
Install Elasticsearch on Ubuntu
Elastic search is an opensource NoSQL database that is scalable and lightning fast. It has a number of applications, the two most commonly used are log analysis/management and as a search backend.
These are my install notes to get it up and running on an ubuntu/debian machine. Elastic provides a very good installation guide if you want to check that out too.
If installing on an Azure machine double check firewall settings and make sure not everyone is allowed to ssh into your box.
The basics
Configure Timezone
dpkg-reconfigure tzdata
Install ntp
apt-get install ntp vim /etc/ntp.conf # add server 0.be.pool.ntp.org server 1.europe.pool.ntp.org server 0.europe.pool.ntp.org service ntp start
Install Java
I opted for Oracle Java, but you could use openjdk too.
add-apt-repository ppa:webupd8team/java apt-get update apt-get install oracle-java8-installer
Check java version
java -version
Create & Mount data disk (for data node)
When configuring this node as a data node, you’ll need to partition the disks that will hold the data/indexes.
reboot fdisk -l #should see a new unpartitioned disk fdisk /dev/sdc #create new partition (n, p, 1, enter, enter, w) mkfs -t ext4 /dev/sdc1 mkdir /mnt/datadisk mount /dev/sdc1 /mnt/datadisk/ df -h umount /mnt/datadisk #List uuid's blkid vim /etc/fstab #add /dev/sdc1 /mnt/datadisk ext4 relatime,errors=remount-ro 0 1 #or add UUID=<insert_blkid> /mnt/datadisk ext4 errors=remount-ro 0 1 mount /mnt/datadisk/
Elastic
Installation
wget -qO - https://packages.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add - vim /etc/apt/sources.list.d/elastic.list #add deb http://packages.elastic.co/elasticsearch/2.x/debian stable main apt-get update apt-get install elasticsearch
If data node
chown -R elasticsearch:elasticsearch /mnt/datadisk
Configuration
update-rc.d elasticsearch defaults 95 10 /usr/share/elasticsearch/bin/plugin install cloud-azure /usr/share/elasticsearch/bin/plugin install mobz/elasticsearch-head /usr/share/elasticsearch/bin/plugin install royrusso/elasticsearch-HQ vim /etc/init.d/elasticsearch # Change heap size to 50% of system memory ! Don't cross the 32G limit. # MAX_LOCKED_MEMORY=unlimited
mv /etc/elasticsearch/elasticsearch.yml /etc/elasticsearch/elasticsearch.yml.org vim /etc/elasticsearch/elasticsearch.yml # Change cluster.name: MyCluster node.name: MyNode node.rack: OptionalLocation path.data: /mnt/datadisk path.repo: /mnt/esarchives network.bind_host: "0.0.0.0" network.publish_host: "local ip" discovery.zen.ping.unicast.hosts: ["xxxx","xxxx","xxxx"] discovery.zen.minimum_master_nodes: 2 discovery.zen.ping.unicast.enabled: false gateway.recover_after_nodes: 3 node.data: true node.master: false bootstrap.memory_lock: true cluster.routing.allocation.disk.watermark.low: "50gb" cluster.routing.allocation.disk.watermark.high: "25gb" http.cors.enabled: false http.cors.allow-origin: "*" http.cors.allow-headers: "X-Requested-With, Content-Type, Content-Length, kbn-version, accept" index.merge.scheduler.max_thread_count: 1 indices.fielddata.cache.size: 20%
Logging
vim /etc/elasticsearch/logging.yml # comment file section # uncomment file with gzip section
Kibana
Install
wget -qO - https://packages.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add - vim /etc/apt/sources.list.d/elastic.list #add deb http://packages.elastic.co/kibana/4.4/debian stable main apt-get update apt-get install kibana update-rc.d kibana defaults 95 10
Configuration
vim /opt/kibana/config/kibana.yml Change "elasticsearch.url" if needed
Logstash
Install
wget -qO - https://packages.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add - vim /etc/apt/sources.list.d/elastic.list #add deb http://packages.elastic.co/logstash/2.2/debian stable main apt-get update apt-get install logstash update-rc.d logstash defaults 95 10
Install Topbeat and others
wget -qO - https://packages.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add - vim /etc/apt/sources.list.d/elastic.list => add deb https://packages.elastic.co/beats/apt stable main apt-get update apt-get install topbeat update-rc.d topbeat defaults 95 10
Install dell open manage
If you are installing on a dell server you might want to install Dell OpenManage Server Administrator.
vim /etc/apt/sources.list.d/linux.dell.com.sources.list #add deb http://linux.dell.com/repo/community/ubuntu trusty openmanage gpg --keyserver pool.sks-keyservers.net --recv-key 1285491434D8786F && gpg -a --export 1285491434D8786F | sudo apt-key add - && sudo apt-get update apt-get install srvadmin-all /opt/dell/srvadmin/sbin/srvadmin-services.sh restart
Install VMWare Extentions/Tools
apt-get install open-vm-tools
Extra optional system tweaks
Bash history with Page up / Page down
vim /etc/inputrc # alternate mappings for "page up" and "page down" to search the history "\e[5~": history-search-backward "\e[6~": history-search-forward
Restart your shell or use Ctrl+X, Ctrl+R to tell it to re-read.
Sudoers
vim /etc/sudoers.d/extra # add user1 ALL=(ALL) ALL user2 ALL=(ALL) ALL