Elastic search is an opensource NoSQL database that is scalable and lightning fast. It has a number of applications, the two most commonly used are log analysis/management and as a search backend.

These are my install notes to get it up and running on an ubuntu/debian machine. Elastic provides a very good installation guide if you want to check that out too.

If installing on an Azure machine double check firewall settings and make sure not everyone is allowed to ssh into your box.

The basics

Configure Timezone

dpkg-reconfigure tzdata

Install ntp

apt-get install ntp
vim /etc/ntp.conf

# add
server 0.be.pool.ntp.org
server 1.europe.pool.ntp.org
server 0.europe.pool.ntp.org

service ntp start

Install Java
I opted for Oracle Java, but you could use openjdk too.

add-apt-repository ppa:webupd8team/java
apt-get update
apt-get install oracle-java8-installer

Check java version

java -version

Create & Mount data disk (for data node)
When configuring this node as a data node, you’ll need to partition the disks that will hold the data/indexes.

reboot
fdisk -l 
#should see a new unpartitioned disk

fdisk /dev/sdc
#create new partition (n, p, 1, enter, enter, w)

mkfs -t ext4 /dev/sdc1
mkdir /mnt/datadisk
mount /dev/sdc1 /mnt/datadisk/
df -h
umount /mnt/datadisk

#List uuid's
blkid

vim /etc/fstab
#add 
/dev/sdc1 /mnt/datadisk ext4 relatime,errors=remount-ro 0 1

#or add
UUID=<insert_blkid> /mnt/datadisk ext4 errors=remount-ro 0 1

mount /mnt/datadisk/

Elastic

Installation

wget -qO - https://packages.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
vim /etc/apt/sources.list.d/elastic.list
#add 
deb http://packages.elastic.co/elasticsearch/2.x/debian stable main

apt-get update
apt-get install elasticsearch

If data node

chown -R elasticsearch:elasticsearch /mnt/datadisk

Configuration

update-rc.d elasticsearch defaults 95 10

/usr/share/elasticsearch/bin/plugin install cloud-azure
/usr/share/elasticsearch/bin/plugin install mobz/elasticsearch-head
/usr/share/elasticsearch/bin/plugin install royrusso/elasticsearch-HQ

vim /etc/init.d/elasticsearch
# Change heap size to 50% of system memory ! Don't cross the 32G limit.
# MAX_LOCKED_MEMORY=unlimited

mv /etc/elasticsearch/elasticsearch.yml /etc/elasticsearch/elasticsearch.yml.org
vim /etc/elasticsearch/elasticsearch.yml

# Change
cluster.name: MyCluster
node.name: MyNode
node.rack: OptionalLocation
path.data: /mnt/datadisk
path.repo: /mnt/esarchives
network.bind_host: "0.0.0.0"
network.publish_host: "local ip"
discovery.zen.ping.unicast.hosts: ["xxxx","xxxx","xxxx"]
discovery.zen.minimum_master_nodes: 2
discovery.zen.ping.unicast.enabled: false
gateway.recover_after_nodes: 3
node.data: true
node.master: false
bootstrap.memory_lock: true
cluster.routing.allocation.disk.watermark.low: "50gb"
cluster.routing.allocation.disk.watermark.high: "25gb"
http.cors.enabled: false
http.cors.allow-origin: "*"
http.cors.allow-headers: "X-Requested-With, Content-Type, Content-Length, kbn-version, accept"
index.merge.scheduler.max_thread_count: 1
indices.fielddata.cache.size: 20%

Logging

vim /etc/elasticsearch/logging.yml
# comment file section
# uncomment file with gzip section

Kibana

Install

wget -qO - https://packages.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
vim /etc/apt/sources.list.d/elastic.list
#add 
deb http://packages.elastic.co/kibana/4.4/debian stable main
apt-get update
apt-get install kibana
update-rc.d kibana defaults 95 10

Configuration

vim /opt/kibana/config/kibana.yml
Change "elasticsearch.url" if needed

Logstash

Install

wget -qO - https://packages.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
vim /etc/apt/sources.list.d/elastic.list
#add 
deb http://packages.elastic.co/logstash/2.2/debian stable main
apt-get update
apt-get install logstash
update-rc.d logstash defaults 95 10

Install Topbeat and others

wget -qO - https://packages.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
vim /etc/apt/sources.list.d/elastic.list
=> add deb https://packages.elastic.co/beats/apt stable main
apt-get update
apt-get install topbeat
update-rc.d topbeat defaults 95 10

Install dell open manage

If you are installing on a dell server you might want to install Dell OpenManage Server Administrator.

vim /etc/apt/sources.list.d/linux.dell.com.sources.list
#add 
deb http://linux.dell.com/repo/community/ubuntu trusty openmanage

gpg --keyserver pool.sks-keyservers.net --recv-key 1285491434D8786F && gpg -a --export 1285491434D8786F | sudo apt-key add - && sudo apt-get update
apt-get install srvadmin-all
/opt/dell/srvadmin/sbin/srvadmin-services.sh restart

Install VMWare Extentions/Tools

apt-get install open-vm-tools

Extra optional system tweaks

Bash history with Page up / Page down

vim /etc/inputrc
# alternate mappings for "page up" and "page down" to search the history
"\e[5~": history-search-backward
"\e[6~": history-search-forward

Restart your shell or use Ctrl+X, Ctrl+R to tell it to re-read.

Sudoers

vim /etc/sudoers.d/extra
# add
user1 ALL=(ALL) ALL
user2 ALL=(ALL) ALL