LogStash, ElasticSearch, Kibana 설치하기

1. elasticsearch 설치

# wget https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.7.1.tar.gz

# tar zxvf ./elasticsearch-1.7.1.tar.gz

2. elasticsearch 실행

: elasticsearch는 기본값으로 9200 포트로 바인딩된다.

# cd elasticsearch-1.7.1

# ./bin/elasticsearch &

[2015-08-21 05:15:10,670][INFO ][node                     ] [Hindsight Lad] version[1.7.1], pid[2450], build[b88f43f/2015-07-29T09:54:16Z]
[2015-08-21 05:15:10,670][INFO ][node                     ] [Hindsight Lad] initializing ...
[2015-08-21 05:15:10,803][INFO ][plugins                  ] [Hindsight Lad] loaded [], sites []
[2015-08-21 05:15:10,855][INFO ][env                      ] [Hindsight Lad] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [5.6gb], net total_space [7.7gb], types [rootfs]
[2015-08-21 05:15:13,589][INFO ][node                     ] [Hindsight Lad] initialized
[2015-08-21 05:15:13,590][INFO ][node                     ] [Hindsight Lad] starting ...
[2015-08-21 05:15:13,953][INFO ][transport                ] [Hindsight Lad] bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/172.31.27.21:9300]}
[2015-08-21 05:15:13,991][INFO ][discovery                ] [Hindsight Lad] elasticsearch/lxbEpNmrT2SxCNI4xHP5lQ
[2015-08-21 05:15:17,782][INFO ][cluster.service          ] [Hindsight Lad] new_master [Hindsight Lad][lxbEpNmrT2SxCNI4xHP5lQ][ip-172-31-27-21][inet[/172.31.27.21:9300]], reason: zen-disco-join (elected_as_master)
[2015-08-21 05:15:17,830][INFO ][http                     ] [Hindsight Lad] bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/172.31.27.21:9200]}
[2015-08-21 05:15:17,831][INFO ][node                     ] [Hindsight Lad] started
[2015-08-21 05:15:17,941][INFO ][gateway                  ] [Hindsight Lad] recovered [0] indices into cluster_state

3. logstash 설치

# sudo su

# wget https://download.elastic.co/logstash/logstash/logstash-1.5.4.tar.gz

# tar zxvf ./logstash-1.5.4.tar.gz

4. logstash 실행

: stdin으로 들어오는 메시지 elasticsearch로 출력

: ./bin/logstash -e 'input { file { type => "log4j" path => "/usr/share/tomcat8/logs/catalina.out" } } output { elasticsearch { host => "54.65.37.240" } }'

# cd ~/logstash-1.5.4

# ./bin/logstash -e 'input { stdin { } } output { elasticsearch { host => localhost } }'

# TEST MESSAGE

5. elasticsearch API 사용하여 output 확인

# curl 'http://localhost:9200/_search?pretty'

{
  "took" : 2,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "failed" : 0
  },
  "hits" : {
    "total" : 15,
    "max_score" : 1.0,
    "hits" : [ {
      "_index" : "logstash-2015.08.21",
      "_type" : "logs",
      "_id" : "AU9OwBbXKOC_sOqlZdpn",
      "_score" : 1.0,
      "_source":{"message":"TEST MESSAGE","@version":"1","@timestamp":"2015-08-21T05:34:15.766Z","host":"ip-172-31-27-21"}
    }, {
      "_index" : "logstash-2015.08.21",
      "_type" : "logs",
      "_id" : "AU9OtoNNKOC_sOqlZdpc",
      "_score" : 1.0,
      "_source":{"message":"","@version":"1","@timestamp":"2015-08-21T05:23:47.754Z","host":"ip-172-31-27-21"}
    } ]
  }
}

6. kibana 설치

: 5601 포트로 바인딩된다.

: http://<ip>:5601 접속

# wget https://download.elastic.co/kibana/kibana/kibana-4.1.1-linux-x64.tar.gz

# tar xvf kibana-*.tar.gz

# cd kibana-4.1.1-linux-x64

# ./bin/kibana

참고 사이트

1. http://logstash.net/docs/1.4.1/tutorials/getting-started-with-logstash