Elasticsearch-5.6.16.tar.gz版本安装配置

发表于2021-03-18更新于2021-03-18

安装JDK1.8

下载链接：https://pan.baidu.com/s/1uvktlm-6AqhQKJQZR1nQuw
上传到机器

1
2
3
4
5
6
7
8
9
10
11
12
13
[root@CentOS7_node1 src]# tar xf jdk-8u161-linux-x64.tar.gz  -C /usr/local/
[root@CentOS7_node1 src]# mv /usr/local/jdk1.8.0_161 /usr/local/java
[root@CentOS7_node1 src]# tail -5 /etc/profile
#################JAVA#################
export JAVA_HOME=/usr/local/java
export JRE_HOME=/usr/local/java/jre
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar:$JRE_HOME/lib
export PATH=$JAVA_HOME/bin:$JRE_HOME/bin:$PATH
[root@CentOS7_node1 src]# source /etc/profile
[root@CentOS7_node1 src]# java -version
java version "1.8.0_161"
Java(TM) SE Runtime Environment (build 1.8.0_161-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.161-b12, mixed mode)

安装Elasticsearch

下载链接：https://pan.baidu.com/s/1hDPam2SWACkbvaz8W_r2SQ
上传到机器

1
2
3
[root@CentOS7_node1 src]# tar xf elasticsearch-5.6.16.tar.gz -C /usr/local/
[root@CentOS7_node1 src]# ls /usr/local/elasticsearch-5.6.16/
bin  config  data  lib  LICENSE.txt  logs  modules  NOTICE.txt  plugins  README.textile
  1. bin目录：二进制脚本启动es程序文件以及启动es插件目录
  2. config目录：elasticsearch配置文件
  3. data目录：默认Elasticsearch生成的索引/切片数据文件存放目录，可以指定多个位置来存储数据
  4. lib目录：一些开发的jar包
  5. logs目录：elasticsearch日志目录
  6. modules目录：模块目录，详情看官网链接：https://www.elastic.co/guide/en/elasticsearch/reference/5.6/modules.html
  7. plugins目录：elasticsearch插件目录，此版本tar包装后默认无插件

bin目录文件介绍

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
[root@CentOS7_node1 elasticsearch-5.6.16]# tree bin/
bin/
├── elasticsearch                                       #Elasticsearch启动脚本
├── elasticsearch.bat                                   #Win启动脚本，tar包也是可以在Win运行的
├── elasticsearch.in.bat
├── elasticsearch.in.sh
├── elasticsearch-keystore
├── elasticsearch-keystore.bat
├── elasticsearch-plugin                                #集成插件启动脚本         
├── elasticsearch-plugin.bat
├── elasticsearch-service.bat
├── elasticsearch-service-mgr.exe
├── elasticsearch-service-x64.exe
├── elasticsearch-service-x86.exe
├── elasticsearch-systemd-pre-exec
├── elasticsearch-translog
└── elasticsearch-translog.bat

0 directories, 15 files

config目录文件介绍

1
2
3
4
5
6
7
8
[root@CentOS7_node1 elasticsearch-5.6.16]# tree config/
config/
├── elasticsearch.yml                              #ES节点集群等相关配置
├── jvm.options                                    #JVM堆内存及GC相关配置
├── log4j2.properties                              #log4j2框架日志输出相关配置
└── scripts                                        #脚本存放目录，默认无脚本

1 directory, 3 files

logs目录文件介绍

1
2
3
4
5
6
7
8
9
log目录默认是没有的，需要你正常启动elasticsearch后，会在你的es目录下自动生成一个log目录
[root@CentOS7_node1 elasticsearch-5.6.16]# tree logs/
logs/
├── elasticsearch_deprecation.log                   #ES描述日志
├── elasticsearch_index_indexing_slowlog.log        #索引日志
├── elasticsearch_index_search_slowlog.log          #搜索日志
└── elasticsearch.log                               #集群名称会生成一个以集群名称命名的日志                                          

0 directories, 4 files

创建elasticsearch所用到的用户及组

elasticsearch默认是不能以root身份去运行的，否则启动会报错误信息为“can not run elasticsearch as root Elasticsearch”，所以我们要创建一个普通用户来管理elasticsearch
创建elasticsearch用户及组

1
2
3
4
5
[root@CentOS7_node1 /]# groupadd elasticsearch 
[root@CentOS7_node1 /]# useradd elasticsearch -g elasticsearch
[root@CentOS7_node1 /]# echo '3edc#EDC' | passwd --stdin elasticsearch
Changing password for user elasticsearch.
passwd: all authentication tokens updated successfully.

更改Elasticsearch目录属性信息

1
2
3
4
5
6
7
8
9
10
11
12
[root@CentOS7_node1 /]# chown -Rf elasticsearch:elasticsearch /usr/local/elasticsearch-5.6.16
[root@CentOS7_node1 /]# ll /usr/local/elasticsearch-5.6.16
total 240
drwxr-xr-x.  2 elasticsearch elasticsearch   4096 May  5 14:57 bin
drwxr-xr-x.  2 elasticsearch elasticsearch   4096 May  5 15:11 config
drwxr-xr-x.  2 elasticsearch elasticsearch   4096 Mar 13 23:34 lib
-rw-r--r--.  1 elasticsearch elasticsearch  11358 Mar 13 23:31 LICENSE.txt
drwxr-xr-x.  2 elasticsearch elasticsearch   4096 May  5 14:02 logs
drwxr-xr-x. 13 elasticsearch elasticsearch   4096 Mar 13 23:35 modules
-rw-r--r--.  1 elasticsearch elasticsearch 194187 Mar 13 23:34 NOTICE.txt
drwxr-xr-x.  2 elasticsearch elasticsearch   4096 Mar 13 23:34 plugins
-rw-r--r--.  1 elasticsearch elasticsearch   9491 Mar 13 23:31 README.textile

启动elasticsearch

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
切换到elasticsearch用户来启动ES
[root@CentOS7_node1 ~]# su elasticsearch
[elasticsearch@CentOS7_node1 root]$ /usr/local/elasticsearch-5.6.16/bin/elasticsearch -h
默认情况下，Elasticsearch在前台运行，将其日志打印到标准输出（stdout），然后按下Ctrl-C即可停止
starts elasticsearch

Option                Description                                              
------                -----------                                              
-E <KeyValuePair>     Configure a setting         
#ES启动时可以指定配置信息，是以Key Value的形式来配置                           
#-E选项实例：./bin/elasticsearch -d -Ecluster.name=my_cluster -Enode.name=node_1
#-d 表示后台启动ES，-Ecluster.name指定ES的集群名称，node.name指定节点名称，这两参数在ES的config/elasticsearch.yml中都有设置
-V, --version         Prints elasticsearch version information and exits       
#显示ES的版本信息
-d, --daemonize       Starts Elasticsearch in the background                    
#-d选项改为以后台守护进程方式启动
-h, --help            show help                                                         
-p, --pidfile <Path>  Creates a pid file in the specified path on start         
#指定ES的PID文件路径 
-q, --quiet           Turns off standard ouput/error streams logging in console     
#关闭控制台中的标准输出/错误流日志记录
-s, --silent          show minimal output                                           
#精简显示启动信息                       
-v, --verbose         show verbose output                                       
#详细显示启动信息
[elasticsearch@CentOS7_node1 /]$ /usr/local/elasticsearch-5.6.16/bin/elasticsearch 
[2019-05-07T23:39:22,140][INFO ][o.e.n.Node               ] [] initializing ...
[2019-05-07T23:39:22,229][INFO ][o.e.e.NodeEnvironment    ] [lPEqF6-] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [18gb], net total_space [24.5gb], spins? [unknown], types [rootfs]
[2019-05-07T23:39:22,229][INFO ][o.e.e.NodeEnvironment    ] [lPEqF6-] heap size [1.9gb], compressed ordinary object pointers [true]
[2019-05-07T23:39:22,230][INFO ][o.e.n.Node               ] node name [lPEqF6-] derived from node ID [lPEqF6-7TaClsuN_NVm9NA]; set [node.name] to override
[2019-05-07T23:39:22,230][INFO ][o.e.n.Node               ] version[5.6.16], pid[17264], build[3a740d1/2019-03-13T15:33:36.565Z], OS[Linux/3.10.0-957.el7.x86_64/amd64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_161/25.161-b12]
[2019-05-07T23:39:22,231][INFO ][o.e.n.Node               ] JVM arguments [-Xms2g, -Xmx2g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -Djdk.io.permissionsUseCanonicalPath=true, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j.skipJansi=true, -XX:+HeapDumpOnOutOfMemoryError, -Des.path.home=/usr/local/elasticsearch-5.6.16]
[2019-05-07T23:39:22,900][INFO ][o.e.p.PluginsService     ] [lPEqF6-] loaded module [aggs-matrix-stats]
[2019-05-07T23:39:22,900][INFO ][o.e.p.PluginsService     ] [lPEqF6-] loaded module [ingest-common]
[2019-05-07T23:39:22,901][INFO ][o.e.p.PluginsService     ] [lPEqF6-] loaded module [lang-expression]
[2019-05-07T23:39:22,901][INFO ][o.e.p.PluginsService     ] [lPEqF6-] loaded module [lang-groovy]
[2019-05-07T23:39:22,901][INFO ][o.e.p.PluginsService     ] [lPEqF6-] loaded module [lang-mustache]
[2019-05-07T23:39:22,901][INFO ][o.e.p.PluginsService     ] [lPEqF6-] loaded module [lang-painless]
[2019-05-07T23:39:22,901][INFO ][o.e.p.PluginsService     ] [lPEqF6-] loaded module [parent-join]
[2019-05-07T23:39:22,901][INFO ][o.e.p.PluginsService     ] [lPEqF6-] loaded module [percolator]
[2019-05-07T23:39:22,901][INFO ][o.e.p.PluginsService     ] [lPEqF6-] loaded module [reindex]
[2019-05-07T23:39:22,901][INFO ][o.e.p.PluginsService     ] [lPEqF6-] loaded module [transport-netty3]
[2019-05-07T23:39:22,901][INFO ][o.e.p.PluginsService     ] [lPEqF6-] loaded module [transport-netty4]
[2019-05-07T23:39:22,902][INFO ][o.e.p.PluginsService     ] [lPEqF6-] no plugins loaded
[2019-05-07T23:39:24,457][INFO ][o.e.d.DiscoveryModule    ] [lPEqF6-] using discovery type [zen]
[2019-05-07T23:39:24,935][INFO ][o.e.n.Node               ] initialized
[2019-05-07T23:39:24,936][INFO ][o.e.n.Node               ] [lPEqF6-] starting ...
[2019-05-07T23:39:27,518][INFO ][o.e.t.TransportService   ] [lPEqF6-] publish_address {127.0.0.1:9300}, bound_addresses {[::1]:9300}, {127.0.0.1:9300}
[2019-05-07T23:39:27,529][WARN ][o.e.b.BootstrapChecks    ] [lPEqF6-] max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536]
[2019-05-07T23:39:27,529][WARN ][o.e.b.BootstrapChecks    ] [lPEqF6-] max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
[2019-05-07T23:39:30,589][INFO ][o.e.c.s.ClusterService   ] [lPEqF6-] new_master {lPEqF6-}{lPEqF6-7TaClsuN_NVm9NA}{O8BQXY-3Q1O9nrC34rDU5w}{127.0.0.1}{127.0.0.1:9300}, reason: zen-disco-elected-as-master ([0] nodes joined)
[2019-05-07T23:39:30,622][INFO ][o.e.h.n.Netty4HttpServerTransport] [lPEqF6-] publish_address {127.0.0.1:9200}, bound_addresses {[::1]:9200}, {127.0.0.1:9200}
[2019-05-07T23:39:30,622][INFO ][o.e.n.Node               ] [lPEqF6-] started
[2019-05-07T23:39:30,635][INFO ][o.e.g.GatewayService     ] [lPEqF6-] recovered [0] indices into cluster_state

以上告警有WARN警告信息，我们一定要注意下，并解决此问题，这涉及到了ES的两种模式，分别为：开发者模式和生产模式

开发者模式可生产模式

默认情况下，Elasticsearch假定您正在开发模式下工作。如果未正确配置上述任何设置，则会向日志文件写入警告，但您将能够启动并运行Elasticsearch节点。
一旦配置了网络设置network.host(此参数在elasticsearch.yml文件中配置)，Elasticsearch就会假定您正在转向生产并将上述警告升级为异常。这些异常将阻止您的Elasticsearch节点启动。这是一项重要的安全措施，可确保您不会因服务器配置错误而丢失数据。
为了更让我们的测试环境贴合生产模式，我们坚决不允许它出半点问题

启动失败问题

1.文件描述符过低
[2019-05-07T23:39:27,529][WARN ][o.e.b.BootstrapChecks ] [lPEqF6-] max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536]

解决：
修改文件描述符数量

1
2
3
4
5
echo '* soft nofile 65536' >> /etc/security/limits.conf
echo '* hard nofile 65536' >> /etc/security/limits.conf
然后重新登录终端
[root@CentOS7_node1 ~]# ulimit -n
65536

2.max_map_count值为65530]太低，至少增加到262144
[2019-05-07T23:39:27,529][WARN ][o.e.b.BootstrapChecks ] [lPEqF6-] max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
在linux系统上，elasticsearch默认使用hybrid mmapfs / niofs来存储索引文件，因此操作系统主要会通过ｍｍａｐ来限制存储的空间，因此如果存储空间满了，那么会抛出异常，我们可以使用如下命令来更改设置。

解决：
修改max_map_count值

1
2
3
4
5
6
7
8
临时设置：
[root@CentOS7_node1 ~]# sysctl -w vm.max_map_count=655360
vm.max_map_count = 655360

永久设置：
[root@CentOS7_node1 ~]# echo 'vm.max_map_count=655360' >> /etc/sysctl.conf 
[root@CentOS7_node1 ~]# sysctl -p   
vm.max_map_count = 655360

3.这个问题我们现在虽然是没遇见，但是建议先把坑填上，不然还是会出现的
在ES启动的时候警告信息为：“Unable to lock JVM Memory: error=12, reason=Cannot allocate memory”，无法分配内存的意思

解决：

1
2
3
4
5
[root@CentOS7_node1 ~]# echo '* soft memlock unlimited' >> /etc/security/limits.conf
[root@CentOS7_node1 ~]# echo '* hard memlock unlimited' >> /etc/security/limits.conf
[root@CentOS7_node1 ~]# tail -2 /etc/security/limits.conf
* soft memlock unlimited
* hard memlock unlimited

再次启动
首先要退出终端，然后重新登录，让elasticsearch用户能够重新加载我们刚修改的配置

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
[root@CentOS7_node1 ~]# su elasticsearch
[elasticsearch@CentOS7_node1 root]$ ulimit -n
65536
[elasticsearch@CentOS7_node1 root]$ /usr/local/elasticsearch-5.6.16/bin/elasticsearch 
[2019-05-08T00:00:43,188][INFO ][o.e.n.Node               ] [] initializing ...
[2019-05-08T00:00:43,265][INFO ][o.e.e.NodeEnvironment    ] [lPEqF6-] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [18.1gb], net total_space [24.5gb], spins? [unknown], types [rootfs]
[2019-05-08T00:00:43,265][INFO ][o.e.e.NodeEnvironment    ] [lPEqF6-] heap size [1.9gb], compressed ordinary object pointers [true]
[2019-05-08T00:00:43,266][INFO ][o.e.n.Node               ] node name [lPEqF6-] derived from node ID [lPEqF6-7TaClsuN_NVm9NA]; set [node.name] to override
[2019-05-08T00:00:43,266][INFO ][o.e.n.Node               ] version[5.6.16], pid[21842], build[3a740d1/2019-03-13T15:33:36.565Z], OS[Linux/3.10.0-957.el7.x86_64/amd64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_161/25.161-b12]
[2019-05-08T00:00:43,266][INFO ][o.e.n.Node               ] JVM arguments [-Xms2g, -Xmx2g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -Djdk.io.permissionsUseCanonicalPath=true, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j.skipJansi=true, -XX:+HeapDumpOnOutOfMemoryError, -Des.path.home=/usr/local/elasticsearch-5.6.16]
[2019-05-08T00:00:44,000][INFO ][o.e.p.PluginsService     ] [lPEqF6-] loaded module [aggs-matrix-stats]
[2019-05-08T00:00:44,000][INFO ][o.e.p.PluginsService     ] [lPEqF6-] loaded module [ingest-common]
[2019-05-08T00:00:44,000][INFO ][o.e.p.PluginsService     ] [lPEqF6-] loaded module [lang-expression]
[2019-05-08T00:00:44,000][INFO ][o.e.p.PluginsService     ] [lPEqF6-] loaded module [lang-groovy]
[2019-05-08T00:00:44,000][INFO ][o.e.p.PluginsService     ] [lPEqF6-] loaded module [lang-mustache]
[2019-05-08T00:00:44,001][INFO ][o.e.p.PluginsService     ] [lPEqF6-] loaded module [lang-painless]
[2019-05-08T00:00:44,001][INFO ][o.e.p.PluginsService     ] [lPEqF6-] loaded module [parent-join]
[2019-05-08T00:00:44,001][INFO ][o.e.p.PluginsService     ] [lPEqF6-] loaded module [percolator]
[2019-05-08T00:00:44,001][INFO ][o.e.p.PluginsService     ] [lPEqF6-] loaded module [reindex]
[2019-05-08T00:00:44,001][INFO ][o.e.p.PluginsService     ] [lPEqF6-] loaded module [transport-netty3]
[2019-05-08T00:00:44,001][INFO ][o.e.p.PluginsService     ] [lPEqF6-] loaded module [transport-netty4]
[2019-05-08T00:00:44,001][INFO ][o.e.p.PluginsService     ] [lPEqF6-] no plugins loaded
[2019-05-08T00:00:45,824][INFO ][o.e.d.DiscoveryModule    ] [lPEqF6-] using discovery type [zen]
[2019-05-08T00:00:46,288][INFO ][o.e.n.Node               ] initialized
[2019-05-08T00:00:46,288][INFO ][o.e.n.Node               ] [lPEqF6-] starting ...
[2019-05-08T00:00:48,555][INFO ][o.e.t.TransportService   ] [lPEqF6-] publish_address {127.0.0.1:9300}, bound_addresses {[::1]:9300}, {127.0.0.1:9300}
[2019-05-08T00:00:51,637][INFO ][o.e.c.s.ClusterService   ] [lPEqF6-] new_master {lPEqF6-}{lPEqF6-7TaClsuN_NVm9NA}{QRumAuUXQpW14l6cEgBUGA}{127.0.0.1}{127.0.0.1:9300}, reason: zen-disco-elected-as-master ([0] nodes joined)
[2019-05-08T00:00:51,672][INFO ][o.e.h.n.Netty4HttpServerTransport] [lPEqF6-] publish_address {127.0.0.1:9200}, bound_addresses {[::1]:9200}, {127.0.0.1:9200}
[2019-05-08T00:00:51,672][INFO ][o.e.n.Node               ] [lPEqF6-] started
[2019-05-08T00:00:51,692][INFO ][o.e.g.GatewayService     ] [lPEqF6-] recovered [0] indices into cluster_state

ES端口详解

再次打开一个终端

1
2
3
4
5
6
7
8
[root@CentOS7_node1 ~]# ps -ef|grep java
elastic+ 21842 21777 14 00:00 pts/1    00:00:09 /usr/local/java/bin/java -Xms2g -Xmx2g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+AlwaysPreTouch -server -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -Djdk.io.permissionsUseCanonicalPath=true -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Dlog4j.skipJansi=true -XX:+HeapDumpOnOutOfMemoryError -Des.path.home=/usr/local/elasticsearch-5.6.16 -cp /usr/local/elasticsearch-5.6.16/lib/* org.elasticsearch.bootstrap.Elasticsearch
root     22132 20125  0 00:01 pts/2    00:00:00 grep --color=auto java
[root@CentOS7_node1 ~]# ss -anplt | grep java
LISTEN     0      128       ::ffff:127.0.0.1:9200                    :::*                   users:(("java",pid=21842,fd=135))
LISTEN     0      128        ::1:9200                    :::*                   users:(("java",pid=21842,fd=134))
LISTEN     0      128       ::ffff:127.0.0.1:9300                    :::*                   users:(("java",pid=21842,fd=121))
LISTEN     0      128        ::1:9300                    :::*                   users:(("java",pid=21842,fd=119))

查看java进程可以看到我们的elasticsearch是以elasticsearch用户所启动
查看端口看到ES默认启用两个端口，分别是TCP端口9200及TCP端口9300，但是只监听在了127.0.0.1的地址上，如需修改监听地址，我们可以通过config/elasticsearch.yml文件来修改。
9200端口是为搜索服务所提供的端口。
9300是ES集群中通信端口，根据9300端口来判定集群各节点的存活性，此端口采用单播的形式进行集群通信。

测试ES是否启动成功

打开一个终端，访问本地9200端口，查看返回信息

1
2
3
4
5
6
7
8
9
10
11
12
13
14
[elasticsearch@CentOS7_node1 root]$ curl http://127.0.0.1:9200 
{
  "name" : "mDAWBTb",                       #本节点名称
  "cluster_name" : "elasticsearch",         #集群名称
  "cluster_uuid" : "DEXGYpVzRhOnXcmykCtJTQ",#集群UID
  "version" : {                             #以下为版本信息
    "number" : "5.6.16",                    #ES版本
    "build_hash" : "3a740d1",               #ES默认会为各个节点分片一个UID标识符，具有唯一性
    "build_date" : "2019-03-13T15:33:36.565Z",#es数据构建时间
    "build_snapshot" : false,                 #是否构建快照
    "lucene_version" : "6.6.1"                #Apache Lucene检索工具库版本
  },
  "tagline" : "You Know, for Search"
}

转载于：

[https://blog.51cto.com/11612299/2405048]: