CentOS 7 部署 Elasticsearch7.4 集群并进行安全认证

释放双眼,带上耳机,听听看~!

Elastic 安全认证介绍

从 **Elastic Stack 6.8 和 7.1 版本开始 **,Elastic 公司开放了一些基础认证功能。

免费版本

  1. TLS 功能,可对通信进行加密
  2. 文件和原生 Realm,可用于创建和管理用户
  3. 基于角色的访问控制,可用于控制用户对集群 API 和索引的访问权限;
  4. 通过针对 Kibana Spaces 的安全功能,还可允许在 Kibana 中实现多租户。

收费版本

  1. 日志审计
  2. IP过滤
  3. LDAP、PKI和活动目录身份验证
  4. 单点登录身份验证(SAML、Kerberos)
  5. 基于属性的权限控制
  6. 字段和文档级别安全性
  7. 静态数据加密支持

集群环境

以下在一台机器上采用伪造集群的方式进行部署elasticserach

IP 监听端口 主机名 系统 es实例名称 es版本 kibana版本
192.168.31.215 9201/9301 elastic CentOS 7.6.1810 elastic_node1 elasticsearch-7.4.0 kibana-7.4.0
192.168.31.215 9202/9302 elastic CentOS 7.6.1810 elastic_node2 elasticsearch-7.4.0 kibana-7.4.0
192.168.31.215 9203/9303 elastic CentOS 7.6.1810 elastic_node3 elasticsearch-7.4.0 kibana-7.4.0

集群部署

准备环境

[root@elastic /]# wget -P /usr/local/src https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.4.0-linux-x86_64.tar.gz
[root@elastic local]# cd /
[root@elastic /]# cd /usr/local/src/
[root@elastic src]# tar xf elasticsearch-7.4.0-linux-x86_64.tar.gz 
[root@elastic src]# mv elasticsearch-7.4.0 /usr/local/elastic_node1
[root@elastic src]# cp -rf /usr/local/elastic_node1 /usr/local/elastic_node2
[root@elastic src]# cp -rf /usr/local/elastic_node1 /usr/local/elastic_node3
[root@elastic src]# mkdir /usr/local/elastic_node1/{data,run}
[root@elastic src]# mkdir /usr/local/elastic_node2/{data,run}
[root@elastic src]# mkdir /usr/local/elastic_node3/{data,run}
[root@elastic src]# useradd -s /bin/bash -U elastic
[root@elastic src]# chown -Rf elastic.elastic /usr/local/elastic_node* 

修改系统配置

#修改elastic系统文件打开数
cat << EOF >> /etc/security/limits.conf
elastic soft nofile 65536
elastic hard nofile 65536
EOF

#修改max_map_count值
sysctl -w vm.max_map_count=655360
echo 'vm.max_map_count=655360' >> /etc/sysctl.conf 
sysctl -p

#修改无法分配内存问题
cat << EOF >> /etc/security/limits.conf
elastic soft memlock unlimited
elastic hard memlock unlimited
EOF

修改配置文件

1.elastic_node1elasticsearch.yml配置文件

[root@elastic src]# su - elastic 
[elastic@elastic ~]$ vim /usr/local/elastic_node1/config/elasticsearch.yml
cluster.name: elastic_cluster
node.name: elastic_node1
node.master: true
node.data: true

path.data: /usr/local/elastic_node1/data
path.logs: /usr/local/elastic_node1/logs

bootstrap.memory_lock: true

network.host: 192.168.31.215
network.tcp.no_delay: true
network.tcp.keep_alive: true
network.tcp.reuse_address: true
network.tcp.send_buffer_size: 256mb
network.tcp.receive_buffer_size: 256mb

transport.tcp.port: 9301
transport.tcp.compress: true

http.max_content_length: 200mb
http.cors.enabled: true
http.cors.allow-origin: "*"
http.port: 9201

discovery.seed_hosts: ["192.168.31.215:9301","192.168.31.215:9302","192.168.31.215:9303"]
cluster.initial_master_nodes: ["192.168.31.215:9301","192.168.31.215:9302","192.168.31.215:9303"]
cluster.fault_detection.leader_check.interval: 15s
discovery.cluster_formation_warning_timeout: 30s
cluster.join.timeout: 30s
cluster.publish.timeout: 90s
cluster.routing.allocation.cluster_concurrent_rebalance: 16
cluster.routing.allocation.node_concurrent_recoveries: 16
cluster.routing.allocation.node_initial_primaries_recoveries: 16

2.elastic_node1修改jvm.options配置文件

[elastic@elastic ~]$ grep -Ev '#|^$' /usr/local/elastic_node1/config/jvm.options | head -2
-Xms4g
-Xmx4g

3.复制配置文件

[elastic@elastic ~]$ \cp -rf /usr/local/elastic_node1/config/elasticsearch.yml /usr/local/elastic_node2/config/
[elastic@elastic ~]$ \cp -rf /usr/local/elastic_node1/config/elasticsearch.yml /usr/local/elastic_node3/config/
[elastic@elastic ~]$ \cp -rf /usr/local/elastic_node1/config/jvm.options /usr/local/elastic_node2/config/
[elastic@elastic ~]$ \cp -rf /usr/local/elastic_node1/config/jvm.options /usr/local/elastic_node3/config/

elastic_node2需要与elastic_node1变更的地方

node.name: elastic_node2                            #es在集群中的名称
path.data: /usr/local/elastic_node2/data            #数据写入地址
path.logs: /usr/local/elastic_node2/logs            #日志写入地址
transport.tcp.port: 9302                            #集群通信端口(我这里为伪集群)
http.port: 9202                                     #es对外调用端口

elastic_node3同上

4.启动集群

[elastic@elastic ~]$ /usr/local/elastic_node1/bin/elasticsearch -d -p /usr/local/elastic_node1/run/elastic_node1.pid
[elastic@elastic ~]$ /usr/local/elastic_node2/bin/elasticsearch -d -p /usr/local/elastic_node2/run/elastic_node2.pid
[elastic@elastic ~]$ /usr/local/elastic_node3/bin/elasticsearch -d -p /usr/local/elastic_node3/run/elastic_node3.pid

5.查看集群节点状态

[elastic@elastic ~]$ curl -XGET 'http://192.168.31.215:9201/_cat/nodes?v'
ip             heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
192.168.31.215           15          68   6    2.32    0.90     0.36 dilm      -      elastic_node3
192.168.31.215           13          68  20    2.32    0.90     0.36 dilm      *      elastic_node2
192.168.31.215           11          68  19    2.32    0.90     0.36 dilm      -      elastic_node1

配置ES集群间 TLS 和 身份验证

1.生成证书
es集群通过证书来安全的组成集群

在主节点elastic_node2上配置TLS,以下命令生成证书到我们指定的位置

[elastic@elastic ~]$ /usr/local/elastic_node2/bin/elasticsearch-certutil cert -out /usr/local/elastic_node2/config/elastic-certificates.p12 -pass ""

2.复制证书到elastic_node1elastic_nod3节点

[elastic@elastic ~]$ cp /usr/local/elastic_node2/config/elastic-certificates.p12 /usr/local/elastic_node1/config/
[elastic@elastic ~]$ cp /usr/local/elastic_node2/config/elastic-certificates.p12 /usr/local/elastic_node3/config/

3.修改配置文件开启TLS集群通信认证
elastic_node1

cat << EOF >> /usr/local/elastic_node1/config/elasticsearch.yml

xpack.security.enabled: true
xpack.license.self_generated.type: basic
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: /usr/local/elastic_node1/config/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: /usr/local/elastic_node1/config/elastic-certificates.p12
EOF 

elastic_node2

cat << EOF >> /usr/local/elastic_node2/config/elasticsearch.yml

xpack.security.enabled: true
xpack.license.self_generated.type: basic
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: /usr/local/elastic_node2/config/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: /usr/local/elastic_node2/config/elastic-certificates.p12
EOF 

elastic_node3

cat << EOF >> /usr/local/elastic_node3/config/elasticsearch.yml

xpack.security.enabled: true
xpack.license.self_generated.type: basic
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: /usr/local/elastic_node3/config/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: /usr/local/elastic_node3/config/elastic-certificates.p12
EOF 

4.重启es节点
停止es节点

[elastic@elastic ~]$ cat /usr/local/elastic_node1/run/elastic_node1.pid | xargs kill -HUP
[elastic@elastic ~]$ cat /usr/local/elastic_node2/run/elastic_node2.pid | xargs kill -HUP
[elastic@elastic ~]$ cat /usr/local/elastic_node3/run/elastic_node3.pid | xargs kill -HUP

启动es节点

[elastic@elastic ~]$ /usr/local/elastic_node1/bin/elasticsearch -d -p /usr/local/elastic_node1/run/elastic_node1.pid
[elastic@elastic ~]$ /usr/local/elastic_node2/bin/elasticsearch -d -p /usr/local/elastic_node2/run/elastic_node2.pid
[elastic@elastic ~]$ /usr/local/elastic_node3/bin/elasticsearch -d -p /usr/local/elastic_node3/run/elastic_node3.pid

5.查看日志及进行确认集群已组成

这个时候使用curl命令会失败的,因为集群通信认证了以后,我们再去curl访问是会询问账号的,以下示例

我们使用浏览器访问测试,es会询问账号密码,下面就使用工具来生成账号密码

为 Elasticserach集群配置密码

一旦我们的集群开始运行以后,就可以配置账号密码
以下两个命令可以来设置连接elasticsearch的密码
bin/elasticsearch-setup-passwords auto为各种内部堆栈用户生成随机密码
bin/elasticsearch-setup-passwords interactive手动定义内部堆栈密码
以下我采用随机生成密码,随机生成的密码请谨慎保管

在集群中的任何一个节点上生成密码都可以,一个节点生成后会同步至集群
以下我在 elastic_node1 上生成密码

[elastic@elastic ~]$ /usr/local/elastic_node1/bin/elasticsearch-setup-passwords auto
Initiating the setup of passwords for reserved users elastic,apm_system,kibana,logstash_system,beats_system,remote_monitoring_user.
The passwords will be randomly generated and printed to the console.
Please confirm that you would like to continue [y/N]y       #确认随机生成密码

Changed password for user apm_system
PASSWORD apm_system = g7JOAUBi4jJh7PAaXdAN

Changed password for user kibana                            #kibana连接es的用户名及密码
PASSWORD kibana = 0B3EINVicVRbsnyJHk99

Changed password for user logstash_system                   #logstash连接es的用户名及密码
PASSWORD logstash_system = b34Aradp6gSqJMe3SbXK

Changed password for user beats_system                      #beats连接es的用户名及密码
PASSWORD beats_system = EWjwNoDZILqCOCjCEjSc

Changed password for user remote_monitoring_user            #远程监控es的用户名及密码
PASSWORD remote_monitoring_user = N92vKgJ4AHrSfhg5mFUK

Changed password for user elastic                           #应用程序连接es API的用户名及密码
PASSWORD elastic = 26tBktGolYCyZD2pPISW

验证Elasticsearch集群密码

1.使用curl命令验证

[elastic@elastic ~]$ curl -u elastic:26tBktGolYCyZD2pPISW -XGET 'http://192.168.31.215:9201/_cat/nodes?v'
ip             heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
192.168.31.215           29          68   2    0.04    0.17     0.26 dilm      -      elastic_node1
192.168.31.215           22          68   1    0.04    0.17     0.26 dilm      *      elastic_node2
192.168.31.215           28          68   1    0.04    0.17     0.26 dilm      -      elastic_node3

2.使用浏览器验证

部署kibana

以下来使用kibana来连接es并进行安全认证

[root@elastic src]# wget https://artifacts.elastic.co/downloads/kibana/kibana-7.4.0-linux-x86_64.tar.gz
[root@elastic src]# tar xf kibana-7.4.0-linux-x86_64.tar.gz -C /usr/local/
[root@elastic src]# mv /usr/local/kibana-7.4.0-linux-x86_64 /usr/local/kibana
[root@elastic src]# mkdir /usr/local/kibana/run
[root@elastic src]# useradd -s /bin/bash -U kibana
[root@elastic src]# chown -Rf kibana.kibana /usr/local/kibana

修改Kibana配置文件

[root@elastic src]# su - kibana
[kibana@elastic ~]$ cp -rf /usr/local/kibana/config/kibana.yml /usr/local/kibana/config/kibana.yml.default
[kibana@elastic ~]$ vim /usr/local/kibana/config/kibana.ym
[kibana@elastic ~]$ vim /usr/local/kibana/config/kibana.yml
server.port: 5601
server.host: "192.168.31.215"
server.name: "kibana"
elasticsearch.hosts: "http://192.168.31.215:9201"
elasticsearch.preserveHost: true
kibana.index: ".kibana"
pid.file: /usr/local/kibana/run/kibana.pid
logging.verbose: true
elasticsearch.username: "kibana"                        #指定刚使用elasticsearch生成的kibana连接的用户名及密码
elasticsearch.password: "0B3EINVicVRbsnyJHk99"

启动并访问Kibana

1.启动kibana
kibana同样不允许root用户直接运行,所以要使用kibana或普通用户运行

[kibana@elastic ~]$ nohup /usr/local/kibana/bin/kibana &

查看kibana进程

[kibana@elastic ~]$ ps -ef|grep kibana | grep -v grep 
root     19034 17071  0 11:23 pts/1    00:00:00 su - kibana
kibana   19035 19034  0 11:23 pts/1    00:00:00 -bash
kibana   19069 19035 99 11:26 pts/1    00:01:19 /usr/local/kibana/bin/../node/bin/node /usr/local/kibana/bin/../src/cli

2.访问kibana
访问Kibana的地址加端口
http://192.168.31.215:5601
下面中的账号密码为我们刚才使用 elasticsearch-setup-passwords auto 命令生成的 elastic 用户密码

如果是像这个例子中的全新安装情况,Kibana 将询问您是否希望加载一些样本数据。

人已赞赏
0 条回复 A文章作者 M管理员
    暂无讨论,说说你的看法吧