atextcrawler/doc/source/elasticsearch.md

120 lines
3 KiB
Markdown
Raw Permalink Normal View History

2021-11-29 09:16:31 +00:00
# Howto elasticsearch
## Prerequisites
On the host (virtualization host) we need:
```
# cat /etc/sysctl.d/virtual_memory.conf
vm.max_map_count=262144
# sysctl -p /etc/sysctl.d/virtual_memory.conf
```
If this cannot be done, change this file after installing or upgrading elasticsearch:
```
/usr/lib/sysctl.d/elasticsearch.conf
```
## Setup
### Install package
In general, see the [elaticsearch reference](https://www.elastic.co/guide/en/elasticsearch/reference/7.10/deb.html).
We do a manual install. If you configure the apt repo instead, also think about setting
`RESTART_ON_UPGRADE=true` in `/etc/default/elasticsearch`.
```
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.15.2-amd64.deb
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.15.2-amd64.deb.sha512
shasum -a 512 -c elasticsearch-7.15.2-amd64.deb.sha512
dpkg -i elasticsearch-7.15.2-amd64.deb
systemctl daemon-reload
systemctl enable elasticsearch.service
systemctl start elasticsearch.service
```
First test:
```
http -j GET 127.0.0.1:9200/
```
### Storage
```
systemctl stop elasticsearch.service
mv /var/lib/elasticsearch/ /srv/
systemctl start elasticsearch.service
```
Edit /etc/elasticsearch/elasticsearch.yml
```
cluster.name: org.a-text.search
node.name: atext1
path.data: /srv/elasticsearch
path.logs: /var/log/elasticsearch
discovery.seed_hosts: ["atext1.multiname.org"]
xpack.security.enabled: true
xpack.security.authc.api_key.enabled: true
```
```
systemctl restart elasticsearch
```
The logfile now is at
```
/var/log/elasticsearch/org.a-text.search.log
```
### Setup passwords
Setup passwords:
```
# /usr/share/elasticsearch/bin/elasticsearch-setup-passwords auto
Initiating the setup of passwords for reserved users elastic,apm_system,kibana,kibana_system,logstash_system,beats_system,remote_monitoring_user.
The passwords will be randomly generated and printed to the console.
Please confirm that you would like to continue [y/N]y
```
Copy output to /etc/elasticsearch/passwords and
```
chmod 400 /etc/elasticsearch/passwords
```
Check login as user elastic:
```
http --auth elastic:************** -j GET http://127.0.0.1:9200/
```
### Memory limitation
To limit memory usage
```
mkdir /etc/systemd/system/elasticsearch.service.d
cat >/etc/systemd/system/elasticsearch.service.d/override.conf <<EOF
[Service]
LimitMEMLOCK=8G
systemctl stop elasticsearch
systemctl daemon-reload
systemctl start elasticsearch
EOF
```
and restart the service.
## Usage
Some useful requests:
### List indices
```
http --auth elastic:$PASS -j GET http://127.0.0.1:9200/_cat/indices
```
### Health
```
http --auth elastic:$PASS -j GET http://127.0.0.1:9200/_cat/health
```
### Node attributes
```
http --auth elastic:$PASS -j GET http://127.0.0.1:9200/_cat/nodeattrs
```
### Create API key
```
http --auth elastic:$PASS -j POST http://127.0.0.1:9200/_security/api_key name=anarchism role_descriptors:='{"anarchism": {"cluster": [], "index": [{"names": ["anarchism_*"], "privileges": ["all"]}]}}'
```