14.10. Grafana Loki¶

Loki is an aggregation system really similar to Prometheus, but instead of reading metrics, it reads logs (via push). Please check the official documentation website for additional information.

14.10.1. Create a system user ¶

# useradd --system --no-create-home --shell /bin/false loki
# useradd --system --no-create-home --shell /bin/false promtail

14.10.2. Download Loki ¶

Download
Extract
Copy to /usr/local/bin
Set ownership and permissions

# cd /tmp
# wget https://github.com/grafana/loki/releases/download/v3.0.0/loki-linux-amd64.zip
# unzip loki-linux-amd64.zip -d .
# mv loki-linux-amd64 loki
# cp loki /usr/local/bin/

14.10.3. Loki configuration file ¶

auth_enabled: false

server:
  http_listen_port: 3100
  grpc_listen_port: 9096

common:
  instance_addr: 127.0.0.1
  path_prefix: /tmp/loki
  storage:
    filesystem:
       chunks_directory: /tmp/loki/chunks
       rules_directory: /tmp/loki/rules
  replication_factor: 1
  ring:
    kvstore:
      store: inmemory

query_range:
  results_cache:
    cache:
      embedded_cache:
        enabled: true
        max_size_mb: 100

schema_config:
  configs:
    - from: 2020-10-24
      store: tsdb
      object_store: filesystem
      schema: v13
      index:
        prefix: index_
        period: 24h

14.10.4. Systemd service file ¶

# Path: /etc/systemd/system/loki.service

[Unit]
Description=Loki service
After=network.target

[Service]
Type=simple
User=loki
ExecStart=/usr/local/bin/loki -config.file /etc/loki/config.yml
# Give a reasonable amount of time for the server to start up/shut down
TimeoutSec = 120
Restart = on-failure
RestartSec = 2

[Install]
WantedBy=multi-user.target

14.10.5. Refresh systemd and restart ¶

# systemctl daemon-reload
# systemctl enable --now loki
# systemctl status loki
# systemctl restart prometheus

14.10.5.1. Check¶

http://ip:3100

14.10.6. Close the 3100 port with Nginx ¶

# Path: /etc/nginx/sites-available/loki.conf

upstream loki {
 server 127.0.0.1:3100; # Loopback
 keepalive 15;
}

server {
 listen 80;
 listen [::]:80;

server_name loki.taler-ops.ch;
root /dev/null;

# LOKI

location / {
 proxy_read_timeout 1800s;
 proxy_connect_timeout 1600s;
 proxy_pass http://loki;
 }

location /ready {
 proxy_pass http://loki;
 proxy_http_version 1.1;
 proxy_set_header Connection "Keep-Alive";
 proxy_set_header Proxy-Connection "Keep-Alive";
 proxy_redirect off;
 auth_basic "off";
 }

}

Enable in Nginx the new virtualhost file

# ln -s /etc/nginx/sites-available/loki.conf /etc/nginx/sites-enabled/loki.conf
# nginx -t
# systemctl reload nginx

14.10.7. Edit the loki configuration file ¶

#Path: /etc/loki/config.yml
server:
  http_listen_port: 3100
  grpc_listen_port: 9096
  # Add this to close the 3100 port
  http_listen_address: 127.0.0.1

14.10.8. Refresh systemd and restart ¶

# systemctl restart nginx
# systemctl restart loki
# systemctl restart prometheus

14.10.8.1. Check¶

Check that the 3100 port is publicly closed by typing in your Web browser http://ip:3100. At the same time change through the Grafana control panel, your loki data source server URL field, so it can connect now through the specified subdomain in the nginx virtualhost file (in our test case loki.taler-ops.ch).

14.10.9. Grafana control panel ¶

You can now to go the Grafana control panel and easily add the new Loki data source.

14.9. Prometheus postgres-exporter

14.11. Grafana Promtail

GNU Taler

14.10. Grafana Loki¶

14.10.5.1. Check¶

14.10.8.1. Check¶