Grafana Promtail

15.12. Grafana Promtail#

Official documentation website can be found here.

Promtail can read two different types of logs. Regular log files, or information from the Systemd journald.

15.12.1. Download promtail#

  • Download

  • Extract

  • Copy to /usr/bin

  • Set ownership and permissions

# cd /tmp
# wget https://github.com/grafana/loki/releases/download/v3.0.0/promtail-linux-amd64.zip
# unzip promtail-linux-amd64.zip -d .
# mv promtail-linux-amd64 promtail
# cp promtail /usr/local/bin

15.12.2. Create system user#

# useradd --system --no-create-home --shell /bin/false promtail
# usermod -G adm -a promtail # Add the "promtail" user, to the "adm" group if necessary.

15.12.3. Promtail configuration file#

# Path: /etc/promtail/config.yml

server:
  http_listen_port: 9080
  grpc_listen_port: 0

positions:
  filename: /tmp/positions.yaml

clients:
  - url: http://localhost:3100/loki/api/v1/push

# Example for log file

scrape_configs:
- job_name: system
  static_configs:
  - targets:
      - localhost
    labels:
      job: nginx
      __path__: /var/log/nginx/*log # List here your log files

# Example for Systemd journald

scrape_configs:
- job_name: journal
  journal:
   json: true
   path: /var/log/journal
   labels:
     job: systemd-journal
 relabel_configs:
   - source_labels: ['__journal__systemd_unit']
     target_label: 'unit'

15.12.4. Promtail systemd service file#

# Path: /etc/systemd/system/promtail.service

[Unit]
Description=Promtail service
After=network.target

[Service]
Type=simple
User=promtail
ExecStart=/usr/local/bin/promtail -config.file /etc/promtail/config.yml
# Give a reasonable amount of time for promtail to start up/shut down
TimeoutSec = 60
Restart = on-failure
RestartSec = 2

[Install]
WantedBy=multi-user.target

15.12.4.1. Start and enable Promtail#

# systemctl enable --now promtail.service
# systemctl status promtail.service

15.12.5. Promtail temporary files#

# cd /tmp
# touch positions.yaml
# chown promtail:promtail /tmp/positions.yaml

15.12.6. Grafana control panel#

To check if Promtail is reading properly either your log files or the systemd journald units, you can click on the “Explore” section in the Grafana control panel, choose the right Loki connector, choose your desired log file or journald unit, and execute the query. If you see this is working (you can see the chunks of the big log files), you can convert this temporary query into a real Grafana “dashboard”, to continue working later with additional filtering options.