12.5. Installation of Prometheus

12.5.1. Download Prometheus

  • Download

  • Extract

  • Copy files to /usr/local/bin

  • Set ownership and permissions

$ wget https://github.com/prometheus/prometheus/releases/download/v2.52.0/prometheus-2.52.0.linux-amd64.tar.gz -O /tmp/prometheus.tar.gz
$ cd /tmp
$ tar -xvzf prometheus.tar.gz
$ mkdir /etc/prometheus
$ mkdir /var/lib/prometheus

$ chown prometheus:prometheus /etc/prometheus
$ chown prometheus:prometheus /var/lib/prometheus/
$ cp prometheus promtool /usr/local/bin/
$ chown prometheus:prometheus /usr/local/bin/prometheus
$ chown prometheus:prometheus /usr/local/bin/promtool
$ cp -R consoles console_libraries /etc/prometheus/
$ chown -R prometheus:prometheus /etc/prometheus/consoles
$ chown -R prometheus:prometheus /etc/prometheus/console_libraries
$ chown prometheus:prometheus /etc/prometheus/prometheus.yml

12.5.2. Create a system user

$ useradd --no-create-home --shell /bin/false prometheus

12.5.3. Configuring Prometheus

The main configuration file of Prometheus is in the YAML format, and it is located on /etc/prometheus/prometheus.yml. Please be aware YAML, is a very sensitive format, where white spaces matter.

Note

If you find that your prometheus service is not working properly, please always check the prometheus.yml configuration file, or the systemd prometheus.service file, as in most of occasions, these are the main source of errors.

12.5.3.1. Prometheus main configuration file

# Path: /etc/prometheus/prometheus.yml

global:
   scrape_interval:     15s
   evaluation_interval: 15s

scrape_configs:
  - job_name: prometheus
     static_configs:
     - targets: ['localhost:9090']

12.5.4. Prometheus Systemd service file

# Path: /etc/systemd/system/prometheus.service

[Unit]
Description=Prometheus
Wants=network-online.target
After=network-online.target

[Service]
User=prometheus
Group=prometheus
Type=simple
ExecStart=/usr/local/bin/prometheus \
   --config.file /etc/prometheus/prometheus.yml \
   --storage.tsdb.path /var/lib/prometheus/ \
   --web.console.templates=/etc/prometheus/consoles \
   --web.console.libraries=/etc/prometheus/console_libraries

[Install]
WantedBy=multi-user.target

12.5.4.1. Refresh systemd file and restart Prometheus

# systemctl daemon-reload
# systemctl start prometheus
# systemctl status prometheus
# systemctl enable prometheus.service

12.5.4.2. Check

The prometheus service should be listening on the 9090 port, check this with your Internet browser, by typing http://ip:9090.

Note

If you find any problem to start the prometheus service, use the “journalctl -u prometheus” command to find any errors. Always check twice the prometheus.yml and the prometheus.service files.

12.5.5. Close the 9090 port with Nginx

Note

This is optional, but increases security, by closing your server 9090 listening port.

In order to reverse proxy the Prometheus default port (9090) to loopback with NGINX, you need to undertake 3 actions:

  • Modify the current systemd service file

  • Create a new Nginx virtualhost file to loopback port 9090 to prometheus

  • Restart the Prometheus systemd service file

12.5.5.1. Modify the Prometheus systemd service file

ExecStart=/usr/local/bin/prometheus \
   --config.file=/etc/prometheus/prometheus.yml \
   --storage.tsdb.path=/var/lib/prometheus \
   --web.console.templates=/etc/prometheus/consoles \
   --web.console.libraries=/etc/prometheus/console_libraries \
   --web.listen-address=127.0.0.1:9090 \ # Add this line ****
   --web.external-url=/prometheus/ # Add this line ****

12.5.5.2. Nginx configuration

# Path: /etc/nginx/sites-available/prometheus.conf

server {
   listen 80;
   listen [::]:80;
   server_name name.domain.tld;
   root /dev/null;

   location /prometheus/ {
      proxy_pass http://localhost:9090;
   }
}

## Create the Nginx symbolic link to enable the virtualhost file
# ln -s /etc/nginx/sites-available/prometheus.conf /etc/nginx/sites-enabled/prometheus
# nginx -t
# systemctl reload nginx

12.5.5.3. Restart the Prometheus service

# systemctl daemon-reload
# systemctl restart prometheus

12.5.5.4. Check

If you have done all steps explained above you won’t be able to reach Prometheus anymore from your web browser (http://ip:9090). To reach the prometheus program from your web browser you will have to access through the subdomain or domain name, that you specified on the nginx virtualhost file.

12.5.6. Grafana control panel (GUI)

To add the server prometheus connector or datasource, to the Taler grafana server go to “Connections->Add new connection” and search for “Prometheus” in the search text field. Once you have found and selected the Prometheus connection type, just press the blue button “Add new data source”.The main 2 fields required to specify are the name of the connector, and the URL. Then just press “Save and test”, and you are done.

Note

Please find the main official documentation in Prometheus official documentation