Comment on page
Prometheus and Grafana Setup
Setup Prometheus and Grafana to view app metrics
Grainite provides a counters/metrics endpoint that includes various counters and gauges to provide insight into applications, as well as the server itself. This page goes over setting up Prometheus to access these metrics in Grafana.
Secured Cluster/Server
Unsecured (w/o TLS)
Note: The following instructions apply to Grainite clusters or servers with TLS enabled.
- 2.Copy the new client certificates to the config directory. Example:
cp ~/.gxscerts/<cluster-name>/client/<user-name>/* /etc/config/gxs-client-secretwhere<cluster-name>is the name of the current cluster. - 3.Grainite exposes the counters at
<server_ip>:5064/export-dashboard. To have Prometheus use this, set up your Prometheus configuration file (prometheus.yml) withschemeandtls_configattributes like in the following example:
prometheus.yml
...
scrape_configs:
- job_name: 'grainite'
scrape_interval: 5s
metrics_path: '/export-dashboard'
params:
aggregate: [1]
scheme: https
tls_config:
ca_file: /etc/config/gxs-client-secret/grainite_ca.crt
key_file: /etc/config/gxs-client-secret/grainite_client.<user-name>.key
cert_file: /etc/config/gxs-client-secret/grainite_client.<user-name>.crt
insecure_skip_verify: true
static_configs:
- targets: [ '<server-IP>:5064' ]
...
Where:
<server-IP>is the IP address of the cluster or server. You can get the IP address by following the instructions linked here.<user-name>corresponds to the user name specified when generating the client certificate.
- 4.After adding the above to the Prometheus configuration file, restart Prometheus for it to pick up the updated targets. Once Prometheus is restarted, confirm that "grainite" is shown as a target for Prometheus:
Prometheus Targets Page
Note: The following instructions apply to Grainite clusters or servers without TLS enabled.
Grainite exposes the counters at
<server_ip>:5064/export-dashboard. To have Prometheus use this, add the following to your Prometheus configuration file (prometheus.yml):prometheus.yml
...
scrape_configs:
- job_name: 'grainite'
scrape_interval: 5s
metrics_path: '/export-dashboard'
params:
aggregate: [1]
static_configs:
- targets: ['<server-IP>:5064']
...
Where:
<server-IP>is the IP address of the server.
After adding the above to the Prometheus configuration file, restart Prometheus for it to pick up the updated targets. Once Prometheus is restarted, confirm that "grainite" is shown as a target for Prometheus:
Prometheus Targets Page
Add Prometheus as a data source in Grafana by going to
Settings -> Data sources -> Add data source and selecting Prometheus, filling in the URL, and clicking on Save & test.Now, app counters can be accessed by using Prometheus as a data source and searching for counters/gauges beginning with
gxsapp. The format for app metrics is the following:- 1.Counters:
gxsapp_<counter name>_total - 2.Gauges:
gxsapp_<gauge name>_current
Last modified 3mo ago