grafana Archives - GeralexGR

Teamcity can be integrated with Grafana in order to gather statistics about builds and agents activity. In order to gather those teamcity metrics you will need a prometheus installation and then a configuration for the teamcity source.

When you connect teamcity as a target on prometheus you have two options regarding an https secure communication. Either you trust the not verified certificate and place that in the configuration file, or you should enable tls communication with certificates between teamcity and prometheus.

If you do not set one of the two options on prometheus you might end up with the issue:

Get /app/metrics remote error: tls handshake failure

The first option as discussed would be to trust the insecure certificate using https.

tls_config:
      insecure_skip_verify: true

The second option would be to communicate through certificates. First create a new folder inside your Prometheus installation and store the certificates for the teamcity server. Then you will need to point the files in the configuration.

    tls_config:
       ca_file:  /var/prometheus/ssl/signalocean.com.crt

The detailed options for tls_config in prometheus can be found below.

Configuration | Prometheus

When using InfluxDB along with grafana you can get some internal statistics for the usage with a build-in internal database. In some cases when you try to create the grafana connection with the influxdb you can notice the error that is mentioned below.

InfluxDB Error: error authorizing query: user not authorized to execute statement ‘SHOW RETENTION POLICIES ON _internal’, requires READ on _internal

This error could be either because of permissions for the user or when the monitoring is not enabled. In order to verify you can login in influxdb

influx -username "user" -password "password"

and then list all available databases.

show databases

If the database _internal is not listed most probably you will not have the internal statistics enabled. You can read more about internal measurements based on your influxdb version in the documentation.

https://docs.influxdata.com/platform/monitoring/influxdata-platform/tools/measurements-internal/

InfluxData does not recommend using the _internal database in a production cluster. It creates unnecessary overhead, particularly for busy clusters, that can overload an already loaded cluster. Metrics stored in the _internal database primarily measure workload performance and should only be tested in non-production environments.

In order to verify if internal metrics are enabled you should go in influxdb configuration file

cat /etc/influxdb/influxdb.conf

and check the monitor section

##
## Controls the system self-monitoring, statistics and diagnostics.
##
## The internal database for monitoring data is created automatically if
## if it does not already exist. The target retention within this database
## is called 'monitor' and is also created with a retention period of 7 days
## and a replication factor of 1, if it does not exist. In all cases the
## this retention policy is configured as the default for the database.

You can then comment out the 3 lines that are showed in order to enable internal monitoring.

Finally after doing so you can go in grafana and create a new connection with the influxdb using the _internal database and the user.

Tag: grafana

Get /app/metrics remote error: tls handshake failure – Promotheus

InfluxDB Error: error authorizing query: user not authorized to execute statement ‘SHOW RETENTION POLICIES ON _internal’, requires READ on _internal