0g-chain/docs/Telemetry.md

87 lines
2.5 KiB
Markdown
Raw Permalink Normal View History

# Telemetry
[example metrics emitted by Kava application](./example-prometheus-metrics.txt)
## Enabling Kava Metric Telemetry
To enable the kava app to emit telemetry during operation, update the relevant config values to enable metrics:
`config.toml`
```toml
[instrumentation]
# When true, Prometheus metrics are served under /metrics on
# PrometheusListenAddr.
# Check out the documentation for the list of available metrics.
prometheus = true
# Address to listen for Prometheus collector(s) connections
prometheus_listen_addr = ":8888"
```
`app.toml`
```toml
[telemetry]
# Prefixed with keys to separate services.
service-name = ""
# Enabled enables the application telemetry functionality. When enabled,
# an in-memory sink is also enabled by default. Operators may also enabled
# other sinks such as Prometheus.
enabled = true
```
Then restart the service with the updated settings
## Running local prometheus collector and grafana services
To collect app metrics and visualize them locally, you can run the prometheus collector and grafana services with docker compose from the repo root directory (after updating `config.toml` and `app.toml` as detailed above)
```bash
docker compose -f prometheus.docker-compose.yml up
```
Navigate to localhost:3000 to view the grafana unix
Login with `admin` as the username and `admin` as the password
Hook up grafana to the local prometheus collector by navigating to `http://localhost:3000/connections/datasources/new`, selecting prometheus, entering `http://prometheus:9090` for the url, and clicking `Save & test` at the bottom of the screen
See [grafana docs](https://grafana.com/docs/grafana/latest/dashboards/) for information on how to construct queries and build dashboards
### Collecting from local host
Update [prometheus config](../prometheus.yml) to collect metrics from your local source, where the port matches the value set for `config.toml/instrumentation.prometheus_listen_addr`
```yaml
metrics_path: /
static_configs:
- targets:
- docker-host:8888
```
### Collecting from remote host
Update the kava config on the host and restart using the instructions from `Enabling Kava Metric Emission`
Install [ngrok](https://ngrok.com/download) on the remote host
Run ngrok on the remote host to forward the prometheus metric port
```bash
ngrok http 8888
```
```yaml
scrape_configs:
- job_name: proxy
scheme: https
metrics_path: /
static_configs:
- targets:
- 4efb-18-207-102-158.ngrok-free.app
```