How to create subdomain metrics-ox

Apps & App Center
#1

Continuing the discussion from Best Practice: Monitor additional services with the UCS-Dashboard:

first, thanks for the article!
I’m stuck in understanding how the subdomain metrics-ox is generated what should go into custom-targets.json
the article reads like there is some automation but I don’t get it…
thanks a lot in advance!

#2

Hey rumprr,

if you have an exporter emitting metrics your Prometheus/UCS dashboard database installation should be able to reach it, there is no definitive need to make this available via subdomain.
The article makes use of the Apache Proxy Pass to make the metrics available via LDAP auth on port 443, to enable this Apache has to be configured accordingly with the given file.

The custom target is defined as follows: If your metrics are reachable under https://dir-node.mydomain.intranet/metrics-ox/metrics/ (scheme: https://A/B/C) the target should look like https://B.A, e.g. metrics-ox.dir-node.mydomain.intranet

You can always look into the defined targets from the Prometheus Web UI too.

#3

thanks for your reply.
what I’m trying to do is: my UCS runs as a Proxmox VM and I’d like to scrape the pve-exporter running outside of UCS with UCS prometheus (cause I’m to lazy to setup a new Prometheus/Grafana instance since you already provide a fully configured one).
But the URL that needs to be scraped is http://192.168.50.2:9221/pve (so it’s running on a custom Port and needs the /pve in the url.
Any hints how to tune custom-targets.json to scrape that url?
thanks a lot in advance

#4

Ok, got your point. Firstly you can look up your prometheus config directly to see how these targets are configured: https://your-prometheus-server/metrics-prometheus/config
The included config has a relabel in place:

  relabel_configs:
  - source_labels: [__address__]
    separator: ;
    regex: ([^\.]+)\..*
    target_label: __metrics_path__
    replacement: /$1/metrics/
    action: replace
  - source_labels: [__address__]
    separator: ;
    regex: '[^\.]+\.(.*)'
    target_label: __address__
    replacement: $1
    action: replace
  - source_labels: [__address__]
    separator: ;
    regex: (.*)
    target_label: instance
    replacement: $1
    action: replace

This is applied to your custom targets too, so enable scraping of your PVE metrics you have to define the URL in a way that the correct path is evaluated after this relabeling.
If this cannot be done a feature request against the app might be appropriate.

You can check the targets definition via the prometheus web UI too: https://your-prometheus-server/metrics-prometheus/targets

Best regards
Jan-Luca

1 Like
#5

If have updated the article to reflect the relabel of targets.

1 Like
Mastodon