I can’t get Prometheus to pick up on my cAdvisor metrics on DOKS. It shows as a successful scrape on the Prometheus dashboard, but I can’t get any container_* metrics to show up in the query. Any ideas?
Here’s my job config for cadvisor:
- job_name: kubernetes-cadvisor honor_timestamps: true scrape_interval: 15s scrape_timeout: 10s metrics_path: /metrics scheme: https bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token tls_config: ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt insecure_skip_verify: true kubernetes_sd_configs: - role: node relabel_configs: - separator: ; regex: __meta_kubernetes_node_label_(.+) replacement: $1 action: labelmap - source_labels: [__address__] separator: ; regex: ([^:]+)(?::\d+)? target_label: __address__ replacement: $1:10250 action: replace - separator: ; regex: (.*) target_label: __metrics_path__ replacement: /metrics/cadvisor action: replace metric_relabel_configs: - source_labels: [namespace] separator: ; regex: ^$ replacement: $1 action: drop - source_labels: [pod_name] separator: ; regex: ^$ replacement: $1 action: drop
I’ve also tried the metric path as /metrics/cadvisor like some issues of Github have suggested.
This config was taken from the tutorial: https://www.digitalocean.com/community/tutorials/how-to-set-up-a-kubernetes-monitoring-stack-with-prometheus-grafana-and-alertmanager-on-digitalocean#step-6-—-configuring-the-monitoring-stack-optional
This textbox defaults to using Markdown to format your answer.
You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!
These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.
container_namewere removed from cadvisor on 1.16. The
pod_nameis dropping the metrics. Try replace it with just
try using /metrics.It might work.
You can debug the target by going to the UI > Status > Targets. See if the target address is correct.