What configmap let's me edit the scrape_configs of the prometheus app on kubernetes

Question

I have installed the one-click app “Prometheus Kubernetes”, with it came an ever so nice stack of software that I am very thankful for, however I have some additional metrics that I would like to be scraped. I am however lost in all of the configmappers that the app installs. I have so far searched the configmaps:

configmap/prometheus-operator-prometheus
configmap/prometheus-prometheus-operator-prometheus-rulefiles-0
configmap/prometheus-operator-prometheus
configmap/prometheus-operator-pod-totoal
configmap/prometheus-operator-pod-total

for the “scrape_configs” string, but could not find any.

Answer 1

RobinDorstijn October 21, 2020

The trick is that this app installs prometheus-operator, which is a separate project. It’s github account provides documentation for the project, in which you can find the description for adding scrape configs.

Reply Report

cpalmer03 December 25, 2020

Robin,

The documnetation linked says after creating the secrets,

“Finally, reference this additional configuration in your prometheus.yaml CRD.”

When I installed monitoring with one click app, I never had a prometheus.yaml file to begin with to apply? So how would I apply these additionalScrapeConfigs?
Reply Report

Answer 2

cpalmer03 December 25, 2020

Robin,

The documnetation linked says after creating the secrets,

“Finally, reference this additional configuration in your prometheus.yaml CRD.”

When I installed monitoring with one click app, I never had a prometheus.yaml file to begin with to apply? So how would I apply these additionalScrapeConfigs?
Reply Report

Answer 3

RobinDorstijn October 21, 2020

Show answer This answer has been marked as resolved by RobinDorstijn.

I made a little bash script in order to search the configmaps:

confs=$(
    kubectl get configmaps -n prometheus-operator |
    grep "prometheus" |
    awk -F " " '{print $1}'
)

for conf_name in $confs
do
    echo "Looking at $conf_name"
    kubectl get -n prometheus-operator configmap/$conf_name -o yaml | grep scrape
done

However this gave no result, so clearly the scrape_configs is in a configmap.

Reply Report

RobinDorstijn October 21, 2020
For one reason or another, it was chosen to store the config file as a secret: secret/prometheus-prometheus-operator-prometheus. To few it run:

$ k get -n prometheus-operator secret/prometheus-prometheus-operator-prometheus -o json | jq -r '.data["prometheus.yaml.gz"]'| base64 -d | gunzip
Reply Report
- RobinDorstijn October 21, 2020
  
  But apparently this is not where the change needs to be made, as a config reloader is available. However it is not clear how this reloader can be set with a new config.
  
  Reply Report

Answer 4

RobinDorstijn October 21, 2020
For one reason or another, it was chosen to store the config file as a secret: secret/prometheus-prometheus-operator-prometheus. To few it run:

$ k get -n prometheus-operator secret/prometheus-prometheus-operator-prometheus -o json | jq -r '.data["prometheus.yaml.gz"]'| base64 -d | gunzip
Reply Report
- RobinDorstijn October 21, 2020
  
  But apparently this is not where the change needs to be made, as a config reloader is available. However it is not clear how this reloader can be set with a new config.
  
  Reply Report

Related

Question

What configmap let's me edit the scrape_configs of the prometheus app on kubernetes

Related

Leave a comment

Looking for something else?

Sign up for our newsletter

Related

Question

What configmap let's me edit the scrape_configs of the prometheus app on kubernetes

Related

Leave a comment

Related

question

Error when approving certificate following Recommended steps to Secure Kubernetes cluster

question

Is possible to connect a DO K8s cluster with a DO Droplet private network?

question

Confirmation email from Wordpress wp-admin password change never arrives

question

Digital Occean Cloud server has user interface logging ?

Looking for something else?