prometheus gets 400 bad request scraping metrics from nextcloud

[BinaryMan32]

Describe your Issue

In prometheus I can see the service monitor tries to get metrics from both nextcloud and nextcloud-metrics.
Scraping nextcloud-metrics works but nextcloud fails with 400 bad request.

Logs and Errors

The nextcloud pod logs look pretty repetitive with messages similar to the below cycling.
I copied from lens with "show resource names" selected so the container name is on the beginning of each line.

[nextcloud-nginx]10.42.13.45 - admin [08/Mar/2026:19:00:44 +0000] "GET /ocs/v2.php/apps/serverinfo/api/v1/info?format=json&skipApps=true&skipUpdate=true HTTP/1.1" 200 1705 "-" "nextcloud-exporter/0.8.0" "-"
[nextcloud]127.0.0.1 -  08/Mar/2026:19:00:50 +0000 "GET /status.php" 200
[nextcloud]127.0.0.1 -  08/Mar/2026:19:00:50 +0000 "GET /status.php" 200
[nextcloud-nginx]10.42.5.1 - - [08/Mar/2026:19:00:50 +0000] "GET /status.php HTTP/1.1" 200 182 "-" "kube-probe/1.35" "-"
[nextcloud-nginx]10.42.5.1 - - [08/Mar/2026:19:00:50 +0000] "GET /status.php HTTP/1.1" 200 182 "-" "kube-probe/1.35" "-"
[nextcloud]127.0.0.1 -  08/Mar/2026:19:00:56 +0000 "GET /index.php" 400
[nextcloud-nginx]10.42.15.202 - - [08/Mar/2026:19:00:56 +0000] "GET /metrics HTTP/1.1" 400 13443 "-" "Prometheus/3.10.0" "-"
[nextcloud]127.0.0.1 -  08/Mar/2026:19:01:00 +0000 "GET /status.php" 200
[nextcloud]127.0.0.1 -  08/Mar/2026:19:01:00 +0000 "GET /status.php" 200
[nextcloud-nginx]10.42.5.1 - - [08/Mar/2026:19:01:00 +0000] "GET /status.php HTTP/1.1" 200 182 "-" "kube-probe/1.35" "-"
[nextcloud-nginx]10.42.5.1 - - [08/Mar/2026:19:01:00 +0000] "GET /status.php HTTP/1.1" 200 182 "-" "kube-probe/1.35" "-"
[nextcloud]127.0.0.1 -  08/Mar/2026:19:01:10 +0000 "GET /status.php" 200
[nextcloud]127.0.0.1 -  08/Mar/2026:19:01:10 +0000 "GET /status.php" 200
[nextcloud-nginx]10.42.5.1 - - [08/Mar/2026:19:01:10 +0000] "GET /status.php HTTP/1.1" 200 182 "-" "kube-probe/1.35" "-"
[nextcloud-nginx]10.42.5.1 - - [08/Mar/2026:19:01:10 +0000] "GET /status.php HTTP/1.1" 200 182 "-" "kube-probe/1.35" "-"
[nextcloud]127.0.0.1 - admin 08/Mar/2026:19:01:13 +0000 "GET /ocs/v2.php" 200

Describe your Environment

• Kubernetes distribution: k3s 1.35.2
• Helm Version (or App that manages helm): ArgoCD 3.3.2 using helm 3.19.4
• Helm Chart Version: 9.0.1
• values.yaml:

# https://github.com/nextcloud/helm/blob/main/charts/nextcloud/values.yaml
# use nginx to serve files instead of apache
image:
  flavor: fpm-alpine
nginx:
  enabled: true
ingress:
  annotations:
    nginx.ingress.kubernetes.io/proxy-body-size: 2g
    nginx.ingress.kubernetes.io/proxy-read-timeout: "180"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "180"
  enabled: true
  className: public
  annotations:
    nginx.org/websocket-services: nextcloud
    cert-manager.io/cluster-issuer: "letsencrypt-prod"
  tls:
    - secretName: nextcloud-tls
      hosts:
      - REDACTED
nextcloud:
  host: REDACTED
  configs:
    protocol.config.php: |-
      <?php
      $CONFIG = array (
        'overwriteprotocol' => 'https'
      );
  existingSecret:
    enabled: true
    secretName: nextcloud-admin
    usernameKey: username
    passwordKey: password
internalDatabase:
  enabled: false
externalDatabase:
  enabled: true
  type: postgresql
  existingSecret:
    enabled: true
    secretName: nextcloud-db-app
    usernameKey: username
    passwordKey: password
    hostKey: host
    databaseKey: dbname
cronjob:
  enabled: true
  type: sidecar
persistence:
  enabled: true
  storageClass: longhorn
  nextcloudData:
    enabled: true
    storageClass: mercury-nfs
startupProbe:
  enabled: true
metrics:
  enabled: true
prometheus:
  serviceMonitor:
    enabled: true
  rules:
    enabled: true

Additional context, if any

The below seem like the most "unusual" things about my setup. I'm wondering whether I should still be using this configuration:

image:
  flavor: fpm-alpine
nginx:
  enabled: true

Can you clarify whether after #834 I should be able to remove the below and scrape all metrics directly from nextcloud? Of course I'd need to solve the above issue first or I would have no metrics at all.

metrics:
  enabled: true

[wrenix]

There was an bug in 9.0.1 please update to 9.0.3 (allowed host of scraping metrics was not configurated correct).

Additional you current need also to set correct trusted_domains see #833

Your Additinal Context, i do not complete understand.
Do you want to have a switch for the nextcloud new/internal metrics?

[BinaryMan32]

Your Additinal Context, i do not complete understand.
Do you want to have a switch for the nextcloud new/internal metrics?

I guess I was asking if nextcloud itself provides all of the metrics that I used to get from the nextcloud-metrics service, so it's ok to set

metrics:
  enabled: false

Or does nextcloud provide different metrics than nextcloud-metrics, so that I should continue to use

metrics:
  enabled: true

[BinaryMan32]

@wrenix thanks for the workaround suggestion, it's working for me.

Now that I can see which metrics are scraped from the nextcloud service, it seems like nextcloud and nextcloud-metrics provide different metrics so it's best to leave metrics: enabled: true from my previous question.

Thanks for your help. I'll close this since you already have other issues tracking this problem.

[rich0]

Additional you current need also to set correct trusted_domains see #833

What is expected to be set here? The IP of the prometheus pod? Is it safe to just trust the entire k8s cluster CIDR, which would include ingresses/etc? I wouldn't think that prometheus would be scraping from a controlled IP/DNS like a service.

[wrenix]

• In trusted_domains the IP of the nextcloud pod
• in nextcloud.openmetrics.allowedClients the ip of prometheus (or a iprange of possible prometheus pods)

[rich0]

• In trusted_domains the IP of the nextcloud pod

  • in nextcloud.openmetrics.allowedClients the ip of prometheus (or a iprange of possible prometheus pods)

Thanks. I'm thinking that either could be anywhere in the cluster CIDR, so it would make sense to just put the cluster CIDR in both. I don't think that creates any vulnerabilities as I don't run anything untrusted in k8s, but it would include stuff like Traefik (which all the nextcloud connections come over anyway).

[wrenix]

Is it possible to set on trusted_domains a cidr?

That are the moments were i thing about implementing networkPolicies here too).