Skip to content

OCPBUGS-60905, OCPBUGS-51196, OCPBUGS-51181: fix(metrics): metrics export through service works now #268

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Cali0707 wants to merge 6 commits into
base: master
Choose a base branch
from
Open

OCPBUGS-60905, OCPBUGS-51196, OCPBUGS-51181: fix(metrics): metrics export through service works now #268

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
58d5e7e
fix(metrics): align service selector with deployment
Cali0707 Jul 21, 2026
faf007d
feat(metrics): serve metrics over TLS using cluster security profile
Cali0707 Jul 21, 2026
6cade1a
fix(metrics): add static Prometheus RBAC for metrics scraping
Cali0707 Jul 21, 2026
474d091
test(metrics): verify service monitor works with openshift-monitoring…
Cali0707 Jul 21, 2026
da505c9
fix(metrics): operator CSV is labelled to indicate metrics
Cali0707 Jul 21, 2026
9cd57b9
chore: wrap errors properly with %w
Cali0707 Jul 21, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
2 changes: 1 addition & 1 deletion Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ func-test: deploy
unit-test:
@echo "Executing unit tests"
go clean -testcache
go test -v ./controllers/...
go test -v ./controllers/... ./pkg/...

build: $(SOURCES)
go build $(GOBUILDFLAGS) -ldflags="$(GOLDFLAGS)" -o ./update-service-operator ./
Expand Down
16 changes: 16 additions & 0 deletions bundle/manifests/prometheus-k8s_rbac.authorization.k8s.io_v1_role.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
creationTimestamp: null
name: prometheus-k8s
rules:
- apiGroups:
- ""
resources:
- services
- endpoints
- pods
verbs:
- get
- list
- watch
13 changes: 13 additions & 0 deletions bundle/manifests/prometheus-k8s_rbac.authorization.k8s.io_v1_rolebinding.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
creationTimestamp: null
name: prometheus-k8s
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: prometheus-k8s
subjects:
- kind: ServiceAccount
name: prometheus-k8s
namespace: openshift-monitoring
17 changes: 16 additions & 1 deletion bundle/manifests/update-service-operator.clusterserviceversion.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,12 +16,14 @@ metadata:
}
]
capabilities: Basic Install
createdAt: "2025-05-14T21:01:52Z"
console.openshift.io/operator-monitoring-default: "true"

@wking wking Aug 17, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are there docs for this one? It sounds a lot like operatorframework.io/cluster-monitoring, but I haven't found anything that looks like docs. I did find this code, but that also sounds a lot like the operatorframework.io/cluster-monitoring semantics. Having a commit message that says "hey, reviewer and future devs, these changes are the appropiate changes to make because..." would help convince me ;) But commit message or not, I'd like to understand what this annotation is about, and I haven't been able to turn up docs around it on my own :/

createdAt: "2026-07-21T18:06:48Z"
description: Creates and maintains an OpenShift Update Service instance
kubernetes.io/description: "This OpenShift Update Service operator Deployment
reconciles local UpdateServices into more fundamental Kubernetes\nand OpenShift
resources like Cincinnati Deployments and Routes, and it reports the status
of those components in \nthe UpdateService status.\n"
operatorframework.io/cluster-monitoring: "true"

@wking wking Aug 17, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From docs:

The operatorframework.io/cluster-monitoring=true annotation. When this annotation is set to true, the OpenShift Console will update the namespace that the operator is being deployed to with the openshift.io/cluster-monitoring=true label. When this annotation is present, the UI will update the OpenShift Monitoring Prometheus Operator ServiceAccount with the appropriate RBAC privileges for the given namespace as well, allowing operators to be scraped by the OpenShift Monitoring Prometheus Operator.

I'm not sure if that's still current, or how it would work on clusters where the console is not installed. but it suggests that 6cade1a might not be necessary.

operatorframework.io/suggested-namespace: openshift-update-service
operators.operatorframework.io/builder: operator-sdk-v1.31.0-ocp
operators.operatorframework.io/project_layout: go.kubebuilder.io/v3
Expand Down Expand Up @@ -124,6 +126,7 @@ spec:
- apiGroups:
- config.openshift.io
resources:
- apiservers
- images
verbs:
- get
Expand Down Expand Up @@ -192,8 +195,20 @@ spec:
image: controller:latest
imagePullPolicy: Always
name: updateservice-operator
ports:
- containerPort: 8443
name: https
protocol: TCP
resources: {}
volumeMounts:
- mountPath: /etc/metrics-certs
name: metrics-certs
readOnly: true
serviceAccountName: updateservice-operator
volumes:
- name: metrics-certs
secret:
secretName: updateservice-operator-metrics-tls
permissions:
- rules:
- apiGroups:
Expand Down
18 changes: 18 additions & 0 deletions bundle/manifests/updateservice-operator-metrics_v1_service.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.openshift.io/serving-cert-secret-name: updateservice-operator-metrics-tls
creationTimestamp: null
labels:
name: updateservice-operator
name: updateservice-operator-metrics
spec:
ports:
- name: https
port: 8443
targetPort: https
selector:
name: updateservice-operator
status:
loadBalancer: {}
1 change: 1 addition & 0 deletions config/default/kustomization.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,4 @@ resources:
- ../crd
- ../rbac
- ../manager
- ../prometheus/rbac
12 changes: 12 additions & 0 deletions config/manager/manager.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,3 +42,15 @@ spec:
value: "updateservice-operator"
- name: RELATED_IMAGE_OPERAND
value: "quay.io/cincinnati/cincinnati:latest"
ports:
- containerPort: 8443
name: https
protocol: TCP
volumeMounts:
- mountPath: /etc/metrics-certs
name: metrics-certs
readOnly: true
volumes:
- name: metrics-certs
secret:
secretName: updateservice-operator-metrics-tls
2 changes: 2 additions & 0 deletions config/manifests/bases/update-service-operator.clusterserviceversion.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,13 @@ metadata:
annotations:
alm-examples: '[]'
capabilities: Basic Install
console.openshift.io/operator-monitoring-default: "true"
description: Creates and maintains an OpenShift Update Service instance
kubernetes.io/description: "This OpenShift Update Service operator Deployment
reconciles local UpdateServices into more fundamental Kubernetes\nand OpenShift
resources like Cincinnati Deployments and Routes, and it reports the status
of those components in \nthe UpdateService status.\n"
operatorframework.io/cluster-monitoring: "true"
operatorframework.io/suggested-namespace: openshift-update-service
operators.operatorframework.io/builder: operator-sdk-v1.9.0
operators.operatorframework.io/project_layout: go.kubebuilder.io/v3
Expand Down
8 changes: 6 additions & 2 deletions config/prometheus/monitor.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,17 @@ apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
labels:
control-plane: updateservice-operator
name: updateservice-operator
name: updateservice-operator-metrics-monitor
namespace: openshift-update-service
spec:
endpoints:
- path: /metrics
port: https
scheme: https
tlsConfig:
caFile: /etc/prometheus/configmaps/serving-certs-ca-bundle/service-ca.crt
serverName: updateservice-operator-metrics.openshift-update-service.svc
selector:
matchLabels:
control-plane: updateservice-operator
name: updateservice-operator
3 changes: 3 additions & 0 deletions config/prometheus/rbac/kustomization.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
resources:
- role.yaml
- role_binding.yaml
16 changes: 16 additions & 0 deletions config/prometheus/rbac/role.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: prometheus-k8s
namespace: openshift-update-service
rules:
- apiGroups:
- ""
resources:
- services
- endpoints
- pods
verbs:
- get
- list
- watch
13 changes: 13 additions & 0 deletions config/prometheus/rbac/role_binding.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: prometheus-k8s
namespace: openshift-update-service
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: prometheus-k8s
subjects:
- kind: ServiceAccount
name: prometheus-k8s
namespace: openshift-monitoring
4 changes: 2 additions & 2 deletions config/rbac/auth_proxy_role_binding.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,5 +8,5 @@ roleRef:
name: proxy-role
subjects:
- kind: ServiceAccount
name: controller-manager
namespace: system
name: updateservice-operator
namespace: openshift-update-service
6 changes: 4 additions & 2 deletions config/rbac/auth_proxy_service.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.openshift.io/serving-cert-secret-name: updateservice-operator-metrics-tls
labels:
control-plane: updateservice-operator
name: updateservice-operator
name: updateservice-operator-metrics
namespace: openshift-update-service
spec:
Expand All @@ -11,4 +13,4 @@ spec:
port: 8443
targetPort: https
selector:
control-plane: updateservice-operator
name: updateservice-operator
10 changes: 6 additions & 4 deletions config/rbac/kustomization.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,12 @@ resources:
- role_binding.yaml
#- leader_election_role.yaml
#- leader_election_role_binding.yaml
# Comment the following 4 lines if you want to disable
# the auth proxy (https://github.com/brancz/kube-rbac-proxy)
# which protects your /metrics endpoint.
#- auth_proxy_service.yaml
# Metrics service is always needed for Prometheus scraping.
- auth_proxy_service.yaml
# TODO: Uncomment the following 3 lines when upgrading controller-runtime to v0.19+
# and enabling FilterProvider: filters.WithAuthenticationAndAuthorization in main.go.
# These grant the operator permission to perform tokenreviews/subjectaccessreviews
# for authenticating metrics scrape requests.
#- auth_proxy_role.yaml
#- auth_proxy_role_binding.yaml
#- auth_proxy_client_clusterrole.yaml
1 change: 1 addition & 0 deletions config/rbac/role.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ rules:
- apiGroups:
- config.openshift.io
resources:
- apiservers
- images
verbs:
- get
Expand Down
2 changes: 1 addition & 1 deletion controllers/updateservice_controller.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ type UpdateServiceReconciler struct {
// +kubebuilder:rbac:groups="apps",resources=deployments,verbs=get;list;watch
// +kubebuilder:rbac:groups=networking.k8s.io,resources=networkpolicies,verbs=get;list;watch
// +kubebuilder:rbac:groups="policy",resources=poddisruptionbudgets,verbs=get;list;watch
// +kubebuilder:rbac:groups=config.openshift.io,resources=images,verbs=get;list;watch

@wking wking Aug 17, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't we need this to support these calls:

$ git grep -B1 'ImageConfigName'
controllers/mapper.go-// Map will return a reconcile request for a UpdateService if the event is for a
controllers/mapper.go:// ImageConfigName Image or a ConfigMap referenced by AdditionalTrustedCA.Name.
--
controllers/mapper.go-          image := &apicfgv1.Image{}
controllers/mapper.go:          err := m.client.Get(ctx, types.NamespacedName{Name: defaults.ImageConfigName, Namespace: ""}, image)
...
controllers/updateservice_controller.go-        image := &apicfgv1.Image{}
controllers/updateservice_controller.go:        err := r.Client.Get(ctx, types.NamespacedName{Name: defaults.ImageConfigName}, image)
...
vendor/github.com/openshift/cluster-image-registry-operator/pkg/defaults/defaults.go-
vendor/github.com/openshift/cluster-image-registry-operator/pkg/defaults/defaults.go:   ImageConfigName   = "cluster"

// +kubebuilder:rbac:groups=config.openshift.io,resources=apiservers;images,verbs=get;list;watch
// +kubebuilder:rbac:groups=route.openshift.io,resources=routes,verbs=get;list;watch
// +kubebuilder:rbac:groups=updateservice.operator.openshift.io,resources=*,verbs=get;list;watch
// +kubebuilder:rbac:groups="",resources=pods;services;services/finalizers;endpoints;persistentvolumeclaims;events;configmaps;secrets,verbs=create;delete;get;list;patch;update;watch,namespace=openshift-update-service
Expand Down
Loading