missing permission in role yaml

[oriyarde]

Hi,

I tried to deploy the replication operator as a sidecar container inside our csi controller pod.

I have a cluster role binding with a service account ("ibm-block-csi-controller-sa") and with a cluster role for the replication operator.
I copied said cluster role rules from the role yaml rules, and the replication operator log started nicely:

I0711 19:40:07.169702 1 common.go:111] Probing CSI driver for readiness
2021-07-11T19:40:07.171Z INFO setup starting manager
2021-07-11T19:40:07.171Z INFO controller-runtime.manager.controller.volumereplication Starting EventSource {"reconciler group": "replication.storage.openshift.io", "reconciler kind": "VolumeReplication", "source": "kind source: /, Kind="}
2021-07-11T19:40:07.271Z INFO controller-runtime.manager.controller.volumereplication Starting Controller {"reconciler group": "replication.storage.openshift.io", "reconciler kind": "VolumeReplication"}
2021-07-11T19:40:07.272Z INFO controller-runtime.manager.controller.volumereplication Starting workers {"reconciler group": "replication.storage.openshift.io", "reconciler kind": "VolumeReplication", "worker count": 1}

but then entered an error loop:

E0711 19:40:07.276673 1 reflector.go:127] pkg/mod/k8s.io/client-go@v0.19.2/tools/cache/reflector.go:156: Failed to watch *v1alpha1.VolumeReplicationClass: failed to list *v1alpha1.VolumeReplicationClass: volumereplicationclasses.replication.storage.openshift.io is forbidden: User "system:serviceaccount:default:ibm-block-csi-controller-sa" cannot list resource "volumereplicationclasses" in API group "replication.storage.openshift.io" at the cluster scope
E0711 19:40:08.432053 1 reflector.go:127] pkg/mod/k8s.io/client-go@v0.19.2/tools/cache/reflector.go:156: Failed to watch *v1alpha1.VolumeReplicationClass: failed to list *v1alpha1.VolumeReplicationClass: volumereplicationclasses.replication.storage.openshift.io is forbidden: User "system:serviceaccount:default:ibm-block-csi-controller-sa" cannot list resource "volumereplicationclasses" in API group "replication.storage.openshift.io" at the cluster scope
...

after adding the following rule (copied from the volumereplicationclass viewer role yaml, which is "for end users"):

- apiGroups:
  - replication.storage.openshift.io
  resources:
  - volumereplicationclasses
  verbs:
  - get
  - list
  - watch

the error was gone.

so I think this rule is missing in the role yaml.

[Madhu-1]

volumereplicationclass rules are defined in different files (these are auto-generated files) and this is expected and you need to apply cluster roles for volumereplicationclass.

cc @sp98

[oriyarde]

@Madhu-1 the issue is that the required rules for the replication operator are defined in different files, instead of one (as in other csi sidecars, e.g. provisioner, attacher, etc...).

and the yaml with the required replication class rule has a comment that says the rule is for end users, which is misleading, since it implies that the rule is not meant for the replication operator

[Madhu-1]

@Madhu-1 the issue is that the required rules for the replication operator are defined in different files, instead of one (as in other csi sidecars, e.g. provisioner, attacher, etc...).

@oriyarde the volume replication is generated from the operator scale folding but the other CSI sidecars are not. that is the reason for having it in different files.

and the yaml with the required replication class rule has a comment that says the rule is for end users, which is misleading, since it implies that the rule is not meant for the replication operator

AFAIK the comment is adding by the operator sdk. lets hear from @raghavendra-talur @sp98 on that

[oriyarde]

@Madhu-1 could all the required rules at least be generated into a single directory?
the information for which rules are required for the operator should be managed somehow anyway (documentation, a single yaml as in the other sidecars, etc...).
currently, how can one know which rules are required (without trial and error)?

[Madhu-1]

Sure we can document the deployment and RBAC in a readme at least of this one.