|
| 1 | +# External Secrets Operator E2E Tests |
| 2 | + |
| 3 | +This directory contains end-to-end tests for the External Secrets Operator. |
| 4 | + |
| 5 | +## Overview |
| 6 | + |
| 7 | +The e2e test suite validates the External Secrets Operator's functionality with AWS secret providers: |
| 8 | + |
| 9 | +- **AWS** (13 test cases) - Secrets Manager and Parameter Store |
| 10 | + |
| 11 | +**Total: 13 comprehensive test cases** |
| 12 | + |
| 13 | +## Test Structure |
| 14 | + |
| 15 | +``` |
| 16 | +test/e2e/ |
| 17 | +├── e2e_suite_test.go # Test suite setup (existing) |
| 18 | +├── e2e_test.go # Original AWS test (existing) |
| 19 | +├── e2e_aws_test.go # Extended AWS tests (NEW) |
| 20 | +└── testdata/ # YAML test data files |
| 21 | + └── aws_*.yaml # AWS test configurations |
| 22 | +
|
| 23 | +test/utils/ |
| 24 | +├── conditions.go # Test utilities (extended with AWS helpers) |
| 25 | +├── dynamic_resources.go # Resource loading utilities (existing) |
| 26 | +└── ... |
| 27 | +``` |
| 28 | + |
| 29 | +## Prerequisites |
| 30 | + |
| 31 | +### 1. Kubernetes Cluster |
| 32 | + |
| 33 | +You need a running Kubernetes cluster with: |
| 34 | +- External Secrets Operator installed |
| 35 | +- Operator pods running in `external-secrets-operator` namespace |
| 36 | +- Operand pods running in `external-secrets` namespace |
| 37 | + |
| 38 | +### 2. Provider Credentials |
| 39 | + |
| 40 | +All provider credentials must be stored as Kubernetes secrets in the `kube-system` namespace: |
| 41 | + |
| 42 | +#### AWS Credentials |
| 43 | + |
| 44 | +```bash |
| 45 | +kubectl create secret generic aws-creds \ |
| 46 | + --from-literal=aws_access_key_id=<YOUR_ACCESS_KEY_ID> \ |
| 47 | + --from-literal=aws_secret_access_key=<YOUR_SECRET_ACCESS_KEY> \ |
| 48 | + -n kube-system |
| 49 | +``` |
| 50 | + |
| 51 | +**Required IAM Permissions:** |
| 52 | +- `secretsmanager:CreateSecret` |
| 53 | +- `secretsmanager:GetSecretValue` |
| 54 | +- `secretsmanager:UpdateSecret` |
| 55 | +- `secretsmanager:DeleteSecret` |
| 56 | +- `ssm:PutParameter` |
| 57 | +- `ssm:GetParameter` |
| 58 | +- `ssm:DeleteParameter` |
| 59 | + |
| 60 | +## Running the Tests |
| 61 | + |
| 62 | +### Run All E2E Tests |
| 63 | + |
| 64 | +```bash |
| 65 | +make test-e2e |
| 66 | +``` |
| 67 | + |
| 68 | +### Run Tests by Provider |
| 69 | + |
| 70 | +```bash |
| 71 | +# Run AWS tests |
| 72 | +make test-e2e E2E_GINKGO_LABEL_FILTER="Platform:AWS" |
| 73 | +``` |
| 74 | + |
| 75 | +### Run Without Make |
| 76 | + |
| 77 | +```bash |
| 78 | +# Set the region (default: ap-south-1) |
| 79 | +export E2E_AWS_REGION=us-east-1 |
| 80 | + |
| 81 | +# Run tests directly |
| 82 | +go test -v -tags=e2e ./test/e2e/... -ginkgo.label-filter="Platform:AWS" |
| 83 | +``` |
| 84 | + |
| 85 | +## Test Coverage |
| 86 | + |
| 87 | +### AWS Tests (e2e_aws_test.go) |
| 88 | + |
| 89 | +#### Basic Operations (3 tests) |
| 90 | +1. **Namespace-scoped SecretStore** - Tests non-cluster scoped SecretStore |
| 91 | +2. **Multiple data keys** - Fetches multiple keys from a single AWS secret |
| 92 | +3. **Binary data handling** - Tests base64-encoded binary data |
| 93 | + |
| 94 | +#### Advanced Features (4 tests) |
| 95 | +4. **Secret rotation** - Tests automatic refresh with 30s interval |
| 96 | +5. **Template transformation** - Tests Go template engine |
| 97 | +6. **dataFrom** - Fetches entire secret without explicit mapping |
| 98 | +7. **JSON path extraction** - Extracts nested values from complex JSON |
| 99 | + |
| 100 | +#### AWS Parameter Store (1 test) |
| 101 | +8. **SSM Parameter Store basic** - Tests Systems Manager Parameter Store integration |
| 102 | + |
| 103 | +#### Error Scenarios (2 tests) |
| 104 | +9. **Non-existent secret** - Verifies error handling for missing secrets |
| 105 | +10. **Invalid credentials** - Tests SecretStore with bad credentials |
| 106 | + |
| 107 | +**Total: 11 new AWS tests** (plus 1 existing test = 12 total) |
| 108 | + |
| 109 | +## Test Pattern |
| 110 | + |
| 111 | +All tests follow this pattern: |
| 112 | + |
| 113 | +```go |
| 114 | +It("should do something", func() { |
| 115 | + // 1. Create remote AWS secret |
| 116 | + |
| 117 | + // 2. Create SecretStore/ClusterSecretStore |
| 118 | + |
| 119 | + // 3. Wait for SecretStore to become Ready |
| 120 | + |
| 121 | + // 4. Create ExternalSecret/PushSecret |
| 122 | + |
| 123 | + // 5. Wait for resource to become Ready |
| 124 | + |
| 125 | + // 6. Verify K8s secret contains expected data |
| 126 | + |
| 127 | + // 7. Cleanup (via defer) |
| 128 | +}) |
| 129 | +``` |
| 130 | + |
| 131 | +## Troubleshooting |
| 132 | + |
| 133 | +### Tests Skip with "credentials not found" |
| 134 | + |
| 135 | +Make sure credentials are created in the `kube-system` namespace: |
| 136 | +- `aws-creds` |
| 137 | + |
| 138 | +Check credentials: |
| 139 | +```bash |
| 140 | +kubectl get secret aws-creds -n kube-system |
| 141 | +``` |
| 142 | + |
| 143 | +### "ClusterSecretStore not becoming Ready" |
| 144 | + |
| 145 | +Check operator logs: |
| 146 | +```bash |
| 147 | +# Operator logs |
| 148 | +kubectl logs -n external-secrets-operator -l app.kubernetes.io/name=external-secrets-operator |
| 149 | + |
| 150 | +# Operand logs |
| 151 | +kubectl logs -n external-secrets -l app.kubernetes.io/name=external-secrets |
| 152 | +``` |
| 153 | + |
| 154 | +### "Secret not created in K8s" |
| 155 | + |
| 156 | +Check ExternalSecret status: |
| 157 | +```bash |
| 158 | +kubectl describe externalsecret <name> -n external-secrets |
| 159 | +``` |
| 160 | + |
| 161 | +Look for error conditions in the status. |
| 162 | + |
| 163 | +### Tests taking too long |
| 164 | + |
| 165 | +- AWS API calls can be slow |
| 166 | +- Default timeout is 2 minutes per resource |
| 167 | +- Ensure good network connectivity to AWS |
| 168 | + |
| 169 | +## Cleanup |
| 170 | + |
| 171 | +Tests automatically cleanup resources using defer blocks. If tests are interrupted, you may need to manually cleanup: |
| 172 | + |
| 173 | +### AWS |
| 174 | +```bash |
| 175 | +# List secrets |
| 176 | +aws secretsmanager list-secrets --query "SecretList[?starts_with(Name, 'eso-e2e-')]" |
| 177 | + |
| 178 | +# Delete secret |
| 179 | +aws secretsmanager delete-secret --secret-id <name> --force-delete-without-recovery |
| 180 | +``` |
| 181 | + |
| 182 | +## CI/CD Integration |
| 183 | + |
| 184 | +Example GitHub Actions workflow: |
| 185 | + |
| 186 | +```yaml |
| 187 | +name: E2E Tests |
| 188 | + |
| 189 | +on: [push, pull_request] |
| 190 | + |
| 191 | +jobs: |
| 192 | + e2e-aws: |
| 193 | + runs-on: ubuntu-latest |
| 194 | + steps: |
| 195 | + - uses: actions/checkout@v4 |
| 196 | + - uses: actions/setup-go@v5 |
| 197 | + with: |
| 198 | + go-version: '1.23' |
| 199 | + |
| 200 | + - name: Create kind cluster |
| 201 | + run: | |
| 202 | + kind create cluster |
| 203 | + kubectl cluster-info |
| 204 | +
|
| 205 | + - name: Install External Secrets Operator |
| 206 | + run: | |
| 207 | + # Install operator |
| 208 | +
|
| 209 | + - name: Create AWS credentials |
| 210 | + run: | |
| 211 | + kubectl create secret generic aws-creds \ |
| 212 | + --from-literal=aws_access_key_id=${{ secrets.AWS_ACCESS_KEY_ID }} \ |
| 213 | + --from-literal=aws_secret_access_key=${{ secrets.AWS_SECRET_ACCESS_KEY }} \ |
| 214 | + -n kube-system |
| 215 | +
|
| 216 | + - name: Run AWS E2E tests |
| 217 | + run: make test-e2e E2E_GINKGO_LABEL_FILTER="Platform:AWS" |
| 218 | +``` |
| 219 | +
|
| 220 | +## Contributing |
| 221 | +
|
| 222 | +When adding new test cases: |
| 223 | +
|
| 224 | +1. Follow the existing pattern in test files |
| 225 | +2. Use descriptive test names |
| 226 | +3. Add appropriate labels (e.g., `Label("Platform:AWS")`) |
| 227 | +4. Create test data YAML files in `testdata/` |
| 228 | +5. Use pattern replacement for dynamic values |
| 229 | +6. Always cleanup resources in defer blocks |
| 230 | +7. Use `Eventually()` for async assertions |
| 231 | +8. Test both success and error scenarios |
| 232 | + |
| 233 | +## Resources |
| 234 | + |
| 235 | +- [External Secrets Operator Documentation](https://external-secrets.io/) |
| 236 | +- [Ginkgo Testing Framework](https://onsi.github.io/ginkgo/) |
| 237 | +- [Gomega Matcher Library](https://onsi.github.io/gomega/) |
0 commit comments