Skip to content

fix: Inject S3 credentials for SAIA v1 and Ray on AWS s3 storage #92

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
kupratyu-splunk merged 3 commits into from
May 21, 2026
Merged

fix: Inject S3 credentials for SAIA v1 and Ray on AWS s3 storage #92

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
716a528
fix: inject static S3 credentials for SAIA v1 and Ray on AWS/s3compat…
kbhos-splunk May 21, 2026
09af9ee
codex comment fix
kbhos-splunk May 21, 2026
50a4119
Apply suggestions from code review
kbhos-splunk May 21, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
169 changes: 169 additions & 0 deletions config/configs/applications.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,19 @@ applications:
S3COMPAT_OBJECT_STORE_ENDPOINT_URL: "{{.S3CompatObjectStoreEndpointUrl}}"
S3COMPAT_OBJECT_STORE_ACCESS_KEY: "{{.S3CompatObjectStoreAccessKey}}"
S3COMPAT_OBJECT_STORE_SECRET_KEY: "{{.S3CompatObjectStoreSecretKey}}"
# AWS / boto3 standard credential names — populated whenever the
# operator can load credentials from spec.objectStorage.secretRef. For
# CLOUD_PROVIDER=aws these are the values boto3 reads (the S3COMPAT_*
# names above are only consumed by the s3compat shim). Both code paths
# share the same source-of-truth Secret keys (s3_access_key /
# s3_secret_key) so emitting both pairs is safe — each provider only
# reads its own. AWS_REGION lets boto3 resolve the default regional S3
# endpoint when no AWS_ENDPOINT_URL is set; required for any AWS S3
# bucket outside us-east-1 to avoid PermanentRedirect on the first call.
AWS_ACCESS_KEY_ID: "{{.S3CompatObjectStoreAccessKey}}"
AWS_SECRET_ACCESS_KEY: "{{.S3CompatObjectStoreSecretKey}}"
AWS_REGION: "{{.Region}}"
AWS_DEFAULT_REGION: "{{.Region}}"
ENABLE_AUTHN: "false"
ENABLE_AUTHZ: "false"
SERVICE_EXTERNAL_NAME: "ai-platform-models"
Expand Down Expand Up @@ -86,6 +99,19 @@ applications:
S3COMPAT_OBJECT_STORE_ENDPOINT_URL: "{{.S3CompatObjectStoreEndpointUrl}}"
S3COMPAT_OBJECT_STORE_ACCESS_KEY: "{{.S3CompatObjectStoreAccessKey}}"
S3COMPAT_OBJECT_STORE_SECRET_KEY: "{{.S3CompatObjectStoreSecretKey}}"
# AWS / boto3 standard credential names — populated whenever the
# operator can load credentials from spec.objectStorage.secretRef. For
# CLOUD_PROVIDER=aws these are the values boto3 reads (the S3COMPAT_*
# names above are only consumed by the s3compat shim). Both code paths
# share the same source-of-truth Secret keys (s3_access_key /
# s3_secret_key) so emitting both pairs is safe — each provider only
# reads its own. AWS_REGION lets boto3 resolve the default regional S3
# endpoint when no AWS_ENDPOINT_URL is set; required for any AWS S3
# bucket outside us-east-1 to avoid PermanentRedirect on the first call.
AWS_ACCESS_KEY_ID: "{{.S3CompatObjectStoreAccessKey}}"
AWS_SECRET_ACCESS_KEY: "{{.S3CompatObjectStoreSecretKey}}"
AWS_REGION: "{{.Region}}"
AWS_DEFAULT_REGION: "{{.Region}}"
ENABLE_AUTHN: "false"
ENABLE_AUTHZ: "false"
SERVICE_EXTERNAL_NAME: "ai-platform-models"
Expand Down Expand Up @@ -170,6 +196,19 @@ applications:
S3COMPAT_OBJECT_STORE_ENDPOINT_URL: "{{.S3CompatObjectStoreEndpointUrl}}"
S3COMPAT_OBJECT_STORE_ACCESS_KEY: "{{.S3CompatObjectStoreAccessKey}}"
S3COMPAT_OBJECT_STORE_SECRET_KEY: "{{.S3CompatObjectStoreSecretKey}}"
# AWS / boto3 standard credential names — populated whenever the
# operator can load credentials from spec.objectStorage.secretRef. For
# CLOUD_PROVIDER=aws these are the values boto3 reads (the S3COMPAT_*
# names above are only consumed by the s3compat shim). Both code paths
# share the same source-of-truth Secret keys (s3_access_key /
# s3_secret_key) so emitting both pairs is safe — each provider only
# reads its own. AWS_REGION lets boto3 resolve the default regional S3
# endpoint when no AWS_ENDPOINT_URL is set; required for any AWS S3
# bucket outside us-east-1 to avoid PermanentRedirect on the first call.
AWS_ACCESS_KEY_ID: "{{.S3CompatObjectStoreAccessKey}}"
AWS_SECRET_ACCESS_KEY: "{{.S3CompatObjectStoreSecretKey}}"
AWS_REGION: "{{.Region}}"
AWS_DEFAULT_REGION: "{{.Region}}"
ENABLE_AUTHN: "false"
ENABLE_AUTHZ: "false"
SERVICE_EXTERNAL_NAME: "ai-platform-models"
Expand Down Expand Up @@ -230,6 +269,19 @@ applications:
S3COMPAT_OBJECT_STORE_ENDPOINT_URL: "{{.S3CompatObjectStoreEndpointUrl}}"
S3COMPAT_OBJECT_STORE_ACCESS_KEY: "{{.S3CompatObjectStoreAccessKey}}"
S3COMPAT_OBJECT_STORE_SECRET_KEY: "{{.S3CompatObjectStoreSecretKey}}"
# AWS / boto3 standard credential names — populated whenever the
# operator can load credentials from spec.objectStorage.secretRef. For
# CLOUD_PROVIDER=aws these are the values boto3 reads (the S3COMPAT_*
# names above are only consumed by the s3compat shim). Both code paths
# share the same source-of-truth Secret keys (s3_access_key /
# s3_secret_key) so emitting both pairs is safe — each provider only
# reads its own. AWS_REGION lets boto3 resolve the default regional S3
# endpoint when no AWS_ENDPOINT_URL is set; required for any AWS S3
# bucket outside us-east-1 to avoid PermanentRedirect on the first call.
AWS_ACCESS_KEY_ID: "{{.S3CompatObjectStoreAccessKey}}"
AWS_SECRET_ACCESS_KEY: "{{.S3CompatObjectStoreSecretKey}}"
AWS_REGION: "{{.Region}}"
AWS_DEFAULT_REGION: "{{.Region}}"
ENABLE_AUTHN: "false"
ENABLE_AUTHZ: "false"
SERVICE_EXTERNAL_NAME: "ai-platform-models"
Expand Down Expand Up @@ -280,6 +332,19 @@ applications:
S3COMPAT_OBJECT_STORE_ENDPOINT_URL: "{{.S3CompatObjectStoreEndpointUrl}}"
S3COMPAT_OBJECT_STORE_ACCESS_KEY: "{{.S3CompatObjectStoreAccessKey}}"
S3COMPAT_OBJECT_STORE_SECRET_KEY: "{{.S3CompatObjectStoreSecretKey}}"
# AWS / boto3 standard credential names — populated whenever the
# operator can load credentials from spec.objectStorage.secretRef. For
# CLOUD_PROVIDER=aws these are the values boto3 reads (the S3COMPAT_*
# names above are only consumed by the s3compat shim). Both code paths
# share the same source-of-truth Secret keys (s3_access_key /
# s3_secret_key) so emitting both pairs is safe — each provider only
# reads its own. AWS_REGION lets boto3 resolve the default regional S3
# endpoint when no AWS_ENDPOINT_URL is set; required for any AWS S3
# bucket outside us-east-1 to avoid PermanentRedirect on the first call.
AWS_ACCESS_KEY_ID: "{{.S3CompatObjectStoreAccessKey}}"
AWS_SECRET_ACCESS_KEY: "{{.S3CompatObjectStoreSecretKey}}"
AWS_REGION: "{{.Region}}"
AWS_DEFAULT_REGION: "{{.Region}}"
ENABLE_AUTHN: "false"
ENABLE_AUTHZ: "false"
SERVICE_EXTERNAL_NAME: "ai-platform-models"
Expand Down Expand Up @@ -330,6 +395,19 @@ applications:
S3COMPAT_OBJECT_STORE_ENDPOINT_URL: "{{.S3CompatObjectStoreEndpointUrl}}"
S3COMPAT_OBJECT_STORE_ACCESS_KEY: "{{.S3CompatObjectStoreAccessKey}}"
S3COMPAT_OBJECT_STORE_SECRET_KEY: "{{.S3CompatObjectStoreSecretKey}}"
# AWS / boto3 standard credential names — populated whenever the
# operator can load credentials from spec.objectStorage.secretRef. For
# CLOUD_PROVIDER=aws these are the values boto3 reads (the S3COMPAT_*
# names above are only consumed by the s3compat shim). Both code paths
# share the same source-of-truth Secret keys (s3_access_key /
# s3_secret_key) so emitting both pairs is safe — each provider only
# reads its own. AWS_REGION lets boto3 resolve the default regional S3
# endpoint when no AWS_ENDPOINT_URL is set; required for any AWS S3
# bucket outside us-east-1 to avoid PermanentRedirect on the first call.
AWS_ACCESS_KEY_ID: "{{.S3CompatObjectStoreAccessKey}}"
AWS_SECRET_ACCESS_KEY: "{{.S3CompatObjectStoreSecretKey}}"
AWS_REGION: "{{.Region}}"
AWS_DEFAULT_REGION: "{{.Region}}"
ENABLE_AUTHN: "false"
ENABLE_AUTHZ: "false"
SERVICE_EXTERNAL_NAME: "ai-platform-models"
Expand Down Expand Up @@ -371,6 +449,19 @@ applications:
S3COMPAT_OBJECT_STORE_ENDPOINT_URL: "{{.S3CompatObjectStoreEndpointUrl}}"
S3COMPAT_OBJECT_STORE_ACCESS_KEY: "{{.S3CompatObjectStoreAccessKey}}"
S3COMPAT_OBJECT_STORE_SECRET_KEY: "{{.S3CompatObjectStoreSecretKey}}"
# AWS / boto3 standard credential names — populated whenever the
# operator can load credentials from spec.objectStorage.secretRef. For
# CLOUD_PROVIDER=aws these are the values boto3 reads (the S3COMPAT_*
# names above are only consumed by the s3compat shim). Both code paths
# share the same source-of-truth Secret keys (s3_access_key /
# s3_secret_key) so emitting both pairs is safe — each provider only
# reads its own. AWS_REGION lets boto3 resolve the default regional S3
# endpoint when no AWS_ENDPOINT_URL is set; required for any AWS S3
# bucket outside us-east-1 to avoid PermanentRedirect on the first call.
AWS_ACCESS_KEY_ID: "{{.S3CompatObjectStoreAccessKey}}"
AWS_SECRET_ACCESS_KEY: "{{.S3CompatObjectStoreSecretKey}}"
AWS_REGION: "{{.Region}}"
AWS_DEFAULT_REGION: "{{.Region}}"
ENABLE_AUTHN: "false"
ENABLE_AUTHZ: "false"
SERVICE_EXTERNAL_NAME: "ai-platform-models"
Expand Down Expand Up @@ -427,6 +518,19 @@ applications:
S3COMPAT_OBJECT_STORE_ENDPOINT_URL: "{{.S3CompatObjectStoreEndpointUrl}}"
S3COMPAT_OBJECT_STORE_ACCESS_KEY: "{{.S3CompatObjectStoreAccessKey}}"
S3COMPAT_OBJECT_STORE_SECRET_KEY: "{{.S3CompatObjectStoreSecretKey}}"
# AWS / boto3 standard credential names — populated whenever the
# operator can load credentials from spec.objectStorage.secretRef. For
# CLOUD_PROVIDER=aws these are the values boto3 reads (the S3COMPAT_*
# names above are only consumed by the s3compat shim). Both code paths
# share the same source-of-truth Secret keys (s3_access_key /
# s3_secret_key) so emitting both pairs is safe — each provider only
# reads its own. AWS_REGION lets boto3 resolve the default regional S3
# endpoint when no AWS_ENDPOINT_URL is set; required for any AWS S3
# bucket outside us-east-1 to avoid PermanentRedirect on the first call.
AWS_ACCESS_KEY_ID: "{{.S3CompatObjectStoreAccessKey}}"
AWS_SECRET_ACCESS_KEY: "{{.S3CompatObjectStoreSecretKey}}"
AWS_REGION: "{{.Region}}"
AWS_DEFAULT_REGION: "{{.Region}}"
ENABLE_AUTHN: "false"
ENABLE_AUTHZ: "false"
SERVICE_EXTERNAL_NAME: "ai-platform-models"
Expand All @@ -453,6 +557,19 @@ applications:
S3COMPAT_OBJECT_STORE_ENDPOINT_URL: "{{.S3CompatObjectStoreEndpointUrl}}"
S3COMPAT_OBJECT_STORE_ACCESS_KEY: "{{.S3CompatObjectStoreAccessKey}}"
S3COMPAT_OBJECT_STORE_SECRET_KEY: "{{.S3CompatObjectStoreSecretKey}}"
# AWS / boto3 standard credential names — populated whenever the
# operator can load credentials from spec.objectStorage.secretRef. For
# CLOUD_PROVIDER=aws these are the values boto3 reads (the S3COMPAT_*
# names above are only consumed by the s3compat shim). Both code paths
# share the same source-of-truth Secret keys (s3_access_key /
# s3_secret_key) so emitting both pairs is safe — each provider only
# reads its own. AWS_REGION lets boto3 resolve the default regional S3
# endpoint when no AWS_ENDPOINT_URL is set; required for any AWS S3
# bucket outside us-east-1 to avoid PermanentRedirect on the first call.
AWS_ACCESS_KEY_ID: "{{.S3CompatObjectStoreAccessKey}}"
AWS_SECRET_ACCESS_KEY: "{{.S3CompatObjectStoreSecretKey}}"
AWS_REGION: "{{.Region}}"
AWS_DEFAULT_REGION: "{{.Region}}"
ENABLE_AUTHN: "false"
ENABLE_AUTHZ: "false"
SERVICE_EXTERNAL_NAME: "ai-platform-models"
Expand Down Expand Up @@ -504,6 +621,19 @@ applications:
S3COMPAT_OBJECT_STORE_ENDPOINT_URL: "{{.S3CompatObjectStoreEndpointUrl}}"
S3COMPAT_OBJECT_STORE_ACCESS_KEY: "{{.S3CompatObjectStoreAccessKey}}"
S3COMPAT_OBJECT_STORE_SECRET_KEY: "{{.S3CompatObjectStoreSecretKey}}"
# AWS / boto3 standard credential names — populated whenever the
# operator can load credentials from spec.objectStorage.secretRef. For
# CLOUD_PROVIDER=aws these are the values boto3 reads (the S3COMPAT_*
# names above are only consumed by the s3compat shim). Both code paths
# share the same source-of-truth Secret keys (s3_access_key /
# s3_secret_key) so emitting both pairs is safe — each provider only
# reads its own. AWS_REGION lets boto3 resolve the default regional S3
# endpoint when no AWS_ENDPOINT_URL is set; required for any AWS S3
# bucket outside us-east-1 to avoid PermanentRedirect on the first call.
AWS_ACCESS_KEY_ID: "{{.S3CompatObjectStoreAccessKey}}"
AWS_SECRET_ACCESS_KEY: "{{.S3CompatObjectStoreSecretKey}}"
AWS_REGION: "{{.Region}}"
AWS_DEFAULT_REGION: "{{.Region}}"
ENABLE_AUTHN: "false"
ENABLE_AUTHZ: "false"
SERVICE_EXTERNAL_NAME: "ai-platform-models"
Expand Down Expand Up @@ -560,6 +690,19 @@ applications:
S3COMPAT_OBJECT_STORE_ENDPOINT_URL: "{{.S3CompatObjectStoreEndpointUrl}}"
S3COMPAT_OBJECT_STORE_ACCESS_KEY: "{{.S3CompatObjectStoreAccessKey}}"
S3COMPAT_OBJECT_STORE_SECRET_KEY: "{{.S3CompatObjectStoreSecretKey}}"
# AWS / boto3 standard credential names — populated whenever the
# operator can load credentials from spec.objectStorage.secretRef. For
# CLOUD_PROVIDER=aws these are the values boto3 reads (the S3COMPAT_*
# names above are only consumed by the s3compat shim). Both code paths
# share the same source-of-truth Secret keys (s3_access_key /
# s3_secret_key) so emitting both pairs is safe — each provider only
# reads its own. AWS_REGION lets boto3 resolve the default regional S3
# endpoint when no AWS_ENDPOINT_URL is set; required for any AWS S3
# bucket outside us-east-1 to avoid PermanentRedirect on the first call.
AWS_ACCESS_KEY_ID: "{{.S3CompatObjectStoreAccessKey}}"
AWS_SECRET_ACCESS_KEY: "{{.S3CompatObjectStoreSecretKey}}"
AWS_REGION: "{{.Region}}"
AWS_DEFAULT_REGION: "{{.Region}}"
ENABLE_AUTHN: "false"
ENABLE_AUTHZ: "false"
SERVICE_EXTERNAL_NAME: "ai-platform-models"
Expand Down Expand Up @@ -606,6 +749,19 @@ applications:
S3COMPAT_OBJECT_STORE_ENDPOINT_URL: "{{.S3CompatObjectStoreEndpointUrl}}"
S3COMPAT_OBJECT_STORE_ACCESS_KEY: "{{.S3CompatObjectStoreAccessKey}}"
S3COMPAT_OBJECT_STORE_SECRET_KEY: "{{.S3CompatObjectStoreSecretKey}}"
# AWS / boto3 standard credential names — populated whenever the
# operator can load credentials from spec.objectStorage.secretRef. For
# CLOUD_PROVIDER=aws these are the values boto3 reads (the S3COMPAT_*
# names above are only consumed by the s3compat shim). Both code paths
# share the same source-of-truth Secret keys (s3_access_key /
# s3_secret_key) so emitting both pairs is safe — each provider only
# reads its own. AWS_REGION lets boto3 resolve the default regional S3
# endpoint when no AWS_ENDPOINT_URL is set; required for any AWS S3
# bucket outside us-east-1 to avoid PermanentRedirect on the first call.
AWS_ACCESS_KEY_ID: "{{.S3CompatObjectStoreAccessKey}}"
AWS_SECRET_ACCESS_KEY: "{{.S3CompatObjectStoreSecretKey}}"
AWS_REGION: "{{.Region}}"
AWS_DEFAULT_REGION: "{{.Region}}"
ENABLE_AUTHN: "false"
ENABLE_AUTHZ: "false"
SERVICE_EXTERNAL_NAME: "ai-platform-models"
Expand Down Expand Up @@ -638,6 +794,19 @@ applications:
S3COMPAT_OBJECT_STORE_ENDPOINT_URL: "{{.S3CompatObjectStoreEndpointUrl}}"
S3COMPAT_OBJECT_STORE_ACCESS_KEY: "{{.S3CompatObjectStoreAccessKey}}"
S3COMPAT_OBJECT_STORE_SECRET_KEY: "{{.S3CompatObjectStoreSecretKey}}"
# AWS / boto3 standard credential names — populated whenever the
# operator can load credentials from spec.objectStorage.secretRef. For
# CLOUD_PROVIDER=aws these are the values boto3 reads (the S3COMPAT_*
# names above are only consumed by the s3compat shim). Both code paths
# share the same source-of-truth Secret keys (s3_access_key /
# s3_secret_key) so emitting both pairs is safe — each provider only
# reads its own. AWS_REGION lets boto3 resolve the default regional S3
# endpoint when no AWS_ENDPOINT_URL is set; required for any AWS S3
# bucket outside us-east-1 to avoid PermanentRedirect on the first call.
AWS_ACCESS_KEY_ID: "{{.S3CompatObjectStoreAccessKey}}"
AWS_SECRET_ACCESS_KEY: "{{.S3CompatObjectStoreSecretKey}}"
AWS_REGION: "{{.Region}}"
AWS_DEFAULT_REGION: "{{.Region}}"
ENABLE_AUTHN: "false"
ENABLE_AUTHZ: "false"
SERVICE_EXTERNAL_NAME: "ai-platform-models"
Expand Down
Loading
Loading