Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions .github/workflows/publish-jenkins.yml
Original file line number Diff line number Diff line change
Expand Up @@ -44,3 +44,29 @@ jobs:
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}

tag:
name: Update Major Version Tags
needs: publish
if: startsWith(github.ref, 'refs/tags/v')
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0

- name: Update Major Tag
run: |
# Extract major version (e.g., v1 from v1.2.3)
MAJOR=$(echo "${{ github.ref_name }}" | cut -d. -f1)

echo "Updating major tag: $MAJOR to point to ${{ github.ref_name }}"

git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"

# Force create/move major tag
git tag -f "$MAJOR" "${{ github.ref_name }}"
git push origin "$MAJOR" --force
30 changes: 0 additions & 30 deletions .github/workflows/release-tagging.yml

This file was deleted.

1 change: 1 addition & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ name: Tests
on:
push:
branches: [ dev ]
tags: [ 'v*' ]
pull_request:
branches: [ dev ]
workflow_dispatch:
Expand Down
3 changes: 1 addition & 2 deletions action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -89,9 +89,8 @@ inputs:
description: 'Custom path for environment files (when structure=custom)'
required: false
env_files_patterns:
description: 'Comma-separated list of .env file patterns (e.g., .env.app,.env.database)'
description: 'Comma-separated list of .env file patterns (e.g., .env.app,.env.database). Only used if env_files_structure is NOT set to "auto".'
required: false
default: '.env.app,.env.database'
env_files_create_root:
description: 'Also create .env files in project root'
required: false
Expand Down
6 changes: 3 additions & 3 deletions changelogs/2026-01-29_16-00-53.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@
### Fixed
- Improved code coverage by updating the `coverage.xml` file to reflect changes in the codebase.

Note: The changes to `coverage.xml` and `tests/unit/test_multi_server.py` are not explicitly mentioned in the changelog as they are not directly related to user-facing changes. However, they are included in the commit and are reflected in the changelog as "Improved code coverage".
Note: The changes to `coverage.xml` and `tests/unit/test_multi_server.py` are not explicitly mentioned in the changelog as they are not directly related to user-facing changes. However, they are included in the commit and are reflected in the changelog as "Improved code coverage".

The changes to `.flake8` and `pytest.ini` are related to code formatting and testing, which are not typically included in a user-facing changelog. However, they are included here as they may be relevant to developers working with the codebase.
The changes to `.flake8` and `pytest.ini` are related to code formatting and testing, which are not typically included in a user-facing changelog. However, they are included here as they may be relevant to developers working with the codebase.

The actual changes to the codebase are not explicitly mentioned in the changelog as they are not directly related to user-facing changes. The changelog only includes changes that are relevant to users or developers working with the codebase.
The actual changes to the codebase are not explicitly mentioned in the changelog as they are not directly related to user-facing changes. The changelog only includes changes that are relevant to users or developers working with the codebase.
12 changes: 12 additions & 0 deletions changelogs/2026-01-29_16-10-08.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# Changelog

## [Unreleased]

### Changed
- The workflow for updating major version tags has been moved from a separate workflow file (`release-tagging.yml`) to the `publish-jenkins.yml` file.
- The `test.yml` workflow now triggers on tags starting with 'v', in addition to its existing triggers.

### Removed
- The `release-tagging.yml` workflow file has been deleted, as its functionality has been merged into `publish-jenkins.yml`.

Note: The changes to the changelog file itself are not explicitly mentioned as they are not directly related to user-facing changes, but rather to the maintenance of the changelog.
21 changes: 21 additions & 0 deletions changelogs/2026-01-29_16-48-08.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Changelog

## [Unreleased]

### Added
- Support for a literal `ENV` secret for non-prefixed global dumps
- Support for custom environment names in prefixes
- File permissions for generated environment files are now `0o604` for better security

### Changed
- The `env_files_patterns` input now only applies when `env_files_structure` is not set to "auto"
- The priority system for environment variables has been updated to include literal `ENV` secrets and to support custom environment names
- Environment variables are now scanned for prefixes including the current environment input, in addition to common environment names like "PROD", "STAGING", "DEV", "TEST", and "PRODUCTION"

### Fixed
- No specific bug fixes mentioned in the diff, but several code changes suggest improvements to the environment variable handling and file generation logic

### Removed
- Default value for `env_files_patterns` has been removed, making it an optional input

Note: The changes to the changelog file itself are not explicitly mentioned as they are not directly related to user-facing changes, but rather to the maintenance of the changelog.
142 changes: 36 additions & 106 deletions docs/env-generation.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,8 @@ MetalDeploy includes powerful environment file generation capabilities that auto
- ✅ **Flexible File Structures** - Single `.env` file, flat `.env.*` files, or nested `.envs/{environment}/` organization
- ✅ **Priority System** - Environment-specific secrets override base secrets automatically
- ✅ **All-in-One Secret Support** - Store multiple variables in single secrets
- ✅ **Secure Handling** - Files created with `0o600` permissions, no secret logging
- ✅ **Global Literal Blob** - Support for a literal `ENV` secret for non-prefixed global dumps
- ✅ **Secure Handling** - Files created with `0o604` permissions, no secret logging
- ✅ **Secrets & Variables** - Supports both GitHub Secrets (encrypted) and GitHub Variables (plaintext)

## Configuration
Expand All @@ -18,20 +19,31 @@ MetalDeploy includes powerful environment file generation capabilities that auto
| `env_files_generate` | Enable environment file generation | `false` |
| `env_files_structure` | File structure: `single`, `flat`, `nested`, `auto`, `custom` | `auto` |
| `env_files_path` | Custom path (when `structure=custom`) | - |
| `env_files_patterns` | Comma-separated patterns (`.env.app,.env.database`) | `.env.app,.env.database` |
| `env_files_patterns` | Comma-separated patterns (`.env.app,.env.database`) | - |
| `env_files_create_root` | Also create a combined `.env` file in project root | `false` |
| `env_files_format` | Format for parsing: `auto`, `env`, `json`, `yaml` | `auto` |

## How it Works

1. **Discovery**: The action scans all environment variables starting with `ENV_`. It treats GitHub Secrets and GitHub Variables exactly the same.
1. **Discovery**: The action scans environment variables starting with `ENV_` or a literal `ENV` secret.
2. **Bucketing**:
- If `env_files_patterns` is provided, the action **only** looks for variables matching those specific buckets (e.g., `env_files_patterns: .env.app` only processes `ENV_APP_...` variables).
- If `env_files_structure` is `auto` (and patterns are default), it automatically discovers all buckets based on your variable prefixes (e.g., `ENV_REDIS_URL` automatically creates a `.env.redis` file).
- **Literal `ENV`**: Treated as a non-prefixed global blob. Its contents go directly into the `.env` file without any modifications.
- **`ENV_COMPONENT_...`**: Treated as part of a specific component (e.g., `ENV_APP_...` goes to `.env.app`).
- **`ENV_ENVIRONMENT_...`**: Automatically identifies prefixes matching common environments (`PROD`, `STAGING`, `DEV`, `TEST`) **plus** your current `environment` input.
3. **Generation**: Files are generated on the remote server with secure permissions.

## Secret Naming Convention

### Literal Global Blob (No Prefix)
If you want to dump a list of variables without any prefixing or component logic, use the literal secret name `ENV`.

```bash
# GitHub Secret: ENV
PORT=8080
DEBUG=true
```
**Result**: `.env` contains `PORT=8080` and `DEBUG=true`.

### Individual Variables

```bash
Expand All @@ -41,6 +53,7 @@ ENV_APP_SECRET_KEY=base-key
ENV_DATABASE_HOST=localhost

# Environment-specific (higher priority)
# If environment input is 'prod', this will override base values
ENV_PROD_APP_SECRET_KEY=prod-secret
ENV_PROD_DATABASE_HOST=prod-host
```
Expand All @@ -62,121 +75,38 @@ SECRET_KEY=dev-secret
"
```

## Usage Examples
## Priority System

### Example 1: Single .env File
The priority system ensures proper variable overriding (last one wins):

```yaml
# GitHub Secrets:
# ENV_APP_DEBUG=false
# ENV_APP_SECRET_KEY=abc123
# ENV_DATABASE_HOST=localhost
1. **Literal `ENV`** (Lowest priority - Base Layer)
2. **Base Component secrets** (e.g., `ENV_APP_...`)
3. **Base All-in-one secrets** (e.g., `ENV_APP="..."`)
4. **Env-specific Component secrets** (e.g., `ENV_PROD_APP_...`)
5. **Env-specific All-in-one secrets** (Highest priority - `ENV_PROD_APP="..."`)

- uses: ./
with:
env_files_generate: 'true'
env_files_structure: 'single'
environment: 'prod'
```

**Result**: Creates `/project/.env` with all variables merged.
## Usage Examples

### Example 2: Flat Mode with Individual Secrets
### Example 1: Custom Environment Names
MetalDeploy automatically supports your custom environment names for prefixes.

```yaml
# GitHub Secrets:
# ENV_APP_DEBUG=false
# ENV_APP_SECRET_KEY=abc123
# ENV_DATABASE_HOST=localhost
# ENV_REDIS_URL=redis://localhost:6379

- uses: ./
with:
env_files_generate: 'true'
env_files_structure: 'flat'
env_files_patterns: '.env.app,.env.database,.env.redis'
environment: 'prod'
environment: 'qa' # Custom environment name
```
**Secrets**:
- `ENV_APP_PORT=8080` (Base)
- `ENV_QA_APP_PORT=9090` (Overrides base because environment is 'qa')

**Result**: Creates `.env.app`, `.env.database`, and `.env.redis` in project root. Use `env_files_path` to override base directory.

### Example 3: Nested Mode with Priority System
**Result**: `.env.app` will have `PORT=9090`.

### Example 2: Single Mode
```yaml
# GitHub Secrets:
# ENV_APP_DEBUG=true
# ENV_PROD_APP_SECRET_KEY=prod-secret
# ENV_PROD_APP="DEBUG=false\nDATABASE_URL=postgresql://..."

- uses: ./
with:
env_files_generate: 'true'
env_files_structure: 'nested'
environment: 'prod'
```

**Result**: Creates `.envs/prod/.env.app` with merged variables:
- `DEBUG=false` (from ENV_PROD_APP)
- `SECRET_KEY=prod-secret` (from ENV_PROD_APP_SECRET_KEY)
- `DATABASE_URL=...` (from ENV_PROD_APP)

**With `env_files_create_root: true`:**
Also creates a single `/project/.env` file containing ALL variables merged together.

## File Structure Examples

### Single Mode
```
project/
├── .env # All variables in one file
├── app.py
└── requirements.txt
```

### Flat Mode
```
project/
├── .env.app # APP_* variables
├── .env.database # DATABASE_* variables
├── .env.redis # REDIS_* variables
└── app.py
```

### Nested Mode
```
project/
├── .envs/
│ ├── dev/
│ │ ├── .env.app
│ │ └── .env.database
│ └── prod/
│ ├── .env.app
│ └── .env.database
└── app.py
env_files_structure: 'single'
```

## Priority System

The priority system ensures proper variable overriding:

1. **Base secrets** (lowest priority):
```
ENV_APP_DEBUG=false
ENV_DATABASE_HOST=localhost
```

2. **Environment-specific secrets** (higher priority):
```
ENV_PROD_APP_SECRET_KEY=prod-secret
ENV_PROD_DATABASE_HOST=prod-host
```

3. **All-in-one environment-specific** (highest priority):
```
ENV_PROD_APP="DEBUG=false\nSECRET_KEY=prod-override"
```

4. **All-in-one base** (fallback):
```
ENV_APP="DEBUG=true\nVERSION=1.0"
```
**Result**: Creates `/project/.env` with all variables merged. Component variables are prefixed (e.g., `APP_PORT`, `DB_HOST`) to prevent collisions, while literal `ENV` variables remain un-prefixed.
28 changes: 20 additions & 8 deletions src/env_manager.py
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,11 @@ def detect_file_patterns(

# Determine if it's environment-specific
matched_env = ""
for env in ["PROD", "STAGING", "DEV", "TEST", "PRODUCTION"]:
# Dynamic check for current environment + common fallbacks
env_candidates = list(set([env_upper, "PROD", "STAGING", "DEV", "TEST", "PRODUCTION"]))
for env in env_candidates:
if not env:
continue
if var_name.startswith(f"ENV_{env}_"):
matched_env = env
break
Expand Down Expand Up @@ -204,10 +208,8 @@ def merge_env_vars_by_priority(
continue

# Skip environment-specific ones
if any(
k.startswith(f"ENV_{e}_")
for e in ["PROD", "STAGING", "DEV", "TEST", "PRODUCTION"]
):
env_candidates = [env_upper, "PROD", "STAGING", "DEV", "TEST", "PRODUCTION"]
if any(k.startswith(f"ENV_{e}_") for e in env_candidates if e):
continue

key = k[4:] # e.g. APP or APP_PORT
Expand Down Expand Up @@ -296,7 +298,7 @@ def detect_environment_secrets() -> Dict[str, Dict[str, str]]:
all_env_vars = {
k: v
for k, v in os.environ.items()
if k.startswith("ENV_") and not k.startswith("ENV_FILES_")
if (k.startswith("ENV_") or k == "ENV") and not k.startswith("ENV_FILES_")
}
if not all_env_vars:
return {}
Expand All @@ -310,10 +312,20 @@ def detect_environment_secrets() -> Dict[str, Dict[str, str]]:
patterns = [p.strip() for p in config.ENV_FILES_PATTERNS if p.strip()]

result = {}
if "ENV" in all_env_vars:
# Treat ENV as a global blob that goes into .env (or first pattern) without prefixing
parsed = parse_all_in_one_secret(all_env_vars["ENV"], config.ENV_FILES_FORMAT)
if parsed:
target_file = patterns[0] if patterns else ".env"
result[target_file] = parsed.copy()

for pattern in patterns:
merged_vars = merge_env_vars_by_priority(all_env_vars, config.ENVIRONMENT, pattern)
if merged_vars:
result[pattern] = merged_vars
if pattern in result:
result[pattern].update(merged_vars)
else:
result[pattern] = merged_vars
return result


Expand All @@ -339,7 +351,7 @@ def generate_env_files(conn) -> None:
all_env_vars = {
k: v
for k, v in os.environ.items()
if k.startswith("ENV_") and not k.startswith("ENV_FILES_")
if (k.startswith("ENV_") or k == "ENV") and not k.startswith("ENV_FILES_")
}
env_file_data = detect_environment_secrets()
if not env_file_data:
Expand Down