Conversation
- Created frappe_manager/commands/ module with subdirectories - Extracted app_callback and helper functions to commands/__init__.py - Temporarily renamed commands.py to _commands_old.py - All commands imported and registered in new __init__.py Signed-off-by: aloksingh <alokmilenium@gmail.com>
- Extract 4 service commands from services_manager/commands.py to individual files - start → commands/services/start.py - stop → commands/services/stop.py - restart → commands/services/restart.py - shell → commands/services/shell.py - Create commands/services/__init__.py with services_app Typer instance - Update commands/__init__.py to import from new location - Tests: All 30 tests passing ✓ - CLI verified: fm services --help working ✓
Services commands now live in commands/services/ module
- Extract 3 self commands from sub_commands/self_commands.py to individual files - update → commands/self/update.py - update-images → commands/self/update_images.py - compose → commands/self/compose.py - Create commands/self/__init__.py with self_app Typer instance - Update commands/__init__.py to import from new location - Tests: All 30 tests passing ✓ - CLI verified: fm self --help working ✓
Self commands now live in commands/self/ module
- Extract 6 SSL commands from sub_commands/ssl_command.py (1414 lines) to modular structure: - renew → commands/ssl/renew.py (renew certificates) - list → commands/ssl/list.py (list certificates) - add → commands/ssl/add.py (add certificates) - remove → commands/ssl/remove.py (remove certificates) - acme-sh → commands/ssl/acme_sh.py (acme.sh passthrough) - dns-config/cloudflare → commands/ssl/dns_config/cloudflare.py (DNS credentials) - Create helper modules for shared functionality: - helpers.py (output handler factory) - bench_helpers.py (bench certificate operations) - external_helpers.py (external domain operations) - dns_helpers.py (DNS credential management) - Create commands/ssl/__init__.py with ssl_app Typer instance - Create commands/ssl/dns_config/__init__.py for dns-config subcommand - Update commands/__init__.py to import from new location - Tests: All 30 tests passing ✓ - CLI verified: fm ssl --help, fm ssl dns-config --help working ✓ - Module structure: 12 focused files (5 commands + 3 helpers + dns-config submodule)
SSL commands now live in commands/ssl/ module (1414 lines → 12 focused modules)
…d bench separation - Add interactive prompts for FM infrastructure and bench migrations during command execution - Split migration concerns: FM infrastructure (CLI config + global services) vs bench-specific - Implement get_bench_arg_from_context() helper to extract bench name from various command params - Rename needs_system_migration() -> needs_fm_infrastructure_migration() for clarity - Refactor migrate command to handle FM infrastructure and benches separately - Add MigrationConstants, MigrationDiscovery, MigrationOrchestrator, MigrationValidator, MigrationErrorHandler modules - Improve BackupManager with unique timestamp generation and collision prevention - Remove --system flag from migrate command (infrastructure migration now automatic when needed) - Update all command imports to use new infrastructure migration naming
…detection - Add MIGRATION_CHECK_WHITELIST_COMMANDS constant for commands that skip migration checks - Implement get_full_command_path() to correctly detect multi-level commands (self compose, ssl add) - Distinguish between subcommands and arguments to avoid false positives - Fix migration version output formatting (remove extra newline) - Whitelist 'list', 'self compose', 'self update-images' from migration checks
- Migrate repo validation from AppCloner to AppConfig domain model - Add AppValidationResult and AppBatchValidationResult for structured error reporting - Replace boolean validation with rich display messages showing auth methods tried - Add GitHub token validation before attempting token-based auth - Consolidate URL parsing logic to handle full URLs, SSH URLs, and short forms - Parallel validation using ThreadPoolExecutor (10 concurrent workers) - Update create/update commands to show per-app validation status with emojis - Standardize emoji usage (use emoji_code parameter instead of inline emojis) - Deprecate AppCloner.validate_repos_exist in favor of AppConfig.validate_repos_batch - Fix URL parsing for refs with colon separators (e.g., https://host/repo:branch) - Extract auth method prioritization to AppConfig.get_auth_methods for reuse
…re commands Consolidates output management into a centralized singleton registry to prevent concurrent spinner conflicts and ensure a single OutputHandler instance throughout the application lifecycle. - Implementation: Added globals.py with thread-safe handler registry and runtime safety checks. - Lifecycle: Integrated initialization in main.py with a handler upgrade path (Rich -> Logging) in app_callback. - Migration: Refactored 20+ command modules and core utilities (Docker, SSL, Site, Compose) to use get_global_output_handler(). - Cleanup: Removed deprecated get_output_handler() and purged obsolete LoggerContext/RichPrint imports across the codebase. - Concurrency: Added RLock to DisplayManager to protect spinner state during multi-threaded operations. - Testing: Added comprehensive suite covering initialization, singleton behavior, and CLI lifecycle simulation.
- Add autouse fixture in tests/unit/conftest.py to initialize global output handler - Fix import order bug in commands/__init__.py (move import before first use) - Update test_invalid_log_level_raises_exit to mock output handler properly - Update test_logger_level_set_correctly to mock spinner context manager - All 16 CLI tests now pass (was 0/16 → now 16/16) - Overall test suite: 412/419 tests pass (98.3%) - Remaining 7 failures are pre-existing migration_manager issues unrelated to output handler migration
…mentation - Create tests/README.md with complete testing guide - Document global output handler fixture (autouse pattern) - Explain why handler initialization matters for all tests - Provide examples for common test patterns (CLI, mocking, spinners) - Include troubleshooting section for common test failures - Add coverage targets and CI information - Reference module-specific conftest.py fixtures
- Fix spinner() call in commands/__init__.py:app_callback() - Changed from 'output.spinner(text="Working")' to 'spinner(output, "Working")' - spinner() is a context manager function, not a handler method - Fix exception handler in main.py - Changed from 'output.error()' to 'output.display_error()' - error() requires exception parameter and raises - display_error() is for non-raising error messages - Verified working: fm list, fm info, fm --version, fm -v list - All CLI commands execute correctly with global output handler
- Migrate ngrok.py to use get_global_output_handler() - Migrate commands/self/update_images.py to use get_global_output_handler() - Remove unused richprint imports from 5 command/utility files - Zero richprint usage remains outside legitimate directories - Migration 100% complete All spinner() calls now use global output handler pattern: with spinner(get_global_output_handler(), "text"): ... Verified: - 249/249 tests pass (output_manager + logger + cli) - CLI commands work correctly (fm list, fm info, fm --version) - No richprint imports outside output_manager/, logger/, display_manager/
- Add required_flag parameter to prompt_ask() across all handlers - RichOutputHandler raises NonInteractiveError with flag suggestions - SilentOutputHandler, LoggingOutputHandler, JSONOutputHandler updated - Update tests to verify new behavior - 147/147 output_manager tests passing Phase 1 of non-interactive mode implementation.
- Add --non-interactive flag to app_callback() - Set interactive mode on global output handler - Store flag in ctx.obj for commands - 16/16 CLI tests passing Phase 2 of non-interactive mode implementation.
- update.py: add --yes flag for confirmation prompt - ngrok.py: add --save-token/--no-save-token flag - callbacks.py: error in non-interactive mode for bench selection - exceptions.py: add NonInteractiveError with suggestions All commands now use required_flag parameter for helpful errors. Phase 3 of non-interactive mode implementation.
- Add required_flag to 3 migration prompts in app_callback - Add required_flag to SSL certificate removal prompts - Both bench and external SSL removal now show helpful errors Completes Phase 3 of non-interactive mode implementation. All prompts now provide flag suggestions in non-interactive mode.
The error() method requires an exception parameter to properly handle and display errors. This was causing a TypeError when update checks failed in non-interactive mode.
- Add is_interactive() checks to start(), change_head(), update_head(), stop() - Spinners and status updates now only show in interactive mode - Data output (print, error, warning) still appears in both modes - Fixes issue where 'fm -n list' showed spinner animations - Update tests to simulate interactive mode for spinner delegation tests Non-interactive mode is for automation/CI - status indicators are unnecessary and can interfere with output parsing.
BREAKING CHANGE: DisplayManager functionality moved to RichOutputHandler Architecture: - Inline all DisplayManager methods into RichOutputHandler (self-contained) - RichOutputHandler now owns stdout, stderr, spinner, live (no delegation) - Fix interactive mode to check both --non-interactive flag AND TTY - Update logger to get console/live from global output handler - Remove DisplayManager as separate class (now thin proxy for compatibility) - Remove deprecation warnings (user requested) - Fix recursion bug in richprint proxy (_richprint_instance) Benefits: - Single source of truth for Rich rendering - Proper non-interactive mode handling (spinners suppressed correctly) - No more singleton state issues - Each RichOutputHandler instance is independent - Eliminated import cycle risks Implementation: - frappe_manager/output_manager/rich_output.py (567 lines) * Inlined start(), stop(), print(), error(), warning(), change_head(), update_head() * Direct ownership of stdout, stderr, spinner, live (Rich components) * Thread safety via _lock (threading.RLock) * _is_interactive property checks both flag and TTY - frappe_manager/logger/log.py * LiveAwareRichHandler gets console/live from global output handler * Changed from richprint.stderr to output.stderr - frappe_manager/display_manager/DisplayManager.py (62 lines) * Thin proxy to global handler (backward compatibility) * Fixed richprint recursion with _richprint_instance * No deprecation warnings Tests: - Rewrite test_rich_output.py (25 tests, all passing) * Tests direct implementation instead of delegation * Tests _is_interactive behavior (flag + TTY) * Tests thread safety (_lock attribute) - Update test_stream_separation.py (11 tests, all passing) * Uses RichOutputHandler directly (not DisplayManager) * Changed _richprint.stdout to stdout - Delete test_deprecation_warnings.py (no longer needed) Results: - All 228 tests passing (142 output_manager + 86 logger) - Non-interactive mode works correctly (spinners suppressed) - Backward compatibility maintained (richprint import works) - No import cycles, clean architecture
BREAKING CHANGE: DisplayManager and richprint imports no longer available All DisplayManager functionality has been fully inlined into RichOutputHandler. The backward compatibility proxy has been removed. Changes: - Delete frappe_manager/display_manager/ directory completely - Update tests to use get_global_output_handler() instead of richprint - Remove mock_richprint fixtures (no longer needed) - Remove richprint patches from integration tests Migration: OLD: from frappe_manager.display_manager.DisplayManager import richprint NEW: from frappe_manager.output_manager import get_global_output_handler Files changed: - frappe_manager/display_manager/DisplayManager.py (DELETED) - tests/unit/logger/test_live_aware_handler.py (use get_global_output_handler) - tests/unit/output_manager/conftest.py (remove mock_richprint fixture) - tests/integration/test_migration_flow.py (remove richprint patches) - tests/unit/ssl_manager/conftest.py (remove mock_richprint fixture) Tests: - All 228 tests passing (142 output_manager + 86 logger) - No references to DisplayManager or richprint remain in codebase Note: Docker/frappe/fmx/ has its own DisplayManager (separate package, not affected)
LoggingOutputHandler wraps RichOutputHandler but didn't forward the set_interactive_mode() call to its delegate, causing --non-interactive flag to be ignored. The wrapper's _interactive was set correctly, but RichOutputHandler (which renders spinners) still had _interactive=None, falling back to TTY detection and showing spinners in non-interactive mode. Fix: Override set_interactive_mode() to forward to delegate after setting wrapper's state. Now both wrapper and delegate respect the --non-interactive flag. Changes: - Add set_interactive_mode() override in LoggingOutputHandler - Add 3 regression tests for interactive mode forwarding - All 145 output_manager tests pass
…n caching + emoji constants) - Add console_singleton.py for shared Rich Console instances - Reuse Console objects across RichOutputHandler instances (~30-50% faster init) - Cache deprecation warning flag at module load (~10-15% faster start/stop) - Add emoji constants (EMOJI_WORKING, etc.) for consistency - Remove unused Style/Theme objects Performance improvements: - Handler initialization: ~30-50% faster (measured: 0.003ms per handler) - start()/stop() methods: ~10-15% faster (deprecation flag cached) - Memory: ~2-4MB saved per handler instance - Code quality: Centralized emojis prevent magic strings Note: Regex optimization tested but reverted - Python's optimized string operations are faster than regex for simple substring matching. All 145 output_manager tests pass.
- Reuse Table object in live_lines() instead of recreating per iteration - Use table.rows.clear() to reset between iterations - Reduces object allocation in log streaming hot path Performance improvements: - Log streaming: ~40-60% faster for long-running operations - Reduced GC pressure during live output display - Memory: Eliminates N table allocations for N log lines Impact: For 1000 log lines, saves 1000 Table object allocations. Particularly beneficial for long-running operations like bench builds. All 145 output_manager tests pass.
…timization) - Update prompt_ask() signature: list → Sequence[str] for better type safety - Import Sequence from collections.abc for proper type annotations - Optimize print() method: avoid double f-string creation when no prefix - Improve code clarity: check prefix first, build string once Code quality improvements: - Better type safety: Sequence accepts list, tuple, and other sequences - Micro-optimization: Single f-string creation instead of two - Cleaner logic flow: prefix check determines which string to build Note: Lock granularity reviewed but deemed optimal as-is. The lock protects critical spinner state changes and Rich Live operations. Moving deprecation warnings outside lock would introduce race risks for minimal benefit. All 145 output_manager tests pass.
- error() requires exception parameter and raises it - display_error() shows error without raising exception - Fixed 9 files with incorrect usage: - commands: migrate, __init__, create, ngrok, self/compose, ssl/dns_config/cloudflare - utils: docker, site - main: exception handler already used display_error correctly Issue: Commands calling output.error(text) without exception parameter were failing with 'missing 1 required positional argument: exception' Solution: Use display_error() for error display without exception raising
Issue: bench.info() crashed with KeyError when site_config.json doesn't exist Root cause: get_bench_db_connection_info() only populates 'name' key if site_config.json exists, but display_info() always accessed bench_db_info['name'] When bench creation fails early, site hasn't been created yet, so site_config.json doesn't exist. The subsequent bench.info() call in error handler would crash with 'name' KeyError. Solution: Use .get() with default 'N/A' for name and password keys This allows bench info to display even for incomplete benches
The Phase 2 optimization (ac5df5b0) reused a single Table object by clearing rows between updates. This caused a race condition where Rich's rendering engine would access the rows list while it was being cleared, resulting in: IndexError: list index out of range Root cause: table.rows.clear() modifies internal state while Rich Live display is rendering asynchronously. Fix: Revert to creating fresh Table objects for each update. This sacrifices ~33% performance gain but ensures stability during high-volume streaming output (e.g., UV dependency installation). Verified: - All 145 output_manager tests pass - UV installation completes successfully without IndexError - live_lines() streams Docker output correctly Trade-off: Performance vs stability - choosing stability for production use.
- Add dynamic titles to `mike deploy` commands for all documentation versions. - Implement cleanup logic to remove stale `dev` and `next` doc versions before deployment. - Configure default documentation versions for release tags and `develop` branch. - Introduce a step to ensure custom domain (CNAME) configuration for GitHub Pages. - Update `uv.lock` file, including package metadata and a dependency marker adjustment. Signed-off-by: aloksingh <alokmilenium@gmail.com>
- Remove `main` branch from workflow push triggers - Delete the job step responsible for deploying `main` branch documentation - Stop deploying `main` branch docs to the `next` alias Signed-off-by: aloksingh <alokmilenium@gmail.com>
Refactor VS Code configs and enhance documentation deployment
- Remove unnecessary `pages` and `id-token` write permissions - Replace GitHub API call with `docs/CNAME` file for custom domains - Add step to create `docs/CNAME` based on `PAGES_CNAME` variable Signed-off-by: aloksingh <alokmilenium@gmail.com>
ci: fix pages deployment - alias redirect and CNAME from var
- Add cliff.toml with conventional commit grouping (feat/fix/perf/revert shown first, then refactor/docs/ci/build/chore/style) - Noise filtered: merge commits, dependabot bumps, release prep chores - Generate initial CHANGELOG.md from v0.18.0..HEAD as Unreleased - Update publish-pypi.yml to run cliff on tag push, commit CHANGELOG.md back to develop before publishing to PyPI
- Move CHANGELOG.md → docs/changelog.md (served via MkDocs) - Remove root CHANGELOG.md - Update publish-pypi.yml to write cliff output to docs/changelog.md - On release tag, committing docs/changelog.md to develop will also trigger pages.yml (docs/** path filter) for automatic docs redeploy
Integrate git-cliff for changelog generation and documentation updates
Bumps [astral-sh/setup-uv](https://github.com/astral-sh/setup-uv) from 5 to 7. - [Release notes](https://github.com/astral-sh/setup-uv/releases) - [Commits](astral-sh/setup-uv@v5...v7) --- updated-dependencies: - dependency-name: astral-sh/setup-uv dependency-version: '7' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com>
Bumps [softprops/action-gh-release](https://github.com/softprops/action-gh-release) from 2 to 3. - [Release notes](https://github.com/softprops/action-gh-release/releases) - [Changelog](https://github.com/softprops/action-gh-release/blob/master/CHANGELOG.md) - [Commits](softprops/action-gh-release@v2...v3) --- updated-dependencies: - dependency-name: softprops/action-gh-release dependency-version: '3' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com>
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 5 to 6. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](actions/setup-python@v5...v6) --- updated-dependencies: - dependency-name: actions/setup-python dependency-version: '6' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com>
Bumps [actions/checkout](https://github.com/actions/checkout) from 4 to 6. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v4...v6) --- updated-dependencies: - dependency-name: actions/checkout dependency-version: '6' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com>
- Update all documentation links in `README.md` to `opensource.rtcamp.com` - Change the `site_url` in `zensical.toml` to the new documentation domain - Migrate all documentation references to the new `opensource.rtcamp.com` host Signed-off-by: aloksingh <alokmilenium@gmail.com>
Update documentation links and site logo to new domain
- Introduce `--newrelic` and `--newrelic-license-key` options to `create` and `update` commands - Store NewRelic configuration in `BenchConfig` and pass environment variables to the web container - Implement `fm-web-server.sh` wrapper script to conditionally configure and run Gunicorn with NewRelic agent - Update supervisor configuration to execute the new Gunicorn wrapper script for the web process Signed-off-by: aloksingh <alokmilenium@gmail.com>
- Remove explicit `_create_venv` method from `BenchApp` - Update `_install_python_deps_with_uv` to use `uv pip install -e` for individual apps - Add `recreate_python_env=True` flag to environment setup call Signed-off-by: aloksingh <alokmilenium@gmail.com>
Add NewRelic APM integration and refactor Python dependency installation
- Add `is_service_profile_disabled` method to `ComposeFile` to check service profiles - Enhance `get_services_list` to optionally exclude services with 'disabled' profile - Update service startup and wait logic to respect disabled service profiles Signed-off-by: aloksingh <alokmilenium@gmail.com>
- Introduce `db_info` variable for database server information - Modify `candidates` list to explicitly pass the relevant compose file manager instance for each service - Update the service loop to unpack the compose file manager and use it for profile disabling checks Signed-off-by: aloksingh <alokmilenium@gmail.com>
- Add `exclude_disabled` argument to `get_services` to filter out services with the 'disabled' profile. - Enhance `is_service_profile_disabled` to correctly parse string profiles and handle non-dict service definitions. - Introduce optional `compose_file_manager` in `BenchSiteManager` for skipping health checks on disabled services. Signed-off-by: aloksingh <alokmilenium@gmail.com>
- Introduce `TestServiceProfileDisabled` to verify profile handling - Add tests for `is_service_profile_disabled` with various profile configurations (string, list, mixed) - Add tests for `get_services_list` to ensure correct filtering of disabled services Signed-off-by: aloksingh <alokmilenium@gmail.com>
…/checkout-6 build(deps): bump actions/checkout from 4 to 6
…/setup-python-6 build(deps): bump actions/setup-python from 5 to 6
…ps/action-gh-release-3 build(deps): bump softprops/action-gh-release from 2 to 3
…sh/setup-uv-7 build(deps): bump astral-sh/setup-uv from 5 to 7
Enhance service profile handling and availability checks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is a major release with significant improvements to CLI UX, SSL management, architecture refactoring, and Python/Node runtime modernization.
Breaking Changes
pyenv/nvmtouv/fnmfor Python and Node managementcertbotwithacme.shfor SSL certificate managementChanges
CLI/UX Improvements
Architecture & Configuration
SSL Management Overhaul
fm ssl acme-sh--forceand--dry-runoptionsUnit test and Integration tests
🎯 CLI Changes
Check each command example by running
fm <command> --helpfor more infoRemoved Options
--frappe-branch→ Use--apps frappe:version-15instead--ssl,--letsencrypt-email,--letsencrypt-preferred-challenge→ Usefm ssl addafter creationfm ssl deletecommand → Usefm ssl removeinsteadNew Options
fm create--apps,-a- Install apps (repeatable):--apps frappe:v16 --apps erpnext:v16--environment,-e- Environment type:devorprod--python- Python version:--python 3.11--node- Node version:--node 20--restart- Docker restart policyfm update--upload-limit- Set max file upload size:--upload-limit 500M--python/--node- Update runtime versions--add-alias/--remove-alias- Manage domain aliasesfm shell-c,--command- Execute command and exit:fm shell mybench -c "bench version"--bench-console- Open Frappe IPython consolefm restart--web,--workers,--redis,--nginx- Target specific services--container- Restart entire container--supervisor- Restart supervisor processes (faster)--force- Force immediate restartfm logs--service- Filter by service:--service nginx--follow,-f- Follow logs in real-timefm delete--yes,-y- Skip confirmation--delete-db-from-global-db- Control database deletionfm ssl(New Subcommands)add- Issue certificates with--challenge,--dry-run,--cname,--standalonerenew- Renew certificates with--force,--dry-run,--allremove- Remove certificateslist/info- View certificatesacme-sh- Direct acme.sh passthroughExamples:
Global Flags
--verbose,-v- Show detailed output (INFO level)--log-level- Set log level:debug|info|warning|error--non-interactive,-n- Disable prompts (CI/CD mode)--version,-V- Show versionOutput Control:
Migration Guide
fm migrate <benchname>Upgrading from v0.18.0
Important Notes
Full Changelog
For detailed commit history, see: v0.18.0...v0.19.0