Dependency Installer#20
Merged
Merged
Conversation
added 30 commits
June 23, 2025 00:05
**Major**: - Rewrote completely the logic of `add_package_to_environment` to leverage the new API in `hatch_validator` - This allowed to completely remove duplicated code for dependency resolution. - In addition, we are making use of the package and registry services to leverage automatic handling of the different schema versions. - Also updated the tests to have a better mock registry as it was necessary with all the checks in place to control the registry's structure.
**Major**: - Acknowledge successfully refactoring to use the new `hatch_validator`
**Major**: - `DependencyInstaller` Abstract base class for dependency installers. - `InstallationContext` (data class), `InstallationStatus` (Enum), `InstallationResult` (data class) - Added `test_installer_base.py` and integrated it with `run_environment_tests.py`
**Major**: - The concrete implementation to install hatch depencies of a hatch package - Added tests to check installation assuming the validation checks were performed before hand (i.e. truly instal-ready)
**Major**: - The concrete implementation to install python depencies of a hatch package relying on "pip" - Added tests to check installation assuming the validation checks were performed before hand (i.e. truly instal-ready) - Tests are covering mocking of `_run_pip_subprocess` to make sure the system to capture some error types is working - Then integration tests are building fake python dependencies and having them being installed in a `venv` created in teh `setup` function
**Major**: - The concrete implementation to install system depencies of a hatch package relying on "apt" - I don't really know how long this one will survive given the limitations of "apt" only running with sudo. If we support them, it's open doors to having MCP servers running only in specific environments. - Added tests to check installation assuming the validation checks were performed before hand (i.e. truly instal-ready) - Tests are mainly using Mockup given the constrainsts of needing `sudo`(as mentioned above). - That being said, the current tests do trigger once a sudo password prompt to the user in basic environment with the NOPASSWORD is not setup. - I think it will be a pain to integrate within the docker container...let's see!
**Major**: - The concrete implementation to install docker depencies of a hatch package relying on `docker-py` - Added the dependency to `docker-py v7.1.0`, currently the latest version of the SDK - Added tests to check installation assuming the validation checks were performed before hand (i.e. truly instal-ready) - Tests are doing mockup of the individual functions - and testing that some small images can be pulled.
**Major**: - Added the singleton `InstallerRegistry` to have access to all dependency installers added previously - Updated every installer to register themselves to the registry at the end of the code. - Meaning installers are registered as soon as they are imported once in the codebase. - Added tests to check registration
**Major**: - Added the singleton `DependencyInstallerOrchestrator` to coordonate the retrieval of deps of a package, install all deps and the package itself. - Added tests to check the dummy orchestrator can install the dummy packages with all types of dependencies.
**Major**: - Was passing `environment_path=env_path / env_name` in `InstallationContext` of `DependencyInstallerOrchestrator` - But later parts of the code already supports adding `/ env_name` so that was building paths with two `env_name` directories
**Major**: - Was returning `status="COMPLETED"` in `InstallationResult` of `HatchInstaller` instead of the enum symbol `InstallationStatus.COMPLETED`
**Major**: - Had forgotten to use `self.validate_dependency(dependency)` at the top of the `install` and `uninstall` functions.
**Major**: - Added an argument `dep_type` to facilitate passing this information when `can_handle` is used.
**Major**: - `test_env_manip.py` - Skipping `test_add_package_with_system_dependency` is system is Windows - `test_hatch_installer.py` - Asserts use `InstallationStatus.COMPLETED` instead of "COMPLETED" as this was updated in 7809d8c - `test_installer_base.py` - Fixed wrong attempt to `str(error)` to using `error.message` - Honestly, this test is not so relevant... - `test_installer_base.py` - Must convert the `Path` with `str()` to be able to compare the values in assert as expected - `test_online_package_loader.py` - Adapted the value of the environment directory to test packages presence as expected - Add the `auto_approve=True` to `add_package_to_environment` to skip user consent - `test_system_installer.py` - skipping `test_verify_installation_success`, `test_simulate_installation_success`, `test_simulate_curl_installation`, `test_install_real_dependency` on Windows **Minor**: - `test_env_manip.py` - Made better use of `HatchEnvironmentManager` initialization arguments rather than changing the member variables manually. - `test_online_package_loader.py` - separated `version` from `version_control` to assert things as expected
**Major**: - Was already doing it locally but pushing the change to test the whole import pipeline.
**Major**! - Updating the toml to discover the packages automatically (setuptool) - Aded the imports to all installer such that the installer registry will be populated immediately upon import
**Major**: - Filtering the installed package insformation at the step of the `environment_manager.py` at the end of `add_package_to_environment` to only add the hatch packages in the environment information file. - Adapted the value of the field `type` of the package installation result to be able toretrieve expected package type. - Adapted the tests `test_env_manip`
**Major**: - WARNING New dependency on conda/mamba environment management features - Added methods for creating, removing, and validating Python environments using conda/mamba. - Implemented cross-platform detection of conda/mamba executables with fallback logic. - Added support for specifying Python versions during environment creation. - Integrated shell launching functionality for interactive or command-based execution within environments. **Minor**: - Enhanced environment metadata with detailed diagnostics, including Python version, packages, and platform info. - Implemented methods for retrieving Python executable paths and environment activation variables. - Added diagnostics methods for both manager-level and environment-level troubleshooting.
**Major**: - Integrated PythonEnvironmentManager for managing Python environments within Hatch environments. - Enhanced environment metadata structure with detailed Python environment information, including: - `python_env` object with fields for conda environment name, Python executable, version, and manager. - Updated `create_environment` and `remove_environment` methods to handle Python environments. - Implemented `create_python_environment_only` and `remove_python_environment_only` methods for standalone Python environment management. - Updated `set_current_environment` to configure the Python executable for dependency installation.
**Major**: - Covers all the new API for python environment management and information printing.
**Major**: - Mocking, Integration, and CLI tests - The tests for the shell commands are passing, but shell activation via hatch command may fail on windows with mamba, if the user has not initiatest its shell first
**Major**: - Mostly to facilitate testing - Differentiated the case where the docker library is available (`DOCKER_AVAILABLE`) and the case where the daemon is avaialble (`DOCKER_DAEMON_AVAILABLE`) - We check availability of the daemon only if the library is installed
**Major**: - Increased the number of places where we are looking for mamba and conda exe to improve robustness of automatic discovery - It still remains deterministic, however. - I searched the options given by mamba, hoping to find a field indicating the path, but I could find anything satisfying. - In the future, we might include a hatch launch setting to let users give a path directly. - Refactored the discovery function to be interchangeable for conda and mamba since it's the same up to a name. **Minor**: - Updated the tests to match the new refactored function.
**Major**: - Using the result of python environment manager, we inject the associated python executable to the pip command used to install pacakges **Minor**: - Updated `run_environment_tests.py` to include all the tests again - Updated `test_env_manip` to set `create_python_env=False` when it is not needed for the test in order to speed up by skipping creation of thee environment
**Major**: - The previous implementation was relying on the python's executable path to hope that the correct environment would be picked to install the packages - To make this more robust, we are passing the general information about the python environment by setting the necessary environment variables - FIXED a bug in the tests where we were not actually switching to the correct environment after creating it. - The tests were passing thanks to the system's python - Now the test `test_add_package_with_mixed_dependency_types` will fail if the expected environment is not used **Minor**: - Deleted deprecated commented code
**Major**: - Added the list of packages and versions - Deleted some logs - Changed the level of logs to debug
**Major**: - Changed the default log level of the docker availability issue from warning to debug. - Changed the logger name to the conventional of the path of the file (indicated manually)
**Major**: - Use the value of `DOCKER_DAEMON_AVAILABLE` instead of rewriting the whole test
**Major**: - The CLI commands were expecting the hatch environment name as a positional argument for the python-related commands - We made the name optional and use the current environment by default **Minor**: - Homogenized the format of the import statements
**Major**: - The checks for the freshness of the registry was performed against a local variable during initialization. - Which means that every single use of hatch via the CLI would trigger download of the registry. - This was fixed/enhanced to save the last update time in a local file
**Major**: - Using `--prefix` to create the python environments associated to hatch environments was causing issues when under the directory of the hatch environment - These issues were encountered specifically during rebuild process of Hatchling image - Therefore it is a global bug for something that was not necessarily an issue with other deployment means - However it just means that we will let miniforge manage the environments which is not a bad thing either - Hence, now the API was updated to use `-n` (name) - Tests were updated to reflect the changes. All are passing.
**Major**: - `_get_conda_env_path` had been removed in `python_environment_manager.py` because it was a simple indirection to `get_environment_path` - But I had missed an occurence of the old function - This revealed that `get_environment_activation_info` was untested - This is now included in the test
**Major**: - `hatch_mcp_server` is the wrapper of the Cracking Shells ecosystem around MCP server. - It was recently added as a package in the ecosystem: https://github.com/CrackingShells/Hatch-MCP-Server - It is necessary in most cases to add this package by default when creating a new hatch environment, which has a python environment - We give CLI arguments control to opt out of the installation of this package - We give a CLI argument to control the git tag (branch name of actual tag) which should be installed.
**Major**: - We were reading the python stdout and std err after it became available by capture which was completely irrelevant for our purpose and was rather clutering the output - Removed them from `_run_pip_subprocess` - Tests slightly adapted for the mock result's return value - Except this everything is passing and printing in the console works as intended
**Major**: - Similarly to how we can control the initialization of a python environment for a single hatch environment, we added the possibility to only add `hatch_mcp_server` - Actually, this will also allow to upgrade it when new version arrives, or pin it to another version. **Minor**: - Uncommented the tests for the environment manipulations
**Major**: - Removed the argument `sudo: bool=True` in `_build_apt_command` because this builds an `apt install` command which always requires to be sudo - Updated the command to include `sudo apt-get update` at the beginning before starting to install any package to make sure the registry copy is up to date. **Minor**: - Default log level in `system_installer.py` moved from `DEBUG` to `INFO` - Removed useles commented code
**Major**: - Reprinting manually the commands' output was clutering the terminal's information - The benefits, on the other side were very low (just some output recording without doing anything with it. - The only function for which capturing the `stdout` was useful was `_verify_installation` to capture the version numbers. - We reimplemented it locally to work the same way but not disturb the general usage of `_run_apt_subprocess` - Updated the tests - All are passing
**Major**: - Given that all the default is to import `hatch_mcp_server` for Hatch MCP packages, any Hatch MCP package expect to have a python environment set up with this package. - So, the default should allow that (after all, it's the default...)
**Major**: - Old bug discovered only now - It happened when we fiddled wiht the Docker availability global parameters
**Major**: - In the `DependencyInstallationOrchestrator` we regorganize the dictionary of deps to guarentee that system depedencies will be processed first **Minor**: - Added debug logs - Added default values for some `.get(...)` calls on dicts
**Major**: - Python environment was configured by default only when no environment files existed. - It would not be correctly configured on second usage forward of the package manager... - The fix is simply to move the `_configure_python_executable(...)` at the end of the `__init__()`
**Major**: - HatchMCP is not in `hatchling` anymore but in `hatch_mcp_server`
**Major**: - Just face the issue on a slow internet connection that I couldn't initialize the environment - It makes no sense to hardcode timeouts without a clear rationale. And there is none for that one. - And it would take too long to make it a parameter instead of the whole Hatch for the current gain.
This logic has been stable for several weeks and used routinely in Hatchling dev branches. Merging the PR.
**Major**: - Copied over directly from Hatchling. - The only difference is that we renamed Hatchlings direcotry `/scripts` to `/versioning` for more clarity about what the folder contains.
Versioning system for automated version bumps
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request introduces solutions that:
This required many fixes and adaptation during the development. (the commit refs above are references to the first introduction of the features.
#10 is also supported only in so far as the docker image can indeed be pulled, but the packe metada is currently lacking information on how the user might want to run the image, which makes hardly usable for now (see CrackingShells/Hatch-Schemas#20)
Other improvements include:
Hatch CLI Enhancements:
hatch env pythonfor initializing, inspecting, removing, and launching Python environments, as well as installing thehatch_mcp_serverwrapper. [1] [2]hatch_mcp_serverinstallation. [1] [2]CI/CD and Versioning Automation:
Added new GitHub Actions workflows:
.github/workflows/commit_version_tag.yml: Automates committing updatedVERSIONfiles and creating version tags on pushes to key branches..github/workflows/tag-release.yml: Publishes GitHub releases for new tags, distinguishing between pre-releases and standard releases based on tag naming..github/workflows/tag-cleanup.yml: Provides a manual (and optionally scheduled) workflow for cleaning up old tags, with a dry-run option..github/workflows/test_build.yml: Standardizes the build and version update process, including updating version files, building, and testing the package.Updated versioning files:
VERSIONandVERSION.meta: Now include structured version information, branch, and build metadata to support automated CI/CD. [1] [2]