Skip to content

Commit a981322

Browse files
kozo2kozo2
committed
Add initial documentation for Hatchling, including README, chat commands, running instructions, and Docker setup
1 parent 5679d23 commit a981322

4 files changed

Lines changed: 299 additions & 1 deletion

File tree

‎README.md‎

Lines changed: 20 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1,20 @@
1-
# CrackingShells.github.io
1+
# CrackingShells.github.io
2+
3+
## Table of Contents (Hatchling)
4+
5+
### Users
6+
7+
#### Main Articles
8+
- Getting Started
9+
- [Chat Commands](hatchling-users/chat_commands.md)
10+
- [Running Hatchling](hatchling-users/running_hatchling.md)
11+
12+
#### Tutorials
13+
- [Docker Setup](hatchling-users/tutorials/docker-setup.md)
14+
15+
### Developers
16+
*Documentation for developers coming soon*
17+
18+
### Appendices
19+
- Glossary
20+

‎hatchling-users/chat_commands.md‎

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
This article is about:
2+
- Available chat commands in Hatchling
3+
- Basic commands for session management
4+
- Hatch environment and package management commands
5+
6+
You will learn about:
7+
- How to use basic commands like help, clear, and exit
8+
- Managing Hatch environments
9+
- Adding and removing packages
10+
11+
# Chat Commands
12+
13+
The following commands are available during chat:
14+
15+
## Basic Commands
16+
17+
| Command | Description | Arguments | Example |
18+
|---------|-------------|----------|---------|
19+
| `help` | Display help for available commands | None | `help` |
20+
| `clear` | Clear the chat history | None | `clear` |
21+
| `enable_tools` | Enable MCP tools | None | `enable_tools` |
22+
| `disable_tools` | Disable MCP tools | None | `disable_tools` |
23+
| `show_logs` | Display session logs | `[n]` - Optional number of log entries to show | `show_logs` or `show_logs 10` |
24+
| `set_log_level` | Change log level | `<level>` - Log level (debug, info, warning, error, critical) | `set_log_level debug` |
25+
| `set_max_tool_call_iterations` | Set maximum tool call iterations | `<n>` - Maximum iterations | `set_max_tool_call_iterations 10` |
26+
| `set_max_working_time` | Set maximum working time in seconds | `<seconds>` - Maximum time | `set_max_working_time 60` |
27+
| `exit` or `quit` | End the chat session | None | `exit` |
28+
29+
## Hatch Environment Management
30+
31+
| Command | Description | Arguments | Example |
32+
|---------|-------------|----------|---------|
33+
| `hatch:env:list` | List all available Hatch environments | None | `hatch:env:list` |
34+
| `hatch:env:create` | Create a new Hatch environment | `<name>` - Environment name <br>`--description <description>` - Environment description | `hatch:env:create my-env --description "For biology tools"` |
35+
| `hatch:env:remove` | Remove a Hatch environment | `<name>` - Environment name | `hatch:env:remove my-env` |
36+
| `hatch:env:current` | Show the current Hatch environment | None | `hatch:env:current` |
37+
| `hatch:env:use` | Set the current Hatch environment | `<name>` - Environment name | `hatch:env:use my-env` |
38+
39+
## Hatch Package Management
40+
41+
| Command | Description | Arguments | Example |
42+
|---------|-------------|----------|---------|
43+
| `hatch:pkg:add` | Add a package to an environment | `<package_path_or_name>` - Path or name of package<br>`--env <env_name>` - Environment name<br>`--version <version>` - Package version | `hatch:pkg:add ./my-package --env my-env` |
44+
| `hatch:pkg:remove` | Remove a package from an environment | `<package_name>` - Name of package to remove<br>`--env <env_name>` - Environment name | `hatch:pkg:remove my-package --env my-env` |
45+
| `hatch:pkg:list` | List packages in an environment | `--env <env_name>` - Environment name | `hatch:pkg:list --env my-env` |
46+
| `hatch:create` | Create a new package template | `<name>` - Package name<br>`--dir <dir>` - Target directory<br>`--description <description>` - Package description | `hatch:create my-package --description "My MCP package"` |
47+
| `hatch:validate` | Validate a package | `<package_dir>` - Path to package directory | `hatch:validate ./my-package` |

‎hatchling-users/running_hatchling.md‎

Lines changed: 149 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,149 @@
1+
This article is about:
2+
- Running Hatchling with Docker
3+
- Setting up and configuring Ollama for local LLMs
4+
- Understanding Hatchling's environment variables and configuration options
5+
6+
You will learn about:
7+
- How to start Ollama with CPU or GPU support
8+
- Downloading and building Hatchling from source
9+
- Configuring environment variables for your needs
10+
- Starting and using Hatchling with Docker Compose
11+
# Running Hatchling
12+
13+
This section assumes you have followed the [docker setup](./docker-setup.md).
14+
15+
In this guide, you will:
16+
17+
1. Start Ollama to have access to local LLMs later in Hatchling.
18+
2. Download, setup, and start Hatchling
19+
20+
## Running Ollama with Docker
21+
22+
### Using CPU or GPU for the LLM
23+
24+
* Windows/Linux **CPU only** & MacOS **on Apple Silicon**:
25+
26+
```bash
27+
docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama
28+
```
29+
30+
* Windows/Linux **NVIDIA GPU support**:
31+
32+
```bash
33+
docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama
34+
```
35+
36+
* Windows/Linux **AMD GPU support**:
37+
38+
```bash
39+
docker run -d --device /dev/kfd --device /dev/dri -v ollama:/ root/.ollama -p 11434:11434 --name ollama ollama/ollama:rocm
40+
```
41+
42+
### Checking that GPU support is enabled as expected
43+
44+
* Go to the `Containers` tab in Docker Desktop (arrow 1) and select your Ollama container
45+
![docker_desktop_find_container](../resources/images/docker-setup/docker_find_container.png)
46+
* Check the logs and search for a message indicating GPU detection, similar to:
47+
48+
```txt
49+
msg="inference compute" id=GPU-a826c853-a49e-a55d-da4d-804bfe10cdcf library=cuda variant=v12 compute=8.6 driver=12.7 name="NVIDIA GeForce RTX 3070 Laptop GPU" total="8.0 GiB" available="7.0 GiB"
50+
```
51+
52+
![docker_desktop_find_container_log](../resources/images/docker-setup/docker_find_container_log.png)
53+
* Alternatively, run the command `docker logs ollama` and search for the message in the output.
54+
55+
For more detailed instructions and options, refer to the [official Ollama Docker documentation](https://github.com/ollama/ollama/blob/main/docs/docker.md).
56+
57+
## Running Hatchling with Docker
58+
59+
### Get the source code
60+
61+
At this step, you will be downloading the content of Hatchling. Currently, we are only using GitHub's interface to install Hatchling.
62+
63+
* Open a terminal
64+
* Navigate to a directory where you want Hatchling to be:
65+
66+
```bash
67+
cd path/to/the/directory/you/want
68+
```
69+
70+
* Then, use Git, to retrieve the source code
71+
72+
```bash
73+
git clone https://github.com/CrackingShells/Hatchling.git
74+
```
75+
76+
### Navigate to the `docker` directory of Hatchling
77+
78+
```bash
79+
cd ./Hatchling/docker
80+
```
81+
82+
### Install Hatchling by building the code
83+
84+
```bash
85+
docker-compose build hatchling
86+
```
87+
88+
This typically takes 20 to 50 seconds depending on your computer.
89+
90+
### Start Hatchling
91+
92+
#### Configure Hatchling's Environment
93+
94+
Modify the variables in the `.env` file to suit your needs.
95+
96+
#### Configuration
97+
98+
Configuration is managed through environment variables or a `.env` file in the `docker` directory:
99+
100+
| Variable | Description | Default |
101+
|----------|-------------|---------|
102+
| `OLLAMA_HOST_API` | URL for the Ollama API | `http://localhost:11434/api` |
103+
| `OLLAMA_MODEL` | Default LLM model to use | `llama3.2` |
104+
| `HATCH_HOST_CACHE_DIR` | Directory where Hatch environments and cache will be stored on the host machine | `./.hatch` |
105+
| `HATCH_LOCAL_PACKAGE_DIR` | Directory where local packages are stored on the host machine to be accessible in the container | `../../Hatch_Pkg_Dev` |
106+
| `NETWORK_MODE` | Docker network mode | `host` (for Linux) |
107+
| `LOG_LEVEL` | The default log level at start up | `INFO` |
108+
| `USER_ID` | User ID for the container user (set on Linux to match host user for permissions) | `1000` |
109+
| `GROUP_ID` | Group ID for the container user (set on Linux to match host group for permissions) | `1000` |
110+
| `USER_NAME` | Username for the container user (optional, defaults to `appuser`) | `appuser` |
111+
112+
##### OLLAMA_HOST_API
113+
114+
You may need to adjust `OLLAMA_HOST_API` to match where your Ollama container is hosted. If you did not change this value and used the default from [earlier](#using-cpu-or-gpu-for-the-llm), then you don't need to change that variable either.
115+
116+
##### OLLAMA_MODEL
117+
118+
For hatchlings, one can change `OLLAMA_MODEL` to any model under the category [tools](https://ollama.com/search?c=tools)
119+
120+
> [!Warning]
121+
> Be mindful of the size of the LLM. Models larger than your GPU's memory (VRAM on GPU cards, or the partition of the shared memory that can be allocated to GPU tasks on Apple Silicon), will not run smoothly. You can check the actual size of a model when selecting a model on the within [the list](https://ollama.com/search?c=tools)
122+
123+
![deepseek_model_size_choice](../resources/images/running-hatchling/deepseek_model_page_example.png)
124+
125+
For example, [earlier](#checking-that-gpu-support-is-enabled--as-expected) the GPU's available memory was indicated to be 7GB. Therefore, this GPU can load up to `deepseek-r1:8b`, which happens to be the default (marked as `latest`).
126+
127+
#### Ollama
128+
129+
* On Docker Desktop, navigate to your containers (arrow 1), and press the play button (arrow 2)
130+
![start_ollama_container](../resources/images/docker-setup/Run_Ollama_Container.png)
131+
* Alternatively, run the command `docker start ollama`
132+
133+
#### Hatchling
134+
135+
> [!Note]
136+
> You can adapt the [environment variables](#configuration) to suit your needs (e.g. change the LLM) before you run the following command:
137+
138+
```bash
139+
# From the docker directory in your project
140+
docker-compose run --rm hatchling #The `--rm` flag ensures the container is removed when you exit the application.
141+
```
142+
143+
If Hatchling successfully connects to Ollama, it will download the specified LLM model. This will materialize by many prints indicating the download progress. Of course, the download time varies based on the model's size: the default model `llama3.2` takes about 2GB.
144+
145+
Here is a screenshot of what Hatchling typically looks like right after start up:
146+
![Typical_Hatchling_CLI_20250627_pt1](../resources/images/running-hatchling/Typical_Hatchling_CLI_20250627_pt1.png)
147+
![Typical_Hatchling_CLI_20250627_pt2](../resources/images/running-hatchling/Typical_Hatchling_CLI_20250627_pt2.png)
148+
149+
You can receive help about all available commands by writing `help` in the chat. Details about the commands are also available in the [documentation](./chat_commands.md)

‎hatchling-users/tutorials/docker-setup.md‎

Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,83 @@
1+
This article is about:
2+
- Setting up Docker for running Hatchling
3+
- Configuring GPU support for accelerated LLM inference
4+
- Platform-specific setup instructions for Windows, Linux, and macOS
5+
6+
You will learn about:
7+
- Installing Docker Desktop and WSL on Windows
8+
- Setting up NVIDIA or AMD GPU support
9+
- Configuring Docker to work with your GPU
10+
- Verifying your Docker and GPU setup
11+
# Docker Setup Instructions
12+
13+
This document provides instructions on how to set up and run the project using Docker.
14+
15+
## Prerequisites
16+
17+
- Docker Desktop: [Install Docker Desktop](https://docs.docker.com/get-docker/)
18+
- On Windows, install Windows Subsystem for Linux (WSL). Latest version is v2: [Official Microsoft Documentation](https://learn.microsoft.com/en-us/windows/wsl/install)
19+
- GPU Support:
20+
- For MacOS users with Apple Silicon chips (typically M series), you can **follow the instructions for CPU and ignore the GPU-related sections**
21+
- For other OSs (mainly Windows & Linux with dedicated GPUs) we strongly recommend enabling GPU support to increase for LLM output speed:
22+
- NVIDIA GPUs: [NVIDIA Container Toolkit Installation](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html)
23+
- AMD GPUs: See instructions below
24+
25+
## Setup with Docker Desktop
26+
27+
1. **Install Docker Desktop**:
28+
- Download and install Docker Desktop following the official instructions: https://docs.docker.com/get-docker/
29+
30+
2. **On Windows, connect Docker to WSL**:
31+
![docker_settings_wsl](../resources/images/docker-setup/docker_settings_position.png)
32+
- In Docker Desktop, follow the arrows numbered 1, 2, and 3 on the screenshot to navigate through `Settings` > `Resources` > `WSL Integration`.
33+
- Either enable integration with your default WSL distro (arrow 4.1) OR select a specific one (arrow 4.2)
34+
- Click "Apply & Restart" if you make changes (arrow 5)
35+
36+
3. **For NVIDIA GPU owners, setup GPU Support (nothing to do for AMD GPU owners at this stage)**:
37+
- Open a terminal
38+
- On Windows, launch the Linux version that was installed via WSL and that Docker is using. For example, in the previous image, that would be `Ubuntu-24.04`; so, run `wsl -d Ubuntu-24.04` to start Ubuntu.
39+
- For NVIDIA GPU support, run:
40+
41+
```bash
42+
# Add NVIDIA repository keys
43+
curl -fsS https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
44+
45+
# Add NVIDIA repository
46+
curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
47+
48+
# Update package lists
49+
sudo apt-get update
50+
51+
# Install NVIDIA container toolkit
52+
sudo apt-get install -y nvidia-container-toolkit
53+
54+
# Configure Docker runtime
55+
sudo nvidia-ctk runtime configure --runtime=docker
56+
```
57+
58+
- Close the terminal
59+
- Restart Docker
60+
- For Docker Desktop, click on the three vertical dots icon (arrow 1), then `Restart` (arrow 2)
61+
![docker_restart](../resources/images/docker-setup/docker_restart_large.png)
62+
- On Linux (Ubuntu, Debian, CentOs, Fedora), running: `systemctl restart docker` should do it. You can prepend with `sudo` if necessary.
63+
64+
4. **Pull Ollama Image**:
65+
- Open a terminal capable of running docker commands.
66+
- In Docker Desktop you can open it by pressing the `Terminal` button:
67+
![docker_terminal_position](../resources/images/docker-setup/docker_terminal_position.png)
68+
- Or any terminal of your system that can access to Docker
69+
- Write `docker pull ollama/ollama` in the terminal and press enter to run it.
70+
- It will download about 1.6GB (as of May 2025)
71+
- Once finished, click on the `Images` tab (arrow 1) of Docker Desktop, and check that `ollama/ollama` is available (arrow 2)
72+
![docker_desktop_find_ollama_image](../resources/images/docker-setup/docker_find_image.png)
73+
- If it does not show up, try closing Docker Desktop (arrow 1, then arrow 2) and launch it again.
74+
![closing_docker_desktop](../resources/images/docker-setup/docker_quit_large.png)
75+
- Alternatively, to check that the image exists, you can run the command `docker images -a`. The output should include a line similar to `ollama/ollama latest d42df3fe2285 11 days ago 4.85GB` (May 2025)
76+
77+
## Next Step
78+
79+
[Running Hatchling](./running_hatchling.md)
80+
81+
## Additional Resources
82+
83+
- [Docker Documentation](https://docs.docker.com/)

0 commit comments

Comments
 (0)