Skip to content

Simplify llama.cpp installation in Dockerfile #11

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
j4ys0n merged 1 commit into from
Nov 4, 2025
Merged

Simplify llama.cpp installation in Dockerfile #11

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 2 additions & 4 deletions docker/Dockerfile.gpu
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,11 +43,9 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
# Download and install pre-compiled llama.cpp binary
RUN curl -L -o /tmp/llama.zip https://github.com/ggml-org/llama.cpp/releases/download/${LLAMA_CPP_VERSION}/llama-${LLAMA_CPP_VERSION}-bin-ubuntu-x64.zip && \
unzip -q /tmp/llama.zip -d /tmp/llama-extract && \
EXTRACTED_DIR=$(find /tmp/llama-extract -maxdepth 1 -type d -name "llama-*" | head -1) && \
if [ -z "$EXTRACTED_DIR" ]; then echo "Error: No llama directory found after extraction"; ls -la /tmp/llama-extract; exit 1; fi && \
mv "$EXTRACTED_DIR" /opt/llama.cpp && \
mv /tmp/llama-extract/build /opt/llama.cpp && \
Copy link

Copilot AI Nov 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The hardcoded path /tmp/llama-extract/build assumes a specific directory structure in the extracted archive. If the archive structure changes or doesn't contain a build directory at this exact path, the installation will fail silently or with a confusing error. The code in src/msquant/core/quantizer/engine.py:350 also expects /opt/llama.cpp/convert-hf-to-gguf.py to exist, but with this new structure it's unclear if the Python scripts will be in the correct location. Consider verifying that both the binaries and Python conversion scripts end up in the expected locations.

Copilot uses AI. Check for mistakes.
rm -rf /tmp/llama.zip /tmp/llama-extract && \
chmod +x /opt/llama.cpp/llama-* && \
chmod +x /opt/llama.cpp/bin/llama-* && \
pip install gguf

# Add llama.cpp to PATH
Expand Down