Optimize PostgreSQL startup with pre-initialized Docker image#4309
Open
kyleconroy wants to merge 2 commits intomainfrom
Open
Optimize PostgreSQL startup with pre-initialized Docker image#4309kyleconroy wants to merge 2 commits intomainfrom
kyleconroy wants to merge 2 commits intomainfrom
Conversation
Pre-initialize the database at Docker build time so container startup only needs to launch the postgres process — no initdb, no entrypoint scripts. Combined with aggressive durability-off settings (fsync=off, synchronous_commit=off, wal_level=minimal), this cuts container startup from ~1-2s to near-instant. Key changes: - Dockerfile.postgres: pre-initialized PG 18 image with test-tuned config - docker-compose.yml: build from Dockerfile.postgres instead of pulling - internal/sqltest/docker/postgres.go: auto-build the fast image on first test run, pipe Dockerfile via stdin to avoid sending build context https://claude.ai/code/session_018Godbm3DFdWJf5jc7NU8Uq
Both the sqlc-test-setup CLI tool and the native test helper now detect postgresql.conf at runtime and append the same speed-optimized settings used in Dockerfile.postgres: fsync=off, synchronous_commit=off, full_page_writes=off, wal_level=minimal, etc. Settings are applied idempotently (a marker comment prevents duplicate entries) and trigger a full restart since wal_level and max_connections require it. This brings CI and non-Docker test environments to parity with the Docker fast-startup image. https://claude.ai/code/session_018Godbm3DFdWJf5jc7NU8Uq
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR optimizes PostgreSQL test database startup by pre-initializing the database at Docker image build time and applying performance-tuned settings. This eliminates the ~1-2 second
initdbcost on every container start and ensures consistent, fast test execution.Key Changes
New Dockerfile.postgres: Creates a fast-startup PostgreSQL image with:
Docker integration improvements:
internal/sqltest/docker/postgres.go: Added logic to build and use the customsqlc-postgresimage instead of pullingpostgres:16buildPostgresImage()to build from Dockerfile.postgres on first usepostgresImageExists()to check if image is already builtfindRepoRoot()to locate the Dockerfile relative to the repositoryNative PostgreSQL setup enhancements:
internal/sqltest/native/postgres.go: AddedpgFastSettingsconfiguration andapplyFastSettings()function to apply performance tuning to existing PostgreSQL installationsrestartPostgres()with fallback logic for different init systems (systemctl, service, pg_ctlcluster)Test setup tool updates:
cmd/sqlc-test-setup/main.go: AddedapplyFastSettings()function to apply performance tuning during manual PostgreSQL setupdetectPgConfigPath()to query the running server for postgresql.conf locationDocker Compose update:
docker-compose.yml: Updated to build the custom Dockerfile.postgres instead of pulling the standard postgres:16 imageImplementation Details
gosufor proper privilege handling and pipes the Dockerfile todocker build -to avoid sending the entire repository as build contexthttps://claude.ai/code/session_018Godbm3DFdWJf5jc7NU8Uq