Operations guides specific to managing Sourcegraph with Docker Compose installations.
Trying to deploy Sourcegraph? Refer to our installation guide.
Configure your Sourcegraph deployment with our deployment reference. Upgrade
Upgrade your deployment to the latest Sourcegraph release. Backup and restore
Back up your Sourcegraph instance and restore from a previous backup.
Refer to our installation guide for more details on how to deploy Sourcegraph.
Migrating from another deployment type? Refer to our migrating to Docker Compose guides.
We strongly recommend that you create and run Sourcegraph from your own fork of the reference repository to track customizations to the Sourcegraph Docker Compose YAML. This will make upgrades far easier.
-
Create a fork of the sourcegraph/deploy-sourcegraph-docker repository.
WARNING: Set your fork to private if you plan to store secrets (SSL certificates, external Postgres credentials, etc.) within the repository.
-
Clone your fork using the repository's URL.
git clone $FORK_URL -
Add the reference repository as an
upstreamremote so that you can get updates.git remote add upstream https://github.com/sourcegraph/deploy-sourcegraph-docker
-
Create a
releasebranch to track all of your customizations to Sourcegraph. This branch will be used to upgrade Sourcegraph and install your Sourcegraph instance.# Specify the version you want to install export SOURCEGRAPH_VERSION="v3.29.1" # Check out the selected version for use, in a new branch called "release" git checkout $SOURCEGRAPH_VERSION -b release
-
Make and commit customizations to the Sourcegraph Docker Compose YAML to your
releasebranch.
- The version argument in the YAML file must be the same as in the standard deployment
- Users should only alter the YAML file to adjust resource limits, or duplicate container entries to add more container replicas
This requires you to have set up configuration for Docker Compose.
When you upgrade, merge the corresponding upstream release tag into your release branch.
# fetch updates
git fetch upstream
# merge the upstream release tag into your release branch
git checkout release
git merge upstream v$SOURCEGRAPH_VERSIONAddress any merge conflicts you might have.
If you are upgrading a live deployment, make sure to check the release upgrade notes for any additional actions you need to take before proceeding. Then run:
# Fully stop the Docker Compose instance of Sourcegraph currently running
docker-compose down --remove-orphans
# Start Docker Compose again, now using the latest contents of the Sourcegraph configuration
docker-compose up -dYou can see what's changed in the Sourcegraph changelog.
The Docker Compose configuration has its own internal PostgreSQL and Redis databases. To preserve this data when you kill and recreate the containers, you can use external services for persistence.
The Sourcegraph Docker Compose definition uses Docker volumes to store its data. These volumes are stored at /var/lib/docker/volumes by default on Linux.
Guides for managing cloud storage and backups are available in our cloud-specific installation guides:
- Storage and backups for Amazon Web Services
- Storage and backups for Google Cloud
- Storage and backups for Digital Ocean
The following instructions are specific to backing up and restoring the sourcegraph databases in a Docker Compose deployment. These do not apply to other deployment types.
WARNING: Only core data will be backed up.
These instructions will only back up core data including user accounts, configuration, repository-metadata, etc. Other data will be regenerated automatically:
- Repositories will be re-cloned
- Search indexes will be rebuilt from scratch
The above may take a while if you have a lot of repositories. In the meantime, searches may be slow or return incomplete results. This process rarely takes longer than 6 hours and is usually much faster.
These instuctions will back up the primary sourcegraph database the codeintel database.
sshfrom your local machine into the machine hosting thesourcegraphdeploymentcdto thedeploy-sourcegraph-docker/docker-composedirectory on the host- Verify the deployment is running:
docker-compose ps
Name Command State Ports
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
caddy caddy run --config /etc/ca ... Up 2019/tcp, 0.0.0.0:443->443/tcp, 0.0.0.0:80->80/tcp
cadvisor /usr/bin/cadvisor -logtost ... Up (health: starting) 8080/tcp
codeinsights-db docker-entrypoint.sh postgres Up 5432/tcp
codeintel-db /postgres.sh Up (healthy) 5432/tcp
github-proxy /sbin/tini -- /usr/local/b ... Up
gitserver-0 /sbin/tini -- /usr/local/b ... Up
grafana /entry.sh Up 3000/tcp, 0.0.0.0:3370->3370/tcp
jaeger /go/bin/all-in-one-linux - ... Up 0.0.0.0:14250->14250/tcp, 14268/tcp, 0.0.0.0:16686->16686/tcp, 5775/udp, 0.0.0.0:5778->5778/tcp,
0.0.0.0:6831->6831/tcp, 6831/udp, 0.0.0.0:6832->6832/tcp, 6832/udp
minio /usr/bin/docker-entrypoint ... Up (healthy) 9000/tcp
pgsql /postgres.sh Up (healthy) 5432/tcp
precise-code-intel-worker /sbin/tini -- /usr/local/b ... Up (health: starting) 3188/tcp
prometheus /bin/prom-wrapper Up 0.0.0.0:9090->9090/tcp
query-runner /sbin/tini -- /usr/local/b ... Up
redis-cache /sbin/tini -- redis-server ... Up 6379/tcp
redis-store /sbin/tini -- redis-server ... Up 6379/tcp
repo-updater /sbin/tini -- /usr/local/b ... Up
searcher-0 /sbin/tini -- /usr/local/b ... Up (healthy)
symbols-0 /sbin/tini -- /usr/local/b ... Up (healthy) 3184/tcp
syntect-server sh -c /http-server-stabili ... Up (healthy) 9238/tcp
worker /sbin/tini -- /usr/local/b ... Up 3189/tcp
zoekt-indexserver-0 /sbin/tini -- zoekt-source ... Up
zoekt-webserver-0 /sbin/tini -- /bin/sh -c z ... Up (healthy)- Stop the deployment, and restart the databases service only to ensure there are no other connections during backup and restore.
docker-compose down
docker-compose -f db-only-migrate.docker-compose.yaml up -d- Generate the database dumps
docker exec pgsql sh -c 'pg_dump -C --username sg sg' > sourcegraph_db.out
docker exec codeintel-db -c 'pg_dump -C --username sg sg' > codeintel_db.out- Ensure the
sourcgraph_db.outandcodeintel_db.outfiles are moved to a safe and secure location.
The following instructions apply only if you are restoring your databases into a new deployment of sourcegraph ie: a new virtual machine
If you are restoring a previously running environment, see the instructions for restoring a previously running deployment
- Copy the database dump files, into the
deploy-sourcegraph-docker/docker-composedirectory. - Start the database services
docker-compose -f db-only-migrate.docker-compose.yaml up -d- Copy the database files into the containers
docker cp sourcegraph_db.out pgsql:/tmp/sourcegraph_db.out
docker cp codeintel_db.out codeintel-db:/tmp/codeintel_db.out- Restore the databases
docker exec pgsql sh -c 'psql -v ERROR_ON_STOP=1 --username sg -f /tmp/sourcegraph_db.out sg'
docker exec codeintel-db sh -c 'psql -v ERROR_ON_STOP=1 --username sg -f /tmp/condeintel_db.out sg'- Start the remaining sourcegraph services
docker-compose up -d- Verify the deployment has started
docker-compose ps
Name Command State Ports
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
caddy caddy run --config /etc/ca ... Up 2019/tcp, 0.0.0.0:443->443/tcp, 0.0.0.0:80->80/tcp
cadvisor /usr/bin/cadvisor -logtost ... Up (health: starting) 8080/tcp
codeinsights-db docker-entrypoint.sh postgres Up 5432/tcp
codeintel-db /postgres.sh Up (healthy) 5432/tcp
github-proxy /sbin/tini -- /usr/local/b ... Up
gitserver-0 /sbin/tini -- /usr/local/b ... Up
grafana /entry.sh Up 3000/tcp, 0.0.0.0:3370->3370/tcp
jaeger /go/bin/all-in-one-linux - ... Up 0.0.0.0:14250->14250/tcp, 14268/tcp, 0.0.0.0:16686->16686/tcp, 5775/udp, 0.0.0.0:5778->5778/tcp,
0.0.0.0:6831->6831/tcp, 6831/udp, 0.0.0.0:6832->6832/tcp, 6832/udp
minio /usr/bin/docker-entrypoint ... Up (healthy) 9000/tcp
pgsql /postgres.sh Up (healthy) 5432/tcp
precise-code-intel-worker /sbin/tini -- /usr/local/b ... Up (health: starting) 3188/tcp
prometheus /bin/prom-wrapper Up 0.0.0.0:9090->9090/tcp
query-runner /sbin/tini -- /usr/local/b ... Up
redis-cache /sbin/tini -- redis-server ... Up 6379/tcp
redis-store /sbin/tini -- redis-server ... Up 6379/tcp
repo-updater /sbin/tini -- /usr/local/b ... Up
searcher-0 /sbin/tini -- /usr/local/b ... Up (healthy)
symbols-0 /sbin/tini -- /usr/local/b ... Up (healthy) 3184/tcp
syntect-server sh -c /http-server-stabili ... Up (healthy) 9238/tcp
worker /sbin/tini -- /usr/local/b ... Up 3189/tcp
zoekt-indexserver-0 /sbin/tini -- zoekt-source ... Up
zoekt-webserver-0 /sbin/tini -- /bin/sh -c z ... Up (healthy)> docker-compose ps- Browse to your sourcegraph deployment, login and verify your existing configuration has been restored
cdto thedeploy-sourcegraph-docker/docker-composeand stop the previous deployment and remove any existing volumes
docker-compose down
docker volume rm docker-compose_pgsql
docker volume rm docker-compose_codeintel-db- Start the databases services only
docker-compose -f db-only-migrate.docker-compose.yaml up -d- Copy the database files into the containers
docker cp sourcegraph_db.out pgsql:/tmp/sourcegraph_db.out
docker cp codeintel_db.out codeintel-db:/tmp/codeintel_db.out- Restore the databases
docker exec pgsql sh -c 'psql -v ERROR_ON_STOP=1 --username sg -f /tmp/sourcegraph_db.out sg'
docker exec codeintel-db sh -c 'psql -v ERROR_ON_STOP=1 --username sg -f /tmp/condeintel_db.out sg'- Start the remaining sourcegraph services
docker-compose up -d- Verify the deployment has started
docker-compose ps
Name Command State Ports
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
caddy caddy run --config /etc/ca ... Up 2019/tcp, 0.0.0.0:443->443/tcp, 0.0.0.0:80->80/tcp
cadvisor /usr/bin/cadvisor -logtost ... Up (health: starting) 8080/tcp
codeinsights-db docker-entrypoint.sh postgres Up 5432/tcp
codeintel-db /postgres.sh Up (healthy) 5432/tcp
github-proxy /sbin/tini -- /usr/local/b ... Up
gitserver-0 /sbin/tini -- /usr/local/b ... Up
grafana /entry.sh Up 3000/tcp, 0.0.0.0:3370->3370/tcp
jaeger /go/bin/all-in-one-linux - ... Up 0.0.0.0:14250->14250/tcp, 14268/tcp, 0.0.0.0:16686->16686/tcp, 5775/udp, 0.0.0.0:5778->5778/tcp,
0.0.0.0:6831->6831/tcp, 6831/udp, 0.0.0.0:6832->6832/tcp, 6832/udp
minio /usr/bin/docker-entrypoint ... Up (healthy) 9000/tcp
pgsql /postgres.sh Up (healthy) 5432/tcp
precise-code-intel-worker /sbin/tini -- /usr/local/b ... Up (health: starting) 3188/tcp
prometheus /bin/prom-wrapper Up 0.0.0.0:9090->9090/tcp
query-runner /sbin/tini -- /usr/local/b ... Up
redis-cache /sbin/tini -- redis-server ... Up 6379/tcp
redis-store /sbin/tini -- redis-server ... Up 6379/tcp
repo-updater /sbin/tini -- /usr/local/b ... Up
searcher-0 /sbin/tini -- /usr/local/b ... Up (healthy)
symbols-0 /sbin/tini -- /usr/local/b ... Up (healthy) 3184/tcp
syntect-server sh -c /http-server-stabili ... Up (healthy) 9238/tcp
worker /sbin/tini -- /usr/local/b ... Up 3189/tcp
zoekt-indexserver-0 /sbin/tini -- zoekt-source ... Up
zoekt-webserver-0 /sbin/tini -- /bin/sh -c z ... Up (healthy)> docker-compose ps- Browse to your sourcegraph deployment, login and verify your existing configuration has been restored