Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/CODEOWNERS
Original file line number Diff line number Diff line change
Expand Up @@ -255,7 +255,7 @@

/docs/deployments/databases/ @OctopusDeploy/team-modern-deployments

# updates to the deployments docker section are reviewed by team-modern-deployments
# updates to the deployments Docker section are reviewed by team-modern-deployments

/docs/deployments/docker/ @OctopusDeploy/team-modern-deployments

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ You should be safe doing an in-place upgrade of 3.x to the latest version of Oct
- Support for Kubernetes was introduced.
- Terraform support was introduced.
- Raised the [minimum requirements for hosting and using Octopus Server](https://octopus.com/blog/raising-minimum-requirements-for-octopus-server) (both Windows and SQL Server).
- Execution containers running on docker on workers were introduced.
- Execution containers running on Docker on workers were introduced.

## Prep work

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ Some important items to note about workers:
- Unlike deployment targets, workers are designed to run multiple tasks concurrently.
- **Octopus Server 2020.1** added the [Worker Pool Variable Type](/docs/projects/variables/worker-pool-variables) making it possible to scope worker pools to environments.
- **Octopus Server 2020.2** added the [execution container for workers](/docs/projects/steps/execution-containers-for-workers) feature, making it easier to manage software dependencies.
- We provide a [Tentacle docker image](https://hub.docker.com/repository/docker/octopusdeploy/tentacle) that can be configured to run as a worker.
- We provide a [Tentacle Docker image](https://hub.docker.com/repository/docker/octopusdeploy/tentacle) that can be configured to run as a worker.

## Provided Workers

Expand Down
2 changes: 1 addition & 1 deletion src/pages/docs/deployments/aws/ecs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ At a high level, the `Deploy Amazon ECS Service` step will:

* Select the Docker image tags for the task definition (version selection is performed when creating a release).
* Build a CloudFormation template with:
* A task definition with the details specific to the deployment for the selected environment and docker image tags.
* A task definition with the details specific to the deployment for the selected environment and Docker image tags.
* A service that references the task definition.
* Perform variable substitution on the CloudFormation template.
* Deploy a CloudFormation stack with the template.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ Octopus Cloud uses a special type of worker pool called a [Dynamic Worker Pool](
To use your own version of the Azure CLI or Azure PowerShell cmdlets when using Dynamic Worker Pools, please do the following:

- Configure your step to use a Dynamic Worker pool that supports [execution containers](/docs/projects/steps/execution-containers-for-workers).
- Configure your step to run in an execution container with a [compatible docker image](/docs/projects/steps/execution-containers-for-workers/#which-image) that contains the versions of the Azure CLI or Azure PowerShell cmdlets that you would like to use.
- Configure your step to run in an execution container with a [compatible Docker image](/docs/projects/steps/execution-containers-for-workers/#which-image) that contains the versions of the Azure CLI or Azure PowerShell cmdlets that you would like to use.

## Run an Azure PowerShell script step

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,7 @@ Octopus Cloud uses a special type of worker pool called a [Dynamic Worker Pool](
To use your own version of the AWS CLI or AWS PowerShell cmdlets when using Dynamic Worker Pools, please do the following:

- Configure your step to use a Dynamic Worker pool that supports [execution containers](/docs/projects/steps/execution-containers-for-workers).
- Configure your step to run in an execution container with a [compatible docker image](/docs/projects/steps/execution-containers-for-workers/#which-image) that contains the versions of the AWS CLI or AWS PowerShell cmdlets that you would like to use.
- Configure your step to run in an execution container with a [compatible Docker image](/docs/projects/steps/execution-containers-for-workers/#which-image) that contains the versions of the AWS CLI or AWS PowerShell cmdlets that you would like to use.

## Older versions

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ Octopus Cloud uses a special type of worker pool called a [Dynamic Worker Pool](
To use your own version of the Azure CLI or Azure PowerShell cmdlets when using Dynamic Worker Pools, please do the following:

- Configure your step to use a Dynamic Worker pool that supports [execution containers](/docs/projects/steps/execution-containers-for-workers).
- Configure your step to run in an execution container with a [compatible docker image](/docs/projects/steps/execution-containers-for-workers/#which-image) that contains the versions of the Azure CLI or Azure PowerShell cmdlets that you would like to use.
- Configure your step to run in an execution container with a [compatible Docker image](/docs/projects/steps/execution-containers-for-workers/#which-image) that contains the versions of the Azure CLI or Azure PowerShell cmdlets that you would like to use.

These scripts are executed on the Octopus Server and will be pre-authenticated using the selected Azure Account. For information about adding a step to the deployment process, see the [add step](/docs/projects/steps) section.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ In a newly created project, click **Add Step ➜ Create a Docker network**. This
![](/docs/img/deployments/docker/images/add-docker-network-step.png)

:::div{.hint}
**docker network support**
**Docker network support**
Keep in mind that as the Docker Network Octopus step simply wraps the `docker network` command, you will need to ensure your installed version of docker-engine supports this command. This was provided as of the [1.9.0 Docker Engine release](https://github.com/docker/docker/blob/master/CHANGELOG/#networking-10).

For detailed information about Docker networking and additional arguments you can provide, we suggest reading the [Understand Docker container networks](https://docs.docker.com/network/) and the [network create](https://docs.docker.com/engine/reference/commandline/network_create/) Docker documentation.
Expand Down Expand Up @@ -98,7 +98,7 @@ The command itself is arbitrary. What is important is that we start a process th

### Step 3: Creating container 2 {#step3-create-container2}

Now we will create a second container, exactly the same as the first using busybox, but this container will connect to the first container to demonstrate how containers can communicate in a docker network.
Now we will create a second container, exactly the same as the first using busybox, but this container will connect to the first container to demonstrate how containers can communicate in a Docker network.

1. Create a new Run Docker Container step (very much like the first one, but notice some subtle differences):
* Set the *Name* to **Second Server**.
Expand Down Expand Up @@ -185,7 +185,7 @@ Looking at the results of a deployment you will see some logging indicating the
**Having problems deploying to Docker?**
Usually this will come down to problems with the Octopus Account you configured accessing the Docker daemon with messages like this:

`Cannot connect to the Docker daemon. Is the docker daemon running on this host?`
`Cannot connect to the Docker daemon. Is the Docker daemon running on this host?`

For steps to grant the Octopus Account the ability to manage the Docker daemon, refer to the [Docker post-installation steps for Linux](https://docs.docker.com/engine/install/linux-postinstall/#manage-docker-as-a-non-root-user).
:::
Expand All @@ -202,7 +202,7 @@ The Docker equivalent of the "Package Acquisition" phase involves retrieving the
![](/docs/img/deployments/docker/images/acquire-package-steps.png)
:::

When creating a network or container, the name and Id as simply echoed to the standard logs. The verbose logs will show the docker version on the target, the full docker command being called and if successful, the results of the inspect command that are passed to an output variable.
When creating a network or container, the name and Id as simply echoed to the standard logs. The verbose logs will show the Docker version on the target, the full Docker command being called and if successful, the results of the inspect command that are passed to an output variable.

:::figure
![](/docs/img/deployments/docker/images/first-and-second-server-task-output.png)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -154,7 +154,7 @@ Octopus does not recommend installing additional software on Dynamic Workers.

By default, every dynamic worker is destroyed after it has been idle for 60 minutes or allocated for over 72 hours. Additionally, Octopus cannot guarantee that the dynamic worker leased to run one step will be the same worker leased to other executing steps in a deployment or runbook run.

For deployments and runbook runs that require additional software dependencies on a dynamic worker, our recommendation is to leverage [execution containers for workers](/docs/projects/steps/execution-containers-for-workers). Octopus provides execution containers with a baseline of tools (`octopusdeploy/worker-tools`) pre-installed. These tools won't include every possible software combination you might need. If you require a specific set of software and tooling we recommend [building your own custom docker images for use with execution containers](/docs/projects/steps/execution-containers-for-workers/#custom-docker-images).
For deployments and runbook runs that require additional software dependencies on a dynamic worker, our recommendation is to leverage [execution containers for workers](/docs/projects/steps/execution-containers-for-workers). Octopus provides execution containers with a baseline of tools (`octopusdeploy/worker-tools`) pre-installed. These tools won't include every possible software combination you might need. If you require a specific set of software and tooling we recommend [building your own custom Docker images for use with execution containers](/docs/projects/steps/execution-containers-for-workers/#custom-docker-images).

:::div{.hint}
**Octopus worker-tools are cached on Dynamic Workers**
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -92,5 +92,5 @@ Ubuntu 22.04 requires a later version of GCloud CLI. We have selected the earlie
This change does not impact the Windows dynamic workers.

### How does this affect Execution Containers?
Although Ubuntu 18.04 docker images, along with [Worker Tools](/docs/infrastructure/workers/worker-tools-versioning-and-caching), can still operate on Ubuntu 22.04 dynamic workers, we will no longer provide support for the ubuntu.18.04 Worker Tools. Instead, we have introduced a new [ubuntu.22.04](https://hub.docker.com/r/octopusdeploy/worker-tools/tags?page=1&name=22.04) image, which is recommended moving forward.
Although Ubuntu 18.04 Docker images, along with [Worker Tools](/docs/infrastructure/workers/worker-tools-versioning-and-caching), can still operate on Ubuntu 22.04 dynamic workers, we will no longer provide support for the ubuntu.18.04 Worker Tools. Instead, we have introduced a new [ubuntu.22.04](https://hub.docker.com/r/octopusdeploy/worker-tools/tags?page=1&name=22.04) image, which is recommended moving forward.

Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ Of note:

| Value | Purpose |
| ----------------------------- | ------------------------------------------------------------------------- |
| scriptPods.worker.image | Specifies the docker container image to be used when running an operation |
| scriptPods.worker.image | Specifies the Docker container image to be used when running an operation |
| scriptPods.resources.requests | Specifies the average cpu/memory usage required to execute an operation |

If you are experiencing difficulties with your Kubernetes Cluster's autoscaling, modifying `scriptPods.resources.requests.*`
Expand All @@ -72,5 +72,5 @@ Which means certain operations which are typically valid, may not be possible.
Specifically:

- Creating an [inline execution container](/docs/projects/steps/execution-containers-for-workers#inline-execution-containers)
- Fetching docker images (when used as secondary packages)
- Fetching Docker images (when used as secondary packages)
- Arbitrary scripts which use docker
Original file line number Diff line number Diff line change
Expand Up @@ -3,16 +3,16 @@ layout: src/layouts/Default.astro
pubDate: 2023-01-01
modDate: 2023-01-01
title: Worker Tools, Versioning and Caching
description: How Octopus creates, versions, caches, and releases the worker-tools docker images for use with the execution containers for workers feature.
description: How Octopus creates, versions, caches, and releases the worker-tools Docker images for use with the execution containers for workers feature.
navOrder: 50
---

Worker Tools are a set of docker images used as [execution containers for workers](https://octopus.com/docs/projects/steps/execution-containers-for-workers) to run deployment processes. Worker Tools include a wide range of software tools to support most deployment scenarios out of the box. This page focuses on how we create these Worker Tool images, version, cache on workers, and release them.
Worker Tools are a set of Docker images used as [execution containers for workers](https://octopus.com/docs/projects/steps/execution-containers-for-workers) to run deployment processes. Worker Tools include a wide range of software tools to support most deployment scenarios out of the box. This page focuses on how we create these Worker Tool images, version, cache on workers, and release them.

## Versioning Worker Tools
Worker Tool images follow a semantic versioning (SemVer) approach of `Major.Minor.Patch-Distro` for their tag format. When we release a new version of Worker Tools to the [Worker Tools Docker Hub repository](https://hub.docker.com/r/octopusdeploy/worker-tools/tags), we also add the following image tags, distribution (`ubuntu.22.04` or `windows.ltsc2022`), `Major-Distro` (e.g. `3-Distro`) and `Major.Minor-Distro` (`3.3-Distro`). We recommend using the fully qualified SemVer as patch updates of Worker Tools could result in an updated tool dependency introducing a breaking change.

The Worker Tools Dockerfiles use a combination of tools pinned to specific versions, such as CLI tools and Frameworks, while other tools pull their latest available release. For Ubuntu, these are pulled with apt-get, and for Windows, chocolatey. You can find the full details of these tools in the docker files for [Windows](https://github.com/OctopusDeploy/WorkerTools/blob/master/windows.ltsc2022/Dockerfile) and [Ubuntu](https://github.com/OctopusDeploy/WorkerTools/blob/master/ubuntu.22.04/Dockerfile) Worker Tools.
The Worker Tools Dockerfiles use a combination of tools pinned to specific versions, such as CLI tools and Frameworks, while other tools pull their latest available release. For Ubuntu, these are pulled with apt-get, and for Windows, chocolatey. You can find the full details of these tools in the Docker files for [Windows](https://github.com/OctopusDeploy/WorkerTools/blob/master/windows.ltsc2022/Dockerfile) and [Ubuntu](https://github.com/OctopusDeploy/WorkerTools/blob/master/ubuntu.22.04/Dockerfile) Worker Tools.

The tools pulling their latest releases for Ubuntu include `wget`, `python3-pip`, `groff`, `unzip`, `apt-utils`, `curl`, `software-properties-common`, `jq`, `yq`, `openssh-client`, `rsync`, `git`, `augeas-tools`, `maven`, `gradle`, `Node 14`, `istioctl`, `linkerd `, `umoci`.

Expand All @@ -39,7 +39,7 @@ In short, we recommend using the full `octopusdeploy/worker-tools:Major.Minor.Pa

Worker Tools are cached on dynamic workers to help improve the performance of deployments. Windows workers cache the latest two sets of Worker Tools while Ubuntu workers cache the latest five.

To understand this cache, it's important to understand a worker's life cycle. Workers are acquired from a dynamic worker pool and leased to a single cloud instance. They are allocated in a round robin fashion to individual deployment steps, storing packages, docker images, and other data on disk. Workers are destroyed after either the worker has been idle for 60 minutes or has existed for 72 hours (3 days).
To understand this cache, it's important to understand a worker's life cycle. Workers are acquired from a dynamic worker pool and leased to a single cloud instance. They are allocated in a round robin fashion to individual deployment steps, storing packages, Docker images, and other data on disk. Workers are destroyed after either the worker has been idle for 60 minutes or has existed for 72 hours (3 days).

When a new worker is acquired, if a new set of Worker Tools has been released, the worker will no longer have the oldest version of Worker Tools, and any other images pulled on the old worker. This is important for the performance of deployments as pull times for uncached Worker Tools are ~1.5 minutes for Ubuntu and ~20 minutes for Windows. We recommend updating to the latest set of Worker Tools available to avoid these pull times. By Caching multiple versions of Worker Tools when using the latest version, any new release of Worker Tools will not result in degraded deployment performance.

Expand All @@ -61,4 +61,4 @@ Using non-cached versions of these worker-tools can result in long downloads.
## Learn more

- [Worker blog posts](https://octopus.com/blog/tag/workers/1)
- [Custom docker images](/docs/projects/steps/execution-containers-for-workers/#custom-docker-images)
- [Custom Docker images](/docs/projects/steps/execution-containers-for-workers/#custom-docker-images)
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,7 @@ docker build -t octopusbob/nginx:1.0.0 -t octopusbob/nginx:latest .
```

### Running the NGINX Container
Then you can run the docker image in a container by running the command.
Then you can run the Docker image in a container by running the command.

```
docker run --name octopus-reverse-proxy -p 443:443 -e OCTOPUS_SERVER=servername:8080 octopusbob/nginx:latest
Expand Down
2 changes: 1 addition & 1 deletion src/pages/docs/kubernetes/resources/kubectl.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,6 @@ navOrder: 100

The [kubectl command-line tool](https://kubernetes.io/docs/reference/kubectl/overview/) is required by Octopus Deploy's Kubernetes features.

`kubectl` is not bundled with Octopus, and must be pre-installed on the [worker](/docs/infrastructure/workers/) or Octopus Server which will execute steps and health checks against a [Kubernetes target](/docs/kubernetes/targets/kubernetes-api). Alternatively, [execution containers](/docs/projects/steps/execution-containers-for-workers) may be used and we recommend using [`octopuslabs/k8s-workertools`](https://hub.docker.com/r/octopuslabs/k8s-workertools) docker image, we also recommend setting the tag to a version that is compatible with the version of your cluster, [read the official version skew policy](https://kubernetes.io/releases/version-skew-policy/#kubectl) for more information.
`kubectl` is not bundled with Octopus, and must be pre-installed on the [worker](/docs/infrastructure/workers/) or Octopus Server which will execute steps and health checks against a [Kubernetes target](/docs/kubernetes/targets/kubernetes-api). Alternatively, [execution containers](/docs/projects/steps/execution-containers-for-workers) may be used and we recommend using [`octopuslabs/k8s-workertools`](https://hub.docker.com/r/octopuslabs/k8s-workertools) Docker image, we also recommend setting the tag to a version that is compatible with the version of your cluster, [read the official version skew policy](https://kubernetes.io/releases/version-skew-policy/#kubectl) for more information.

By default, Octopus assumes `kubectl` is available in the PATH environment variable. A specific location for `kubectl` can be supplied by setting a `Octopus.Action.Kubernetes.CustomKubectlExecutable` variable in the Octopus project (an example value is `c:\tools\kubectl\version\kubectl.exe`).
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,7 @@ The default image used for workloads when running as a worker.

:::div{.hint}

For ARM workloads use **octopuslabs/k8s-workertools** or your own [custom docker image](/docs/projects/steps/execution-containers-for-workers#custom-docker-images).
For ARM workloads use **octopuslabs/k8s-workertools** or your own [custom Docker image](/docs/projects/steps/execution-containers-for-workers#custom-docker-images).

### octopuslabs/k8s-workertools

Expand Down
Loading