Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
0782a2c
[DOCS-1927] Update Registry role permission table for protected aliases
mdlinville Dec 29, 2025
d65bd5f
Automations doc revamp
mdlinville Mar 6, 2026
da1f3ec
Merge branch 'main' into DOCS-1917
mdlinville Mar 9, 2026
f25da49
Merge branch 'main' into DOCS-1917
mdlinville Mar 10, 2026
7a12b22
Apply suggestions from code review
mdlinville Mar 10, 2026
e21b685
Merge remote-tracking branch 'origin/main' into DOCS-1917
mdlinville Mar 10, 2026
58d001f
Merge remote-tracking branch 'origin/DOCS-1927' into DOCS-1917
mdlinville Mar 10, 2026
10fc830
Merge remote-tracking branch 'origin/DOCS-1917' into DOCS-1917
mdlinville Mar 10, 2026
ec3d4ad
Drop notebook in favor of wandb/examples PR
mdlinville Mar 10, 2026
039d1f4
Merge branch 'main' into DOCS-1917
mdlinville Mar 11, 2026
958ea4a
Merge branch 'main' into DOCS-1917
mdlinville Mar 11, 2026
7f19636
Merge branch 'main' into DOCS-1917
mdlinville Mar 16, 2026
2f5dd71
Merge remote-tracking branch 'origin/main' into DOCS-1917
mdlinville Mar 16, 2026
8b1418a
Merge remote-tracking branch 'origin/DOCS-1917' into DOCS-1917
mdlinville Mar 16, 2026
c0ae8cd
Split tutorial into two to avoid <View> and nested <Tabs> components
mdlinville Mar 16, 2026
958c2e8
Split Mermaid diagram into two snippets, include in tutorial landing …
mdlinville Mar 16, 2026
a51b94d
Merge branch 'main' into DOCS-1917
mdlinville Mar 16, 2026
551d993
Update TOC structure, adjust mental model and Mermaid diagram includes
mdlinville Mar 16, 2026
869fa7e
Merge remote-tracking branch 'origin/DOCS-1917' into DOCS-1917
mdlinville Mar 16, 2026
c23779f
More adjustments
mdlinville Mar 16, 2026
708480e
Merge remote-tracking branch 'origin/main' into DOCS-1917
mdlinville Mar 19, 2026
aa9eff4
Noah's feedback
mdlinville Mar 19, 2026
0f26329
Automations are no longer limited to Enterprise Cloud-only
mdlinville Mar 19, 2026
d95fa61
Merge branch 'main' into DOCS-1917
mdlinville Mar 20, 2026
ad3d46b
Merge branch 'main' into DOCS-1917
mdlinville Mar 26, 2026
cc6fb7d
Merge remote-tracking branch 'origin/main' into DOCS-1917
mdlinville Mar 26, 2026
c5868aa
Merge remote-tracking branch 'origin/DOCS-1917' into DOCS-1917
mdlinville Mar 26, 2026
25a0ab7
Trying to add more space around mermaid
mdlinville Mar 26, 2026
c9d377d
Merge branch 'main' into DOCS-1917
mdlinville Mar 27, 2026
82fad9a
Merge remote-tracking branch 'origin/DOCS-1917' into DOCS-1917
mdlinville Mar 27, 2026
67bf99d
Add vertical space around Mermaid diagrams
mdlinville Mar 27, 2026
49c75df
Merge branch 'main' into DOCS-1917
mdlinville Mar 27, 2026
f223f49
Merge branch 'main' into DOCS-1917
mdlinville Mar 27, 2026
b8bc364
Merge branch 'main' into DOCS-1917
mdlinville Mar 30, 2026
af562c1
Merge branch 'main' into DOCS-1917
mdlinville Mar 31, 2026
709bee3
Merge branch 'main' into DOCS-1917
mdlinville Apr 1, 2026
baba37e
Merge remote-tracking branch 'origin/main' into DOCS-1917
mdlinville Apr 7, 2026
be66f85
Merge origin/DOCS-1917 into DOCS-1917
mdlinville Apr 7, 2026
ae21529
Merge branch 'main' into DOCS-1917
mdlinville Apr 8, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion docs.json
Original file line number Diff line number Diff line change
Expand Up @@ -391,8 +391,17 @@
"models/automations/create-automations/webhook"
]
},
{
"group": "Tutorials",
"pages": [
"models/automations/tutorial",
"models/automations/project-automation-tutorial",
"models/automations/registry-automation-tutorial"
]
},
"models/automations/view-automation-history",
"models/automations/automation-events"
"models/automations/automation-events",
"models/automations/api"
]
},
{
Expand Down
1 change: 0 additions & 1 deletion ja/models/automations/create-automations/webhook.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -291,7 +291,6 @@ W&B が `POST` リクエストに使用する形式の詳細については、**
* `PAYLOAD`
* `API_ENDPOINT`


```bash webhook_test.sh
#!/bin/bash

Expand Down
1 change: 0 additions & 1 deletion ko/models/automations/automation-events.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,6 @@ z-score 자동화를 생성하려면 다음을 구성합니다:
- z-score가 -2.0이면 메트릭이 평균보다 2 표준 편차 낮음을 의미합니다.
- ±2를 벗어나는 값은 종종 통계적으로 유의미한 이상치(outlier)로 간주됩니다.


#### Run 필터
이 섹션에서는 자동화가 평가할 run을 선택하는 방법에 대해 설명합니다.

Expand Down
1 change: 0 additions & 1 deletion ko/models/automations/create-automations/webhook.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -291,7 +291,6 @@ webhook 문제를 해결하려면 아래 코드를 쉘 스크립트에 복사하
* `PAYLOAD`
* `API_ENDPOINT`


```bash webhook_test.sh
#!/bin/bash

Expand Down
71 changes: 43 additions & 28 deletions models/automations.mdx
Original file line number Diff line number Diff line change
@@ -1,57 +1,72 @@
---
title: Automations overview
sidebarTitle: Overview
description: Use W&B Automations for triggering workflows based on events in W&B
---
import AutomationsMentalModel from "/snippets/en/_includes/automations/mental-model.mdx";
import AutomationsActionsList from "/snippets/en/_includes/automations/actions-list.mdx";
import AutomationsBestPractices from "/snippets/en/_includes/automations/best-practices.mdx";
import AutomationsWhereToFind from "/snippets/en/_includes/automations/where-to-find-automations.mdx";

This page describes _automations_ in W&B. [Create an automation](/models/automations/create-automations/) to trigger workflow steps, such as automated model testing and deployment, based on an event in W&B.
Automations exist for both **projects** and **registries**. Where you create an automation, which events you can use, and how scope works all differ. For event types by scope, see [Automation events and scopes](/models/automations/automation-events).

For example, an automation can notify a Slack channel when a new version is created, trigger an automated testing webhook when the `production` alias is added to an artifact, or start a validation job only when a run's `loss` is within acceptable bounds.
<AutomationsMentalModel/>

<Info>
Looking for companion tutorials for automations?
- [Learn to automatically trigger a Github Action for model evaluation and deployment](https://wandb.ai/wandb/wandb-model-cicd/reports/Model-CI-CD-with-W-B--Vmlldzo0OTcwNDQw).
- [Watch a video demonstrating automatically deploying a model to a Sagemaker endpoint](https://www.youtube.com/watch?v=s5CMj_w3DaQ).
- [Watch a video series introducing automations](https://youtube.com/playlist?list=PLD80i8An1OEGECFPgY-HPCNjXgGu-qGO6&feature=shared).
</Info>
```mermaid
flowchart LR
Event[Event] --> Condition[Condition]
Condition --> Action[Action]
```

**Example:** Run fails (event) and optional run name filter (condition) then Slack notification (action). Or: alias `production` added (event) then webhook (action).

## Automation events
## Where to create automations

Automations can be triggered by events related to runs or artifacts in a collection, project, or registry. An automation can start:
<AutomationsWhereToFind/>

- When a run metric meets a defined absolute or relative threshold. For example, trigger when a run's `loss` is less than 0.01 or when a run's `accuracy` improves by 5%.
- When a run's z-score (standard score) deviates from the mean by a given amount. A z-score of 0 indicates that the result aligns with the mean. A z-score of +2 means that the result is two standard deviations higher than the mean, and a z-score of -2 means that the result is two standard deviations lower than the mean.
- When an event occurs in a project or registry. For example, trigger when a new version of a model artifact is created or when the `production` alias is added to a model artifact.
## Use cases

For more details, see [Automation events and scopes](/models/automations/automation-events/).
- **Run monitoring and alerting**: Notify the team when a run fails or when a metric crosses a threshold (for example, loss goes to NaN or accuracy drops).
- **Registry CI/CD**: When a new model version is linked or an alias (such as `staging` or `production`) is added, trigger a webhook to run tests or deploy.
- **Project artifact workflows**: When a new artifact version is created or an alias is added in a project, run a downstream job or post to Slack.

For full event and scope details, see [Automation events and scopes](/models/automations/automation-events).

## Automation actions

When an event triggers an automation, it can perform one of these actions:

### Slack notification
Send a message to a Slack channel with details about the triggering event. The message summarizes the event, with a link to view more details in W&B.

### Webhook
Call a webhook URL with a JSON payload containing information about the triggering event. This enables integration with external systems like CI/CD pipelines, model deployment services, or custom workflows. The body of the webhook request may be any JSON-serializable payload.
<AutomationsActionsList/>

For implementation details, see:
- [Create a Slack automation](/models/automations/create-automations/slack/)
- [Create a webhook automation](/models/automations/create-automations/webhook/)
For implementation details, see [Create a Slack automation](/models/automations/create-automations/slack) and [Create a webhook automation](/models/automations/create-automations/webhook).

## How automations work

To [create an automation](/models/automations/create-automations/), you:
To [create an automation](/models/automations/create-automations), you:

1. If required, configure [secrets](/platform/secrets/) for sensitive strings the automation requires, such as access tokens, passwords, or sensitive configuration details. Secrets are defined in your **Team Settings**. Secrets are most commonly used in webhook automations to securely pass credentials or tokens to the webhook's external service without exposing it in plain text or hard-coding it in the webhook's payload.
1. If required, configure [secrets](/platform/secrets) for sensitive strings the automation requires, such as access tokens, passwords, or sensitive configuration details. Secrets are defined in your **Team Settings**. Secrets are most commonly used in webhook automations to securely pass credentials or tokens to the webhook's external service without exposing it in plain text or hard-coding it in the webhook's payload.
1. Configure team-level webhook or Slack integrations to authorize W&B to post to Slack or run the webhook on your behalf. A single automation action (webhook or Slack notification) can be used by multiple automations. These actions are defined in your **Team Settings**.
1. In the project or registry, create the automation:
1. Define the [event](#automation-events) to watch for, such as when a new artifact version is added.
1. Define the [event](/models/automations/automation-events) to watch for, such as when a new artifact version is added.
1. Define the action to take when the event occurs (posting to a Slack channel or running a webhook). For a webhook, specify a secret to use for the access token and/or a secret to send with the payload, if required.

## Recommendations

<AutomationsBestPractices/>

## Limitations
[Run metric automations](/models/automations/automation-events/#run-metrics-events) and [run metrics z-score change automations](/models/automations/automation-events/#run-metrics-z-score-change-automations) are currently supported only in [W&B Multi-tenant Cloud](/platform/hosting/#wb-multi-tenant-cloud).

## Next steps
- [Create an automation](/models/automations/create-automations/).
- Learn about [Automation events and scopes](/models/automations/automation-events/).
- [Create a secret](/platform/secrets/).
- [Automations tutorial](/models/automations/tutorial): Guides you to create a project automation to alert on run failures and a Registry automation to run a webhoook when an alias is added. The tutorial provides both UI and API instructions.
- [Create an automation](/models/automations/create-automations).
- [Manage automations with the API](/models/automations/api).
- [Automation events and scopes](/models/automations/automation-events).
- [Create a secret](/platform/secrets).

<Info>
Looking for companion tutorials for automations?
- [Learn to automatically trigger a Github Action for model evaluation and deployment](https://wandb.ai/wandb/wandb-model-cicd/reports/Model-CI-CD-with-W-B--Vmlldzo0OTcwNDQw).
- [Watch a video demonstrating automatically deploying a model to a Sagemaker endpoint](https://www.youtube.com/watch?v=s5CMj_w3DaQ).
- [Watch a video series introducing automations](https://youtube.com/playlist?list=PLD80i8An1OEGECFPgY-HPCNjXgGu-qGO6&feature=shared).
</Info>
132 changes: 132 additions & 0 deletions models/automations/api.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,132 @@
---
title: Manage automations with the API
description: Create, list, get, update, and delete W&B automations using the Python API
---

You can manage W&B [automations](/models/automations) programmatically with the public Python API. This page summarizes the main operations. For full method signatures and parameters, see the [Automations API reference](/models/ref/python/public-api/automations).

## List automations

Use `api.automations()` to iterate over automations. You can filter by entity and optionally by name.

```python
import wandb

api = wandb.Api()

# All automations for an entity (team or user)
for automation in api.automations(entity="my-team"):
print(automation.name, automation.scope)

# Filter by name (returns matching automations)
for automation in api.automations(entity="my-team", name="my-automation"):
print(automation.name)
```

Automations can be scoped to a [project](/models/automations/automation-events/#project) or a [registry](/models/automations/automation-events/#registry). The returned `Automation` objects include scope information.

## Get one automation

Use `api.automation()` when you know the automation name. The call returns the only automation matching the name (and entity, if provided).

```python
import wandb

api = wandb.Api()

automation = api.automation(name="my-automation")
print(automation.description, automation.enabled)

# From a specific entity
automation = api.automation(name="my-automation", entity="my-team")
```

If zero or multiple automations match, the API raises `ValueError`.

## Create an automation

Create an automation by defining an **event** (what triggers it) and an **action** (what runs when it triggers), then calling `api.create_automation()`. Events and actions are built from classes in `wandb.automations`; the scope (project or registry/collection) determines which event types you can use.

**Example: project automation (run state to Slack)**

This example creates a project-scoped automation that sends a Slack notification when a run in the project finishes in a **Failed** state.

```python
import wandb
from wandb.automations import OnRunState, RunEvent, SendNotification

api = wandb.Api()

project = api.project("my-project", entity="my-team")
slack_integration = next(api.slack_integrations(entity="my-team"))

event = OnRunState(
scope=project,
filter=RunEvent.state.in_(["failed"]),
)
action = SendNotification.from_integration(slack_integration)

automation = api.create_automation(
event >> action,
name="run-failure-alert",
description="Notify the team when a run fails.",
)
```

**Example: run metric threshold to Slack (project)**

The [Api reference](/models/ref/python/public-api/api/#api-create_automation) shows creating an automation that triggers when a run metric exceeds a threshold. You can combine other event types (for example, [OnAddArtifactAlias](/models/ref/python/automations/onaddartifactalias) or [OnCreateArtifact](/models/ref/python/automations/oncreateartifact)) with [SendNotification](/models/ref/python/automations/sendnotification) or [SendWebhook](/models/ref/python/automations/sendwebhook) in the same way: build the event and action, then pass `event >> action` to `api.create_automation()`.

## Update an automation

Fetch the automation, change its attributes, then call `api.update_automation()`. You can pass keyword arguments to update in place without mutating the object.

```python
import wandb

api = wandb.Api()

automation = api.automation(name="my-automation")
automation.enabled = False
automation.description = "Temporarily disabled for maintenance."
api.update_automation(automation)
```

Or pass updates as keyword arguments:

```python
updated = api.update_automation(
automation,
enabled=False,
description="Temporarily disabled for maintenance.",
)
```

## Delete an automation

Pass the automation object or its ID to `api.delete_automation()`.

```python
import wandb

api = wandb.Api()

automation = api.automation(name="my-automation")
api.delete_automation(automation)
```

Or delete by ID:

```python
api.delete_automation(automation.id)
```

{/* TODO: Fix when https://github.com/wandb/examples/pull/618 merges */}
For a runnable walkthrough that includes these operations and the two tutorial automations (project run failure alert and registry alias to webhook), see the [Automations tutorial notebook](https://raw.githubusercontent.com/mdlinville/examples/a43b31213c8e0642a30a202f82e174772eb687f6/colabs/automations/automations-tutorial.ipynb) on GitHub.

## Next steps

- [Automations overview](/models/automations)
- [Automation events and scopes](/models/automations/automation-events) (project vs registry events)
- [Create a Slack automation](/models/automations/create-automations/slack) or [Create a webhook automation](/models/automations/create-automations/webhook) (UI)
- [Automations API reference](/models/ref/python/public-api/automations)
1 change: 0 additions & 1 deletion models/automations/automation-events.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -130,7 +130,6 @@ For example, trigger an automation when `accuracy` has a z-score above 2, which
- A z-score of -2.0 means the metric is 2 standard deviations below average.
- Values beyond ±2 are often considered statistically significant outliers.


#### Run filters
This section describes how the automation selects runs to evaluate.

Expand Down
22 changes: 12 additions & 10 deletions models/automations/create-automations.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,24 +3,24 @@ title: Overview
description: Create and manage W&B automations to streamline your ML workflows
---

This page gives an overview of creating and managing W&B [automations](/models/automations/). For more detailed instructions, refer to [Create a Slack automation](/models/automations/create-automations/slack/) or [Create a webhook automation](/models/automations/create-automations/webhook/).
This page gives an overview of creating and managing W&B [automations](/models/automations). For more detailed instructions, refer to [Create a Slack automation](/models/automations/create-automations/slack) or [Create a webhook automation](/models/automations/create-automations/webhook).

<Note>
Looking for companion tutorials for automations?
- [Learn to automatically triggers a Github Action for model evaluation and deployment](https://wandb.ai/wandb/wandb-model-cicd/reports/Model-CI-CD-with-W-B--Vmlldzo0OTcwNDQw).
- [Learn to automatically trigger a Github Action for model evaluation and deployment](https://wandb.ai/wandb/wandb-model-cicd/reports/Model-CI-CD-with-W-B--Vmlldzo0OTcwNDQw).
- [Watch a video demonstrating automatically deploying a model to a Sagemaker endpoint](https://www.youtube.com/watch?v=s5CMj_w3DaQ).
- [Watch a video series introducing automations](https://youtube.com/playlist?list=PLD80i8An1OEGECFPgY-HPCNjXgGu-qGO6&feature=shared).
</Note>

## Requirements
- A team admin can create and manage automations for the team's projects, as well as components of their automations, such as webhooks, secrets, and Slack integrations. Refer to [Team settings](/platform/app/settings-page/teams/).
- A team admin can create and manage automations for the team's projects, as well as components of their automations, such as webhooks, secrets, and Slack integrations. Refer to [Team settings](/platform/app/settings-page/teams).
- To create a registry automation, you must have access to the registry. Refer to [Configure Registry access](/models/registry/configure_registry/#registry-roles).
- To create a Slack automation, you must have permission to post to the Slack instance and channel you select.

## Create an automation
Create an automation from the project or registry's **Automations** tab. At a high level, to create an automation, follow these steps:

1. If necessary, [create a W&B secret](/platform/secrets/) for each sensitive string required by the automation, such as an access token, password, or SSH key. Secrets are defined in your **Team Settings**. Secrets are most commonly used in webhook automations.
1. If necessary, [create a W&B secret](/platform/secrets) for each sensitive string required by the automation, such as an access token, password, or SSH key. Secrets are defined in your **Team Settings**. Secrets are most commonly used in webhook automations.
1. Configure team-level webhook or Slack integrations to authorize W&B to post to Slack or run the webhook on your behalf. A single webhook or Slack integration can be used by multiple automations. These actions are defined in your **Team Settings**.
1. In the project or registry, create the automation, which specifies the event to watch for and the action to take (such as posting to Slack or running a webhook). When you create a webhook automation, you configure the payload it sends.

Expand All @@ -35,8 +35,8 @@ Or, from a line plot in the workspace, you can quickly create a [run metric auto

For details, refer to:

- [Create a Slack automation](/models/automations/create-automations/slack/)
- [Create a webhook automation](/models/automations/create-automations/webhook/)
- [Create a Slack automation](/models/automations/create-automations/slack)
- [Create a webhook automation](/models/automations/create-automations/webhook)

## View and manage automations
View and manage automations from a project or registry's **Automations** tab.
Expand All @@ -47,7 +47,9 @@ View and manage automations from a project or registry's **Automations** tab.
- To delete an automation, click its action `...` menu, then click **Delete automation**.

## Next steps
- Learn more about [automation events and scopes](/models/automations/automation-events/)
- [Create a Slack automation](/models/automations/create-automations/slack/).
- [Create a webhook automation](/models/automations/create-automations/webhook/).
- [Create a secret](/platform/secrets/).
- [Automations tutorials](/models/automations/tutorial): Build a [project run failure alert](/models/automations/project-automation-tutorial) or a [registry alias-to-webhook automation](/models/automations/registry-automation-tutorial) (UI or API).
- [Manage automations with the API](/models/automations/api).
- [Automation events and scopes](/models/automations/automation-events).
- [Create a Slack automation](/models/automations/create-automations/slack).
- [Create a webhook automation](/models/automations/create-automations/webhook).
- [Create a secret](/platform/secrets).
Loading
Loading