Skip to content

Commit 0a8c562

Browse files
nicktrnnicktrn
authored
Merge branch 'main' into improvement/admin-tabs-preserve-search
2 parents 7a7a82d + d144220 commit 0a8c562

4 files changed

Lines changed: 175 additions & 8 deletions

File tree

‎.github/workflows/pr_checks.yml‎

Lines changed: 136 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -3,33 +3,162 @@ name: 🤖 PR Checks
33
on:
44
pull_request:
55
types: [opened, synchronize, reopened]
6-
paths-ignore:
7-
- "docs/**"
8-
- ".changeset/**"
9-
- "hosting/**"
10-
- ".github/workflows/helm-prerelease.yml"
116

127
concurrency:
138
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
149
cancel-in-progress: true
1510

1611
permissions:
1712
contents: read
13+
pull-requests: read
1814

1915
jobs:
16+
changes:
17+
name: Detect changes
18+
runs-on: ubuntu-latest
19+
outputs:
20+
code: ${{ steps.filter.outputs.code }}
21+
webapp: ${{ steps.filter.outputs.webapp }}
22+
packages: ${{ steps.filter.outputs.packages }}
23+
internal: ${{ steps.filter.outputs.internal }}
24+
cli: ${{ steps.filter.outputs.cli }}
25+
sdk: ${{ steps.filter.outputs.sdk }}
26+
steps:
27+
- uses: dorny/paths-filter@fbd0ab8f3e69293af611ebaee6363fc25e6d187d # v4.0.1
28+
id: filter
29+
with:
30+
filters: |
31+
code:
32+
- '**'
33+
- '!docs/**'
34+
- '!.changeset/**'
35+
- '!hosting/**'
36+
- '!.github/workflows/helm-prerelease.yml'
37+
webapp:
38+
- 'apps/webapp/**'
39+
- 'packages/**'
40+
- 'internal-packages/**'
41+
- '.github/workflows/pr_checks.yml'
42+
- '.github/workflows/unit-tests-webapp.yml'
43+
- '.github/workflows/e2e-webapp.yml'
44+
- '.configs/**'
45+
- 'package.json'
46+
- 'pnpm-lock.yaml'
47+
- 'pnpm-workspace.yaml'
48+
- 'turbo.json'
49+
packages:
50+
- 'packages/**'
51+
- '.github/workflows/pr_checks.yml'
52+
- '.github/workflows/unit-tests-packages.yml'
53+
- '.configs/**'
54+
- 'package.json'
55+
- 'pnpm-lock.yaml'
56+
- 'pnpm-workspace.yaml'
57+
- 'turbo.json'
58+
internal:
59+
- 'internal-packages/**'
60+
- 'packages/**'
61+
- '.github/workflows/pr_checks.yml'
62+
- '.github/workflows/unit-tests-internal.yml'
63+
- '.configs/**'
64+
- 'package.json'
65+
- 'pnpm-lock.yaml'
66+
- 'pnpm-workspace.yaml'
67+
- 'turbo.json'
68+
cli:
69+
- 'packages/cli-v3/**'
70+
- 'packages/build/**'
71+
- 'packages/core/**'
72+
- 'packages/schema-to-json/**'
73+
- '.github/workflows/pr_checks.yml'
74+
- '.github/workflows/e2e.yml'
75+
- '.configs/**'
76+
- 'package.json'
77+
- 'pnpm-lock.yaml'
78+
- 'pnpm-workspace.yaml'
79+
- 'turbo.json'
80+
sdk:
81+
- 'packages/trigger-sdk/**'
82+
- 'packages/core/**'
83+
- '.github/workflows/pr_checks.yml'
84+
- '.github/workflows/sdk-compat.yml'
85+
- '.configs/**'
86+
- 'package.json'
87+
- 'pnpm-lock.yaml'
88+
- 'pnpm-workspace.yaml'
89+
- 'turbo.json'
90+
2091
typecheck:
92+
needs: changes
93+
if: needs.changes.outputs.code == 'true'
2194
uses: ./.github/workflows/typecheck.yml
2295

23-
units:
24-
uses: ./.github/workflows/unit-tests.yml
96+
webapp:
97+
needs: changes
98+
if: needs.changes.outputs.webapp == 'true'
99+
uses: ./.github/workflows/unit-tests-webapp.yml
100+
secrets:
101+
DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
102+
DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
103+
104+
e2e-webapp:
105+
needs: changes
106+
if: needs.changes.outputs.webapp == 'true'
107+
uses: ./.github/workflows/e2e-webapp.yml
108+
secrets:
109+
DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
110+
DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
111+
112+
packages:
113+
needs: changes
114+
if: needs.changes.outputs.packages == 'true'
115+
uses: ./.github/workflows/unit-tests-packages.yml
116+
secrets:
117+
DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
118+
DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
119+
120+
internal:
121+
needs: changes
122+
if: needs.changes.outputs.internal == 'true'
123+
uses: ./.github/workflows/unit-tests-internal.yml
25124
secrets:
26125
DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
27126
DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
28127

29128
e2e:
129+
needs: changes
130+
if: needs.changes.outputs.cli == 'true'
30131
uses: ./.github/workflows/e2e.yml
31132
with:
32133
package: cli-v3
33134

34135
sdk-compat:
136+
needs: changes
137+
if: needs.changes.outputs.sdk == 'true'
35138
uses: ./.github/workflows/sdk-compat.yml
139+
140+
all-checks:
141+
name: All PR Checks
142+
needs:
143+
- changes
144+
- typecheck
145+
- webapp
146+
- e2e-webapp
147+
- packages
148+
- internal
149+
- e2e
150+
- sdk-compat
151+
if: always()
152+
runs-on: ubuntu-latest
153+
steps:
154+
- name: Verify all checks
155+
run: |
156+
if [[ "${{ contains(needs.*.result, 'failure') }}" == "true" ]]; then
157+
echo "One or more checks failed"
158+
exit 1
159+
fi
160+
if [[ "${{ contains(needs.*.result, 'cancelled') }}" == "true" ]]; then
161+
echo "One or more checks were cancelled"
162+
exit 1
163+
fi
164+
echo "All checks passed or were skipped due to path filters"

‎docs/self-hosting/docker.mdx‎

Lines changed: 16 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -189,6 +189,7 @@ docker compose up -d
189189
To create additional worker groups beyond the bootstrap group, use the admin API endpoint. This requires admin privileges.
190190
191191
**Making a user admin:**
192+
192193
- **New users**: Set `ADMIN_EMAILS` environment variable (regex pattern) before user creation.
193194
- **Existing users**: Set `admin = true` in the `user` table in your database.
194195
@@ -341,6 +342,18 @@ By default, the images will point at the latest versioned release via the `lates
341342
TRIGGER_IMAGE_TAG=v4.0.0
342343
```
343344
345+
## Task events
346+
347+
By default, task events (timeline, logs, spans) are stored in PostgreSQL. For production deployments we recommend storing them in ClickHouse instead, it scales to much higher volumes and avoids unbounded growth of the `TaskEvent` table.
348+
349+
To enable, set on the webapp in your `.env`:
350+
351+
```bash
352+
EVENT_REPOSITORY_DEFAULT_STORE=clickhouse_v2
353+
```
354+
355+
This only affects new runs; existing runs continue to read from wherever their events were originally stored.
356+
344357
## Troubleshooting
345358
346359
- **Deployment fails at the push step.** The machine running `deploy` needs registry access. See the [registry setup](#registry-setup) section for more details.
@@ -359,7 +372,9 @@ TRIGGER_IMAGE_TAG=v4.0.0
359372
- **ClickHouse migrations say "no migrations to run" but schema is missing.** The goose migration tracker is out of sync. Exec into the webapp container, set the GOOSE env vars (from webapp startup logs), and run `goose reset && goose up`.
360373
361374
<Warning>
362-
**Data Loss Warning:** The `goose reset` command is destructive and will drop the entire schema. Make sure to backup your data and confirm you are running this in a non-production environment before executing this command.
375+
**Data Loss Warning:** The `goose reset` command is destructive and will drop the entire schema.
376+
Make sure to backup your data and confirm you are running this in a non-production environment
377+
before executing this command.
363378
</Warning>
364379
365380
## CLI usage

‎docs/self-hosting/env/webapp.mdx‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -123,6 +123,8 @@ mode: "wide"
123123
| `TRIGGER_OTEL_ATTRIBUTE_PER_LINK_COUNT_LIMIT` | No | 10 | OTel attribute per link count limit. |
124124
| `TRIGGER_OTEL_ATTRIBUTE_PER_EVENT_COUNT_LIMIT` | No | 10 | OTel attribute per event count limit. |
125125
| `SERVER_OTEL_SPAN_ATTRIBUTE_VALUE_LENGTH_LIMIT` | No | 8192 | OTel span attribute value length limit. |
126+
| **Task events** | | | |
127+
| `EVENT_REPOSITORY_DEFAULT_STORE` | No | postgres | Where to store task events. Set to `clickhouse_v2` to store in ClickHouse (recommended for production). |
126128
| **Realtime** | | | |
127129
| `REALTIME_STREAM_MAX_LENGTH` | No | 1000 | Realtime stream max length. |
128130
| `REALTIME_STREAM_TTL` | No | 86400 (1d) | Realtime stream TTL (s). |

‎docs/self-hosting/kubernetes.mdx‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -354,6 +354,27 @@ webapp:
354354
- Compatible with secret management tools (External Secrets Operator, etc.)
355355
- Follows Kubernetes security best practices
356356

357+
## DNS performance
358+
359+
For production clusters we recommend deploying [NodeLocal DNSCache](https://kubernetes.io/docs/tasks/administer-cluster/nodelocaldns/). DNS queries — especially to managed Postgres or Redis endpoints — can be very slow under Kubernetes' default resolver, and a node-local cache typically gives a large step change in latency and throughput across the cluster.
360+
361+
The default `ndots: 5` setting also forces every cluster search domain to be tried before resolving hostnames with fewer dots (the case for most external database hosts). Lowering `ndots` to `1` on the webapp and supervisor pods avoids those extra round-trips.
362+
363+
## Task events
364+
365+
By default, task events (timeline, logs, spans) are stored in PostgreSQL. For production deployments we recommend storing them in ClickHouse instead, it scales to much higher volumes and avoids unbounded growth of the `TaskEvent` table.
366+
367+
ClickHouse is already deployed by the chart, so no extra services are required. To enable, set `EVENT_REPOSITORY_DEFAULT_STORE` on the webapp via `extraEnvVars`:
368+
369+
```yaml
370+
webapp:
371+
extraEnvVars:
372+
- name: EVENT_REPOSITORY_DEFAULT_STORE
373+
value: "clickhouse_v2"
374+
```
375+
376+
This only affects new runs; existing runs continue to read from wherever their events were originally stored.
377+
357378
## Worker token
358379

359380
When using the default bootstrap configuration, worker creation and authentication is handled automatically. The webapp generates a worker token and makes it available to the supervisor via a shared volume.

0 commit comments

Comments
 (0)