Skip to content

feat(api): Add generic webhook source for arbitrary event-driven task spawning #902

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
gjkim42 merged 1 commit into from
Apr 16, 2026
Merged

feat(api): Add generic webhook source for arbitrary event-driven task spawning #902

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 59 additions & 0 deletions api/v1alpha1/taskspawner_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,13 @@ type When struct {
// LinearWebhook triggers task spawning on Linear webhook events.
// +optional
LinearWebhook *LinearWebhook `json:"linearWebhook,omitempty"`

// GenericWebhook triggers task spawning from arbitrary HTTP POST payloads.
// Any system that can send an HTTP POST with a JSON body can trigger
// tasks through this source. The URL path is /webhook/<source> and
// the HMAC secret is read from the <SOURCE>_WEBHOOK_SECRET env var.
// +optional
GenericWebhook *GenericWebhook `json:"webhook,omitempty"`
}

// Cron triggers task spawning on a cron schedule.
Expand Down Expand Up @@ -424,6 +431,58 @@ type LinearWebhookFilter struct {
ExcludeLabels []string `json:"excludeLabels,omitempty"`
}

// GenericWebhook configures webhook-driven task spawning from arbitrary HTTP
// POST payloads with JSON bodies. Any system that can send an HTTP POST can
// trigger tasks through this source. The URL path is /webhook/<source> and
// the HMAC secret is read from the <SOURCE>_WEBHOOK_SECRET env var (e.g.,
// source "notion" uses NOTION_WEBHOOK_SECRET).
// +kubebuilder:validation:XValidation:rule="'id' in self.fieldMapping",message="fieldMapping must include an 'id' key for deduplication and task naming"
type GenericWebhook struct {
// Source is a short identifier for this webhook source (e.g., "notion",
// "sentry", "drata"). It determines:
// - The URL path: /webhook/<source>
// - The env var for HMAC validation: <SOURCE>_WEBHOOK_SECRET
// Must be lowercase alphanumeric with optional hyphens.
// +kubebuilder:validation:Required
// +kubebuilder:validation:Pattern=`^[a-z0-9]([a-z0-9-]*[a-z0-9])?$`
Source string `json:"source"`
Copy link
Copy Markdown
Collaborator

@gjkim42 gjkim42 Apr 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The source should be unique in cluster scope, right?

Copy link
Copy Markdown
Contributor Author

@knechtionscoding knechtionscoding Apr 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. The source is unique in the cluster, but we aren't spawning a new webhook server per source (at least not currently).

The current implementation spawns a single GenericWebhook taskSpawner that then listens for everything that isn't routed to one of our specific handlers. This means on the taskSpawner you have to declare which source you care to listen too.

For example, if you are listening to both notion and salesforce on the taskspawners for those you need to declare what source you care about.

Copy link
Copy Markdown
Contributor Author

@knechtionscoding knechtionscoding Apr 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gjkim42 Any concerns here? Otherwise I'll rework the other thread and push for final review

Copy link
Copy Markdown
Collaborator

@gjkim42 gjkim42 Apr 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a validation that prevents duplicate sources?

Copy link
Copy Markdown
Contributor Author

@knechtionscoding knechtionscoding Apr 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, because it isn't declared anywhere as a universal.

You are allowed to have multiple task spawners listening to the same source.

And on the webhook server side it just sees the path and reads for the secret. There's no ability for duplication anywhere.

Because once deployed you can send to webhook/test, webhook/notion, webhook/drata without any further configuration on the server side.

Once the generic source is deployed it then just checks for an existing taskSpawner with source that matches what it received.

So there's nothing to ensure isn't duplicated.


// FieldMapping maps JSONPath expressions to WorkItem template variables.
// Each key is a template variable name (available as {{.Key}} in
// promptTemplate and branch), and each value is a JSONPath expression
// evaluated against the request body.
// The "id" key is required — it provides the unique identifier used for
// deduplication and task naming.
// +kubebuilder:validation:Required
FieldMapping map[string]string `json:"fieldMapping"`

// Filters define conditions that must ALL match for a webhook delivery
// to trigger a task (AND semantics across filters). Each filter extracts
// a field via JSONPath and matches it against an exact value or regex
// pattern. If empty, all deliveries trigger tasks.
// +optional
Filters []GenericWebhookFilter `json:"filters,omitempty"`
}

// GenericWebhookFilter defines a condition for filtering generic webhook payloads.
// Exactly one of Value or Pattern must be set.
// +kubebuilder:validation:XValidation:rule="has(self.value) != (has(self.pattern) && size(self.pattern) > 0)",message="exactly one of value or pattern must be set"
type GenericWebhookFilter struct {
// Field is a JSONPath expression selecting the payload field to match.
// +kubebuilder:validation:Required
Field string `json:"field"`

// Value requires an exact string match against the extracted field value.
// Mutually exclusive with Pattern.
// +optional
Value *string `json:"value,omitempty"`

// Pattern requires a regex match against the extracted field value.
// Mutually exclusive with Value.
// +optional
Pattern string `json:"pattern,omitempty"`
}

// TaskTemplateMetadata holds optional labels and annotations for spawned Tasks.
type TaskTemplateMetadata struct {
// Labels are merged into the spawned Task's labels. Values support Go
Expand Down
54 changes: 54 additions & 0 deletions api/v1alpha1/zz_generated.deepcopy.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

13 changes: 10 additions & 3 deletions cmd/kelos-webhook-server/main.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,9 +65,11 @@ func main() {
webhookSource = webhook.GitHubSource
case "linear":
webhookSource = webhook.LinearSource
case "generic":
webhookSource = webhook.GenericSource
default:
setupLog.Error(fmt.Errorf("invalid source: %s", source),
"Source must be 'github' or 'linear'")
"Source must be 'github', 'linear', or 'generic'")
os.Exit(1)
}

Expand Down Expand Up @@ -98,9 +100,14 @@ func main() {
os.Exit(1)
}

// Set up HTTP server for webhooks
// Set up HTTP server for webhooks.
// Generic source uses /webhook/<source> paths; others use root.
mux := http.NewServeMux()
mux.Handle("/", handler)
if webhookSource == webhook.GenericSource {
mux.Handle("/webhook/", handler)
} else {
mux.Handle("/", handler)
}

webhookServer := &http.Server{
Addr: webhookAddr,
Expand Down
2 changes: 2 additions & 0 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ module github.com/kelos-dev/kelos
go 1.25.0

require (
github.com/PaesslerAG/jsonpath v0.1.1
github.com/go-logr/logr v1.4.3
github.com/google/go-github/v66 v66.0.0
github.com/google/uuid v1.6.0
Expand Down Expand Up @@ -36,6 +37,7 @@ require (
github.com/Masterminds/goutils v1.1.1 // indirect
github.com/Masterminds/semver/v3 v3.4.0 // indirect
github.com/Masterminds/sprig/v3 v3.3.0 // indirect
github.com/PaesslerAG/gval v1.0.0 // indirect
github.com/beorn7/perks v1.0.1 // indirect
github.com/bmatcuk/doublestar/v4 v4.7.1 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
Expand Down
5 changes: 5 additions & 0 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,11 @@ github.com/Masterminds/semver/v3 v3.4.0 h1:Zog+i5UMtVoCU8oKka5P7i9q9HgrJeGzI9SA1
github.com/Masterminds/semver/v3 v3.4.0/go.mod h1:4V+yj/TJE1HU9XfppCwVMZq3I84lprf4nC11bSS5beM=
github.com/Masterminds/sprig/v3 v3.3.0 h1:mQh0Yrg1XPo6vjYXgtf5OtijNAKJRNcTdOOGZe3tPhs=
github.com/Masterminds/sprig/v3 v3.3.0/go.mod h1:Zy1iXRYNqNLUolqCpL4uhk6SHUMAOSCzdgBfDb35Lz0=
github.com/PaesslerAG/gval v1.0.0 h1:GEKnRwkWDdf9dOmKcNrar9EA1bz1z9DqPIO1+iLzhd8=
github.com/PaesslerAG/gval v1.0.0/go.mod h1:y/nm5yEyTeX6av0OfKJNp9rBNj2XrGhAf5+v24IBN1I=
github.com/PaesslerAG/jsonpath v0.1.0/go.mod h1:4BzmtoM/PI8fPO4aQGIusjGxGir2BzcV0grWtFzq1Y8=
github.com/PaesslerAG/jsonpath v0.1.1 h1:c1/AToHQMVsduPAa4Vh6xp2U0evy4t8SWp8imEsylIk=
github.com/PaesslerAG/jsonpath v0.1.1/go.mod h1:lVboNxFGal/VwW6d9JzIy56bUsYAP6tH/x80vjnCseY=
github.com/antlr4-go/antlr/v4 v4.13.0 h1:lxCg3LAv+EUK6t1i0y1V6/SLeUi0eKEKdhQAlS8TVTI=
github.com/antlr4-go/antlr/v4 v4.13.0/go.mod h1:pfChB/xh/Unjila75QW7+VU4TSnWnnk9UTnmpPaOR2g=
github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
Expand Down
2 changes: 1 addition & 1 deletion internal/controller/taskspawner_controller.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ func isCronBased(ts *kelosv1alpha1.TaskSpawner) bool {

// isWebhookBased returns true if the TaskSpawner is webhook-driven.
func isWebhookBased(ts *kelosv1alpha1.TaskSpawner) bool {
return ts.Spec.When.GitHubWebhook != nil || ts.Spec.When.LinearWebhook != nil
return ts.Spec.When.GitHubWebhook != nil || ts.Spec.When.LinearWebhook != nil || ts.Spec.When.GenericWebhook != nil
}

// Reconcile handles TaskSpawner reconciliation.
Expand Down
16 changes: 16 additions & 0 deletions internal/controller/taskspawner_controller_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,22 @@ func TestIsWebhookBased(t *testing.T) {
},
want: false,
},
{
name: "generic webhook TaskSpawner",
ts: &kelosv1alpha1.TaskSpawner{
Spec: kelosv1alpha1.TaskSpawnerSpec{
When: kelosv1alpha1.When{
GenericWebhook: &kelosv1alpha1.GenericWebhook{
Source: "notion",
FieldMapping: map[string]string{
"id": "$.data.id",
},
},
},
},
},
want: true,
},
}

for _, tt := range tests {
Expand Down
68 changes: 68 additions & 0 deletions internal/manifests/charts/kelos/templates/crds/taskspawner-crd.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -995,6 +995,74 @@ spec:
required:
- types
type: object
webhook:
description: |-
GenericWebhook triggers task spawning from arbitrary HTTP POST payloads.
Any system that can send an HTTP POST with a JSON body can trigger
tasks through this source. The URL path is /webhook/<source> and
the HMAC secret is read from the <SOURCE>_WEBHOOK_SECRET env var.
properties:
fieldMapping:
Comment thread
cubic-dev-ai[bot] marked this conversation as resolved.
additionalProperties:
type: string
description: |-
FieldMapping maps JSONPath expressions to WorkItem template variables.
Each key is a template variable name (available as {{ "{{.Key}}" }} in
promptTemplate and branch), and each value is a JSONPath expression
evaluated against the request body.
The "id" key is required — it provides the unique identifier used for
deduplication and task naming.
type: object
filters:
description: |-
Filters define conditions that must ALL match for a webhook delivery
to trigger a task (AND semantics across filters). Each filter extracts
a field via JSONPath and matches it against an exact value or regex
pattern. If empty, all deliveries trigger tasks.
items:
description: |-
GenericWebhookFilter defines a condition for filtering generic webhook payloads.
Exactly one of Value or Pattern must be set.
properties:
field:
description: Field is a JSONPath expression selecting
the payload field to match.
type: string
pattern:
description: |-
Pattern requires a regex match against the extracted field value.
Mutually exclusive with Value.
type: string
value:
description: |-
Value requires an exact string match against the extracted field value.
Mutually exclusive with Pattern.
type: string
required:
- field
type: object
x-kubernetes-validations:
- message: exactly one of value or pattern must be set
rule: has(self.value) != (has(self.pattern) && size(self.pattern)
> 0)
type: array
source:
description: |-
Source is a short identifier for this webhook source (e.g., "notion",
"sentry", "drata"). It determines:
- The URL path: /webhook/<source>
- The env var for HMAC validation: <SOURCE>_WEBHOOK_SECRET
Must be lowercase alphanumeric with optional hyphens.
pattern: ^[a-z0-9]([a-z0-9-]*[a-z0-9])?$
type: string
required:
- fieldMapping
- source
type: object
x-kubernetes-validations:
- message: fieldMapping must include an 'id' key for deduplication
and task naming
rule: '''id'' in self.fieldMapping'
type: object
required:
- taskTemplate
Expand Down
12 changes: 11 additions & 1 deletion internal/manifests/charts/kelos/templates/webhook-gateway.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
{{- if .Values.webhookServer.gateway.enabled }}
{{- $githubEnabled := .Values.webhookServer.sources.github.enabled }}
{{- $linearEnabled := .Values.webhookServer.sources.linear.enabled }}
{{- if or $githubEnabled $linearEnabled }}
{{- $genericEnabled := .Values.webhookServer.sources.generic.enabled }}
{{- if or $githubEnabled $linearEnabled $genericEnabled }}
---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
Expand Down Expand Up @@ -82,5 +83,14 @@ spec:
- name: kelos-webhook-linear
port: 8443
{{- end }}
{{- if $genericEnabled }}
- matches:
- path:
type: PathPrefix
value: /webhook/
backendRefs:
- name: kelos-webhook-generic
port: 8443
{{- end }}
{{- end }}
{{- end }}
12 changes: 11 additions & 1 deletion internal/manifests/charts/kelos/templates/webhook-ingress.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
{{- if .Values.webhookServer.ingress.enabled }}
{{- $githubEnabled := .Values.webhookServer.sources.github.enabled }}
{{- $linearEnabled := .Values.webhookServer.sources.linear.enabled }}
{{- if or $githubEnabled $linearEnabled }}
{{- $genericEnabled := .Values.webhookServer.sources.generic.enabled }}
{{- if or $githubEnabled $linearEnabled $genericEnabled }}
---
apiVersion: networking.k8s.io/v1
kind: Ingress
Expand Down Expand Up @@ -49,5 +50,14 @@ spec:
port:
number: 8443
{{- end }}
{{- if $genericEnabled }}
- path: /webhook/
pathType: Prefix
backend:
service:
name: kelos-webhook-generic
port:
number: 8443
{{- end }}
{{- end }}
{{- end }}
Loading
Loading