Skip to content

Design: Key-Value Store for Automation State Persistence#68

Closed
jpshackelford wants to merge 1 commit into
mainfrom
feature/kv-store-design
Closed

Design: Key-Value Store for Automation State Persistence#68
jpshackelford wants to merge 1 commit into
mainfrom
feature/kv-store-design

Conversation

@jpshackelford
Copy link
Copy Markdown
Contributor

Summary

This PR adds the design document for a built-in key-value store API that enables automations to persist state between runs.

Problem

Automations often need to store small amounts of data between runs—for example, tracking the last processed timestamp for incremental data sync. Currently, users must set up external storage services (JSONBin.io, Redis Cloud, etc.), which undermines the "batteries included" value of the automation platform.

Solution

Provide a built-in KV store API scoped per-automation with:

  • Simple Redis-like semantics — GET, SET, DELETE, INCR/DECR, list operations (LPUSH, RPUSH, LPOP, RPOP)
  • Application-level encryption — All values encrypted with JWE before storage
  • Strict isolation — JWT tokens scoped per-automation, one automation cannot access another's data
  • Opt-in — Automations enable via enable_kv_store flag

Design Highlights

  • Storage: PostgreSQL with encrypted BYTEA values
  • Auth: Per-run JWT tokens passed via AUTOMATION_KV_TOKEN env var
  • Client: Separate openhands-kv package installed in sandbox
  • API: /api/automation/v1/kv/{key} endpoints

Related

Closes #67


This PR was created by an AI agent (OpenHands) on behalf of the user.

@jpshackelford can click here to continue refining the PR

Design for a built-in key-value store API for automation state persistence.

Key features:
- Scoped per-automation with strict isolation
- Redis-like API semantics (GET, SET, INCR, list operations)
- Application-level encryption (JWE) for all values
- JWT-based authentication per automation run

Closes #67

Co-authored-by: openhands <openhands@all-hands.dev>
@github-actions
Copy link
Copy Markdown

🚀 Deploy Preview PR Created/Updated

A deploy preview has been created/updated for this PR.

Deploy PR: https://github.com/OpenHands/deploy/pull/3918
Automation SHA: cec19335606b5fe2716a22b1efe9eeb913805177
Last updated: Apr 24, 2026, 05:09:38 PM ET

Once the deploy PR's CI passes, the automation service will be deployed to the feature environment.

@github-actions
Copy link
Copy Markdown

Coverage

Coverage Report
FileStmtsMissCoverMissing
__init__.py10100% 
app.py1175949%36, 39, 42, 48, 50, 53, 56–59, 62–63, 66, 73–74, 77–78, 82, 90–91, 94, 101–102, 104, 107–108, 111, 116–124, 126–128, 208–210, 215–216, 218–219, 221, 224–226, 228–229, 231–232, 237–238, 242, 245, 247
auth.py111595%71, 113, 270, 278–279
config.py470100% 
constants.py120100% 
db.py442640%37–39, 48–49, 51–52, 54, 62, 69, 79, 82–83, 87–88, 96, 104, 109, 114, 117–123
dispatcher.py1373574%53, 73, 85, 87–88, 157, 161, 183, 190, 209–211, 239–241, 244–246, 281–282, 306–313, 334–335, 345–346, 353–354, 356
event_router.py591967%83, 88, 119–121, 137–138, 156, 158, 160–161, 163, 173, 179–181, 184, 186, 188
exceptions.py40100% 
execution.py22212742%45–47, 56, 89–92, 100–102, 110, 115–119, 133, 135, 137–140, 142–147, 149, 151–158, 160–161, 163, 178, 183, 194, 200–202, 213, 219–221, 268, 276, 280, 282–283, 288–289, 294, 342, 344, 362, 365–368, 390–393, 395, 403–404, 407, 413, 476–482, 485–486, 488, 490–492, 495, 498, 501–504, 506, 509–510, 513–515, 519, 522, 526–529, 531, 539–540, 544–546, 548–554, 558, 560, 569–571, 573–575
filter_eval.py50296%161–162
logger.py531866%25–26, 36, 46–47, 49–55, 68, 88, 90–93
models.py800100% 
preset_router.py1825669%124–126, 234–235, 240–247, 252, 255, 257–258, 269–272, 274–278, 283, 292, 353–355, 466–467, 472–479, 484, 487, 489–490, 501–504, 506–510, 515, 525
router.py1136443%74–75, 95, 97, 100, 102, 116, 129, 131–132, 134–135, 138–140, 151–153, 171–174, 193, 196, 199, 206, 208, 237, 242–244, 247–249, 253–254, 259, 263–266, 268, 276, 278–279, 284–285, 288, 290, 292–294, 297–300, 305, 307–308, 317, 338–340, 344
scheduler.py57984%124–125, 162–163, 178–179, 189–190, 192
schemas.py2681893%150, 156–158, 217–219, 221, 283, 312–313, 316, 321, 326, 332, 478, 486, 493
trigger_matcher.py28389%72–74
uploads.py1075944%138–141, 149–151, 157–158, 161, 170–171, 174–175, 183–184, 186–189, 192–195, 197, 199–201, 203–206, 208–209, 211, 226, 232–233, 236, 239, 242, 245, 247, 260–261, 275, 278–280, 282–283, 285, 291–292, 305, 313–315, 319
watchdog.py1074756%63–64, 76–77, 82–84, 96–97, 241–242, 244, 246, 255, 257–259, 261, 268–270, 272–274, 276–277, 279, 294, 296, 301–304, 306–311, 313–319, 321
webhook_router.py804840%57, 82–83, 107–108, 110, 113–114, 116, 126, 128–132, 137, 139, 151, 154, 157, 164–165, 167, 180, 182–183, 188, 204, 206–207, 213–215, 217–218, 220, 235, 237–238, 243–244, 261, 263–264, 270–271, 273, 275
event_schemas
   __init__.py29196%53
   custom.py33584%52–53, 64–66
   detection.py320100% 
   github.py125496%306, 311, 456, 483
presets
   __init__.py00100% 
storage
   __init__.py50100% 
   factory.py110100% 
   file_store.py18572%11, 20, 25, 30, 54
   google_cloud.py751086%103–108, 142–143, 196, 198
   s3.py1151487%100, 102–103, 107, 109, 190, 213–215, 269–270, 275, 337–338
utils
   __init__.py40100% 
   api_key.py322425%40–41, 46–48, 50, 55, 60, 62–65, 67–68, 70–71, 73, 79, 81–82, 89, 91–92, 98
   cron.py45686%39, 45, 74, 80, 123, 140
   run.py751284%74–76, 172–174, 179–181, 228, 234–235
   sandbox.py1088224%32–37, 51–52, 57–60, 62–64, 66–72, 84, 86, 96–97, 99–101, 103–104, 107–108, 114, 120–122, 132–133, 138–144, 167–168, 170–174, 176–180, 217–218, 220, 222–225, 230–231, 234, 236–237, 243–245, 250–252, 257–258, 266–268, 270
   tarball_validation.py480100% 
   time.py30100% 
   webhook.py511668%46, 51, 119, 129–131, 137, 174, 177–183, 189
TOTAL268877471% 

Copy link
Copy Markdown
Contributor Author

Out of Scope: Admin Debugging Tooling

We are calling out of scope for this implementation:

  • An automation-specific role/permission that would allow an engineer to obtain a key to list, retrieve, and manipulate KV data stored for an automation
  • Admin CLI or API endpoints for debugging encrypted KV store contents

Context: Automations are linked to organizations (org_id), but the automation service delegates role/permission management to the main OpenHands platform. Adding admin debugging capabilities would require coordination with the platform team to define appropriate permissions (e.g., automation:kv:admin).

For now: All KV data is encrypted at rest. Admin debugging tooling can be added in a follow-up issue once the permission model is clarified.


This comment was created by an AI agent (OpenHands) on behalf of the user.

Copy link
Copy Markdown
Contributor Author

Closing this PR as the design document has been merged into PR #69, which now contains both the design document and the implementation.


This PR was closed by an AI agent (OpenHands) on behalf of the user.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature: Key-Value Store for Automation State Persistence

2 participants