Skip to content

Feature: Support jsontemplate in http-bulk#2

Open
PranavSathy wants to merge 2 commits intohalon-extras:mainfrom
PranavSathy:feature/support_json_template
Open

Feature: Support jsontemplate in http-bulk#2
PranavSathy wants to merge 2 commits intohalon-extras:mainfrom
PranavSathy:feature/support_json_template

Conversation

@PranavSathy
Copy link
Copy Markdown

WARNING: AI Generated, utilized Codex 5.4 High Reasoning

Prompt:

This is a C++ repository for a plugin to "Halon" (https://halon.io) to support bulk sending HTTP requests
  to external systems while reading queued JSON objects located on Disk.

  The logic reads from `http-bulk.schema.json` in C++ (halon configuration) to construct the HTTP requests.

  Currently, the issue is that the only two formats that are really usable are `ndjson` and `jsonarray`.

  I want to introduce a new plugin configuration value (similar to format) called "jsontemplate" which supports
  formatting the JSON objects based on a template string before sending. My end goal (which needs to be unit testable) is generating the following HTTP request:

  {
      workflowType: { name: "handleHalonEmails" },
      taskQueue: { name: "lbx-live-halon-workflows", kind: "TASK_QUEUE_KIND_NORMAL" },
      identity: "halon-http-bulk",
      requestId: $UUIDV7,
      workflowTaskTimeout: "10s",
      input: {
          payloads: [
              {
                  metadata: { encoding: "anNvbi9wbGFpbg==" },
                  data: $DATA_BASE64
              }
          ]
      }
  }

  In this situation, I want to support a couple of custom tags:

  $UUIDV7 - A uuidv7 generator that subs out a UUIDv7 value within the template string.
  $DATA_BASE64 - The jsonarray into a base64 encoded json string.

This also wrote some extra tests to validate that this works.

NOTE: I am not C++ expert, I reviewed the code to the best of my ability, I suspect that this will result in some more mallocs considering the liberal use of std::string. I would encourage a more seasoned C++ reviewer take a look.

However, this totally works and offers some flexibility to the http-bulk plugin to interact with more modern systems like SQS, Temporal, etc. to enqueue tasks directly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant