Skip to content

feat: add cron orchestration for argo workflows#1422

Open
YufanKambang wants to merge 5 commits intodevelopfrom
cron-orch-argo-workflow
Open

feat: add cron orchestration for argo workflows#1422
YufanKambang wants to merge 5 commits intodevelopfrom
cron-orch-argo-workflow

Conversation

@YufanKambang
Copy link
Copy Markdown
Contributor

@YufanKambang YufanKambang commented Mar 17, 2026

Description

This is for this epic with a stfc cron orchestration service: UserOfficeProject/issue-tracker#1540
This will close this sub-issue: UserOfficeProject/issue-tracker#1552
This is to demo the use of argo-workflow with our own services to schedule in CRON the asyncJobs that

Motivation and Context

How Has This Been Tested

Fixes

Changes

Depends on

Tests included/Docs Updated?

  • I have added tests to cover my changes.
  • All relevant doc has been updated

@YufanKambang YufanKambang requested a review from a team as a code owner March 17, 2026 13:30
@YufanKambang YufanKambang requested review from Bhaswati1148 and removed request for a team March 17, 2026 13:30
@YufanKambang YufanKambang changed the title feature: cron orchestration argo-wroflow entrypoint script feat: cron orchestration argo-wroflow entrypoint script Mar 17, 2026
@YufanKambang
Copy link
Copy Markdown
Contributor Author

In the apps/backend/package.json we have defined new scripts.

When commanding:

cd apps/backend
npm run test:job:local -- checkAllCallsEnded

We where able to know that our code compiled successfully, the DI container initialized ("Invalid or no value was provided for the DEPENDENCY_CONFIG"), the job tried to run.
However, it hit a native library issue (muhammara for PDFs) that requires musl libc, which your local Ubuntu doesn't have. I think this is fine as the docker container will have the right libc.

@YufanKambang YufanKambang changed the title feat: cron orchestration argo-wroflow entrypoint script feat: cron-orchestration-argo-workflow Mar 17, 2026
@YufanKambang YufanKambang changed the title feat: cron-orchestration-argo-workflow feat(argo): cron-orchestration-workflow Mar 17, 2026
@YufanKambang YufanKambang changed the title feat(argo): cron-orchestration-workflow feat: add cron orchestration for argo workflows Mar 17, 2026
@YufanKambang
Copy link
Copy Markdown
Contributor Author

✅ Created runJob.ts a CLI entrypoint for Argo jobs
✅ Built Dockerfile.worker a lean Docker image for job execution
✅ Built and deployed the image to my local Minikube
✅ Created Kubernetes Secret for database credentials, local database needs to be running for this to express
✅ Created and deployed CronWorkflow to Argo, made a specific one for user-office-core/apps/backend/src/asyncJobs/jobs/checkAllCallsEnded.ts
✅ Tested the job successfully in Argo UI and the terminal
✅ Verified logs showing job execution in the Argo UI and terminal

Copy link
Copy Markdown
Contributor

@Scott-James-Hurley Scott-James-Hurley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a proof of concept or something intended to be merged?

RUN cd apps/backend && npm run build

# Set working directory to backend
WORKDIR /app/apps/backend
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couldn't this be moved up to when you start running backend scripts?

);

await checkAllCallsEndedJob.functionToRun(callDataSource);
console.log('Job completed successfully');
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we probably want to use the user office logger instead of console log.

@YufanKambang
Copy link
Copy Markdown
Contributor Author

This is a POC.
I have presented the demo successfully today to Deepak and Hassan.
I have been guided to look more into docker-orchestration to see how this can be integrated. As we use argo-CD and that lives in docker-orchestration

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants