Conversation
There was a problem hiding this comment.
Pull request overview
This PR optimizes the janitor’s “biggest old job ID” lookup to avoid an expensive scan/sort on updated_at by leveraging the indexed jobs.system_job_id primary key.
Changes:
- Change the
old_job_biggest_id.sqlquery to order bysystem_job_id DESC(instead ofupdated_at DESC) to fetch the largest old job ID more efficiently.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 1 out of 1 changed files in this pull request and generated 1 comment.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| WHERE updated_at < $1 | ||
| ORDER BY updated_at desc | ||
| ORDER BY system_job_id desc | ||
| LIMIT 1 No newline at end of file |
There was a problem hiding this comment.
For consistency with other janitor SQL queries (e.g. stale_and_canceling_jobs_select.sql, old_jobs_delete.sql), this statement should end with a trailing semicolon. That also avoids accidental issues if the embedded SQL ever gets concatenated with additional statements.
| LIMIT 1 | |
| LIMIT 1; |
Optimized query, my thought was that DB could find the job ID in mid table <10M of rows during secods, it's not true because it causes a whole table scan.
This update started to utilize the fact that a PK is indexed and it's a system_job_id bigint generated always as identity.
Now query is ~500 times faster
Tested with database