Conversation
Agent-Logs-Url: https://github.com/Analytical-Guide/Datalake-Guide/sessions/12994727-fdb4-452b-be6e-98d1e863bd19 Co-authored-by: moshesham <7207587+moshesham@users.noreply.github.com>
…ection Agent-Logs-Url: https://github.com/Analytical-Guide/Datalake-Guide/sessions/12994727-fdb4-452b-be6e-98d1e863bd19 Co-authored-by: moshesham <7207587+moshesham@users.noreply.github.com>
Agent-Logs-Url: https://github.com/Analytical-Guide/Datalake-Guide/sessions/12994727-fdb4-452b-be6e-98d1e863bd19 Co-authored-by: moshesham <7207587+moshesham@users.noreply.github.com>
Copilot created this pull request from a session on behalf of
moshesham
April 27, 2026 19:22
View session
Contributor
|
@copilot Review the content and the structure and beef up and improve the quality of the content |
Agent-Logs-Url: https://github.com/Analytical-Guide/Datalake-Guide/sessions/ec4d1303-fd4a-4a4b-af0a-36c444e7a501 Co-authored-by: moshesham <7207587+moshesham@users.noreply.github.com>
Contributor
Author
Done in commit README.md
docs/awesome-list.md
docs/comparisons/feature-matrix.md
docs/tutorials/migration-guide.md
docs/architecture/system-overview.md
QUICKSTART.md
|
Copilot
AI
changed the title
Stabilize scheduled GitHub Actions and enable weekly automated content curation updates
Stabilize scheduled GitHub Actions, enable weekly automated content curation, and improve documentation quality
Apr 27, 2026
moshesham
approved these changes
Apr 27, 2026
There was a problem hiding this comment.
Pull request overview
This PR stabilizes several scheduled GitHub Actions (leaderboard, quiz leaderboard, stale-content bot, and resource aggregation), hardens the weekly content aggregation path to work under current repo permissions, and refreshes/cleans up key documentation pages.
Changes:
- Made scheduled automations more resilient (quiz leaderboard issue resolution; stale-docs timezone handling; fixed trusted-source endpoints).
- Switched the awesome-list aggregator from PR creation to direct commit/push updates.
- Refreshed and corrected documentation content/structure (README, awesome list resources, feature matrix metadata, migration guide cleanup, architecture diagram updates).
Reviewed changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
scripts/update_quiz_leaderboard.py |
Adds fallback issue resolution via quiz-leaderboard label when ISSUE_NUMBER is absent. |
scripts/find_stale_docs.py |
Normalizes datetimes to UTC for correct comparisons; adjusts directories scanned. |
scripts/find_new_articles.py |
Fixes Iceberg blog URL to a valid endpoint. |
scripts/config/trusted_sources.json |
Updates trusted sources to valid Iceberg RSS/blog endpoints. |
docs/tutorials/migration-guide.md |
Removes leaked Windows file-path/XML artifact from the end of the doc. |
docs/comparisons/feature-matrix.md |
Replaces placeholder last-reviewed date; adds explicit version coverage note. |
docs/awesome-list.md |
Replaces placeholder resources with current real links; updates books/benchmarks and last-updated date. |
docs/architecture/system-overview.md |
Updates the aggregation sequence diagram to reflect the new push-based update flow. |
README.md |
Major doc restructure + adds required leaderboard anchors for automation; updates links and workflow descriptions. |
QUICKSTART.md |
Updates workflow listing and expands automation descriptions (resource aggregation + quiz leaderboard). |
.github/workflows/awesome-list-aggregator.yml |
Replaces PR creation with direct git commit/push of aggregated content updates. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
17
to
19
| STALE_THRESHOLD_MONTHS = 12 | ||
| DIRECTORIES_TO_CHECK = ["docs/", "tutorials/"] | ||
| DIRECTORIES_TO_CHECK = ["docs/", "docs/tutorials/"] | ||
| STALE_LABEL = "stale-content" |
Comment on lines
+85
to
+89
| for issue in repo.get_issues(state="open"): | ||
| if any(label.name == QUIZ_LABEL for label in issue.labels): | ||
| return issue.number | ||
|
|
||
| return None |
| if git diff --cached --quiet; then | ||
| echo "ℹ️ No file changes to commit" | ||
| else | ||
| git commit -m "Add new curated resources to awesome list [skip ci]" |
| AI-->>Agg: Summaries (or simple fallback) | ||
| Agg->>Repo: Update docs/awesome-list.md | ||
| Agg->>Repo: Update community/processed_urls.json | ||
| Repo->>Repo: git commit & push [skip ci] |
Comment on lines
+80
to
+88
| | Automation | Trigger | What It Does | | ||
| |-----------|---------|-------------| | ||
| | **Code Recipe CI** | Every PR | Lints Python, runs `validate.sh` per recipe | | ||
| | **Documentation CI** | Every PR | Markdownlint, link checker, Mermaid diagram validation | | ||
| | **Stale Content Bot** | Weekly (Mon) | Opens issues for docs untouched > 12 months | | ||
| | **Resource Aggregator** | Weekly (Sun) | Discovers new articles from RSS feeds, commits to awesome list | | ||
| | **Leaderboard Update** | Daily | Regenerates top-10 contributor table in README | | ||
| | **Gamification Engine** | PR/Review/Issue | Awards points and updates contributor stats | | ||
| | **Quiz Leaderboard** | Issue comment | Updates quiz scores in the leaderboard issue | |
| - [🏗️ **Production Readiness**](docs/best-practices/production-readiness.md) — Best practices for production | ||
| - [🤝 **Contributing Guide**](CONTRIBUTING.md) — Earn points, join the community | ||
| - [📜 **Code of Conduct**](CODE_OF_CONDUCT.md) — Community standards | ||
| - [🏆 **Community Leaderboard**](#-community-leaderboard) — Top contributors |
| │ ├── gamification-engine.yml # Points on PR/review/issue events | ||
| │ ├── update-leaderboard.yml # Daily: regenerate README leaderboard | ||
| │ ├── update-quiz-leaderboard.yml # Update quiz scores in leaderboard issue | ||
| │ └── awesome-list-aggregator.yml # Weekly: discover & commit new articles |
Comment on lines
159
to
163
| issue_body = f"""## 📅 Stale Content Detected | ||
|
|
||
| **File:** `{filepath}` | ||
| **Last Updated:** {last_modified.strftime('%Y-%m-%d')} ({(datetime.now() - last_modified).days} days ago) | ||
| **Last Updated:** {last_modified.strftime('%Y-%m-%d')} ({age_days} days ago) | ||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR addresses recurring failures in scheduled workflows, advances the weekly content-agent path by making automated discovery/updates robust under current repository permissions and runtime behavior, and improves the quality, accuracy, and structure of repository content.
Workflow breakage fixes (current failures)
README.mdsoscripts/generate_leaderboard.pycan update content in-place.ISSUE_NUMBERis absent (common onschedule/workflow_dispatch) by resolving the target issue viaquiz-leaderboardlabel.Awesome List agent flow hardening
awesome-list-aggregator.ymlwith direct commit/push for discovered content updates, avoiding failure mode where Actions cannot open PRs in this repo configuration.Content quality and structure improvements
README.md: Replaced vague badge labels with version-specific ones (Delta Lake 3.x, Iceberg 1.5+); added a sharp one-liner description; replaced the verbose Vision Statement with a focused needs→content table; added a full repository tree with inline descriptions; converted the automation bullet list into a structured table with trigger and schedule for each of the 7 workflows; merged duplicated "Getting Started" sections; fixed brokenquiz/internal links →quiz.md.docs/awesome-list.md: Replaced 2024-01-01 placeholder articles with four current real resources (Delta Lake 3.2 Liquid Clustering, Iceberg 1.5, 2025 format comparison, AWS lakehouse guide); updated books section with full publication details including the O'Reilly Iceberg Definitive Guide (2024); added Dremio benchmark reference; updated Last Updated date.docs/comparisons/feature-matrix.md: Replaced[CURRENT_DATE]placeholder with2026-04-27and added explicit Delta 3.2 / Iceberg 1.5 version coverage note.docs/tutorials/migration-guide.md: Removed leaked Windows file-path artifact (</content><parameter name="filePath">c:\Users\...) from end of file.docs/architecture/system-overview.md: Updated the Resource Aggregation Flow sequence diagram to reflect the current push-based commit approach instead of the now-removed PR-creation step.QUICKSTART.md: Addedupdate-quiz-leaderboard.ymlto the workflow listing; expanded Resource Aggregation and Quiz Leaderboard automation descriptions to match current behavior.