Convert datasets to parquet (un-shard USPTO) and rework validation CI

[skearnes]

Summary

Adds parquet copies of every dataset alongside the existing pb.gz files,
consolidates two shard groups into single un-sharded parquet files, and
restructures CI to validate the new format efficiently.

Conversion (commit ca97f95)

• 489 monthly uspto-grants-YYYY_MM pb.gz files → one un-sharded uspto-grants parquet (1.77M reactions, ~1.0 GB). Per-month CML filename provenance dropped from the description; per-reaction patent provenance is preserved.
• 10 (N/10) Training-data shards from doi.org/10.1039/C8SC04228D → one parquet (409 035 reactions).
• 47 other datasets converted 1:1, keeping name/description/dataset_id.
• pb.gz inputs kept in place.
• Merged-output dataset_ids derived deterministically from the sorted source ids, so re-runs are idempotent.
• New scripts/convert_to_parquet.py is the script that did this; left in-tree as the audit trail and for future re-conversions.

CI restructure (commit ca97f95)

• Renames validate_database → validate_pb (the input glob has always matched all *.pb* extensions). Adds fail-fast: false so one shard failure does not cancel the others. The old 9-way hex-prefix matrix is otherwise unchanged.
• Removes the parquet pass that briefly lived inside that job (added in Bump ORD_SCHEMA_TAG to v0.6.1 and add parquet support #240).
• Adds a new validate_parquet job with a 2-element matrix:
  • uspto: filters to the un-sharded USPTO parquet only. Row-group parallelism in validate_dataset.py saturates the 4-CPU runner on this single ~1.77M-reaction file.
  • other: negative lookahead on the USPTO id; the remaining 48 parquet files validate in parallel at file + row-group level.
• Local validation of all 49 parquet files (n_jobs=8 on an 8-CPU machine) completed in 19m58s with zero errors.

Tag bump (commit e0523b6)

• ORD_SCHEMA_TAG v0.6.1 → v0.6.3. Required by the new validate_parquet job: row-group parallelism in validate_dataset.py landed in v0.6.2 (Parallelize parquet dataset validation over row groups ord-schema#812). v0.6.3 is the latest release.

HF mirror scope (commit ff91745)

• scripts/upload_to_huggingface.py and huggingface_mirror.yml widened to mirror data/, .gitattributes, README.md, LICENSE, CITATION.cff, CONTRIBUTING.md, CONTRIBUTORS.md — matches what already exists on the HF dataset. GitHub-side infrastructure (.github/, scripts/, badges/) stays excluded.
• One-time bootstrap already done: .gitattributes was pushed to the HF dataset out-of-band so future parquet uploads preupload as LFS regardless of size. (Without this, small singleton parquets would have landed as regular blobs.)

Test plan

• validate_pb (9 shards) all pass.
• validate_parquet (uspto) passes.
• validate_parquet (other) passes.
• process_submission validates the 49 added parquet files cleanly.
• huggingface_mirror dry-run shows 49 parquet uploads, 0 deletions.

Notes for reviewers

• This PR will trigger 11 LFS-heavy CI checkouts (9 pb shards + 2 parquet shards) plus process_submission and the HF mirror dry-run.
• validate_pb is renamed → required-checks in branch protection (if any) will need updating after merge.

🤖 Generated with Claude Code

[bdeadman]

Git LFS usage in current period is at 207.3 GB on 13th May.

Our monthly LFS bandwidth budget is 250 GB (according to the billing portal) but in March we used 554 GB, and in April it was 354 GB so we may have additional allowance that isn't shown in the portal. The meter resets in 19 days so if we are worried about bandwidth we can delay until the end of the month and make a decision to used up the remaining May bandwidth, or get this job in first at the start of June.