Skip to content

feat: add SCROLL statement for pagination support in QQL#26

Merged
pavanjava merged 1 commit into
pavanjava:mainfrom
srimon12:feat/scroll-pagination
May 13, 2026
Merged

feat: add SCROLL statement for pagination support in QQL#26
pavanjava merged 1 commit into
pavanjava:mainfrom
srimon12:feat/scroll-pagination

Conversation

@srimon12
Copy link
Copy Markdown
Collaborator

Summary

This PR adds first-class pagination support to QQL via a new SCROLL statement.

Supported syntax:

SCROLL FROM docs LIMIT 50
SCROLL FROM docs WHERE year >= 2024 LIMIT 50
SCROLL FROM docs AFTER 'cursor-id' LIMIT 50
SCROLL FROM docs WHERE year >= 2024 AFTER 42 LIMIT 50

This closes a usability gap for browsing large collections and paginated exports using SDK-native scroll behavior.

Why

  • Qdrant SDK already exposes scroll(...)
  • QQL already uses scroll(...) internally in dumper flows
  • Users need a direct query-level pagination primitive for browsing and operational workflows

Changes

Language / Parsing

  • Added SCROLL as a top-level statement
  • Added AFTER clause support for cursor-based pagination
  • Added ScrollStmt AST node with:
    • collection
    • limit
    • optional query_filter
    • optional after cursor (string or integer)

Execution

  • Added _execute_scroll(...) in executor using QdrantClient.scroll(...)
  • Supports:
    • optional filter (scroll_filter)
    • optional cursor (offset)
    • payload-only fetch (with_payload=True, with_vectors=False)
  • Returns structured result:
    • points: list of {id, payload}
    • next_offset: cursor for the next page (or None)

CLI / Script

  • Added SCROLL help text in CLI command reference
  • Added pretty-print output for SCROLL points + next_offset
  • Added SCROLL to script statement splitting so .qql scripts can execute SCROLL statements cleanly

Documentation

Updated docs and examples to include SCROLL:

  • README quick syntax + feature summary
  • search docs (SEARCH, SCROLL, RECOMMEND...)
  • programmatic usage result shape
  • getting started examples
  • reference error table
  • docs homepage card text

Tests

Added/updated tests for:

  • lexer: SCROLL/AFTER tokenization
  • parser:
    • basic SCROLL
    • SCROLL + WHERE
    • SCROLL + AFTER
    • SCROLL + WHERE + AFTER
  • executor:
    • successful scroll result shape
    • forwarding filter and cursor
    • missing collection error
  • script splitter:
    • SCROLL recognized as top-level statement boundary

Validation

  • Focused test suites passed
  • Full suite passed:
pytest -q
# 409 passed

@srimon12 srimon12 requested a review from pavanjava May 13, 2026 08:38
@pavanjava pavanjava merged commit 572924f into pavanjava:main May 13, 2026
2 checks passed
@srimon12 srimon12 deleted the feat/scroll-pagination branch May 13, 2026 15:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants