docs: add LSQB benchmark tutorial and example scripts

[longbinlai]

Summary

This PR adds comprehensive documentation and scripts for reproducing the LSQB (Labelled Subgraph Query Benchmark) performance results.

• Add tutorial documentation at doc/source/tutorials/lsqb_benchmark.rst
• Add benchmark scripts at examples/lsqb_benchmark/
• Include data loading, query execution, and result reporting
• Provide dataset download link and reproducibility instructions

Changes

• doc/source/index.rst - Add link to new tutorial
• doc/source/tutorials/lsqb_benchmark.rst - New tutorial document
• examples/lsqb_benchmark/run_neug_benchmark.py - Main benchmark script
• examples/lsqb_benchmark/README.md - Usage instructions
• examples/lsqb_benchmark/requirements.txt - Python dependencies

Dataset

The LDBC SNB SF1 dataset used in this benchmark is available at:
https://neug.oss-cn-hangzhou.aliyuncs.com/datasets/ldbc-snb-sf1-lsqb.tar.gz

Test Plan

# Download dataset
wget https://neug.oss-cn-hangzhou.aliyuncs.com/datasets/ldbc-snb-sf1-lsqb.tar.gz
tar -xzf ldbc-snb-sf1-lsqb.tar.gz

# Install dependencies
pip install neug

# Run benchmark
python examples/lsqb_benchmark/run_neug_benchmark.py --data-dir social_network-sf1-CsvComposite-StringDateFormatter

🤖 Generated with Claude Code

[greptile-apps]

Your free trial has ended. If you'd like to continue receiving code reviews, you can add a payment method here.

[qodo-code-review]

Review Summary by Qodo

Add LSQB benchmark tutorial and example scripts for NeuG

📝 Documentation ✨ Enhancement

Walkthroughs

Description

• Add comprehensive LSQB benchmark tutorial with 9 complex subgraph queries
• Implement complete benchmark script for LDBC SNB SF1 dataset loading and execution
• Include CSV preprocessing utilities for data schema compatibility
• Provide reproducibility instructions and expected performance results

Diagram

flowchart LR
  A["LDBC SNB SF1<br/>Dataset"] -->|"CSV Preprocessing"| B["Derived CSVs<br/>with Headers"]
  B -->|"COPY Statements"| C["NeuG Database<br/>Schema & Data"]
  C -->|"9 LSQB Queries"| D["Benchmark Results<br/>JSON Report"]
  E["Tutorial Doc"] -->|"References"| F["Benchmark Scripts"]
  F -->|"Executes"| D

Loading

File Changes

1. examples/lsqb_benchmark/run_neug_benchmark.py ✨ Enhancement +472/-0

Main LSQB benchmark script with data loading

• Implements complete LSQB benchmark script with 9 complex subgraph matching queries
• Provides CSV preprocessing functions to handle LDBC SNB SF1 data format
• Includes data loading pipeline with schema creation and COPY statements
• Implements query execution with warmup runs and statistical result reporting

examples/lsqb_benchmark/run_neug_benchmark.py

---

2. doc/source/tutorials/lsqb_benchmark.rst 📝 Documentation +242/-0

LSQB benchmark tutorial documentation

• Comprehensive tutorial documenting LSQB benchmark setup and execution
• Includes dataset download instructions and directory structure overview
• Provides simplified code examples and expected performance results table
• Documents reproducibility instructions and comparison methodology

doc/source/tutorials/lsqb_benchmark.rst

---

3. doc/source/index.rst 📝 Documentation +1/-0

Add LSQB tutorial to documentation index

• Adds reference to new LSQB benchmark tutorial in documentation index
• Integrates tutorial into Tutorials section of main documentation

doc/source/index.rst

---

View more (2)

4. examples/lsqb_benchmark/README.md 📝 Documentation +74/-0

Benchmark directory README with quick start

• Provides quick start guide for running the benchmark
• Documents dataset information and expected results table
• Lists query descriptions and file organization
• Includes references to LSQB and LDBC SNB benchmarks

examples/lsqb_benchmark/README.md

---

5. examples/lsqb_benchmark/requirements.txt Dependencies +1/-0

Python dependencies for benchmark

• Specifies NeuG package dependency for benchmark execution

examples/lsqb_benchmark/requirements.txt

---

[qodo-code-review]

Code Review by Qodo

🐞 Bugs (1)   📘 Rule violations (0)   📎 Requirement gaps (0)   🎨 UX Issues (0)

🐞\ ≡ Correctness (1)

1. Unsafe database deletion ☑ 🐞 ⛨

Description

In load_data(), any existing path provided via --db-path is unconditionally deleted with
shutil.rmtree, so a typo (e.g., ".", a parent directory, or another important folder) can cause
irreversible data loss. This is especially risky because --db-path is user-controlled CLI input and
there is no validation/confirmation.

Code

examples/lsqb_benchmark/run_neug_benchmark.py[R273-276]

+    # Clean up existing database
+    if db_path.exists():
+        print(f"  Removing existing database...")
+        shutil.rmtree(db_path)

Evidence

The script checks only that db_path exists and then recursively deletes it, without ensuring it is a
NeuG database directory or otherwise safe to remove.

examples/lsqb_benchmark/run_neug_benchmark.py[265-277]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`load_data()` uses `shutil.rmtree(db_path)` on any existing `--db-path`, which can delete arbitrary directories if the user passes an unsafe path by mistake.
## Issue Context
This is a benchmark/example script, but it still runs destructive filesystem operations based on user input. It should refuse to delete obviously-dangerous paths and ideally require an explicit confirmation/flag.
## Fix Focus Areas
- examples/lsqb_benchmark/run_neug_benchmark.py[265-277]
## Suggested changes
- Add a `--force` (or `--overwrite-db`) flag; only delete when explicitly provided.
- Add safety checks before deletion, e.g.:
- resolve to absolute path and reject if it is `/`, the user home dir, the current working directory, or a parent of the repo.
- ensure the target looks like a NeuG DB directory (e.g., contains expected metadata/files) before removal.
- If `db_path` exists but is a file, handle with `unlink()` (or fail with a clear error) instead of calling `rmtree()`.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

2. Tutorial run command wrong 🐞 ≡

Description

The tutorial tells users to run python run_lsqb_benchmark.py, but the repository provides
examples/lsqb_benchmark/run_neug_benchmark.py and it requires --data-dir. Following the tutorial
as written will fail unless users create a new file themselves and/or guess the required CLI
arguments.

Code

doc/source/tutorials/lsqb_benchmark.rst[R191-193]

+   # 3. Run the benchmark
+   python run_lsqb_benchmark.py
+

Evidence

The tutorial’s run command references a script name not present in the repo, while the actual
benchmark README and script indicate run_neug_benchmark.py and require a --data-dir argument.

doc/source/tutorials/lsqb_benchmark.rst[52-54]
doc/source/tutorials/lsqb_benchmark.rst[191-193]
examples/lsqb_benchmark/README.md[20-32]
examples/lsqb_benchmark/run_neug_benchmark.py[427-439]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The new tutorial instructs running `python run_lsqb_benchmark.py`, but the repo ships `examples/lsqb_benchmark/run_neug_benchmark.py` which requires `--data-dir`. The tutorial currently leads to a broken reproduction flow.
## Issue Context
The tutorial already references `examples/lsqb_benchmark/` as the location of the complete script, so the run instructions should match that script and its CLI.
## Fix Focus Areas
- doc/source/tutorials/lsqb_benchmark.rst[52-54]
- doc/source/tutorials/lsqb_benchmark.rst[191-193]
- examples/lsqb_benchmark/run_neug_benchmark.py[427-439]
## Suggested changes
- Update the tutorial caption/script name to `run_neug_benchmark.py` (or clearly label the snippet as pseudocode and remove the direct run command for the non-existent file).
- Replace `python run_lsqb_benchmark.py` with the real invocation, e.g.:
- `python examples/lsqb_benchmark/run_neug_benchmark.py --data-dir social_network-sf1-CsvComposite-StringDateFormatter`
- or instruct users to `cd examples/lsqb_benchmark` and then run `python run_neug_benchmark.py --data-dir ...`.
- Optionally point to the README as the authoritative command reference.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

3. Derived CSV path mismatch ☑ 🐞 ☼

Description

preprocess_csvs() records a derived CSV path in the mapping even when the source CSV is missing, and
get_copy_statements() will then prefer the derived path solely based on key membership. This can
generate COPY statements pointing at non-existent derived files and cause data loading to fail on
dataset layout variations.

Code

examples/lsqb_benchmark/run_neug_benchmark.py[R161-168]

+    for relpath, new_header in DEDUP_CSVS.items():
+        out = derived_dir / Path(relpath).name
+        mapping[relpath] = out
+        if out.exists():
+            continue
+        src = data_dir / relpath
+        if not src.exists():
+            continue

Evidence

The mapping is populated before verifying src.exists(), so missing source files still get an entry
in dedup_map. Later, get_copy_statements() chooses the derived path whenever `relpath in
dedup_map`, without checking that the derived output file actually exists.

examples/lsqb_benchmark/run_neug_benchmark.py[156-180]
examples/lsqb_benchmark/run_neug_benchmark.py[221-229]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`preprocess_csvs()` adds `mapping[relpath] = out` before checking whether the source CSV exists. `get_copy_statements()` then routes COPY to the derived path whenever the key is present, even if the derived file was never created.
## Issue Context
This fails loudly at COPY time (file-not-found), but it is avoidable and makes the script brittle to dataset variations.
## Fix Focus Areas
- examples/lsqb_benchmark/run_neug_benchmark.py[156-180]
- examples/lsqb_benchmark/run_neug_benchmark.py[221-229]
## Suggested changes
- Populate `mapping[relpath] = out` only after confirming `src.exists()` (and ideally after successfully writing `out`).
- Alternatively (or additionally), in `f(relpath)` prefer the derived path only if `dedup_map[relpath].exists()`; otherwise fall back to `data_dir/relpath`.
- Consider logging a warning when an expected CSV is missing so users get immediate, actionable context.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

4. Zero shown as N/A ☑ 🐞 ≡

Description

print_results() formats the result column using r.result or 'N/A', so valid results of 0 are
rendered as N/A. This contradicts the documented expected output where Q3 is 0 and can mislead users
into thinking the query failed.

Code

examples/lsqb_benchmark/run_neug_benchmark.py[R393-395]

+    for r in results:
+        status = "OK" if r.ok else "FAIL"
+        print(f"| Q{r.query_id}    | {r.elapsed_ms:8.2f} | {r.result or 'N/A':>12} | {status} |")

Evidence

The code uses a truthiness check (or) rather than a None check, and the docs explicitly expect a 0
result for Q3.

examples/lsqb_benchmark/run_neug_benchmark.py[386-396]
examples/lsqb_benchmark/README.md[34-48]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`print_results()` uses `r.result or 'N/A'`, which prints `'N/A'` when `r.result == 0`.
## Issue Context
The benchmark includes queries that can legitimately return 0 (the docs list Q3 as 0), so the summary table becomes misleading.
## Fix Focus Areas
- examples/lsqb_benchmark/run_neug_benchmark.py[386-396]
## Suggested changes
- Change formatting to check for None explicitly, e.g.:
- `result_str = 'N/A' if r.result is None else str(r.result)`
- or inline: `{('N/A' if r.result is None else r.result):>12}` (ensure it’s a string for alignment).
- Keep `QueryResult.ok` semantics unchanged (0 is a valid result).

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

ⓘ The new review experience is currently in Beta. Learn more

[liulx20]

LGTM

[longbinlai]

这里不是 完整的 LDBC SNB Interactive Benchmark，只是拿了其中的 complex read 的workload 做评测。完整的 LDBC SNB Interactive Benchmark 我们会在后续推出

[longbinlai]

为什么上面叫 run_interactive_benchmark, 底下 叫 run_neug_benchmark? 要么 就叫 run_lsqb_benchmark

[liulx20]

There is a difference compared to LSQB Q2 provided by LDBC: https://github.com/ldbc/lsqb/blob/main/cypher/q2.cypher

[liulx20]

https://github.com/ldbc/lsqb/blob/main/cypher/q6.cypher

[liulx20]

same as q2 https://github.com/ldbc/lsqb/blob/main/cypher/q3.cypher

[liulx20]

https://github.com/ldbc/lsqb/blob/main/cypher/q9.cypher

[longbinlai]

@liulx20 I have added the notes according to your comments.

[liulx20]

LGTM