Scrappy

Scrappy is a Python project that collects NEPSE market and IPO data in a clean, ML-friendly format for downstream analysis and automation. like -> https://ipo-alert-red.vercel.app/

What it collects

• Market snapshots to CSV:
  • data/nepse/YYYY/MM/market_summary_YYYY-MM-DD.csv
  • data/nepse/YYYY/MM/today_price_YYYY-MM-DD.csv
• IPO feed to JSON:
  • data/ipo/ipo_feed.json

data

https://raw.githubusercontent.com/prabinbessie/scrappy/dev/data/ipo/ipo_feed.json

Sources

• NEPSE : https://www.nepalstock.com/
• Merolagani : https://merolagani.com/
• ShareSansar: https://www.sharesansar.com/
• NepseLink : https://nepselink.com/ipo-opening

Setup

Create and activate a virtual environment:

python3 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install -e ".[dev]"

Run jobs

# market only
python -m scraper.cli market

# ipo only
python -m scraper.cli ipo

# both
python -m scraper.cli all

Dev workflow

source .venv/bin/activate && python -m pip install -e ".[dev]" && pre-commit install && pre-commit run --all-files

Install and enable git hooks for pre-commit checks:

pre-commit install
pre-commit run --all-files

After pre-commit install, every commit will automatically run lint, formatting, and tests (pytest -q tests).

Manual quality checks:

ruff check .
black --check .
pytest -q tests