Skip to content

START_HERE.md

Latest commit

 

History

History
339 lines (255 loc) · 9.62 KB

START_HERE.md

File metadata and controls

339 lines (255 loc) · 9.62 KB

START HERE - Documentation Index

Last Updated: November 1, 2025 Project: Product Trend Automation System Status: ALL SYSTEMS OPERATIONAL

Documentation Navigation

For Next Developer - Read in This Order:

1. PROJECT_STATE.md - START HERE FIRST

Read Time: 15-20 minutes Purpose: Complete system state, architecture, all changes, testing results

This is the MASTER document. It contains:

  • Current system state (services, database, schedules)
  • Complete architecture diagrams
  • All changes made this session
  • Testing results with commands
  • Troubleshooting guide
  • File structure
  • API configuration
  • Next steps

READ THIS FIRST to understand complete project state

2. QUICK_REFERENCE.md

Read Time: 5 minutes Purpose: Essential commands and quick operations

Contains:

  • Quick start commands (30 seconds)
  • Service management commands
  • Database queries
  • Manual task triggers
  • Health check commands
  • Common troubleshooting

Use this as your command cheat sheet

3. SYSTEM_ARCHITECTURE.md

Read Time: 10 minutes Purpose: Visual architecture and system design

Contains:

  • System architecture diagrams
  • Data flow visualizations
  • Component relationships
  • Database schema
  • Performance metrics

Read this for architectural understanding

4. AGENTIC_AI_SETUP.md

Read Time: 15 minutes Purpose: AI agent system details

Contains:

  • AI agent architecture
  • Agent roles and responsibilities
  • Enhanced agent instructions
  • Testing procedures
  • Performance metrics

Read this for deep dive on AI agents

Quick Start for Next Session

If you need to start working immediately (5 minutes):

# 1. Navigate to project
cd C:\Users\timud\Documents\product-trend-automation

# 2. Verify services running
docker ps --filter "name=product-trend"
# Should show 6 containers: frontend, backend, celery, celery-beat, db, redis

# 3. Check system health
docker logs product-trend-celery --tail 50
docker logs product-trend-backend --tail 50

# 4. Verify feedback loop data
docker exec product-trend-backend python -c "
from models.database import SessionLocal, TrendingKeyword
db = SessionLocal()
count = db.query(TrendingKeyword).count()
print(f'[COMPLETE] Keywords in database: {count}')
db.close()
"

# 5. Access the system
# Frontend: http://localhost:3000
# Backend: http://localhost:8000
# API Docs: http://localhost:8000/docs

# 6. You're ready! Read PROJECT_STATE.md for full context

All Documentation Files

Core Documentation

  • START_HERE.md - You are here (navigation guide)
  • PROJECT_STATE.md - Master state document
  • README.md - Project overview
  • QUICK_REFERENCE.md - Command reference
  • DOCUMENTATION_INDEX.md - Complete file index

Technical Documentation

  • SYSTEM_ARCHITECTURE.md - Architecture details
  • AGENTIC_AI_SETUP.md - AI agent architecture
  • AGENT_ROLES_STRUCTURE.md - Agent roles and responsibilities
  • MONITOR_AUTONOMOUS_AGENTS.md - Monitoring guide
  • RESTART_GUIDE.md - Service restart procedures

Configuration Files

  • .env - API keys and environment variables
  • docker-compose.yml - Service orchestration
  • backend/tasks/celery_app.py - Task schedules

What This System Does

Product Trend Automation - AI-powered platform that:

  1. Discovers trending products from 7 sources (Amazon, TikTok, Reddit, etc.)
  2. Analyzes products using 12 AI agents (11 Groq + 1 Perplexity)
  3. Learns from discoveries via Perplexity feedback loop
  4. Recommends products with APPROVE/REVIEW/REJECT decisions
  5. Auto-posts to marketplaces (requires manual approval)

Key Innovation: Self-improving feedback loop where Perplexity discovers trending keywords from the web, stores them in the database, and TrendScanner uses them to prioritize future product searches.

Current System State (Snapshot)

Services Status

[RUNNING] 6 Docker containers running
[OPERATIONAL] 12 AI agents operational (enhanced with detailed instructions)
[ACTIVE] Perplexity feedback loop active
[STORED] 28 trending keywords discovered and stored
[DATABASE] 399 products in database
[SCHEDULED] All scheduled tasks running (1hr, 6hr, 15min intervals)

Recent Changes (Nov 1, 2025)

  • [COMPLETE] Added Perplexity as 12th agent
  • [COMPLETE] Created feedback loop (Perplexity → DB → TrendScanner)
  • [COMPLETE] Enhanced all agents with ultra-detailed instructions
  • [COMPLETE] Added TrendingKeyword database table
  • [COMPLETE] Scheduled automatic discovery every 6 hours
  • [COMPLETE] Tested end-to-end (all systems passing)

What's Next

  1. Monitor Perplexity automatic discovery (runs every 6 hours)
  2. Build frontend product review dashboard
  3. Configure platform APIs (Amazon, eBay, TikTok)
  4. Implement keyword-based scoring boost
  5. Create analytics dashboard

Essential Information

API Keys (Active)

GROQ_API_KEY=your_groq_api_key_here
PERPLEXITY_API_KEY=your_perplexity_api_key_here
HUGGINGFACE_API_KEY=your_huggingface_api_key_here

Ports

Database

  • Type: PostgreSQL
  • URL: postgresql://postgres:postgres@localhost:5432/product_trends
  • Tables: products, trending_keywords, trend_sources, platform_listings, audit_logs

Technology Stack

  • Frontend: React + TypeScript
  • Backend: FastAPI + Python
  • Workers: Celery + Redis
  • Database: PostgreSQL
  • AI: 11 Groq agents + 1 Perplexity agent
  • Containerization: Docker + Docker Compose

Project Structure

product-trend-automation/
├── START_HERE.md              ← You are here
├── PROJECT_STATE.md           ← Read this first
├── QUICK_REFERENCE.md         ← Command cheat sheet
├── README.md                  ← Project overview
├── DOCUMENTATION_INDEX.md     ← Complete file index
│
├── .env                       ← API keys
├── docker-compose.yml         ← Services
│
├── backend/
│   ├── main.py
│   ├── models/database.py     ← Database models
│   ├── services/
│   │   ├── ai_analysis/agentic_system.py     ← 12 AI agents
│   │   └── trend_discovery/
│   │       ├── trend_scanner.py              ← Trend scanning
│   │       └── perplexity_discovery.py       ← Perplexity integration
│   └── tasks/
│       ├── celery_app.py      ← Task schedules
│       ├── trend_tasks.py     ← Trend scanning tasks
│       └── analysis_tasks.py  ← AI analysis tasks
│
└── frontend/
    └── src/                   ← React components

Verification Checklist

Before starting work, verify:

# [CHECK] All services running?
docker ps --filter "name=product-trend"

# [CHECK] Database accessible?
docker exec product-trend-backend python -c "from models.database import SessionLocal; db = SessionLocal(); print('[SUCCESS] DB Connected'); db.close()"

# [CHECK] Trending keywords stored?
docker exec product-trend-backend python -c "from models.database import SessionLocal, TrendingKeyword; db = SessionLocal(); print(f'[INFO] Keywords: {db.query(TrendingKeyword).count()}'); db.close()"

# [CHECK] Backend healthy?
curl http://localhost:8000/health

# [CHECK] Recent activity?
docker logs product-trend-celery --tail 20

# If all checks pass, you're ready to work!

Learning Path

If you're new to this project:

  1. Start: Read PROJECT_STATE.md (20 min)
  2. Understand: Run verification checklist above (5 min)
  3. Explore: Check QUICK_REFERENCE.md for commands (5 min)
  4. Deep Dive: Read SYSTEM_ARCHITECTURE.md (10 min)
  5. Start Coding: Pick a task from "What's Next" section

If you're continuing development:

  1. Catch Up: Read PROJECT_STATE.md (15 min)
  2. Verify: Run verification checklist (2 min)
  3. Code: Continue from "What's Next" in PROJECT_STATE.md

Important Notes

What's Working

  • [COMPLETE] All 12 AI agents with enhanced instructions
  • [COMPLETE] Perplexity feedback loop (discovery → storage → usage)
  • [COMPLETE] TrendScanner loading keywords from database
  • [COMPLETE] Automated schedules (every 1hr, 6hr, 15min)
  • [COMPLETE] Docker stack stable
  • [COMPLETE] All tests passing

Known Limitations

  • [WARNING] Groq rate limits during heavy testing (expected on free tier)
  • [WARNING] Platform APIs not configured (Amazon, eBay, TikTok)
  • [WARNING] Frontend needs development (product review dashboard)

Critical Files (Don't Modify Without Understanding)

  • backend/services/ai_analysis/agentic_system.py - 12 agent system
  • backend/models/database.py - Database schema
  • backend/tasks/celery_app.py - Task schedules
  • .env - API keys (keep secret!)

Need Help?

Documentation Priority

  1. Quick question? → QUICK_REFERENCE.md
  2. Complete context? → PROJECT_STATE.md
  3. Architecture? → SYSTEM_ARCHITECTURE.md
  4. AI agents? → AGENTIC_AI_SETUP.md

Common Issues

  • Services not starting? → Check PROJECT_STATE.md → Troubleshooting section
  • API errors? → Check .env file has valid keys
  • Database issues? → Run init_db() (see QUICK_REFERENCE.md)

You're Ready!

Everything is documented. Everything is tested. All systems operational.

Next Steps:

  1. Read PROJECT_STATE.md for full context
  2. Run verification checklist
  3. Start development!

Project Location: C:\Users\timud\Documents\product-trend-automation

Happy Coding!

Last Updated: November 1, 2025 Session: Perplexity Integration Complete Status: Production Ready