ShelfSense AI

ShelfSense AI is a visual inventory and retail decision-support project built around computer vision, classical machine learning, and reinforcement learning. The repository now includes a production-style Streamlit dashboard, an OpenCV pipeline explorer, trained models, sample datasets, and a reinforcement learning module for restock recommendations.

What the project can do

• Monitor shelf inventory using structured product and stock data.
• Predict low-stock vs healthy-stock shelf states with a supervised learning model.
• Analyze uploaded images, browser camera captures, or desktop webcam frames.
• Detect product-like regions using an 8-phase OpenCV pipeline.
• Use label-text hints to classify products with a text-based model.
• Fall back to visual clustering for unlabeled or weakly labeled products.
• Rank products for restocking using inventory pressure, sales behavior, and RL signals.
• Show reinforcement learning recommendations and priority insights.
• Suggest weather-aware restocking boosts for temperature-sensitive categories.
• Surface nearby or co-located products from retail basket patterns.
• Simulate and support inventory updates through the RL subsystem.

Main entry points

main.py

Primary Streamlit app for day-to-day demo and analysis. It includes:

• Dashboard
• Shelf Monitoring
• Product Detection
• RL Insights
• Weather Intelligence
• Store Room Finder

Run it with:

streamlit run main.py

app.py

Extended Streamlit interface focused on the OpenCV pipeline walkthrough, model integration status, camera controls, stock thresholds, and detailed phase-by-phase computer vision outputs.

Run it with:

streamlit run app.py

Project structure

visual_inventory_system/
|-- app.py                  # OpenCV pipeline and integrated visual inventory demo
|-- main.py                 # Main Streamlit dashboard
|-- opencv_pipeline.py      # 8-phase image-processing pipeline
|-- shelfsense.html         # Static HTML asset/demo page
|-- requirements.txt        # Python dependencies
|-- bg.jpg                  # UI/background asset
|
|-- camera/
|   `-- capture.py          # Webcam and frame capture helpers
|
|-- data/
|   |-- product_labels.csv  # Label-text training data
|   |-- product_profiles.csv# Visual/profile feature data
|   |-- retail_dataset.csv  # Retail transactions for analytics
|   `-- sample_data.csv     # Inventory/sample shelf data
|
|-- models/
|   |-- supervised_model.pkl
|   |-- clustering_model.pkl
|   |-- label_text_model.pkl
|   |-- rf_model.pkl
|   |-- scaler.pkl
|   `-- vectorizer.pkl
|
|-- utils/
|   |-- data_loader.py      # Dataset loading and feature preparation
|   |-- image_analyzer.py   # Image-to-feature analysis helpers
|   `-- model_loader.py     # Model loading, training, and prediction helpers
|
|-- rl/
|   `-- rl_model.py         # Priority scoring and RL-driven ranking helpers
|
|-- part3_rl/
|   |-- shelfsense_api.py   # RL API layer for restock recommendations
|   |-- train_rl.py         # RL training entry point
|   |-- environment.py      # RL environment
|   |-- q_agent.py          # Q-learning agent
|   |-- inventory_manager.py
|   |-- decay_predictor.py
|   |-- inventory.csv
|   |-- q_table.npy
|   |-- state_visits.npy
|   |-- restock_log.json
|   `-- README.md
|
`-- notebooks/
    |-- supervised.ipynb
    |-- unsupervised.ipynb
    `-- unsupervised_retail.ipynb

ML and analytics components

Supervised learning

• Uses inventory and stock-derived features to predict shelf status.
• utils/model_loader.py trains or loads supervised_model.pkl.

Label-text classification

• Uses TF-IDF plus engineered text features to predict a product category from label hints.
• Useful when a filename or OCR-like text clue is available.

Unsupervised visual clustering

• Uses product profile features with scaling, PCA, and K-Means.
• Helps identify likely product families when labels are weak or unavailable.

Reinforcement learning

• The part3_rl/ module manages restock recommendations with a Q-learning workflow.
• rl/rl_model.py also blends RL outputs with stock pressure and sales signals to build priority rankings.

OpenCV pipeline

The visual pipeline in opencv_pipeline.py includes:

1. Read and resize
2. Color-space conversion
3. Image blurring
4. Thresholding and edge detection
5. Morphology and contour detection
6. ROI extraction
7. Color filtering and dominant-color features
8. Histogram generation

Data used in the project

• data/sample_data.csv: inventory-style shelf dataset used by the dashboard and supervised model flow.
• data/product_labels.csv: text samples for label/category prediction.
• data/product_profiles.csv: structured physical and color features for clustering.
• data/retail_dataset.csv: retail history used for product analytics, basket patterns, and time-based insights.
• part3_rl/inventory.csv: RL-specific inventory state used by the reinforcement learning module.

Setup

1. Create a virtual environment

python -m venv venv
venv\Scripts\activate

2. Install dependencies

pip install -r requirements.txt

3. Run the app

Use one of the following:

streamlit run main.py

streamlit run app.py

Notes

• The repo already contains trained model artifacts inside models/.
• main.py is the best starting point for most demos.
• app.py is useful when you want to inspect the OpenCV stages in detail.
• Weather intelligence in main.py uses an online API when available and falls back gracefully if it is not.
• Some optional integrations referenced in app.py are loaded defensively, so missing modules do not always break the whole app.

Contributors

@ssktech3geek @anuj122511 @dubey00sujal

License

This project was built as part of a hackathon and internal project work. Add a formal license here if you plan to distribute it publicly.