Agentic RAG: What should I wear? - Personal Wardrobe Assistant (Demo Project)

This repository contains code for Agentic RAG that helps users choose garments from their wardrobe to create a complete outfit according to their requests, leveraging both knowledge of an LLM itself and its ability to use tools to retrieve the necessary information.

This project was developed to gain hands-on experience in building Agentic RAG using modern AI frameworks and state-of-the-art Vision and Large Language Models, and to demonstrate its application to solve a real-world problem.

How It Works

1. Vector Store: The user places all images of garments they plan to use in a special folder. They then create a vector store representing their wardrobe and containing indexed descriptions of these images. These descriptions are generated by a Vision Language Model (VLM) according to the provided instruction (Image-Text-to-Text task).

2. User Information: The user provides brief information about themselves, which is saved in store.

3. Initial User Message: The user asks a question, such as "What should I wear today?"

4. Agent: The agent begins working according to its graph:

   

   The Large Language Model (LLM) analyzes the question and decides whether and which tools to use. The final answer is displayed to the user along with images of the garments, if any were selected.

5. New User Message (Optional): If desired, the user can continue the conversation, as the agent remembers the conversation.

Dataset Used

To create the user's wardrobe, 100 images were selected from the Clothing dataset (full, high resolution): images_compressed so that the subsample contained garments of different types. A list of the images used can be found in the data/used_images.txt file.

Technologies Used

• Language: Python 3.12
• Core Frameworks:
  • LangChain: Loading documents, connecting to the vector store as a retriever, defining tools, and interacting with chat models.
  • LangGraph: Building the stateful agent with memoization (short-term and long-term memory).
• Model Provider: Hugging Face
  • VLM (local): HuggingFaceTB/SmolVLM-256M-Instruct
  • Text Embedding Model (local): sentence-transformers/all-mpnet-base-v2
  • LLM (via API): Qwen/Qwen3-235B-A22B-Instruct-2507 or Qwen/Qwen3-4B-Thinking-2507
• Vector Store: Chroma
• Tools:
  • Garment Description Retriever (local): the vector store as the retriever.
  • Weather Forecast (via API): OpenWeatherMap
  • Web Search (via API): Tavily
• Other Key Platforms and Libraries:
  • Google Colab: Developing the notebook in a browser-based Jupyter Notebook environment.
  • Gradio: Interacting with the agent via the UI application.
  • LangSmith: Tracing the agent behavior.

Project Structure

.
├── agent/
|   ├── components/
|   |   ├── __init__.py
|   |   ├── context_memory.py    # Agent graph context schema, state, store, and checkpointer
|   |   ├── nodes_edges.py       # LLM, graph node and conditional edge functions
|   |   ├── prompts.py           # Prompts for the VLM and LLM
|   |   ├── tools.py             # Tool functions
|   |   └── vector_store.py      # VLM, Text Embedding Model, image description generation, and vector store
|   ├── utils/
|   |   ├── __init__.py
|   |   └── utils.py             # Helper functions
|   ├── __init__.py
|   ├── graph.py                 # Agentic RAG graph built using LangGraph
|   └── requirements.txt         # Graph dependencies
├── data/
|   ├── garment_images           # Images of garments whose descriptions are used to create the vector store
|   ├── wardrobe_db/             # Persist directory for the vector store
|   |   ├── ***                  # UUID collection name
|   |   └── chroma.sqlite3       # Vector DB itself representing a user's wardrobe
|   └── used_images.txt          # Images that were used in this project
├── demo/
|   ├── app.py                   # Gradio UI app for step-by-step data preparation and agent interaction
|   └── requirements.txt         # UI application dependencies
├── notebooks/
|   └── Agentic_RAG_Graph_(What_Should_I_Wear).ipynb    # Jupyter Notebook for running in Google Colab
├── static/
|   └── agent_graph.png          # Agent graph visualization
├── tests
├── .env                         # Environment variables
├── .env.example                 # Example .env file
├── .flake8
├── .gitignore
├── .pre-commit-config.yaml
├── .pytest.ini
├── dev-requirements.txt         # Project dependencies
├── langgraph.json               # LangGraph application configurations
├── LICENSE
└── README.md

Getting Started

Follow these steps to set up and run the demo project:

1. Prerequisites

• Python 3.12
• Images of garments in the data/garment_images folder
• An API token for Hugging Face Hub
• An API key for OpenWeatherMap
• An API key for Tavily
• (Optional) An API key for LangSmith

2. Running the Agentic RAG

You can interact with the agent in three ways:

A. Step-by-Step Notebook

To explore the code in detail:

1. Copy the data folder and the notebook Agentic_RAG_Graph_(What_Should_I_Wear).ipynb to Google Drive.
2. Open the notebook in Google Colab and fill in Colab Secrets with your API keys and tokens (their names correspond to environment variables).
3. Run the notebook.

B. Interactive Gradio Application

To walk through the Agentic RAG process:

1. Clone the repository, create and activate a virtual environment, and install the required dependencies from demo/requirements.txt.
2. Create a .env file similar to the .env.example ​with your API keys and tokens.
3. Execute the demo/app.py file directly to launch the app.
4. Follow the app's steps through the entire process, from creating a vector store of your wardrobe and providing information about yourself to chatting with your assistant.

Note: Generating image descriptions, and therefore creating the vector store, takes time! You can skip first step in the app if you already have one.

C. Python Scripts

To create the vector store and run the predefined agent:

1. Clone the repository, create and activate a virtual environment, and install the required dependencies from agent/requirements.txt.
2. Create a .env file similar to the .env.example ​with your API keys and tokens.
3. Execute the agent/components/vector_store.py file directly to create a vector store of your wardrobe. Note: Generating image descriptions, and therefore creating the vector store, takes time! You can skip this step if you already have one.
4. Execute the agent/graph.py file directly to run the agent graph with the user message already defined.