Real-Time E-Commerce Analytics System

Real-time analytics platform for e-commerce order processing using Apache Kafka, ksqlDB, and stream processing.

Project Overview

This system processes e-commerce order events in real-time, providing instant analytics on sales performance, product trends, and customer behavior.

Features

• Real-time order event processing
• Sales analytics by category, location, and product
• Trending product detection with time windows
• Windowed aggregations (5-minute, hourly)
• Multi-dimensional analytics (10 products, 5 locations, 4 categories)

Architecture

E-Commerce Generator (Python)
    ↓
Kafka Topic: ecommerce-events
    ↓
ksqlDB Stream Processing
    ↓
    ├→ Sales Metrics Table
    ├→ Category Sales Table
    ├→ Location Sales Table
    ├→ Product Sales Table
    └→ Trending Products Table (windowed)

Tech Stack

• Apache Kafka 7.5.0
• ksqlDB 0.29.0
• Python 3.8+
• Docker Compose

Quick Start

1. Create Kafka Topic

docker exec -it kafka bash
cd /bin

kafka-topics --create \
  --bootstrap-server localhost:9092 \
  --topic ecommerce-events \
  --partitions 3 \
  --replication-factor 1

exit

2. Generate Events

pip install kafka-python
python src/ecommerce_generator.py

3. Create Analytics in ksqlDB

docker exec -it ksqldb-cli ksql http://ksqldb-server:8088

CREATE STREAM ecommerce_stream (
    event_type VARCHAR,
    order_id VARCHAR,
    customer_id VARCHAR,
    product_id VARCHAR,
    product_name VARCHAR,
    category VARCHAR,
    quantity INT,
    unit_price DOUBLE,
    total_amount DOUBLE,
    location VARCHAR,
    payment_method VARCHAR,
    timestamp VARCHAR
) WITH (
    KAFKA_TOPIC='ecommerce-events',
    VALUE_FORMAT='JSON'
);

CREATE TABLE sales_metrics AS
    SELECT 
        'TOTAL' as metric_key,
        COUNT(*) as order_count,
        SUM(total_amount) as total_revenue,
        AVG(total_amount) as avg_order_value
    FROM ecommerce_stream
    WHERE event_type = 'order'
    GROUP BY 'TOTAL'
    EMIT CHANGES;

Product Catalog

• iPhone 15 Pro (Electronics) - N450,000
• Samsung Galaxy S24 (Electronics) - N380,000
• Nike Air Max (Fashion) - N45,000
• Adidas Ultraboost (Fashion) - N52,000
• HP Laptop (Electronics) - N280,000
• Dell Monitor (Electronics) - N85,000
• Sony Headphones (Electronics) - N25,000
• Kitchen Blender (Home) - N15,000
• Coffee Maker (Home) - N28,000
• Study Desk (Furniture) - N35,000

Project Structure

ecommerce-analytics/
├── src/
│   └── ecommerce_generator.py
├── .gitignore
└── README.md

License

MIT License - See LICENSE file for details

Author

Conrad Mba - Software Engineer | Data Engineering & Real-Time Systems | Kafka Specialist

• LinkedIn: [https://www.linkedin.com/in/conrad-mba/]
• Email: [mbaconrad@gmail.com]
• GitHub: [https://github.com/ezechimere]