Big Data Applications

Real-Time Insurance Fraud Detection Pipeline

Overview

End-to-end real-time fraud detection system for insurance claims processing. Combines streaming data ingestion, in-stream ML inference, and low-latency storage for sub-second decision making.

Architecture

Claims Stream → Apache Kafka → PyFlink (XGBoost inference)
                                      ↓
                             Redis (feature cache)
                                      ↓
                            PostgreSQL (audit log)

Key Components

Component	Role
Apache Kafka	Event streaming & ingestion
PyFlink	Stream processing & feature computation
XGBoost	Real-time fraud scoring
Redis	Sub-millisecond feature caching
PostgreSQL	Persistent audit and results storage

---

Maestría en Inteligencia Artificial · Big Data · Universidad Politécnica Metropolitana de Hidalgo