Easy open source monitoring for ML models.
Agents for monitoring Python webservers - e.g Flask, Django, FastAPI running ML models.
MetricRule agents are designed to be deployed with a serving model endpoint to generate input feature and output distribution metrics based on the model's endpoint usage. Integrations with Python WSGI and ASGI servers, like Django, Flask and FastAPI are supported.
The motivation of this project is to make it easier to monitor feature distributions in production to better catch real world ML issues like training-serving skew, feature drifts, poor model performance on specific slices of input.
Please see the example directory to see usages.
Please refer to metricrule.com for more information.