An end-to-end MLOps pipeline for network intrusion detection using Airflow, Docker, and MLflow. CyberFlow automates data ingestion, drift detection, A/B model testing, and deployment, ensuring real-time monitoring and high accuracy.
Dual-Pipeline System
-
One-Time Training Pipeline: Trains the initial model.
-
Weekly Ingestion Pipeline: Handles new data while maintaining 98.89% accuracy.
Automated Drift Detection
-
Weekly drift detection with detailed reports and visualizations.
-
Triggers model retraining if significant drift is detected.
A/B Testing & Deployment
-
Compares multiple models in production.
-
Automatically deploys the best-performing model.
Dashboard for Monitoring
- Real-time performance visualization.
- Model metrics and drift detection reports.
Streamlit Report: View Dashboard
- Python
- Apache Airflow
- Docker
- MLflow
- Streamlit