I've completed the review of the third MLOPS Zoomcamp project organized by DataTalksClub. This project is quite interesting and comprehensive as it utilizes several technologies such as MLflow and Airflow, in addition to deploying the model on the cloud provider AWS.
#MLOpsZoomcamp #DataTalksClub
I've started reviewing the third MLOPS Zoomcamp project organized by DataTalksClub. This project focuses on trip prediction using the NYC taxi dataset.
#MLOpsZoomcamp #DataTalksClub
I've completed the review of the second MLOPS ZoomCamp project organized by DataTalksClub. This project is quite interesting and comprehensive as it uses several technologies such as mlflow, prefect, optuna, and fastapi.
#MLOpsZoomcamp #DataTalksClub
I've started reviewing the second MLOPS Zoomcamp project organized by DataTalksClub. This project focuses on predicting Eurovision results based on the song's characteristics, the country's context, and the characteristics of the competition.
#MLOpsZoomcamp #DataTalksClub
I've completed the review of the third MLOPS Zoomcamp project organized by DataTalksClub. Unfortunately, the project repository was empty, and I wasn't able to award any points to the author.
#MLOpsZoomcamp #DataTalksClub
I have started reviewing the third MLOPS Zoomcamp project organized by DataTalksClub.
#MLOpsZoomcamp #DataTalksClub
🧪 Phase 7: Integration Testing
Wrapping up with integration tests to ensure everything works smoothly. Time to verify that all pipelines (training, deployment, and monitoring) integrate well together. #MLOpsZoomcamp #DataTalksClub
📊 Phase 6: Monitoring with Evidently & Grafana
Setting up the monitoring pipeline to detect data and model drift using Evidently 🚨. Visualizing performance metrics with Grafana in real time for better model monitoring. #MLOpsZoomcamp #DataTalksClub
🚀 Phase 5: Deployment Pipeline
Creating the deployment pipeline! The optimized model is now in production, and I’m using BentoML to serve the model as an API. Time to make it scalable in Docker 🐳. #MLOpsZoomcamp #DataTalksClub
⚙️ Phase 4: Training Pipeline
Building and fine-tuning the training pipeline for my optimized model 🏆. It's been registered in MLflow and ready for promotion to production! Let’s get this model ready for deployment! #MLOpsZoomcamp #DataTalksClub
🛠️ Phase 3: Stack Configuration
Setting up the tech stack for my project using Docker-Compose, ZenML, MLflow, Optuna, and more! Time to integrate tools for optimal MLOps workflow 🚀. #MLOpsZoomcamp #DataTalksClub
🔧 Phase 2: Feature Engineering & Model Training
Moving forward with feature engineering and model training! Using XGBoost for predictions of bike trips 🚴♂️. Now it's time to dive into Hyperparameter Optimization (HPO) with Optuna for the best model performance. #MLOpsZoomcamp #DataTalksClub
🌍 Phase 1: Data Analysis & EDA
Kicking off my #MLOps project by analyzing the raw CitiBike and weather data from NYC 🌦️. Exploring the dataset, cleaning it, and combining the information to build a powerful prediction model for bike trips every hour. #MLOpsZoomcamp #DataTalksClub
🔄 Set up CI/CD pipeline with GitHub Actions for automated testing and deployment! Every commit triggers: linting → testing → building → deployment.
Automation is the key to reliable ML systems!
#MLOPSZOOMCAMP #CICD #GitHubActions #MLOps
🎯 My energy prediction model achieves 15-20% energy savings potential! Applied ML to solve real-world problems: reducing electricity bills and carbon footprint.
Tech for good! 💚
#MLOPSZOOMCAMP #SustainableTech #EnergyEfficiency #MLOps
☁️ Deployed my ML model to AWS with complete infrastructure: S3, ECR, ECS, ALB, VPC - all managed with Terraform IaC!
From code to production in the cloud - what a journey!
#MLOPSZOOMCAMP #AWS #CloudMLOps #Infrastructure
🧪 Implemented comprehensive testing for my ML pipeline! Unit tests, integration tests, and API tests ensure my model works correctly before deployment.
Testing in ML is not optional - it's essential!
#MLOPSZOOMCAMP #MLTesting #MLOps #QualityAssurance
⚡ Built a complete data pipeline that processes 2M+ smart meter readings! From raw data → feature engineering → model training → deployment.
Using Prefect for workflow orchestration made it scalable and maintainable!
#MLOPSZOOMCAMP #DataPipeline #Prefect #MLOps
📊 Implemented data drift detection using Evidently! Now my ML model can automatically detect when incoming data patterns change and alert for potential model
retraining needs.
Real-time monitoring = Production-ready ML!
#MLOPSZOOMCAMP #DataDrift #MLOps #Evidently
🐳 Containerized my ML model with Docker and deployed it to AWS ECS with Fargate! From local development to production cloud deployment - the power of
infrastructure as code with Terraform!
#MLOPSZOOMCAMP #Docker #AWS #Terraform #MLOps
💡 Learned how to implement experiment tracking with MLflow for ML model versioning and hyperparameter optimization. Game-changer for keeping track of model
performance across different experiments!
Production API: smart-energy-alb-795435159.us-east-1.elb.amazonaws.com
#MLOPSZOOMCAMP
🚀 Just completed my first end-to-end MLOps project! Built a smart energy prediction system that processes 2M+ measurements to predict household energy
consumption.
Features: Data pipeline, ML training, API deployment, monitoring & AWS infrastructure!
#MLOPSZOOMCAMP #MachineLearning #MLOps
📊 Building robust ML pipelines:
Today's project: NYC taxi duration predictions with proper testing
Data validation
Feature transformation tests
S3 integration testing
Error handling
CI/CD ready
Key learning: Tests are not afterthoughts in ML, they're essential! #MLOPSzoomcamp
🏗️ Refactoring ML code for testability:
Move global vars to function params
Separate I/O from transformations
Make paths configurable via env vars
Add proper type hints
Handle edge cases explicitly
Result: Maintainable, testable ML code that's production-ready! #MLOPSzoomcamp
"💡 Game-changer: Using Localstack for ML pipeline testing!
No more depending on real AWS services or credentials. Just spin up a local S3 mock, run your tests, and tear it down. Perfect for CI/CD pipelines and local development.
docker-compose makes it super easy! #MLOPSzoomcamp"
🔍 Testing ML pipelines done right:
Unit tests for data transformations
Integration tests with Localstack for S3
Proper error handling for edge cases
Duration filtering (1-60 mins)
Categorical value handling
Real-world ML testing scenarios in action! #MLOPSzoomcamp"
🧪 Deep dive into ML testing today! Learned how to properly structure a batch prediction service with pytest. Key takeaway: Separating data preparation from model inference makes unit testing a breeze. The prepare_data() function can be tested independently without loading the model! #MLOPSzoomcamp
🧪 Wrapped up this week's homework by testing the preprocessing and prediction pipeline with pytest.
From raw data to tested models — full MLOps loop complete!
#MLOpsZoomcamp #DataTalksClub
🧹 Preprocessed the taxi dataset and trained a baseline Linear Regression model. Then pushed the trained model to S3.
Now we’re talking MLOps in action!
#MLOpsZoomcamp #DataTalksClub
🚖 Downloaded the NYC Yellow Taxi dataset for March 2023 and uploaded it to an AWS S3 bucket as part of our pipeline setup.
Getting comfortable with cloud data workflows!
#MLOpsZoomcamp #DataTalksClub
