Data Engineering · Analytics · ML/AI · MLOps

Hi, I'm Harish

Data & Analytics Engineer with 2+ years of experience building modern data platforms and applied AI systems across financial services and manufacturing.

My work spans data pipelines, analytics, machine learning, MLOps, RAG, and agentic workflows, from ingestion and modeling through deployment.

Chicago, IL

Microsoft Certified: Fabric Data Engineer Associate

Microsoft · Exam DP-700

Google Cloud Associate Cloud Engineer

Google Cloud

HNM

About

I like the part of data work where things are still messy - source systems don't line up, reporting is manual, pipelines are slow, or the business question isn't completely clear yet.

That's usually where I do my best work: understanding what's actually wrong, getting the data trustworthy, and deciding whether the answer should be a pipeline, analytical model, dashboard, predictive system, or AI application.

My background spans KYC data pipelines, manufacturing sensor analytics, BI, predictive modeling, and RAG/agentic AI. I work mainly with Python, SQL, and modern data platforms, but I also care about the last mile - testing, CI/CD, deployment, documentation, and making the solution something people can use.

Where I Add Value

From raw data to decisions, deployed models, and intelligent systems.

Data Engineering & Modernization

Build reliable pipelines, data models, and cloud platforms while replacing fragmented or manual workflows with scalable tooling.

ETL / ELT · Spark · Kafka · Airflow · dbt · Fabric · Azure · Snowflake

Analytics & Business Intelligence

Turn operational data into trusted metrics, dashboards, analytical models, and decision-ready insights.

SQL · Power BI · Tableau · Grafana · KPIs · Root cause analysis

Data Science & MLOps

Build and evaluate predictive models while connecting feature engineering and experimentation to testing, deployment, and monitoring.

scikit-learn · XGBoost · Forecasting · Model evaluation · CI/CD

Applied AI & Agentic Systems

Build grounded AI applications with retrieval, LLM workflows, tool integration, and agentic patterns for practical tasks.

RAG · LangChain · MCP · Agentic workflows · Vector search

Work Experience

Built data-quality analysis, vehicle risk scoring, predictive models, and real-time manufacturing quality monitoring across two production stages.

  • Analyzed 2,000+ automotive sensor records across two manufacturing stages, identifying 15+ data-quality issues and establishing a cleaner foundation for downstream risk modeling.
  • Engineered 10+ tolerance-margin features and a vehicle-level risk score to surface high-risk units before Final Assembly.
  • Built and benchmarked Linear Regression, Random Forest, and Gradient Boosting models in scikit-learn, reaching up to approximately 85% prediction accuracy.
  • Built a real-time Grafana dashboard tracking quality KPIs, pass/fail rates, and anomaly alerts for manufacturing monitoring.

Built SQL-based ETL and BI solutions for a KYC compliance platform serving a US-based banking client.

  • Engineered and maintained SQL-based ETL workflows for a KYC compliance platform processing 1M+ onboarding records across source, staging, and reporting layers.
  • Optimized SQL queries, stored procedures, views, and indexing strategies, improving ETL processing performance by approximately 40%.
  • Developed Power BI reporting for onboarding and operational KPIs, contributing to approximately 15% higher account activation and approximately 30% lower manual reporting effort.
  • Performed data-quality validation, reconciliation, profiling, and root cause analysis with QA, business, and compliance teams.

Automated recurring reporting and improved customer-data reconciliation through SQL analysis and validation.

  • Automated recurring reporting with 30+ SQL queries across three databases, reducing manual reporting effort by approximately 25%.
  • Profiled and validated 50K+ customer records, resolving inconsistencies and supporting approximately 98% reconciliation accuracy.

Technical Skills

Core Stack

  • Python
  • SQL
  • Apache Spark
  • Apache Kafka
  • Apache Airflow
  • dbt
  • Microsoft Fabric
  • Snowflake
  • Azure
  • Power BI
  • Databricks
  • scikit-learn
  • Docker

Data Engineering

Core technologies

  • Python
  • SQL
  • R
  • PySpark
  • Pandas
  • NumPy
  • Apache Spark
  • Apache Kafka
  • Apache Airflow
  • dbt
  • Delta Lake
  • Parquet
  • Hadoop
  • HDFS
  • PostgreSQL
  • MySQL
  • Microsoft SQL Server
  • Oracle
  • Snowflake
  • BigQuery
  • SQLite
  • Elasticsearch
  • DuckDB

Concepts & methods

CTEs / Window Functions / Stored Procedures / Query Optimization / Complex Joins / Data Manipulation / ETL / ELT / Data Pipelines / Data Modeling / Dimensional Modeling / Star Schema / Fact & Dimension Design / Data Warehousing / Medallion Architecture / Data Validation / Data Profiling / Data Reconciliation / Data Quality / Pipeline Testing / REST API Integration / Event-Driven Workflows

Analytics & ML

Core technologies

  • Power BI
  • Tableau
  • Grafana
  • Excel
  • Streamlit
  • Plotly
  • Matplotlib
  • Seaborn
  • scikit-learn
  • XGBoost

Concepts & methods

KPI Development / Ad Hoc Reporting / Data Storytelling / Trend Analysis / Root Cause Analysis / RFM Segmentation / Drill-Down Analysis / Regression / Linear Regression / Random Forest / Gradient Boosting / Feature Engineering / Predictive Modeling / Model Evaluation / Time-Series Forecasting / Holt-Winters / Statistical Analysis / Hypothesis Testing / A/B Testing / Forecast Accuracy / WAPE / Risk Scoring / Segmentation

Cloud & Data Platforms

Core technologies

  • Microsoft Fabric
  • AWS
  • Google Cloud Platform
  • Snowflake
  • Databricks

Microsoft Fabric: OneLake · Lakehouse · Warehouse · Fabric Data Pipelines · Fabric Data Factory · Fabric Spark · Real-Time Intelligence

AWS: S3 · EC2 · Redshift · IAM

Google Cloud Platform: BigQuery

Snowflake: Warehouse and analytics workflows

Databricks: Spark · Delta · notebook environment

Concepts & methods

KQL / DAX / Power Query

Applied AI

Core technologies

  • LangChain
  • FAISS
  • Elasticsearch

Concepts & methods

RAG / Vector Search / Embeddings / Elasticsearch Retrieval / MCP / Model Context Protocol / AI Agents / Agentic Workflows / LLM Applications / Document Chunking / Metadata-Aware Retrieval / Prompt Engineering / LLM Evaluation / Transformers / LoRA / Federated Learning

Supporting Tools & Delivery

Core tools

  • Git
  • GitHub
  • Docker
  • Linux

Engineering & delivery

MLOps / CI/CD / Automated Testing / Pipeline Testing / Model Evaluation / Deployment Workflows / Code Reviews / Technical Documentation / REST API Integration

Collaboration

Jira / Confluence / Agile / Scrum

Selected Work

Projects

A focused selection of data systems, analytical products, predictive models, and retrieval applications.

Featured Projects

Data Engineering
Analytics
BI

RetailIQ - Retail Data Platform & Revenue Analytics

100K+

Retail transactions

Problem
Retail transaction data needed to be transformed into structured, decision-ready analytics.
What I built
Designed a modern retail analytics pipeline and dimensional model for revenue, product, and customer analysis.
Scale
100K+ retail transactions
Outcome
Created analytics-ready models and interactive BI reporting across 8+ business KPIs.
Python
SQL
Airflow
dbt
Snowflake
Power BI
Star Schema
RFM Segmentation
AI
RAG

CampusGuide RAG - Grounded Institutional Knowledge Assistant

Grounded Q&A

Semantic document retrieval

Problem
Institutional policies and student-services information were distributed across documents and difficult to retrieve quickly.
What I built
Built a retrieval-augmented assistant that indexes institutional documents, retrieves relevant context through semantic search, and generates grounded responses.
Outcome
Created a semantic retrieval workflow that returns contextual answers grounded in institutional content.
Python
LangChain
Elasticsearch
Vector Search
Document Ingestion
Chunking
Embeddings
Prompt Workflows
Data Science
Forecasting
Analytics

PartsFlow - Demand Forecasting & Replenishment Engine

19.1% WAPE

Compared with 21.6% baseline

Problem
Improve SKU-level demand forecasting and inventory replenishment decisions.
What I built
Built Holt-Winters forecasts and an accuracy-versus-cost dashboard.
Scale
Simulated 40-SKU, three-year demand network
Outcome
Connected forecast accuracy to operational inventory outcomes.
  • 19.1% WAPE vs. 21.6% naive baseline
  • Outperformed the baseline on 30 of 40 SKUs
  • Simulated fill rate improved from 82% to 89%
  • Approximately 40% lower lost sales
Python
DuckDB
statsmodels
Holt-Winters
Streamlit
AI
Healthcare
Hackathon

Bandaid Maps - AI-Assisted Healthcare Navigation

Winner - Melissa Data Challenge, LA Hacks 2025

Winner

LA Hacks 2025 Melissa Data Challenge

Problem
Help users understand health readiness and locate appropriate nearby healthcare resources.
What I built
Created an AI-assisted healthcare navigation application during a 36-hour LA Hacks build at UCLA.
Outcome
Won the Melissa Data Challenge at LA Hacks 2025.
FastAPI
React
Google Gemini API
Melissa APIs
MongoDB
Health Readiness Score
Emergency Action Plan
Interactive Map

More Projects

Data Engineering
Machine Learning

ChurnShield - Customer Churn Prediction & Analytics Pipeline

Reached approximately 89% classification accuracy and produced interpretable customer-risk segments.

PythonSQLKafkaSnowflakescikit-learn
AI
RAG

FinDocs RAG - Financial Document Q&A System

Enabled contextual question answering with retrieval traceability and source-aware responses.

PythonLangChainFAISSOpenAI APIStreamlit
Analytics
Data Engineering

CTA TransitPulse - Chicago Transit Analytics

Enabled route-level KPI monitoring, trend analysis, heatmaps, and geographic exploration.

PythonSQLPandasSQLiteStar Schema
Data Science
Analytics

MediCost - Healthcare Cost Prediction & Risk Analytics

Built healthcare cost-prediction models and delivered an executive analytics dashboard with 10+ KPIs.

PythonSQLRPandasscikit-learn
Data Engineering
Financial Analytics

FinStream - Credit Card Transaction Analytics Pipeline

Created a structured analytics layer supporting transaction behavior and anomaly analysis.

PythonSQLSnowflakedbtPandas
Big Data
Data Engineering

GridWatt - Smart Meter Energy Analytics Pipeline

Created scalable aggregate analytics for consumption, peak demand, and household segments.

PySparkPythonAWS S3ParquetSnowflake
Data Engineering
Databricks

HealthSync - Healthcare Claims Data Platform

Reached approximately 97% data completeness and enabled analytics for claims cost, patient trends, and quality monitoring.

PythonDatabricksPySparkKafkaREST API
Hackathon
Applied Data Science

Road Sign Validation & Geospatial Intelligence

Reached the finalist stage while applying computer vision, routing data, and geospatial pipelines to urban transportation infrastructure under hackathon time constraints.

OpenCVObject DetectionComputer VisionClusteringHERE APIs
Database Engineering

Scalable Food Delivery Database System

Created a normalized database structure supporting scalable transaction handling and analytical queries.

SQLRelational ModelingNormalization3NFQuery Optimization

Research & Recognition

IEEE Research · ICAIC 2026
Research
Machine Learning
Cybersecurity

Hybrid Transformer and XGBoost Model for Federated IoT Intrusion Detection

2026 IEEE 5th International Conference on AI in Cybersecurity (ICAIC)

~98% detection

~90% lower communication overhead

Problem
IoT intrusion detection needs strong predictive performance while federated learning introduces privacy and communication constraints.
What I built
Developed a hybrid Transformer and XGBoost intrusion-detection architecture for distributed IoT learning.
Outcome
Achieved approximately 98% detection performance with approximately 90% lower communication overhead.
Transformer
XGBoost
Federated Learning
LoRA
IoT
Cybersecurity
Machine Learning
View Paper
Winner
2025

Melissa Data Challenge Winner

LA Hacks 2025 · UCLA

Project: Bandaid Maps

Won the Melissa Data Challenge for Bandaid Maps, an AI-assisted healthcare navigation application built during a 36-hour hackathon.

Participant · Datathon
2026

AnDackaThon 2026

Analytics & Data Summit 2026 · Oracle Redwood Shores / San Jose State University

Participated in the Datathon track, working with real-world datasets and Oracle analytics and data technologies under a compressed hackathon timeline.

Finalist
2025

HERE Technologies Chicago Hackathon Finalist

HERE Technologies Chicago Hackathon

Reached the finalist stage with a road sign validation and geospatial intelligence project.

Education

Illinois Institute of Technology

Master of Data Science

GPA: 3.81

Chicago, IL

Aug 2024 to May 2026

Anna University

Bachelor of Engineering

Electronics and Communication Engineering

Affiliated college: St. Joseph's College of Engineering

GPA: 3.65

Chennai, India

Aug 2018 to May 2022

Certifications

Microsoft Certified: Fabric Data Engineer Associate

Microsoft · Exam DP-700

Google Cloud Associate Cloud Engineer

Google Cloud

Credential ID: 73403999

May 2023

Leadership & Activities

IIT Product Management Club

Vice PresidentAug 2025 to May 2026

Co-led the club with the President, supporting product-building activities, AI-oriented initiatives, workshops, case studies, and product work connecting strategy, data, and engineering.

Indian Student Association - Illinois Tech

Vice PresidentJan 2026 to May 2026
Finance HeadAug 2025 to Dec 2025
Finance Team MemberFeb 2025 to Jul 2025

Supported event management, budgeting, expense tracking, reimbursements, organization operations, and executive leadership across three roles.

ACM Illinois Tech

TreasurerMay 2025 to May 2026
Vice TreasurerNov 2024 to May 2025

Managed budget planning, expense tracking, reimbursements, sponsor and vendor payments, and chapter operations, including ScarletHacks organizing.

The Optical Society (OSA)

PresidentJun 2021 to May 2022
TreasurerJun 2020 to Jun 2021

As President, led a 16 to 17 member student organization and coordinated approximately 10 to 12 technical events and workshops with faculty, members, and sponsors. As Treasurer, managed budgeting and financial coordination for chapter activities and supported sponsorship-related planning.

Let's Connect

Let's Connect

Open to conversations around data engineering, analytics, ML/AI, projects, and opportunities.

Start a Conversation
Schedule a conversationFor opportunities, project discussions, or a quick introduction.Email me to arrange a time
Email meharishnamasivayam@gmail.comSend EmailConnect on LinkedInProfessional profile and experienceConnect on LinkedInView GitHubCode, projects, and experimentsView GitHub
LinkedIn