Job Search Status
Open to workDesired start date: Open to discussion
Desired Job Title
Data EngineerMachine Learning EngineerAI DeveloperWork Type
Full TimeLocation Preference
On-SiteRemoteHybridOpen to relocation: Yes
Open To Work
Rajesh Mandapati
Data Engineer
Bengaluru,KA
Work Preference
Summary
Data Engineer with 3 years of experience building scalable ETL and data processing pipelines across GCP and Hadoop environments. Strong in SQL, Python, BigQuery, Spark, and microservice-based workflow orchestration, with hands-on experience in privacy-compliant data workflows for PII scanning, masking, and deletion. Proven impact in reducing Spark runtime by 20%+, lowering scanning and deletion costs by $5,000+ per month, and enabling self-service onboarding across 100+ datasets. Experienced in FastAPI-based microservices, event-driven architectures, CI/CD-supported releases, data quality, production support, and stakeholder collaboration.
Overview
3
3
years of professional experience
2
2
Certificates
Work History
Data Engineer
Lowe’s India
06.2025 - Current
- Redesigned scanning and deletion pipelines by introducing a lookup-based delete strategy aligned with stakeholder workflows, reducing costs by $5,000+/month.
- Optimized Spark jobs using tier-based configurations, reducing runtime by 20%+.
- Designed a multi-factor scoring model using weighted parameters (SLA, request state, submission date) to dynamically prioritize DSR requests, improving processing efficiency by 40%.
- Led migration from on-prem Hadoop to GCP (Dataproc Serverless, BigQuery, GCS) for privacy compliance platform.
- Designed scalable BigQuery data models supporting PII workflows across 100+ datasets.
- Built event-driven architecture using Kafka producers and consumers to handle asynchronous request processing.
- Developed microservice-based ETL workflows for PII scanning, masking, and deletion across distributed Hadoop and GCP environments.
- Built configuration & computation services using MongoDB to manage datasets, PII columns, and Spark configs and the core computation logic.
- Owned end-to-end production releases including deployment validation and pipeline testing, ensuring zero SLA breaches.
- Took ownership of sprint execution and JIRA workflow management during lead’s absence, ensuring on-time delivery of planned work.
- Acted as primary data engineer in a lean team, handling critical deliveries post team transitions.
- Mentored junior engineers through knowledge transfer (KT) sessions and onboarding support.
- Led bi-annual audit processes ensuring compliance through documentation and validation.
- Resolved high-priority production issues ensuring system stability and business continuity.
- Enabled platform adoption by onboarding and guiding 100+ data custodians on KAIZEN workflows including scanning, masking, and deletion processes.
- Supported containerized deployment and release validation of microservices using Docker, YAML-based deployment configs, internal CI/CD tooling, and Kubernetes pod monitoring.
Associate Data Engineer
Lowe’s India
05.2023 - 05.2025
- Automated Airflow DAG workflows, reducing manual effort by 60% and saving ~2 FTE bandwidth.
- Built end-to-end automation for request processing (task grouping, preprocessing, scanning, deletion).
- Implemented data masking solutions aligned with privacy policies, avoiding hard deletions.
- Optimized Spark configurations across datasets improving processing efficiency.
Programmer Analyst Trainee Intern
Cognizant
10.2022 - 12.2022
- Trained in Python and AWS fundamentals (EC2, S3, Lambda, SNS)
Projects
LLM-based Data Operation Chatbot
- Built hybrid chatbot using RAG architecture with vector database (Pinecone)
- Enabled natural language querying for operational insights (DSR status, failures, cost metrics)
- Implemented metadata-based filtering to improve retrieval relevance
- Integrated SQL and MongoDB querying with agent-based routing
Education
B.Tech -
JNTUK University College of Engineering
Vizianagaram, India 04.2001 -
Skills
Languages: Python, SQL, Java (Intermediate)
Cloud & Big Data: GCP (BigQuery, GCS, Dataproc Serverless), Hadoop, Hive, Apache Spark, AWS (EC2, S3, Lambda, SNS)
Databases: BigQuery, PostgreSQL, MongoDB, Pinecone, Weaviate, FAISS
Data Engineering: ETL Pipelines, Data Processing, Data Modeling, Dataframe-Based Transformations, Microservices
API & Frameworks: FastAPI, Apache Airflow
Messaging & Distributed Systems: Kafka
DevOps & Deployment: Docker, Kubernetes, Git, CI/CD, YAML-based deployment configuration, Postman, JIRA
Analytics & Visualization: Pandas, NumPy, Tableau, Power BI, Matplotlib, seaborn, scikit-learn, PyTorch
AI/ML: Machine learning, Deep learning, NLP, LLMs, RAG Pipelines, LangChain, LangGraph, Vector Databases
Accomplishments
- Team Excellence-Award
- Spot-Award – 3 times
Certification
Ekeeda - School of Data Science
Timeline
Data Engineer
Lowe’s India
06.2025 - Current
Associate Data Engineer
Lowe’s India
05.2023 - 05.2025
Ekeeda - School of Data Science
02-2023
Programmer Analyst Trainee Intern
Cognizant
10.2022 - 12.2022
Data Analytics with python
04-2022
B.Tech -
JNTUK University College of Engineering
04.2001 -
AWARDS & CERTIFICATIONS
Team Excellence-Award, Spot-Award – 3 times, Data Science training - Ekeeda school of data science, Data Analytics with python – NPTEL, B and C certificate - NCC
DASHBOARDS
Customer Analysis Dashboard | Tableau
- Built a dashboard to analyze customer segments, business trends, and performance metrics.
- Link: dashboard link
HR Analytics Dashboard | Power BI
- Built a dashboard to analyze attrition, hiring, and workforce trends
- Link: dashboard link