Sudit Patnaik
Summary
Data Engineer with 12+ years of experience building large-scale big data platforms, data warehouses, and domain-driven data solutions. Brings 8+ years in the retail domain, specializing in Customer Data Management,Customer Data Platforms (CDP), marketing analytics, and product master data management. Highly proficient in Spark (PySpark), Hadoop, GCP, BigQuery, Hive, Presto, and Python. Proven expertise in entity resolution and customer matching, designing fact and dimension models for historical analysis and auditability, and enforcing privacy governance including CCPA and other consumer data protection regulations through compliant data deletion and PII remediation.
Overview
12
12
years of professional experience
Work History
Lead Data Engineer
Lowe's India
Bengaluru
06.2017 - Current
- Led cross-functional data engineering initiatives across Customer MDM, Customer Data Warehouse, CDP, Marketing Data Hub, Loyalty, and Product MDM platforms.
- Managed and mentored a team of up to 12 engineers, driving technical execution, solution design, and alignment with business and product requirements.
- Architected scalable, fault-tolerant data pipelines processing over 200 million customer records and over 12 billion order-line records, delivering Gold-layer datasets for Marketing, Analytics, Salesforce CDP, and in-house audience platforms.
- Designed and implemented RFM (Recency, Frequency, Monetary) pipelines enabling advanced analytics and data science models including Customer Lifetime Value (CLV), churn propensity, shopping behavior, and brand affinity modeling.
- Built automated privacy enforcement pipelines compliant with CCPA and US state privacy regulations, supporting customer data erasure, consent management, and PII updates across enterprise data platforms.
- Led Cloudera-to-GCP migration, converting 100+ Spark/Hive jobs to BigQuery & Dataproc, migrating 50+ datasets, and ensuring uninterrupted operations via 50+ cross-platform data pipelines.
- Led feature engineering for 10+ data science models (shopping propensity, churn propensity, CLV and more), enabling scalable model development; partnered with data scientists to build CI/CD pipelines, schedule training and inference, and operationalize model outputs into Gold tables for business consumption.
Senior Software Engineer
Wipro
Bengaluru
11.2013 - 06.2017
- Worked for a US based Banking and Brokerage company to build Mainframe systems for loading customer banking accounts to brokerage firm, Money transfers between banking and brokerage firm, maintaining 5 test environments for UAT, QA, Preprod testing.
- Worked on automations to perform data transfers between environments, applying masking rules over PII columns to adhere to privacy governance.
- Worked for a US based Banking and Brokerage company to build Mainframe systems for loading customer banking accounts to brokerage firm, Money transfers between banking and brokerage firm, maintaining 5 test environments for UAT, QA, Preprod testing.
Education
B.TECH - Electronic and communication Engineering
Biju Patnaik University Of Technology
05.2013
Skills
- Apache Spark and Hadoop
- SQL and Hive
- Python , Pyspark, Dataproc
- Google Cloud Platform and BigQuery
- Cloud Composer / Airflow
- Kafka , pub-sub
- AI and machine learning
- Teradata
- Mainframe
- GIT, Bitbucket
Personal Information
Title: Lead Data Engineer
Websites
Timeline
Lead Data Engineer
Lowe's India
06.2017 - Current
Senior Software Engineer
Wipro
11.2013 - 06.2017
B.TECH - Electronic and communication Engineering
Biju Patnaik University Of Technology