Shubhanshu Mani
Gurugram,
Summary
Experienced Data Engineer with a demonstrated proficiency in devising and implementing data solutions within the Amazon Web Services (AWS) framework. Actively pursuing a stimulating role to leverage my extensive background in data engineering, proficiency in ETL (Extract, Transform, Load) processes, and expertise in cloud technologies. Eager to contribute to innovative initiatives and provide impactful data solutions for a progressive and forward-looking organization. Proven success in leading Cross Functional Teams for the development and delivery of multiple projects
Overview
3
3
years of professional experience
Work History
Data Engineer
Decision Point Analytics
Gurugram
03.2021 - Current
- Conceptualized, constructed, and executed end-to-end data pipelines catering to fast-moving consumer goods (FMCG) scenarios using AWS Services and Azure Data Factory. • • Oversaw data ingestion, integration, transformation, storage, and delivery processes. Collaborated with cross-functional teams, including data scientists, analysts, and business stakeholders, to outline data requirements, deliver actionable insights, and enhance overall team efficiency by 10%. • • Led the successful completion of a complex data promotional project, ensuring minimal downtime and zero data loss. Pioneered the adoption of a novel data compression algorithm, optimizing pipeline memory usage by 60% and reducing storage costs by 20%. • Headed the Spark optimization initiative, resulting in a notable 50% reduction in ETL pipeline execution time. Achieved this through targeted SQL tuning, memory optimization, and adjustments in parallelism, consequently enhancing data processing and analysis capabilities for the team. • Established real-time logging and alerting systems, documented best practices, and collaborated cross-functionally to ensure seamless operations. • • Identified and resolved a bottleneck in the data processing pipeline, leading to a 50% improvement in overall system performance. • Developed and maintained ETL pipelines using SQL queries, Python, and Spark scripts on Azure Data Factory (ADF). • Proficient in AWS data services (S3, CloudWatch, Glue), SageMaker for ML/AI automation, Athena, and Azure Databricks. • Skilled in crafting scalable and cost-effective architectures.
Education
Bachelor of Technology - Information Technology
ABES Engineering College
Ghaziabad01-2021
Skills
- Database Management: MySQL, PostgreSQL
- Python, Spark
- AWS, Azure, GCP : AWS Glue, Cloudwatch, Athena, S3, AWS Lambda, Crawler, AWS Sagemaker, Azure Databricks, Azure Blob, GCP Storage
- Data Validation, Metadata management
Internship
- Data Analyst intern at SAPIO Analytics
Timeline
Data Engineer
Decision Point Analytics
03.2021 - Current
Bachelor of Technology - Information Technology
ABES Engineering College
Similar Profiles
Jithesh JanardhananJithesh Janardhanan
Analytics Consultant at Decision Point AnalyticsAnalytics Consultant at Decision Point Analytics
Sahil MittalSahil Mittal
Lead Data Analyst at Decision Tree AnalyticsLead Data Analyst at Decision Tree Analytics
ASEER JESAN AASEER JESAN A
Senior Data Engineer at Decision PointSenior Data Engineer at Decision Point