Aditya Roy Choudhury
Data Engineer
pune
Summary
Experienced Data Engineer with 4 years of hands-on experience in implementing Data Engineering, Cloud Technologies, and SAP Business Warehouse solutions. Demonstrates a curious mindset, strong problem-solving skills, and an unwavering passion for learning. Excels at adapting to emerging technologies and leveraging them to effectively tackle complex business challenges. Committed to fostering collaboration, innovation, and delivering impactful data-driven solutions.
Overview
4
4
years of professional experience
6
6
years of post-secondary education
Work History
Systems Engineer - Data Engineer
Tata Consultancy Service Ltd
10.2023 - Current
- Collaborated with a US-based food wholesale company to modernize and migrate ETL pipelines from AWS Redshift and DBT to Databricks, enhancing system performance and scalability
- Implemented Medallion Architecture on Databricks using Autoloader for seamless raw data ingestion, ensuring data integrity through conversion and cleaning, optimizing query performance for analytics, developed Delta Live Table (DLT) pipelines, and configured jobs to streamline data-loading workflows
- Developed dashboards and reports in Power BI and Databricks, integrating Databricks data for actionable insights and improved decision-making
- Worked extensively with Databricks Bundle and GitLab for version control and collaborative development
- Exercised strong problem-solving skills to identify and resolve data inconsistencies, improving client satisfaction and data accuracy
- Conducted client demos and maintained regular client interactions to gather feedback, align on requirements, and ensure successful project delivery
- Contributed to two additional projects post-migration, focusing on Gold Layer query development, enhancement, optimization, and validation of query issues to ensure consistency and reliability across workflows
Senior Systems Engineer
Infosys Ltd
10.2022 - 09.2023
- Worked as a Data Engineer for one of the biggest beverage company client
- Developed, deployed, and maintained data pipelines, ETL processes, and data integration workflows using tools like Azure Databricks, Azure Data Lake Storage, Airflow, GitHub, and Jenkins
- Skillfully handled diverse data sources and formats, such as CSV, JSON, and Delta tables
- Automated data pipelines and workflows, validating retail and customer data using SQL and PySpark in Databricks
- Conducted thorough analysis to identify and resolve system problems
- Implemented and maintained data analytics protocols, standards, and manuals, including conducting training sessions to educate co-workers
Systems Engineer
Infosys Ltd
03.2021 - 09.2022
- Successfully managed ETL (Extract, Load & Transform) operations on the ecom data pipeline for a prominent sports company, handling vast data volumes
- Utilized Databricks, PySpark, SQL, GIT, Bitbucket, and Jenkins in an Agile methodology to ingest data from Legacy SAP Business Warehouse into Databricks Lakehouse (ADLS Gen 2)
- Completed Infosys Training on SQL, SAP BW/BI, and Data Engineering with High-performance grades
- Created Jobs for scheduling daily data loads to the Lakehouse and performed data modeling for specific data assets
- Enhanced the HR system in SAP BW by adding metrics and attributes to tables, resulting in faster query results, tested through Microstrategy
Education
Bachelor's of Technology - Biotechnology
08.2016 - 07.2020
Indian School Certificate (ICSE/ISC) - Science
04.2014 - 03.2016
Skills
Python, SQL & Pyspark
undefinedAccomplishments
- Microsoft Certified: AZ-900, DP-900, and AI-900
- AWS Certified Cloud Practitioner
- Databricks Lakehouse Fundamentals Accreditation
- 5-Star HackerRank in Python and SQL
- GATE 2020 Qualified (GEN)
Timeline
Systems Engineer - Data Engineer - Tata Consultancy Service Ltd
10.2023 - Current
Senior Systems Engineer - Infosys Ltd
10.2022 - 09.2023
Systems Engineer - Infosys Ltd
03.2021 - 09.2022
Haldia Institute of Technology - Bachelor's of Technology, Biotechnology
08.2016 - 07.2020
St. Xavier's School - Indian School Certificate (ICSE/ISC), Science
04.2014 - 03.2016
Similar Profiles
Sachin ChakrabortySachin Chakraborty
QA Analyst at TATA CONSULTANCY SERVICE LTDQA Analyst at TATA CONSULTANCY SERVICE LTD
Kavindra ShrivastavaKavindra Shrivastava
Technical Project Manager at Tata Consultancy Service LTDTechnical Project Manager at Tata Consultancy Service LTD
Gowthaman AGowthaman A
Global Tower Lead at Tata Consultancy Service LtdGlobal Tower Lead at Tata Consultancy Service Ltd