Aditya Shivam
Patna
Summary
Innovative and adaptable Data Engineer with 4.8 years of experience specializing in building robust data pipelines, scalable data architecture, and high-performance ETL processes. Proficient in Python, SQL, and cloud technologies like AWS, with expertise in modern tools such as Snowflake and Apache Airflow. Adept at collaborating with cross-functional teams to deliver end-to-end data solutions, improve data quality, and support decision-making through actionable insights. Passionate about leveraging data engineering to drive organizational growth and optimize business processes.
Overview
5
5
years of professional experience
Work History
Data Engineer
Funding Societies
08.2023 - Current
- Developed data pipelines, streamlining ingestion processes and enhancing analysis availability.
- Designed and maintained scalable database architectures for optimized data storage solutions.
- Automated validation processes, improving data integrity and reducing manual errors.
- Conducted performance tuning on SQL queries to enhance response times and resource efficiency.
- Integrated diverse third-party data sources via REST APIs to increase interoperability and system reliability.
- Resolved discrepancies through analysis, implementing corrective actions to maintain data accuracy.
- Optimized query performance by creating indexes on tables, significantly improving execution speed.
- Built and optimized ETL pipelines with Apache Airflow, reducing job runtimes by 30%.
Data Engineer
TCS
Kolkata
01.2021 - 07.2023
- Developed real-time and batch data ingestion pipelines, increasing the availability of actionable data
by 40% - Implemented S3 lifecycle rules to automate storage optimization, leading to a 30% reduction in AWS
costs. - Improved query performance by 25% through efficient partitioning and bucketing strategies in Spark-
based table storage.
Designed and deployed CI/CD workflows in GitLab for seamless integration and faster deployment of data pipelines. - Cleaned and manipulated raw data.
- Utilized PySpark to process and transform large-scale datasets, reducing data latency and improving scalability.
- Monitored and optimized ETL processes to improve data loading efficiency and reduce processing times.
Education
Bachelor of Technology - Computer Science And Engineering
Parul University
Vadodara, Gujarat05-2020
Intermediate Certificate -
St. Karen’s High School
Patna, Bihar03-2016
Matriculation -
Rose Bud School
Patna, Bihar03-2014
Skills
- Data pipeline development
- Database architecture
- ETL optimization
- Performance tuning
- API integration
- Data analysis
- Data migration
- ETL development
- SQL and databases
- Query optimization
- Python
- Snowflake (data warehousing, modeling)
- Apache Airflow (ETL orchestration)
- Spark (big data processing)
- AWS (S3, EC2, Lambda)
- Databricks (Data Engineering)
- Qlik Sense (data visualization)
- Rest APIs (integration)
- CI/CD (GitLab, automation)
Timeline
Data Engineer
Funding Societies
08.2023 - Current
Data Engineer
TCS
01.2021 - 07.2023
Bachelor of Technology - Computer Science And Engineering
Parul University
Intermediate Certificate -
St. Karen’s High School
Matriculation -
Rose Bud School