Aaryan Singh
Bengaluru,Karnataka
Summary
Highly skilled Data Engineer with 9 years of experience. Expert in building and managing data pipelines, covering all aspects from data ingestion to loading into warehouses. Develops efficient ETL and BI solutions by analyzing business needs. Proficient in Snowflake, MySQL, SQL Server, AWS, Azure, Talend, Informatica, Stitch, DBT, Spark, SQL, Python, and Unix.
Overview
9
9
years of professional experience
1
1
Certification
Work History
Senior Data Engineer
Qlik
07.2022 - Current
- Managed Snowflake administration, integrating it with Talend Cloud and handling large-scale data with advanced SQL and Python.
- Developed cloud-based ETL pipelines using AWS, Snowflake, and Spark.
- Led performance tuning, capacity planning, and cloud cost optimization for efficient Snowflake and Spark operations.
- Created reusable data ingestion frameworks using Python, ensuring data security, governance, and disaster recovery strategies.
.
Solution Lead
MResult
09.2019 - 07.2022
- Designed and developed a reusable data ingestion framework using Python and Spark, automating the extraction of data from multiple dynamically changing sources.
- Integrated Snowflake with Talend Cloud on AWS, processing large-scale data from JDE, Salesforce, file systems, and various APIs (SOAP, REST).
- Optimized Snowflake for performance and scalability, including query optimization, resource management, and cost-efficient data processing for large datasets.
- Delivered end-to-end data engineering solutions for Howard Hughes, leveraging Snowflake, AWS, and Spark to optimize data workflows and processing efficiency.
Associate Consultant
Capgemini
05.2018 - 09.2019
- Developed data pipelines for an Enterprise Data Warehouse using Talend Big Data on Hadoop, ensuring efficient data flow and processing.
- Migrated shell scripts to Talend ETL jobs, streamlining processes and improving automation.
- Administered Talend and the data platform end-to-end, ensuring smooth operations and optimized performance.
- Converted MapReduce jobs to Spark, significantly reducing task execution times and improving overall system efficiency.
System Engineer
Tata Consultancy Services
10.2015 - 05.2018
- Trained in Informatica, SQL, and SAP Business Objects during ILP, gaining expertise in data integration and management.
- Developed a Corporate Account Planning solution for Mashreq Bank using Informatica and SQL, improving account management processes.
- Designed and developed a Common Data Model for Johnson & Johnson's ASPAC region, utilizing Talend DI, S3, and AWS Redshift for optimized data handling.
- Implemented a dynamic load framework for multiple sources, including Salesforce and MySQL, recognized as the Express Ingestion module for its efficiency.
Education
Bachelor of Engineering (B.E) - Telecommunications Engineering
Sir M Visvesvaraya Institute Of Technology
Bengaluru05.2015
Skills
- Databases: Snowflake, Redshift, MySQL, SQL Server
- Cloud Platforms: AWS, Azure
- Integration Tools: Talend Cloud, Talend DI, Talend Big Data, Informatica, Stitch, DBT
- Programming Languages: SQL, Python, Unix
- Big Data Technologies: Kafka, Spark, MapReduce (MR), Hive, HDFS
- Orchestration Tools: Apache Airflow, Talend Management Console
Certification
- Snowflake - Snow Pro Core Certified
- Azure Data Fundamentals Certified
- Talend Cloud Administrator Certified
- AWS Cloud Practitioner Certified
- Talend Data Integration Certified
Timeline
Senior Data Engineer
Qlik
07.2022 - Current
Solution Lead
MResult
09.2019 - 07.2022
Associate Consultant
Capgemini
05.2018 - 09.2019
System Engineer
Tata Consultancy Services
10.2015 - 05.2018
Bachelor of Engineering (B.E) - Telecommunications Engineering
Sir M Visvesvaraya Institute Of Technology
Similar Profiles
Prasun Ghosh HazraPrasun Ghosh Hazra
Customer Success Manager at QlikCustomer Success Manager at Qlik
Hemal PatelHemal Patel
Manager at QlikManager at Qlik
Dominique ThomasDominique Thomas
Customer Success Manager at QlikCustomer Success Manager at Qlik