Practical data engineer possessing in-depth knowledge of data manipulation techniques and computer programming paired with expertise in integrating and implementing new software packages and new products into system. Offering more then 4 years of background managing various aspects of development, design and delivery of database solutions. Tech-savvy and independent professional bringing outstanding communication and organizational abilities.
Overview
5
5
years of professional experience
Work History
Data Engineer
Guardian life insurance company of America
Delhi
11.2019 - Current
Optimized existing queries to improve query performance by creating indexes on tables.
Analyzed user requirements, designed and developed ETL processes to load enterprise data into the Data Warehouse.
Designed and developed a PySpark-based ETL pipeline to integrate data from diverse sources, resulting in a unified and cohesive dataset for business analytics
Utilized Presto for interactive querying and analysis of distributed data sets, providing real-time insights for business.
Optimized PySpark jobs by tuning configurations and partitioning strategies, reducing resource utilization and improving overall performance by 25%.
Conducted performance tuning of Informatica jobs, optimizing resource utilization and improving overall system efficiency.
Collaborated with data architects and business analysts to understand data requirements and implemented scalable and flexible ETL solutions to meet business needs.
• Conducted performance tuning of DMX jobs, optimizing resource utilization and improving overall system efficiency.
• Actively participated in the design and development of a data warehouse architecture, ensuring alignment with business
goals and scalability for future growth.
• Developed Shell Scripts for automation of daily tasks.
• Collaborated Managed file transfer team to send or get files through AXWAY system from multiple teams.
• Schedule Control-M runbooks in Production Environment.
Education
Bachelor of Science - Information Technology
Maharaja Agrasen Institute of Technology
New Delhi, DL
05-2018
Skills
Data Migration
Data Warehousing
SQL
SQL and Databases
Spark Framework
AWS
GIT
Bitbucket
Timeline
Data Engineer
Guardian life insurance company of America
11.2019 - Current
Bachelor of Science - Information Technology
Maharaja Agrasen Institute of Technology
Similar Profiles
Gulshan GoyalGulshan Goyal
Lead Engineer at Guardian Life Insurance Company of America (A Fortune 250 Company)Lead Engineer at Guardian Life Insurance Company of America (A Fortune 250 Company)
Infrastructure Service Management Engineer at Guardian Life Insurance Company of AmericaInfrastructure Service Management Engineer at Guardian Life Insurance Company of America