LUCKY SHARMA
New Delhi
Summary
Overall 5+ Years and Relevant 4.6 years of experience in Data Engineering, Design, Data on Cloud, Deployment using various AWS Cloud Service tools & technologies, Python, SQL and PySpark. Implements best practices to create Data pipeline, Data Migration from on-premises to Cloud and developed ETL pipeline applications. Deep understanding of AWS Serverless Services, Python, SQL, Terraform, Data Infrastructure.
Overview
5
5
years of professional experience
1
1
Certification
Work History
Technology - Data Engineer
Iris Software
Noida
03.2024 - Current
- Designed and implemented a Proof of Concept (POC) for Data Quality Framework, focusing on accuracy, timeliness, validity, consistency, and completeness.
- Proficiently utilized Python, PySpark, and AWS Glue to streamline the read and write operations of data into AWS Redshift tables.
- Performed schema comparison between schema registry and files uploaded in S3 Bucket, then loaded the results into the desired table of a Redshift database.
Senior Software Engineer
GlobalLogic Noida
Noida
04.2021 - 03.2024
- Data Requirements and Modelling: Involved in requirements gathering, analysis, design, implementation, and testing.
- Data Extraction and Validation: Implement code to extract raw data from images and csv files. perform require transformation and validation and record into the database.
- Implement Orchestration technique using AWS Glue workflow, validate data, perform parallel processing for optimization of ETL Pipeline.
- Data Monitoring and Troubleshooting: Monitor data pipeline health, share insight through SNS to respective email users and teams channel, Troubleshoot and fix problems.
- Configure ETL Pipeline for respective clients to run only respective glue jobs for customers.
- Deploy code on QA/Dev/Prod environment using Terraform.
- Involved into UAT and PROD support to resolve the PROD issues.
- Enhanced efficiency of pipeline through the division of large files into smaller chunks and implementation of parallel processing techniques.
Software Engineer
GlobalLogic Noida
Noida
10.2019 - 03.2021
- Involved in requirements gathering, analysis, design, implementation and testing.
- Understanding the Read and Write IOPS using Window Monitoring tool to decide the replication instance engine.
- Prepare the support and unsupported schema using Schema Conversion tool.
- Configure Source endpoint, target endpoint and DMS.
- Implemented a two-step migration process involving backup & restore as well as ongoing data replication with DMS.
- Performed live cutover ensuring minimal downtime.
Intern Trainee
Servosys Solution
New Delhi
05.2019 - 08.2019
- Completed a 4-month internship focused on Frontend development.
Internship Trainee
Cognizant
Pune
02.2019 - 04.2019
- Gained hands-on experience in implementing Data on AWS Cloud during a 3-month internship.
Education
BE in Computer Science -
CHANDIGARH UNIVERSITY
06-2019
12th -
Sarvodaya Bal Vidyalaya
05-2014
10th -
Govt. Boys Sr. Sec School
05-2012
Skills
- Python
- SQL
- Pandas
- PySpark
- Data on AWS Cloud
- AWS Cloud Services
- Data Pipeline Design
- Data Migration
- Glue ETL
- SQL and NOSQL Databases
- Terraform
- Problem Solving
- Redshift
- Communication and Team Collaboration
Awards
- Spot Award of the Month April' 23, AccessCorp, GlobalLogic
- Spot Award of the Month May' 21, AccessCorp, GlobalLogic
- Spot Award of the Month Sep' 20, AccesssCorp, GlobalLogic
Certification
- AWS Certified Developer Associative, Udemy, 08/2021
- AWS Certified Developer Associative, AWS Training & Certification, 10/2021
Languages
- English
- Hindi
Timeline
Technology - Data Engineer
Iris Software
03.2024 - Current
Senior Software Engineer
GlobalLogic Noida
04.2021 - 03.2024
Software Engineer
GlobalLogic Noida
10.2019 - 03.2021
Intern Trainee
Servosys Solution
05.2019 - 08.2019
Internship Trainee
Cognizant
02.2019 - 04.2019
BE in Computer Science -
CHANDIGARH UNIVERSITY
12th -
Sarvodaya Bal Vidyalaya
10th -
Govt. Boys Sr. Sec School
Similar Profiles
Harshitha J SalianHarshitha J Salian
Lead Developer at IRIS SoftwareLead Developer at IRIS Software
Sumit NavinSumit Navin
Senior Software Engineer at IRIS SoftwareSenior Software Engineer at IRIS Software
Himaanshu SachdevaHimaanshu Sachdeva
Product Manager / Product Owner at Iris SoftwareProduct Manager / Product Owner at Iris Software