MANJOT SINGH
Lead Data Engineer
Gurgaon
Summary
Experienced Data Engineer skilled in designing and implementing data pipelines and quality frameworks using various tools including Pyspark, SQL, Apache Spark, Azure DataFactory, Azure Databricks, and Apache Hive. Seeking a Senior/Lead Data Engineer role to leverage advanced analytics techniques and technologies for driving business objectives.
Overview
8
8
years of professional experience
4
4
years of post-secondary education
4
4
Certificates
Work History
Lead Data Engineer
Publicis Sapient
12.2023 - Current
- Leading a team of 6 Data Engineers to ensure delivery of high-quality, scalable, and performant data integration solutions
- Develop and maintain documentation, including technical specifications, data flow diagrams, and data mappings.
- Involved in architectural discussions and solution of design patterns to consider various factors such as requirements of system, scalability needs, performance considerations, security concerns.
- Designed and reviewed data pipelines, which included data ingestion, extraction, and transformation services, that increased data quality by 50%.
Data Engineer
IBM
03.2022 - 12.2023
- Collaborated with system architects, design analysts and others to understand business and industry requirements, by facilitating weekly meetings to discuss current and future projects
- Evaluated and prepared metadata for tables and columns, to be used in use case, reviewing and enhancing table structures
- Built complex data pipelines and identified trends, and risks to provide useful insights to improve data, reducing time for data preparation by 30%.
Software Engineer
Fidelity Investments
05.2020 - 03.2022
- Analyzed large datasets to identify trends and patterns in customer insurance enrollments by analysis past trends in electable and elected choices.
- Prepared documentation and analytic reports, delivering summarized results analysis and conclusions to stakeholders.
- Using advance hive techniques partitioning and bucketing for improving performance of queries.
System Engineer
Tata Consultancy Services
04.2016 - 05.2020
- Gathered fair understanding on HRMS functionalities while working on minor enhancement requests/bug fixes in custom PL/SQL packages, functions, and procedures
- Expertise in Root Cause Analysis and support strategies for significantly affecting production incidents
- Conducted regression testing and submitted observations to team which reduced potential issues in production by 20%.
Education
Bachelor of Technology, Electronics and Communication Engineering -
Punjab Technical University
Jalandhar 05.2011 - 05.2015
Skills
Azure Data Factory
Azure Data Lake
Azure Databricks
PySpark
Hadoop
Big Data Analytics
Hive
SQL
PL/SQL
Apache Spark
Python
Certification
Amazon Web Services Cloud Practitioner
Timeline
Lead Data Engineer
Publicis Sapient
12.2023 - Current
Data Engineer
IBM
03.2022 - 12.2023
Software Engineer
Fidelity Investments
05.2020 - 03.2022
System Engineer
Tata Consultancy Services
04.2016 - 05.2020
Bachelor of Technology, Electronics and Communication Engineering -
Punjab Technical University
05.2011 - 05.2015
Awards
- Star Award
- Employee of Month and Multiple ITQ points.
Awards
- Star Award
- Employee of Month and Multiple ITQ points.
Awards
- Star Award
- Employee of Month and Multiple ITQ points.