Summary
Overview
Work History
Education
Skills
Certification
Personal Information
Awards
Timeline
Generic

Harsh Prateek Singh

Senior Data Engineer II
Bengaluru

Summary

Transitioning from data-centric environment with focus on developing efficient data solutions and optimizing workflows. Skilled in data architecture, database management, SQL, and Scala/Python, with track record of enhancing data-driven decision-making processes. Seeking to apply these transferrable skills in new field, bringing consultative approach to solving complex problems and improving operational efficiency.

Overview

10
10
years of professional experience
4
4
years of post-secondary education
7
7
Certifications

Work History

Senior Data Engineer - II

Tesco
11.2018 - Current
  • Automated data extraction and transformation, reducing manual effort by 40% and error rates by 30%
  • Designed a scalable data pipeline handling terabytes of customer data, supporting business growth
  • Optimized Spark jobs, reducing processing time by 50% and improving efficiency
  • Collaborated with cross-functional teams, ensuring data accuracy for analytics and machine learning
  • Strengthened data security and compliance, protecting sensitive customer information
  • Directly contributed to £50M in additional revenue through improved analytics and reporting
  • Developed and maintained batch and streaming ingestion pipelines for structured and unstructured data
  • Managed daily data loads, reducing failures by 25% through proactive monitoring and optimization
  • Built late-data handling mechanisms, ensuring seamless processing and accurate reporting
  • Used Sqoop to migrate high-volume data between Hadoop and RDBMS systems (SQL Server, Teradata)
  • Orchestrated GDPR-compliant Spark jobs for automated data deletion
  • Implemented Kafka-based real-time data ingestion with NiFi and Spark Streaming, improving data availability by 35%
  • Unified in-store and online shopping data, enabling a single customer view and increasing personalization
  • Processed complex data formats (XML, JSON, CSV, Avro) to enhance data interoperability
  • Built clickstream and order processing applications, improving analytics insights
  • Architected and executed schema evolution strategies for frequently changing schema
  • Migrated 50+ Oozie workflows to Dolphin Scheduler, enhancing job reliability
  • Built and integrated real-time processing solutions using Kafka and Spark Structured Streaming

Big Data R&D Engineer

Nokia Corporation
09.2018 - 11.2018
  • Planned and executed Spark Scala jobs processing billions of records, optimizing data transformation workflows
  • Improved query performance by 40% using techniques like broadcast joins and repartitioning
  • Ensured reliability with robust error handling, logging, and monitoring mechanisms
  • Integrated Spark pipelines into enterprise ETL workflows, reducing data processing delays by 60%
  • Deployed and scheduled Spark jobs using Oozie and AutoSys, streamlining data operations

Senior Engineer

Mindtree Limited
06.2015 - 08.2018
  • Led the migration of legacy mainframe batch jobs to Spark Structured Streaming, reducing runtime by 50%
  • Engineered and streamlined Kafka-based event-driven data pipelines
  • Fine-tuned Spark code, improving processing speed and resource efficiency
  • Build unit test cases using Scala Test, improving code coverage and reliability
  • Upgraded and maintained SQL applications, improving system stability
  • Debugged and resolved performance issues, reducing downtime by 30%
  • Design, develop, and optimize SQL queries, generating reports and creating views
  • Collaborated with teams to implement new features, increasing system efficiency
  • Develop and optimize ETL processes to extract, transform, and load data from various sources

Education

Bachelor of Technology - Computer Science

UP Technical University
08.2010 - 07.2014

Skills

undefined

Certification

AWS Certified Solutions Architect Associate

Personal Information

Awards

  • Top Performer Award, 2025
  • Team Player Award, 2024

Timeline

Senior Data Engineer - II

Tesco
11.2018 - Current

Big Data R&D Engineer

Nokia Corporation
09.2018 - 11.2018

Senior Engineer

Mindtree Limited
06.2015 - 08.2018

Bachelor of Technology - Computer Science

UP Technical University
08.2010 - 07.2014

Similar Profiles

CREATE PROFILE
Harsh Prateek SinghSenior Data Engineer II