Summary
Overview
Work History
Education
Certification
Timeline
Technicaltools
Technicallanguages
Generic

SURAJ TALEKAR

Lead Data Engineer

Summary

Data Engineer with 4+ years of professional experience and 6 years of education in Computer Science. Experienced in developing and optimizing ETL pipelines, working with cloud technologies such as Azure Data Factory, Azure Synapse, Databricks, and Amazon AWS, and integrating data across multiple platforms. Skilled in data transformation using ABAP, SQL, and PySpark, with hands-on experience in building scalable data solutions using REST APIs and OData. Proficient in setting up and managing data environments, including Self-hosted Integration Runtime for ADF, and utilizing CDM Utils and Synapse Link for real-time data synchronization. Dedicated to achieving career growth through hard work, consistency, and collaboration to help organizations achieve their goals and objectives.

Overview

5
5
years of professional experience
4
4
Certificates

Work History

Lead Data Engineer

ABFRL
06.2022 - Current
  • Designed and implemented a data architecture and created a metadata driven generic ETL template using Delta Live Tables (DLT), Azure Data Factory (ADF), and SQL in Databricks, streamlining the ETL process for all data sources.
  • Spearheaded the migration of over 50 SharePoint lists to Databricks using Data Lake, delta lake, Databricks notebooks ,Databricks workflows, improving data processing performance and reducing data read times significantly.
  • Led the adoption and integration of Unity Catalog to enhance platform modernization and streamline data governance within Databricks.
  • Worked extensively with advanced Databricks technologies, including Delta Live Tables (DLT), to modernize ETL workflows, improving data reliability, quality, and processing efficiency.
  • Led the team of 4 members to integrate data from 70+ surveys using a Delta Live Tables (DLT) template, utilizing modern tools like Azure Data Factory and Databricks for efficient data processing.
  • Developed and implemented efficient Python code to integrate encrypted CSV files using the PGP public-private key protocol, ensuring secure and reliable data handling.
  • Gained expertise in selecting optimal cluster types for development, testing, and job runs to optimize performance and reduce execution costs.
  • Utilized Azure Logic Apps to automate notifications and failure alerts for task/pipeline failures, enabling quick action to minimize the impact of any issues.
  • Identified areas for improvement within existing ETL frameworks to maximize efficiency without sacrificing accuracy or integrity of results produced during transformations applied on ingested datasets.
  • Set up and configured Self-hosted Integration Runtime for Azure Data Factory (ADF), ensuring high availability by deploying multiple nodes to improve reliability and performance of data pipelines.
  • Worked with CDM (Common Data Model) Utils in Azure Synapse to standardize and manage data models for better integration and data consistency across platforms.
  • Leveraged Synapse Link to enable real-time data synchronization between Azure Synapse Analytics and operational data stores, improving data accessibility for analytics and reporting.


Software Engineer/ETL Developer

Company: GERP Tech.
12.2020 - 05.2022
  • Gained 1 year and 4 months of hands-on experience as an ETL Data Engineer, working extensively with technologies such as SSIS and Teradata SQL for the development of ETL processes to load data into the data warehouse.
  • Utilized SSIS to extract data from various sources and load it into the data warehouse, and leveraged Teradata SQL to transform the data according to project requirements before loading it into production tables.
  • Deployed ETL packages in SSMS (SQL Server Management Studio) using Visual Studio, performing scheduling and monitoring of deployed packages, and providing support for failed ETL jobs to ensure minimal disruption.
  • Developed strong expertise in working with relational databases in a business environment, with a focus on writing complex SQL queries for data extraction, transformation, and analysis.
  • Gained practical experience with the Azure cloud platform, developing ETL pipelines using PaaS services like Azure Data Factory and Azure Synapse, and leveraging technologies such as PySpark and Spark SQL for data processing and transformation.

SAP ABAP Consultant/ Junior Data Engineer

Company: PEOL Technologies.
06.2019 - 08.2020
  • Worked as an SAP ABAP Consultant for 1 year and 2 months, focusing on data extraction, transformation, and reporting using ABAP and SQL to manage and process data from SAP systems.
  • Developed ABAP reports and Smart Forms to create custom data solutions, ensuring accurate and efficient data processing for reporting and business needs.
  • Built OData services to enable data access between SAP systems and external applications, improving data integration and flow.
  • Created and maintained REST APIs for the e-invoice portal using Python and Amazon AWS, allowing smooth data exchange between the backend and external systems.
  • Worked with SQL to extract, move, and manipulate data across different systems, supporting data migration and integration tasks.

Education

Diploma - Computer engineering

Maharashtra State Board Of Technical Education

BE - Information Technology

University Of Mumbai

Certification

Python/SQL

Timeline

Lead Data Engineer

ABFRL
06.2022 - Current

Software Engineer/ETL Developer

Company: GERP Tech.
12.2020 - 05.2022

SAP ABAP Consultant/ Junior Data Engineer

Company: PEOL Technologies.
06.2019 - 08.2020

Diploma - Computer engineering

Maharashtra State Board Of Technical Education

BE - Information Technology

University Of Mumbai

Technicaltools

  • Azure Data Factory
  • Azure Databricks
  • Azure Synapse
  • SSIS
  • MS Azure Storage Explorer
  • Teradata SQL Assistant
  • SQL Server Management Studio
  • MS Visual Studio


Technicallanguages

  • PYTHON
  • PySpark
  • SQL
  • JAVA
  • SAP ABAP

Similar Profiles

CREATE PROFILE
SURAJ TALEKARLead Data Engineer