Highly competent Data Engineer with nearly 4 years of experience in building and implementing efficient ETL workflows using Azure Data Factory and Azure Databricks. Skilled in big data processing with PySpark. Possess strong skills in designing, developing,optimizing and maintaining data management systems. Adept at enhancing query performance and optimizing complex database systems. Seeking to apply technical expertise to contribute to a dynamic data-centric environment while continuing to grow as a Data Engineer.
Overview
4
4
years of professional experience
1
1
Certification
Work History
Data Engineer
Tata Consultancy Services (TCS)
11.2023 - Current
Designed conceptual, logical, and physical data models and data mapping templates. Developed data pipelines for efficient data extraction and transformation process.
Implemented database backup and recovery strategies to ensure data protection, availability, and business continuity.
Enhanced database performance through advanced SQL query optimization and tuning, resulting in faster data retrieval and improved application efficiency.
Built stored procedures and functions in multi-database environments to support data analysis and automate processes like data archival and purging.
Conducted comprehensive unit testing for stored procedures and functions to validate logic, data integrity, and business rule compliance.
Assisted in designing scalable solutions to handle increasing data volumes efficiently.
Managed data synchronization across multiple databases using SYMDS.
Utilized the Dataloader utility to import XML and JSON data into databases, optimizing data ingestion workflows.
Designed, developed, and maintained database architectures aligned with business requirements to ensure scalability, performance, and data integrity.
Refactored legacy database schemas by applying normalization principles, enhancing scalability and achieving a 40% boost in query performance.
Designed and implemented scalable and efficient data pipelines using Azure Data Factory (ADF) for orchestrating complex data workflows across cloud and on-premise systems.
Designed and automated robust ETL workflows using PySpark on Azure Databricks, resulting in a 30% reduction in pipeline runtime and enhanced data reliability.
Utilized Azure Data Lake Storage (ADLS) for storing and managing large volumes of structured and unstructured data in a scalable cloud environment.
Developed and maintained real-time and batch data pipelines using Delta Live Tables (DLT), improving pipeline reliability and reducing manual intervention by 40%.
Configured and managed Delta Sharing for secure, scalable data exchange with external partners, improving collaboration without data duplication.
Developed interactive visualizations and reports using Apache Superset to support business intelligence and analytics initiatives.
Integrated Azure SQL into ETL workflows for reliable data processing and analytics in a cloud environment.
Project: TCS Optumera
Education
Bachelor of Technology -B.Tech - Dept of Civil Engineering
SRK Institute of Technology
Skills
Pyspark
DeltaLake
Azure Data Factory
Azure Databricks
Azure Datalake
PostgreSQL
MySQL
SQLServer
Azure SQL
PowerPoint
Word
Excel
GitLab
Certification
DP 900 certified Data Engineer
Award of Appreciation for enabling SAAS for a client (TCS Omnistore)
Service and commitment award from TCS
Special initiative award from TCS RSI
Languages
English
Proficient
C2
Telugu
Proficient
C2
Hindi
Beginner
A1
Tamil
Elementary
A2
Timeline
Data Engineer
Tata Consultancy Services (TCS)
11.2023 - Current
Azure Data Engineer
Tata Consultancy Services (TCS)
01.2022 - 11.2023
Bachelor of Technology -B.Tech - Dept of Civil Engineering
Business Analyst at Diligenta (TCS- UK Subsidiary), TATA CONSULTANCY SERVICES(TCS) PVT.LTDBusiness Analyst at Diligenta (TCS- UK Subsidiary), TATA CONSULTANCY SERVICES(TCS) PVT.LTD