Swarna Latha Modem
Azure Data Engineer
Bangalore
Summary
- Highly experienced Data Engineer with 8 years of professional experience in developing secure, cost-efficient data solutions.
- Experienced Data Engineer with a passion for problem-solving and a track record of delivering successful data projects. Expertise in designing and implementing data solutions in the Azure Cloud using ADF,ADB,ADLS,PYSPARK. Looking to join a dynamic team to continue developing technical and personal skills.
- Having 3 years of experience in software development using Azure Data Factory, Azure Data Bricks, Azure KeyVault, ADLS Gen 2, Blob Storage,Pyspark.
- Having 5 years of experience in ETL Tool Informatica Power Center and 2 years of experience in C and perl language.
- Hands on experience on Azure Data Factory V2.
- Hands on experience on ADF activities such as Copy, Stored Procedure, Look-up and Data Flow activities.
- Hands on experience on Databricks, Databricks Workspace User interface,Managing Databricks Notebooks.
- Hands on experience on Creating ADF Pipeline for Full Load and Incremental Load.
- Hands on experience on Creating the linked services, datasets for the different services.
- Hands on experience on installing the self-hosted Integration runtime to migrate data from on premise to cloud and creating dynamic pipelines.
- Processed flat files in various file formats CSV and Parquet etc.
- Having experience to transform data in Azure Data Bricks.
- Experience in Designing Data Lake storage layer (Landing/staging, raw, trusted/curated zones)
- Created Azure Data Factory V2 Linked Services to connect to different sources like on premise SQL server, blob storage and SQL Data warehouse.
- Good knowledge on Data Warehouse Concepts.
- Extensively worked on Informatica Designer Components – Source Analyzer, Transformation Developer, Mapplet and Mapping Designer.
- Emphasized on performance tuning in Informatica Power Center.
- Hands on experience with complex mappings from varied transformation logics like connected lookups, Router, Aggregator, Joiner, and Update Strategy.
Overview
12
12
years of professional experience
Work History
Azure Data Engineer
Capgemini
06.2022 - 12.2023
- Worked on Azure services like ADF, ADB and Pyspark.
- Collaborated with cross-functional teams for seamless integration of data sources into the company''s data ecosystem.
- Managed cloud-based infrastructure to ensure optimal performance, security, and cost-efficiency of the company''s data platform.
- Increased efficiency of data-driven decision making by creating user-friendly dashboards that enable quick access to key metrics.
- .Fine-tuned query performance and optimized database structures for faster, more accurate data retrieval and reporting.
Azure Data Engineer
CTS
12.2021 - 05.2022
- Implemented real-time data processing solutions for timely decision-making and improved customer experience.
- Enhanced data quality by performing thorough cleaning, validation, and transformation tasks.
- Designed scalable and maintainable data models to support business intelligence initiatives and reporting needs.
- Collaborated on ETL (Extract, Transform, Load) tasks, maintaining data integrity and verifying pipeline stability.
IT Analyst
TCS
01.2016 - 11.2021
Worked on Informatica development projects and also support projects.
Project Engineer
Wipro
03.2012 - 12.2015
Worked on C language and perl scripting.
Education
Bachelor of Engineering - Electrical, Electronics And Communications Engineering
Sir Vishveswaraiah Institute of Science And Techno
Madanapall 04.2001 -
Skills
Microsoft SQL server
Informatica Power Center versions 8,9 &10
Azure Data Factory
Azure Databricks
Pyspark
Data Warehousing
ETL development
Data Pipeline Design
Data Analysis
SQL Programming
Data Migration
Companies Worked
- Wipro, March 2012 – Dec 2015
- TCS, Jan 2016 – November 2021
- CTS, Dec 2021 - May 2022
- Capgemini, June 2022 – Dec 2023
Projects
Harmony, Piramal,Glenmark,Sun pharma etc., June 2022 - till now, Azure Data Factory Developer, Agile (Scrum), SQL Server, Azure Data Factory V2, Azure Blob Storage, ADLS Gen2,Pyspark., Attending the DSM (Daily Stand up Meetings) to update the task status, Installed Self hosted IR in on premise machine and created the connection from ADF, Created Azure Data Factory V2 Linked Services to connect to different sources like on premise sql server, blob storage and SQL Data warehouse., Ingested data to Azure Storage services like ADLS Gen2 using ADF and processed the data in Azure Databricks ., Transformed the data in Azure Databricks using Pyspark notebooks., Designed and implemented highly performant data pipelines from multiple sources using Azure Databricks, Created triggers in azure data factory V2 to run pipelines on scheduled time, Created the dynamic pipelines to copy multiple tables using dynamic parameters. Lifestyle, Landmark Group, December 2020 to May 2022, Azure Data Engineer, ADF, ADB, PYSPARK, Good Experience working on ingesting various file formats like CSV,Parquet, Structured and non-structered data., Experience in performing Design,Development using Azure Services(Data Factory, databricks, Pyspark,SQL).., Develop and maintain scalable data pipelines and build new data Source integrations to support increasing data volume and Complexity., Designed and implemented highly performant data pipelines from multiple sources using Apache Spark and/or Azure Databricks OKR Tool, Swiss Re, Sep 2018 to November 2020, ETL Developer, Prepared Technical document and Operational document for the OKR Tool., Frequent calls with Vendors and Business Stakeholders., Involved in Development and support of Informatica job for the Jira Integration to the OKR Tool., Implemented ADO Plugin for the ADO integration to the OKR Tool., Guiding the users for implementing OKR’s in their Projects., Developed mappings and workflows for the Jira Integration. US Payroll, HBC, Mar 2016 - Aug 2018, Developer, Developed Informatica mappings and workflows for Incremental load., Worked on different sources and Targets like Oracle and Flatfiles., Worked on flatfiles, unix,oracle and clearcase., Prepared testcase documents,KT documents and technical design documents., Performed Unit testing Component Tracking System, Cisco, March 2012 to Jan 2016, Developer, Perl ,Oracle and Unix, Helps in web development part of CTS., Resolved some perl related bugs., Developed some standalone Perl scripts., Analysing the project and fixing the Bugs.Testing
Disclaimer
I hereby declare that the information given above correct to the best of my knowledge.
Timeline
Azure Data Engineer
Capgemini
06.2022 - 12.2023
Azure Data Engineer
CTS
12.2021 - 05.2022
IT Analyst
TCS
01.2016 - 11.2021
Project Engineer
Wipro
03.2012 - 12.2015
Bachelor of Engineering - Electrical, Electronics And Communications Engineering
Sir Vishveswaraiah Institute of Science And Techno
04.2001 -