Dhanasekhar Danduprolu
Hyderabad
Summary
Experienced and Microsoft Certified Azure Data Engineer with 4.7 years of experience in designing, developing, and implementing data solutions on the Azure cloud platform. Adept at leveraging Azure services such as Azure Data Factory, Databricks, Data Lake, Synapse Analytics, Snowflake, and a strong background in Python, Pyspark, and SQL. Used control flow tasks like Copy activity, For Each Activity, Get Metadata Activity, If Activity and Lookup Activity. Expertise in gathering business requirements, identifying source, target and applying transformation logic as per the functional requirements. Having experience in connecting different sources like Azure SQL database, data lake storage and blob storage from ADF. Good understanding of the principles and best practices of Software Configuration Management (SCM) in Agile, scrum, and Waterfall methodologies. Worked with delta tables and data frames, pyspark functions and Joins queries. Good understanding in Software Configuration Management, and Agile Methodologies. Extensive experience in all phases of Software Development Life Cycle. Extensive working experience on integration and automation for all deployments. Extensive working experience on logic apps Worked on different types of on-prem source systems like File system, Sql server and RestAPI. Creating Sub programmes Stored procedures, functions and trigger and collections Writing Complex SQL Queries Extensively worked on PLSQL Development
Overview
4
4
years of professional experience
1
1
Certification
Work History
Engineer
DISH Network Technologies India Pvt Ltd
05.2024 - 08.2024
- Migrating the data from On-prem (SAP) to Azure data lake storage
- Setup the Azure Purview Account
- Understand the data governance policies and apply to the metadata
- Scheduling the Incremental & Full Scan as per the requirement
- Monitoring the assets
- Creating Glossaries
- Provide the data Stewart and curator roles as per the customer requirement
- Integrate with Azure services and tools like Azure Data Factory, Azure Data bricks, and Power BI for data analytics and processing
- Regular monitoring for on-going metadata management and data governance
- Understand and amylase the newly added assets and help the business to check the lineage
- Creating ADF Pipeline with copy activity to make mapping between SAP to Azure Data Lake
- Creating ADF Pipeline with Data Flow activity to make transformation between to Azure Data Lake Azure Synapse Pool
- Using copy activity copy the data from oracle as source and using notebook as a sink
- We apply the transformation, parsing the data based on requirement and storing data in to Data Lake
- I use to write the SQL Quires on Azure notebook to analyse the data
- Creating Dashboards and Visualization based on insights that we read from Azure Delta Lake
- Creating Manual and Automatic lineage between Source SAP to Azure Synapse to show table and column level lineage
- Creating Jobs to scan the Azure Synapse, Azure Blob Storage and Azure Data Lake Gen2
Software Engineer
Turbito Infotainment Pvt Ltd
Hyderabad
08.2022 - 04.2024
- Created Pipeline to extract data from on-premises source systems to Azure cloud data lake storage; Extensively worked on copy activities and implemented the copy behaviors such as flattening hierarchy, preserving hierarchy, and Merge hierarchy; Implemented Error Handling concept through copy activity
- Exposure to Azure Data Factory activities such as Lookups, Stored procedures, if condition, for each, Set Variable, Append Variable, Get Metadata, Filter, and Wait
- Configured the logic apps to handle email notifications to the end users and key shareholders with the help of web services activity; created a dynamic pipeline to handle multiple sources extracting to multiple targets; extensively used Azure key vaults to configure the connections in linked services
- Configured and implemented the Azure Data Factory Triggers and scheduled the Pipelines; monitored the scheduled Azure Data Factory pipelines and configured the alerts to get notification of failure pipelines
- Extensively worked on Azure Data Lake Analytics with the help of Azure Data bricks to implement SCD-1 and SCD-2 approaches
- Created Azure Stream Analytics Jobs to replication the real-time data to load to Azure SQL Data warehouse
- Writing complex queries in snowflake
- Implemented delta logic extractions for various sources with the help of a control table; implemented the Data Frameworks to handle the deadlocks, recovery, and logging of the data of pipelines
- Deployed the codes to multiple environments with the help of the CI/CD process and worked on code defects during the SIT and UAT testing and provided support to data loads for testing; Implemented reusable components to reduce manual interventions
Software Developer
Miscot Systems Pvt Ltd
Mumbai
10.2021 - 07.2022
- Created Linked Services for multiple source systems (i.e.: Azure SQL Server, ADLS, BLOB, Rest API)
- Writing complex queries in snowflake
- Creating data pipelines using Azure data factory
- Processing the data using spark on top Azure data bricks
- We do auto scaling for cluster stability
- Write the data in to Blob (which are images and Json data)
- Using Pyspark for some use cases on Azure data bricks
- Using SQL to write the query and make the dashboard using Power BI tool
PLSQL Developer
Lincesoft Pvt Ltd
03.2020 - 09.2021
- Involved understanding project requirements and specifications
- Writing complex SQL Queries using joins and subqueries
- Developed the DB objects(Procedures, Packages, Tables, Views)
- Involved in creation of indexes, Views, Sequences as per design document
- Development of database objects like Cursors, Triggers, Procedures and Functions etc
- Developed, Modified, and optimized complex queries for data retrieval and modification
- Worked for UAT Bug’s
- Interacting with the team and lead to discuss about the issues and standup calls with Functions and BA team
Education
B.Tech - ME
Pragathi Engineering College
Surampalem01.2018
Skills
- Snowflake Data Warehouse
- ETL Development
- Data Modeling
- SQL and Stored Procedures
- Functions & Collections
- PLSQL Development
- Pyspark
- Azure Data Factory
- Azure Databricks
- Azure Synapse
- Python Scripting
- Cloud Platforms (Azure, AWS)
- Data Integration Tools (Informatica)
- Data Visualization (Power BI)
- Performance Tuning and Optimization
- Agile Methodologies
- Data Flows
- Microsoft Fabric
- Logic Apps
- Function Apps
Certification
Microsoft Certified: Azure Data Engineer Associate (DP-203)
Disclaimer
I hereby declare that all the statements made in this application are true to the best of my knowledge and belief.
Languages
English
First Language
Telugu
Proficient (C2)
C2
Hindi
Intermediate (B1)
B1
Timeline
Engineer
DISH Network Technologies India Pvt Ltd
05.2024 - 08.2024
Software Engineer
Turbito Infotainment Pvt Ltd
08.2022 - 04.2024
Software Developer
Miscot Systems Pvt Ltd
10.2021 - 07.2022
PLSQL Developer
Lincesoft Pvt Ltd
03.2020 - 09.2021
B.Tech - ME
Pragathi Engineering College