Project: 01
Technologies: Python, Databricks, SQL, PySpark, Microsoft Azure Cloud, SQL Server, Pandas, MySQL, Power BI, GIT, Azure DevOps
Functional Responsibilities:-
- Understand and analyze customer and business requirements.
- Create, design, develop, test data pipeline by using Azure Data Factory.
- Monitor datatpipeline by using power BI dashboard.
- Daily activities include development, coding, test, deploy.
- Search new ways to get data sources and assess their accuracy.
- Perform data quality assurance, quality checks and suggest improvements to improve data quality.
- Building baseline modes for requirements with necessary data preparation.
- Importing data from various sources using numerous APIs.
- Perform exploratory data analysis on data to pinpoint trends, find patterns.
- Coordinate with various technical/functional teams to implement and monitor results.
- Validated already developed python reports, fixed identified bugs and re-deployed the same.
- Attending weekly meetings to discuss the project milestones.
- Writing SQL queries to work with data.