Deepjyoti Mitra

Tech stack: Azure Data Factory, SQL Server, Data Lake, Databricks Pyspark/SQL, Python

Responsibilities:

• Designing ETL pipelines using Azure Data Factory.
• Building complex data transformation logic using Python, Databricks Pyspark/SQL.
• Develop Stored Procedure in SQL Server.
• Troubleshoot complex issues, data analysis, co-ordinating with multiple stakeholders & provide fix.
• Design CI/CD pipelines for deployment.

Tech stack:

Azure: Data Factory, Data Lake, Databricks Pyspark, Spark SQL, Logic App

MSBI: Microsoft SSIS, SQL Server, Power BI.

Programming: Python

Responsibilities:

• Design ETL pipelines using Azure Data Factory.
• Data transformation using Databricks Pyspark/Spark SQL.
• Automation using Logic App.
• SQL tuning for long running query’s on a regular interval in production.
• Production deployment using Azure DevOps CI/CD.
• Design ETL solutions using SSIS, SQL Server.
• Design dashboards and visualizations using Power BI.
• Sensitive Data Analysis and System Improvement Program.
• Implementing GDPR policy to protect customer’s PII information.
• Resolve and troubleshoot complex issues, bug analysis, related product deficiencies and recommending process improvements.
• Provision VM, ADF, databricks & it's cluster configuration, SQL Server, Data Lake, key kault, configuring SSIS IR, IAM, ACL.
• Project development documentation.

Tech stack: UNIX, Oracle PL/SQL.

Responsibilities:

• Develop batch scripts using Unix Shell Scripting and Oracle PL/SQL.
• Transform & load data using grep, awk, sed, tr, ssh, sftp, sqlldr etc.
• Create proof-of-concepts to demonstrate viability of solutions.
• Configure .profile, install/configure packages using rpm, mounting NAS volume, configuring secured SSH connection.
• Server health status automation.