Ashish Kumar Patra

Azure Data Engineering Project:

• Developed ADF Pipelines to load data from various sources such as Oracle, SQL Server, REST APIs, GraphQL APIs, and the File system into Azure Data Lake Gen2, SQL, PostgreSQL, and S3 Bucket destinations. Configured auditing tables in Azure SQL Database.
• Utilized Dataflow and Databricks Notebooks within Azure Data Factory for data transformation tasks.
• Established External and Internal tables in Azure SQL Data Warehouse employing distribution methods like ROUND_ROBIN, HASH, and REPLICATE, alongside Polybase for the data serving layer.
• Set up Event Hub and IoT Hub, along with Azure Stream Analytics jobs for real-time data processing, directing outputs to Azure Data Lake Storage and SQL Server destinations.

Azure Databricks Project:

• Set up Azure Databricks from scratch, which entails creating clusters, configuring access policies, managing the workspace, Azure Storage, Blob Storage, and Databricks table mounting, as well as upgrading Databricks runtime versions.
• Additionally, developed Databricks libraries, Databricks jobs, and various notebooks for performing data transformations using Spark, Python, Scala, and SQL.

Azure DevOps Project:

• Set up DevOps, including repository creation, access group creation, user onboarding, policy and rules creation, and project management using Azure DevOps Board, following the best DevOps practices provided by Microsoft.
• Established DevOps frameworks for multiple clients and projects, initiated CI/CD pipelines for Azure Data Factory (ADF), Databricks (JAR Library, Notebooks), Infrastructure as Code (IaaC), Azure Stream Analytics, Azure Functions, and facilitated App Service and WordPress website deployments.

Azure Vision AI Project:

• Developed an API utilizing Azure Vision AI Form Recognizer to extract JSON data from various invoices and receipts.
• Integrated a custom model for a distinct receipt template type, extracting, transforming, and storing the data in JSON format within both Azure Data Lake Storage (ADLS) and SQL Server.
• Successfully deployed the API within an Azure Function App.

Time Series Insights (TSI) to Azure Data Explorer (ADX) Migration Project:

• Migrated TSI data to ADX using Kusto Query Language (KQL).
• Created views and various graph representations of data in ADX.

AWS to Azure Migration Project:

• Migrated AWS Lambda Functions to Azure Functions and created a Standard Operating Procedure (SOP) document accordingly.
• Transferred AWS S3 bucket contents to Azure Storage.

ETL using Python Project:

• Developed a Python web application that parses bank statements in PDF format and extracts the bank information into JSON format.
• Initially, individual parser scripts were created for each bank. Subsequently, a generic parser script was developed and deployed in a Docker container, enabling extraction of data from multiple bank statements.
• Additionally, the web application was designed to be configurable and capable of plug-and-play functionality for new bank statements. Python's class and object concepts were utilized, allowing for the creation of new classes by extending the base class according to the requirements of new bank statements.

Project : Automated Trading using Predictive Analysis

• Developed python modules (and some C++ functions) for detecting varieties of chart-patterns and harmonic analysis patterns on minute level EUR/USD data.
• Besides this, performed other ancillary support tasks for the development of other AI/Machine Learning based modules performing the same.
• Also implemented a web-scrapper that performed sentiment analysis from Twitter to gauge the trends of popular crypto currencies (Bitcoin, Ethereum, Litecoin).