SHUBHABRATA PAL

Worked in multiple projects as a senior developer with Azure CloudServices( ADF, AKS, Key Vault, ADLS GEN2, Delta Lake , Azure Data Flow, Azure Databricks, AKV, PySpark,Azure Synapse, Unty Catalog).

• Led a team of 10 members.
• As a senior developer, approach to client to understands and gathers the requirements.
• Based on the requirements propose Azure cloud based solutions and convince the client to opt for the solution.
• Designing an effective, optimised and secure solution based on the requirements
• Connect several cloud and on premise sources through ADF Linked Service (SHIR/AIRIR) and Azure Databricks. Get incremental records based on logic and dumps into ADLS Gen2 to form Unified Data Lake.
• Design and building pipeline using Azure Data Factory (ADF), where transformations are being performed using Azure Databricks(Pyspark).
• Perform Data quality Check(DQC) and Business Validation(BV) on the source data to clean and send it to higher layers for further layers. This data are considered to be the input for the data science team to perform featurization and inference model building.
• One more important activity is to identify PII data as take care of that
• Further the data are being used to act as a source for Power BI to generate day to day reports.
• On another project we used, Azure Synapse( Dedicate Sql Pool) to build required datasets for further process and produce report in Power BI.
• Here we took help of Azure Data Flow(Mapping) in Synapse for required transformations.

• Technology Stack: Azure service(ADLS Gen 2, Databricks)
• As a Spark developer I am supposed to convert a TDS Ledger document to an automated spark jobs, that will automatically read file and do all types of complex logic and produce output in another file.
• Worked in the project as a senior developer with Azure Cloud Services, Azure Databricks, PySpark, Python.
• Managed a team of 5 consultants to successfully complete multiple projects within tight deadlines.
• Conducted training of various technical topics to the colleagues.

• Technology Stack: Oracle Sql Developer, Hadoop, Hive, Sqqop, Spark, Scala, OEDQ.
• As a Hadoop developer we need to provide scripts or queries that efficiently able to handle the dataflow. Business team takes this processed data to generate report.
• Handling various data sources like RDBMS, File base data, JSON data.
• Importing and exporting of data between HDFS and RDBMS using SQOOP.
• Creating Hive tables (Internal or External).
• Creating Hive tables with appropriate indexing, bucketing and partitions.
• Creating transformation scripts & developing API using SPARK and SCALA.
• Use ODI as and ELT. Load the transformed data to RDBMS.
• Using EDQ to prepare necessary data and schema for analysis of raw data.

• Technology Stack: ASP.Net MVC, Angular JS, IIS 7.5, SQL Server 2012.
• Designed and implemented user interfaces for web applications using ASP.NET and HTML/CSS.
• Developed complex business logic and data access layers for web applications using C#.
• Utilised AJAX, jQuery, and JSON to develop web applications with a modern, responsive UI.
• Debugged and fixed existing code to resolve production issues.
• As a Web Developer worked i Production release and CR solving.