Summary
Overview
Work History
Education
Skills
Timeline
Certification
Generic

Jinesh Choudhary

Hyderabad

Summary

Dedicated and highly skilled Data Engineering professional with 13 years of industry experience, including 7 years of expertise in Azure Data Engineering, Oracle Cloud Services, AWS Data Services, and Big Data technologies. Strong foundation in Data Warehousing concepts with a proven track record of implementing BI solutions using ETL tools such as IBM Infosphere Datastage and Python during the non-cloud era. Committed to continuously updating IT skills and staying informed about industry trends in Data Engineering and Data Science.

Overview

14
14
years of professional experience

Work History

Associate Manager

Ernst & Young
Bengaluru
06.2021 - Current
  • Involved in designing and developing an omnichannel platform to ingest data from multiple sources like Azure Blob Storage, SFTP locations, Oracle, Teradata etc. to different layers like Azure datalake, Hive and Azure SQL DW as destination using Azure Databricks jobs, Azure data Factory pipelines, SQL procedures and Azure Logicapps.
  • Designed and developed the workload management in Azure Synapse Analytics (formerly SQL DW) as part of Optimization technique for the performance improvement and for prioritizing the workloads.
  • Actively involved in the migration of Apache Kafka pipelines to Azure EventHub to minimize the cost occurring due to Kafka clusters
  • Responsible for setting the deployment process in PowerShell for the promotion of the different components like Azure Databricks, Azure Data Factory, Azure key vaults and Azure Logicapp code from development environment to test and production environment.
  • Involved in setting up the service principle and Managed identity to provision the access in Databricks and ADF pipeline through the Azure Key Vaults to maintain the secret keys, certificates and passwords defined in different scopes.
  • Led the data warehouse/ETL team in developing a prototype and detailed design for a large system using oracle cloud.
  • Developed the data pipelines to ingest data from object storage bucket using Oracle cloud and exposed to BDC spark cluster, Oracle ADW and Oracle EDW.
  • Worked as developer in AWS to pull data from s3 using python to process the tech debt data in ec2 and AWS lambda and ingest it in Postgres to calculate the overall tech debt in the organization. Here exposed to IaC pipelines which were using scalr workspace and terraform to deploy infrastructure.
  • MVP in snowflake to implement the slowly changing dimension and change data capture concepts

Software Engineer

Tech Mahindra
Hyderabad
02.2015 - 05.2021
  • Led the team from onshore and Gathered new business requirement and provided the system and design documents for solution at onsite.
  • Designed and developed the framework in Azure to trigger the multiple dependent pipelines in Azure Data Factory using the Azure logicapps and Azure Queues
  • Involved in setting up the polybase method of data load in Azure Synapse Analytics (formerly SQL DW) using Azure Databricks
  • Led the process of configuring SCIM provisioning for Databricks using Azure Active Directory (Azure AD) to establish the access process within Databricks through azure.
  • Hands-on experience in IBM infosphere DataStage, Unix scripting and Control-m
  • Tuned the long running job by improving the jobs performance and reduced the running time for the Datastage jobs.

Software Engineer

HSBC Software Development Pvt Ltd
Hyderabad
06.2011 - 01.2015
  • Implemented the data warehouse concepts for designing new Datastage jobs, Sequences, UNIX scripts for extracting, cleaning, transforming and loading data into data warehouse for the new business requirement
  • Worked on the different CRs and developed the Datastage jobs for the project
  • I worked in multiple support roles as well, which includes support activities i.e. housekeeping of the servers which includes bouncing the servers when it is required, cleaning the space on servers by deleting the unwanted files, managing support activities raised by users for Cognos reports, PJP Reports this role includes run the business activities i.e fixing the project related L1 issues and bugs that may come up during the production phase, helping the application users in understanding the application.

Education

Bachelor of Engineering - Computer Science

Rajiv Gandhi Technical University Bhopal (M.P.)
05-2011

Skills

  • Azure Databricks
  • Azure Data Factory
  • Azure LogicApps
  • Azure Data Lake
  • Azure Functions
  • Azure Synapse (formerly Azure Data Warehouse)
  • Oracle BDS Cluster
  • Oracle EDW
  • Oracle ADW
  • AWS Lambda
  • AWS EC2
  • PySpark
  • Hive
  • NoSQL (CosmosDB)
  • Snowflake
  • IBM Infosphere Datastage (ETL Tool)
  • Control-M
  • Unix Scripting

Timeline

Associate Manager

Ernst & Young
06.2021 - Current

Software Engineer

Tech Mahindra
02.2015 - 05.2021

Software Engineer

HSBC Software Development Pvt Ltd
06.2011 - 01.2015

Bachelor of Engineering - Computer Science

Rajiv Gandhi Technical University Bhopal (M.P.)

Certification

  • Databricks Certified Data Engineer

Similar Profiles

CREATE PROFILE
Jinesh Choudhary