Data engineer with 9+ years of experience with expertise in AWS cloud data engineering, Snowflake, SQL Server, Python, and Pyspark and proficient in creating dashboards and analytics with Tableau.
Overview
10
10
years of professional experience
Work History
Associate
JPMorganChase
Bangalore
06.2023 - Current
Worked on the Data Platform for Merchant Services to ingest data from multiple sources, like Kafka, MFTS, etc., and manage a data warehouse for various SCD types on Databricks hosted on AWS.
Developed an export functionality to move data from Databricks to Snowflake for consumer teams to generate reports and use for analytics.
Developed multiple Glue, Lambda, EMR, and Step Functions for improving and stabilizing the Data Platform.
Provided direction, guidance, and support to team members whenever required
Senior Data Engineer
Carelon Global Solutions
Bangalore
07.2021 - 06.2023
Company Overview: Formely Legato Heath Tech.
Designed a user-friendly solution which proactively identified additional opportunities for recovery or prevention of claim level fraud, waste, abuse and overpayments
Leveraged AWS Lambda to automate various events, such as crawlers runs, step function trigger within the AWS ecosystem.
Leveraged Snowpark API and Spark Snowflake connector to extract and import data from AWS S3 and Snowflake, transforming it into a specified format for storage in S3 or loading into Snowflake tables for analysis
Utilized either Glue Crawler or DDL in Athena to trigger a Lambda Function for creating tables in Glue Catalog.
Used AWS CloudWatch to monitor the logs
Used AWS Simple Notification Service to send the email via notifications to the subscribers
Employed AWS StepFunctions to automate and orchestrate end-to-end workflows.
Utilized Pyspark within the AWS Glue ecosystem to process and then persist the final data either into AWS RDS, Snowflake, or a windows server path.
Spearheaded efforts to optimize code performance, achieving faster execution times while reducing costs for product owners.
Developed and implemented reusable frameworks to streamline workflows and increase efficiency
Business Technology Analyst
Deloitte Digital
Hyderabad
11.2018 - 07.2021
Gathered business requirements, analyzed and designed software solution based on the same.
Employed advanced data extraction techniques to efficiently retrieve data from Facebook, Campaign Manager, and Basis data sources. Processed and analyzed the data in Snowflake to uncover valuable insights for the client.
Collaborated in the creation of essential database objects such as tables, views, procedures, and functions using T-SQL to establish structure, provide definitions, and ensure efficient data management.
Developed Redshift queries to facilitate the loading of data into DW tables, utilizing S3 as the transient storage, and implemented the Merge scenario within the Customer Data platform architecture.
Senior Systems Engineer
Infosys Ltd.
Hyderabad
10.2015 - 10.2018
Possessed 3 years of experience in designing, analyzing, developing, and implementing automation for various application environments using BI tools such as ETL, database management, and dashboarding.
Demonstrated expertise in data modeling, analysis, ETL, data warehousing, data visualization, and data migration projects.
Utilized optimal ETL techniques, with extensive experience in extracting, transforming, and loading data from various sources into target systems using SSIS and SQL Server 2012.
Developed engaging and informative dashboards, data visualizations, and analytics using Tableau.
Education
Bachelor of Technology -
National Institute of Science And Technology
Berhampur
05.2015
Higher Secondary School -
Chinmaya Vidyalaya
Bokaro Steel City
05.2010
Skills
Python
SQL
TSQL
Terraform
Pyspark
ETL
Data Modelling
Data Warehousing
Databricks
AWS Glue
Data Catalog
AWS Lambda
Step Functions
AWS Athena
AWS Redshift
SNS
SQS
Event Bridge Rule
Cloud Watch
Snowflake Datawarehousing
Snowpark API
Talend
Tableau
Version Control
Bitbucket
Accomplishments
Impact Go Above Award, Carelon Global Solutions, 02/01/23, Implemented the data foundation layer and developed a tokenization solution, successfully handling the processing of multiple months of data for all application analytics.
Impact Go Above Award, Carelon Global Solutions, 09/01/22, Developed and implemented a Python-based deployment process that replaced dependencies on Bamboo for script deployment.
Spot Award, Deloitte Digital USI, 05/01/21, Pioneered the implementation of a solution in Snowflake for a client using unstructured JSON data, resulting in increased efficiency and cost savings.