Data Enthusiastic, eager to contribute to team success through hard work and have excellent organizational skills. Motivated to learn, grow and excel on day to day basis. Having 7.6 years of experience in IT industry. Enriched with the ability to learn new concepts & technology within a short span of time.
Overview
7
7
years of professional experience
3
3
Certification
Work History
Senior Data Engineer
MAERSK
Bangalore
07.2021 - Current
Compiled, cleaned and manipulated data for proper handling.
Implemented business logic using Databricks and loaded transformed data loaded into Azure Synapse SQL DW using polybase.
Implemented Spark transformations like Posexplode, Windows function, pivot etc.
Wrote Store procedures in Synapse to convert data into appropriate datatypes which helped the downstream teams for reports.
Utilized Synapse distributions like hash,round robin for better performance on data retrievals from table.
Developed, implemented and maintained data analytics protocols, standards and documentation.
Used delta tables in databricks with optimize, zorder for further tuning.
Data Engineer
GENERAL ELECTRIC (GE Healthcare)
Bangalore
09.2019 - 07.2021
Collaborated on building a petabyte scale data warehouse solution and data pipeline on AWS for Healthcare assets and its metadata
Created AWS Glue jobs which compacts data thus providing optimum data storage and integrity
Implemented business logic using AWS Glue and made use of Dynamic Dataframes to carry out operations
Used PySpark for CDC logic and data analysis, improved downstream performance by handing over the data in partition fashion
Implemented Spark transformations like Union, Join, Filter, Coalesce etc
Wrote multiple stored procedures on AWS Redshift by following core principles as distribution key, sort key etc
Senior Product Development Engineer
HARMAN
Bangalore
09.2018 - 09.2019
Analysis of retail data by following fact and dimension framework in Dataware house
Performed data validations on source with target and created Hive queries to load table in desired format as required by clients
Created shell scripts to automate jobs for loading and processing data for fact and dimension load as per Snowflake schema
Wrote hqls to perform hive jobs incorporated with shell scripting
Implemented performance tuning techniques in hive with help of setting up parameters and query optimization, partitioned and bucketed tables and orc with zlib format for table storage.
Senior Systems Engineer
INFOSYS
Bangalore
12.2017 - 08.2018
Migrated MS Sql Server data to AWS RDS cloud
Analysis on Loan services
Created S3 bucket, EC2 instance, EMR cluster based on requirements
Handled meta data of tables using IIG(Data transfer tool) and storing in Amazon RDS Aurora instance
Scheduled DAGs using Airflow scheduler which are created as part of Data pipelining for S3 ingestion and staging in Aurora
Implemented Spark joins, Performance tuning , Data validation and debugged issues.
Systems Engineer
TATA CONSULTANCY SERVICES
Bangalore
12.2014 - 12.2017
Retailer project for day to day analytics
Importing and exporting data into HDFS and Hive using Sqoop
Improved performance of Hive queries with various tuning techniques
Involved in implementing of Spark SQL to improve performance
Involved in bug fixes and support for product when moved to production
Supported testing team (Unit Testing & Integration Testing) in each release and handling their queries