Summary
Overview
Work History
Education
Skills
Timeline
AccountManager
Pawan Negi

Pawan Negi

Data Engineer
New Delhi

Summary

A business professional having more than 3 years of experience in Information Technology with expertise in Big Data development working on various Big Data tools such as HIVE, SQOOP, SPARK(using Python), AWS with good understanding of HDFS.

Overview

3
3
years of professional experience
4
4
years of post-secondary education

Work History

Senior Consultant

Birlasoft
New Delhi
04.2022 - 09.2022


  • Designing and configuring end to end Data Pipeline from On-premise servers and cloud databases as well
  • Developing and unit testing ETL Python transformations at various layers in AWS Glue using pyspark, pandas, numpy libraries
  • Deploying, managing and configuring AWS RDS instances and Redshift clusters
  • Establishing and configuring connections through different types of APIs with Lambda functions in AWS and ingested data into S3
  • Configuring and managing EC2 instances in public and private subnets.
  • Developed Step functions to orchestrate the AWS glue jobs in a sequential order and glue triggers, cloudwatch events if needed to execute at a scheduled interval.
  • Implemented and configured CI/CD pipelines across different environments using AWS CodeCommit, CodeBuild, CloudFormation and CodePipeline.
  • Implemented email notification mechanism on failure/succession of a AWS Glue job or a lambda function using AWS SNS and AWS EventBridge rules.

Application Development Analyst

Accenture Solutions Pvt Ltd
New Delhi
09.2019 - 03.2022


  • Developed ETL processes in AWS Glue to migrate data from on-premise SQL servers into AWS environment
  • Developed AWS Glue JDBC connections to ingest data from Oracle/SQL servers into AWS S3
  • Unit testing of transformations developed via SQL on AWS Athena
  • Configuration of CICD pipeline in AWS among different environments
  • Deploying and managing EC2 instances and resolving security vulnerabilities over them.
  • Used Sqoop to import large data from traditional RDMS to HDFS and also handled incremental Sqoop.

Education

Bachelor Of Technology - Electronics And Communications Engineering

Maharaja Agrasen Institute Of Technology, Guru Gobind Singh Indraprastha University
New Delhi,India
08.2015 - 07.2019

Skills

Python

undefined

Timeline

Senior Consultant

Birlasoft
04.2022 - 09.2022

Application Development Analyst

Accenture Solutions Pvt Ltd
09.2019 - 03.2022

Bachelor Of Technology - Electronics And Communications Engineering

Maharaja Agrasen Institute Of Technology, Guru Gobind Singh Indraprastha University
08.2015 - 07.2019

Similar Profiles

CREATE PROFILE
Pawan NegiData Engineer