Venkata Narasimha Murthy Donthala
Senior Data Engineer
Visakhapatnam
Summary
Senior Data Engineer with 7+ years of experience in ETL and Data warehousing using various bigdata technologies like Hadoop, Spark, Kubernetes and EMR. Also, active participant in digital transformation projects, predictive analytics implementation strategies and build robust ETL data pipelines with high data quality standards.
Overview
8
8
years of professional experience
10
10
years of post-secondary education
Work History
EXPERIENCE , Senior Data Engineer
Comcast India Engineering Centre
Chennai
04.2021 - Current
- Working as a Data Engineer in Comcast to process Equipment data for the business for reporting..
- Working on spark to process large amount of data using spark transformations and optimizations to load data for the datalake(raw data journaling layer).
- Have been involved in migration of application from on premise hadoop to cloud databricks for processing.
- Have developed applications for processing PI data in databricks by creating new jobs which are run as jar, and using databricks api command for job run and status retrieval.
- Have worked on AWS cloud platform which involves VPC setup for the team AWS account with the help of platform admin , creation of IAM roles and policies, security groups, running ec2 instances etc.
- Have launched ec2 instances for end to end application run from the cloud ec2 instances.
- Have also worked in containerization of application for process involving non-PI data, and running in Spark on K8 for data processing and loading onto MinIo storage
Data Engineer
Comcast India Engineering Centre
Chennai
02.2020 - 04.2021
- Worked as a Hadoop, Spark developer. Loading comcast equipment data from inventory and biller to various ETL layers.
- Have written scoop scripts to import and export structured data in and out of hdfs.
- Worked on Spark to process large amount of data, performing various spark transformations for processing the data to hive tables.
- Have written Spark application in Scala language using IntelliJ IDE, built the application into jar and deployed the same to run in Hadoop environment.
- Have worked on tuning and optimization of spark jobs through configuration of spark parameters at spark submit level as well as at application coding level.
- Have worked in hive extensively with various hive commands, written hql scripts to perform various etl transformations in Hadoop and performed various optimizations of jobs in hive.
- Have worked on end to end automation of the process by developing Unix scripts and using schedular - uc4.
- Have created images in Docker and pushing of images into docker hub repository, Knowledge of various Docker and K8 commands.
- Have created various frameworks involving cloud and Docker/K8 platform for data export from hdfs to mino and aws.
Software Development Engineer
HTC Global Services
Chennai
07.2015 - 02.2020
- Started my career as production support engineer for Windows based data pipelines
- Got a chance to work in migrating current pipelines to Hadoop based data pipelines
- Imported data from Upstream data base servers to HDFS (Data Mart) using Sqoop
- Performed data standardization, Created Hive External tables and implemented slowly changing dimensions
- Converted existing SSIS scripts into HQL scripts to pull out key business metrics
- Created hive semantic tables for downstream applications and reports
- Day to day support and maintenance of the process and resolve issues.
Education
B.Tech - Electrical, Electronics And Communications Engineering
CITM
Vizianagaram 01.2010 - 01.2014
Intermediate Education - Maths, Physics And Chemistry
Narayana Junior College
01.2008 - 01.2010
Secondary Education -
SSVV School
01.2004 - 01.2008
Skills
Sqlundefined
Timeline
EXPERIENCE , Senior Data Engineer
Comcast India Engineering Centre
04.2021 - Current
Data Engineer
Comcast India Engineering Centre
02.2020 - 04.2021
Software Development Engineer
HTC Global Services
07.2015 - 02.2020
B.Tech - Electrical, Electronics And Communications Engineering
CITM
01.2010 - 01.2014
Intermediate Education - Maths, Physics And Chemistry
Narayana Junior College
01.2008 - 01.2010
Secondary Education -
SSVV School
01.2004 - 01.2008
Similar Profiles
BHARATH MOHANBHARATH MOHAN
DEVELOPMENT ENGINEER 3 at Comcast India Engineering CentreDEVELOPMENT ENGINEER 3 at Comcast India Engineering Centre
Andrews JosephAndrews Joseph
Engineering Manager at Comcast India Engineering CenterEngineering Manager at Comcast India Engineering Center
KARANKUMAR BKARANKUMAR B
Development Engineer 3 at Comcast India Engineering Pvt LtdDevelopment Engineer 3 at Comcast India Engineering Pvt Ltd