Ganesh Anand Goparaju
Associate
Hyderabad
Summary
Data Engineering professional with 6+ years of experience in building and maintaining stable ETL pipelines that can handle huge amounts of data using a variety of open source and enterprise tools in on-premise Infrastructure under challenging timelines. Self-motivated, Hard Working and Passion Driven. TECH EXCITES ME.
Overview
6
6
years of professional experience
8
8
years of post-secondary education
2
2
Certifications
Work History
Associate
JP Morgan & Chase
Hyderabad
09.2020 - Current
- Created Ingestion, Computation and Validation Spark driven
frameworks to migrate data from various sources to data lake. - Underlying logic has been developed with Java 8 by adhering to spark optimized coding standards to avoid long running jobs in spark.
- Good understanding of CICD deployment pipeline in DEV, SIT and UAT Environment with real time production deployment experience.
- Worked in hybrid environment where legacy and data warehouse
applications and new big-data applications co-existed. - Used Control-M as a tool to design workflows and trigger pipelines.
- Involved in peer reviews in an Agile methodology.
Technology Analyst
Deloitte Consulting
Hyderabad
08.2019 - 08.2020
- Achieved good experience with different types of domains like retail and investment banking
- Implement framework driven validation checks on the data which is generated for reporting which to be provided to the client.
- Validation checks are performed using Spring Boot using Scala and reporting done in Spark Scala
- Implemented a framework to develop an application on Google Cloud Platform in the DataProc Cluster to maintain Customer and Proposition related information and then providing a proper offer or discount to the customer and the data is in BigQuery.
- Using GCP, we have used PySpark as the Big-data framework and implemented the pipelines using Apache AirFlow
- Implemented a framework to develop an application on AWS in the EMR Cluster to maintain data related to different information sources and migrating to Snow flake using spark with Java.
Systems Engineer
TCS
Hyderabad
01.2016 - 07.2019
- Created Ingestion, Computation and Validation Metadata driven frameworks to migrate oracle data to data lake.
- Implemented Ingestion Component using NiFi, Spark for ingesting data into HDFS (from different sources, batch files and file formats) by leveraging metadata mapping placed as part of MariaDB SQL Database.
- Spark as Service: SPARK Job server provides restful interface for submitting and maintaining APACHE SPARK Jobs, jars and Contexts. Implement Spark Job Server to run over 9000+ jobs Jobs on same context.
- Designed and Implemented Computation Component to place metadata Source to Target mapping as part of MariaDB and loaded data into target tables.
- Optimized jobs by caching regularly used Intermittent data that is written to disk to avoid re-computation of DAG for downstream jobs by using shared spark context for all jobs using Spark as Rest Service.
- POC on replaceing the existing AG’s APAMA CEP Engine with the open source Spark Engine for the business transactions data and generate Alerts accordingly to the business requirement within seconds of time.
Education
Master of Technology - Systems And Signal Processing
JNTUCEH
Hyderabad 09.2013 - 10.2015
Bachelor of Technology - Electrical, Electronics And Communications Engineering
JNTU-H
Siddipet 07.2008 - 05.2012
High School - Mathematics, Physics And Chemistry
NJC
Hyderabad 07.2006 - 05.2008
No Degree - High School
BSRKV
Hyderabad 04.2001 -
Skills
Apache Spark
Accomplishments
On the Spot Award in TCS
Certification
AWS Solution Architect
Scuba Diving
Completed 10 Dives of Scuba in Andaman Islands, it was magical to see the creatures of the Ocean.
Work Availability
monday
tuesday
wednesday
thursday
friday
saturday
sunday
morning
afternoon
evening
swipe to browse
Timeline
Advanced Open Water Scuba
12-2021
AWS Solution Architect
06-2021
Associate
JP Morgan & Chase
09.2020 - Current
Technology Analyst
Deloitte Consulting
08.2019 - 08.2020
Systems Engineer
TCS
01.2016 - 07.2019
Master of Technology - Systems And Signal Processing
JNTUCEH
09.2013 - 10.2015
Bachelor of Technology - Electrical, Electronics And Communications Engineering
JNTU-H
07.2008 - 05.2012
High School - Mathematics, Physics And Chemistry
NJC
07.2006 - 05.2008
No Degree - High School
BSRKV
04.2001 -
Similar Profiles
Lisa Olivia GrenviczLisa Olivia Grenvicz
Private Client Banker at JP Morgan Chase & Company (Chase Bank)Private Client Banker at JP Morgan Chase & Company (Chase Bank)
Miranda GuerreroMiranda Guerrero
Associate Banker Market Expansion at Jp Morgan Chase ChaseAssociate Banker Market Expansion at Jp Morgan Chase Chase
COREY L MOSLEYCOREY L MOSLEY
Security Project Manager and Scrum Master at JP Morgan ChaseSecurity Project Manager and Scrum Master at JP Morgan Chase