AMRUTH KUMAR KONDAPALLY
Senior Data Engineer
Summary
Around 10 years of professional IT work experience as an astute Senior Data Engineer with data-driven and technology-focused approach. Communicates clearly with stakeholders and builds consensus around well-founded models. Talented in writing optimized processes and applications. Organized and dependable candidate successful at managing multiple priorities with a positive attitude. Willingness to take on added responsibilities to meet team goals. Good programmer with strong Engineering principles and a knack for code quality.
Overview
12
12
years of professional experience
6
6
years of post-secondary education
Work History
Senior Associate
DBS Asia Hub 2
05.2021 - Current
- Worked varied hours to meet seasonal and business needs.
- Improved customer satisfaction by quickly and effectively addressing inquiries and complaints.
- Defined work plans in alignment with stakeholder requirements.
- Design and develop derived rules (custom rules) created by data stewards to provide special access privileges to specific users.
- Read Presto audit logs residing in ElasticSearch index , parse queries run by users and extract DB, table, user information and write to a Kafka topic
- Automate Data Backup process to read from ElasticSearch index and upload to AWS S3 .
- Generate metrics required by business from Audit data and create Grafana dashboard to show Metrics in a graphical representation.
Associate
JP Morgan Chase
12.2020 - 04.2021
- Created Python scripts to automate 50% of existing manual maintenance and deployment jobs.
- Write Spark Jobs in Scala for migrating data from multiple RDBMS Databases to Cassandra for archival process.
- Helped developers optimize existing Spark jobs for better performance and cluster resource utilization.
- Perform code reviews and add JUnit Tests for existing Projects.
Senior Software Engineer
Quaero, India Pvt. Ltd
05.2019 - 12.2020
- Processed real time data coming from Kafka along with batch data coming from different sources in a Lambda architecture to form attributes and run segmentation using Spark Structured Streaming , Apache Phoenix and HBase
- Create Looker explores dynamically of select few Hive tables
- Write data processing pipelines using Spark in Scala/Java
- Developed python script for automating HDInsight cluster creation in Azure
- Write Airflow DAGs for creating workflows using Python
- Developed features like fetching a segment population size from a total population of 10Million in less than 20 seconds using Apache Livy
- Participate in Agile processes like sprint planning, scrum and retrospective
- Write Jenkins jobs for CI/CD Pipelines
- Write custom Docker images and deployed applications on Kubernetes .
Data Engineer
UnitedHealth Group India Solutions (UHGIS) pvt. Ltd
10.2016 - 04.2019
- Responsible for enabling Predictive Model in a distributed environment using technologies like MaprDB , Spark Streaming , Spark SQL and Hive .
- Developed REST API using Spring Boot for processing, validate inbound data, store in MaprDB, process and create response XML using XStream Parser and write into RabbitMQ.
- Responsible for developing jobs using Spark with Scala for preparing training data for Machine learning models.
- Actively participated in designing data model for storing data in Hive and HBase .
- Optimizing Hive queries by using dynamic partitioning, query optimizations and using different data formats like ORC and Parquet wherever it was required
- Improving existing code in Scala and Java to handle multiple failure scenarios.
- Responsible for writing Bash shell scripts to automate maintenance processes and also to execute different spark and hive jobs.
- Presented new design and operational guidelines to senior engineers, resulting in 30% improvement in task completion.
Technology Engineer
Virtusa Consulting Services Pvt. Ltd
11.2013 - 09.2016
- Responsible for migration of Ingestion code (data extraction, cleaning and loading) from MapReduce to Spark .
- Responsible for Spark job optimization for better performance in ingestion jobs.
- Responsible for improving Hive Query performance which are used for generating custom reports.
- Write Hive/MapReduce jobs to calculate complex metrics asked by Business and write to AWS S3 in Parquet / ORC
- Fix or implementing a new Change request in ingestion and preprocessing using Apache Spark.
- Work on defects raised by Customer on Production Datasets.
- Exporting data from HBase to Hive.
Software Engineer Trainee
Nexwave
05.2013 - 07.2013
- Trained on Java/J2EE, My-SQL
- Prepared and submitted reports and other documentation to assist development team members.
- Collaborated effectively with members of software development team and personnel in other departments.
- Wrote clear, clean code for various projects.
- Documented technical workflows.
- Proofread technical documentation and user manuals.
- Brainstormed with engineering team to determine appropriate code testing processes.
Education
B.Tech - Computer Science
Nishitha College of Engineering (Affiliated To JNTU
Hyderabad, TG 09.2008 - 05.2012
Intermediate -
Narayana Junior College
Hyderabad, TG 07.2006 - 03.2008
SSC -
T.R.R High School
Hyderabad, TG 04.2005 - 05.2006
Skills
Spark (Core,SQL and Streaming), Hive, HBase, Phoenix, Presto, Kafka, Airflow, Livy
HBase, SQL Server, My SQL, ElasticSearch
RabbitMQ, Kafka
Jenkins, Kubernetes, Docker
Java / J2EE Frameworks
Spring Boot
Git, GitHub, Liquibase
Confluence, swagger
Looker, Grafana
Timeline
Senior Associate
DBS Asia Hub 2
05.2021 - Current
Associate
JP Morgan Chase
12.2020 - 04.2021
Senior Software Engineer
Quaero, India Pvt. Ltd
05.2019 - 12.2020
Data Engineer
UnitedHealth Group India Solutions (UHGIS) pvt. Ltd
10.2016 - 04.2019
Technology Engineer
Virtusa Consulting Services Pvt. Ltd
11.2013 - 09.2016
Software Engineer Trainee
Nexwave
05.2013 - 07.2013
B.Tech - Computer Science
Nishitha College of Engineering (Affiliated To JNTU
09.2008 - 05.2012
Intermediate -
Narayana Junior College
07.2006 - 03.2008
SSC -
T.R.R High School
04.2005 - 05.2006