Uday Vangara
Data Engineer
Bengaluru
Summary
With over 8+ years of experience in the IT industry, specializing in ETL for the past 6+ years. Expertise in gathering, analyzing, developing, and maintaining data integration solutions using Big Data. Successfully executed solutions for complex business problems utilizing ETL On-Prem with Cloudera, HDFS, and various AWS services including Glue, Lambda, S3, Crawlers, and Cloud Watch. Proficient in Python, PySpark, SQL, PSQL, and Unix. Hands-on experience with MPP Databases like Greenplum and OLAP Databases like AWS Redshift. Successfully migrated data from Greenplum to AWS RDS using PySpark jobs and resolved Glue job failures by analyzing issues at AWS Cloud Watch. Sound knowledge of performance tuning for Glue/Spark jobs, optimizing Glue jobs by analyzing their performance through Spark UI. Written multiple Jenkins Pipelines using Groovy script to extract console output information based on business logic using certain patterns and send emails to end users. Notable achievement includes developing a Python automation of a File Monitoring Application that alerts end users via email if client files are stuck in the pre-processing environment according to the configured settings. Successfully converted multiple scripts in PowerShell to Python. Personable Technology Specialist with 2 years' success employing technological expertise to assist businesses. Adept in providing high-quality technical support by troubleshooting, resolving issues and communicating results to the user. Gifted listener and problem-solver with a penchant for determining best plan of action and most effective solutions.
Overview
8
8
years of professional experience
4
4
years of post-secondary education
Work History
Technology Specialist
Broadridge Financial Solutions
Bangalore
06.2023 - Current
- BRCC Report Mining, an archival solution for processing and storing multiple reports using Systemware Application.
- Developed and implemented a Python-based File Monitoring Application that recursively fetches client file information from multiple servers, directories, and AWS, and sends email alerts to end users if files are stuck in the pre-processing environment, based on configurable time frames.
- Developed a Python wrapper to fetch 350 PowerShell scripts from a Windows drive, eliminate duplicates and unnecessary scripts, archive them, and identify 16 master scripts, which were then committed to a Git repository.
- Collaborated with the team member and architect to develop an RM Support Query Tool with a plug-and-play architecture, incorporating multiple features.
- Updated two PowerShell scripts to Python as part of the new "Internal Validation" feature in the RM Support Query Tool, with plans to convert the remaining 14 PowerShell scripts to Python.
- Written multiple Jenkins pipelines using Groovy script to extract console output Information based on business logic using certain patterns and send emails to end users.
Technology Lead
Broadridge Financial Solutions
Hyderabad
12.2022 - 05.2023
- tPR-Lite is the lite application of PRU Hvy and it is a project to migrate on-premises project to AWS Cloud and storing the data into AWS RDS database
- Clients: Ctra, Scmca
- Migrated data from Greenplum database to AWS RDS database with the help of PySpark jobs
- Resolved issue of null datatype of fields while writing to RDS database, JDBC is not accepting null data types
- Optimized long running Glue jobs and analyzed performance of Glue jobs with the help of Spark UI
- Partitioned the data into multiple bucket tables according to client specific
- In this project will directly load the data to AWS RDS database with the help of AWS Data connections and AWS Crawlers to read the RDS database schema
- Identified multiple flaws in AWS Glue, Crawlers, and Informed AWS Support Team to bring them as feature request
Senior Member Technical
Broadridge Financial Solutions
Hyderabad
03.2020 - 11.2022
- PRU Hvy is Performance Reporting Application, which reports the performance of investments
- Client: UBS
- Written multiple PySpark jobs from scratch based on the business logic as per client requirement and store the data into AWS Redshift database
- Created multiple scenarios for unit testing of jobs and tested the jobs according to the requirement and fixed the bugs on it
- Migrated AWS Glue 2.0 to Glue 3.0 and identified multiple issues like adding configuration parameters at Glue parameters, date format pattern changes at Glue 3.0
- Ran multiple batches using UC4 WF with both Glue 2.0 & 3.0 and compared the counts and compared the data fields like date, price or amounts
- Implemented Auto scaling feature in AWS Glue, reduced the cost for job run based on the workload
- Analyzed and fixed multiple bugs in code and sometimes identified issues in data pattern
- Resolved multiple jobs issues related to OOM
- Having sound knowledge on Performance Tuning of Glue/Spark jobs and optimized the Glue jobs and analyzed performance of Glue jobs with the help of Spark UI
- Have resolved multiple Glue jobs failures by analyzing the issue at AWS Cloud Watch
Senior Software Engineer
Mindtree
Bangalore
05.2019 - 02.2020
- We used to analyze historical sales information and generate reports on monthly basis and compare sales between months, compare sales of different product categories, analyze growth, and decline in sales
- Client: Walmart, USA
- Experienced in handling large datasets using partitions, Spark in-memory capabilities, Effective & efficient Joins, Transformations
- Having excellent knowledge in Spark Architecture
- Read multiple formats of source files like CSV, Pipe separated, JSON
- After processed data stored in different file formats like PARQUET and ORC, AVRO
- Performed Data Frames pre-processing like Aggregation, adding, and removing columns, column renaming and sorting, merging, Joins two data frames, type casting column data type
- Resolved multiple Spark jobs issues by debugging the code
- Created multiple scenarios by prepping source files for unit testing of jobs for minimal bugs and written strong code
Software Engineer
Mindtree
Bangalore
02.2017 - 04.2019
- Building a data lake for All State Benefits for everyday policy transactions and processed by using spark and creating dimensions for each of the policy details and storing into Hive
- Client: All State
- Extensively worked on Spark RDD, Spark SQL using Python
- Used Spark SQL to process the huge amount of structured data to perform SQL operations
- Performance Analysis using Spark web UI
- Created Hive external tables to store the results on top of HDFS
- Wrote Spark job to process the semi-structured data and stored into Hive tables
- Importing data from RDBMS to HDFS, Hive and Exporting data from Hive, HDFS to RDBMS by using Apache SQOOP
Education
Bachelors - Electrical Electronics Engineering
JNTUH
Hyderabad 06.2010 - 05.2014
Skills
- Python
- Big Data services like PySpark, Spark SQL, Hive, HDFS, SQOOP
- AWS Services like Glue, Lambda, Crawlers, Cloud Watch, S3, EC2
- Databases like RDS, Redshift, SQL, Oracle, Greenplum
- Groovy Programming Basics
- Basic PowerShell Scripting
- UC4
Awards
Rock Star, Strong player in WMAP PRU Project, Broadridge. Majority of the security load related fixes are mastered in the WMAP PRU journey.
Timeline
Technology Specialist
Broadridge Financial Solutions
06.2023 - Current
Technology Lead
Broadridge Financial Solutions
12.2022 - 05.2023
Senior Member Technical
Broadridge Financial Solutions
03.2020 - 11.2022
Senior Software Engineer
Mindtree
05.2019 - 02.2020
Software Engineer
Mindtree
02.2017 - 04.2019
Bachelors - Electrical Electronics Engineering
JNTUH
06.2010 - 05.2014
Similar Profiles
Tyler WilsonTyler Wilson
Financial Analyst at Broadridge Financial Solutions, Matrix Financial SolutionsFinancial Analyst at Broadridge Financial Solutions, Matrix Financial Solutions
Matthew P Gutierrez Matthew P Gutierrez null
Relationship Manager at Broadridge Financial Solutions, Matrix Financial SolutionsRelationship Manager at Broadridge Financial Solutions, Matrix Financial Solutions
Rosaura AlvarezRosaura Alvarez
Operator at Broadridge Financial Solutions, Matrix Financial SolutionsOperator at Broadridge Financial Solutions, Matrix Financial Solutions