Summary
Overview
Work History
Education
Skills
Work Availability
Languages
Timeline
Generic
Venkateswara R Vallampudi
Open To Work

Venkateswara R Vallampudi

Senior Data Engineer
Hyderabad

Summary

Results-driven Data Engineer with over 10 years of experience in designing, building, and optimizing scalable data pipelines, cloud-based solutions, and analytics platforms. Expertise includes implementing data pipelines, ETL development, data warehousing, real-time streaming, and navigating big data ecosystems. Proven ability to collaborate with cross-functional teams to deliver actionable insights that drive business growth. Seeking opportunities across Europe to leverage skills in cloud-based solutions and drive innovation in scalable data architectures.

Overview

10
10
years of professional experience

Work History

Senior Data Engineer

Tata Consultancy Services
08.2023 - Current
  • Client: London Stock Exchange Group, UK
  • Tech Stack: Pyspark, Snowflake, Snowpark, AWS (S3, EC2, RDS, lambda, Cloud watch), Hadoop (HDFS, Hive, Impala, HBASE, Oozie, Zookeeper), Jenkins, Kubernetes
  • Migrated legacy on-prem Applications (Trade Performance Reports) to AWS architecture, reducing infrastructure costs. Major contributor in re-writing the applications in AWS
  • Designed and implemented scalable Data pipelines to handle data from various sources like Files, API calls and various databases.
  • Developed real-time data streaming pipelines using snowflake for data Ingestion
  • Enhanced existed API code to be utilized by larger clients

Senior Data Engineer

Mphasis Limited
04.2022 - 08.2023
  • Client: JPMC, USA
  • Tech Stack: Pyspark, AWS (S3, EC2, Glue, Athena), Hadoop (HDFS, Hive, Tez, HBase), Jules, Oracle, Teradata, Control-M, Unix
  • Built and optimized Data pipelines (Spark-Scala) to transfer the data from Oracle to Teradata
  • Implemented reconciliation process to avoid manual validations and in turn it reduced manual efforts of 1 hour to 5 minutes
  • Improved SQL query performance, reducing reporting runtime from 2 hours to 15 minutes.
  • Analyzed the data chucks which are not been utilized by the client and moved it to Glacier

Data Engineer

Optum Global Solutions
07.2020 - 04.2022
  • Client: United Health Care, USA
  • Tech Stack: Spark-Scala, sparkSQL, Hadoop (HDFS, Hive, HBase), Jenkins, TWS - Scheduler, Unix
  • Designed and maintained spark-scala based extracts to generate the business level reports from the data which reside on Datalake. I contributed majorly in this by using Data frames and Datasets to implement the core business logic
  • Generated the feeds which will be used by US State Governments officials. Directly worked with vendor to understand the requirements along with BA and Customized the reports / data feed based on the client requirement
  • Worked with multiple teams (Mainframe and DB teams) to deliver the output in customer requested formats like standard 834 or Custom 834 formats.
  • Maintained cross team collaborations effectively to deliver the things in smoother way
  • Conducted data quality checks to ensure accuracy and consistency.

Data Engineer

Capgemini
11.2018 - 06.2020
  • Client: DBS Bank, Singapore
  • Tech Stack: Pyspark, Spark-Scala, sparkSQL, Hadoop, Airflow, AWS S3, Jenkins, Unix
  • Plays a major role in the implementation of Cash in Transit (CIT) project – Pyspark based application which generates the data feed from ATM, BTM and CRS machines to be utilized by data scientist to generate the forecast of these Machines
  • Collaborated with Data Scientists and Analysts to deploy ML-ready datasets for predictive models.
  • By creating the custom DAGs in Airflow, scheduled these jobs in the server and reduced major manual efforts
  • Migrated over 160+ reports from Teradata to PySpark

ETL Developer

United Health Group
07.2015 - 11.2018
  • Client: United Health Care, USA
  • Tech Stack: DataStage, Talend, Hadoop (HDFS, MapReduce, Sqoop, Hive, HBASE), Unix
  • Designed and maintained ETL pipelines using DataStage/Talend.
  • Migrated some reports from DataStage to Talend using the big data features in Talend
  • Migrated existing DataStage applications to Hadoop based approach (Sqoop- Hive).
  • Involved in several POC’s to implement the low cost-based application to migrate the existing application
  • Learned multiple technologies like Talend, Hadoop, DataStage to meet the project requirements and to become vital player in the team
  • Involved in implementation of DDL and DML scripts to transform Data and populate in data warehouse tables (Fact tables and Dimension Tables)

Education

Masters in Computer Applications -

JNTU
Kakinada
09.2014

Skills

  • Python, SQL, Scala, Java and Shell scripting
  • Spark (Pyspark, Spark-scala, SparkSQL)
  • AWS – S3, EM2, EMR, Glue, Athena, Lambda, CloudWatch
  • Hadoop - HDFS, Sqoop, Hive, MapReduce, HBase, Oozie, Zookeeper, YARN
  • Snowflake, Snowpark, Databricks
  • Airflow, Talend, Datastage
  • Teradata, Oracle, PostgreSQL, MySQL
  • Git Version Control, Maven, SBT
  • CI/CD, Jenkins, JIRA, Kubernetes, Docker
  • Big data processing

Work Availability

monday
tuesday
wednesday
thursday
friday
saturday
sunday
morning
afternoon
evening
swipe to browse

Languages

English
Advanced (C1)

Timeline

Senior Data Engineer

Tata Consultancy Services
08.2023 - Current

Senior Data Engineer

Mphasis Limited
04.2022 - 08.2023

Data Engineer

Optum Global Solutions
07.2020 - 04.2022

Data Engineer

Capgemini
11.2018 - 06.2020

ETL Developer

United Health Group
07.2015 - 11.2018

Masters in Computer Applications -

JNTU
Venkateswara R VallampudiSenior Data Engineer