Open To Work
Venkateswara R Vallampudi
Senior Data Engineer
Hyderabad
Summary
Results-driven Data Engineer with over 10 years of experience in designing, building, and optimizing scalable data pipelines, cloud-based solutions, and analytics platforms. Expertise includes implementing data pipelines, ETL development, data warehousing, real-time streaming, and navigating big data ecosystems. Proven ability to collaborate with cross-functional teams to deliver actionable insights that drive business growth. Seeking opportunities across Europe to leverage skills in cloud-based solutions and drive innovation in scalable data architectures.
Overview
10
10
years of professional experience
Work History
Senior Data Engineer
Tata Consultancy Services
08.2023 - Current
- Client: London Stock Exchange Group, UK
- Tech Stack: Pyspark, Snowflake, Snowpark, AWS (S3, EC2, RDS, lambda, Cloud watch), Hadoop (HDFS, Hive, Impala, HBASE, Oozie, Zookeeper), Jenkins, Kubernetes
- Migrated legacy on-prem Applications (Trade Performance Reports) to AWS architecture, reducing infrastructure costs. Major contributor in re-writing the applications in AWS
- Designed and implemented scalable Data pipelines to handle data from various sources like Files, API calls and various databases.
- Developed real-time data streaming pipelines using snowflake for data Ingestion
- Enhanced existed API code to be utilized by larger clients
Senior Data Engineer
Mphasis Limited
04.2022 - 08.2023
- Client: JPMC, USA
- Tech Stack: Pyspark, AWS (S3, EC2, Glue, Athena), Hadoop (HDFS, Hive, Tez, HBase), Jules, Oracle, Teradata, Control-M, Unix
- Built and optimized Data pipelines (Spark-Scala) to transfer the data from Oracle to Teradata
- Implemented reconciliation process to avoid manual validations and in turn it reduced manual efforts of 1 hour to 5 minutes
- Improved SQL query performance, reducing reporting runtime from 2 hours to 15 minutes.
- Analyzed the data chucks which are not been utilized by the client and moved it to Glacier
Data Engineer
Optum Global Solutions
07.2020 - 04.2022
- Client: United Health Care, USA
- Tech Stack: Spark-Scala, sparkSQL, Hadoop (HDFS, Hive, HBase), Jenkins, TWS - Scheduler, Unix
- Designed and maintained spark-scala based extracts to generate the business level reports from the data which reside on Datalake. I contributed majorly in this by using Data frames and Datasets to implement the core business logic
- Generated the feeds which will be used by US State Governments officials. Directly worked with vendor to understand the requirements along with BA and Customized the reports / data feed based on the client requirement
- Worked with multiple teams (Mainframe and DB teams) to deliver the output in customer requested formats like standard 834 or Custom 834 formats.
- Maintained cross team collaborations effectively to deliver the things in smoother way
- Conducted data quality checks to ensure accuracy and consistency.
Data Engineer
Capgemini
11.2018 - 06.2020
- Client: DBS Bank, Singapore
- Tech Stack: Pyspark, Spark-Scala, sparkSQL, Hadoop, Airflow, AWS S3, Jenkins, Unix
- Plays a major role in the implementation of Cash in Transit (CIT) project – Pyspark based application which generates the data feed from ATM, BTM and CRS machines to be utilized by data scientist to generate the forecast of these Machines
- Collaborated with Data Scientists and Analysts to deploy ML-ready datasets for predictive models.
- By creating the custom DAGs in Airflow, scheduled these jobs in the server and reduced major manual efforts
- Migrated over 160+ reports from Teradata to PySpark
ETL Developer
United Health Group
07.2015 - 11.2018
- Client: United Health Care, USA
- Tech Stack: DataStage, Talend, Hadoop (HDFS, MapReduce, Sqoop, Hive, HBASE), Unix
- Designed and maintained ETL pipelines using DataStage/Talend.
- Migrated some reports from DataStage to Talend using the big data features in Talend
- Migrated existing DataStage applications to Hadoop based approach (Sqoop- Hive).
- Involved in several POC’s to implement the low cost-based application to migrate the existing application
- Learned multiple technologies like Talend, Hadoop, DataStage to meet the project requirements and to become vital player in the team
- Involved in implementation of DDL and DML scripts to transform Data and populate in data warehouse tables (Fact tables and Dimension Tables)
Education
Masters in Computer Applications -
JNTU
Kakinada09.2014
Skills
- Python, SQL, Scala, Java and Shell scripting
- Spark (Pyspark, Spark-scala, SparkSQL)
- AWS – S3, EM2, EMR, Glue, Athena, Lambda, CloudWatch
- Hadoop - HDFS, Sqoop, Hive, MapReduce, HBase, Oozie, Zookeeper, YARN
- Snowflake, Snowpark, Databricks
- Airflow, Talend, Datastage
- Teradata, Oracle, PostgreSQL, MySQL
- Git Version Control, Maven, SBT
- CI/CD, Jenkins, JIRA, Kubernetes, Docker
- Big data processing
Work Availability
monday
tuesday
wednesday
thursday
friday
saturday
sunday
morning
afternoon
evening
swipe to browse
Languages
English
Advanced (C1)
Timeline
Senior Data Engineer
Tata Consultancy Services
08.2023 - Current
Senior Data Engineer
Mphasis Limited
04.2022 - 08.2023
Data Engineer
Optum Global Solutions
07.2020 - 04.2022
Data Engineer
Capgemini
11.2018 - 06.2020
ETL Developer
United Health Group
07.2015 - 11.2018
Masters in Computer Applications -
JNTU
Similar Profiles
Vrunda ChindhadeVrunda Chindhade
HR Manager at Tata Consultancy Services, TCS, Tata Consultancy Services LimitedHR Manager at Tata Consultancy Services, TCS, Tata Consultancy Services Limited
Sreya BasuSreya Basu
Procare Analyst at Tata Steel(Tata Consultancy Services)Procare Analyst at Tata Steel(Tata Consultancy Services)
Arun VArun V
Program Manager at Tata consultancy services | Financial Ombudsman ServicesProgram Manager at Tata consultancy services | Financial Ombudsman Services
Naimish AgnihotriNaimish Agnihotri
Assistant Delivery Manager at Tata Consultancy Services, Global Shared ServicesAssistant Delivery Manager at Tata Consultancy Services, Global Shared Services
Nithin SaraNithin Sara
Sr. Product Engineer at TECHMAHINDRA LTDSr. Product Engineer at TECHMAHINDRA LTD