Summary
Overview
Work History
Education
Skills
Certification
Accomplishments
Timeline
Generic

Vasudeva Reddy S

Consultant Data Engineer
Bengaluru

Summary

Results-driven Data Engineer with 5+ years of experience, including 1.5 years onsite and 4+ years in the Big Data ecosystem. Adept at designing, building, and maintaining data-intensive applications using AWS, Azure, Spark, Hadoop, Python, and Scala. Proven expertise in scalable ETL pipelines, Medallion architectures, cloud cost-performance optimization, and automated data quality assurance.

Overview

6
6
years of professional experience
2
2
Certifications

Work History

Consultant

Affine Analytics
02.2024 - Current
  • Led 3 cross-functional teams (25+ engineers, analysts, and business users) to build a scalable AWS data platform, migrating 100+ manual Excel reports to Qlik, reducing manual effort by ~40 hours/week and improving data accessibility for 1,000+ users.
  • Implemented Medallion architecture (Bronze–Silver–Gold) using Apache Iceberg with partitioning and bucketing, enabling schema evolution, time travel, and delivering a 5x query performance boost, reducing average query time from 15 mins to under 3 mins.
  • Optimized Redshift ETL pipelines, improving data load efficiency by 60%, cutting daily job runtime from 10 hours to 4 hours, and lowering compute costs by ~30%.
  • Designed star schema-based data models for the Super App and regulatory reporting, achieving 75% reduction in processing time.
  • Refactored 40+ complex SQL transformations and optimized Redshift queries, eliminating 3 redundant staging layers, reducing storage usage by ~30%, and improving query latency by up to 60%.
  • Reduced AWS Glue monthly compute costs by 45% by minimizing data movement, leveraging pushdown predicates, and migrating to Iceberg-based storage, enabling scalable and cost-efficient processing of 100M+ records daily and reduced 40% storage of reduced by introducing iceberg.
  • Data Platform Development & Architecture (AWS | Redshift | Iceberg | Qlik)
  • Data Quality, Monitoring & Automation (AWS Glue | CloudWatch | Jenkins | Terraform)
  • Super App Notification Engine (AWS Glue | PySpark | CDC | Redshift)

Data Engineer

Prescience Decision Solutions
12.2021 - 02.2024
  • Translated business requirements into technical solutions and data pipelines.
  • Designed Python and Spark-based ETL using AWS Glue and Lambda to migrate data from Postgres to Redshift, maintaining performance and integrity.
  • Automated end-to-end ETL pipelines using CloudFormation with external parameterization.
  • Built data migration pipelines from Azure Blob Storage to MySQL using Azure Data Factory.
  • Designed and developed LookML models, Explores, and dashboards for interactive and self-service analytics across multiple domains.
  • Created scheduled and ad hoc reports, exception handling, and visualizations tailored to stakeholders.
  • Provided end-to-end production support, including LookML refactoring, performance optimization, and resolution of complex data issues.
  • Involved in full SDLC phases (Development, QA, UAT, Production) for multiple client-facing Looker implementations.
  • Used JIRA for efficient issue tracking and agile workflows.
  • Applied branching, naming, and versioning conventions using CodeCommit for organized version control.
  • Data Migration and Report Automation
  • Looker BI Development
  • Tools & Collaboration
  • Optimized data processing by implementing efficient ETL pipelines and streamlining database design.

Software Engineer

Tata Consultancy Services
11.2019 - 12.2021
  • Performed ETL and data engineering tasks for banking product development.
  • Gathered and analyzed system requirements with BA teams to align technical solutions.
  • Supported SIT and UAT phases, applied defect resolution best practices.
  • Demonstrated proficiency in core Python concepts: Identifiers, Reserved Words, Operators, Variables, and custom exception handling.
  • Collaborated with designers and QA teams to analyze, triage, and resolve issues affecting functionality or test coverage.
  • SQL | Shell Scripting | Python

Electrical Engineer

Honor labs
08.2019 - 10.2021
  • Worked as an Electrical Engineer on MATLAB and PLC automation in Honor Labs to automate the seven machines running parallelly.

Education

Bachelor of Technology - Electrical & Electronics Engineering

Jawaharlal Nehru Technological University
Andhra Pradesh
01.2019

Skills

  • Big Data & Ecosystem: Apache Spark, Hadoop, Databricks, Hive
  • Cloud: AWS (Glue, S3, Lambda, DynamoDB, EMR, SQS, SNS, EC2, CloudFormation, EKS), Azure
  • Distribution: Cloudera, EMR, Azure Data Factory
  • Databases: MS SQL, PostgreSQL, DB2, Oracle, DynamoDB
  • Programming Languages: Python, SQL, JavaScript, Scala
  • Messaging Queue: Apache Kafka, SQS
  • Frameworks: Flask
  • DevOps & CI/CD: Jenkins, Terraform, CodeCommit
  • BI & Visualization: Looker, Qlik

Certification

AWS Certified Cloud Practitioner

Accomplishments

  • Transformer of the Team (Apache Iceberg Optimization) – Twice at Affine.
  • Distinctive Performer of the Year – 2022, Prescience Decision Solutions.
  • Recognized for client-endorsed cost-saving initiatives with financial impact.
  • Data wizard of the year 2023 - Prescience Decision Solutions.

Timeline

Consultant

Affine Analytics
02.2024 - Current

Data Engineer

Prescience Decision Solutions
12.2021 - 02.2024

Software Engineer

Tata Consultancy Services
11.2019 - 12.2021

Electrical Engineer

Honor labs
08.2019 - 10.2021

Bachelor of Technology - Electrical & Electronics Engineering

Jawaharlal Nehru Technological University

Similar Profiles

CREATE PROFILE
Vasudeva Reddy SConsultant Data Engineer