Summary
Overview
Work History
Education
Skills
Websites
Certification
Accomplishments
Personal Information
Timeline
Generic

VARUN CHOWDARY GURRAM

Bangalore

Summary

Cloud Data Engineer with over 4 years of experience designing and implementing scalable data pipelines, cloud-based data warehousing, and real-time analytics solutions. Proficient in Microsoft Azure, AWS, and Apache Spark (PySpark, Scala) with expertise in Extract, Transform, Load (ETL) and Extract, Load, Transform (ELT) frameworks. Skilled in orchestrating workflows using Apache Airflow and optimizing data processes for business intelligence. Experienced in collaborating with stakeholders to deliver data pipelines that improve processing speed and data governance.

Overview

4
4
years of professional experience
1
1
Certification

Work History

Azure & AWS Data Engineer

Epsilon
Bangalore
10.2023 - Current
  • Architected ELT pipelines using Medallion architecture to migrate 5TB of SAP HANA data to AWS Databricks, ensuring complete data integrity.
  • Developed AWS Glue ETL jobs to extract and transform vendor data, integrating with AWS Lambda for event-driven ingestion, reducing costs by 15%.
  • Implemented AWS CloudWatch monitoring and logging framework to track pipeline performance and data lineage, enhancing data governance.
  • Tuned PySpark and Scala transformations in Databricks, improving query performance by 30% through partitioning and bucketing.
  • Connected API data (Google Analytics, YouTube Analytics, Facebook Ads) to AWS Databricks using AWS Lambda triggers, enabling near real-time analytics.
  • Automated PySpark code generation with Vertex AI, reducing development time by 25% and improving transformation consistency.
  • Managed CI/CD pipelines using Azure Repos for seamless deployment and version control.
  • Designed data pipelines for machine learning models using Apache Airflow on AWS EC2 Docker containers, supporting daily model training.
  • Processed raw data with AWS Glue and automated workflows via AWS Lambda, reducing latency by 20%.
  • Enforced IAM policies for secure data access and monitored pipeline health with AWS CloudWatch.
  • Scheduled workflows using CRON expressions, achieving 99.9% uptime for data processing.
  • Partnered with data scientists to integrate PySpark transformations, improving recommendation accuracy by 10%.

Data Engineer

Infosys Limited
Bangalore
03.2021 - 09.2023
  • Created batch data pipelines using Azure Data Factory and Databricks to integrate 10TB of Oracle and Teradata data into Azure Data Lake monthly.
  • Deployed ELT processes with Medallion architecture for Azure Synapse Analytics, supporting Power BI and Tableau dashboards.
  • Constructed real-time pipelines with Azure Event Hubs and Databricks Structured Streaming, processing 1M events daily.
  • Enhanced Spark (PySpark, Scala) performance, reducing pipeline runtime by 35%.
  • Oversaw version control with GitHub, ensuring zero-defect deployments.

Education

Bachelor of Technology - Electronics and Communications Engineering

Hindustan University
Chennai, Tamil Nadu
01.2020

Skills

  • Microsoft Azure
  • Data Factory
  • Databricks
  • Data Lake Gen 2
  • Event Hubs
  • Synapse Analytics
  • Logic Apps
  • Amazon Web Services
  • AWS Glue
  • Lambda
  • S3
  • Redshift
  • EC2
  • CloudWatch
  • IAM
  • Apache Spark
  • PySpark
  • Scala
  • Apache Airflow
  • Delta Lake
  • Azure Synapse Analytics
  • Azure SQL Database
  • SAP HANA
  • Oracle
  • Teradata
  • Python
  • SQL
  • GitHub
  • Azure Repos
  • Continuous Integration
  • Continuous Deployment
  • CI/CD
  • CRON
  • AWS Lambda

Certification

  • Microsoft Certified: Azure Fundamentals
  • Microsoft Certified: Azure Data Fundamentals
  • Databricks Certified Associate Developer for Apache Spark (2024)

Accomplishments

  • Received Quarterly Data Science Awards for creating efficient data pipelines and meeting project deadlines.
  • DNA Rise Award, Q3 2021: Awarded for streamlining Azure data pipelines at Infosys.
  • Directed phase-1 SAP HANA migration to AWS Databricks, achieving 100% data accuracy.
  • Built automated PySpark code framework, saving 20 hours of development time weekly.

Personal Information

Title: Cloud Data Engineer

Timeline

Azure & AWS Data Engineer

Epsilon
10.2023 - Current

Data Engineer

Infosys Limited
03.2021 - 09.2023

Bachelor of Technology - Electronics and Communications Engineering

Hindustan University

Similar Profiles

CREATE PROFILE
VARUN CHOWDARY GURRAM