Summary
Overview
Work History
Education
Skills
Timeline
Generic
Rahul Gautam

Rahul Gautam

Sr. Data Engineer
Shimla

Summary

Detail-oriented data Engineer designs, develops and maintains highly scalable, secure and reliable data structures. Accustomed to working closely with system architects, software architects and design analysts to understand business or industry requirements to develop comprehensive data models. Proficient at developing database architectural strategies at the modeling, design and implementation stages.

Overview

9
9
years of professional experience
4
4
years of post-secondary education

Work History

Sr. Data Engineer

Apollo.io
08.2022 - Current
  • Built ETL framework to ingest data from Application database, in batch, into data warehouse, enabling analytics and reporting on Application datasets.
  • Designed and Implemented config driven ingestion framework to get data from External data source, via APIs.
  • Implemented the ELT via DBT.
  • Implemented the data quality monitoring tool via dbt elementary.
  • Implemneted the integration for extrnal data real time into Snowflake via snowpark.
  • Improvbements into snowflake, automated db house keeping.
  • Governance

Analytical Developer Data Engineer

Airbnb
Gurugram
05.2021 - 08.2022
  • Automated the schema management for
    data warehouse tables. Solution was GIT-managed, hence version controlled, and improved team efficiency by 19%, and opened ways for away team members to work on team's platform.
  • Remodeled existing data model into revamped model that was optimized for storage and compute. Achieved this using intermediate aggregated tables. This resulted in $2400/month savings in team's budget, improvement of 34%.
  • Migrated legacy HQL-based codebase to optimized Scala-based pipelines. This helped with compute costs savings of $900/month.
  • Implemented Data Quality Framework for anomaly detection and reporting. Proactively identified 233 issues in span of 7 months, improving reliability of datasets.
  • Led design and implementation of in-house Push reporting framework for insights delivery through various channels.

Data Engineer

Amazon
Bengaluru
08.2019 - 04.2021
  • Delivered data pipelines handling terabytes scale data, loaded daily into data warehouse.
  • Migrated legacy data ingestion(SQL-based system) to Spark
    based pipelines, on AWS EMR, using Scala for programming and Airflow for orchestration, resulting in availability of data in near real time.
  • Led implementation of Away team enablement, to simplify and automate general data warehousing requests for customers, leading to 27% improvement in team efficiency.
  • Designed data model for transactional data for Amazon Music, overhauling existing data models, leading to effective storage and retrieval of data.
  • Implemented automated data archival system, that offloaded data from
    redshift to S3 for archival, saving $100k/yr in overall costs.

Senior Associate Technical Consultant

SAS R & D
Pune
01.2018 - 08.2019
  • Led global deployment of SAS Anti-Money Laundering Solutions for 5 banking clients across different geographies.
  • Designed and optimized ETL framework for efficient data warehousing and analytics in Oracle/MySQL, leveraging SAS Visual Analytics for predictive modelling.
  • Established AML monitoring scenarios for financial institutions, enhancing transaction surveillance and compliance with integrated alert system for case management.
  • Improved data pipeline efficiency using SAS Data Integration Studio and Oracle tuning, and optimizations lead to 44% reduction in processing times for data loads.
  • Developed customized reports for AML-based analytics, which was appreciated by clientele, and was integrated into base product.

Project Engineer

Wipro Infotech
Mumbai
09.2015 - 01.2018
  • Developed ETL pipeline to land banking data from disparate source systems into Central Data Warehouse(CDW).
  • Integrated warehouse with third-party solution, SAS DDS, for Automated reporting for ADF reporting to RBI, and generating internal reporting on Credit Risk, Market Risk and asset-liabilities management system. Implemented SAS Anti-Money laundering solution for banking clients, for screening potentially illicit transactions and alerting investigators.
  • Optimized whole data flow applications between Bank's Oracle Data Warehouse and SAS systems. Reduced processing time by 90% for overall data load.
  • Mentoring new joiners and new team members on data
    engineering practices in SQL and Oracle DB.

Education

Bachelor of Technology - Electronics And Communications Engineering

Jawaharlal Nehru Government Engineering College
Sundernagar, Mandi, Himachal Pradesh, India
08.2011 - 07.2015

Skills

Spark, Scala

undefined

Timeline

Sr. Data Engineer

Apollo.io
08.2022 - Current

Analytical Developer Data Engineer

Airbnb
05.2021 - 08.2022

Data Engineer

Amazon
08.2019 - 04.2021

Senior Associate Technical Consultant

SAS R & D
01.2018 - 08.2019

Project Engineer

Wipro Infotech
09.2015 - 01.2018

Bachelor of Technology - Electronics And Communications Engineering

Jawaharlal Nehru Government Engineering College
08.2011 - 07.2015

Similar Profiles

CREATE PROFILE
Rahul GautamSr. Data Engineer