GAYATHRI RAJAVARAPU

Project: Sunbird Obsrv

• Contributed to introducing lake house capabilities using Apache Hudi, benchmarking pipelines to enhance data management and processing efficiency.

Project: NPCI - Data Products

• Contributed to development of scalable data pipelines, processing 300 million daily transactions using Spark and Python, to generate business aggregates.
• Optimized workflows with Airflow and DBT, and Trino for high-performance querying.

Project: Sunbird Obsrv

• Played key role in shaping backend, developing and deploying APIs, optimizing Flink jobs, and implementing automation scripts to enhance efficiency and scalability.
• Made significant contributions to OBSRV pipeline testing, designing and executing comprehensive test suites to ensure robustness and reliability.

• Gained hands-on experience with various data engineering tools, such as Apache Kafka, Apache Spark, Apache Flink, and Druid.
• Utilized these tools to develop an end-to-end project using Python, understanding ETL pipelines.