Directed and supervised team of 6 engaged in Kafka to Data Warehouse pipeline development.
Built S3 to Snowflake Operator which will move data from S3 to Snowflake in near real time (~2mins) and open sourced it. https://github.com/apache/airflow/pull/6469
Hosted and maintained apache airflow in house on AWS and scaled to run millions of jobs per month and automated all infra by automating infra operators and took care of DR by implementing Chef and Terraform.
Built Monitoring and Tooling around Systems to act proactively while System failures.
Maintained and hosted Snowflake and Redshift Data wareshouse which consist of ~100TB of data. Taken care of High availability
Lead the team to implement Bule Green Deploymnets, involved in optimizing production systems cost.
Software Engineer
Freshworks
Chennai, Tamil nadu
06.2017 - 12.2019
Kafka & Kafka Consumers: We use Kafka and apache airflow as pipeline to move events from upstream to Data warehouses. Developed and scaled generic consumers for Kafka, where single consumer with 8GB ram can process up to ~40k rpm with response time under 5ms.
Business rule engine: Developed rule engine for data transformation from upstream to multiple downstream where we can apply business rules where transformation takes approximately 2ms
Trained and mentored junior developers(interns) to build Self-service Portals.
Software Engineering Intern
Skillenza
Bangaloer, Karnataka
06.2016 - 12.2016
Test case generator Which involves multiple REST API ‘s using Flask. Which can generate big set of test cases for competitive programming, Which is faster then 2X compared to what is there in Industry (https://test-case-generator.herokuapp.com/)
Education
Bachelor of Technology - Computer Science And Programming
Rajiv Gandhi University Of Knowledge Technologies
Nuzvid, Andhra Pradesh
06.2010 - 05.2017
Skills
Python
undefined
Timeline
Senior Software Engineer
Freshworks
01.2019 - Current
Software Engineer
Freshworks
06.2017 - 12.2019
Software Engineering Intern
Skillenza
06.2016 - 12.2016
Bachelor of Technology - Computer Science And Programming
Senior Software Engineer/Senior Software Test Engineer at Bosch Global Software SolutonsSenior Software Engineer/Senior Software Test Engineer at Bosch Global Software Solutons