Venkat Chilamala
Data Engineer
Hyderabad
Summary
With 11+ years of experience in Database(Oracle) and Data Engineering, I specialize in AWS services including Glue, S3, IAM, Lambda,Athena and CloudFormation. My expertise extends to Snowflake Cloud Data Warehouse and ETL/ELT tools such as Apache NiFi and WherescapeRED,as well as PySpark and Python with hands-on experience spanning 7 years. A proactive team player, I excel at independently managing responsibilities while thriving in a collaborative environment. My knowledge also encompasses data governance tools like Collibra and Alation, focusing on business terms, feed inventory, and heat maps. Additionally, I have extensive experience in version control using Git-Stash and source code management through Jenkins for seamless project building and deployment via CI/CD pipelines. Throughout my career, I have made significant contributions to the logical and physical design of systems, ensuring adherence to architectural standards following thorough technical reviews.
Overview
11
11
years of professional experience
2012
2012
years of post-secondary education
3
3
Certifications
Work History
Senior Advisory Technical Services Specialist (Data Engineer)
IBM India Private Limited
11.2021 - Current
- Created data pipelines from AWS (S3) to Snowflake using AWS S3, Glue, Athena, Snow pipe and WherescapeRED(ELT)
- Created AWS Glue Workflow, for orchestrating glue jobs
- Created schema files for processing raw files to prepared bucket based on source standards
- Modified the existing lambda python code to perform ad hoc data files process
- Verify the data in Athena tables and check the logs in cloud watch if there are any errors in file processing
- Created snowflake tables, Views, Secure views, data share listing and custom procedures
- Involved in data modelling and created DV/DV2 tables like Hub, link and SAT and star schema objects like Dimension and fact tables in presentation layer in snowflake
- Involved in masking PII data using Glue and snowflake masking policies
- Identify the root cause of the issues by trackback reports/diagrams generated from WherescapeRED and fix
- Identify and fix the performance issues in snowflake
Application Development Senior Analyst (Data Engineer)
Accenture Solutions Pvt. Ltd
06.2018 - 11.2021
- Onboarded multiple data feeds by structuring them in accordance with the Data Vault modeling architecture, utilizing the Event-Driven Framework on AWS
- Used AWS DMS for data base migration from on-premises
- Implemented AWS step functions for job orchestration
- Created python and pyspark scripts to migrate on-premises data to AWS cloud s3
- Created Apache NiFi Process flows to consume S3 bucket files and Kafka topics from different sources
- Implemented the pyspark script to read JSON and csv files landed in on-premises
- Maintain proper data governance using Collibra and Alation
- Perform data profiling to understand nature of data and align with source and customers on the same
- Create data models in Oracle, Snowflake like creating DV and dimensional modelling
- Analyzing and understanding the user stories and designs
- Assign story points to the user stories based on complexity and start designing and implementing for the new features in the sprint
Sr. Software Engineer
HCL Technologies
10.2017 - 06.2018
Software Engineer
HCL Technologies
04.2014 - 10.2017
- Have experience in upgrading the Fermat Credit risk product from 10.9 to RAY CR 4.1.0
- Code changes to maintain the existing customization of the application
- Creating new stored Procedures, Packages, Functions in Oracle 11g and enhancing existing PL/SQL objects
- Performance tuning to make the batch run timings better and efficient to achieve SLA
- Implemented the Process Profile setup for Moody’s RAY 4.1
- Implemented the RRT reports generation
- Implemented the Oracle compression for storage savings
- Completed SAT, OAT for new servers
- Automation of housekeeping jobs to have the servers healthy
- Import and Export of Fermat 10.9 Schema s and RAY 4.1 for supporting various testing cycles
- Implemented Customization in Check error process
Education
MCA -
Sri Venkateswara University(SVU)
Tirupati, IndiaSkills
Snowflake CDW
AWS(AWS Glue,AWS Lambda,AWS Athena,AWS S3,AWS IAM,AWS Cloud Formation,AWS Cloud Watch etc)
Oracle PL/SQL
Pyspark
Python
Unix
WherescapeRED
Apache NiFi
ServiceNow
Jira
Apache Airflow
CI/CD (Git,Bitbucket,Jenkins)
Certification
AWS Developer associate, 12/01/20
Timeline
Senior Advisory Technical Services Specialist (Data Engineer)
IBM India Private Limited
11.2021 - Current
Application Development Senior Analyst (Data Engineer)
Accenture Solutions Pvt. Ltd
06.2018 - 11.2021
Sr. Software Engineer
HCL Technologies
10.2017 - 06.2018
Software Engineer
HCL Technologies
04.2014 - 10.2017
MCA -
Sri Venkateswara University(SVU)