SANTHOSH REDDY
Sr Data Engineer
Bangalore
Summary
Senior Data Engineer with 5+ years of experience in designing and implementing scalable data solutions, optimizing ETL pipelines, and enhancing data architecture to drive business insights and efficiency.
Overview
6
6
years of professional experience
4
4
years of post-secondary education
2
2
Certifications
Work History
Senior Data Engineer
Synechron
Bangalore
07.2024 - Current
- Designed and implemented scalable data pipelines using Snowflake Cloud and DBT to transform raw data into structured datasets for analytics and reporting
- Developed and optimized complex SQL queries using Snowflake SQL for efficient data extraction, transformation, and performance tuning
- Automated ETL/ELT workflows by using Python and Snowflake's native features like Streams, Tasks, and Snowpipes for seamless data ingestion and transformation
- Built modular and reusable data models using DBT, enabling efficient data transformations and version control for better pipeline management
- Ensured data quality through rigorous testing, validation, and monitoring of all data assets, minimizing inaccuracies and inconsistencies.
Senior Data Engineer
CGI
Bangalore
09.2022 - 06.2024
- As a technical architect, led the design of a Snowflake cloud data warehouse solution for finance domain, ensuring top-notch performance and scalability
- Designed and implemented end-to-end ETL pipelines, encompassing data extraction from API calls, extensive transformations in AWS EC2 instances using Python scripts, and bulk loading the data from external stages - AWS S3 buckets into Snowflake, with structured layers including landing, transformed and consumption layers for a robust layered architecture
- Implemented data orchestration within Snowflake using stages, streams, and tasks, ensuring a consistent and reliable data flow across all layers of the data warehouse
- Formulated comprehensive testing strategies to ensure data pipeline quality and correctness, enhancing overall data quality and reliability
Software Engineer
Tata Steel
Jamshedpur
07.2019 - 09.2022
- Successfully led a Proof of Concept (POC) initiative that used Python scripts and Snowflake tools to create ETL pipelines, resulting in winning the project for the team
- Developed a custom test automation framework using Python scripting and Pandas data frames, seamlessly integrated with Azure DevOps (ADO) pipelines and test plans
- Engineered a Python-based key-hierarchy report comparison tool, achieving 100% accuracy and enhancing performance by 90% in complex report comparison tasks
Education
B.E - computer science and engineering
Birla Institute of Technology
Jharkhand 08.2015 - 05.2019
Skills
- Data Engineering
- Cloud Technologies
- Snowflake
- Snowsight
- SnowSQL
- ETL
- ELT
- Data Orchestration
- Snowpark
- AWS
- S3
- IAM
- SQS
- Azure DevOps
- ADO Pipelines
- Test Plans
Websites
Technical Expertise
- Proficient in Snowflake cloud data warehouse, Snowsight, and SnowSQL.,
- In-depth knowledge of Snowflake features such as time travel, zero-copy cloning and data sharing.
- Experience in using Snowflake SQL for proficient database management, including maintaining databases, warehouses, schemas, tables, views, and stored procedures.
- Experience in implementing ETL/ELT processes and data orchestration within Snowflake by using objects such as stages, snow pipes, streams, and tasks.
- Additionally, have good knowledge of Snowpark for advanced data processing., Familiar with AWS services including S3, IAM and cloud notification services like SQS queues.
- Experience in working with Azure Devops services such as ADO Pipelines, Test Plans, Repos etc. Familiar with DBT for streamlined data transformations.
- Proficient in Python for data engineering tasks such as data extraction, transformation and analysis.
- Pandas library for data manipulation tasks, including cleaning, transformation, and analysis.
- Selenium and requests libraries for web scraping and data extraction tasks.
- Pyodbc, cx_oracle, azure-cosmos libraries for interacting with and retrieving data from databases.
- Proficient in SQL for database querying and manipulation.
- Experience in working with MS SQL Server, Azure SQL, Cosmos DB, and Oracle databases.
- Proficient in Agile (Scrum) methodologies for efficient project management.
- Experienced in Git for version control and collaborative development.
- Familiar with Power BI for creating interactive visualizations and insightful dashboards.
Certification
SnowPro Core Certification, Snowflake, 02/01/24, https://achieve.snowflake.com/eee369c6-b2ac-458c-8b56-6b4866d8d05e#acc.OMVCAk9W
Awards
- GOLD Award, Recognized for exceptional performance and contributions to CGI for the fiscal years 2023 and 2024.
- KEY CONTRIBUTOR AWARD, Honoured for making significant contributions to the business unit, exemplifying excellence at CGI in the fiscal year 2023.
Timeline
Senior Data Engineer
Synechron
07.2024 - Current
Senior Data Engineer
CGI
09.2022 - 06.2024
Software Engineer
Tata Steel
07.2019 - 09.2022
B.E - computer science and engineering
Birla Institute of Technology
08.2015 - 05.2019
SnowPro Core Certification, Snowflake, 02/01/24, https://achieve.snowflake.com/eee369c6-b2ac-458c-8b56-6b4866d8d05e#acc.OMVCAk9W
Power BI Data Analyst Associate Certification, Microsoft, https://learn.microsoft.com/en-us/users/patlolasanthoshreddy-5226/transcript/drlyxiexq3g0mgr
Similar Profiles
Sasmita LenkaSasmita Lenka
Project Manager at SynechronProject Manager at Synechron
Pranav PatilPranav Patil
Senior Associate at SynechronSenior Associate at Synechron
SAURABH KUMARSAURABH KUMAR
Site Reliability Engineer at SynechronSite Reliability Engineer at Synechron