SIDDAMREDDY SATHEESH REDDY
Senior Data Engineer
Hyderabad
Summary
Certified SnowPro Core and Advanced Data Engineer with over 5 years of experience in designing and implementing scalable cloud data platforms using Snowflake, Databricks, AWS, Azure, DBT, and PySpark. Proficient in developing comprehensive end-to-end ELT/ETL pipelines, executing seamless cloud migrations, and integrating real-time APIs to deliver tailored analytics solutions across various sectors including Maritime, MarTech, FMCG, Healthcare, Hospitality, and Agro. Committed to leveraging data-driven insights to enhance business performance and drive strategic decision-making. Recognized for the ability to adapt to evolving technologies and industry trends while maintaining a focus on delivering high-quality results.
Overview
5
5
years of professional experience
Work History
Senior Data Engineer
Aion Tech Solutions Ltd
01.2023 - Current
- Project 1: Instagram Ads, Twitter Ads, LinkedIn Ads, Adobe Analytics, Fivetran, Azure Synapse, Microsoft Fabric
- Domain: Marketing Technology (MarTech) Industry
- Responsibilities:
- Gathered business requirements and created traceability matrix to map end-to-end data flows, define project scope, and ensure alignment across cross-functional teams.
- Designed and orchestrated scalable ELT pipelines using Fivetran integrating LinkedIn, Salesforce, Google AdSense, Google Search Console, Instagram Ads, Facebook Ads, Twitter Ads, Adobe Analytics, and Marketo into Microsoft Azure Synapse and Microsoft Fabric.
- Developed transformation logic in Azure Synapse Analytics, enhancing downstream reporting and analytics for business intelligence solutions.
- Implemented business rules, KPI calculations, and reporting logic delivering accurate, actionable analytical insights for stakeholders.
- Project 2: PostgreSQL, AWS, ECR, Lambda, Docker, S3, Tableau
- Domain: Agro Technology Industry
- Responsibilities:
- Migrated Salesforce and SAP data into PostgreSQL using AWS Lambda and AWS AppFlow, enabling scalable and automated data ingestion.
- Built and deployed Lambda container images using Docker; pushed to Amazon ECR and deployed to AWS Lambda.
- Configured Amazon EventBridge schedules to automate and orchestrate end-to-end data pipelines.
- Developed PostgreSQL transformation logic using MERGE operations, views, and optimized SQL queries for Tableau reporting.
- Project 3: Snowflake, Databricks, SQL Server, T-SQL, Matillion, Azure, Fivetran, DBT
- Domain: Maritime & Shipping Logistics — Voyage Optimization, Vessel Management, Fuel Consumption Analytics
- Responsibilities:
- Migrated on-premises maritime databases to Snowflake using Fivetran and Azure Data Factory (ADF); redesigned legacy schemas into star schema optimal for analytical workloads.
- Designed and managed Matillion ETL workflows for multi-source ingestion into Snowflake; implemented Snowpipe, Dynamic Tables, Streams, Stored Procedures, and Tasks for real-time data movement.
- Implemented API integrations using Snowflake External Access Integration, Network Rules, Snowpark, Snow SQL API, and Async APIs with AWS services.
- Built Databricks PySpark pipelines with Delta Lake (Bronze → Silver → Gold medallion architecture) for real-time and batch maritime data processing.
- Implemented incremental loads, CDC, partitioning, and clustering strategies — achieving 40% reduction in pipeline processing time and 60% improvement in query performance.
- Integrated external systems via REST APIs; automated data loads from Databricks to SQL Server using JDBC connections.
- Created views, alerts, and Power BI reporting layers on Snowflake for voyage, fuel, and vessel performance dashboards.
- Sub-Project: Maritime Gen AI Platform — Intelligent Vessel Data Insights (Data 360)
- Tech Stack: Databricks, Python, Snowflake, Delta Lake, Power BI, Streamlit, RAG, NL2SQL, Azure VM
- Responsibilities:
- NL2SQL for business users to query Snowflake conversationally
- AI anomaly detection on fuel, navigation & sensor telemetry
- Databricks + PySpark + Delta Lake pipelines with DCP/DMP Python validation
- Enterprise RBAC, lineage, governance & CI/CD
- Migration + 40% performance improvement
- Project 4: Snowflake, API, SQL Server, T-SQL, SSIS, AWS S3, Pentaho
- Domain: FMCG Industry
- Responsibilities:
- Led end-to-end Snowflake migration from requirement gathering to go-live — converted 700+ stored procedures and integrated 20 APIs via AWS Lambda and S3 for automated data ingestion.
- Implemented CDC logic using T-SQL and SQL Server Change Tracking features for high-fidelity data replication.
- Performed comprehensive data validation and reconciliation between SQL Server and Snowflake using checksum functions and row count validations.
- Designed and deployed AWS infrastructure including S3, IAM, SNS, VPC, Lambda, Event Bridge, and API Gateway for scalable pipeline automation.
Senior Consultant
Apps Associates
01.2022 - 01.2023
- Project: Snowflake, Matillion, Fivetran, Tableau
- Domain: Hospitality Industry
- Responsibilities:
- Transformed JSON files into VARIANT data types and flattened tables to convert semi-structured data into relational format in Snowflake, enhancing data accessibility.
- Integrated Salesforce, Google Analytics, Search Console, and Google AdWords via Fivetran, streamlining data ingestion into Snowflake with source-level aggregations.
- Integrated Salesforce, Google Analytics, Search Console, and Google AdWords via Fivetran; applied source-level aggregations to reduce load before ingestion into Snowflake.
- Integrated Matillion with Snowflake to create seamless ETL workflows and collaborated with stakeholders to establish source system connections, improving data flow.
Snowflake Developer
Hexaware Technologies
09.2021 - 01.2022
- Project: SQL Server, T-SQL, Power BI, Azure Blob, IICS
- Domain: Healthcare, Insurance & Health Services
- Responsibilities:
- Designed and implemented data pipeline from Azure Blob Storage to Snowflake DWH using Snowpipe and bulk loading to enable Tableau and Power BI visualization.
- Created and managed Azure resources — Storage Accounts and Azure Functions; integrated Azure Cloud with Snowflake to streamline data accessibility.
- Developed T-SQL stored procedures and automated ETL workflows for data transformation, cleansing, and aggregation using IICS to enhance data quality.
Education
Post Graduate (Engineering) -
Siddharth Institute of Engineering & Technology
01-2020
Skills
Data pipeline development
Data integration tools
Data analytics tools
Gen AI technologies
Datawarehouse: Snowflake, Databricks, Fabric
Databases: Oracle, MySQL, PostgreSQL, SQL Server , Vector wise db
Cloud platforms expertise
Visualization: Power BI, Tableau, Streamlit
Other Tools: Docker, Postman, Swagger, Git, Agile/Scrum
SnowPro certifications
Additional Information
Guided junior developers in technical best practices and professional growth, leading to a stronger, more collaborative team dynamic.
Participated in Technical Evaluation Panels, recruiting high-performing technical talent by assessing and selecting qualified candidates, ensuring optimal project alignment.
Contributed to organizational initiatives and volunteered for company-led programs, enhancing community engagement and corporate culture.
Timeline
Senior Data Engineer
Aion Tech Solutions Ltd
01.2023 - Current
Senior Consultant
Apps Associates
01.2022 - 01.2023
Snowflake Developer
Hexaware Technologies
09.2021 - 01.2022
Post Graduate (Engineering) -
Siddharth Institute of Engineering & Technology