P.S.Kalyan Rama Reddy
Anaparthy
Summary
Highly skilled and results-driven Senior Data Engineer with over five years of experience in designing, building, and managing scalable data pipelines and analytical platforms across Azure and AWS ecosystems. Proven expertise in ETL development, data lake/lake house architecture, data modeling, and real-time streaming. Hands-on experience with PySpark, Spark SQL, ADF, Azure Databricks, Snowflake, Microsoft Fabric, Redshift, Kafka, and Airflow. Adept at translating complex business requirements into optimized data solutions, delivering high-quality data for analytics, BI, and AI/ML initiatives. Strong collaborator with cross-functional teams, with a passion for data quality, performance optimization, and automation.
Overview
6
6
years of professional experience
Work History
Senior Data Engineer
Infobell IT Solution
Bangalore
11.2024 - Current
- Built a hybrid data lake house architecture using Microsoft Fabric, Delta Lake, and Azure Synapse for batch and streaming analytics.
- Engineered streaming pipelines using AWS Glue, Kinesis, Kafka, and Airflow for ingesting clickstream and IoT data.
- Transformed and enriched structured and unstructured data using PySpark and Scala on EMR and Databricks.
- Integrated NoSQL stores like MongoDB and Cassandra, with Redshift and Snowflake, for multi-source data fusion.
- Built real-time dashboards using Power BI, used Athena and PostgreSQL for ad-hoc queries, and secured pipelines with IAM roles and GitHub CI/CD.
Technical Analyst
Infinito Technologies
Bangalore
05.2024 - 08.2024
- Architected ingestion pipelines using ADF for multiple business domains like Sales, HR, Marketing, and Logistics.
- Processed data using PySpark in Azure Databricks to create unified dimensional models for 360-degree views.
- Defined and implemented KPIs such as customer acquisition cost and churn rates in Power BI dashboards.
- Used DBT to manage transformations, along with CI pipelines for deployment and versioning using Git.
- Enhanced performance with optimized Spark code and intelligent caching strategies.
Technical Analyst
Clear Sense India Private Limited
Hyderabad
04.2023 - 04.2024
- Developed ADF pipelines for batch ingestion, and executed complex transformations using Databricks.
- Implemented SCD Type 1 and Type 2 logic using PySpark, following dimensional modeling principles.
- Integrated sources like flat files, SQL Server, and APIs to load data into Snowflake and Azure SQL DW.
- Built BI dashboards using Power BI, focusing on KPIs, and opportunity sizing across markets.
- Applied modular DBT models for transformation logic, version-controlled via GitHub, and deployed using CI/CD pipelines.
Data Analyst
Dodla Dairy Limited
Hyderabad
07.2022 - 03.2023
- Built end-to-end ETL workflows using ADF, copying data from on-premises to Azure Blob, and Data Lake.
- Created custom Python scripts for data cleansing, and loaded the final data into SQL Server.
- Developed dashboards in Power BI, and performed interim data validation using Excel and Power Query.
- Automated alerts using Azure Logic Apps and managed secrets with Key Vault.
- Participated in daily stand-ups, retrospectives, and release planning using Jira and Azure DevOps.
Data Process Executive
Cognizant Technology Solutions
Hyderabad
10.2019 - 11.2021
- Developed dynamic and parameterized pipelines using ADF to migrate data from legacy project systems to ADLS Gen1.
- Transformed data using PySpark and Spark SQL, integrated with SQL Server and visualized in Power BI dashboards.
- Built reusable components with SSIS for pre-Azure legacy workloads.
- Orchestrated workflows and alerts using Azure Logic Apps; secured access using Key Vault.
- Delivered sprint stories using Agile methodology and versioned code using Git & Azure DevOps.
Education
BTech - Electronic And Communication Engineering
Aditya College of Engineering
Kakinada, Andhra Pradesh07-2019
Skills
- Azure Data Factory
- Azure Databricks
- Azure Synapse Analytics
- Microsoft Fabric
- S3
- EC2
- Lambda
- IAM
- Aws glue
- Athena
- Redshift
- EMR
- DynamoDB
- Kinesis
- Data pipeline
- CloudWatch
- CloudTrail
- PySpark
- Spark SQL
- Snowflake
- SQL
- Python
- Data modeling
- Excel
- DBT
- Scala
- SSIS
- Kafka
- Amazon Kinesis
- Airflow
- Power BI
- Alteryx
- Azure DevOps
Affiliations
• Actively contributed to internal knowledge-sharing sessions on Azure Data Engineering and PySpark best practices.
• Participated in code review panels and automation initiative teams to improve pipeline quality and performance.
• Mentored junior engineers on cloud-native data pipeline development and performance optimization.
• Regular participant in technical meetups and webinars related to Big Data, Azure, and modern data platforms.
Timeline
Senior Data Engineer
Infobell IT Solution
11.2024 - Current
Technical Analyst
Infinito Technologies
05.2024 - 08.2024
Technical Analyst
Clear Sense India Private Limited
04.2023 - 04.2024
Data Analyst
Dodla Dairy Limited
07.2022 - 03.2023
Data Process Executive
Cognizant Technology Solutions
10.2019 - 11.2021
BTech - Electronic And Communication Engineering
Aditya College of Engineering
Similar Profiles
Vijay HSVijay HS
Junior Software Engineer at Infobell IT SolutionJunior Software Engineer at Infobell IT Solution
M PrashanthaM Prashantha
Software Developer at Infobell IT SolutionsSoftware Developer at Infobell IT Solutions
Prajwal BhopalePrajwal Bhopale
Jr. Software Engineer at Infobell IT Solutions | Client AMDJr. Software Engineer at Infobell IT Solutions | Client AMD
Mehar Ankith BobbiliMehar Ankith Bobbili
Data Engineering at AccentureData Engineering at Accenture
Ajay Manikanta Reddy NallamilliAjay Manikanta Reddy Nallamilli
Imports &Exports at Ajay Manikanta Reddy NallamilliImports &Exports at Ajay Manikanta Reddy Nallamilli