Hi, I’m
SINGAMPALLI KRISHNAMOULYA
Senior Data Engineer
Bangalore,KA
Summary
Lead Data Engineer delivering Azure-based data platforms and AI-powered workflow automation across ETL, validation, and production support. Led a team of 4 to build automated pipelines that cut data processing time by 40%, while improving warehouse query efficiency by 25% through PySpark, Azure Databricks, and Synapse tuning. Integrates LLMs, Python, and REST APIs into production solutions for document analysis, data validation, and intelligent query handling.
Overview
9
years of professional experience
Work History
Tiger AnalyticsBengaluru
Lead Data Engineer
07.2024 - 07.2026
Job overview
- Led a team of 4 to design an automated ETL process, resulting in 40% faster data processing times.
- Developed AI-powered solutions using Large Language Models (LLMs) to automate business workflows.
- Designed and implemented AI agents for document analysis, data validation, and intelligent query processing.
- Integrated LLMs with enterprise applications using REST APIs and Python.
- Developed and tested AI agent workflows for document understanding, summarization, and information retrieval.
- Worked on Deep Insight and ZESP (Zero-Shot Profiler) applications to analyze datasets and generate data-driven insights.
- Built prompt-based AI workflows to improve response quality and automate repetitive tasks.
- Collaborated with business stakeholders to gather requirements and deliver tailored AI solutions.
- Tested, debugged, and optimized AI applications, enhancing performance and accuracy.
- Participated in Agile ceremonies, conducted code reviews, and provided production support.
Tiger AnalyticsBengaluru
Senior Data Engineer
07.2024 - 07.2026
Job overview
- Optimized complex data warehouse structures, achieving a 25% increase in query efficiency.
- Served as Data Engineer and Data Validator for Johnson G Johnson data migration project, ensuring data integrity throughout the process.
- Developed PySpark notebooks for data cleansing, transformation, aggregation, and incremental data loading.
- Worked with Delta Lake and implemented partitioning strategies to improve data processing performance.
- Created Azure Data Factory pipelines using Copy, Lookup, Metadata, ForEach, If Condition, and Stored Procedure activities.
- Validated source and target data to ensure data completeness, accuracy, and consistency during migration.
- Developed Azure Data Factory pipelines utilizing Copy, Lookup, Metadata, ForEach, If Condition, and Stored Procedure activities to streamline data workflows.
- Worked as a Data Engineer and Data Validator in the Johnson G Johnson data migration project.
- Participated in the migration of Spark applications from Amazon EMR (Elastic MapReduce) to Amazon EKS (Elastic Kubernetes Service) by updating PySpark applications, validating business logic, and ensuring compatibility with the new execution environment.
- Analyzed existing ETL workflows and modified code to support Kubernetes-based Spark execution while maintaining existing functionality.
- Collaborated with developers, QA teams, and business stakeholders to troubleshoot data issues, facilitating timely project releases.
- Supported production deployments, performance tuning, defect resolution, and post-production validation.
InformaticaBengaluru
Data Engineer
01.2024 - 05.2024
Job overview
- Developed and maintained ETL/ELT pipelines using Azure Data Factory, Azure Databricks, PySpark, Azure Synapse Analytics, ADLS Gen2, and Azure SQL to streamline data ingestion and processing workflows.
- Developed PySpark notebooks for data cleansing, transformation, aggregation, and incremental data loading.
- Worked with Delta Lake and implemented partitioning strategies to improve data processing performance.
- Created Azure Data Factory pipelines using Copy, Lookup, Metadata, ForEach, If Condition, and Stored Procedure activities.
- Performed SQL-based data validation, reconciliation, and data quality checks.
- Migrated Spark applications from Amazon EMR to Amazon EKS by updating PySpark applications and validating business logic for compatibility with the new execution environment.
- Analyzed existing ETL workflows and modified code to support Kubernetes-based Spark execution while maintaining existing functionality.
- Collaborated with developers, QA teams, and business stakeholders to resolve data issues, ensuring timely delivery of project releases.
NeudesicBengaluru
Data Engineer
10.2022 - 09.2023
Job overview
- Worked on coding PySpark in Databricks and loading data into Azure Data Lake.
- Created and validated data loading processes using notebooks and SQL for accuracy and reliability.
- Created workspaces and ingested data using data frames from raw to enriched layers.
- Implemented transformations and partitions.
- Created, validated, and tested data loaded using notebooks and validated the scripts using SQL.
- Implemented data transformations and partitioning strategies to enhance data processing efficiency.
- Experience with SQL Server and T-SQL objects like triggers, stored procedures, tables, table variables, user-defined functions, views, indexes, CTEs, jobs, backups, and restores.
- Developed and optimized notebooks to apply data transformations using Spark SQL in Azure Databricks.
- Understanding of the latest Azure Service Fabric and its usages, with trainings attended.
- Worked on an auction management system use case.
L&T Technology ServicesBengaluru
Data Engineer
09.2021 - 08.2022
Job overview
- Experience in designing ADF pipelines for ingestion of data from SAP (HANA) to Snowflake tables.
- Executed metadata management, conditional logic, copy operations, For Each loops, and Lookup activities in Azure Data Factory.
- Designed Azure ADF pipelines to move data from blob to blob and to Azure Data Lake Gen2.
- Designed Azure ADF pipelines to move data from SAP BW Open Hubs to Azure Data Lake Gen2.
- In Data migration project with Azure ADF, Synapse Analytics, Data Lake Store Gen2 (ADLS Gen2), Azure SQL.
- Worked on activities like metadata, IF, copy, For Each, Lookup, etc., in Azure Data Factory.
- Implemented activities such as Copy activity, Execute Pipeline, Get Metadata, If Condition, Lookup, Variable, Filter, and For Each pipeline activities for on-cloud ETL processing.
- Implemented activities such as Copy activity, Execute Pipeline, Get Metadata, If Condition, Lookup, Variable, Filter, and For Each pipeline activities for on-cloud ETL processing.
- Implemented Copy activity, Execute Pipeline, Get Metadata, If Condition, Lookup, Variable, Filter, and For Each activities for on-cloud ETL processing.
- Used dataflows for transformations and notebooks for data validations.
- Developed notebooks and applied transformations using Spark SQL in Azure Data Bricks.
- Tested the notebooks and validated the scripts using SQL.
- Prepared documents for coding standards and DMPV.
- Implemented CI/CD for code migration to other environments.
- Utilized dataflows for data transformations and notebooks for validating datasets.
CLINZECHBengaluru
Data Engineer and BI Developer
07.2017 - 10.2021
Job overview
- Designed and developed scalable ETL/ELT pipelines using Azure Data Factory (ADF) to ingest data from multiple sources into Azure Data Lake Storage (ADLS Gen2).
- Developed data transformation workflows using Azure Databricks (PySpark) to clean, validate, and transform data.
- Designed and maintained data warehouse solutions using Azure Synapse Analytics and Azure SQL Database.
- Created Bronze, Silver, and Gold data layers to enhance data accessibility and usability in accordance with Medallion Architecture.
- Implemented incremental data loading, Change Data Capture (CDC), and merge strategies for efficient data processing.
- Implemented incremental data loading, Change Data Capture (CDC), and merge strategies to streamline data processing workflows.
- Developed complex SQL queries, stored procedures, views, and performance-optimized database objects.
- Scheduled and monitored data pipelines using Azure Data Factory triggers and alerts.
- Implemented data quality checks, validation rules, and reconciliation processes to ensure data accuracy.
- Optimized Power BI report performance through efficient data modeling and query optimization techniques, resulting in faster report generation.
- Collaborated with business analysts, product owners, and stakeholders to gather reporting and data requirements.
- Created interactive dashboards and reports using Power BI for business users and leadership.
- Designed Power BI data models, relationships, DAX measures, calculated columns, and KPIs.
- Optimized Power BI report performance using efficient data modeling and query optimization techniques.
- Performed production support activities, root cause analysis, bug fixes, and performance tuning.
- Maintained technical documentsation, deployment guides, and data mapping document.
Education
Vikrama Simhapuri University (VSU)Nellore, India
MBA from HR
05-2012
SRI KRISHNA CHAITANYA DEGREE COLLEGENellore, India
B.SC from Computers
04-2009
Skills
Python
Azure Data Factory
Azure Databricks
PySpark
SQL
Data pipeline design
Data transformation workflows
Data quality validation
Accomplishments
- Successful Team Mentorship - Created a mentorship program that boosted junior engineers' productivity by 15% over a 6-month period.
- Enhancing Query Performance - Initiated a query optimization project, resulting in 25% increased efficiency, enhancing overall analytics capability.
Timeline
Lead Data Engineer
Tiger Analytics
07.2024 - 07.2026
Senior Data Engineer
Tiger Analytics
07.2024 - 07.2026
Data Engineer
Informatica
01.2024 - 05.2024
Data Engineer
Neudesic
10.2022 - 09.2023
Data Engineer
L&T Technology Services
09.2021 - 08.2022
Data Engineer and BI Developer
CLINZECH
07.2017 - 10.2021
Vikrama Simhapuri University (VSU)
MBA from HR
SRI KRISHNA CHAITANYA DEGREE COLLEGE
B.SC from Computers