Bittu Kumar

Project: Enterprise Data Platform & Analytics Pipeline for FinOps, HR and Sales

• Designed and developed scalable data pipelines using Azure Data Factory, Databricks, and PySpark to support business-critical dashboards and analytics use cases.
• Collaborated with Operations and FinOps teams to gather and analyze requirements, translating business needs into efficient and scalable data pipelines and data models.
• Built end-to-end pipelines using Azure Data Factory to ingest, transform, and load large-scale datasets into Delta Lake for reporting and analytics.
• Developed modular and reusable PySpark notebooks in Databricks to implement complex transformation logic and standardize data processing.
• Optimized data processing performance using Spark techniques such as partitioning, caching, and broadcast joins, improving execution efficiency and reducing processing time.
• Integrated Azure Data Factory pipelines with MicroStrategy dashboards to enable efficient reporting and analytics.
• Automated workflow orchestration and scheduling using Azure Data Factory pipelines and Databricks Workflows, improving reliability and reducing manual intervention.

Project: Large-Scale ETL Modernization and Data Migration

• Developed and maintained complex ETL workflows using Informatica PowerCenter for large-scale data migration across CRM and OMS systems.
• Performed migration analysis by analyzing Informatica PowerCenter XML files to understand transformation logic such as Lookup, Joiner, Router, Aggregator, Expression, Filter, and Update Strategy, along with workflows, dependencies, and parameterization.
• Modernized ETL pipelines by migrating legacy Informatica workflows to Azure Data Factory and Databricks using PySpark and Spark SQL for scalable and distributed data processing.
• Converted Informatica mappings into modular PySpark and Spark SQL-based Databricks notebooks, replacing legacy ETL logic with optimized Spark transformations to improve performance, scalability, and maintainability, achieving around 40 percent processing efficiency improvement while reducing Informatica licensing costs and optimizing cloud resource utilization.
• Designed and implemented Azure Data Factory pipelines to orchestrate end-to-end data workflows, replacing Informatica scheduling and improving reliability and monitoring.
• Developed scalable data ingestion frameworks using PySpark with JDBC connectors to load Data from Oracle into Delta Lake following Medallion architecture.
• Optimized data transformation logic using Spark-based processing, improving overall performance and reducing execution time.
• Migrated job orchestration from Informatica to Databricks Workflows and Azure Data Factory triggers, enabling better scheduling, dependency management, and monitoring.