Teja Chowdhary

Description: Worked on end-to-end data engineering projects, developing reliable data pipelines and reporting solutions. Successfully migrated data from legacy systems to modern platforms, improved data quality and security, and created impactful dashboards for enhanced decision-making. Also supported automation, performance tuning, and collaborated effectively with teams in agile environments.

Responsibilities:

§ Developed scalable data engineering solutions in Azure utilizing services like Azure Data Factory (ADF), Data Lake Analytics, HDInsight, Azure Synapse, and Azure Databricks for both batch and streaming data pipelines.

§ Designed and implemented sophisticated ETL/ELT solutions with ADF v2, orchestrating data integration from both on-premise and cloud endpoints(Blob Storage, Azure SQL, Oracle, REST APIs) using a comprehensive suite of activities and Databricks notebooks.

§ Designed and maintained ADF Linked Services, Datasets, and Data Flows for efficient ETL operations, moving data into Azure Data Lake Storage Gen2, Azure SQL, andSynapse Analytics.

§ Automated data ingestion and transformation processes in ADF through the strategic use of Schedule, Tumbling Window, and Event-Based Triggers.

§ Developed PySpark/Scala/Spark SQL notebooks in Azure Databricks for the cleansing, transformation, and enrichment of large-scale datasetsstored in ADLS Gen2.

§ Implemented robust real-time data pipelineswith Azure Stream Analytics, Event Hub, and Service Bus, delivering processed data to Power BI and SQL Data Warehouse for dynamic insights.

§ Designed and implemented Slowly Changing Dimension (SCD) Type 1 & Type 2 data integration frameworks within Azure Data Lake using U-SQL, PySpark, and Databricks workflows.

§ Performed data cleansing, feature engineering, and missing value imputation on structured and semi-structured datasets using Python and Spark.

§ Successfully migrated legacy ETL workloads toAzure cloud by developing SSIS packages and ADF-SSIS IR integrations.

§ Developed and optimized Azure Synapse views, stored procedures, triggers, and partitioned tables to enhance performance and query efficiency.

§ Implemented end-to-end observability using Azure Monitor, Log Analytics, and Spark UI to track pipeline health, performance, and data quality.

§ Developed interactive Power BI dashboardsfor banking executives and operations teams, enabling real-time monitoring of loan disbursements, customer on boarding funnel, account activity trends, and compliance SLAs using data from Azure SQL, Synapse, andADLS.

§ Designed and deployed Power BI's Row-Level Security (RLS) to enforce strict data access policies for various banking roles, such as branch managers, financial advisors, and audit teams, directly contributing to enhanced data confidentiality and adherence to regulatory standards.

§ Created DAX measures, calculated columns, and custom visuals in Power BI Desktop to address complex business logic and data modeling needs.

§ Automated Power BI dataset refreshesvia Dataflows and Power BI Service, integrating real-time data from Azure Event Hubs to monitor banking transactions, fraud detection alerts, and customer interactions across digital channels.

§ Utilized Power BI Embedded to integrate Power BI reports directly intoSharePoint and custom applications, ensuring seamless data access for various business units.

§ Collaborated with business teams to gather visualization requirements and delivered self-service BI solutions using Power BI, Excel Power Query, andSSRS.

§ Successfully transitioned on-premise data pipelines to Azure cloud, implementing solutions with ARM Templates, ADF, and Databricks.

§ Developed and optimized Spark jobs for efficient batch processing of large datasets in various formats, including JSON, CSV, ORC, and Parquet.

§ Optimized data storage and query performance by designing effective partitioning and bucketing strategies inHive and Spark.

§ Maintained data governance by integrating solutions with Azure Key Vault, RBAC, and managed identities.

§ Actively participated in Agile/Scrum sprints, contributing to backlog grooming and effort estimationfor new data platform enhancements.

Description: Developed and deployed enterprise-grade data pipelines for a large banking initiative, facilitating the migration of legacy systems to Azure and establishing real-time analytics capabilities for financial operations. Managed multi-system data integration, implemented real-time risk analytics processing, and built Power BI dashboards for executive insights. Actively contributed to data solutions supporting KYC/AML, Basel reporting, credit risk modeling, and customer behavior analysis.

Responsibilities:

§ Designed and implemented end-to-end data pipelines using Azure Data Factory, Azure Data Lake Gen2, and Azure SQL to support scalable data ingestion and transformation.

§ Built and deployed ELT pipelines for batch and real-time processing using PySpark, Azure Databricks, and Azure Stream Analytics.

§ Successfully transitioned data from legacy systems and on-premise SQL Servers to Azure SQL Database and Synapse Analytics, utilizing ADF and SSIS.

§ Developed Power BI dashboards to track credit utilization, risk exposure, loan performance, and regulatory compliance trends.

§ Designed and implemented robust real-timedata pipelines leveraging Kafka, Event Hubs, and Spark Structured Streamingto power dynamic tax computation engines.

§ Created and scheduled ADF pipelines to extract data from heterogeneous sources (RDBMS, flat files, APIs) into Azure Data Lake.

§ Developed and executed data transformations within Databricks notebooks using PySpark, incorporating parameterized control through ADF widgets.

§ Developed highly optimized dimensional models and star schemas to enhance Power BI reporting for credit risk scoring, transaction history, and customer segmentation.

§ Built Hive tables and executedHiveQL for pre-aggregated datasets, significantly improving reporting performance.

§ Utilized Informatica for ETL developmentand integrated it with Azure resources for legacy-to-cloud migration.

§ Designed and orchestrated CI/CD pipelines using Git, Azure DevOps, and environment-specific configurations.

§ Utilized HDInsight, HDFS, HBase, Pig, Hive, and Sqoop to manage hybrid big data workloads and facilitate cloud migration efforts.

§ Implemented data validation rules to ensure accuracy of financial data pipelines supporting KYC, AML, Basel reporting, and regulatory compliance.

Description: As a Data Engineer, I was part of a project focused on updating and improving data systems. I built solutions for data movement and cleansing, created visual reports for better understanding, and assisted with real-time data processing, leading to clearer insights.

Responsibilities:

§ Assisted in developing ETL pipelines using SSIS and T-SQL for data ingestion into the enterprise Data Warehouse.

§ Contributed to data cleansing and file format conversion efforts through PySpark and Spark SQL transformations.

§ Participated in the migration of on-premise data(SQL Server, Oracle) to Azure Data Lake using ADF (V2).

§ Created PySpark scripts focused on fundamental data extraction, transformation, and aggregation tasks.

§ Assisted in the configuration and management of Databricks clusters to support batch processing workloads.

§ Designed and built straightforward dashboards inPower BI and Tableau to support internal business reporting requirements.

§ Utilized Azure SQL and Azure Storage Explorer for managing and validating ingested datasets.

§ Gained exposure to Kafka-based data streaming into Databricks using JSON format files.

§ Created real-time Power BI dashboards integrating with Kafka-streamed data to track product movement and inventory fluctuations across regional warehouses.

§ Developed Power BI visuals based on Talend-processed data to monitor data ingestion success rates and error patterns.

§ Collaborated with the team to handle data stored in Avro and Parquet formats.

§ Used Python libraries (Pandas, NumPy)to support small data exploration tasks.

§ Maintained thorough documentation of data lineage and addressed data quality issues identified during ETL job execution.