Prajnaya Prakash Nayak
Bengaluru
Summary
Certified Azure Data Engineer and Databricks Spark Developer with over 9 years of experience in data engineering and ETL development. Expertise includes 6+ years in Azure Data Engineering, utilizing services like Azure Data Factory, Azure Databricks, and Azure Data Lake Storage (ADLS) to design and optimize scalable data pipelines. Proficient in transforming and integrating data across diverse environments with 3+ years of ETL experience using Informatica and PL/SQL. Led cloud migration projects, achieving significant cost savings for clients. Adept at performance optimization, automation, and delivering robust, secure, and efficient data solutions.
Overview
2025
2025
years of professional experience
Work History
Data Engineer
Société Générale
1 2022 - Current
- Company Overview: Société Générale aims to be a leader in the French banking industry, providing various services to its clients, including global investment management
- Gathered requirements from business analysts and translated them into technical design documents for team implementation
- Collaborated with regulatory teams to understand data requirements and implemented solutions to meet regulatory reporting standards
- Managed team engagement, work distribution, and presented to management for deployments and upcoming projects
- Developed and optimized scalable data pipelines using PySpark dataframe API, Spark SQL with Azure Databricks using Azure Data lake Storage
- Responsible for estimating the cluster size, monitoring, and troubleshooting of the Spark Databricks cluster
- Worked on performance improvement of data pipeline
- Société Générale aims to be a leader in the French banking industry, providing various services to its clients, including global investment management
Technology Lead
Infosys
12.2020 - 12.2021
- Company Overview: Liberty Global is a world leader in converged broadband, video, and mobile communications services
- Develop Spark applications using PySpark and Spark SQL for data extraction, transformation, and aggregation from multiple file formats for analyzing and transforming the data to uncover insight into the customer usage patterns
- Extract Transform and Load data from source systems to Azure Data Storage services using a combination of Informatica Cloud and Spark SQL, then ingestion of data to one or more Azure services (Azure Data Lake, Azure Storage, Azure SQL, Azure DW) and processing the data in Azure Databricks
- Prepared design documents and unit test documents for the developed code
- Liberty Global is a world leader in converged broadband, video, and mobile communications services
Associate IT Consultant
ITC Infotech
05.2019 - 12.2020
- Company Overview: BMC HealthNet Plan is an organization that offers health insurance coverage
- As part of this project develop Spark applications using PySpark and Spark SQL for data extraction, transformation, and aggregation from multiple file formats (parquet, CSV, JSON)
- Monitor system health and logs and respond accordingly to any warning or failure conditions
- Import data from various systems/sources like MySQL, Oracle to ADLS
- Prepared design document, unit test scripts for developed code
- BMC HealthNet Plan is an organization that offers health insurance coverage
ETL Developer
Tata Consultancy Services
03.2016 - 05.2018
- Company Overview: Santander Consumer Bank AS, Sweden
- Developed Informatica mappings to load the data from different file (VSAM, XML, Flat) types and different RDMS Systems (Oracle, SQL Server) till Staging, ODS, data marts
- Developed mappings/SQL Scripts to generate XML files from RDMS and flat files as per the Business requirement
- Implemented GDPR (General Data Protection Regulation) successfully
- Migrated SAS code (PROC SQL) to Informatica, PL/SQL scripts
- Prepared design document, Unit Test plan and unit test document along with Unit test cases for the developed code
- Prepared detail SOP for team to use for known production issues
- Santander Consumer Bank AS, Sweden
Informatica Developer
Tata Consultancy Services
03.2014 - 02.2016
- Company Overview: GE Healthcare
- Developed Informatica mappings to load the data from disparate source systems and load it at one place for analysis and reporting purpose
- Prepared Unit Test plan and Unit test cases for the developed code
- Rectify issues associated with load failures
- Solve incidents/Service request/ Change requests raised by business
- Prepared document /Job run book for support team
- GE Healthcare
Skills
PySpark
Hadoop
Azure Data Bricks
Elastic Search
Azure Data Factory
IICS/Informatica PowerCenter
Azure
Oracle
SQL Server
Teradata
DB2
Hive
PostgreSQL
ADLS
Python
Shell Scripting
PowerShell
SQL
Git
CI/CD
Kibana
Grafana
Erwin
Timeline
Technology Lead
Infosys
12.2020 - 12.2021
Associate IT Consultant
ITC Infotech
05.2019 - 12.2020
ETL Developer
Tata Consultancy Services
03.2016 - 05.2018
Informatica Developer
Tata Consultancy Services
03.2014 - 02.2016
Data Engineer
Société Générale
1 2022 - Current