Rajasankar Radhakrishnan

A seasoned Data Architect and Data Engineer with over 12 years of experience designing and implementing data platforms on hyperscalers like Azure and GCP. Skilled in utilizing cloud-native and open-source technologies to support leading automotive, manufacturing, and BFSI clients. Proficient in modern data architectures such as Lakehouse, DataMesh, and active metadata management, with a strong focus on data quality, automation, and integrity. Adept at designing metadata-driven solutions based on the Medallion architecture with integrated data governance. Additionally, possesses extensive knowledge of traditional dimensional modeling and data warehousing, along with expertise in Python/SQL-based analytics solutions. A proven leader, having successfully led cross-functional teams, driven digital transformation, and defined data roadmaps to deliver strategic value for clients.

1.Bosch Semantic Stack Data Layer(Oct 2023- till date):

Role: Data Architect/Senior Data Engineer

Project Description: Unified Data Platform Connective Automotive use cases in Bosch.

• Defined, Architected and Implemented a self-serve multi-tenant data platform to Fasttrack advanced analytics for automotive and manufacturing use cases.
• Analyzed multiple options(Fabric, Snowflake, Databricks) and shortlisted the best tools and technology based on different parameters in collaboration with product owners (PO) and management, ensuring alignment with business objectives.
• Led the development with two PODs focusing on delivery of Features such as Configurable Data pipeline Orchestration Framework, Delta Lakehouse Optimization, COVESA Standardized semantic layer etc.
• Implemented a user-friendly Framework for Role based access control, Row level and column level security using Databricks Unity Catalog on the data Layer.
• Provided technical mentorship, fostering team development through code reviews and guidance on best practices across API development Life cycle, Data archival framework.
• Collaborated with cross-functional teams to design and implement data extraction, transformation, and loading (ETL) processes in alignment with business objectives.

Tech stack: Azure, Databricks, ADF,ADLS Gen2,Azure Devops

2.CIMB-DataCore(Jan 2023-Nov 2023)

Role: Data Architect/Senior Data Engineer

Project Description: Real Time Lakehouse for Gold standard Master data consumption

• Setting up Unity catalog and the different data layers of the Lakehouse.
• Design and Implementation of real time data pipelines for data ingestion from kafka till consumption layer using Delta Live tables for low latency data availability.
• Implementation of different Data governance capabilities such as Role based access control, RLS and CLS using Databricks dynamic views and SQL functions.
• Implementation of configuration driven DLT module for easier onboarding on new data pipelines.

Tech stack: GCP Databricks, Unity catalog, GCS, Confluent Kafka, Spark   structured streaming.

3.VHIT-Lake house implementation(July 2022 – April 2023)

Role: Data Architect/Senior Data Engineer

Project Description: Real Time Lakehouse for Gold standard Master data consumption

• Architecting the solution for Data-lake implementation
• Creation of Technical design document, Test case document, user manual etc.
• Designing and implementing the dynamic data pipelines using ADB, ADF, SQL server, ADLS gen2 for validating the files, checking the data quality for completely metadata driven data lake.
• Creation of mount points, authenticating access between Azure resource using Service principle, managed identities, and key vault.
• Analyzing and optimizing the data loads for larger tables and files by tuning spark configuration, applying zorder, optimize, vacuum, partitioning.
• Monitoring spark jobs using Ganglia UI, Spark UI to understand the bottlenecks and optimize the jobs.
• Creation of Pyspark notebooks for validating files, data and business logic transformations and creation of wheel files for reusable code.
• Creation of ADF pipelines for orchestrating the various

Tech stack: Azure Databricks, Azure data factory

4.DFL-2.1(Nov 2021– June 2022)

Role: Data Architect/Senior Data Engineer

Project Description: Migration of existing Big data application from Synapse to Databricks and implementation of Data lake for processing data from 5 different applications.

• Conversion of existing Synapse stored procedures as Pyspark and Spark SQL notebooks.
• Reduction of data load time by 75 % and cost savings by over 80% compared to the existing Data Warehouse.
• Implementation of custom framework to do incremental load from Source databases even in the absence of watermark columns in source systems.
• Smart spark job parallelization framework based on source table row count to reduce load time.
• Implemented Json read module to dynamically identify and decode encrypted json files from Iothub.
• Implemented Lifecycle management for Datalake to set appropriate retention periods and layers (hot, cold, archive) to reduce storage cost.
• Creation of Grafana monitoring reports using Azure log analytics as source and KQL for transforming data.

Tech Stack: Azure Databricks , Azure data factory, ADLS gen2,Azure SQL server, Pyspark , IOTHUB, PowerBI , Grafana

5.DataNexus: Factory of Future(Feb 2020– June 2021 )

Role: Data Engineer Specialist

Project Description: Migration of existing Big data application from Synapse to Databricks and implementation of Data lake for processing data from 5 different applications.

• Implementation of Authentication using which multiple services can access each other securely using managed identities, service principle and key vault.
• Creation of File validator module using higher order functions which lets users configure dynamic File validation rules.
• Creation of Data quality validator module which lets users configure dynamic Data quality validation rules.
• Processing IOT json files using Databricks Autoloader(cloudfiles) for maintaining a near real time gold layer.
• Creation of Mock data generation framework for removing the data non-availability hindrance.
• Creation of Alert framework to inform stakeholders of job failures/validation failures as per alert configuration.

Tech Stack:

Azure Databricks , Azure data factory, ADLS gen2,Azure SQL server, Azure functions. Pyspark

Mixing 2.0(Jan 2018 – Aug 2020)

Role: Data Engineer

Project Description: Implementation of recipe generation algorithm to shortlist best set of raw materials (from half a million possibilities) from stock warehouse for Push belt production. The objective to is to enhance the pushbelt quality and reduce scrap.

• Created a custom algorithm(using numpy , pandas and statsmodels library) in Python to shortlist best set of raw materials(from half a million possibilities) from stock warehouse for Push belt production, resulting in time reduction of stock preparation stage by 75%.
• Created and deployed a suite of microservices in python for cleaning stock data, predicting missing measurements and identifying best set of stock data.

Tech Stack: Python, pandas, numpy , Flask, Postman, pycharm

Titan MIS/GHS Reporting(Jan 2016 – Jun 2016)

Role: Sql Developer/ETL Developer

Project Description: Development and maintenance of MIS system that hosts sales,inventory,after service reports of various business segments.

• Designed and developed the data warehouse for Titan Management information system.
• Upgraded existing Crystal reports to SSRS and Power BI reports resulting in 75% decreased runtime of reports.
• Used SSIS SCD components to capture slowly changing dimensions to generate historically accurate reports.
• Upgraded existing C# /HTML reports by Converting inefficient Microsoft SQL Queries to Recursive CTEs and efficient Stored procedures.
• Used Snapshot and Cache mechanism of SSRS to reduce the load on data warehouse.

Supported and maintained existing report portals and data base ecosystem by debugging complex SQL queries and fixing bugs when needed

Tech Stack:SSIS, SSRS, SQL server, C#,ASP.net,Python