LOKESHWARAN R

Pharmaceutical client.

• Implemented an ETL pipeline in PySpark using Databricks and MySQL, with Azure Data Lake, transforming raw data into meaningful insights for pharmaceutical health analytics.
• Strengthened Delta Lake frameworks using PySpark and Spark SQL in Databricks, improving data accessibility and reliability.
• Utilized various data file formats, reducing data processing time by 25%, and enhancing workflow efficiency.
• Engineered data governance in Unity Catalog, streamlining access for 200+ users, and increasing data accuracy.
• Executed a proof of concept for the Databricks Asset bundle, enhancing workflow deployments, and achieving a 25% efficiency improvement.
• Implemented a proof of concept for a data pipeline from the source to the Gold layer with Delta Live Tables, reducing latency by 20%.
• Optimized query performance using Adaptive Query Execution (AQE), reducing data retrieval times across complex datasets.

Banking Client

• Designed and developed data ingestion into ADLS Gen2 using Synapse Spark spool with recon.
• Developed and optimized transformation logic using PySpark, resulting in a 25% reduction in processing time for reporting views
• Automated data processing systems efficiently handle over 10 terabytes of data daily, reducing processing time by 40%.
• Proficient in handling various data file formats, including Parquet, CSV, JSON, and Delta tables.
• Optimized query performance by 40% through effective partitioning and bucketing strategies in large-scale data sets
• Implemented a CI/CD pipeline using Azure DevOps, reducing deployment time, and enhancing release frequency.
• Developed a proof of concept with Databricks Streaming DataFrames, reducing data processing latency in real-time applications.
• Prioritized and organized tasks to efficiently accomplish service goals.

Healthcare Client.

• De-identification of metadata fields from various applications to classify: PHI, PII, PCI.
• Process the raw data from the Enterprise Data Warehouse and transform it based on the requirements.
• Worked on Informatica PowerCenter and Test Data Manager tools.
• Develop data masking and data subsetting techniques.
• Design and develop Power Center mappings, mapplets, sessions, and workflows.
• Migrate code moves from the Test to the QA repositories.
• Create metadata patterns, policy creation in ILM, TDM.
• Perform unit testing, fix issues, and validate target tables.
• Develop Informatica workflow automation, leveraging Shell scripts.
• Configure the scheduling of workflows to run in particular time frames.
• Worked on various transformations and performance tuning in Informatica PowerCenter.
• Analysis of HL7/EDI transaction files in B2B Informatica Developer.
• Develop data transformation in legacy systems: IBM Mainframe using Power Exchange.