Harshanth DS

Client : PesiCo

Domain: Data Governance

Team Size: 10

Environment: Python, FastAPI, OpenAI, Azure OpenAI, LangChain, LangGraph, Pandas, Pydantic, REST APIs, Git, GitHub Actions.

• Developed an Assisted Data Module for PepsiCo to automate metadata generation and source-to-target mapping using Generative AI (LLM) with FastAPI, LangChain, and LangGraph, enhancing data lineage traceability and accelerating onboarding of new data assets.
• Built LLM-powered agents to extract schema intelligence and semantic relationships from source systems, enabling smart metadata enrichment and integration workflows.
• Designed and optimized data ingestion workflows using Cloud Dataflow to process and load streaming data from sources like Apache Kafka or Cloud Pub/Sub into BigQuery in near real-time.
• Implemented Change Data Capture (CDC) mechanisms to capture real-time data changes from relational databases and stream them into BigQuery for up-to-date analytics and reporting.

Client : Citi Bank

Domain: Banking

Team Size: 20

Environment: Hadoop, HDFS, Hive, PySpark, Spark SQL, Cloudera, GCP, AirFlow, Terraform, Qlik Replicate, DDIT

Project Details:

• Built end-to-end data pipelines using munging, wrangling, curation, enrichment, and windowing techniques; created views for Data Governance and handled SCD Types 1, 2, and 3.
• Migrated Hadoop Spark jobs to GCS and DataProc; designed optimized Spark jobs for large-scale processing.
• Used Cloud Composer (Airflow) for orchestration; integrated with BigQuery, GCS, and Pub/Sub; stored outputs in the curated layer.
• Used Qlik Replicate to load Oracle/MySQL into BigQuery; leveraged Terraform for GCP resources; built ingestion framework for Rocket data into EAP, creating 3 key tables.

Client : Standard Chartered

Domain: Banking

Team Size: 20

Environment: Hadoop, Hive, SQL, PySpark, Airflow, Jupyter, Putty, Oracle, Unix, GIt, Control-M

Project Details:

• Conducted TPSA-MFU security risk assessments to evaluate Information and Cyber Security (ICS) controls for third parties with access to the bank's non-public information.
• Managed MOSAIC | HR Feed Sourcing, extracting HR data from the COO Data Lake and loading it into the MOSAIC system for streamlined data integration.
• Led LK ITRS regulatory project for the Central Bank of Sri Lanka, ensuring compliance by reporting all cross-border and domestic foreign currency transactions using ITRS-specified formats.
• Implemented ACC Dashboard, integrating scan and schedule data to provide a high-level compliance view of CAAT controls across applications.

Client : ICICI Lombard

Domain: Ecommerce, Yield , Price Management

Team Size: 10

Environment: Hadoop, Hive, SQL, Scala Spark, Airflow, Jupyter, Putty, Oracle, Unix, Git

Project Details:

• Collaborated on a sales analysis application for Co-operative Group Ltd., one of the UK's largest retailers, focusing on analyzing sales and profits across major brands and promotional schemes using Hadoop's historical data.
• Skilled in Hive script development, including creating and altering databases and tables, with a strong understanding of Hive execution for performance optimization.
• Handled various file formats such as CSV, AVRO, XML, JSON, and PARQUET for efficient data processing.
• Developed and optimized Sqoop jobs for incremental data loads from heterogeneous RDBMS to HDFS.

EWatch, IT Infrastructure Management Platform

Client: Fullerton

Domain: Retail

Team Size: 20

Environment: Oracle, Sqoop, Hive, SQL, Oozie, Hadoop, HDFS

Project Details:

• Contributed to MWatch, an integrated IT infrastructure management platform, by writing DDL and DML scripts for data transformation and populating target tables.
• Automated data loading tasks in HDFS using Control-M workflows and shell scripts, and developed Oozie workflows for scheduled batch processing and reporting.
• Experienced in HiveQL, creating external tables, moving data between layers, and performing data enrichment tasks like filtering, sorting, and aggregation.

Talisma (CRM) Management System

Client: ICICI Lombard

Package: Customer Relationship Management (CRM)

Team Size: 10

Environment: Vb. Net, Win form, SQL Server 2000, Oracle 9i, CRM tool

Project Details:

• Worked on Talisma CRM to customize and extend its capabilities for the insurance domain, utilizing ActiveX technologies and the Talisma SDK to meet specific business requirements.
• Encapsulated core business logic within components to ensure seamless integration and functionality across the CRM platform.