Jayanth Penumarthi

Project: Mars Project - Centralized Data Lake Implementation on AWS

• Orchestrated a structured Big Data Storage Solution on AWS S3, establishing Landing, Raw, and Staging Zones with lifecycle management and versioning.
• Engineered a scalable ETL pipeline using Apache Airflow on EC2, managing end-to-end data workflows.
• Developed EMR-Spark jobs for large-scale data transformation, applying SHA-2 hashing for data masking and SCD2 logic for historical data accuracy.
• Integrated Apache Livy with EMR clusters for real-time analytics via REST-based Spark job submissions.
• Reduced data inconsistencies by 95% and cut infrastructure costs by 25% through optimized EMR resource provisioning.

Project: Zebra Client - Migration of Reporting Systems

• Led the migration of reporting systems from Oracle Analytics Cloud (OAC) to Power BI (PBI), ensuring 99.9% data accuracy.
• Implemented data validation frameworks using GCP BigQuery and Databricks, ensuring cross-platform consistency.

Project: Merck Client - Data Automation & Analytics Enhancement

• Automated data ingestion & transformation using advanced Python scripting, reducing manual effort by 50%.
• Contributed to AWS Data Fabric, implementing serverless architectures and event-driven processing for real-time analytics.

• Built an automated file processing pipeline using AWS Lambda & S3 triggers for structured CSV data storage in Amazon RDS.
• Designed Python-based Lambda functions with Boto3, executing parameterized SQL queries to prevent SQL injection.
• Enhanced query execution & indexing, integrated CloudWatch logging, and secured Lambda-to-RDS connectivity using IAM roles, improving data processing speed by 70%.