Yukti Singh

Amazon – Data Spark Automation Projects (Offshore_1)
Duration: February 2025 – May 2025
Client: Amazon
Role: PySpark Developer

Designed and implemented scalable PySpark solutions to support the training of a language model that automates the analysis of migration changes between Apache Spark versions.

Built modular, high-performance PySpark scripts for processing large-scale datasets used in model training and evaluation.

Authored clear, structured documentation detailing script logic, data pipelines, and output formats for client reference and maintenance.

Conducted comprehensive data quality assessments to ensure output accuracy, consistency, and alignment with project requirements.

Client-Corten Logistics

• Objective: Optimize vehicle transportation for clients like Hyundai and BMW by automating data workflows, improving efficiency, and reducing costs.
• AWS Lambda: Serverless processing for data validation and integration.
• Amazon S3: Storage for raw data.
• Amazon DynamoDB: Scalable NoSQL database for structured data storage.
• AWS Glue: ETL processing for data transformation.
• AWS Step Functions: Workflow automation and orchestration.

Key Features:

• Automated data ingestion from emails using Amazon SES.
• Real-time data processing with AWS Lambda and SageMaker.
• Scalable data storage and management in S3 and DynamoDB.
• Workflow automation with Step Functions and CloudWatch for monitoring.
• Data-driven decision-making with QuickSight reporting.
• Outcome: Streamlined cargo management, enhanced efficiency, cost reduction, and improved customer satisfaction.

• Company Overview: United Healthcare Data Management Project Summary
• Objective: Streamline customer records and improve data quality, reporting accuracy, and operational efficiency.
• Challenges:
• Fragmented data systems (Salesforce, Market, NetSuite).
• Poor data quality and misalignment.
• Inefficiencies in reporting and operations.
• Phases:
• Client Admin: Manage broker and client admin access, including ACA features.
• Partner Admin ACA: Provide comprehensive data views for associated companies.
• Non-Partner Integration: Integrate SSO and HR Logics for seamless authentication and data exchange.
• Data Framework:
• Storage: AWS Glue (new data), Salesforce (old data).
• Processing: Normalization, security enhancements, and quality checks.
• Tools: Openpyxl, FuzzyWuzzy, and Parquet formats for efficiency.
• Workflow:
• Roles: UHC – Broker – Client – User.
• Revenue: UHC profits, brokers earn commissions, clients receive policies.
• Logic: Handle scenarios for missing/existing partners or clients.
• Integration: ThinkHR tracks policy updates and renewals.
• Outcome: A unified, secure, and scalable data management system enabling better business decisions.
• United Healthcare Data Management Project Summary

• Company Overview: Optimization of Project Costing - SummaryTech Stack
• Objective: Leverage machine learning to predict budgets, estimate operational costs, and enhance resource management.
• Key Goals:
• Forecast project costs to avoid over/underspending.
• Estimate daily operational expenses.
• Improve resource allocation and efficiency.
• Process:
• Clean and transform data for analysis.
• Use XGBoost and regression models for predictions.
• Deploy models via Django and monitor with AWS CloudWatch.
• Outcomes:
• Achieved 94% accuracy with XGBoost.
• Provided reliable cost ranges for budgeting.
• Result: Accurate budgeting, cost control, and enhanced project efficiency.
• Optimization of Project Costing - SummaryTech Stack