Sandeep V K S

Part of the Hadoop Engineering team under the Strategic Data Platform, working on secure data ingestion, job orchestration, automation, and platform modernization initiatives.

Key Responsibilities:

• Designed and maintained Hive-based ETL workflows to support large-scale data processing across the enterprise data lake.
• Automated ingestion processes and validations using Python scripting, enhancing reliability and reducing manual effort.
• Built and scheduled complex batch workflows using Autosys, ensuring efficient orchestration, dependency handling, and SLA compliance.
• Managed file-to-Hadoop and table-to-table data loads using internal ingestion frameworks, including SEE extraction and Sqoop ingestion.
• Applied Protegrity data masking to safeguard sensitive data and meet internal compliance policies.
• Collaborated with cross-functional teams to onboard and support new data feeds, ensuring quality, consistency, and schedule alignment.

Migration and Upgrade Contributions:

• Participated in multiple major infrastructure upgrades, including: Spark 2 to Spark 3 migration, Cloudera CDH to CDP platform migration, and RHEL 7 to RHEL 9 OS migration.
• Actively involved in testing and validating job compatibility, performance, and output correctness post-migration, to ensure a smooth transition with zero data loss or SLA breaches.

Key Achievements:

• Maintained SLA compliance across 50+ business-critical Autosys workflows post-migration with minimal impact.
• Identified and resolved compatibility issues in Hive and Python scripts during Spark and CDP transitions.