Dharshini SS

• Implementing robust Cloud Data Fusion pipelines for validating dynamic file attributes (name, extension, headers, trailers), with runtime parameterization
• Designing efficient JavaScript Transformations to process files from GCS buckets with consolidated error handling mechanisms
• Successfully deploying Airflow DAGs with TaskGroup, streamlining orchestration and workflow automation
• Working extensively with Denodo, enabling seamless data integration, virtualization, and governance across multiple platforms
• Trained and supported new team members, maintaining culture of collaboration.

• Architected and implemented an end-to-end data cataloging solution, improving metadata accuracy and accessibility
• Optimized Spark jobs to process vast amounts of data efficiently, reducing processing time and enhancing system performance
• Automated ETL workflows using Autosys, ensuring seamless data ingestion, transformation, and catalog updates
• Designed scalable Hive data models, enhancing query performance and ensuring compatibility with business intelligence tools
• Provided governance and compliance solutions by building metadata lineage and auditing capabilities within the data catalog

• Involved in analyzing business requirements and prepared detailed specifications that follow project guidelines required for project development
• Developed Hive SQL queries, Mappings, tables, external tables in Hive for analysis across different banners and worked on partitioning, optimization, compilation, and execution
• Wrote complex queries to get the data into HBase and responsible for executing hive queries using Hive Command Line
• Automated workflows using shell scripts to pull data from various data bases into Hadoop
• Wrote Hive jobs to parse the logs and structure them in tabular format to facilitate effective querying on the log data
• Implemented Spark using Scala and Spark SQL for faster testing and processing of data

• Involved in analyzing business requirements and prepared detailed specifications that follow project guidelines required for project development
• Created and maintained technical documentation for launching Hadoop Clusters and for executing Hive queries
• Developed reusable transformations to load data from Flat files and other data sources to the Data Warehouse
• Implemented Spark using Scala and Spark SQL for faster testing and processing of data
• Used Spark Streaming to divide streaming data into batches as an input to Spark engine for batch processing
• Designed batch processing jobs using Apache Spark to increase speed compared to that of MapReduce jobs
• Wrote Spark applications for data validation, cleansing, transformation, custom aggregation
• Imported data from disparate sources into Spark RDD for processing
• Developed custom aggregate functions using Spark SQL and performed interactive querying

· Worked as a Test Lead handling multiple projects simultaneously, implementing new automation frameworks solution, Learning new technologies for the migration of one technology to another.

· Developed various Test frameworks libraries and maintain the old frameworks in optimistic way Has been part of creating dashboards using Kibana and using Grafana (For GCP Services)

· Designing the Test cases and executing the same in various testing cycles.

· Raising Defects, coordinating with developers for clarifications of defects raised, Performing Regression Testing and closing the defects

• Worked as EDI developer using WTX 8.4
• Worked as Quality Assurance Professional for Testing API and Middle Ware Services
• Has developed Automation Framework using RestAssured and Java