Gowtham Kumar M R
Bangalore
Summary
Big data engineer with over 5 years of experience in Spark and AWS, focusing on optimizing data pipelines for AML compliance. Demonstrated ability to enhance performance and accuracy through effective data transformation and collaboration across teams. Strong analytical and problem-solving skills contribute to increased efficiency and high-quality datasets for analytics and compliance reporting. Experienced in designing and managing large-scale distributed data processing systems, consistently delivering solutions that improve performance and reliability.
Overview
5
5
years of professional experience
1
1
Certification
Work History
BIG Data Engineer
Cognizant Technology Services
02.2024 - 02.2026
- Designed and developed Spark (Scala) based ETL pipelines to process large-scale structured and semi-structured datasets.
- Optimized Spark jobs for performance, enhancing processing time through partitioning, caching, and memory tuning.
- Executed data profiling, validation, and quality checks, ensuring accuracy of AML entity resolution results.
- Performed data cleansing, standardization, enrichment, and schema alignment prior to AML entity resolution processing.
- Implemented Quantexa Entity Resolution platform for Anti-Money Laundering (AML) use cases.
- Built entity matching, deduplication, survivorship, and relationship modeling logic within the Quantexa data model.
- Supported production deployment and monitoring, resolving issues in AML data pipelines to maintain operational integrity.
- Delivered investigation-ready AML entity datasets for compliance and risk teams.
- Technologies: Spark, Scala, Quantexa, HDFS, AWS
BIG Data Engineer
Girmiti Software Pvt Ltd
02.2022 - 09.2023
- Developed data processing workflows using Scala Spark and Spark SQL in Eclipse IDE.
- Designed and created a scalable data processing application and packaged it as a JAR.
- Extracted and integrated data from Amazon S3 and Web APIs, ensuring data consistency.
- Performed data transformation, deduplication, and complex data manipulations.
- Orchestrated final step of pipeline to store processed data in Amazon S3, delivering optimized data to BI Engineer for downstream analytics and reporting.
- Conducted comprehensive testing, monitoring, and debugging of data pipeline, ensuring reliability and accuracy.
- Worked closely with the DevOps team to ensure successful code deployment to Amazon EMR clusters.
Big Data Engineer
BigPerl Solutions Pvt Ltd
10.2020 - 01.2022
- Expertise in using Spark SQL to process large-scale structured and semi-structured data sets, including querying, filtering, mapping, reducing, grouping, and aggregating data.
- Skilled in using Spark SQL persistency and caching mechanisms to reduce data processing overhead and improve query performance.
- Utilized technologies such as Spark, Scala, Python, Hadoop, and Hive for data processing.
- Skilled in working with textual data formats in Spark, such as CSV, JSON, and XML, and their serialization and deserialization using Spark DataFrames and RDDs.
- Contributed to Hadoop migration project, improving data processing capabilities.
- Optimized Hive queries to reduce project costs.
- Conducted data analysis, data quality assessments, and data profiling to support business operations.
Education
S J C Institute of Technology -
S J C Institute of Technology
Chickballapur06-2020
Vijaya Chetana PU College -
Vijaya Chetana PU College
Kolar03-2016
Vimala Hridaya Composite High School -
Vimala Hridaya Composite High School
Gollahalli04-2014
Skills
- Hadoop
- Sqoop
- Hive
- Apache Spark
- AWS
- Cloudera
- SQL Server
- Scala
- Python
- SQL
- Linux
- Windows
- Spark programming
Certification
Databricks Data engineer Associate
Timeline
BIG Data Engineer
Cognizant Technology Services
02.2024 - 02.2026
BIG Data Engineer
Girmiti Software Pvt Ltd
02.2022 - 09.2023
Big Data Engineer
BigPerl Solutions Pvt Ltd
10.2020 - 01.2022
S J C Institute of Technology -
S J C Institute of Technology
Vijaya Chetana PU College -
Vijaya Chetana PU College
Vimala Hridaya Composite High School -
Vimala Hridaya Composite High School