Accomplished Big Data Engineer with over 9.5 years of experience in managing complex data systems and utilizing Hadoop tools including HDFS, Hive, Sqoop, and Spark. Specializes in developing and optimizing Spark RDD workflows with Scala and Python to improve query performance and resource efficiency. Skilled in processing large structured and semi-structured datasets through Spark DataFrame APIs, ensuring data quality and integrity.
Hi, I’m
Monalisa Bhambal
Data Analyst
Pune
Summary
Overview
6
years of professional experience
Work History
ID Medical
Big Data Engineer
06.2022 - Current
Job overview
- Developed Spark applications to enable efficient distributed data processing.
- Developed custom Spark connectors for data ingestion.
- Created and managed Spark clusters for distributed computing.
- Utilized Spark for real-time recommendation systems.
- Conducted Spark job performance tuning.
- Debugged complex Spark data transformations in PySpark jobs on AWS EMR.
- Developed custom Spark aggregations for reporting.
- Conducted performance tuning for Spark jobs on AWS EMR to optimize processing speed and resource utilization.
- Worked with Spark's data serialization formats (Avro, Parquet, JSON, etc)
- Debugged complex data transformations in PySpark jobs on AWS EMR to ensure reliable data output.
- Managed Spark job orchestration on AWS EMR using AWS Step Functions.
- Automated cluster creation and job submission on AWS EMR using PySpark.
- Implemented data aggregation and transformation in PySpark jobs on AWS EMR.
- Debugged memory and performance issues in PySpark jobs running on AWS EMR.
- Configured EC2 instances as part of AWS EMR clusters for running PySpark jobs.
- Used AWS Hive to query structured data within AWS EMR jobs.
- Technologies: Spark, Pyspark, Python, HDFS, Hive, AWS
- Technologies: Spark, Pyspark, Python, HDFS, Hive, AWS
- Integrated real-time streaming technologies for accurate monitoring of critical business metrics.
- Automated routine tasks through scripting languages, reducing manual effort and human error risks.
- Proactively addressed potential bottlenecks in the ETL process through regular monitoring, enabling seamless workflow operations.
Tech Mahindra
ETL Tester
11.2021 - 05.2022
Job overview
- Enhanced data quality by identifying and resolving ETL-related issues in a timely manner.
- Validated data accuracy through comprehensive testing of source-to-target mappings and transformations.
- Mentored team members on best practices in ETL testing methodologies, enhancing overall team competency.
- Elevated team performance by providing mentorship and guidance on best practices in ETL testing methodologies.
- Maintained high-quality documentation of test plans, scripts, and defects to facilitate knowledge sharing among team members.
- Analyzed ETL-related metrics and trends, contributing to informed business intelligence decisions.
- Validated cube processing functions to ensure accuracy in calculations and aggregations in reports.
Altruist Technologies
ETL Tester
12.2019 - 12.2020
Job overview
- Proficient in SQL queries and scripting for data validation and verification during ETL testing.
- Expertise in testing ETL metadata repositories and data catalogs.
- Documented test plans, test cases, and test results to ensure comprehensive ETL testing coverage.
- Experienced in conducting data validation and reconciliation between source and target systems in ETL testing.
- Conducted data migration testing and validated data conversion to ensure data integrity during ETL processes.
- Applied ETL performance tuning techniques to enhance system optimization and data processing efficiency.
- Expertise in testing data warehousing concepts, such as dimensional modeling and star schemas.
- Proficient in testing ETL data lineage and data traceability.
- Experienced in testing ETL metadata management tools and data dictionaries.
- Skilled in testing data synchronization and replication across distributed systems.
- Strong understanding of data deduplication and data consolidation techniques in ETL testing.
- Knowledgeable about testing data migration from legacy systems to modern platforms using ETL processes.
- Familiarity with testing ETL processes in real-time analytics and reporting systems.
- Expertise in testing data transformation rules and business logic applied during ETL processes.
- Proficient in using ETL testing tools and frameworks, such as QuerySurge, Talend Data Quality, or Informatica Data Validation Option.
- Led project to verify data integrity and accuracy.
- Utilized SQL for data analysis and query execution.
- Ensured data integrity by validating end-to-end data flows in complex, multi-tier systems.
- Optimized ETL processes for faster data extraction, transformation, and loading.
Education
AISSMS College of EngineeringPune
Bachelor of Science
04.2001
University Overview
GPA: First Class
Sardar Dastur Junior CollegePune
High School Diploma
04.2001
University Overview
GPA: First Class
ST. Felix High SchoolPune
01-2015
University Overview
GPA: First Class
Skills
Big data analytics
Data engineering
Data pipeline design
ETL development
Data migration
Data integration
Data lake management
Spark development
Hadoop ecosystem
Apache flink
Real-time processing
Stream processing
Data quality
Data modeling
Databases: SQL Server, MySQL, Oracle
NoSQL databases
SQL programming
Database optimization
SQL replication
Data warehousing
Data visualization
Python programming
Data analysis
Data strategy
Data-driven decisions
Analytical thinking
Active listening
Critical thinking
Decision-making
Adaptability
Detail-oriented
Effective communication
Excellent communication
Active listening
Teamwork
Time management
Team building
Multitasking
Time management
Workflow optimization
Business understanding
Team building
Detail-oriented
Adaptability
Data-driven decisions
Analytical thinking
DECLARATION
DECLARATION
I hereby declare that the above statements are true and complete to the best of my knowledge and belief.Timeline
Big Data Engineer
ID Medical
06.2022 - Current
ETL Tester
Tech Mahindra
11.2021 - 05.2022
ETL Tester
Altruist Technologies
12.2019 - 12.2020
AISSMS College of Engineering
Bachelor of Science
04.2001
Sardar Dastur Junior College
High School Diploma
04.2001
ST. Felix High School