Harshita Jain
Summary
Data engineer analyst with over 2.5 years of experience in developing batch and streaming ETL pipelines utilizing SQL, PySpark, Airflow, and Azure. Proven track record in processing large datasets, optimizing job performance, and resolving production issues. Expertise in constructing reliable data pipelines to enhance reporting capabilities.
Overview
1
1
Certification
5
5
years of professional experience
Work History
Data Engineer Analyst
Accenture
Gurugram
11.2023 - Current
- Designed and developed batch-based incremental ETL pipeline, reducing unnecessary full data loads and improving processing efficiency.
- Optimized complex SQL transformations involving joins, aggregations, and data validation rules; leveraged indexes, PARALLEL hints, and query tuning techniques to improve ETL performance and reduce processing time by 40%.
- Implemented idempotent data pipelines using checkpointing, control tables, and restart mechanisms to ensure safe job reruns without duplicate data generation.
- Designed data quality frameworks for null validation, duplicate detection, schema validation, and business rule checks, improving downstream data reliability.
- Analyzed and debugged production ETL pipelines to resolve data quality issues including missing data, duplicate records, and constraint errors, ensuring reliable downstream data availability.
Data scientist Intern
Shubhdeep Techies I.T. Solutions Pvt ltd
Gwalior
09.2021 - 12.2021
- Conducted research and analysis to build machine learning models with Python, utilizing TensorFlow and Scikit-learn to enhance predictive accuracy.
- Implemented computer vision techniques using OpenCV for image preprocessing, object detection, and feature extraction, improving model performance.
- Applied deep learning algorithms to solve image classification, object recognition, and facial recognition problems.
- Preprocessed large datasets for model training, ensuring high data quality and effective transformations with Python libraries like Pandas and NumPy.
Education
B.Tech - Information Technology
Institute of Technology & Management (ITM)
Gwalior08-2023
Skills
- ETL & Orchestration tools : Pentaho Data Integration (PDI), Apache Airflow, Pyspark, SparkSQL
- Big Data & Cloud: Azure Databricks, AWS, Hadoop, Spark
- Databases & SQL: MySQL, Oracle Database, SQL, CTEs, Window Functions, Query Optimization
- Programming Languages: Python, SQL, C
- Tools & Technologies: Git, Jira, Postman, Linux, Unix
Certification
- Databricks Certified Associate Developer, 02/01/24, 02/01/27
- AWS Certified Cloud Practitioner, 05/01/22, 05/01/25
- Microsoft Technology Associate (Introduction to Programming using Python), 06/01/22, 06/01/25
Timeline
Data Engineer Analyst
Accenture
11.2023 - Current
Data scientist Intern
Shubhdeep Techies I.T. Solutions Pvt ltd
09.2021 - 12.2021
B.Tech - Information Technology
Institute of Technology & Management (ITM)