Summary
Overview
Work History
Education
Skills
Websites
Certification
Languages
Timeline
Generic

Jagadeesh Pamarthi

Chennai

Summary

Data engineer with 8+ years of expertise in data warehousing, analytics, and big data technologies. Skilled in SQL, UNIX, and ETL tools, with proficiency in programming languages such as HQL, GCP, and Scala. Demonstrated experience with Big Data technologies including HDFS, Apache Spark, and Kafka, along with a strong focus on process improvement and data manipulation. Knowledgeable in Agile methodologies and tools like Git, JIRA, and Confluence, promoting continuous development and integration.

Overview

9
9
years of professional experience
1
1
Certification

Work History

Senior Data Engineer

Paypal
Chennai
09.2023 - Current
  • Designed data pipelines to facilitate efficient integration of systems.
  • Developed ETL processes transforming raw data into actionable insights.
  • Collaborated with cross-functional teams to establish data architecture and standards.
  • Implemented quality checks ensuring accuracy and reliability of datasets.
  • Optimized database performance through indexing and query tuning.
  • Mentored junior engineers on data engineering and analytics best practices.
  • Automated report generation for timely information access using scripting languages.
  • Analyzed system performance metrics to identify infrastructure improvement opportunities.

Senior Consultant

Deloitte India LLP
Chennai
06.2020 - 09.2023
  • Enhanced transactional model and customer profile data for regulatory reporting on ad-hoc, daily, monthly, and quarterly schedules.
  • Gathered business inputs from management to develop optimized ETL jobs using HiveQL and Shell scripting.
  • Ensured timely delivery of ad-hoc requests and requirements from business stakeholders.
  • Delivered Zettle transactions and financial regulatory data quality reports punctually to downstream teams.
  • Improved Fin-Reg data quality report by reconciling metrics from Zettle transactions using HiveQL and Shell scripting.
  • Analyzed and raised queries to maintain high data quality and alignment across all processes.
  • Resolved data issues swiftly to support business needs within established timelines.
  • Optimized Hive scripts to enhance code quality and efficiency, reducing memory usage while handling large data volumes.

Data Engineer

Foray Software
Chennai
04.2019 - 05.2020
  • Developed claims pipeline using Spark/Scala for airline domain, enhancing data processing capabilities.
  • Performed requirement analysis and prepared design documents to ensure project alignment.
  • Aggregated data from parquet files and loaded it into HDFS using Spark/Scala solutions.
  • Utilized checkpoint nodes to maintain and persist file system metadata within HDFS.
  • Executed extensive data loading into Hive warehouse through SparkSQL/Scala.
  • Re-designed applications and scaled database structures to improve efficiency.
  • Optimized SparkSQL/Scala scripts to enhance code quality and reduce memory usage for large datasets.
  • Conducted unit testing, data validation, and UAT bug fixes to ensure data integrity.

Software Engineer

Experis IT
Pune
02.2017 - 02.2019
  • Revamped application using Spring MVC Architecture, transitioning from Struct Framework.
  • Applied Agile Methodology across all phases, including Sprint Planning and UAT Testing.
  • Delivered detailed data analysis reports by aligning solutions with business and market needs.
  • Conducted tool evaluations and Proof of Concept (POC) assessments, outlining advantages and limitations.
  • Troubleshot issues to ensure timely delivery of sprint objectives.
  • Supported development and operational tasks, occasionally addressing urgent needs outside regular hours.

Education

P.G. - Data science

International Institute of Information Technology
Bangalore
09.2021

B.E. - computer science and engineering

Jawaharlal Nehru Technological University
Kakinada
05.2016

Skills

  • Hive and HiveQL
  • DB2 and SQL
  • Data analysis and warehousing
  • Apache Spark and MapReduce
  • Scala programming
  • Version control with GitHub
  • Unix and Linux systems
  • Agile methodologies
  • HDFS management
  • SQL tuning
  • Microsoft Excel proficiency
  • GCP BigQuery
  • Business requirement analysis
  • Python programming

Websites

Certification

  • Cloudera certified Spark and Hadoop Developer (CCA - 175)
  • GCP Professional Data Engineering badges: link
  • Hadoop & Spark Developer Certification Program | E&ICT IIT Roorke

Languages

Telugu
First Language
English
Advanced (C1)
C1
Hindi
Upper Intermediate (B2)
B2
Tamil
Upper Intermediate (B2)
B2

Timeline

Senior Data Engineer

Paypal
09.2023 - Current

Senior Consultant

Deloitte India LLP
06.2020 - 09.2023

Data Engineer

Foray Software
04.2019 - 05.2020

Software Engineer

Experis IT
02.2017 - 02.2019

P.G. - Data science

International Institute of Information Technology

B.E. - computer science and engineering

Jawaharlal Nehru Technological University
Jagadeesh Pamarthi