Data Engineer at JPMorgan Chase & Co delivering high-impact data solutions for regulatory and executive reporting. Skilled in building observability-driven systems and optimizing data workflows for reliability and efficiency. Passionate about applying AI and analytics to solve complex business problems.
Hi, I’m
ANURAAG B
Software Engineer 1
Summary
Overview
2
years of professional experience
Work History
J P Morgan Chase & Co
Software Engineer 1
Current
Job overview
- Developed and optimized large-scale ETL pipelines, enhancing reporting capabilities for Consumer & Business Banking and achieving a 40% reduction in data processing time.
- Designed and implemented metadata-driven workflow orchestration, improving data pipeline reliability and automation across systems.
- Built custom Airflow DAGs enabling dynamic execution of upstream/downstream dependencies, bidirectional job runs, and parallelized workflows, reducing pipeline execution time by 70% and eliminating manual job planning through change-based execution.
- Engineered an intelligent observability platform delivering comprehensive table- and attribute-level data lineage, job run metrics, and cross-team impact analysis, increasing data transparency by 80%.
- Built a Dev Suite Portal consolidating automation utilities for data modeling, dataset registration, DDL generation, data migration, and compliance checks, improving operational efficiency and reducing manual effort by 60%.
- Accelerated CI/CD processes by optimizing data pipelines and automating release workflows, achieving a 25% faster delivery cycle.
- Developed cost management dashboards, enabling leadership to optimize resource utilization and reduce cloud spending by 15%.
- Designed and delivered Loan Originations reporting tables, enabling daily reporting of business banking loans into Essbase cubes for downstream financial analytics.
- Earned multiple organization-level hackathon awards for innovative automation and AI-driven solutions.
JPMorgan Chase & Co.
SEP Intern
01.2024 - 06.2024
Job overview
- Led migration of ETL workflows from on-premises Hadoop (Hive/Impala) to cloud-native Databricks (Spark), advancing enterprise data platform modernization.
- Refactored legacy Hive queries into high-performance Spark SQL and PySpark pipelines, improving query efficiency by up to 30%.
- Conducted rigorous performance benchmarking and validation to ensure 100% data accuracy and parity between on-prem and cloud environments.
- Migrated batch workflows and mapped complex dependencies to cloud orchestration frameworks, improving pipeline reliability and maintainability.
- Developed Databricks monitoring dashboards for daily and monthly pipeline tracking, significantly reducing manual querying efforts and improving operational visibility.
- Enhanced platform scalability and reduced infrastructure constraints, enabling seamless cloud-native processing and accelerating data delivery timelines by 25%.
Education
Chaitanya Bharathi Institute of Technology
B.Tech/B.E. - Bachelor of Technology / Engineering
04-2024
University Overview
Information Technology undergraduate at Chaitanya Bharathi Institute of Technology with hands-on experience in backend development, data engineering, and machine learning. Skilled in Python, Java, SQL, and big-data tools such as Spark and Databricks. Interested in designing scalable data systems and applying AI/ML techniques to extract meaningful insights from large datasets.
- 9.36 CGPA
Skills
- SQL
- Python
- ETL Development
- Data Warehousing
- Spark Framework
- Databricks
Professional Work history
Professional Work history
01-2024Projects
Projects
Metaweave – Intelligent Observability Platform
- Built a full-stack intelligent observability platform for data lineage, job monitoring, and runbook-driven operations, significantly improving transparency across analytics workflows.
- Implemented comprehensive lineage capabilities including table, attribute, and temporal lineage, along with cross-team impact analysis and RCA, strengthening enterprise data governance.
- Developed LLM-powered features including code understanding (CTE interpretation, business rule extraction) and a conversational assistant enabling users to query job logic and dependencies, improving developer productivity and debugging efficiency.
- Integrated job performance analytics and monitoring dashboards, reducing job failures by 15% and enabling proactive, data-driven optimization.
- Built advanced code analysis capabilities (join graphs, column-level transformations, dependency tracking) to enable deep introspection of ETL pipelines.
- Automated ingestion and processing of job metadata across systems, improving operational efficiency by 30%.
- Widely adopted across teams, with strong positive feedback from business stakeholders for improving data transparency, impact analysis, and incident resolution speed.
- Developed a centralized automation and governance platform consolidating data modeling validation, dataset registration, DDL generation, data migration, and compliance reconciliation workflows.
- Automated schema consistency checks across environments and metadata systems, ensuring alignment between logical and physical data models and improving release confidence.
- Built utilities for RDS metadata management and automated job onboarding, eliminating manual configuration and reducing onboarding effort by 60%.
- Implemented cross-environment compliance validation to detect schema drift, missing dependencies, and inconsistencies prior to deployment, reducing production risks.
- Streamlined data movement and environment synchronization processes, improving data availability and operational efficiency.
- Significantly enhanced developer productivity and governance adherence by standardizing critical data platform workflows.
Certification
- Amazon Web Services Cloud Practitioner
- Databricks Associate
- IBM Data Engineering
- IBM Data Science
Accomplishments
Accomplishments
Key Accomplishments
- Achieved a 40% reduction in ETL processing time by optimizing large-scale data pipelines for regulatory and executive reporting.
- Reduced pipeline execution time by 70% by designing dynamic, dependency-aware workflow orchestration, eliminating manual job planning.
- Built an intelligent observability platform (Metaweave) adopted across teams, improving data transparency and reducing job failures by 15%.
- Developed a Dev Suite Portal automating data platform workflows, reducing onboarding and operational effort by 60%.
- Improved cloud cost efficiency by 15% through data-driven monitoring dashboards and resource optimization.
- Delivered Loan Originations reporting pipelines enabling daily business-critical reporting for financial analytics systems.
- Accelerated release cycles by 25% through CI/CD and deployment automation improvements.
- Earned multiple organization-level hackathon awards for building innovative automation and AI-driven solutions.
- Successfully contributed to enterprise-scale cloud modernization, ensuring 100% data accuracy and seamless migration from on-prem to cloud.
Availability
See my work availability
Not Available
Available
monday
tuesday
wednesday
thursday
friday
saturday
sunday
morning
afternoon
evening
swipe to browse
Affiliations
Affiliations
- IIT Ropar AI for All
Timeline
SEP Intern
JPMorgan Chase & Co.
01.2024 - 06.2024
Software Engineer 1
J P Morgan Chase & Co
Current
Chaitanya Bharathi Institute of Technology
B.Tech/B.E. - Bachelor of Technology / Engineering