Summary
Overview
Work History
Education
Skills
Websites
Accomplishments
Timeline
Generic

RAHUL GOYAL

Senior Data Engineer | Cloud Data Pipelines | Azure ADF | SQL Performance | GDPR Compliance
Noida

Summary

Experienced Data Engineer with 10+ years of expertise in designing and enhancing scalable data pipelines using Azure Data Factory (ADF) and cloud platforms like Azure and AWS. Proven track record in automating data ingestion, transformation, and orchestration for diverse industries. Adept at ensuring data security, governance, and compliance with GDPR/HIPAA regulations. Skilled in optimizing SQL performance, reducing pipeline downtime, and promoting collaboration for actionable insights and business outcomes.

Overview

10
10
years of professional experience
4
4
years of post-secondary education
2
2
Languages

Work History

Senior Associate

Cognizant
09.2021 - Current
  • Designed and optimized end-to-end data pipelines using Azure Data Factory (ADF), automating ingestion and transformation processes for enterprise applications and reducing data latency by 30%.
  • Managed and administered PostgreSQL databases with replication and tuning strategies, increasing system availability and supporting SLA compliance for business-critical applications.
  • Implemented role-based access controls and data masking for sensitive data across 10+ systems, ensuring full compliance with GDPR and HIPAA policies.
  • Developed ETL pipelines using Python and ADF to process multi-source structured/semi-structured data (>10GB/day), improving BI insights and reducing data prep time.
  • Enhanced data governance using Azure Purview and schema validation scripts, reducing anomalies by 25% and increasing audit traceability.
  • Optimized SQL queries and implemented indexing strategies that reduced average execution time by 40% across critical reporting workloads.
  • Partnered with data scientists and architects to deliver scalable data platforms, accelerating analytics delivery by 20% for key business units.
  • Led root-cause analysis and performance tuning for ADF pipelines using Azure Monitor and Log Analytics, reducing pipeline failure rate by 35%.
  • Automated database maintenance tasks via Python/PowerShell scripts, cutting manual intervention by 60% and reducing operational overhead.
  • Integrated and secured enterprise data infrastructure using Azure SQL, Storage, Key Vault, and Monitor, resulting in a 20% improvement in system reliability.
  • Conducted weekly training sessions for junior engineers, improving team ramp-up time by 25% and fostering a strong DevOps learning culture.
  • Collaborated with cross-functional teams to drive successful completion of complex projects within deadlines.

Senior Engineer

Successive Technologies
08.2020 - 09.2021


  • Led a team of 15 engineers to build a data ingestion and transformation platform, enabling real-time Excel/CSV processing via an interactive UI, reducing manual reporting time by 50%.
  • Architected a scalable backend in Node.js with MongoDB and SQLDB integration, handling 1M+ data records/month with 99.9% system uptime.
  • Built a responsive React.js frontend that improved user engagement by 40%, enabling seamless data exploration and filtering for business users.
  • Developed automated pipelines for ingesting and transforming 10,000+ files/month, improving data accuracy and reducing processing time by 60%.
  • Collaborated with clients and architects to define project roadmaps, delivering 3 major releases on time with 95% stakeholder satisfaction.
  • Managed engineering workflows including sprint planning and code reviews, increasing team throughput by 30% and reducing bugs in UAT by 40%.
  • Enforced security protocols and data governance using JWT auth and encrypted storage, ensuring compliance with ISO 27001 standards.
  • Mentored 6 junior developers, accelerating onboarding by 40% and promoting 2 to senior roles within 12 months.

Software Design Engineer

Alphaa AI
03.2019 - 08.2020
  • Designed and deployed ETL pipelines to ingest and transform structured/semi-structured data into cloud databases (Redshift, Snowflake, Cosmos DB), enabling faster analytics and 99% data availability.
  • Built scalable data workflows using Airflow and Druid on Amazon S3, improving data processing speed by 45% and enabling high-availability batch jobs.
  • Developed self-service dashboards in Power BI/Tableau/QlikView, used by 5+ departments to drive real-time operational decisions.
  • Developed NLP-based query interface using Python and spaCy, allowing non-technical users to generate charts/tables via natural language, increasing report access by 60%.
  • Integrated Azure Data Lake and Amazon S3 for secure data storage, reducing retrieval latency by 25% across analytics workloads.
  • Used Python and Node.js to build automation scripts and APIs, reducing manual data handling by 70% across the ETL lifecycle.
  • Improved database query speeds by 35% through indexing and tuning strategies across SQL, PostgreSQL, and MongoDB environments.
  • Partnered with analysts and engineers to deliver scalable cloud data solutions, cutting reporting delays from hours to minutes.

Software Design Engineer

RealBox Data Analytics Private Limited
05.2015 - 03.2019
  • Led data-driven retention and acquisition initiatives, increasing customer retention by 50% through targeted segmentation and loyalty campaigns.
  • Optimized query performance in PostgreSQL, reducing average response time by 30% for key client reporting dashboards.
  • Built microservices-based APIs enabling real-time data ingestion, reducing end-to-end data sync latency by 40%.
  • Designed a full-stack POS system used by 50+ businesses across retail and food industries, integrating billing, inventory, and attendance modules.
  • Built a white-labeled online food ordering platform for 10+ restaurants, increasing customer orders by 60% within 3 months of deployment.
  • Coordinated closely with QA teams throughout the development lifecycle, ensuring timely identification and resolution of defects or bugs that could impact product quality or delivery timelines.

Education

Bachelor of Technology (B.Tech.) -

G.L Bajaj Institute Of Technology And Management
Greater Noida, India
08.2011 - 05.2015

Skills

ADF

Apache Airflow

amazon s3

Microservices

Azure Data

azure data lake

Azure SQL

Business Intelligence

clickhouse

Data Engineering

Data Visualization

Apache Druid

ETL

Google Analytics

Javascript

undefined

Websites

Accomplishments

  • Enhanced SQL query performance by 35% through advanced optimization and strategic indexing, accelerating critical report generation.
  • Reduced data pipeline downtime by 40% by implementing proactive monitoring and automated troubleshooting protocols.
  • Boosted engineering team efficiency by 50% through agile sprint planning, process optimization, and workflow automation.
  • Attained 100% GDPR compliance by implementing robust data security controls, including access management and data masking strategies.
  • Enabled self-service BI by building dashboards and automating reports, reducing ad-hoc analyst requests by 60%.
  • Delivered a high-impact cloud data migration project ahead of schedule, moving 5TB+ of data to Azure with zero downtime.
  • Mentored 6 junior engineers, improving team productivity and accelerating onboarding time by 40%.

Timeline

Senior Associate

Cognizant
09.2021 - Current

Senior Engineer

Successive Technologies
08.2020 - 09.2021

Software Design Engineer

Alphaa AI
03.2019 - 08.2020

Software Design Engineer

RealBox Data Analytics Private Limited
05.2015 - 03.2019

Bachelor of Technology (B.Tech.) -

G.L Bajaj Institute Of Technology And Management
08.2011 - 05.2015

Similar Profiles

CREATE PROFILE
RAHUL GOYALSenior Data Engineer | Cloud Data Pipelines | Azure ADF | SQL Performance | GDPR Compliance