RAHUL DAS
Kolkata
Summary
Dynamic Senior Data Engineer with nearly 5 years of experience in developing robust data pipelines that enhance data-driven decision-making. Proficient in Azure Data Factory, Databricks, and Spark, ensuring automated workflows and high-quality data standards. Demonstrated expertise in designing scalable architectures and integrating modern technologies to optimize efficiency. Strong track record of aligning technological solutions with business objectives while fostering cross-functional collaboration.
Overview
5
5
years of professional experience
1
1
Certification
Work History
Senior Data Engineer
Accenture
Kolkata
03.2022 - Current
- Designed and orchestrated scalable ELT pipelines using Azure Data Factory, PySpark, SQL, and Spark SQL on Databricks, within a metadata-driven framework, leveraging Lakehouse and Medallion architecture (Bronze, Silver, and Gold layers). Automated workflows ensure robust data lineage, high data quality, and reusability, achieving a 30% runtime reduction by migrating from Synapse 2-tier to Lakehouse 3-tier architecture, and implementing business rules to standardize and ensure compliant data processing.
- Utilized PySpark and SQL for efficient data processing and transformation, enabling seamless integration and optimization across the data pipeline.
- Utilized Azure Data Lake Services in designing and implementing end-to-end ETL pipelines and Lakehouse architectures, incorporating Delta Lake for scalable, high-performance data storage and processing.
- Implemented data masking and leveraged Unity Catalog for robust data security, governance, and access control across the data platform.
- Performed ETL processes across multiple data sources, including Oracle EPM, ARCS, EBS tools, and real estate data, to bolster regulatory compliance, and enhance business analytics capabilities. Integrated business rules and compliance requirements into data workflows, ensuring seamless and accurate data transformation.
- Collaborated with cross-functional teams, including data scientists, analysts, and other stakeholders, to communicate the benefits of data solutions, emphasizing operational efficiency, KPI gatherings and driving alignment across technical and business objectives.
- Established a Global Resilient Dimension Model to enhance data model standardization.
- Led data engineering initiatives across multiple high-impact use cases, ensuring seamless alignment with organizational goals. Trained junior team members, fostering their growth, and ensuring successful delivery of data solutions that support business objectives and operational growth.
- Established CI/CD pipelines with Azure DevOps, dramatically cut time-to-production for key data updates.
Programmer Analyst
Cognizant
Bengaluru
09.2020 - 03.2022
- Worked as an Azure Data Engineer in a fast-paced, agile environment, collaborating closely with Data Scientists, API teams, and UI teams to deliver data solutions tailored to their requirements.
- Automated data pipelines to eliminate manual interventions and improve efficiency.
- Developed end-to-end ETL pipelines for migrating data from third-party APIs to Azure Database, with data cleaning and transformation performed using Azure Data Factory (ADF), Azure Databricks (ADB), SQL Server, and Azure Data Explorer (ADX).
- Managed a significant volume of live weather data (approximately 20 GB) processed on a time-series database (Azure Data Explorer), enabling API teams and UI teams to consume and analyze real-time data effectively.
Education
Bachelor of Technology - Information Technology
Future Insititute of Engineering And Management
Kolkata07-2020
Skills
- Data Pipeline Development
- Azure Databricks
- Azure Data Factory
- SQL
- T-SQL
- Spark
- Pyspark
- Data Warehousing
- Lakehouse Architecture
- Data warehousing
- ELT process orchestration
- Team leadership
- Azure Data Lake Services
- Delta Lake
Accomplishments
Champion of the Quarter FY24 Q3 - Above and Beyond - Accenture
- Received Quarterly Champion Award for Apr-Jun'24
Champion of the Quarter FY24 Q1 - Above and Beyond - Accenture
- Received Quarterly Champion Award for Sep-Dec'23
Certification
Microsoft Certified: Azure Data Engineer Associate DP-203
- Microsoft - E997E5612F40096
Databricks Certified Data Engineer Associate
- Databricks - 106860511
Microsoft Certified: Azure Data Fundamentals
- Microsoft - 146750B95046DB19
Languages
Bengali
First Language
English
Proficient (C2)
C2
Hindi
Proficient (C2)
C2
Timeline
Senior Data Engineer
Accenture
03.2022 - Current
Programmer Analyst
Cognizant
09.2020 - 03.2022
Bachelor of Technology - Information Technology
Future Insititute of Engineering And Management
Similar Profiles
Uttara MerupulaUttara Merupula
Team Leader at AccentureTeam Leader at Accenture
Anchitha K RAnchitha K R
Senior Software Engineer at AccentureSenior Software Engineer at Accenture
PHANI TEJA NALLAMOTHUPHANI TEJA NALLAMOTHU
Senior Manager, Technology Architecture at AccentureSenior Manager, Technology Architecture at Accenture