Soumyajit Das
Bengaluru
Summary
Dynamic data engineering professional with approximately 16 years of experience in leveraging modern technologies to drive organizational efficiency and effectiveness. Proven track record in developing and implementing strategic plans that align with business objectives. Expertise in translating complex business requirements into innovative technical solutions. Adept at building and nurturing strong relationships with stakeholders to foster collaboration and achieve shared goals.
Overview
15
15
years of professional experience
Work History
Data Engineer
Amazon
05.2022 - Current
- Led GenAI initiatives leveraging AWS Bedrock, enabling managers, project managers, and business analysts to ask business questions and receive data-driven insights. Integrated multi-agent collaboration for Root Cause Analysis (RCA), anomaly detection, and automated query execution using Athena, DynamoDB, and S3, improving decision-making efficiency.
- Data Pipeline Development: Led the design and implementation of data pipelines and data lakes, establishing robust infrastructure to support model training for various models in Amazon Music
- Efficiency Optimization: Streamlined model training processes, reducing runtime from 7-8 hours to approximately 30 minutes, significantly boosting operational efficiency
- Product development: Designed and built the data engineering pipeline that served as the backbone for the successful release of 'Summer Soundtrack' and 'Year in Review' features in Amazon Music, demonstrating expertise in processing approximately 30TB of data leveraging EMR
- DMA Compliance Leadership: Spearheaded efforts to align data-sharing processes with organizational standards for DMA compliance, achieving successful adoption within established deadlines
- Cross-Team Documentation: Created comprehensive documentation, including an FAQ wiki, to enhance understanding of new processes and effectively address queries across multiple teams
Lead Data Engineer
HSBC
11.2018 - 04.2022
- Collaborated with the leadership team to plan, develop, and implement technical capabilities in alignment with the company's vision, mission, and strategic goals
- Acted as a Subject Matter Expert (SME), successfully designing, developing, and deploying global data pipelines using Apache Spark as the primary data processing engine and Python for scripting
- Additionally, facilitated the establishment of impactful data visualizations for the bank
- Spearheaded the migration of traditional SAS-based ETL processes to PySpark, resulting in an annual cost reduction and a performance boost of approximately 45%
- Demonstrated expertise in handling large-scale datasets, with hands-on experience addressing performance optimization and scalability challenges
- Led and managed key data engineering initiatives for the UK Collections at HSBC, delivering high-impact results and operational efficiencies
- Recognized by the Global CCS Head of HSBC for outstanding contributions
- Earned the 'Top Performer' rating for the year 2021
Team Lead
Accenture
07.2015 - 10.2018
- Successfully led and managed a team of over 5 data engineers, driving high-impact project deliveries
- Developed a Marketing Automation Framework that enabled clients to execute outbound and inbound campaigns, manage multi-channel communications, and maintain omni-channel customer responses efficiently
- Prepared comprehensive functional and technical documentation for data warehousing solutions, ensuring clarity and effective knowledge transfer
- Fostered a collaborative and professional team environment, resolving conflicts swiftly and ensuring smooth project execution
Consultant
Capgemini
09.2013 - 06.2015
- Leveraged expertise in business tools to design, develop, and deploy Business Intelligence (BI) capabilities, improving data-driven decision-making
- Developed and managed SAS applications to extract data from RDBMS source systems and load it into SAS/SPDS datasets, driving BI functionalities through SAS Web Report Studio (WRS) and SAS Information Delivery Portal
- Consistently provided exceptional service to clients, strengthening relationships and fostering future business opportunities
- Maintained clear and effective communication with stakeholders, ensuring transparency on key project milestones and deliverables
Data Management Associate
Tech Mahindra
04.2010 - 08.2013
- Built and deployed comprehensive plans, reports, and validation procedures to meet diverse data management needs
- Developed ETL jobs using SAS DI Studio and created SAS reports via SAS Web Report Studio, publishing them on the SAS Information Delivery Portal to enhance data accessibility
- Increased customer satisfaction by promptly identifying and resolving data-related issues, contributing to smoother operations
- Awarded 'PAT ON BACK' in recognition of exceptional performance
Education
Bachelor of Technology -
West Bengal University of Technology
08.2009
Executive MBA - undefined
Indian Institute of Management - Sambalpur
05.2026
Skills
- Big Data
- SQL
- Python
- Hadoop
- Apache Spark
- AWS Bedrock
- AWS - EMR
- AWS-MWAA
- AWS-S3
- AWS-Lambda
- AWS-Redshift
- AWS-Glue
- AWS-SNS
- SQS
Timeline
Data Engineer
Amazon
05.2022 - Current
Lead Data Engineer
HSBC
11.2018 - 04.2022
Team Lead
Accenture
07.2015 - 10.2018
Consultant
Capgemini
09.2013 - 06.2015
Data Management Associate
Tech Mahindra
04.2010 - 08.2013
Executive MBA - undefined
Indian Institute of Management - Sambalpur
Bachelor of Technology -
West Bengal University of Technology