Seasoned lead in data engineering with strong background in managing complex data infrastructure projects. Known for innovative approaches to solve challenging problems and ability to oversee team operations effectively. Possess strength in Spark SQL, Python, Java, and AWS skills coupled with keen business acumen. Have made significant impacts by driving efficiency and performance improvements in previous roles.
Energetically led Data Engineering at JP Morgan Chase & Co, achieving an 80% data migration to AWS, optimizing costs by decommissioning legacy systems. Expert in big data processing and guiding teams, I excel in developing innovative solutions like event-driven pipelines and enhancing data protection.
Excel in problem-solving, teamwork, and communication, ensuring seamless project execution and collaboration within diverse teams. Passionate about continuous learning and professional development.
● Designed and developed an Event Driven pipeline with capabilities like self-healing, Data Lineage, etc.
● Achieved 80% data set migration from proprietary data warehouse to
AWS data lake and subsequently to consumption platforms such as snowflake which is a cloud native data warehouse using event driven architecture, leveraging the capabilities of AWS serverless architecture involving step functions, lambdas and Event Bridge.
● Helped firm in achieving the goal of migrating on premise data to cloud by Q4 2023 which helped in strategic decommission of legacy system and optimize cost.
● Responsible for guiding teams in identifying the MVP features to be able to focus on building the core system for the migration.
● To ease user-onboarding developed a chatbot using RASA framework.
● Developed ETL pipelines which process data on-prem.