Mayuresh Pandey

As an RSA at Databricks, I am one of trusted advisors of APAC region Databricks customers, some of my recent engagements are:

• Data governance implementation using unity catalog for largest Insurance company in GCR: Migrated customer's existing metadata, roles, permissions model and data governance from existing HMS based workspaces to Unity Catalog based workspaces with all the data assets and Jobs. Defined processes for data access for ETL and serving/consumption layers to multiple stakeholders. Defined and architected migration process, cutover and go live approaches without business interruption.
• Data platform migration for OTT platform (40B USD annual revenue customer): Migrated OSS based Big data stacks (Spark, Hive, Hadoop, Presto, Airflow) based data services to Databricks Platform with batch and streaming workflows and DB SQL. Migration architecture design and proposal and sign off from customer, technical reviews and running tech spikes for complex deliverable items as per Databricks best practices, defining technical roadmap and migration phases with Databricks preferred partner, ensuring migrated job code quality and performance on DBR, Driving sprint velocity, code maturity and knowledge sharing for successful delivery.
• Streaming Data platform modernization of a big glass packaging MNC ( customer with 6B USD annual revenue customer) for their productions floor IOT devices:

Designed and architected end to end streaming and batch jobs data flow. Enabled medallion architecture the data platform with unity catalog and data governance structures and permission models best practices, near realtime reporting, dashboards and on demand data quality matrices as part of delivery and hands on work.

• Streaming Data platform development for an ASEAN region popular Airline:

Designed and architected end to end streaming and batch jobs data flow. Enabled medallion architecture the data platform with unity catalog and data governance structures and permission models best practices, near realtime reporting, dashboards, user onboarding with custom notebooks and requirements as part of delivery and hands on work.

• Leading and contributing technology efforts for Data Engineering in the multi-cloud Tokopedia ecosystem
• Owning and running cost and performance optimizations practices for the data engineering pipelines, Data lake, DWH, Data Catalog, Data Quality framework and downstream systems for the specified budgets
• Driving internal and external PoC, Pilots and evaluating products for functional and nonfunctional requirements
• Team mentoring and helping on day to day problems solving in a fast paced multi stakeholder environment
• Planning and executing for an advanced data engineering roadmap with emphasis on local efforts and stakeholders matrix.
• Designed internal process improvements to automate repetitive tasks, shortening data delivery times.

• Leading proof of value and platform onboarding efforts for Qubole's prospects and customers who are on public clouds (AWS,GCP and Azure).Helping on Solutions engineering and proposing overall big data architecture
• Performance and cost optimizations on Spark, Hive and Presto engines for ETL, Analytics and Data Science workloads
• Reviewing and advising on best cost effective and secure data lake architecture
• Post sales customer engagement and driving new big data workloads on platform
• Tech Evangelism and professional services (ETL, Analytics and Data science use cases) for new age big data on cloud architecture.

• SOC Data Ingestion and DWH: This is developed as a data platform for telecom operator's probe data (binary event data) pulled from various third party endpoints through standard api's and ingested into HDFS for data processing
• Designed and owned the overall data pipeline from ingesting to creating DWH and enabling data access for various use cases (querying, dashboarding, machine learning) run by dierent stakeholders
• Analysis, Designing and coding: Team mentoring, Designing architecture for data and application platforms
• POC and pilot development and coding, capacity planning, integration and migration planning, Impact analysis of bug fixes, features & enhancements
• Defining future development strategy.

• Lead the development eorts for next gen CRM and sales application, integrating CRM with other state of art platform services like product search services, oer, recommendations, inventory and checkout services including Lead and case generation integration
• Dynamic Show Scheduling and Sales Dashboard: Architected and authored the technical framework for TV schedule planning and sales/performance dashboard which broadcasts on screen live sales data feed
• Predicted show sales vs Actual show, and custom alerts.

• Authored & built SDK platform for terrain analysis services, on SOA based portal application
• Oracle spatial is being used to define the geometry database
• Web services are being used to bring the dierent algorithmic computation analysis on the portal
• JSR-286 and spring integration has been used for exposing the Analysis Layer layers on the portal
• JAX-WS and REST based is being used for web services Analysis & Designing: Designing architecture for databases as well as the application
• Impact Analysis of bug fixes, features & enhancements
• Defining future development strategy.

• Worked on various Java / J2ee and Open source tools and OSS ERP projects in multiple capacities and roles (Programmer, Technical Lead and Project lead)
• Specsavers UK: RetailFinance, ERP and Store POS application development
• Rental Results UK PLC: Product Development
• Delhi Trac Police & Crime Branch: SMS Based online challan delivery system
• TCI transport data transfer: Cargo and consignment movement tracking.