SUDIPTA KAR

• Designed & automated the ML model deployment cycle using customized Airflow operators, AWS, MLflow, & Jenkins - eliminating 2 hrs of daily manual work across teams (~25% effort reduction).
• Engineered high-performance complex ETLs (Python, Scala, SQL) on AWS EMR using Spark, processing 10M+ daily events from the social commerce ecosystem to power data-driven insights.
• Developed an API-based microservice integrating ML models to detect spam comments, to protect new users from potential scams.

• Developed a customized AWS Fleet operator in Airflow, optimizing EMR job execution and reducing overall runtime and compute cost by 30%.
• Architected an hourly ETL pipeline that ingests data from Snowflake, computes user insight metrics over rolling 7, 14, 30, & 365 days window, and publishes metric deltas to consumers via RabbitMQ
• Led the enhancement and upgrade of the Grafana application used for data visualization & monitoring across organization
• Developed an API-based microservice to detect duplicated listings, improving data integrity and reducing marketplace abuse.

• Migrated on-prem datasets to Azure Data Lake, optimizing ingestion and transformation workflows.
• Reduced Spark SQL query latency by 40% using efficient data modeling and partitioning in Databricks.
• Designed Power BI dashboards with 100+ KPIs using DAX and integrated Analysis Services models.
• Monitored and improved Azure Data Factory ETL workflows and data integrity in Azure SQL.

Tech: Azure, Databricks, Power BI, DAX, SSMS, SSAS, SQL, Python

• Integrated Nuxeo with AEM for content management and AWS S3 for large asset uploads.
• Customized Nuxeo UI and REST APIs; built reusable components in Polymer.js and JQuery.
  Tech: Java, AEM, Nuxeo, AWS S3, MongoDB, Polymer.js

Worked on “Information security & GRC Automation using archer Tool”.Learned basics of Information security & ISO 27001,Data privacy,regulatory compliances,PCI-DSS,SOX