Ganesh Gurusiddaiah

Technology: Azure ADF, Azure Databricks, Azure App services, Storage, Azure Dev Ops, Azure Container Registries, Pyspark, Python, Flask.

• Description: Building ADF pipeline for Sales SRM analytical use case which involve data preparation notebook, model training, model scoring and drift analysis notebooks

Responsibilities:

• Design and create data preparation pyspark notebooks
• Read data from azure storage accounts
• Design data model on delta, SQL, parquet file system
• Built end to end pipeline using Azure data factory
• Scheduled and monitored ADF pipelines.

Technology: AWS Glue, Redshift, Abinitio GDE, AbInitio EME, S3, cloudera, Hive

• Designed and developed analytical data structures.
• Analyzed complex data and identified anomalies, trends, and risks to provide useful insights to improve internal controls.
• DE pipeline creation and orchestration using cloudera stack

• Technology: Amazon Redshift, AWS Glue, AWS S3,Parquet Hive tables
• Description: Real estate origination data is extracted and staged in Hive raw tables and refined using
• Source include Mongo collection and PostgreSQL tables
• Refined data is transformed into dimension and fact table target using AWS Glue and loaded into Redshift tables
• Lead a group of 10 developers to deploy the ETL solution in prod
• Data Lake Analytics Enablement and usage