Summary
Overview
Work History
Education
Skills
Certifications
Projects
Work Availability
Quote
Timeline
OperationsManager
Anup S. Pillai

Anup S. Pillai

Supervising Associate (Lead Azure Data Engineer)
Trivandrum

Summary

Lead Data Engineer with 10+ years of experience in developing data pipelines for large scale data movement and transformation.

Experienced in developing data storage solutions across Azure Cloud

Experienced in managing team of data engineer working across product development pipeline

Highly analytic team player, with the aptitude for prioritization of needs/risks. Constantly striving to streamlining process and experimenting with optimizing and benchmarking solutions

Overview

10
10
years of professional experience

Work History

Supervising Associate

EY GDS
Trivandrum
12.2019 - Current
  • Developed pipelines to move and transform raw data
  • Collaborated on ETL (Extract, Transform, Load) tasks, maintaining data integrity and verifying pipeline stability.
  • Designed and implemented effective database solutions(Azure blob storage, ADLS GEN2) to store and retrieve data.
  • Deployed data factory for creating data pipeline to orchestrate the data into SQL database
  • Develop the end to end project plans with the execution team including Global Process Lead,
  • Facilitate creation of the tasks for the Sprint with dependencies identified
  • Looking constantly for development as well as continuous improvement for the entire team
  • Mentoring and training colleagues on data storage, ADF and Databricks
  • Identifying process improvement areas such as scope for implementing OCR, Chat Bot
  • Developed Power Apps for internal and external clients
  • Participate in requirements gathering and design from clients

Test Specialist

Allianz Techniology
Trivandrum
02.2019 - 12.2019
  • Maintain data quality and establish data governance
  • Developed ADF pipelines to move data to test environment
  • Structured the data into MYSQL
  • Designed compliance frameworks for multi-site data warehousing efforts to verify conformity with restaurant supply chain and data security guidelines.
  • Performed large-scale data conversions for integration into MYSQL

Senior Systems Engineer

Infosys Limited
Trivandrum
08.2013 - 02.2019
  • Developed ETL process for data transformation and loading
  • Performed root cause analysis to provide resolutions for production issues.
  • Worked on data warehouse solutions
  • Liaison with the product managers, delivery leads, business analysts
  • Identifying data quality issues
  • Defining and developing database schemas
  • Identified, reviewed and evaluated data management metrics to recommend ways to strengthen data across enterprise.
  • Communicated new or updated data requirements to global team.
  • Developed Power BI reports

Education

B.Tech - Electrical & Electronics Engineering

Biju Patnaik University of Technology
09.2008 - 09.2012

Skills

Azure Data Factory

undefined

Certifications

Microsoft Power Platform -PL 900

Aure AI Fundamentals- AI 900

Microsoft Technology Associate- Software Development Fundamentals

Projects

Description: Worked with various service lines and external clients towards developing and implementing ETL and ELT pipelines by utilizing ADF. Developed pipelines for moving on-prem data to Azure cloud storage such as BLOB and ADLS Gen2. Depending on data volume utilized Synapse Analytics for developing data transformation pipelines. Collaborated on ETL tasks, maintain data integrity and pipeline stability. Worked on establishing data governance and data modelling. Deployed data factory for creating data pipeline to orchestrate data into identified Azure storage.


Responsibilities: Analyze the business requirements, develop and maintain data pipelines responsible for data movement and transformation.  
Collaborate with global business teams to guide requirements discussions around identifying key problem and improvement areas. Manage team of 5 data engineers. Creating and updating ADO tasks.

Work Availability

monday
tuesday
wednesday
thursday
friday
saturday
sunday
morning
afternoon
evening
swipe to browse

Quote

The problem is not to find the answer; it’s to face the answer.
Terence McKenna

Timeline

Supervising Associate

EY GDS
12.2019 - Current

Test Specialist

Allianz Techniology
02.2019 - 12.2019

Senior Systems Engineer

Infosys Limited
08.2013 - 02.2019

B.Tech - Electrical & Electronics Engineering

Biju Patnaik University of Technology
09.2008 - 09.2012

Similar Profiles

CREATE PROFILE
Anup S. PillaiSupervising Associate (Lead Azure Data Engineer)