Summary
Overview
Work History
Education
Skills
Certification
Languages
Hobbies and Interests
Career Experience
Interests
Timeline
Generic
Sheik Irfan

Sheik Irfan

Bengaluru

Summary

Being an AWS Data Engineer, I bring nearly 8 years of experience in designing, building, and maintaining data pipelines. I have a proven track record in leveraging AWS cloud services to develop scalable and efficient data pipelines. My goal is to contribute to a progressive company by providing expert data engineering solutions and robust analytical insights.

Overview

8
8
years of professional experience
1
1
Certification

Work History

Senior Data Engineer

HTC Global Services
05.2020 - Current
  • Strong experience in designing and implementing data pipelines for large-scale, complex data systems using AWS Stack(S3, Glue, Redshift, Athena, SNS, SQS, Step Functions, Lambda, DynamoDB, and Terraform) which enhanced business decision-making capability to a great level
  • Designed and implemented robust ETL pipelines using AWS Glue to integrate with S3, Redshift, and Glue catalog with Glue optimization techniques namely push_down_predicate, hash field, hash partitions, bulk size and columnar data format, etc., for fast and scalable processing of data
  • Orchestrate Glue resources using Glue workflow, Glue triggers, and Step Functions
  • Created Athena Iceberg table for S3 data modifications and wrote time travel queries for data analytics and developed Athena table with partitions and bucketing the data for improved query performance
  • Worked on Python modules namely Pyspark, Pandas, DataWrangler, boto3, PyAthena, Snowparks, redshift_connector, and SQL for data analytics and ETL Transformations
  • Generated S3 bucket events notifications with SNS Topic, SQS Queue, and Lambda function to send Email notifications to the customers and automate the Glue Job
  • Deployed the resources in AWS for our team and other teams using Visual Studio code, Gitlab, Terraform and Scalr workspace
  • Created cross-account permissions through IAM Roles, Inline policies, and Trust Relationships and also imposed firewall rules to access Redshift Clusters
  • Worked on ELT Transformations in Snowflakes using DBT Core
  • Good knowledge in Azure Data Factory, Azure SQL, Data Bricks and Cosmos DB.
  • Good exposure with Big Query, Big Table, Cloud SQL and Cloud Storage.
  • Acquainted with Kubernetes and Airflow

Software Developer

3i Infotech Pvt Ltd
01.2019 - 04.2020
  • Worked on writing Oracle SQL Queries and provided them to the Oracle Report developers
  • Developed Glue Job to integrate with Redshift and S3 for ETL Pipeline
  • Automated Glue resources using Glue workflow and Glue triggers
  • Worked on creating Athena external tables for the requirements for data analytics
  • Created External schema in Redshift for S3 data.
  • Hands on experience on creating IAM Roles and Polcies

Junior Developer

SCIO Health Analytics
01.2017 - 01.2019
  • Analyzing the business requirements in the Ad-hoc forms
  • Worked on writing Spark SQL Queries using Glue Job based on the requirement receiving in Ad-hoc forms and provided the query results to customers' S3 buckets

Education

MBA - Information Management Systems

Bharathiyar University
10.2023

Bachelor of Engineering - Electrical and Electronics Engineering

Anna University
05.2015

Skills

  • AWS
  • SQL
  • Python/PySpark
  • Pandas
  • Datawrangler
  • Git
  • Terraform/Kubernetes
  • Snowflakes
  • DBT
  • BigQuery/Cloud SQL/Cloud Storage/Bigtable
  • Data Bricks/Data Factory/Cosmos DB
  • Apache Spark
  • Airflow

Certification

  • AWS
  • Python
  • SQL
  • Snowflakes
  • DBT
  • Terraform
  • Gitlab

Languages

Tamil
Urdu
English

Hobbies and Interests

  • Exploring new technologies
  • Data Analysis Projects
  • Mentoring or Teaching
  • Routine Excellence

Career Experience

  • Senior Data Engineer, HTC Global Services, Chennai, Tamil Nadu, 05/01/20, Enttech data analytics, Statefarm Insurance Company, Designed and implemented data pipelines for large-scale, complex data systems using AWS Stack (S3, Glue, Redshift, Athena, SNS, SQS, Step Functions, Lambda, DynamoDB, and Terraform)., Designed and implemented robust ETL pipelines using AWS Glue to integrate with S3, Redshift, and Glue catalog., Orchestrated Glue resources using Glue workflow, Glue triggers, and Step Functions., Created Athena Iceberg table for S3 data modifications and wrote time travel queries., Worked on Python modules for data analytics and ETL Transformations., Generated S3 bucket events notifications with SNS Topic, SQS Queue, and Lambda function., Deployed resources in AWS using Visual Studio code, Gitlab, Terraform, and Scalr workspace., Created cross-account permissions through IAM Roles and imposed firewall rules.
  • Software Developer, 3i Infotech Pvt Ltd, Chennai, Tamil Nadu, 01/01/19, 04/30/20, Premia Life Insurance, National Takaful Insurance Company, Worked on writing Oracle SQL Queries., Developed Glue Job to integrate with Redshift and S3 for ETL Pipeline., Automated Glue resources using Glue workflow and Glue triggers.
  • Junior Developer, SCIO Health Analytics, Chennai, Tamil Nadu, 01/01/17, 01/31/19, Ad-Hoc Data Analytics, Humana Insurance, Analyzed business requirements in the Ad-hoc forms., Worked on writing SQL Queries based on the requirements.

Interests

Exploring new technologies
Data Analysis Projects
Mentoring or Teaching
Routine Excellence

Timeline

Senior Data Engineer

HTC Global Services
05.2020 - Current

Software Developer

3i Infotech Pvt Ltd
01.2019 - 04.2020

Junior Developer

SCIO Health Analytics
01.2017 - 01.2019

Bachelor of Engineering - Electrical and Electronics Engineering

Anna University

MBA - Information Management Systems

Bharathiyar University
Sheik Irfan