Data Engineer with six years of experience in designing and maintaining robust data pipelines. Expertise in Big Data technologies, cloud platforms, and data warehousing. Proven skills in ETL processes and DevOps practices, driving efficiency and scalability in data management. Ready to leverage technical knowledge to enhance organizational data strategies.
Overview
6
6
years of professional experience
Work History
Operations Engineer 2 (Data Engineer)
Comcast
06.2022 - Current
Developed and optimized data pipelines using Hadoop, Spark, and Databricks to process large-scale broadcast data.
Implemented Kafka-based streaming solutions to handle real-time event data.
Created and managed data warehousing solutions on Snowflake and BigQuery for reporting and analytics.
Automated ETL workflows using Informatica and DBT to streamline data transformation and ingestion.
Designed Airflow workflows for efficient scheduling, monitoring, and pipeline orchestration.
Built and maintained Azure cloud-based data solutions to ensure scalability and reliability.
Applied DevOps concepts by managing infrastructure with Docker, Kubernetes, Terraform, and version control with GitHub & Git.
Analyzed user requirements, designed and developed ETL processes to load enterprise data into the Data Warehouse.
Participated in agile development processes, contributing to sprint planning, stand-ups, and reviews to ensure timely delivery of data projects.
Collaborated with cross-functional teams to gather requirements and translate business needs into technical specifications for data solutions.
Monitored data systems performance, identifying bottlenecks and implementing solutions to maintain system efficiency.
Managed version control and deployment of data applications using Git, Docker, and Jenkins.
Implemented and optimized big data storage solutions, including Hadoop and NoSQL databases, to improve data accessibility and efficiency.
Senior System Engineer (Data Engineer)
Infosys
06.2021 - 06.2022
Designed and developed ETL pipelines to process structured and unstructured retail business data.
Worked extensively with Hadoop, Hive, and Spark for large-scale batch data processing.
Managed and optimized data storage solutions using MongoDB and Cassandra for fast retrieval and indexing.
Automated data integration workflows using Airflow for seamless scheduling and monitoring.
Developed efficient SQL queries for business intelligence reporting on Snowflake and BigQuery.
Implemented Kafka-based streaming to capture real-time retail transactions.
System Engineer (Data Analyst)
Infosys
05.2019 - 06.2021
Created and maintained SQL-based analytical reports for business decision-making.
Built foundational ETL processes for data cleaning and transformation.
Collected, tracked and reviewed data to evaluate business and market trends.
Generated reports and obtained data to develop analytics on key performance and operational metrics.
Performed data entry, data cleaning, and data coding for analysis.
Developed dashboards with Tableau to monitor key performance indicators.
Built visualizations that enabled users to quickly interpret results from complex analyses.
Education
Bachelor of Engineering - Mechanical Engineering
Anna University
Chennai
04-2019
12th - HSC
SRV Boys Higher Secondary School
03-2015
10th - SSLC
Ramkrishna Vidyalaya Matric School
03-2013
Skills
Languages: Python, SQL
Big Data & Storage: Hadoop, Hive, Kafka, MongoDB, Cassandra, Spark
Cloud & Data Warehousing: Databricks, Airflow, Snowflake, BigQuery, Azure
ETL Tools: Informatica, DBT (Low-code ETL)
DevOps & Infrastructure: DevOps concepts for Big Data, GitHub & Git, Docker, Kubernetes, Terraform