Experienced Data Engineer with expertise in SQL & ETL development, cloud & data engineering, and data science. Proficient in PySpark, Hive, AWS, Denodo, and Snowflake. Skilled in automating ETL pipelines, constructing serverless architectures, and optimizing data workflows.
Overview
2
2
years of professional experience
Work History
Cloud and Data Engineer
InfoCepts Technologies
Chennai
02.2023 - 06.2024
Proficient in analyzing Denodo workflows, sessions, and mappings, resolving issues through backtracking in Denodo and SQL.
Documented analyses on Confluence and utilized Snowflake for efficient table querying.
Repointed the views from Netezza to Snowflake .
Refactored views in snowflake and Repointed them in Denodo according to business requirement.
Developed SQL queries to address specific challenges effectively.
Built an end-to-end ETL pipeline for restaurant data, utilizing PySpark to convert JSON to CSV.
Created structured DB tables with Hive for efficient data storage.
Automated ETL pipeline using shell scripting, reducing manual effort.
Handled large CSV files through Amazon S3 and automated data transformations with Glue jobs.
Constructed a serverless AWS Lambda architecture for seamless data processing.
Integrated DynamoDB for metadata and data retrieval.
Worked with technologies including HDFS, Hive, PySpark, and Apache Hadoop,linux,AWS during training.
Worked with technologies including python , data science ,time series forecasting models in data science coe.
Web-Developer Intern
HighRadius Technologies
chennai, tamilnadu
01.2022 - 04.2022
Education
B-tech - Big data analytics
SRM University
Chennai
05-2023
Skills
Mysql/Data warehousing
Python/Pyspark
Snowflake
Denodo
Data Science
AWS cloud
Machine Learning
Linux
Data analyst
ETL
Apache Hadoop/HDFS
Data workflow management
Soft Skills
Analytical Thinking
Problem-Solving
Team Collaboration
Adaptability
Project Management
Critical Thinking
Time Management
Attention to Detail
Project Profiles
InfoCepts Technologies Intern, Developed SQL queries to overcome specific challenges effectively., Built end-to-end ETL pipeline for restaurant data , Utilized PySpark to convert JSON to CSV, easing processing., Created structured DB tables with Hive for efficient storage., Automated the ETL pipeline using shell scripting, reducing manual effort., Handled large CSV files via Amazon S3., Automated data transformations with Glue jobs., Constructed serverless AWS Lambda architecture., Integrated DynamoDB for metadata and data retrieval., Worked on technologies like HDFS, Hive, PySpark, Apache Hadoop.
InfoCepts Snowflake Developer, Analyzing the Denodo workflows, sessions and their mappings., Created analysis document on Confluence., Used Snowflake to query the required tables., Backtrack the mapping in Denodo and SQL if there is any issue in analyzing., Working on Repointing from Netezza to Snowflake., Modified Some Cubes, Tested and validated changes., Helped in Deploying those changes., Denodo VM, Snowflake VM, Confluence, Jira
InfoCepts DataScience COE Python Developer, Analyzing the Dataworkflows, sessions and their mappings., Created analysis document on JupyterNotebook., Used Anaconda to query the required tables., Backtrack the mapping in Python using machine learning concepts, if there is any issue in analyzing., Working on Training the model with SARIMAX., Modified Some Cubes, Tested and validated changes., Helped in Deploying those changes., AnacondaVM, Jupyternotebook, Dataspell
Affiliations
Student commitee leader - Directorate of student affairs - SRM university
Certification
Data science certification - PADAI
SQL - udemy
Linux - udemy
Data bricks - data bricks
Languages
Telugu
First Language
English
Proficient (C2)
C2
Hindi
Intermediate (B1)
B1
Tamil
Advanced (C1)
C1
Operating Sysytems
Windows 32bit / 64 bit
Linux
Ubuntu
Timeline
Cloud and Data Engineer
InfoCepts Technologies
02.2023 - 06.2024
Web-Developer Intern
HighRadius Technologies
01.2022 - 04.2022
B-tech - Big data analytics
SRM University
Similar Profiles
Ishan ShendeIshan Shende
Assistant Manager (Data Analyst/DQE) at InfoCepts TechnologiesAssistant Manager (Data Analyst/DQE) at InfoCepts Technologies