MAGESHWAR R
#30115, Sobha Dream Acres, Wing 30, Balagare Road, VarthurBangalore
Summary
To excel as Software professional and hold up a strong position in an organization through diligence and dedication.
Overview
10
10
years of professional experience
12
12
years of post-secondary education
Work History
SENIOR DATA ENGINEER
TIGER ANALYTICS
1 2024 - Current
- Worked closely with various stakeholders like Project Managers, Data Architects and Client Architects in planning and developing data pipelines
- Managing a team of 11 employees, providing guidance and support to ensure successful completion of projects
- Lead weekly team meetings to update team members of project goals and objectives
- Developed strategies to increase team productivity and efficiency
- Developed Azure Data Factory (ADF) pipelines responsible for fetching the Logic App Run history details for each Logic app within a specific resource group via REST API call
- These pipelines runs hourly and merges the collected data into a target Delta table using a Databricks notebook
- Designed ADF pipelines and Logic apps to automate the monthly process of sending email notifications to stakeholders
- These emails contain information about newly acquired hospital contracts with key retailers from the previous month
- Designed and implemented a tiered architecture (Bronze, Silver, Gold) in Databricks on Azure, using Delta sharing as the source
- Developed a logic to efficiently handle deletion of records during incremental fetch, prioritizing removal of records where the deletion feed_date exceeds the source feed_date before loading into the target gold table
- Crafted reusable PySpark and Spark SQL code for various data processing and analysis tasks
- Collaborated closely with Data Analysts and Data Scientists, fostering a multidisciplinary approach to data projects.
DATA ENGINEERING SPECIALIST
ACCENTURE SOLUTIONS PVT LTD
09.2021 - 01.2024
- Experience in creating, configuring and managing BigData ecosystem on-premise
- Profound experience in performing Data Ingestion, Data Processing (Transformations, enrichment & aggregations)
- Experience in Amazon services like EC2, S3, Glue, Lambda, EMR, Redshift, DynamoDB, Athena, Step functions & other services of AWS family
- Experience in designing and developing data processing module using Pyspark
- Experience in Creating, Debugging, Scheduling & Monitoring tasks in Airflow
- Experience in improving and optimizing Pyspark and HiveQL jobs
- Efficient in understanding business requirement and transforming into data model
- Created Multiple Incremental and History Load pipelines and reduced the execution time of the pipelines by 80% by implementing parallel execution logic
- Worked on creating the Big Data Pipelines for one of the Banking Sector from US
- Responsible for creating Pipelines and triggering it using Azure Data Factory
- Responsible for writing Spark Scala codes to flow data from Various layers in Azure
- Worked on creating Databricks Notebooks to have the data into various zones
- Worked on writing Spark codes to perform CDC, SCD operation
- Involved in End-to-End data modelling and data mapping on the various use cases
- Worked closely with Business Analyst team to design the data modelling
- Have good experience on creating dashboards in Tableau
- Worked on creating the Tableau reports using views created in MySQL
- Involved in merging dashboards from various live sources
- Developed Tableau workbooks from multiple data sources using Data Blending
- Gathering the Test Input data
- Test Input data includes Pre-condition, Test Input, Test Result, Test Regression data
- Prepared and conducted timely performances and appraisals of direct reports
- Mentor junior team members in customer service project
- Monitor project progress and adjust as needed
- Managed full project life cycle for clients and ensured top quality deliveries
- Coordinated with development and quality assurance teams to maintain schedules
- Actively listened to client objectives and concerns to maximize satisfaction.
TECHNOLOGY ANALYST
INFOSYS LIMITED
09.2018 - 09.2021
- Worked on the Internal Solution development of Infosys
- Worked on creating Spark and Scala codes for the Data Ingestion
- Worked on creating Data Pipelines for the Solutions built using Spark and Scala
- Installed and configured Solution on different environments
- Created the data pipelines in the Solution tool
- Written an automated script to configure the mount point to the instance, so that each time an instance is restarted mount point gets attached automatically
- Tested the pipelines created for different source systems
- Created buckets and tested the data retrieval and data loading using the Solution Pipelines
- Created a metadata using AWS Glue and queried using AWS Athena for the analysis purpose.
APPLICATION DEVELOPMENT ANALYST
ACCENTURE SOLUTIONS PVT LTD
01.2014 - 09.2018
- Responsible for building scalable distributed data solutions using Hadoop
- Handled importing of data from various data sources, performed transformations using Hive, MapReduce, loaded data into HDFS and Extracted the data from MySQL into HDFS using Sqoop
- Analyzed the data by performing Hive queries and running Pig scripts to know user behavior
- Continuous monitoring and managing the Hadoop cluster through Cloudera Manager
- Created various node range of CDH/HDP clusters on AWS cloud for the performance benchmarking
- Developed Hive queries to process the data and generate the data cubes for visualizing
- Experience working independently and as part of the team to debug application issues working with configuration files/databases and application log files
- Have good knowledge on Optimization and Performance tuning
- Experience in working with “GIT” shared code repository
- Experience in Performance Benchmarking for various customers
- Worked on the databases of the Amazon RDS and carried out functionalities for creating instances as per the requirements
- Developed Shell Scripts for automation purpose
- Done complete End to End testing with Alteryx for Full load
- Building, publishing customized interactive reports and dashboards, report scheduling using Tableau server
- Developed Tableau visualizations and dashboards using Tableau Desktop
- Developed Tableau workbooks from multiple data sources using Data Blending
- Created Batch Stored Procedures for in the Report Scheduler according to the monthly, weekly or daily.
APPLICATION DEVELOPMENT ASSOCIATE
ACCENTURE SOLUTIONS PVT LTD
01.2014 - 01.2016
- Deals with maintaining the database of the users involve in betting
- Maintaining the loyalty and payment for the active users involved in betting
- This project deals with providing monetary rewards instantly to players based on certain conditions
- Involves Anti Money Laundering for users
- Provided maintenance support
- Analyzing requirements.
Education
BE -
SRI RAMAKRISHNA ENGINEERING COLLEGE
COIMBATORE 04.2001 - 05.2013
HSC - undefined
sri vijay vidyalaya matriculation higher secondary SCHOOL
Post Graduate CERTIFICATION IN BUSINESS ANALYTICS AND APPLICATIONS - undefined
IIM TRICHY
Skills
PySpark
Spark
Scala
PowerBI
AWS - S3, Glue, EMR, Redshift, Athena, Step Functions
Python
Hadoop
Tableau
Azure – Data Bricks, ADLS Gen2, ADF, Azure SQL, Cosmos DB
SQL
Basics of Machine Learning
Capstone Project
After Sales Customer Retention Strategy – Heavy Duty Vehicles, PYTHON, R, TABLEAU, PREDICTIVE ANALYSIS USING MACHINE LEARNING MODEL, A heavy-duty commercial vehicle dealership wishes to retain its customers for Service after they buy a vehicle from them. The project must identify the root cause of the problem and recommend a solution to keep these current customers even after the free Service days are over. The best customer retention strategies allow a company to build long-term relationships with customers who will become brand loyalists. As a result, the company's revenue will increase., Customer retention is vital in driving repeat purchases and ongoing value from your customer base. One oft-cited rule of thumb is that it costs five times as much to acquire a new customer as it does to retain an existing customer. Two of the most important factors in improving customer retention is understanding your customers’ satisfaction and loyalty. Businesses also need to understand any operations that may turn off potential and existing customers, such as slow or poor customer Service or a faulty product. To develop a solution that analyses customer Sales and Services to determine the root cause of the low Service turnover. Also, recommend a set of target customers so that the after-Sales Service team can contact and bring them back for Service.
Disclaimer
I hereby declare that the above-mentioned information is true to my knowledge and I bear the responsibility for the correctness of the above-mentioned.
Timeline
DATA ENGINEERING SPECIALIST
ACCENTURE SOLUTIONS PVT LTD
09.2021 - 01.2024
TECHNOLOGY ANALYST
INFOSYS LIMITED
09.2018 - 09.2021
APPLICATION DEVELOPMENT ANALYST
ACCENTURE SOLUTIONS PVT LTD
01.2014 - 09.2018
APPLICATION DEVELOPMENT ASSOCIATE
ACCENTURE SOLUTIONS PVT LTD
01.2014 - 01.2016
BE -
SRI RAMAKRISHNA ENGINEERING COLLEGE
04.2001 - 05.2013
SENIOR DATA ENGINEER
TIGER ANALYTICS
1 2024 - Current
HSC - undefined
sri vijay vidyalaya matriculation higher secondary SCHOOL
Post Graduate CERTIFICATION IN BUSINESS ANALYTICS AND APPLICATIONS - undefined
IIM TRICHY