Summary
Overview
Work History
Education
Skills
Certification
Timeline
Generic

Venkata Ganesh

Inida

Summary

10+ years of IT Experienced and results-oriented Data Engineer with 5+ years of hands-on expertise in designing, implementing, and maintaining scalable data pipelines. Proficient in various ETL tools, cloud platforms, and programming languages. Adept at collaborating with cross-functional teams to deliver data-driven solutions that meet business objectives. Strong analytical skills coupled with a deep understanding of database management and optimization techniques.

Overview

10
10
years of professional experience
1
1
Certification

Work History

Senior Data Engineer

DXC Technology
02.2022 - 08.2022
  • Project 1: Supply Chain Reporting Analysis
  • Role: Senior Data Engineer
  • Objective: The project aims to leverage data engineering techniques to integrate process, analyze, and visualize supply chain data, enabling stakeholders to optimize operations, mitigate risks, and make data-driven decisions
  • Description: The Supply Chain Reporting Analysis project aims to streamline the collection, processing, and analysis of supply chain data by leveraging Azure cloud services
  • The primary objective is to build an end-to-end data engineering solution that collects data from various source systems, processes it using Azure Databricks (ADB) notebooks, and loads it into SQL Data Warehouse (SQLDWH) tables for reporting and analysis purposes
  • Roles and Responsibilities:
  • Design and develop data pipelines in Azure Data Factory to ingest and load data from diverse sources into target data stores
  • Experience in Configure and optimize data movement and transformation activities to ensure efficient and reliable data processing
  • Experience in Monitor pipeline executions, troubleshoot errors, and optimize performance using ADF monitoring and diagnostic tools
  • Experience in implementing Spark applications using PySpark and Spark-SQL for Data extraction, transformation and aggregation from multiple file formats for analyzing and transforming data to cover insights into customer usage patterns using Azure Databricks
  • Developed and optimize Spark jobs, transformations, and workflows to cleanse, transform, and enrich raw data into structured datasets
  • Experience in Identify performance bottlenecks in pyspark jobs and optimize Spark configurations, resource allocation, and parallelism settings to improve job performance
  • Experience in implementation of Meta-data driven Framework approach for handling data in multiple layer configurations (SRC-RDS-SDM-CDM-DWH)
  • Experience in Medallion architecture pattern
  • Experience in Data Lake (ADLS Gen2) and Lake house storage accounts
  • Good Experience in working on Agile Projects and Involved on Program Implementation planning to create a feature for all Business requirements
  • Experience in CI/CD pipelines for deploying the code from DEV to QA to PROD using Azure Devops
  • Experience in Collaborate with data analysts and data scientists to define data requirements, validate analytical results, and derive actionable insights
  • Good Experience on providing Ad-hoc models manage POC’s for customer to improve business KPI’s
  • Good Experience in Data modeling and Data warehouse Concepts.

Data Engineer

Cognizant Technology
05.2018 - 01.2022
  • Project 2: Customer Experience and Analytics
  • Role: Data Engineer
  • Objective: Managing the dataflow from upstream sources and providing feed files to the survey engine for sending the surveys and prepare the data using Databricks for performs the analytics
  • Description: Business capture feedback everywhere their customers are, understand it in real-time, and deliver insights and action across their organization
  • This empowers organizations and their employees to embed pulse of customer into their daily decisions
  • Through this Experience Cd, companies can create more loyal customers, grow faster, reduce costs and improve corporate culture
  • Roles and Responsibilities: Developing and managing data factory pipelines responsible for end-to-end orchestration from copying data from source to target
  • Handling all Parquet and CSV from Source location to Land Layer
  • Handling structure and contents of nested JSON data to identify key fields, arrays, struct and nested objects
  • Handling Curated layer which has more transformation is involved1.Identify corrupt records which can’t be survey for that batch.2.Analyze bad data and perform additional validations based on corrupted data and provide work around fix and process data and also Responsible for Processed layer which has used for reporting
  • Debugging stored procedure and identifying data which is not coming as expected or causing failure in load
  • Reaching out to sources in case of data inflow more than expected and in cases of any table load failure from their end
  • Testing changes in lower environments and production during a bug fix
  • Connecting with product owners for gathering requirements for future enhancements and implementing same
  • Assisting source and target teams by conducting the impact analysis of their changes on our data flows and Landing layer and also providing RCA to business for that failure
  • Involved in Sprint Backlog and Refinement meeting for creating and assigning stories to planned resources
  • Create ad-hoc Notebooks using Data bricks for business that wants data in single flow and should be loaded data in to on demand run
  • Working with Subject Matter Experts to understands requirements and create business requirement document.

ETL Developer

Capgemini
03.2016 - 04.2018
  • Project 3: FMS-Analytics
  • Role: ETL Developer
  • Description: In this project implementing data warehouse for Forecasting, Marketing and Sales Performance reports.The data is obtained from Relational tables and flat files.I was involved in cleansing and transformation data in staging area and loading into oracle data marts.it is used for extensive reporting.
  • Responsibilities:
  • Used Informatica Power Center for (ETL) extraction, transformation and loading data from heterogeneous source systems into target databases
  • Participated in ETL Design of new or changing mappings and workflows with the team and prepares technical specifications
  • Creates ETL Mappings, Mapplets, Workflows, and Worklets using Informatica PowerCenter and prepare corresponding documentation
  • Designs and builds integrations supporting standard data warehousing objects (type-2 dimensions, aggregations, star schema and snow-flake schema etc.)
  • Worked on source system analysis to better understanding business model
  • Worked with DBAs and Data Architects to plan and implement appropriate data partitioning strategy in Enterprise Data Warehouse
  • Implements versioning of the ETL repository and supporting code as necessary
  • Develops stored procedures, database triggers and SQL queries
  • Implements best practices and tunes SQL code for optimization
  • Worked with Informatica Data Quality (Analyst and Developer)
  • Worked in preparing TDD and Test cases
  • Experienced in performance tuning techniques.

Reporting Developer

Birlasoft
12.2012 - 10.2015
  • Project 4: Cost Service Analytics
  • Role: Reporting Developer
  • Description: GE Aviation is among the top aircraft engine supplier.CSA_WIP_Shop_Analytics tool analyze amount of each ESN cost services analytics.This tool calculates different plant code information like Malaysia, Celma, Caledonian, strother and EATS. End users’ analyses Each ESN cost analytics on different gates stages
  • Roles and Responsibilities:
  • Understanding the requirements on user story based and communicates with End users and ERP Team to Build the Query and Creating Reports and Dash Boards
  • Deploying, Migration of Code from one Environment to Other Environment like Non-Prod and Prod
  • Involved in creating complex and innovative reports as per customer need using various types of visualization present in Spotfire taking advantage of all Spot fire functions including Data functions, Python Scripting etc
  • Applying Write-Back Features to facilitates the end-user update the Data from Spotfire End
  • Applying Row-Level Security Based on their Group privileges
  • Design and development of report queries like Business Views using ADS
  • Creating Test Cases based of the design specifications
  • Expertise in review and documentation of system Test plans and creating Test cases on the basis of products releases features, client's requirements and related technical document
  • Involved Unit testing and Error Handling for the Reports and Data Validation for the Required Results.

Education

Master of Computer Applications -

BVCITS
JNTUK
06.2010

Skills

  • ADF,ADB(pyspark,sparksql),devops
  • Python
  • OBIEE,Tibco Spotfire
  • SQL Server,Oracle

Certification

Az-900

Timeline

Senior Data Engineer

DXC Technology
02.2022 - 08.2022

Data Engineer

Cognizant Technology
05.2018 - 01.2022

ETL Developer

Capgemini
03.2016 - 04.2018

Reporting Developer

Birlasoft
12.2012 - 10.2015

Master of Computer Applications -

BVCITS

Similar Profiles

CREATE PROFILE
Venkata Ganesh