Sheshkumar K
Senior Data Engineer Lead
Hyderabad
Summary
Dynamic and results-driven Data Engineer with over 19 years of IT industry experience and 5+ years specializing in Azure Data Engineering. Proficient in designing, implementing, and optimizing scalable data solutions using Azure technologies such as Azure Data Factory, Azure Databricks, Azure SQL Database, and Azure Synapse Analytics. Demonstrated expertise in database design and development, with a strong focus on data ingestion through Azure Data Factory, storage in Azure Data Lake, processing with Azure Databricks, and data loading into Azure Synapse. Skilled in managing Databricks workspaces, developing and maintaining notebooks, clusters, and Delta Lake with PySpark and Spark SQL. Proven ability to collaborate with stakeholders, translate business requirements into technical solutions, and lead teams to deliver high-quality results. Certified PMP and Professional Scrum Master.
Overview
9
9
years of professional experience
4
4
Certifications
Work History
Senior Data Engineer and Lead
Vantage Data Centers
05.2024 - Current
- Engineered and implemented high-performance data solutions using Azure Databricks and PySpark
- Designed and implemented a robust medallion architecture (bronze, silver, gold layers) to streamline data processing and enhance data quality
- Developed complex SQL queries for performance optimization and custom business transformations
- Employed best practices for writing efficient PySpark SQL code, significantly improving query performance
- Managed Azure Data Factory, including activities like ForEach and Get Metadata, and Azure Data Lake Storage to resolve data loading, query performance, and system errors, ensuring seamless data flow
- Conducted comprehensive performance tuning to enhance the efficiency of data pipelines
- Provided expert support during troubleshooting and issue resolution, successfully resolving critical issues in the HR Workday application
Senior Data Engineer
Carestream Health - Data Migration
09.2023 - 03.2024
- Architected and implemented database solutions in Azure SQL Data Warehouse, optimizing data storage and retrieval
- Migrated stored procedures, tables, views, and indexes to Azure Synapse SQL dedicated pool, enhancing data accessibility and performance
- Developed and optimized SQL queries for high performance and complex business transformations
- Implemented best practices for writing efficient T-SQL code, improving system reliability and performance
- Managed Azure Data Factory activities, including For Each and Get Metadata, and Azure Data Lake Storage, addressing data loading, query performance, and system errors
- Conducted performance tuning to optimize data workflows and enhance system efficiency
Senior Data Engineer
Wolters Kluwer - Customer and Entity Management System
10.2021 - 07.2023
- Creating File formats based on file types like (comma Separated, Pipe separated and JSON).
- Analytics team request basis creating of sample tables to reduce costing and storage.
- Experience in reviewing Python code for running the troubleshooting test-cases and bug issues.
- Validating data by using validation mode option.
- Taking care of data loading from inbound to outbound.
- In this process I have to load data from AWS S3 bucket to snowflake tables.
- Data masking implementation to hide secured information.
- Creating storage integration objects to establish relation between AWS S3 and Snowflake.
- Views and materialized for reporting purpose.
- Creating internal and external stages based on requirement. Designed and automated data
- ingestion processes into Snowflake using DBT.
- Designed and developed DBT models aligned with specific business requirements and
- objectives.
- Developed DBT models, materializations, and seed concepts within the DBT development IDE.
- Maintained and enhanced DBT models to transform raw data into structured datasets for analytical purposes.
- Involved in creation of data sampling for reader account.
- Designed and automated data ingestion processes into Snowflake using DBT.
- In data quality we have got one house in data quality framework which we maintain
- We have got that support central form where we gathered the data quality information
- from the track team and based on that we prepared input file we posted it we monitor it
Senior Data Engineer
Arthrex
11.2019 - 08.2021
- Design and Implement Scalable Database Solutions:
- Use Hadoop to design and deploy scalable database architectures that can handle large volumes of data efficiently.
- Implement distributed storage and processing to ensure data scalability and reliability.
- Develop databases that provide easy and quick access to data.
- Utilize Hadoop's HDFS to store and manage data, ensuring high availability and accessibility.
- Optimize Database Performance:
- Use Hadoop Map-reduce to optimize data processing workflows.
- Streamline Data Management:
- Implement techniques to reduce data redundancy and improve data management efficiency.
- Use Hadoop tools like Hive and Pig to manage and query data effectively.
SAP BO Admin
Zurich UK
07.2015 - 10.2019
- Migrated SAP BO reports from Dev to Production using Promotion Management, ensuring smooth transition and data integrity
- Migrated and optimized tables in Azure SQL MI, enhancing data storage and performance
- Participated in project planning sessions, collaborating with project managers and business analysts to design and implement effective data solutions
- Resolved daily production issues, maintaining system stability and performance
- Upgraded BO from SP4 to SP6 and applied service packs and fix packs to ensure system reliability
- Conducted performance tuning to optimize system performance and data workflows
- Investigated and resolved system issues, providing effective support to prevent recurring problems
Education
Masters of Computer Applications(OU) - Computers
Rajiv Gandhi Degree & PG College
Hyderabad06-1997
Skills
ADF, ADLS, Azure Synapse Analytic, Azure
Certification
Azure Data Engineer Associate
Timeline
Senior Data Engineer and Lead
Vantage Data Centers
05.2024 - Current
Senior Data Engineer
Carestream Health - Data Migration
09.2023 - 03.2024
Senior Data Engineer
Wolters Kluwer - Customer and Entity Management System
10.2021 - 07.2023
Senior Data Engineer
Arthrex
11.2019 - 08.2021
SAP BO Admin
Zurich UK
07.2015 - 10.2019
Masters of Computer Applications(OU) - Computers
Rajiv Gandhi Degree & PG College
Similar Profiles
Varma PinnamarajuVarma Pinnamaraju
Senior Energy Manager at Vantage Data Centers APAC Pte LtdSenior Energy Manager at Vantage Data Centers APAC Pte Ltd
Lexee MarshallLexee Marshall
Talent Acquisition Partner at Vantage Data CentersTalent Acquisition Partner at Vantage Data Centers
Garry GilesGarry Giles
Critical Facilities Engineer at Vantage Data CentreCritical Facilities Engineer at Vantage Data Centre
Sunil SinghSunil Singh
Internship Student at ForageInternship Student at Forage
Venkata Krishna Reddy TallapureddyVenkata Krishna Reddy Tallapureddy
Institutional Sales Manager at DSP Asset Managers Private LimitedInstitutional Sales Manager at DSP Asset Managers Private Limited