B. Yashwanth Kumar
Summary
- 7+ years of experience in IT with over 4+ years specializing in Azure data engineering.
- Expertise in developing and managing data pipelines using Azure Data Factory, Azure Databricks, and Python.
- Skilled in transforming raw data into actionable insights using PySpark and Scala.
- Proficient in SQL for querying and managing relational databases and handling large datasets.
- Experienced in Apache Kafka for streaming data solutions and real-time data processing.
- Strong understanding and hands-on experience with Azure DevOps for CI/CD pipeline automation and version control.
- Knowledgeable in monitoring and logging using Datadog, and ITSM platforms like ServiceNow for incident management and troubleshooting.
- Experience in working with Azure SQL Databases, Cosmos DB, Azure Blob Storage, and Azure Synapse Analytics for data storage and analytics solutions.
- Excellent communicator with the ability to document processes, collaborate with cross-functional teams, and ensure compliance with data governance policies.
Overview
8
8
years of professional experience
Work History
Cloud Data Engineer
3k Softech Solutions PVT LTD
02.2025 - 02.2026
- Worked with Databricks to develop and optimize scalable data processing workflows.
- Utilized Teradata and SQL for efficient data querying, transformation, and performance tuning.
- Developed data pipelines using PySpark and Python for large-scale data processing and analytics.
- Built and maintained monitoring dashboards in Grafana to track system performance and data pipeline health.
- Managed version control and collaborative development using GitHub.
- Used PuTTY and WinSCP for secure server access and file transfers.
- Wrote complex SQL queries for data extraction, validation, and reporting across multiple data sources.
Azure Data Engineer
Abilio IT Solutions Pvt Ltd
09.2023 - 01.2025
- Develop and manage end-to-end data pipelines using Azure Data Factory for large-scale data integration projects.
- Design and implement Azure Databricks solutions for data processing and real-time analytics, utilizing PySpark and Scala.
- Optimize and automate ETL processes to improve data flow and reduce latency using Azure SQL Database and Azure Synapse Analytics.
- Work with Apache Kafka for stream processing and handling real-time data ingestion and processing.
- Collaborate with data scientists and analysts to ensure seamless data flow and integration for advanced analytics.
- Set up and manage Azure DevOps pipelines for code deployment, version control, and continuous integration.
- Monitor and troubleshoot data pipeline performance using Datadog and Azure Monitor.
- Maintain documentation on data workflows and configurations, ensuring data security and governance compliance.
Azure Data Engineer
Experis IT Pvt Ltd
03.2022 - 06.2023
- Worked with Azure Data Factory to automate data movement and transformation across various environments.
- Implemented PySpark and Scala solutions for large-scale data transformations and processing.
- Conducted data profiling and analysis on raw data sources, ensuring data accuracy and consistency.
- Utilized Azure Blob Storage and Azure SQL Database for data storage and extraction tasks.
- Built and maintained monitoring dashboards in Datadog to track the performance of pipelines and services.
- Assisted in deploying code updates and maintaining data infrastructure using Azure DevOps.
Data Engineer
HCL Technologies
11.2021 - 02.2022
- Assisted in the design and implementation of ETL pipelines using Azure Data Factory.
- Worked with the data engineering team to streamline data ingestion, transformation, and loading processes.
- Collaborated with the business intelligence team for data visualization and reporting.
- Ensured that data workflows were efficient, scalable, and met SLAs.
- Used ServiceNow to manage change requests and incident management for data issues.
Analyst
Amazon Development Center
09.2017 - 06.2021
- Worked on migrating legacy data pipelines to Azure Data Factory and Azure Databricks.
- Designed and implemented scalable, reliable data solutions for multiple internal teams using SQL, Python, and PySpark.
- Integrated Apache Kafka for stream-based data processing and ingestion.
- Conducted real-time data analytics and troubleshooting to ensure optimal performance and reliability.
- Created and optimized batch and real-time data pipelines, ensuring efficient data flow across multiple cloud services.
Education
B.Tech - Computer Science Engineering
Mallareddy Engineering College
Hyderabad01-2017
Skills
- Azure Data Factory
- Azure Databricks
- Apache Kafka
- PySpark
- SQL
- Scala
- Python
- Microsoft Azure
- Azure Synapse Analytics
- Azure Blob Storage
- Azure SQL Database
- Azure Cosmos DB
- Azure DevOps
- Git
- Jenkins
- CI/CD Pipeline Automation
- ETL processes
- Data Transformation
- Data Modeling
- Data Integration
- Datadog
- Azure Monitor
- ServiceNow
- GitHub
- Hadoop
- Airflow
- DAG creation
- Spark
- Power BI
- Windows Server
- Linux
- Azure Linux VM
Languages Known
Telugu
Hindi
English
Disclaimer
I do hereby declare that the above-mentioned details are true to the best of my knowledge.
Timeline
Cloud Data Engineer
3k Softech Solutions PVT LTD
02.2025 - 02.2026
Azure Data Engineer
Abilio IT Solutions Pvt Ltd
09.2023 - 01.2025
Azure Data Engineer
Experis IT Pvt Ltd
03.2022 - 06.2023
Data Engineer
HCL Technologies
11.2021 - 02.2022
Analyst
Amazon Development Center
09.2017 - 06.2021
B.Tech - Computer Science Engineering
Mallareddy Engineering College