- Gaming
- Listening to music
- Films & TV series
- Playing Badminton
- Playing Volleyball
Anup Kumar Vishwakarma
Summary
Microsoft Certified Azure/AWS Data Engineer with 10.5 years of progressive experience in the end-to-end lifecycle of cloud and ETL data warehousing projects. Proven expertise in analysis, design, development, monitoring, implementation, and maintenance of cloud-based solutions. Highly self-motivated and passionate about learning cutting-edge technologies, seeking challenging opportunities to leverage cloud-based technologies and drive innovative data solutions.
Overview
11
11
years of professional experience
Work History
SENIOR TECHNICAL CONSULTANT
Nityo – British Petroleum
07.2023 - 10.2023
- Lead the design and implementation of data architectures on AWS, leveraging S3 for scalable and cost-effective storage solutions
- Well-acquainted with database engineering, data warehouses, and data processing
- Knowledgeable in Python frameworks related to data
- Collaborate with cross-functional teams to gather requirements, design data models, and implement ETL processes for efficient data transformation
- Implement version control best practices using GitHub to manage and track changes in data pipelines and analytics code
- Utilize AWS Lambda functions to build serverless data processing workflows, ensuring optimal performance and resource utilization
- Design and optimize data pipelines for large-scale data processing, ensuring data quality, reliability, and timely delivery
- Implement and maintain security and access controls for sensitive data stored in S3 buckets.
SUBJECT MATTER EXPERT CONSULTANT
Quess Corp – Shell India
07.2022 - 07.2023
- We must move the existing enterprise integration tool, which is SSIS from on Prem to a cloud version to be more compliant with Shell
- A tough guideline and the cloud first strategy is nothing but the target operating environment
- And why are we doing this because the current version would go end of life in July 2026, which ideally means we still have around 1800 odd interfaces
- Those interfaces must go for a migration, which will bring in the migration cost to migration team.
TECHNICAL CONSULTANT
Chevron as FreeLancer – PowerCore
04.2022 - 07.2022
- Built a real-time data processing pipeline using Apache Kafka and Apache Spark to identify fraudulent transactions
- Integrated with ML models for fraud detection
- Offer services to design, build, and optimize data pipelines for clients who need to extract, transform, and load (ETL) data from various sources into their data warehouses or analytics platforms
- Collaborated with senior team members to troubleshoot data integration issues and ensure data integrity
- Assist clients in designing and optimizing their database structures for efficient data storage, retrieval, and analysis
- Help clients migrate their on-premises databases or data warehouses to cloud platforms like AWS, Google Cloud, or Azure.
SENIOR CONSULTANT
Value Global Services – Hilti, Pune
02.2020 - 03.2022
- Building a Domain Data Store with relevant data which subsequently is getting used by the Power App
- Design and Development of ADF, Databricks and Azure SQL database
- Understand business requirements from business stakeholders and assess the timelines of disclosing the source till production
- Create re-usable Data Factory pipelines/activities for moving source files
- Integrating with Databricks for processing and transforming data
- Handling the Operations part during failure in the pipelines
- Bringing source and Business together for a good collaboration
- Create re-usable Data Factory pipelines/activities for moving source files
- Integrating with Databricks for processing and transforming data
- Handling the Operations part during failure in the pipelines
- Collaborated with Web Application Engineers, used Python scripts to load the data into Azure Cloud database
- Created automation test script in Python
- Strong Experience and skills in API (rest, web) Development using .Net.
CONSULTANT
Altius Services – Albert Heijn, Pune
10.2017 - 01.2020
- Setting up a dev-ops team at offshore consisting of 10 associates
- Interviewing profiles, finding right skill sets and candidates, grooming and nurturing them technically and professionally, merging existing development and operations team to work in a dev-ops way
- Creating the project documents, requirement documents, drafting knowledge transition plans and knowledge articles for new associates
- Working as a Dev-Ops Engineer, I was responsible for developing and maintaining the software components on Oracle, Teradata, DB2 and Unix
- Also started with Cloud migration projects where in I was responsible for moving data from On-premises to Cloud with Data Factory of Microsoft Azure
- Creating Data Factory pipelines with the necessary validations as required by Business users
- Creating integration runtimes, Linked services, and triggers
- Creating blob storages for storing files
- Maintaining secrets and credentials in Azure Active Directory.
SENIOR SOFTWARE ENGINEER
Clover InfoTech Services – Avantha ERGO / IDFC Bank, Mumbai
11.2015 - 09.2017
- As a Senior coordinator for a Data warehousing project, I was responsible for maintaining a smooth and efficient co-ordination between Client, Onshore and Offshore
- Regular meetings with clients/stakeholders to understand the Business requirements, Issues, Concerns, Feedback
- Also worked as functional maintenance in the client's team wherein I was responsible for the functional aspects of the project, meetings with Business users to understand the functional requirements, transforming functional requirements to technical design for the development team, assessing impact of planned changes on infrastructure or application level, Go/NoGo decision authority for changes, reducing recurring issues and Incident counts
- Maintaining co-ordination and collaboration between infrastructure teams, development teams, Operations teams, Business users and stakeholders
- Responsible for monitoring daily batches and ensuring data deliveries to users within agreed SLA's
- Responsible for handling major crisis with minimum possible downtime, finding RCA's with required stakeholders and closing the RCA's with a permanent fix
- Developing solutions and providing ideations for automating manual tasks, reducing recurring incidents, reducing batch timings and cost savings
- Responsible for following Agile way of working
- Was also responsible for grooming new associates
- Involvement in multiple POC's for shifting existing on-premises architecture to cloud (Microsoft Azure).
SOFTWARE ENGINEER
Primebiz Info Solutions Services – ABN AMRO Bank, Mumbai
10.2010 - 08.2014
- Joined as a development engineer for a Data warehousing project of ABN AMRO Bank which is a client of TCS
- As a developer I was responsible for understanding the Business requirements and its mapping to technical documentations, understanding nature of sources and desired output, create ETL Informatica mappings, creating necessary validations in the form of Data Quality checks, working with multiple teams to bring the software components together, creating service now changes for deployments, creating runbooks for efficient deployments, periodic meetings with stakeholders, aftercare
- Also worked as operations engineer in the same team wherein, I was responsible for Batch monitoring, Incidents Problems and Change Management in Service now, have worked on multiple major crisis situations, experienced in handling all types of Incidents, periodic status reporting to client and stakeholders, working on RCA's of major issues and putting permanent fixes
- Was responsible for solving customer queries and issues within agreed SLA.
Education
BACHELOR OF TECHNOLOGY (IT) -
PCE Rourkela BPUT
Centre For Development of Advanced Computing (Design in System Software) -
Bangalore
Skills
- AWS
- Azure Data Factory
- Azure Databricks
- Azure DevOps
- Microsoft SQL
- S3
- DB2
- Power Apps
- Service Now
- Jira
- Power BI
- Confluence
- Python
- PySpark
- MSBI
- P > Data analysis
- P > Solutions-focused sales
Interests
Timeline
SENIOR TECHNICAL CONSULTANT
Nityo – British Petroleum
07.2023 - 10.2023
SUBJECT MATTER EXPERT CONSULTANT
Quess Corp – Shell India
07.2022 - 07.2023
TECHNICAL CONSULTANT
Chevron as FreeLancer – PowerCore
04.2022 - 07.2022
SENIOR CONSULTANT
Value Global Services – Hilti, Pune
02.2020 - 03.2022
CONSULTANT
Altius Services – Albert Heijn, Pune
10.2017 - 01.2020
SENIOR SOFTWARE ENGINEER
Clover InfoTech Services – Avantha ERGO / IDFC Bank, Mumbai
11.2015 - 09.2017
SOFTWARE ENGINEER
Primebiz Info Solutions Services – ABN AMRO Bank, Mumbai
10.2010 - 08.2014
BACHELOR OF TECHNOLOGY (IT) -
PCE Rourkela BPUT
Centre For Development of Advanced Computing (Design in System Software) -
Bangalore
Similar Profiles
Anup Kumar VishwakarmaAnup Kumar Vishwakarma
SENIOR TECHNICAL CONSULTANT at Nityo – British PetroleumSENIOR TECHNICAL CONSULTANT at Nityo – British Petroleum
Reda ZaghloulReda Zaghloul
Project Manager at Pharaonic Petroleum Co. PhPC (British Petroleum bp - JV)Project Manager at Pharaonic Petroleum Co. PhPC (British Petroleum bp - JV)
Puravi MhatrePuravi Mhatre
Learning Services Lead at British PetroleumLearning Services Lead at British Petroleum
Sarath Chandu VatturiSarath Chandu Vatturi
Senior Technical Consultant at Hitachi VantaraSenior Technical Consultant at Hitachi Vantara
Ganesh JosyulaGanesh Josyula
Senior Technical Consultant at InfovitySenior Technical Consultant at Infovity