Rachna Dongre
Bengaluru
Summary
Experienced Data Engineer capable of converting difficult data into useful insights. Proficient in using Azure cloud services, including Databricks, Data Factory, and SQL, for data collection, processing, and analysis.Expertise in Python, SQL, Apache Spark, Apache Kafka, EDA, and statistical data analysis. Successfully led data engineering initiatives, managed cross-functional teams, and delivered significant solutions. Seeking new challenges in a dynamic, growth-oriented setting.
Overview
11
11
years of professional experience
1
1
Certification
Work History
Senior Software Engineer Manager
MORGAN STANLEY ADVANTAGE DEPARTMENT
10.2022 - 07.2023
- Project name: Data distribution platform DDP
- This project is about Data Distribution and we have maintaining big data of Employee, we receive data from Workday, by scheduled Autosys Jobs, which further distributes this data in MongoDB collection, each specific data goes to specific collection
- My roles in this project to maintain production environment keep the data refreshed in lower non production environment and keep the code base clean and up to date.
- This Role also involve Peoples management, I have handled a team of 4 people, to maintain Production support
- Multiple AWS buckets were used to store the data that the application gathered from various sources. The data was filtered and formatted using Pyspark and Databricks to read from Hadoop cluster.To run and deploy builds, we have Jenkins.
- In addition, we have more than 80 micro services that we must comprehend. As a data engineer, it was my responsibility to leverage those APIs and microservices and move the data from raw to platform-readable form.
- Go live Production support with all the releases
- Mentor to My team and build a timely proper solution to each agile task my team has committed
- Technology Used: Autosys system, Apigee, MongoDB, Jenkins Python, Groovy, Docker , AWS , Hadoop cluster
Team lead
Mindtree India Pvt Ltd
12.2021 - 09.2022
- Project name: kaiser Permanente search relevancy Project
- This project is about Data search and its relevancy
- Raw data will be converted and feed to SQL tables as Step0 pipeline, in Data Lake, from there onwards, Azure ML experiments will calculate the relevancy terms as step 1to5 in pipeline and feed it back to ADLS, and ADLS will publish the data to Tableau as step07 pipeline
- My role was to get the data flow using Jenkins's pipeline automation and I have designed Data pipeline for Azure data factory .
- All the steps were converted into Jenkins pipeline and they are connected in sequence.
- I have managed and guided team of 3 people to get this work done fast and proficient.
- Worked end to end creation of Pipeline in ADF right from creating resource group, Migration of on-premise data to data flow, ETL and final publishing the data to web page
- We have used Azure cycle cloud for high performance and orchestration and monitoring data
- Technology Used: Autosys system, Apigee, MongoDB, Jenkins Python, Groovy, Docker, Azure MLOP, Azure DevOps
Team lead
Mindtree India Pvt Ltd
08.2021 - 12.2021
- This project is about real estate solution to Jumeirah group
- Here we have used AWS Cloud to run Jumeirah ecommerce website.
- Here we used EC2 instance and AWS Lambda as data streaming
- We have used Apache kafka for messaging queue
- Design CI/CD pipeline for Build deployment to Kubernetes cluster
- My roles and Responsibility: as a data engineer I have worked on data pipeline , which include data to be rolling from AWS s3 Bucket to get populated in DB.
- Created all the important build pipeline and and data pipeline
- Build deploy and approving of release with proper criteria assessment
- Modification of release pipeline if in case any new task is been added up to release
- Go live support to all release
- Mentor to junior Production support SRE team and Production support troubleshooting
- Technology Used: Aws DevOps, EKS server/Kubernetes, Pyspark, CI/CD pipeline
Team lead
Mindtree India Pvt Ltd
05.2021 - 08.2021
- Project name: Digital marketing for Bekaert
- Here Azure DevOps is used to maintain code checking, version control and other functional pipeline
- We were having 4 subscriptions to Azure, my work was to handle all these subscriptions, cost analysis.
- All the release pipeline were present in Azure DevOps.
- We have used Enterprise Terraform in this project for basic infrastructure coding, later all these were maintained in Azure DevOps git version control.
- Here we have used Kubernetes of Azure services AKS, for orchestration
- My roles and responsibility: I have created pipeline to build the code and release them to production, Maintained AKS server, debug if in case of error.
- Created branching strategy for team
- Participate in the code approval process for higher branches and be a crucial individual for the git version of the code.
- Go live support and release work.
- Mentoring to Junior SRE.
- Technology Used: Azure DevOps, git version control, SonarQube Kubernetes
System Analyst Level III
Smarsh India Pvt. Ltd
10.2016 - 04.2021
- Alcatraz Storage Compliance
- Roles and responsibility in Project
- This project is about, compliance audit solution to Big data, I have contributed, in building test suite for functional and smoke test. Configuring YAML test plan to deploy specific functionality in Azure environment
- This deployment had many spring boot apps to be deployed in decomposed way, then storm deployment and finally topology bring up for various operation
- To have deployment build ready to promote we had functional test .it was basically an API test
- One the test passes this build pass the CI and one YAML plan can deploy the build
- After build deployment, we run smoke test to ensure deployment success
- This project has used Big data technology Kafka as producer and storm as consumer to data
- Part of data(metadata) stores in Mongo DB and some metadata in Elasticsearch (for faster data access) and entire data with attachment stored in Azure storage
- We have used Azure Blog storage for compliance and storage
- My roles and Responsibilities: Maintain the Build and deploy pipeline and change if its needed or in project demand.
- Go live support to All release. Sometime this support also requires quick Python scripting solution to quick MongoDB
- Help and Mentor our Junior SRE Team member in Debugging. Providing smart solution
- Technology Used: Azure blog storage, git version control, Docker, Kafka, Elasticsearch, strom topologies, zookeeper
Software Test Engineer
SANDISK DEVICE DESIGN CENTER, BRISA TECHNOLOGY
11.2014 - 07.2016
- Emerging Marketing Solutions (EMS)
- Working in SSD software solution called Strom it's a software developed in open-source technology ceph. Ceph is further developed to adapt all changes according to SSD box called IGLOO
- To make the system faster in process, we have replaced XFS file system used in software with FDF that is Flash Data Fabric
- My work is to test basic functionality of file system
- I was involved in Testing the patch for ceph open-source software and the functionally that is been added to it for SSD support.
- Go live support in all the release
- Technology Used: Ceph, SSD Storage
Software Test Engineer
HEWLETT PACKARD
11.2012 - 10.2014
- DVG SOLUTION, Chennai
- Working on maintains of Perl script that used for test automation of Service guard system
- Also have design entire test suit for XDC system testing in shell scripting
- Service guard system is used for creating highly availability cluster of HP server system
- My work is to monitor regular Tron automation test run on entire source code of Service guard and design new Perl script to test new feature enable/ disable on service guard
- Raise fault reports when appropriate and Setup test lab and test data
- Maintaining dedicated cluster and storage related to it
- Go live support in all the release
- Technology Used: Automation suite in Shell script and HP storage
Education
Bachelor of Engineering -
BIT Durg
Skills
- Terraform
- Azure DevOps
- Azure data Migration
- Python
- Pyspark
- Data analytics
- Azure Data Factory
- Azure Data bricks
- Elasticsearch
- Linux
- Amazon EC2
- Amazon S3
- No SQL Database
- Big Data
- MongoDB
- Agile Software Development
- JIRA
- Microservices
- Kubernetes
- Docker
- Team building and Management
- Data Lakes
- Production Support
- Cloudbee Enterprise Jenkins CI/CD pipeline
- Groovy
- Infrastructure (Terraform)
- System Monitoring tools (Graffana)
- Version Control (git)
- Azure Storage
- Landing zones configuration
- Securities and compliance
- Troubleshooting
- Deployment and Release activity
- Site reliability
- Databricks
Certification
Udemy Kubernetes Admin course
Awards
Spot award from Smarsh, Spot award from Mindtree, Team A Award from Mindtree
Personal Information
Timeline
Senior Software Engineer Manager
MORGAN STANLEY ADVANTAGE DEPARTMENT
10.2022 - 07.2023
Team lead
Mindtree India Pvt Ltd
12.2021 - 09.2022
Team lead
Mindtree India Pvt Ltd
08.2021 - 12.2021
Team lead
Mindtree India Pvt Ltd
05.2021 - 08.2021
System Analyst Level III
Smarsh India Pvt. Ltd
10.2016 - 04.2021
Software Test Engineer
SANDISK DEVICE DESIGN CENTER, BRISA TECHNOLOGY
11.2014 - 07.2016
Software Test Engineer
HEWLETT PACKARD
11.2012 - 10.2014
Bachelor of Engineering -
BIT Durg
Similar Profiles
Vinod GVinod G
Vice President at Morgan Stanley Advantage Services Ltd.Vice President at Morgan Stanley Advantage Services Ltd.
Aditya KarkiAditya Karki
Marketing Operations Specialist at Morgan Stanley Advantage Services Private Limited (on Payroll Of Russell Tobin Associates)Marketing Operations Specialist at Morgan Stanley Advantage Services Private Limited (on Payroll Of Russell Tobin Associates)
Karan PanikalKaran Panikal
Associate (5.5 Years) at Morgan Stanley Advantage ServicesAssociate (5.5 Years) at Morgan Stanley Advantage Services
SHARMILA SSHARMILA S
IT Audit Consultant at SeimensIT Audit Consultant at Seimens
Ritik Kumar SinghRitik Kumar Singh
Intern at Airline Management SystemIntern at Airline Management System