Pradnya Koli
mumbai
Summary
As a Cloud Data Engineer Enthusiast, Creative Integrator, and Automation expert, I have experience in automating various tasks in the Cloud environment, including migration, load balancing, and critical deployments. As a result-oriented person, I always strive to achieve the desired outcome of a project or task. I possess good problem-solving skills, which enable me to identify and resolve issues quickly and effectively .
Overview
3
3
year of professional experience
3
3
Certification
Work History
DATA ENGINEER
SEARCE COSOURCING PVT
PUNE
08.2022 - Current
- I work in the Center of Excellence (COE) team at Searce as a, I have added value to the company by building IPs and accelerators that can migrate any type of data to the cloud ecosystem
- I developed the data pipeline from scratch, which integrates various data sources into the Google Cloud Storage (GCS) ecosystem
- Additionally, I have developed code for schema evolution and real-time streaming I explore and leverage open-source technologies such as Apache Hive, Spark, Kafka, and Airflow when building accelerators
- Until now, I have gained hands-on experience with various managed cloud services such as BigQuery, Dataproc, Dataplex, Cloud SQL, Pub/Sub, and Dataflow on the Google Cloud Platform As well as AWS Redshift, S3 buckets, and Lambda on Amazon Web Services
- I have hands-on experience writing DAG scripts on Airflow Operators using Python.
- I Designed and implemented Real Time streaming pipeline on pubsub(GCP).
PROGRAMMER ANALYST TRAINEE
COGNIZANT SOLUTION PVT
01.2022 - 07.2022
- I have an intermediate knowledge of cloud infrastructure migration in Microsoft Azure
- I am proficient in configuring availability sets, virtual machine sets with load balancers, virtual networks, virtual security groups (NSGs)
- I am familiar with the configuration of Azure Virtual
- Networks, Subnets, DHCP, DNS, Network Security
- Groups, Load Balancer, Application gateway, and Traffic
- Manager, as well as provisioning, managing, and monitoring storage accounts
- I am also familiar with Git commands and GitHub repositories
- Additionally, I have knowledge in writing
- SQL queries and am comfortable with joins and subqueries.
Education
Bachelor of Engineering - Electronics
K.J Somaiya college of Engineering
2021
Diploma - Higher Education Electronics
Premlila Vithaldas Polytechnic
2018
Skills
- Programming:-
- Python, Spark, Scala
- Linux Scripting :- Bash shell scripting
- Database Expertise:- SQL, Postgres,
- Teradata, MySQL
- Big Data Framework Query Engine:-
- Hive, Spark
- Scheduler Tool: Apache Airflow
- Cloud Expertise:- Google Cloud, AWS
- Data warehouse Expertise:- Snowflake,
- Redshift, Bigquery
Additional Information
1- Implemented pipeline on Bank Project for Real Time Stock data:
Tech:-Python,Google cloud Pubsub, Google cloud Bigquery, Google cloud Composer, Dataproc Pyspark.
- Got the Daily real time stock data from API and used to fetch the data from using pubsub with python language.
- Implemented the schema evolution for any changes in data like column drop or add will reflect to bigquery.
- Scheduled pipeline using Airflow for continuous training
- Used Dataproc to submit all the dag process to Pyspark job.
2 - Generic Schema Conversion From x sources to Bigquery
Tech:-Python,Bigquery,Google cloud storage,Compute engine
- Used the Google client library to build this approach where the schema from oltp and olap database schema can be converted to bigquery with the minimal efforts
- the approach is suitable for hundreds and thousand of table schema can be migrated with multi-threading approach.
3 - Build the solution for data migration from AWS RDS Postgres to Bigquery.
Tech:-Python, AWS RDS Postgres, Google cloud storage, Bigquery
- The solution is implemented for client to migrate the data from AWS to bigquery.
- Implemented the historic load,Incremental load and schema evolution pipeline.
4 - Build The pipeline for Realtime update of RSS feeds
Tech:-Python,Cloud Function, Send Grid, Bigquery,Cloud Composer
- Client wants to build the pipeline to get the realtime update of GCP Services Health(for eg which service is down,Region,Zone)
- Build the End to End pipeline to get the RSS feed update and trigger the mail if any services is down, coding is used as python
- To trigger the code cloud Function.
- Composer is used to keep running the code trigger the cloud function if there is any update.
Certification
- Google Cloud Certified Professional Cloud Database Engineer
- Google Cloud Certified Professional Cloud Data Engineer
Timeline
DATA ENGINEER
SEARCE COSOURCING PVT
08.2022 - Current
PROGRAMMER ANALYST TRAINEE
COGNIZANT SOLUTION PVT
01.2022 - 07.2022
Bachelor of Engineering - Electronics
K.J Somaiya college of Engineering
Diploma - Higher Education Electronics
Premlila Vithaldas Polytechnic
Similar Profiles
Simmy SahiSimmy Sahi
Account Executive at Searce Cosourcing Pvt LtdAccount Executive at Searce Cosourcing Pvt Ltd
Nirav SavaniNirav Savani
Associate Cloud Architect at Searce Cosourcing Services PVT LTDAssociate Cloud Architect at Searce Cosourcing Services PVT LTD
Nirali TrivediNirali Trivedi
Client Success Partner at Searce Cosourcing Services (P) LtdClient Success Partner at Searce Cosourcing Services (P) Ltd
Neel YagnikNeel Yagnik
Client Referential Officer (Associate Level 1) at BNP Paribas ISPLClient Referential Officer (Associate Level 1) at BNP Paribas ISPL
Aneri NanduAneri Nandu
Media Insight Analyst. at OnclusiveMedia Insight Analyst. at Onclusive