Rahul Parmar

• Data warehouse migration of Teradata to GCP
• Utilized Apache Airflow to create and manage DAGs for efficient data workflow management.
• Optimized bigquery tables by using clustering & partitioning.
• Built end-to-end data ingestion pipelines using Dataproc and Airflow for job scheduling and
  orchestration.
• Converted Teradata load utility into BigQuery SQL.
• Validated table data using the Pelican tool.
• Expertise in GCP and BigQuery for processing and analyzing large datasets.
• Complete Ownership of Developing Batch and Streaming Data Ingestion pipeline using
  Airflow-Dag,Dataflow, python and shell scripts.And given Handover and support to the team.
• Root cause analysis of failures performed workaround and permanent fixes as well.

Responsibilities: -

• Working on Data Quality.
• Configured data mapping using MapForce by Altova
• RCA Analysis and familiar with the incident, tickets, and changes handling, creation on ServiceNow.
• Hands-on experience on Apache Airflow to manage and monitor DAGs.
• Monitoring and troubleshooting.
• Root cause analysis of failures performed workaround and permanent fixes as well.
• Monitoring & Optimization Migration scripts and Queries for improving performance and timelines.
• IWS Scheduling for Monitoring the Teradata and Big Query Jobs.

Responsibilities: -

• Worked as an engineer with varied knowledge in data migration, cloud technology, basic SQL, BigQuery, data ingestion, and Cloud Composer, Python, etc.
• Data ingestion through the cloud shell and scripts.
• Hands on Experience on Apache Airflow to create and manage & monitor DAGs.
• Created an alert using python Script in GCP cloud Function
• Familiar with Tidal scheduler.