Experienced Data Engineer with 6+ years of experience in all aspects of software development(Data Engineering) life cycle, including requirement gathering,implementation,Testing,Maintenance and Development.
•Worked on PNR Module of Air-france Project in the process of migration to GCP environment and loading the data into Big-Query with the tech stack being used as python in cloud function, airflow in cloud composer and with pub-sub to send the data as an event with metadata information to big-query.
• Testing the solution to analyze the processing speed of pub-sub in loading the data to big-query.
• Development of scripts based on python for cloud run,dataflow,cloud composer and related services.
• Setting up the GCP environment with the necessary IAM permissions to access specific functionalities in GCP
• Resolution of issues related to python and fixing the issues to process the data.
Data-proc,Cloud Function,Pub/sub,Cloud Composer,
Big-Query,Data-flow,Big-Table,Cloud Storage,IAM