Vivek Kumar

• Re-architecting existing data product and building new data product at current engagement.
• Focused on enriching and building design documents and DBT models in day to day activities.
• Utilized Snowflake for focused data analysis on user queries.
• Leveraged Looker's PDT feature to optimize data persistence, resulting in faster dashboard loading.
• I designed a dbt model for loading and processing JSON data within Snowflake.

• Architected and constructed a data platform/pipeline on AWS that enhances decision-making capabilities, resulting in improved customer experience.
• Facilitated stakeholder communication to ensure accurate representation of technical requirements in the specification.
• Architect high volume ETL pipelines on AWS, employing Kafka/ Kinesis, Python, Pandas, AWS Lambda, S3, and Snowflake.
• Obtained approvals from stakeholders, security team, and vendors for end-to-end implementation of data pipeline architecture utilizing AWS CloudFormation.
• Implemented overall data platform onboarding stack, dramatically reducing new team or data source onboarding
• Implemented data pipelines for various data sources including external vendor data, real-time streaming from Kafka and Kinesis, and SFTP file transfers to the data platform
• I architected and implemented a solution to ingest data from JSON files in S3 into Snowflake, transforming it for BI reporting purposes.
• Implemented AWS API gateway for seamless integration with the data platform.
• Designed and integrated GenAI into business operations, providing stakeholders with a convenient chat-bot showcasing real-time data visualization.

• Developed Airflow dags to orchestrate and manage data pipelines.
• Developed a real-time DAG to monitor ongoing DMS tasks, taking immediate action in case of data loading errors in the final layer by halting the process and alerting the team.
• Created a DAG to monitor data loading from Snowflake to Postgres using an efficient and reliable method.
• Developed efficient data transfer processes using DAGs between Snowflake and S3 for seamless integration.
• Created logs to capture real-time sensor data for process analysis.
• Developed utility file to efficiently perform batch insert, update, and delete operations from Postgres to DynamoDB.

As a part of DNA unit I was working for NIKE to develop and maintain the data factory which contains dimensional data product, purchase order, sales order and wholesale data As a part of team my roles and responsibilities are below:-

• To develop pyspark script to perform ETL operations.
• To develop Airflow dag for data orchestration and pipeline.
• To track down the missing data from the raw, cleansed and curated layer.
• To monitor the airflow DAGs.