Naveen Gade

• Building efficient ETL pipelines using dagster as orchestrator and dbt.
• Building efficient dbt models for the table transformations. It helped in reducing the credits in snowflake
• Extracting and loading data from flat files into snowflake and creating dashboards
• Monitoring and fixing the pipeline issues

Data Warehouse automation:

• Automated DDL and schema creation for on-premise environment over 600+ DDL's can be deployed with GIT LAB CI/CD with single deployment
• Automated open source MINIO object storage buckets creation, deletion and files migration into it.
• Worked on Trino query engine with both Hive and Iceberg connectors to get better outcome or optimization of queries and performance.

Metabase migration:

• Migrated metabase report collections, cards and nested dashboards from one environment to another environment such as pre-prod or staging to production vice versa using metabase API's

Snowflake Automation:

• Automated regular snowflake tasks or operations such as user creations, Access permission, Role grants, Database creation ..etc with single deployment using open source pulumi frame work

Access and Data masking policies for Trino:

• Discovered and Implemented access and data-masking policies for all tables using apache ranger for trino which made environment more stable and secure.

Airflow DAG deployment:

• DAG's deployed as per business requirement to gather data for reports
• Maintaining healthy data pipeline line to run business as usual

Achievement:

• 800+ DDL's and schemas got created with single click deployment. Easy to maintain code base
• Metabase migration saves 100+ human hours while migrating reports between different environment
• Most of snowflake operational tasks automated with single configuration file
• Enhanced security is implemented on all on-premise environment with apache ranger

• As Backend Python Automation Engineer supported in automating COE operational tasks to reduce Human efforts. As team developed common platform and integrated all developments into it to make use of all operations. Developed APIs to extract data and generated auto triggered emails and tickets to alert teams.
• Expertise in working closely with business and IT stakeholders to define problems and design large-scale, enterprise class solutions
• Implemented processes and systems to monitor data quality, ensuring production data is always accurate and available for key stakeholders and business processes that depend on it
• Efficient in gathering and understanding requirements of clients & other multiple stakeholders, followed by Automation driven solutions.
• Developed APIs using Flask to extract data and for auto creation of monitoring tickets and auto generated reports
• Developed template-based engine to automate all regular operations tasks performed by COE owners. Such as Linux team (Administration tasks), VM ware team(snap shot creation, deletion etc) and DBA team (MS SQL , Oracle, Postgres)
• Automated security agents on large number of servers to overcome security vulnerabilities

Achievement:

• Achieved significant saving of over 4,000+ man hours by building template- based engine

• Learnt about Network related concepts, Windows Administration, MS Azure Fundamentals and Virtualisation Concepts
• Received placement offer from company for qualifying specific assessment conducted after training