Kaustav Kumar Das

Project : Smith & Nephew Data Analytics and Enterprise Architecture

Design and build data ingestion pipelines, orchestration, conversion of full Loads to Incremental, custom warehousing solutions - Savings of 80% Snowflake cost & execution.

• Experience in dbt, AWS S3, AWS Glue, AWS Lambda, AWS Step Functions, SNP Glue, and Snowflake, and overall release management process using Git CICD and implementing DevOps strategies.
• Develop, maintain, and oversee 300+ automated and scalable ELT data pipelines (dbt,AWS S3,AWS Glue,AWS Lambda,AWS Step Function,SNP Glue and Snowflake)
• Source, process, validate, transform, aggregate, analyze and distributed data from 10+ sources (Oracle, SQL Server, Snowflake, CSV files, JSON files, APIs)
• Develop Unified Data Platform (UDP) which would address the bouquet of data needs for the entire organization.
• Optimizing various non-performant queries, and data pipelines. Converting models to run on incremental base in place of full loads.
• Documenting 30+ data sources, data structures, data flows, and data infrastructure
• Create Python scripts to implement GitLab automation, Snowflake Scripts deployment.
• Design the orchestration framework for ingesting data from varied sources.
• Creation and maintenance of AWS Step functions which helps in the orchestration processes.
• Help in standardizing release management process using Git CICD, Confluence, Jira and DevOps practices.
• Helping design test cases and scripts for Snowflake to do data reconciliation and auditing.
• Working in an Agile environment, using Jira to track the user stories on a day-to-day basis.
• Assisted to select tools of choice for creation of data pipelines based on needs.
• Ensured data quality through rigorous testing, validation, and monitoring of all data assets, minimizing inaccuracies and inconsistencies.
• Reengineered existing ELT workflows to improve performance by identifying bottlenecks.

Project : USAA CFO Treasury Team

Data engineer building/designing data pipeline to feed the information required for Liquidity team reports. Data pipelines are designed using IBM Datastage, dbt and Snowflake.

• Lead 5 member team to develop, maintain, and oversee 20+ automated and scalable ETL/ELT data pipelines (IBM Datastage, Snowflake-dbt)
• Source, process, validate, transform, aggregate, analyze and distributed data from 10+ sources (Oracle, Snowflake, CSV files, JSON files, APIs)
• Tableau dashboards for CFO Treasury to report details like Available Collateral Report, Bank Product Reporting etc.
• Migration from Datastage 11.5 to 11.7.
• Modify Python scripts to implement Cyberark credential management
• Modify Unix shell scripts for supporting the data pipelines
• Creation and maintenance of Control-M jobs
• Maintaining codebases in Git and promoting codebases to higher environment using Urban Code Deploy
• Creation of test cases using Qtest and create code review sessions in SmartBear
• Jira to track the user stories on a day-to-day basis
• Multinational technology service provider - www.ibm.com
• Optimized data processing by implementing efficient ETL pipelines and streamlining database design.
• Collaborated on ETL (Extract, Transform, Load) tasks, maintaining data integrity and verifying pipeline stability.

Project : PepsiCo LATAM & Mattel (USA) Development Team

ETL development executive for IBM DataStage (8.5) processes (SAP system integrations with Legacy and ETL applications for loading Data Warehouses) and Informatica PowerCenter (9.1,10.2) processes (SAP system integrations with Legacy and ETL applications for loading Data Warehouses).

• Experience in working in Agile Environment (PSM1 Certified)
• Monitor system integrity through daily system checks, to ensure a high standard of service is provided to clients.
• Provide Root Cause Analysis for issues related to jobs and business logic validations.
• Coordinate with vendors and customers to troubleshoot and resolve issues.
• Support the Client Reporting Production Environment as a first priority.
• Resolve on-going maintenance issues and fixing existing bugs; monitoring Informatica sessions, IBM DataStage sessions as well as performance tuning of mappings and sessions.
• Using Automation Scheduling tools like Control-M and Windows Task Scheduler.
• Checking file transfer activities via FTP and SFTP (SSH key).
• Documenting the resolution for known issues and responsible for updating and maintaining the articles in the Knowledge Base.
• Managing the incident teams so that the information is provided to the vendors and customers in Mexico, Brazil.
• Designed DataStage server and parallel jobs.
• Managed cross-functional teams for successful project completion within budgetary constraints and deadlines.
• Lead Offshore development team and working in an offshore - onshore model.
• RPA tool Athus to create automated jobs for alert management with Oracle SQL query integration.
• Microsoft Flow (Power Automate) jobs for replacing iterative operations with automation (Alert Management, Monitoring).
• Microsoft Power BI, IBM Cognos.
• Tune DataStage jobs, SQL queries within DataStage as required for DataStage (Level 3)/Informatica (Level 2) environment.
• Migration of Informatica Jobs from version 9.1 to 10.2.
• Windows batch commands and UNIX commands.
• Multiple process improvements and enhancement solutions to customer.
• Service Now Incident Management Tool.
• Developer for multiple ETL/ESB tools.
• Development of Informatica PowerCenter. TIBCO BW codes, codes in Globalscape EFT Server Enterprise 7.3, & data profiling in IBM Infosphere Information Server 11.5, data profiling with IBM IIS.
• Parsed high-level design specification to simple ETL(Informatica) coding and mapping standards.
• Involved in building the ETL architecture and Source to Target mapping to load data into Data warehouse.
• Experienced in using Automation Scheduling tool CA Workload for scheduling Informatica Jobs.
• Developed Transformation Logic to cleanse the source data of inconsistencies before loading the data into staging area.
• Participated in the full development life cycle within the reporting framework, from developing business intelligence requirements to QA and production deployment.
• Develop processes for extracting, cleansing, transforming, integrating, and loading data into data warehouse database and enforcing data Integrity and business rules.
• Experienced in working with JMS Queues, As400/DB2 database, Oracle Database.
• Responsible for designing, deployment, testing of TIBCO BW codes. Also included monitoring the jobs via TIBCO Administrator.
• Created data extracts using complex Oracle SQL queries.
• Responsible for documentation of technical specifications and design documents.
• Responsible for building Globalscape EFT codes to transfer files from one server to another based upon complex logics provided by customers.