G Shashi Kumar Reddy

Client: Dun & Bradstreet.

The IBM team is responsible for data services and technical solutions for clients.

Contribution

Replatforming from legacy systems to GCP.

• Extracted data from legacy systems to the modern platform, GCP buckets, maintaining complete accuracy, and preventing any data loss during the data movement.
• Designed DDL structures and built ingestion pipelines aligned with source systems; loaded data into tables, and conducted audits and variance checks to validate data accuracy.
• Developed fabrication queries (PostgreSQL) to implement complex business logic from the existing system, replace outdated logic, and enhance the code wherever applicable.
• Provided final data results to the customer from GCS in the specified format, ensuring proper encoding, and validating all transfer-related requirements for project/account-specific requirements.

Migration to Databricks.

• Generated artifacts to ingest input data from various sources, and supply them as inputs for the required data products.
• Work with the STP team. Configured STP IDs to orchestrate secure data movement from mainframe environments to distributed servers, enabling multiplatform file accessibility for subsequent workflows.
• Developed reusable Databricks notebooks to ingest raw data, implement complex transformation logic using Python, PySpark/Spark SQL, and SQL, and write processed outputs to DBFS for downstream consumption and customer delivery.
• Maintained a code repository to ensure all the changes are captured for auditing purposes.

Delivery fullfilment.

• Collaborate with business stakeholders, the sales team, to gather requirements, validate data sources, and build data models, and translate business requirements.
• Optimize the existing mainframe-based SAS codes for better performance, and adhere to best coding standards.
• Perform peer reviews, revise the checklist, and formally document all process changes.
• Proficient in writing complex SQL queries to retrieve data from large-scale data sources for ad hoc and scheduled deliverables.
• Identify the feasibilities and challenges of migrating the process to a new platform for faster and cost-effective delivery fulfillment.
• Collaborated with cross-functional teams to define requirements and develop end-to-end solutions for complex data engineering projects.