Santosh Kumar Bugude

FTPS (PTR BOS)

• Worked on the data ingestion flow to read UBM, UTM from AMPS an OG sources into PTR Kafka topic using the kafka API
• Worked on Streaming Loader job to read data from kafka and write to HDFS storage handling the duplicate check but storing the key in hbase
• Worked on DBWriter component to read the hbase metadata table and process the new batch parquet files from hdfs and write to oracle database
• Worked on idet process to read the json from oracle and process impact related data to mq
• Migrating the existing IDET spring boot application into aws eks optimizing the code by configuring the Batch size
• Provisioned aws infrastructure using the terraform
• Involved in the aws design plans for the application
• Migrating the legacy PLSQL stored proc EOD reports into spark sql using generic service.

sac-collection-azure-software-log-parser

• Spark-ETL job, read the messages from Azure Event Hubs, by reading manually maintained offset file in blob storage with that we can determine from which offset to read for all the partitions
• Decompressing the EventHub message using google proto Buffer
• Applying the data massaging and Storing the file Parquet file in the Blob Storage
• Accumulating all the row wise errors into Spark Dataset and at the end of application run pushing it Error EventHub
• Data validation for the final data frame are captured and stored as csv file for reporting purpose
• Logging the Job status in Azure Application insights using Telemetry API
• Validating the invalid collection and pushing to event Hubs
• Deploying spark-scala application on PKS
• Real time Streaming application read data from eventhubs and update the latest status of the file into elastic search DB
• On-prem data to into Azure cloud using the Spark Streaming solution deployed into on-prem Kubernetes solution
• Gathering all the weekly issues and addressing them in next sprint
• Querying the Appinsights telemetry using KQL and created the Azure Dashboards to understand the errored records and success ratios.

BNPP-Limit Grants

• Analyzing the functional specifications and providing the estimations
• Connect with Data Architects and Users for designing reports
• Ingestion of data into hive tables and extraction
• Implementing hive tables to store the flow of daily data
• Used Spark SQL to process the huge amount of structured data available in Hive Tables
• Analyzing the spark job and tune it if any performance issues are seen
• Data cleansing activities
• Experienced in implementing Spark RDD transformations, actions to implement business analysis
• Worked on Spark Data frames, Spark Data sources, Spark SQL using Scala
• Preparing SFTP and FTP scripts to import data from different source systems
• Write Custom UDF for some of the requirements
• Validating the datasets and sending for UAT signoff and moving to production.

• Involved in all phases of the SDLC from analysis, design, development, testing, implementation and maintenance with timely delivery
• Migrating the data using Sqoop from HDFS to Relational Database System and vice-versa
• Migrating Oracle PLSQL stored procedures into spark applications using data frames and datasets
• Developing spark application using both Data frames/SQL/Datasets and RDD 2.3 for Data Aggregation, queries and writing data back into OLTP system through Sqoop
• Loading data from flat files into Hive tables
• Designing the database data model for building an application to meet the business requirements, Customizing the application using PL SQL techniques and using normalizing Techniques
• Analyzing the best Access methods, join methods predicates, using hints for complex query using the execution plan
• Creating indexes on the table based on the cardinality, have created B-trees, Bit-map and unique, functional, composite, global, prefixed local non prefixed local index
• Worked on partitioning the table for the SQL performance as well as archiving the old data
• Used PL SQL Tables, Cursors to process huge volumes of data and used bulk collect for mass insert as performance improvement process
• Loading the data from external source into database using SQL loader, external tables
• Gathering table stats when bulk updates, deletes and insert happen one the table, rebuilding the indexes of the table if necessary
• Having basic knowledge on SQL trace and TKPROF.

• Creating views for the QlikView team to pull the data into QlikView qvd’s
• Developed Stored procedure, function and package for the migration of Virgin client data into Tesco system
• Developed onetime SQL and Anonymous Block scripts to handle data issues in the production environment
• Worked on data pumps (EXDP, IMPED) and SQL loader for processing of Tata sky data into Tesco system
• Worked on Linux environment for getting secured data files using sftps and stp commands got at.

• Working with Business Analyst and understanding the requirement
• Preparing Low level design document
• Defect fixing
• Production Support of the application
• Involved in Testing.

A two-month training program on Java J2EE

Understanding the fundamentals of Java syntax, variables, data types, and control structures.
Exploring the principles of object-oriented programming, including classes, objects, inheritance, polymorphism, encapsulation, and abstraction.

Introduction to HTML, CSS, and JavaScript for front-end development.
Learning the basics of servlets and JavaServer Pages (JSP) for server-side web development.
Building simple dynamic web applications using servlets and JSP.

Understanding the role of EJB in enterprise applications and learning how to develop EJB components.
Introduction to JPA for database access and management.
Building a small-scale enterprise application integrating servlets, JSP, EJB, and JPA.