SHUBHAM BHATIA

Currently working with a leading UK Bank to create various risk based KPI , providing the data engineering solution for model development in the field of credit risk , worked on both secured and unsecured portfolios & different kind of data i.e cards , loans , created final data by collaborating different variables from various data sources.

Tech Stack : SAS , Hive , Pyspark , SQL , Excel

• Domain: Banking)
• Working with one of the major Banking client, responsible for understanding of requirements , analysis and then provide them with the technical solutions for Market Risk , Operational Risk , Reporting
• Worked on various kind of data in banking i.e Market Risk for both domestic & forex across different categories i.e Portfolio , Equity , securities
• Creation of Jobs , Reports and design document to transfer data from SAS to PySpark ,Scala,Hbase and applied various options for parallelism and use of analysis for faster extraction of data & reporting purpose
• Working on cloud migration along with data modelers & data architect , mainly using snowflake for dimensions/ facts for EDW , using AWS GLUE for transformations & Redshift for OLAP and EMR for Data lake as a service

• The primary objective of this project is to integrate Hadoop (Big Data) with the Relationship Care Application to leverage the raw/processed data that the big data platform owns
• It will provide an enriched customer experience by delivering customer insights, profile information and customer journey
• This will allow us to prioritize conversations to drive value generation and a 360 degree view of an account member
• Role and Responsibilities :
• Responsible for building scalable distributed data solutions using Hadoop (Big Data)
• Developed Spark scripts by using PySpark shell commands as per the requirement
• Used Spark API over Cloudera Hadoop YARN to perform analytics on data in Hive
• Experienced in performance tuning of Spark Applications for setting right Batch Interval time, correct level of Parallelism and memory tuning
• Optimizing of existing algorithms in Hadoop using Spark Context, Spark-SQL, Data Frames and Pair RDD's
• Worked on a POC to compare processing time of Impala with Apache Hive for Analyzed the SQL scripts and designed the solution to implement using Pyspark
• Responsible for developing data pipeline with Amazon AWS to extract the data from weblogs and store in HDFS
• Involved in creating Hive tables, and loading and analyzing data using hive queries
• Implemented Partitioning, Dynamic Partitions, Buckets in HIVE
• Extraction of data from source in csv text format & finally loading it into hdfs
• Have created Job workflows & done setup as per requirements
• Project Name: Hadoop Multinodecluster & DATA Ingestion & Monitoring to HDFS, Involved in Clustering of machines through Hadoop fully distributed mode
• Use of PigLatin & Hive0.8.0 to simplified MapReduce Task
• Administration, Managing and Monitoring 20 node each two Hadoop clusters, cluster tune, settings and cluster maintenance
• Developing parser and loader map reduce application to store and retrieve data from HDFS and store to Hbase and Installed & Configured Hadoop for storing and retrieving data
• The individual nodes (computer systems) were first configured for single-node setup of Hadoop
• It involved setting up of environment variables, paths etc
• Single node configuration is then succeeded by the multi-node configuration
• One of the nodes is assigned to be the master while the others are the slaves
• Checked the health of HDFS, the block report on Browser, block health of HDFS
• Map/Reducing programming for ETL for new application on the top Hadoop platform
• A US Major retail chain with more than 400 stores, Developed a solution & created different tables i.e
• Dimensions (stores, customers , products etc) & facts (transaction etc) , Created different segmentations as per market demand i.e Baby segmentation (Targeting Customers who majorly buy Baby food) , Loyalty segmentation (It identify loyal customers of different category)
• Technology Used: SAS-EG,Spark, Python,Hbase ,AWS S3 , Step functions , EMR
• A data management project undertaken to move all the Retail & Customer data from SAS to Hadoop
• Project required team of four to perform data mapping, gap and impact analysis of customer data on SAS and Hadoop platform to confirm successful data migration
• Compared the data with the SAS
• Technology Used: Hadoop, Python , SQL ,Spark , Cassandra , OOZIE , AWS LAMBDA , EC

• Worked closely with the Data manger & onshore team by handling there dashboards which covers MIS needs to provide business insights for analytics based solutions.

• Digital :- Part of, Media Team, Marketing ROI (Aligning Reach and MROI Benchmarks for Media Planning) : The relationship between sales & investment, results in MROI metrics
• Aligning Reach and other KPIs for Media planning for cross seasonal campaigns for predictive analytics and decision making.