Priyanshu Sharma
Sr Data Engineer
Banalore,Karnataka
Summary
To seek and maintain a full-time position that offers professional challenges utilizing interpersonal skills, excellent time management and problem-solving skills.
Overview
3
3
Certifications
4
4
years of post-secondary education
7
7
years of professional experience
Work History
Senior Data Engineer- Mesh Contra
Societe Generale
Bangalore, Karnataka
04.2019 - Current
- Designed and developed continuous data pipelines using Spark-Scala performing data wrangling, cleansing & transformations.
- Re-Designed MCG architecture, enabling it to feed multiple input files, unlike the old design. This change decreased the batch processing latency by 400%.
- Worked with terabytes of semi-structured and structured data.
- Used SOLID principles while code refraction and development.
- Developed ETL platforms using Ansible, Oozie, Jenkins, Git.
- Designed RDBMS focused process tracker cum workflow launcher for monthly jobs.
- Analyzed complex data and identified anomalies, bugs and risks to provide useful insights.
- Wrote Ansible scripts to ease/automate day-to-day developer activities
Big Data Engineer- Experian Sandbox
Tavant
Bangalore, Karnataka
12.2016 - 03.2019
- Implemented Iterative Broadcast to improve the performance of CCA job by 20%.
- Developed Fixed-Width parser for the files having no delimiter.
- Performance tuned spark/hive jobs which increased the performance by 50%.
- Built CI-CD enabled spark jobs for handling complex transformations.
- Migrated jobs from Cloudera to AWS resulting in annual cost saving of $60000
- Developed automation tasks and workflow using AWS Lamda and step function.
- Addressed ad hoc analytics requests and facilitated data acquisitions to support internal projects investigations.
Hadoop Developer- Citi QLM
L&T Infotech
Bangalore, Karnataka
12.2013 - 11.2016
- Cleaned server logs using hive & pig.
- Decreased Jobs latency by 30%, by converting pig scripts to hive UDFs .
- Wrote Java APIs to generate 10 reports per week.
- Met with key stakeholders to discuss and understand all major aspects of project, including scope, tasks required and deadlines.
Education
BTech - Electronics And Communications Engineering
Sir M Visvesvaraya Institute of Technology
Bangalore 08.2009 - 07.2013
Skills
Programming: Scala, Python & Java
Big Data: Spark, Hive, Kafka, HBase
Scheduler: Oozie
Scripting: Shell , Python
Databases: MySql
Datawarehouse: Teradata, Redshift
AWS: EMR, S3, Lambda, Step function
DevOps: Jenkins, Ansible, Git
Certification
Databricks Spark Certified
Timeline
Senior Data Engineer- Mesh Contra
Societe Generale
04.2019 - Current
Databricks Spark Certified
05-2018
Big Data Engineer- Experian Sandbox
Tavant
12.2016 - 03.2019
Java BrainBench
06-2014
NCFM certification on financial derivatives
01-2014
Hadoop Developer- Citi QLM
L&T Infotech
12.2013 - 11.2016
BTech - Electronics And Communications Engineering
Sir M Visvesvaraya Institute of Technology
08.2009 - 07.2013