Siddhant Srivastava
Data Engineer
Bangalore ,KA
Summary
Dynamic Data Engineer with around 2.1 years at Tata Consultancy Services Limited, specializing in real-time data solutions and advanced analytics. Proven expertise in PySpark and SQL, coupled with strong teamwork and communication skills. Successfully optimized data processes, ensuring business continuity and enhancing data security through innovative techniques.
Overview
2
2
years of professional experience
3
3
Languages
Work History
Data Engineer
Tata Consultancy Services Limited
06.2023 - Current
- Have worked on real time data scenarios like incremental data by setting watermark columns and applying lookup activity by setting a particular date value by defining a new sink with that date folder and archiving hist data in archive folder.
- Worked on SCD( slowing changing dimensions for keeping the track of minor changes.
- Have handled security features like salting by adding a salt key with hash value ensuring the sensitive data become impossible to be hacked and adding features azure security vault so that some crucial information like passwords do not get hardcoded
- Worked on delta repository by enabling merge schema to handle versatility in updation and defining versioning of data so that previous data can be utilised according to buisness needs and analytical requirements.
- Have worked on real time scenarios like partitioning, bucketing, using of broadcast joins and caching features( for storing intermediate results which can be used multiple times) for optimization purposes.
- Know to handle failures by documenting all previous logs with a solution and adding an notification object alarm very sensitive job failures that are very crucial in real time scenarios to avoid escalations and meet SLA.
- I have adhered and thoroughly gained the insight of the my real time responsibilities and role as a crucial resource by developing a sense of understanding of smooth communication and importance of buisness continuity management.
- As being a part of a Life Cycle and health care buisness unit for around 2.1 years I have gained an excellency and expertise in coordination with the team and members that are involved in buisness, know to prioritise the issues and escalations as per their requirement and needs, and how a message that has to be communicated between more number of communicating components has to cautiously communicated as it more prone to error which can effect the deliverables.
- Have worked on pyspark, Hadoop and azure data factory and knows the complex query handling in sql.
- Handles various classes like spark context and spark session in pyspark and to create a spark job, other various libraries like pandas for specific tasks.
Education
Bachelors Of Engineering - Electronics And Communications Engineering
Don Bosco Institute of Technology
Bangalore 07.2022
Skills
REAL TIME STREAMING
undefinedTimeline
Data Engineer
Tata Consultancy Services Limited
06.2023 - Current
Bachelors Of Engineering - Electronics And Communications Engineering
Don Bosco Institute of Technology
Similar Profiles
Vrunda ChindhadeVrunda Chindhade
HR Manager at Tata Consultancy Services, TCS, Tata Consultancy Services LimitedHR Manager at Tata Consultancy Services, TCS, Tata Consultancy Services Limited
Arun PandeyArun Pandey
Head of Digital Innovation, Manufacturing Business at Tata Consultancy Services LimitedHead of Digital Innovation, Manufacturing Business at Tata Consultancy Services Limited
Bhuvana RamamurthyBhuvana Ramamurthy
Project Manager and Scrum Manager(AUS Airline) at Tata Consultancy Services LimitedProject Manager and Scrum Manager(AUS Airline) at Tata Consultancy Services Limited
Aparajita GhoshAparajita Ghosh
SENIOR DATA & BI ANALYST at Tata Consultancy Services LimitedSENIOR DATA & BI ANALYST at Tata Consultancy Services Limited
Madhan A KMadhan A K
Product Security Expert / Security Researcher at Intel CorporationProduct Security Expert / Security Researcher at Intel Corporation