SATYANAND GUPTA
Noida,India
Summary
Senior Data Engineer with extensive experience in optimising data pipelines and migrating datasets across cloud platforms, including Snowflake and Databricks. Proven track record in enhancing data processing efficiency by 30% through innovative use of Apache Airflow and Python. Skilled in big data technologies such as Hadoop, Spark, and Hive, with a strong background in Java development for financial applications. Adept at leveraging cloud storage solutions like AWS S3 to streamline operations. Career goals include advancing expertise in scalable data solutions and contributing to cutting-edge projects that drive business success.
Overview
11
11
years of professional experience
1
1
Certification
Work History
Senior Data Engineer
Capgemini
Noida
12.2024 - Current
- Led the migration of ETL datasets from Snowflake to Databricks, utilizing Apache Airflow DAGs written in Python to orchestrate scalable and efficient data pipelines.
- Transitioned data from Snowflake’s standard format to Snowflake-managed Iceberg format, enabling Databricks ETL jobs to directly read from Snowflake without requiring intermediate curator copies.
- This optimization eliminated redundant data transfers, reduced storage, and processing overhead by 30%, and improved data pipeline performance and reliability.
- Role: Data Engineer
- Tech Stack: Python, SQL, Snowflake, Databricks, Apache Airflow, Git, Sigma (reporting), AWS S3 (data storage).
Senior Data Engineer
Capgemini
12.2022 - 12.2024
- Working in production engineering team along with developers to process and analyze big datasets (.avro and .parquet files) using Pig Scripts, UDFs, MapReduce, PySpark, Java and Shell Script jobs scheduled through Oozie job scheduler.
- The datasets are ingested onto HDFS clusters from several ads and media applications owned by the client. These datasets are click events which contribute to revenue generation through ads and media data.
- Ads and Media
- Tech Stack: MapReduce, PySpark, Pig Scripts, UDFs, Hive, HDFS commands, shell scripting, Java, Oozie workflow jobs, Kafka, and Storm real-time streaming engines.
IT Analyst
TATA Consultancy Services
Gurugram
11.2018 - 08.2022
- A financial organization using the NetReveal platform can send real-time messages containing details of customers and financial transactions which can be checked against lists in the Watch List Manager or detection scenarios in the Scenario Manager.
- End User Reporting (EUR) is a sub-product integrated with ‘TCS BaNCS’ which can be used by BaNCS customers to configure and generate adhoc as well as scheduled reports as per their requirements.
- Email dispatch of subscribed reports and index file creation.
- Copied subscribed reports (PDF/Text/Excel/CSV) from one server location to another and created an index file having details of each subscribed reports, which can be used further for report printing and dispatch to the subscribed users.
- Audit Functionality to track user activity on EUR Reports.
- Working on Non-Regression Functional Automation Testing using Selenium Webdriver and Selenium IDE for recording.
- Migrated database from Oracle to Hive, using Sqoop.
- Methodology: SAFe Agile
- Technologies: Java 1.7-8, Struts2, Spring, Spring Batch, WAS Server, Oracle 11g, SQOOP, Hive, etc.
- CI/CD Tool: Jenkins
Senior Software Engineer
Sopra Steria
Noida, India
08.2014 - 10.2018
- Worked on customization and maintenance of Finacle corporate e-banking applications for Co- Operative Bank, UK in Corporate banking domain.
- FCM stands for Financial Crime Mitigation under which ATO project went live, in which the user login was split into two phases. If any suspect login is detected then all payments and transactions during that session is declined and that customer is blocked for any further login.
- The Single Euro Payments Area (SEPA) is a payment-integration initiative of the European Union for simplification of bank transfers denominated in euro.
- Role: Developer
- Methodology: Waterfall
- Technologies: Java 1.6, Struts 1.2, EJB 2.1, Servlet, JSP, MQ, XML, WAS Server, Oracle 10g.
Education
B. Tech - CSE
Krishna Institute of Engineering & Technology
Ghaziabad, U.P.01.2014
Intermediate - CBSE
Cambridge School
Greater Noida, U.P.01.2010
High School - ICSE
Lucknow Public School/College Jankipuram
Lucknow, U.P.01.2008
Skills
- Data pipeline optimization
- Big data processing
- Cloud data storage
- Data migration
- Snowflake
- Databricks
- Java
- Python
- Scala
- Big Data
- Hadoop
- HDFS
- Map Reduce
- Spark
- Hive
- Sqoop
- MongoDB
- Sonar
- Fortify
- Struts
- Spring
- JDBC
- Hibernate
- Web Services
- SOAP-WSDL
- Oracle 11g
- SQL
- MySQL
- HTML5
- CSS3
- JavaScript
- JIRA
- Splunk
Certification
- SCALED AGILE – Certified SAFe 5 Practitioner, 2022-06-08
- Big Data and Hadoop Plus, EduPristine, 2018-11-12, 2018-02-25, Trainee, Studied about Big Data and Hadoop technologies such as HDFS, MapReduce, Spark, Hive, Pig, Sqoop, Oozie, Flume and Kafka. Also built 'Customer 360 Degree' project under this training.
- AppStudioz Technologies Private Limited, 2013-06-17, 2013-08-14, Intern, Full time developer, developed an Android Application named ‘LogoQuiz’ in Java using Collections API and Generics.
- Hewlett Packard Educational Services (HPES), 2012-06-05, 2012-07-20, Trainee, Went through Summer Training program conducted by HPES. The project developed was – Online Polling System, in J2EE – Struts with Hibernate.
Areas Of Interest
- Code development to implement new innovative ideas.
- Sketching and portrait making in spare times.
Disclaimer
I hereby declare that all the information furnished by me above is true and best to my knowledge. Any discrepancy found in the data provided will solely be my responsibility.
Languages
English
Proficient
C2
Hindi
Proficient
C2
Interests
- Sketching and Portrait making in leisure times
- Travelling
Timeline
Senior Data Engineer
Capgemini
12.2024 - Current
Senior Data Engineer
Capgemini
12.2022 - 12.2024
IT Analyst
TATA Consultancy Services
11.2018 - 08.2022
Senior Software Engineer
Sopra Steria
08.2014 - 10.2018
B. Tech - CSE
Krishna Institute of Engineering & Technology
Intermediate - CBSE
Cambridge School
High School - ICSE
Lucknow Public School/College Jankipuram