SURAJ GUPTA
Senior Principal MTS
Bangalore
Summary
Ạ quality focused Lead / Architect with over 16 + years of experience in the Technological domain , Software Development & maintenance experience. Extensive Experience in building Data Engineering & analytical solutions.
Overview
17
17
years of professional experience
4
4
years of post-secondary education
3
3
Languages
Work History
Sr PMTS
Oracle Cloud
05.2022 - Current
- Led the initiatives and actively involved in recommendation for re architect existing framework and pipelines using modern data architecture and framework.
- Used strong analytical and problem solving skills to develop effective solutions for varied situations.
- Designed and Demonstrated working prototypes of DataBricks Delta Lake integrated with Spark and Kafka .
- Actively involved along side internal partners to help them migrate to our new Horizon 2.0 platform .
- Developed robust API's for seamless integration for cross realm data transfers across realms using mutual TLS cert based authentication .
- Demonstrated creativity in identifying and recommending architecture design for re architecting the current HDP platform using modern data lake architecture and tools.
Lead MTS
Salesforce
04.2021 - 11.2021
- Contributed to business growth by identifying areas where technology could be leveraged more effectively to meet project requirements.
- Mentored junior team members in various aspects of DTS execution, fostering a culture of continuous learning within the organization.
- Managed and actively involved with Vendors like AWS, Fivetran.
- Building proof of concepts within the organizations technical frameworks / expansions to solve business problems.
- Designed and Developed data migration tool for customers to perform data transfer from customer's on-prem or non-aws data lake to aws data lake.
- Worked on building data processing pipelines in Spark for building analytical datasets for various customer.
- Worked on improving Spark performances and overcoming limitations of Spark framework by leveraging various open source frameworks like Iceberg, DeltaLake etc
- Worked closely with Product owners & salesforce partners (AWS) to fulfill and manage the business
Staff Software Engineer
Walmart Labs
11.2019 - 02.2021
- Reduced Application load times and Streamlined Data retrieval frameworks by building modern Streaming pipelines using Spark ,Kafka Connect integration framework.
- Enhanced Team performance by providing periodic training sessions and constructive feedbacks on individual tasks.
- Provided technical guidance and mentorship to junior team members, fostering a collaborative learning environment within the organization.
- Proactively identified opportunities for process improvements, addressing inefficiencies through innovative solutions and implementing best practices in the field of data engineering.
- Actively involved with cross functional and cross geographical teams to priorities action plans and resource utilization.
- Worked on understanding and streamlining Walmart existing processes for better coordination and alignment among teams
- Leading the efforts in migration of existing traditional internal ETL framework and scheduling tools to advanced processing framework like Spark along with building CICD pipelines.
- Designed & built Modern Datalake in collaboration with cross geo teams and successfully lead teradata migration into datalake.
- Maintained comprehensive documentations of technical specifications, enabling smooth knowledge transfer within the team during transitions or new additions.
Software Development Engineer
TESCO
09.2016 - 10.2019
- Worked in a fast-paced agile development environment to quickly analyze, develop, and test potential use cases for the business
- Work closely together with global tech, product and data science teams to develop new ideas, implement and test them, and measure success
- Building CICD pipelines and developing project frameworks
- Contributes design, code, configurations and documentation for components that manage data ingestion, real time streaming, batch processing, data extraction, transformation
- End to End Design & development of Data platform for building a master source of Tesco Products
- Involved in Design and analysis of new features and product using Scala & Spark/Kafka Frameworks to provide better and effective solution to business.
- Maintaining the existing application framework and ensure smooth executions of various processes running on cluster
- Cross-trains, Mentoring team members on technologies being developed e.g. Scala functional programming, Spark programming etc while also continuously learning new technologies as per project needs.
- Participated and demonstrated working prototype solving business problems as a Single individual on TESCO HACKATHON and won high appreciation.
Module Lead Engineer
IMPETUS INFOTECH
09.2015 - 09.2016
- Design and Developed new modules and demonstrated abilities in key decision makings
- Maintenance and Enhancements of existing modules and involved in re designing implementations
- Involved in developing modules for new client involving technologies like Scala 2.10, Spark 1.5, Kafka 0.9 and Cassandra 2.2
- Successfully developed and delivered couple of initiatives on big data analytics and large data processing using SparkSQL, Scala and Hive
- Developed code on XML processing using Spark/Scala
- Training and mentoring junior team members in Big Data technologies
Technical Consultant
E2 Open, Amber Road Softwares
07.2012 - 09.2015
- Understanding HDFS and Hadoop mechanism.
- Co coordinating with Architects and being part of initial set up of data pipelines using Hadoop map reduce, sqoop etc.
- Participating in design discussions and analyze the problem in detail to understand the key points.
- Interfacing with client for requirement gathering, finalizing technical specifications for our product.
- Working closely with Functional Team to support finalizing specification design document.
- Ensuring smooth implementation and testing of the application
Associate
COGNIZANT TECHNOLOGY SOLUTIONS
06.2011 - 07.2012
- Interfacing with Onsite team for business gathering and finalizing technical / functional specifications.
- Involved in high level and Low level design for the project as lead.
- Monitoring multiple teams and working closely to support production issues as Lead.
- Fixing functional and performance defects.
Associate Consultant
ORACLE FINANCIAL SERVICES SOFTWARE
10.2010 - 06.2011
- Involved in development of software component for banking client.
- Developed multiple modules across projects and delivered on time.
- Provided support for production issues and fixes.
Senior Software Engineer
MINDTREE LTD
07.2007 - 09.2010
- Interfacing with clients for business gathering, conducting system analysis and finalizing technical / functional specifications and high level design documents for the project.
- Independently handled module/Client and was solely responsible for the functionality and technical specifications of the module in the offshore team.
- Developed custom reports and SDD (Solution Description Documents) and successfully implemented and ensured proper unit testing before delivery.
- Participating in project management and testing as per requirement in the project.
Education
BE - Electrical, Electronics And Communications Engineering
SATHYABAMA UNIVERSITY
Chennai, TAMIL NADU 08.2003 - 06.2007
Skills
Java
Scala
HiveMap ReduceApache Spark 16/23RESTApache KafkaHadoop/YARN 26Elastic Search (ELK)Struts 11Hibernate 30SpringRDBMS
HBaseCassandra
AWSMs AzureApache NifiMavenGitJenkinsCI/CDDockerDatabricks Delta LakePLSQL
Presto
Flume
Sqoop
Terraform
Apache Hudi
Apache iceberg
Timeline
Sr PMTS
Oracle Cloud
05.2022 - Current
Lead MTS
Salesforce
04.2021 - 11.2021
Staff Software Engineer
Walmart Labs
11.2019 - 02.2021
Software Development Engineer
TESCO
09.2016 - 10.2019
Module Lead Engineer
IMPETUS INFOTECH
09.2015 - 09.2016
Technical Consultant
E2 Open, Amber Road Softwares
07.2012 - 09.2015
Associate
COGNIZANT TECHNOLOGY SOLUTIONS
06.2011 - 07.2012
Associate Consultant
ORACLE FINANCIAL SERVICES SOFTWARE
10.2010 - 06.2011
Senior Software Engineer
MINDTREE LTD
07.2007 - 09.2010
BE - Electrical, Electronics And Communications Engineering
SATHYABAMA UNIVERSITY
08.2003 - 06.2007