Talend
Rupesh Agrawal
Sr Data Architect
Pune
Summary
Accomplished data management and quality professional with 19 years of expertise in designing, implementing, and maintaining comprehensive data management frameworks that ensure accuracy, completeness, and consistency. Experienced with designing and implementing complex data architectures to support enterprise-level applications in cloud migration strategy. Passionate about leveraging data asset to empower organizations and foster informed decision-making.
Overview
19
19
years of professional experience
1
1
Certification
Work History
Senior Data Architect(First Citizen Bank)
Tata Consultancy Services
10.2019 - Current
- Led the data management team to design and implement a data quality architecture to ensure data accuracy, completeness, and consistency.
- Developed a highly scalable and configurable metadata-driven data quality-as-a-service framework for each hop of the Medallion architecture.
- Designed and developed a strategy for the data dictionary, data catalog, and metadata repository
- Conducted a data profiling and data cleansing exercise.
- Designed the reporting dashboard for data quality monitoring.
- Developed project plans and managed project scope using methodologies to guide projects from conceptualization to implementation, and maintenance
- Collaborated with business users, technical teams, database administrators, and testing teams during kickoff meetings, joint application design, and planning sessions to validate requirements
- Participated in the architecture review board meeting to review the design proposed by the development team and ensure it is in line with long-term strategy
- Implemented effective data governance policies, ensuring consistent use of data elements across the organization.
- Actively participated in regular code reviews helping improve overall code quality across the team.
- Designing and developing processes for platform stabilization, which includes performance improvement, data center migration, and process automation.
Cloud Data Lead(The Home Depot)
Tata Consultancy services
08.2015 - 09.2019
- Led and managed strategic programs and cross-functional project teams, demonstrating a sense of urgency to drive programs and projects to completion, and being successful in a collaborative environment.
- Developed a hybrid data quality framework to work with the data pipeline and as an independent process.
- Develop plans and track the progress of projects to successful completion.
- Design and development of ETL mapping in Talend based on functional requirements.
- Leading a team for the cloud migration project, which delivered customer data and processes from on-premises to Google Cloud on Google BigQuery.
- Communicate status, issues, and risks to executive-level stakeholders.
- Managed changes to project scope, project schedule, and project costs using appropriate verification techniques.
- Coordinate internal resources and other teams for flawless execution of projects.
- Monitoring project deadlines, quality control, and change management.
ETL Lead Developer(Sears Holding Corporation)
Tata Consultancy services
06.2009 - 07.2015
- Developing Talend Big Data ETL mapping to handle large set of click stream data on hourly basis
- Development of ETL job using Hadoop components HDFS and HIVE as source and targets
- Lead the team to develop the service-based architecture for Data pipeline processing
- Developing Talend mapping to implement functionality similar to ab Initio Graphs using mapping documents
- Developing job to sync data between Hadoop and Teradata using Sqoop
- Developed Talend mapping to read Source data using restful web services
- Parsing JSON and XML data before loading into Hive
- Working on Agile environment using continuous integration
- Tuning jobs and queries for better performance by applying best practices
- Performing Unit testing, Functional testing and Integration testing
ETL Developer(Crysler)
Tata Consultancy Services
10.2006 - 05.2009
- Developed DataStage Jobs using required stages, and the obtained data from different sources were formatted, Cleansed, summarized, aggregated, transformed and loaded into target
- Used the Ascential Data Stage 7.5 and 8.1 both server and parallel extensively to achieve business needs
- Tuned DataStage jobs for better performance by applying best practices
- Wrote Complex SQL quesries, UNIX scripts for the Delta Process
- Worked as Datastage Admin, created projects, Users and project level parameters
- Involved in design and development of parallel jobs, sequences using the Designer
- Worked with SCD stage for implementing slowly changing dimensions
- Data staging of the ETL solutions from different source systems to target database
- Designed the job templates and created new jobs from existing templates
Education
Bachelor of Engineering - Electronics and Communication Engineering
Samrat Ashok Technological Institute
Vidisha, MP06.2006
Skills
Data Quality framework
Master data management
ETL development
Data warehousing solutions
Software
Datastage
SAP BODS
SSIS
Database (Teradata,Oracle, DB2,Netezza))
SQL
Informatica
SSAS
Redwood
Google Bigquery
GCP Migration
Snowflake
Databricks
AWS Glue
Certification
Aws solution architect associate
Accomplishments
- Improved data quality by 98% through implementation of data quality framework and data cleansing processes.
- Reduced data-related issues by 95% through data governance and data management initiatives.
- Developed and implemented a data governance framework, resulting in improved data accuracy and compliance.
- Successfully led a team of 12 people to migrate 600 TB of data from om premise data center to Google cloud, achieving 99.5 % data accuracy and zero downtime.
- Reduced security vulnerability issue by 99% by implementing the security scan and automatic security ticket creation process
Timeline
Aws solution architect associate
02-2023
Senior Data Architect(First Citizen Bank)
Tata Consultancy Services
10.2019 - Current
Cloud Data Lead(The Home Depot)
Tata Consultancy services
08.2015 - 09.2019
ETL Lead Developer(Sears Holding Corporation)
Tata Consultancy services
06.2009 - 07.2015
ETL Developer(Crysler)
Tata Consultancy Services
10.2006 - 05.2009
Bachelor of Engineering - Electronics and Communication Engineering
Samrat Ashok Technological Institute