Gaurang Patel
Data Architect
Ahmedabad
Summary
Having 14+ years of work experience in IT Industry and having total experience in Data engineering is 12 + years.
Overview
13
13
years of professional experience
Work History
Data Architect
MG Marketsoft
Ahmedabad
11.2011 - Current
- Automated ETL Process, making easier to clean data that has reduced time by 60% percent
- Avoided operational Loss of $40K for the company
- Ensured end to end process of data from extraction to reporting and noted efficiency close to 30%
- Ingested data from the 7 different sources from the compromising data logs from payment history, structured data from user details
- Migrated slow Processing ETL Pipelines to spark which resulted in faster execution of pipeline
- Expanded and optimized data and data pipeline architecture resulting in 30% efficiency in resource utilization
- Optimized the existing pipeline to deal with the growing data requirements which resulted in reducing resources by 50% and speed up the process by 10X
- Led to migration of Old server and architecture to Azure which saved annual cost of the company by 27%
- Worked with team of 3 data engineers and collaborated with company management, Client interaction
- Recommended changes on based of data history and tests
- Managed 3 cross functional team to work in close collaboration with stakeholders, data engineers and client technical team
- Created many flow charts, system design diagrams and design documentation to assist problem analysis
- Worked closely with team members, stake holders and solution architects to incorporate changes to the existing pipelines which are running map reduce
- Built scalable and fault tolerance systems and oversaw the complete ETL process
- Become a point of operational excellence in ensuring that job failures are sorted out before SLA
- Become point of innovation pioneers by building new frameworks.
MS SQL Server DBA/Developer
NCodeTechnologies Inc
01.2011 - 11.2011
- Developing , Monitoring, Optimizing Database Objects like Tables, Stored Procedures, Views, Triggers, Rules, Defaults, user defined data types and functions for prospect finder project
- Troubleshooting query performance, procedure performance with various parameter
- Developing ETL Processes in for Transferring Data from Flat file and other sources to SQLServer and MySQL Database
- Developed a backup and recovery plan based on the data availability requirements
- Designed database maintenance plans to set up core maintenance tasks to make sure that the database performs well, is regularly backed up in case of system failure and is checked for inconsistencies.
MS SQL Server Database Analyst
Shakti Informatics Pvt LTD
11.2010 - 01.2011
- Analyzing Call Center data of large bank and insurance house – exporting, compiling and manipulating data and summarizing results in various formats, as well as providing recommendations based on data analysis
- As part of a team, created various reports according to the business requirements for the company
- Individually authored, installed, deployed and managed reports using SQL Server Reporting Services for a call center database
- Collaborating with the Personnel Analysis team on projects and key tasks
- Reporting results to management, including Firm Partners
- Work with users and IS management in loading and understanding data and data flows.
Jr.Software Engineer
Value Chain Solution India Pvt LTD
07.2008 - 11.2009
- Analyzed the business requirements and systematically designed a MS SQL database for Decision Support system based on supply chain solution
- Employed best practices in creating a complete schema including tables, relationships, stored procedures, views, clustered and non-clustered indexes and triggers, and implemented the importation and exportation of data
- Individually designed a E-cubix(Dashboard) application using ASP.NET, C# according to business requirements, and implemented stored procedures, functions, and triggers to implement required functionalities
- Developed and designed the GUI with asp.net and AJAX for C-Logix, Created Report by using DevExpress Report Tool.
Education
M.Sc - Information Science
B.Sc - Computer Science
CERTIFICATE - undefined
mcp.microsoft. - undefined
Skills
Python, Scala, SQL, C#BIG Data : Hadoop, Hive, Spark, HBASE, My-SQL, Kafka, Sqoop, MSSQL ServerCloud services : AWS Redshift, Glue, Athena, Kinesis, Databricks, Azure Data factory, EMR , AZURESQLScheduler : AirflowIDE : Eclipse, Intellij, DatabricksProject Planning: Jira, agile
Timeline
Data Architect
MG Marketsoft
11.2011 - Current
MS SQL Server DBA/Developer
NCodeTechnologies Inc
01.2011 - 11.2011
MS SQL Server Database Analyst
Shakti Informatics Pvt LTD
11.2010 - 01.2011
Jr.Software Engineer
Value Chain Solution India Pvt LTD
07.2008 - 11.2009