Around 9 years of IT experience as on Azure Cloud. Experience on Migrating SQL database to Azure data Lake, Azure data lake Analytics, Azure SQL Database, Data Bricks and Azure SQL Data warehouse and Controlling and granting database access and Migrating On premise databases to Azure Data lake store using Azure Data factory.
Experience in Developing Spark applications using Spark - SQL in Databricks for data extraction, transformation and aggregation from multiple file formats for analyzing & transforming the data to uncover insights into the customer usage patterns.
Good understanding of Spark Architecture including Spark Core, Spark SQL, Data Frames, Spark Streaming, Driver Node, Worker Node, Stages, Executors and Tasks.
Expertise in various phases of project life cycles (Design, Analysis, Implementation and testing).
Have designed and developed ETL mapping for data collection from various data feeds using REST API. The data sources include feeds from mobile, social Facebook Query Language, You tube, Twitter, web and other partner feeds.
Created Azure SQL database, performed monitoring and restoring of Azure SQL database.
Performed migration of Microsoft SQL server to Azure SQL database.
Excellent communication skills with excellent work ethics and a proactive team player with a positive attitude.
Worked in mixed role DevOps: Azure Arcitect/System Engineering, network operations and data engineering.
Azure Data Factory (ADF), Integration Run Time (IR), File System Data Ingestion, Relational Data Ingestion.
Experience in Database Design and development with Business Intelligence using SQL Server 2014/2016, Integration Services (SSIS), DTS Packages, SQL Server Analysis Services (SSAS), DAX, OLAP Cubes, Star Schema and Snowflake Schema.
Design and implement database solutions in Azure SQL Data Warehouse, Azure SQL
Work History
Sr Data Engineer | GCP
American Express
Analyzed user requirements, designed and developed ETL processes to load enterprise data into the Data Warehouse.
Trained non-technical users and answered technical support questions.
Developed Python scripts for extracting data from web services API's and loading into databases.
Collaborated with cross-functional teams to gather requirements and translate business needs into technical specifications for data solutions.
Wrote and coded logical and physical database descriptions, specifying identifiers of database to management systems.
Prioritized and organized tasks to efficiently accomplish service goals.
Worked successfully with diverse group of coworkers to accomplish goals and address issues related to our products and services.
Achieved cost-savings by developing functional solutions to problems.
Managed household errands and other essential duties.
Design and implement scalable data pipelines using Google Cloud Dataflow.
Develop and optimize BigQuery datasets for efficient data analysis and reporting.
Collaborate with cross-functional teams to integrate data solutions with business processes.
Automate data workflows using Cloud Composer and Apache Airflow.
Ensure data security and compliance with GCP Identity and Access Management.
Mentor junior engineers in best practices for cloud-based data engineering.
Utilize machine learning models with AI Platform for predictive analytics.
Implement real-time data processing with Google Cloud Pub/Sub and Dataflow.
Continuously evaluate and adopt new GCP tools and technologies.
Lead data architecture discussions to align with strategic business goals.
Participate in agile ceremonies to enhance team collaboration and project delivery.
Data Engineer | AWS
IGATE Global Solutions
Documented AWS architecture and operational procedures, ensuring clear understanding and compliance.
Optimized AWS resource usage and costs through detailed analysis and application of best practices.
Defined technical integration strategy and developed integration plans.
Integrated third-party services with the help of APIs into the existing architecture running on AWS Cloud Platforms.
Planned, implemented, and integrated updated system-oriented projects.
Conducted vulnerability assessments and applied security patches to safeguard AWS environments.
Collaborated with other teams within the organization to ensure seamless integration between legacy systems and new cloud deployments.
Designed and implemented scalable, secure cloud infrastructure using AWS services to meet client requirements.
Maintained high availability of applications by setting up auto scaling groups across multiple regions.
Coordinated and enhanced existing databases and established new databases as part of initiative.
Evaluated emerging technology factors around cost comparison, portability or usability.
Performed analysis of existing cloud environment and developed strategies for migrating workloads to AWS Cloud.
Deployed applications on AWS Cloud using services like Elastic Beanstalk, CloudFormation and OpsWorks.
Configured backup plans for data stored in Amazon S3 buckets by scheduling regular backups and establishing retention periods.
Built and oversaw network infrastructure comprised of various virtual products.
Automated infrastructure deployment using AWS CloudFormation and Terraform to enhance operational efficiency.
Created IAM policies for users and groups with proper access control levels according to organizational security requirements.
Data Engineer Jr | GCP
IGATE Global Solutions
Collaborated in the development of scalable ETL processes for over 5TB of data daily, improving data ingestion efficiency by 30%
Created comprehensive data dictionaries and catalogues that served as essential references for cross-departmental teams
Supported the enhancement of the company's data lake by automating data validation, saving over 15 hours of manual checking per week
Built basic ETL that ingested transactional and event data from a web app with 12,000 daily active users that saved over $85,000 annually in external vendor costs
Worked with client to understand business needs and translate those business needs into actionable reports in Tableau, saving 17 hours of manual work each week
Used Spark in Python to distribute data processing on large streaming datasets, improving ingestion and speed by 67%
Supported implementation and active monitoring of controls and programs for precision and efficacy
Maintained data pipeline up-time of 99.8% while ingesting streaming and transactional data across 8 different primary data sources using Spark, Redshift, S3, and Python
Automated ETL processes across billions of rows of data, which reduced manual workload by 29% monthly
Ingested data from disparate data sources using a combination of SQL, Google Analytics API, and Salesforce API using Python to create data views to be used in BI tools like Tableau
Communicated with project managers and analysts about data pipelines that drove efficiency KPIs up by 26%
Set up a CI/CD pipeline on AWS CodePipeline resulting in a 40% increase in release speed
Managed containerized solutions using Kubernetes and Docker, improving system stability by 25%
Designed an intricate logging system with Amazon CloudWatch, increasing system observability by 30%
Used Python scripts and CloudFormation to automate deployment, reducing deployment time by 35%
Developed Python scripts for extracting data from web services API's and loading into databases.
Analyzed user requirements, designed and developed ETL processes to load enterprise data into the Data Warehouse.
Created and implemented complex business intelligence solutions.
Implemented data visualization tools like Tableau and Power BI to create dashboards and reports for business stakeholders.
Wrote and coded logical and physical database descriptions, specifying identifiers of database to management systems.
Identified, protected and leveraged existing data.
Trained non-technical users and answered technical support questions.
Collaborated with cross-functional teams to gather requirements and translate business needs into technical specifications for data solutions.
Approached customers and engaged in conversation through use of effective interpersonal and people skills.
Completed day-to-day duties accurately and efficiently.
Worked successfully with diverse group of coworkers to accomplish goals and address issues related to our products and services.
Software Engineer Support
BOB Tech Solutions
Addressed users tickets regarding hardware, software, and networking
Maintained and administered computer networks and related computing environments
Managed file servers and printer servers
Managed hardware and software assets
Able to create accurate network diagrams and documentation for designing and planning
Network communication systems
Responsible for handling Outlook and mailbox-related issues
Exhibited strong logging and troubleshooting skills, promptly identifying, and resolving network issues using automation tools and scripting techniques
Handled communications with different domains or engineering and operation teams
Took ownership of network-related problems, ensured timely resolution, and provided detailed incident reports
Helped resolve software and technical questions for the customer efficiently and effectively
Gathered the required information necessary in order to best handle customer software and technical inquiries
Managed customer expectations regarding estimated response times for issue resolution
Resolved the issues through phone chat and email communication channels
Education
Education
Master of Science - Data Science
Saint Peter's University
Jersey City, NJ
02.2024
Bachelor of Science - Electrical, Electronics And Communications Engineering
