Faujveer Singh
Chennai
Summary
Experienced Data Engineer/Analyst with 2+ years of expertise in designing and implementing robust data solutions. Proficient in ETL processes and optimizing data workflows for enhanced business intelligence. Skilled in leveraging various data tools and technologies to drive actionable insights supporting decision-making. Adept at collaborating with cross-functional teams to deliver scalable, efficient data solutions.
Overview
2
2
years of professional experience
3
3
Certification
Work History
Azure Data Engineer
COGNIZANT TECHNOLOGY SOLUTIONS
01.2024 - Current
- Worked on data ingestion and curation using Azure Databricks for processing and Azure Data Factory for orchestration
- Developed efficient data pipelines using Azure Data Factory (ADF) to ingest data into Delta Lake for efficient storage and querying.
- Designed Medallion architecture (bronze, silver, gold layers) in Azure Databricks for effective data transformation.
- Implemented incremental load in the ingestion pipeline to efficiently process and load only new or updated data, improving performance and reducing processing time
- Developed Databricks notebooks using PySpark and Spark SQL for data extraction, transformation and aggregation from AWS S3 sources, including JSON, CSV, Text and Parquet files, to uncover insights into customer usage patterns
- Created and managed tasks and user stories in Azure DevOps for project tracking and collaboration, while utilizing GitHub for version control and code management
- Worked on Data Export jobs using Bash Shell, Hive and pyspark scripts to perform Address Matching and Customer Matching and WHD framework to generate unique address and party keys for the targeted campaigns.
- Worked on Hive to PySpark conversion jobs to optimize data processing by converting HQL scripts into PySpark using DataFrame, API, and Databricks SQL reducing job runtime by approximately 45%.
- Responsible for analyzing, data mapping and orchestrating the seamless migration of existing ETL jobs to leverage Dynamics source data
- Managed data transfers by importing and exporting data into HDFS and Hive scripts using Sqoop
Data Engineer
COGNIZANT TECHNOLOGY SOLUTIONS
01.2023 - 12.2023
- Developed ETL processes for migrating data from legacy systems to Teradata and other big data platforms, also migrated over 20+ scripts to be compatible with the big data systems
- Experience with tools such as PuTTY, WinSCP, Ambari, qlik task HDI compute
- Contributed to various SDLC phases, including planning, design, implementation, Unit testing, and deployment, while working in an Agile-driven team environment
- Design and optimize data pipelines for real-time and batch processing across multiple data sources
- Participate in code reviews and contribute to improving the overall quality of the development team's work
- Troubleshoot performance issues and optimize queries running on big data platforms and developed a code using pyspark
- Developed automation scripts for the table audit, data validation and reconciliation.
- Skilled in effective client communication, ensuring a clear understanding of requirements to deliver tailored, impactful solutions.
Education
B.Tech - IT
Sikkim Manipal Institute of Technology
Sikkim05-2022
Skills
- Azure Databricks
- PySpark
- HiveQL/SQL
- Hadoop
- Azure Data Factory
- ETL
- Python
- AWS S3
- Shell scripting
- Power BI
- Azure Devops
- GitHub/GitLab
- Agile methodology
Certification
- Databricks Certified Data Engineer Associate
- Microsoft Certified: Azure Data Fundamentals
- Microsoft Certified: Azure Data Engineer
Accomplishments
- Certificate of Appreciation - Above & Beyond: Recognized by clients for extraordinary work in Re-platforming project.
- Coral Award - Acknowledged for flexibility in working with Data squad, leading end-to-end work & making outstanding contributions to ETL jobs.
Languages
- English
- Hindi
Timeline
Azure Data Engineer
COGNIZANT TECHNOLOGY SOLUTIONS
01.2024 - Current
Data Engineer
COGNIZANT TECHNOLOGY SOLUTIONS
01.2023 - 12.2023
B.Tech - IT
Sikkim Manipal Institute of Technology
Work Preference
Work Type
Full TimeWork Location
On-SiteRemoteHybridSimilar Profiles
PRAKASH NARAYANA VIGNESH BPRAKASH NARAYANA VIGNESH B
Business Analyst- Technical Lead at Cognizant technology SolutionsBusiness Analyst- Technical Lead at Cognizant technology Solutions
Raja RathinamadasamyRaja Rathinamadasamy
QA Manager at Cognizant Technology SolutionsQA Manager at Cognizant Technology Solutions
Josephine Arockiya Rani SJosephine Arockiya Rani S
Technical Lead -AWS at Cognizant Technology Solutions, CTSTechnical Lead -AWS at Cognizant Technology Solutions, CTS