Rajasekar BS
Senior Data Engineer
Chennai,TN
Summary
Results-driven Data Engineer with six years of experience in technology and engineering services, specializing in data engineering and analytics. Proven expertise in designing, building, and optimizing end-to-end data pipelines to enhance scalability, performance, and reliability. Skilled in leading teams and implementing modern data architectures that facilitate data-driven decision-making through actionable insights. Committed to leveraging advanced analytical skills to drive innovation and improve business outcomes.
Overview
6
6
years of professional experience
4
4
Languages
Work History
Senior Data Engineer
Ciklum
12.2024 - Current
- Data Engineer for the P&G R&D Data Platform, managing ingestion and transformation of IoT and consumer data.
- Designed and implemented Lakeflow-based declarative pipelines in Databricks to ingest and process IoT and consumer data.
- Governed and secured data using Azure Data Lake (ADLS) and Unity Catalog, ensuring data lineage, access control, and compliance.
- Built and optimized pipelines to ingest and synchronize data from AWS S3 to ADLS using PySpark.
- Implemented CI/CD pipelines in Databricks via GitHub Actions, integrating Snyk and SonarQube for vulnerability scanning and code quality checks.
- Automated Power BI dataset refreshes via REST APIs within Databricks workflows.
- Led and mentored a team of five junior engineers, providing technical guidance and upskilling initiatives to enhance team performance.
Senior Software Analyst (Data Engineering)
Bounteous
02.2024 - 12.2024
- Enhanced and supported legacy systems using Airflow, Snowflake, and GCP, ensuring data pipeline efficiency and reliability.
- Encouraged continuous improvement initiatives within the team, driving innovation in process efficiencies and data quality.
- Optimized software development processes by implementing Agile methodologies, reducing time-to-market for data solutions.
Data Engineer II
Elsevier
02.2022 - 02.2024
- Optimized data processing by implementing efficient ETL pipelines and streamlining database design using Redshift.
- Enhanced data quality through comprehensive data cleaning, validation, and transformation using AWS Glue.
- Provided technical guidance and mentorship to junior team members, fostering a collaborative environment and knowledge sharing.
Systems/ Senior Systems Engineer
Infosys
11.2019 - 02.2022
- Developed and optimized backend logic for Sales Planning and Analysis applications under the Verizon account.
- Built ETL packages using SSIS to extract, transform, and load data from multiple upstream systems.
- Improved query performance and database design using SQL Server.
- Partnered with stakeholders for seamless system integrations and project delivery.
Education
MBA - Business Analytics
SRM University
Chennai, India08-2024
B.Tech - Electrical & Electronics Engineering
Amrita Vishwa Vidyapeetham
Chennai, India 04.2001 -
Skills
Programming & Scripting: Python, SQL, PySpark
Data Engineering & ETL: Databricks, Apache Airflow, Azure Data Factory (ADF), AWS Glue, SSIS
Cloud Platforms:Microsoft Azure: Azure Data Lake (ADLS), Azure Databricks, Synapse Analytics, Unity CatalogAmazon Web Services (AWS): S3, Glue, IAM, EC2Google Cloud Platform (GCP): CloudSQL, Airflow integrations
Data Warehousing: Snowflake, Azure Synapse Analytics
Data Modeling & Processing: Delta Lake, ETL/ELT pipeline design, Data Quality & Validation Frameworks
Data Visualization: Power BI (including Power BI REST API automation)
Version Control & DevOps: GitHub, GitHub Actions, CI/CD pipelines, SonarQube, Snyk
Orchestration & Workflow Automation: Airflow DAGs, Databricks Workflows, Lakeflow Pipelines
Timeline
Senior Data Engineer
Ciklum
12.2024 - Current
Senior Software Analyst (Data Engineering)
Bounteous
02.2024 - 12.2024
Data Engineer II
Elsevier
02.2022 - 02.2024
Systems/ Senior Systems Engineer
Infosys
11.2019 - 02.2022
B.Tech - Electrical & Electronics Engineering
Amrita Vishwa Vidyapeetham
04.2001 -
MBA - Business Analytics
SRM University