Rakshita Shetty
Data Engineer
Udupi
Summary
Results-oriented data professional with extensive experience designing and delivering data solutions for enterprise clients. Skilled in Databricks Lakehouse, Apache Spark™, and Azure, with strong proficiency in Python, SQL, and Spark for building scalable, high-performance ETL pipelines. Proven success in migrating legacy systems to modern big data platforms, optimising data workflows, and translating complex business needs into robust technical architectures. Collaborative team player passionate about leveraging data to drive meaningful business outcomes.
Overview
4
4
years of professional experience
3
3
Certifications
Work History
Solutions Consultant
Databricks
08.2022 - Current
- Assisted in the end-to-end design, development, and deployment of large-scale data solutions using Databricks Lakehouse and Apache Spark™ for enterprise clients across multiple industries.
- Built and optimised scalable, high-performance ETL pipelines using Python, Spark, and Databricks best practices, streamlining data processing and database design.
- Successfully migrated legacy data systems to modern big data platforms, significantly improving scalability, performance, and reliability while minimising business disruption.
- Enhanced data quality by performing thorough data cleaning, validation, and transformation tasks, ensuring robust analytics and reporting.
- Streamlined complex workflows by breaking them down into manageable, reusable components for easier implementation and maintenance.
- Collaborated closely with cross-functional teams—including project management and customer stakeholders—to deliver technical components on time and within scope.
- Provided technical enablement, knowledge transfer, and mentoring to customer teams and partners, enabling knowledge transfer and adoption of Databricks technologies.
- Conducted extensive troubleshooting and root cause analysis to identify and resolve issues, maintaining data pipeline stability and integrity.
- Developed reusable project artifacts and accelerators to streamline solution delivery and maximise customer value.
- Increased efficiency of data-driven decision-making by creating user-friendly dashboards and reporting solutions that enable quick access to key metrics.
Research and Development Intern
Signify (Formerly Known as Philips Lighting)
08.2021 - 06.2022
- Developed an evaluation metric to quantitatively measure the performance of a Geo-Localisation System designed for detecting street lamps in urban environments.
- Conducted research and analysis to identify relevant performance indicators, ensuring accurate assessment of system capabilities.
- Supported the enhancement of the Geo-Localisation System by providing actionable insights based on metric-driven evaluations.
- Created a user-friendly dashboard using the Streamlit package to visualise system performance metrics and detection outcomes for stakeholders.
Education
Master of Engineering - Big Data And Data Analytics
Manipal School of Information Sciences
Manipal, Karnataka, India 10.2020 - 10.2022
Bachelor of Engineering - Computer Science And Engineering
NMAM Institute of Technology
Nitte, Karkala, Karnataka, India 08.2013 - 07.2017
Skills
Databricks
Python
SQL
Pyspark
Apache Spark
Apache Airflow
Kafka
Azure
MLOPS
Streaming
GitHub
Data Governance
ETL development
Data pipeline design
Data warehousing
Data modeling
Data migration
Machine learning
Additional Information
Migration from Informatica and Netezza to Databricks
Technologies: Informatica Workflow Manager, Databricks, Python, PySpark, SQL, Git
- Migrated business logic and workflows from Informatica to Databricks, significantly enhancing performance.
- Re-implemented and optimised ETL processes in Databricks using PySpark and Python, modularising code for maintainability and reusability.
- Translated Informatica workflows into Databricks Jobs, streamlining orchestration and automation of data pipelines.
- Diagnosed and resolved production issues, ensuring robust, reliable, and efficient data operations post-migration.
Migration from SQL Server to Delta Lake
Technologies: SSIS, Databricks, Python, PySpark, SQL, Git
- Migrated ~1600 tables with historical data load from SQL Server to Databricks Delta, ensuring accuracy and completeness of transferred data.
- Designed and implemented an automated framework for DDL creation, supporting various operational modes.
- Built a robust historical load framework using JDBC for seamless data ingestion from SQL Server to Delta Lake.
- Developed a comprehensive validation framework utilising internal tools to ensure data quality; all frameworks are metadata-driven, support multiple operational modes (all, failure, catchup, adhoc), and are designed for easy use by users without Python or PySpark knowledge.
Migration of Hive, Pig, and Shell Scripts from Hadoop to Databricks
Technologies: Databricks, Python, PySpark, SQL, HiveQL, Azure Repos
- Migrated and re-engineered data pipelines from Hive, Pig, and Shell scripts to Databricks, implementing equivalent business logic in Python and SQL.
DBU Consumption Dashboard (Internal Use Case)
Technologies: Databricks AI/BI Dashboards
- Developed a comprehensive internal dashboard to monitor and track Databricks DBU (Databricks Unit) consumption, improving cost visibility and resource management.
DPP (Delivery Partner Program) Process Automation (Internal Use Case)
Technologies: Airtable, Databricks AI/BI Dashboards
- Automated the DPP process to streamline partner candidate tracking, interview scheduling, and communications across global regions. This enabled 130+ resource onboarding in Q1.
- Built dashboards for leadership to monitor partner resource utilisation and process efficiency.
- Recognised by leadership for significant contributions to process automation and operational improvement.
Certification
Databricks Certified Data Engineer Associate
Work Availability
monday
tuesday
wednesday
thursday
friday
saturday
sunday
morning
afternoon
evening
swipe to browse
Work Preference
Work Type
Full TimeLocation Preference
RemoteImportant To Me
Career advancementCompany CultureWork-life balanceFlexible work hoursPersonal development programsHealthcare benefitsWork from home optionPaid time offTeam Building / Company RetreatsPaid sick leaveStock Options / Equity / Profit Sharing401k match4-day work weekTimeline
Solutions Consultant
Databricks
08.2022 - Current
Research and Development Intern
Signify (Formerly Known as Philips Lighting)
08.2021 - 06.2022
Master of Engineering - Big Data And Data Analytics
Manipal School of Information Sciences
10.2020 - 10.2022
Bachelor of Engineering - Computer Science And Engineering
NMAM Institute of Technology
08.2013 - 07.2017