LAVANYA YARAGALA
London
Summary
Data Engineer with extensive experience at Lexis Nexis Risk Solutions Group, specializing in cloud-native applications and data warehousing. Proficient in AWS and Python, driving efficiency through CI/CD automation and data governance initiatives. Demonstrated leadership in teams and delivery of impactful data solutions that enhance business decision-making.
Overview
7
7
years of professional experience
1
1
Certification
Work History
Data Engineer III
Lexis Nexis Risk Solutions Group
London
05.2022 - Current
- Led the design, development, and delivery of cloud-native applications, ensuring alignment with business objectives and industry best practices.
- Designed, developed, and deployed cloud-native applications using Python and AWS cloud platforms.
- Developed and maintained CI/CD pipelines using Jenkins and GitLab CI/CD, ensuring automated testing, building, and deployment of applications.
- Orchestrated workflows using Apache Airflow and AWS Step Functions, ensuring efficient and scalable workflow execution.
- Scripted in Python, SQL, Bash, Python, Terraform to automate data workflows and processes.
- Provided technical leadership and guidance to development teams, ensuring adherence to industry best practices, design patterns, and coding standards.
- Collaborated with DevOps teams to ensure seamless deployment, monitoring, and maintenance of cloud-native applications using Docker, Kubernetes.
- Proactively used metrics to inform performance optimizations and improve data pipeline efficiency.
- Collaborated with cross-functional teams to identify and prioritize project requirements and develop solutions that meet business needs.
- Collaborated with stakeholders to understand business requirements and delivered data-driven insights to inform decision-making.
- Developed and maintained technical documentation, including system design documents, architecture diagrams, and technical guides.
- Designed and implemented scalable data warehousing solutions, leveraging cloud-based platforms like Snowflake to optimize data storage, access, and analytical performance.
- Spearheaded initiatives for data governance and quality assurance, implementing frameworks and policies, including the use of platforms like Collibra, to ensure data integrity and compliance across pipelines.
- Developed and deployed Infrastructure as Code (IAC) using Terraform to provision and manage AWS resources, ensuring consistent, repeatable, and scalable cloud infrastructure for data applications.
Hadoop Developer
Tata Consultancy Services
10.2018 - 04.2022
- The project consists of various applications that imports data from various sources and processes it to provide standardized data for analytics purpose by business.
- Responsibilities include to maintain value and integrity of output data as well as timely availability to avoid business impact.
- Understood the insights of business flow and usage of various processes to achieve 100% SLA.
- Implemented various processes and automation that resulted in hours of time savings and CPU utilization and received multiple appreciations for the same.
- Implemented various bug fixes in production code and introduced automations to save effort.
- Worked with clients on their needs and provided optimal solutions for various issues.
- Prepared various technical and function documents of processes.
- Development of hive scripts using HQL.
- Analyzed jobs and added hive parameters/properties to decrease the runtime and increase performance of jobs.
- Created space alerts and avoiding the job failures because of space issue.
- Perform code quality checks as and when required.
- Apply transformations on the data as per the clients requirements and load the data into target tables in hive.
- Develop Scala code to load the data into target tables.
- Develop CDC framework to update the updated records and also to remove duplicate records through spark shell.
- Implemented various bug fixes in production code and introduced automations to save effort.
- Working with clients and getting appreciations for the design flow and the coding scripts I provided and deployed in production environment.
- Prepared various technical and function documents of processes.
Education
Velagapudi Ramakrishna Siddhartha Engineering College - Electronics and Communication Engineering
Velagapudi Ramakrishna Siddhartha Engineering College
06-2018
Skills
- AWS
- Data migration
- Hadoop ecosystem
- S3
- Glue
- EMR
- Athena
- Lambda
- Step Functions
- Snowflake
- Hadoop
- Hive
- Apache Spark
- Python
- Py Spark
- SQL
- Scala
- Unix Shell Scripting
- Bash
- Terraform
- Apache Airflow
- Stream lit
- Collibra
- Jenkins
- GitLab CI/CD
- Docker
- Kubernetes
- ETL Pipelining
- Data Lake
- CDC Frameworks
- Parquet
- ORC
- Grafana
- Splunk
Personal Information
Visa Status: PBS dependent visa
Certification
AWS Certified Developer - Associate
Sponsorship Required
No sponsorship required
Core Technical Skills
AWS (S3, Glue, EMR, Athena, Lambda, Step Functions), Snowflake, Hadoop, Hive, Apache Spark, Python, Py Spark, SQL, Scala, Unix Shell Scripting, Bash, Terraform, Apache Airflow, Streamlit, Collibra, Jenkins, GitLab CI/CD, CI/CD, Docker, Kubernetes, ETL Pipelining, Data Lake, CDC Frameworks, Parquet, ORC, Grafana, Splunk
Timeline
Data Engineer III
Lexis Nexis Risk Solutions Group
05.2022 - Current
Hadoop Developer
Tata Consultancy Services
10.2018 - 04.2022
Velagapudi Ramakrishna Siddhartha Engineering College - Electronics and Communication Engineering
Velagapudi Ramakrishna Siddhartha Engineering College
Similar Profiles
Simon HebdonSimon Hebdon
Regional Director of Sales Performance at Lexis Nexis Risk Solutions (part of Relx)Regional Director of Sales Performance at Lexis Nexis Risk Solutions (part of Relx)
Brayton WyattBrayton Wyatt
Technical Support Analyst II at Lexis Nexis Risk SolutionsTechnical Support Analyst II at Lexis Nexis Risk Solutions
Sam EganSam Egan
Customer Operations Manager at Lexis Nexis Risk SolutionsCustomer Operations Manager at Lexis Nexis Risk Solutions
Jibril MekrousJibril Mekrous
Sales associate / Workshop manager at EURAUTOSales associate / Workshop manager at EURAUTO
Hanga LovasHanga Lovas
Executive Assistant & Office Administrator at The Last MileExecutive Assistant & Office Administrator at The Last Mile