Pooja Pilani

• Developed an ELT process to load data from MongoDB to Databricks into Bronze and Silver databases
• Utilized Kafka for real-time data streaming from MongoDB to Databricks using the KafkaMongoSource connector
• Optimized data processing by correctly balancing job loads to ensure real-time performance
• Handled client-related queries and provided solutions to meet client requirements
• Implemented data validation and quality checks to ensure data integrity and accuracy
• Developed data cleansing scripts to improve data quality and consistency
• Created and maintained data models and schemas to support analytical reporting
• Conducted root cause analysis and troubleshooting for data-related issues

Member Data Ingestion

• Developed pipelines to ingest data from diverse sources such as client EMRs, CSVs, and Amazon S3
• Designed and optimized the architecture for accurate auditing to maintain data integrity for future reference
• Ensured compliance with healthcare data regulations and standards during data transformation and storage
• Automated data validation and quality checks to improve the reliability and consistency of ingested data
• Collaborated with cross-functional teams to define requirements and optimize data flow.

MPHRX In-house Product

• Provided product support to enhance the usage of in-house software, Mirth, used to ingest data from various sources in different formats and perform product transformations for multiple releases
• Worked on a dedicated framework for generating Looker reports based on client requests by transforming data from MongoDB to MySQL
• Upgraded MongoDB versions and provided framework support for two clients
• Optimized data transformation processes to ensure seamless data flow between MongoDB and MySQL for accurate reporting
• Collaborated with product teams to identify and implement improvements in the Mirth data ingestion process
• Conducted performance tuning and troubleshooting to maintain high availability and efficiency of the data ingestion and transformation systems.