Dinesh Singh Negi

Data Engineer:

• Collaborated on ETL (Extract, Transform, Load) tasks, maintaining data integrity, and verifying pipeline stability.
• Fine-tuned query performance and optimized database structures for faster, more accurate data retrieval and reporting.
• Designed and implemented a real-time data pipeline to process semi-structured data by integrating 150 million raw records from 30+ data sources, using Kafka and Spark.
• Cost optimization on the Snowflake side, with Kafka topics going from an average of $500 per day down to approximately $100 per day on the production warehouse, and saving around $145,000 per year.
• Managed AWS EC2 instances and S3 buckets. Developed automated jobs for HL7 processing, ensuring accurate FHIR conversion, and displayed clinical resources on the UI for enhanced visibility, reducing manual efforts by 20%.
• Optimized data processing by implementing efficient ETL pipelines, and streamlining database design.
• Automated real-time status updates, reducing manual intervention by 90%, and ensuring seamless data ingestion into MongoDB.
• Engineered and automated job that enabled efficient and accurate data fetching of failed CPF data from MongoDB, and production of the report. Saved 20+ hours of manual work per week.
• Provided technical guidance and mentorship to junior team members, fostering a collaborative learning environment within the organization.
• Increased efficiency of data-driven decision-making by creating user-friendly dashboards that enable quick access to key metrics.
• Led end-to-end implementation of multiple high-impact projects, from requirements gathering through deployment, and post-launch support stages.
• Automated routine tasks using Python scripts, increasing team productivity, and reducing manual errors.
• Reviewed project requests describing database user needs to estimate the time and cost required to accomplish the projects.

Operations:

• Incident Identification, Incident Logging, and Escalation, Investigation and Diagnosis, Resolution, and Recovery.
• Optimizing the scheduling of batch jobs to minimize resource conflicts, maximize system efficiency, and ensure timely execution.
• Monitoring implementation through various tools like Zabbix, Grafana, Prometheus, Loki, Airflow, etc. To check application and system health involving 24/7 coverage efforts, along with monitoring set-up and pre-existing alert enhancement.
• Data extraction and reporting based on internal and external requirements from production while ensuring PHI and PII compliance is met.
• Trained the new joiners in the product and workflow, and helped the team with new deployments and internal requests.
• Experience with Ansible for automating the infrastructure tasks.
• Ensuring SLAs are met, and making necessary escalations and follow-ups for the resolutions.
• Transitioning new implementations into production using the Jenkins pipeline and automating scripts as per the requirements.
• Tracking job/pipeline execution, identifying any issues or delays, and generating reports or notifications to keep stakeholders informed about job completion, success, or failure.

• Delivered comprehensive reports highlighting key trends, patterns, anomalies, presenting findings to senior management for informed decision-making purposes.
• Analyzed large amounts of data to identify trends and find patterns, signals and hidden stories within data.
• Partnered with IT teams to ensure seamless integration between databases and analytical tools, maximizing system efficiency across departments.
• Served as a subject matter expert on various projects, sharing valuable insights derived from extensive industry experience as a Senior Data Analyst.
• Reduced manual data entry errors by designing and deploying automated ETL processes to transform raw data into usable formats.
• Spearheaded the implementation of dashboard visualization tools[Tableau/PowerBI], providing actionable insights for stakeholders and decisionmakers.
• Trained junior analysts in best practices for statistical modeling techniques, fostering professional growth within the team.
• Used business objects, business intelligence and other reporting tools to extract data from data solutions and data warehouses.