Madhukar Danthurthi

1.Real-Time Log Retrieval from Vector-Dev Instances Using Python

Developed a Python-based tool that interacts with Vector (a high-performance observability data pipeline) to query and retrieve real-time logs from running Vector-Dev instances. The project involved building HTTP clients to communicate with Vector's API, parsing structured log data (e.g., JSON or text), and displaying or storing the logs for analysis. Integrated features included real-time streaming, filtering logs based on conditions (like log level or source), and exporting logs to files or dashboards for monitoring and debugging purposes.

Key Technologies: Python, Vector.dev, JSON, Log Processing

2.Local Log Intelligence Agent for Vector-Dev using LLaMA 3 via Ollama

Built a local Python-based intelligent agent that connects to a locally running Vector-Dev instance to retrieve and validate real-time logs. The agent parses log events, validates Vector Remap Language (VRL) parser outputs, and automates the extraction of ECS (Elastic Common Schema) fields. It uses a locally hosted LLaMA 3 model via Ollama to generate clear, concise descriptions for each extracted field. The final output is a structured schema documentation to support observability and log analysis.

Key Responsibilities:

• Connected to local Vector-Dev instance to stream and process logs in real time.
• Validated VRL parser logic and output structure against expected schemas.
• Automatically extracted ECS fields and inferred data types from parsed events.
• Leveraged LLaMA 3 model (via Ollama) to generate meaningful field descriptions.
• Produced documentation in CSV format: field_name, data_type, description.

Key Technologies: Python, Vector.dev, VRL, ECS, Ollama, LLaMA 3, JSON, REST API, Log Analysis, LLMs

3.Building a Custom Observability Pipeline Using Vector.dev

Designed and deployed a custom observability pipeline using Vector.dev , a high-performance tool for collecting, transforming, and routing logs. The project involved setting up Vector as a local service , configuring sources, transforms (using VRL) , and sinks to process structured and unstructured log data. Successfully built reusable Vector configurations to ingest logs from file directories or network sources, apply custom parsing and enrichment, and output to file, console, or cloud destinations.

Key Responsibilities:

• Installed and configured Vector.dev as a local log processor.
• Created custom Vector configuration files with sources, transforms, and sinks.
• Used Vector Remap Language (VRL) for parsing, filtering, and enriching log events.
• Routed processed logs to files and other observability tools.
• Validated pipeline reliability and performance for real-time log flows.

Key Technologies: Vector.dev, VRL, YAML, JSON, Log Processing, Observability Pipelines, Local Deployment