Skip to content

Guides

Use a Vellum assistant as the model

The local Ollama quick start works because Ollama serves a real OpenAI-compatible /v1/chat/completions endpoint. A locally hatched Vellum assistant does not expose anything like that.

This tests Vellum's model, not its memory

The history is still pasted into the prompt here. To evaluate how Vellum remembers, see Evaluate a personal assistant.

Its closest equivalent is a synchronous, single-shot CLI/IPC call (assistant llm send, aka assistant inference send) with its own {message, systemPrompt?, model?, ...} shape, not OpenAI's {model, messages}. vf-run --client.base-url can't point at it directly.

scripts/vellum_adapter.py bridges the gap: a small, dependency-free (stdlib-only) local HTTP server that speaks OpenAI's /v1/chat/completions on the outside and shells out to assistant llm send --json on the inside, translating the response back into an OpenAI-shaped JSON body.

Scope — read before using: - Only convomem-arena's context memory mode is supported. assistant llm send has no tools parameter, so it cannot do real tool calls — don't point tools mode at it. - Each request blocks on one subprocess hitting one local daemon; there's no real parallelism behind this adapter. Run vf-run with a low --max-concurrent against it (e.g. -c 1 or -c 2), not the default 128. - assistant llm send does not necessarily use your instance's everyday chat activeProfile — it can fall back to a different, advisor-oriented profile that needs a connection you may not have configured (seen locally: it defaulted to an Anthropic profile and failed with provider_connection "..." not found, even though the instance's real chat profile was a working, keyless Ollama one). If you hit that, check assistant config get llm for a profile that IS configured and set VELLUM_PROFILE to its name — see below. - Verified working end to end — 6/6, mean reward 1.000 — using a local Ollama-backed Vellum profile (qwen3:30b-a3b) as both the agent (through this adapter) and the judge.

# 1. Have a hatched Vellum instance with a working model profile. A keyless local one is
#    easiest to get working for this, e.g.:
#      assistant config set services.inference.provider ollama   # only if you don't
#      assistant config set services.inference.model qwen3:30b-a3b   # already have one
#    then confirm which profile name is actually configured and keyless:
#      assistant config get llm   # look for a profile with "provider": "ollama"

# 2. Point the adapter at that instance and your vellum-assistant checkout:
export ASSISTANT_IPC_SOCKET_DIR=...     # same vars your shell already uses to talk
export VELLUM_WORKSPACE_DIR=...         # to the hatched instance (e.g. via a `vellum-env` helper)
export VELLUM_ASSISTANT_DIR=/path/to/vellum-assistant/assistant   # dir containing src/index.ts
export VELLUM_PROFILE=qwen-local        # the keyless profile name from step 1

# 3. Start the adapter (separate terminal, left running):
python3 scripts/vellum_adapter.py       # listens on http://127.0.0.1:8800/v1

# 4. Run convomem-arena against it. The adapter only stands in for the *agent* model —
#    you still need a judge endpoint (PRIME_API_KEY, or a local judge as in the Ollama
#    example in the quick start).
export VELLUM_FAKE_KEY=unused   # value is ignored; vf-run just needs the var set
export PRIME_API_KEY=...

uv run vf-run convomem-arena -n 6 -r 1 -m vellum -c 1 \
  --client.base-url http://127.0.0.1:8800/v1 --client.api-key-var VELLUM_FAKE_KEY \
  --env.agent.harness.id null --env.agent.runtime.type subprocess \
  --env.taskset.context-sizes 10