JEV as a Memory Relevance Scorer
The problem was not memory. It was memory navigation.
Long-running agents eventually remember too much. In my case, the problem became visible at an absurd scale: the local agent runs on a Qwen 3.8 27B with a large context window, and one session recently crossed 200 million tokens. Long-term memory is spread across dozens of Markdown files, while only a small fraction is relevant to any given task.
My first instinct was to treat JEV as a judge or a second brain. That turned out not to be the most useful interpretation. The more interesting question was smaller: given the context the agent is about to continue from, which memories would be useful to consult?
I wasn't looking for a better judge. I was looking for a cheaper way to tell my agent where to look.
I wasn't looking for a better judge. I was looking for a cheaper way to tell my agent where to look.
The resulting architecture is deliberately narrow: Pi-VCC describes the current state of the work, JEV ranks the available memory blocks, and the main agent decides which files it actually wants to read. Nothing is injected automatically.
From JEV-like classification to memory ranking
Before using JEV this way, I needed to know whether the mechanism was viable on the model that already runs my agent: a local Qwen 3.8 27B in Unsloth's Q4_K_XL dynamic quantization — about 5.1 bits per weight effective, a per-tensor mix of Q5_K, IQ4_XS, Q4_K and Q6_K rather than a stock Q4_K — running through llama.cpp.
This distinction mattered. On the FEVER out-of-sample run, the local model and JEV both struggled with the benchmark's task distribution, while the local confidence still behaved as a useful escalation signal. Agreement between the local and JEV three-way decisions was 92.4%, and the local contradiction probability tracked JEV with a Pearson correlation of 0.9209. The lesson was not “Qwen replaces JEV”. It was “Qwen can provide a useful decision signal.”
The memory query is intentionally boring
The JEV prompt does not ask for facts from the memory. It does not ask for a summary. It does not ask the classifier to verify anything. It asks one thing:
At compaction, the input is simply the raw Pi-VCC output plus a lightweight JSON index describing each memory block; at cold start — before any compaction exists — the input is the user's own message. The contents of the Markdown files are not provided to JEV in either case.
Pi-VCC becomes the query
This is the part that made the whole design click. Pi-VCC already produces a structured representation of the session — including the session goal, files and changes, commits, and recent context. In one observed session it compacted roughly 191,560 tokens into a much smaller state. That output is already a description of what the agent is doing. There is no obvious reason to place another LLM between VCC and the classifier.
For each memory, JEV gets the same shared VCC context and a separate relevance question. The output is metadata only:
The important part is what does not happen next. The system does not automatically open the top five files. The ranking is appended as a hint for the main agent. The model can read the high-scoring files, ignore them, or decide that a medium score is enough to justify a lookup.
JEV answers “where might useful context be?” The main agent still answers “what do I actually need to read?” The ranking is injected as an invisible hint message — the classifier never becomes the owner of memory.
The cache constraint changed where JEV belongs
There was one more design constraint hiding underneath the idea. With a stateful local model and a KV cache, adding memory after the model has already started is expensive: you risk invalidating the prefix you just computed.
The result is two clean retrieval points rather than a classifier running all the time: cold start on the first user message, and context rebuild after a VCC compaction.
What the system actually does
Pi-VCC already contains the current task, recent changes, and enough structure to describe what the agent is trying to continue. No second summarizer is required.
JEV scores short descriptions of the available memory blocks against that context. The result is a small relevance vector rather than a forced retrieval decision.
The main Qwen still decides which files to open. The classifier only makes the search space easier to navigate.
This is also where the local implementation becomes practical. An initial 24-branch implementation took about 2 minutes 30 seconds because the local CLI was recomputing the shared prefix of every branch. Shared-prefix reuse via an anchor reduced the same workload to about 28 seconds. The production shape — 25 memory blocks scored against a real session state — runs in 14–22 seconds, the anchor phase itself in 1.7–5.4 seconds; the first real post-VCC run scored all 25 blocks in 17.4 seconds.
That is not free, but it is a very different cost profile from invoking a classifier on every turn. A memory relevance pass only needs to happen when the context is constructed or reconstructed.
Timeline
The local Qwen 3.8 27B reproduced the JEV-style canary directions 6/6 under the same protocol, making a local classifier path worth testing.
The benchmark ruled out a simple “calibrated local drop-in replacement” interpretation, while preserving strong local↔JEV decision agreement and a useful confidence signal.
The first real post-VCC ranking surfaced the memory areas most aligned with the live session and pushed the remaining candidates close to zero.
The relevance index was integrated as a sidecar hint, with local scoring and lazy memory reads rather than forced retrieval.
I made it easier for the agent to find what it already remembers.
The experiment is still running. I have not established a general retrieval benchmark across hundreds of unrelated sessions yet, and the relevance scores should not be confused with calibrated probabilities of “memory usefulness”. The next useful measurement is simple: across real VCC checkpoints, does the top-k ranking consistently predict the memories the agent actually ends up reading?
This article separates three things: controlled classifier experiments (ANLI and FEVER), live canaries and workflow tests, and the current memory-ranking prototype. The local model is Qwen 3.8 27B in Unsloth's Q4_K_XL dynamic quantization (about 5.1 bits per weight effective; per-tensor mix of Q5_K/IQ4_XS/Q4_K/Q6_K) running through llama.cpp. Benchmark protocols were frozen where stated; the FEVER result is treated as out-of-sample evidence rather than as a tuning set. The memory-ranking experiment is explicitly presented as an ongoing engineering test, not as a general retrieval benchmark.
REFERENCES
- featherless-ai/simple-jev — JEV-style classifier implementation and shared-prefix scoring design.
- sting8k/pi-vcc — the session compactor that produces the structured state JEV scores: pure extraction and formatting, no LLM calls, transcript-preserving summaries.
- JEV Decision Index — reproduction tracker and benchmark context used for the initial experiment.