Selective guideline retrieval allows artificial intelligence agents to learn from past trajectories while consuming significantly fewer tokens than full-playbook memory systems, according to technical details published on Hugging Face by researcher Ashwath Vaithinathan Aravindan on August 11, 2026.
The report compared ALTK-Evolve against Agentic Context Engineering (ACE) using identical base ReAct agents across multi-step software tasks. Both systems function as agentic memory, converting previous execution histories into reusable lessons without needing human feedback labels or parameter weight updates. Both architectures explicitly decline to summarize lessons down to short rules, preserving exact support counts to track how many independent episodes produced each guideline.
The systems differ in how they deliver stored experience to an agent at inference time. ACE sends its complete, itemized playbook on every execution step regardless of task complexity. ALTK-Evolve delivers a fixed core of high-support guidelines alongside a small set of task-specific rules chosen through priority-weighted cosine or model-guided retrieval.
AppWorld Benchmark Performance
In-house evaluations on the 168-task AppWorld benchmark showed distinct token efficiency differences between the two delivery methods. Paired with DeepSeek-V3.2, ALTK-Evolve achieved an 89.3 percent Task Goal Completion (TGC) rate and an 80.4 percent Scenario Goal Completion (SGC) rate while using 263,000 tokens. ACE recorded 80.4 percent TGC and 73.2 percent SGC while consuming 634,000 tokens on the same model.
When evaluated on gpt-oss-120b, ALTK-Evolve recorded 56.0 percent TGC and 37.5 percent SGC using 116,000 tokens. ACE achieved 54.8 percent TGC and 35.7 percent SGC while using 777,000 tokens.
ALTK-Evolve builds its store by clustering near-duplicate lessons and merging them while conserving support counts, ensuring surviving guidelines inherit combined instance records. It extracts strategy, recovery, and optimization rules with causal attribution and direct provenance links back to source trajectories at subtask granularity.
Task difficulty breakdowns revealed that ACE maintained higher goal completion on easy and medium tasks when paired with gpt-oss-120b. However, ALTK-Evolve won the hard task tier on that model, securing the overall aggregate completion lead while injecting fewer tokens into the model context.
