AI Agent Systems: Memory and Evaluation
Follow state, attribution and evaluation through small agent systems.
Start with state and failures
BotSim introduces a small local community. The orchestration article follows streamed events, failures and cancellation. Both distinguish deterministic test providers from live language models.
Test coordination
The swarm experiment measures neighbour coupling in a tiny grammar. Mixture of Perspectives separates hard constraints from preference disagreements. Neither fixture establishes general intelligence or better decisions in the world.
Evaluate revision and training
Read self correction before adding repeated answer revisions, then self rewarding training for preference loss and independent evaluation. The Minecraft article is a personal game building account, not a model benchmark.
Articles in this reading path
- BotSim: A Local AI Agent Community SimulatorExplore scheduling, memory and restart behaviour in a small local agent community, with deterministic fixture checks kept separate from live model behaviour.
- Multi Agent Orchestration: Streaming, Failures and ReplayTrace streamed output, cancellation and injected failures through a replayable orchestration fixture before evaluating real model answer quality.
- Swarm Sequence Generation: Neighbour Coupling TestsTest neighbour coupling in a small finite grammar, with baseline coverage and clear boundaries around untested communication and routing proposals.
- Mixture of Perspectives: Decision Support with DisagreementSeparate hard constraints from competing preferences and retain useful disagreement in a synthetic decision fixture with explicit utility assumptions.
- LLM Self Correction: Evaluation and StoppingEvaluate helpful and harmful answer revisions, frozen stopping rules and baseline competence before drawing conclusions about LLM self correction.
- Self Rewarding Training: DPO and EvaluationCheck preference loss direction, answer masks and judge bias, and distinguish objective tests from a toy training run that failed to generalise.
- Building a Minecraft Style Browser Game with AstraA personal browser game experiment with one playable original, successive screenshots and notes on the Astra and Codex revisions.