I think this space is very untapped. Models are interesting, but I am absolutely obsessed with some things I've been researching/working on for the past few years:

Fractal tool discovery: tool taxonomy where an agent can "drill deeper" to find what specific tool it's looking for. Helps if/when polluting context with a zillion (mostly unnecessary) tools.

Leveraging splay trees: this is my favorite data structure and I think relatively unused in the context of agents/harnesses. A lot of times, recently-used workflows/tool-chains will be used again, so having those at the top of the search hierarchy is an awesome optimization.

Virtual containerized notebooks: models working in sandboxed (WASI) Python notebooks is incredible. Even local models (if given enough time) will usually converge on a good solution. Being able to mount tools/resources/fs is again, imo quite untapped. Some problems here are running native things (thing numpy/pandas) in containers is a nightmare (or impossible).

Anyway, happy to see other folks seriously doing stuff in this space. If anyone wants to collaborate on anything don't hesitate to reach out :) I'm also actively looking for a job or some contract gigs.

Fun times ahead.

In my projects i use unreal and me and my team have tested out multiple things. We have settled on no MCP complications whatsoever other than simply exposing the python scripting from the editor + a export step that can write the blueprints and asset data into plaintext so that the bot can grep them. This has given by far the strongest results, and we now have no issues having the agents edit game code and do operations.

We found this massively outperforms any kind of agent like this and the official unreal MCP systems. Its similar to the Blender MCP which also just exposes scripting + very minimal api to claude code/others.

The headline graph is kind of bizarre.

For some reason they're comparing their harness running on Astra xhigh to Codex with Astra max?

---

Also worth noting that OpenAI just added support for async tool calling to their harness, which isn't 1:1 with this approach, but is slowly ramping up in being able to provide something similar.

A big part of why Codex uses so many tokens is that it basically hot loops on polling tasks it starts for... absolutely no good reason: https://www.reddit.com/r/codex/comments/1wdlp7q/weve_discove...

I fixed it on my fork of Codex too, also back in Jan/Feb – I keep this patch rebased, for anyone who wants it: https://github.com/tekacs/codex/commit/9ffcf8db9078eae43d411...

It results in token savings similar in scale to those displayed here by Unreal.

---

My harness has used a slightly fancier version of the approach that Unreal is using since ~Feb, and... it definitely works excellently, but it's also assuredly smoother with Astra and other recent models that are more aware of async tool calling.

If the frontier labs (well, I guess just Anthropic) would go full OAuth support even on a subscription we would see, an even bigger, explosion in harness improvements. I maintain that there is a ton of low-hanging fruit and new ideas/concepts that should be tried but the costs are holding people back (using API pricing only).

It's both expensive to test alternative harnesses and it's expensive to develop them (if using API pricing).

Sounds like a trademark issue when Epic ships a wildly popular Unreal Engine

Does it support subscriptions ? If not it's a no go for me.

I’m sad to discover that this is not a mod for Unreal to extend TacOps with a clandestine gameplay mode which prioritizes stealth and spy craft.

I think this is what I’ve been waiting for! Live interactions with models are fundamentally asynchronous! This will be great for interactivity.

What a good time for harness design. Just while OpenCode is growing up a bit and focusing on their harness. I’m delighted to see people focusing on good general solid harness design principles.

Someone make an OpenCode API (the best general agent end UI API I know) compatible server for it!

Sucks that I haven’t upgraded my client to V2 yet and I don’t really want to build on an outdated API ...

Edit: It would be cool to get streaming responses :) But I understand (and actually applaud) that the authors seem to have been very focused on the core mechanics.

Also note that this works with the responses API!

happy to see more harnesses in the Go space, i've had a fun time building https://github.com/ChaseRensberger/wingman

i think the language primitives map well to the natural desire to use these tools across networks and in concurrent workloads.

another Go harness i keep up to date with is: https://github.com/boldsoftware/shelley (exe.dev team)

I don't really understand how async tool calls translate to token savings

It says that it removes tokens wasted while a model is waiting on synchronous tool calls. What tokens exactly is Pi using when waiting?

that's like the worst name you could have picked

Why don't you explain what's different?

Honestly the fact that we're still modeling agent harnesses like chat and strapping them into shell sessions instead of building async actor systems with sandboxed OS functionality access actors is bananas to me. So much easy wins can be had just by building on right abstractions... Hopefully will get enough time to play with this idea soon on my own.

This feels like it's begging for a lawsuit from Epic.

Can I use it with my codex sub?

This is not an Unreal Engine agents? Very disappointed

omp.sh does this way better by just allowing structural toolcall execution in eval with python/js.

Happy with swival.dev ...

Mildly disappointed that this has nothing to do with Unreal Engine, the popular game engine.