I have experimented with this type of modeling for a RTS game I'm building: https://playwarpact.com

Few things:

1. Opus 5 is just as good for this tbh and cheaper.

2. They don't generate optimized 3d models. They have high poly count for simple geometries.

3. A better approach I've utilized for game ready assets is to use the model to create low poly silhouettes of the 3d model and then bake textures that include a lot more details like windows, doors etc with tools like Meshy. Can post a tutorial if there's interest.

The models need some more RL to be able to do this autonomously.

These things always look impressive but I fail to see how they are usable beyond demos or simple games. In my experience the topology is often a mess and texturing is difficult as well

This is the kind of demo that would really benefit from a youtube demo of 2-3 min

Fake edit: there is a longer video here 1 min long: https://github.com/PhiloLabs/fable51-worlds/blob/main/union-...

I would be especially curious to see the NPC person/car logic and if they're on rails or what, that's a pretty good NPC density for a demo.

This isn't what I expected when I heard "world model"!

I wish you would provide more information. About time, costs, reliability (how many duds did you have? did you need to do any babysitting or etc) and so on But it is a very impressive looking demo, for sure.

Fundamentally it’s an image model predicting out frames of a sequence.

It’s not enough to call it just an image model though. Not quite video either because it’s so specific to rendering images of the world.

Thus, “world model”?

The problem I have with “world” is it should imply so much more depth. To me this is actually a “POV image model” or “First-person perspective model” (maybe we call this an FPM).

There could also be TPMs (third-person models).

This maps better to modern game and 3D dev phrasing, where “world model” might imply 3D object collections spatially organized in a cohesive file format - complete with interactivity, audio, and all the baseline constituents of what one might call a “world”.

FPMs and TPMs, think about it. Reserve POVMs for photographic realism.

This is beautiful! I would love if we could model out a whole city, ideally using a much cheaper model.

This reminds me of the experience VRML wanted to be 3 decades ago. The only difference is that links went to other worlds living on another server. But that seems trivial to implement with this.

Cool demo! That's funny. I literally sketched out something similar 30 minutes ago on paper. I wonder if there was some recent, semi-related article/link that subconsciously prompted the relevance.

Be nice to turn some of these into Counter Strike maps... mhmm...

the 'most ai games want me to sign in' bit is the real world model. the game world is fine, it's the account creation funnel that's fully realized.

this is really neat. I wonder if people will start building open world games or "AR" games in the vein of Pokemon Go and Ingress based on similar tech. I've had a similar idea for a long time but don't think it was feasible before now due to AI

Started the Matrix

This looks great.

Very good

“Fable 5.1 World Modeling” is an incredibly misleading title! This has very little to do with Fable and using an unexpected sense of “world model”

Cool. But the question remains, what is the point of this? I don't mean to pick on this project specifically, but it's indicative of AI usage in general. You end up with something that's like "wow an AI made this" but it's not really anything that interesting or useful in and of itself. I've yet to see a single agent generated piece of software above that bar.

[flagged]

fable 5.1 generated an interactive 3D union square, and the agent filmed its own tour guide vid inside it. you can walk Powell to Stockton, read the actual storefronts, cross a working intersection, watch a cable car go by. went inside Apple and the Nintendo store, lower level included

check source code + more worlds soon

[dead]

I shitpost on this website constantly and trash Anslopic on a daily basis.

I am your biggest hater.

But god dammit if I’m not also your biggest respecter.

reaches out for a handshake

You guys are onto something with this one.

Keep going.

You crazy bastards!

You crazy fuckin bastards you hear me?! Hahahahah

WOO!

WOO!