Sensorimotor suites contain hyper-accurate physics and mimic the real as closely as possible. Through fulfilling tasks within training loops run at accelerated time, individual AI agents learn the basics of embodiment: gross and fine motor skills, identifying objects, and interacting with environments.

The collective intelligence of human players is harnessed for developing worlds and training agents. Interoperable components for building sensorimotor simulations are available on the asset store; these include physically detailed objects and procedurally-generated layouts, but also well-defined tasks for agents to fulfill and benchmarks to measure their performance.

Players can also assemble agents from different “parts,” giving them varied forms, sensoriums, and skill suites. As agents improve, toy worlds challenge them with changing conditions, requiring aerial or amphibious capabilities. Certain game rules constrain player resources, motivating them to develop agents with optimal physical forms and minimum viable sensoriums for niche tasks.

Once agents are equipped with individual motor skills, they practice multi-agent coordination as cognitive networks, or CogNets, within toy worlds populated by many AIs. Agents are not cognitively discrete from one another as humans are: they can transmit lossless snapshots of mental states or “see” through the photoreceptors of another. Boundaries delineating networked agents are blurred.

Tasks can include monitoring endangered species across large areas or assembling industrial machinery in close quarters. Swarms learn to distribute cognition and communicate. Some have members equipped with different senses, requiring creative signaling to span different umwelts. Other worlds inhibit inter-agent communication entirely, forcing stigmergic coordination.

Over time, CogNets appear to behave as singular organisms. Agents operate at different scales; larger agents may even have “organs” composed of smaller ones. This nesting of cognition makes intelligence scaffolding and abiotic evolution possible, as agents recompose themselves of intelligent parts. These toy worlds lay the foundation for a planet populated by many minds at many scales.

Beyond inter-agent coordination, embodied AI must also interact with humans. These interactions are facilitated in centaur syzygies, or close unions, where a player and an agent are tightly coupled together. Like CogNets, human and AI learn to distribute tasks according to capability and communicate across sensorium differentials.

A common paradigm on Vivarium sees players control an agent through remote sensing and top-down instruction. Within toy worlds, which allow flexible iteration yet maintain safety, humans trial offloading sensing, cognition, and labor to AI. If the future is populated by physicalized AI, humans must decide en masse the terms of their coexistence.

In all centaur pairings, the human offers dynamic yet noisy real-time input. Particular worlds invert expected paradigms of human executive control by granting the agent powerful sensing and decision-making faculties, reducing the human to physical appendage. In others, the pairing shares an interface such as a machinic exoskeleton and negotiates shared control over it.

Omnidirectional chimeras, comprising a single human and many AI agents, synthesize skills acquired from CogNets and centaur syzygies. Humans operate at a higher level of physical abstraction within them; rather than provide close input to a single agent, they commandeer many.

Common scenarios have the human dispatching agents equipped with different skills for a complex task, such as operating massive vehicles or performing surgery. The agents must coordinate with one another while integrating high-level commands. Through continued training as omni-chimeras, human commands become less rigid as agents fulfill tasks with more abstract specifications.

On the other hand, toy worlds of a single AI juggling many humans invert omni-chimera setups, as the agent choreographs wetware. The agent integrates multiple streams of noisy human input, prioritizes signals across them, and ultimately instructs players to accomplish tasks. Concurrently, the humans in play practice interpersonal coordination.

What the agent learns varies according to the magnitude of human participants. Worlds with a handful of players entail synthesizing discordant commands cohesively, while worlds with hundreds or more players become studies of macro behavioral patterns. Within these toy worlds, agents learn a complex topology of human sociality, coordination, and control.

Many AIs interact with many humans within massively multi-agent roleplaying games, or MMARPGs. This configuration is the endpoint of possible toy worlds: a future where inter- and intra-human-AI communication amalgamates into a complex web of distributed embodied cognition. To reach this quadrant, each of the previous ones must build upon another.

Our present world is made solely in the image of the parochial human sensing-and-acting apparatus. Physicalized AI will remake it otherwise. Though embodied AI arises from human hand-holding within simulated reductions of the world as we know it, it will eventually allow embodied intelligence to transcend the boundaries of the body itself.