Any-to-Any: The Unified Model Bet

The unified-model bet: every modality becomes a token stream — text as BPE, images as VQ codes, audio as codec tokens, video as spacetime latents — and one transformer does next-token prediction over all of it. Modality becomes a property of the token (and its stream markers), not of the architecture.

The payoff is genuine:

• Generation in any modality — text-in, image-out; audio-in, speech-out; image-and-text interleaved output
• Cross-modal reasoning flows through one representation — no adapter seam between 'seeing' and 'thinking'
• Emergent transfer — skills learned in one modality bleed into others through shared weights

This is the architecture frontier labs converged on for their omnimodels.