Browser Runtime
Step the latent dynamics model in-browser with learned action codes. Each step corresponds to roughly five frames of underlying continuous control.
Past context from the dataset and the sequence you choose here.
The latent state is decoded after each dynamics step.
From the first frame, nearly every button sends the blue controller directly toward the T. That is not because the buttons are meaningless. It is because the dynamics model has absorbed a very strong regularity from the training data: the blue controller usually needs to get to the T before it can do anything useful to it.
So in the data there are very few examples of the controller just wandering around aimlessly away from the T. As a result, the earliest step is dominated by this shared “go to the T” behaviour across almost all learned action codes.
After that first move, the buttons begin to separate into more distinct semantic roles. Once the controller has reached the T, the training distribution contains many more possibilities: pushing it, moving around it, approaching it from different sides, or setting up for the next contact. That is where the learned codes start to look less like a single attractor and more like reusable action types.