Rivard, Luke2026-04-302026-04-302026-04-302026-04-14https://hdl.handle.net/10012/23119We introduce NeuralOS, a neural framework that simulates graphical user inter- faces (GUIs) of operating systems by directly predicting screen frames in response to user in- puts such as mouse movements, clicks, and keyboard events. NeuralOS combines a recur- rent neural network (RNN), which tracks computer state, with a diffusion-based neural renderer that generates screen images. The model is trained on a large-scale dataset of Ubuntu XFCE recordings, which include both randomly generated interactions and real- istic interactions produced by AI agents. Experiments show that NeuralOS successfully renders realistic GUI sequences, accurately captures mouse interactions, and reliably pre- dicts state transitions like application launches. Although modeling fine-grained keyboard interactions precisely remains challenging, NeuralOS offers a step toward creating fully adaptive, generative neural interfaces for future human-computer interaction systems.engenaidiffusionlatent diffusionrnnlstmdeep learningworld modelsMaster Thesis