Visionary

Visionary: The World Model Carrier Built on WebGPU-Powered Gaussian Splatting Platform

Shanghai AI Laboratory · Sichuan University · The University of Tokyo · Shanghai Jiao Tong University · Northwestern Polytechnical University

TL;DR: Visionary is an open, web-native platform built on WebGPU and ONNX Runtime. Enabling real-time rendering of diverse Gaussian Splatting variants (3DGS, MLP-based 3DGS, 4DGS, Neural Avatars and ✨any future algorithms✨), and traditional 3d Mesh, directly in the browser. We also support post-processing using feed-forward networks.

Abstract

Neural rendering, particularly 3D Gaussian Splatting (3DGS), has evolved rapidly and become a key component for building world models. In this work, we present Visionary, an open, web-native platform for real-time various Gaussian Splatting and meshes rendering. Built on an efficient WebGPU renderer with per-frame ONNX inference, Visionary enables dynamic neural processing while maintaining a lightweight, "click-to-run" browser experience. It introduces a standardized Gaussian Generator contract, which not only supports standard 3DGS rendering but also allows plug-and-play algorithms to generate or update Gaussians each frame. Such inference also enables us to apply feedforward generative post-processing. The platform further offers a plug in three.js library with a concise TypeScript API for seamless integration into existing web applications. Experiments show that, under identical 3DGS assets, Visionary achieves superior rendering efficiency compared to current Web viewers due to GPU-based primitive sorting. It already supports multiple variants, including MLP-based 3DGS, 4DGS, neural avatars, and style transformation or enhancement networks. By unifying inference and rendering directly in the browser, Visionary significantly lowers the barrier to reproduction, comparison, and deployment of 3DGS-family methods, serving as a unified World Model Carrier for both reconstructive and generative paradigms.

We have developed a powerful Online Editor based on Visionary to help users easily manage and edit 3D scenes in one click. If you want to develop your own web project using this project, please refer to Quick Start.

visionary_demo.mp4

✨ Feature

⚡️ Native WebGPU Powered: Utilizes webgpu to achieve high-performance parallel sorting and rendering of millions of Gaussian particles.
🎨 Hybrid Rendering Architecture: Automatically handles depth mixing (Depth Compositing) between Gaussian point clouds and standard Meshes, perfectly solving occlusion issues and supporting complex scene compositions.
📦 Universal Asset Loader: Single interface to intelligently identify and load multiple formats:
- Static Gaussians: PLY, SPLAT, KSplat, SPZ, SOG
- Standard Mesh Models: GLB, GLTF, FBX, OBJ
- 4DGS/Avatar/scaffold-GS: ONNX
- 🔥Your own algorithom: See Export your algorithm to ONNX in detail.

News

We have updated a comprehensive demonstration for how to export your own customized onnx model: https://drive.google.com/file/d/1W8C1OINxDPUoiYtDi9KKz4AmwTcuVbKX/view

WebGPU Requirements & Known Issues

Browser: A recent version of Chrome (or Chromium-based browser) with WebGPU enabled is required.
Recommended platform: Windows 10/11 with a discrete GPU (NVIDIA / AMD) is strongly recommended for stable performance.
Ubuntu: Currently not supported. Due to a Chrome WebGPU bug, fp16 is not properly supported on Ubuntu, which breaks our fp16 ONNX pipelines. We will enable official Ubuntu support after the upstream issue is fixed.
macOS: GPU performance on most Macs is limited for heavy 3D Gaussian Splatting workloads. Unless you are using a high-end chip (e.g., M4 Max or better), we do not recommend macOS as the primary platform, as the viewer may run very slowly or stutter.

🚀 Quick Start

1. Install Dependencies

Ensure that Node.js (v18+ recommended) is installed in your environment.

# Clone the repository
git clone https://github.com/Visionary-Laboratory/visionary.git
cd visionary

# Install dependencies
npm install

2. Start Development Server

npm run dev

After successful startup, visit the following address to view the example: 👉 http://localhost:3000/demo/simple/index.html

Feel free to try all our demos.

3. Model Assets

You can import our provided example assets (1,Neural Avater + 4DGS), (2,LHM Example), (3,LHM Example2) or your own 3DGS/4DGS/Avater assets in the page. For details on creating 4DGS/Avater/Custom assets, see Convert to ONNX.

🛠️ Export your algorithm to ONNX

This project supports rendering of various 3DGS/4DGS/Avater/Custom representations. To achieve this, trained 3D representations need to be exported to the ONNX format. This project provides conversion examples for 4DGS/Dynamic Avatar/Scaffold-GS, see export instruction for details.

🤝 Contributions & Acknowledgments

This project is deeply inspired and supported by the following open source projects:

📄 Citation

If you use Visionary in your research or projects, please consider citing:

@article{gong2025visionary,
      title={Visionary: The World Model Carrier Built on WebGPU-Powered Gaussian Splatting Platform}, 
      author={Gong, Yuning and Liu, Yifei and Zhan, Yifan and Niu, Muyao and Li, Xueying and Liao, Yuanjun and Chen, Jiaming and Gao, Yuanyuan and Chen, Jiaqi and Chen, Minming and Zhou, Li and Zhang, Yuning and Wang, Wei and Hou, Xiaoqing and Huang, Huaxi and Tang, Shixiang and Ma, Le and Zhang, Dingwen and Yang, Xue and Yan, Junchi and Zhang, Yanchi and Zheng, Yinqiang and Sun, Xiao and Zhong, Zhihang},
      journal={arXiv preprint arXiv:2512.08478},
      year={2025}
}

📝 License

This project is licensed under the Apache-2.0 license.

visionary