Skip to main content

Getting Started

Overview​

WebSpatial is a set of minimal extensions to HTML/CSS/DOM APIs plus a polyfill-style open-source SDK. It brings spatialized UI capabilities equivalent to native spatial apps and a 2D containing 3D developer experience into Web standards and mainstream Web frameworks. This allows HTML content on spatial computing platforms to break free from the screen, enter real space, gain real volume, support natural spatial interactions and flexible 3D programming, while preserving the Web's original cross-platform nature, mental model, and development workflow. The goal is to let the mainstream Web ecosystem and Web developers move seamlessly into the era of spatial computing and multimodal AI.

Features​

WebSpatial API​

  1. Spatial scenes: The start page of a Web App (PWA), and every page of the same app opened in a new window, becomes a spatial scene container integrated with the surrounding space. Different initialization settings can be applied to these containers.
  2. Material backplates: For webpages using the window scene type, the background panel can use a native-feeling translucent material rendered dynamically based on viewpoint and environment, or it can be made fully transparent with no visible border so that elements appear to float separately in space.
  3. Volumetric windows: A webpage window can change its behavior in space from serving 2D GUI needs first to simulating a real-world object, so the window behaves like a "box" with real volume and depth.
  4. Spatialized HTML elements: HTML elements can be "lifted" into the 3D space in front of the page plane while still participating in the CSS layout system. These spatialized HTML elements keep their original state and APIs on the X and Y axes, while also becoming 2D planes floating in a spatial scene. They can have material backplates, be laid out and positioned along the Z axis with CSS, receive 3D transformations such as rotation, and expose related state through DOM APIs.
  5. Material backgrounds: The background of an HTML element can use a real-time rendered translucent material, instead of being limited to fixed colors.
  6. 3D container elements: Two new kinds of 3D HTML elements act as containers for 3D content with real volume. These 3D container elements still participate in the CSS layout system as 2D planes, support Z-axis layout and transforms, and also create a local space in front of the 2D plane based on a 3D development paradigm. Real volumetric 3D content can be rendered there so it fits into the 2D layout system and the rendering model of 2D GUI frameworks, enabling a 2D containing 3D development paradigm.
  7. Static 3D container elements: The 3D content inside a container can be rendered from prebuilt 3D model asset files. The API for this kind of 3D container element is fully based on the Web standard model element.
  8. Dynamic 3D container elements: The 3D content inside a container can be rendered dynamically through a flexibly programmable, HTML-style 3D engine API.
  9. HTML-style 3D engine APIs: These APIs include 3D asset declarations such as models and materials, built-in capability modules, and ready-to-use 3D entities such as primitive shapes. These entities can be freely composed in a 3D coordinate system using a tree structure and Transform props, enabling arbitrary 3D scenes and animations.
  10. 2D attachment entities: 2D HTML content can be attached to plane-shaped 3D entities, so fully functional 2D content can also be embedded inside 3D content.
  11. Spatial interactions: New spatial interaction events such as tap, drag, and rotate can be triggered on the 2D planes corresponding to spatialized HTML elements and on 3D content inside 3D container elements (on 3D mesh surfaces or bounding boxes), providing interaction information in 3D space such as positions in a 3D coordinate system.
  12. Mixed 2D + 3D content: 2D content based on the CSS layout system and dynamic 3D container content based on the 3D engine can be combined through APIs such as coordinate conversion and unit conversion, enabling alignment, coupling, and other integration patterns.

WebSpatial SDK​

  1. Forward-looking pre-implementation: Together with a native runtime implementation, the proposed HTML/DOM/CSS APIs are polyfilled (or "prollyfilled") within the JSX, refs, and CSS of React projects. This makes the WebSpatial API available immediately, without waiting for browser engines on each platform to formally support these APIs.
  2. Cross-version compatibility: The SDK shields developers from instability, changes, and platform differences while WebSpatial APIs enter Web standards (HTML/CSS/DOM). The APIs exposed by the SDK remain backward compatible, so older code keeps working.
  3. Cross-platform compatibility: On platforms that support spatial computing and unified rendering, platform differences are hidden as much as possible so apps get unified spatial app concepts and spatialized UI features. On platforms that do not support spatial computing and unified rendering, the SDK will automatically ignore WebSpatial APIs, skip loading the full SDK implementation, and avoid affecting the behavior or performance of webpages in ordinary browsers on desktop computers, phones, and other screen-based devices.
  4. Custom cross-platform logic: The SDK provides feature detection and runtime detection, so developers can add custom cross-platform handling for the small number of JS API / DOM API calls that can't be ignored automatically, and enable custom enhancements and platform-specific features on spatial computing platforms.
  5. App packaging: A PWA can be packaged as a native app bundle with its own WebSpatial Runtime and no external dependency, such as a visionOS app. Like a native app, it can be installed on simulators or real devices for preview and debugging, and submitted to app stores such as the visionOS App Store.

Philosophy​

  1. WebSpatial adopts a new "2D containing 3D" development paradigm. It is built on the 2D Web ecosystem and HTML/CSS APIs, continuing the 2D mental model and development style developers already know. It only adds Z-axis APIs for 2D elements, plus APIs that let a local 3D space behave like a 2D element. The 3D development paradigm is confined inside local 3D spaces, and those local 3D renderings are integrated into the overall 2D rendering model. Developers can use 3D capabilities only where needed, instead of rebuilding an entire app as a 3D app and disconnecting from the mainstream 2D ecosystem.
  2. WebSpatial only makes minimal extensions to HTML/CSS/DOM APIs. For X/Y-axis-related functionality, it continues to use existing Web standard APIs, which can be combined with the newly added Z-axis-related APIs.
  3. Inside local 3D spaces, WebSpatial avoids doing independent rendering with low-level 3D graphics APIs as WebXR does. Instead, it uses a declarative 3D engine API that combines ECS and HTML, so spatial computing platforms can understand this 3D content and render it together with content from other apps in the same space.
  4. WebSpatial avoids implementing all content for a spatial app inside a single webpage the way a WebXR session does. Instead, it preserves standard Web development patterns such as multi-page sites, hyperlinks, and PWAs, and uses the Web App Manifest and "open links in a new window" to provide spatial app and spatial container capabilities.
  5. The SDK should keep its simulated pre-implementation of the proposed HTML/CSS/DOM APIs moderate, not overly complex or uncontrollable. For that reason, modifying spatial styles/state and listening to spatial events is only supported in declarative code that follows the Rules of React, and reading spatial styles/state (read-only) is only supported on objects obtained through Hook APIs, such as Refs. Directly selecting DOM objects with imperative code is not supported for these operations.
  6. The SDK should integrate into existing standard Web projects with as little cost as possible, in a way that is close to one-click installation, without changing the project's original development, build, or deployment workflow, and while ensuring that the site's existing behavior, performance, and debugging experience on desktop/mobile platforms and ordinary browsers are not affected.
  7. With WebSpatial APIs and the SDK, building a new spatial app should feel just like building a normal website. If developers want, the app can still be distributed as a standard website, preserving the Web's original cross-platform capability and its URL-based usage model.

Supported Web Projects​

The open-source WebSpatial SDK currently provides a React SDK. Any standard website project built with React should be able to enable the WebSpatial API by integrating this SDK.

The React SDK needs to be integrated into the JSX Runtime. In React projects that use TypeScript, this requires only one line of configuration (a few web build tools need extra configuration). In JavaScript projects, you can only configure the JSX Runtime in the web build tool. The web build tools that have been tested and used most often so far are:

Current limitation

The latest version of WebSpatial SDK currently has a bug that temporarily prevents support for styled-components.

Supported Platforms​

The spatial computing platforms currently supported are:

Spatial computing platforms currently working toward WebSpatial Runtime implementations are:

Spatial computing platforms planned for priority support, but currently waiting for stable native APIs, are:

Installation​

Step 1: Runtime SDK​

To enable the WebSpatial API in a React project, install the React SDK and the underlying Core SDK provided by the WebSpatial SDK project:

npm install @webspatial/react-sdk @webspatial/core-sdk

Step 2 (Optional): Builder​

For spatial computing platforms that do not have a built-in WebSpatial Runtime, the website must be packaged as a native app that includes WebSpatial Runtime. That requires these additional installation steps:

PICO OS 6 does not need packaging

The Web App Runtime in PICO OS 6 already includes WebSpatial Runtime and can make the WebSpatial APIs on a website work directly, so packaging is not required and none of the extra installation steps below are needed.

  1. Install the packaging tool (WebSpatial Builder) provided by the WebSpatial SDK project in the React project:
npm install -D @webspatial/builder
  1. The WebSpatial Runtime implementation for each platform is provided as a separate npm package and must be installed independently. For example, to support visionOS, install:
npm install -D @webspatial/platform-visionos
  1. To generate native app bundles, WebSpatial Builder must invoke the native development tools for the target platform. For visionOS, that means you must install Xcode and visionOS-related components.

Set Up Your Project​

Step 1: JSX Runtime​

Before using the WebSpatial API, integrate the React SDK into the current React project's JSX Runtime.

For TypeScript React projects, you only need to configure the jsxImportSource field in tsconfig:

tsconfig.json
{
  "compilerOptions": {
    "jsx": "react-jsx",
    "jsxImportSource": "@webspatial/react-sdk",
// ...
Related setup guides

Step 2: Minimal PWA​

To provide the app information required by a spatial app and the settings for the start window, the current site must provide a Web App Manifest following the PWA standard.

Sample files

webspatial-icon-examples.zip contains icon files and manifest file that already satisfy the PWA requirements. You just need to unzip it into a directory on your website that can be publicly accessed via URL, then embed the manifest file's URL in your HTML with <link rel="manifest" href="/app.webmanifest" />.

Already a PWA?

If the current site is already a PWA and can be installed as a PWA in Chrome, you can skip this step.

Boilerplate​

If you want to try WebSpatial quickly, use the WebSpatial Starter repository. It is a project template based on React + TypeScript + Vite, already integrated with WebSpatial SDK, and includes documentation optimized for coding agents so AI can quickly help add WebSpatial features.

Preview​

To preview and debug WebSpatial effects, first run the current Web project as a website just as you would in normal website development, and get an accessible URL. For example, using the Dev Server from Vite:

Start the local dev server
vite dev
# result:
# -> Local: http://localhost:5173/

Website Mode​

For spatial computing platforms with a built-in WebSpatial Runtime (for example, PICO OS 6):

Just visit the current website URL in the PICO Browser on the official simulator or on a real PICO OS 6 device, then click the "Run as a standalone app" button in the address bar - the website will run as a Web App with WebSpatial enabled, and you can see the spatialized UI effects and volumetric 3D content produced by WebSpatial.

PICO OS 6 simulator networking

When previewing a local URL as a Web App on the PICO OS 6 simulator, the URL is not required to use HTTPS, and you need to use the default IP address 10.0.2.2. For example, http://10.0.2.2:5173/.

To access the Vite dev server via 10.0.2.2, you need to set server: { host: true } in Vite so it allows IPs beyond localhost or 127.0.0.1.

Packaged App Mode​

For spatial computing platforms without a built-in WebSpatial Runtime (for example, visionOS):

You cannot preview and debug directly from a website URL. You need WebSpatial Builder:

Running Builder's run command gives you a "one-click preview" workflow. It automatically packages the current website as a native app bundle that includes WebSpatial Runtime, launches the official simulator (for example, the visionOS simulator), transfers the app bundle into the simulator, installs it, and launches the app.

Preview the app in the simulator
webspatial-builder run --base="http://localhost:5173/"
run can use a temporary manifest

The run command allows the Web project to skip providing a Web App Manifest during the early development and debugging phase. In that case, Builder will provide temporary default app information (start_url is '').

Offline packaging slows iteration

If start_url is incomplete and missing the domain, and you do not use --base to fill in the base part for start_url, Builder will automatically package the website output into the native app bundle. During runtime, website files are loaded offline from local files inside the app bundle.

During development and debugging, this mode means every code change requires rerunning Builder and waiting for packaging and installation to finish, which is inefficient. It is recommended to always provide the --base parameter during development and debugging.

To preview on a personal test device, run Builder's build command to generate an app bundle. For visionOS devices, you need to obtain extra parameters from App Store Connect, register the test device in Xcode, and install the app.

Debug​

Whether you run on a simulator or a test device, whether you access the website URL directly or package and install the website as a standalone app with WebSpatial Builder, you can remotely connect from DevTools in a browser on your local computer to the Web Runtime in the spatial computing environment for debugging.

WebSpatial apps on visionOS are implemented on top of WebView. You can use Safari Web Inspector on macOS to remotely debug the state and Web code inside each WebView instance, just as you would when debugging webpages on iOS devices. See Apple's official documentation: Inspecting iOS and iPadOS

For Android-based spatial computing platforms such as PICO OS 6, you can use Chrome DevTools on a computer to remotely debug Web Apps with WebSpatial enabled. See Google's official documentation: Remote debug Android devices

Distribution​

There are two ways to distribute a Web App that includes WebSpatial:

The first is to distribute it as a cross-platform website, spread it through URLs, and get traffic that way. On spatial computing platforms with a built-in WebSpatial Runtime, it can be accessed on demand by URL, used temporarily, or installed onto a device as a PWA.

Web Apps on PICO OS 6

On PICO OS 6 devices, a Web App can also run independently outside the browser without being installed, and WebSpatial effects are enabled automatically.

The second is app-store distribution, where users discover the app in an app store, install it first, and then use it.

Publishing to the visionOS App Store requires using the publish command from WebSpatial Builder to generate the app bundle and upload it automatically to App Store Connect. This requires some extra parameters and registration steps.

The PICO app store supports publishing a Web App directly by submitting a URL, without packaging and uploading it through WebSpatial Builder first. However, at the current stage, the PICO developer platform does not yet provide self-service Web App submission, so Web Apps must be published to the PICO app store through PICO's business partnership channels.