# Spatial Video

[English](README.md) | [简体中文](README_zh.md)

A reference application for PICO OS 6 demonstrating best practices for spatial video playback with the PICO Spatial SDK, showcasing 3D video rendering, immersive playback, and Spatial UI components.

## Overview

Spatial Video enables immersive video playback in a spatial environment with interactive controls and immersive scenes. It serves as a practical guide for implementing video playback, Spatial UI design, and immersive media experiences using the Spatial SDK.

![Screenshot](.github/screenshot.png)

## Features

Spatial Video Sample highlights the following core capabilities of the PICO Spatial SDK:

- **Immersive Video Playback**: Side-by-Side (SBS) stereoscopic video in 3D with 4K/8K support
- **Dual Display Modes**: Planar video panel and immersive 360° sphere modes
- **Interactive Controls**: Play/pause, seek, volume adjustment, and immersive mode controls
- **Spatial UI Design**: 3D UI elements following PICO design guidelines
- **Advanced Video Management**: Playback state handling, progress tracking, and lifecycle management
- **WindowContainer & Stage Management**: Spatial layout management using PICO's spatial containers

## Requirements

To build and run this project, configure your development environment by following the official PICO Spatial SDK setup guide:

- [PICO Spatial SDK Setup Guide (Outside Chinese Mainland)](https://developer.picoxr.com/document/spatial-sdk/set-up-development-environment/)
- [PICO Spatial SDK Setup Guide (Chinese Mainland)](https://developer-cn.picoxr.com/document/spatial-sdk/set-up-development-environment/)

## Getting Started

### 1. Clone the Repository

```bash
git clone <repository-url>
cd SpatialVideo
```

### 2. Open in Android Studio

Launch Android Studio, select **Open an Existing Project**, and choose the `SpatialVideo` directory.

### 3. Run the App

#### On Emulator

1. Open **Device Manager** (toolbar or **Tools > Device Manager**)
2. Click **Add a new device** > **Create a PICO Emulator**
3. Start the emulator from **Device Manager**
4. Click **Run 'app'** to launch the app

#### On Physical Device

1. Connect your PICO device to your development machine
2. Enable **USB Debugging** on the device
3. Click **Run 'app'** to deploy directly to your device

## Project Structure

Spatial Video follows a modular architecture optimized for video playback applications:

```txt
SpatialVideo/
├── app/                                  # Main application module
│   └── src/main/
│       ├── java/com/pico/spatial/sample/spatialvideo/ # Kotlin source files
│       │   ├── Main.kt                     # App entry point
│       │   ├── data/                       # Data and state management
│       │   │   ├── PlaybackManager.kt      # Video playback control
│       │   │   ├── PlaybackState.kt        # Playback state definitions
│       │   │   ├── VideoAssetBundle.kt     # Video asset management
│       │   │   └── VideoEffectManager.kt   # Video effect control
│       │   ├── di/                         # Dependency injection
│       │   │   └── VideoModule.kt          # Video-related DI module
│       │   ├── ecs/                        # ECS for video panels
│       │   │   └── VideoEntityAssembler.kt # Video entity creation
│       │   ├── platform/                   # Platform-specific implementations
│       │   │   ├── LaunchActivity.kt       # Android launch activity
│       │   │   └── SpatialApplication.kt   # App initialization
│       │   └── ui/                         # UI components
│       │       ├── ImmersiveScene.kt        # Immersive 3D scene
│       │       ├── PlaybackToolbar.kt       # Playback control toolbar
│       │       ├── SpatialVideoScreen.kt    # Main video screen
│       │       └── VideoViewModel.kt        # ViewModel for UI state
│       ├── res/                            # Android resources
│       └── AndroidManifest.xml             # App manifest with Spatial component declarations
├── editor-asset/                         # Editor assets for spatial development
│   └── src/main/res3d/videoSample/       # Spatial Editor project
│       ├── Build/                         # Built asset bundles
│       ├── Sources/                       # Source assets and scenes
│       │   ├── Assets/                    # Textures and 3D models
│       │   ├── Components/                # ECS components
│       │   └── Scenes/                    # Scene definitions (USDZ files)
│       └── ModelView                      # 3D model views
├── gradle/                               # Gradle dependencies and scripts
├── build.gradle.kts                      # Root build configuration
└── settings.gradle.kts                   # Project settings and SDK configuration
```

## Editor Assets & Shader Graph Materials

The `editor-asset/` directory contains Spatial Editor project files and shader graph materials that define advanced video effects:

### Shader Graph Materials

The application includes three specialized shader graph materials:

#### 1. DefaultVideo (`DefaultEffect.usda`)

Basic video playback with standard rendering. Uses simple texture sampling and conversion as the default material for normal video playback scenarios.

#### 2. PortalEffectMaterial (`PortalEffect.usda`)

Creates a portal-like video projection effect with a geometry modifier. Key features include ray-sphere intersection calculations for 3D projection, dynamic UV generation based on view angle and position, and customizable parameters (sphere radius, FOV, distance scale).

#### 3. StereoMaterial (`StereoEffect.usda`)

Handles stereoscopic video rendering for immersive experiences. Key features include left/right eye detection and switching, video mask blending using `blurMask2.png`, IPD adjustment for 3D effect, and advanced UV warping based on viewport depth and camera position.

### Customizing Shader Graph Materials

To modify or customize the shader graph materials:

1. Open `editor-asset/src/main/res3d/videoSample/videoSample.spatialproject` in Spatial Editor
2. Navigate to the **Scenes** folder to access the USD files
3. Edit materials using the Shader Graph editor
4. Rebuild the asset bundle for use in the application

## 3D Assets

The application uses the following additional 3D assets located in `editor-asset/src/main/res3d/videoSample/Sources/Assets/`:

- **sphere.usdz**: 3D sphere model for immersive 360° video projection
- **blurMask2.png**: Alpha mask for smooth video edges and transitions

## Video Source

This sample streams a Side-by-Side (SBS) stereoscopic video from a remote HTTPS URL.

- Default source for PICO Emulator: `https://lf-devtools.picoxr.com/obj/spatial-toolbox/examples/cave_4096x2048_sbs.mp4`
- Higher-resolution source for physical devices: `https://lf-devtools.picoxr.com/obj/spatial-toolbox/examples/cave_8192x4096_sbs.mp4`
- Network access is required for playback.

## Learning Resources

To deepen your understanding of spatial app development, explore the resources on our official developer websites:

- [PICO Developer Website (Outside Chinese Mainland)](https://developer.picoxr.com/)
- [PICO Developer Website (Chinese Mainland)](https://developer-cn.picoxr.com/)

## License

© 2026 PICO. The project's source code is licensed under the [Apache 2.0 License](LICENSE.txt). All assets, including textures, 3D models, audio, and others, are licensed via [Creative Commons BY 4.0](https://creativecommons.org/licenses/by/4.0/).

The streamed spatial video content used by this sample remains the exclusive property of its respective copyright holders and is licensed for **limited use within this sample project only**.

Restrictions:

- Any form of modification, editing, adaptation, or creation of derivative works
- Any reproduction, distribution, publication, or upload to external platforms or services
- Any commercial, promotional, or otherwise unauthorized use beyond the scope of this project

Use of these assets constitutes acceptance of the above restrictions. **All rights reserved by PICO.**

## Support

For questions, issues, or feature requests, please contact us through the [PICO Developer Support Portal](https://picodevsupport.freshdesk.com/support/home) and submit a ticket.

---

Developed as a reference for spatial application developers
