Vision
Images, video, capture, and models on one GPU-first path.
OA Vision combines image operations, Vulkan Video decode and encode, timestamped capture, media containers, and direct oa::Matrix interoperability.
Rust · PythonVideo sessionsCapability-gated codecs
The frame contract
oa::VideoFrame carries the image or buffer resource, format, dimensions, presentation timestamp, color metadata, and GPU completion token. Decode, capture, processing, display, and encode use that same contract instead of copying through an intermediate CPU frame abstraction.
Image operations
The public inventory is generated from Vision schemas; complex graph lowering remains handwritten and tested.
Vulkan Video
Codec support and resource paths are selected from device capabilities, queue topology, formats, and profile limits.
Media sessions
oa::video provides open, next, seek, flush, playback state, frame reordering, and optional synchronized audio.
ML interop
as_matrix and oa::image preserve the direct path from decoded or captured imagery to model input.
Operation families
| Family | Coverage |
|---|---|
Pixel | Thresholding, range tests, clamp, inversion, tone controls, compositing, channel operations, and noise. |
Geometric | Resize, crop, flip, rotate, pad, remap, affine warp, and perspective warp. |
Filter | Convolution, blur, Sobel, Scharr, Laplacian, morphology, bilateral, sharpening, and adaptive thresholding. |
Color | Per-channel normalization plus semantic image color conversion. |
Image processing example
The semantic oa::Image contract and oa::image operations are documented from the current Rust and Python example sources.
1use anyhow::Context;2use oa::{Engine, ImageFormat, Path, image};34fn main() -> anyhow::Result<()> {5 let engine = Engine::new().context("create OA engine")?;67 let path = Path::asset_rel("image/visionTestPattern320x180.jpg");89 let img = image::decode_file(&engine, &path, ImageFormat::Rgb)10 .with_context(|| format!("decode {}", path.display()))?;11 let small = image::resize(&img, 160, 90, image::InterpolationMode::Bilinear)?;12 let adjusted = image::brightness_contrast(&small, 0.05, 1.1)?;1314 let matrix = adjusted.as_matrix();15 let values = matrix.read_f32()?;1617 assert_eq!(matrix.shape(), [1, 3, 90, 160]);18 assert_eq!(values.len(), 3 * 90 * 160);1920 let min = values.iter().copied().fold(f32::INFINITY, f32::min);21 let max = values.iter().copied().fold(f32::NEG_INFINITY, f32::max);22 println!("{:?} min={min:.4} max={max:.4}", matrix.shape());23 Ok(())24}25
Codec boundary
| Format | Route | Output contract |
|---|---|---|
| H.264 / AVC | Vulkan Video when supported | Timestamped NV12 or converted RGBA frame |
| H.265 / HEVC | Consult current decoder capabilities | Timestamped NV12 or converted RGBA frame |
| AV1 | Consult current decoder capabilities | Timestamped NV12 or converted RGBA frame |
| VP9 | Consult current decoder capabilities | Timestamped NV12 or converted RGBA frame |
| JPEG / PNG | File decode and GPU upload | Texture, image planes, or oa::Matrix |
Support is queried at runtime. Codec availability must be checked against the selected release and device; it is not a claim that every Vulkan device exposes every profile, format, or queue route.