oa::ml::advantage

Module published by the oa::ml module.

MdRust public API

Public Types

Public Functions

pub fn gae( reward: &Matrix, value: &Matrix, next_value: &Matrix, terminated: &Matrix, truncated: &Matrix, config: GaeConfig, ) -> Result<GaeResult>

pub fn gae_into( reward: &Matrix, value: &Matrix, next_value: &Matrix, terminated: &Matrix, truncated: &Matrix, advantage: &mut Matrix, returns: &mut Matrix, config: GaeConfig, ) -> Result<()>

pub fn normalize(advantage: &Matrix, epsilon: f32) -> Result<Matrix>

Function Documentation

pub fn gae(
	reward: &Matrix,
	value: &Matrix,
	next_value: &Matrix,
	terminated: &Matrix,
	truncated: &Matrix,
	config: GaeConfig
) -> Result<GaeResult>

Compute generalized advantage estimates over a fixed rollout. Termination disables value bootstrap. Termination and truncation both stop the reverse trace, while truncation deliberately retains one-step bootstrap. Errors Returns an error unless all inputs share one engine and one nonempty [time, environments] shape, values are F32, boundary masks are U8, both discounts are finite in [0, 1], and GPU recording succeeds.

pub fn gae_into( reward: &Matrix, value: &Matrix, next_value: &Matrix, terminated: &Matrix, truncated: &Matrix, advantage: &mut Matrix, returns: &mut Matrix, config: GaeConfig, ) -> Result<()>

Record generalized advantage estimation into caller-owned output storage. This is the allocation-free form used by fixed-capacity rollout storage. It has the same semantic identity and numerical contract as gae. Errors Returns the errors from gae, and rejects output matrices that do not match the rollout’s engine, F32 dtype, and shape or alias any input/each other.

pub fn normalize(
	advantage: &Matrix,
	epsilon: f32
) -> Result<Matrix>

Standardize a nonempty FP32 advantage tensor over all elements. The result is (advantage - mean) / sqrt(variance + epsilon). The operation remains device-resident and differentiable, and records one semantic identity over its composite Matrix lowering. Errors Returns an error when advantage is empty or not FP32, epsilon is not positive and finite, a size exceeds the admitted shader ABI, allocation fails, or composite lowering fails.