oa::ml::nn::Moe
Sparse top-k mixture-of-experts feed-forward module. The implementation preserves OA’s expert-major route plan and stacked parameter layout. Routing decisions remain detached while gradients flow through selected gate magnitudes, expert projections, RMSNorm, and the residual input.
Declaration
pub struct Moe { /* private fields */ }Public Rust declaration.
Public Methods
Associated Functions
pub fn new_with_seed( model_width: usize, hidden_width: usize, num_experts: usize, experts_per_token: usize, epsilon: f32, seed: u64, ) -> Result<Self>pub fn new_with_seed_on( engine: &Engine, model_width: usize, hidden_width: usize, num_experts: usize, experts_per_token: usize, epsilon: f32, seed: u64, ) -> Result<Self>pub fn new_with_seed_and_shared_experts( model_width: usize, hidden_width: usize, num_experts: usize, experts_per_token: usize, epsilon: f32, num_shared_experts: usize, seed: u64, ) -> Result<Self>Public Method Documentation
No public source comment is attached to this declaration.
No public source comment is attached to this declaration.
No public source comment is attached to this declaration.
No public source comment is attached to this declaration.
No public source comment is attached to this declaration.
No public source comment is attached to this declaration.
No public source comment is attached to this declaration.
No public source comment is attached to this declaration.
No public source comment is attached to this declaration.
No public source comment is attached to this declaration.
No public source comment is attached to this declaration.
No public source comment is attached to this declaration.
No public source comment is attached to this declaration.
No public source comment is attached to this declaration.
No public source comment is attached to this declaration.
No public source comment is attached to this declaration.
No public source comment is attached to this declaration.
No public source comment is attached to this declaration.
No public source comment is attached to this declaration.
No public source comment is attached to this declaration.
No public source comment is attached to this declaration.
No public source comment is attached to this declaration.
No public source comment is attached to this declaration.
No public source comment is attached to this declaration.
No public source comment is attached to this declaration.
No public source comment is attached to this declaration.
No public source comment is attached to this declaration.
No public source comment is attached to this declaration.
No public source comment is attached to this declaration.
No public source comment is attached to this declaration.
No public source comment is attached to this declaration.
No public source comment is attached to this declaration.
No public source comment is attached to this declaration.
No public source comment is attached to this declaration.
No public source comment is attached to this declaration.
No public source comment is attached to this declaration.
No public source comment is attached to this declaration.
No public source comment is attached to this declaration.
No public source comment is attached to this declaration.
Construct a deterministically initialized sparse MoE. experts_per_token is clamped to 1..=num_experts, matching OA C++. The current GPU route planner admits at most 256 experts. Errors Returns an error for zero dimensions, more than 256 experts, invalid epsilon, shape overflow, allocation failure, or registration failure.
No public source comment is attached to this declaration.
No public source comment is attached to this declaration.
Construct a deterministically initialized sparse MoE. experts_per_token is clamped to 1..=num_experts, matching OA C++. The current GPU route planner admits at most 256 experts. Errors Returns an error for zero dimensions, more than 256 experts, invalid epsilon, shape overflow, allocation failure, or registration failure.
No public source comment is attached to this declaration.