RealmOA Library

oa::ml::nn::Transformer

Ready-to-train causal Transformer language model. This is the idiomatic Rust spelling of donor oa::NnTransformer: token and position embeddings, registered Transformer blocks, final normalization, and a vocabulary projection.

StRust public API

Declaration

pub struct Transformer { /* private fields */ }
Public Rust declaration.

Public Methods

pub fn forward(&self, tokens: &Matrix) -> Result<Matrix>
pub const fn vocab_size(&self) -> usize
pub const fn context_length(&self) -> usize
pub const fn model_width(&self) -> usize
pub const fn hidden_width(&self) -> usize
pub fn num_layers(&self) -> usize
pub const fn num_heads(&self) -> usize
pub fn is_moe(&self) -> bool
pub fn block(&self, index: usize) -> Option<&TransformerBlock>
fn forward(&self, input: &Matrix) -> Result<Matrix>
fn registry(&self) -> &ModuleRegistry
fn parameters(&self) -> Vec<Parameter>
fn named_parameters(&self) -> Vec<NamedParameter>
fn buffers(&self) -> Vec<Matrix>
fn named_buffers(&self) -> Vec<NamedBuffer>
fn all_parameters(&self) -> Result<Vec<Parameter>>
fn all_named_parameters(&self) -> Result<Vec<NamedParameter>>
fn all_named_buffers(&self) -> Result<Vec<NamedBuffer>>
fn all_named_state_u32(&self) -> Result<Vec<NamedStateU32>>
fn num_parameters(&self) -> Result<usize>
fn train(&self, training: bool)
fn eval(&self)
fn is_training(&self) -> bool
fn scoped_eval(&self) -> ScopedEval<'_>

Associated Functions

pub fn with_seed( engine: &Engine, vocab_size: usize, context_length: usize, model_width: usize, hidden_width: usize, num_layers: usize, num_heads: usize, epsilon: f32, seed: u64, ) -> Result<Self>
pub fn with_seed_moe( engine: &Engine, vocab_size: usize, context_length: usize, model_width: usize, expert_hidden_width: usize, num_layers: usize, num_heads: usize, num_experts: usize, experts_per_token: usize, epsilon: f32, seed: u64, ) -> Result<Self>

Public Method Documentation

pub fn forward( &self, tokens: &Matrix ) -> Result<Matrix>
No public source comment is attached to this declaration.
pub const fn vocab_size( &self ) -> usize
Construct a deterministic causal language model. Errors Returns an error unless all dimensions are positive, the model width is divisible by the head count, positions fit U32, and every child can be constructed and registered.
pub const fn context_length( &self ) -> usize
Construct a deterministic causal language model. Errors Returns an error unless all dimensions are positive, the model width is divisible by the head count, positions fit U32, and every child can be constructed and registered.
pub const fn model_width( &self ) -> usize
Construct a deterministic causal language model. Errors Returns an error unless all dimensions are positive, the model width is divisible by the head count, positions fit U32, and every child can be constructed and registered.
pub const fn hidden_width( &self ) -> usize
Construct a deterministic causal language model. Errors Returns an error unless all dimensions are positive, the model width is divisible by the head count, positions fit U32, and every child can be constructed and registered.
pub fn num_layers( &self ) -> usize
Construct a deterministic causal language model. Errors Returns an error unless all dimensions are positive, the model width is divisible by the head count, positions fit U32, and every child can be constructed and registered.
pub const fn num_heads( &self ) -> usize
Construct a deterministic causal language model. Errors Returns an error unless all dimensions are positive, the model width is divisible by the head count, positions fit U32, and every child can be constructed and registered.
pub fn is_moe( &self ) -> bool
No public source comment is attached to this declaration.
pub fn block( &self, index: usize ) -> Option<&TransformerBlock>
No public source comment is attached to this declaration.
fn forward( &self, input: &Matrix ) -> Result<Matrix>
No public source comment is attached to this declaration.
fn registry( &self ) -> &ModuleRegistry
No public source comment is attached to this declaration.
fn parameters( &self ) -> Vec<Parameter>
No public source comment is attached to this declaration.
fn named_parameters( &self ) -> Vec<NamedParameter>
No public source comment is attached to this declaration.
fn buffers( &self ) -> Vec<Matrix>
No public source comment is attached to this declaration.
fn named_buffers( &self ) -> Vec<NamedBuffer>
No public source comment is attached to this declaration.
fn all_parameters( &self ) -> Result<Vec<Parameter>>
No public source comment is attached to this declaration.
fn all_named_parameters( &self ) -> Result<Vec<NamedParameter>>
No public source comment is attached to this declaration.
fn all_named_buffers( &self ) -> Result<Vec<NamedBuffer>>
No public source comment is attached to this declaration.
fn all_named_state_u32( &self ) -> Result<Vec<NamedStateU32>>
No public source comment is attached to this declaration.
fn num_parameters( &self ) -> Result<usize>
No public source comment is attached to this declaration.
fn train( &self, training: bool )
No public source comment is attached to this declaration.
fn eval( &self )
No public source comment is attached to this declaration.
fn is_training( &self ) -> bool
No public source comment is attached to this declaration.
fn scoped_eval( &self ) -> ScopedEval<'_>
No public source comment is attached to this declaration.
pub fn with_seed( engine: &Engine, vocab_size: usize, context_length: usize, model_width: usize, hidden_width: usize, num_layers: usize, num_heads: usize, epsilon: f32, seed: u64 ) -> Result<Self>
No public source comment is attached to this declaration.
pub fn with_seed_moe( engine: &Engine, vocab_size: usize, context_length: usize, model_width: usize, expert_hidden_width: usize, num_layers: usize, num_heads: usize, num_experts: usize, experts_per_token: usize, epsilon: f32, seed: u64 ) -> Result<Self>
No public source comment is attached to this declaration.