Building Non-Transformer Language Models from Scratch in Rust

A new project called PSSA just hit the top of Hacker News, and it’s doing something most ML engineers have forgotten is even possible: building a language model without transformers. Written entirely from scratch in Rust, PSSA represents a fascinating counter-movement to the transformer monoculture that has dominated NLP since 2017. But beyond the novelty, this project highlights a critical skill that separates great AI engineers from code copiers: understanding how to implement alternative architectures when transformers aren’t the answer.

Let’s use PSSA as a springboard to explore why non-transformer models still matter, how to architect them, and what Rust brings to the table for serious machine learning systems work.

Table of Contents

• Why Non-Transformer Architectures Still Matter
• Architecture Alternatives to Transformers
• Rust for Machine Learning Systems
• Implementation Patterns for Custom Models
• Performance Considerations and Trade-offs

Why Non-Transformer Architectures Still Matter

Transformers have become the default choice for language modeling, but they’re not universally optimal. Their quadratic attention complexity makes them computationally expensive for long sequences, and their massive parameter counts require substantial infrastructure. Projects like PSSA remind us that alternative architectures can offer compelling trade-offs: lower memory footprint, faster inference on specific tasks, or better performance in resource-constrained environments.

More importantly, understanding non-transformer approaches forces you to think about the fundamental problems in sequence modeling rather than just tuning hyperparameters on pre-built architectures. When you’re working on edge deployment, real-time systems, or specialized domains where transformers are overkill, this knowledge becomes invaluable. Many professionals deepen this foundation through structured programs on DataCamp, where hands-on architecture courses help bridge theory and implementation.

The PSSA Approach

While the PSSA repository doesn’t provide exhaustive documentation yet, the concept of building a “non-transformer language model” typically involves architectures like recurrent neural networks (RNNs), state space models, or hybrid approaches that combine convolutional layers with recurrent elements. The key innovation often lies in how these models handle long-range dependencies without the quadratic complexity of attention mechanisms.

Architecture Alternatives to Transformers

Let’s examine several viable alternatives that you might encounter or implement when building custom language models. Each has distinct characteristics that make it suitable for different scenarios.

State Space Models

State space models like S4 (Structured State Spaces) have emerged as powerful alternatives, offering linear or near-linear complexity while maintaining the ability to capture long-range dependencies. They work by parameterizing continuous-time systems and discretizing them for sequence processing.

Here’s a conceptual Rust structure showing how you might organize a state space layer:

// Basic structure for a state space layer in Rust
struct StateSpaceLayer {
    // A, B, C, D matrices defining the state space
    a_matrix: Vec<Vec<f32>>,  // State transition matrix
    b_matrix: Vec<f32>,        // Input-to-state matrix
    c_matrix: Vec<f32>,        // State-to-output matrix
    d_matrix: f32,             // Direct feedthrough (usually zero)
    state_size: usize,
}

impl StateSpaceLayer {
    // Forward pass: processes input sequence through state space dynamics
    fn forward(&self, input: &[f32]) -> Vec<f32> {
        let mut state = vec![0.0; self.state_size];
        let mut output = Vec::with_capacity(input.len());
        
        for &x in input {
            // Update state: x_{t+1} = A * x_t + B * u_t
            let new_state: Vec<f32> = self.a_matrix
                .iter()
                .map(|row| {
                    row.iter().zip(&state).map(|(a, s)| a * s).sum::<f32>()
                        + self.b_matrix[0] * x
                })
                .collect();
            
            // Compute output: y_t = C * x_t + D * u_t
            let y: f32 = self.c_matrix.iter()
                .zip(&state)
                .map(|(c, s)| c * s)
                .sum::<f32>() + self.d_matrix * x;
            
            state = new_state;
            output.push(y);
        }
        output
    }
}

Gated Recurrent Architectures

LSTMs and GRUs may seem outdated, but they’re still highly effective for specific applications. Their sequential processing can be a feature rather than a bug when you need guaranteed temporal ordering or when working with streaming data where you can’t look ahead.

Advanced practitioners exploring these architectures often complement hands-on coding with theoretical foundations available through specialized courses on Coursera, particularly in sequence modeling and recurrent network optimization.

Rust for Machine Learning Systems

The choice to implement PSSA in Rust isn’t arbitrary—it reflects growing recognition that Rust offers compelling advantages for production ML systems. Memory safety without garbage collection, zero-cost abstractions, and excellent performance make it ideal for inference engines and custom model implementations.

Rust’s ownership system prevents entire classes of bugs that plague C++ implementations, while its performance rivals or exceeds Python/C++ hybrids. For systems that need to handle high-throughput inference or edge deployment, Rust increasingly makes sense.

Practical Rust Patterns for ML

When building ML models in Rust, you’ll typically structure your code around traits that define common operations. Here’s how you might define a generic layer interface:

// Generic trait for neural network layers
trait Layer {
    // Forward propagation with owned input
    fn forward(&self, input: Vec<f32>) -> Vec<f32>;
    
    // Backward propagation returning gradients
    fn backward(&mut self, grad_output: &[f32]) -> Vec<f32>;
    
    // Update parameters given learning rate
    fn update_params(&mut self, learning_rate: f32);
}

// Example: Dense layer implementation
struct DenseLayer {
    weights: Vec<Vec<f32>>,
    biases: Vec<f32>,
    last_input: Vec<f32>,  // Cached for backprop
    weight_gradients: Vec<Vec<f32>>,
    bias_gradients: Vec<f32>,
}

impl Layer for DenseLayer {
    fn forward(&self, input: Vec<f32>) -> Vec<f32> {
        // Matrix multiplication: output = weights * input + biases
        self.weights.iter()
            .zip(&self.biases)
            .map(|(w_row, &bias)| {
                w_row.iter()
                    .zip(&input)
                    .map(|(w, i)| w * i)
                    .sum::<f32>() + bias
            })
            .collect()
    }
    
    // Implementation details for backward and update_params omitted for brevity
    fn backward(&mut self, grad_output: &[f32]) -> Vec<f32> { vec![] }
    fn update_params(&mut self, learning_rate: f32) {}
}

Implementation Patterns for Custom Models

Building a language model from scratch requires careful attention to several key components: tokenization, embedding, the core architecture (where PSSA’s innovation lies), and the output layer. Each component needs to be optimized for both correctness and performance.

Tokenization and Vocabulary Management

Efficient tokenization in Rust typically involves hash maps for vocabulary lookup and careful buffer management. Unlike Python where you might casually create temporary strings, Rust pushes you toward reusing allocations and working with string slices.

Embedding Layers

Embeddings in non-transformer models work identically to transformers—you map token IDs to dense vectors. The difference comes in how you process these embeddings through your architecture. Without positional encodings added elementwise, recurrent and state space models naturally capture position through their sequential processing.

Performance Considerations and Trade-offs

When implementing custom architectures, performance isn’t just about raw speed—it’s about the entire trade-off space: memory usage, latency, throughput, training stability, and inference efficiency. Non-transformer models often win on memory and can offer lower latency for shorter sequences, but may struggle with very long-range dependencies that transformers handle naturally.

Benchmarking Your Architecture

Serious implementation work requires rigorous benchmarking. In Rust, you’ll want to use the criterion crate for microbenchmarks and flamegraph tools for profiling. Pay particular attention to cache efficiency—many ML operations are memory-bound rather than compute-bound, so optimizing memory access patterns matters more than you might expect.

When to Choose Non-Transformer Architectures

Choose non-transformer models when you have: constrained memory budgets (edge devices), very long sequences where quadratic attention is prohibitive, real-time streaming requirements, or tasks where simpler models achieve comparable results. The PSSA project demonstrates that there’s still substantial room for innovation outside the transformer paradigm.

For production deployment, consider that non-transformer models often have simpler inference graphs, making them easier to optimize with standard compiler techniques. They’re also easier to understand and debug when things go wrong—a non-trivial advantage in mission-critical systems.