Skip to main content
SMB-v1-Structure is our flagship biological world model. Unlike traditional LLMs that predict tokens, it predicts patient states in latent space, modeling how patients evolve over time and respond to interventions.
Read more about our modeling approach and benchmarks at our blog.

​
Key Differentiators

State Prediction

Predicts future patient states in latent space, not text tokens

Causal Learning

Learns cause-and-effect: (Pre-State + Intervention) → Post-State

Multimodal Fusion

Unifies genomics, imaging, EHR, and proteomics

​
Environment Activation

source standard_model/bin/activate

​
Usage

from transformers import AutoModel, AutoTokenizer
import torch

# Load model
model = AutoModel.from_pretrained("standardmodelbio/SMB-v1-1.7B-Structure")
tokenizer = AutoTokenizer.from_pretrained("standardmodelbio/SMB-v1-1.7B-Structure")

# Move to GPU
device = "cuda" if torch.cuda.is_available() else "cpu"
model = model.to(device)
model.eval()

​
Architecture

The Standard Model uses Joint-Embedding Predictive Architecture (JEPA) — treating the patient as a dynamic “world” and treatments as interventions that change that world.
Standard Model Architecture

​
How It Works

1

Modality Ingestion

Raw signals — genomics, proteomics, imaging, EHR data — pass through modality-specific encoders. Each encoder is trained to extract meaningful representations from its data type.
2

Fusion Layer

A specialized projector maps these encodings into a universal latent space. This creates a “fused” patient state embedding that retains both high-level semantic context and low-level biological granularity.
3

State Prediction

Given the current patient state S(t) and an intervention A(t), the model predicts the future state S(t+1) in latent space — not as text, but as a dense embedding.
4

Hybrid Optimization

The model combines supervised fine-tuning (anchoring to clinical outcomes) with JEPA objectives (learning dynamics), preventing training collapse.

​
Extracting Embeddings

Get patient state embeddings for downstream tasks: 
from transformers import AutoModel, AutoTokenizer
import torch

model = AutoModel.from_pretrained("standardmodelbio/SMB-v1-1.7B-Structure")
tokenizer = AutoTokenizer.from_pretrained("standardmodelbio/SMB-v1-1.7B-Structure")

device = "cuda" if torch.cuda.is_available() else "cpu"
model = model.to(device)
model.eval()

# Prepare input
inputs = tokenizer(
    "patient clinical data here",
    return_tensors="pt",
    padding=True,
    truncation=True
).to(device)

# Extract embeddings
with torch.no_grad():
    outputs = model(**inputs)
    embeddings = outputs.last_hidden_state
    
    # Pool to get single patient embedding
    patient_embedding = embeddings.mean(dim=1)  # [batch, hidden_dim]

print(f"Embedding shape: {patient_embedding.shape}")

​
Use Cases

Treatment Simulation

Simulate how a tumor would evolve under Treatment A versus Treatment B by conditioning on different interventions.

Trajectory Prediction

Predict disease progression over 3, 6, or 12 month windows.

Digital Twins

Create evolving patient representations that update as new data arrives.

Response Prediction

Model probability of response to specific therapies.

​
Memory Optimization

SMB-v1-Structure requires 16GB GPU memory at full precision. Memory usage can be reduced in several ways: 
model = AutoModel.from_pretrained(
    "standardmodelbio/SMB-v1-1.7B-Structure",
    torch_dtype=torch.float16,
    device_map="auto"
)
Memory: ~8GB

​
Hardware Requirements

PrecisionGPU MemoryRecommended GPU
float3216 GBA100, A6000
float168 GBRTX 4090, A10
8-bit4 GBRTX 3080, T4
4-bit2 GBRTX 3060