Skip to main content
Model: standardmodelbio/SMB-v1-1.7B-Structure
Parameters: 1.7B
Architecture: Joint-Embedding Predictive Architecture (JEPA)
SMB-v1-Structure is our flagship biological world model. Unlike traditional LLMs that predict tokens, it predicts patient states — modeling how patients evolve over time given interventions.

​
Key Differentiators

State Prediction

Predicts future patient states in latent space, not text tokens

Causal Learning

Learns cause-and-effect: (Pre-State + Intervention) → Post-State

Multimodal Fusion

Unifies genomics, imaging, EHR, and proteomics

​
Environment Activation

source standard_model/bin/activate
See the Quickstart Guide for environment creation and usage.

​
Usage

from transformers import AutoModel, AutoTokenizer
import torch

# Load model
model = AutoModel.from_pretrained("standardmodelbio/SMB-v1-1.7B-Structure")
tokenizer = AutoTokenizer.from_pretrained("standardmodelbio/SMB-v1-1.7B-Structure")

# Move to GPU
device = "cuda" if torch.cuda.is_available() else "cpu"
model = model.to(device)
model.eval()

​
Architecture

The Standard Model uses Joint-Embedding Predictive Architecture (JEPA) — treating the patient as a dynamic “world” and treatments as interventions that change that world.
Standard Model Architecture

​
How It Works

1

Modality Ingestion

Raw signals — genomics, proteomics, imaging, EHR data — pass through modality-specific encoders. Each encoder is trained to extract meaningful representations from its data type.
2

Fusion Layer

A specialized projector maps these encodings into a universal latent space. This creates a “fused” patient state embedding that retains both high-level semantic context and low-level biological granularity.
3

State Prediction

Given the current patient state S(t) and an intervention A(t), the model predicts the future state S(t+1) in latent space — not as text, but as a dense embedding.
4

Hybrid Optimization

The model combines supervised fine-tuning (anchoring to clinical outcomes) with JEPA objectives (learning dynamics), preventing training collapse.

​
Extracting Embeddings

Get patient state embeddings for downstream tasks: 
from transformers import AutoModel, AutoTokenizer
import torch

model = AutoModel.from_pretrained("standardmodelbio/SMB-v1-1.7B-Structure")
tokenizer = AutoTokenizer.from_pretrained("standardmodelbio/SMB-v1-1.7B-Structure")

device = "cuda" if torch.cuda.is_available() else "cpu"
model = model.to(device)
model.eval()

# Prepare input
inputs = tokenizer(
    "patient clinical data here",
    return_tensors="pt",
    padding=True,
    truncation=True
).to(device)

# Extract embeddings
with torch.no_grad():
    outputs = model(**inputs)
    embeddings = outputs.last_hidden_state
    
    # Pool to get single patient embedding
    patient_embedding = embeddings.mean(dim=1)  # [batch, hidden_dim]

print(f"Embedding shape: {patient_embedding.shape}")
See the Embeddings Guide for advanced embedding extraction and pooling strategies.

​
Use Cases

Treatment Simulation

Simulate how a tumor would evolve under Treatment A versus Treatment B by conditioning on different interventions.

Trajectory Prediction

Predict disease progression over 3, 6, or 12 month windows.

Digital Twins

Create evolving patient representations that update as new data arrives.

Response Prediction

Model probability of response to specific therapies.

​
Memory Optimization

SMB-v1-Structure requires approximately 16GB GPU memory at full precision. Use these techniques to reduce memory: 
model = AutoModel.from_pretrained(
    "standardmodelbio/SMB-v1-1.7B-Structure",
    torch_dtype=torch.float16,
    device_map="auto"
)
Memory: ~8GB

​
Hardware Requirements

PrecisionGPU MemoryRecommended GPU
float3216 GBA100, A6000
float168 GBRTX 4090, A10
8-bit4 GBRTX 3080, T4
4-bit2 GBRTX 3060

​
Research

The Patient is Not a Document

Read the announcement blog post explaining the motivation and architecture behind SMB-v1-Structure.

Embeddings Guide

Extract and use embeddings

Linear Probing

Train classifiers on embeddings

All Models

View full model catalog