Model: standardmodelbio/smb-ehr-4bParameters: 4BTask: Clinical event prediction, temporal reasoning
SMB-EHR is a foundation model trained on longitudinal electronic health records. It reframes EHRs as timestamped chains of clinical events and predicts future events to enable temporal reasoning over disease trajectories.
Key Features
Next Event Prediction Predicts upcoming clinical events based on patient history
Temporal Reasoning Understands time-based relationships between events
Disease Trajectories Models how conditions evolve over time
Environment Activation source standard_model/bin/activate
Usage from transformers import AutoModel, AutoTokenizer import torch # Load model model = AutoModel.from_pretrained( "standardmodelbio/smb-ehr-4b" ) tokenizer = AutoTokenizer.from_pretrained( "standardmodelbio/smb-ehr-4b" ) # Move to GPU device = "cuda" if torch.cuda.is_available() else "cpu" model = model.to(device) model.eval()
How It Works SMB-EHR treats electronic health records as sequences of clinical events, each with a timestamp. The model learns to:
Encode clinical history — Convert sequences of diagnoses, procedures, medications, and labs into embeddingsReason temporally — Understand the timing and order of eventsPredict next events — Forecast what clinical events are likely to occur next
Get patient embeddings from clinical history:
from transformers import AutoModel, AutoTokenizer import torch model = AutoModel.from_pretrained( "standardmodelbio/smb-ehr-4b" ) tokenizer = AutoTokenizer.from_pretrained( "standardmodelbio/smb-ehr-4b" ) device = "cuda" if torch.cuda.is_available() else "cpu" model = model.to(device) model.eval() # Example: encode clinical history clinical_history = "DX:E11.9 LAB:A1C:7.2 RX:metformin DX:I10 LAB:BP:142/90" inputs = tokenizer( clinical_history, return_tensors = "pt" , padding = True , truncation = True , max_length = 2048 ).to(device) with torch.no_grad(): outputs = model( ** inputs) embeddings = outputs.last_hidden_state # Pool to get patient embedding patient_embedding = embeddings.mean( dim = 1 ) print ( f "Embedding shape: { patient_embedding.shape } " )
Batch Processing Process multiple patients efficiently:
# Multiple patient histories patient_histories = [ "DX:E11.9 LAB:A1C:7.2 RX:metformin" , "DX:I10 DX:I25.10 RX:lisinopril RX:aspirin" , "DX:C34.90 PROC:surgical_resection RX:carboplatin" ] inputs = tokenizer( patient_histories, return_tensors = "pt" , padding = True , truncation = True , max_length = 2048 ).to(device) with torch.no_grad(): outputs = model( ** inputs) embeddings = outputs.last_hidden_state.mean( dim = 1 ) print ( f "Batch embeddings shape: { embeddings.shape } " ) # [3, hidden_dim]
Use Cases
Risk Stratification Identify high-risk patients based on their clinical history patterns.
Event Prediction Predict upcoming hospitalizations, diagnoses, or adverse events.
Cohort Discovery Find similar patients based on embedding similarity.
Clinical Phenotyping Automatically classify patients into clinical phenotypes.
Memory Optimization SMB-EHR-4B requires approximately 24GB GPU memory at full precision.
Float16
8-bit Quantization
model = AutoModel.from_pretrained( "standardmodelbio/smb-ehr-4b" , torch_dtype = torch.float16, device_map = "auto" )
Memory: ~12GBmodel = AutoModel.from_pretrained( "standardmodelbio/smb-ehr-4b" , load_in_8bit = True , device_map = "auto" )
Memory: ~6GBHardware Requirements Precision GPU Memory Recommended GPU float32 24 GB A100 40GB float16 12 GB RTX 4090, A10 8-bit 6 GB RTX 3080, T4
Research
Building the EHR Foundation Model via Next Event Prediction Read the paper describing the training methodology and architecture.
