Inferno blog

Tensor Deduplication for Multi-Model Inference

Multi-model workloads are the norm: A/B tests, customer fine-tunes, safety variants, multi-stage pipelines. GPU memory requirements scales linearly with model count, and VRAM is the limiting resource. Tensor dedup can make a big difference here.

Shared Backbones: Loading Weights Once, Serving Many Models

Many multimodal and multi-task models share the same underlying text encoder or LLM backbone. This post explores loading shared backbones once and letting multiple heads reuse them.