olmo_tap.inference

Modules

loading_weights	Helper file to load our 10 head Hydra model (9 LLM heads + 1 Uncertainty head)
poe	Implements the Spec-Decode PoE method detailed here: https://www.overleaf.com/7351696474ggfyybskyttm#e97251 This provides a security guarantee that no harmful token is ever sampled provided there exists at least 1 honest head in the jury which assigns negligible probability mass to the harmful token.
poe_demo_no_kv	Implements the Spec-Decode PoE method detailed here: https://www.overleaf.com/read/cjnxtpqjbbcx#4f6a0e This provides a security guarantee that no harmful token is ever sampled provided there exists at least 1 honest head in the jury which assigns negligible probability mass to the harmful token.