SALT: Smart LLM Context Compression Using Tries — Same Meaning, 70% Fewer Tokens

One of the biggest pain points when working with LLMs is the context window — you pay for every token you send, and the more the model reads, the slower it gets. SALT (Semantic-Aware Lossless Trie) solves this by compressing long documents into shorter prompts, keeping only the sentences that carry the most information.

How It Works

SALT builds a trie (prefix tree) from the input text. Each sentence is scored based on how much unique information it contributes — redundant or low-information sentences are dropped. The result is a significantly shorter prompt that preserves core meaning.

The killer feature: saltChat keeps the trie in DRAM across conversation turns, meaning a document is indexed once and reused for the entire chat — no re-reading on every message.

Installation

git clone https://github.com/oteomamo/SALT.git
cd SALT
pip install -e .

# Basic compression
python -m salt.cli compress document.txt --output prompt.txt

# Chat with long documents
python -m salt.cli chat --document research_paper.pdf

Real-World Experience

u/No_Sky9786, SALT’s creator, shared on Reddit: “SALT shrinks a long document down to a fixed size before it is sent to a language model, keeping the sentences that carry the most information. It works with any model, produces a shorter plain-text prompt, and cuts the compute, memory, and wait time that long inputs cost.”

This tool pairs especially well with local-first MCP servers like Synapse — index your codebase with Synapse, then compress the output with SALT before it hits your context window.

Bottom Line

SALT isn’t the hottest repo on GitHub today, but it solves a real problem: context window economics. As models increasingly support massive context windows (1M+ tokens), intelligent compression becomes just as critical as raw processing power. If you regularly work with long documents through LLMs, SALT deserves a spot in your toolkit.

Sources: GitHub (oteomamo/SALT), Reddit r/artificial