Key Takeaways
- Marktechpost has released Token Saver, an open-source extension for Claude Desktop.
- The tool cuts token usage significantly, reducing it by up to 99% for large PDF documents.
- Token Saver works by sending only relevant sections of the document to Claude.
Marktechpost has introduced Token Saver, an open-source extension designed to drastically reduce the number of tokens used when working with large PDF documents. The tool, currently at version 1.0 and released under the MIT license, aims to address a common issue faced by users who repeatedly question AI models about extensive documents.
According to Marktechpost, adding a 200-page PDF to a conversation does not create only a one-time context cost; the document remains part of the ongoing discussion, increasing token usage as the dialogue continues. The report also notes that processing large PDFs can involve both extracted text and page images, with each page potentially containing between 1,500 to 3,000 tokens.
Token Saver operates by running as a local Model Context Protocol (MCP) server, allowing Claude Desktop to use it as a document-search tool. The PDF itself remains on the user’s computer, and when a question is asked, Token Saver searches the document locally before sending only relevant sections to Claude. This approach ensures that Claude receives a smaller collection of page-cited passages rather than the complete document.
The Hybrid RAG system employed by Token Saver combines two retrieval methods: BM25 keyword search and semantic search. The former uses SQLite FTS5 for finding passages containing relevant terms, while the latter employs the local all-MiniLM-L6-v2 embedding model to find passages based on meaning. Both methods are weighted at 0.4 and 0.6 respectively, with the embedding component being optional.
The tool’s eight-stage process runs locally before information reaches Claude. It includes stages such as extraction, chunking, scoring, gating, deduplication, trimming, budgeting, and envelope addition. This ensures that only relevant sections are sent to Claude, significantly reducing token usage. The report states that up to 99.4% token savings can be achieved.
Token Saver was developed by Arnav Rai, a computer science student at Rochester Institute of Technology, during his internship under the supervision of Jean-Marc Mommessin and Asif Razzaq. The tool is part of Marktechpost AI Media Inc.'s efforts to enhance user experience with Claude Desktop.
While prompt caching and Claude Projects can reduce some overhead, Marktechpost argues that they do not eliminate the need to send the document to the provider. Token Saver addresses this issue by ensuring that only relevant sections are sent, thereby saving tokens and improving efficiency.
