Goal: ask a local assistant questions about one small document and verify the answer against the source. Begin with harmless sample text, not your tax records or customer files. Allow time for the initial application and model downloads; speed depends on your computer.
What you need
AnythingLLM Desktop, a local language model that fits your hardware, a local embedding model and a local vector database. Its Desktop edition can provide these components, but verify the actual selections in your version. A local desktop interface connected to a cloud model is not an all-local setup.
Build a small, verifiable test
- Install Desktop from the official download linked in the AnythingLLM setup guide. Select local inference, local embeddings and local storage in its provider settings. Complete model downloads before attempting offline use.
- Create a new workspace for this test. Keep web search, external tools, MCP integrations and cloud providers out of the experiment. Review the telemetry setting if you want to opt out.
- Make a text file with three invented facts: "The demonstration pantry opens at 09:30. Its supervisor is Rowan. There are 12 blue storage boxes." Upload it into the workspace and embed it for reuse across threads. A one-chat attachment has a different scope.
- Ask: "What time does the pantry open, and which source says so?" Compare the answer with your file. Then ask a question the file cannot answer, such as the supervisor's phone number. A useful assistant should admit the information is absent rather than invent it.
- Once downloads are finished, optionally disconnect internet access and repeat the test in a new chat. An offline success is evidence for that workflow, not an audit of every plugin or future configuration.
Know the limits
Document retrieval selects relevant excerpts; it does not guarantee that every page was considered or that a quotation is accurate. Check important answers yourself. Cloud embeddings, cloud language models and external tools can change where content goes. Local storage also does not protect against other people with access to your computer or backups.
For cleanup, removing a document from a workspace is not complete deletion. AnythingLLM's current documentation distinguishes that from deleting it in My Documents, which removes the parsed content and cached embeddings across workspaces. Keep your original sample file if you want to repeat the test.
Documentation checked November 14, 2026. This is a documented project with an explicit acceptance test, not a claim of a completed fresh installation on every supported platform.