Sean Pedersen

A short list of free AI chat apps for local LLM execution that work offline and promise privacy first (no user data collection).

Jan - https://github.com/menloresearch/jan

• Standalone app: easy setup
• Open source
• Not many features

ChatBox - https://github.com/chatboxai/chatbox

• Standalone app: easy setup
• Open source
• Features: (not yet tested)

Msty - https://msty.app/

• Standalone app: easy setup
• Closed Source
• Small bootstrapped company
• Features: RAG over files, split (copy) conversations, web search (unreliable right now)

Deep Chat - https://deepchat.thinkinai.xyz/

• Standalone app: easy setup
• Open source
• Features: integrated web search (Brave Search, Google)

LM Studio - https://lmstudio.ai/

• Standalone app: easy setup
• Closed Source
• VC Funded (might clash with privacy focus in the future)
• Features: Clean UI, split (copy) conversations

Open WebUI - https://docs.openwebui.com/

• Web app: hard setup (Install is complicated, requires sign-up)
• Open source

LibreChat - https://www.librechat.ai/

• Web app: hard setup (requires Docker)
• Open source

Conclusion

To me Jan and Msty stand out. Jan being a functional basic open-source chat app, that is easy to install and use. While Msty being also easy to setup but with more advanced features - Msty only lacks open-source. I recommend the Llama and Gemma model family for local basic use. DeepSeek R1 for advanced reasoning and coding capabilities.

Msty LLM CLI Tool Integration

Set in Msty options under section “Local AI”, the service endpoint (which uses Ollama) to http://localhost:11434 (the default port for Ollama) or just install Ollama. Next install the CLI tool llm, which lets you interact with LLM’s from the CLI.

Now you can access (while Msty or Ollama is running), your local LLM models from the CLI using f.e.: $ llm -m llama3.2:3b-instruct-q8_0 "Tell me a joke about two sausages in a bar." (specify the model you installed with -m option - list intalled models like so: $ curl http://localhost:11434/api/tags | jq .)

And the cool thing about llm is, you can pipe into it: $ echo "omnia mirari, gaudium explorandi" | llm -s "Translate into English:"

Enabling following use case: summarize youtube video transcripts (install https://pypi.org/project/youtube-transcript-api/) and use this bash function:

yt-summarize () {
 local video_id
 if [[ "$1" == http* ]]
 then
  video_id=$(echo "$1" | sed -E 's/.*[?&]v=([^&]+).*/\1/; s/.*youtu\.be\/([^?&]+).*/\1/')
 else
  video_id="$1"
 fi
 youtube_transcript_api --format text $video_id | llm -m llama3.2:3b-instruct-q8_0 -s "Summarize this transcript. Exclude any mentions of sponsors, advertisements, product placements, or promotional content (e.g., 'This video is sponsored by', 'free trial', '30-day trial', 'promo code', 'Use code', 'Check out', 'Brilliant.org', 'Brilliant', 'NordVPN'). Only include the main educational or informational content and remove the rest:"
}

#tutorial #privacy