pumFORGE

Getting started

Speak and it replies out loud. Or type in the box at the bottom. Two modes (side panel → Mode):

• Listening — hands-free. Just talk; a pause ends your turn.
• Push-to-talk — hold the mic button (or Space) to talk.

How voice works

Your voice flows through four stages, shown live when you open the Voice window:

Mic → Whisper → LLM → Voice

• Mic — captures your speech (energy VAD detects when you talk). It's active automatically in the Voice window.
• Whisper — transcribes speech to text (faster-whisper, GPU with CPU fallback).
• LLM — Built-in pumFORGE engine or a custom OpenAI-compatible server generates the reply (streaming).
• Voice — Piper, or your cloned F5-TTS/XTTS voice.

Use the pill at the bottom of the Voice window to exit to chat, mute, or open voice settings.

Voice cloning (your own voice)

Click + next to Voice, name it, and record 15s of natural sentences. It's cloned with F5-TTS (most accurate) or XTTS (faster).

• Speak naturally — don't count or read lists (that causes "bleed" of the reference words).
• After recording, check/edit the reference text so it matches exactly what you said.
• Voice adherence (Generation panel) — higher = sticks to the target text (stops the bleed), lower = more expressive.

Models & LLM

• Backend — pick pumFORGE (built-in) or a custom OpenAI-compatible server, then set the server URL and API key.
• Download model — pull a GGUF model for pumFORGE or any OpenAI-compatible server (with live % and speed). HF token supports gated models.
• Eject (header) unloads the model from GPU; trash deletes it.
• Generation panel — temperature, top-p/k, penalties, max tokens, context.
• Prompt panel — system prompt + stop sequences.

Built-in tools

Turn on Web search in the Tools panel to activate them. The model calls the right tool automatically — you can also just ask normally:

• search_web — "search the web for X" (SearXNG).
• ddg_answer — quick facts, e.g. "who won the 1998 World Cup?"
• fetch_url — "open and summarise this page: URL" (reads a link).
• get_weather — "what's the weather in Paris?"
• calculator — "what is sqrt(256) + 7*3?" (safe math).
• youtube_transcript — "summarise this video: URL".
• arxiv_search — "find papers on quantum computing".
• query_wikidata — factual SPARQL lookups.
• get_current_time — "what time is it?" / "what's today's date?"
• send_email — enable in Settings (SMTP), then "email Bob the following…".

Microphone & camera

• The mic, camera and speaker run in your browser when it can provide them (auto-detected). If the browser can't (no HTTPS, permission denied), pumFORGE falls back to the server's own mic, webcam and speakers.
• Pick the right microphone; the meter's yellow line is the speech threshold — if your voice doesn't cross it, raise Sensitivity.
• Audio health (Settings) opens a diagnostics panel: live waveform, input level, device picker, level suggestions, and one-click "Copy diagnostics".
• The mic pauses while the AI talks, so it won't hear itself.
• Camera toggles the webcam preview (and your face in Voice mode).
• The server only opens the mic/camera while a browser tab is connected — close the app and they're released.

Controls & shortcuts

• Stop button or Esc — interrupt the AI.
• Chat / Voice / Notebook toggle — switch between text chat, the face view, or the Notebook (source-based Q&A).
• Barge-in — talk loudly to interrupt (when enabled).
• Space — push-to-talk (in Push-to-talk mode).

Desktop control (PC)

Turn on Settings → Desktop control and pumFORGE can use your computer for you:

• Commands — run terminal commands, open apps and URLs, close apps.
• Files — list (including full recursive trees with sizes), read, write, move, rename, delete, search, and find/remove duplicate files by content.
• Screen & input — screenshot, read the screen (OCR), type, click, press keys.
• System — CPU/RAM/disk/battery, volume, brightness, clipboard.
• Risky actions always ask first — pick Trust once, Trust always, or Cancel.

Remote access (internet)

By default pumFORGE only listens on your own machine. To reach it from anywhere, run it behind a reverse proxy (Nginx Proxy Manager, Caddy, etc.):

• Start with PUMFORGE_HOST=0.0.0.0 PUMFORGE_TOKEN=your-secret ./start.sh.
• Point your proxy at http://localhost:15080, enable WebSockets (upgrade headers), and force HTTPS.
• Open https://your-domain/?token=your-secret — the token is remembered.
• The mic, camera and speaker run in your browser, so you can talk to your home PC from any device.
• Every browser keeps its own chat history — logging into the same account from two devices won't mix conversations.

Notebook (pumbookLM)

The Notebook is a source-based Q&A workspace — add documents and ask questions with cited answers.

• Projects — create a project (folder icon) to group related sources. Each project has its own documents, conversations, and chat history.
• Add sources — click + to add PDFs, text files, URLs, YouTube transcripts, notes, or audio. Each source is chunked and embedded for search.
• Ask questions — type in the chat bar at the bottom. The LLM searches your sources and answers with citations showing which document each fact came from.
• Conversations — switch between chat threads within a project (chat bubble icon).
• Studio — click the star icon in the top bar to open Studio:
  • Generate Summary — AI summary of all project sources.
  • Generate Audio — reads the summary aloud via browser TTS.
  • Generate Mind Map — interactive visual map of project topics (zoom, pan, download as PNG or Markdown).
  • Export — download project as Markdown, JSON, or plain text.
• Collapse sidebar — click the chevron to collapse/expand the project list.
• The Notebook uses all enabled tools when answering — so the LLM can search the web, do math, check the weather, create documents, and more, all while citing your sources.

Document tools

• create_document — "create a PDF/DOCX/PPTX/XLSX report about X". Creates real document files you can open. DOCX/PPTX/XLSX work out of the box; PDF needs a self-hosted OnlyOffice server.
• convert_document — "convert this file to PDF". Uses OnlyOffice when configured (Settings → OnlyOffice).
• OnlyOffice setup — in Settings → OnlyOffice, enter your Document Server URL, JWT secret, and pumFORGE's public URL. This enables PDF creation and document conversion.
• get_stock — "what's the stock price of Apple?" Quick Yahoo Finance lookup, no API key needed.
• analyze_stock — "analyze Tesla stock". Deep analysis with RSI, MACD, trends, volatility, and recent news.

Desktop file management

• create_folder — "create a folder called Projects on the desktop".
• copy_file — "copy report.pdf to the Documents folder".
• compress — "zip the folder Projects into an archive".
• decompress — "extract this zip file".
• batch_rename — "rename all JPG files in Downloads to lowercase".

Mode & listening

• Listening — hands-free. The mic stays on and a short pause ends your turn.
• Push-to-talk — hold the mic button (or your PTT key) while you talk, release to send.
• PTT key — click the field, then press any key (or combo like Shift+A) to set the hold-to-talk trigger.
• Wake word — the assistant ignores everything until you say "Hey Pum", "Hi Pum", or "Yo Pum" first.
• Microphone — which input device to listen from. The meter above shows live level; the yellow line is the speech threshold.
• Barge-in — talk over the assistant to interrupt it mid-reply.
• Vision — the assistant can look at the camera when you ask about what you see.

Voice & speech

• Voice — the voice that talks back. The + button records/clones your own voice; the trash deletes the selected one.
• Speech-to-text — the Whisper model size. Tiny/Base = faster but less accurate; Small = balanced; Medium = most accurate but slowest.
• GPU for STT & TTS — off runs speech & voices on the CPU (leaves your GPU free for the LLM). Turn on only if you have VRAM to spare.
• Auto-mute when audio plays — pauses the mic when another app is playing sound, so the assistant won't overhear music/videos.
• Speed — how fast the assistant speaks (1.00× = normal).
• Sensitivity — how quiet a sound counts as speech. Raise it if your voice isn't being picked up; lower it if background noise triggers it.
• Voice adherence — (cloned voices) how closely it sticks to your words. Higher = more accurate, lower = more expressive.
• Bass cut — removes low rumble from cloned voices; 0 = off.

Model & backend

• Backend — pumFORGE (built-in, default) or any OpenAI-compatible server.
• Network access — "Local only" keeps it on this PC; "All interfaces" lets other devices reach it (restart required).
• Server URL — where the LLM server lives. Leave blank until you know it (e.g. http://localhost:15083). Refresh reloads the model list, Test checks the connection.
• API key — only needed for a remote or protected OpenAI-compatible server.
• Model — which model answers you. The trash deletes it from disk; the eject icon unloads it from GPU memory.
• Download model — download a GGUF model for pumFORGE or any OpenAI-compatible server.

Generation

• Thinking (reasoning) — lets the model think step-by-step before answering (slower, sometimes better).
• Flash attention — GPU speedup for pumFORGE engine (on supported NVIDIA cards).
• Temperature — randomness. 0 = predictable, higher = more creative.
• Top P — limits choices to the most likely words (lower = more focused).
• Top K — caps how many word choices are considered at each step.
• Repeat penalty — higher = the model repeats itself less.
• Frequency penalty — discourages over-used words.
• Presence penalty — encourages new topics/words.
• Max tokens — the longest a reply can be.
• Context — how much conversation the model remembers (up to 128k). More context = more memory but more VRAM.

Prompt

• System prompt — the standing instructions that define the assistant's personality and style.
• Stop sequences — comma-separated text that ends a reply early when the model writes it.

MCP Tools

• Enable MCP — attach external Model Context Protocol tool servers.
• Servers — list of attached servers; Add server takes a name and the command/URL to launch it.

Email

• Enable email — lets the assistant send mail on your behalf.
• SMTP host / port / address / password — your outgoing mail server details. Gmail needs an "app password", not your normal password.

Users

• Only admins see this section. Create accounts, pick User or Admin roles, disable, or delete users. The last admin can't be removed.
• Factory reset (bottom) wipes all accounts, chats, memories, knowledge and settings back to first-run.

What are tools?

Tools are things the assistant can do — search the web, get the weather, do maths, remember facts, control your computer. When you ask something that needs one, the assistant calls it automatically. You enable each one with its toggle below.

1 · Web search

• Web search — the master switch. Turn it on so the search tools can run.
• SearXNG URL — your own SearXNG server address (for private search). Leave the default if you don't run one — ddg_answer still works without it.
• Results / Pages — how many results to fetch, and how many SearXNG pages to scan.
• OpenWeather API key — optional key for better weather + multi-day forecasts (from openweathermap.org). Blank = weather still works via a free no-key service.

2 · Agents

• Sub-agent — lets the assistant delegate a complex, multi-step task to a separate reasoning loop.
• Sub-agent steps — how many tool calls the sub-agent may make before stopping.
• LLM council — several models answer independently, rank each other, and a chairperson picks the best answer.
• Council models / Chairperson / size — which models take part, who synthesizes, and how many.

3 · Memory

• Enable memory — lets the assistant remember facts you tell it (say "remember that …") and recall them across chats.
• Memories list — everything it currently remembers; delete entries with the ✕.
• Add — type a fact and click Add to store it manually.

4 · Built-in tools

• search_web — search the internet via SearXNG.
• ddg_answer — instant answers from DuckDuckGo (no setup).
• fetch_url — read and summarise a web page's text.
• get_weather — current weather for a city (or your location).
• get_forecast — today's hourly forecast.
• get_extended_forecast — 5-day forecast (needs the OpenWeather key).
• calculator — safe math evaluation.
• youtube_transcript — pull a YouTube video's transcript.
• arxiv_search — find research papers.
• query_wikidata — factual SPARQL lookups.
• get_current_time — date & time.
• get_location — your approximate location (from your IP).

5 · Desktop control

• Enable PC control — lets the assistant operate your computer: run commands, manage files, take screenshots, read the screen (OCR), type/click, report system info, set volume/brightness, use the clipboard.
• Risky actions (deleting files, running commands, typing) ask for your approval first — pick Trust once, Trust always, or Cancel.
• On Wayland, typing/mouse/clipboard need ydotool + wl-clipboard installed.

Email

• Configured in Settings → Email. Once enabled, the assistant can send mail on your behalf (Gmail needs an app password).

MCP Tools

• Configured in Settings → MCP Tools. Attach external Model Context Protocol servers to give the assistant even more abilities.