vps-gpu Quickstart — GPU-accelerated DepthFusion + research tools

Note: This doc describes V1 behavior. V2 server setup is documented in docs/v2/admin-runbooks.md (§1 Initial Server Setup + First-User Enrollment). The V2 installer adds OIDC configuration, database initialization, and TLS setup steps not covered here. If you are running the v2-enterprise branch, refer to the V2 admin runbooks instead.

Target: Hetzner GEX44 (NVIDIA RTX 4000 SFF Ada) or any CUDA-12+ host Time: ~1-1.5 hours (mostly GPU-driver setup + Gemma model pull — Ollama's install + pull replaces vLLM's from-source CUDA toolchain build, which was most of the old ~4 hour estimate; not independently re-timed end-to-end, so treat this as an estimate rather than a measured figure). Produces: working DepthFusion in vps-gpu mode + weekly regression monitor + initial prompt corpus Estimated first-run size on disk: ~20-25 GB (Ollama + Gemma 4 26B + sentence-transformers). Gemma 4 26B is ~19 GB as pulled by Ollama (gemma4:26b). For cards with < 20 GB VRAM, use gemma4:e4b instead (~9.6 GB on disk).

This is the GPU-enabled counterpart to vps-cpu-quickstart.md. At the end you'll have:

• DepthFusion in vps-gpu mode (local Gemma LLM + local embeddings)
• Ollama serving Gemma 4 26B (already runs as its own systemd service — no DepthFusion-specific unit needed)
• Weekly regression monitor scheduled
• Initial prompt corpus mined

If you are migrating from an existing vps-cpu install (moving data from a current VPS to the new GPU box), follow ../runbooks/gpu-vps-migration.md instead — it includes the data-migration steps this quickstart omits.

---

0. Prerequisites

Pick a Gemma 4 variant matching your hardware

The recommended model is Gemma 4 26B (gemma4:26b, an MoE model, Ollama's default quantization), which is what the rest of this guide assumes. Use the table below to pick the right tag if your VRAM differs from the GEX44 reference. All sizes are Ollama's default-quantization download size.

GPU VRAM	Recommended tag	Size on disk	Notes
12-16 GB	gemma4:e4b	~9.6 GB	Edge model, 4.5B effective params; smallest competent variant
16-20 GB	gemma4:e4b or gemma4:12b	~9.6 GB / ~7.6 GB	Safer fit than the 26B MoE on the RTX 4000 SFF Ada (19.55 GiB) — see note below
20-24 GB	gemma4:26b (this guide)	~19 GB	MoE, 4B active params; GEX44 reference target
40+ GB	gemma4:31b	~20 GB	Dense workstation model; highest quality

MoE memory note (counter-intuitive): Gemma 4 26B is a Mixture-of-Experts model with ~4B active parameters per forward pass. MoE is a compute optimization, NOT a memory optimization — every expert weight must still be resident in VRAM (or CPU-offloaded by Ollama, at a latency cost). If you assume "4B active means it'll fit on a 12 GB card" you'll get heavy CPU offload instead of a clean GPU fit.

RTX 4000 SFF Ada (19.55 GiB) note: gemma4:26b (~19 GB on disk) leaves very little headroom for KV cache and context on this card. Ollama will offload what doesn't fit rather than OOMing outright, but if you see high latency or thrashing, switch to gemma4:e4b or gemma4:12b instead — Ollama manages VRAM allocation automatically, no --gpu-memory-utilization flag to tune.

For the rest of this guide, "Gemma 4" without qualification means the 26B variant (gemma4:26b).

System checks

# Hardware
nvidia-smi                  # must show GPU; ≥ 20 GB VRAM for gemma4:26b
nvcc --version              # CUDA 12.0+

# Python — 3.11 or newer; "or newer" means any modern Python is fine.
# Ubuntu 24.04 ships 3.12 as default. 3.11 is the minimum required.
# Don't try to install 3.11 specifically on 24.04 — 3.12 is the default.
python3 --version

# Build tools + venv module (fresh-install gotcha — venv isn't
# pre-installed on Ubuntu 24.04, and chromadb / hnswlib need compile tools)
sudo apt update
sudo apt install -y python3-full python3-venv build-essential python3-dev

# systemd — user-level only, for the weekly timer. Ollama installs and
# manages its own root-level systemd unit automatically (no manual setup).
systemctl --user status || sudo loginctl enable-linger $USER

# Network bandwidth — model download is ~19 GB (gemma4:26b);
# plan for metered transit

You'll also need: - DEPTHFUSION_API_KEY — strongly recommended even on GPU hosts; powers the Haiku fallback path when Gemma is down or OOM - ≥ 25 GB free disk space (Ollama + Gemma weights + sentence-transformers)

---

1. Clone, create a venv, and install with GPU extras

On modern Ubuntu, pip refuses system-wide installs (PEP 668). You must install into a virtualenv.

1a. Clone and venv

git clone https://github.com/gregdigittal/depthfusion.git ~/projects/depthfusion
cd ~/projects/depthfusion

python3.12 -m venv ~/venvs/depthfusion  # pin an explicit interpreter — S-316
source ~/venvs/depthfusion/bin/activate
# Prompt should now show (depthfusion) at the front

1b. Install with vps-gpu extras + Ollama

pip install --upgrade pip

# [vps-gpu] pulls in sentence-transformers + chromadb
pip install -e '.[vps-gpu]'

# Ollama serves Gemma. Install it (official install script — verify
# https://ollama.com/download/linux still documents this before running it
# on a host you care about):
curl -fsSL https://ollama.com/install.sh | sh

# Pull the model matching your VRAM tier (see §0 table above):
ollama pull gemma4:26b

The Ollama installer sets up and starts its own systemd service (ollama.service) automatically — nothing DepthFusion-specific to configure here.

Verify:

python3 -c "import depthfusion, sentence_transformers; print('ok')"
# -> ok

ollama list
# -> should show gemma4:26b (or whichever tag you pulled)

If ollama pull fails, the most common cause is a CUDA/driver version mismatch preventing Ollama from detecting the GPU. Check nvidia-smi succeeds and ollama ps reports GPU usage once a model is loaded (see Ollama's GPU docs).

1c. Make the venv auto-activate in new shells

grep -q "# depthfusion venv auto-activate" ~/.bashrc || cat >> ~/.bashrc <<'EOF'

# depthfusion venv auto-activate
if [ -z "$VIRTUAL_ENV" ] && [ -f "$HOME/venvs/depthfusion/bin/activate" ]; then
    source "$HOME/venvs/depthfusion/bin/activate"
fi

# depthfusion PS1 prefix enforcement — robust against whatever the
# activate script does or doesn't do with PS1.
if [ -n "$VIRTUAL_ENV" ] && [[ "$PS1" != *"(depthfusion)"* ]]; then
    PS1="(depthfusion) $PS1"
fi
EOF

⚠ Do NOT source ~/.bashrc while the venv is already active. Use deactivate; exec bash instead — sourcing on an active venv leaves the shell in a half-activated state ($VIRTUAL_ENV set but $PATH clobbered).

Test in a fresh shell:

deactivate 2>/dev/null; exec bash
which python3       # should print ~/venvs/depthfusion/bin/python3
echo "$VIRTUAL_ENV" # should print /home/$USER/venvs/depthfusion

---

2. Ollama serves Gemma — no service setup needed

Unlike vLLM, Ollama manages its own systemd service and model store. The install in §1b already started ollama.service and pulled the model — there is no DepthFusion-specific unit to write or enable here.

The model pull in §1b (ollama pull gemma4:26b) downloads the ~19 GB weights and blocks until it completes, so by the time it returns the server is already up and the model is on disk (unlike vLLM, there's no separate "wait for the server to load the model" step — ollama pull and ollama serve are decoupled, but ollama run/API calls trigger an on-demand load of an already-pulled model in a few seconds).

Verify:

systemctl status ollama --no-pager
# Should show active (running)

curl -s http://127.0.0.1:11434/v1/models | python3 -m json.tool
# Should list gemma4:26b

ollama ps
# Shows the model + GPU/CPU split once it's been queried at least once

If Ollama fails to start or the model won't load, common causes: - Driver mismatch → nvidia-smi should succeed; check journalctl -u ollama --since="10 minutes ago" for CUDA errors - Port 11434 conflict → sudo lsof -i :11434 to find the squatter - Heavy CPU offload / high latency on a tight-VRAM card → switch to gemma4:e4b or gemma4:12b (see §0 table) instead of gemma4:26b - Partial/corrupt pull → ollama rm gemma4:26b && ollama pull gemma4:26b

If depthfusion-mcp fails to bind after a restart (port already in use):

# A stale process may be holding the MCP port from the previous run
sudo fuser -k 7301/tcp
sudo systemctl start depthfusion-mcp

---

3. Run the installer and add credentials

The installer merges with any existing ~/.claude/depthfusion.env — credentials you add before or after are preserved across re-installs.

3a. Run the installer in vps-gpu mode

python3 -m depthfusion.install.install --mode=vps-gpu

The installer auto-probes for: - nvidia-smi (should pass, GPU is present) - sentence-transformers (pass, installed via [vps-gpu] extra) - Ollama at DEPTHFUSION_GEMMA_URL (pass, service is running)

Then runs run_vps_gpu_smoke(): three-probe smoke test that actually executes nvidia-smi, imports sentence-transformers, and issues a one-shot Gemma completion. Failure is non-fatal — install completes, smoke can be re-run later.

The installer also registers the DepthFusion MCP server with Claude Code via claude mcp add (if the claude CLI is present on this host).

3b. Add credentials + enable flag to the env file

Even on a GPU host where Gemma is primary, setting DEPTHFUSION_API_KEY is strongly recommended so the FallbackChain can fall through to Haiku if Gemma is OOM, rate-limited, or down. You can do this before or after the installer runs; re-running the installer will not wipe these lines.

DEPTHFUSION_GEMMA_URL/DEPTHFUSION_GEMMA_MODEL are already set correctly by both the installer (§3a) and GemmaBackend's own built-in default (http://127.0.0.1:11434/v1, gemma4:e4b). Only add them below if you pulled a different tag (e.g. gemma4:26b per the §0 VRAM table) and need to override the installer's default.

cat >> ~/.claude/depthfusion.env <<'EOF'
DEPTHFUSION_API_KEY=sk-ant-api03-your-real-key-here
DEPTHFUSION_HAIKU_ENABLED=true
DEPTHFUSION_GEMMA_URL=http://127.0.0.1:11434/v1
DEPTHFUSION_GEMMA_MODEL=gemma4:26b
EOF
chmod 600 ~/.claude/depthfusion.env

⚠ Billing safety — use DEPTHFUSION_API_KEY, NOT ANTHROPIC_API_KEY. Setting ANTHROPIC_API_KEY flips Claude Code's billing to pay-per-token for all usage, not just DepthFusion. The installer refuses to use ANTHROPIC_API_KEY by design (E-12 S-22).

3c. Make the shell auto-load the env file

Writing to depthfusion.env doesn't put variables in your shell environment — nothing auto-sources it. Python tools read os.environ, so without this block the factory check in 3e will route LLM capabilities to null despite the env file being correct.

grep -q "# depthfusion env auto-source" ~/.bashrc || cat >> ~/.bashrc <<'EOF'

# depthfusion env auto-source — export vars from the config file
if [ -f "$HOME/.claude/depthfusion.env" ]; then
    set -a
    source "$HOME/.claude/depthfusion.env"
    set +a
fi
EOF

3d. Reload the shell

deactivate 2>/dev/null; exec bash

# Confirm the env vars are live
echo "haiku:     $DEPTHFUSION_HAIKU_ENABLED"
echo "key:       ${DEPTHFUSION_API_KEY:0:16}..."
echo "gemma URL: ${DEPTHFUSION_GEMMA_URL:-not set}"

3e. Verify the install

python3 -c "
from depthfusion.backends.factory import get_backend
for cap in ('reranker', 'extractor', 'linker', 'summariser', 'decision_extractor', 'embedding'):
    b = get_backend(cap)
    print(f'{cap:22} -> {b.name:20} (healthy={b.healthy()})')
"

Expected: - LLM capabilities → gemma (healthy) - embedding → local (healthy)

If LLM capabilities route to haiku instead of gemma, Ollama isn't reachable at DEPTHFUSION_GEMMA_URL — check curl http://127.0.0.1:11434/v1/models and systemctl status ollama. This is expected behaviour (the factory falls back to Haiku when Gemma is unhealthy) so the install is usable, just not running on GPU until Ollama comes up.

---

4. Install the research tools

bash ~/projects/depthfusion/scripts/install-research-tools.sh

This is the same script as the vps-cpu path — the tools are mode-agnostic.

Verify:

systemctl --user list-timers ciqs-weekly.timer --no-pager
ls -lh ~/.local/share/depthfusion/corpus/

---

5. Smoke test the full pipeline

# End-to-end recall via the real MCP interface
python3 -c "
from depthfusion.mcp.server import _tool_recall
import json
result = json.loads(_tool_recall({'query': 'GPU verification test', 'top_k': 3}))
print(f'blocks returned: {len(result.get(\"blocks\", []))}')
print(f'error: {result.get(\"error\", \"none\")}')
"

# Sanity-check latency: p95 on GPU should be much lower than on CPU
python3 -c "
from depthfusion.metrics.collector import MetricsCollector
from depthfusion.metrics.aggregator import MetricsAggregator
import json
s = MetricsAggregator(MetricsCollector()).backend_summary()
print(json.dumps(s, indent=2, default=str))
"

---

6. Record the baseline CIQS run

If you're running the parallel-comparison plan, now is the moment to capture a 3-run CIQS baseline on this GPU host:

for i in 1 2 3; do
  python3 scripts/ciqs_harness.py run --mode vps-gpu --run $i
done
# Then score each via the operator/judge workflow — see docs/benchmarks/README.md

Scored files land in docs/benchmarks/. When you've also collected baseline runs on the vps-cpu host, scripts/ciqs_compare.py produces the honest delta report. See the end of the vps-cpu quickstart for the exact invocation.

---

7. Enable HNSW vector index (recommended)

HNSW is an approximate nearest-neighbour index that fuses with BM25 to give vector-similarity-boosted recall. hnswlib is already installed by the [vps-gpu] extras — you only need to add two env vars and restart.

Non-vps-gpu installs (e.g. Mac local mode): hnswlib is not in the local or vps-cpu extras. Install it manually: bash pip install 'hnswlib>=0.7' zsh users: quote the version specifier — without quotes, zsh interprets >= as an output-redirection operator and you get zsh: 0.7 not found instead of a pip install. Single quotes ('hnswlib>=0.7') prevent this.

# Add to ~/.claude/depthfusion.env
cat >> ~/.claude/depthfusion.env <<'EOF'
DEPTHFUSION_HNSW_ENABLED=true
DEPTHFUSION_VECTOR_SEARCH_ENABLED=true
EOF

Reload and verify:

# Reload the shell so systemd services pick up the new vars
deactivate 2>/dev/null; exec bash

# Confirm HNSW is active on the MCP server (restart it first):
systemctl --user restart depthfusion-mcp   # if running as a service
# or restart manually: kill $(lsof -ti:7301) && nohup ... &

python3 -c "
from depthfusion.retrieval.hnsw_store import HNSWStore
s = HNSWStore()
print('hnsw_available:', s.is_available())
"
# Expected: hnsw_available: True

After HNSW is active, recall responses include strategy: "bm25+hnsw-fused" and hnsw_available: true. The index is built incrementally — each publish_context upserts into HNSW automatically when enabled.

---

8. Start the REST API as a persistent systemd service (optional)

The REST API (127.0.0.1:7300) exposes all 29 MCP tools over HTTP. It is required for the depthfusion-pp-cli generated CLI (§9). The service file is included in the repo:

# Install the user-level systemd service
cp ~/projects/depthfusion/infra/systemd/depthfusion-rest.service \
   ~/.config/systemd/user/

systemctl --user daemon-reload
systemctl --user enable --now depthfusion-rest

# Verify it started:
systemctl --user status depthfusion-rest
curl -s http://127.0.0.1:7300/status | python3 -m json.tool
# Expected: {"status": "ok", ...}

Required env vars — the service reads ~/.claude/depthfusion.env via systemd EnvironmentFile. Ensure the following are set before starting: DEPTHFUSION_REST_API=true, DEPTHFUSION_HNSW_ENABLED=true, DEPTHFUSION_VECTOR_SEARCH_ENABLED=true, DEPTHFUSION_MODE=vps-gpu.

See infra/systemd/README.md for the full install reference and upgrade notes.

---

9. Install the generated CLI (optional)

depthfusion-pp-cli is a Go CLI exposing all 29 API endpoints as subcommands, plus three compound commands (discovery-audit, graph-inspect, batch-recall). It requires the REST API (§8) to be running.

The binary ships pre-built at:

~/printing-press/library/depthfusion/build/stage/bin/depthfusion-pp-cli
~/printing-press/library/depthfusion/build/stage/bin/depthfusion-pp-mcp

Add it to your PATH:

# Add to ~/.zshrc or ~/.bashrc — already present if you used the agent-ops installer
export PATH="$HOME/printing-press/library/depthfusion/build/stage/bin:$PATH"
source ~/.zshrc

# Verify
depthfusion-pp-cli status
depthfusion-pp-cli recall relevant --query "test" --json

The MCP HTTP server is fail-closed: every request requires a Bearer token. Set DEPTHFUSION_MCP_TOKEN on the server and export the same value as DEPTHFUSION_API_KEY in your client environment before running CLI commands. To expose the server on a non-loopback interface, set DEPTHFUSION_MCP_HOST=0.0.0.0 in the env file (and ensure DEPTHFUSION_MCP_TOKEN is configured and a firewall rule is in place).

The companion MCP server (depthfusion-pp-mcp) mirrors all commands as agent tools. Register it with Claude Code:

claude mcp add --scope user depthfusion-cli \
  ~/printing-press/library/depthfusion/build/stage/bin/depthfusion-pp-mcp

Full CLI reference: docs/cli.md.

---

Done

You now have:

• ✅ DepthFusion in vps-gpu mode (local Gemma + local embeddings)
• ✅ Ollama serving Gemma as a systemd service (root-level, self-managed)
• ✅ Weekly regression monitor scheduled (user-level)
• ✅ Initial prompt corpus mined
• ✅ All three research tools available under scripts/
• ✅ HNSW vector index enabled (§7) — BM25+HNSW fused recall active
• ✅ REST API running as a persistent systemd service on 127.0.0.1:7300 (§8)
• ✅ depthfusion-pp-cli and depthfusion-pp-mcp on PATH (§9)

Operational notes

Ollama service health — check occasionally:

sudo systemctl status ollama
sudo journalctl -u ollama -n 100 --no-pager

If Ollama OOMs, crashes, or the model unloads, GemmaBackend.healthy() returns False and the factory returns NullBackend for LLM capabilities until Ollama recovers. The FallbackChain class (shipped in v0.6.0a1, wired-by-default in v0.6.0 stable) will make this graceful — until then, check systemctl status ollama if recall quality drops suddenly.

Model updates — pulling a newer Gemma checkpoint or switching tags:

ollama pull gemma4:26b       # re-pull to update an existing tag
# or edit DEPTHFUSION_GEMMA_MODEL in ~/.claude/depthfusion.env to switch tags,
# then restart depthfusion-rest to pick up the change

Rollback — if this GPU install misbehaves and you need to fall back to the vps-cpu host temporarily:

# On the GPU host
sudo systemctl stop ollama
sudo systemctl disable ollama
# Traffic directed to the other host by whatever controls your routing

Data on the GPU host (~/.claude/shared/discoveries/, etc.) is preserved — rollback is a routing decision, not a data migration.