Chat Direct to your local model — each send is a curl on your Mac
Training reading heartbeat…
Local data sync M5 → MSI · RAG serving data, kept separate from training
—
Failover monitor auto-ping · primary M5, MSI picks up if M5 is offline
The Studio pings each node every 15s (real liveness, catches dead and hung). Down nodes are skipped so requests never wait on a dead M5; if the monitor itself stops, routing safely reverts to per-request failover.
The training pipeline how customer reality becomes the next HutFin AI — every step gated
Live progress lands in the card above the moment a run starts (the GB10 wrapper POSTs its heartbeat to /api/admin/training). Full detail: Models ▾ → Plane / Architecture.
Machines who does what
Measured 08-08, identical weights: M5 82.9 tok/s single · MSI 60.3 single / 225.6 tok/s aggregate @ 8 users.
Products in the stack every tool that touches the model lifecycle
| Product | Role | Where |
|---|---|---|
| HutFin AI 1.0 | the model — Qwen3.6-35B-A3B MoE (Q6_K · MLX 6-bit · CUDA) | M5 · MSI |
| Unsloth | LoRA fine-tuning engine (Unsloth-first doctrine) | MSI GB10 |
| NGC PyTorch + CUDA 13 | training container, 96.3 TFLOPS verified | MSI GB10 |
| Ollama | serving engine — the fleet-wide insurance policy | M5 :11434 · MSI :11434 |
| mlx-vlm / MLX-LM | Apple-GPU serving + vision tower (v5: :11902 batching) | M5 :11700/:11902 |
| llama-server | slot-based concurrency (the 225 tok/s @8 result) | MSI (bench) |
| Qdrant + BGE-M3 | ONE vector engine — listings + knowledge, hybrid + rerank | M5 :11903 |
| Valkey | L1 events · session pub/sub | M5 :11905 |
| DuckDB | Tier-0 exact counts + deterministic CRE calc | M5 |
| XGBoost | price estimator — banded, R² gate, monthly retrain | M5 :11600 |
| DataGate | custody: passport + vault + journal, permanent | M5 :11951 |
| SearXNG | Tier-3 web fallback, cited | Studio :8787 |
| logging-router | every turn → S3 · failover · model affinity | Studio :8090 |
| S3 | query-log corpus · fleet registry · training heartbeat | AWS |
| Caddy + Cognito | edge TLS + team-only SSO (hutfin-portal group) | gateway |
Server checking…
These act on the currently active Mac (named above). To switch which Mac serves, or turn an individual Mac on/off, use the Server Fleet tab.
Pull a model
Downloads from the Ollama registry onto your Mac. Big models take a while.
Installed models —
MODEL SETTINGS
Defaults are Ollama's. Recommended is the model vendor's guidance for hutfin-ai-m5:1.0 (temperature 0.7, top_p 0.8, top_k 20) — what this panel uses. Leave a custom field blank to fall back to the Ollama default.
| SETTING | DEFAULT | RECOMMENDED | YOUR VALUE |
|---|
Save settings keeps these as your per-request options in this browser. Apply to server bakes temperature/top_p/top_k/context into the selected model on its Mac for everyone (persistent). The system prompt also ships as the built-in default, so every device starts with it.
AVAILABILITY SCHEDULE
Keeps this model loaded during the window via a cron job on your Mac. Runs even when this app is closed. Times use your Mac's local clock.
No schedule set for this model.
Connect from anywhere
All traffic flows: your browser → CloudFront (WAF + TLS) → Caddy on the AWS gateway → reverse SSH tunnel → Ollama on the Mac. Requests are rate-limited at the edge; backend origins never reach the browser bundle.
none — public endpointList models Chat (streaming)
Keys are stored on the gateway and listed here for the authenticated admin. Copy a key when you create it. (The public chat path is open; these keys are for the developer API surface.)
Server Fleet Macs serving models, reached via reverse tunnels
Add a server
Sessions Every user session — logged to S3 (location fills in once CloudFront fronts it)
Model Console Fixed commands only — list · ps · load · unload · pull · run
Type "help" and press Enter.