router: add forge-hidream and forge-liveportrait (Hunter's, unlisted)

moserhu commented

2026-10-09 19:00:12 -05:00

Collaborator

Hey Riley, one llama-swap entry so spark-swapper can actually load forge-hidream.

Why: spark-swapper already lists forge-hidream, but llama-swap's config doesn't have it, so a request for it gets queued and then comes back 404. I'm using it for Truesight (the robot head): you can ask the page to "generate a face", and HiDream paints two front-facing portraits to pick from. The Forge's picture area uses the same server.

What it is: HiDream-O1-Image-Dev (MIT) in forge-pic:hidream-serve, the image I already built on the Spark. One small HTTP server: GET /health, POST /paint (multipart in, zip of PNGs out). It's unlisted, so no chat client shows it, and it's only reached through /upstream/forge-hidream/.

How it shares the box:

It's in spark-memory, so loading it evicts deepseek, the same as the coder models. It needs the GPU alone (33 GB of weights). It loads in about 10-16 s, and a picture takes about 36 s at 1024.
ttl 600: after 10 idle minutes it unloads and the swapper brings deepseek back.
Truesight's runner doesn't wait for that. When its pictures are done, it unloads forge-hidream and asks for deepseek straight away.
Runs as hunter (uid 1001). Weights are read-only from /home/hunter/forge/weights, the only writable folder is /home/hunter/forge/cache, it's bound to 127.0.0.1, and it needs no network or Hugging Face token at runtime.

Heads-up: the config deployed on the Spark seems to differ from master. The swapper knows qwen3.6-35b-a3b-nvfp4, which master doesn't have. This PR only adds lines, so it should merge cleanly on top of whatever you have.

Test after deploy:

curl -s http://127.0.0.1:8081/upstream/forge-hidream/health

That should answer 200 once deepseek has been idle about 2 minutes, which is the swapper's handoff rule.

Second commit: forge-liveportrait. Truesight's face frames. LivePortrait (MIT code) takes a saved portrait and re-poses it into the mouth, blink and expression frames the robot's screen plays while someone watches.

The image forge-face:liveportrait-serve is already built on the Spark. It built on the CPU only, nothing ran on the GPU, and an import check passed. It's also tested end to end on a Mac.
It's small (about 630 MB of weights) but still takes the GPU alone in spark-memory. One face is about a minute. ttl 300, and the runner hands the slot back straight away.
It's added to spark-swapper's models list in router/spark-swapper.toml, so both configs ship together.
Its first load on the GB10 is the real test: cuDNN is off inside it, the same as the Forge's servers.

Update: the forge-liveportrait image also does an eye pass with Talking Head(?) Anime 3 for drawn and anime faces (code MIT, models CC BY 4.0). Same entry, same image name. Its weights are another ~700 MB, read-only under /home/hunter/forge/weights. It still takes the GPU alone, and one face takes a minute or two.

🤖 Generated with Claude Code

Hey Riley, one llama-swap entry so spark-swapper can actually load `forge-hidream`. **Why:** spark-swapper already lists `forge-hidream`, but llama-swap's config doesn't have it, so a request for it gets queued and then comes back **404**. I'm using it for Truesight (the robot head): you can ask the page to "generate a face", and HiDream paints two front-facing portraits to pick from. The Forge's picture area uses the same server. **What it is:** HiDream-O1-Image-Dev (MIT) in `forge-pic:hidream-serve`, the image I already built on the Spark. One small HTTP server: `GET /health`, `POST /paint` (multipart in, zip of PNGs out). It's `unlisted`, so no chat client shows it, and it's only reached through `/upstream/forge-hidream/`. **How it shares the box:** - It's in **`spark-memory`**, so loading it evicts deepseek, the same as the coder models. It needs the GPU alone (33 GB of weights). It loads in about 10-16 s, and a picture takes about 36 s at 1024. - **ttl 600**: after 10 idle minutes it unloads and the swapper brings deepseek back. - Truesight's runner doesn't wait for that. When its pictures are done, it unloads `forge-hidream` and asks for deepseek straight away. - Runs as `hunter` (uid 1001). Weights are read-only from `/home/hunter/forge/weights`, the only writable folder is `/home/hunter/forge/cache`, it's bound to 127.0.0.1, and it needs no network or Hugging Face token at runtime. **Heads-up:** the config deployed on the Spark seems to differ from master. The swapper knows `qwen3.6-35b-a3b-nvfp4`, which master doesn't have. This PR only adds lines, so it should merge cleanly on top of whatever you have. **Test after deploy:** ``` curl -s http://127.0.0.1:8081/upstream/forge-hidream/health ``` That should answer 200 once deepseek has been idle about 2 minutes, which is the swapper's handoff rule. --- **Second commit: `forge-liveportrait`.** Truesight's face frames. LivePortrait (MIT code) takes a saved portrait and re-poses it into the mouth, blink and expression frames the robot's screen plays while someone watches. - The image `forge-face:liveportrait-serve` is already built on the Spark. It built on the CPU only, nothing ran on the GPU, and an import check passed. It's also tested end to end on a Mac. - It's small (about 630 MB of weights) but still takes the GPU alone in `spark-memory`. One face is about a minute. `ttl 300`, and the runner hands the slot back straight away. - It's added to spark-swapper's `models` list in `router/spark-swapper.toml`, so both configs ship together. - Its first load on the GB10 is the real test: cuDNN is off inside it, the same as the Forge's servers. **Update:** the `forge-liveportrait` image also does an eye pass with Talking Head(?) Anime 3 for drawn and anime faces (code MIT, models CC BY 4.0). Same entry, same image name. Its weights are another ~700 MB, read-only under `/home/hunter/forge/weights`. It still takes the GPU alone, and one face takes a minute or two. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

moserhu added 1 commit

2026-10-09 19:00:12 -05:00

router: add forge-hidream (Hunter's picture server, unlisted) 565deae611

HiDream-O1-Image-Dev (MIT) behind the Forge's picture server image
(forge-pic:hidream-serve, already built on the Spark). spark-swapper
already lists forge-hidream, but llama-swap didn't have it, so requests
for it came back 404. Used by the Forge and by Truesight's "generate a
face" (two portraits per request).

- In spark-memory: it needs the GPU alone (33 GB weights), so loading it
  evicts deepseek; ~10-16 s to load, ~36 s per 1024 picture.
- unlisted; reached only through /upstream/forge-hidream/.
- ttl 600, then the swapper's default (deepseek) returns. Truesight's
  runner unloads it and asks for deepseek as soon as its pictures are done.
- Runs as hunter (1001), weights read-only, one cache folder, 127.0.0.1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

moserhu changed title from ~~router: add forge-hidream (Hunter's picture server, unlisted)~~ to router: add forge-hidream and forge-liveportrait (Hunter's, unlisted)

2026-10-09 19:29:08 -05:00

moserhu referenced this pull request

2026-10-11 18:02:43 -05:00

router + litellm: all of the Forge models, and its LiteLLM routes (replaces #1) #2

moserhu commented

2026-10-11 18:02:57 -05:00

Author

Collaborator

Replaced by #2, which carries both commits here plus the rest of the Forge models and its LiteLLM routes. This one shows broken because Forgejo never moved its stored head to the second commit; #2 explains it.

moserhu closed this pull request

2026-10-11 18:02:57 -05:00

moserhu referenced this pull request

2026-10-11 19:33:21 -05:00

litellm: a route for forge-liveportrait (Truesight face frames) #3

Pull request closed

Please reopen this pull request to perform a merge.

Rows
Columns

router: add forge-hidream and forge-liveportrait (Hunter's, unlisted) #1

Pull request closed