rssnyder/spark-utilities
1
0
Fork 0
Code Issues Pull requests 2 Projects Releases Packages Wiki Activity Actions

router + litellm: all of the Forge models, and its LiteLLM routes (replaces #1) #2

Open
moserhu wants to merge 0 commits from forge-models into master
pull from: forge-models
merge into: rssnyder:master
Conversation 0 Commits - Files changed -
moserhu commented 2026-10-11 18:02:43 -05:00
Collaborator
Copy link

@rssnyder this replaces #1. It carries #1's two commits (forge-hidream, forge-liveportrait) plus one more, and records everything I changed on your Spark this weekend.

Why #1 says "broken due to missing fork information"

There's no fork involved. Forgejo shows that message when a PR's stored head (refs/pull/1/head) doesn't match its branch. #1's head is still 565deae, its first commit. The second commit (257a297, liveportrait) was pushed 29 minutes later, and Forgejo never moved the PR to it. That update runs on Forgejo's background queue, which has been stuck since the 09-22 outage (Actions runs are stuck the same way). This PR's branch was pushed complete before the PR was opened, so it should read correctly. If it still shows broken, git merge origin/forge-models and a push does the same thing. The queue itself needs a restart from you as admin.

What happened on the Spark

~/llama-swap/config.yaml was pushed on 2026-10-07 without the Forge's entries, and llama-swap then answered 404 model not found for them.

  • Every Forge picture failed (8 jobs, 22 tries on 10-11).
  • Your deepseek got blocked. spark-swapper still listed those models, so each time the Forge worker polled a missing one, the swapper treated it as loading. It refused deepseek for each 15-minute wait: 220 refusals on 10-11. That's fixed in spark-swapper (moserhu/spark-swapper@0535b6a), and the worker now gives up on a missing model at once.

What I changed on the Spark, live

Each change has a backup beside the file and was made with the Spark otherwise idle.

When (UTC) File Change Restart
10-10 21:21 ~/llama-swap/config.yaml + forge-pixal3d, forge-trellis2 and group members (.bak-worlds-forge-20261010T212118Z) SIGHUP reload
10-11 ~20:33 ~/llama-swap/config.yaml + forge-hidream, forge-ideogram4 and group members (.bak-forge-pictures-…) SIGHUP reload (deepseek cold-started; it was idle)
10-11 18:57 ~/litellm/config.yaml + the Forge pass-through routes (.bak-forge-passthrough-20261011T185757Z) spark-litellm restart, ~1 min
10-11 21:58 ~/litellm/config.yaml + read-only /swapper/status (.bak-swapper-status-20261011T215838Z) spark-litellm restart, ~1 min

I also minted two keys in your LiteLLM, each limited by allowed_routes to its own routes. Both get 403 on your chat models.

  • forge-worker: the Forge's worker.
  • worlds-assets: Worlds' asset workshop.

The point is that our Spark traffic goes through your LiteLLM, logged per key. Neither key reaches spark-swapper directly.

What this PR changes

  • router/config.yaml:
    • forge-pixal3d, forge-trellis2 and forge-ideogram4 are added in spark-memory, beside #1's forge-hidream and forge-liveportrait.
    • Their commands match what runs live, word for word.
    • The file passes llama-swap v249's own parse on a loopback port; nothing was loaded.
  • litellm/config.yaml: the pass-through routes, identical to what has been live since 10-11.
  • From #1:
    • forge-hidream gets ttl: 600 and a 900 s response timeout. Live has ttl: 900 and no timeout.
    • forge-liveportrait isn't live yet. Its image is built on the Spark.
    • router/spark-swapper.toml's models gains forge-liveportrait.

Differences that are yours, not mine

Live has qwen3.6-35b-a3b-nvfp4 in ~/litellm/config.yaml and in /etc/spark-swapper/swapper.toml's models, and master has neither. I left both alone. If you deploy this repo's files as they are, you'd drop them, so add them first if you want to keep them.

To deploy (when deepseek is idle)

  • router/config.yaml → ~/llama-swap/config.yaml, then systemctl kill -s HUP --kill-whom=main llama-swap. The reload restarts deepseek (~3 min).
  • litellm/config.yaml → ~/litellm/config.yaml, then systemctl restart spark-litellm (~1 min).

🤖 Generated with Claude Code

@rssnyder this replaces #1. It carries #1's two commits (forge-hidream, forge-liveportrait) plus one more, and records everything I changed on your Spark this weekend. ## Why #1 says "broken due to missing fork information" There's no fork involved. Forgejo shows that message when a PR's stored head (`refs/pull/1/head`) doesn't match its branch. #1's head is still `565deae`, its first commit. The second commit (`257a297`, liveportrait) was pushed 29 minutes later, and Forgejo never moved the PR to it. That update runs on Forgejo's background queue, which has been stuck since the 09-22 outage (Actions runs are stuck the same way). This PR's branch was pushed complete before the PR was opened, so it should read correctly. If it still shows broken, `git merge origin/forge-models` and a push does the same thing. The queue itself needs a restart from you as admin. ## What happened on the Spark `~/llama-swap/config.yaml` was pushed on 2026-10-07 without the Forge's entries, and llama-swap then answered 404 `model not found` for them. - **Every Forge picture failed** (8 jobs, 22 tries on 10-11). - **Your deepseek got blocked.** spark-swapper still listed those models, so each time the Forge worker polled a missing one, the swapper treated it as loading. It refused deepseek for each 15-minute wait: 220 refusals on 10-11. That's fixed in spark-swapper ([moserhu/spark-swapper@0535b6a](https://github.com/moserhu/spark-swapper/commit/0535b6a)), and the worker now gives up on a missing model at once. ## What I changed on the Spark, live Each change has a backup beside the file and was made with the Spark otherwise idle. | When (UTC) | File | Change | Restart | |---|---|---|---| | 10-10 21:21 | `~/llama-swap/config.yaml` | + `forge-pixal3d`, `forge-trellis2` and group members (`.bak-worlds-forge-20261010T212118Z`) | SIGHUP reload | | 10-11 ~20:33 | `~/llama-swap/config.yaml` | + `forge-hidream`, `forge-ideogram4` and group members (`.bak-forge-pictures-…`) | SIGHUP reload (deepseek cold-started; it was idle) | | 10-11 18:57 | `~/litellm/config.yaml` | + the Forge pass-through routes (`.bak-forge-passthrough-20261011T185757Z`) | `spark-litellm` restart, ~1 min | | 10-11 21:58 | `~/litellm/config.yaml` | + read-only `/swapper/status` (`.bak-swapper-status-20261011T215838Z`) | `spark-litellm` restart, ~1 min | I also minted two keys in your LiteLLM, each limited by `allowed_routes` to its own routes. Both get 403 on your chat models. - `forge-worker`: the Forge's worker. - `worlds-assets`: Worlds' asset workshop. The point is that our Spark traffic goes through your LiteLLM, logged per key. Neither key reaches spark-swapper directly. ## What this PR changes - `router/config.yaml`: - `forge-pixal3d`, `forge-trellis2` and `forge-ideogram4` are added in `spark-memory`, beside #1's `forge-hidream` and `forge-liveportrait`. - Their commands match what runs live, word for word. - The file passes llama-swap v249's own parse on a loopback port; nothing was loaded. - `litellm/config.yaml`: the pass-through routes, identical to what has been live since 10-11. - From #1: - `forge-hidream` gets `ttl: 600` and a 900 s response timeout. Live has `ttl: 900` and no timeout. - `forge-liveportrait` isn't live yet. Its image is built on the Spark. - `router/spark-swapper.toml`'s `models` gains `forge-liveportrait`. ## Differences that are yours, not mine Live has `qwen3.6-35b-a3b-nvfp4` in `~/litellm/config.yaml` and in `/etc/spark-swapper/swapper.toml`'s `models`, and `master` has neither. I left both alone. If you deploy this repo's files as they are, you'd drop them, so add them first if you want to keep them. ## To deploy (when deepseek is idle) - `router/config.yaml` → `~/llama-swap/config.yaml`, then `systemctl kill -s HUP --kill-whom=main llama-swap`. The reload restarts deepseek (~3 min). - `litellm/config.yaml` → `~/litellm/config.yaml`, then `systemctl restart spark-litellm` (~1 min). 🤖 Generated with [Claude Code](https://claude.com/claude-code) 
HiDream-O1-Image-Dev (MIT) behind the Forge's picture server image
(forge-pic:hidream-serve, already built on the Spark). spark-swapper
already lists forge-hidream, but llama-swap didn't have it, so requests
for it came back 404. Used by the Forge and by Truesight's "generate a
face" (two portraits per request).

- In spark-memory: it needs the GPU alone (33 GB weights), so loading it
  evicts deepseek; ~10-16 s to load, ~36 s per 1024 picture.
- unlisted; reached only through /upstream/forge-hidream/.
- ttl 600, then the swapper's default (deepseek) returns. Truesight's
  runner unloads it and asks for deepseek as soon as its pictures are done.
- Runs as hunter (1001), weights read-only, one cache folder, 127.0.0.1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
LivePortrait re-poses a saved portrait into the mouth, blink and
expression frames the robot's screen plays (forge-face:liveportrait-serve,
already built on the Spark). In spark-memory like forge-hidream, ttl 300,
and listed in spark-swapper's models.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
router/config.yaml: forge-pixal3d, forge-trellis2 and forge-ideogram4 in
spark-memory, beside forge-hidream and forge-liveportrait from the commits
before this. The live config on the Spark lost all the Forge entries on
2026-10-07; they were put back by hand on 10-10 and 10-11, so this makes
the repo say what runs. Their commands match the live ones word for word,
and the file passes llama-swap v249's own parse on a loopback port.

litellm/config.yaml: pass-through routes for the four Forge model servers
and a read-only /swapper/status, so the Forge's worker and Worlds reach
the Spark through LiteLLM with their own route-limited keys. Identical to
what has been live since 2026-10-11.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch is already included in the target branch. There is nothing to merge.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin forge-models:forge-models
git switch forge-models

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch master
git merge --no-ff forge-models
git switch forge-models
git rebase master
git switch master
git merge --ff-only forge-models
git switch forge-models
git rebase master
git switch master
git merge --no-ff forge-models
git switch master
git merge --squash forge-models
git switch master
git merge --ff-only forge-models
git switch master
git merge forge-models
git push origin master
Sign in to join this conversation.
Reviewers
No reviewers
rssnyder
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
rssnyder/spark-utilities!2
No description provided.