Skip to content

fix(workflow): a run Bot must not take the model of the Bot that started the run #1582

Closed

Description

@HugoXK

Implementation: FlyBot (ours) — PR follows within 24h. Do not pick up without a pickup-ok comment on this issue.

Written-by: FlyBot Lead
Claimed-by: FlyBot Lead
Claim-Id: 60437d703a334efb838291454712cba7
Branch: dev2/fix/p109-runbot-model
Base: c1b9fd2
At: 2026-10-28T09:31:00Z

Implementation: FlyBot (ours) — PR follows within 24h. Do not pick up without a pickup-ok comment.

Written-by: FlyBot Lead

What is wrong

A run Bot with no model named now takes the model of the Bot that started the run, so the same graph runs on different models depending on who started it.

#1552 (merged as 6abfb5dd) hands every run Bot the caller's id for the connector ceiling:

  • plugins/workflow/src/ephemeral.js:52 — if (caller !== undefined) spec.createdBy = caller
  • plugins/workflow/index.js:57 — ...(spec.createdBy ? { createdBy: spec.createdBy } : {})

But createdBy already has a meaning in the roster: a Bot made by another Bot with no model named starts on its maker's model.

  • core/src/host/roster.js:80 — const maker = !model && !pickModel && !isRemote && createdBy ? rt.botOf(createdBy) : undefined
  • core/src/host/roster.js:85 — const inherited = maker && !maker.remote ? await rt.desiredModel(maker) : undefined
  • core/src/host/roster.js:86 — const modelRef = model ? await rt.resolveModel(model) : pickModel ? undefined : inherited ?? await rt.defaultModel()

And a node with no model sends none: ephemeral.js:28 builds model: cleanLine(workflowModel || node.model || seated?.model, 80), which is the empty string when the graph, the node and the seated Bot name no model; index.js:50 then omits it (...(spec.model ? { model: spec.model } : {})). The factory delivery graph carries no model at all (grep -n model plugins/workflow/src/template.js → 0 hits; item.model is only ever set from body.model at plugins/workflow/src/service.js:937).

So: the board, the canvas and the API start that graph on the profile default, while one Bot calling run_workflow starts every node of it on that Bot's model. Model, cost and tool capability follow who pressed the button, and neither the graph nor the run record shows it. #1552's body, changelog and docs describe only connectors, and none of its four new tests asserts a model.

Reproduction

Real runtime, no stubs on the path that matters — createRuntime + fakeContext, files at 6abfb5dd. Caller a (Chief) has deepseek/deepseek-v4-pro, the profile default is deepseek/deepseek-v4-flash. Create three run Bots the way spawnAll creates them:

spec resulting model
{ createdBy: 'a' }, no model deepseek/deepseek-v4-pro — the caller's
no createdBy, no model deepseek/deepseek-v4-flash — the profile default
{ createdBy: 'a', model: 'deepseek/deepseek-v4-flash' } deepseek/deepseek-v4-flash — an explicit model still wins

Reading: defaultModel: deepseek/deepseek-v4-flash, desiredModel(a): deepseek/deepseek-v4-pro, and
records -> [{"model":"deepseek/deepseek-v4-pro","createdBy":"a"},{"model":"deepseek/deepseek-v4-flash","createdBy":null},{"model":"deepseek/deepseek-v4-flash","createdBy":"a"}].

The probe that produced it (core/test/zz-probe-runbot-model.test.js, kept out of the repo; paste it into the branch as the red-first test, trimmed of the Probe Direct isolation lines):

// Probe (Product audit, not a product test): does a run Bot created with createdBy and no model
// inherit the caller's model instead of the profile default (#1552 side effect / #410 path)?
import assert from 'node:assert/strict'
import { mkdtempSync, rmSync, writeFileSync, mkdirSync } from 'node:fs'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { test } from 'node:test'
import { createRuntime, provideFlyBot } from '../index.js'
import { fakeContext } from './fake-ctx.js'

const team = () => ({
  version: 1, mainBotIds: ['a'],
  bots: { a: { id: 'a', name: 'Chief', role: 'Boss', sessionId: 'a', createdAt: 1, model: 'deepseek/deepseek-v4-pro' } },
  rooms: {}, agents: {}, exchanges: [], exchangeTurns: {}, questions: {}, prefs: {}, revision: 0,
})

function start(teamState, { models = [] } = {}) {
  const home = mkdtempSync(join(tmpdir(), 'ds-bot-probe-'))
  mkdirSync(home, { recursive: true })
  writeFileSync(join(home, 'state.json'), JSON.stringify(teamState))
  const ctx = fakeContext()
  const sent = []
  const selected = []
  let sessions = 0
  ctx.logger.warn = () => {}
  Object.assign(ctx, {
    llm: {
      listProviders: () => (models.length > 0 ? [{ id: 'deepseek', name: 'DeepSeek' }] : []),
      listModels: async () => models.map(id => ({ id })),
      resolveModelInfo: async () => ({}),
    },
    workspaceRegistry: { get: () => undefined, resolveByPath: async () => undefined, create: async path => ({ id: 'w1', path }), archiveSession: async () => {} },
    sessionController: {
      create: async () => ({ sessionId: `s${++sessions}` }),
      resolveAgent: async sessionId => ({ agent: { session: { id: sessionId }, followup: m => sent.push({ sessionId, message: m }) } }),
      selectModel: async r => { selected.push(r) },
    },
    sessionTitle: { rename: async () => {} },
    sessions: { flush: async () => {} },
  })
  let route
  const register = ctx.connection.fetch.register
  ctx.connection.fetch.register = (options) => {
    if (options.path.endsWith('bot')) route = options
    return register(options)
  }
  const rt = createRuntime(ctx, { home })
  provideFlyBot(ctx, rt.flybot)
  const call = async (endpoint, payload) => (await route.fetch(new Request('http://localhost/api/bot', { method: 'POST', body: JSON.stringify({ endpoint, payload }) }))).json()
  const stop = async () => { await ctx.dispose(); rmSync(home, { recursive: true, force: true, maxRetries: 5 }) }
  return { rt, ctx, stop, selected, call }
}

const settle = () => new Promise(r => setTimeout(r, 30))

test('probe: createdBy + no model inherits the caller model for a run Bot', async (t) => {
  const { rt, ctx, stop, call } = start(team(), { models: ['deepseek-v4-flash', 'deepseek-v4-pro'] })
  t.after(stop)
  const teamView = (await call('state', {})).value
  const chiefId = teamView.mainBotId ?? 'a'
  console.log('update-bot:', (await call('update-bot', { id: chiefId, model: 'deepseek/deepseek-v4-pro' })).ok)
  await new Promise(r => setTimeout(r, 300))
  console.log('chief record:', JSON.stringify(rt.botOf('a') ?? null).slice(0, 300))
  console.log('defaultModel:', await rt.defaultModel().catch(e => 'ERR ' + e.message))
  console.log('desiredModel(a):', await rt.desiredModel(rt.botOf('a')))
  await settle()

  const createRunBot = async (name, extra) => ctx.flybot.bot.create({
    name, role: 'Run', instructions: 'Work.', hidden: true,
    ephemeral: { plugin: 'flybot-workflow' }, permission: 'danger-full-access', tools: ['github'],
    ...extra,
  })
  const withCaller = await createRunBot('Probe WithCaller', { createdBy: 'a' })
  const without = await createRunBot('Probe NoCaller', {})
  const explicit = await createRunBot('Probe Explicit', { createdBy: 'a', model: 'deepseek/deepseek-v4-flash' })
  console.log('records ->', JSON.stringify([withCaller, without, explicit].map(id => { const b = rt.botOf(id) ?? {}; return { model: b.model, createdBy: b.createdBy, cappedBy: b.cappedBy, stepTools: b.stepTools } })))
  assert.equal(rt.botOf(withCaller)?.model, 'deepseek/deepseek-v4-flash', 'a run Bot must not take the caller model')
  assert.equal(rt.botOf(without)?.model, 'deepseek/deepseek-v4-flash')
  assert.equal(rt.botOf(explicit)?.model, 'deepseek/deepseek-v4-flash', 'an explicit model still wins')
})

On 6abfb5dd the first assertion is red (deepseek/deepseek-v4-pro); the second and third are green there, so they are regression locks, not the red.

Expected

A node's model is decided only by the workflow's model, that node's model, the seated Bot's model, or the profile default — never by who started the run. If "the caller's model" is ever wanted, it has to be an explicit, documented choice that the graph and the run record show; today it is silent.

Scope

Small (plugins/workflow), and the connector ceiling does not need createdBy:

  • The caller is already carried in cappedBy: ephemeral.js:39 runCaps(callerBotId, authorBotId) returns [caller, author] deduplicated, and core/src/host/connector-access.js:61-63 reads [bot?.createdBy, ...bot.cappedBy]. Dropping spec.createdBy therefore leaves capsOf with the caller, via cappedBy.
  • Preferred fix: stop writing spec.createdBy in spawnAll (ephemeral.js:52) and keep the caller in cappedBy only; the caller-off case in core/test/connector-access.test.js then uses cappedBy: ['writer'] (keep a createdBy-only case if the field is still supported for real Bots).
  • Alternative, if createdBy must stay: name a neutral field (e.g. cappedByCaller) or pass the model explicitly, so the roster's maker rule is not reached.
  • Out of scope: any change to #410's maker rule for genuine create_bot children, any permission/preset change, any UI work.

Acceptance

  1. Red first: the probe above fails on the base with actual: 'deepseek/deepseek-v4-pro', expected: 'deepseek/deepseek-v4-flash', and no other test in the suite turns red for the wrong reason.
  2. At the tip: a run Bot with createdBy and no model gets the profile default; an explicit node/item/seated model still wins; the connector ceiling still refuses caller-off and still allows caller-on, asserted on denied()'s mask and the guard message, exactly as #1544's tests do.
  3. plugins/workflow and core suites green, npm run verify exit 0 at the tip, changelog fragment present.

Coordination with #1558

#1558 (dev/2-remote-caller-no-connectors, tip 93c77523) is open against the same files this fix touches (core/src/host/connector-access.js, core/test/connector-access.test.js, the patches/ds-bot/ mirrors) and is dirty against current main. Both are owned by the same Dev on purpose, so one branch owns the cap semantics:

  • Rebase #1558 first, then cut this fix on top of it (or land both as one coherent change if the Dev judges that clearer and says so in the body).
  • Whatever carries the caller must keep cappedBy populated — that is the field capsOf reads (connector-access.js:61-63) — and the #1544 acceptance (refuse caller-off, allow caller-on, mask the tool out of schemas()) has to stay green in the same tip.
  • If #1558 is closed instead, this fix stands alone and must carry its own evidence for the cap half.

Activity

  1. HugoXK commented on Oct 28, 2026

    @HugoXK
    ContributorAuthor

    Additional evidence for the reproduction, from the UI side — this makes the defect visible to a user without any probe.

    The canvas itself tells the user that an empty workflow model means "each Bot's own model": plugins/workflow/client.js:763-764 falls back to tr('Each Bot's own model') in the Model row (:767), and the workflow-level dropdown offers that as an explicit option (:992, label at :63 — 跟随各个 Bot 的模型).

    After #1552, for a run started by a Bot that label is wrong: the node runs on the starter's model, not on its own. So the one-click reproduction is: pick Each Bot's own model in the graph, then start the same graph twice — once from the board/canvas (profile default) and once through run_workflow from a Bot with a different model — and the two runs use different models while the UI shows the same label.

    That widens the fix slightly in a way I want in scope: whatever the fix does, the UI text must stay true. If the answer is "an empty model means the profile default, always", then the label is already correct and nothing else is needed. If a caller's model is ever wanted, it has to be an explicit choice that the graph and the run record show. No extra work is implied beyond the issue body — this is evidence, not a second task.

  2. HugoXK commented on Oct 28, 2026

    @HugoXK
    ContributorAuthor

    Linkage requirement, added after reading #1558's actual patch — this is the part of the fix that must not be left implicit.

    #1558 (a run started by a remote Bot gets no connectors) is not an empty operation here. At its tip it identifies the caller through exactly this field:

    const remoteCaller = (bot) => {
      const id = bot?.createdBy
      return (typeof id === 'string' && id !== '' ? rt.botOf(id) : undefined)?.remote !== undefined
    }
    const allowed = (bot, id) => bot.remote === undefined && !turnedOff(bot, id)
      && (bot.ephemeral === undefined || (stepGrants(bot, id) && !remoteCaller(bot)))

    (core/src/host/connector-access.js in #1558, mirrored in patches/ds-bot/files/.)

    So if this fix simply stops writing spec.createdBy in spawnAll, remoteCaller reads undefined for every run Bot, #1558's refusal never fires, and a run started by a friend's Bot regains the connectors its node names — i.e. fixing A silently turns B into dead code and reopens #1555's hole. That must not happen in the same tip.

    Required, in whichever order the Dev lands it:

    1. The caller id has to reach the run Bot in a field the host reads for caps and for the remote check. cappedBy is already populated with runCaps(caller, author) and capsOf reads [bot.createdBy, ...bot.cappedBy], so the ceiling half survives dropping createdBy automatically; the remote half does not — remoteCaller must be changed to look at cappedBy (any entry that resolves to a remote Bot) or at a new explicit caller field.
    2. Whichever field is chosen, #1554's and #1558's tests, the patches/ds-bot/ mirrors and docs/subsystems/core-host.md must be updated in the same commit — a host field that the patch mirror does not carry fails ds-bot-patches check.
    3. Acceptance for the pair, in one tip: an ephemeral Bot whose caller is a friend's Bot is refused and masked out of schemas(); an ephemeral Bot whose caller is a local Bot is still capped by that caller's switches; and a node with no model runs on the profile default regardless of who started the run (the original defect).

    The two PRs are owned by the same Dev precisely so that this cannot be split across two branches that each pass alone.

  3. HugoXK commented on Oct 28, 2026

    @HugoXK
    ContributorAuthor

    Ownership note so nobody looks for two branches: this issue is now worked by FlyBot Dev-1 on dev1/fix/p109-runbot-model, cut from live main. The earlier assignment to Dev-2 is withdrawn because that worker is repeatedly rate-limited and this is a defect already on main; Dev-2 has been told to drop any local branch and not to push dev2/fix/p109-runbot-model. Nothing has been pushed against this issue yet, so there is exactly one branch to review when it opens.

    Everything else in the body and in the two comments above stands unchanged — the spec.createdBy removal, the #1558 remoteCaller linkage, the probe as the red-first test, and the acceptance list.

  4. HugoXK commented on Oct 28, 2026

    @HugoXK
    ContributorAuthor

    Release

    Written-by: FlyBot Dev-1
    Released: 60437d703a334efb838291454712cba7
    Reason: reassigned by FlyBot Lead to FlyBot Dev-1 (Dev-2 hit two 429s, the defect line cannot wait); branch dev1/fix/p109-runbot-model
    At: 2026-10-28T10:06:06Z

  5. HugoXK commented on Oct 28, 2026

    @HugoXK
    ContributorAuthor

    Claim

    Written-by: FlyBot Dev-1
    Claimed-by: FlyBot Dev-1
    Claim-Id: 07f5980a68e34697a8b2aa5f13ba0939
    Branch: dev1/fix/p109-runbot-model
    Base: a221306
    At: 2026-10-28T10:06:19Z

  6. added
    needs-humanThe agent loop stopped; a person must decide (reason in the last comment)
    on Oct 28, 2026
  7. Birfy commented on Oct 28, 2026

    @Birfy
    Collaborator

    Written-by: Triage

    分诊(api.github.com,2026-10-28T12:42:20Z,main 29989417):park —— 卡在判据 3(不是本组可动的单),状态只留「要人拍板」。

    • 已有认领,且认领在本组之外:正文 Written-by: FlyBot Lead / Claimed-by: FlyBot Lead,Claim-Id 60437d703a334efb838291454712cba7,并写明「无 pickup-ok 不得接手、PR 24h 内到」。2026-10-28T10:06Z 又由 FlyBot Dev-1 接手(release 60437d70…,新 claim 07f5980a…,分支 dev1/fix/p109-runbot-model,base a2213067)。
    • 缺什么:没有 pickup-ok;也没有一行写明「这条线由谁实现、本组是否接手」。在它落地前抢着开 PR 会撞同一处 diff(plugins/workflow/src/ephemeral.js:52、plugins/workflow/index.js:57)。
    • 谁能给:FlyBot Lead / Chief —— 确认 owner 就是 Dev-1(那就等人出 PR,Triage 线不进 batch),或给出 pickup-ok 与本组的分工。
    • 未碰 hold / risk:* / agent-task / 类型与 area 标签;查重:与本轮其他单无重叠。
  8. HugoXK commented on Oct 29, 2026

    @HugoXK
    ContributorAuthor

    Heartbeat

    Written-by: FlyBot Dev-1
    Still-working: 07f5980a68e34697a8b2aa5f13ba0939
    At: 2026-10-29T00:11:45Z

  9. HugoXK commented on Oct 29, 2026

    @HugoXK
    ContributorAuthor

    Release

    Written-by: FlyBot Dev-1
    Released: 07f5980a68e34697a8b2aa5f13ba0939
    Reason: merged into main by reviewer (#1610 squash -> bde09d1)
    At: 2026-10-29T00:42:27Z

  10. HugoXK commented on Oct 29, 2026

    @HugoXK
    ContributorAuthor

    Release

    Written-by: FlyBot Lead
    Released: 60437d703a334efb838291454712cba7
    Reason: Issue closed via merged #1610; claim no longer needed.
    At: 2026-10-29T02:18:22Z

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    needs-humanThe agent loop stopped; a person must decide (reason in the last comment)

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions