diff --git a/.github/scripts/snippets/README.md b/.github/scripts/snippets/README.md new file mode 100644 index 000000000..6c2b628e1 --- /dev/null +++ b/.github/scripts/snippets/README.md @@ -0,0 +1,75 @@ +# Partner-model Code pages + +Every Router-addressable partner model can carry a `code.mdx` page under +`development/comfy-router/models///`, showing how to call the +model through Comfy Router from Python, TypeScript and cURL. + +These pages live in the **developer** section, not under `tutorials/`: the +tutorials tree is for end users driving the nodes in the app, and mixing API +reference into it makes both harder to find. Each tutorial page links across to +its Code page instead. + +`code.mdx` is **generated**. The source of truth is the `code.yaml` next to it; +the page shape lives in `gen-code-pages.ts` and nowhere else. + +```text +development/comfy-router/models/black-forest-labs/flux-1-kontext/ + code.yaml spec: name, variants, example body, result path <- edit this + code.mdx generated from code.yaml <- never edit + +tutorials/partner-nodes/black-forest-labs/flux-1-kontext.mdx Overview (hand-written, links to the Code page) +``` + +## Commands + +```bash +pnpm code-pages:gen # regenerate every code.mdx from its code.yaml +pnpm code-pages:check # CI: fail if any code.mdx is stale, and syntax-check the emitted snippets +``` + +`code-pages-check.yml` runs the check on any PR touching a spec, a generated +page, the shared Router snippets or the generator. + +## Schema sections + +Every Code page ends with a Schema section (Input, Output) and an Examples +section (Input, Output). They render from `router-schemas//.json`, +which is the exact body of `GET https://api.comfy.org/v2/models///openapi.json` +(a standalone OpenAPI document; the spec-sync bot drops these in, do not hand +write them). When the file is absent, or reports +`x-comfy-input-schema-authored: false`, the page falls back to the spec's +`input` / `output` JSON Schema blocks (rendered as the same ParamField / +ResponseField list) and `example` / `result.example`, with a note that Router +has not published the schema yet. Variants that resolve to the same +schema share one block; variants with different schemas get tabs. + +## Provider drift check + +`pnpm code-pages:check-providers` (`check-provider-schemas.ts`) fetches each +provider's own published API specification (`provider_spec.url` in the +`code.yaml`: BFL and Ideogram publish OpenAPI, Google a discovery document) +and compares the documented `input` / `output` against it: unknown fields, +type, default, enum, bound and required-ness mismatches fail; provider fields +we leave undocumented are warnings (`--strict` makes them errors, `omit:` lists +the deliberate ones). It runs in CI as the `provider-schemas` job, so the +fallback schemas are tested rather than trusted until Router publishes its own. + +## Adding a model + +1. Create `code.yaml` in the model's directory (copy the Kontext one). +2. Set `variants` to the Router model IDs (`provider/model`, from `GET /v2/models`). +3. Write `summary`: one sentence describing what the model does. It becomes the + opening line of the generated page's body (`API Reference for . + `) unless `intro` overrides it. +4. Put the smallest request body that produces a result in `example`. A value of + `"@file:"` is read from disk and base64 encoded by every snippet. +5. Set `result.path` to where the output lives in the provider's native + response and `result.example` to a representative response. +6. Run `pnpm code-pages:gen`, add the page to the model's group in `docs.json`, + and link it from the overview's "Use it" cards. + +Python, TypeScript and cURL are all emitted from the same `example`, so the +three snippets cannot disagree about the body. `--validate` compiles each +emitted snippet (`py_compile`, `bun build`, `bash -n`); nothing is executed and +nothing is billed. Live verification against Router is a separate, nightly, +credentialed job. diff --git a/.github/scripts/snippets/check-provider-schemas.ts b/.github/scripts/snippets/check-provider-schemas.ts new file mode 100644 index 000000000..0efa7c16c --- /dev/null +++ b/.github/scripts/snippets/check-provider-schemas.ts @@ -0,0 +1,203 @@ +#!/usr/bin/env bun +/** + * Check every code.yaml `input` / `output` schema against the PROVIDER's own + * published API specification, so the fields we document are the fields the + * provider actually accepts and returns. + * + * bun .github/scripts/snippets/check-provider-schemas.ts # report drift, exit 1 on errors + * bun .github/scripts/snippets/check-provider-schemas.ts --strict # also fail on fields we omit + * + * Each spec (or variant) names where its provider publishes the contract: + * + * provider_spec: + * url: https://api.bfl.ai/openapi.json # OpenAPI 3 document, or a Google discovery document + * operation: POST /v1/flux-kontext-pro # request body = this operation's application/json schema + * response_operation: GET /v1/get_result # optional: where the FINAL result shape lives (poll route) + * request: GenerateContentRequest # google-discovery: schema names instead of operations + * response: GenerateContentResponse + * omit: [webhook_url, webhook_secret] # provider fields we deliberately do not document + * + * Errors (exit 1): a documented field the provider does not have; a type, default, + * enum, bound or required-ness that disagrees with the provider. Warnings: provider + * fields we do not document (errors under --strict), and provider shapes that are + * opaque (`{}`) where we document structure. + */ +import { readFileSync } from "node:fs"; +import { join, dirname } from "node:path"; + +const ROOT = join(import.meta.dir, "../../.."); +const strict = process.argv.includes("--strict"); +const verbose = process.argv.includes("--verbose"); + +type ProviderSpec = { url: string; operation?: string; response_operation?: string; request?: string; response?: string; omit?: string[] }; + +const FETCH_TIMEOUT_MS = 20_000; +const cache = new Map>(); +async function fetchOnce(url: string): Promise { + const r = await fetch(url, { signal: AbortSignal.timeout(FETCH_TIMEOUT_MS) }); + if (!r.ok) throw new Error(`${url}: HTTP ${r.status}`); + return r.json(); +} +/** Transient = the request never produced an HTTP status (timeout, DNS, reset). Those are worth one retry; an HTTP error is not. */ +const transient = (e: unknown) => !(e instanceof Error) || !/: HTTP \d+$/.test(e.message); +function fetchDoc(url: string): Promise { + // A provider that never answers would otherwise hold this promise open for the job's whole runner limit. + if (!cache.has(url)) cache.set(url, fetchOnce(url).catch((e) => { if (!transient(e)) throw e; return fetchOnce(url); })); + return cache.get(url)!; +} + +// ---- normalise provider schemas (OpenAPI 3 or Google discovery) into plain JSON-schema-ish objects +function normalizer(doc: any) { + const isDiscovery = !!doc.schemas && !doc.openapi; + const comps: Record = isDiscovery ? doc.schemas : doc.components?.schemas ?? {}; + const seen = new Set(); + const norm = (s: any, depth = 0): any => { + if (!s || depth > 12) return {}; + if (s.$ref) { + const name = String(s.$ref).split("/").pop()!; + if (seen.has(name) && depth > 6) return { type: "object", opaque: true }; + seen.add(name); + return norm(comps[name], depth + 1); + } + if (s.anyOf || s.oneOf) { + const alts = (s.anyOf ?? s.oneOf).map((a: any) => norm(a, depth + 1)).filter((a: any) => a.type !== "null"); + if (alts.length === 1) return { ...alts[0], nullable: true, ...(s.default !== undefined ? { default: s.default } : {}) }; + // discriminated unions (FLUX 3 Video): union of properties, required = intersection + const props: Record = {}; let req: Set | null = null; + for (const a of alts) { + Object.assign(props, a.properties ?? {}); + const r = new Set(a.required ?? []); req = req ? new Set([...req].filter((x) => r.has(x))) : r; + } + const types = [...new Set(alts.map((a: any) => a.type).filter(Boolean))]; + return { type: types.length === 1 ? types[0] : types, properties: props, required: [...(req ?? [])], union: true, ...(s.default !== undefined ? { default: s.default } : {}) }; + } + if (s.allOf) return s.allOf.map((a: any) => norm(a, depth + 1)).reduce((acc: any, d: any) => ({ ...acc, ...d, properties: { ...(acc.properties ?? {}), ...(d.properties ?? {}) }, required: [...(acc.required ?? []), ...(d.required ?? [])] }), {}); + const out: any = { type: s.type }; + if ((s.type === "object" || s.type === undefined) && !s.properties && !s.items && !s.enum && !s.anyOf) out.opaque = true; + for (const k of ["default", "enum", "minimum", "maximum", "format", "description"]) if (s[k] !== undefined) out[k] = s[k]; + if (s.properties) { out.properties = {}; for (const [k, v] of Object.entries(s.properties)) out.properties[k] = norm(v, depth + 1); } + if (s.required) out.required = s.required; + if (s.items) out.items = norm(s.items, depth + 1); + if (isDiscovery && s.type === undefined && s.properties) out.type = "object"; + return out; + }; + return { norm, isDiscovery, comps }; +} + +function findOperation(doc: any, op: string) { + const [method, path] = op.split(" "); + const node = doc.paths?.[path]?.[method.toLowerCase()]; + if (!node) throw new Error(`operation ${op} not found in ${doc.info?.title ?? "spec"}`); + return node; +} + +async function providerShapes(ps: ProviderSpec): Promise<{ input: any; output: any }> { + const doc = await fetchDoc(ps.url); + const { norm, isDiscovery, comps } = normalizer(doc); + if (isDiscovery) { + // Google discovery documents do not express `required`, so required-ness cannot be checked against them. + const pick = (key: "request" | "response") => { + const name = ps[key]; + if (!name) throw new Error(`provider_spec.${key} is required: ${ps.url} is a discovery document, which names schemas rather than operations`); + if (!comps[name]) throw new Error(`provider_spec.${key}: \`${name}\` is not a schema in ${ps.url}`); + return { ...norm({ $ref: name }), noRequiredInfo: true }; + }; + return { input: pick("request"), output: pick("response") }; + } + if (!ps.operation) throw new Error(`provider_spec.operation is required: ${ps.url} is an OpenAPI document, which names operations rather than schemas`); + const op = findOperation(doc, ps.operation); + const req = op.requestBody?.content?.["application/json"]?.schema; + const resOp = ps.response_operation ? findOperation(doc, ps.response_operation) : op; + const res = resOp.responses?.["200"]?.content?.["application/json"]?.schema; + return { input: norm(req), output: norm(res) }; +} + +// ---- comparison +type Report = { errors: string[]; warnings: string[] }; +const eq = (a: unknown, b: unknown) => JSON.stringify(a) === JSON.stringify(b); +const baseType = (t: unknown) => (Array.isArray(t) ? t : [t]).map(String); +const compatible = (ours: unknown, theirs: unknown) => { + if (theirs === undefined) return true; + const o = baseType(ours), t = baseType(theirs); + return o.some((x) => t.includes(x) || (x === "integer" && t.includes("number")) || (x === "number" && t.includes("integer"))); +}; + +/** + * Collapse a documented `anyOf` / `oneOf` (e.g. `integer | "auto"`) into one comparable shape, + * the way `norm` already collapses the provider's: union of types and properties, intersection + * of `required`, widest bounds. Without this a documented union reads as an untyped field. + */ +function flatten(s: any): any { + const alts: any[] = s?.anyOf ?? s?.oneOf; + if (!Array.isArray(alts) || !alts.length) return s; + const types = [...new Set(alts.flatMap((a) => (Array.isArray(a.type) ? a.type : [a.type])).filter(Boolean))]; + const nums = (k: string) => alts.map((a) => a[k]).filter((n) => n !== undefined).map(Number); + const props: Record = {}; let req: Set | null = null; + for (const a of alts) { + Object.assign(props, a.properties ?? {}); + const r = new Set(a.required ?? []); req = req ? new Set([...req].filter((x) => r.has(x))) : r; + } + const [mins, maxs] = [nums("minimum"), nums("maximum")]; + return { + ...s, anyOf: undefined, oneOf: undefined, + type: s.type ?? (types.length === 1 ? types[0] : types), + ...(mins.length ? { minimum: Math.min(...mins) } : {}), + ...(maxs.length ? { maximum: Math.max(...maxs) } : {}), + ...(Object.keys(props).length ? { properties: { ...props, ...(s.properties ?? {}) }, required: [...(req ?? [])] } : {}), + }; +} + +function compare(ours: any, theirs: any, where: string, omit: Set, rep: Report, prefix = "", noRequired = false) { + ours = flatten(ours); + noRequired = noRequired || !!theirs?.noRequiredInfo; + if (!theirs || theirs.opaque) { if (ours?.properties && Object.keys(ours.properties).length) rep.warnings.push(`${where}: provider declares \`${prefix || "body"}\` as an opaque object; cannot verify ${Object.keys(ours.properties).length} documented field(s) beneath it`); return; } + const ourReq = new Set(ours.required ?? []); const theirReq = new Set(theirs.required ?? []); + for (const [name, raw] of Object.entries(ours.properties ?? {})) { + const o = flatten(raw); + const path = prefix ? `${prefix}.${name}` : name; + const t = theirs.properties?.[name]; + if (!t) { rep.errors.push(`${where}: \`${path}\` is documented but the provider spec has no such field`); continue; } + if (!compatible(o.type, t.type)) rep.errors.push(`${where}: \`${path}\` type ${JSON.stringify(o.type)} vs provider ${JSON.stringify(t.type)}`); + if (!noRequired && ourReq.has(name) !== theirReq.has(name) && !theirs.union) rep.errors.push(`${where}: \`${path}\` required=${ourReq.has(name)} vs provider required=${theirReq.has(name)}`); + if (t.default !== undefined && o.default !== undefined && !eq(o.default, t.default)) rep.errors.push(`${where}: \`${path}\` default ${JSON.stringify(o.default)} vs provider ${JSON.stringify(t.default)}`); + if (t.default !== undefined && o.default === undefined && !prefix.includes("[]")) rep.warnings.push(`${where}: \`${path}\` provider default ${JSON.stringify(t.default)} is not documented`); + if (o.default !== undefined && t.default === undefined) rep.warnings.push(`${where}: \`${path}\` documents default ${JSON.stringify(o.default)} but the provider spec declares none`); + if (o.enum && t.enum) { const extra = o.enum.filter((v: unknown) => !t.enum.includes(v)); if (extra.length) rep.errors.push(`${where}: \`${path}\` enum values ${JSON.stringify(extra)} are not in the provider's ${JSON.stringify(t.enum)}`); const missing = t.enum.filter((v: unknown) => !o.enum.includes(v) && !String(v).endsWith("UNSPECIFIED")); if (missing.length) rep.warnings.push(`${where}: \`${path}\` provider also allows ${JSON.stringify(missing)}`); } + if (t.enum && !o.enum && t.enum.length <= 12) rep.warnings.push(`${where}: \`${path}\` provider enumerates ${JSON.stringify(t.enum)} but we document a free ${o.type}`); + for (const b of ["minimum", "maximum"] as const) if (o[b] !== undefined && t[b] !== undefined && Number(o[b]) !== Number(t[b])) rep.errors.push(`${where}: \`${path}\` ${b} ${o[b]} vs provider ${t[b]}`); + if (o.properties) compare(o, t, where, omit, rep, path, noRequired); + if (o.items?.properties) compare(o.items, t.items, where, omit, rep, `${path}[]`, noRequired); + } + for (const name of Object.keys(theirs.properties ?? {})) { + const path = prefix ? `${prefix}.${name}` : name; + if (!ours.properties?.[name] && !omit.has(path) && !omit.has(name) && !prefix.includes("[]") && prefix.split(".").length <= 1) { + (strict ? rep.errors : rep.warnings).push(`${where}: provider field \`${path}\` is not documented${theirReq.has(name) ? " (and the provider marks it required)" : ""}`); + } + } +} + +// ---- main +const glob = new Bun.Glob("development/comfy-router/models/**/code.yaml"); +const rep: Report = { errors: [], warnings: [] }; +let checked = 0, skipped = 0; +for (const specPath of glob.scanSync({ cwd: ROOT })) { + const spec = Bun.YAML.parse(readFileSync(join(ROOT, specPath), "utf8")) as any; + for (const v of spec.variants) { + const ps: ProviderSpec | undefined = v.provider_spec ?? spec.provider_spec; + const input = v.input ?? spec.input, output = v.output ?? spec.output; + if (!ps) { skipped++; rep.warnings.push(`${dirname(specPath)} (${v.model}): no provider_spec, cannot verify`); continue; } + const where = `${dirname(specPath).replace("development/comfy-router/models/", "")} (${v.model})`; + try { + const shapes = await providerShapes(ps); + const omit = new Set(ps.omit ?? []); + if (input) compare(input, shapes.input, `${where} input`, omit, rep); + if (output) compare(output, shapes.output, `${where} output`, omit, rep); + checked++; + } catch (e) { rep.errors.push(`${where}: ${(e as Error).message}`); } + } +} +if (verbose) for (const w of rep.warnings) console.log(`warn ${w}`); +else if (rep.warnings.length) console.log(`(${rep.warnings.length} warning(s); run with --verbose to list them)`); +for (const e of rep.errors) console.log(`ERROR ${e}`); +console.log(`\n${checked} model(s) checked against provider specs, ${skipped} skipped, ${rep.errors.length} error(s), ${rep.warnings.length} warning(s)`); +process.exit(rep.errors.length ? 1 : 0); diff --git a/.github/scripts/snippets/gen-code-pages.ts b/.github/scripts/snippets/gen-code-pages.ts new file mode 100644 index 000000000..ec81e4c84 --- /dev/null +++ b/.github/scripts/snippets/gen-code-pages.ts @@ -0,0 +1,530 @@ +#!/usr/bin/env bun +/** + * Generate the per-model "Code" pages (development/comfy-router/models///code.mdx) + * from their code.yaml specs. + * + * bun .github/scripts/snippets/gen-code-pages.ts # write every code.mdx + * bun .github/scripts/snippets/gen-code-pages.ts --check # exit 1 if any code.mdx is stale + * bun .github/scripts/snippets/gen-code-pages.ts --validate # also syntax-check the emitted snippets + * + * The template below is the only place the page shape lives. Python, TypeScript + * and cURL are all emitted from the same `example` object, so the three cannot + * disagree about the request body. + */ +import { readFileSync, writeFileSync, existsSync, mkdtempSync, rmSync } from "node:fs"; +import { join, dirname, relative } from "node:path"; +import { tmpdir } from "node:os"; + +const ROOT = join(import.meta.dir, "../../.."); +const SPEC_GLOB = "development/comfy-router/models/**/code.yaml"; +const BASE_URL = "https://api.comfy.org"; +const ROUTE = "/v2/models"; + +type Variant = { title: string; model: string; example?: Record; input?: any; output?: any; provider_spec?: { url: string } }; +type Spec = { + name: string; + provider: string; + description: string; + task?: string; + variants: Variant[]; + example: Record; + fields?: string; + input?: any; + output?: any; + provider_spec?: { url: string }; + result: { path: string; label: string; example: unknown; note?: string }; + summary: string; + intro?: string; +}; + +// --------------------------------------------------------------------------- +// Snippet emitters. `@file:` values are read from disk and base64 encoded. +// --------------------------------------------------------------------------- + +type FileInput = { key: string; path: string; varName: string }; + +function fileInputs(example: Record): FileInput[] { + return Object.entries(example) + .filter(([, v]) => typeof v === "string" && v.startsWith("@file:")) + .map(([key, v]) => ({ key, path: (v as string).slice("@file:".length), varName: key })); +} + +function camel(s: string): string { + return s.replace(/_([a-z])/g, (_, c) => c.toUpperCase()); +} + +function shellVar(s: string): string { + return s.toUpperCase(); +} + +/** Result path like `candidates[0].content.parts[0].inlineData.data` -> segments. */ +function pathSegments(path: string): (string | number)[] { + const out: (string | number)[] = []; + for (const part of path.split(".")) { + const m = part.match(/^([^[]+)((?:\[\d+\])*)$/); + if (!m) throw new Error(`bad result path segment: ${part}`); + out.push(m[1]); + for (const idx of m[2].matchAll(/\[(\d+)\]/g)) out.push(Number(idx[1])); + } + return out; +} + +function pyPath(path: string): string { + return pathSegments(path).map((p) => (typeof p === "number" ? `[${p}]` : `[${JSON.stringify(p)}]`)).join(""); +} + +function tsPath(path: string): string { + return pathSegments(path).map((p) => (typeof p === "number" ? `[${p}]` : `.${p}`)).join(""); +} + +function tsResultType(path: string): string { + const segs = pathSegments(path); + let t = "string"; + for (let i = segs.length - 1; i >= 0; i--) { + const p = segs[i]; + t = typeof p === "number" ? `${t}[]` : `{ ${p}: ${t} }`; + } + return t; +} + +/** JSON value -> Python literal, multi-line, at the given indent. */ +function pyLiteral(v: unknown, indent: number, files: FileInput[], topKey?: string): string { + const pad = " ".repeat(indent); + const f = topKey !== undefined ? files.find((x) => x.key === topKey) : undefined; + if (f) return f.varName; + if (v === null) return "None"; + if (typeof v === "boolean") return v ? "True" : "False"; + if (typeof v === "number" || typeof v === "string") return JSON.stringify(v); + if (Array.isArray(v)) { + if (v.every((x) => typeof x !== "object" || x === null)) return `[${v.map((x) => pyLiteral(x, indent, files)).join(", ")}]`; + return `[\n${v.map((x) => `${pad} ${pyLiteral(x, indent + 4, files)},`).join("\n")}\n${pad}]`; + } + const entries = Object.entries(v as Record); + return `{\n${entries.map(([k, x]) => `${pad} ${JSON.stringify(k)}: ${pyLiteral(x, indent + 4, files)},`).join("\n")}\n${pad}}`; +} + +/** JSON value -> TypeScript object literal, multi-line, at the given indent. */ +function tsLiteral(v: unknown, indent: number, files: FileInput[], topKey?: string): string { + const pad = " ".repeat(indent); + const f = topKey !== undefined ? files.find((x) => x.key === topKey) : undefined; + if (f) return camel(f.varName); + if (v === null || typeof v !== "object") return JSON.stringify(v); + if (Array.isArray(v)) { + if (v.every((x) => typeof x !== "object" || x === null)) return `[${v.map((x) => tsLiteral(x, indent, files)).join(", ")}]`; + return `[\n${v.map((x) => `${pad} ${tsLiteral(x, indent + 2, files)},`).join("\n")}\n${pad}]`; + } + const entries = Object.entries(v as Record); + const key = (k: string) => (/^[a-zA-Z_$][\w$]*$/.test(k) ? k : JSON.stringify(k)); + return `{\n${entries.map(([k, x]) => `${pad} ${key(k)}: ${tsLiteral(x, indent + 2, files)},`).join("\n")}\n${pad}}`; +} + +function pythonSnippet(model: string, example: Record, files: FileInput[], resultPath: string, label: string): string { + const reads = files + .map((f) => `with open(${JSON.stringify(f.path)}, "rb") as f:\n ${f.varName} = base64.b64encode(f.read()).decode()`) + .join("\n\n"); + const body = Object.entries(example) + .map(([k, v]) => ` ${JSON.stringify(k)}: ${pyLiteral(v, 12, files, k)},`) + .join("\n"); + return `${files.length ? "import base64\n\n" : ""}from comfy_sdk import Comfy +${reads ? `\n${reads}\n` : ""} +# Reads COMFY_API_KEY from the environment. Each call sends a fresh +# Idempotency-Key and waits up to 10 minutes for the finished result. +with Comfy() as client: + result = client.models.run( + "${model}", + { +${body} + }, + ) + +print("${label}:", result${pyPath(resultPath)})`; +} + +function typescriptSnippet(model: string, example: Record, files: FileInput[], resultPath: string, label: string): string { + const imports = `import { comfy } from "@comfyorg/sdk";\n${files.length ? `import { readFile } from "node:fs/promises";\n` : ""}`; + const reads = files + .map((f) => `const ${camel(f.varName)} = (await readFile(${JSON.stringify(f.path)})).toString("base64");`) + .join("\n"); + const body = Object.entries(example) + .map(([k, v]) => ` ${/^[a-zA-Z_$][\w$]*$/.test(k) ? k : JSON.stringify(k)}: ${tsLiteral(v, 2, files, k)},`) + .join("\n"); + return `${imports} +${reads ? `${reads}\n\n` : ""}// Reads COMFY_API_KEY from the environment. Each call sends a fresh +// Idempotency-Key and waits up to 10 minutes for the finished result. +type Result = ${tsResultType(resultPath)}; +const { data } = await comfy.models.run("${model}", { +${body} +}); + +console.log("${label}:", data${tsPath(resultPath)});`; +} + +function curlSnippet(model: string, example: Record, files: FileInput[]): string { + const reads = files.map((f) => `${shellVar(f.varName)}=$(base64 < ${f.path} | tr -d '\\n')`).join("\n"); + const esc = (v: unknown) => JSON.stringify(v).replace(/[\\$`"]/g, (c) => `\\${c}`); + const entries = Object.entries(example).map(([k, v]) => { + const f = files.find((x) => x.key === k); + const value = f ? `\\"$${shellVar(f.varName)}\\"` : esc(v); + return `${esc(k)}: ${value}`; + }); + const json = `{${entries.join(", ")}}`; + return `${reads ? `${reads}\n\n` : ""}curl ${BASE_URL}${ROUTE}/${model} \\ + -H "X-API-Key: $COMFY_API_KEY" \\ + -H "Idempotency-Key: $(uuidgen)" \\ + -H "Content-Type: application/json" \\ + -d "${json}"`; +} + +function possessive(name: string): string { + return name.endsWith("s") ? `${name}'` : `${name}'s`; +} + + +// --------------------------------------------------------------------------- +// Per-model schema documents. `router-schemas//.json` is the +// exact body of `GET /v2/models///openapi.json` (a standalone +// OpenAPI document), dropped in by the spec-sync bot. When present and +// authored, the Input schema / Input example sections render from it; when +// absent or unauthored, the page falls back to the spec's hand-written fields. +// --------------------------------------------------------------------------- + +type SchemaDoc = { + paths: Record }; responses?: Record }> } }>; + components?: { schemas?: Record }; + "x-comfy-router-model-id"?: string; + "x-comfy-input-schema-authored"?: boolean; +}; + +type ModelSchema = { + authored: boolean; + input?: any; + inputExample?: unknown; + output?: any; + outputExample?: unknown; + components: Record; +}; + +function loadModelSchema(model: string): ModelSchema | null { + const file = join(ROOT, "router-schemas", `${model}.json`); + if (!existsSync(file)) return null; + let doc: SchemaDoc; + try { + doc = JSON.parse(readFileSync(file, "utf8")) as SchemaDoc; + } catch (e) { + throw new Error(`router-schemas/${model}.json: cannot parse JSON: ${(e as Error).message}`); + } + const op = doc.paths?.[`${ROUTE}/${model}`]?.post; + if (!op) throw new Error(`router-schemas/${model}.json: no POST ${ROUTE}/${model} operation`); + const req = op.requestBody?.content?.["application/json"]; + const res = op.responses?.["200"]?.content?.["application/json"]; + return { + authored: doc["x-comfy-input-schema-authored"] !== false, + input: req?.schema, + inputExample: req?.example ?? req?.schema?.example, + output: res?.schema, + outputExample: res?.example ?? res?.schema?.example, + components: doc.components?.schemas ?? {}, + }; +} + +function deref(schema: any, components: Record, depth = 0): any { + if (!schema || depth > 8) return schema ?? {}; + if (schema.$ref) { + const name = String(schema.$ref).split("/").pop()!; + return deref(components[name], components, depth + 1); + } + if (schema.allOf) { + return schema.allOf.reduce((acc: any, part: any) => { + const d = deref(part, components, depth + 1); + return { ...acc, ...d, properties: { ...(acc.properties ?? {}), ...(d.properties ?? {}) }, required: [...(acc.required ?? []), ...(d.required ?? [])] }; + }, {}); + } + return schema; +} + +function typeLabel(schema: any, components: Record): string { + const s = deref(schema, components); + if (s.oneOf || s.anyOf) return (s.oneOf ?? s.anyOf).map((x: any) => typeLabel(x, components)).join(" | "); + if (s.const !== undefined) return JSON.stringify(s.const); + if (s.enum) return s.enum.map((v: unknown) => `\`${String(v)}\``).join(", "); + if (s.type === "array") return `${typeLabel(s.items ?? {}, components)}[]`; + if (s.type === "string" && s.format) return `string (${s.format})`; + return s.type ?? "object"; +} + +function constraints(s: any): string { + const out: string[] = []; + if (s.default !== undefined) out.push(`default \`${JSON.stringify(s.default)}\``); + if (s.minimum !== undefined || s.maximum !== undefined) out.push(`${s.minimum ?? ""}..${s.maximum ?? ""}`); + if (s.minLength !== undefined || s.maxLength !== undefined) out.push(`length ${s.minLength ?? ""}..${s.maxLength ?? ""}`); + if (s.minItems !== undefined || s.maxItems !== undefined) out.push(`items ${s.minItems ?? ""}..${s.maxItems ?? ""}`); + return out.join(", "); +} + +const attr = (v: unknown) => String(v ?? "").replace(/"/g, """).replace(/\s+/g, " ").trim(); + +/** Escape the characters MDX reads as syntax in prose, leaving `code spans` alone. */ +const mdxText = (v: unknown) => + String(v).split(/(`[^`]*`)/).map((part, i) => (i % 2 ? part : part.replace(/[{<]/g, "\\$&"))).join(""); + +/** Render a JSON Schema object as Mintlify ParamField (input) or ResponseField (output) blocks. */ +function schemaFields(schema: any, components: Record, kind: "param" | "response"): string { + const blocks: string[] = []; + const walk = (s: any, prefix: string, depth: number) => { + s = deref(s, components); + const required = new Set(s.required ?? []); + for (const [name, raw] of Object.entries(s.properties ?? {})) { + const prop = deref(raw, components); + const path = prefix ? `${prefix}.${name}` : name; + const tag = kind === "param" ? "ParamField" : "ResponseField"; + const nameAttr = kind === "param" ? `body="${path}"` : `name="${path}"`; + const type = attr(typeLabel({ ...prop, enum: undefined }, components)); + const attrs = [nameAttr, `type="${type}"`]; + if (required.has(name)) attrs.push("required"); + if (prop.default !== undefined) attrs.push(`default="${attr(JSON.stringify(prop.default))}"`); + const body: string[] = []; + // An array property usually carries no description of its own: the prose sits on the + // component its `items` point at (OpenAPI 3.0 ignores a sibling `description` next to a + // `$ref`, so that is the only place an author can put it). Without this fallback the whole + // Gemini request body renders as empty ParamFields. + const itemsDesc = prop.type === "array" && prop.items ? deref(prop.items, components).description : undefined; + const description = prop.description ?? itemsDesc; + if (description) body.push(mdxText(String(description).trim())); + if (prop.enum) body.push(`Possible values: ${prop.enum.map((v: unknown) => `\`${String(v)}\``).join(", ")}`); + // A union (`integer | "auto"`) carries its bounds on the numeric branch, not on the field. + const alts: any[] = prop.anyOf ?? prop.oneOf ?? []; + const bound = (k: "minimum" | "maximum") => { + if (prop[k] !== undefined) return prop[k]; + const ns = alts.map((a) => a[k]).filter((n) => n !== undefined).map(Number); + return ns.length ? (k === "minimum" ? Math.min(...ns) : Math.max(...ns)) : undefined; + }; + const [min, max] = [bound("minimum"), bound("maximum")]; + if (min !== undefined || max !== undefined) body.push(`Range: \`${min ?? "…"}\` to \`${max ?? "…"}\``); + if (prop.format) body.push(`Format: \`${prop.format}\``); + blocks.push(`<${tag} ${attrs.join(" ")}>\n ${body.join("\n\n ") || " "}\n`); + if (depth < 4) { + if (prop.type === "object" || prop.properties) walk(prop, path, depth + 1); + else if (prop.type === "array") { + const items = deref(prop.items ?? {}, components); + if (items.properties) walk(items, `${path}[]`, depth + 1); + } + } + } + }; + walk(schema, "", 0); + if (!blocks.length) return "_The schema declares no fixed fields: any JSON object is accepted._"; + return blocks.join("\n\n"); +} + +// --------------------------------------------------------------------------- +// Page template +// --------------------------------------------------------------------------- + +function sectionBlocks(v: Variant, spec: Spec) { + const s = loadModelSchema(v.model); + const example = v.example ?? spec.example; + const specInput = v.input ?? spec.input; + const specOutput = v.output ?? spec.output; + const fields = (spec.fields ?? "").replace(/\s+/g, " ").trim(); + const checked = !!(v.provider_spec ?? spec.provider_spec); + const notPublished = checked + ? `_Fields follow ${possessive(spec.provider)} published API specification and are checked against it in CI. Router's own schema for this model is not published yet, so requests are forwarded to the provider unvalidated._` + : `\nRouter has not published an authored input schema for this model yet: \`GET ${ROUTE}/${v.model}/openapi.json\` returns an open object with \`x-comfy-input-schema-authored: false\`. The fields below follow the provider's own API documentation and are not yet validated server side.\n`; + // `x-comfy-input-schema-authored: false` disqualifies the whole served document, not just its input + // half: the page then reads its fields AND its examples from the spec, as the README describes. + const published = s?.authored ? s : null; + let input: string; + if (published?.input) { + input = `${schemaFields(published.input, published.components, "param")}\n\nGenerated from the schema Router serves at \`GET ${ROUTE}/${v.model}/openapi.json\`, the same document it validates a call against before the request reaches the provider.`; + } else if (specInput) { + input = `${notPublished}\n\n${schemaFields(specInput, {}, "param")}`; + } else { + input = `${notPublished}\n\n${fields}`; + } + const inputExample = JSON.stringify(published?.inputExample ?? example, null, 2).replace(/"@file:([^"]+)"/g, '""'); + let output: string; + if (published?.output) { + output = schemaFields(published.output, published.components, "response"); + } else if (specOutput) { + output = `Router returns ${possessive(spec.provider)} native response unchanged. The ${spec.result.label} is at \`${spec.result.path}\`.\n\n${schemaFields(specOutput, {}, "response")}`; + } else { + output = `Router returns ${possessive(spec.provider)} native output unchanged and does not publish an output schema for this model. The ${spec.result.label} is at \`${spec.result.path}\`; the example below is representative of the provider's response.`; + } + const outputExample = JSON.stringify(published?.outputExample ?? spec.result.example, null, 2); + return { input, inputExample, output, outputExample }; +} + +/** Model ID, endpoint and the three snippets for one variant. */ +function quickStart(v: Variant, spec: Spec): string { + const example = v.example ?? spec.example; + const files = fileInputs(example); + const label = spec.result.label; + return `**Model ID:** \`${v.model}\` + +**Endpoint:** \`POST ${BASE_URL}${ROUTE}/${v.model}\` + + +\`\`\`python Python +${pythonSnippet(v.model, example, files, spec.result.path, label)} +\`\`\` + +\`\`\`typescript TypeScript +${typescriptSnippet(v.model, example, files, spec.result.path, label)} +\`\`\` + +\`\`\`bash cURL +${curlSnippet(v.model, example, files)} +\`\`\` +`; +} + +/** Schema + Examples for one variant. `html` headings keep them out of the TOC when rendered inside tabs. */ +function sections(v: Variant, spec: Spec, html: boolean): string { + const h2 = (t: string) => (html ? `

${t}

` : `## ${t}`); + const h3 = (t: string) => (html ? `

${t}

` : `### ${t}`); + const b = sectionBlocks(v, spec); + return `${h2("Schema")} + +${h3("Input")} + +${b.input} + +${h3("Output")} + +${b.output} + +${h2("Examples")} + +${h3("Input")} + +\`\`\`json +${b.inputExample} +\`\`\` + +${h3("Output")} + +\`\`\`json +${b.outputExample} +\`\`\`${spec.result.note ? `\n\n${spec.result.note}` : ""}`; +} + +function variantsShareSections(spec: Spec): boolean { + const key = (v: Variant) => JSON.stringify([v.input ?? spec.input, v.output ?? spec.output, v.example ?? spec.example, loadModelSchema(v.model)]); + return spec.variants.every((v) => key(v) === key(spec.variants[0])); +} + +function renderPage(spec: Spec, dir: string): string { + const both = spec.variants.length > 1; + // The one-time setup a snippet cannot run without. Everything else that is + // shared across models (idempotency, deadline, request IDs) lives on the + // headers page the footer links to. + const setup = `Create a key at [platform.comfy.org/profile/api-keys](https://platform.comfy.org/profile/api-keys) and export it as \`COMFY_API_KEY\`. The Python and TypeScript snippets use the Comfy SDKs (\`pip install comfy-sdk\`, \`npm install @comfyorg/sdk\`); the cURL snippet is the same call over raw HTTP.`; + let body: string; + if (!both) { + body = `## Quick start\n\n${setup}\n\n${quickStart(spec.variants[0], spec)}\n\n${sections(spec.variants[0], spec, false)}`; + } else if (variantsShareSections(spec)) { + // Only the snippets differ: one selector under Quick start, the shared schema and examples once below. + body = `## Quick start\n\n${setup}\n\nPick the model you want to call. The models share one request and response shape, documented once below.\n\n\n${spec.variants.map((v) => ` \n${quickStart(v, spec)}\n `).join("\n")}\n\n\n${sections(spec.variants[0], spec, false)}`; + } else { + // The models take different inputs: one selector switches the whole page. + body = `## Quick start\n\n${setup}\n\nPick the model you want to call. Everything below, from the snippets to the schema and examples, follows your choice.\n\n\n${spec.variants.map((v) => ` \n${quickStart(v, spec)}\n\n${sections(v, spec, true)}\n `).join("\n")}\n`; + } + return `--- +title: ${JSON.stringify(`Use ${spec.name} with Comfy Router`)} +description: ${JSON.stringify(spec.description.replace(/\s+/g, " ").trim())} +sidebarTitle: ${JSON.stringify(spec.name)} +--- + +{/* GENERATED FILE. Edit code.yaml in this directory and run \`pnpm code-pages:gen\`. */} + +import RouterPreviewNotice from "/snippets/comfy-router/preview-notice.mdx"; +import RouterCodeFooter from "/snippets/comfy-router/model-code-footer.mdx"; + +${spec.intro ?? `API Reference for ${spec.name}. ${spec.summary.replace(/\s+/g, " ").trim()}`} + + + +${body} + + +`; +} + +// --------------------------------------------------------------------------- +// Validation of emitted snippets (syntax only; nothing is executed or billed) +// --------------------------------------------------------------------------- + +function validate(page: string, rel: string): string[] { + const problems: string[] = []; + const tmp = mkdtempSync(join(tmpdir(), "code-pages-")); + try { + const fences = [...page.matchAll(/```(python|typescript|bash)[^\n]*\n([\s\S]*?)```/g)]; + fences.forEach((m, i) => { + const [, lang, code] = m; + const file = join(tmp, `s${i}.${lang === "python" ? "py" : lang === "typescript" ? "ts" : "sh"}`); + writeFileSync(file, code); + const cmd = + lang === "python" + ? ["python3", "-m", "py_compile", file] + : lang === "typescript" + ? ["bun", "build", "--target=node", "--no-bundle", file, "--outfile", `${file}.out.js`] + : ["bash", "-n", file]; + const r = Bun.spawnSync(cmd, { stderr: "pipe", stdout: "pipe" }); + if (r.exitCode !== 0) problems.push(`${rel}: ${lang} snippet #${i + 1} failed syntax check:\n${r.stderr.toString()}`); + }); + } finally { + rmSync(tmp, { recursive: true, force: true }); + } + return problems; +} + +// --------------------------------------------------------------------------- + +const check = process.argv.includes("--check"); +const doValidate = process.argv.includes("--validate"); +const glob = new Bun.Glob(SPEC_GLOB); +let stale: string[] = []; +let problems: string[] = []; +let count = 0; +for (const specPath of glob.scanSync({ cwd: ROOT })) { + count++; + let spec: Spec; + try { + spec = Bun.YAML.parse(readFileSync(join(ROOT, specPath), "utf8")) as Spec; + } catch (e) { + problems.push(`${specPath}: cannot parse YAML: ${(e as Error).message}`); + continue; + } + for (const key of ["name", "provider", "description", "summary", "variants", "example", "result"] as const) { + if (spec[key] === undefined) problems.push(`${specPath}: missing required key \`${key}\``); + } + if (problems.some((m) => m.startsWith(specPath))) continue; + const dir = dirname(specPath); + const out = join(ROOT, dir, "code.mdx"); + let page: string; + try { + // A malformed `result.path`, or a router-schemas document we cannot read, must not abandon the + // remaining specs half written; report it against this spec and carry on, as YAML errors do. + page = renderPage(spec, dir); + } catch (e) { + problems.push(`${specPath}: cannot render: ${(e as Error).message}`); + continue; + } + if (doValidate) problems.push(...validate(page, relative(ROOT, out))); + if (check) { + if (!existsSync(out) || readFileSync(out, "utf8") !== page) stale.push(relative(ROOT, out)); + } else { + writeFileSync(out, page); + console.log(`wrote ${relative(ROOT, out)}`); + } +} +if (count === 0) { + console.error(`no specs matched ${SPEC_GLOB}`); + process.exit(1); +} +if (stale.length) { + console.error(`stale generated pages (run \`pnpm code-pages:gen\`):\n ${stale.join("\n ")}`); +} +if (problems.length) console.error(problems.join("\n")); +if (stale.length || problems.length) process.exit(1); +if (check) console.log(`${count} code page(s) fresh`); diff --git a/.github/workflows/code-pages-check.yml b/.github/workflows/code-pages-check.yml new file mode 100644 index 000000000..e0bf86b99 --- /dev/null +++ b/.github/workflows/code-pages-check.yml @@ -0,0 +1,51 @@ +name: Code Pages Freshness + +on: + pull_request: + paths: + - 'development/comfy-router/models/**/code.yaml' + - 'development/comfy-router/models/**/code.mdx' + - 'snippets/comfy-router/**' + - 'router-schemas/**' + - '.github/scripts/snippets/**' + - '.github/workflows/code-pages-check.yml' + - 'package.json' + +permissions: + contents: read + +jobs: + code-pages: + runs-on: ubuntu-latest + timeout-minutes: 10 + steps: + - name: Checkout code + uses: actions/checkout@v4 + with: + persist-credentials: false + + - name: Setup Bun + uses: oven-sh/setup-bun@v2 + with: + bun-version: latest + + - name: Check generated code pages are fresh and their snippets parse + run: bun run code-pages:check + + provider-schemas: + runs-on: ubuntu-latest + # Fetches each provider's published spec, so a stalled provider host must not hold the runner. + timeout-minutes: 15 + steps: + - name: Checkout code + uses: actions/checkout@v4 + with: + persist-credentials: false + + - name: Setup Bun + uses: oven-sh/setup-bun@v2 + with: + bun-version: latest + + - name: Check documented input/output schemas against each provider's published spec + run: bun run code-pages:check-providers diff --git a/development/comfy-router/headers.mdx b/development/comfy-router/headers.mdx new file mode 100644 index 000000000..b935c10fc --- /dev/null +++ b/development/comfy-router/headers.mdx @@ -0,0 +1,101 @@ +--- +title: "Comfy Router headers" +sidebarTitle: "Headers" +description: "The request headers you can send to Comfy Router and the response headers it returns, for every model: authentication, idempotency, request IDs, error buckets, retry pacing and spend limits." +--- + +import RouterPreviewNotice from "/snippets/comfy-router/preview-notice.mdx"; + + + +Every model behind Comfy Router is called the same way: `POST /v2/models/{provider}/{model}` with the model's own JSON body. What is common to all of them lives in the headers, and this page is the one place they are described. The per-model Code pages link here instead of repeating it; the [API reference](/development/comfy-router/reference) carries the same definitions in generated form. + +The Comfy SDKs (`comfy-sdk` for Python, `@comfyorg/sdk` for TypeScript) send the request headers for you and surface the response headers as fields on results and errors. If you call Router over raw HTTP, you send and read them yourself. + +## Request headers + + + A Comfy API key, `comfyui-...`, created at [platform.comfy.org/profile/api-keys](https://platform.comfy.org/profile/api-keys). Keys are per workspace and carry that workspace's model entitlements and credit balance. The same key is also accepted as `Authorization: Bearer comfyui-...`; the `comfyui-` prefix, not the header, is what marks it as an API key. If both headers are sent, `X-API-Key` wins. + + + + `Bearer `. A `comfyui-` API key is accepted here exactly as in `X-API-Key`. A value without that prefix is treated as a Comfy Cloud JWT, which is what the generated reference means by "bearer token" and what the OAuth-based SDK clients send. + + + + Your own key for one logical call, 1 to 255 characters; a UUID is the intended shape. A call that reached you with an answer is recorded against its key for 24 hours, and a retry carrying the same key is answered from that record instead of dispatching, and charging, the provider a second time. Send it on every paid call, reuse it for every retry of that call, and mint a new one for a new call. The same key with a different request (body, model path, query or method) is a `409` rather than a silent overwrite. The guarantee is a billing one: a key is charged at most once. It does not make a lost call resumable. The SDKs mint one per call and let you pass your own (`idempotency_key=` in Python, `idempotencyKey` in TypeScript). + + + + `application/json`. Router forwards the body to the provider unchanged, so the body is the provider's native JSON and nothing else is accepted. + + + + On `GET /v2/models/{provider}/{model}/openapi.json` only. Send the `ETag` you hold from an earlier `200`; when it still matches, the answer is a bodyless `304` with the same `ETag`. Cache a model's schema for the life of your process and revalidate it this way rather than re-reading it before every call. + + +## Response headers + + + On every response, success and error alike. The ID to quote in a support request; the same value is written into the call's usage record, which is what lets a question about a charge be joined to the charge. The TypeScript SDK returns it as `requestId`; the Python SDK exposes `request_id` on every error. + + + + On every error response. One of fifteen buckets: `invalid_input`, `content_policy_violation`, `provider_error`, `provider_timeout`, `insufficient_credits`, `model_not_found`, `unauthorized`, `forbidden`, `concurrency_limit_exceeded`, `client_disconnected`, `internal_error`, `deadline_exceeded`, `not_enabled`, `service_unavailable`, `rate_limited`. It repeats the body's `error_type`, and on a `422` it is the only machine-readable bucket, because that body is the per-field `detail[]` shape and has no `error_type` of its own. Branch on this header, never on the status alone: `409`, `429` and `504` each carry two different buckets that call for opposite actions. Treat an unrecognised value as `internal_error`. The SDKs raise a typed error per bucket. + + + + Present, and `true`, when the response was served from an `Idempotency-Key`'s record rather than by running the model again. It carries the original call's status, body and content type and is not billed a second time. Absent on a fresh run rather than sent as `false`, so branch on its presence. + + + + Seconds to wait before re-sending the same request with the same `Idempotency-Key`. Set on the two answers such a retry can actually collect from: a `409` with `concurrency_limit_exceeded` (the original call for that key is still running) and a `504` with `deadline_exceeded` (Router stopped holding the connection but still holds a handle to the running generation). Re-sending the same key after the wait collects that result instead of starting, and paying for, a second one. It is also sent on a `429` with `rate_limited`, where it says when the allowance window rolls. Absent when there is nothing to collect: an unkeyed call, or a `409` with `invalid_input` that refuses the key outright. + + + + USD cents. On a `429` refused by the in-flight spend ceiling rather than by the concurrent-call count: the ceiling on partner spend you may have committed to calls still running. + + + + USD cents currently committed to your calls still in flight, not counting the refused one. Sent alongside `X-Committed-Spend-Limit`. + + + + USD cents of headroom left under the ceiling, floored at zero. It can be positive on a refusal: the refused call cost more than what was left, and a cheaper call would still be admitted. + + + + On `GET /v2/models/{provider}/{model}/openapi.json`. A strong validator over the document's bytes; store it and send it back as `If-None-Match`. + + + + On the schema route: `private, must-revalidate`. The document is not caller-specific, but the route is authenticated, so a shared cache must not hold it, and a stale copy is revalidated against the `ETag` rather than served on. + + +## Status codes that carry two meanings + +Three statuses are shared by two buckets, and the header is what tells them apart: + +| Status | `X-Comfy-Error-Type` | What to do | +| --- | --- | --- | +| `409` | `concurrency_limit_exceeded` | The original call for this key is still running. Wait `Retry-After`, re-send the same key. | +| `409` | `invalid_input` | The key cannot serve this request: a different request under the same key, or an original that cannot be replayed. Use a new key; do not re-send this one. | +| `429` | `concurrency_limit_exceeded` | Too many calls in flight, or the committed-spend ceiling (see the `X-Committed-Spend-*` headers). Clears when one of your own calls finishes. | +| `429` | `rate_limited` | A windowed allowance is spent. Nothing you do drains it early; wait `Retry-After`. | +| `504` | `deadline_exceeded` | Router's own 10-minute bound. With `Retry-After`, re-send the same key to collect the running generation. | +| `504` | `provider_timeout` | The partner did not answer in time. The request itself was fine; a fresh call may succeed. | + +## What Router does not offer as headers + +Readers coming from other hosted-model APIs sometimes look for these; Router does not have them, by design. + +1. **No client-set timeout or priority.** Router holds the connection until the generation finishes, up to its own server deadline (10 minutes by default), and answers `504` / `deadline_exceeded` at that bound. Set your client timeout above it, as the SDKs do, so you keep the typed error and the request ID. +2. **No retry or no-retry controls.** Retrying is the client's decision; the SDKs retry inside a bounded budget with the same `Idempotency-Key`, which is what makes a retry safe. +3. **No output retention or storage knobs.** Router returns the provider's native response unchanged; result URLs are the provider's and expire on the provider's schedule. +4. **No cost on the response.** Usage is reported through your workspace's billing, not per call. See [limitations](/development/comfy-router/limitations). + +## Next + +- [Quick Start](/development/comfy-router/quickstart): typed error handling in Python and TypeScript, reading the `422`, retrying safely with your own key. +- [API reference](/development/comfy-router/reference): the generated contract these headers are defined in. +- [Limitations](/development/comfy-router/limitations): what Router does not do today, and what to use instead. diff --git a/development/comfy-router/models/black-forest-labs/flux-1-1-pro-ultra-image/code.mdx b/development/comfy-router/models/black-forest-labs/flux-1-1-pro-ultra-image/code.mdx new file mode 100644 index 000000000..938692013 --- /dev/null +++ b/development/comfy-router/models/black-forest-labs/flux-1-1-pro-ultra-image/code.mdx @@ -0,0 +1,438 @@ +--- +title: "Use Flux 1.1 Pro Ultra Image with Comfy Router" +description: "Python, TypeScript and cURL snippets for calling FLUX 1.1 [pro] Ultra and FLUX 1.1 [pro] over HTTP through Comfy Router, plus the request fields and the result shape" +sidebarTitle: "Flux 1.1 Pro Ultra Image" +--- + +{/* GENERATED FILE. Edit code.yaml in this directory and run `pnpm code-pages:gen`. */} + +import RouterPreviewNotice from "/snippets/comfy-router/preview-notice.mdx"; +import RouterCodeFooter from "/snippets/comfy-router/model-code-footer.mdx"; + +API Reference for Flux 1.1 Pro Ultra Image. FLUX 1.1 [pro] is a text-to-image model from Black Forest Labs. Ultra mode generates images at up to 4MP resolution. + + + +## Quick start + +Create a key at [platform.comfy.org/profile/api-keys](https://platform.comfy.org/profile/api-keys) and export it as `COMFY_API_KEY`. The Python and TypeScript snippets use the Comfy SDKs (`pip install comfy-sdk`, `npm install @comfyorg/sdk`); the cURL snippet is the same call over raw HTTP. + +Pick the model you want to call. Everything below, from the snippets to the schema and examples, follows your choice. + + + +**Model ID:** `bfl/flux-pro-1.1-ultra` + +**Endpoint:** `POST https://api.comfy.org/v2/models/bfl/flux-pro-1.1-ultra` + + +```python Python +from comfy_sdk import Comfy + +# Reads COMFY_API_KEY from the environment. Each call sends a fresh +# Idempotency-Key and waits up to 10 minutes for the finished result. +with Comfy() as client: + result = client.models.run( + "bfl/flux-pro-1.1-ultra", + { + "prompt": "a single red maple leaf on a plain white background, studio lighting", + "aspect_ratio": "16:9", + "raw": False, + }, + ) + +print("image:", result["result"]["sample"]) +``` + +```typescript TypeScript +import { comfy } from "@comfyorg/sdk"; + +// Reads COMFY_API_KEY from the environment. Each call sends a fresh +// Idempotency-Key and waits up to 10 minutes for the finished result. +type Result = { result: { sample: string } }; +const { data } = await comfy.models.run("bfl/flux-pro-1.1-ultra", { + prompt: "a single red maple leaf on a plain white background, studio lighting", + aspect_ratio: "16:9", + raw: false, +}); + +console.log("image:", data.result.sample); +``` + +```bash cURL +curl https://api.comfy.org/v2/models/bfl/flux-pro-1.1-ultra \ + -H "X-API-Key: $COMFY_API_KEY" \ + -H "Idempotency-Key: $(uuidgen)" \ + -H "Content-Type: application/json" \ + -d "{\"prompt\": \"a single red maple leaf on a plain white background, studio lighting\", \"aspect_ratio\": \"16:9\", \"raw\": false}" +``` + + +

Schema

+ +

Input

+ + + Aspect ratio of the image between 21:9 and 9:21, e.g. 16:9. + + + + Optional base64-encoded image to remix. + + + + Blend between the prompt and the image prompt, from 0 (prompt only) to 1 (image prompt only). + + Range: `0` to `1` + + + + Output image format. + + Possible values: `jpeg`, `png`, `webp` + + + + Text prompt for image generation. + + + + Whether to upsample the prompt. If active, the prompt is automatically modified for more creative generation. + + + + Generate less processed, more natural-looking images. + + + + Tolerance level for input and output moderation, between 0 (most strict) and 6 (least strict). + + Range: `0` to `6` + + + + Optional seed for reproducibility. A random seed is used when omitted. + + + + Optional secret for webhook signature verification. + + + + URL to receive webhook notifications. + + Format: `uri` + + +Generated from the schema Router serves at `GET /v2/models/bfl/flux-pro-1.1-ultra/openapi.json`, the same document it validates a call against before the request reaches the provider. + +

Output

+ + + Provider-reported cost in credits, populated once the task is Ready. + + Format: `float` + + + + BFL task identifier. + + + + Optional generation progress reported by BFL. + + Range: `0` to `1` + + Format: `float` + + + + The finished generation. Not nullable here: this component's `required` entry is a promise that a `200` carries the result, and a nullable `result` would reduce it to a key-presence check. + + + + Provider-reported cost of the generation. This is BFL's number, not the Comfy charge. + + Format: `double` + + + + Provider-reported generation duration in seconds. + + Format: `double` + + + + Provider-reported completion time of the generation, in seconds since the Unix epoch. `double` for the same reason as `start_time`. + + Format: `double` + + + + The prompt the generation actually ran, after any prompt upsampling. + + + + Signed URL for the generated asset. Router re-hosts the asset onto Comfy storage and rewrites this field, so it is normally a Comfy-hosted URL valid for up to 24 hours - signed for 24 hours when minted, and replayed from a 23-hour memo, so a later poll can hand back one with as little as an hour left; a leaf whose re-host could not be performed keeps BFL's own short-lived delivery URL instead - roughly two hours for video, roughly ten minutes for images. Either way the link expires, so download the asset rather than storing the URL. + + Format: `uri` + + + + The seed the generation used, whether supplied or chosen by the provider. Declared `int64` because BFL returns seeds above 2^31 (e.g. 2784347701), which an unformatted `integer` generates as a 32-bit field in many SDK generators. + + Format: `int64` + + + + Provider-reported start time of the generation, in seconds since the Unix epoch. `double`, not `float`: float32 spacing near a present-day epoch value is ~128 seconds, which collapses a whole generation's span to a single decoded value. + + Format: `double` + + + + Task status: Pending, Reasoning, Generating, Ready, Request Moderated, Content Moderated, Error, or Task not found. + + +

Examples

+ +

Input

+ +```json +{ + "aspect_ratio": "16:9", + "prompt": "A lighthouse on a rocky coast at golden hour, cinematic" +} +``` + +

Output

+ +```json +{ + "cost": null, + "id": "b2e0c1a4-0f2f-4a55-9f2e-2f9a1c0d4e77", + "progress": null, + "result": { + "cost": null, + "duration": 3.4, + "end_time": 1767225603.4, + "prompt": "A watercolor painting of a lighthouse at dawn, soft light on the water", + "sample": "https://example.invalid/bfl/flux-pro-1.1/sample.png", + "seed": 2784347701, + "start_time": 1767225600 + }, + "status": "Ready" +} +``` + +The URL is temporary. Download the image promptly if you need to keep it. +
+ +**Model ID:** `bfl/flux-pro-1.1` + +**Endpoint:** `POST https://api.comfy.org/v2/models/bfl/flux-pro-1.1` + + +```python Python +from comfy_sdk import Comfy + +# Reads COMFY_API_KEY from the environment. Each call sends a fresh +# Idempotency-Key and waits up to 10 minutes for the finished result. +with Comfy() as client: + result = client.models.run( + "bfl/flux-pro-1.1", + { + "prompt": "a single red maple leaf on a plain white background, studio lighting", + "width": 1024, + "height": 768, + }, + ) + +print("image:", result["result"]["sample"]) +``` + +```typescript TypeScript +import { comfy } from "@comfyorg/sdk"; + +// Reads COMFY_API_KEY from the environment. Each call sends a fresh +// Idempotency-Key and waits up to 10 minutes for the finished result. +type Result = { result: { sample: string } }; +const { data } = await comfy.models.run("bfl/flux-pro-1.1", { + prompt: "a single red maple leaf on a plain white background, studio lighting", + width: 1024, + height: 768, +}); + +console.log("image:", data.result.sample); +``` + +```bash cURL +curl https://api.comfy.org/v2/models/bfl/flux-pro-1.1 \ + -H "X-API-Key: $COMFY_API_KEY" \ + -H "Idempotency-Key: $(uuidgen)" \ + -H "Content-Type: application/json" \ + -d "{\"prompt\": \"a single red maple leaf on a plain white background, studio lighting\", \"width\": 1024, \"height\": 768}" +``` + + +

Schema

+ +

Input

+ + + Height of the generated image in pixels. Must be a multiple of 32. + + Range: `256` to `1440` + + + + Optional base64-encoded image to use with FLUX Redux. + + + + Output image format. + + Possible values: `jpeg`, `png`, `webp` + + + + Text prompt for image generation. + + + + Whether to upsample the prompt. If active, the prompt is automatically modified for more creative generation. + + + + Tolerance level for input and output moderation, between 0 (most strict) and 6 (least strict). + + Range: `0` to `6` + + + + Optional seed for reproducibility. A random seed is used when omitted. + + + + Optional secret for webhook signature verification. + + + + URL to receive webhook notifications. + + Format: `uri` + + + + Width of the generated image in pixels. Must be a multiple of 32. + + Range: `256` to `1440` + + +Generated from the schema Router serves at `GET /v2/models/bfl/flux-pro-1.1/openapi.json`, the same document it validates a call against before the request reaches the provider. + +

Output

+ + + Provider-reported cost in credits, populated once the task is Ready. + + Format: `float` + + + + BFL task identifier. + + + + Optional generation progress reported by BFL. + + Range: `0` to `1` + + Format: `float` + + + + The finished generation. Not nullable here: this component's `required` entry is a promise that a `200` carries the result, and a nullable `result` would reduce it to a key-presence check. + + + + Provider-reported cost of the generation. This is BFL's number, not the Comfy charge. + + Format: `double` + + + + Provider-reported generation duration in seconds. + + Format: `double` + + + + Provider-reported completion time of the generation, in seconds since the Unix epoch. `double` for the same reason as `start_time`. + + Format: `double` + + + + The prompt the generation actually ran, after any prompt upsampling. + + + + Signed URL for the generated asset. Router re-hosts the asset onto Comfy storage and rewrites this field, so it is normally a Comfy-hosted URL valid for up to 24 hours - signed for 24 hours when minted, and replayed from a 23-hour memo, so a later poll can hand back one with as little as an hour left; a leaf whose re-host could not be performed keeps BFL's own short-lived delivery URL instead - roughly two hours for video, roughly ten minutes for images. Either way the link expires, so download the asset rather than storing the URL. + + Format: `uri` + + + + The seed the generation used, whether supplied or chosen by the provider. Declared `int64` because BFL returns seeds above 2^31 (e.g. 2784347701), which an unformatted `integer` generates as a 32-bit field in many SDK generators. + + Format: `int64` + + + + Provider-reported start time of the generation, in seconds since the Unix epoch. `double`, not `float`: float32 spacing near a present-day epoch value is ~128 seconds, which collapses a whole generation's span to a single decoded value. + + Format: `double` + + + + Task status: Pending, Reasoning, Generating, Ready, Request Moderated, Content Moderated, Error, or Task not found. + + +

Examples

+ +

Input

+ +```json +{ + "height": 768, + "prompt": "An impressionist landscape of rolling hills under a summer sky", + "width": 1024 +} +``` + +

Output

+ +```json +{ + "cost": null, + "id": "b2e0c1a4-0f2f-4a55-9f2e-2f9a1c0d4e77", + "progress": null, + "result": { + "cost": null, + "duration": 3.4, + "end_time": 1767225603.4, + "prompt": "A watercolor painting of a lighthouse at dawn, soft light on the water", + "sample": "https://example.invalid/bfl/flux-pro-1.1/sample.png", + "seed": 2784347701, + "start_time": 1767225600 + }, + "status": "Ready" +} +``` + +The URL is temporary. Download the image promptly if you need to keep it. +
+
+ + diff --git a/development/comfy-router/models/black-forest-labs/flux-1-1-pro-ultra-image/code.yaml b/development/comfy-router/models/black-forest-labs/flux-1-1-pro-ultra-image/code.yaml new file mode 100644 index 000000000..d4f4b69cd --- /dev/null +++ b/development/comfy-router/models/black-forest-labs/flux-1-1-pro-ultra-image/code.yaml @@ -0,0 +1,174 @@ +# Source of truth for code.mdx in this directory. Edit this file, then run +# `pnpm code-pages:gen`; never edit code.mdx by hand (CI checks it is fresh). +# `input` / `output` are the provider's documented shapes (JSON Schema in YAML) and +# are the fallback until Router publishes the model's schema in router-schemas/. +name: Flux 1.1 Pro Ultra Image +provider: Black Forest Labs +description: >- + Python, TypeScript and cURL snippets for calling FLUX 1.1 [pro] Ultra and FLUX 1.1 [pro] over HTTP through + Comfy Router, plus the request fields and the result shape +summary: >- + FLUX 1.1 [pro] is a text-to-image model from Black Forest Labs. Ultra mode generates images at up to 4MP resolution. +task: generation +variants: +- title: FLUX 1.1 [pro] Ultra + model: bfl/flux-pro-1.1-ultra + example: + prompt: a single red maple leaf on a plain white background, studio lighting + aspect_ratio: '16:9' + raw: false + input: + type: object + required: [] + properties: + prompt: + type: string + description: >- + Text description of the image to generate. Optional in the schema (defaults to an empty prompt), + required in practice. + aspect_ratio: + type: string + description: Output aspect ratio, from `21:9` to `9:21`. + default: '16:9' + raw: + type: boolean + description: Generate with the less processed, more natural aesthetic. + default: false + image_prompt: + type: string + description: Optional base64 image to blend with the prompt. + image_prompt_strength: + type: number + description: How strongly `image_prompt` steers the result. + minimum: 0 + maximum: 1 + default: 0.1 + seed: + type: integer + description: Seed for reproducible results. Omit for random. + safety_tolerance: + type: integer + description: Moderation tolerance for inputs and outputs. 0 is strictest. + minimum: 0 + maximum: 6 + default: 2 + output_format: + type: string + description: Format of the returned image. + enum: + - jpeg + - png + - webp + default: jpeg + prompt_upsampling: + type: boolean + description: Rewrite the prompt for richer results. Results are not reproducible when enabled. + default: false + provider_spec: + url: https://api.bfl.ai/openapi.json + operation: POST /v1/flux-pro-1.1-ultra + response_operation: GET /v1/get_result + omit: + - webhook_url + - webhook_secret +- title: FLUX 1.1 [pro] + model: bfl/flux-pro-1.1 + example: + prompt: a single red maple leaf on a plain white background, studio lighting + width: 1024 + height: 768 + input: + type: object + required: [] + properties: + prompt: + type: string + description: >- + Text description of the image to generate. Optional in the schema (defaults to an empty prompt), + required in practice. + width: + type: integer + description: Output width in pixels, a multiple of 32. + minimum: 256 + maximum: 1440 + default: 1024 + height: + type: integer + description: Output height in pixels, a multiple of 32. + minimum: 256 + maximum: 1440 + default: 768 + prompt_upsampling: + type: boolean + description: Rewrite the prompt for richer results. Results are not reproducible when enabled. + default: false + image_prompt: + type: string + description: Optional base64 image to use as an image prompt. + seed: + type: integer + description: Seed for reproducible results. Omit for random. + safety_tolerance: + type: integer + description: Moderation tolerance for inputs and outputs. 0 is strictest. + minimum: 0 + maximum: 6 + default: 2 + output_format: + type: string + description: Format of the returned image. + enum: + - jpeg + - png + - webp + default: jpeg + provider_spec: + url: https://api.bfl.ai/openapi.json + operation: POST /v1/flux-pro-1.1 + response_operation: GET /v1/get_result + omit: + - webhook_url + - webhook_secret +example: + prompt: a single red maple leaf on a plain white background, studio lighting +output: + type: object + required: + - id + - status + - result + properties: + id: + type: string + description: BFL task id for this generation. + status: + type: string + description: Terminal task status. Router only returns once this is `Ready`. + enum: + - Ready + result: + type: object + required: + - sample + properties: + sample: + type: string + description: 'Signed URL of the generated image. Temporary: download it promptly.' + format: uri + prompt: + type: string + description: The prompt that was run, after any upsampling. + seed: + type: integer + description: Seed used for this generation. +result: + path: result.sample + label: image + example: + id: 0a1b2c3d-... + status: Ready + result: + sample: https://.../out.jpeg + prompt: a single red maple leaf on a plain white background, studio lighting + seed: 1234567890 + note: The URL is temporary. Download the image promptly if you need to keep it. diff --git a/development/comfy-router/models/black-forest-labs/flux-1-kontext/code.mdx b/development/comfy-router/models/black-forest-labs/flux-1-kontext/code.mdx new file mode 100644 index 000000000..86f1d434d --- /dev/null +++ b/development/comfy-router/models/black-forest-labs/flux-1-kontext/code.mdx @@ -0,0 +1,461 @@ +--- +title: "Use Flux.1 Kontext with Comfy Router" +description: "Python, TypeScript and cURL snippets for calling Flux.1 Kontext Pro and Kontext Max over HTTP through Comfy Router, plus the request fields and the result shape" +sidebarTitle: "Flux.1 Kontext" +--- + +{/* GENERATED FILE. Edit code.yaml in this directory and run `pnpm code-pages:gen`. */} + +import RouterPreviewNotice from "/snippets/comfy-router/preview-notice.mdx"; +import RouterCodeFooter from "/snippets/comfy-router/model-code-footer.mdx"; + +API Reference for Flux.1 Kontext. Flux.1 Kontext is Black Forest Labs' instruction-driven image editing model: send an image and a text instruction, get the edited image back with the rest of the scene preserved. + + + +## Quick start + +Create a key at [platform.comfy.org/profile/api-keys](https://platform.comfy.org/profile/api-keys) and export it as `COMFY_API_KEY`. The Python and TypeScript snippets use the Comfy SDKs (`pip install comfy-sdk`, `npm install @comfyorg/sdk`); the cURL snippet is the same call over raw HTTP. + +Pick the model you want to call. Everything below, from the snippets to the schema and examples, follows your choice. + + + +**Model ID:** `bfl/flux-kontext-pro` + +**Endpoint:** `POST https://api.comfy.org/v2/models/bfl/flux-kontext-pro` + + +```python Python +import base64 + +from comfy_sdk import Comfy + +with open("input.jpg", "rb") as f: + input_image = base64.b64encode(f.read()).decode() + +# Reads COMFY_API_KEY from the environment. Each call sends a fresh +# Idempotency-Key and waits up to 10 minutes for the finished result. +with Comfy() as client: + result = client.models.run( + "bfl/flux-kontext-pro", + { + "prompt": "replace the background with a sunlit beach, keep the subject unchanged", + "input_image": input_image, + "aspect_ratio": "1:1", + }, + ) + +print("image:", result["result"]["sample"]) +``` + +```typescript TypeScript +import { comfy } from "@comfyorg/sdk"; +import { readFile } from "node:fs/promises"; + +const inputImage = (await readFile("input.jpg")).toString("base64"); + +// Reads COMFY_API_KEY from the environment. Each call sends a fresh +// Idempotency-Key and waits up to 10 minutes for the finished result. +type Result = { result: { sample: string } }; +const { data } = await comfy.models.run("bfl/flux-kontext-pro", { + prompt: "replace the background with a sunlit beach, keep the subject unchanged", + input_image: inputImage, + aspect_ratio: "1:1", +}); + +console.log("image:", data.result.sample); +``` + +```bash cURL +INPUT_IMAGE=$(base64 < input.jpg | tr -d '\n') + +curl https://api.comfy.org/v2/models/bfl/flux-kontext-pro \ + -H "X-API-Key: $COMFY_API_KEY" \ + -H "Idempotency-Key: $(uuidgen)" \ + -H "Content-Type: application/json" \ + -d "{\"prompt\": \"replace the background with a sunlit beach, keep the subject unchanged\", \"input_image\": \"$INPUT_IMAGE\", \"aspect_ratio\": \"1:1\"}" +``` + + +

Schema

+ +

Input

+ + + Aspect ratio of the output between 21:9 and 9:21, e.g. 16:9. Defaults to the input image's aspect ratio when one is given, otherwise 1:1. + + + + Image to edit, as a base64-encoded image or an http(s) URL. Optional; without it the model generates from the prompt alone. + + + + Additional reference image, base64-encoded or an http(s) URL (experimental multi-reference). + + + + Additional reference image, base64-encoded or an http(s) URL (experimental multi-reference). + + + + Additional reference image, base64-encoded or an http(s) URL (experimental multi-reference). + + + + Output image format. + + Possible values: `jpeg`, `png`, `webp` + + + + Text prompt describing the edit to apply to input_image, or the image to generate when no input_image is given. + + + + Whether to upsample the prompt. If active, the prompt is automatically modified for more creative generation. + + + + Tolerance level for input and output moderation, between 0 (most strict) and 6 (least strict). + + Range: `0` to `6` + + + + Optional seed for reproducibility. A random seed is used when omitted. + + + + Optional secret for webhook signature verification. + + + + URL to receive webhook notifications. + + Format: `uri` + + +Generated from the schema Router serves at `GET /v2/models/bfl/flux-kontext-pro/openapi.json`, the same document it validates a call against before the request reaches the provider. + +

Output

+ + + Provider-reported cost in credits, populated once the task is Ready. + + Format: `float` + + + + BFL task identifier. + + + + Optional generation progress reported by BFL. + + Range: `0` to `1` + + Format: `float` + + + + The finished generation. Not nullable here: this component's `required` entry is a promise that a `200` carries the result, and a nullable `result` would reduce it to a key-presence check. + + + + Provider-reported cost of the generation. This is BFL's number, not the Comfy charge. + + Format: `double` + + + + Provider-reported generation duration in seconds. + + Format: `double` + + + + Provider-reported completion time of the generation, in seconds since the Unix epoch. `double` for the same reason as `start_time`. + + Format: `double` + + + + The prompt the generation actually ran, after any prompt upsampling. + + + + Signed URL for the generated asset. Router re-hosts the asset onto Comfy storage and rewrites this field, so it is normally a Comfy-hosted URL valid for up to 24 hours - signed for 24 hours when minted, and replayed from a 23-hour memo, so a later poll can hand back one with as little as an hour left; a leaf whose re-host could not be performed keeps BFL's own short-lived delivery URL instead - roughly two hours for video, roughly ten minutes for images. Either way the link expires, so download the asset rather than storing the URL. + + Format: `uri` + + + + The seed the generation used, whether supplied or chosen by the provider. Declared `int64` because BFL returns seeds above 2^31 (e.g. 2784347701), which an unformatted `integer` generates as a 32-bit field in many SDK generators. + + Format: `int64` + + + + Provider-reported start time of the generation, in seconds since the Unix epoch. `double`, not `float`: float32 spacing near a present-day epoch value is ~128 seconds, which collapses a whole generation's span to a single decoded value. + + Format: `double` + + + + Task status: Pending, Reasoning, Generating, Ready, Request Moderated, Content Moderated, Error, or Task not found. + + +

Examples

+ +

Input

+ +```json +{ + "prompt": "A watercolor painting of a lighthouse at dawn, soft light on the water" +} +``` + +

Output

+ +```json +{ + "cost": null, + "id": "b2e0c1a4-0f2f-4a55-9f2e-2f9a1c0d4e77", + "progress": null, + "result": { + "cost": null, + "duration": 3.4, + "end_time": 1767225603.4, + "prompt": "A watercolor painting of a lighthouse at dawn, soft light on the water", + "sample": "https://example.invalid/bfl/flux-pro-1.1/sample.png", + "seed": 2784347701, + "start_time": 1767225600 + }, + "status": "Ready" +} +``` + +The URL is temporary. Download the image promptly if you need to keep it. +
+ +**Model ID:** `bfl/flux-kontext-max` + +**Endpoint:** `POST https://api.comfy.org/v2/models/bfl/flux-kontext-max` + + +```python Python +import base64 + +from comfy_sdk import Comfy + +with open("input.jpg", "rb") as f: + input_image = base64.b64encode(f.read()).decode() + +# Reads COMFY_API_KEY from the environment. Each call sends a fresh +# Idempotency-Key and waits up to 10 minutes for the finished result. +with Comfy() as client: + result = client.models.run( + "bfl/flux-kontext-max", + { + "prompt": "replace the background with a sunlit beach, keep the subject unchanged", + "input_image": input_image, + "aspect_ratio": "1:1", + }, + ) + +print("image:", result["result"]["sample"]) +``` + +```typescript TypeScript +import { comfy } from "@comfyorg/sdk"; +import { readFile } from "node:fs/promises"; + +const inputImage = (await readFile("input.jpg")).toString("base64"); + +// Reads COMFY_API_KEY from the environment. Each call sends a fresh +// Idempotency-Key and waits up to 10 minutes for the finished result. +type Result = { result: { sample: string } }; +const { data } = await comfy.models.run("bfl/flux-kontext-max", { + prompt: "replace the background with a sunlit beach, keep the subject unchanged", + input_image: inputImage, + aspect_ratio: "1:1", +}); + +console.log("image:", data.result.sample); +``` + +```bash cURL +INPUT_IMAGE=$(base64 < input.jpg | tr -d '\n') + +curl https://api.comfy.org/v2/models/bfl/flux-kontext-max \ + -H "X-API-Key: $COMFY_API_KEY" \ + -H "Idempotency-Key: $(uuidgen)" \ + -H "Content-Type: application/json" \ + -d "{\"prompt\": \"replace the background with a sunlit beach, keep the subject unchanged\", \"input_image\": \"$INPUT_IMAGE\", \"aspect_ratio\": \"1:1\"}" +``` + + +

Schema

+ +

Input

+ + + Aspect ratio of the output between 21:9 and 9:21, e.g. 16:9. Defaults to the input image's aspect ratio when one is given, otherwise 1:1. + + + + Image to edit, as a base64-encoded image or an http(s) URL. Optional; without it the model generates from the prompt alone. + + + + Additional reference image, base64-encoded or an http(s) URL (experimental multi-reference). + + + + Additional reference image, base64-encoded or an http(s) URL (experimental multi-reference). + + + + Additional reference image, base64-encoded or an http(s) URL (experimental multi-reference). + + + + Output image format. + + Possible values: `jpeg`, `png`, `webp` + + + + Text prompt describing the edit to apply to input_image, or the image to generate when no input_image is given. + + + + Whether to upsample the prompt. If active, the prompt is automatically modified for more creative generation. + + + + Tolerance level for input and output moderation, between 0 (most strict) and 6 (least strict). + + Range: `0` to `6` + + + + Optional seed for reproducibility. A random seed is used when omitted. + + + + Optional secret for webhook signature verification. + + + + URL to receive webhook notifications. + + Format: `uri` + + +Generated from the schema Router serves at `GET /v2/models/bfl/flux-kontext-max/openapi.json`, the same document it validates a call against before the request reaches the provider. + +

Output

+ + + Provider-reported cost in credits, populated once the task is Ready. + + Format: `float` + + + + BFL task identifier. + + + + Optional generation progress reported by BFL. + + Range: `0` to `1` + + Format: `float` + + + + The finished generation. Not nullable here: this component's `required` entry is a promise that a `200` carries the result, and a nullable `result` would reduce it to a key-presence check. + + + + Provider-reported cost of the generation. This is BFL's number, not the Comfy charge. + + Format: `double` + + + + Provider-reported generation duration in seconds. + + Format: `double` + + + + Provider-reported completion time of the generation, in seconds since the Unix epoch. `double` for the same reason as `start_time`. + + Format: `double` + + + + The prompt the generation actually ran, after any prompt upsampling. + + + + Signed URL for the generated asset. Router re-hosts the asset onto Comfy storage and rewrites this field, so it is normally a Comfy-hosted URL valid for up to 24 hours - signed for 24 hours when minted, and replayed from a 23-hour memo, so a later poll can hand back one with as little as an hour left; a leaf whose re-host could not be performed keeps BFL's own short-lived delivery URL instead - roughly two hours for video, roughly ten minutes for images. Either way the link expires, so download the asset rather than storing the URL. + + Format: `uri` + + + + The seed the generation used, whether supplied or chosen by the provider. Declared `int64` because BFL returns seeds above 2^31 (e.g. 2784347701), which an unformatted `integer` generates as a 32-bit field in many SDK generators. + + Format: `int64` + + + + Provider-reported start time of the generation, in seconds since the Unix epoch. `double`, not `float`: float32 spacing near a present-day epoch value is ~128 seconds, which collapses a whole generation's span to a single decoded value. + + Format: `double` + + + + Task status: Pending, Reasoning, Generating, Ready, Request Moderated, Content Moderated, Error, or Task not found. + + +

Examples

+ +

Input

+ +```json +{ + "prompt": "A watercolor painting of a lighthouse at dawn, soft light on the water" +} +``` + +

Output

+ +```json +{ + "cost": null, + "id": "b2e0c1a4-0f2f-4a55-9f2e-2f9a1c0d4e77", + "progress": null, + "result": { + "cost": null, + "duration": 3.4, + "end_time": 1767225603.4, + "prompt": "A watercolor painting of a lighthouse at dawn, soft light on the water", + "sample": "https://example.invalid/bfl/flux-pro-1.1/sample.png", + "seed": 2784347701, + "start_time": 1767225600 + }, + "status": "Ready" +} +``` + +The URL is temporary. Download the image promptly if you need to keep it. +
+
+ + diff --git a/development/comfy-router/models/black-forest-labs/flux-1-kontext/code.yaml b/development/comfy-router/models/black-forest-labs/flux-1-kontext/code.yaml new file mode 100644 index 000000000..2cf579dde --- /dev/null +++ b/development/comfy-router/models/black-forest-labs/flux-1-kontext/code.yaml @@ -0,0 +1,122 @@ +# Source of truth for code.mdx in this directory. Edit this file, then run +# `pnpm code-pages:gen`; never edit code.mdx by hand (CI checks it is fresh). +# `input` / `output` are the provider's documented shapes (JSON Schema in YAML) and +# are the fallback until Router publishes the model's schema in router-schemas/. +name: Flux.1 Kontext +provider: Black Forest Labs +description: >- + Python, TypeScript and cURL snippets for calling Flux.1 Kontext Pro and Kontext Max over HTTP through + Comfy Router, plus the request fields and the result shape +summary: >- + Flux.1 Kontext is Black Forest Labs' instruction-driven image editing model: send an image and a text instruction, get the edited image back with the rest of the scene preserved. +task: image editing +variants: +- title: Kontext Pro + model: bfl/flux-kontext-pro + provider_spec: + url: https://api.bfl.ai/openapi.json + operation: POST /v1/flux-kontext-pro + response_operation: GET /v1/get_result + omit: + - webhook_url + - webhook_secret +- title: Kontext Max + model: bfl/flux-kontext-max + provider_spec: + url: https://api.bfl.ai/openapi.json + operation: POST /v1/flux-kontext-max + response_operation: GET /v1/get_result + omit: + - webhook_url + - webhook_secret +example: + prompt: replace the background with a sunlit beach, keep the subject unchanged + input_image: '@file:input.jpg' + aspect_ratio: '1:1' +input: + type: object + required: + - prompt + properties: + prompt: + type: string + description: What to change in the image. Describe the edit, not the whole scene. + input_image: + type: string + description: >- + The image to edit, base64 encoded or an HTTPS URL. Optional in the schema, but Kontext is an editing + model: without it you get text-to-image. + aspect_ratio: + type: string + description: Output aspect ratio, from `21:9` to `9:21`. + seed: + type: integer + description: Seed for reproducible results. Omit for random. + prompt_upsampling: + type: boolean + description: Rewrite the prompt for richer results. Results are not reproducible when enabled. + default: false + safety_tolerance: + type: integer + description: Moderation tolerance for inputs and outputs. 0 is strictest. + minimum: 0 + maximum: 6 + default: 2 + output_format: + type: string + description: Format of the returned image. + enum: + - jpeg + - png + - webp + default: png + input_image_2: + type: string + description: Additional reference image (base64 or HTTPS URL). Experimental multi-reference input. + input_image_3: + type: string + description: Additional reference image (base64 or HTTPS URL). Experimental multi-reference input. + input_image_4: + type: string + description: Additional reference image (base64 or HTTPS URL). Experimental multi-reference input. +output: + type: object + required: + - id + - status + - result + properties: + id: + type: string + description: BFL task id for this generation. + status: + type: string + description: Terminal task status. Router only returns once this is `Ready`. + enum: + - Ready + result: + type: object + required: + - sample + properties: + sample: + type: string + description: 'Signed URL of the generated image. Temporary: download it promptly.' + format: uri + prompt: + type: string + description: The prompt that was run, after any upsampling. + seed: + type: integer + description: Seed used for this generation. +result: + path: result.sample + label: image + example: + id: 0a1b2c3d-... + status: Ready + result: + sample: https://.../out.png + prompt: replace the background with a sunlit beach, keep the subject unchanged + seed: 1234567890 + note: The URL is temporary. Download the image promptly if you need to keep it. diff --git a/development/comfy-router/models/black-forest-labs/flux-3-video/code.mdx b/development/comfy-router/models/black-forest-labs/flux-3-video/code.mdx new file mode 100644 index 000000000..29c618efa --- /dev/null +++ b/development/comfy-router/models/black-forest-labs/flux-3-video/code.mdx @@ -0,0 +1,203 @@ +--- +title: "Use FLUX 3 Video with Comfy Router" +description: "Python, TypeScript and cURL snippets for generating video with synchronized audio from FLUX 3 over HTTP through Comfy Router, plus the request fields and the result shape" +sidebarTitle: "FLUX 3 Video" +--- + +{/* GENERATED FILE. Edit code.yaml in this directory and run `pnpm code-pages:gen`. */} + +import RouterPreviewNotice from "/snippets/comfy-router/preview-notice.mdx"; +import RouterCodeFooter from "/snippets/comfy-router/model-code-footer.mdx"; + +API Reference for FLUX 3 Video. FLUX 3 Video is Black Forest Labs' video generation model, turning a text prompt into a short clip with synchronized audio. + + + +## Quick start + +Create a key at [platform.comfy.org/profile/api-keys](https://platform.comfy.org/profile/api-keys) and export it as `COMFY_API_KEY`. The Python and TypeScript snippets use the Comfy SDKs (`pip install comfy-sdk`, `npm install @comfyorg/sdk`); the cURL snippet is the same call over raw HTTP. + +**Model ID:** `bfl/flux-3-video` + +**Endpoint:** `POST https://api.comfy.org/v2/models/bfl/flux-3-video` + + +```python Python +from comfy_sdk import Comfy + +# Reads COMFY_API_KEY from the environment. Each call sends a fresh +# Idempotency-Key and waits up to 10 minutes for the finished result. +with Comfy() as client: + result = client.models.run( + "bfl/flux-3-video", + { + "mode": "t2v", + "prompt": "a single red maple leaf falling onto still water, slow motion", + "duration": 5, + "aspect_ratio": "16:9", + "generate_audio": True, + }, + ) + +print("video:", result["result"]["sample"]) +``` + +```typescript TypeScript +import { comfy } from "@comfyorg/sdk"; + +// Reads COMFY_API_KEY from the environment. Each call sends a fresh +// Idempotency-Key and waits up to 10 minutes for the finished result. +type Result = { result: { sample: string } }; +const { data } = await comfy.models.run("bfl/flux-3-video", { + mode: "t2v", + prompt: "a single red maple leaf falling onto still water, slow motion", + duration: 5, + aspect_ratio: "16:9", + generate_audio: true, +}); + +console.log("video:", data.result.sample); +``` + +```bash cURL +curl https://api.comfy.org/v2/models/bfl/flux-3-video \ + -H "X-API-Key: $COMFY_API_KEY" \ + -H "Idempotency-Key: $(uuidgen)" \ + -H "Content-Type: application/json" \ + -d "{\"mode\": \"t2v\", \"prompt\": \"a single red maple leaf falling onto still water, slow motion\", \"duration\": 5, \"aspect_ratio\": \"16:9\", \"generate_audio\": true}" +``` + + +## Schema + +### Input + + + Output aspect ratio: auto, 21:9, 2:1, 16:9, 4:3, 1:1, 3:4, or 9:16. auto lets BFL choose from the prompt and any references. + + + + Draft mode: generate a fast preview whose result includes a draft_cache download URL. Send that bundle back with mode draft_enhance to render the full-quality version of the same generation. + + + + draft_enhance only. Encrypted draft-cache bundle from a prior draft generation, as the base64-encoded downloaded bundle or its still-valid http(s) URL. The original inputs are embedded in the bundle. + + + + Video duration in seconds (any whole second from 5 to 20), or auto to fit the content. + + Range: `5` to `20` + + + + Generate synchronized audio alongside the video. + + + + i2v only. Images that become frames of the video, each an http(s) URL or base64, one to ten total. Accepts a single image, a list of images (one starts the video, two start and end it, more spread evenly and need a set duration), or timestamped [seconds, image] pairs in time order, e.g. [[0, "..."], [3.5, "..."]]. + + + + Generation mode: t2v (text-to-video), i2v (image-continuation), v2v (video-continuation), or draft_enhance (full-quality render of a prior draft). Spelled-out aliases such as text-to-video are accepted. + + + + Free-form prompt describing the video. Required for every mode except draft_enhance. + + + + Video resolution class: hd, or fhd for a higher-resolution result finished by the video upsampler. Exact dimensions vary with the aspect ratio. + + + + Tolerance level for input and output harm moderation, 0 strictest. Sexual content is limited to level 3 and hate content to level 2 regardless of the requested tolerance; requests with conditioning media are limited to level 2. + + Range: `0` to `4` + + + + v2v only. The video to continue, an http(s) URL or base64 MP4; the generated clip carries on from its final frames. + + + + Endpoint version. latest serves the current release; dated pinnable release tags are added as they are published. + + +Generated from the schema Router serves at `GET /v2/models/bfl/flux-3-video/openapi.json`, the same document it validates a call against before the request reaches the provider. + +### Output + + + Provider-reported cost in credits, populated once the task is Ready. + + Format: `float` + + + + BFL task identifier. + + + + Optional generation progress reported by BFL. + + Range: `0` to `1` + + Format: `float` + + + + The finished generation. Exactly one of the two URL leaves is populated: `sample` in the default mode, `draft_cache` in `draft: true` mode. + + + + Provider-reported task cost. This is BFL's number, not the Comfy charge. + + Format: `double` + + + + Signed URL returned INSTEAD of `sample` by the `draft: true` mode, re-hosted onto Comfy storage the same way `sample` is: normally a Comfy-hosted URL valid for up to 24 hours, and BFL's own roughly two-hour delivery URL when the re-host could not be performed. + + Format: `uri` + + + + Signed URL for the generated MP4. Router re-hosts the asset onto Comfy storage and rewrites this field, so it is normally a Comfy-hosted URL valid for up to 24 hours - signed for 24 hours when minted and replayed from a 23-hour memo, so a later poll can hand back one with as little as an hour left; a leaf whose re-host could not be performed keeps BFL's own roughly two-hour delivery URL instead. Absent in `draft: true` mode. + + Format: `uri` + + + + Task status: Pending, Reasoning, Generating, Ready, Request Moderated, Content Moderated, Error, or Task not found. Compare case-insensitively; Router forwards BFL's spelling unchanged. + + +## Examples + +### Input + +```json +{ + "mode": "t2v", + "prompt": "A slow dolly shot through a rain-soaked neon street at night" +} +``` + +### Output + +```json +{ + "cost": null, + "id": "3f7a1b28-5c0d-4e91-8a6f-1b2c3d4e5f60", + "progress": null, + "result": { + "cost": null, + "draft_cache": "https://example.invalid/bfl/flux-3-video/draft.mp4" + }, + "status": "Ready" +} +``` + +`result.sample` is a signed URL that expires roughly two hours after the result is ready. Download the MP4 promptly. + + diff --git a/development/comfy-router/models/black-forest-labs/flux-3-video/code.yaml b/development/comfy-router/models/black-forest-labs/flux-3-video/code.yaml new file mode 100644 index 000000000..e5d445320 --- /dev/null +++ b/development/comfy-router/models/black-forest-labs/flux-3-video/code.yaml @@ -0,0 +1,144 @@ +# Source of truth for code.mdx in this directory. Edit this file, then run +# `pnpm code-pages:gen`; never edit code.mdx by hand (CI checks it is fresh). +# `input` / `output` are the provider's documented shapes (JSON Schema in YAML) and +# are the fallback until Router publishes the model's schema in router-schemas/. +name: FLUX 3 Video +provider: Black Forest Labs +description: >- + Python, TypeScript and cURL snippets for generating video with synchronized audio from FLUX 3 over HTTP + through Comfy Router, plus the request fields and the result shape +summary: >- + FLUX 3 Video is Black Forest Labs' video generation model, turning a text prompt into a short clip with synchronized audio. +task: video generation +variants: +- title: FLUX 3 Video + model: bfl/flux-3-video +example: + mode: t2v + prompt: a single red maple leaf falling onto still water, slow motion + duration: 5 + aspect_ratio: '16:9' + generate_audio: true +input: + type: object + required: + - mode + properties: + mode: + type: string + description: >- + Generation mode: `t2v` text-to-video, `i2v` continue from images in `keyframes`, `v2v` continue + the video in `start_video`, `draft_enhance` full-quality render of a prior draft. + enum: + - t2v + - i2v + - v2v + - draft_enhance + prompt: + type: string + description: Free-form description of the video. Required in every mode except `draft_enhance`. + keyframes: + type: array + description: >- + `i2v` only. One to ten images (HTTPS URLs or base64) that become frames: one starts the video, + two start and end it, more are spread evenly. + items: + type: string + start_video: + type: string + description: '`v2v` only. The video to continue, as an HTTPS URL or base64 MP4.' + duration: + description: >- + Video length in whole seconds, or `auto` to fit the content. `v2v` caps the range at 15 + seconds; the other modes accept up to 20. + anyOf: + - type: integer + minimum: 5 + maximum: 20 + - type: string + const: auto + default: auto + aspect_ratio: + type: string + description: Output aspect ratio. `auto` lets BFL choose from the inputs. + enum: + - auto + - '21:9' + - '2:1' + - '16:9' + - '4:3' + - '1:1' + - '3:4' + - '9:16' + default: auto + resolution: + type: string + description: '`fhd` (default) is finished by the video upsampler; `hd` is faster.' + enum: + - hd + - fhd + default: fhd + generate_audio: + type: boolean + description: Generate synchronized audio alongside the video. + default: true + safety_tolerance: + type: integer + description: Moderation tolerance for inputs and outputs. 0 is strictest. + minimum: 0 + maximum: 4 + default: 2 + draft: + type: boolean + description: Generate a fast preview instead of the full render. + default: false + version: + type: string + description: Endpoint version. `latest` serves the current release. + default: latest + draft_cache: + type: string + description: '`draft_enhance` only. The draft-cache bundle returned by a prior draft generation.' +output: + type: object + required: + - id + - status + - result + properties: + id: + type: string + description: BFL task id for this generation. + status: + type: string + description: Terminal task status. Router only returns once this is `Ready`. + enum: + - Ready + result: + type: object + required: + - sample + properties: + sample: + type: string + description: Signed URL of the generated MP4. Expires roughly two hours after the result is + ready. + format: uri +result: + path: result.sample + label: video + example: + id: 0a1b2c3d-... + status: Ready + result: + sample: https://.../out.mp4 + note: >- + `result.sample` is a signed URL that expires roughly two hours after the result is ready. Download + the MP4 promptly. +provider_spec: + url: https://api.bfl.ai/openapi.json + operation: POST /v1/flux-3-video + response_operation: GET /v1/get_result + omit: + - webhook_url + - webhook_secret diff --git a/development/comfy-router/models/black-forest-labs/flux-video-upscale/code.mdx b/development/comfy-router/models/black-forest-labs/flux-video-upscale/code.mdx new file mode 100644 index 000000000..5752e6a97 --- /dev/null +++ b/development/comfy-router/models/black-forest-labs/flux-video-upscale/code.mdx @@ -0,0 +1,212 @@ +--- +title: "Use FLUX Video Upscale with Comfy Router" +description: "Python, TypeScript and cURL snippets for upscaling a video with FLUX Video Upscale over HTTP through Comfy Router, plus the request fields and the result shape" +sidebarTitle: "FLUX Video Upscale" +--- + +{/* GENERATED FILE. Edit code.yaml in this directory and run `pnpm code-pages:gen`. */} + +import RouterPreviewNotice from "/snippets/comfy-router/preview-notice.mdx"; +import RouterCodeFooter from "/snippets/comfy-router/model-code-footer.mdx"; + +API Reference for FLUX Video Upscale. FLUX Video Upscale is Black Forest Labs' video upscaler: send a video, get a higher-resolution version back. + + + +## Quick start + +Create a key at [platform.comfy.org/profile/api-keys](https://platform.comfy.org/profile/api-keys) and export it as `COMFY_API_KEY`. The Python and TypeScript snippets use the Comfy SDKs (`pip install comfy-sdk`, `npm install @comfyorg/sdk`); the cURL snippet is the same call over raw HTTP. + +**Model ID:** `bfl/video-upscale-v1` + +**Endpoint:** `POST https://api.comfy.org/v2/models/bfl/video-upscale-v1` + + +```python Python +from comfy_sdk import Comfy + +# Reads COMFY_API_KEY from the environment. Each call sends a fresh +# Idempotency-Key and waits up to 10 minutes for the finished result. +with Comfy() as client: + result = client.models.run( + "bfl/video-upscale-v1", + { + "input_video": "https://your-host.example/clip.mp4", + "upscale_factor": 2, + "creativity": 1, + }, + ) + +print("video:", result["result"]["sample"]) +``` + +```typescript TypeScript +import { comfy } from "@comfyorg/sdk"; + +// Reads COMFY_API_KEY from the environment. Each call sends a fresh +// Idempotency-Key and waits up to 10 minutes for the finished result. +type Result = { result: { sample: string } }; +const { data } = await comfy.models.run("bfl/video-upscale-v1", { + input_video: "https://your-host.example/clip.mp4", + upscale_factor: 2, + creativity: 1, +}); + +console.log("video:", data.result.sample); +``` + +```bash cURL +curl https://api.comfy.org/v2/models/bfl/video-upscale-v1 \ + -H "X-API-Key: $COMFY_API_KEY" \ + -H "Idempotency-Key: $(uuidgen)" \ + -H "Content-Type: application/json" \ + -d "{\"input_video\": \"https://your-host.example/clip.mp4\", \"upscale_factor\": 2, \"creativity\": 1}" +``` + + +## Schema + +### Input + + + 0 preserves the source precisely and sharpens it; 1 allows creative detail enhancement, which does not strictly preserve faces or products. + + Range: `0` to `1` + + + + Video to upscale, either an HTTP(S) URL or a base64-encoded MP4. At most 20 seconds of source footage and 50MB. + + + + Optional description of the clip's content, steering the enhanced detail. Leave empty for a neutral upscale. + + + + Tolerance level for prompt and output frame moderation, 0 being most strict. + + Range: `0` to `4` + + + + Output scaling relative to the source resolution. The output preserves the source aspect ratio and is capped at roughly 14.4 megapixels per frame, so very large sources scale by less than the requested factor. + + Range: `1.5` to `3` + + Format: `float` + + + + Optional secret for webhook signature verification. + + + + URL to receive webhook notifications. + + Format: `uri` + + +Generated from the schema Router serves at `GET /v2/models/bfl/video-upscale-v1/openapi.json`, the same document it validates a call against before the request reaches the provider. + +### Output + + + Provider-reported cost in credits, populated once the task is Ready. + + Format: `float` + + + + BFL task identifier. + + + + Optional generation progress reported by BFL. + + Range: `0` to `1` + + Format: `float` + + + + The finished generation. Not nullable here: this component's `required` entry is a promise that a `200` carries the result, and a nullable `result` would reduce it to a key-presence check. + + + + Provider-reported cost of the generation. This is BFL's number, not the Comfy charge. + + Format: `double` + + + + Provider-reported generation duration in seconds. + + Format: `double` + + + + Provider-reported completion time of the generation, in seconds since the Unix epoch. `double` for the same reason as `start_time`. + + Format: `double` + + + + The prompt the generation actually ran, after any prompt upsampling. + + + + Signed URL for the generated asset. Router re-hosts the asset onto Comfy storage and rewrites this field, so it is normally a Comfy-hosted URL valid for up to 24 hours - signed for 24 hours when minted, and replayed from a 23-hour memo, so a later poll can hand back one with as little as an hour left; a leaf whose re-host could not be performed keeps BFL's own short-lived delivery URL instead - roughly two hours for video, roughly ten minutes for images. Either way the link expires, so download the asset rather than storing the URL. + + Format: `uri` + + + + The seed the generation used, whether supplied or chosen by the provider. Declared `int64` because BFL returns seeds above 2^31 (e.g. 2784347701), which an unformatted `integer` generates as a 32-bit field in many SDK generators. + + Format: `int64` + + + + Provider-reported start time of the generation, in seconds since the Unix epoch. `double`, not `float`: float32 spacing near a present-day epoch value is ~128 seconds, which collapses a whole generation's span to a single decoded value. + + Format: `double` + + + + Task status: Pending, Reasoning, Generating, Ready, Request Moderated, Content Moderated, Error, or Task not found. + + +## Examples + +### Input + +```json +{ + "input_video": "https://example.com/clip.mp4", + "upscale_factor": 2 +} +``` + +### Output + +```json +{ + "cost": null, + "id": "b2e0c1a4-0f2f-4a55-9f2e-2f9a1c0d4e77", + "progress": null, + "result": { + "cost": null, + "duration": 3.4, + "end_time": 1767225603.4, + "prompt": "A watercolor painting of a lighthouse at dawn, soft light on the water", + "sample": "https://example.invalid/bfl/flux-pro-1.1/sample.png", + "seed": 2784347701, + "start_time": 1767225600 + }, + "status": "Ready" +} +``` + +`result.sample` is a signed URL that expires roughly two hours after the result is ready. Download the MP4 promptly. + + diff --git a/development/comfy-router/models/black-forest-labs/flux-video-upscale/code.yaml b/development/comfy-router/models/black-forest-labs/flux-video-upscale/code.yaml new file mode 100644 index 000000000..defd916cd --- /dev/null +++ b/development/comfy-router/models/black-forest-labs/flux-video-upscale/code.yaml @@ -0,0 +1,96 @@ +# Source of truth for code.mdx in this directory. Edit this file, then run +# `pnpm code-pages:gen`; never edit code.mdx by hand (CI checks it is fresh). +# `input` / `output` are the provider's documented shapes (JSON Schema in YAML) and +# are the fallback until Router publishes the model's schema in router-schemas/. +name: FLUX Video Upscale +provider: Black Forest Labs +description: >- + Python, TypeScript and cURL snippets for upscaling a video with FLUX Video Upscale over HTTP through + Comfy Router, plus the request fields and the result shape +summary: >- + FLUX Video Upscale is Black Forest Labs' video upscaler: send a video, get a higher-resolution version back. +task: upscale +variants: +- title: FLUX Video Upscale + model: bfl/video-upscale-v1 +example: + input_video: https://your-host.example/clip.mp4 + upscale_factor: 2 + creativity: 1 +input: + type: object + required: + - input_video + properties: + input_video: + type: string + description: The video to upscale, as an HTTPS URL or a base64-encoded MP4. At most 20 seconds and + 50MB. + upscale_factor: + type: number + description: Resolution multiplier. + minimum: 1.5 + maximum: 3.0 + default: 2.0 + creativity: + type: integer + description: >- + `0` preserves faces and products faithfully; `1` lets the model add detail, suited to generated + footage. + enum: + - 0 + - 1 + default: 1 + prompt: + type: string + description: Optional guidance for the added detail. + default: '' + safety_tolerance: + type: integer + description: Moderation tolerance for inputs and outputs. 0 is strictest. + minimum: 0 + maximum: 4 + default: 2 +output: + type: object + required: + - id + - status + - result + properties: + id: + type: string + description: BFL task id for this generation. + status: + type: string + description: Terminal task status. Router only returns once this is `Ready`. + enum: + - Ready + result: + type: object + required: + - sample + properties: + sample: + type: string + description: Signed URL of the generated MP4. Expires roughly two hours after the result is + ready. + format: uri +result: + path: result.sample + label: video + example: + id: 0a1b2c3d-... + status: Ready + result: + sample: https://.../upscaled.mp4 + note: >- + `result.sample` is a signed URL that expires roughly two hours after the result is ready. Download + the MP4 promptly. +provider_spec: + url: https://api.bfl.ai/openapi.json + operation: POST /v1/flux-tools/video-upscale-v1 + response_operation: GET /v1/get_result + omit: + - webhook_url + - webhook_secret diff --git a/development/comfy-router/models/google/gemini/code.mdx b/development/comfy-router/models/google/gemini/code.mdx new file mode 100644 index 000000000..f17e2d396 --- /dev/null +++ b/development/comfy-router/models/google/gemini/code.mdx @@ -0,0 +1,793 @@ +--- +title: "Use Google Gemini with Comfy Router" +description: "Python, TypeScript and cURL snippets for calling Google Gemini text models over HTTP through Comfy Router, plus the request fields and the result shape" +sidebarTitle: "Google Gemini" +--- + +{/* GENERATED FILE. Edit code.yaml in this directory and run `pnpm code-pages:gen`. */} + +import RouterPreviewNotice from "/snippets/comfy-router/preview-notice.mdx"; +import RouterCodeFooter from "/snippets/comfy-router/model-code-footer.mdx"; + +API Reference for Google Gemini. Google Gemini is Google's family of multimodal text models, covering fast drafting through deep reasoning across the Flash and Pro tiers. + + + +## Quick start + +Create a key at [platform.comfy.org/profile/api-keys](https://platform.comfy.org/profile/api-keys) and export it as `COMFY_API_KEY`. The Python and TypeScript snippets use the Comfy SDKs (`pip install comfy-sdk`, `npm install @comfyorg/sdk`); the cURL snippet is the same call over raw HTTP. + +Pick the model you want to call. The models share one request and response shape, documented once below. + + + +**Model ID:** `vertexai/gemini-3.1-pro-preview` + +**Endpoint:** `POST https://api.comfy.org/v2/models/vertexai/gemini-3.1-pro-preview` + + +```python Python +from comfy_sdk import Comfy + +# Reads COMFY_API_KEY from the environment. Each call sends a fresh +# Idempotency-Key and waits up to 10 minutes for the finished result. +with Comfy() as client: + result = client.models.run( + "vertexai/gemini-3.1-pro-preview", + { + "contents": [ + { + "role": "user", + "parts": [ + { + "text": "Describe a single red maple leaf on a white background in one sentence.", + }, + ], + }, + ], + "generationConfig": { + "temperature": 0.7, + "maxOutputTokens": 256, + }, + }, + ) + +print("text:", result["candidates"][0]["content"]["parts"][0]["text"]) +``` + +```typescript TypeScript +import { comfy } from "@comfyorg/sdk"; + +// Reads COMFY_API_KEY from the environment. Each call sends a fresh +// Idempotency-Key and waits up to 10 minutes for the finished result. +type Result = { candidates: { content: { parts: { text: string }[] } }[] }; +const { data } = await comfy.models.run("vertexai/gemini-3.1-pro-preview", { + contents: [ + { + role: "user", + parts: [ + { + text: "Describe a single red maple leaf on a white background in one sentence.", + }, + ], + }, + ], + generationConfig: { + temperature: 0.7, + maxOutputTokens: 256, + }, +}); + +console.log("text:", data.candidates[0].content.parts[0].text); +``` + +```bash cURL +curl https://api.comfy.org/v2/models/vertexai/gemini-3.1-pro-preview \ + -H "X-API-Key: $COMFY_API_KEY" \ + -H "Idempotency-Key: $(uuidgen)" \ + -H "Content-Type: application/json" \ + -d "{\"contents\": [{\"role\":\"user\",\"parts\":[{\"text\":\"Describe a single red maple leaf on a white background in one sentence.\"}]}], \"generationConfig\": {\"temperature\":0.7,\"maxOutputTokens\":256}}" +``` + + + +**Model ID:** `vertexai/gemini-3.5-flash` + +**Endpoint:** `POST https://api.comfy.org/v2/models/vertexai/gemini-3.5-flash` + + +```python Python +from comfy_sdk import Comfy + +# Reads COMFY_API_KEY from the environment. Each call sends a fresh +# Idempotency-Key and waits up to 10 minutes for the finished result. +with Comfy() as client: + result = client.models.run( + "vertexai/gemini-3.5-flash", + { + "contents": [ + { + "role": "user", + "parts": [ + { + "text": "Describe a single red maple leaf on a white background in one sentence.", + }, + ], + }, + ], + "generationConfig": { + "temperature": 0.7, + "maxOutputTokens": 256, + }, + }, + ) + +print("text:", result["candidates"][0]["content"]["parts"][0]["text"]) +``` + +```typescript TypeScript +import { comfy } from "@comfyorg/sdk"; + +// Reads COMFY_API_KEY from the environment. Each call sends a fresh +// Idempotency-Key and waits up to 10 minutes for the finished result. +type Result = { candidates: { content: { parts: { text: string }[] } }[] }; +const { data } = await comfy.models.run("vertexai/gemini-3.5-flash", { + contents: [ + { + role: "user", + parts: [ + { + text: "Describe a single red maple leaf on a white background in one sentence.", + }, + ], + }, + ], + generationConfig: { + temperature: 0.7, + maxOutputTokens: 256, + }, +}); + +console.log("text:", data.candidates[0].content.parts[0].text); +``` + +```bash cURL +curl https://api.comfy.org/v2/models/vertexai/gemini-3.5-flash \ + -H "X-API-Key: $COMFY_API_KEY" \ + -H "Idempotency-Key: $(uuidgen)" \ + -H "Content-Type: application/json" \ + -d "{\"contents\": [{\"role\":\"user\",\"parts\":[{\"text\":\"Describe a single red maple leaf on a white background in one sentence.\"}]}], \"generationConfig\": {\"temperature\":0.7,\"maxOutputTokens\":256}}" +``` + + + +**Model ID:** `vertexai/gemini-2.5-pro` + +**Endpoint:** `POST https://api.comfy.org/v2/models/vertexai/gemini-2.5-pro` + + +```python Python +from comfy_sdk import Comfy + +# Reads COMFY_API_KEY from the environment. Each call sends a fresh +# Idempotency-Key and waits up to 10 minutes for the finished result. +with Comfy() as client: + result = client.models.run( + "vertexai/gemini-2.5-pro", + { + "contents": [ + { + "role": "user", + "parts": [ + { + "text": "Describe a single red maple leaf on a white background in one sentence.", + }, + ], + }, + ], + "generationConfig": { + "temperature": 0.7, + "maxOutputTokens": 256, + }, + }, + ) + +print("text:", result["candidates"][0]["content"]["parts"][0]["text"]) +``` + +```typescript TypeScript +import { comfy } from "@comfyorg/sdk"; + +// Reads COMFY_API_KEY from the environment. Each call sends a fresh +// Idempotency-Key and waits up to 10 minutes for the finished result. +type Result = { candidates: { content: { parts: { text: string }[] } }[] }; +const { data } = await comfy.models.run("vertexai/gemini-2.5-pro", { + contents: [ + { + role: "user", + parts: [ + { + text: "Describe a single red maple leaf on a white background in one sentence.", + }, + ], + }, + ], + generationConfig: { + temperature: 0.7, + maxOutputTokens: 256, + }, +}); + +console.log("text:", data.candidates[0].content.parts[0].text); +``` + +```bash cURL +curl https://api.comfy.org/v2/models/vertexai/gemini-2.5-pro \ + -H "X-API-Key: $COMFY_API_KEY" \ + -H "Idempotency-Key: $(uuidgen)" \ + -H "Content-Type: application/json" \ + -d "{\"contents\": [{\"role\":\"user\",\"parts\":[{\"text\":\"Describe a single red maple leaf on a white background in one sentence.\"}]}], \"generationConfig\": {\"temperature\":0.7,\"maxOutputTokens\":256}}" +``` + + + +**Model ID:** `vertexai/gemini-2.5-flash` + +**Endpoint:** `POST https://api.comfy.org/v2/models/vertexai/gemini-2.5-flash` + + +```python Python +from comfy_sdk import Comfy + +# Reads COMFY_API_KEY from the environment. Each call sends a fresh +# Idempotency-Key and waits up to 10 minutes for the finished result. +with Comfy() as client: + result = client.models.run( + "vertexai/gemini-2.5-flash", + { + "contents": [ + { + "role": "user", + "parts": [ + { + "text": "Describe a single red maple leaf on a white background in one sentence.", + }, + ], + }, + ], + "generationConfig": { + "temperature": 0.7, + "maxOutputTokens": 256, + }, + }, + ) + +print("text:", result["candidates"][0]["content"]["parts"][0]["text"]) +``` + +```typescript TypeScript +import { comfy } from "@comfyorg/sdk"; + +// Reads COMFY_API_KEY from the environment. Each call sends a fresh +// Idempotency-Key and waits up to 10 minutes for the finished result. +type Result = { candidates: { content: { parts: { text: string }[] } }[] }; +const { data } = await comfy.models.run("vertexai/gemini-2.5-flash", { + contents: [ + { + role: "user", + parts: [ + { + text: "Describe a single red maple leaf on a white background in one sentence.", + }, + ], + }, + ], + generationConfig: { + temperature: 0.7, + maxOutputTokens: 256, + }, +}); + +console.log("text:", data.candidates[0].content.parts[0].text); +``` + +```bash cURL +curl https://api.comfy.org/v2/models/vertexai/gemini-2.5-flash \ + -H "X-API-Key: $COMFY_API_KEY" \ + -H "Idempotency-Key: $(uuidgen)" \ + -H "Content-Type: application/json" \ + -d "{\"contents\": [{\"role\":\"user\",\"parts\":[{\"text\":\"Describe a single red maple leaf on a white background in one sentence.\"}]}], \"generationConfig\": {\"temperature\":0.7,\"maxOutputTokens\":256}}" +``` + + + + +## Schema + +### Input + + + The content of the current conversation with the model. For single-turn queries, this is a single instance. For multi-turn queries, this is a repeated field that contains conversation history and the latest request. + + + + + + + + URI based data. + + + + URI + + + + The media type of the file specified in the data or fileUri fields. Acceptable values include the following. For gemini-2.0-flash-lite and gemini-2.0-flash, the maximum length of an audio file is 8.4 hours and the maximum length of a video file (without audio) is one hour. For more information, see Gemini audio and video requirements. Text files must be UTF-8 encoded. The contents of the text file count toward the token limit. There is no limit on image resolution. + + Possible values: `application/pdf`, `audio/mpeg`, `audio/mp3`, `audio/wav`, `image/png`, `image/jpeg`, `image/webp`, `text/plain`, `video/mov`, `video/mpeg`, `video/mp4`, `video/mpg`, `video/avi`, `video/wmv`, `video/mpegps`, `video/flv` + + + + Inline data in raw bytes. For gemini-2.0-flash-lite and gemini-2.0-flash, you can specify up to 3000 images by using inlineData. + + + + The base64 encoding of the image, PDF, or video to include inline in the prompt. When including media inline, you must also specify the media type (mimeType) of the data. Size limit: 20MB + + Format: `byte` + + + + The media type of the file specified in the data or fileUri fields. Acceptable values include the following. For gemini-2.0-flash-lite and gemini-2.0-flash, the maximum length of an audio file is 8.4 hours and the maximum length of a video file (without audio) is one hour. For more information, see Gemini audio and video requirements. Text files must be UTF-8 encoded. The contents of the text file count toward the token limit. There is no limit on image resolution. + + Possible values: `application/pdf`, `audio/mpeg`, `audio/mp3`, `audio/wav`, `image/png`, `image/jpeg`, `image/webp`, `text/plain`, `video/mov`, `video/mpeg`, `video/mp4`, `video/mpg`, `video/avi`, `video/wmv`, `video/mpegps`, `video/flv` + + + + A text prompt or code snippet. + + + + Indicates this part is a thinking/reasoning step from the model. + + + + Possible values: `user`, `model` + + + + Sampling, length and output settings for the generation. Every field is optional: the fields below that declare a `default` apply it when omitted, and the rest fall back to the model's own behaviour. + + + + Configuration for image generation + + + + Aspect ratio for generated images + + + + Optional. The image output format for generated images. + + + + Optional. The compression quality of the output image. + + + + Optional. The image format that the output should be saved as. + + + + Optional. Specifies the size of generated images. Supported values are 1K, 2K, 4K. If not specified, the model will use default value 1K. + + + + Maximum number of tokens that can be generated in the response. A token is approximately 4 characters. 100 tokens correspond to roughly 60-80 words. + + Range: `16` to `8192` + + + + + + + + When seed is fixed to a specific value, the model makes a best effort to provide the same response for repeated requests. Deterministic output isn't guaranteed. Also, changing the model or parameter settings, such as the temperature, can cause variations in the response even when you use the same seed value. By default, a random seed value is used. Available for the following models:, gemini-2.5-flash, gemini-2.5-pro, gemini-2.5-flash-preview-04-1, gemini-2.5-pro-preview-05-0, gemini-2.0-flash-lite-00, gemini-2.0-flash-001 + + + + + + + + The temperature is used for sampling during response generation, which occurs when topP and topK are applied. Temperature controls the degree of randomness in token selection. Lower temperatures are good for prompts that require a less open-ended or creative response, while higher temperatures can lead to more diverse or creative results. A temperature of 0 means that the highest probability tokens are always selected. In this case, responses for a given prompt are mostly deterministic, but a small amount of variation is still possible. If the model returns a response that's too generic, too short, or the model gives a fallback response, try increasing the temperature + + Range: `0` to `2` + + Format: `float` + + + + Optional. Configuration for thinking features. Thinking is a process where the model breaks down a complex task into smaller steps to generate a higher-quality response. + + + + Optional. If true, the model will include its thoughts in the response. + + + + Optional. The token budget for the model's thinking process. The model will make a best effort to stay within this budget. + + + + Optional. The thinking level for the model. + + Possible values: `THINKING_LEVEL_UNSPECIFIED`, `LOW`, `MEDIUM`, `HIGH`, `MINIMAL` + + + + Top-K changes how the model selects tokens for output. A top-K of 1 means the next selected token is the most probable among all tokens in the model's vocabulary. A top-K of 3 means that the next token is selected from among the 3 most probable tokens by using temperature. + + Range: `1` to `…` + + + + If specified, nucleus sampling is used. +Top-P changes how the model selects tokens for output. Tokens are selected from the most (see top-K) to least probable until the sum of their probabilities equals the top-P value. For example, if tokens A, B, and C have a probability of 0.3, 0.2, and 0.1 and the top-P value is 0.5, then the model will select either A or B as the next token by using temperature and excludes C as a candidate. +Specify a lower value for less random responses and a higher value for more random responses. + + Range: `0` to `1` + + Format: `float` + + + + Per request settings for blocking unsafe content. Enforced on GenerateContentResponse.candidates. + + + + Possible values: `HARM_CATEGORY_SEXUALLY_EXPLICIT`, `HARM_CATEGORY_HATE_SPEECH`, `HARM_CATEGORY_HARASSMENT`, `HARM_CATEGORY_DANGEROUS_CONTENT` + + + + Possible values: `OFF`, `BLOCK_NONE`, `BLOCK_LOW_AND_ABOVE`, `BLOCK_MEDIUM_AND_ABOVE`, `BLOCK_ONLY_HIGH` + + + + Instructions for the model to steer it toward better performance. For example, "Answer as concisely as possible" or "Don't use technical terms in your response". The text strings count toward the token limit. The role field of systemInstruction is ignored and doesn't affect the performance of the model. Note: Only text should be used in parts and content in each part should be in a separate paragraph. + + + + A list of ordered parts that make up a single message. Different parts may have different IANA MIME types. For limits on the inputs, such as the maximum number of tokens or the number of images, see the model specifications on the Google models page. + + + + A text prompt or code snippet. + + + + The identity of the entity that creates the message. The following values are supported: user: This indicates that the message is sent by a real person, typically a user-generated message. model: This indicates that the message is generated by the model. The model value is used to insert messages from the model into the conversation during multi-turn conversations. For non-multi-turn conversations, this field can be left blank or unset. + + Possible values: `user`, `model` + + + + A piece of code that enables the system to interact with external systems to perform an action, or set of actions, outside of knowledge and scope of the model. See Function calling. + + + + + + + + + + + + + + + + JSON schema for the function parameters + + + + If true, generated images will be uploaded to cloud storage and returned as signed URLs instead of inline base64 data. The URLs expire after 24 hours. + + + + For video input, the start and end offset of the video in Duration format. For example, to specify a 10 second clip starting at 1:00, set "startOffset": \{ "seconds": 60 } and "endOffset": \{ "seconds": 70 }. The metadata should only be specified while the video data is presented in inlineData or fileData. + + + + Represents a duration offset for video timeline positions. + + + + Signed fractions of a second at nanosecond resolution. Negative second values with fractions must still have non-negative nanos values. + + Range: `0` to `999999999` + + + + Signed seconds of the span of time. Must be from -315,576,000,000 to +315,576,000,000 inclusive. + + Range: `-315576000000` to `315576000000` + + + + Represents a duration offset for video timeline positions. + + + + Signed fractions of a second at nanosecond resolution. Negative second values with fractions must still have non-negative nanos values. + + Range: `0` to `999999999` + + + + Signed seconds of the span of time. Must be from -315,576,000,000 to +315,576,000,000 inclusive. + + Range: `-315576000000` to `315576000000` + + +Generated from the schema Router serves at `GET /v2/models/vertexai/gemini-3.1-pro-preview/openapi.json`, the same document it validates a call against before the request reaches the provider. + +### Output + + + + + + + + + + + + + + + + + + + + + + + + + + + Format: `date` + + + + + + + + + + + + + + + + The content of the current conversation with the model. For single-turn queries, this is a single instance. For multi-turn queries, this is a repeated field that contains conversation history and the latest request. + + + + + + + + URI based data. + + + + URI + + + + The media type of the file specified in the data or fileUri fields. Acceptable values include the following. For gemini-2.0-flash-lite and gemini-2.0-flash, the maximum length of an audio file is 8.4 hours and the maximum length of a video file (without audio) is one hour. For more information, see Gemini audio and video requirements. Text files must be UTF-8 encoded. The contents of the text file count toward the token limit. There is no limit on image resolution. + + Possible values: `application/pdf`, `audio/mpeg`, `audio/mp3`, `audio/wav`, `image/png`, `image/jpeg`, `image/webp`, `text/plain`, `video/mov`, `video/mpeg`, `video/mp4`, `video/mpg`, `video/avi`, `video/wmv`, `video/mpegps`, `video/flv` + + + + Inline data in raw bytes. For gemini-2.0-flash-lite and gemini-2.0-flash, you can specify up to 3000 images by using inlineData. + + + + The base64 encoding of the image, PDF, or video to include inline in the prompt. When including media inline, you must also specify the media type (mimeType) of the data. Size limit: 20MB + + Format: `byte` + + + + The media type of the file specified in the data or fileUri fields. Acceptable values include the following. For gemini-2.0-flash-lite and gemini-2.0-flash, the maximum length of an audio file is 8.4 hours and the maximum length of a video file (without audio) is one hour. For more information, see Gemini audio and video requirements. Text files must be UTF-8 encoded. The contents of the text file count toward the token limit. There is no limit on image resolution. + + Possible values: `application/pdf`, `audio/mpeg`, `audio/mp3`, `audio/wav`, `image/png`, `image/jpeg`, `image/webp`, `text/plain`, `video/mov`, `video/mpeg`, `video/mp4`, `video/mpg`, `video/avi`, `video/wmv`, `video/mpegps`, `video/flv` + + + + A text prompt or code snippet. + + + + Indicates this part is a thinking/reasoning step from the model. + + + + Possible values: `user`, `model` + + + + + + + + + + + + Possible values: `HARM_CATEGORY_SEXUALLY_EXPLICIT`, `HARM_CATEGORY_HATE_SPEECH`, `HARM_CATEGORY_HARASSMENT`, `HARM_CATEGORY_DANGEROUS_CONTENT` + + + + The probability that the content violates the specified safety category + + Possible values: `NEGLIGIBLE`, `LOW`, `MEDIUM`, `HIGH`, `UNKNOWN` + + + + Timestamp when the response was created. + + + + The model version used to generate the response. + + + + + + + + + + + + + + + + + + + + Possible values: `HARM_CATEGORY_SEXUALLY_EXPLICIT`, `HARM_CATEGORY_HATE_SPEECH`, `HARM_CATEGORY_HARASSMENT`, `HARM_CATEGORY_DANGEROUS_CONTENT` + + + + The probability that the content violates the specified safety category + + Possible values: `NEGLIGIBLE`, `LOW`, `MEDIUM`, `HIGH`, `UNKNOWN` + + + + Unique identifier for the response. + + + + + + + + Output only. Number of tokens in the cached part in the input (the cached content). + + + + Number of tokens in the response(s). + + + + Breakdown of candidate tokens by modality. + + + + Type of input or output content modality. + + Possible values: `MODALITY_UNSPECIFIED`, `TEXT`, `IMAGE`, `VIDEO`, `AUDIO`, `DOCUMENT` + + + + Number of tokens for the given modality. + + + + Number of tokens in the request. When cachedContent is set, this is still the total effective prompt size meaning this includes the number of tokens in the cached content. + + + + Breakdown of prompt tokens by modality. + + + + Type of input or output content modality. + + Possible values: `MODALITY_UNSPECIFIED`, `TEXT`, `IMAGE`, `VIDEO`, `AUDIO`, `DOCUMENT` + + + + Number of tokens for the given modality. + + + + Number of tokens present in thoughts output. + + + + Number of tokens present in tool-use prompt(s). + + + + Total number of tokens (prompt + candidates). + + + + Traffic type used for the request (e.g., PROVISIONED_THROUGHPUT). + + +## Examples + +### Input + +```json +{ + "contents": [ + { + "parts": [ + { + "text": "Describe a robot learning to paint, in two sentences." + } + ], + "role": "user" + } + ] +} +``` + +### Output + +```json +{ + "candidates": [ + { + "content": { + "parts": [ + { + "text": "A lighthouse stands at the edge of the harbour, its lamp still turning as the sun comes up." + } + ], + "role": "model" + }, + "finishReason": "STOP" + } + ], + "modelVersion": "gemini-2.5-flash", + "responseId": "0d1f2a3b-4c5d-6e7f-8a9b-0c1d2e3f4a5b", + "usageMetadata": { + "candidatesTokenCount": 21, + "promptTokenCount": 12, + "totalTokenCount": 33 + } +} +``` + + diff --git a/development/comfy-router/models/google/gemini/code.yaml b/development/comfy-router/models/google/gemini/code.yaml new file mode 100644 index 000000000..7a568dcfb --- /dev/null +++ b/development/comfy-router/models/google/gemini/code.yaml @@ -0,0 +1,227 @@ +# Source of truth for code.mdx in this directory. Edit this file, then run +# `pnpm code-pages:gen`; never edit code.mdx by hand (CI checks it is fresh). +# `input` / `output` are the provider's documented shapes (JSON Schema in YAML) and +# are the fallback until Router publishes the model's schema in router-schemas/. +name: Google Gemini +provider: Google +description: >- + Python, TypeScript and cURL snippets for calling Google Gemini text models over HTTP through Comfy Router, + plus the request fields and the result shape +summary: >- + Google Gemini is Google's family of multimodal text models, covering fast drafting through deep reasoning across the Flash and Pro tiers. +task: response +variants: +- title: Gemini 3.1 Pro + model: vertexai/gemini-3.1-pro-preview +- title: Gemini 3.5 Flash + model: vertexai/gemini-3.5-flash +- title: Gemini 2.5 Pro + model: vertexai/gemini-2.5-pro +- title: Gemini 2.5 Flash + model: vertexai/gemini-2.5-flash +example: + contents: + - role: user + parts: + - text: Describe a single red maple leaf on a white background in one sentence. + generationConfig: + temperature: 0.7 + maxOutputTokens: 256 +input: + type: object + required: + - contents + properties: + contents: + type: array + description: >- + The conversation so far, oldest first. A part is `text`, or `inlineData` to send an image, audio + or video alongside the prompt. + items: + type: object + required: + - role + - parts + properties: + role: + type: string + description: Who authored the turn. + enum: + - user + - model + parts: + type: array + description: The turn's content parts. + items: + type: object + properties: + text: + type: string + description: A text part. + inlineData: + type: object + description: An inline media part. + properties: + mimeType: + type: string + description: Media type, for example `image/png`. + data: + type: string + description: Base64-encoded media bytes. + systemInstruction: + type: object + description: System prompt applied to the whole conversation. + properties: + parts: + type: array + items: + type: object + properties: + text: + type: string + generationConfig: + type: object + description: Sampling and length settings. + properties: + temperature: + type: number + description: Randomness of sampling. Defaults are model specific. + minimum: 0 + maximum: 2 + topP: + type: number + description: Nucleus sampling threshold. Defaults are model specific. + topK: + type: integer + description: Top-k sampling cutoff. Defaults are model specific. + maxOutputTokens: + type: integer + description: Upper bound on generated tokens. + seed: + type: integer + description: Seed for reproducible sampling. + stopSequences: + type: array + description: Strings that end generation when produced. + items: + type: string + thinkingConfig: + type: object + description: Reasoning effort controls on models that support it. + properties: + thinkingLevel: + type: string + enum: + - MINIMAL + - LOW + - MEDIUM + - HIGH + includeThoughts: + type: boolean + description: Return the model's reasoning alongside the answer. + tools: + type: array + description: Function declarations the model may call. + items: + type: object + properties: + functionDeclarations: + type: array + items: + type: object + properties: + name: + type: string + description: + type: string + parameters: + type: object + description: JSON Schema for the function's arguments. +output: + type: object + properties: + candidates: + type: array + description: Generated candidates; one unless you asked for more. + items: + type: object + properties: + content: + type: object + properties: + role: + type: string + enum: + - model + parts: + type: array + items: + type: object + properties: + text: + type: string + description: The generated text. + functionCall: + type: object + description: Present when the model chose to call one of your `tools`. + properties: + name: + type: string + args: + type: object + finishReason: + type: string + description: 'Why generation stopped: `STOP`, `MAX_TOKENS`, `SAFETY`, ...' + usageMetadata: + type: object + description: Token accounting for the call. + properties: + promptTokenCount: + type: integer + description: Tokens in the prompt. + candidatesTokenCount: + type: integer + description: Tokens in the generated candidates. + promptFeedback: + type: object + description: >- + Present when the prompt itself was blocked. It is the only field returned in that case, so + check for it before reading the result. + properties: + blockReason: + type: string + description: Why the prompt was blocked. No candidates are returned; rephrase the prompt and retry. + enum: + - SAFETY + - OTHER + - BLOCKLIST + - PROHIBITED_CONTENT + - IMAGE_SAFETY +result: + path: candidates[0].content.parts[0].text + label: text + example: + candidates: + - content: + role: model + parts: + - text: A single red maple leaf rests on a plain white background, its edges sharp and its color + deep. + finishReason: STOP + usageMetadata: + promptTokenCount: 18 + candidatesTokenCount: 24 +provider_spec: + url: https://generativelanguage.googleapis.com/$discovery/rest?version=v1beta + request: GenerateContentRequest + response: GenerateContentResponse + omit: + - promptFeedback.safetyRatings + - model + - store + - serviceTier + - cachedContent + - toolConfig + - modelVersion + - modelStatus + - responseId diff --git a/development/comfy-router/models/google/nano-banana-2-lite/code.mdx b/development/comfy-router/models/google/nano-banana-2-lite/code.mdx new file mode 100644 index 000000000..0955799e3 --- /dev/null +++ b/development/comfy-router/models/google/nano-banana-2-lite/code.mdx @@ -0,0 +1,586 @@ +--- +title: "Use Nano Banana 2 Lite with Comfy Router" +description: "Python, TypeScript and cURL snippets for generating images with Nano Banana 2 Lite (Gemini 3.1 Flash-Lite Image) over HTTP through Comfy Router, plus the request fields and the result shape" +sidebarTitle: "Nano Banana 2 Lite" +--- + +{/* GENERATED FILE. Edit code.yaml in this directory and run `pnpm code-pages:gen`. */} + +import RouterPreviewNotice from "/snippets/comfy-router/preview-notice.mdx"; +import RouterCodeFooter from "/snippets/comfy-router/model-code-footer.mdx"; + +API Reference for Nano Banana 2 Lite. Nano Banana 2 Lite (Gemini 3.1 Flash-Lite Image) is the Flash-Lite tier of Google's Nano Banana image generation family, tuned for lower latency and cost. + + + +## Quick start + +Create a key at [platform.comfy.org/profile/api-keys](https://platform.comfy.org/profile/api-keys) and export it as `COMFY_API_KEY`. The Python and TypeScript snippets use the Comfy SDKs (`pip install comfy-sdk`, `npm install @comfyorg/sdk`); the cURL snippet is the same call over raw HTTP. + +**Model ID:** `vertexai/gemini-3.1-flash-lite-image` + +**Endpoint:** `POST https://api.comfy.org/v2/models/vertexai/gemini-3.1-flash-lite-image` + + +```python Python +from comfy_sdk import Comfy + +# Reads COMFY_API_KEY from the environment. Each call sends a fresh +# Idempotency-Key and waits up to 10 minutes for the finished result. +with Comfy() as client: + result = client.models.run( + "vertexai/gemini-3.1-flash-lite-image", + { + "contents": [ + { + "role": "user", + "parts": [ + { + "text": "a single red maple leaf on a plain white background, studio lighting", + }, + ], + }, + ], + "generationConfig": { + "responseModalities": ["IMAGE"], + "imageConfig": { + "aspectRatio": "1:1", + }, + }, + }, + ) + +print("image (base64):", result["candidates"][0]["content"]["parts"][0]["inlineData"]["data"]) +``` + +```typescript TypeScript +import { comfy } from "@comfyorg/sdk"; + +// Reads COMFY_API_KEY from the environment. Each call sends a fresh +// Idempotency-Key and waits up to 10 minutes for the finished result. +type Result = { candidates: { content: { parts: { inlineData: { data: string } }[] } }[] }; +const { data } = await comfy.models.run("vertexai/gemini-3.1-flash-lite-image", { + contents: [ + { + role: "user", + parts: [ + { + text: "a single red maple leaf on a plain white background, studio lighting", + }, + ], + }, + ], + generationConfig: { + responseModalities: ["IMAGE"], + imageConfig: { + aspectRatio: "1:1", + }, + }, +}); + +console.log("image (base64):", data.candidates[0].content.parts[0].inlineData.data); +``` + +```bash cURL +curl https://api.comfy.org/v2/models/vertexai/gemini-3.1-flash-lite-image \ + -H "X-API-Key: $COMFY_API_KEY" \ + -H "Idempotency-Key: $(uuidgen)" \ + -H "Content-Type: application/json" \ + -d "{\"contents\": [{\"role\":\"user\",\"parts\":[{\"text\":\"a single red maple leaf on a plain white background, studio lighting\"}]}], \"generationConfig\": {\"responseModalities\":[\"IMAGE\"],\"imageConfig\":{\"aspectRatio\":\"1:1\"}}}" +``` + + +## Schema + +### Input + + + The content of the current conversation with the model. For single-turn queries, this is a single instance. For multi-turn queries, this is a repeated field that contains conversation history and the latest request. + + + + + + + + URI based data. + + + + URI + + + + The media type of the file specified in the data or fileUri fields. Acceptable values include the following. For gemini-2.0-flash-lite and gemini-2.0-flash, the maximum length of an audio file is 8.4 hours and the maximum length of a video file (without audio) is one hour. For more information, see Gemini audio and video requirements. Text files must be UTF-8 encoded. The contents of the text file count toward the token limit. There is no limit on image resolution. + + Possible values: `application/pdf`, `audio/mpeg`, `audio/mp3`, `audio/wav`, `image/png`, `image/jpeg`, `image/webp`, `text/plain`, `video/mov`, `video/mpeg`, `video/mp4`, `video/mpg`, `video/avi`, `video/wmv`, `video/mpegps`, `video/flv` + + + + Inline data in raw bytes. For gemini-2.0-flash-lite and gemini-2.0-flash, you can specify up to 3000 images by using inlineData. + + + + The base64 encoding of the image, PDF, or video to include inline in the prompt. When including media inline, you must also specify the media type (mimeType) of the data. Size limit: 20MB + + Format: `byte` + + + + The media type of the file specified in the data or fileUri fields. Acceptable values include the following. For gemini-2.0-flash-lite and gemini-2.0-flash, the maximum length of an audio file is 8.4 hours and the maximum length of a video file (without audio) is one hour. For more information, see Gemini audio and video requirements. Text files must be UTF-8 encoded. The contents of the text file count toward the token limit. There is no limit on image resolution. + + Possible values: `application/pdf`, `audio/mpeg`, `audio/mp3`, `audio/wav`, `image/png`, `image/jpeg`, `image/webp`, `text/plain`, `video/mov`, `video/mpeg`, `video/mp4`, `video/mpg`, `video/avi`, `video/wmv`, `video/mpegps`, `video/flv` + + + + A text prompt or code snippet. + + + + Indicates this part is a thinking/reasoning step from the model. + + + + Possible values: `user`, `model` + + + + Sampling, length and output settings for the generation. Every field is optional: the fields below that declare a `default` apply it when omitted, and the rest fall back to the model's own behaviour. + + + + Configuration for image generation + + + + Aspect ratio for generated images + + + + Optional. The image output format for generated images. + + + + Optional. The compression quality of the output image. + + + + Optional. The image format that the output should be saved as. + + + + Optional. Specifies the size of generated images. Supported values are 1K, 2K, 4K. If not specified, the model will use default value 1K. + + + + Maximum number of tokens that can be generated in the response. A token is approximately 4 characters. 100 tokens correspond to roughly 60-80 words. + + Range: `16` to `8192` + + + + + + + + When seed is fixed to a specific value, the model makes a best effort to provide the same response for repeated requests. Deterministic output isn't guaranteed. Also, changing the model or parameter settings, such as the temperature, can cause variations in the response even when you use the same seed value. By default, a random seed value is used. Available for the following models:, gemini-2.5-flash, gemini-2.5-pro, gemini-2.5-flash-preview-04-1, gemini-2.5-pro-preview-05-0, gemini-2.0-flash-lite-00, gemini-2.0-flash-001 + + + + + + + + The temperature is used for sampling during response generation, which occurs when topP and topK are applied. Temperature controls the degree of randomness in token selection. Lower temperatures are good for prompts that require a less open-ended or creative response, while higher temperatures can lead to more diverse or creative results. A temperature of 0 means that the highest probability tokens are always selected. In this case, responses for a given prompt are mostly deterministic, but a small amount of variation is still possible. If the model returns a response that's too generic, too short, or the model gives a fallback response, try increasing the temperature + + Range: `0` to `2` + + Format: `float` + + + + Optional. Configuration for thinking features. Thinking is a process where the model breaks down a complex task into smaller steps to generate a higher-quality response. + + + + Optional. If true, the model will include its thoughts in the response. + + + + Optional. The token budget for the model's thinking process. The model will make a best effort to stay within this budget. + + + + Optional. The thinking level for the model. + + Possible values: `THINKING_LEVEL_UNSPECIFIED`, `LOW`, `MEDIUM`, `HIGH`, `MINIMAL` + + + + Top-K changes how the model selects tokens for output. A top-K of 1 means the next selected token is the most probable among all tokens in the model's vocabulary. A top-K of 3 means that the next token is selected from among the 3 most probable tokens by using temperature. + + Range: `1` to `…` + + + + If specified, nucleus sampling is used. +Top-P changes how the model selects tokens for output. Tokens are selected from the most (see top-K) to least probable until the sum of their probabilities equals the top-P value. For example, if tokens A, B, and C have a probability of 0.3, 0.2, and 0.1 and the top-P value is 0.5, then the model will select either A or B as the next token by using temperature and excludes C as a candidate. +Specify a lower value for less random responses and a higher value for more random responses. + + Range: `0` to `1` + + Format: `float` + + + + Per request settings for blocking unsafe content. Enforced on GenerateContentResponse.candidates. + + + + Possible values: `HARM_CATEGORY_SEXUALLY_EXPLICIT`, `HARM_CATEGORY_HATE_SPEECH`, `HARM_CATEGORY_HARASSMENT`, `HARM_CATEGORY_DANGEROUS_CONTENT` + + + + Possible values: `OFF`, `BLOCK_NONE`, `BLOCK_LOW_AND_ABOVE`, `BLOCK_MEDIUM_AND_ABOVE`, `BLOCK_ONLY_HIGH` + + + + Instructions for the model to steer it toward better performance. For example, "Answer as concisely as possible" or "Don't use technical terms in your response". The text strings count toward the token limit. The role field of systemInstruction is ignored and doesn't affect the performance of the model. Note: Only text should be used in parts and content in each part should be in a separate paragraph. + + + + A list of ordered parts that make up a single message. Different parts may have different IANA MIME types. For limits on the inputs, such as the maximum number of tokens or the number of images, see the model specifications on the Google models page. + + + + A text prompt or code snippet. + + + + The identity of the entity that creates the message. The following values are supported: user: This indicates that the message is sent by a real person, typically a user-generated message. model: This indicates that the message is generated by the model. The model value is used to insert messages from the model into the conversation during multi-turn conversations. For non-multi-turn conversations, this field can be left blank or unset. + + Possible values: `user`, `model` + + + + A piece of code that enables the system to interact with external systems to perform an action, or set of actions, outside of knowledge and scope of the model. See Function calling. + + + + + + + + + + + + + + + + JSON schema for the function parameters + + + + If true, generated images will be uploaded to cloud storage and returned as signed URLs instead of inline base64 data. The URLs expire after 24 hours. + + + + For video input, the start and end offset of the video in Duration format. For example, to specify a 10 second clip starting at 1:00, set "startOffset": \{ "seconds": 60 } and "endOffset": \{ "seconds": 70 }. The metadata should only be specified while the video data is presented in inlineData or fileData. + + + + Represents a duration offset for video timeline positions. + + + + Signed fractions of a second at nanosecond resolution. Negative second values with fractions must still have non-negative nanos values. + + Range: `0` to `999999999` + + + + Signed seconds of the span of time. Must be from -315,576,000,000 to +315,576,000,000 inclusive. + + Range: `-315576000000` to `315576000000` + + + + Represents a duration offset for video timeline positions. + + + + Signed fractions of a second at nanosecond resolution. Negative second values with fractions must still have non-negative nanos values. + + Range: `0` to `999999999` + + + + Signed seconds of the span of time. Must be from -315,576,000,000 to +315,576,000,000 inclusive. + + Range: `-315576000000` to `315576000000` + + +Generated from the schema Router serves at `GET /v2/models/vertexai/gemini-3.1-flash-lite-image/openapi.json`, the same document it validates a call against before the request reaches the provider. + +### Output + + + + + + + + + + + + + + + + + + + + + + + + + + + Format: `date` + + + + + + + + + + + + + + + + The content of the current conversation with the model. For single-turn queries, this is a single instance. For multi-turn queries, this is a repeated field that contains conversation history and the latest request. + + + + + + + + URI based data. + + + + URI + + + + The media type of the file specified in the data or fileUri fields. Acceptable values include the following. For gemini-2.0-flash-lite and gemini-2.0-flash, the maximum length of an audio file is 8.4 hours and the maximum length of a video file (without audio) is one hour. For more information, see Gemini audio and video requirements. Text files must be UTF-8 encoded. The contents of the text file count toward the token limit. There is no limit on image resolution. + + Possible values: `application/pdf`, `audio/mpeg`, `audio/mp3`, `audio/wav`, `image/png`, `image/jpeg`, `image/webp`, `text/plain`, `video/mov`, `video/mpeg`, `video/mp4`, `video/mpg`, `video/avi`, `video/wmv`, `video/mpegps`, `video/flv` + + + + Inline data in raw bytes. For gemini-2.0-flash-lite and gemini-2.0-flash, you can specify up to 3000 images by using inlineData. + + + + The base64 encoding of the image, PDF, or video to include inline in the prompt. When including media inline, you must also specify the media type (mimeType) of the data. Size limit: 20MB + + Format: `byte` + + + + The media type of the file specified in the data or fileUri fields. Acceptable values include the following. For gemini-2.0-flash-lite and gemini-2.0-flash, the maximum length of an audio file is 8.4 hours and the maximum length of a video file (without audio) is one hour. For more information, see Gemini audio and video requirements. Text files must be UTF-8 encoded. The contents of the text file count toward the token limit. There is no limit on image resolution. + + Possible values: `application/pdf`, `audio/mpeg`, `audio/mp3`, `audio/wav`, `image/png`, `image/jpeg`, `image/webp`, `text/plain`, `video/mov`, `video/mpeg`, `video/mp4`, `video/mpg`, `video/avi`, `video/wmv`, `video/mpegps`, `video/flv` + + + + A text prompt or code snippet. + + + + Indicates this part is a thinking/reasoning step from the model. + + + + Possible values: `user`, `model` + + + + + + + + + + + + Possible values: `HARM_CATEGORY_SEXUALLY_EXPLICIT`, `HARM_CATEGORY_HATE_SPEECH`, `HARM_CATEGORY_HARASSMENT`, `HARM_CATEGORY_DANGEROUS_CONTENT` + + + + The probability that the content violates the specified safety category + + Possible values: `NEGLIGIBLE`, `LOW`, `MEDIUM`, `HIGH`, `UNKNOWN` + + + + Timestamp when the response was created. + + + + The model version used to generate the response. + + + + + + + + + + + + + + + + + + + + Possible values: `HARM_CATEGORY_SEXUALLY_EXPLICIT`, `HARM_CATEGORY_HATE_SPEECH`, `HARM_CATEGORY_HARASSMENT`, `HARM_CATEGORY_DANGEROUS_CONTENT` + + + + The probability that the content violates the specified safety category + + Possible values: `NEGLIGIBLE`, `LOW`, `MEDIUM`, `HIGH`, `UNKNOWN` + + + + Unique identifier for the response. + + + + + + + + Output only. Number of tokens in the cached part in the input (the cached content). + + + + Number of tokens in the response(s). + + + + Breakdown of candidate tokens by modality. + + + + Type of input or output content modality. + + Possible values: `MODALITY_UNSPECIFIED`, `TEXT`, `IMAGE`, `VIDEO`, `AUDIO`, `DOCUMENT` + + + + Number of tokens for the given modality. + + + + Number of tokens in the request. When cachedContent is set, this is still the total effective prompt size meaning this includes the number of tokens in the cached content. + + + + Breakdown of prompt tokens by modality. + + + + Type of input or output content modality. + + Possible values: `MODALITY_UNSPECIFIED`, `TEXT`, `IMAGE`, `VIDEO`, `AUDIO`, `DOCUMENT` + + + + Number of tokens for the given modality. + + + + Number of tokens present in thoughts output. + + + + Number of tokens present in tool-use prompt(s). + + + + Total number of tokens (prompt + candidates). + + + + Traffic type used for the request (e.g., PROVISIONED_THROUGHPUT). + + +## Examples + +### Input + +```json +{ + "contents": [ + { + "parts": [ + { + "text": "Describe a robot learning to paint, in two sentences." + } + ], + "role": "user" + } + ] +} +``` + +### Output + +```json +{ + "candidates": [ + { + "content": { + "parts": [ + { + "inlineData": { + "data": "PGJhc2U2ND4=", + "mimeType": "image/png" + } + } + ], + "role": "model" + }, + "finishReason": "STOP" + } + ], + "modelVersion": "gemini-2.5-flash-image", + "responseId": "7c6b5a49-3827-1605-f4e3-d2c1b0a99887", + "usageMetadata": { + "candidatesTokenCount": 1290, + "promptTokenCount": 11, + "totalTokenCount": 1301 + } +} +``` + +The image comes back inline as base64 in `inlineData.data`, with its `mimeType` beside it. Decode it and write it to a file; there is no URL to download. + + diff --git a/development/comfy-router/models/google/nano-banana-2-lite/code.yaml b/development/comfy-router/models/google/nano-banana-2-lite/code.yaml new file mode 100644 index 000000000..0013eb811 --- /dev/null +++ b/development/comfy-router/models/google/nano-banana-2-lite/code.yaml @@ -0,0 +1,203 @@ +# Source of truth for code.mdx in this directory. Edit this file, then run +# `pnpm code-pages:gen`; never edit code.mdx by hand (CI checks it is fresh). +# `input` / `output` are the provider's documented shapes (JSON Schema in YAML) and +# are the fallback until Router publishes the model's schema in router-schemas/. +name: Nano Banana 2 Lite +provider: Google +description: >- + Python, TypeScript and cURL snippets for generating images with Nano Banana 2 Lite (Gemini 3.1 Flash-Lite + Image) over HTTP through Comfy Router, plus the request fields and the result shape +summary: >- + Nano Banana 2 Lite (Gemini 3.1 Flash-Lite Image) is the Flash-Lite tier of Google's Nano Banana image generation family, tuned for lower latency and cost. +task: generation +variants: +- title: Nano Banana 2 Lite + model: vertexai/gemini-3.1-flash-lite-image +example: + contents: + - role: user + parts: + - text: a single red maple leaf on a plain white background, studio lighting + generationConfig: + responseModalities: + - IMAGE + imageConfig: + aspectRatio: '1:1' +input: + type: object + required: + - contents + properties: + contents: + type: array + description: >- + The conversation so far. For a single image, one `user` turn with a text part; add an `inlineData` + image part to edit an existing image. + items: + type: object + required: + - role + - parts + properties: + role: + type: string + description: Who authored the turn. + enum: + - user + - model + parts: + type: array + description: The turn's content parts. + items: + type: object + properties: + text: + type: string + description: A text part. + inlineData: + type: object + description: An inline media part. + properties: + mimeType: + type: string + description: Media type, for example `image/png`. + data: + type: string + description: Base64-encoded media bytes. + generationConfig: + type: object + description: Generation settings. + properties: + responseModalities: + type: array + description: Ask for an image with `["IMAGE"]`, or `["TEXT", "IMAGE"]` to also get a caption. + items: + type: string + enum: + - TEXT + - IMAGE + imageConfig: + type: object + description: Image output settings. + properties: + aspectRatio: + type: string + description: Output aspect ratio, for example `1:1`, `16:9`, `9:16`. + seed: + type: integer + description: Seed for reproducible results. + safetySettings: + type: array + description: Per-category harm thresholds. + items: + type: object + properties: + category: + type: string + enum: + - HARM_CATEGORY_SEXUALLY_EXPLICIT + - HARM_CATEGORY_HATE_SPEECH + - HARM_CATEGORY_HARASSMENT + - HARM_CATEGORY_DANGEROUS_CONTENT + threshold: + type: string + enum: + - BLOCK_NONE + - BLOCK_LOW_AND_ABOVE + - BLOCK_MEDIUM_AND_ABOVE + - BLOCK_ONLY_HIGH +output: + type: object + properties: + candidates: + type: array + description: Generated candidates; one unless you asked for more. + items: + type: object + properties: + content: + type: object + properties: + role: + type: string + enum: + - model + parts: + type: array + items: + type: object + properties: + inlineData: + type: object + description: The generated image, inline. + properties: + mimeType: + type: string + description: Media type of the image, typically `image/png`. + data: + type: string + description: Base64-encoded image bytes. Decode and write to a file; there is + no URL. + text: + type: string + description: Present when `TEXT` was among the requested modalities. + finishReason: + type: string + description: Why generation stopped, for example `STOP`. + usageMetadata: + type: object + description: Token accounting for the call. + properties: + promptTokenCount: + type: integer + description: Tokens in the prompt. + candidatesTokenCount: + type: integer + description: Tokens in the generated candidates. + promptFeedback: + type: object + description: >- + Present when the prompt itself was blocked. It is the only field returned in that case, so + check for it before reading the result. + properties: + blockReason: + type: string + description: Why the prompt was blocked. No candidates are returned; rephrase the prompt and retry. + enum: + - SAFETY + - OTHER + - BLOCKLIST + - PROHIBITED_CONTENT + - IMAGE_SAFETY +result: + path: candidates[0].content.parts[0].inlineData.data + label: image (base64) + example: + candidates: + - content: + role: model + parts: + - inlineData: + mimeType: image/png + data: iVBORw0KGgoAAAANSUhEUgAA... + finishReason: STOP + usageMetadata: + promptTokenCount: 12 + candidatesTokenCount: 1290 + note: >- + The image comes back inline as base64 in `inlineData.data`, with its `mimeType` beside it. Decode + it and write it to a file; there is no URL to download. +provider_spec: + url: https://generativelanguage.googleapis.com/$discovery/rest?version=v1beta + request: GenerateContentRequest + response: GenerateContentResponse + omit: + - promptFeedback.safetyRatings + - model + - store + - serviceTier + - cachedContent + - toolConfig + - modelVersion + - modelStatus + - responseId diff --git a/development/comfy-router/models/google/nano-banana-2/code.mdx b/development/comfy-router/models/google/nano-banana-2/code.mdx new file mode 100644 index 000000000..9a2e6d4f4 --- /dev/null +++ b/development/comfy-router/models/google/nano-banana-2/code.mdx @@ -0,0 +1,586 @@ +--- +title: "Use Nano Banana 2 with Comfy Router" +description: "Python, TypeScript and cURL snippets for generating images with Nano Banana 2 (Gemini 3.1 Flash Image) over HTTP through Comfy Router, plus the request fields and the result shape" +sidebarTitle: "Nano Banana 2" +--- + +{/* GENERATED FILE. Edit code.yaml in this directory and run `pnpm code-pages:gen`. */} + +import RouterPreviewNotice from "/snippets/comfy-router/preview-notice.mdx"; +import RouterCodeFooter from "/snippets/comfy-router/model-code-footer.mdx"; + +API Reference for Nano Banana 2. Nano Banana 2 (Gemini 3.1 Flash Image) is Google's image generation and editing model, balancing quality and speed. + + + +## Quick start + +Create a key at [platform.comfy.org/profile/api-keys](https://platform.comfy.org/profile/api-keys) and export it as `COMFY_API_KEY`. The Python and TypeScript snippets use the Comfy SDKs (`pip install comfy-sdk`, `npm install @comfyorg/sdk`); the cURL snippet is the same call over raw HTTP. + +**Model ID:** `vertexai/gemini-3.1-flash-image` + +**Endpoint:** `POST https://api.comfy.org/v2/models/vertexai/gemini-3.1-flash-image` + + +```python Python +from comfy_sdk import Comfy + +# Reads COMFY_API_KEY from the environment. Each call sends a fresh +# Idempotency-Key and waits up to 10 minutes for the finished result. +with Comfy() as client: + result = client.models.run( + "vertexai/gemini-3.1-flash-image", + { + "contents": [ + { + "role": "user", + "parts": [ + { + "text": "a single red maple leaf on a plain white background, studio lighting", + }, + ], + }, + ], + "generationConfig": { + "responseModalities": ["IMAGE"], + "imageConfig": { + "aspectRatio": "1:1", + }, + }, + }, + ) + +print("image (base64):", result["candidates"][0]["content"]["parts"][0]["inlineData"]["data"]) +``` + +```typescript TypeScript +import { comfy } from "@comfyorg/sdk"; + +// Reads COMFY_API_KEY from the environment. Each call sends a fresh +// Idempotency-Key and waits up to 10 minutes for the finished result. +type Result = { candidates: { content: { parts: { inlineData: { data: string } }[] } }[] }; +const { data } = await comfy.models.run("vertexai/gemini-3.1-flash-image", { + contents: [ + { + role: "user", + parts: [ + { + text: "a single red maple leaf on a plain white background, studio lighting", + }, + ], + }, + ], + generationConfig: { + responseModalities: ["IMAGE"], + imageConfig: { + aspectRatio: "1:1", + }, + }, +}); + +console.log("image (base64):", data.candidates[0].content.parts[0].inlineData.data); +``` + +```bash cURL +curl https://api.comfy.org/v2/models/vertexai/gemini-3.1-flash-image \ + -H "X-API-Key: $COMFY_API_KEY" \ + -H "Idempotency-Key: $(uuidgen)" \ + -H "Content-Type: application/json" \ + -d "{\"contents\": [{\"role\":\"user\",\"parts\":[{\"text\":\"a single red maple leaf on a plain white background, studio lighting\"}]}], \"generationConfig\": {\"responseModalities\":[\"IMAGE\"],\"imageConfig\":{\"aspectRatio\":\"1:1\"}}}" +``` + + +## Schema + +### Input + + + The content of the current conversation with the model. For single-turn queries, this is a single instance. For multi-turn queries, this is a repeated field that contains conversation history and the latest request. + + + + + + + + URI based data. + + + + URI + + + + The media type of the file specified in the data or fileUri fields. Acceptable values include the following. For gemini-2.0-flash-lite and gemini-2.0-flash, the maximum length of an audio file is 8.4 hours and the maximum length of a video file (without audio) is one hour. For more information, see Gemini audio and video requirements. Text files must be UTF-8 encoded. The contents of the text file count toward the token limit. There is no limit on image resolution. + + Possible values: `application/pdf`, `audio/mpeg`, `audio/mp3`, `audio/wav`, `image/png`, `image/jpeg`, `image/webp`, `text/plain`, `video/mov`, `video/mpeg`, `video/mp4`, `video/mpg`, `video/avi`, `video/wmv`, `video/mpegps`, `video/flv` + + + + Inline data in raw bytes. For gemini-2.0-flash-lite and gemini-2.0-flash, you can specify up to 3000 images by using inlineData. + + + + The base64 encoding of the image, PDF, or video to include inline in the prompt. When including media inline, you must also specify the media type (mimeType) of the data. Size limit: 20MB + + Format: `byte` + + + + The media type of the file specified in the data or fileUri fields. Acceptable values include the following. For gemini-2.0-flash-lite and gemini-2.0-flash, the maximum length of an audio file is 8.4 hours and the maximum length of a video file (without audio) is one hour. For more information, see Gemini audio and video requirements. Text files must be UTF-8 encoded. The contents of the text file count toward the token limit. There is no limit on image resolution. + + Possible values: `application/pdf`, `audio/mpeg`, `audio/mp3`, `audio/wav`, `image/png`, `image/jpeg`, `image/webp`, `text/plain`, `video/mov`, `video/mpeg`, `video/mp4`, `video/mpg`, `video/avi`, `video/wmv`, `video/mpegps`, `video/flv` + + + + A text prompt or code snippet. + + + + Indicates this part is a thinking/reasoning step from the model. + + + + Possible values: `user`, `model` + + + + Sampling, length and output settings for the generation. Every field is optional: the fields below that declare a `default` apply it when omitted, and the rest fall back to the model's own behaviour. + + + + Configuration for image generation + + + + Aspect ratio for generated images + + + + Optional. The image output format for generated images. + + + + Optional. The compression quality of the output image. + + + + Optional. The image format that the output should be saved as. + + + + Optional. Specifies the size of generated images. Supported values are 1K, 2K, 4K. If not specified, the model will use default value 1K. + + + + Maximum number of tokens that can be generated in the response. A token is approximately 4 characters. 100 tokens correspond to roughly 60-80 words. + + Range: `16` to `8192` + + + + + + + + When seed is fixed to a specific value, the model makes a best effort to provide the same response for repeated requests. Deterministic output isn't guaranteed. Also, changing the model or parameter settings, such as the temperature, can cause variations in the response even when you use the same seed value. By default, a random seed value is used. Available for the following models:, gemini-2.5-flash, gemini-2.5-pro, gemini-2.5-flash-preview-04-1, gemini-2.5-pro-preview-05-0, gemini-2.0-flash-lite-00, gemini-2.0-flash-001 + + + + + + + + The temperature is used for sampling during response generation, which occurs when topP and topK are applied. Temperature controls the degree of randomness in token selection. Lower temperatures are good for prompts that require a less open-ended or creative response, while higher temperatures can lead to more diverse or creative results. A temperature of 0 means that the highest probability tokens are always selected. In this case, responses for a given prompt are mostly deterministic, but a small amount of variation is still possible. If the model returns a response that's too generic, too short, or the model gives a fallback response, try increasing the temperature + + Range: `0` to `2` + + Format: `float` + + + + Optional. Configuration for thinking features. Thinking is a process where the model breaks down a complex task into smaller steps to generate a higher-quality response. + + + + Optional. If true, the model will include its thoughts in the response. + + + + Optional. The token budget for the model's thinking process. The model will make a best effort to stay within this budget. + + + + Optional. The thinking level for the model. + + Possible values: `THINKING_LEVEL_UNSPECIFIED`, `LOW`, `MEDIUM`, `HIGH`, `MINIMAL` + + + + Top-K changes how the model selects tokens for output. A top-K of 1 means the next selected token is the most probable among all tokens in the model's vocabulary. A top-K of 3 means that the next token is selected from among the 3 most probable tokens by using temperature. + + Range: `1` to `…` + + + + If specified, nucleus sampling is used. +Top-P changes how the model selects tokens for output. Tokens are selected from the most (see top-K) to least probable until the sum of their probabilities equals the top-P value. For example, if tokens A, B, and C have a probability of 0.3, 0.2, and 0.1 and the top-P value is 0.5, then the model will select either A or B as the next token by using temperature and excludes C as a candidate. +Specify a lower value for less random responses and a higher value for more random responses. + + Range: `0` to `1` + + Format: `float` + + + + Per request settings for blocking unsafe content. Enforced on GenerateContentResponse.candidates. + + + + Possible values: `HARM_CATEGORY_SEXUALLY_EXPLICIT`, `HARM_CATEGORY_HATE_SPEECH`, `HARM_CATEGORY_HARASSMENT`, `HARM_CATEGORY_DANGEROUS_CONTENT` + + + + Possible values: `OFF`, `BLOCK_NONE`, `BLOCK_LOW_AND_ABOVE`, `BLOCK_MEDIUM_AND_ABOVE`, `BLOCK_ONLY_HIGH` + + + + Instructions for the model to steer it toward better performance. For example, "Answer as concisely as possible" or "Don't use technical terms in your response". The text strings count toward the token limit. The role field of systemInstruction is ignored and doesn't affect the performance of the model. Note: Only text should be used in parts and content in each part should be in a separate paragraph. + + + + A list of ordered parts that make up a single message. Different parts may have different IANA MIME types. For limits on the inputs, such as the maximum number of tokens or the number of images, see the model specifications on the Google models page. + + + + A text prompt or code snippet. + + + + The identity of the entity that creates the message. The following values are supported: user: This indicates that the message is sent by a real person, typically a user-generated message. model: This indicates that the message is generated by the model. The model value is used to insert messages from the model into the conversation during multi-turn conversations. For non-multi-turn conversations, this field can be left blank or unset. + + Possible values: `user`, `model` + + + + A piece of code that enables the system to interact with external systems to perform an action, or set of actions, outside of knowledge and scope of the model. See Function calling. + + + + + + + + + + + + + + + + JSON schema for the function parameters + + + + If true, generated images will be uploaded to cloud storage and returned as signed URLs instead of inline base64 data. The URLs expire after 24 hours. + + + + For video input, the start and end offset of the video in Duration format. For example, to specify a 10 second clip starting at 1:00, set "startOffset": \{ "seconds": 60 } and "endOffset": \{ "seconds": 70 }. The metadata should only be specified while the video data is presented in inlineData or fileData. + + + + Represents a duration offset for video timeline positions. + + + + Signed fractions of a second at nanosecond resolution. Negative second values with fractions must still have non-negative nanos values. + + Range: `0` to `999999999` + + + + Signed seconds of the span of time. Must be from -315,576,000,000 to +315,576,000,000 inclusive. + + Range: `-315576000000` to `315576000000` + + + + Represents a duration offset for video timeline positions. + + + + Signed fractions of a second at nanosecond resolution. Negative second values with fractions must still have non-negative nanos values. + + Range: `0` to `999999999` + + + + Signed seconds of the span of time. Must be from -315,576,000,000 to +315,576,000,000 inclusive. + + Range: `-315576000000` to `315576000000` + + +Generated from the schema Router serves at `GET /v2/models/vertexai/gemini-3.1-flash-image/openapi.json`, the same document it validates a call against before the request reaches the provider. + +### Output + + + + + + + + + + + + + + + + + + + + + + + + + + + Format: `date` + + + + + + + + + + + + + + + + The content of the current conversation with the model. For single-turn queries, this is a single instance. For multi-turn queries, this is a repeated field that contains conversation history and the latest request. + + + + + + + + URI based data. + + + + URI + + + + The media type of the file specified in the data or fileUri fields. Acceptable values include the following. For gemini-2.0-flash-lite and gemini-2.0-flash, the maximum length of an audio file is 8.4 hours and the maximum length of a video file (without audio) is one hour. For more information, see Gemini audio and video requirements. Text files must be UTF-8 encoded. The contents of the text file count toward the token limit. There is no limit on image resolution. + + Possible values: `application/pdf`, `audio/mpeg`, `audio/mp3`, `audio/wav`, `image/png`, `image/jpeg`, `image/webp`, `text/plain`, `video/mov`, `video/mpeg`, `video/mp4`, `video/mpg`, `video/avi`, `video/wmv`, `video/mpegps`, `video/flv` + + + + Inline data in raw bytes. For gemini-2.0-flash-lite and gemini-2.0-flash, you can specify up to 3000 images by using inlineData. + + + + The base64 encoding of the image, PDF, or video to include inline in the prompt. When including media inline, you must also specify the media type (mimeType) of the data. Size limit: 20MB + + Format: `byte` + + + + The media type of the file specified in the data or fileUri fields. Acceptable values include the following. For gemini-2.0-flash-lite and gemini-2.0-flash, the maximum length of an audio file is 8.4 hours and the maximum length of a video file (without audio) is one hour. For more information, see Gemini audio and video requirements. Text files must be UTF-8 encoded. The contents of the text file count toward the token limit. There is no limit on image resolution. + + Possible values: `application/pdf`, `audio/mpeg`, `audio/mp3`, `audio/wav`, `image/png`, `image/jpeg`, `image/webp`, `text/plain`, `video/mov`, `video/mpeg`, `video/mp4`, `video/mpg`, `video/avi`, `video/wmv`, `video/mpegps`, `video/flv` + + + + A text prompt or code snippet. + + + + Indicates this part is a thinking/reasoning step from the model. + + + + Possible values: `user`, `model` + + + + + + + + + + + + Possible values: `HARM_CATEGORY_SEXUALLY_EXPLICIT`, `HARM_CATEGORY_HATE_SPEECH`, `HARM_CATEGORY_HARASSMENT`, `HARM_CATEGORY_DANGEROUS_CONTENT` + + + + The probability that the content violates the specified safety category + + Possible values: `NEGLIGIBLE`, `LOW`, `MEDIUM`, `HIGH`, `UNKNOWN` + + + + Timestamp when the response was created. + + + + The model version used to generate the response. + + + + + + + + + + + + + + + + + + + + Possible values: `HARM_CATEGORY_SEXUALLY_EXPLICIT`, `HARM_CATEGORY_HATE_SPEECH`, `HARM_CATEGORY_HARASSMENT`, `HARM_CATEGORY_DANGEROUS_CONTENT` + + + + The probability that the content violates the specified safety category + + Possible values: `NEGLIGIBLE`, `LOW`, `MEDIUM`, `HIGH`, `UNKNOWN` + + + + Unique identifier for the response. + + + + + + + + Output only. Number of tokens in the cached part in the input (the cached content). + + + + Number of tokens in the response(s). + + + + Breakdown of candidate tokens by modality. + + + + Type of input or output content modality. + + Possible values: `MODALITY_UNSPECIFIED`, `TEXT`, `IMAGE`, `VIDEO`, `AUDIO`, `DOCUMENT` + + + + Number of tokens for the given modality. + + + + Number of tokens in the request. When cachedContent is set, this is still the total effective prompt size meaning this includes the number of tokens in the cached content. + + + + Breakdown of prompt tokens by modality. + + + + Type of input or output content modality. + + Possible values: `MODALITY_UNSPECIFIED`, `TEXT`, `IMAGE`, `VIDEO`, `AUDIO`, `DOCUMENT` + + + + Number of tokens for the given modality. + + + + Number of tokens present in thoughts output. + + + + Number of tokens present in tool-use prompt(s). + + + + Total number of tokens (prompt + candidates). + + + + Traffic type used for the request (e.g., PROVISIONED_THROUGHPUT). + + +## Examples + +### Input + +```json +{ + "contents": [ + { + "parts": [ + { + "text": "Describe a robot learning to paint, in two sentences." + } + ], + "role": "user" + } + ] +} +``` + +### Output + +```json +{ + "candidates": [ + { + "content": { + "parts": [ + { + "inlineData": { + "data": "PGJhc2U2ND4=", + "mimeType": "image/png" + } + } + ], + "role": "model" + }, + "finishReason": "STOP" + } + ], + "modelVersion": "gemini-2.5-flash-image", + "responseId": "7c6b5a49-3827-1605-f4e3-d2c1b0a99887", + "usageMetadata": { + "candidatesTokenCount": 1290, + "promptTokenCount": 11, + "totalTokenCount": 1301 + } +} +``` + +The image comes back inline as base64 in `inlineData.data`, with its `mimeType` beside it. Decode it and write it to a file; there is no URL to download. + + diff --git a/development/comfy-router/models/google/nano-banana-2/code.yaml b/development/comfy-router/models/google/nano-banana-2/code.yaml new file mode 100644 index 000000000..4e57279d5 --- /dev/null +++ b/development/comfy-router/models/google/nano-banana-2/code.yaml @@ -0,0 +1,203 @@ +# Source of truth for code.mdx in this directory. Edit this file, then run +# `pnpm code-pages:gen`; never edit code.mdx by hand (CI checks it is fresh). +# `input` / `output` are the provider's documented shapes (JSON Schema in YAML) and +# are the fallback until Router publishes the model's schema in router-schemas/. +name: Nano Banana 2 +provider: Google +description: >- + Python, TypeScript and cURL snippets for generating images with Nano Banana 2 (Gemini 3.1 Flash Image) + over HTTP through Comfy Router, plus the request fields and the result shape +summary: >- + Nano Banana 2 (Gemini 3.1 Flash Image) is Google's image generation and editing model, balancing quality and speed. +task: generation +variants: +- title: Nano Banana 2 + model: vertexai/gemini-3.1-flash-image +example: + contents: + - role: user + parts: + - text: a single red maple leaf on a plain white background, studio lighting + generationConfig: + responseModalities: + - IMAGE + imageConfig: + aspectRatio: '1:1' +input: + type: object + required: + - contents + properties: + contents: + type: array + description: >- + The conversation so far. For a single image, one `user` turn with a text part; add an `inlineData` + image part to edit an existing image. + items: + type: object + required: + - role + - parts + properties: + role: + type: string + description: Who authored the turn. + enum: + - user + - model + parts: + type: array + description: The turn's content parts. + items: + type: object + properties: + text: + type: string + description: A text part. + inlineData: + type: object + description: An inline media part. + properties: + mimeType: + type: string + description: Media type, for example `image/png`. + data: + type: string + description: Base64-encoded media bytes. + generationConfig: + type: object + description: Generation settings. + properties: + responseModalities: + type: array + description: Ask for an image with `["IMAGE"]`, or `["TEXT", "IMAGE"]` to also get a caption. + items: + type: string + enum: + - TEXT + - IMAGE + imageConfig: + type: object + description: Image output settings. + properties: + aspectRatio: + type: string + description: Output aspect ratio, for example `1:1`, `16:9`, `9:16`. + seed: + type: integer + description: Seed for reproducible results. + safetySettings: + type: array + description: Per-category harm thresholds. + items: + type: object + properties: + category: + type: string + enum: + - HARM_CATEGORY_SEXUALLY_EXPLICIT + - HARM_CATEGORY_HATE_SPEECH + - HARM_CATEGORY_HARASSMENT + - HARM_CATEGORY_DANGEROUS_CONTENT + threshold: + type: string + enum: + - BLOCK_NONE + - BLOCK_LOW_AND_ABOVE + - BLOCK_MEDIUM_AND_ABOVE + - BLOCK_ONLY_HIGH +output: + type: object + properties: + candidates: + type: array + description: Generated candidates; one unless you asked for more. + items: + type: object + properties: + content: + type: object + properties: + role: + type: string + enum: + - model + parts: + type: array + items: + type: object + properties: + inlineData: + type: object + description: The generated image, inline. + properties: + mimeType: + type: string + description: Media type of the image, typically `image/png`. + data: + type: string + description: Base64-encoded image bytes. Decode and write to a file; there is + no URL. + text: + type: string + description: Present when `TEXT` was among the requested modalities. + finishReason: + type: string + description: Why generation stopped, for example `STOP`. + usageMetadata: + type: object + description: Token accounting for the call. + properties: + promptTokenCount: + type: integer + description: Tokens in the prompt. + candidatesTokenCount: + type: integer + description: Tokens in the generated candidates. + promptFeedback: + type: object + description: >- + Present when the prompt itself was blocked. It is the only field returned in that case, so + check for it before reading the result. + properties: + blockReason: + type: string + description: Why the prompt was blocked. No candidates are returned; rephrase the prompt and retry. + enum: + - SAFETY + - OTHER + - BLOCKLIST + - PROHIBITED_CONTENT + - IMAGE_SAFETY +result: + path: candidates[0].content.parts[0].inlineData.data + label: image (base64) + example: + candidates: + - content: + role: model + parts: + - inlineData: + mimeType: image/png + data: iVBORw0KGgoAAAANSUhEUgAA... + finishReason: STOP + usageMetadata: + promptTokenCount: 12 + candidatesTokenCount: 1290 + note: >- + The image comes back inline as base64 in `inlineData.data`, with its `mimeType` beside it. Decode + it and write it to a file; there is no URL to download. +provider_spec: + url: https://generativelanguage.googleapis.com/$discovery/rest?version=v1beta + request: GenerateContentRequest + response: GenerateContentResponse + omit: + - promptFeedback.safetyRatings + - model + - store + - serviceTier + - cachedContent + - toolConfig + - modelVersion + - modelStatus + - responseId diff --git a/development/comfy-router/models/google/nano-banana-pro/code.mdx b/development/comfy-router/models/google/nano-banana-pro/code.mdx new file mode 100644 index 000000000..89e11b2f2 --- /dev/null +++ b/development/comfy-router/models/google/nano-banana-pro/code.mdx @@ -0,0 +1,586 @@ +--- +title: "Use Nano Banana Pro with Comfy Router" +description: "Python, TypeScript and cURL snippets for generating images with Nano Banana Pro (Gemini 3 Pro Image) over HTTP through Comfy Router, plus the request fields and the result shape" +sidebarTitle: "Nano Banana Pro" +--- + +{/* GENERATED FILE. Edit code.yaml in this directory and run `pnpm code-pages:gen`. */} + +import RouterPreviewNotice from "/snippets/comfy-router/preview-notice.mdx"; +import RouterCodeFooter from "/snippets/comfy-router/model-code-footer.mdx"; + +API Reference for Nano Banana Pro. Nano Banana Pro (Gemini 3 Pro Image) is the Pro tier of Google's Nano Banana image generation family, aimed at complex scenes and legible text. + + + +## Quick start + +Create a key at [platform.comfy.org/profile/api-keys](https://platform.comfy.org/profile/api-keys) and export it as `COMFY_API_KEY`. The Python and TypeScript snippets use the Comfy SDKs (`pip install comfy-sdk`, `npm install @comfyorg/sdk`); the cURL snippet is the same call over raw HTTP. + +**Model ID:** `vertexai/gemini-3-pro-image` + +**Endpoint:** `POST https://api.comfy.org/v2/models/vertexai/gemini-3-pro-image` + + +```python Python +from comfy_sdk import Comfy + +# Reads COMFY_API_KEY from the environment. Each call sends a fresh +# Idempotency-Key and waits up to 10 minutes for the finished result. +with Comfy() as client: + result = client.models.run( + "vertexai/gemini-3-pro-image", + { + "contents": [ + { + "role": "user", + "parts": [ + { + "text": "a single red maple leaf on a plain white background, studio lighting", + }, + ], + }, + ], + "generationConfig": { + "responseModalities": ["IMAGE"], + "imageConfig": { + "aspectRatio": "1:1", + }, + }, + }, + ) + +print("image (base64):", result["candidates"][0]["content"]["parts"][0]["inlineData"]["data"]) +``` + +```typescript TypeScript +import { comfy } from "@comfyorg/sdk"; + +// Reads COMFY_API_KEY from the environment. Each call sends a fresh +// Idempotency-Key and waits up to 10 minutes for the finished result. +type Result = { candidates: { content: { parts: { inlineData: { data: string } }[] } }[] }; +const { data } = await comfy.models.run("vertexai/gemini-3-pro-image", { + contents: [ + { + role: "user", + parts: [ + { + text: "a single red maple leaf on a plain white background, studio lighting", + }, + ], + }, + ], + generationConfig: { + responseModalities: ["IMAGE"], + imageConfig: { + aspectRatio: "1:1", + }, + }, +}); + +console.log("image (base64):", data.candidates[0].content.parts[0].inlineData.data); +``` + +```bash cURL +curl https://api.comfy.org/v2/models/vertexai/gemini-3-pro-image \ + -H "X-API-Key: $COMFY_API_KEY" \ + -H "Idempotency-Key: $(uuidgen)" \ + -H "Content-Type: application/json" \ + -d "{\"contents\": [{\"role\":\"user\",\"parts\":[{\"text\":\"a single red maple leaf on a plain white background, studio lighting\"}]}], \"generationConfig\": {\"responseModalities\":[\"IMAGE\"],\"imageConfig\":{\"aspectRatio\":\"1:1\"}}}" +``` + + +## Schema + +### Input + + + The content of the current conversation with the model. For single-turn queries, this is a single instance. For multi-turn queries, this is a repeated field that contains conversation history and the latest request. + + + + + + + + URI based data. + + + + URI + + + + The media type of the file specified in the data or fileUri fields. Acceptable values include the following. For gemini-2.0-flash-lite and gemini-2.0-flash, the maximum length of an audio file is 8.4 hours and the maximum length of a video file (without audio) is one hour. For more information, see Gemini audio and video requirements. Text files must be UTF-8 encoded. The contents of the text file count toward the token limit. There is no limit on image resolution. + + Possible values: `application/pdf`, `audio/mpeg`, `audio/mp3`, `audio/wav`, `image/png`, `image/jpeg`, `image/webp`, `text/plain`, `video/mov`, `video/mpeg`, `video/mp4`, `video/mpg`, `video/avi`, `video/wmv`, `video/mpegps`, `video/flv` + + + + Inline data in raw bytes. For gemini-2.0-flash-lite and gemini-2.0-flash, you can specify up to 3000 images by using inlineData. + + + + The base64 encoding of the image, PDF, or video to include inline in the prompt. When including media inline, you must also specify the media type (mimeType) of the data. Size limit: 20MB + + Format: `byte` + + + + The media type of the file specified in the data or fileUri fields. Acceptable values include the following. For gemini-2.0-flash-lite and gemini-2.0-flash, the maximum length of an audio file is 8.4 hours and the maximum length of a video file (without audio) is one hour. For more information, see Gemini audio and video requirements. Text files must be UTF-8 encoded. The contents of the text file count toward the token limit. There is no limit on image resolution. + + Possible values: `application/pdf`, `audio/mpeg`, `audio/mp3`, `audio/wav`, `image/png`, `image/jpeg`, `image/webp`, `text/plain`, `video/mov`, `video/mpeg`, `video/mp4`, `video/mpg`, `video/avi`, `video/wmv`, `video/mpegps`, `video/flv` + + + + A text prompt or code snippet. + + + + Indicates this part is a thinking/reasoning step from the model. + + + + Possible values: `user`, `model` + + + + Sampling, length and output settings for the generation. Every field is optional: the fields below that declare a `default` apply it when omitted, and the rest fall back to the model's own behaviour. + + + + Configuration for image generation + + + + Aspect ratio for generated images + + + + Optional. The image output format for generated images. + + + + Optional. The compression quality of the output image. + + + + Optional. The image format that the output should be saved as. + + + + Optional. Specifies the size of generated images. Supported values are 1K, 2K, 4K. If not specified, the model will use default value 1K. + + + + Maximum number of tokens that can be generated in the response. A token is approximately 4 characters. 100 tokens correspond to roughly 60-80 words. + + Range: `16` to `8192` + + + + + + + + When seed is fixed to a specific value, the model makes a best effort to provide the same response for repeated requests. Deterministic output isn't guaranteed. Also, changing the model or parameter settings, such as the temperature, can cause variations in the response even when you use the same seed value. By default, a random seed value is used. Available for the following models:, gemini-2.5-flash, gemini-2.5-pro, gemini-2.5-flash-preview-04-1, gemini-2.5-pro-preview-05-0, gemini-2.0-flash-lite-00, gemini-2.0-flash-001 + + + + + + + + The temperature is used for sampling during response generation, which occurs when topP and topK are applied. Temperature controls the degree of randomness in token selection. Lower temperatures are good for prompts that require a less open-ended or creative response, while higher temperatures can lead to more diverse or creative results. A temperature of 0 means that the highest probability tokens are always selected. In this case, responses for a given prompt are mostly deterministic, but a small amount of variation is still possible. If the model returns a response that's too generic, too short, or the model gives a fallback response, try increasing the temperature + + Range: `0` to `2` + + Format: `float` + + + + Optional. Configuration for thinking features. Thinking is a process where the model breaks down a complex task into smaller steps to generate a higher-quality response. + + + + Optional. If true, the model will include its thoughts in the response. + + + + Optional. The token budget for the model's thinking process. The model will make a best effort to stay within this budget. + + + + Optional. The thinking level for the model. + + Possible values: `THINKING_LEVEL_UNSPECIFIED`, `LOW`, `MEDIUM`, `HIGH`, `MINIMAL` + + + + Top-K changes how the model selects tokens for output. A top-K of 1 means the next selected token is the most probable among all tokens in the model's vocabulary. A top-K of 3 means that the next token is selected from among the 3 most probable tokens by using temperature. + + Range: `1` to `…` + + + + If specified, nucleus sampling is used. +Top-P changes how the model selects tokens for output. Tokens are selected from the most (see top-K) to least probable until the sum of their probabilities equals the top-P value. For example, if tokens A, B, and C have a probability of 0.3, 0.2, and 0.1 and the top-P value is 0.5, then the model will select either A or B as the next token by using temperature and excludes C as a candidate. +Specify a lower value for less random responses and a higher value for more random responses. + + Range: `0` to `1` + + Format: `float` + + + + Per request settings for blocking unsafe content. Enforced on GenerateContentResponse.candidates. + + + + Possible values: `HARM_CATEGORY_SEXUALLY_EXPLICIT`, `HARM_CATEGORY_HATE_SPEECH`, `HARM_CATEGORY_HARASSMENT`, `HARM_CATEGORY_DANGEROUS_CONTENT` + + + + Possible values: `OFF`, `BLOCK_NONE`, `BLOCK_LOW_AND_ABOVE`, `BLOCK_MEDIUM_AND_ABOVE`, `BLOCK_ONLY_HIGH` + + + + Instructions for the model to steer it toward better performance. For example, "Answer as concisely as possible" or "Don't use technical terms in your response". The text strings count toward the token limit. The role field of systemInstruction is ignored and doesn't affect the performance of the model. Note: Only text should be used in parts and content in each part should be in a separate paragraph. + + + + A list of ordered parts that make up a single message. Different parts may have different IANA MIME types. For limits on the inputs, such as the maximum number of tokens or the number of images, see the model specifications on the Google models page. + + + + A text prompt or code snippet. + + + + The identity of the entity that creates the message. The following values are supported: user: This indicates that the message is sent by a real person, typically a user-generated message. model: This indicates that the message is generated by the model. The model value is used to insert messages from the model into the conversation during multi-turn conversations. For non-multi-turn conversations, this field can be left blank or unset. + + Possible values: `user`, `model` + + + + A piece of code that enables the system to interact with external systems to perform an action, or set of actions, outside of knowledge and scope of the model. See Function calling. + + + + + + + + + + + + + + + + JSON schema for the function parameters + + + + If true, generated images will be uploaded to cloud storage and returned as signed URLs instead of inline base64 data. The URLs expire after 24 hours. + + + + For video input, the start and end offset of the video in Duration format. For example, to specify a 10 second clip starting at 1:00, set "startOffset": \{ "seconds": 60 } and "endOffset": \{ "seconds": 70 }. The metadata should only be specified while the video data is presented in inlineData or fileData. + + + + Represents a duration offset for video timeline positions. + + + + Signed fractions of a second at nanosecond resolution. Negative second values with fractions must still have non-negative nanos values. + + Range: `0` to `999999999` + + + + Signed seconds of the span of time. Must be from -315,576,000,000 to +315,576,000,000 inclusive. + + Range: `-315576000000` to `315576000000` + + + + Represents a duration offset for video timeline positions. + + + + Signed fractions of a second at nanosecond resolution. Negative second values with fractions must still have non-negative nanos values. + + Range: `0` to `999999999` + + + + Signed seconds of the span of time. Must be from -315,576,000,000 to +315,576,000,000 inclusive. + + Range: `-315576000000` to `315576000000` + + +Generated from the schema Router serves at `GET /v2/models/vertexai/gemini-3-pro-image/openapi.json`, the same document it validates a call against before the request reaches the provider. + +### Output + + + + + + + + + + + + + + + + + + + + + + + + + + + Format: `date` + + + + + + + + + + + + + + + + The content of the current conversation with the model. For single-turn queries, this is a single instance. For multi-turn queries, this is a repeated field that contains conversation history and the latest request. + + + + + + + + URI based data. + + + + URI + + + + The media type of the file specified in the data or fileUri fields. Acceptable values include the following. For gemini-2.0-flash-lite and gemini-2.0-flash, the maximum length of an audio file is 8.4 hours and the maximum length of a video file (without audio) is one hour. For more information, see Gemini audio and video requirements. Text files must be UTF-8 encoded. The contents of the text file count toward the token limit. There is no limit on image resolution. + + Possible values: `application/pdf`, `audio/mpeg`, `audio/mp3`, `audio/wav`, `image/png`, `image/jpeg`, `image/webp`, `text/plain`, `video/mov`, `video/mpeg`, `video/mp4`, `video/mpg`, `video/avi`, `video/wmv`, `video/mpegps`, `video/flv` + + + + Inline data in raw bytes. For gemini-2.0-flash-lite and gemini-2.0-flash, you can specify up to 3000 images by using inlineData. + + + + The base64 encoding of the image, PDF, or video to include inline in the prompt. When including media inline, you must also specify the media type (mimeType) of the data. Size limit: 20MB + + Format: `byte` + + + + The media type of the file specified in the data or fileUri fields. Acceptable values include the following. For gemini-2.0-flash-lite and gemini-2.0-flash, the maximum length of an audio file is 8.4 hours and the maximum length of a video file (without audio) is one hour. For more information, see Gemini audio and video requirements. Text files must be UTF-8 encoded. The contents of the text file count toward the token limit. There is no limit on image resolution. + + Possible values: `application/pdf`, `audio/mpeg`, `audio/mp3`, `audio/wav`, `image/png`, `image/jpeg`, `image/webp`, `text/plain`, `video/mov`, `video/mpeg`, `video/mp4`, `video/mpg`, `video/avi`, `video/wmv`, `video/mpegps`, `video/flv` + + + + A text prompt or code snippet. + + + + Indicates this part is a thinking/reasoning step from the model. + + + + Possible values: `user`, `model` + + + + + + + + + + + + Possible values: `HARM_CATEGORY_SEXUALLY_EXPLICIT`, `HARM_CATEGORY_HATE_SPEECH`, `HARM_CATEGORY_HARASSMENT`, `HARM_CATEGORY_DANGEROUS_CONTENT` + + + + The probability that the content violates the specified safety category + + Possible values: `NEGLIGIBLE`, `LOW`, `MEDIUM`, `HIGH`, `UNKNOWN` + + + + Timestamp when the response was created. + + + + The model version used to generate the response. + + + + + + + + + + + + + + + + + + + + Possible values: `HARM_CATEGORY_SEXUALLY_EXPLICIT`, `HARM_CATEGORY_HATE_SPEECH`, `HARM_CATEGORY_HARASSMENT`, `HARM_CATEGORY_DANGEROUS_CONTENT` + + + + The probability that the content violates the specified safety category + + Possible values: `NEGLIGIBLE`, `LOW`, `MEDIUM`, `HIGH`, `UNKNOWN` + + + + Unique identifier for the response. + + + + + + + + Output only. Number of tokens in the cached part in the input (the cached content). + + + + Number of tokens in the response(s). + + + + Breakdown of candidate tokens by modality. + + + + Type of input or output content modality. + + Possible values: `MODALITY_UNSPECIFIED`, `TEXT`, `IMAGE`, `VIDEO`, `AUDIO`, `DOCUMENT` + + + + Number of tokens for the given modality. + + + + Number of tokens in the request. When cachedContent is set, this is still the total effective prompt size meaning this includes the number of tokens in the cached content. + + + + Breakdown of prompt tokens by modality. + + + + Type of input or output content modality. + + Possible values: `MODALITY_UNSPECIFIED`, `TEXT`, `IMAGE`, `VIDEO`, `AUDIO`, `DOCUMENT` + + + + Number of tokens for the given modality. + + + + Number of tokens present in thoughts output. + + + + Number of tokens present in tool-use prompt(s). + + + + Total number of tokens (prompt + candidates). + + + + Traffic type used for the request (e.g., PROVISIONED_THROUGHPUT). + + +## Examples + +### Input + +```json +{ + "contents": [ + { + "parts": [ + { + "text": "Describe a robot learning to paint, in two sentences." + } + ], + "role": "user" + } + ] +} +``` + +### Output + +```json +{ + "candidates": [ + { + "content": { + "parts": [ + { + "inlineData": { + "data": "PGJhc2U2ND4=", + "mimeType": "image/png" + } + } + ], + "role": "model" + }, + "finishReason": "STOP" + } + ], + "modelVersion": "gemini-2.5-flash-image", + "responseId": "7c6b5a49-3827-1605-f4e3-d2c1b0a99887", + "usageMetadata": { + "candidatesTokenCount": 1290, + "promptTokenCount": 11, + "totalTokenCount": 1301 + } +} +``` + +The image comes back inline as base64 in `inlineData.data`, with its `mimeType` beside it. Decode it and write it to a file; there is no URL to download. + + diff --git a/development/comfy-router/models/google/nano-banana-pro/code.yaml b/development/comfy-router/models/google/nano-banana-pro/code.yaml new file mode 100644 index 000000000..678592963 --- /dev/null +++ b/development/comfy-router/models/google/nano-banana-pro/code.yaml @@ -0,0 +1,210 @@ +# Source of truth for code.mdx in this directory. Edit this file, then run +# `pnpm code-pages:gen`; never edit code.mdx by hand (CI checks it is fresh). +# `input` / `output` are the provider's documented shapes (JSON Schema in YAML) and +# are the fallback until Router publishes the model's schema in router-schemas/. +name: Nano Banana Pro +provider: Google +description: >- + Python, TypeScript and cURL snippets for generating images with Nano Banana Pro (Gemini 3 Pro Image) + over HTTP through Comfy Router, plus the request fields and the result shape +summary: >- + Nano Banana Pro (Gemini 3 Pro Image) is the Pro tier of Google's Nano Banana image generation family, aimed at complex scenes and legible text. +task: generation +variants: +- title: Nano Banana Pro + model: vertexai/gemini-3-pro-image +example: + contents: + - role: user + parts: + - text: a single red maple leaf on a plain white background, studio lighting + generationConfig: + responseModalities: + - IMAGE + imageConfig: + aspectRatio: '1:1' +input: + type: object + required: + - contents + properties: + contents: + type: array + description: >- + The conversation so far. For a single image, one `user` turn with a text part; add an `inlineData` + image part to edit an existing image. + items: + type: object + required: + - role + - parts + properties: + role: + type: string + description: Who authored the turn. + enum: + - user + - model + parts: + type: array + description: The turn's content parts. + items: + type: object + properties: + text: + type: string + description: A text part. + inlineData: + type: object + description: An inline media part. + properties: + mimeType: + type: string + description: Media type, for example `image/png`. + data: + type: string + description: Base64-encoded media bytes. + generationConfig: + type: object + description: Generation settings. + properties: + responseModalities: + type: array + description: Ask for an image with `["IMAGE"]`, or `["TEXT", "IMAGE"]` to also get a caption. + items: + type: string + enum: + - TEXT + - IMAGE + imageConfig: + type: object + description: Image output settings. + properties: + aspectRatio: + type: string + description: Output aspect ratio, for example `1:1`, `16:9`, `9:16`. + imageSize: + type: string + description: Output resolution class. + enum: + - 1K + - 2K + - 4K + seed: + type: integer + description: Seed for reproducible results. + safetySettings: + type: array + description: Per-category harm thresholds. + items: + type: object + properties: + category: + type: string + enum: + - HARM_CATEGORY_SEXUALLY_EXPLICIT + - HARM_CATEGORY_HATE_SPEECH + - HARM_CATEGORY_HARASSMENT + - HARM_CATEGORY_DANGEROUS_CONTENT + threshold: + type: string + enum: + - BLOCK_NONE + - BLOCK_LOW_AND_ABOVE + - BLOCK_MEDIUM_AND_ABOVE + - BLOCK_ONLY_HIGH +output: + type: object + properties: + candidates: + type: array + description: Generated candidates; one unless you asked for more. + items: + type: object + properties: + content: + type: object + properties: + role: + type: string + enum: + - model + parts: + type: array + items: + type: object + properties: + inlineData: + type: object + description: The generated image, inline. + properties: + mimeType: + type: string + description: Media type of the image, typically `image/png`. + data: + type: string + description: Base64-encoded image bytes. Decode and write to a file; there is + no URL. + text: + type: string + description: Present when `TEXT` was among the requested modalities. + finishReason: + type: string + description: Why generation stopped, for example `STOP`. + usageMetadata: + type: object + description: Token accounting for the call. + properties: + promptTokenCount: + type: integer + description: Tokens in the prompt. + candidatesTokenCount: + type: integer + description: Tokens in the generated candidates. + promptFeedback: + type: object + description: >- + Present when the prompt itself was blocked. It is the only field returned in that case, so + check for it before reading the result. + properties: + blockReason: + type: string + description: Why the prompt was blocked. No candidates are returned; rephrase the prompt and retry. + enum: + - SAFETY + - OTHER + - BLOCKLIST + - PROHIBITED_CONTENT + - IMAGE_SAFETY +result: + path: candidates[0].content.parts[0].inlineData.data + label: image (base64) + example: + candidates: + - content: + role: model + parts: + - inlineData: + mimeType: image/png + data: iVBORw0KGgoAAAANSUhEUgAA... + finishReason: STOP + usageMetadata: + promptTokenCount: 12 + candidatesTokenCount: 1290 + note: >- + The image comes back inline as base64 in `inlineData.data`, with its `mimeType` beside it. Decode + it and write it to a file; there is no URL to download. +provider_spec: + url: https://generativelanguage.googleapis.com/$discovery/rest?version=v1beta + request: GenerateContentRequest + response: GenerateContentResponse + omit: + - promptFeedback.safetyRatings + - model + - store + - serviceTier + - cachedContent + - toolConfig + - modelVersion + - modelStatus + - responseId diff --git a/development/comfy-router/models/ideogram/ideogram-v4/code.mdx b/development/comfy-router/models/ideogram/ideogram-v4/code.mdx new file mode 100644 index 000000000..4b00fc8fe --- /dev/null +++ b/development/comfy-router/models/ideogram/ideogram-v4/code.mdx @@ -0,0 +1,162 @@ +--- +title: "Use Ideogram 4.0 with Comfy Router" +description: "Python, TypeScript and cURL snippets for generating images with Ideogram 4.0 over HTTP through Comfy Router, plus the request fields and the result shape" +sidebarTitle: "Ideogram 4.0" +--- + +{/* GENERATED FILE. Edit code.yaml in this directory and run `pnpm code-pages:gen`. */} + +import RouterPreviewNotice from "/snippets/comfy-router/preview-notice.mdx"; +import RouterCodeFooter from "/snippets/comfy-router/model-code-footer.mdx"; + +API Reference for Ideogram 4.0. Ideogram 4.0 is Ideogram's text-to-image model, which renders legible text inside generated images. + + + +## Quick start + +Create a key at [platform.comfy.org/profile/api-keys](https://platform.comfy.org/profile/api-keys) and export it as `COMFY_API_KEY`. The Python and TypeScript snippets use the Comfy SDKs (`pip install comfy-sdk`, `npm install @comfyorg/sdk`); the cURL snippet is the same call over raw HTTP. + +**Model ID:** `ideogram/ideogram-v4` + +**Endpoint:** `POST https://api.comfy.org/v2/models/ideogram/ideogram-v4` + + +```python Python +from comfy_sdk import Comfy + +# Reads COMFY_API_KEY from the environment. Each call sends a fresh +# Idempotency-Key and waits up to 10 minutes for the finished result. +with Comfy() as client: + result = client.models.run( + "ideogram/ideogram-v4", + { + "text_prompt": "a single red maple leaf on a plain white background, studio lighting", + "resolution": "1024x1024", + "rendering_speed": "DEFAULT", + }, + ) + +print("image:", result["data"][0]["url"]) +``` + +```typescript TypeScript +import { comfy } from "@comfyorg/sdk"; + +// Reads COMFY_API_KEY from the environment. Each call sends a fresh +// Idempotency-Key and waits up to 10 minutes for the finished result. +type Result = { data: { url: string }[] }; +const { data } = await comfy.models.run("ideogram/ideogram-v4", { + text_prompt: "a single red maple leaf on a plain white background, studio lighting", + resolution: "1024x1024", + rendering_speed: "DEFAULT", +}); + +console.log("image:", data.data[0].url); +``` + +```bash cURL +curl https://api.comfy.org/v2/models/ideogram/ideogram-v4 \ + -H "X-API-Key: $COMFY_API_KEY" \ + -H "Idempotency-Key: $(uuidgen)" \ + -H "Content-Type: application/json" \ + -d "{\"text_prompt\": \"a single red maple leaf on a plain white background, studio lighting\", \"resolution\": \"1024x1024\", \"rendering_speed\": \"DEFAULT\"}" +``` + + +## Schema + +### Input + + + Opt into post-generation copyright detection (Hive likeness and logo checks). + + + + Structured V4 prompt. Disables Magic Prompt; consumed directly. Supply exactly one of text_prompt or json_prompt. + + + + The rendering speed setting that controls the trade-off between generation speed and quality + + Possible values: `DEFAULT`, `TURBO`, `QUALITY` + + + + Output resolution in WIDTHxHEIGHT. Omit to let the model pick an aspect ratio. Supported 2K values: 2048x2048, 1440x2880, 2880x1440, 1664x2496, 2496x1664, 1792x2240, 2240x1792, 1440x2560, 2560x1440, 1600x2560, 2560x1600, 1728x2304, 2304x1728, 1296x3168, 3168x1296, 1152x2944, 2944x1152, 1248x3328, 3328x1248, 1280x3072, 3072x1280. + + + + Natural-language prompt. Enables Magic Prompt automatically. Supply exactly one of text_prompt or json_prompt. + + +Generated from the schema Router serves at `GET /v2/models/ideogram/ideogram-v4/openapi.json`, the same document it validates a call against before the request reaches the provider. + +### Output + + + Timestamp when the generation was created. + + Format: `date-time` + + + + Array of generated image information. + + + + Indicates whether the image is considered safe. + + + + The prompt used to generate this image. + + + + The resolution of the generated image (e.g., '1024x1024'). + + + + The seed value used for this generation. + + + + The style type used for generation (e.g., 'REALISTIC', 'ANIME'). + + + + URL to the generated image. + + +## Examples + +### Input + +```json +{ + "rendering_speed": "DEFAULT", + "text_prompt": "A poster for a jazz festival, bold typography, warm colours" +} +``` + +### Output + +```json +{ + "created": "2026-01-01T00:00:00Z", + "data": [ + { + "is_image_safe": true, + "prompt": "A poster for a jazz festival, bold typography, warm colours", + "resolution": "2048x2048", + "seed": 918273645, + "style_type": "REALISTIC", + "url": "https://example.invalid/ideogram/ideogram-v4/generated.png" + } + ] +} +``` + +The URL is temporary. Download the image promptly if you need to keep it. + + diff --git a/development/comfy-router/models/ideogram/ideogram-v4/code.yaml b/development/comfy-router/models/ideogram/ideogram-v4/code.yaml new file mode 100644 index 000000000..f98237bf7 --- /dev/null +++ b/development/comfy-router/models/ideogram/ideogram-v4/code.yaml @@ -0,0 +1,103 @@ +# Source of truth for code.mdx in this directory. Edit this file, then run +# `pnpm code-pages:gen`; never edit code.mdx by hand (CI checks it is fresh). +# `input` / `output` are the provider's documented shapes (JSON Schema in YAML) and +# are the fallback until Router publishes the model's schema in router-schemas/. +name: Ideogram 4.0 +provider: Ideogram +description: >- + Python, TypeScript and cURL snippets for generating images with Ideogram 4.0 over HTTP through Comfy + Router, plus the request fields and the result shape +summary: >- + Ideogram 4.0 is Ideogram's text-to-image model, which renders legible text inside generated images. +task: generation +variants: +- title: Ideogram 4.0 + model: ideogram/ideogram-v4 +example: + text_prompt: a single red maple leaf on a plain white background, studio lighting + resolution: 1024x1024 + rendering_speed: DEFAULT +input: + type: object + properties: + text_prompt: + type: string + description: Text description of the image. Provide this or `json_prompt`. + json_prompt: + type: object + description: Structured prompt, as an alternative to `text_prompt`. + resolution: + type: string + description: Output size as `WIDTHxHEIGHT`, for example `1024x1024` or `1536x1024`. + rendering_speed: + type: string + description: Speed / quality trade-off. + enum: + - TURBO + - BALANCED + - DEFAULT + - QUALITY + default: DEFAULT + enable_copyright_detection: + type: boolean + description: Reject outputs that match known copyrighted material. +output: + type: object + required: + - created + - data + properties: + response_type: + type: string + description: How the images are delivered. + enum: + - url + created: + type: string + description: Timestamp of the generation. + format: date-time + data: + type: array + description: Generated images. + items: + type: object + properties: + url: + type: string + description: Temporary URL of the image. Download it promptly. + format: uri + prompt: + type: string + description: The prompt that was run. + resolution: + type: string + description: Output size as `WIDTHxHEIGHT`. + seed: + type: integer + description: Seed used for this image. + is_image_safe: + type: boolean + description: Whether the image passed Ideogram's safety filter. + required: + - prompt + - resolution + - seed + - is_image_safe +result: + path: data[0].url + label: image + example: + response_type: url + created: '2026-08-27T21:00:00Z' + data: + - url: https://.../image.png + prompt: a single red maple leaf on a plain white background, studio lighting + resolution: 1024x1024 + seed: 1234567890 + is_image_safe: true + note: The URL is temporary. Download the image promptly if you need to keep it. +provider_spec: + url: https://api.ideogram.ai/openapi.json + operation: POST /v1/ideogram-v4/generate + omit: + - magic_prompt_system_prompt_config_id diff --git a/docs.json b/docs.json index 79f6017a8..7f1a3dd58 100644 --- a/docs.json +++ b/docs.json @@ -3032,8 +3032,23 @@ "group": "Comfy Router", "pages": [ "development/comfy-router/quickstart", + "development/comfy-router/headers", "development/comfy-router/reference", - "development/comfy-router/limitations" + "development/comfy-router/limitations", + { + "group": "Models", + "pages": [ + "development/comfy-router/models/black-forest-labs/flux-1-1-pro-ultra-image/code", + "development/comfy-router/models/black-forest-labs/flux-1-kontext/code", + "development/comfy-router/models/black-forest-labs/flux-3-video/code", + "development/comfy-router/models/black-forest-labs/flux-video-upscale/code", + "development/comfy-router/models/google/nano-banana-pro/code", + "development/comfy-router/models/google/nano-banana-2/code", + "development/comfy-router/models/google/nano-banana-2-lite/code", + "development/comfy-router/models/google/gemini/code", + "development/comfy-router/models/ideogram/ideogram-v4/code" + ] + } ] }, { @@ -13897,4 +13912,4 @@ "destination": "/tutorials/partner-nodes/wan/wan3-0" } ] -} \ No newline at end of file +} diff --git a/package.json b/package.json index 5e8b9ebf2..fc9dbc487 100644 --- a/package.json +++ b/package.json @@ -33,7 +33,10 @@ "analytics:fetch:all": "bun .github/scripts/analytics/fetch-assistant-insights.ts --all", "analytics:fetch:assistant": "bun .github/scripts/analytics/fetch-assistant-insights.ts --assistant-only", "analytics:fetch:resume": "bun .github/scripts/analytics/fetch-assistant-insights.ts --resume", - "analytics:fetch:dry-run": "bun .github/scripts/analytics/fetch-assistant-insights.ts --dry-run" + "analytics:fetch:dry-run": "bun .github/scripts/analytics/fetch-assistant-insights.ts --dry-run", + "code-pages:gen": "bun .github/scripts/snippets/gen-code-pages.ts", + "code-pages:check": "bun .github/scripts/snippets/gen-code-pages.ts --check --validate", + "code-pages:check-providers": "bun .github/scripts/snippets/check-provider-schemas.ts --verbose" }, "devDependencies": { "@executeautomation/playwright-mcp-server": "^1.0.5" diff --git a/snippets/comfy-router/model-code-footer.mdx b/snippets/comfy-router/model-code-footer.mdx new file mode 100644 index 000000000..ba3d64663 --- /dev/null +++ b/snippets/comfy-router/model-code-footer.mdx @@ -0,0 +1,15 @@ +## Before you ship + +The snippets above are the shortest working call. Three things are the same for every model and are documented once on the [Comfy Router headers](/development/comfy-router/headers) page: send an `Idempotency-Key` on every paid call and reuse it when you retry, expect the connection to be held up to Router's 10 minute deadline, and keep `X-Comfy-Request-Id` from every response. The SDKs do all three for you; the cURL tab does none of them. On failure, `X-Comfy-Error-Type` names the bucket, and a `422` means the body failed the model's schema and was never billed. + + + + Authentication, idempotency, request IDs, error buckets, retry pacing, spend limits. + + + Typed error handling in Python and TypeScript, reading the 422, walking the catalog. + + + What Router does not do today, and what to use instead. + + diff --git a/snippets/comfy-router/preview-notice.mdx b/snippets/comfy-router/preview-notice.mdx new file mode 100644 index 000000000..33ff3dda2 --- /dev/null +++ b/snippets/comfy-router/preview-notice.mdx @@ -0,0 +1,3 @@ + +**Comfy Router is not generally available yet.** `POST /v2/models/{provider}/{model}` and its catalog and schema siblings are not serving requests yet: an authenticated call answers `404` today. The snippets on this page document the contract those routes will serve, published ahead of the rollout so your integration is ready to write against. + diff --git a/tutorials/partner-nodes/black-forest-labs/flux-1-1-pro-ultra-image.mdx b/tutorials/partner-nodes/black-forest-labs/flux-1-1-pro-ultra-image.mdx index a084eb406..acd331203 100644 --- a/tutorials/partner-nodes/black-forest-labs/flux-1-1-pro-ultra-image.mdx +++ b/tutorials/partner-nodes/black-forest-labs/flux-1-1-pro-ultra-image.mdx @@ -6,7 +6,6 @@ sidebarTitle: "Flux 1 1 Pro Ultra Image" import ReqHint from "/snippets/tutorials/partner-nodes/req-hint.mdx"; import UpdateReminder from "/snippets/tutorials/update-reminder.mdx"; - FLUX 1.1 Pro Ultra is a high-performance AI image generation tool by BlackForestLabs, featuring ultra-high resolution and efficient generation capabilities. It supports up to 4MP resolution (4x the standard version) while keeping single image generation time under 10 seconds - 2.5x faster than similar high-resolution models. The tool offers two core modes: @@ -31,6 +30,14 @@ Image-to-image remixing, blending features from an input image into the output: ![Flux 1.1 pro Image-to-Image Remix](https://raw.githubusercontent.com/Comfy-Org/example_workflows/main/api_nodes/bfl/flux_1_1_pro_i2i.png) +## Use it + + + Call it over HTTP through Comfy Router, with copy-paste Python, TypeScript and cURL snippets + + +This runs on the same Comfy account and is billed in the same credits as running it in ComfyUI below. See [Partner Nodes pricing](/tutorials/partner-nodes/pricing) for per-model rates and [Concurrency limits](/tutorials/partner-nodes/concurrency-limits) for how many requests you can have in flight. + diff --git a/tutorials/partner-nodes/black-forest-labs/flux-1-kontext.mdx b/tutorials/partner-nodes/black-forest-labs/flux-1-kontext.mdx index f63627276..cb46c8dc4 100644 --- a/tutorials/partner-nodes/black-forest-labs/flux-1-kontext.mdx +++ b/tutorials/partner-nodes/black-forest-labs/flux-1-kontext.mdx @@ -7,7 +7,6 @@ import ReqHint from "/snippets/tutorials/partner-nodes/req-hint.mdx"; import PromptTechniques from "/snippets/tutorials/flux/prompt-techniques.mdx"; import UpdateReminder from "/snippets/tutorials/update-reminder.mdx"; - FLUX.1 Kontext is a professional image-to-image editing model developed by Black Forest Labs, focusing on intelligent understanding of image context and precise editing. It can perform various editing tasks without complex descriptions, including object modification, style transfer, background replacement, character consistency editing, and text editing. The core advantage of Kontext lies in its excellent context understanding ability and character consistency maintenance, ensuring that key elements such as character features and composition layout remain stable even after multiple iterations of editing. @@ -39,6 +38,14 @@ Image editing with Kontext Max, which pushes the limits on typography, prompt pr ![Flux.1 Kontext Max image editing example](https://raw.githubusercontent.com/Comfy-Org/example_workflows/main/api_nodes/bfl/flux_1_kontext_max_image.png) +## Use it + + + Call Kontext Pro or Max over HTTP through Comfy Router, with copy-paste Python, TypeScript and cURL snippets + + +This runs on the same Comfy account and is billed in the same credits as running it in ComfyUI below. See [Partner Nodes pricing](/tutorials/partner-nodes/pricing) for per-model rates and [Concurrency limits](/tutorials/partner-nodes/concurrency-limits) for how many requests you can have in flight. + diff --git a/tutorials/partner-nodes/black-forest-labs/flux-3-video.mdx b/tutorials/partner-nodes/black-forest-labs/flux-3-video.mdx index 5fc47f0bb..8c372ad25 100644 --- a/tutorials/partner-nodes/black-forest-labs/flux-3-video.mdx +++ b/tutorials/partner-nodes/black-forest-labs/flux-3-video.mdx @@ -6,7 +6,6 @@ sidebarTitle: "Flux 3 Video" import ReqHint from "/snippets/tutorials/partner-nodes/req-hint.mdx"; import UpdateReminder from "/snippets/tutorials/update-reminder.mdx"; - FLUX 3 is Black Forest Labs' multimodal foundation model, announced July 23, 2026 and currently in Early Access. It jointly learns from images, videos, and audio within a unified architecture built on Self-Flow, their approach for aligning multimodal generation and understanding in the same model. Instead of treating each modality in isolation, FLUX 3 learns a shared representation of the world: how objects hold together, how things move, and how events sound. Capabilities and limits may change during the Early Access rollout. For video, FLUX 3 creates highly diverse clips with native audio up to 20 seconds long in a single generation. All outputs come with synchronized audio generation, including ambient sound, speech, and effects. It supports text-to-video and image-to-video generation, with multi-shot output that chains individual clips into longer sequences and flexible aspect ratios. @@ -29,6 +28,14 @@ Image-to-video generation from a starting frame and a text prompt: +## Use it + + + Call it over HTTP through Comfy Router, with copy-paste Python, TypeScript and cURL snippets + + +This runs on the same Comfy account and is billed in the same credits as running it in ComfyUI below. See [Partner Nodes pricing](/tutorials/partner-nodes/pricing) for per-model rates and [Concurrency limits](/tutorials/partner-nodes/concurrency-limits) for how many requests you can have in flight. + diff --git a/tutorials/partner-nodes/black-forest-labs/flux-video-upscale.mdx b/tutorials/partner-nodes/black-forest-labs/flux-video-upscale.mdx index 5fe432657..14e37f106 100644 --- a/tutorials/partner-nodes/black-forest-labs/flux-video-upscale.mdx +++ b/tutorials/partner-nodes/black-forest-labs/flux-video-upscale.mdx @@ -6,7 +6,6 @@ sidebarTitle: "Flux Video Upscale" import ReqHint from "/snippets/tutorials/partner-nodes/req-hint.mdx"; import UpdateReminder from "/snippets/tutorials/update-reminder.mdx"; - FLUX Video Upscale is Black Forest Labs' FLUX 3 powered super-resolution tool for video, available in ComfyUI as a partner node. It upscales clips to 1080p, 2K, or 4K while keeping the source aspect ratio and its audio track. ## What FLUX Video Upscale is good at @@ -22,6 +21,14 @@ Upscaling an existing clip while keeping its aspect ratio and audio track: +## Use it + + + Call it over HTTP through Comfy Router, with copy-paste Python, TypeScript and cURL snippets + + +This runs on the same Comfy account and is billed in the same credits as running it in ComfyUI below. See [Partner Nodes pricing](/tutorials/partner-nodes/pricing) for per-model rates and [Concurrency limits](/tutorials/partner-nodes/concurrency-limits) for how many requests you can have in flight. + diff --git a/tutorials/partner-nodes/google/gemini.mdx b/tutorials/partner-nodes/google/gemini.mdx index b3b7fd335..74e4254cf 100644 --- a/tutorials/partner-nodes/google/gemini.mdx +++ b/tutorials/partner-nodes/google/gemini.mdx @@ -6,7 +6,6 @@ sidebarTitle: "Gemini" import ReqHint from "/snippets/tutorials/partner-nodes/req-hint.mdx"; import UpdateReminder from "/snippets/tutorials/update-reminder.mdx"; - Google Gemini is a powerful AI model developed by Google, supporting conversational and text generation functions. Currently, ComfyUI has integrated the Google Gemini API, allowing you to directly use the related nodes in ComfyUI to complete conversational functions. ## What Google Gemini is good at @@ -16,6 +15,14 @@ Google Gemini is a powerful AI model developed by Google, supporting conversatio - **Image-to-prompt interpretation**: The official template ships with a prompt that turns your images into corresponding drawing prompts - **Multi-image input**: Use `Batch Images` to send several images for AI interpretation in a single run +## Use it + + + Call it over HTTP through Comfy Router, with copy-paste Python, TypeScript and cURL snippets + + +This runs on the same Comfy account and is billed in the same credits as running it in ComfyUI below. See [Partner Nodes pricing](/tutorials/partner-nodes/pricing) for per-model rates and [Concurrency limits](/tutorials/partner-nodes/concurrency-limits) for how many requests you can have in flight. + diff --git a/tutorials/partner-nodes/google/nano-banana-2-lite.mdx b/tutorials/partner-nodes/google/nano-banana-2-lite.mdx index fa9ea4170..8e910af11 100644 --- a/tutorials/partner-nodes/google/nano-banana-2-lite.mdx +++ b/tutorials/partner-nodes/google/nano-banana-2-lite.mdx @@ -6,7 +6,6 @@ sidebarTitle: "Nano Banana 2 Lite" import ReqHint from "/snippets/tutorials/partner-nodes/req-hint.mdx"; import UpdateReminder from "/snippets/tutorials/update-reminder.mdx"; - Nano Banana 2 Lite is Google DeepMind's fastest and most cost-efficient Gemini Image model, designed for rapid ideation and high-volume workflows. Powered by `gemini-3.1-flash-lite-image`, it delivers text-to-image generation in approximately 4 seconds at $0.041 per image, making it ideal for quick concept visualization, rapid prototyping, and iterative design exploration. ## What Nano Banana 2 Lite is good at @@ -27,6 +26,14 @@ Image editing from a text instruction: ![Nano Banana 2 Lite Image Edit - Output](https://raw.githubusercontent.com/Comfy-Org/workflow_templates/main/output/api_nano_banana_2_lite_image_edit.png) +## Use it + + + Call it over HTTP through Comfy Router, with copy-paste Python, TypeScript and cURL snippets + + +This runs on the same Comfy account and is billed in the same credits as running it in ComfyUI below. See [Partner Nodes pricing](/tutorials/partner-nodes/pricing) for per-model rates and [Concurrency limits](/tutorials/partner-nodes/concurrency-limits) for how many requests you can have in flight. + diff --git a/tutorials/partner-nodes/google/nano-banana-2.mdx b/tutorials/partner-nodes/google/nano-banana-2.mdx index 493daf5ed..f8bbe3460 100644 --- a/tutorials/partner-nodes/google/nano-banana-2.mdx +++ b/tutorials/partner-nodes/google/nano-banana-2.mdx @@ -6,7 +6,6 @@ sidebarTitle: "Nano Banana 2" import ReqHint from "/snippets/tutorials/partner-nodes/req-hint.mdx"; import UpdateReminder from "/snippets/tutorials/update-reminder.mdx"; - Nano Banana 2 is now available in ComfyUI through Partner Nodes. This release fundamentally changes the default choice for most creators, delivering Pro-level quality at Flash speed. ## What Nano Banana 2 is good at @@ -59,6 +58,14 @@ Text rendering and translation into localized variants: ![Mandarin translation](https://substack-post-media.s3.amazonaws.com/public/images/5f4a52f9-1995-440f-8394-245e43f3a9a2_2784x1536.png) *Translate text to Mandarin.* +## Use it + + + Call it over HTTP through Comfy Router, with copy-paste Python, TypeScript and cURL snippets + + +This runs on the same Comfy account and is billed in the same credits as running it in ComfyUI below. See [Partner Nodes pricing](/tutorials/partner-nodes/pricing) for per-model rates and [Concurrency limits](/tutorials/partner-nodes/concurrency-limits) for how many requests you can have in flight. + diff --git a/tutorials/partner-nodes/google/nano-banana-pro.mdx b/tutorials/partner-nodes/google/nano-banana-pro.mdx index 8b9cbe065..95afbf270 100644 --- a/tutorials/partner-nodes/google/nano-banana-pro.mdx +++ b/tutorials/partner-nodes/google/nano-banana-pro.mdx @@ -6,7 +6,6 @@ sidebarTitle: "Nano Banana Pro" import ReqHint from "/snippets/tutorials/partner-nodes/req-hint.mdx"; import UpdateReminder from "/snippets/tutorials/update-reminder.mdx"; - Nano Banana Pro is Google DeepMind's flagship image model (Gemini 3 Pro Image) for studio-quality image generation and editing. It pushes beyond casual creation into production-ready visuals with advanced capabilities including native 4K resolution, precise text rendering in 10 languages, multi-image blending, and search-grounded world knowledge for accurate real-world depictions. ## What Nano Banana Pro is good at @@ -24,6 +23,14 @@ Multi-image blending, combining two reference images into one studio-quality res ![Nano Banana Pro example](https://raw.githubusercontent.com/Comfy-Org/workflow_templates/main/output/api_nano_banana_pro.png) +## Use it + + + Call it over HTTP through Comfy Router, with copy-paste Python, TypeScript and cURL snippets + + +This runs on the same Comfy account and is billed in the same credits as running it in ComfyUI below. See [Partner Nodes pricing](/tutorials/partner-nodes/pricing) for per-model rates and [Concurrency limits](/tutorials/partner-nodes/concurrency-limits) for how many requests you can have in flight. + diff --git a/tutorials/partner-nodes/ideogram/ideogram-v4.mdx b/tutorials/partner-nodes/ideogram/ideogram-v4.mdx index b338b0ee7..604589884 100644 --- a/tutorials/partner-nodes/ideogram/ideogram-v4.mdx +++ b/tutorials/partner-nodes/ideogram/ideogram-v4.mdx @@ -6,7 +6,6 @@ sidebarTitle: "Ideogram V4" import ReqHint from "/snippets/tutorials/partner-nodes/req-hint.mdx"; import UpdateReminder from "/snippets/tutorials/update-reminder.mdx"; - Ideogram 4.0 is the latest text-to-image model from Ideogram, offering superior photorealistic quality, accurate text rendering, and precise style control. You can use either plain natural language or **structured JSON prompts** for fine-grained control over layout, colors, and in-image text. ## What Ideogram 4.0 is good at @@ -36,6 +35,14 @@ A special live conversation with Mohammad Norouzi (CEO, Ideogram) and Yoland Yan allowFullScreen > +## Use it + + + Call it over HTTP through Comfy Router, with copy-paste Python, TypeScript and cURL snippets + + +This runs on the same Comfy account and is billed in the same credits as running it in ComfyUI below. See [Partner Nodes pricing](/tutorials/partner-nodes/pricing) for per-model rates and [Concurrency limits](/tutorials/partner-nodes/concurrency-limits) for how many requests you can have in flight. +