Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,9 @@ python -m blacki.server
Telegram long polling, and exposes `/live`, `/ready`, and `/health`.
It also mounts the package-backed private observability dashboard at
`/dashboard` with APIs for aggregate statistics, users, sessions, local logs,
and local traces.
and local traces. LiteLLM-backed requests additionally write content-free
usage and cost records to the local SQLite ledger so the dashboard can show
per-user and per-session spend without parsing prompts or responses.

`src/blacki/agent.py` creates the `LlmAgent`, selects a native Gemini model or a
LiteLLM/OpenRouter model from the environment, registers tools, and assembles
Expand Down
1 change: 1 addition & 0 deletions docs/base-infra/environment-variables.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ surface on every host interface.
| `ALLOW_ORIGINS` | local origins JSON | JSON array of CORS origins |
| `AGENT_ENGINE` | unset | Optional Agent Engine identifier |
| `SQLITE_PATH` | `{AGENT_DIR}/.adk/tools.db` | SQLite file for application tools |
| `BLACKI_COST_LEDGER_PATH` | `{AGENT_DIR}/.adk/costs.db` | Content-free model usage and cost ledger |

For `ALLOW_ORIGINS`, use a JSON array string:

Expand Down
12 changes: 12 additions & 0 deletions docs/base-infra/observability.md
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,18 @@ Telegram `/reset` starts the next versioned ADK session and leaves earlier
session rows available. The append-only log and trace files are not deleted by
`/reset`, so the dashboard continues to show that history.

For LiteLLM-backed requests, Blacki also writes a content-free SQLite usage
ledger at `{AGENT_DIR}/.adk/costs.db` (override with
`BLACKI_COST_LEDGER_PATH`). The ledger stores identity, model, token, provider
response, and fixed-point cost fields, but never prompts, responses, or tool
arguments. Provider-reported OpenRouter account cost and upstream inference
cost are kept separately; a LiteLLM catalog calculation is labelled estimated.
The dashboard uses UTC calendar months for monthly totals and averages. The
average is across users with an exact or estimated cost in the current month;
users with unavailable cost are excluded and the reported coverage is shown.
Records created before cost capture, or responses without a provider cost,
remain unavailable rather than being treated as zero.

This is an admin-only, private-data surface. The application does not add
HTTP Basic auth, cookies, or Tailscale identity-header authentication. For
direct tailnet access, set `HOST_BIND_IP` to the host's Tailscale IPv4 address
Expand Down
3 changes: 3 additions & 0 deletions src/blacki/agent.py
Original file line number Diff line number Diff line change
Expand Up @@ -176,6 +176,9 @@ def _build_model() -> str | LiteLlm:
litellm_kwargs: dict[str, Any] = {}
if model_name.lower().startswith("openrouter/") and openrouter_api_key:
litellm_kwargs["api_key"] = openrouter_api_key
from .llm_costs import CostAwareLiteLLMClient

litellm_kwargs["llm_client"] = CostAwareLiteLLMClient()

logger.info("Using LiteLlm for model: %s", model_name)
return LiteLlm(model=model_name, **litellm_kwargs)
Expand Down
23 changes: 23 additions & 0 deletions src/blacki/callbacks.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
from google.adk.tools import ToolContext
from google.adk.tools.base_tool import BaseTool

from .llm_costs import attach_cost_metadata, begin_cost_capture
from .privacy import is_private_tool, private_tool_privacy_enabled
from .telegram.api import TelegramApiClient, TelegramApiError
from .telegram.formatting import format_for_telegram
Expand Down Expand Up @@ -531,6 +532,12 @@ def before_model(
f"*** Before LLM call for agent '{callback_context.agent_name}' "
f"with invocation_id '{callback_context.invocation_id}' ***"
)
session = getattr(callback_context, "session", None)
begin_cost_capture(
user_id=getattr(callback_context, "user_id", None),
session_id=getattr(session, "id", None),
invocation_id=getattr(callback_context, "invocation_id", None),
)
self.logger.debug(f"State keys: {callback_context.state.to_dict().keys()}")

redact_content = private_tool_privacy_enabled()
Expand Down Expand Up @@ -578,6 +585,22 @@ def after_model(
response_data = llm_content.model_dump(exclude_none=True, mode="json")
self.logger.debug(f"LLM response: {response_data}")

cost_observation = attach_cost_metadata(llm_response)
if cost_observation:
self.logger.debug(
"LLM cost captured: %s",
{
key: cost_observation[key]
for key in (
"cost_usd",
"upstream_cost_usd",
"estimated_cost_usd",
"cost_kind",
)
if key in cost_observation
},
)

return None

def before_tool(
Expand Down
Loading
Loading