Skip to content

HYBIM-898 Update migration tool README #224

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ridhima-splunk wants to merge 4 commits into
base: main
Choose a base branch
from
+82 −8
Open

HYBIM-898 Update migration tool README #224

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
81bd6c2
Update migration tool README
ridhima-splunk Aug 13, 2026
1c9738e
Applied review comments
ridhima-splunk Aug 18, 2026
f6a756a
Added BuiltInEvaluators
ridhima-splunk Aug 18, 2026
74e7ec4
applied new comment fixes
ridhima-splunk Aug 19, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
88 changes: 81 additions & 7 deletions splunk-ao-migration-tool/README.md
View file
Open in desktop
Comment thread
ridhima-splunk marked this conversation as resolved.
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ The migration touches four areas:
| **Class / symbol names** | `Galileo*` prefix → `SplunkAO*` |
| **Environment variables** | `GALILEO_*` → `SPLUNK_AO_*` |

Additionally there are a handful of **removed features** (Protect, `GalileoScorers`) and a **Python version floor bump** (3.10 → 3.11).
Additionally there are a handful of **removed features** (Protect, `GalileoScorers`), a **Python version floor bump** (3.10 → 3.11), and **domain entity renames**: "Metrics" → "Evaluators" and "Log Streams" → "Agent Streams".

---

Expand Down Expand Up @@ -141,8 +141,11 @@ All sub-module paths follow the same rename pattern:
|-----|-----|
| `from galileo.decorator import …` | `from splunk_ao.decorator import …` |
| `from galileo.logger import …` | `from splunk_ao.logger import …` |
| `from galileo.metric import …` | `from splunk_ao.metric import …` |
| `from galileo.schema.metrics import …` | `from splunk_ao.schema.metrics import …` |
| `from galileo.metric import …` | `from splunk_ao.evaluator import …` |
| `from galileo.metrics import …` | `from splunk_ao.evaluators import …` |
| `from galileo.log_stream import …` | `from splunk_ao.agent_stream import …` |
| `from galileo.log_streams import …` | `from splunk_ao.agent_streams import …` |
| `from galileo.schema.metrics import …` | `from splunk_ao.schema.metrics import …` (module path unchanged) |
| `from galileo.shared.exceptions import …` | `from splunk_ao.shared.exceptions import …` |
| `from galileo.exceptions import …` | `from splunk_ao.exceptions import …` |
| `from galileo.configuration import …` | `from splunk_ao.configuration import …` |
Expand Down Expand Up @@ -193,17 +196,81 @@ All sub-module paths follow the same rename pattern:

### 3.3 Metric Classes

> **Domain rename:** "Metrics" → "Evaluators". The object-API classes (from `galileo.metric`) and the built-in scorer accessor are renamed. The Pydantic schema models in `splunk_ao.schema.metrics` (`Metric`, `LocalMetricConfig`) and the `MetricSpec` type alias keep their existing names — do **not** blanket find-and-replace "Metric" → "Evaluator".

| Old | New |
|-----|-----|
| `GalileoMetric` | `SplunkAOMetric` |
| `GalileoMetric` | `SplunkAOEvaluator` |
| `GalileoMetrics` | `SplunkAOEvaluators` |
| `GalileoScorers` | **Removed** (see §5.2) |
| `Metric` (OO base class from `galileo.metric`) | `Evaluator` |
| `LlmMetric` | `LlmEvaluator` |
| `LocalMetric` | `LocalEvaluator` |
| `CodeMetric` | `CodeEvaluator` |
| `BuiltInMetrics` | `BuiltInEvaluators` — `from splunk_ao.evaluator import BuiltInEvaluators` (not re-exported at top level; typically accessed via `Evaluator.metrics`) |
| `Metrics` | `Evaluators` |
| `MetricSpec` | `MetricSpec` — **not renamed** (still `from splunk_ao import MetricSpec`) |
| `LocalMetricConfig` | `LocalMetricConfig` — **not renamed** (still `from splunk_ao.schema.metrics import LocalMetricConfig`) |

```diff
- from galileo import GalileoMetric, GalileoMetrics
+ from splunk_ao import SplunkAOMetric, SplunkAOEvaluators
+ from splunk_ao import SplunkAOEvaluator, SplunkAOEvaluators

- from galileo.metric import Metric, LlmMetric, LocalMetric, CodeMetric
+ from splunk_ao.evaluator import Evaluator, LlmEvaluator, LocalEvaluator, CodeEvaluator
```

### 3.3a Log Stream / Agent Stream Classes

> **Domain rename:** "Log Streams" → "Agent Streams". All classes and methods containing "LogStream" or "log_stream" are renamed to use "AgentStream" / "agent_stream".

**Class renames:**

| Old | New |
|-----|-----|
| `LogStream` | `AgentStream` |
| `LogStreams` | `AgentStreams` |

**Method and property renames:**

| Class | Old method / property | New method / property |
|-------|-----------------------|-----------------------|
| `AgentStream` (`from splunk_ao import AgentStream`) | `set_metrics()` / `get_metrics()` | **unchanged** |
| `AgentStream` returned by `get_agent_stream()` / `create_agent_stream()` | `enable_metrics()` | `enable_evaluators()` |
| `AgentStreams` service / `splunk_ao.agent_streams` module-level helper | `enable_metrics()` | `enable_evaluators()` |
| `splunk_ao.evaluators` module-level helper (was `galileo.metrics`) | `get_metrics()` | `get_evaluators()` |
| `Project` | `create_log_stream()` | `create_agent_stream()` |
| `Project` | `list_log_streams()` | `list_agent_streams()` |
| `Project` | `.logstreams` | `.agent_streams` |

> `AgentStream.set_metrics()` and `AgentStream.get_metrics()` are **unchanged** — they still set and return the names of the evaluators enabled on the stream.

```diff
- from galileo.log_stream import LogStream
+ from splunk_ao.agent_stream import AgentStream

- project.create_log_stream("prod")
+ project.create_agent_stream("prod")

- streams = project.list_log_streams()
+ streams = project.list_agent_streams()

- project.logstreams
+ project.agent_streams
```
Comment thread
ridhima-splunk marked this conversation as resolved.

**Module-level convenience function renames:**

| Old (`galileo`) | New (`splunk_ao`) |
|-----------------|-------------------|
| `get_log_stream()` | `get_agent_stream()` (`splunk_ao.agent_streams`) |
| `list_log_streams()` | `list_agent_streams()` (`splunk_ao.agent_streams`) |
| `create_log_stream()` | `create_agent_stream()` (`splunk_ao.agent_streams`) |
| `enable_metrics()` | `enable_evaluators()` (`splunk_ao.agent_streams`) |
| `get_metrics()` | `get_evaluators()` (`splunk_ao.evaluators`) |
| `delete_metric()` | `delete_evaluator()` (`splunk_ao.evaluators`) |
| `create_custom_llm_metric()` | `create_custom_llm_evaluator()` (`splunk_ao.evaluators`) |

### 3.4 Handlers & Middleware

| Old | New |
Expand Down Expand Up @@ -413,7 +480,7 @@ with splunk_ao_context(project="my-project", agent_stream="production"):
result = call_llm("Hello")

# Direct logger approach
# project/log_stream are constructor args, not start_session args
# project/agent_stream are constructor args, not start_session args
logger = SplunkAOLogger(project="my-project", agent_stream="production")
logger.start_session(name="my-session")
logger.add_llm_span(input="Hello", output="Hi", model="gpt-4")
Expand Down Expand Up @@ -449,9 +516,16 @@ The following are **unchanged** between galileo and splunk-ao and require no mig
- [ ] Rename `GalileoAPIError` → `SplunkAOAPIError`
- [ ] Rename `GalileoLoggerException` → `SplunkAOLoggerException`
- [ ] Rename `GalileoFutureError` → `SplunkAOFutureError`
- [ ] Rename `GalileoMetric` → `SplunkAOMetric`
- [ ] Rename `GalileoMetric` → `SplunkAOEvaluator`
- [ ] Rename `GalileoMetrics` → `SplunkAOEvaluators`
- [ ] Replace `GalileoScorers` with `SplunkAOEvaluators`
- [ ] Rename domain evaluator classes: `Metric` → `Evaluator` (OO class only, **not** `splunk_ao.schema.metrics.Metric`), `LlmMetric` → `LlmEvaluator`, `LocalMetric` → `LocalEvaluator`, `CodeMetric` → `CodeEvaluator`
- [ ] Rename `BuiltInMetrics` → `BuiltInEvaluators` (`from splunk_ao.evaluator import BuiltInEvaluators`; not re-exported at top level — note: `MetricSpec` and `LocalMetricConfig` are **not** renamed)
- [ ] Update evaluator module imports: `galileo.metric` → `splunk_ao.evaluator`
- [ ] Rename `LogStream` → `AgentStream`, `LogStreams` → `AgentStreams`
- [ ] Update `Project` method calls: `create_log_stream()` → `create_agent_stream()`, `list_log_streams()` → `list_agent_streams()`, `.logstreams` → `.agent_streams`
- [ ] `AgentStream.set_metrics()` and `AgentStream.get_metrics()` are **unchanged**; replace `enable_metrics()` with `enable_evaluators()` on the `AgentStream` returned by `get_agent_stream()`/`create_agent_stream()`, on the `AgentStreams` service, and on the module-level helper
- [ ] Rename module-level helpers: `get_metrics()` → `get_evaluators()`, `delete_metric()` → `delete_evaluator()`, `create_custom_llm_metric()` → `create_custom_llm_evaluator()`, `get_log_stream()`/`list_log_streams()`/`create_log_stream()` → `get_agent_stream()`/`list_agent_streams()`/`create_agent_stream()`
- [ ] Rename `GalileoAgentControlBridge` → `SplunkAOAgentControlBridge`
- [ ] Rename all `GALILEO_*` environment variables to `SPLUNK_AO_*`
- [ ] Update `.env`, `.env.example`, CI/CD secrets, and deployment configs
Expand Down
2 changes: 1 addition & 1 deletion splunk-ao-migration-tool/examples/after_splunk_ao.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ def call_llm(prompt: str) -> str:
result = call_llm("Hello")

# Direct logger approach
# project/log_stream are constructor args, not start_session args
# project/agent_stream are constructor args, not start_session args
logger = SplunkAOLogger(project="my-project", agent_stream="production")
logger.start_session(name="my-session")
logger.add_llm_span(input="Hello", output="Hi", model="gpt-4")
Expand Down
Loading