Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 46 additions & 2 deletions .claude/skills/build-plugin/references/data-streams.md
Original file line number Diff line number Diff line change
Expand Up @@ -578,7 +578,7 @@ Always use `displayName` β€” column names in scripts are often terse API field n
- **Explicit set only**: list each column; unlisted columns are hidden.
- **Mix**: list specific columns with shapes/roles, then add the pattern for the rest.

**Type inference**: SquaredUp infers column types from the JS primitive returned. Return the correct primitive type from your script β€” don't coerce to string. Declared `shape` overrides inference.
**Type inference**: SquaredUp infers column types from the JS primitive returned. Return the correct primitive type from your script β€” don't coerce to string. Declared `shape` overrides inference. A list is the exception β€” it is never inferred; an array is serialised to a JSON string unless the column declares the `array` shape (see [List columns](#list-columns-array-shape)).

### Shapes

Expand Down Expand Up @@ -626,7 +626,13 @@ _Special_
| `"guid"` | GUID |
| `"customunit"` | Custom unit label. Options: `prefix`, `separator` |

**Array form** β€” use when you need formatting options:
_List_

| Shape | Notes |
| --------- | ------------------------------------------------------------------------------------------------------ |
| `"array"` | A list of values. Options: `itemShape` (any shape above). See [List columns](#list-columns-array-shape) |

**With options** β€” use the `[name, options]` form when you need formatting options:

```json
{ "name": "price", "shape": ["number", { "decimalPlaces": 2 }] }
Expand All @@ -636,6 +642,38 @@ _Special_
{ "name": "cost", "shape": ["currency", { "code": "jpy" }] }
```

### List columns (`array` shape)

When a field holds several values β€” tags, topics, IP addresses, environments β€” and should stay a list, declare the `array` shape:

```json
{ "name": "topics", "displayName": "Topics", "shape": ["array", { "itemShape": "string" }] }
{ "name": "ports", "displayName": "Ports", "shape": ["array", { "itemShape": ["number", { "thousandsSeparator": false }] }] }
{ "name": "seenAt", "displayName": "Seen at", "shape": ["array", { "itemShape": "date" }] }
```

Without it the platform serialises the array to a JSON string (`"[\"devops\",\"cli\"]"`). Don't work around that by joining to a comma-separated string or adding a display twin of the column: one list column feeds table display, tile filtering, the import and `objectPropertyPath` read-back.

**`itemShape`** takes any shape from the tables above, in either form: a name (`"string"`, `"number"`, `"boolean"`, `"date"`, `"url"`, `"state"`, `"bytes"`, `"seconds"`, `"percent"`, …) or `[name, options]`. Each member is converted to and formatted as that shape. `"shape": "array"` with no `itemShape` infers it from the first non-null member. Declare it anyway, so the column's type doesn't depend on whichever row came first.

**The value** is a real JS array of primitives: the raw API array via `pathToData`, or an array your script builds (split a delimited string here, rather than in the tile). A serialised JSON array string, for example from a `valueExpression`, is parsed back into members too.

| Member | Result |
| ---------------------------------------------- | ------------------------------------------------------------------------------------------------------ |
| `null` | Dropped |
| An object, alongside primitives | Dropped |
| Every member an object (`[{ "name": "a" }]`) | The whole cell stays a JSON string. Map to primitives in the script first: `tags.map((t) => t.name)` |
| Not convertible (`"abc"` with a `number` item) | The whole cell becomes no data, not a shorter list |
| Empty list | An empty list, rendered blank |

**Downstream behaviour:**

- **Display:** the members are joined with `, `.
- **Tile filters** test each member. `contains` and `equals` keep a row if any member matches, and a negated filter drops a row if any member matches. Numeric and date comparisons need a `number` or `date` `itemShape`. They are refused on a list of strings, just as on a string column.
- **Not a scalar:** a list column is never picked as a tile's value column, so scalar, gauge and bar tiles still need a separate numeric column.
- **Import:** a list column mapped into `objectMapping.properties` is written as a multi-value graph property. See [index-defs.md](index-defs.md).
- **Read-back:** an `objectPropertyPath` column that reads a list property must declare the `array` shape as well. Without it you get only the first value. See [Object property lookup](#object-property-lookup).

### Roles

| Role | Description |
Expand Down Expand Up @@ -738,8 +776,14 @@ Replace a raw ID column with a human-readable property from a related indexed ob
// Combine properties
{ "name": "AgentLabel", "sourceId": "AgentID", "sourceType": "my-agent",
"objectPropertyPath": "name", "valueExpression": "{{ object.name }} ({{ object.company }})" }

// Read back a multi-value property β€” the array shape returns every value, not just the first
{ "name": "repoTopics", "displayName": "Repository topics", "sourceId": "repoId", "sourceType": "Repository",
"objectPropertyPath": "topics", "shape": ["array", { "itemShape": "string" }] }
```

Appending a related object's list property this way lets tiles filter and group a stream's rows by it (e.g. issues filtered by their repository's topics). See [List columns](#list-columns-array-shape).

---

## Post-request scripts
Expand Down
2 changes: 2 additions & 0 deletions .claude/skills/build-plugin/references/index-defs.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,10 +59,12 @@ Defines what gets imported into the SquaredUp graph.
- `objectMapping.type` maps to the `sourceType` column. It can also be a fixed string: `{ "value": "My Device" }` β€” use when all rows are the same type, rather than a computed column.
- `objectMapping.properties` are extra fields stored on the graph node, accessible in scripts as `object.propName`.
- Use `{ "targetProp": "sourceProp" }` syntax when the column name differs from the desired property name.
- **A list-valued property needs a list column.** Map a column that declares `"shape": ["array", { "itemShape": "string" }]` and the import writes a real multi-value property, one value per member. A list without that shape arrives as a JSON string and is stored as one text value. See [List columns](data-streams.md#list-columns-array-shape).
- **Never map a column into `properties` that is already mapped as `id`, `name`, or `type`.** The id's raw value is always available on every object as `rawId` (`{{object.rawId}}` in templates, `context.objects[N].rawId` in scripts β€” and as a scalar, unlike user-defined properties, which arrive as arrays), and the name as `name`. Adding e.g. `{ "projectId": "id" }` to `properties` when `"id": "id"` already exists creates a duplicate that has to be re-indexed to take effect and otherwise sits as dead config β€” use `rawId`/`name` instead.
- **Never use a reserved name as a property name.** The property name β€” a string entry, or the *key* in `{ "targetProp": "sourceProp" }` β€” becomes a property on the graph object, and the graph owns some names. Using one deploys cleanly, then fails the import with a raw schema error that doesn't mention your mapping:
- `id`, and any `__`-prefixed name β€” always rejected: `Payload failed schema validation` … `/vertices/0/id` … `boolean schema is false`. You don't need it β€” the object's id is already `rawId` (above).
- `state` β€” reserved by the platform; an API status (`"active"`) or a postal state (`"OH"`) fails with `must be equal to one of the allowed values`. Rename it: `{ "status": "state" }`, or `{ "region": "state" }` for an address.
- `tags` β€” reserved by the graph, with its own schema; a plugin's tag list fails with `/vertices/0/tags` … `must be array`. Prefix it: `{ "checkTags": "tags" }`.
- `rawId`, `sourceId`, `sourceType`, `name`, `label` β€” silently overwritten by the platform, so the mapping does nothing.

A reserved name is fine as the *source column* (`{ "status": "state" }` reads a column called `state`). `squaredup validate` rejects `id`, `state` and `__` names; it can't catch the silent overwrites.
Expand Down
2 changes: 1 addition & 1 deletion .claude/skills/build-plugin/references/test-agent.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ For **build-mode**, also pass the **build spec** distilled from the Phase 2 plan

- Endpoint path + HTTP method, and scoping β†’ `baseDataSourceName`: single-object β†’ `httpRequestScopedSingle`, multi-object β†’ `httpRequestScoped`, unscoped/global β†’ `httpRequestUnscoped`.
- Candidate `pathToData` (where the row array lives in the response).
- Planned columns with `displayName`, `shape`, and any `role` (label/value/timestamp/sourceId…).
- Planned columns with `displayName`, `shape`, and any `role` (label/value/timestamp/sourceId…). Flag any list-valued field as `["array", { "itemShape": … }]` (see [List columns](data-streams.md#list-columns-array-shape)). Otherwise the sub-agent sees a JSON string in the shaped rows and "fixes" it by joining.
- Any `ui` parameters, `timeframes` setting, paging mode, and the `matches` selector.
- The endpoint's **data granularity** (from Phase 1) and, when the stream supports a time range coarser than the default (`timeframes` is an array or `true` at daily/monthly granularity), a **first-test `--timeframe` hint**. `squaredup test` defaults to `last1hour`, which 404s against an endpoint aggregated to daily/monthly (e.g. billing, usage), so tell the sub-agent the timeframe to pass on its first test β€” e.g. "daily granularity: first test with `--timeframe last7days`". This stops every sub-agent from independently rediscovering the default-timeframe 404. Omit the hint for streams with no time-range control (`timeframes: false`) β€” there is no timeframe to pass.

Expand Down
8 changes: 8 additions & 0 deletions .claude/skills/build-plugin/references/testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,14 @@ Example cell (a shaped currency value):
{ "raw": "1234.567", "value": 1234.57, "formatted": "$1,234.57" }
```

Example cell (a [list column](data-streams.md#list-columns-array-shape), `"shape": ["array", { "itemShape": "string" }]`):

```json
{ "raw": ["devops", "cli"], "value": ["devops", "cli"], "formatted": "devops, cli" }
```

For a list column, check `value`, not `formatted`. `formatted` is always the members joined with `, ` for display, so a plain string there is expected. `value` must be an array (its column's `shapeName` is `shape_array`). If `value` is a JSON string such as `"[\"devops\",\"cli\"]"`, there are two causes. If the column is missing the `array` shape, add it. If every member is an object, the shape alone won't fix it: map the members to primitives in the script first. Either way, don't join the members in the script.

`data` is **absent** when shaping produced nothing (e.g. the request errored). `test` sets a non-zero exit code when any request errored β€” inspect the error and the `[Response] Body`, fix the stream, and re-test.

### Zooming in
Expand Down
3 changes: 2 additions & 1 deletion REVIEW.md
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,7 @@ When suggesting changes:
- timeframe - Data streams that don't support time-based filtering must explicitly declare `"none"` as an available timeframe. Omitting this causes UI warnings and incorrect default behaviour in OOB dashboards.
- Numeric values - Return raw numeric values rather than pre-formatted strings (e.g. return `1234.56` not `"Β£1,234.56"`). Use column format expressions in dashboards for display formatting.
- Column shapes - Use the `bytes` shape for columns representing byte sizes; SquaredUp will automatically display the most appropriate unit (e.g. 4.2 GB). Check for other applicable semantic shapes such as `timestamp` and `duration`.
- List values - A column holding several values (tags, topics, addresses) uses `"shape": ["array", { "itemShape": "<shape>" }]`, not a comma-joined string, and not a second display column alongside it. An `objectPropertyPath` column reading a list property declares the same shape.
- Timestamps - SquaredUp expects ISO 8601 strings for timestamp columns. If the upstream API returns Unix timestamps, the script must convert them.
- Expressions - Where a transformation can be expressed using `map`, prefer it over a mustache-style value expression, or complex processing in a script for performance.
- Deduplication - If multiple data streams share the same API endpoint and differ only by a filter, consider merging them into a single stream with a UI parameter to control the filter.
Expand All @@ -105,7 +106,7 @@ When suggesting changes:
### Source types (indexDefinitions/)

- Name source types after how they are referred to in the upstream product or API (e.g. `agent`, `device`). Do not prefix them with the plugin name (e.g. avoid `NinjaOne Device`). A separate friendly display name can be configured if needed (via custom_types.json).
- objectMapping.properties - Never use `id`, `state`, or a `__`-prefixed name as a property name; they fail the import at graph write. Rename instead, e.g. `{ "status": "state" }`. `rawId`, `sourceId`, `sourceType`, `name` and `label` are silently overwritten, so mapping them does nothing.
- objectMapping.properties - Never use `id`, `state`, `tags`, or a `__`-prefixed name as a property name; they fail the import at graph write. Rename instead, e.g. `{ "status": "state" }` or `{ "checkTags": "tags" }`. `rawId`, `sourceId`, `sourceType`, `name` and `label` are silently overwritten, so mapping them does nothing.
- pageSize - For APIs where a tenant can hold tens of thousands of objects, check the import stream's `pageSize` against both import limits: ~6MB per page and ~220 pages per import run. See the build-plugin skill's index-defs.md, "Large imports".

### Documentation - (docs/README.md)
Expand Down
Loading