Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 10 additions & 10 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,8 @@ crate-type = ["cdylib"]
# with (ADR 0002), so a revision is the honest way to say which one.
# A local checkout is used instead with a `paths` override in
# `.cargo/config.toml`, which is untracked on purpose.
zudb = { package = "zu", git = "https://github.com/tamnd/zu", rev = "130f67db924bcd0f766ee814b0da2edae32150d4" }
zu-common = { git = "https://github.com/tamnd/zu", rev = "130f67db924bcd0f766ee814b0da2edae32150d4" }
zudb = { package = "zu", git = "https://github.com/tamnd/zu", rev = "6753ded13a215bf5e8fe70ce41e69b35321e3e73" }
zu-common = { git = "https://github.com/tamnd/zu", rev = "6753ded13a215bf5e8fe70ce41e69b35321e3e73" }
# N-API by way of napi-rs (ADR 0002). `napi9` is the version of N-API
# this addon declares it needs, which is what makes one binary work
# across Node 24, Node 26, Electron and Bun without a rebuild: the
Expand Down
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -290,6 +290,8 @@ The same memory is on both sides of that: `table.getChild("age").data[0].values`

Two things are not buffers, and both are named by the type rather than found out by looking. A column of nodes, rels, paths, lists or records has no fixed width cell, so it arrives as `items`, holding the same JavaScript values `query` would have made. A column of nothing but nulls has a length and nothing else, because there is nothing to put in a buffer. A column that mixes two types is refused, naming the column and the row that did it, since a columnar result holds one type per column and a column that quietly became strings is worse than one that would not build.

A statement that matched nothing still comes back with its columns, each one the type the plan declared and each one empty: a string column of no rows has its one starting offset, an integer column of no rows has a buffer of no elements. So a table built from an empty answer has the schema the same statement would have had with rows in it, and a loop that concatenates a page at a time does not have to hold the first page that had anything in it as a special case.

`bigIntMode` says nothing here. A columnar read has one physical layout per type and an INT64 column is 64 bit cells however a caller would rather read one, which is the difference between a buffer and a value. The mode still decides what is inside `items`, where this client is making objects anyway.

## Preparing a statement
Expand Down
1 change: 1 addition & 0 deletions binding.d.cts
Original file line number Diff line number Diff line change
Expand Up @@ -593,6 +593,7 @@ export interface ZuProgressOptions {
* already been left behind.
*/
export interface ZuProgress extends Disposable {
/** Stops the watch. The callback is not called again. */
stop(): void
}

Expand Down
1 change: 0 additions & 1 deletion etc/zudb.api.md
Original file line number Diff line number Diff line change
Expand Up @@ -373,7 +373,6 @@ export interface ZuProfile {

// @public
export interface ZuProgress extends Disposable {
// (undocumented)
stop(): void
}

Expand Down
36 changes: 28 additions & 8 deletions test/columnar.test.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -240,14 +240,26 @@ test('what no buffer covers arrives as the values themselves', async (t) => {

test('a statement that matched nothing is columns of no rows', async (t) => {
const { conn } = await twoPeople(t)
const read = await conn.columnar("MATCH (p:person) WHERE p.name = 'nobody' RETURN p.id AS id")
const read = await conn.columnar(
"MATCH (p:person) WHERE p.name = 'nobody' RETURN p.id AS id, p.name AS name",
)
const { id, name } = named(read)

assert.equal(read.rows, 0)
assert.equal(read.columns.length, 1)
assert.equal(read.columns[0].length, 0)
// Nothing settled the type, so it is the type of nothing, which is
// the one every columnar format has for exactly this.
assert.equal(read.columns[0].type, 'null')
assert.equal(read.columns.length, 2)
// No row settled the type, so it is the type the plan declared and
// not the type of nothing: an empty answer has the schema the same
// statement would have had with rows in it, which is what makes a
// table built from one appendable to a table built from the other.
assert.equal(id.type, 'int')
assert.equal(id.length, 0)
assert.equal(id.values.length, 0)
// A string column of no rows still carries the offset every string
// column starts from, so the subarray arithmetic holds at zero rows
// rather than being a case a reader has to special case.
assert.equal(name.type, 'string')
assert.equal(name.data.length, 0)
assert.deepEqual([...name.offsets], [0])
})

test('a statement that projects nothing has no columns and says so', async (t) => {
Expand Down Expand Up @@ -351,15 +363,23 @@ test('a million rows come back down one buffer and the loop stays free', async (

let ticks = 0
const timer = setInterval(() => (ticks += 1), 1)
const at = performance.now()
const read = await conn.columnar('MATCH (n:number) RETURN n.at AS at')
const took = performance.now() - at
clearInterval(timer)

assert.equal(read.rows, rows)
assert.equal(read.columns[0].values.length, rows)
assert.equal(read.columns[0].values[rows - 1], BigInt(rows))
// The whole read is on the threadpool, so the timer kept firing
// throughout it rather than queueing behind it.
assert.ok(ticks > 20, `the event loop ticked ${ticks} times`)
// throughout it rather than queueing behind it. The bar is a tick
// every ten milliseconds of the read and not a fixed count, because
// a blocked loop fires none however long the read takes and a fixed
// count turns every speedup into a failure.
assert.ok(
ticks > took / 10,
`the event loop ticked ${ticks} times in ${took.toFixed(0)} ms`,
)
})

// The types every fixed-width column maps to, which is the whole of
Expand Down
11 changes: 10 additions & 1 deletion test/register.test.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -520,5 +520,14 @@ test('registering costs the same whatever the frame holds', async (t) => {
best[rows] = Math.min(best[rows], Number(process.hrtime.bigint() - started) / 1e6)
}
}
assert.ok(best[5_000_000] < 2, `registering 5m rows took ${best[5_000_000].toFixed(2)} ms`)
// Against the ten row call rather than against a millisecond count,
// because the claim is that the cost does not follow the rows and a
// fixed budget measures the machine's load as much as the code: a way
// in that walked the rows would be five hundred thousand times the
// small frame and not twenty.
assert.ok(
best[5_000_000] < best[10] * 20 + 2,
`registering 5m rows took ${best[5_000_000].toFixed(2)} ms ` +
`against ${best[10].toFixed(2)} ms for ten`,
)
})
Loading