feat(sipflow): export raw-file flush and general sqlite pipeline metrics - #270
Open
ftong2010 wants to merge 3 commits into
Open
feat(sipflow): export raw-file flush and general sqlite pipeline metrics#270ftong2010 wants to merge 3 commits into
ftong2010 wants to merge 3 commits into
Conversation
added 3 commits
September 2, 2026 08:50
The raw capture file already recorded write latency (sipflow_raw_write_seconds); add the flush counter and a per-flush byte-size histogram so the data.raw write path can be correlated with sqlite flush metrics (sipflow_flush_rows_total / flush_batch_size / flush_db_seconds) against cgroup IO counters.
…w_* ones Instrument the sqlite usage with a general taxonomy — connections (opened/open/errors/latency, role=read|write), statements (statements_total, rows read/written, latency by kind select/insert/pragma/ddl), transactions (commit/rollback + latency), WAL checkpoints (passive/truncate + busy) and file gauges (db/wal bytes, page_count, freelist). All carry a database label (value: sipflow) so the same taxonomy generalizes to other sqlite usage later. Existing domain metrics (sipflow_flush_*, sipflow_wal_checkpoint_*, sipflow_db_file_bytes) are unchanged.
…sted
The bucket-rotate path released sqlite_connections_open{role=write}
unconditionally; on the first rotate with no open connection (startup
layout pass) that drove the gauge negative.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Investigating sustained disk IO on the collector (measured ~7,200 write
IOPS / ~100 MB/s at 50 cps ingest) required ad-hoc probing of sqlite and
raw-file behavior. This PR makes the write/read pipeline permanently
observable so IO attribution is a metrics query, not an investigation.
Existing domain metrics (
sipflow_flush_*,sipflow_wal_checkpoint_*,sipflow_db_file_bytes,sipflow_raw_write_seconds) are unchanged; thenew metrics sit alongside them with a general taxonomy.
What this PR adds
1. Raw capture file (
data.raw)sipflow_raw_flush_totaldata.rawsipflow_raw_flush_bytessipflow_raw_write_seconds2. General sqlite taxonomy (
sqlite_*)Every metric carries a
databaselabel (valuesipflow) so the samenames apply to other sqlite usage later.
sqlite_connections_opened_total{role=read|write}sqlite_connections_open{role}sqlite_connection_open_seconds{role}sqlite_connection_errors_total{phase}sqlite_statements_total{kind=select|insert|pragma|ddl}sqlite_statement_rows_total{kind,direction=read|written}sqlite_statement_seconds{kind}sqlite_errors_total{kind,error=busy|locked|constraint|other}sqlite_transactions_total{outcome=commit|rollback}+sqlite_transaction_secondssqlite_wal_checkpoint_total{kind=passive|truncate}+_seconds+_busy_totalsqlite_db_bytes,sqlite_wal_bytes,sqlite_page_count,sqlite_freelist_pagesVerification
50 cps × 300 s load against a rebuilt collector, all new metrics live:
sqlite_statement_rows_total{written}10,730,964 vssipflow_flush_rows_total10,700,976 — Δ 0.28 % (same rows, twoindependent counters)
sqlite_transactions_total{commit}5,965 → 1,800 rows/tx,matching the observed ~1,795-row flush batches
sqlite_wal_checkpoint_total{passive}33 in 345 s — exactly thethrottled 1 per 10 s
sqlite_connections_opened_total{read}194,530 × 4 PRAGMAs each =778,120, matching
sqlite_statements_total{pragma}778,126 (the +6are the single write connection's setup)
sipflow_raw_flush_total30,811 (~89/s),sipflow_raw_flush_bytesaveraging 62.8 KB — the 64 KB batch threshold working as designed
Correlated against cgroup
io.statduring the run: ~7,200 write IOPS /~100 MB/s device writes sustained, dominated by WAL page appends +
checkpoint copying of ~33 k
media_msgsrows/s.Notes
(
backend::remote::tests::test_report_endpoint_returns_receive_counters)fails identically on unmodified
main; unrelated.flush_sliceis split into a thin wrapper +flush_slice_innertoscope the transaction timer around the existing
?error paths; thesilent
.ok()on bucket-creation DDL now also incrementssqlite_errors_total(still non-fatal).