BE-522: HashQL: Expand stdlib type system with temporal types#8674
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
2 Skipped Deployments
|
PR SummaryMedium Risk Overview Refactors stdlib type construction by extracting reusable Updates graph/entity types and tests to use the new temporal metadata structures (e.g. Heap/runtime improvements add a thread-shareable bump allocator pool ( Reviewed by Cursor Bugbot for commit 5d01d1e. Bugbot is set up for automated code reviews on this repo. Configure here. |
Merging this PR will degrade performance by 16.09%
Performance Changes
Comparing |
89ae7c2 to
9047680
Compare
feat: make Int size aware feat: postgres serialization feat: checkpoint feat: checkpoint feat: checkpoint feat: errors feat: psql codec feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: move bridge -> orchestrator feat: move bridge -> orchestrator feat: organize the runtime a bit more feat: organize the runtime a bit more feat: checkpoint feat: checkpoint feat: checkpoint feat: checkpoint feat: organize the runtime a bit more feat: finish orchestrator first version feat: finish orchestrator first version feat: feat: move out suspension requests to own module chore: clippy feat: split out into more manageable functions feat: split out into more manageable functions chore: orchestrator docs feat: organize the runtime a bit more feat: test orchestrator feat: better symbol repr feat: orchestrator test harness feat: orchestrator test harness feat: orchestrator test harness feat: orchestrator test harness feat: orchestrator test harness feat: test orchestrator feat: pipeline feat: bless snapshots chore: tests chore: masking chore: tests fix: SSA repair chiore: ssa repair snapshots feat: orchestrator test harness chore: remove spec fix: library features chore: update snapshots chore: update docs
9047680 to
5d01d1e
Compare
Merge activity
|
Benchmark results
|
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 2002 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 1001 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: high, policies: 3314 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: medium, policies: 1526 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 2078 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 1033 | Flame Graph |
policy_resolution_medium
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 102 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 51 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: high, policies: 269 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: medium, policies: 107 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 133 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 63 | Flame Graph |
policy_resolution_none
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 2 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 8 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 3 | Flame Graph |
policy_resolution_small
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 52 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 25 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: high, policies: 94 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: medium, policies: 26 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 66 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 29 | Flame Graph |
read_scaling_complete
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_id;one_depth | 1 entities | Flame Graph | |
| entity_by_id;one_depth | 10 entities | Flame Graph | |
| entity_by_id;one_depth | 25 entities | Flame Graph | |
| entity_by_id;one_depth | 5 entities | Flame Graph | |
| entity_by_id;one_depth | 50 entities | Flame Graph | |
| entity_by_id;two_depth | 1 entities | Flame Graph | |
| entity_by_id;two_depth | 10 entities | Flame Graph | |
| entity_by_id;two_depth | 25 entities | Flame Graph | |
| entity_by_id;two_depth | 5 entities | Flame Graph | |
| entity_by_id;two_depth | 50 entities | Flame Graph | |
| entity_by_id;zero_depth | 1 entities | Flame Graph | |
| entity_by_id;zero_depth | 10 entities | Flame Graph | |
| entity_by_id;zero_depth | 25 entities | Flame Graph | |
| entity_by_id;zero_depth | 5 entities | Flame Graph | |
| entity_by_id;zero_depth | 50 entities | Flame Graph |
read_scaling_linkless
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_id | 1 entities | Flame Graph | |
| entity_by_id | 10 entities | Flame Graph | |
| entity_by_id | 100 entities | Flame Graph | |
| entity_by_id | 1000 entities | Flame Graph | |
| entity_by_id | 10000 entities | Flame Graph |
representative_read_entity
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/block/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/book/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/building/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/organization/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/page/v/2
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/person/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/playlist/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/song/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/uk-address/v/1
|
Flame Graph |
representative_read_entity_type
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| get_entity_type_by_id | Account ID: bf5a9ef5-dc3b-43cf-a291-6210c0321eba
|
Flame Graph |
representative_read_multiple_entities
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_property | traversal_paths=0 | 0 | |
| entity_by_property | traversal_paths=255 | 1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true | |
| link_by_source_by_property | traversal_paths=0 | 0 | |
| link_by_source_by_property | traversal_paths=255 | 1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true |
scenarios
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| full_test | query-limited | Flame Graph | |
| full_test | query-unlimited | Flame Graph | |
| linked_queries | query-limited | Flame Graph | |
| linked_queries | query-unlimited | Flame Graph |

🌟 What is the purpose of this PR?
This PR expands the HashQL standard library's temporal type system and refactors how standard library types are defined and exposed. The
QueryTemporalAxestype, previously a stub in thegraphmodule, is replaced with a fully-specifiedtemporalmodule containing a rich set of temporal bound and interval types (Timestamp,TemporalBound,Interval,LeftClosedTemporalInterval,RightBoundedTemporalInterval,PinnedTransactionTimeTemporalAxes,PinnedDecisionTimeTemporalAxes,QueryTemporalAxes, etc.). Entity types in the knowledge graph are updated to use real temporal metadata (usingLeftClosedTemporalInterval) instead of opaqueUnknownwrappers.Alongside this, type construction logic is extracted into public
typessubmodules for each standard library module, enabling external consumers to construct monomorphized type instances without going through the full module registration machinery. Several modules are madepubto support this. TheScratchPoolallocator is introduced for multi-threaded bump allocation, andFromIn<&str, A> for Rc<str, A>is implemented. TheRuntimeReprsymbol representation is refactored to useextern "C"unsized types and raw pointer field access to avoid provenance issues.🔗 Related links
🔍 What does this change?
graph::temporalas a new standard library module with a full set of temporal types:Timestamp,DecisionTime,TransactionTime,UnboundedTemporalBound,InclusiveTemporalBound,ExclusiveTemporalBound,TemporalBound,FiniteTemporalBound,OpenTemporalBound,Interval,LeftClosedTemporalInterval,RightBoundedTemporalInterval,PinnedTransactionTimeTemporalAxes,PinnedDecisionTimeTemporalAxes, andQueryTemporalAxes.QueryTemporalAxes(previously an empty opaque struct) fromgraph::Graphand moves it to the newtemporalmodule.entityknowledge types to depend ongraph::temporal::Temporaland useLeftClosedTemporalIntervalforTemporalMetadatainstead of an opaqueUnknownwrapper.typessubmodules forcore::option,core::url,core::uuid,graph::temporal,graph::types::ontology,graph::types::principal::actor_group,graph::types::principal::actor_group::web, andgraph::types::knowledge::entity, each accepting optional dependency structs for injection.std_lib,std_lib::core,std_lib::graph,graph::types,graph::types::knowledge,graph::types::ontology,graph::types::principal,graph::types::principal::actor_group,graph::types::principal::actor_group::web,graph::temporal,module::std_lib, andmodule::std_lib::core::option/url/uuidpublic.ScratchPoolandScratchPoolGuardto the heap module, providing aSync-capable pool of bump allocators backed bybump_scope::BumpPool.FromIn<&str, A> for Rc<str, A>with tests covering unicode, empty strings, and clone sharing.RuntimeReprto use anextern "C"unsizedUnsizetype and raw pointer field access (&raw const/mut) to avoid provenance narrowing issues, replacingstr_from_raw_partswithstr::from_utf8_uncheckedandget_mut_unchecked/extern_typesnightly features.sym::constants throughout, eliminating mostheap.intern_symbol("...")calls.symsymbol table, including type names, field names, and fully-qualified path symbols for all new and existing types.🛡 What tests cover this?
FromIn<&str, A> for Rc<str, A>: basic conversion, unicode, empty string, and clone pointer equality.❓ How to test this?
cargo test -p hashql-coreand confirm all tests pass.temporalmodule types are accessible and correctly structured via the standard library manifest.