Relationship Bridge V2 — Open Questions

Status: Under review (2026-05-05) Companion doc: relationship-bridge-architecture.md Purpose: Tracks the decisions still required after reframing the bridge as a deterministic SECS semantic kernel.

---

How to Read This Document

The prior open-questions document was useful as exploration, but it inherited the v1 mistake: it treated SECS.Native as a primitive-first Flecs-inspired layer and then tried to decide details inside that framing.

V2 changes the order of reasoning:

1. Define the SECS semantic kernel contract.
2. Define identity, relation metadata, deterministic iteration, execution lanes, and host boundaries.
3. Derive storage primitives from that contract.
4. Migrate existing engine subsystems behind the semantic boundary.

The questions below are therefore not a continuation of the old 23 decisions as-is. Several old decisions collapse, several become mechanical constraints, and several need redesign.

---

V2 Commitments Already Made

The companion architecture document now commits these points:

• The bridge is a SECS Kernel, not a primitive-first Flecs clone.
• SECS ids remain 64-bit; packed Flecs pair ids are rejected as the semantic identity model.
• WorldSchema is the compiler/runtime contract the kernel consumes.
• Relation metadata drives storage and invalidation.
• Host integration is split by responsibility instead of one HostBridge*.
• Execution lanes are required.
• Query planning is SECS-first; a full backtracking VM is optional implementation detail.
• Determinism is specified at semantic iterator boundaries.
• Observability ABI is host-agnostic.
• Migration is contract-first, with native backing after semantic tests exist.

Open questions below are the remaining choices inside that v2 shape.

---

Retired or Reframed V1 Decisions

V1 Topic	V2 Treatment
PairId packed as 31-bit relation + 32-bit target	Rejected. SECS ids remain 64-bit. Use explicit composite keys; any packing is internal only.
Public RawHashMap / RawMultiHashMap tier	Reframed. Raw collections are implementation details unless adapter authors prove a stable low-level API is needed.
One giant HostBridge* replacing ISecsHostReads	Rejected. Split into field, callable, command, entity, trace, and instrumentation bridges.
Unity ProfilerMarker fields inside L4 structs	Rejected. Kernel uses opaque marker ids; Unity adapter maps them to Unity profiling types.
"Build all SECS.Native primitives in isolation"	Rejected. Build semantic APIs and correctness tests first; native backing follows.
Hashmap iteration order as the full determinism answer	Reframed. Semantic iterators define order; deterministic containers are an internal requirement.
Flecs-like query VM as the query architecture	Reframed. Query planner is SECS-first; VM/backtracking is an implementation strategy for complex plans only.
ReadPrevTickInt missing from host bridge	Reframed. Prev-tick reads are runtime/kernel snapshot service, not a host field callback.
ISecsHostReads flip as the only migration event	Reframed. Generated signature flip remains major, but runtime can prepare with compatibility facades and split bridges first.

---

Question 1 — Kernel Name and Public API Tier

Question: Is the new layer named SECS.Kernel, SECS.Native, or something else, and what is public?

Options

Option	Shape	Tradeoff
A	Rename concept and assembly to SECS.Kernel	Clearest architecture; bigger project rename
B	Keep assembly SECS.Native, document concept as "SECS Kernel"	Lowest churn; name still suggests primitives
C	Split SECS.Kernel from SECS.UnsafeCollections	Cleanest long-term tiering; more assemblies

Recommendation

Use SECS.Kernel as the architectural name. Assembly naming can be decided later. If the assembly remains SECS.Native, its README must say "native means GC-free/blittable internals, not a public raw collection product."

Public API should expose semantic kernel operations: schema load, relation mutation/query, command journal, query plan execution, modifier/channel index access, snapshots, diagnostics. Raw collections should remain internal until a concrete adapter requirement forces them public.

Decision Needed

Pick the assembly naming policy and decide whether raw collection types are public in v1.

---

Question 2 — Entity Identity and Handle Layout

Question: What entity handle shape does the kernel commit to?

Constraints

• Current EntityHandle is a 64-bit long.
• SECS declaration ids are 64-bit ulong FNV-1a hashes.
• Unity DOTS Entity is index/version shaped.
• The kernel must not adopt Flecs's 32-bit target limit as its semantic contract.

Options

Option	Shape	Tradeoff
A	Keep long EntityHandle.Value as the semantic id	Matches current code; easy server migration
B	Introduce EntityRef { int Index; int Generation; }	Maps naturally to DOTS; requires current engine migration
C	Introduce EntityRef { long StableId; int Generation; }	Preserves 64-bit space and stale-handle detection; larger key
D	Adapter-local mapping: kernel uses long, Unity adapter maps DOTS index/version	Keeps kernel host-neutral; adapter owns mapping complexity

Recommendation

Adopt D for v1: kernel semantic identity remains 64-bit, and adapters maintain host-native mappings. Consider adding generation tracking internally only if stale-handle bugs become a real failure mode.

Decision Needed

Confirm 64-bit semantic entity ids and reject any packed pair/target encoding that truncates them.

---

Question 3 — RelationDefinition Schema

Question: What metadata must every relation carry?

Required Fields

At minimum:

• RelationId
• source scope id
• target scope id
• relation kind
• cardinality: one-to-one, one-to-many, many-to-one, many-to-many
• ordering: unordered/stable-by-target, ordered-list, keyed-dictionary
• inverse index policy
• lifecycle policy
• mutation policy
• acyclicity / self-edge policy
• save/load policy
• mod visibility policy
• host-backed flag

Relation Kinds

Initial required kinds:

• ScopeWalk
• CollectionMembership
• Ownership
• TemplateIsA
• ModifierBindingOwner
• SavedScope
• HostReference

Recommendation

Define relation metadata before storage. Storage tables should be generated/configured from RelationDefinition, not hand-coded per use case.

Decision Needed

Review the companion doc's proposed field set and enum axes. Decide exact enum names and whether mod visibility is a separate policy field or derived from relation kind.

---

Question 4 — Relation Mutation and Command Journal

Question: Are relation mutations immediate, deferred, or both?

Options

Option	Shape	Tradeoff
A	Immediate mutation	Simpler mental model; hard to parallelize and cache safely
B	Deferred journal, single canonical apply phase	Deterministic and cache-safe; one-tick semantics where writes are observed later
C	Immediate during activation/destruction, deferred during systems	Practical hybrid; requires clear phase rules

Recommendation

Use C:

• Structural setup/teardown inside activation/destruction may apply immediately because it happens inside controlled runtime phases.
• System/event/activity writes go through a deterministic command journal and apply at declared boundaries.
• Unity adapter maps the journal to EntityCommandBuffer where appropriate.

Proposed Apply Phases

Phase	Applies
ActivationImmediate	template activation, child spawn wiring, own-root channel sources
SystemEnd	commands emitted by one system before next system observes state, if current semantics require this
TickEnd	batched host writes, dirty sync, deferred relation changes safe to observe next tick
HotReloadCommit	schema diff application and rebind/reindex work
LoadCommit	save-game state materialization after schema validation

The hard question is SystemEnd vs TickEnd. Current generated bodies often flush commands inside bodies, so a pure tick-end journal is a semantic change unless the language explicitly adopts deferred visibility.

Decision Needed

Define exact apply points in the tick pipeline and document when .secs authors observe writes.

---

Question 5 — Scope Walk Semantics and Reachable Cache

Question: What does a scope walk mean once scope edges live in kernel relation indexes?

Required Decisions

• Is walks_to always explicit-only? Current docs say yes.
• Are multi-hop shortcuts declared edges or derived transitive paths?
• When multiple targets exist for a scope id, is that illegal, first-by-order, or query-only?
• Is a failed walk a null result plus diagnostic, or a hard runtime error in some contexts?
• What invalidates a cached walk?

Recommendation

Keep explicit-only walks. A WalkScope(entity, Settlement) lookup resolves a declared ScopeWalk relation to a single target. Multi-target traversals should be query operations, not scalar walks.

ReachableCache should cache semantic walk results keyed by (entity, scopeId, worldGeneration/relationGeneration). It must invalidate on:

• source relation add/remove
• target destruction
• relation definition hot reload
• collection membership changes that affect declared walks
• template transitions if template IsA participates in scope predicates

Decision Needed

Define cardinality errors and null/diagnostic policy for scalar walks.

---

Question 6 — Query Plan IR

Question: What does the compiler lower query-like .secs expressions into?

Candidate Plan Ops

• ScanContract
• ScanRelationTargets
• ScanRelationSources
• FilterHasTemplate
• FilterHasContract
• FilterHasTag
• FilterScalarCompare
• SortByChannel
• Limit
• ProjectEntity
• later: BindVariable, JoinVariable, RandomPick, MaterializeSavedScopeList

Recommendation

Design a small SECS QueryPlan IR first. Do not commit to a general Flecs-style query VM as v1. Most early Valenar needs are scans, filters, ordered materialization, and simple joins.

Backtracking is an implementation detail for plans with multiple open variables, not the core public model.

Decision Needed

Confirm the initial op set above and specify whether RandomPick is v1 or deferred until the deterministic RNG contract is redesigned.

---

Question 7 — Execution Lanes

Question: How does generated code declare whether it can run in native/Burst-compatible paths?

Proposed Lanes

Lane	Meaning
NativeEligible	Blittable, no managed callbacks, no generic dispatch, no allocation, adapter can Burst compile
KernelEligible	Uses kernel services but stays inside deterministic SECS APIs
ManagedOnly	Uses managed records, generic callables, template metadata objects, or host-managed callbacks

Recommendation

Make execution lane metadata part of generated output. The compiler should classify formulas, triggers, systems, queries, and contract methods. Hand-written stand-ins must declare the same metadata manually until the compiler exists.

Decision Needed

Decide policy for formulas/triggers that call ResolveChannel*: allow as KernelEligible when dependencies are declared, or mark as ManagedOnly until a native channel-reentry path exists.

---

Question 8 — Host Boundary Split

Question: What replaces ISecsHostReads?

Proposed Bridges

Bridge	Responsibilities
HostFieldBridge	scalar/entity field reads and writes
HostCommandSink	host side effects and command application
HostCallableBridge	declared scope methods and contract callables implemented by host
HostEntityBridge	entity creation/destruction/id mapping
TraceBridge	diagnostics, walk/query/modifier traces
InstrumentationBridge	opaque performance markers/counters

Recommendation

Split the runtime boundary by responsibility. Template metadata lookup, channel resolution, template-field resolution, saved scopes, and prev-tick snapshots remain runtime/kernel services.

Decision Needed

Specify exact methods and ownership for each bridge. Confirm which bridges have native function-pointer forms and which are managed-only.

Bridge Review Checklist

For each bridge, decide:

• Is it callable from NativeEligible bodies?
• Does it accept only blittable arguments?
• Does it return a view or a scalar?
• Is it allowed during query evaluation?
• Is it allowed during channel resolution?
• Does it record commands or mutate immediately?
• Does it require adapter state pinning?

This checklist is more important than the final type names. The old boundary failed because it allowed unrelated capabilities to accrete without classifying them.

---

Question 9 — Channel/Modifier Ownership Boundary

Question: Which parts of channel resolution move into the kernel?

Options

Option	Shape	Tradeoff
A	Kernel owns only indexes; ChannelResolver stays in L3	Lowest semantic risk; Burst benefits delayed
B	Kernel owns modifier bucket preparation and dependency graph; L3 owns value math	Balanced
C	Kernel owns full channel resolution	Highest performance potential; largest rewrite and lane complexity

Recommendation

Use B. Preserve the six-phase resolver semantics in L3 while moving the deterministic binding indexes, dependency graph, invalidation, snapshots, and prepared buckets into the kernel.

Decision Needed

Define the kernel APIs ChannelResolver consumes and what prepared bucket data looks like for all scalar types.

---

Question 10 — Save/Load Format and Content Manifest

Question: What save format does the kernel use?

Stable Decisions

• Content manifest is mandatory from day one.
• Save envelope records schema hash/version, active content packages, declaration name/id mappings, mod ids, and relation/table section versions.
• Kernel state must be host-neutral.

Required Save Sections

Section	Required?	Notes
header/schema	yes	schema id, kernel version, endian/version flags
content manifest	yes	package ids, load order, declaration name/id maps, mod ids
entity table	yes	semantic entity ids and optional adapter mapping token
relation tables	yes	runtime-owned relations; host-backed relations may rebuild
modifier bindings	yes	owner, target, modifier id, state, duration, mod id
channel snapshots	yes for tracked channels	prev-tick values and any save-worthy cache sections
command journal	optional	only if mid-tick saves are supported
diagnostics/replay	optional	debug builds or deterministic replay mode

Format Options

Format	Tradeoff
Custom TLV	Small, explicit, no dependency; more hand maintenance
FlatBuffers	Strong schema evolution and near-zero-copy reads; schema/tooling complexity
MessagePack IntKey	Fast and familiar to server stack; less zero-copy
Hybrid binary + JSON manifest	Fast state plus debuggable manifest; two formats

Recommendation

Pick the format after the kernel state sections are specified. Do not choose serialization before knowing the tables.

Decision Needed

Confirm the required save sections above, then pick TLV/FlatBuffers/MessagePack.

---

Question 11 — Hot Reload Commit Policy

Question: How does a new WorldSchema replace an old one during development?

Recommended Policy

Use the CK3-style split:

• Data changes patch declarations and invalidate affected caches at a tick boundary.
• Structural changes pause, rebuild affected kernel indexes, re-activate/rebind affected entities, then resume.
• Generated managed code changes still require assembly reload until a deliberate ALC/hot-swap design exists.

Proposed Diff Categories

Category	Examples	Commit behavior
data-only	modifier numeric value, translation, clamp text, tag display name	patch declaration, invalidate affected caches
relation-structural	new walks_to, collection type change, relation cardinality change	rebuild relation indexes; may require host support
channel-structural	new channel, channel kind change, source field change, track_prev change	rebuild channel metadata, dirty affected channels, snapshot policy update
callable-signature	scope method/contract query parameter or return change	reload generated/host callable bindings
generated-code	formula/system/event body change	assembly reload until ALC hot-swap is designed
mod-manifest	package load order or id/name map change	validate save compatibility and rebuild manifest indexes

Decision Needed

Confirm the categories above and define which are allowed during a running session.

---

Question 12 — Deterministic Iteration Contract

Question: Where is determinism specified?

Recommendation

Specify determinism on semantic APIs, not only containers:

• entity sets: ascending entity id unless a declared order exists
• contract sets: stable schema/activation order or explicit id order
• relation targets/sources: relation ordering metadata
• ScopedList: list order
• ScopedDictionary: stable key order
• query results: explicit order_by, then plan-defined fallback order
• command journal: tick step, system order, sequence number, entity id
• save sections: schema order, then stable key order

Internal hashmaps and allocators must also be deterministic, but they are subordinate to API-level rules.

Decision Needed

Adopt the exact fallback ordering for each semantic iterator.

Residual Choice

The companion architecture proposes exact defaults. The one remaining policy choice is whether entities by contract should be activation order or ascending entity id. Activation order preserves current generated-system behavior better; ascending id is simpler for deterministic replay. Pick one and make generated query lowering request the other explicitly when needed.

---

Question 13 — Observability ABI

Question: What observability hooks must exist from day one?

Required Slots

• ModId on runtime modifier bindings/prepared entries
• walk trace callback
• query trace callback
• modifier trace callback
• diagnostic sink
• opaque performance marker ids

Rejected Shape

Do not put Unity ProfilerMarker fields in kernel structs. Unity adapter maps opaque ids to Unity markers.

Decision Needed

Define TraceBridge event payload structs and PerfMarkerId.

Trace Volume Policy

Also decide the volume model:

• always-on counters only
• sampled traces
• opt-in per entity/channel/query
• full trace in dev/editor only

Without this policy, a correct trace ABI can still become too expensive to leave wired in production.

---

Question 14 — Migration Sequence

Question: How does the current engine move to the kernel without breaking Valenar/server work?

Recommended Sequence

1. Generate/build WorldSchema from current SecsRegistry tables.
2. Add semantic kernel interfaces with a managed reference implementation.
3. Add deterministic entity and relation indexes.
4. Route WalkScope and child collection queries through kernel indexes.
5. Route InstanceStore through deterministic kernel entity/contract/scope tables.
6. Move modifier binding indexes and propagation mappings into kernel.
7. Move channel dependency graph and prev-tick snapshots into kernel.
8. Add query-plan execution for hand-written stand-ins.
9. Split host bridges behind compatibility facades.
10. Flip generated signatures once the bridge split is proven.

Decision Needed

Confirm whether a managed reference kernel is required before unsafe/native backing. It is slower but gives a correctness oracle for fuzz/differential tests.

Milestone Definitions

Milestone	Exit condition
M1: schema mirror	current registry can produce WorldSchema with no behavior change
M2: deterministic entity index	InstanceStore behavior matches current tests, with deterministic iteration specified
M3: relation index	current WalkScope and child collection reads route through kernel indexes
M4: modifier index	modifier add/remove/query behavior matches current store
M5: channel dependency/snapshot	ChannelCache and prev-tick behavior match current engine
M6: query plans	hand-written Valenar stand-ins can execute one kernel query path
M7: bridge split	compatibility facades replace direct ISecsHostReads usage internally
M8: generated flip	generated signatures move to v2 lanes/bridges

These milestones are intentionally behavior-preserving until M6. New syntax waits until after M6.

---

Question 15 — Compiler and Syntax Sequencing

Question: Does v2 commit new .secs syntax now?

Recommendation

No. Commit kernel semantics before syntax expansion.

Stand-ins first, compiler second still holds:

• ordered iteration can be hand-written against query plans
• saved-scope lists can be hand-written once kernel storage exists
• multi-hop predicates can be hand-written as query plans
• cross-hop aggregation waits for transitive traversal semantics
• template inheritance waits for TemplateIsA semantics

Decision Needed

Confirm no new author-facing syntax lands until the kernel can execute the underlying semantics and the lowering contract has been updated.

---

V2 Cross-Cutting Findings

Finding A — The Kernel Is Semantic, Not Primitive

The central layer is the SECS Kernel. Raw native collections are an implementation strategy. Public architecture must discuss SECS concepts first.

Finding B — Host Boundary Must Split

ISecsHostReads currently combines field reads, graph navigation, template metadata, generic callables, channel re-entry, template-field resolution, and snapshots. One HostBridge* would preserve the design flaw. Split bridges by responsibility.

Finding C — Determinism Is an API Contract

Stable hashmap iteration is required, but semantic iterators must also define stable order. Otherwise deterministic containers can still expose nondeterministic gameplay behavior.

Finding D — SECS Identity Is 64-Bit

Do not use Flecs packed pair identity as the SECS identity contract. Relation ids and entity ids must not be truncated.

Finding E — Relation Metadata Drives Storage

The kernel needs relation definitions with cardinality, ordering, lifecycle, inverse, and save/load policy. Storage follows metadata.

Finding F — Execution Lanes Prevent Burst Ambiguity

Generated code must say whether it is native eligible, kernel eligible, or managed only. Burst compatibility is not implied by replacing an interface with function pointers.

Finding G — Observability Must Be Host-Agnostic

Reserve trace and profiling hooks from day one, but use opaque ids and bridge callbacks. Unity profiling types belong in the Unity adapter.

Finding H — Migration Is Contract-First

Build semantic contracts and tests first. Then implement native backing. Avoid a large isolated primitive library that may not match SECS.

---

Summary of Decisions Required

#	Decision	Recommendation	Status
1	Kernel name/API tier	Use SECS Kernel concept; raw collections internal by default	Needs choice
2	Entity identity	Preserve 64-bit semantic ids; adapters map host handles	Needs confirmation
3	RelationDefinition schema	Adopt v2 axes; decide exact enum names/mod visibility field	Needs review
4	Relation mutation	Immediate in controlled lifecycle phases; deferred journal for systems/events/actions	Needs confirmation
5	Scope walk semantics	Explicit-only scalar walks; multi-target is query-only	Needs confirmation
6	QueryPlan IR	Adopt initial op set; defer VM until concrete joins require it	Needs review
7	Execution lanes	NativeEligible / KernelEligible / ManagedOnly	Needs confirmation
8	Host bridge split	Field, command, callable, entity, trace, instrumentation	Needs design
9	Channel/modifier boundary	Kernel owns indexes/deps/buckets; L3 owns value semantics initially	Needs confirmation
10	Save/load	Adopt section list first; choose binary format later	Needs review
11	Hot reload	Adopt diff categories; decide allowed running-session classes	Needs review
12	Deterministic iteration	Choose contract-set fallback order	Needs choice
13	Observability ABI	Opaque marker ids + trace callbacks + ModId + volume policy	Needs review
14	Migration	Contract-first, optional managed reference kernel, native backing after	Needs confirmation
15	Compiler/syntax sequencing	No new syntax until kernel semantics exist	Needs confirmation

---

Recommended Review Packets

Packet A — Kernel Contract

Questions 1-3 and 12. Outcome: name/API tier, 64-bit identity confirmation, relation schema, deterministic iterator defaults.

Packet B — Runtime Semantics

Questions 4-6 and 9. Outcome: command apply phases, scalar walk policy, initial query IR, channel/modifier ownership boundary.

Packet C — Host and Execution

Questions 7-8 and 13. Outcome: execution lane classifier, split bridge method ownership, observability ABI and volume policy.

Packet D — Lifecycle

Questions 10-11 and 14. Outcome: save sections, hot reload categories, migration milestones.

Packet E — Language Sequencing

Question 15. Outcome: explicit commitment that syntax waits until kernel semantics and hand-written stand-ins prove the runtime path.