Skip to content

feat!: Add per-execution runId, at-most-once tracking, and cross-process tracker resumption #133

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
jsonbailey wants to merge 22 commits into
base: main
Choose a base branch
from
Open

feat!: Add per-execution runId, at-most-once tracking, and cross-process tracker resumption #133

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
2b997fa
feat!: Add per-execution runId and at-most-once event tracking
jsonbailey Apr 15, 2026
211ead4
feat!: Add per-execution runId, at-most-once tracking, and cross-proc…
jsonbailey Apr 15, 2026
6237d6c
refactor: Move UUID generation from tracker constructor to factory cl…
jsonbailey Apr 16, 2026
d895e64
fix: Fix CI lint errors and add run_id to provider tests
jsonbailey Apr 16, 2026
7839104
refactor: Use LDAIMetricSummary fields as at-most-once guards
jsonbailey Apr 16, 2026
df722f9
refactor: Reorder LDAIConfigTracker __init__ params to match spec
jsonbailey Apr 16, 2026
a6e9612
refactor: Move context before model/provider params, fix resumption t…
jsonbailey Apr 16, 2026
59c574e
chore: Include track data in at-most-once warning logs
jsonbailey Apr 16, 2026
6bf91fa
feat: Add from_resumption_token classmethod to LDAIConfigTracker
jsonbailey Apr 16, 2026
ba5421a
fix: Omit variationKey from track data when empty
jsonbailey Apr 16, 2026
4c0451b
feat: Always set create_tracker as callable that returns a tracker
jsonbailey Apr 17, 2026
82cb40a
feat!: Remove config.tracker field — use create_tracker() instead
jsonbailey Apr 17, 2026
7050ab0
feat: Include graphKey in resumption token when set
jsonbailey Apr 17, 2026
31dcc8f
feat!: Remove tracker param from Judge and ManagedAgentGraph
jsonbailey Apr 17, 2026
08da63a
chore: Use ldai.log instead of logging module in tracker
jsonbailey Apr 17, 2026
ae5d752
fix: Update provider packages for tracker factory pattern
jsonbailey Apr 17, 2026
fc814c6
feat!: Replace AgentGraphDefinition.get_tracker() with create_tracker…
jsonbailey Apr 17, 2026
5313ce5
fix: Cache node trackers per execution to preserve runId correlation
jsonbailey Apr 17, 2026
04f14eb
fix: Create graph tracker once per run, not twice
jsonbailey Apr 17, 2026
dd44577
feat!: Return Result from from_resumption_token instead of raising
jsonbailey Apr 17, 2026
84a1ab1
fix: Type create_tracker as Optional to fix mypy errors
jsonbailey Apr 17, 2026
a41c7e3
feat!: Remove public disabled() classmethod from AIConfigDefault
jsonbailey Apr 17, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions packages/ai-providers/server-ai-langchain/src/ldai_langchain/langgraph_agent_graph_runner.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -260,7 +260,7 @@ def route(state: WorkflowState) -> str:

self._graph.traverse(fn=handle_traversal)

tracker = self._graph.get_tracker()
tracker = self._graph.create_tracker() if self._graph.create_tracker is not None else None
Copy link
Copy Markdown

@cursor cursor bot Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wasted graph tracker created solely for debug logging

Low Severity

_build_graph calls self._graph.create_tracker() and instantiates a full AIGraphTracker just to read its graph_key property for a debug log message. The graph key is readily available from self._graph._agent_graph.key without creating a tracker object. This adds an unnecessary side effect in a method that otherwise only builds the graph structure.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit a41c7e3. Configure here.

graph_key_str = tracker.graph_key if tracker else 'unknown'
log.debug(
f"LangGraphAgentGraphRunner: graph='{graph_key_str}', root='{root_key}', "
Expand All @@ -281,7 +281,7 @@ async def run(self, input: Any) -> AgentGraphResult:
:param input: The string prompt to send to the agent graph
:return: AgentGraphResult with the final output and metrics
"""
tracker = self._graph.get_tracker()
tracker = self._graph.create_tracker() if self._graph.create_tracker is not None else None
start_ns = time.perf_counter_ns()

try:
Expand Down
2 changes: 1 addition & 1 deletion packages/ai-providers/server-ai-langchain/src/ldai_langchain/langgraph_callback_handler.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -200,7 +200,7 @@ def flush(self, graph: AgentGraphDefinition) -> None:
node = graph.get_node(node_key)
if not node:
continue
config_tracker = node.get_config().tracker
config_tracker = node.get_config().create_tracker()
Copy link
Copy Markdown

@cursor cursor bot Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Callback handler creates new tracker per flush, losing runId correlation

Medium Severity

The flush() method calls node.get_config().create_tracker() for each node in the path. In production, this factory (from client.py) generates a new UUID runId each time it's called. This means if the same node's tracker was already obtained elsewhere during the run (e.g., for logging in _build_graph), the flush() creates an entirely separate tracker with a different runId. More importantly, if flush() were ever called more than once, each call would create new trackers with new runIds and emit duplicate events (the at-most-once guards are per-tracker-instance, and each call gets a fresh instance). The OpenAI runner avoids this by caching trackers in _node_trackers, but the LangGraph callback handler has no such caching.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit a41c7e3. Configure here.

if not config_tracker:
continue

Expand Down
2 changes: 0 additions & 2 deletions packages/ai-providers/server-ai-langchain/tests/test_langchain_provider.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -536,7 +536,6 @@ def sync_tool(x: str = '') -> str:
),
provider=ProviderConfig(name='openai'),
instructions='',
tracker=MagicMock(),
)
tools = build_structured_tools(cfg, {'my_tool': sync_tool})
assert len(tools) == 1
Expand All @@ -559,7 +558,6 @@ async def async_tool(x: str = '') -> str:
),
provider=ProviderConfig(name='openai'),
instructions='',
tracker=MagicMock(),
)
tools = build_structured_tools(cfg, {'my_tool': async_tool})
assert len(tools) == 1
Expand Down
8 changes: 4 additions & 4 deletions packages/ai-providers/server-ai-langchain/tests/test_langgraph_agent_graph_runner.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,13 +11,13 @@


def _make_graph(enabled: bool = True) -> AgentGraphDefinition:
graph_tracker = MagicMock()
root_config = AIAgentConfig(
key='root-agent',
enabled=enabled,
model=ModelConfig(name='gpt-4'),
provider=ProviderConfig(name='openai'),
instructions='You are a helpful assistant.',
tracker=MagicMock(),
)
graph_config = AIAgentGraphConfig(
key='test-graph',
Expand All @@ -31,7 +31,7 @@ def _make_graph(enabled: bool = True) -> AgentGraphDefinition:
nodes=nodes,
context=MagicMock(),
enabled=enabled,
tracker=MagicMock(),
create_tracker=lambda: graph_tracker,
)


Expand Down Expand Up @@ -78,7 +78,7 @@ async def test_langgraph_runner_run_raises_when_langgraph_not_installed():
@pytest.mark.asyncio
async def test_langgraph_runner_run_tracks_failure_on_exception():
graph = _make_graph()
tracker = graph.get_tracker()
tracker = graph.create_tracker()
runner = LangGraphAgentGraphRunner(graph, {})

with patch.dict('sys.modules', {'langgraph': None, 'langgraph.graph': None}):
Expand All @@ -92,7 +92,7 @@ async def test_langgraph_runner_run_tracks_failure_on_exception():
@pytest.mark.asyncio
async def test_langgraph_runner_run_success():
graph = _make_graph()
tracker = graph.get_tracker()
tracker = graph.create_tracker()

mock_message = MagicMock()
mock_message.content = "langgraph answer"
Expand Down
23 changes: 12 additions & 11 deletions packages/ai-providers/server-ai-langchain/tests/test_langgraph_callback_handler.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ def _make_graph(mock_ld_client: MagicMock, node_key: str = 'root-agent', graph_k
model_name='gpt-4',
provider_name='openai',
context=context,
run_id='test-run-id',
graph_key=graph_key,
)
graph_tracker = AIGraphTracker(
Expand All @@ -50,7 +51,7 @@ def _make_graph(mock_ld_client: MagicMock, node_key: str = 'root-agent', graph_k
model=ModelConfig(name='gpt-4', parameters={}),
provider=ProviderConfig(name='openai'),
instructions='Be helpful.',
tracker=node_tracker,
create_tracker=lambda: node_tracker,
)
graph_config = AIAgentGraphConfig(
key=graph_key,
Expand All @@ -64,7 +65,7 @@ def _make_graph(mock_ld_client: MagicMock, node_key: str = 'root-agent', graph_k
nodes=nodes,
context=context,
enabled=True,
tracker=graph_tracker,
create_tracker=lambda: graph_tracker,
)


Expand Down Expand Up @@ -320,7 +321,7 @@ def test_flush_emits_token_events_to_ld_tracker():
"""flush() calls track_tokens on the node's config tracker."""
mock_ld_client = MagicMock()
graph = _make_graph(mock_ld_client, node_key='root-agent', graph_key='g1')
tracker = graph.get_tracker()
tracker = graph.create_tracker()

handler = LDMetricsCallbackHandler({'root-agent'}, {})
node_run_id = uuid4()
Expand All @@ -339,7 +340,7 @@ def test_flush_emits_duration():
"""flush() calls track_duration when duration was recorded."""
mock_ld_client = MagicMock()
graph = _make_graph(mock_ld_client)
tracker = graph.get_tracker()
tracker = graph.create_tracker()

handler = LDMetricsCallbackHandler({'root-agent'}, {})
run_id = uuid4()
Expand All @@ -355,7 +356,7 @@ def test_flush_emits_tool_calls():
"""flush() calls track_tool_call for each recorded tool invocation."""
mock_ld_client = MagicMock()
graph = _make_graph(mock_ld_client)
tracker = graph.get_tracker()
tracker = graph.create_tracker()

handler = LDMetricsCallbackHandler({'root-agent'}, {'fn_search': 'search'})
# The agent node must be started first so it appears in the path for flush()
Expand All @@ -377,7 +378,7 @@ def test_flush_includes_graph_key_in_node_events():
"""flush() passes graph_key to the node tracker so graphKey appears in events."""
mock_ld_client = MagicMock()
graph = _make_graph(mock_ld_client, graph_key='my-graph')
tracker = graph.get_tracker()
tracker = graph.create_tracker()

handler = LDMetricsCallbackHandler({'root-agent'}, {})
node_run_id = uuid4()
Expand All @@ -402,14 +403,15 @@ def test_flush_with_no_graph_key_on_node_tracker():
model_name='gpt-4',
provider_name='openai',
context=context,
run_id='test-run-id',
)
node_config = AIAgentConfig(
key='root-agent',
enabled=True,
model=ModelConfig(name='gpt-4', parameters={}),
provider=ProviderConfig(name='openai'),
instructions='Be helpful.',
tracker=node_tracker,
create_tracker=lambda: node_tracker,
)
graph_config = AIAgentGraphConfig(
key='test-graph',
Expand All @@ -423,7 +425,7 @@ def test_flush_with_no_graph_key_on_node_tracker():
nodes=nodes,
context=context,
enabled=True,
tracker=None,
create_tracker=lambda: None,
)

handler = LDMetricsCallbackHandler({'root-agent'}, {})
Expand All @@ -441,7 +443,7 @@ def test_flush_skips_nodes_not_in_path():
"""flush() only emits events for nodes that were actually executed."""
mock_ld_client = MagicMock()
graph = _make_graph(mock_ld_client)
tracker = graph.get_tracker()
tracker = graph.create_tracker()

# Handler with 'root-agent' in node_keys but never started
handler = LDMetricsCallbackHandler({'root-agent'}, {})
Expand All @@ -463,7 +465,6 @@ def test_flush_skips_node_without_tracker():
model=ModelConfig(name='gpt-4', parameters={}),
provider=ProviderConfig(name='openai'),
instructions='',
tracker=None,
)
graph_config = AIAgentGraphConfig(
key='g', root_config_key='no-track', edges=[], enabled=True
Expand All @@ -474,7 +475,7 @@ def test_flush_skips_node_without_tracker():
nodes=nodes,
context=context,
enabled=True,
tracker=None,
create_tracker=lambda: None,
)

handler = LDMetricsCallbackHandler({'no-track'}, {})
Expand Down
Loading
Loading