feat(litellm): Support span streaming#6317
Conversation
Codecov Results 📊✅ 2205 passed | ⏭️ 154 skipped | Total: 2359 | Pass Rate: 93.47% | Execution Time: 8m 34s All tests are passing successfully. ❌ Patch coverage is 0.00%. Project has 13488 uncovered lines. Files with missing lines (1)
Generated by Codecov Action |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 49ec553. Configure here.
| "sentry.op": ( | ||
| consts.OP.GEN_AI_CHAT | ||
| if operation == "chat" | ||
| else consts.OP.GEN_AI_EMBEDDINGS |
There was a problem hiding this comment.
I know this existed in the previous code, but this fallback seems like an interesting choice. I would've expected the inverse where chat is the fallback since I suspect it's the most common operation that would be performed with an AI provider 🤔
| client.completions._client._client, | ||
| "send", | ||
| return_value=model_response, | ||
| ), start_transaction(name="litellm test"): |
There was a problem hiding this comment.
Just double-checking that the start_transaction is intentional here, and that we wouldn't rather move to starting a segment when span_streaming is set to true.
2 participants
Description
Use
sentry_sdk.traces.start_spanand replaceSpan.set_data()withStreamedSpan.set_attribute()when in span streaming mode.Parametrize tests on the trace lifecycle option.
Issues
Closes #6034
Reminders
tox -e linters.feat:,fix:,ref:,meta:)