Update Google Gemini embedding model from text-embedding-004 to gemin…#433
Closed
mariadb-CalebTerry wants to merge 11 commits intomariadb-corporation:mainfrom
Closed
Conversation
…o configurable env var exists anywhere in the codebase. Gemini 2.5 Flash's thinking/reasoning mode consumes ~478 of those 500 tokens internally, leaving only ~19 tokens for actual text output → truncated answers. The fix is simple: Change LLM_MODEL from gemini-2.5-flash to gemini-2.5-flash-lite: gemini-2.5-flash-lite has no thinking/reasoning tokens — all 500 token budget goes to text output Same API key, same LiteLLM path, no other changes required Faster and cheaper than 2.5-flash
Contributor
|
In config.env.template, we need to have Database configuration details @mariadb-CalebTerry |
…rise Server and, of 13.0, Community Server).
Contributor
Author
@mariadb-tauseefkhan Neither of my commits removed those. Looks like this commit to rebase main onto this branch brought those changes in: |
…o configurable env var exists anywhere in the codebase. Gemini 2.5 Flash's thinking/reasoning mode consumes ~478 of those 500 tokens internally, leaving only ~19 tokens for actual text output → truncated answers. The fix is simple: Change LLM_MODEL from gemini-2.5-flash to gemini-2.5-flash-lite: gemini-2.5-flash-lite has no thinking/reasoning tokens — all 500 token budget goes to text output Same API key, same LiteLLM path, no other changes required Faster and cheaper than 2.5-flash
…b.com/mariadb-CalebTerry/mariadb-docs into docs/text-embedding-004-is-deprecated
Contributor
Author
|
I'll abandon this PR and create a new one for the proposed changes. I think some commits were made directly against main, which I didn't have in my branch yesterday. |
Contributor
|
That's a good idea because the launch is around the corner for AI RAG. I am adding too many changes to the section. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
text-embedding-004withgemini-embedding-001due to deprecation oftext-embedding-004