-
Notifications
You must be signed in to change notification settings - Fork 3.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add Anthropic Models to Cache Prompt #3775
Add Anthropic Models to Cache Prompt #3775
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the fix!
This list we have will be a pain to keep updated... 😅 I hope liteLLM will support this feature in a more generic way in the future. For now it's good to have it working.
This shouldn't be necessary, i.e. the model name should be compared to be in the array entries' name (not equality), so we don't need to add prefixed versions.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please see my comment above.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Thank you for your contribution! 👍
Short description of the problem this fixes or functionality that this introduces. This may be used for the CHANGELOG
Adds support for Anthropic models with 'anthropic/' prefix in cache prompting.
Give a summary of what the PR does, explaining any non-trivial design decisions
This pull request addresses an issue where Anthropic models with the 'anthropic/' prefix were not being correctly identified for cache prompting support. The changes include:
Added two new entries to the
cache_prompting_supported_models
list:Retained the existing entries without the 'anthropic/' prefix to maintain backwards compatibility:
This change ensures that the cache prompting feature correctly supports Anthropic models regardless of whether they are referenced with or without the 'anthropic/' prefix. This dual-entry approach allows for flexibility in model naming conventions while maintaining support for existing configurations.
The decision to keep both prefixed and non-prefixed versions was made to:
Link of any specific issues this addresses
Fixes #3774: Incorrect identification of Anthropic models in prompt cache supported models