Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Chore: clean up LLM (prompt caching, supports fn calling), leftover renames #6095

Open
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

enyst
Copy link
Collaborator

@enyst enyst commented Jan 6, 2025

Give a summary of what the PR does, explaining any non-trivial design decisions

This PR does a few clean-ups/leftovers

  • Anthropic prompt caching is no longer in beta, so the header is not needed
  • max cache markers is 4, remove potential 5th
  • compute whether native function calling is enabled only once per LLM instance
  • refactor out Cloudflare exception from llm.py

Also

  • remove leak from runtime
  • some leftover renaming eventstream->docker (for the runtime)

Link of any specific issues this addresses
Fix #4484
Fix #6085

To run this PR locally, use the following command:

docker run -it --rm   -p 3000:3000   -v /var/run/docker.sock:/var/run/docker.sock   --add-host host.docker.internal:host-gateway   -e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:5b795d1-nikolaik   --name openhands-app-5b795d1   docker.all-hands.dev/all-hands-ai/openhands:5b795d1

@enyst enyst marked this pull request as draft January 6, 2025 22:39
@enyst enyst marked this pull request as ready for review January 6, 2025 23:34
@enyst enyst requested review from xingyaoww and neubig January 6, 2025 23:35
@xingyaoww xingyaoww self-assigned this Jan 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug]: Export GitHub is not hidden from the agent (feat) Improve retry_decorator with blacklist exceptions
2 participants