Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(feat) Add trajectory replay for headless mode #6215

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

li-boxuan
Copy link
Collaborator

@li-boxuan li-boxuan commented Jan 13, 2025

End-user friendly description of the problem this fixes or functionality that this introduces

Add trajectory replay feature for headless mode.

  • Include this change in the Release Notes. If checked, you must provide an end-user friendly description for your change below

Give a summary of what the PR does, explaining any non-trivial design decisions

To test it out, add replay_trajectory_path to config.toml, and then

poetry run python openhands/core/main.py

TODOs in this PR:

TODOs in next PR:

  • support GUI mode
  • add integration tests with trajectory replay

Link of any specific issues this addresses

#6049


To run this PR locally, use the following command:

docker run -it --rm   -p 3000:3000   -v /var/run/docker.sock:/var/run/docker.sock   --add-host host.docker.internal:host-gateway   -e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:0dd31bf-nikolaik   --name openhands-app-0dd31bf   docker.all-hands.dev/all-hands-ai/openhands:0dd31bf

@li-boxuan
Copy link
Collaborator Author

Demo:

Step 1. Run OpenHands and generate a trajectory

workspace_base="./workspace"
save_trajectory_path="./traj/demo.json"

With command: poetry run python openhands/core/main.py -t "Please goto GitHub trending and clone the most popular repo" -l claude

Step 2. Run OpenHands with trajectory replay

workspace_base="./workspace2"
replay_trajectory_path="./traj/demo.json"

With command: poetry run python openhands/core/main.py -l wrongkey (LLM config is not required, here's to showcase that a wrong key doesn't matter)

Step 3. Check the result

Screenshot 2025-01-13 at 11 26 45 PM

@li-boxuan li-boxuan marked this pull request as ready for review January 14, 2025 07:28
@li-boxuan li-boxuan requested review from xingyaoww and enyst January 14, 2025 07:29
@@ -108,6 +109,7 @@ def __init__(
is_delegate: Whether this controller is a delegate.
headless_mode: Whether the agent is run in headless mode.
status_callback: Optional callback function to handle status updates.
replay_logs: A list of logs to replay.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about we call it replay_events, since it is really a list of events

self.replay_logs = replay_logs
self.replay_index = 0
if self.replay_logs:
logger.info(f'Replay logs loaded, events length = {len(self.replay_logs)}')
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we consolidate these replay related thing to a separate class? eg something like StuckDetector to help reduce the complexity of AgentController?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, and it's a very self-contained feature

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Although, it does execute the actions, right?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants