[Bug]: Agent finishes before tasks are fully complete

[aditya-choudhary599]

Is there an existing issue for the same bug?

• I have checked the existing issues.

Describe the bug and reproduction steps

Description:

When the prompt to build a Conversational CRM Application using the MERN stack, augmented with Python for NLP capabilities was provided, the generated code failed to meet expectations. The output was incomplete, lacked functionality, and did not adhere to the requirements specified in the prompt.

Prompt Used:

Build a Conversational CRM Application using the MERN stack, augmented with Python for NLP capabilities, to provide a modern, user-friendly interface for managing sales and customer interactions.
Requirements:

• Chatbot Interface: Develop a chatbot or virtual assistant as the core feature, enabling users to interact conversationally. It should support tasks like asking questions, scheduling meetings, and managing sales activities through chat.
• Lead and Customer Management: Ensure the chatbot can update customer records, add new leads, and manage tasks based on natural conversational inputs.
• Automated Follow-ups: Implement automated reminders and follow-up emails triggered by user interactions, improving workflow efficiency.
• Reporting and Analytics: Create a system to generate and present sales reports and analytics through conversational commands, simplifying data access for users.
• Third-Party Tool Integration: Enable integration with tools like email, calendars, and external CRMs to provide a seamless experience.
  Tech Stack:
• MongoDB: Use MongoDB to store customer data, chat history, and related information.
• Express.js: Develop the backend for handling API requests and bridging the front-end with chatbot services.
• React.js: Build a dynamic and engaging user interface for the chatbot and CRM functionalities.
• Node.js: Implement the server-side logic, manage APIs, and integrate the chatbot engine.
• Python: Use Python for NLP tasks, powering the chatbot's natural language understanding and conversational flow.
  Objective:
  The application should deliver a conversational approach to CRM, transforming the way sales teams and customers interact while boosting productivity and improving the user experience.

Expected Behavior:

The generated code should have included:

1. A fully functional and structured MERN stack application.
2. A chatbot interface integrated with Python for NLP tasks, capable of handling conversational inputs and performing CRM tasks.
3. Business logic for lead and customer management, automated follow-ups, reporting and analytics, and third-party tool integration.
4. All necessary files, with non-empty, functional implementations for both backend and frontend components.
5. A clear setup process to ensure the code is runnable without requiring significant manual intervention.

Observed Behavior:

1. Many files were empty or contained only placeholder comments.
2. Core business logic, including chatbot functionality, customer management, and analytics, was absent.
3. The code was not runnable due to missing dependencies, configurations, and incomplete implementations.
4. Generated files lacked proper folder structure or adherence to MERN stack conventions.
5. Python NLP integration was either missing or implemented superficially, without demonstrating any meaningful capabilities.

OpenHands Installation

Development workflow

OpenHands Version

No response

Operating System

Linux

Logs, Errors, Screenshots, and Additional Context

sales-crm.tar.gz

[enyst]

@aditya-choudhary599 you can give the agent this prompt, full with expectations and feedback, and tell it to fix it.

The LLM decides when to stop, in normal behavior, not openhands: the LLM may think it was finished before it should.

[neubig]

I agree that the LLM decides when to stop, but I think we need to have a better mechanism to double-check that the user's task has actually been fully completed.

[neubig]

Also highly related to #2221