Enhance Workflow to Support Checkpoint Restart from Failure Points

[Woo0ood]

Self Checks

• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
• Please do not modify this template :) and fill in all the required fields.

1. Is this request related to a challenge you're experiencing? Tell me about your story.

Currently, when a long-running workflow encounters an error and terminates unexpectedly, restarting the workflow requires re-executing all the steps from the beginning, even those that previously completed successfully. This results in a significant amount of time being spent on re-executing known results, which is inefficient and time-consuming.

2. Additional context or comments

We are thrilled with the recent improvements and optimizations Dify has made to handle errors during workflow execution. These enhancements have significantly improved the robustness and reliability of our workflows. However, there is still room for further optimization to enhance the user experience and efficiency.

Proposed Enhancement:

To address this issue, we propose implementing a checkpoint restart mechanism for workflows. This feature would allow workflows to resume execution from the point of failure, using the results of the previously completed steps as inputs and outputs. By doing so, the workflow can continue from the error node onward, saving time and resources.

Benefits:

• Reduced execution time by avoiding redundant processing of previously successful steps.
• Improved efficiency and resource utilization.
• Enhanced user experience by providing a more resilient and fault-tolerant workflow execution process.

This feature will greatly benefit users who rely on long-running workflows, such as video generation, image generation, and large document processing.

We plan to initiate this enhancement and ultimately contribute the code to the Dify community. We hope you will consider our proposal and look forward to your feedback on our future pull requests.🎉

3. Can you help us with this feature?

• I am interested in contributing to this feature.