[SDK] Support streaming in eager mode #2248

D-W- · 2024-03-07T07:38:52Z

Description

Please add an informative description that covers that changes made by the pull request and link all relevant issues.

This pull request involves changes across multiple files in the src/promptflow directory, primarily focusing on improving generator output handling and testing. The main changes include the introduction of generator output handling in test_submitter.py and _script_executor.py, addition of new test cases for generator outputs, and modification of existing methods to support generator outputs.

Generator output handling:

src/promptflow/promptflow/_sdk/_submitter/test_submitter.py: The flow_test method has been simplified to always call _get_generator_outputs on line_result.output [1]. The _get_generator_outputs method has been expanded to handle both dictionary and generator type outputs [2].
src/promptflow/promptflow/executor/_script_executor.py: Imported GeneratorType from the types module [1]. Updated the exec_line and _exec_line methods to accept a new allow_generator_output parameter [2]. Depending on the value of allow_generator_output, the output is either stringified or left as is [3]. Added a new _stringify_generator_output method to convert generator output to a string [4].

Test case additions and modifications:

src/promptflow/tests/sdk_cli_test/conftest.py: Added a new fixture stream_output for testing generator outputs.
src/promptflow/tests/sdk_cli_test/e2etests/test_flow_serve.py: Added a new test case test_eager_flow_stream_output to test different response types for generator outputs.
src/promptflow/tests/sdk_cli_test/e2etests/test_flow_test.py: Added two new test cases test_eager_flow_stream_output and test_stream_output_with_builtin_llm to test generator outputs in eager flows and with built-in language models respectively.

Additional changes:

src/promptflow/promptflow/_sdk/operations/_flow_operations.py: A comment has been added in the _test method to indicate that is_chat_flow is set to True to allow generator output.
src/promptflow/tests/test_configs/eager_flows/builtin_llm/builtin_call.py: The flow_entry function has been updated to pass a stream parameter to the chat function.
src/promptflow/tests/test_configs/eager_flows/stream_output/stream_output.py: A new Python file has been added that defines a generator function my_flow.

All Promptflow Contribution checklist:

The pull request does not introduce [breaking changes].
CHANGELOG is updated for new features, bug fixes or other significant changes.
I have read the contribution guidelines.
Create an issue and link to the pull request to get dedicated review from promptflow team. Learn more: suggested workflow.

General Guidelines and Best Practices

Title of the pull request is clear and informative.
There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

Pull request includes test coverage for the included changes.

github-actions · 2024-03-07T07:42:24Z