Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Support default input index when collect package tools for custom-llm tool #1502

Merged
merged 27 commits into from
Jan 31, 2024

Conversation

chenslucky
Copy link
Contributor

@chenslucky chenslucky commented Dec 18, 2023

Description

Reason

Previously, customers complained that the order of custom-llm tool inputs displayed in the UI was inconsistent with the order defined in tool interface or yaml file. This was because the UI received the inputs as a dict, which resulted in an alphabetical order display.
In this PR, we will automatically add "ui_hints": {"index": XX} to every input based on the order in tool interface or yaml file for custom_llm tool. When UI received tool inputs, they can display the inputs according to this ui_hints["index"] field to make the order consistent. Both portal and extension have completed the code change to support ui_hints["index"].

Cases which will not be corrected by this pr:

  • Existing flow with flow.tools.json
  • All examples in promptflow gallery which also have flow.tools.json file

Test

For more test cases, please go to this link

  • custom-llm tool
    image

    • in portal
      image
    • in extension
      image
  • built in custom-llm tool

    • in portal:
      image
    • in extension:
      image
  • Test cases
    The behavior of the following test cases are as expected. Only custom_llm tool's inputs' order are corrected, other tool's inputs' order keep same as before.

    • Create a new flow
    • Clone a flow
    • Upload a flow without .promptflow folder

All Promptflow Contribution checklist:

  • The pull request does not introduce [breaking changes].
  • CHANGELOG is updated for new features, bug fixes or other significant changes.
  • I have read the contribution guidelines.
  • Create an issue and link to the pull request to get dedicated review from promptflow team. Learn more: suggested workflow.

General Guidelines and Best Practices

  • Title of the pull request is clear and informative.
  • There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

  • Pull request includes test coverage for the included changes.

Copy link

github-actions bot commented Dec 18, 2023

SDK CLI Global Config Test Result chesi/support-input-index

2 tests   2 ✅  45s ⏱️
1 suites  0 💤
1 files    0 ❌

Results for commit af9a1b7.

♻️ This comment has been updated with latest results.

Copy link

github-actions bot commented Dec 18, 2023

Executor Unit Test Result chesi/support-input-index

702 tests   702 ✅  41s ⏱️
  1 suites    0 💤
  1 files      0 ❌

Results for commit af9a1b7.

♻️ This comment has been updated with latest results.

Copy link

github-actions bot commented Dec 18, 2023

promptflow SDK CLI Azure E2E Test Result chesi/support-input-index

  4 files    4 suites   3m 14s ⏱️
128 tests 109 ✅ 19 💤 0 ❌
512 runs  436 ✅ 76 💤 0 ❌

Results for commit af9a1b7.

♻️ This comment has been updated with latest results.

Copy link

github-actions bot commented Dec 18, 2023

Executor E2E Test Result chesi/support-input-index

188 tests   185 ✅  3m 55s ⏱️
  1 suites    3 💤
  1 files      0 ❌

Results for commit af9a1b7.

♻️ This comment has been updated with latest results.

Copy link

github-actions bot commented Dec 18, 2023

SDK CLI Test Result chesi/support-input-index

   12 files     12 suites   37m 42s ⏱️
  416 tests   408 ✅  8 💤 0 ❌
1 664 runs  1 632 ✅ 32 💤 0 ❌

Results for commit af9a1b7.

♻️ This comment has been updated with latest results.

@chenslucky chenslucky requested a review from a team as a code owner December 26, 2023 03:26
update

update
update

update

add debug info

add debug info

add debug info

add debug

delete debug

debug

update

delete debug

add test collect tools from directory

add debug

update

update

delete debug

delete test in promptflow-tools
update

update
@jiazengcindy jiazengcindy merged commit 500eee0 into main Jan 31, 2024
79 checks passed
@jiazengcindy jiazengcindy deleted the chesi/support-input-index branch January 31, 2024 05:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
executor The changes related to the execution of the flow promptflow
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants