-
Notifications
You must be signed in to change notification settings - Fork 362
Rename generate/stream/full to generate/full and add filter_steps parameter #164
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Signed-off-by: Anuradha Karuppiah <anuradhak@nvidia.com>
Also add a special filter "none" to suppress all steps Signed-off-by: Anuradha Karuppiah <anuradhak@nvidia.com>
Signed-off-by: Anuradha Karuppiah <anuradhak@nvidia.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This pull request renames the streaming endpoint from "/generate/stream/full" to "/generate/full" and adds a new query parameter, filter_steps, to allow callers to control which intermediate steps are returned.
- The test suite has been updated to reflect the new endpoint.
- The FastAPI response helpers and plugin worker now support the filter_steps parameter.
- Documentation has been updated to instruct users on using filter_steps.
Reviewed Changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.
Show a summary per file
File | Description |
---|---|
tests/aiq/eval/test_remote_evaluate.py | Updated test routes to use "/generate/full" |
src/aiq/front_ends/fastapi/response_helpers.py | Added filter_steps parameter and logic to filter intermediate steps based on allowed types |
src/aiq/front_ends/fastapi/fastapi_front_end_plugin_worker.py | Updated API endpoint functions to include the filter_steps parameter and updated route path |
src/aiq/eval/remote_workflow.py | Updated endpoint string to reflect the renamed route |
docs/source/guides/using-aiqtoolkit-ui-and-server.md | Updated documentation to instruct users on using the new "/generate/full" endpoint with filter_steps |
Comments suppressed due to low confidence (2)
src/aiq/front_ends/fastapi/response_helpers.py:125
- [nitpick] Consider validating or stripping whitespace from the comma-separated filter_steps values to avoid potential issues with unexpected spaces.
def generate_streaming_response_raw(..., output_type: type | None = None, filter_steps: str | None = None) -> AsyncGenerator[AIQResponseSerializable]:
src/aiq/front_ends/fastapi/response_helpers.py:179
- The new parameter 'suppress_steps' is currently not used in the function logic; consider either integrating it into the response generation logic or removing it if unnecessary.
async def generate_streaming_response_raw_as_str(..., filter_steps: str | None = None, suppress_steps: bool = False) -> AsyncGenerator[str]:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR renames the streaming endpoint from "/generate/stream/full" to "/generate/full" and adds a new query parameter (filter_steps) to allow filtering or suppression of intermediate steps in streamed responses. Key changes include updating route paths in tests, front-end plugin worker, and evaluation workflows; and updating response helper functions to handle the new filter_steps parameter.
Reviewed Changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.
Show a summary per file
File | Description |
---|---|
tests/aiq/eval/test_remote_evaluate.py | Updated test routes to use "/generate/full" instead of "/generate/stream/full". |
src/aiq/front_ends/fastapi/response_helpers.py | Updated function signatures to incorporate filter_steps and suppress_steps parameters. |
src/aiq/front_ends/fastapi/fastapi_front_end_plugin_worker.py | Adjusted endpoints and parameter passing to align with the new route and filtering logic. |
src/aiq/eval/remote_workflow.py | Modified the endpoint used for remote evaluation to use "/generate/full". |
docs/source/guides/using-aiqtoolkit-ui-and-server.md | Revised documentation to reflect the renamed endpoint and filter_steps functionality. |
Signed-off-by: Anuradha Karuppiah <anuradhak@nvidia.com>
Signed-off-by: Anuradha Karuppiah <anuradhak@nvidia.com>
…se_full[] Signed-off-by: Anuradha Karuppiah <anuradhak@nvidia.com>
/merge |
…ameter (NVIDIA#164) This pull request renames the streaming endpoint from "/generate/stream/full" to "/generate/full" and adds a new query parameter, filter_steps, to allow callers to control which intermediate steps are returned. The test suite has been updated to reflect the new endpoint. The FastAPI response helpers and plugin worker now support the filter_steps parameter. Documentation has been updated to instruct users on using filter_steps. Sample Usage: ``` # Get all intermediate steps and final output (default behavior) curl --request POST \ --url http://localhost:8000/generate/full \ --header 'Content-Type: application/json' \ --data '{ "input_message": "What is LangSmith?" }' # Suppress all intermediate steps (only get final output) curl --request POST \ --url "http://localhost:8000/generate/full?filter_steps=none" \ --header 'Content-Type: application/json' \ --data '{ "input_message": "What is LangSmith?" }' # Get only specific step types curl --request POST \ --url "http://localhost:8000/generate/full?filter_steps=LLM_END,TOOL_END" \ --header 'Content-Type: application/json' \ --data '{ "input_message": "What is LangSmith?" }' ``` ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/AIQToolkit/blob/develop/docs/source/advanced/contributing.md). - We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license. - Any contribution which contains commits that are not Signed-Off will not be accepted. - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. Authors: - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah) Approvers: - Eric Evans II (https://github.com/ericevans-nv) URL: NVIDIA#164 Signed-off-by: Yuchen Zhang <134643420+yczhang-nv@users.noreply.github.com>
…ameter (NVIDIA#164) This pull request renames the streaming endpoint from "/generate/stream/full" to "/generate/full" and adds a new query parameter, filter_steps, to allow callers to control which intermediate steps are returned. The test suite has been updated to reflect the new endpoint. The FastAPI response helpers and plugin worker now support the filter_steps parameter. Documentation has been updated to instruct users on using filter_steps. Sample Usage: ``` # Get all intermediate steps and final output (default behavior) curl --request POST \ --url http://localhost:8000/generate/full \ --header 'Content-Type: application/json' \ --data '{ "input_message": "What is LangSmith?" }' # Suppress all intermediate steps (only get final output) curl --request POST \ --url "http://localhost:8000/generate/full?filter_steps=none" \ --header 'Content-Type: application/json' \ --data '{ "input_message": "What is LangSmith?" }' # Get only specific step types curl --request POST \ --url "http://localhost:8000/generate/full?filter_steps=LLM_END,TOOL_END" \ --header 'Content-Type: application/json' \ --data '{ "input_message": "What is LangSmith?" }' ``` ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/AIQToolkit/blob/develop/docs/source/advanced/contributing.md). - We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license. - Any contribution which contains commits that are not Signed-Off will not be accepted. - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. Authors: - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah) Approvers: - Eric Evans II (https://github.com/ericevans-nv) URL: NVIDIA#164 Signed-off-by: Eric Evans <194135482+ericevans-nv@users.noreply.github.com>
…ameter (NVIDIA#164) This pull request renames the streaming endpoint from "/generate/stream/full" to "/generate/full" and adds a new query parameter, filter_steps, to allow callers to control which intermediate steps are returned. The test suite has been updated to reflect the new endpoint. The FastAPI response helpers and plugin worker now support the filter_steps parameter. Documentation has been updated to instruct users on using filter_steps. Sample Usage: ``` # Get all intermediate steps and final output (default behavior) curl --request POST \ --url http://localhost:8000/generate/full \ --header 'Content-Type: application/json' \ --data '{ "input_message": "What is LangSmith?" }' # Suppress all intermediate steps (only get final output) curl --request POST \ --url "http://localhost:8000/generate/full?filter_steps=none" \ --header 'Content-Type: application/json' \ --data '{ "input_message": "What is LangSmith?" }' # Get only specific step types curl --request POST \ --url "http://localhost:8000/generate/full?filter_steps=LLM_END,TOOL_END" \ --header 'Content-Type: application/json' \ --data '{ "input_message": "What is LangSmith?" }' ``` ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/AIQToolkit/blob/develop/docs/source/advanced/contributing.md). - We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license. - Any contribution which contains commits that are not Signed-Off will not be accepted. - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. Authors: - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah) Approvers: - Eric Evans II (https://github.com/ericevans-nv) URL: NVIDIA#164 Signed-off-by: Eric Evans <194135482+ericevans-nv@users.noreply.github.com>
…ameter (NVIDIA#164) This pull request renames the streaming endpoint from "/generate/stream/full" to "/generate/full" and adds a new query parameter, filter_steps, to allow callers to control which intermediate steps are returned. The test suite has been updated to reflect the new endpoint. The FastAPI response helpers and plugin worker now support the filter_steps parameter. Documentation has been updated to instruct users on using filter_steps. Sample Usage: ``` # Get all intermediate steps and final output (default behavior) curl --request POST \ --url http://localhost:8000/generate/full \ --header 'Content-Type: application/json' \ --data '{ "input_message": "What is LangSmith?" }' # Suppress all intermediate steps (only get final output) curl --request POST \ --url "http://localhost:8000/generate/full?filter_steps=none" \ --header 'Content-Type: application/json' \ --data '{ "input_message": "What is LangSmith?" }' # Get only specific step types curl --request POST \ --url "http://localhost:8000/generate/full?filter_steps=LLM_END,TOOL_END" \ --header 'Content-Type: application/json' \ --data '{ "input_message": "What is LangSmith?" }' ``` ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/AIQToolkit/blob/develop/docs/source/advanced/contributing.md). - We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license. - Any contribution which contains commits that are not Signed-Off will not be accepted. - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. Authors: - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah) Approvers: - Eric Evans II (https://github.com/ericevans-nv) URL: NVIDIA#164
…ameter (NVIDIA#164) This pull request renames the streaming endpoint from "/generate/stream/full" to "/generate/full" and adds a new query parameter, filter_steps, to allow callers to control which intermediate steps are returned. The test suite has been updated to reflect the new endpoint. The FastAPI response helpers and plugin worker now support the filter_steps parameter. Documentation has been updated to instruct users on using filter_steps. Sample Usage: ``` # Get all intermediate steps and final output (default behavior) curl --request POST \ --url http://localhost:8000/generate/full \ --header 'Content-Type: application/json' \ --data '{ "input_message": "What is LangSmith?" }' # Suppress all intermediate steps (only get final output) curl --request POST \ --url "http://localhost:8000/generate/full?filter_steps=none" \ --header 'Content-Type: application/json' \ --data '{ "input_message": "What is LangSmith?" }' # Get only specific step types curl --request POST \ --url "http://localhost:8000/generate/full?filter_steps=LLM_END,TOOL_END" \ --header 'Content-Type: application/json' \ --data '{ "input_message": "What is LangSmith?" }' ``` ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/AIQToolkit/blob/develop/docs/source/advanced/contributing.md). - We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license. - Any contribution which contains commits that are not Signed-Off will not be accepted. - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. Authors: - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah) Approvers: - Eric Evans II (https://github.com/ericevans-nv) URL: NVIDIA#164
Description
This pull request renames the streaming endpoint from "/generate/stream/full" to "/generate/full" and adds a new query parameter, filter_steps, to allow callers to control which intermediate steps are returned.
The test suite has been updated to reflect the new endpoint.
The FastAPI response helpers and plugin worker now support the filter_steps parameter.
Documentation has been updated to instruct users on using filter_steps.
Sample Usage:
By Submitting this PR I confirm: