Skip to content

Conversation

AnuradhaKaruppiah
Copy link
Contributor

@AnuradhaKaruppiah AnuradhaKaruppiah commented Apr 29, 2025

Description

This pull request renames the streaming endpoint from "/generate/stream/full" to "/generate/full" and adds a new query parameter, filter_steps, to allow callers to control which intermediate steps are returned.

The test suite has been updated to reflect the new endpoint.
The FastAPI response helpers and plugin worker now support the filter_steps parameter.
Documentation has been updated to instruct users on using filter_steps.

Sample Usage:


# Get all intermediate steps and final output (default behavior)
curl --request POST \
  --url http://localhost:8000/generate/full \
  --header 'Content-Type: application/json' \
  --data '{
    "input_message": "What is LangSmith?"
}'

# Suppress all intermediate steps (only get final output)
curl --request POST \
  --url "http://localhost:8000/generate/full?filter_steps=none" \
  --header 'Content-Type: application/json' \
  --data '{
    "input_message": "What is LangSmith?"
}'

# Get only specific step types
curl --request POST \
  --url "http://localhost:8000/generate/full?filter_steps=LLM_END,TOOL_END" \
  --header 'Content-Type: application/json' \
  --data '{
    "input_message": "What is LangSmith?"
}'

By Submitting this PR I confirm:

  • I am familiar with the Contributing Guidelines.
  • We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license.
    • Any contribution which contains commits that are not Signed-Off will not be accepted.
  • When the PR is ready for review, new or existing tests cover these changes.
  • When the PR is ready for review, the documentation is up to date with these changes.

Signed-off-by: Anuradha Karuppiah <anuradhak@nvidia.com>
@AnuradhaKaruppiah AnuradhaKaruppiah self-assigned this Apr 29, 2025
@AnuradhaKaruppiah AnuradhaKaruppiah added improvement Improvement to existing functionality non-breaking Non-breaking change labels Apr 29, 2025
@AnuradhaKaruppiah AnuradhaKaruppiah marked this pull request as draft April 29, 2025 20:54
Also add a special filter "none" to suppress all steps

Signed-off-by: Anuradha Karuppiah <anuradhak@nvidia.com>
Signed-off-by: Anuradha Karuppiah <anuradhak@nvidia.com>
@AnuradhaKaruppiah AnuradhaKaruppiah changed the title Rename generate/stream/full to generate/full and add query parameter Rename generate/stream/full to generate/full and add filter_steps parameter Apr 29, 2025
@AnuradhaKaruppiah AnuradhaKaruppiah marked this pull request as ready for review April 29, 2025 21:54
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This pull request renames the streaming endpoint from "/generate/stream/full" to "/generate/full" and adds a new query parameter, filter_steps, to allow callers to control which intermediate steps are returned.

  • The test suite has been updated to reflect the new endpoint.
  • The FastAPI response helpers and plugin worker now support the filter_steps parameter.
  • Documentation has been updated to instruct users on using filter_steps.

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.

Show a summary per file
File Description
tests/aiq/eval/test_remote_evaluate.py Updated test routes to use "/generate/full"
src/aiq/front_ends/fastapi/response_helpers.py Added filter_steps parameter and logic to filter intermediate steps based on allowed types
src/aiq/front_ends/fastapi/fastapi_front_end_plugin_worker.py Updated API endpoint functions to include the filter_steps parameter and updated route path
src/aiq/eval/remote_workflow.py Updated endpoint string to reflect the renamed route
docs/source/guides/using-aiqtoolkit-ui-and-server.md Updated documentation to instruct users on using the new "/generate/full" endpoint with filter_steps
Comments suppressed due to low confidence (2)

src/aiq/front_ends/fastapi/response_helpers.py:125

  • [nitpick] Consider validating or stripping whitespace from the comma-separated filter_steps values to avoid potential issues with unexpected spaces.
def generate_streaming_response_raw(..., output_type: type | None = None, filter_steps: str | None = None) -> AsyncGenerator[AIQResponseSerializable]:

src/aiq/front_ends/fastapi/response_helpers.py:179

  • The new parameter 'suppress_steps' is currently not used in the function logic; consider either integrating it into the response generation logic or removing it if unnecessary.
async def generate_streaming_response_raw_as_str(..., filter_steps: str | None = None, suppress_steps: bool = False) -> AsyncGenerator[str]:

Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR renames the streaming endpoint from "/generate/stream/full" to "/generate/full" and adds a new query parameter (filter_steps) to allow filtering or suppression of intermediate steps in streamed responses. Key changes include updating route paths in tests, front-end plugin worker, and evaluation workflows; and updating response helper functions to handle the new filter_steps parameter.

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
tests/aiq/eval/test_remote_evaluate.py Updated test routes to use "/generate/full" instead of "/generate/stream/full".
src/aiq/front_ends/fastapi/response_helpers.py Updated function signatures to incorporate filter_steps and suppress_steps parameters.
src/aiq/front_ends/fastapi/fastapi_front_end_plugin_worker.py Adjusted endpoints and parameter passing to align with the new route and filtering logic.
src/aiq/eval/remote_workflow.py Modified the endpoint used for remote evaluation to use "/generate/full".
docs/source/guides/using-aiqtoolkit-ui-and-server.md Revised documentation to reflect the renamed endpoint and filter_steps functionality.

Signed-off-by: Anuradha Karuppiah <anuradhak@nvidia.com>
Signed-off-by: Anuradha Karuppiah <anuradhak@nvidia.com>
…se_full[]

Signed-off-by: Anuradha Karuppiah <anuradhak@nvidia.com>
@AnuradhaKaruppiah
Copy link
Contributor Author

/merge

@rapids-bot rapids-bot bot merged commit 2f67b00 into NVIDIA:develop Apr 30, 2025
10 checks passed
@AnuradhaKaruppiah AnuradhaKaruppiah deleted the generate-full branch May 6, 2025 00:48
yczhang-nv pushed a commit to yczhang-nv/NeMo-Agent-Toolkit that referenced this pull request May 8, 2025
…ameter (NVIDIA#164)

This pull request renames the streaming endpoint from "/generate/stream/full" to "/generate/full" and adds a new query parameter, filter_steps, to allow callers to control which intermediate steps are returned.

The test suite has been updated to reflect the new endpoint.
The FastAPI response helpers and plugin worker now support the filter_steps parameter.
Documentation has been updated to instruct users on using filter_steps.

Sample Usage:
```

# Get all intermediate steps and final output (default behavior)
curl --request POST \
  --url http://localhost:8000/generate/full \
  --header 'Content-Type: application/json' \
  --data '{
    "input_message": "What is LangSmith?"
}'

# Suppress all intermediate steps (only get final output)
curl --request POST \
  --url "http://localhost:8000/generate/full?filter_steps=none" \
  --header 'Content-Type: application/json' \
  --data '{
    "input_message": "What is LangSmith?"
}'

# Get only specific step types
curl --request POST \
  --url "http://localhost:8000/generate/full?filter_steps=LLM_END,TOOL_END" \
  --header 'Content-Type: application/json' \
  --data '{
    "input_message": "What is LangSmith?"
}'
```

## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/AIQToolkit/blob/develop/docs/source/advanced/contributing.md).
- We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license.
  - Any contribution which contains commits that are not Signed-Off will not be accepted.
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.

Authors:
  - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah)

Approvers:
  - Eric Evans II (https://github.com/ericevans-nv)

URL: NVIDIA#164
Signed-off-by: Yuchen Zhang <134643420+yczhang-nv@users.noreply.github.com>
ericevans-nv pushed a commit to ericevans-nv/agent-iq that referenced this pull request Jun 3, 2025
…ameter (NVIDIA#164)

This pull request renames the streaming endpoint from "/generate/stream/full" to "/generate/full" and adds a new query parameter, filter_steps, to allow callers to control which intermediate steps are returned.

The test suite has been updated to reflect the new endpoint.
The FastAPI response helpers and plugin worker now support the filter_steps parameter.
Documentation has been updated to instruct users on using filter_steps.

Sample Usage:
```

# Get all intermediate steps and final output (default behavior)
curl --request POST \
  --url http://localhost:8000/generate/full \
  --header 'Content-Type: application/json' \
  --data '{
    "input_message": "What is LangSmith?"
}'

# Suppress all intermediate steps (only get final output)
curl --request POST \
  --url "http://localhost:8000/generate/full?filter_steps=none" \
  --header 'Content-Type: application/json' \
  --data '{
    "input_message": "What is LangSmith?"
}'

# Get only specific step types
curl --request POST \
  --url "http://localhost:8000/generate/full?filter_steps=LLM_END,TOOL_END" \
  --header 'Content-Type: application/json' \
  --data '{
    "input_message": "What is LangSmith?"
}'
```

## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/AIQToolkit/blob/develop/docs/source/advanced/contributing.md).
- We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license.
  - Any contribution which contains commits that are not Signed-Off will not be accepted.
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.

Authors:
  - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah)

Approvers:
  - Eric Evans II (https://github.com/ericevans-nv)

URL: NVIDIA#164
Signed-off-by: Eric Evans <194135482+ericevans-nv@users.noreply.github.com>
ericevans-nv pushed a commit to ericevans-nv/agent-iq that referenced this pull request Jun 3, 2025
…ameter (NVIDIA#164)

This pull request renames the streaming endpoint from "/generate/stream/full" to "/generate/full" and adds a new query parameter, filter_steps, to allow callers to control which intermediate steps are returned.

The test suite has been updated to reflect the new endpoint.
The FastAPI response helpers and plugin worker now support the filter_steps parameter.
Documentation has been updated to instruct users on using filter_steps.

Sample Usage:
```

# Get all intermediate steps and final output (default behavior)
curl --request POST \
  --url http://localhost:8000/generate/full \
  --header 'Content-Type: application/json' \
  --data '{
    "input_message": "What is LangSmith?"
}'

# Suppress all intermediate steps (only get final output)
curl --request POST \
  --url "http://localhost:8000/generate/full?filter_steps=none" \
  --header 'Content-Type: application/json' \
  --data '{
    "input_message": "What is LangSmith?"
}'

# Get only specific step types
curl --request POST \
  --url "http://localhost:8000/generate/full?filter_steps=LLM_END,TOOL_END" \
  --header 'Content-Type: application/json' \
  --data '{
    "input_message": "What is LangSmith?"
}'
```

## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/AIQToolkit/blob/develop/docs/source/advanced/contributing.md).
- We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license.
  - Any contribution which contains commits that are not Signed-Off will not be accepted.
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.

Authors:
  - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah)

Approvers:
  - Eric Evans II (https://github.com/ericevans-nv)

URL: NVIDIA#164
Signed-off-by: Eric Evans <194135482+ericevans-nv@users.noreply.github.com>
AnuradhaKaruppiah added a commit to AnuradhaKaruppiah/oss-agentiq that referenced this pull request Aug 4, 2025
…ameter (NVIDIA#164)

This pull request renames the streaming endpoint from "/generate/stream/full" to "/generate/full" and adds a new query parameter, filter_steps, to allow callers to control which intermediate steps are returned.

The test suite has been updated to reflect the new endpoint.
The FastAPI response helpers and plugin worker now support the filter_steps parameter.
Documentation has been updated to instruct users on using filter_steps.

Sample Usage:
```

# Get all intermediate steps and final output (default behavior)
curl --request POST \
  --url http://localhost:8000/generate/full \
  --header 'Content-Type: application/json' \
  --data '{
    "input_message": "What is LangSmith?"
}'

# Suppress all intermediate steps (only get final output)
curl --request POST \
  --url "http://localhost:8000/generate/full?filter_steps=none" \
  --header 'Content-Type: application/json' \
  --data '{
    "input_message": "What is LangSmith?"
}'

# Get only specific step types
curl --request POST \
  --url "http://localhost:8000/generate/full?filter_steps=LLM_END,TOOL_END" \
  --header 'Content-Type: application/json' \
  --data '{
    "input_message": "What is LangSmith?"
}'
```

## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/AIQToolkit/blob/develop/docs/source/advanced/contributing.md).
- We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license.
  - Any contribution which contains commits that are not Signed-Off will not be accepted.
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.

Authors:
  - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah)

Approvers:
  - Eric Evans II (https://github.com/ericevans-nv)

URL: NVIDIA#164
scheckerNV pushed a commit to scheckerNV/aiq-factory-reset that referenced this pull request Aug 22, 2025
…ameter (NVIDIA#164)

This pull request renames the streaming endpoint from "/generate/stream/full" to "/generate/full" and adds a new query parameter, filter_steps, to allow callers to control which intermediate steps are returned.

The test suite has been updated to reflect the new endpoint.
The FastAPI response helpers and plugin worker now support the filter_steps parameter.
Documentation has been updated to instruct users on using filter_steps.

Sample Usage:
```

# Get all intermediate steps and final output (default behavior)
curl --request POST \
  --url http://localhost:8000/generate/full \
  --header 'Content-Type: application/json' \
  --data '{
    "input_message": "What is LangSmith?"
}'

# Suppress all intermediate steps (only get final output)
curl --request POST \
  --url "http://localhost:8000/generate/full?filter_steps=none" \
  --header 'Content-Type: application/json' \
  --data '{
    "input_message": "What is LangSmith?"
}'

# Get only specific step types
curl --request POST \
  --url "http://localhost:8000/generate/full?filter_steps=LLM_END,TOOL_END" \
  --header 'Content-Type: application/json' \
  --data '{
    "input_message": "What is LangSmith?"
}'
```

## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/AIQToolkit/blob/develop/docs/source/advanced/contributing.md).
- We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license.
  - Any contribution which contains commits that are not Signed-Off will not be accepted.
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.

Authors:
  - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah)

Approvers:
  - Eric Evans II (https://github.com/ericevans-nv)

URL: NVIDIA#164
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
improvement Improvement to existing functionality non-breaking Non-breaking change
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants