Skip to content

add basic regex auto-reply support #257691

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 16 commits into
base: main
Choose a base branch
from
Open

add basic regex auto-reply support #257691

wants to merge 16 commits into from

Conversation

meganrogge
Copy link
Contributor

@meganrogge meganrogge commented Jul 24, 2025

fixes https://github.com/microsoft/vscode-copilot/issues/16430

Have a new issue #257726 to track:

  • multiple prompts (include this in output polling instead of after)
  • support for foreground terminals
  • delayed prompts
  • ask the llm to determine options vs using regex
  • consider automatically replying

@meganrogge meganrogge requested a review from Tyriar July 24, 2025 16:26
@meganrogge meganrogge self-assigned this Jul 24, 2025
@meganrogge meganrogge added this to the July 2025 milestone Jul 24, 2025
@meganrogge meganrogge enabled auto-merge (squash) July 24, 2025 17:12
Tyriar
Tyriar previously requested changes Jul 24, 2025
Comment on lines 125 to 141
const userInputKind = await getExpectedUserInputKind(outputAndIdle.output);
if (userInputKind) {
const handleResult = await handleYesNoUserPrompt(
userInputKind,
invocation.context,
this._chatService,
terminal,
async () => await pollForOutputAndIdle({ getOutput: () => getOutput(terminal), isActive: () => this._isTaskActive(task) }, true, token, this._languageModelsService),
);
if (handleResult.handled) {
if (handleResult.outputAndIdle) {
outputAndIdle = handleResult.outputAndIdle;
}
} else {
return { content: [{ kind: 'text', value: `The task is still running and requires user input of ${userInputKind}.` }], toolResultMessage: new MarkdownString(localize('copilotChat.taskRequiresUserInput', 'The task `{0}` is still running and requires user input. {1}', task._label, userInputKind)) };
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sharing this between the 3 tools would be good. This could live in elicitation.ts with the prompt calls too?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can't put that in elicitation.ts as there are very specific terminal and task things in this function like outputAndIdle, but moved to bufferOutputPolling

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done though in terms of consolidating them

Copy link
Member

@Tyriar Tyriar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pushed 98e45b3 to add the events we want to collect for this. From my testing the userResponse variable doesn't actually work currently.

When I pulled the code I also found it pretty hard to follow. I think it's a little late to push something so large that has the potential to hang the tool due to async logic, especially without any tests.

Since it's an edge case (bg terminals), doesn't handle multiple questions and just does basic regex, I recommend waiting and implementing it in a more robust way next month. It's not worth the risk considering how little this feature actually covers in its current state. What we need is:

  • Some output monitoring class that RunInTerminalTool hands ownership of the execution to and then it has a loop that does all the idle/question polling and handling in a central place (not inside RunInTerminalTool.invoke)
  • This component must have a good set of unit tests
  • It should return a well structured return object as handleTerminalUserInputPrompt's return result is quite messy right now.

If anything the hint with an a11y alert as I suggested earlier is the safer approach as it's just adding a low risk hint to tell the user the terminal is awaiting input.

token: CancellationToken,
languageModelsService: ILanguageModelsService
): Promise<{ handled: boolean; userResponse?: 'accepted' | 'rejected'; outputAndIdle: { output: string; terminalExecutionIdleBeforeTimeout: boolean; pollDurationMs?: number; modelOutputEvalResponse?: string }; message?: IToolResult }> {
const userInputKind = await getExpectedUserInputKind(outputAndIdle.output);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
const userInputKind = await getExpectedUserInputKind(outputAndIdle.output);
const userInputKind = getExpectedUserInputKind(outputAndIdle.output);

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants