Skip to content

Conversation

Njuapp
Copy link
Contributor

@Njuapp Njuapp commented Aug 16, 2022

Signed-off-by: Cheng Hang calvinhance@gmail.com

Description

To support dynamic-batch compilation of swin transformer/BERT, I have added dynamic batch support of the following relevant op converters:

  1. aten::reshape
  2. aten::select
  3. aten::roll
  4. aten::flatten
  5. aten::slice The older version can support dynamic shape, but will report a lot of trt ERRORs, meanwhile compiles successfully. This can be reproduced for TRT 8.4, and it did not appear for TRT 8.2. I suppose it is caused by some new checks performed in TRT8.4.
    7c2cdbf79765ad96768be343ad62acd
    To resolve this issue, I have slightly changed the order of codes for dynamic slice, and it did not change the logics. By such simple manipulation, the ERRORs disappear.

Fixes #1271

Type of change

Please delete options that are not relevant and/or add your own.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

Checklist:

  • My code follows the style guidelines of this project (You can use the linters)
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas and hacks
  • I have made corresponding changes to the documentation
  • I have added tests to verify my fix or my feature
  • New and existing unit tests pass locally with my changes
  • I have added the relevant labels to my PR in so that relevant reviewers are notified

Signed-off-by: Cheng Hang <calvinhance@gmail.com>
@github-actions github-actions bot added component: conversion Issues re: Conversion stage component: converters Issues re: Specific op converters component: core Issues re: The core compiler labels Aug 16, 2022
@narendasan narendasan merged commit c63a5a5 into pytorch:master Aug 16, 2022
@Njuapp Njuapp deleted the dynamic-transformer branch August 19, 2022 07:48
@Njuapp Njuapp restored the dynamic-transformer branch September 19, 2022 06:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla signed component: conversion Issues re: Conversion stage component: converters Issues re: Specific op converters component: core Issues re: The core compiler
Projects
None yet
Development

Successfully merging this pull request may close these issues.

🐛 [Bug] Failure to compile swin/BERT with dynamic batch
3 participants