feat (//cpp): Using atol and rtol based tolerance threshold for torchtrtc #1052

andi4191 · 2022-05-10T21:45:53Z

Description

Added rtol + atol tolerance check for torchtrtc. Similar to what we have for test benchmarks.

This PR is a replica of #1038

Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

Fixes # (issue)
#1030

Type of change

Please delete options that are not relevant and/or add your own.

Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist:

My code follows the style guidelines of this project (You can use the linters)
I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas and hacks
I have made corresponding changes to the documentation
I have added tests to verify my fix or my feature
New and existing unit tests pass locally with my changes

github-actions

There are some changes that do not conform to C++ style guidelines:

diff --git a/workspace/cpp/bin/torchtrtc/main.cpp b/tmp/changes.txt
index e5a6c28..48d9eff 100644
--- a/workspace/cpp/bin/torchtrtc/main.cpp
+++ b/tmp/changes.txt
@@ -446,8 +446,8 @@ int main(int argc, char** argv) {
          threshold_ss << "atol: " << atol_val << " rtol: " << rtol_val;
          torchtrt::logging::log(
              torchtrt::logging::Level::kWARNING,
-              std::string("Maximum numerical deviation for output exceeds tolerance thresholds (") + threshold_ss.str() +
-                  std::string(")"));
+              std::string("Maximum numerical deviation for output exceeds tolerance thresholds (") +
+                  threshold_ss.str() + std::string(")"));
        }
      }
    } else {
ERROR: Some files do not conform to style guidelines

github-actions

There are some changes that do not conform to C++ style guidelines:

diff --git a/workspace/cpp/bin/torchtrtc/main.cpp b/tmp/changes.txt
index e5a6c28..48d9eff 100644
--- a/workspace/cpp/bin/torchtrtc/main.cpp
+++ b/tmp/changes.txt
@@ -446,8 +446,8 @@ int main(int argc, char** argv) {
          threshold_ss << "atol: " << atol_val << " rtol: " << rtol_val;
          torchtrt::logging::log(
              torchtrt::logging::Level::kWARNING,
-              std::string("Maximum numerical deviation for output exceeds tolerance thresholds (") + threshold_ss.str() +
-                  std::string(")"));
+              std::string("Maximum numerical deviation for output exceeds tolerance thresholds (") +
+                  threshold_ss.str() + std::string(")"));
        }
      }
    } else {
ERROR: Some files do not conform to style guidelines

github-actions

Code conforms to Python style guidelines

github-actions

Code conforms to Python style guidelines

narendasan · 2022-05-11T16:23:34Z

/blossom-ci

cpp/bin/torchtrtc/accuracy.cpp

cpp/bin/torchtrtc/main.cpp

cpp/bin/torchtrtc/accuracy.cpp

cpp/bin/torchtrtc/main.cpp

github-actions

There are some changes that do not conform to C++ style guidelines:

diff --git a/workspace/cpp/bin/torchtrtc/main.cpp b/tmp/changes.txt
index e5a6c28..48d9eff 100644
--- a/workspace/cpp/bin/torchtrtc/main.cpp
+++ b/tmp/changes.txt
@@ -446,8 +446,8 @@ int main(int argc, char** argv) {
          threshold_ss << "atol: " << atol_val << " rtol: " << rtol_val;
          torchtrt::logging::log(
              torchtrt::logging::Level::kWARNING,
-              std::string("Maximum numerical deviation for output exceeds tolerance thresholds (") + threshold_ss.str() +
-                  std::string(")"));
+              std::string("Maximum numerical deviation for output exceeds tolerance thresholds (") +
+                  threshold_ss.str() + std::string(")"));
        }
      }
    } else {
diff --git a/workspace/cpp/bin/torchtrtc/accuracy.cpp b/tmp/changes.txt
index 94592b0..4f99d17 100644
--- a/workspace/cpp/bin/torchtrtc/accuracy.cpp
+++ b/tmp/changes.txt
@@ -27,12 +27,9 @@ bool almost_equal(const at::Tensor& a, const at::Tensor& b, float atol, float rt
  auto result = diff.abs().max().item<float>();
  auto threshold = atol + (rtol * b.abs().max().item<float>());

+  torchtrt::logging::log(torchtrt::logging::Level::kDEBUG, std::string("Max Difference: ") + std::to_string(result));
  torchtrt::logging::log(
-      torchtrt::logging::Level::kDEBUG,
-      std::string("Max Difference: ") + std::to_string(result));
-  torchtrt::logging::log(
-      torchtrt::logging::Level::kDEBUG,
-      std::string("Acceptable Threshold: ") + std::to_string(threshold));
+      torchtrt::logging::Level::kDEBUG, std::string("Acceptable Threshold: ") + std::to_string(threshold));

  return result <= threshold;
}
ERROR: Some files do not conform to style guidelines

github-actions

Code conforms to Python style guidelines

github-actions

Code conforms to Python style guidelines

github-actions

Code conforms to C++ style guidelines

github-actions

Code conforms to Python style guidelines

github-actions

Code conforms to C++ style guidelines

github-actions

Code conforms to C++ style guidelines

github-actions

Code conforms to Python style guidelines

…ce test BREAKING CHANGE: The flag `--threshold` has been removed in favor of two flags `--atol` and `--rtol` which control the maximum absolute and relative tolerances for numberical deviation Signed-off-by: Naren Dasan <naren@narendasan.com> Signed-off-by: Naren Dasan <narens@nvidia.com> Signed-off-by: Anurag Dixit <a.dixit91@gmail.com> fix: Fix for rtol and atol tolerance limit in torchtrtc Signed-off-by: Anurag Dixit <a.dixit91@gmail.com> feat(//cpp)!: Using logger instead of std::cout Signed-off-by: Anurag Dixit <a.dixit91@gmail.com> chore!: Applying C++ lint Signed-off-by: Anurag Dixit <a.dixit91@gmail.com> chore!: Updated the tensor names as per review comments Signed-off-by: Anurag Dixit <a.dixit91@gmail.com>

github-actions

Code conforms to Python style guidelines

github-actions

Code conforms to C++ style guidelines

andi4191 requested a review from narendasan May 10, 2022 21:45

andi4191 self-assigned this May 10, 2022

github-actions bot added component: api [C++] Issues re: C++ API documentation Improvements or additions to documentation labels May 10, 2022

andi4191 mentioned this pull request May 10, 2022

feat: torchtrtc using atol and rtol for tolerance test #1038

Closed

6 tasks

github-actions bot requested changes May 10, 2022

View reviewed changes

github-actions bot approved these changes May 10, 2022

View reviewed changes

andi4191 changed the title ~~Anuragd/torchtrtc atol rtol~~ Using atol and rtol based tolerance threshold for torchtrtc May 10, 2022

andi4191 changed the title ~~Using atol and rtol based tolerance threshold for torchtrtc~~ feat: Using atol and rtol based tolerance threshold for torchtrtc May 10, 2022

andi4191 changed the title ~~feat: Using atol and rtol based tolerance threshold for torchtrtc~~ feat (//cpp): Using atol and rtol based tolerance threshold for torchtrtc May 10, 2022

facebook-github-bot added the cla signed label May 12, 2022

narendasan reviewed May 20, 2022

View reviewed changes

cpp/bin/torchtrtc/accuracy.cpp Show resolved Hide resolved

cpp/bin/torchtrtc/main.cpp Show resolved Hide resolved

cpp/bin/torchtrtc/accuracy.cpp Outdated Show resolved Hide resolved

cpp/bin/torchtrtc/main.cpp Show resolved Hide resolved

narendasan mentioned this pull request May 20, 2022

feat(//tests): Update rtol and atol based tolerance for test cases #1055

Merged

6 tasks

github-actions bot requested changes May 21, 2022

View reviewed changes

github-actions bot approved these changes May 21, 2022

View reviewed changes

andi4191 requested a review from narendasan May 25, 2022 21:11

github-actions bot approved these changes May 25, 2022

View reviewed changes

narendasan force-pushed the anuragd/torchtrtc_atol_rtol branch from 3cbe2e2 to 664f117 Compare June 22, 2022 19:54

github-actions bot approved these changes Jun 22, 2022

View reviewed changes

narendasan merged commit a94e493 into master Jun 22, 2022

narendasan deleted the anuragd/torchtrtc_atol_rtol branch June 22, 2022 19:59

feat (//cpp): Using atol and rtol based tolerance threshold for torchtrtc #1052

feat (//cpp): Using atol and rtol based tolerance threshold for torchtrtc #1052

Conversation

andi4191 commented May 10, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Type of change

Checklist:

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

narendasan commented May 11, 2022

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

andi4191 commented May 10, 2022 •

edited

Loading