Skip to content

Create tests for measuring glitches and latency #551

@gavv

Description

@gavv

Roc already has unit and integration tests, but they, by design, have limitations:

  • they are run frequently during development, so they should not take long
  • they are run everywhere (on developer machine, on CI), so should not be sensitive to real-time (e.g. tests should pass even on loaded machine like CI VM worker)
  • they are yes/no tests, i.e. should either succeed or fail

We want to develop a new set of real-time tests:

  • running on real hardware on unloaded system
  • running long if needed
  • providing metrics, not just success or failure
  • testing various real-time aspects of the toolkit, for example latency, number of glitches, etc

These new tests will live in a separate repo: rt-tests.

Currently, the new repo has a build system, and a skeleton of one test that implements full loopback: writes audio stream to roc sender, obtains the stream from roc receiver, and runs a very basic check.

Now we need to develop a few actual tests:

  • service quality test (measure number of glitches to check how roc copes with packet losses and delays)
  • latency test (measure delay from sender to receiver)
  • stress test (check invariants under intense load)
  • and others

Please refer to rt-tests repo issues for more detailed description of each test:

rt-tests repo uses C++17, cmake, and Google Test.

Metadata

Metadata

Assignees

No one assigned

    Labels

    C-rt-testscategory: Real-time testsC-testscategory: Writing or improving testshelp wantedLooking for contributorsmost wantedNeeded most among other help-wanted issues

    Projects

    Status

    Help wanted

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions