Errata for SuperTest Vermeer Release Update 2

Errata for SuperTest Vermeer Release Update 2

Note: this page is no longer maintained since a new update of SuperTest was released. If you have any issues with specific tests, these issues might have been fixed in a newer SuperTest version, or the issue might be mentioned on the errata page of a newer SuperTest version. Click here for an overview of all errata pages for specific versions.

  • [20211004] The following test-files rely on an incorrect assumption about the default constructor of some iterators. To receive updated versions of the test-files, please contact support@solidsands.nl.
    • Cxx11/23/3/6/1/t_insert_id69.C
    • Cxx11/23/3/7/t_insert_id66.C
    • Cxx11/23/3/7/t_insert_id68.C
    • Cxx11/23/5/5/1/t_begin_id64.C
    • Cxx11/23/5/5/1/t_end_id66.C
    • Cxx11/23/5/6/1/t_begin_id63.C
    • Cxx11/23/5/6/1/t_end_id65.C
    • Cxx11/23/5/7/1/t_begin_id63.C
    • Cxx11/23/5/7/1/t_end_id65.C
    • Cxx14/23/3/6/1/t_insert_id70.C
    • Cxx14/23/3/7/t_insert_id70.C
    • Cxx14/23/3/7/t_insert_id72.C
  • [20210909b] The following test-files deallocate memory using a delete expression that is incompatible with the new expression that allocated the memory. To receive updated versions of the test-files, please contact support@solidsands.nl.
    • iso14882/3/7/3/2/t_0005a.C
    • iso14882/3/7/3/2/t_0011.C
    • iso14882/3/7/3/2/t_0001a.C
    • iso14882/3/7/3/2/t_0010.C
    • iso14882/3/7/3/1/t_0006.C
    • iso14882/3/7/3/1/t_0005.C
    • iso14882/3/7/3/1/t_0011.C
    • Cxx17/8/3/4/t_0006.C
    • Cxx17/6/7/4/t_0040.C
    • Cxx17/6/7/4/t_0041.C
    • Cxx17/8/3/4/t_0002.C
  • [20210909a] The test-files iso14882/3/7/3/2/t_0005.C and iso14882/3/7/3/1/t_0010.C are retracted from SuperTest. The tests contain undefined behavior. Therefor, if the test result is FAILED, it can be ignored.
  • [20210723] The test-files Cxx11/17/6/4/7/t_0003.C and Cxx11/18/8/3/3/t_0001.C include header <new>, while they should include header <exception>. To receive updated versions of the test-files, please contact support@solidsands.nl.
  • [20210722] The test-file Cxx17/23/2/1/t_0007.C contains test cases that expect the conversion specifier f to be used, while the e conversion specifier is used instead. To receive an updated version of the test-file, please contact support@solidsands.nl.
  • [20210614] The following test-files are incompatible with data models that have 16-bit integers. To receive an updated version of the test-files, please contact support@solidsands.nl.
    • 3/2/1/1/t102.c
    • 3/2/1/1/t103.c
    • 3/2/1/2/0/c/t17.c
    • 3/2/1/2/0/c/t19.c
  • [20210531] The test-file Cxx11/25/1/t_0001.C contains unintended out of bound array access. To receive an updated version of the test-file, please contact support@solidsands.nl.
  • [20210413] The following test-files generate C files with names that do not match the names of the corresponding generator files. This is incompatible with the valid test driver. To receive an updated version of the test-files, please contact support@solidsands.nl.
    • 3/1/1/ximplfunc.gen
    • C99/7/24/4/1/1/t2.gen
    • C99/7/24/4/1/1/t3.gen
    • C99/7/3/9/5/t2.gen
  • [20210204] The following test-files lack a preprocessing check for some floating-point environment macros before its usage. To receive an updated version of the test-files, please contact support@solidsands.nl.
    • C99/7/12/9/6/t1.c
    • C99/7/12/9/6/t2.c
    • C99/7/12/9/6/t3.c
    • Cxx11/26/3/1/t_0001.C
  • [20210203] The test-file 4/9/6/3/tprintfreturnerror.c tries to use a stream that has already been closed, which is undefined behavior. To receive an updated version of the test-file, please contact support@solidsands.nl.
  • [20210201] The following test-files use subnormal floating point values without verifying that the implementation supports IEC 60559 floating point arithmetic. Therefore, the tests might FAIL incorrectly on implementations that do not support IEC 60559 floating point arithmetic. To receive updated versions of the test-files, please contact support@solidsands.nl.
    • 4/5/4/2/t1.c
    • 4/5/4/4/t1.c
    • C99/7/12/10/3/t1.c
    • C99/7/12/10/3/t2.c
    • C99/7/12/13/1/t1.c
    • C99/7/12/13/1/t2.c
    • C99/7/12/13/1/t3.c
    • C99/7/12/6/10/t1.c
    • C99/7/12/6/10/t2.c
    • C99/7/12/6/2/t2.c
    • C99/7/12/6/4/t2.c
    • C99/7/12/6/5/t1.c
    • C99/7/12/6/5/t2.c
    • C99/7/12/6/7/t2.c
    • C99/7/12/6/8/t1.c
    • C99/7/12/6/8/t2.c
    • C99/7/12/7/3/t2.c
    • C99/7/12/7/3/t3.c
    • C99/7/12/9/2/t2.c
  • [20210125] The menu options to clear the log panes in the SuperTest GUI do not work. To receive an updated version of the SuperTest GUI, please contact support@solidsands.nl.
  • [20210114] The test-files C99/7/19/6/12/t1.c and C99/7/24/2/7/t1.c are updated to avoid possible POSIX compatibility issues. Please replace occurrences of SIZE_MAX with INT_MAX to update these test-files.
  • [20210113] The test-file 4/10/7/2/t1.c may PASS incorrectly, since it’s requirements on the mbtowc function are not strict enough if n == 0. To receive an updated version of the test-file, please contact support@solidsands.nl.
  • [20210104] The test-file Cxx11/20/8/11/2/1/t_0002.C is incorrect and will fail because the template class definition does not have an allocator template-parameter as it is used in the test. The test can be fixed by removing the allocator argument so that the line calling the constructor becomes:
    std::function< decltype(f1) > f{selector, ra, f1};

    NOTE: this fixed version also fails to compile on all compilers known to us. We think the reason for this is that the feature that this test verifies, was added in C++11 and removed again in C++17, and so it was never implemented by the library developers.

  • [20201224b] The execution of test-file Cxx11/27/7/2/1/3/t_explicit_bool.C may not terminate on some implementations when using the ‘valid’ test driver. In that case we recommend that you skip the test, or ask us for an update of the test.
  • [20201224a] The following test-files depend on undefined behavior on implementations for which the char data type is unsigned by default. To receive updated versions of the test-files, please contact support@solidsands.nl.
    • 3/2/1/2/0/b/t150.c
    • 3/2/1/2/0/b/t151.c
    • 3/2/1/2/0/b/t152.c
    • 3/2/1/2/0/b/t153.c
    • 3/2/1/2/0/b/t208.c
    • 3/2/1/2/0/b/t209.c
    • 3/2/1/2/0/b/t210.c
    • 3/2/1/2/0/b/t211.c
    • 3/6/5/3/tspr1827.c
    • 4/4/t2.c
    • 4/9/9/4/t1.c
    • C99/7/19/6/1/t5.c
    • stress/tspr1827.c
  • [20201215] Some executables in the SuperTest\bin installation directory (‘calltest’, ‘cvalsplit’, ‘tempest’) may trigger a false positive virus scanner warning on Windows, despite our continued efforts to avoid this. These executables are created by us from our own source code and are signed with our certificate. If you run into this problem, you can whitelist the SuperTest\bin directory in your virus scanner as a workaround. Alternatively, please contact support@solidsands.nl to receive updated executables that do not trigger virus warnings.
  • [20201210b] The test-files 4/9/9/4/t1.c and C99/7/19/6/1/t5.c are incompatible with implementations for which the char type is unsigned by default. To solve the issue, replace char with int in the declaration of variable c.
  • [20201210a] The following test-files try to use a stream that has already been closed, which is undefined behavior. To receive an updated version of the test-files, please contact support@solidsands.nl.
    • 4/9/7/1/t1.c
    • 4/9/7/2/t2.c
    • 4/9/7/5/t1.gen
    • 4/9/7/6/t1.gen
    • 4/9/7/7/t1.gen
    • 4/9/7/9/t1.c
    • 4/9/7/10/t1.c
    • 4/9/9/1/t1.c
    • 4/9/9/2/t2.c
    • 4/9/9/3/t1.c
    • 4/9/9/4/t1.c
    • 4/9/9/4/t2.c
  • [20201124b] The test-file iso14882/9/0/x16.C is retracted from SuperTest. The test depends on undefined behavior and not on a constraint of the C++ programming language that requires a diagnostic. Therefor, if the test result is FAILED, it can be ignored.
  • [20201124a] The test-file Cxx14/14/7/2/xvtstcei_b.C is incompatible with C++17. Therefor, if the test result is FAILED when using a C++17 compiler, it can be ignored.
  • [20201123] The test-files 3/1/2/1/xspr3120.c, 3/3/3/3/x1.c, 3/3/3/3/x2.c, 3/3/3/3/x3.c, 3/3/6/xspr3112a.c, 3/5/7/xspr4443.c, C11/6/9/1/xsakrfirst.c and C11/6/9/1/xsakrlater.c are incompatible with C++ implementations. To receive an updated version of the test-files, please contact support@solidsands.nl.
  • [20201120] The test-file 3/3/8/xspr3112.c is incompatible with C11 and C18 implementations. Therefor, if the test result is FAILED when using a C11 or C18 compiler, it can be ignored.
  • [20201119] Some test sets, such as the set for C++ exceptions, list too many tests. To receive updated test sets, please contact support@solidsands.nl.
  • [20201101] Due to an internal error in the Tcl/Tk implementation, stgui is known to crash on some Linux hosts when its window is resized to (close to) full screen size.