SuperTest Examples

SuperTest Examples

SuperTest contains a very large collection of C and C++ tests. Here we show a few pretty ones from our collection. SuperTest has two kinds of tests, positive and negative. Positive tests are correct C/C++ programs for which the behavior is defined by the language specification. These test are self-testing and should compile and run successfully in order to pass. For negative tests a diagnostic is expected by the compiler, and compilation is expected to fail. So a negative test passes when its compilation fails, and it is a failure when the compiler compiles without an error. No attempt is made to run a negative test (because it has no defined semantics). Negative tests are also known in SuperTest as diagnostic tests or x-tests. The filenames of positive tests all start with a ‘t’. The filenames of all negative tests start with an ‘x’.

Most test programs include the file def.h. This file defines several macros that link the test program to SuperTest’s diagnostic library which is responsible for creating comprehensive test log-files. SuperTest’s streport program turns these log-files into readable reports.

The path of each test refers to the section in the language specification to which it applies.

SIMPLE POSITIVE TEST: C99/6/5/2/5/T7.C

/* 
 * (c) Copyright 2015 by Solid Sands B.V., 
 * Amsterdam, the Netherlands. All rights reserved. 
 * Subject to conditions in the RESTRICTIONS file. 
 */ 

#include "def.h" 

MAIN{ 
    CVAL_HEADER("compound literal (sizeof incomplete array)"); 
    CVAL_VERIFY(sizeof((int []) {1, 2, 3, 4}) == 4 * sizeof (int)); 
}

This test performs a comparison of sizes using the sizeof() operator. On the left side of the comparison there is a compound literal, in this case an integer array of unknown size, which’ size is determined by the initializer list of four elements. Therefore, the size is that of four integers. On the right side the size of an integer is multiplied by four.

C99 Standard (ISO/IEC 9899:1999), Section 6.5.2.5, Paragraph 4:

Paragraph 4. A postfix expression that consists of a parenthesized type name followed by a brace-enclosed list of initializers is a compound literal. It provides an unnamed object whose value is given by the initializer list.

C99 Standard (ISO/IEC 9899:1999), Section 6.5.2.5, Paragraph 5:

If the type name specifies an array of unknown size, the size is determined by the initializer list as specified in 6.7.8, and the type of the compound literal is that of the completed array type.

The test starts with the inclusion of the SuperTest include file “def.h”. This file contains the definition of SuperTest’s diagnostic macros. The macros used here are:

• MAIN: used to specify the entry-point of the test. This expands to the header of a function that is called from the validation library. The entry-point function has a return type of void, so no value is returned by it.
• CVAL_HEADER: defines the purpose of the test. This string is copied into the log-files. Every tests must contain exactly one one-line CVAL_HEADER.
• CVAL_VERIFY: evaluates its argument. If the argument evaluates to false, a diagnostic message is produced. This macro is the core of SuperTest’s self-checking structure. Every run-time behavior that the test wants to verify must be passed as an argument to this macro. Unlike C’s own assert, the test program does not terminate if the argument to CVAL_VERIFY is false.

SIMPLE NEGATIVE TEST: 3/1/2/5/X0.C

/*
 * (c) Copyright 2015 by Solid Sands B.V.,
 * Amsterdam, the Netherlands. All rights reserved.
 * Subject to conditions in the RESTRICTIONS file.
 */

int v [2][ ][1] = { {{3}, {2}}, {{1}, {0}} };

#include "def.h" 

MAIN{
    CVAL_HEADER("Only 1st dimension may be omitted.");
    CVAL_VERIFY(0);
}

Since an array of incomplete type cannot be constructed, arrays with more than one dimension can only have unknown bound in the first dimension. Therefore, this x-test is expected to fail at compilation.

The name of this test refers to Section 3.2.1.5 of the C90 standard, which corresponds to Section 6.2.5 in C99.

C99 Standard (ISO/IEC 9899:1999), Section 6.2.5, Paragraph 20

An array type describes a contiguously allocated nonempty set of objects with a particular member object type, called the element type (since object types do not include incomplete types, an array of incomplete type cannot be constructed).

C99 Standard (ISO/IEC 9899:1999), Section 6.2.5, Paragraph 22

An array type of unknown size is an incomplete type.

C99 Standard (ISO/IEC 9899:1999), Section 6.7.5.2, Paragraph 4

If the size is not present, the array type is an incomplete type.

COMPLEX POSITIVE TEST: 3/3/7/T1.C

/*
 * (c) Copyright 2015 by Solid Sands B.V.,
 * Amsterdam, the Netherlands. All rights reserved.
 * Subject to conditions in the RESTRICTIONS file.
 */

#include "def.h"

int two = 2;
int six = 6;

MAIN{
	int      i1, i2;
	unsigned int ui;
	unsigned int u1, u2;

	CVAL_HEADER("the '<<' operator");
	CVAL_SECTION("semantic of '<<' operator");
	CVAL_DOTEST(i1 = six << two);
	CVAL_VERIFY(i1 == 24);

	CVAL_SECTION("shift the leftmost bit");
#if CVAL_INTSIZE == 64
	u1 = 0x8000000000000000;
#elif CVAL_INTSIZE == 32
	u1 = 0x80000000;
#elif CVAL_INTSIZE == 24
	u1 = 0x800000;
#elif CVAL_INTSIZE == 16
	u1 = 0x8000;
#else
	u1 = 0;
#endif
	CVAL_DOTEST(u2 = u1 << 1);
	CVAL_VERIFY(u2 == 0);

	CVAL_SECTION("shift an unsigned int");
	ui = (unsigned) -1;
#if CVAL_INTSIZE == 64
	u2 = 0xfffffffffffffffe;
#elif CVAL_INTSIZE == 32
	u2 = 0xfffffffe;
#elif CVAL_INTSIZE == 24
	u2 =0xfffffe;
#elif CVAL_INTSIZE == 16
	u2 = 0xfffe;
#endif
	CVAL_DOTEST(u1 = ui << 1);
	CVAL_VERIFY(u1 == u2);

	CVAL_SECTION("shift 0 bits");
#if CVAL_INTSIZE == 64
	i1 = 0x70f0f0f0f0f0f0f0;
#elif CVAL_INTSIZE == 32
	i1 = 0x70f0f0f0;
#elif CVAL_INTSIZE == 24
	i1 = 0x70f0f0;
#elif CVAL_INTSIZE == 16
	i1 = 0x70f0;
#endif
	CVAL_DOTEST(i2 = i1 << 0);
	CVAL_VERIFY(i2 == i1);
}

Example with CVAL_INTSIZE = 16
    ui = (unsigned) - 1 = 0xFFFF = 65535
    u1 = ui << 1 
    65535 * 2 = 0xFFFF * 0x2 = 0x1FFFE = 131070 
    1 + 65535 = 0x1 + 0xFFFF = 0x10000 = 65536
    131070 % 65536 = 0x1FFFE % 0x10000 = 65534 = 0xFFFE

OPTIMIZATION POSITIVE TEST: 3/6/5/TSPR4541.C

/*
 * (c) Copyright 2015 by Solid Sands B.V.,
 * Amsterdam, the Netherlands. All rights reserved.
 * Subject to conditions in the RESTRICTIONS file.
 */

#include "def.h"

int test(int j, int k, int recurs){
    int x = 0;
    int result = 0;
    if (recurs)    { test(0, 0, 0); } /* prevent inlining */
    bb2:
    if(j == 0)     { x = 1;       goto bb12; }
    else {
        bb4:
        if(k == 0) {              goto bb13; }
        else       { k = 0;       goto bb12; }
    }
    bb12:
    if(x == 0)     { result = 20; goto bb2; }
    else           { result = 10; goto bb4; }
    bb13:
    return result;
}

MAIN{
    CVAL_HEADER("irreducible flow and constant propagation");
    CVAL_VERIFY(test(1, 1, 0) == 20);
    CVAL_VERIFY(test(0, 1, 0) == 10);
}