For me as a software engineer, it is difficult to suppress the urge to automate. Regardless the complexity of a task, in my ideal world everything would be automated. Generating tests is no exception to this rule, so naturally we have a few tricks up our sleeves to make our jobs (and lives) easier. The trick I will dive into here is Tempest.

The Solid Sands tools have a generic mechanism for incorporating test generators. Tempest makes extensive use of this mechanism. It is used to generate tests from a context-free grammar and a code template. The grammar assigns a set of values to keywords, and during the test generation process, instances of these keywords in the code template are replaced with a value from the value set defined for the keyword. This makes Tempest ideal for creating many syntactically similar tests from a single template. For example, when generating a test set to test a requirement that applies to different types, the template will only contain a test for a placeholder type. Using this template, Tempest will then generate a test for every type specified in the grammar. Our customers also use Tempest for creating their own implementation-specific regression tests, such as tests covering a wide range of different value combinations.

Tempest is an easy-to-use tool for automating repetitive test patterns. It is based on context-free grammar. As a consequence, if a C++ test developer wants to create an instance of container A storing objects of type A’ in one test, and container B storing objects of type B’ in another test, Tempest would also generate tests with the combinations {A, B’} and {B, A’}. However, this is not always the desired result.

To address this, we are working on a new version of Tempest, called Tempest2, that introduces the concept of ‘ties’ – a way to bind together multiple literals. In the example above, ‘tying’ container-type pairs ensures that an expansion of a tie rule will only yield literals from a single pair. In addition, you can skip the generation of certain expansions using regular expressions. These two features give the test developer precise control over the manner in which tests are generated. But that’s not all – we plan to add a few more quality of life improvements, such as ways to indicate the generated tests are only applicable for a certain language version, and ways to share grammars between multiple generator files.

Tempest2 will be part of the next release of SuperTest and SuperGuard. If you are a Tempest user and have any suggestions, don’t hesitate to contact us!

Ivaylo Ivanov, Software Engineer

Contact us


Subscribe to our monthly blog!