Test framework

Basalt includes its own test framework, basalt_test. It is part of the framework rather than a third-party dependency, so testing code built on Basalt needs nothing downloaded, vendored or version-matched.

It is built to stay out of your build’s way. Test.h includes six Basalt headers and no system headers at all — no STL, and no <regex> behind the filtering. Complete, though, in the ways that decide whether a test suite is pleasant to live in: suites, cases and nested sub-cases; the full assertion matrix with both operand values reported on failure; glob filtering at every level; machine-readable output; and a way to test that an assertion fires.

You write a test as a named block, assert with CHECK or REQUIRE, and run the resulting executable. There is no main to write, no fixture class to declare, and no registration call to remember.

Your first test

tests/core/string_tests.cpp

#include "basalt/test/Test.h"
#include "basalt/core/String.h"

using namespace bs;

TEST_SUITE("strings")
{
    TEST_CASE("Str8Trim removes surrounding whitespace")
    {
        CHECK(Str8Equal(Str8Trim(Str8FromCString("  hello  ")), Str8FromCString("hello")));
        CHECK(Str8IsEmpty(Str8Trim(Str8FromCString("   "))));
    }
}

That is a complete test file. Add it to the test executable, build, and run it:

$ basalt_core_tests --test-case="Str8Trim*"
PASSED — 1 cases (1 passed, 0 failed), 2 checks (0 failed), 0 ms

When a check fails

Change the second assertion to something wrong on purpose, and use the operand form so the values are reported:

CHECK_EQUAL(Str8Trim(Str8FromCString("  hi  ")).size, 3u);      // it is 2

$ basalt_core_tests --test-case="Str8Trim*"
CHECK failed [strings/Str8Trim removes surrounding whitespace]
  tests/core/string_tests.cpp:13
  Str8Trim(Str8FromCString("  hi  ")).size == 3u
    lhs = 2, rhs = 3
FAILED — 1 cases (0 passed, 1 failed), 2 checks (1 failed), 0 ms

The report gives you the path to the case, the file and line, the expression as you wrote it, and both operand values. The process exit code is non-zero, so a build system or CI step fails without needing to parse anything.

What that gave you

Three things you did not have to write:

• No main, and no registration call. The case made itself known to the runner; basalt_test supplies main, argument parsing and the reporters.
• The failing CHECK did not stop the run. It recorded the failure and execution continued to the next line.
• Stable output order. Cases are sorted before running, so two runs of the same binary produce the same lines in the same order — diffable, and safe to assert on in CI.

Setting it up in your project

Link basalt_test and compile your test sources into the test executable:

add_executable(my_tests
        tests/parser_tests.cpp
        tests/geometry_tests.cpp
)
target_link_libraries(my_tests PRIVATE basalt_test my_library)
add_test(NAME my_tests COMMAND my_tests)

basalt_test provides main() and depends only on basalt_core.

One flag matters: your test target needs exceptions enabled (/EHsc, or the absence of -fno-exceptions), because REQUIRE throws internally to abandon a case from arbitrary depth. Basalt itself ships with exceptions off, and the library you are testing keeps its own flags — only the test target changes.

Never put test cases in a static library

A test case makes itself known through a static object, and a static library only contributes object files whose symbols something references. Test cases placed in one are stripped by the linker and disappear silently — a green run that tested nothing. Always compile test sources straight into the test executable.

If your project has several modules, give each one its own test executable linking only that module’s dependencies. An accidental dependency then shows up as a link error rather than passing unnoticed.

CHECK reports, REQUIRE stops

Every assertion exists in both forms. The choice is whether the rest of the case is still worth running once this one has failed.

	On failure	Use when
CHECK...	Records it, execution continues	The following assertions still mean something
REQUIRE...	Records it and abandons the current pass	Continuing would crash or test nothing

Result<Arena*, MemoryError> created = Arena::Create();
REQUIRE(created.ok);              // without an arena, every line below dereferences null
Arena* arena = created.value;

A CHECK on a pointer followed by a dereference is the mistake this distinction prevents.

Prefer the operand forms

Both styles work; they differ only in what the failure tells you.

// Expression form: prints the source text.
CHECK(list.size() == 4);

//   list.size() == 4

// Operand form: prints the values too.
CHECK_EQUAL(list.size(), 4u);

//   list.size() == 4u
//     lhs = 7, rhs = 4

What the value actually was is usually the whole question, so use the operand forms for comparisons and keep the expression form for conditions that are not comparisons.

Available with both the CHECK_ and REQUIRE_ prefixes: EQUAL, NE, LT, LE, GT, GE, plus TRUE, FALSE, NULL and NOT_NULL.

Sub-cases: fresh setup without a fixture

A SUB_CASE is a named section inside a case. The case body re-runs once per sub-case, entering exactly one of them each time.

TEST_CASE("arena reuse")
{
    Result<Arena*, MemoryError> created = Arena::Create();
    REQUIRE(created.ok);
    Arena* arena = created.value;          // runs for every sub-case

    SUB_CASE("push zero-initialises")
    {
        Span<u32> values = arena->Push<u32>(4);
        CHECK_EQUAL(values.size, 4u);
    }

    SUB_CASE("position is monotonic")
    {
        // ...
    }

    arena->Release();                      // runs for every sub-case
}

The runner executes this body twice. Each pass creates a fresh arena, enters one sub-case, and releases it. So neither sub-case can be polluted by the other, and three things follow:

• Shared setup is ordinary straight-line code, written above the sub-cases.
• Isolation comes from re-execution, so no state survives between passes and nothing needs resetting.
• Adding a case costs three lines and no new declarations.

Everything outside the sub-cases runs on every pass

A case with eight sub-cases runs its prologue eight times, so keep expensive setup in mind. Sub-cases nest, but past two levels the re-run count grows faster than the clarity. The depth limit is 8; exceeding it fails the case rather than skipping silently.

Every case also starts from empty scratch arenas: the runner clears each one before a case runs, so scratch memory a case forgot to rewind cannot change the next.

Testing that an assertion fires

Basalt uses BS_ASSERT for contract violations — an out-of-range index, an append to a full Array<T>, a Pop on an empty one. Those paths need testing, and an assertion normally aborts the process.

ExpectAssert arms an expectation for the current thread. While it is alive, a failing BS_ASSERT records itself and returns instead of aborting.

TEST_CASE("Pop on an empty array asserts")
{
    u32 storage[4] = {};
    Array<u32> array = Array<u32>::FromSpan(Span<u32>(storage, 4));

    {
        ExpectAssert expected;
        array.Pop();                       // would normally abort the process

        CHECK(expected.Fired());
        CHECK_EQUAL(expected.Count(), 1u);
    }                                      // disarmed here
}

The expectation is thread-local and bounded by the scope. Arming it in one test does not change assertion behaviour on any other thread, and a test that forgets to check Fired() cannot leave assertions disarmed for the rest of the run.

LastInfo() returns the AssertInfo of the most recent firing, when you need to know which assertion fired.

Running a subset

Filters use a small glob: * matches any run of characters, and | separates alternatives.

Flag	Effect
--test-case=<glob>	Run only matching cases. Also --test-suite= and --test-subcase=.
--test-case-exclude=<glob>	Skip matching cases. Also -suite- and -subcase-.
--list-test-cases	Print the suite/case names the other filters select, and exit.
--reporter=console|json	Output format. console by default.
--no-color	Disable colour. Also off automatically when output is not a terminal.
--allow-empty	Accept an empty selection: exit 0 instead of 3.
--test-shard-count=<n> --test-shard-index=<i>	Run shard i of n: the selected cases whose position is i modulo n.
--help, -h	Print the flag list and exit.

A case runs when it matches the includes — an empty include list means all — and matches none of the excludes. Give each option once: to select several patterns, join them with | in one option.

The exit code tells a script what happened without parsing the output:

Code	Meaning
0	Every selected case passed.
1	At least one case failed.
3	Nothing was selected: the filters matched no case, or a --test-subcase include matched no sub-case. The run reports an error, never a pass.
4	The command line was not understood — an unknown option or reporter, or a valued option given twice. Nothing ran; the reason is on stderr.

A mistyped filter therefore fails the step instead of passing it with nothing tested. The console reports it as ERROR — no test case matched, followed by the filters that were set; no summary follows. Pass --allow-empty when an empty selection is legitimate. --list-test-cases always exits 0, even when it lists nothing.

Running in parallel

Launch the same executable n times, each with its own --test-shard-index from 0 to n - 1 and the same --test-shard-count=n and filters. The shards split the selected cases between them, each case in exactly one shard, so the processes run side by side and their summaries add up to one full run. Cases are dealt round-robin in the sorted order, so a suite’s slow cases spread over the shards instead of landing in one.

$ my_tests --test-shard-count=4 --test-shard-index=0 &
$ my_tests --test-shard-count=4 --test-shard-index=1 &
  ...

A shard with nothing to run — more shards than selected cases — reports 0 cases and exits 0. An index at or past the count, a count of 0, or a value that is not a whole number is a usage error: exit 4, nothing runs.

Machine-readable output

--reporter=json emits one JSON object per line, which is the mode to use from a script or an agent:

{"type":"test_case_begin","path":"strings/Str8Trim removes surrounding whitespace"}
{"type":"assertion_failed","path":"strings/Str8Trim removes surrounding whitespace","severity":"check","file":"tests/core/string_tests.cpp","line":13,"expr":"Str8Trim(Str8FromCString(\"  hi  \")).size == 3u","op":"==","lhs":"2","rhs":"3"}
{"type":"test_case_end","path":"strings/Str8Trim removes surrounding whitespace","status":"fail","checks":2,"failures":1,"duration_ns":41200}

A run ends with a run_summary line carrying the counts. An empty selection ends instead with a single {"type":"error","message":…} line and no summary. With --list-test-cases, each case is one {"type":"test_case","path":…,"file":…,"line":…} line.

Automated consumers are an expected reader of test output here, not an afterthought.

Threads

The active test context is thread-local and installed on the runner thread, so CHECK, REQUIRE and SUB_CASE are valid only on that thread.

A concurrency test spawns its workers, joins them, and asserts afterwards:

TEST_CASE("fetch-add converges")
{
    AtomicU32 counter = {};
    // ... spawn threads that each add 1000 ...
    // ... join them all ...

    CHECK_EQUAL(counter.Load(Ordering::Relaxed), 8000u);
}

Assertions from inside a worker thread are not supported.

API summary

Macro behaviour and reporting details are documented in basalt/test/Test.h. The full specification, for contributors: test-framework.md.

Structure

Macro	Purpose
TEST_SUITE(name)	Names the suite for cases defined inside the block.
TEST_CASE(name)	A test case. Self-registers; the unit of execution.
SUB_CASE(name)	A section. Re-runs the case body, entering one section per pass.

Assertions

Macro	Purpose
CHECK(expr) / CHECK(expr, message)	Report on failure and continue.
REQUIRE(expr) / REQUIRE(expr, message)	Report and abandon the pass.
CHECK_EQUAL CHECK_NOT_EQUAL CHECK_LESS CHECK_LESS_EQUAL CHECK_GREATER CHECK_GREATER_EQUAL	Comparisons that print both operands.
REQUIRE_EQUAL … REQUIRE_GREATER_EQUAL	The same, fatal.
CHECK_TRUE CHECK_FALSE CHECK_NULL CHECK_NOT_NULL	Single-operand forms.
REQUIRE_TRUE … REQUIRE_NOT_NULL	The same, fatal.

Assertion testing

Signature	Purpose
ExpectAssert	RAII: while alive, a failing BS_ASSERT records and returns.
Fired()	Whether an assertion fired during the scope.
Count()	How many fired.
LastInfo()	The AssertInfo of the most recent firing.

Build detection

Macro	Purpose
BS_TEST_THREAD_SANITIZER	1 when the test binary is built under ThreadSanitizer, else 0. Use it to skip a test that measures how much address space a reservation can take.