Another thing that I’ve been working on in my project, otter, is a basic unit testing framework. There doesn’t seem to be a popular and standard unit testing framework in C, as far as I’m aware. It also didn’t feel right to write my source code in C and then using something like Google Test, Boost.Test, or [insert your favorite framework here] in C++ to test it. Sure, it’s quite possible to do and probably even recommended for a project that needs to write something in C. A lot of the ergonomics are likely better. However, since I can do whatever I want with this project, I tried my hand at making something that works for me.
nathan@nathans-laptop:~/Dev/otter$ ./debug/otter_test --help
Usage: ./debug/otter_test [OPTIONS] <shared_object.so>
Options:
-l, --list
List available tests
-t, --test <test>
Run a single test by name
-h, --help
Print this message
-L, --license
Show license information
To go over the basics of my unit test framework: there is a test driver executable that can be given a path to a shared object containing the tests to be ran. The test driver then loads the shared object, finds all the tests inside, and then optionally filters and executes them one by one. You can find the source code for the full test driver here, but here’s the important bit.
/* otter_test_driver.c */
typedef void (*otter_test_list_fn)(otter_allocator *, const char ***test_names,
int *test_count);
const char *so_path = argv[optind];
void *handle = dlopen(so_path, RTLD_LAZY);
if (!handle) {
fprintf(stderr, "dlopen error: %s\n", dlerror());
otter_allocator_free(allocator);
return EXIT_FAILURE;
}
otter_test_list_fn list_func = dlsym(handle, "otter_test_list");
if (!list_func) {
fprintf(stderr, "dlsym error: %s\n", dlerror());
dlclose(handle);
otter_allocator_free(allocator);
return EXIT_FAILURE;
}
int count = 0;
const char **testnames = NULL;
list_func(allocator, &testnames, &count);
for (int i = 0; i < count; ++i) {
run_test(handle, allocator, testnames[i]);
}
The function “otter_test_list” is the only well-known function that is expected to be found in each shared object that the driver loads. Once that function has successfully been loaded and called, it can then return a list of test names to the driver which can then iterate through them, loading and running each test function. Each shared object becomes essentially its own single test suite.
The interesting bit is how does each test suite know what all of it’s test names are? You could have a big hard coded array where entries are added manually. Every time a test is added or removed though this list will have to be updated manually;
/* otter_example_tests.c */
OTTER_TEST(passing_test) {
OTTER_ASSERT(true);
OTTER_TEST_END();
}
OTTER_TEST(failing_test) {
OTTER_ASSERT(1 == 2);
OTTER_TEST_END();
}
void otter_test_list(otter_allocator *allocator, const char ***test_names,
int *test_count) {
*test_count = 2;
size_t alloc_size = sizeof(char *) * *test_count;
*test_names = otter_malloc(allocator, alloc_size);
if (*test_names == NULL) {
return;
}
(*test_names)[0] = "passing_test";
(*test_names)[1] = "failing_test";
}The way that you could “automatically” register and keep track of the test functions in an executable is by taking advantage of static variables initializing which runs before main when a program is loaded or, in this case, right when a shared object is loaded. There could be some macro magic within OTTER_TEST that creates a class definition whose constructor registers the test name (or possibly even the test function’s address) to some global list and then creates a static variable of that class type so that the constructor gets run at startup. This is how Google Test works as far as I understand. And probably many other C++ based testing frameworks. Since we don’t have access to this constructor trick in C, we need a different way.
That way in C still relies on static variables, mind you. We just need some other way to automatically register things. The gcc compiler (and I think clang as well) supports the section attribute which allows data to be placed in a custom section of the generated object file instead of the text section where code normally goes. Here’s the full macro for OTTER_TEST which uses this attribute.
/* otter_test.h */
#define OTTER_TEST(name) \
bool name(otter_test_context *); \
__attribute__((used, section(OTTER_TEST_STRINGIFY( \
OTTER_TEST_SECTION_NAME)))) static otter_test_entry \
name##_entry = {#name}; \
bool name(__attribute__((unused)) \
otter_test_context *OTTER_TEST_CONTEXT_VARNAME)
Based on the test name static entry variable in the OTTER_TEST_SECTION_NAME is created to hold a string literal of the test’s stringified name. So for each test, we’ll have a corresponding otter_test_entry implicitly created. Here’s an example of a test being expanded:
OTTER_TEST(passing_test) {
OTTER_ASSERT(true);
OTTER_TEST_END();
}
/* becomes */
bool passing_test(otter_test_context *);
__attribute__((used, section("otter_test_section"))) static otter_test_entry
passing_test_entry = {"passing_test"};
bool passing_test(__attribute__((unused)) otter_test_context *otter_test_ctx) {
/* ... */
}So then, in otter_test.c, we can iterate through all of the otter_test_entry’s in the custom section to get all of the names of the tests. So as long as the otter_test.o (which contains this iteration code) is linked with the rest of the files to create the shared object test suite, this “autodiscover” of all of the tests within the shared object is provided for free and no manual list management needs to happen. The actual test source code doesn’t even need to know about or create its own otter_test_list function.
/* otter_test.c */
#include "otter_test.h"
#include <limits.h>
void otter_test_list(otter_allocator *allocator, const char ***test_names,
int *test_count) {
if (allocator == NULL || test_names == NULL || test_count == NULL) {
return;
}
/* Linker guarantees __start and __stop symbols point to the same section */
/* NOLINTNEXTLINE(clang-analyzer-security.PointerSub) */
intptr_t num_tests = __stop_otter_test_section - __start_otter_test_section;
if (num_tests > (intptr_t)INT_MAX) {
return;
}
*test_names = otter_malloc(allocator, sizeof(char *) * (size_t)num_tests);
if (*test_names == NULL) {
*test_count = 0;
return;
}
*test_count = (int)num_tests;
for (intptr_t i = 0; i < num_tests; ++i) {
(*test_names)[i] = __start_otter_test_section[i].name;
}
}
Finally, here’s an example of the test driver being used to run some tests. I’ve got most of this stuff hooked up to a Makefile just to make running all of the test suites easier. I quite like how the driver executable almost never needs to be recompiled. When a test suite changes, only that shared object itself is recompiled and relinked which keeps the amount of work and wait time to a minimum.
nathan@nathans-laptop:~/Dev/otter$ ./debug/otter_test ./debug/otter_cstring_tests.so
Running strdup_works...
passed
Running strdup_allocator_null...
passed
Running strdup_str_null...
passed
Running strdup_empty...
passed
Running strndup_zero_size...
passed
Running strndup_malloc_returns_null...
passed
Running strndup_str_null...
passed
Running strndup_allocator_null...
passed
Running strndup_partial_copy...
passed
Running vasprintf_allocator_null...
passed
Running vasprintf_str_null...
passed
Running vasprintf_fmt_null...
passed
Running vasprintf_works...
passed
Running vasprintf_malloc_fails...
passed
Running asprintf_allocator_null...
passed
Running asprintf_str_null...
passed
Running asprintf_fmt_null...
passed
Running asprintf_works...
passed
Running asprintf_empty_format...
passed
Leave a Reply