s_test framework
For the past couple of weeks I've been playing around with writing simple C libraries from scratch. Right now I’ve written several containers: expandable arrays, as well as doubly and singly linked lists. I have no immediate use for them other than the fun I had writing them and trying to make them optimal for storing arbitrary types. Because there is no immediate consumer of them, and because I want to make sure I wrote these containers correctly, the first consumer of these libraries should be unit tests!
In the spirit of writing everything myself in C for these libraries, I decided to write a testing library/framework myself to test my code which I am calling s_test for the name of the project as a whole, Sapphire (placeholder, but semi permanent name).
s_test library
#ifndef SAPPHIRE_TEST_H_
#define SAPPHIRE_TEST_H_
#include "status/status_code.h"
typedef enum s_status_code (*s_test)(void *context);
struct s_test_suite;
/**
* Initialize a test suite.
*
* @param suite The test suite to initialize.
* @param name The name of the test suite.
* @return A status code representing the success/failure of the operation.
*/
enum s_status_code s_test_init(struct s_test_suite **suite, const char *name);
/**
* Adds a test to the test suite.
*
* @param suite The test suite to add to.
* @param test The test function to add.
* @param name The name of the test.
* @returns A status code representing the success/failure of the operation.
*/
enum s_status_code s_test_add_named(struct s_test_suite *suite, s_test test, const char *name);
/**
* Adds a test to the test suite.
*
* @param suite The test suite to add to.
* @param test The test function to add.
* @returns A status code representing the success/failure of the operation.
*/
#define s_test_add(suite, test) s_test_add_named(suite, test, #test)
/**
* Deletes a test suite and all of its underlying data.
*
* @param suite The test suite to delete.
*/
void s_test_delete(struct s_test_suite *suite);
/**
* Run all the tests in a test suite.
*
* @param suite The test suite to run.
* @return Returns s_ok when all tests pass. Otherwise, s_failed.
*/
enum s_status_code s_test_run(struct s_test_suite *suite);
/**
* Internal to test suite implementation.
*
* Marks a test as failed.
*
* @param context Test context.
* @param expr The expression that caused the test to fail.
* @param line The line where the test failed.
*/
void s_test_fail(void *context, const char *expr, int line);
/**
* Declares a test function.
*
* @param name The name of the function.
*/
#define S_TEST(name) enum s_status_code name(void *s_context_)
/**
* The final statement in every test function. Marks the test as finished.
*/
#define S_TEST_FINISH return s_ok
/**
* Asserts that an expression is true. If false, the test function is finished
* early and marked as failed.
*
* @param expr The expression that should evaluate to true.
*/
#define S_ASSERT(expr) \
do \
{ \
if (!(expr)) \
{ \
s_test_fail(s_context_, #expr, __LINE__); \
return s_failed; \
} \
} while (0);
#endif // SAPPHIRE_TEST_H_
Putting it to use
Here is a bare-bones example of how the testing framework can be used.
#include "s_test/s_test.h"
S_TEST(a_failing_test)
{
S_ASSERT(1 == 2);
S_TEST_FINISH;
}
S_TEST(a_passing_test)
{
S_ASSERT(2 == 2 && 3 == 3);
S_TEST_FINISH;
}
int main()
{
struct s_test_suite *suite;
enum s_status_code status;
status = s_test_init(&suite, "example tests");
S_CHECK(status);
status = s_test_add(suite, a_failing_test);
S_CHECK_AND(status, s_test_delete, suite);
status = s_test_add(suite, a_passing_test);
S_CHECK_AND(status, s_test_delete, suite);
status = s_test_run(suite);
s_test_delete(suite);
return status;
}
Design philosophy
Information hiding
There are several design philosophies that I tried to adhere to in this library. For one, all internal information that is only needed by the implementation of s_test should not be exposed to library consumers. For example, struct s_test_suite is an opaque struct which is only defined in the implementation file of s_test.h, s_test.c. There are other places where this concept is harder to apply, but attempts were made. The function, s_test_fail, should never be called by a consumer of this library directly, but the macros that are used to assert inside tests sure need to! Therefore, the function is publicly exposed. Also, tests need to sneak some internal context around so that s_test_fail knows enough to mark the correct test as failed. Internally, this is a struct s_test_context, but is obfuscated in the public api and given to the test function and whatever is called by the test function as a void pointer.
Explicit error information
I wanted to make any errors that occurred inside the implementation of the test framework are exposed in a way that indicated if the suite was okay to keep using. Because of this, any function that can fail returns an enum s_status_code whose value indicates what went wrong which could be anything from allocation failures to a test failure when running the suite. Utility macros S_CHECK and S_CHECK_AND were added to make status checking less verbose but, hopefully, still clear.
Things I don't like
The only thing that really irks me when using s_test so far is having to write the test and, additionally, be sure to add it to the test suite. Testing frameworks in C++ have this solved by automatically registering all declared tests in some global static class that has a constructor that runs before main. The key here is getting code to run before main so that a library consumer doesn't have to write it themselves. It is unclear to me how this can be done in C without resorting to compiler extensions (e.g., something like attribute((constructor) in gcc).
Improvements
Additional assert macros
I want better failure messages when tests don't work as intended. Right now I always have to break out the debugger when something like this fails: S_ASSERT(a == b). The failure message will be: "test failed because assertion a == b was not true". There is no ability to print what a or b were so that it's known why they weren't equal. So, at the very least S_ASSERT_EQ needs to be added.