Using Check with the Autotools

16 May 2003

Goals

Check is a C unit testing framework that is easily integrated with projects built using autoconf, automake, and their related utilities. Setting it up, however, does take a few steps, which are best done when first creating the project. Not only is is simpler to start with minimal changes, but if you want to use test-driven development, you might as well set up the test harness before you start trying to write code.

When working with Check, there are a couple of goals that make life a bit more difficult:

  1. Don't require Check to be installed by people who just want to compile and run the package.
  1. Don't pollute the package's interfaces, either internal or external, with either unneeded internal functions or bits of test harness.

Requiring Check would go against the portability goals of using the autotools in the first place and though it would be good if everyone had it and used the tests to validate the package, that is not the case now.

However, this goal is handled easily enough by providing a warning in the AM_PATH_CHECK macro call and defining a HAVE_CHECK automake conditional to conditionally compile the code dealing with the tests.

Not polluting interfaces, however, requires more work. Not only does this require careful control of what appears in header files, but also care in what symbols are visible from the source code itself. C provides the static keyword to limit the scope of an identifier to the source file the identifier is defined in. Limiting the scope of functions and variables prevents the internal details of an implementation from being exposed to the outside world---always a good idea. However, those internal details are precisely what you would like to test.

That requirement means that the tests of the internal details need to be part of the same file where those internal details are defined and used. Not in itself a problem, but then the parts of the test harness in the file need to be exposed. But not exposed everywhere, but only to the rest of the test harness, and thus not polluting the package's interface.

To do that, we need a CHECKING cpp macro and to convince the Makefile to compile a special version of the files with the internal details to be tested.

In the rest of this file, I will go through the steps I have used to set up a package to use Check while satisfying the goals I described.

Steps

  1. Create configure.in. Typically, an addition to the AC_INIT, AM_INIT, and AC_OUTPUT macros, the package will need a config file and the C compiler:

    AM_CONFIG_HEADER(config.h)
    AC_PROG_CC
    
  2. Add AM_PATH_CHECK to configure.in (and get it into aclocal.m4). AM_PATH_CHECK is a macro supplied with Check:

    AM_PATH_CHECK(,[have_check="yes"],
      AC_MSG_WARN([Check not found; cannot run unit tests!])
      [have_check="no"])
    AM_CONDITIONAL(HAVE_CHECK, test x"$have_check" = "xyes")
    

    You may need to put the AM_PATH_CHECK definition into acinclude.m4 if aclocal doesn't find it.

    The AM_CONDITIONAL macro will set HAVE_CHECK in the Makefile if Check is available. The AC_MSG_WARN serves two purposes; it tells AM_PATH_CHECK that not having Check in all right, and notifies the end user that "make check" will not do anything useful if Check is not available.

  3. Prepare to build the test harness. Into Makefile.am:

    INCLUDES                    = @CHECK_CFLAGS@
    if HAVE_CHECK
    TESTS                       = check_libapdns
    else
    TESTS                       =
    endif
    noinst_PROGRAMS             = $(TESTS)
    check_libapdns_SOURCES      = check_libapdns.c
    check_libapdns_LDADD        = @CHECK_LIBS@
    CLEANFILES                  = check_libapdns.log
    

    Right now, this is the only thing you need in Makefile.am. The directory and package I am working on is "libapdns", but that is not important right now. The program check_libapdns is the actual program that runs the unit tests. The CHECK_CFLAGS and CHECK_LIBS are provided by AM_PATH_CHECK, and setting TESTS tells automake that check_libapdns is to be run when someone does a "make check". The automake HAVE_CHECK conditional chooses between the two TESTS definitions when configure is run. I like to have the test log file around, but putting it in the CLEANFILES variable will remove it on a "make clean".

  4. Create the harness sources. In my example, this is the file "check_libapdns.c". This file is fairly simple, especially since it will only be compiled if Check is available.

    In check_libapdns.c, get config.h and check.h:

    #include "config.h"
    #include <check.h>
    

    Create the test harness main function:

    /*
     * Main
     */
    int
    main()
    {
      int nf;
      Suite *s = libapdns_suite();
      SRunner *sr = srunner_create(s);
      srunner_set_log(sr, "check_libapdns.log");
      srunner_run_all(sr, CK_NORMAL);
      nf = srunner_ntests_failed(sr);
      srunner_free(sr);
      suite_free(s);
      return (nf == 0) ? 0 : 1;
    }
    

    Finally, create a core test case (just to show we've been here). This is an important part of the test-driven development---getting this test working means that the build system is working. It should probably have been the first thing in this list, but everything else is mostly boilerplate and probably should be handled by some template system that also creates configure.in and Makefile.am:

    /*
     * Core test suite
     */
    START_TEST(test_core)
    {
      fail_unless(1==1, "core test suite");
    }
    END_TEST
    Suite *
    libapdns_suite(void)
    {
      Suite *s = suite_create("libapdns");
      TCase *tc = tcase_create("core");
      tcase_add_test(tc, test_core);
      suite_add_tcase(s, tc);
      return s;
    }
    

    By the way, this should be in check_libapdns.c above the main function.

  5. Set up the autoconf and automake generated files, configure the project, and run "make check". It should succeed, with one test.

  6. Now for real work. Create showdns.c, which is the first of the package's actual components:

    #include "config.h"
    #include "showdns.h"
    
    /* Check-based unit tests */
    #if defined(CHECKING)
    #include <check.h>
    Suite *
    showdns_suite(void)
    {
      Suite *s = suite_create("showdns");
      TCase *tc = tcase_create("base");
      suite_add_tcase(s, tc);
      return s;
    }
    #endif /* CHECKING */
    

    Notice that the suite function is wrapped in tests for CHECKING (which we will get to in a minute).

  7. Create showdns.h and a check_showdns.h. The first should be empty (modulo the typical comment header and multiple-inclusion incantations), but will be filled in later with the interfaces exported from showdns.c. The second should have:

    Suite *showdns_suite(void);
    

    It will be used by the test harness, check_libapdns.c.

  8. Arrange for the test harness to call the new suite. Add the following to the main function in check_libapdns.c:

    srunner_add_suite(sr, showdns_suite());
    

    Also, include check_showdns.h into check_libapdns.c.

  9. Arrange for the Makefile to create check_showdns.o and link it with the test harness. In the Makefile.am:

    check_libapdns_LDADD        = @CHECK_LIBS@ check_showdns.o
    check_%.o : $(srcdir)/%.c
                $(COMPILE) -DCHECKING -c -o $@ $^
    

    This is where CHECKING enters the picture. The CHECKING cpp test prevents the test suite from leaking into the interface, while the special Makefile rule builds a special version of showdns.c with the test suite exposed.

    Try compiling and running the tests. Everything should be happy.

  10. Add a test for a new function. Put the START_TEST definition between the CHECKING test and the showdns_suite function definition:

    START_TEST(test_showquery)
    {
      showquery(22,                    /* size */
    166,                    /* id */
    0,                      /* opcode */
    0, 0, 1, 0,             /* aa, tc, rd, ra */
    0,                      /* rcode */
    1, 0, 0, 0,             /* qd-, an-, ns-, arcount */
               "\003foo\003com\000\000");
    }
    END_TEST
    

    Then add the test to the showdns suite:

    tcase_add_test(tc, test_showquery);
    
  11. Try compiling the project. It'll fail, because the showquery function is not defined. And that is test-driven development in a nutshell.

From now on, through the development of the project, use "make check" rather than just "make". That should compile everything that needs compiling and run the unit tests.

Thanks

Arien Malec, the author of Check, sent me the pointer to AM_CONDITIONAL, which made this thing much simpler.

History

  • Created 10 Apr 2003
  • Updated 16 May 2003

gloria i ad inferni
faciamus opus

Return to Top | About this site...
Last edited Sat Aug 8 03:29:10 2009.
Copyright © 2005-2012 Tommy M. McGuire