On Fri, Oct 9, 2015 at 12:22 PM, Jakub Hrozek <jhrozek_at_redhat.com> wrote:
> On Fri, Oct 09, 2015 at 11:14:30AM +0100, David Drysdale wrote:
>> Hi folks,
>>
>> I've made a start on a unit test suite for c-ares, and I'd be
>> interested in any feedback.
>>
>> The changes so far are in a branch at
>> https://github.com/daviddrysdale/c-ares/commits/test, with some key
>> points being:
>> - The tests are in C++11 (mostly so I can use gTest/gMock, using a
>> local copy of 1.7.0), which needed some extra autoconf macros.
>> - However, I've deliberately not joined up test/configure.ac to the
>> main configure.ac, so users of the library need not be blocked by a
>> missing C++ compiler.
>> - I've added the ability to inject a user-specified malloc/free, so
>> that allocation failures can be tested.
>> - I've also added a Travis config to get automatic builds on at least
>> Linux/OSX happening: https://travis-ci.org/daviddrysdale/c-ares
>> - I've also set up an initial coveralls.io page to track code
>> coverage from the Travis builds:
>> https://coveralls.io/github/daviddrysdale/c-ares
>>
>> The test/README.md file also has some description of the different
>> types of test that are possible (although there might only be a couple
>> of examples of each so far).
>>
>> Thoughts? Useful?
>
> Unit tests are always useful and this is a heap of work, very well done!
>
> While I realize the choice of unit test framework is totally subjective,
> have you considered something in pure C, like cmocka?:
> https://cmocka.org/
I didn't really consider alternatives to gTest/gMock I'm afraid -- I'm pretty
familiar with them, and I like being able to use C++ features (like RAII and
the STL).
> In addition, I was considering using socket_wrapper and running a real DNS
> server in test environment. We use something similar for SSSD tests
> where we run an OpenLDAP server during test...but it looks like your
> tests already run a mock server?
Yeah, given that c-ares effectively allows dependency injection of the
server port I haven't yet found it necessary to do more invasive things
like wrapping sockets. I did add the ability to do injection of malloc/free,
which seemed straightforward enough not to consider anything more
clever (like library preloading).
cwrap & socket_wrapper do look very interesting though; I've idly wondered
about doing similar things with Linux namespaces, but that wouldn't be
very portable. Do you know how portable the preloading approach is?
(E.g. would it work if I ever got a Windows build of the test suite running?)
And does it turn out to be brittle across library/environment changes etc?
> I applaud the choice of travis and coveralls.
>
> The fuzzer is really, really nice, do you have some documentation link
> to share or did you find the official docs
> (http://llvm.org/docs/LibFuzzer.html) sufficient?
I found the official docs for libfuzzer (and for AFL) were enough to get
going. (BTW, if you have any difficulties with it I imagine the author
would be keen to help -- let me know and I can put you in touch.)
Received on 2015-10-09