Software Automated Testing and Framework

Software automated testing and framework

The idea of software automated testing is always there, just how to implement it.

Java has a JUnit testing framework, which looks ok. Other programming languages have their own testing framework, which looks similar.

This is a sample of the output of JUnit:

Tests run: 20, Failures: 0, Errors: 0, Skipped: 0

JUnit 4

JUnit has different versions like version 3, 4, and 5. I am using version 4 as an example here, because it looks simple.

JUnit follow some naming convention and style. The testing program always end with Test, the test case function starts with test. eg. CalculatorTest.testPlus().

If you use maven, JUnit is integrated into the system, just $ mvn test; will do the job. I am using the command line and manual way to shows to create and run the test case.

Let's say this is the program you want to test:

class Calculator {
    int plus(int a, int b) {
        return a+b;
    }
}
    

This is your JUnit test case:

import static org.junit.Assert.assertEquals;
import org.junit.Test;

public class CalculatorTest
{
    @Test
    public void testPlus() {
        Calculator cal= new Calculator();
        assertEquals(50, cal.plus(20,30) );
    }
}

Note: JUnit 4 looks simple, need Java 1.5 or above to compile.

$ javac -classpath junit-4.11.jar:. CalculatorTest.java 
⇒ CalculatorTest.class  Calculator.class
$ java -classpath junit-4.11.jar:. CalculatorTest 
⇒ Error: Main method not found in class CalculatorTest
$ java -classpath junit-4.11.jar:. org.junit.runner.JUnitCore CalculatorTest
Exception in thread "main" java.lang.NoClassDefFoundError: org/hamcrest/SelfDescribing
  

$ java -classpath junit-4.11.jar:hamcrest-core-1.3.jar:. org.junit.runner.JUnitCore CalculatorTest
Time: 0.01
OK (1 test)

The JUnit 4 software can be downloaded from https://github.com/junit-team/junit4/wiki/Download-and-Install

Cmocka for C programm

There is a google testing framework which is more powerful. I choose cmocka, because it is small and simple.

To install:
Debian: $ sudo apt-get install cmocka
macOS: $ brew install cmocka

$ edit calculator.c

int plus(int a, int b)
{
	return a+b;
}

$ edit calculator.h

int plus(int, int);
  

$ edit calculator_test.c

#include 
#include 
#include 
#include "cmocka.h"
#include 
#include 
#include "calculator.h"

static void test_plus(void **state) {
    (void) state; /* unused */
    
    assert_int_equal(50,plus(20,30));
    assert_int_equal(8,plus(5,3));
}

int main(void)
{
    const struct CMUnitTest tests[] = {
        cmocka_unit_test(test_plus),
    };

    return cmocka_run_group_tests(tests, NULL, NULL);
}

To compile and test:
$ cc -c calculator.c
==> calculator.o
$ cc -c `pkg-config --cflags cmocka` calculator_test.c
$ cc -o calculator_test calculator_test.o calculator.o `pkg-config --libs cmocka`

If you don't have pkg-config:
$ cc -c calculator_test.c
$ cc -o calculator_test calculator_test.o calculator.o -lcmocka

==> calculator_test*

Run test case:

./calculator_test 
[==========] Running 1 test(s).
[ RUN      ] test_plus
[       OK ] test_plus
[==========] 1 test(s) run.
[  PASSED  ] 1 test(s).

If integrate with automake. Use libgrowl as an example:
$ git clone https://github.com/fuyichin/libgrowl
$ make check

======================================
Testsuite summary for libgrowl 0.1.0
======================================
# TOTAL: 6
# PASS:  6
# SKIP:  0
# XFAIL: 0
# FAIL:  0

Note: As the time of writing this now, the automake of libgrowl is not integrated into the main branch yet. Can try with the branch autogen.

Other programming like Python and Bourne shell has their own testing framework as well.