Software Automated Testing and Framework 2

Software Automated Testing and Framework 2

Python and bourne shell script

Follow up with the previous tutorial (a href="http://fuyichin.blogspot.com/2022/03/software-automated-testing-and-framework.html">here), testing framework for Java (JUnit) and C (cmocka). This tutorial continue with Python and bourne shell script.

Python

$ edit calculator.py

#!/usr/local/bin/python3
def plus(a,b):
	return a+b
  

$ edit calculator_test.py

import unittest
import calculator

class CalculatorTest(unittest.TestCase):
	def test_plus(self):
		self.assertEqual(51,calculator.plus(20,30))

if __name__ == '__main__':
	unittest.main()
  

Run the test case:
$ python3 calculator_test.py

  F
======================================================================
FAIL: test_plus (__main__.CalculatorTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/hooichee/workspace/test_maven/python/calculator_test.py", line 6, in test_plus
    self.assertEqual(51,calculator.plus(20,30))
AssertionError: 51 != 50
  

Change the result and run the test case again.

Ran 1 test in 0.000s
OK

Bourne shell script

I am using Shunit2, can be downloaded from https://github.com/kward/shunit2. There are many other testing framework are available, I just use this as an example.

$ edit calculator.sh

#!/bin/sh
plus() {
	echo `expr $1 + $2`
}

$ edit calculator_test.sh

#!/bin/sh
source calculator.sh

testPlus() {
  assertEquals 100 `plus 99 1`
  assertEquals 'Test plus' 50 `plus 20 30`
}

# Load and run shUnit2.
#. ../shunit2
. /usr/local/bin/shunit2

Run test:

$ sh calculator_test.sh 
testPlus

Ran 1 test.

OK

Software Automated Testing and Framework

Software automated testing and framework

The idea of software automated testing is always there, just how to implement it.

Java has a JUnit testing framework, which looks ok. Other programming languages have their own testing framework, which looks similar.

This is a sample of the output of JUnit:

Tests run: 20, Failures: 0, Errors: 0, Skipped: 0

JUnit 4

JUnit has different versions like version 3, 4, and 5. I am using version 4 as an example here, because it looks simple.

JUnit follow some naming convention and style. The testing program always end with Test, the test case function starts with test. eg. CalculatorTest.testPlus().

If you use maven, JUnit is integrated into the system, just $ mvn test; will do the job. I am using the command line and manual way to shows to create and run the test case.

Let's say this is the program you want to test:

class Calculator {
    int plus(int a, int b) {
        return a+b;
    }
}
    

This is your JUnit test case:

import static org.junit.Assert.assertEquals;
import org.junit.Test;

public class CalculatorTest
{
    @Test
    public void testPlus() {
        Calculator cal= new Calculator();
        assertEquals(50, cal.plus(20,30) );
    }
}

Note: JUnit 4 looks simple, need Java 1.5 or above to compile.

$ javac -classpath junit-4.11.jar:. CalculatorTest.java 
⇒ CalculatorTest.class  Calculator.class
$ java -classpath junit-4.11.jar:. CalculatorTest 
⇒ Error: Main method not found in class CalculatorTest
$ java -classpath junit-4.11.jar:. org.junit.runner.JUnitCore CalculatorTest
Exception in thread "main" java.lang.NoClassDefFoundError: org/hamcrest/SelfDescribing
  

$ java -classpath junit-4.11.jar:hamcrest-core-1.3.jar:. org.junit.runner.JUnitCore CalculatorTest
Time: 0.01
OK (1 test)

The JUnit 4 software can be downloaded from https://github.com/junit-team/junit4/wiki/Download-and-Install

Cmocka for C programm

There is a google testing framework which is more powerful. I choose cmocka, because it is small and simple.

To install:
Debian: $ sudo apt-get install cmocka
macOS: $ brew install cmocka

$ edit calculator.c

int plus(int a, int b)
{
	return a+b;
}

$ edit calculator.h

int plus(int, int);
  

$ edit calculator_test.c

#include 
#include 
#include 
#include "cmocka.h"
#include 
#include 
#include "calculator.h"

static void test_plus(void **state) {
    (void) state; /* unused */
    
    assert_int_equal(50,plus(20,30));
    assert_int_equal(8,plus(5,3));
}

int main(void)
{
    const struct CMUnitTest tests[] = {
        cmocka_unit_test(test_plus),
    };

    return cmocka_run_group_tests(tests, NULL, NULL);
}

To compile and test:
$ cc -c calculator.c
==> calculator.o
$ cc -c `pkg-config --cflags cmocka` calculator_test.c
$ cc -o calculator_test calculator_test.o calculator.o `pkg-config --libs cmocka`

If you don't have pkg-config:
$ cc -c calculator_test.c
$ cc -o calculator_test calculator_test.o calculator.o -lcmocka

==> calculator_test*

Run test case:

./calculator_test 
[==========] Running 1 test(s).
[ RUN      ] test_plus
[       OK ] test_plus
[==========] 1 test(s) run.
[  PASSED  ] 1 test(s).

If integrate with automake. Use libgrowl as an example:
$ git clone https://github.com/fuyichin/libgrowl
$ make check

======================================
Testsuite summary for libgrowl 0.1.0
======================================
# TOTAL: 6
# PASS:  6
# SKIP:  0
# XFAIL: 0
# FAIL:  0

Note: As the time of writing this now, the automake of libgrowl is not integrated into the main branch yet. Can try with the branch autogen.

Other programming like Python and Bourne shell has their own testing framework as well.