data driven fpc unit tests

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

data driven fpc unit tests

Vincent Snijders
Hi,

Suppose I have a component TMyDataProcessor with the following declaration:

type
   TMyDataProcessor = class
      function process(const s: string) : string;
   end;

Now I want to test it with different strings, but instead of hard coding
them I put them in a inifile, so I can easily extend the test.

[test 1]
Input=Test
Output=tEST

[test 2]
Input=1234
Output=1234

Now I write a fpcunit test that reads those tests input/output pairs
from the ini file and executes them, see following (psuedo) code:

procedure TMyTestCase.TestOutputs;
var
   MyDataProcessor: TMyDataProcessor;
begin
   for each section in the ini-file do begin
      MyDataProcessor := TMyDataProcessor.Create;
      Load Input, Load Output
      AssertEquals(SectionName + 'failed. ',
         Output, MyDataProcessor(Input));
      MyDataProcessor.Free;
   end;
end;

As far as I can see, the drawback of using this method is that if there
is a failure in test 1, test 2 won't run at all. I will loose
information about what could have been wrong.

A solution would be to have different test methods for each test in the
ini file, but I want to avoid that, because then adding a test, will
mean that I need to change the code (after having created the data).

Do you know how I can create data driven tests, so that it will display
the test results of the second test, even if the first test fails.

Vincent
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: data driven fpc unit tests

Dean Zobec
Vincent Snijders pravi:

> Hi,
>
> Suppose I have a component TMyDataProcessor with the following declaration:
>
> type
>   TMyDataProcessor = class
>      function process(const s: string) : string;
>   end;
>
> Now I want to test it with different strings, but instead of hard coding
> them I put them in a inifile, so I can easily extend the test.
>
> [test 1]
> Input=Test
> Output=tEST
>
> [test 2]
> Input=1234
> Output=1234
>
> Now I write a fpcunit test that reads those tests input/output pairs
> from the ini file and executes them, see following (psuedo) code:
>
> procedure TMyTestCase.TestOutputs;
> var
>   MyDataProcessor: TMyDataProcessor;
> begin
>   for each section in the ini-file do begin
>      MyDataProcessor := TMyDataProcessor.Create;
>      Load Input, Load Output
>      AssertEquals(SectionName + 'failed. ',
>         Output, MyDataProcessor(Input));
>      MyDataProcessor.Free;
>   end;
> end;
>
> As far as I can see, the drawback of using this method is that if there
> is a failure in test 1, test 2 won't run at all. I will loose
> information about what could have been wrong.
>
> A solution would be to have different test methods for each test in the
> ini file, but I want to avoid that, because then adding a test, will
> mean that I need to change the code (after having created the data).
>
> Do you know how I can create data driven tests, so that it will display
> the test results of the second test, even if the first test fails.
>
> Vincent
Yes, there is an elegant solution. Joost has already asked me the same
question two months ago and I've prepared in that occasion a small
example of parameterized testcase that can be used for data driven
tests. The file with a short comment is attached.
HTH,
Dean


unit paramstests;

{$mode objfpc}{$H+}

interface

uses
  Classes, SysUtils, fpcunit, testutils, testregistry;

type
{sometimes the different tests to run have a lot of common functionality
and they differ only for the presence of a different parameter, or a different
property of the class under test. To remove duplication and unnecessary code
there is the possibility to manually build a parameterized test suite, see the
following example that can be used as a template. The steps are simple, create
a new constructor for the testcase that has the parameters you'll want to pass
in the tests (in the following examples the parameters are a string input parameter
and an integer parameter to use for the testing of the results). In this constructor
you have to call the inherited CreateWithName constructor passing the name of the
actual test (in the following code the actual test is the DoTest procedure that
performs a simple test of the lenght function for the string passed as parameter to
the testcase). The parameters are then stored in some private fileds of the testcase.
Then you'll have to construct the test suite manually, see the
Suite class function, passing the constructor of the testcase with the wanted parameters.
Finally you'll add the constructed test to the test registry using the
GetTestRegistry.AddTest function in the initialization section.
In this example you'll see that four isolated different tests with the same name (DoTest)
will be created. The only drawback is in the fact that in case of failure it's not
immediate to see which test went wrong, as the tests have the same name. I suggest
to pass some information in the string description of the extended assertequals function
as a hint (in the DoTest example below I've inserted the parameter that was passed into the test)

The same parameterized testcase could be constructed by loading the parameters
from an xml file or from an ini file instead of storing them in the private fields
of the testcase. I leave this simple implementation as an exercise for the reader :)


  { TParameterizedTestCase }

  TParameterizedTestCase = class(TTestCase)
  private
    aParam: string;
    aOut: integer;
  public
    constructor Create(const aParameter: string; aOutput: integer); virtual; overload;
    class function Suite: TTestSuite;
  published
    procedure DoTest;
  end;

implementation

{ TParameterizedTestCase }

constructor TParameterizedTestCase.Create(const aParameter: string; aOutput: integer);
begin
  //create a new DoTest testcase
  inherited CreateWithName('DoTest');
  //store the parameters
  aParam := aParameter;
  aOut := aOutput;
end;

class function TParameterizedTestCase.Suite: TTestSuite;
begin
//manually create the test suite
  Result := TTestSuite.Create('TParameterizedTestCase');
  Result.AddTest(TParameterizedTestCase.Create('', 0));
  Result.AddTest(TParameterizedTestCase.Create('o', 1));
  Result.AddTest(TParameterizedTestCase.Create('two', 3));
  Result.AddTest(TParameterizedTestCase.Create('three', 5));
end;

procedure TParameterizedTestCase.DoTest;
begin
//insert here the common functionality of the parameterized test
//here for example we are testing the Length function:
  AssertEquals('test ' + aParam, aOut, length(aParam));
  // notice that the 'test ' + aParam description was added to be able to
  //distinguish which test went wrong
end;

initialization
  //register the manually created suite
  GetTestRegistry.AddTest(TParameterizedTestCase.Suite);
end.


_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: data driven fpc unit tests

Vincent Snijders
Dean Zobec wrote:

> Vincent Snijders pravi:
>
>>Hi,
>>
>>Suppose I have a component TMyDataProcessor with the following declaration:
>>
>>type
>>  TMyDataProcessor = class
>>     function process(const s: string) : string;
>>  end;
>>
>>Now I want to test it with different strings, but instead of hard coding
>>them I put them in a inifile, so I can easily extend the test.
>>
>>[test 1]
>>Input=Test
>>Output=tEST
>>
>>[test 2]
>>Input=1234
>>Output=1234
>>
>>Now I write a fpcunit test that reads those tests input/output pairs
>>from the ini file and executes them, see following (psuedo) code:
>>
>>procedure TMyTestCase.TestOutputs;
>>var
>>  MyDataProcessor: TMyDataProcessor;
>>begin
>>  for each section in the ini-file do begin
>>     MyDataProcessor := TMyDataProcessor.Create;
>>     Load Input, Load Output
>>     AssertEquals(SectionName + 'failed. ',
>>        Output, MyDataProcessor(Input));
>>     MyDataProcessor.Free;
>>  end;
>>end;
>>
>>As far as I can see, the drawback of using this method is that if there
>>is a failure in test 1, test 2 won't run at all. I will loose
>>information about what could have been wrong.
>>
>>A solution would be to have different test methods for each test in the
>>ini file, but I want to avoid that, because then adding a test, will
>>mean that I need to change the code (after having created the data).
>>
>>Do you know how I can create data driven tests, so that it will display
>>the test results of the second test, even if the first test fails.
>>
>>Vincent
>
>
> Yes, there is an elegant solution. Joost has already asked me the same
> question two months ago and I've prepared in that occasion a small
> example of parameterized testcase that can be used for data driven
> tests. The file with a short comment is attached.
> HTH,
> Dean

Yes, reading it, I think I understand it.

Of course the proof will be when I have used and implemented it for my
purposes.

Thanks a lot.

Vincent

P.S. Maybe you can add this the wiki.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal