FPCUnit article/tutorial online.

classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

FPCUnit article/tutorial online.

Michael Van Canneyt
Hello,

The Editor of Toolbox Magazine has allowed me to put an article
about FPCUnit online.

FPCUnit is (similarly to JUnit, DUnit) an Object Pascal testing
framework. It comes standard with your Free Pascal/Lazarus installation.
It was written by Dean Zobec, and Vincent Snijders made
the integration in Lazarus.

The article covers basic usage, just as well as usage and
installation in Lazarus. It can be considered a tutorial
(till a proper tutorial appears)

Here is the URL:
http://www.freepascal.org/docs-html/fpcunit.pdf

Enjoy!

Michael.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: FPCUnit article/tutorial online.

Marc Santhoff
Am Sonntag, den 09.10.2005, 13:39 +0200 schrieb Michael Van Canneyt:

> The article covers basic usage, just as well as usage and
> installation in Lazarus. It can be considered a tutorial
> (till a proper tutorial appears)
>
> Here is the URL:
> http://www.freepascal.org/docs-html/fpcunit.pdf
>
> Enjoy!

Thank you very much! I hesitated to start with FPCUnit because of
lacking time for wading through source and docs, but now I'll start
testing. :)

Marc


_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: FPCUnit article/tutorial online.

L505
Since I don't exactly know what test frameworks are, even after reading about them for the
past few years, I'm going to ask some risky questions. This is not a flame thrower attempt
at the test framework advocates, I'm just trying to understand what exactly they do, from a
"newbie to test frameworks" perspective.

Do test frameworks
 -cause you to spend lots of time writing test frameworks instead of program code?
 -even apply to languages with compilers and strong typing?
 -cause you to write your programs to conform to the test framework, instead of the program
framework?
 -have any actual statistics of success?
 -have popularity statistics that prove they are popular (in the Delphi community) as stated
in the article?
 -have any real world examples of how they helped personX fix problemY in significantly less
time?

Hoping the answers are no, yes, no yes, yes, yes.

--
L505
http://z505.com

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: FPCUnit article/tutorial online.

Marc Santhoff
Am Sonntag, den 09.10.2005, 14:09 -0700 schrieb L505:

> Since I don't exactly know what test frameworks are, even after reading about them for the
> past few years, I'm going to ask some risky questions. This is not a flame thrower attempt
> at the test framework advocates, I'm just trying to understand what exactly they do, from a
> "newbie to test frameworks" perspective.
>
> Do test frameworks
>  -cause you to spend lots of time writing test frameworks instead of program code?
>  -even apply to languages with compilers and strong typing?
>  -cause you to write your programs to conform to the test framework, instead of the program
> framework?
>  -have any actual statistics of success?
>  -have popularity statistics that prove they are popular (in the Delphi community) as stated
> in the article?
>  -have any real world examples of how they helped personX fix problemY in significantly less
> time?
>
> Hoping the answers are no, yes, no yes, yes, yes.

Too much to explain here imo. I think this article will put some light
on what testing is good for and how it works:

http://junit.sourceforge.net/doc/testinfected/testing.htm

There is more starters documentation at the JUnit site if you like reading...

HTH,
Marc


_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: FPCUnit article/tutorial online.

Tony Pelton
In reply to this post by L505
i started to write a rather lengthy reply to this note, and realized i
was starting to rant.

so i'm gonna shorten my reply up quite a bit.

i'm a java web app programmer by trade. i use Pascal for fun stuff.

up until a couple of years ago, i went through a series of companies
where i couldn't escape the _nightmare_ of unit testing mantra, being
forced down the development teams throat ... _usually_ with a whole
helping of other ridicoulous development "paradigms" freshly read by
the manager right out of "CIO" magazine.

i will offer this opinion ...

i think generally, and especially the way they've been implemented in
the past where i've worked, they (unit tests) have been a huge waste
of time and a soul sucking experience for the development team.

my last 2 years, i've worked with ONE other senior developer, for a
smallish company, with a web app in production, having done ~6
releases in the last 1.5 years since we went 1.0 with the product.

in terms of scale, i would say it is just a smidgen smaller than the
web application at my last job, where there were 12 developers, 3
managers and a "technical lead".

let me express that in a formula :

webapp1 + 2 developers == 12 developers, 3 managers, lead + webapp2

the current application i work on is light years ahead of the other
one i worked on in terms of stability and quality

what is the difference ?

well, currently, me and my peer don't have several middle managers
dictating our every move.

we don't use RAD "tools"

we use simple, best of breed open source technologies where we can

we write code to solve problems when we think the investment is a
better choice for us in the long run, rather than buying some "kitchen
sink" solution from some vendor whose fancy sales guy took the boss
out to lunch one day.

we don't spend time writing unit tests.

my $0.02 ...

On 10/9/05, L505 <[hidden email]> wrote:

> Since I don't exactly know what test frameworks are, even after reading about them for the
> past few years, I'm going to ask some risky questions. This is not a flame thrower attempt
> at the test framework advocates, I'm just trying to understand what exactly they do, from a
> "newbie to test frameworks" perspective.
>
> Do test frameworks
>  -cause you to spend lots of time writing test frameworks instead of program code?
>  -even apply to languages with compilers and strong typing?
>  -cause you to write your programs to conform to the test framework, instead of the program
> framework?
>  -have any actual statistics of success?
>  -have popularity statistics that prove they are popular (in the Delphi community) as stated
> in the article?
>  -have any real world examples of how they helped personX fix problemY in significantly less
> time?
>
> Hoping the answers are no, yes, no yes, yes, yes.
>
> --
> L505
> http://z505.com
>
> _______________________________________________
> fpc-pascal maillist  -  [hidden email]
> http://lists.freepascal.org/mailman/listinfo/fpc-pascal
>
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: FPCUnit article/tutorial online.

L505

> i'm a java web app programmer by trade. i use Pascal for fun stuff.

Well I hope you try some of the Pascal web development projects too, and not JUST java for
all the web apps... ;)

I demand at least some fun pascal web apps out of you, minimum!

--
L505
http://z505.com

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: FPCUnit article/tutorial online.

Michael Van Canneyt
In reply to this post by Tony Pelton


On Sun, 9 Oct 2005, Tony Pelton wrote:

> i started to write a rather lengthy reply to this note, and realized i
> was starting to rant.
>
> so i'm gonna shorten my reply up quite a bit.
>
> i'm a java web app programmer by trade. i use Pascal for fun stuff.
>
> up until a couple of years ago, i went through a series of companies
> where i couldn't escape the _nightmare_ of unit testing mantra, being
> forced down the development teams throat ... _usually_ with a whole
> helping of other ridicoulous development "paradigms" freshly read by
> the manager right out of "CIO" magazine.
>

Your story is an example of how a 'good' technology can be ruined by
applying it too much or in an inappropriate way. Alas, this happens too
often when people who actually don't have a clue start dicating the
rules.

But that doesn't mean this technology is bad. FPC uses a testsuite
(not based on FPCunit) to test the compiler. And luckily so, because
this has helped the compiler team to eliminate bugs on many an occasion.

As in all things: use the right tools for the right job, and with moderation.

I personally wouldn't recommend using FPCunit in all cases.
The purpose of the article is to show that it is present in FPC for those who
want to use such a technology.

Michael.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: FPCUnit article/tutorial online.

Michael Van Canneyt
In reply to this post by L505


On Sun, 9 Oct 2005, L505 wrote:

> Since I don't exactly know what test frameworks are, even after reading about them for the
> past few years, I'm going to ask some risky questions. This is not a flame thrower attempt
> at the test framework advocates, I'm just trying to understand what exactly they do, from a
> "newbie to test frameworks" perspective.
>
> Do test frameworks
> -cause you to spend lots of time writing test frameworks instead of program code?

That depends entirely on you. You should not write test frameworks for
visual aspects of your application, it will take a lot of time. But any
nonvisual code can be tested with little code.

> -even apply to languages with compilers and strong typing?

Yes, of course.

> -cause you to write your programs to conform to the test framework, instead of the program
> framework?

No.

> -have any actual statistics of success?

Yes.

> -have popularity statistics that prove they are popular (in the Delphi community) as stated
> in the article?

I'm not sure what you mean by this question ?

> -have any real world examples of how they helped personX fix problemY in significantly less
> time?

I personally don't, but as far as I know, for instance InstantObjects
use unit testing and in particular FPCUnit extensively to test whether
their objects code functions correctly.

Michael.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: FPCUnit article/tutorial online.

Florian Klämpfl
In reply to this post by Michael Van Canneyt
Michael Van Canneyt wrote:

>
>
> On Sun, 9 Oct 2005, Tony Pelton wrote:
>
>> i started to write a rather lengthy reply to this note, and realized i
>> was starting to rant.
>>
>> so i'm gonna shorten my reply up quite a bit.
>>
>> i'm a java web app programmer by trade. i use Pascal for fun stuff.
>>
>> up until a couple of years ago, i went through a series of companies
>> where i couldn't escape the _nightmare_ of unit testing mantra, being
>> forced down the development teams throat ... _usually_ with a whole
>> helping of other ridicoulous development "paradigms" freshly read by
>> the manager right out of "CIO" magazine.
>>
>
> Your story is an example of how a 'good' technology can be ruined by
> applying it too much or in an inappropriate way. Alas, this happens too
> often when people who actually don't have a clue start dicating the
> rules.
>
> But that doesn't mean this technology is bad. FPC uses a testsuite
> (not based on FPCunit) to test the compiler. And luckily so, because
> this has helped the compiler team to eliminate bugs on many an occasion.

Without the regression tests of our testsuite, FPC would have much less quality
than it has now. Also code reviewing is something which helps a lot for complex
stuff.

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: FPCUnit article/tutorial online.

Dean Zobec
In reply to this post by L505
L505 ha scritto:

>Since I don't exactly know what test frameworks are, even after reading about them for the
>past few years, I'm going to ask some risky questions. This is not a flame thrower attempt
>at the test framework advocates, I'm just trying to understand what exactly they do, from a
>"newbie to test frameworks" perspective.
>  
>
I did not understand the full benefits of unit tests till I've tried them.
And it took a lot to get accustomed to them.

>Do test frameworks
> -cause you to spend lots of time writing test frameworks instead of program code?
>  
>
The function of the xUnit testing framework is to let you write tests
faster and with less effort. It's a matter of personal development
habits and preferences of course, but the greatest benefit of unit
testing I've experienced was the reduction of time spent with the
debugger (I still don't know how to use it effectively with Free Pascal
and in Lazarus I usually keep it disabled :) A failing test usually
leads me directly to the problem. The safety net provided by the tests,
written at the same time as I write the code gives me courage to do the
refactoring that I find usefull when I review the code to improve the
design,  or the changes that I need to apply when I do some profiling.
It assures  me I've not introduced collateral damage.

> -even apply to languages with compilers and strong typing?
>  
>
I has nothing to do with strong typing IMHO, I love strong typing but it
does not solve all my problems. I'm not a perfect programmer, I'm quite
successful in making mistakes and I need to be sure that a piece of code
does exactly what I thought it should do, without making assumptions and
discover the problem later, when the code has been used in other higher
level functions or frameworks

> -cause you to write your programs to conform to the test framework, instead of the program
>framework?
>  
>
Not all kind of code can benefit from unit testing, it's no use in my
opinion to try to test the GUI code, and it's difficult to test some
parts of web or db applications.
But in my case, when I'm doing unit testing of non visual code in more
complex frameworks and I find a class that is difficult to test in
isolation, I've noticed that in most occasions this is due to some flaw
in my design, often the classes proved to be too tightly coupled or had
a wrong interface. So, from my experience, writing tests has the side
effect to improve the design of my code.

> -have any actual statistics of success?
>  
>
The Free Pascal Compiler is the best example of the success of testing,
have you seen how many test have our compiler gods put in place in their
testing suite? It's not fpcunit but the effect is the same. It was the
first think I've noticed when I checked out the code from cvs the first
time. Then I've noticed that each time a bug was found it was pinned
down in a test to prevent it to emerge again, and I've begin to learn
some other good tricks from them.

> -have popularity statistics that prove they are popular (in the Delphi community) as stated
>in the article?
>  
>
I'm quite sure that the folks at Borland don't do a lot of unit testing
;) after all DUnit and NUnit are open source products :), but yes, in
the open source Delphi community they are quite popular, see the Jedi
JCL project that uses DUnit for example:
http://cvs.sourceforge.net/viewcvs.py/jcl/qa/automated/dunit/units/
The developers that took over the InstantObjects framework development
are now successfully using FPCUnit to begin to understand the complex IO
framework, pinpointing the code to be able to do some changes safely and
maybe to pave the way for the  Lazarus port. (See
http://cvs.sourceforge.net/viewcvs.py/instantobjects/Source/Tests/)

The Mono project is successfully using NUnit too.


> -have any real world examples of how they helped personX fix problemY in significantly less
>time?
>  
>
It's not about fixing a problem in my experience. It's about maintaining
the same level of productivity when the system grows. To be able to do
fearless refactoring in the phase of  reviewing the code when
discovering some "bad smells", to be able to improve the design.
I find it difficult to begin some large refactoring without a solid base
of tests in place. When I publish some code I prefer to thoroughly test
it before and when I submit a bug report I found practical to provide a
failing test that isolates it. Unit Tests are not a panacea to all the
problems, but they make my work easier.  It helped  me to drastically
reduce the  time I've spent debugging.  It took some time and effort to
learn to write good tests but for me it was a good investment.  By doing
the tests as I write the code I've learned to spend less time in
developing good tests (it's like mowing a lawn, when the grass is small
it's easier). It keeps my productivity stable even when the code is
getting complex and it even helps me to isolate a reported bug in my
code. Now I'm really having fun in coding.

>Hoping the answers are no, yes, no yes, yes, yes.
>  
>
The answer was quite more articulated and subjective :) there's no use
in saying that unit testing is good for everyone. If the developer
enjoys doing it, as I do, it's a good thing in my opinion. But I'm
convinced that there's not one true way of writing good quality
software, every one has to find the one best suited to his needs.

Dean

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: FPCUnit article/tutorial online.

Alan Mead
In reply to this post by Michael Van Canneyt
Michael Van Canneyt <[hidden email]> wrote:

> Hello,
>
> The Editor of Toolbox Magazine has allowed me to put an article
> about FPCUnit online.
> [...]

I found the article and the discussion on this list very helpful.
Thanks for writing it and making it available.

These are probably stupid questions, but all the examples I've read
about use silly tests like 1+1=2 or checking that a list is empty.
I'm having trouble seeing how I would write tests for my code.

I have an ugly little hack that reads some logged data and counts
certain things.  The code is fragile so I'd love to add unit
testing...  But the code is mostly procedures for reading and parsing
the log data.  Would the unit tests create fake input and compare it
to known output?  (How do I fake reading data from disk?)  For that
matter, how do I test the data-reading procedures?  (Write fake data
and then see if the routine reads it correctly?)

Or, I've been messing around with SQLite... to add unit tests for
this code, would I need to create fake databases?

And what about the GUI code?  How can that be tested?  

Thanks again for the article and stimulating discussion.

-Alan
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: FPCUnit article/tutorial online.

Sebastian Günther
In reply to this post by Florian Klämpfl
Florian Klaempfl schrieb:
>
> Without the regression tests of our testsuite, FPC would have much less quality
> than it has now. Also code reviewing is something which helps a lot for complex
> stuff.

Some weeks ago I've discussed unit testing with a friend (who is working
in a larger team), as I had a light suspiciosly view on this topic. For
_complex_ systems with _clear_ structure and interfaces, they absolutely
make sense and are productive. But the entry level where they are worth
the effort is quite high. I guess that even for average industrial
automation applications (where I have used FPC in practice), the usage
of unit tests is a little bit exaggerated and won't help much.
As Michael stated: It depends on the usage case... And it's something
which I would call in German: "Da wird mal wieder eine neue Sau durchs
Dorf getrieben." (Probably not translateable to English, sorry). It's a
buzzword, managers like it, but there are cases where they really make
sense. But by far not in all cases.


- Sebastian
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: FPCUnit article/tutorial online.

L505
> Some weeks ago I've discussed unit testing with a friend (who is working
> in a larger team), as I had a light suspiciosly view on this topic. For
> _complex_ systems with _clear_ structure and interfaces, they absolutely
> make sense and are productive. But the entry level where they are worth
> the effort is quite high. I guess that even for average industrial
> automation applications (where I have used FPC in practice), the usage
> of unit tests is a little bit exaggerated and won't help much.
> As Michael stated: It depends on the usage case... And it's something
> which I would call in German: "Da wird mal wieder eine neue Sau durchs
> Dorf getrieben." (Probably not translateable to English, sorry). It's a
> buzzword, managers like it, but there are cases where they really make
> sense. But by far not in all cases.
>
>
> - Sebastian


Those were some of my suspicions too, since on for example the C2 wiki, I've read a lot
about these "tests" but I never see any or many actual code or real world examples on the C2
wiki.  Even if these tests are extremely useful in several areas, the inventors of the "test
cases" would be better off explaining the tests with real world examples, instead of all the
tricky English and tricky wording.

So in other words, "tests" could be very useful, but the suspicious human needs to see the
usefulness demonstrated first. The suspicious human also needs to clarify whether the word
"test" really means anything, since a test is a vague word (kind of like the phrase DotNet).
You just can't believe something because it was in a book written by some celebrity
programmer. We have seen the compiler can use "tests" in order to test the validity of the
compiler.  Are these "unit tests" or just "tests". It all seems vague to me. A compiler is
very specific and a special program which obviously needs to be "tested". As for whether the
FPC compiler tests have anything to do with "extreme programming", "test cases" and "unit
tests" I'm not sure.  It may be just a vague term for "general tests to do on your code", or
it may actually be something more than just that. It's their job to make it more clear.. or
if not, maybe a "unit test consulting business" will call you in for an appointment on
Tuesday.

--
L505
http://z505.com

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: FPCUnit article/tutorial online.

Dean Zobec
In reply to this post by Alan Mead
Alan Mead ha scritto:

>>Hello,
>>
>>The Editor of Toolbox Magazine has allowed me to put an article
>>about FPCUnit online.
>>[...]
>>    
>>
>
>I found the article and the discussion on this list very helpful.
>Thanks for writing it and making it available.
>
>These are probably stupid questions, but all the examples I've read
>about use silly tests like 1+1=2 or checking that a list is empty.
>I'm having trouble seeing how I would write tests for my code.
>  
>
Take a look at some real world tests to see how they were written and
then try to write yours to make practice. When writing test you should
learn to identify  the border conditions (strange cases that could give
problems to your code), you know, bugs generally live in the edges :)
like wrong input values, badly formatted data, values in excess of any
reasonable expectation, empty of missing values (null, 0, ""), unordered
values passed in input where the routine expects ordered items ... Ask
yourself some questions as Does the value conform to my expected format?
Is the value within a reasonable range of values? Is something happening
in the right order in time? Are there exactly enough parameters?
Some methods can be tested by applying their inverse: like the result of
y=sqrt(x) could be tested applying y^2 to see if it equals to x. To see
if an item was deleted from a list you perform a search after the deletion.
For some examples of more complex unit tests with fpcunit you can look
at the tests we wrote for  the EOS framework: a sort of simple xml
object persistence framework  that automatically saves the published
properties of objects and complex structures of objects to xml and
automatically recreates the objects from the xml text (it was used to
transport those objects over http from the server to the client and vice
versa)
Here you can find the code:
http://cvs.sourceforge.net/viewcvs.py/camelos/EOS/fpcEOS/
and here are the related tests:
http://cvs.sourceforge.net/viewcvs.py/camelos/EOS/fpcEOS/tests/

>I have an ugly little hack that reads some logged data and counts
>certain things.  The code is fragile so I'd love to add unit
>testing...  But the code is mostly procedures for reading and parsing
>the log data.  Would the unit tests create fake input and compare it
>to known output?  (How do I fake reading data from disk?)  For that
>matter, how do I test the data-reading procedures?  (Write fake data
>and then see if the routine reads it correctly?)
>  
>
Yes exactly, obviously you'll also write strange faked data to see how
your procedures and functions respond. It is much more difficult to
write tests when all the code is already written, so begin with writing
tests for low level simple methods first and when they pass you can
begin to test higher level methods. This is the natural way of
proceeding when you write tests at the same time that you write the code .
You'll see that the tests you'll find here
http://cvs.sourceforge.net/viewcvs.py/camelos/EOS/fpcEOS/tests/
are testing functionality very similar to your case hope you'll find
them useful. Some where even written over legacy code like in your case.
If you have specific questions you can even write me in private mail,
I'll try to help you if I can (here we would be off-topic).

>Or, I've been messing around with SQLite... to add unit tests for
>this code, would I need to create fake databases?
>  
>
>And what about the GUI code?  How can that be tested?
>
This are two of the cases where testing is most difficult. Some suggest
writing mock objects that mock the real behaviour
http://www.mockobjects.com .  But is generally difficult to implement in
case of databases.
For the GUI I'm against testing it using unit testing, it's a waste of time.

If you should need a book to learn more about the testing techniques
from a very practical point of view, with many examples, I suggest this:

Pragmatic Unit Testing
in Java with JUnit
by Andy Hunt and Dave Thomas



Dean
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: FPCUnit article/tutorial online.

Olle Raab
In reply to this post by L505
05-10-11 23.52, skrev L505 följande:

> programmer. We have seen the compiler can use "tests" in order to test the
> validity of the
> compiler.  Are these "unit tests" or just "tests". It all seems vague to me. A
> compiler is

A unit test should test a small module of a program, like a pascal unit.
Since most of the tests in the FPC compiler test suite do test the compiler
as a whole, they are not unit tests, rather some kind of system tests (I
think). Which does not make them less useful.

Olle

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/mailman/listinfo/fpc-pascal