Challenge accepted

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Challenge accepted

Joost van der Sluis
Hi all,

Few months ago I read somewhere on the fpc-mailinglists that someone
stated that FPC is fine, but it could use more additional stuff.

I really searched, but I can not find the original mail anymore. But I
want to accept that challenge.

One of the thinks I think the Pascal-world is still missing is a good
streaming library. Or, in fact I mean more something for
(de-)serializing objects.

Normally I just start coding on such a project, but maybe we can make it
more of a group-project.

First of all, we have to get some functional specifications. Let me start:

At least it should be able to:
- stream to XML, Json and binary formats
- Selecting the format should be done by some sort of dependency-injection
- It should be possible to omit fields based on the whole class, but
also based on a single instance of the class
- It should be possible to stream to/from *all* classes
- Classes can modify how they are streamed by implementing some
interfaces. (Maybe later also based on attributes)
- It should also be possible to let the 'outer world' decide how a class
should be streamed.
- It must be possible to change how properties are 'formatted'

I can go on, but these are the most important I can think of now.

At the moment I'm thinking about a design in which there is some class
which defines how the class has to be mapped to the stream. Then this
'map' and the instance of the class are given to some other class that
does the streaming, using the right (injected) stream-format.

Normally I would submit such code to fpcprojects (svn) but maybe Michael
can help setting up a git-repository in the new fpc git-environment we
are currently testing with?

And who else wanna help?

Regards.

Joost.


_______________________________________________
fpc-pascal maillist  -  [hidden email]
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Challenge accepted

Marco van de Voort-2

Op 2019-08-10 om 17:30 schreef Joost van der Sluis:
>
>
> And who else wanna help?
>
Why not simply port superobject?
_______________________________________________
fpc-pascal maillist  -  [hidden email]
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Challenge accepted

Michael Van Canneyt


On Sat, 10 Aug 2019, Marco van de Voort wrote:

>
> Op 2019-08-10 om 17:30 schreef Joost van der Sluis:
>>
>>
>> And who else wanna help?
>>
> Why not simply port superobject?

2 reasons:
1. Interface based. Really bad idea.
2. JSON only.

And if you need more reasons: the jsonrtti has just this.
I still need to add some functionality to it from the restbase unit (dynamic
arrays) but other than that I've been using it since ages.

Michael.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Challenge accepted

Joost van der Sluis
Op 10-08-19 om 18:37 schreef Michael Van Canneyt:

> On Sat, 10 Aug 2019, Marco van de Voort wrote:
>> Op 2019-08-10 om 17:30 schreef Joost van der Sluis:
>>>
>>>
>>> And who else wanna help?
>>>
>> Why not simply port superobject?
>
> 2 reasons:
> 1. Interface based. Really bad idea.

There can be a really good use for interfaces regarding streaming. Like
you saw earlier in the fppkg-repository code.

> 2. JSON only.

3. I didn't really know it.

> And if you need more reasons: the jsonrtti has just this. I still need
> to add some functionality to it from the restbase unit (dynamic
> arrays) but other than that I've been using it since ages.

I use jsonrtti now, and with some additions it works like I want. But
the goal is somewhat more generic, yes.

Regards,

Joost.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Challenge accepted

Michael Van Canneyt


On Sun, 11 Aug 2019, Joost van der Sluis wrote:

> Op 10-08-19 om 18:37 schreef Michael Van Canneyt:
>> On Sat, 10 Aug 2019, Marco van de Voort wrote:
>>> Op 2019-08-10 om 17:30 schreef Joost van der Sluis:
>>>>
>>>>
>>>> And who else wanna help?
>>>>
>>> Why not simply port superobject?
>>
>> 2 reasons:
>> 1. Interface based. Really bad idea.
>
> There can be a really good use for interfaces regarding streaming. Like
> you saw earlier in the fppkg-repository code.

I think the only case when interfaces are truly needed is in IPC/RPC.
For the rest it just complicates otherwise simple things.

I will use them when useful, which I think is far less than commonly assumed.
As such I'm violently opposed to the 'you must always program an interface' attitude.

>
>> 2. JSON only.
>
> 3. I didn't really know it.

You don't miss out on that. It's better than the mess Embarcadero produced,
but still needlessly complicated IMHO.

>
>> And if you need more reasons: the jsonrtti has just this. I still need
>> to add some functionality to it from the restbase unit (dynamic
>> arrays) but other than that I've been using it since ages.
>
> I use jsonrtti now, and with some additions it works like I want. But
> the goal is somewhat more generic, yes.

I understood that, I just compared it to superobject, not to your eventual
goal.

What additions are you talking about ? Can we incorporate it ? Improvements are always welcome.

Note that the SQLDB rest bridge has dataset result streaming in various formats.
It's somewhat similar to your goal, but is limited to datasets.

Michael.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Challenge accepted

Joost van der Sluis
Op 11-08-19 om 12:28 schreef Michael Van Canneyt:
>
> What additions are you talking about ? Can we incorporate it ?
> Improvements are always welcome.

Maybe the ability to handle read- or write-only properties. And a flag
to allow that properties which are not available in the JSON, are simply
skipped. (Not by using the allow-errors flag)

> Note that the SQLDB rest bridge has dataset result streaming in various
> formats.
> It's somewhat similar to your goal, but is limited to datasets.

Does remind me. It should also be able to stream datasets to something
else. And offcourse to stream objects to a dataset.

Another thing, can you help me setting up some kind of git-repository on
the FPC-infrastructure? Or don't you have any plans in that direction?

Regards,

Joost.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Challenge accepted

Cyrax
On 15/08/2019 00:14, Joost van der Sluis wrote:
> Op 11-08-19 om 12:28 schreef Michael Van Canneyt:
[snip]

>
> Another thing, can you help me setting up some kind of git-repository on
> the FPC-infrastructure? Or don't you have any plans in that direction?
>
> Regards,
>
> Joost.
> _______________________________________________
> fpc-pascal maillist  -  [hidden email]
> https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

You could do that by yourself by using local svn mirror of FPC (so you
don't stress orginal source if the git mirror creation goes awry)
repository and subgit (<url:https://subgit.com/> needs java (!!!) [use
64-bit one] and decent amount memory, 16GB+ is suffcient and some swap
space, too).

subgit saves whole SVN history beatifully and have some speed
improvements and other optimizations over standard git-svn command.

_______________________________________________
fpc-pascal maillist  -  [hidden email]
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Challenge accepted

Joost van der Sluis
In reply to this post by Joost van der Sluis
Hi all,

I've put some code, documentation and examples online for the
serialization-library.

Very rough, but I think it shows my ideas. You can find it here:
https://wiki.freepascal.org/cnocstream

The name, classnames etc may change. Everyone with good ideas, they are
welcome.

There is no support for the serialization of object-lists or collections
(only arrays). Adding this might lead some large refactorings. (I'm also
not sure if I'm content with the property and items classes)

But please have a look. I'm off for vacation now, so I'll respond only
next week.

Regards,

Joost.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal