FCL-WEB and PostgreSQL Support: Current Status

classic Classic list List threaded Threaded
18 messages Options
Reply | Threaded
Open this post in threaded view
|

FCL-WEB and PostgreSQL Support: Current Status

African Wild Dog
Hello,

I am about to start a new project for a very small web application.

I have some questions:

1 - Are the fcl-web and postgres packages stable?
2 - Are they recommended for use in production environments?
3 - Do they have any serious bug (for example, memory leaks)?

Regards

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: FCL-WEB and PostgreSQL Support: Current Status

Michael Van Canneyt


On Wed, 27 Sep 2017, African Wild Dog wrote:

> Hello,
>
> I am about to start a new project for a very small web application.
>
> I have some questions:
>
> 1 - Are the fcl-web and postgres packages stable?

Since 8 years.

> 2 - Are they recommended for use in production environments?

Absolutely.
I use them in very high-load (and we're talking VERY high load)
environments, with 24/7 availability.

> 3 - Do they have any serious bug (for example, memory leaks)?

None that I know of.

Michael.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: FCL-WEB and PostgreSQL Support: Current Status

Karoly Balogh (Charlie/SGR)
Hi,

On Wed, 27 Sep 2017, Michael Van Canneyt wrote:

> > 2 - Are they recommended for use in production environments?
>
> Absolutely.
> I use them in very high-load (and we're talking VERY high load)
> environments, with 24/7 availability.

Can you ellaborate a bit more on this Michael? Like how and where it is
used? (Not mentioning company names, if it's a problem.) I was thinking
about using it in various places, but as I don't know how widely they're
tested and used and under what conditions, I didn't had the balls to offer
this as viable alternative against competing technology X.

We definitely need more use case studies...

Charlie
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: FCL-WEB and PostgreSQL Support: Current Status

African Wild Dog
In reply to this post by Michael Van Canneyt
2017-09-27 12:29 GMT-03:00 Michael Van Canneyt <[hidden email]>:

2 - Are they recommended for use in production environments?

Absolutely. I use them in very high-load (and we're talking VERY high load)
environments, with 24/7 availability.


Could you tell which web server you are using in this environment? Are you using FastCGI?

Regards

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: FCL-WEB and PostgreSQL Support: Current Status

Michael Van Canneyt


On Wed, 27 Sep 2017, African Wild Dog wrote:

> 2017-09-27 12:29 GMT-03:00 Michael Van Canneyt <[hidden email]>:
>
>>
>> 2 - Are they recommended for use in production environments?
>>>
>>
>> Absolutely. I use them in very high-load (and we're talking VERY high load)
>> environments, with 24/7 availability.
>>
>>
> Could you tell which web server you are using in this environment? Are you
> using FastCGI?

Apache, indeed using FastCGI.
Used both mod_fastcgi as mod_fcgid. mod_proxy should work as well.

Michael.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: FCL-WEB and PostgreSQL Support: Current Status

Michael Van Canneyt
In reply to this post by Karoly Balogh (Charlie/SGR)


On Wed, 27 Sep 2017, Karoly Balogh (Charlie/SGR) wrote:

> Hi,
>
> On Wed, 27 Sep 2017, Michael Van Canneyt wrote:
>
>> > 2 - Are they recommended for use in production environments?
>>
>> Absolutely.
>> I use them in very high-load (and we're talking VERY high load)
>> environments, with 24/7 availability.
>
> Can you ellaborate a bit more on this Michael? Like how and where it is
> used? (Not mentioning company names, if it's a problem.) I was thinking
> about using it in various places, but as I don't know how widely they're
> tested and used and under what conditions, I didn't had the balls to offer
> this as viable alternative against competing technology X.

It is extremely viable.

I have used it in 2 companies, first production dating back to 2010.

Mainly it's used to offer read/write REST apis, with access to Firebird or
Postgres databases. You can add some Cassandra, MemCached and Redis into
the mix, and even (yeeeeeecch) mysql.

It's also used as a middle layer on top Microsoft/Google APIS to offer a
single, unified API to access google docs or onedrive to the REST client.
Thus, the oauth2, json apis of FPC are well tested, as well as the webclient
units.

It also offers a variety of export data (CSV/XML/HTML/Excel).

All 24/7.

The new fpreport is used to provide reporting services - outputting PDF and
sending that directly to the browser.

All this has been tested heavily on Linux and Windows.

So yes, I can say it is safe to use in production.

And it will only get better once pas2js is released (expected somewhere in
December). FPC will offer a full web stack that sends NodeJS reeling...

Michael.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: FCL-WEB and PostgreSQL Support: Current Status

Mark Morgan Lloyd-5
In reply to this post by Karoly Balogh (Charlie/SGR)
On 27/09/17 15:45, Karoly Balogh (Charlie/SGR) wrote:
> Hi,
> On Wed, 27 Sep 2017, Michael Van Canneyt wrote:
>>> 2 - Are they recommended for use in production environments?>> Absolutely.> I use them in very high-load (and we're talking VERY high load)> environments, with 24/7 availability.
> Can you ellaborate a bit more on this Michael? Like how and where it isused? (Not mentioning company names, if it's a problem.) I was thinkingabout using it in various places, but as I don't know how widely they'retested and used and under what conditions, I didn't had the balls to offerthis as viable alternative against competing technology X.
> We definitely need more use case studies...

The only problems I've experienced with FPC's support for PostgreSQL
were related to handling SQL queries (e.g. spurious parameter
substitution) rather than to dataflow. As such they were predictable and
could be worked round.

--
Mark Morgan Lloyd
markMLl .AT. telemetry.co .DOT. uk

[Opinions above are the author's, not those of his employers or colleagues]
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: FCL-WEB and PostgreSQL Support: Current Status

Marco van de Voort
In our previous episode, Mark Morgan Lloyd said:
> > Can you ellaborate a bit more on this Michael? Like how and where it isused? (Not mentioning company names, if it's a problem.) I was thinkingabout using it in various places, but as I don't know how widely they'retested and used and under what conditions, I didn't had the balls to offerthis as viable alternative against competing technology X.
> > We definitely need more use case studies...
>
> The only problems I've experienced with FPC's support for PostgreSQL
> were related to handling SQL queries (e.g. spurious parameter
> substitution) rather than to dataflow. As such they were predictable and
> could be worked round.

Zeljan once explained an interesting practical problem he had.

He said the connection per transaction model could be dangerous, specially
in a mobile era where connections are often broken, and thus must wait on
timeout.  This means during rush times, a server can run out of a fairly
limited ephemeral port inventory before abandoned connections start to time
out.

That is a postgres problem though, not the FPC components.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: FCL-WEB and PostgreSQL Support: Current Status

Michael Van Canneyt


On Wed, 27 Sep 2017, Marco van de Voort wrote:

> In our previous episode, Mark Morgan Lloyd said:
>> > Can you ellaborate a bit more on this Michael? Like how and where it isused? (Not mentioning company names, if it's a problem.) I was thinkingabout using it in various places, but as I don't know how widely they'retested and used and under what conditions, I didn't had the balls to offerthis as viable alternative against competing technology X.
>> > We definitely need more use case studies...
>>
>> The only problems I've experienced with FPC's support for PostgreSQL
>> were related to handling SQL queries (e.g. spurious parameter
>> substitution) rather than to dataflow. As such they were predictable and
>> could be worked round.
>
> Zeljan once explained an interesting practical problem he had.
>
> He said the connection per transaction model could be dangerous, specially
> in a mobile era where connections are often broken, and thus must wait on
> timeout.  This means during rush times, a server can run out of a fairly
> limited ephemeral port inventory before abandoned connections start to time
> out.
>
> That is a postgres problem though, not the FPC components.

1. In a REST setting, if you use FastCGI and connnection pooling,
    this should not be a problem.
    The webserver normally has a reliable connection with the DB server:)

2. On Postgres specifically, you can use PGBouncer.
    It mitigates the number of connections to the server.

Michael.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: FCL-WEB and PostgreSQL Support: Current Status

Mark Morgan Lloyd-5
In reply to this post by Marco van de Voort
On 27/09/17 18:30, Marco van de Voort wrote:
> In our previous episode, Mark Morgan Lloyd said:> > Can you ellaborate a bit more on this Michael? Like how and where it isused? (Not mentioning company names, if it's a problem.) I was thinkingabout using it in various places, but as I don't know how widely they'retested and used and under what conditions, I didn't had the balls to offerthis as viable alternative against competing technology X.> > We definitely need more use case studies...> > The only problems I've experienced with FPC's support for PostgreSQL > were related to handling SQL queries (e.g. spurious parameter > substitution) rather than to dataflow. As such they were predictable and > could be worked round.
> Zeljan once explained an interesting practical problem he had.
> He said the connection per transaction model could be dangerous, speciallyin a mobile era where connections are often broken, and thus must wait ontimeout.  This means during rush times, a server can run out of a fairlylimited ephemeral port inventory before abandoned connections start to timeout.
> That is a postgres problem though, not the FPC components
Which I suppose really starts getting to be an issue when somebody tries
to "put something in the cloud" without careful consideration of that
type of scenario.

--
Mark Morgan Lloyd
markMLl .AT. telemetry.co .DOT. uk

[Opinions above are the author's, not those of his employers or colleagues]
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: FCL-WEB and PostgreSQL Support: Current Status

Marcos Douglas B. Santos
In reply to this post by African Wild Dog
On Wed, Sep 27, 2017 at 12:16 PM, African Wild Dog
<[hidden email]> wrote:
> Hello,
>
> I am about to start a new project for a very small web application.
>
> I have some questions:
>
> 1 - Are the fcl-web and postgres packages stable?
> 2 - Are they recommended for use in production environments?
> 3 - Do they have any serious bug (for example, memory leaks)?

I have been using fcl-web in production about 2 years.
However, my environment is Windows and Microsoft IIS, running FastCGI.
Most of these apps are used to provide REST API's.
I don't have any problems. These apps are fast and I can use shared
code between Desktop and Web. Very good.

Best regards,
Marcos Douglas
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: FCL-WEB and PostgreSQL Support: Current Status

Daniel Gaspary
In reply to this post by Michael Van Canneyt
On Wed, Sep 27, 2017 at 12:53 PM, Michael Van Canneyt
<[hidden email]> wrote:

> Apache, indeed using FastCGI. Used both mod_fastcgi as mod_fcgid. mod_proxy
> should work as well.

What do you use to debug this environment, just log messages / log server..?

Sometime ago I have seen someone mentioned developing with a separated
application which communicated to the cgi / fastcgi part. This Cgi /
fastcgi part was used just to receive / response requests from / to
the web server. This way the debugging became much easier.

My question: How this application communicate with the Cgi / FastCgi part?

TSimpleIpc seens to only communicate one way. Anyone using two
TSimpeIpc's components to simulate a duplex channel?

Any chance of extend the component / create a new one, with support to
receive and send data ?

Are you using Sockets(Ip not unix sockets, like SimpleIpc)? I would
like to avoid that, because of firewall restrictions to open a server
socket
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: FCL-WEB and PostgreSQL Support: Current Status

Michael Van Canneyt


On Thu, 28 Sep 2017, Daniel Gaspary wrote:

> On Wed, Sep 27, 2017 at 12:53 PM, Michael Van Canneyt
> <[hidden email]> wrote:
>
>> Apache, indeed using FastCGI. Used both mod_fastcgi as mod_fcgid. mod_proxy
>> should work as well.
>
> What do you use to debug this environment, just log messages / log server..?

See below.

>
> Sometime ago I have seen someone mentioned developing with a separated
> application which communicated to the cgi / fastcgi part. This Cgi /
> fastcgi part was used just to receive / response requests from / to
> the web server. This way the debugging became much easier.
>
> My question: How this application communicate with the Cgi / FastCgi part?

You don't need that at all, that is way too complicated.

You have 3 options to debug:

1. You can perfectly start the fastcgi in the debugger if you use proxy server
or if you use mod_fastcgi confgured with fastcgiexternalserver configuration
setting. I believe nginx only uses the proxy mechanism.

2. If that is not an option, there is also the following:
FPC contains a cgi to fastcgi request converter, you can use that to debug
the fastcgi by running it inside the debugger, and let apache forward the
request using the cgi bridge.

3. Third option:
    FCL-Web is transparant in the sense that the code for CGI/FastCGI/Apache
    module or standalone HTTP server is 100% the same. Only the used
    TCustomApplication descendent differs for these 4 environments.

That means you can debug your code using the HTTP server, and switch to
using FastCGI in production. I use an {$IFDEF } in the project file to
switch the hosting environment. For the business code this is transparant.


> TSimpleIpc seens to only communicate one way. Anyone using two
> TSimpeIpc's components to simulate a duplex channel?
>
> Any chance of extend the component / create a new one, with support to
> receive and send data ?

No. I don't see the need.
See above, there are plenty of mechanisms to debug a web environment.

Michael.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: FCL-WEB and PostgreSQL Support: Current Status

Daniel Gaspary
On Thu, Sep 28, 2017 at 6:11 PM, Michael Van Canneyt
<[hidden email]> wrote:
> 1. You can perfectly start the fastcgi in the debugger if you use proxy
> server
> or if you use mod_fastcgi confgured with fastcgiexternalserver configuration
> setting. I believe nginx only uses the proxy mechanism.

Nice to know about this configuration. It seens similar to what I was
imagining, using unix sockets(optionally)

> 2. If that is not an option, there is also the following:
> FPC contains a cgi to fastcgi request converter, you can use that to debug
> the fastcgi by running it inside the debugger, and let apache forward the
> request using the cgi bridge.

Could you elaborate about this bridge? Is some apache/nginx
configuration, an external tool, using some apache mod.. ??

> 3. Third option:
>    FCL-Web is transparant in the sense that the code for CGI/FastCGI/Apache
>    module or standalone HTTP server is 100% the same. Only the used
>    TCustomApplication descendent differs for these 4 environments.
>
> That means you can debug your code using the HTTP server, and switch to
> using FastCGI in production. I use an {$IFDEF } in the project file to
> switch the hosting environment. For the business code this is transparant.

This and the first appear to be the simpler options.

>> Any chance of extend the component / create a new one, with support to
>> receive and send data ?
>
> No. I don't see the need. See above, there are plenty of mechanisms to debug
> a web environment.

I mean .. Not just for this case, but to have a more capable IPC.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: FCL-WEB and PostgreSQL Support: Current Status

Michael Schnell
In reply to this post by Michael Van Canneyt

On 28.09.2017 23:11, Michael Van Canneyt wrote:
> You have 3 options to debug:
>
> 1. You can perfectly start the fastcgi in the debugger if you use
> proxy server
Is this "Fast CGI via a TCP socket" ?

Does FCL not support this internally ?

-Michael

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: FCL-WEB and PostgreSQL Support: Current Status

Michael Van Canneyt
In reply to this post by Daniel Gaspary


On Thu, 28 Sep 2017, Daniel Gaspary wrote:

> On Thu, Sep 28, 2017 at 6:11 PM, Michael Van Canneyt
> <[hidden email]> wrote:
>> 1. You can perfectly start the fastcgi in the debugger if you use proxy
>> server
>> or if you use mod_fastcgi confgured with fastcgiexternalserver configuration
>> setting. I believe nginx only uses the proxy mechanism.
>
> Nice to know about this configuration. It seens similar to what I was
> imagining, using unix sockets(optionally)
>
>> 2. If that is not an option, there is also the following:
>> FPC contains a cgi to fastcgi request converter, you can use that to debug
>> the fastcgi by running it inside the debugger, and let apache forward the
>> request using the cgi bridge.
>
> Could you elaborate about this bridge? Is some apache/nginx
> configuration, an external tool, using some apache mod.. ??

See unit fcgigate.pp. It has some comments explaining usage.

To the outside world and the webserver, your application is a CGI app.
The cgi application simply repacks the request, sends it to the fastcgi
server, retrieves the response and sends it back to the webserver.

You just need to set the port your FastCGI app listens on. It uses a default
of 2015, so if you use that, no need to configure anything. Just compile the
gateway, drop it wherever you want/need it, and you're ready to go.

>
>> 3. Third option:
>>    FCL-Web is transparant in the sense that the code for CGI/FastCGI/Apache
>>    module or standalone HTTP server is 100% the same. Only the used
>>    TCustomApplication descendent differs for these 4 environments.
>>
>> That means you can debug your code using the HTTP server, and switch to
>> using FastCGI in production. I use an {$IFDEF } in the project file to
>> switch the hosting environment. For the business code this is transparant.
>
> This and the first appear to be the simpler options.

2 is also quite simple. It doesn't even need a recompile, and no need to
configure anything in the webserver.

>
>>> Any chance of extend the component / create a new one, with support to
>>> receive and send data ?
>>
>> No. I don't see the need. See above, there are plenty of mechanisms to debug
>> a web environment.
>
> I mean .. Not just for this case, but to have a more capable IPC.

Well, it's called SimpleIPC for a reason. The current implementation is already
far too complicated to my taste...

But there is nothing preventing us to add a 2-way communication mechanism.
but I suggest using WST or somesuch instead of SimpleIPC.
WST has lots of communication channels available,
and it's very easy to set up a simple general-purpose messaging system.

FPC really has all you need, even in 2017 ;)

Michael.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: FCL-WEB and PostgreSQL Support: Current Status

Daniel Gaspary
On Fri, Sep 29, 2017 at 5:37 AM, Michael Van Canneyt
<[hidden email]> wrote:

> FPC really has all you need, even in 2017 ;)
>

I agree, just needed some details. :)

Thank you for the help.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: FCL-WEB and PostgreSQL Support: Current Status

Michael Van Canneyt
In reply to this post by Michael Schnell


On Fri, 29 Sep 2017, Michael Schnell wrote:

>
> On 28.09.2017 23:11, Michael Van Canneyt wrote:
>> You have 3 options to debug:
>>
>> 1. You can perfectly start the fastcgi in the debugger if you use
>> proxy server
> Is this "Fast CGI via a TCP socket" ?

Yes.

>
> Does FCL not support this internally ?

It does support this.

Michael.
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal