Android porting ideas

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Android porting ideas

Ryan Joseph
I’m going to attempt to port a game to Android but I have some questions about potential problems. Now at least I think building a library and loading using the JNI would be best but using the JVM could be helpful also (the entire project couldn’t be compiled though). Not sure which way is best or if they can be mixed perhaps.

1) All the file I/O in the FPC RTL is not available so what possible work arounds are there for loading data from files?

2) OpenGL functions are not available so how can I call into the Android library? Callback wrappers perhaps?

3) I use ReadXMLFile from XMLRead but that uses file loading internally so is there anyway to load XML on Android?

Since there are limitations with the JVM right now building a static library seems easiest but if I can’t wrap what’s missing from the RTL and call from Java there’s no way to get 1 and 2 working. Callbacks make sense here but I’m not sure.

Any ideas? Thanks.

Regards,
        Ryan Joseph

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Android porting ideas

leledumbo
Administrator
> Any ideas?

Learn how Castle Game Engine and ZenGL managed to do it. They don't use JVM, but the ARM target of the compiler.
Reply | Threaded
Open this post in threaded view
|

Re: Android porting ideas

Ryan Joseph
Good idea, thanks.

> On Feb 21, 2016, at 9:08 PM, leledumbo <[hidden email]> wrote:
>
> Learn how Castle Game Engine and ZenGL managed to do it. They don't use JVM,
> but the ARM target of the compiler.

Regards,
        Ryan Joseph

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Android porting ideas

Michalis Kamburelis-3
In reply to this post by Ryan Joseph
2016-02-21 4:36 GMT+01:00 Ryan Joseph <[hidden email]>:
> I’m going to attempt to port a game to Android but I have some questions about potential problems. Now at least I think building a library and loading using the JNI would be best but using the JVM could be helpful also (the entire project couldn’t be compiled though). Not sure which way is best or if they can be mixed perhaps.

In Castle Game Engine the whole Pascal code is compiled into an .so
library (for Android+Arm, though we could also target Android+x86). It
is then packaged into an apk, with the activity class set to the
"NativeActivity" Java class.

You can communicate with the Java code using JNI. In the base version,
the NativeActivity (part of Android since Android 2.3) already does a
lot for you, and you don't even need to touch the "Java side". For
more complicated stuff, you need to write some Java code, and
communicate with it using JNI. Of course if you use an engine like
Castle Game Engine, you get a lot of help with this:) --- most things
are already implemented for you, and for adding your own Java
extensions we have CastleMessaginig unit and a "component" system for
Android projects,
https://github.com/castle-engine/castle-engine/wiki/Android-Project-Components-Integrated-with-Castle-Game-Engine
.

See http://castle-engine.sourceforge.net/engine.php and download the
engine and try it:) You can use the stable release 5.2.0, or the (soon
to be released as 5.3.0) version from GitHub on
https://github.com/castle-engine/castle-engine .

We have a lot of documentation, some Android/mobile specific stuff:

- http://castle-engine.sourceforge.net/tutorial_mobile.php (tutorial
showing how to write cross-platform game code using CGE --- for
standalone, Android, iOS, web browser plugin...)

- https://github.com/castle-engine/castle-engine/wiki/Android - how to
install Android development environment

- https://github.com/castle-engine/castle-engine/wiki/Android-Project-Components-Integrated-with-Castle-Game-Engine
- extra Android components, e.g. for in-app purchases or analytics.

>
> 1) All the file I/O in the FPC RTL is not available so what possible work arounds are there for loading data from files?

Actually, all the IO in FPC works perfectly. You can read / write
files just like on any other Unix (although you need to declare
appropriate permissions in the AndroidManifest.xml first ---
READ_EXTERNAL_STORAGE , WRITE_EXTERNAL_STORAGE, otherwise access to
"/sdcard/..." files will be restricted). For example, config files are
just normal files on Android. You only use a special function to get
the proper config file location, but this is done for you by CGE, you
just use CastleFilesUtils.ApplicationConfig:)

The important thing is that your program data (what you put in apk) is
*not* stored on disk as normal files. You need to use AssetManager to
read it. With some helpers, you can just open an "asset file" as a
Pascal TStream, and then read from there. In CGE, you just use URL
like "assets:/some_subdirectory/blahblah.png", pass it to any
file-reading function and it will read the file
"some_subdirectory/blahblah.png" from your data. More info on
http://castle-engine.sourceforge.net/tutorial_network.php .

>
> 2) OpenGL functions are not available so how can I call into the Android library? Callback wrappers perhaps?

You use OpenGL ES for rendering (version 2 or 3, for modern devices).
Pascal bindings are available already in FPC, see unit GLES20. Of
course engines like CGE  give you more comfortable API for rendering,
on top of this.

>
> 3) I use ReadXMLFile from XMLRead but that uses file loading internally so is there anyway to load XML on Android?

As said above --- you can still use these functions to read / write
regular files on Android.

To read XML in your apk data, you need to get a TStream from
AssetManager, and read / write XML using this stream. This works
perfectly:)

We use XML files in CGE a lot, and we have helpers like URLReadXML
that reads TXMLDocument from an URL, and URL can be "file:/..." or
"assets:/..." (or even http or others, see the mentioned tutorial
about networking:)

Regards,
Michalis
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Android porting ideas

Sven Barth-2

Am 21.02.2016 19:36 schrieb "Michalis Kamburelis" <[hidden email]>:
> >
> > 1) All the file I/O in the FPC RTL is not available so what possible work arounds are there for loading data from files?
>
> Actually, all the IO in FPC works perfectly. You can read / write
> files just like on any other Unix (although you need to declare
> appropriate permissions in the AndroidManifest.xml first ---
> READ_EXTERNAL_STORAGE , WRITE_EXTERNAL_STORAGE, otherwise access to
> "/sdcard/..." files will be restricted). For example, config files are
> just normal files on Android. You only use a special function to get
> the proper config file location, but this is done for you by CGE, you
> just use CastleFilesUtils.ApplicationConfig:)

I think he means on FPC's JVM target. There file I/O using Write/Read is indeed not implemented.

Regards,
Sven


_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Android porting ideas

Michalis Kamburelis-3
2016-02-21 23:00 GMT+01:00 Sven Barth <[hidden email]>:

> Am 21.02.2016 19:36 schrieb "Michalis Kamburelis"
> <[hidden email]>:
>> >
>> > 1) All the file I/O in the FPC RTL is not available so what possible
>> > work arounds are there for loading data from files?
>>
>> Actually, all the IO in FPC works perfectly. You can read / write
>> files just like on any other Unix (although you need to declare
>> appropriate permissions in the AndroidManifest.xml first ---
>> READ_EXTERNAL_STORAGE , WRITE_EXTERNAL_STORAGE, otherwise access to
>> "/sdcard/..." files will be restricted). For example, config files are
>> just normal files on Android. You only use a special function to get
>> the proper config file location, but this is done for you by CGE, you
>> just use CastleFilesUtils.ApplicationConfig:)
>
> I think he means on FPC's JVM target. There file I/O using Write/Read is
> indeed not implemented.
>

Oh, OK. My comments were indeed about the native target (Android+Arm
communicating through JNI), not JVM. Well, at least I can confirm that
I/O works cool there:)

Regards,
Michalis
_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Android porting ideas

Ryan Joseph
In reply to this post by Michalis Kamburelis-3
Hey Michalis, thanks for your detailed answers I really appreciate it.

> In Castle Game Engine the whole Pascal code is compiled into an .so
> library (for Android+Arm, though we could also target Android+x86). It
> is then packaged into an apk, with the activity class set to the
> "NativeActivity" Java class.

I see you made a builder tool I’ll have to check out. From the Android tutorials I’ve seen building Android packages seems be pretty forward. You can run/install apps from the terminal also I think based on your documents. That’d be nice to not be in Android Studio if I don’t have to be.

It looks like the majority of the startup code is in castlewindow_android but I was expected to see an OpenGL context (surface view I think in Android). How do you render OpenGL then?

>>
>> 1) All the file I/O in the FPC RTL is not available so what possible work arounds are there for loading data from files?

> The important thing is that your program data (what you put in apk) is
> *not* stored on disk as normal files. You need to use AssetManager to
> read it. With some helpers, you can just open an "asset file" as a
> Pascal TStream, and then read from there. In CGE, you just use URL
> like "assets:/some_subdirectory/blahblah.png", pass it to any
> file-reading function and it will read the file
> "some_subdirectory/blahblah.png" from your data. More info on
> http://castle-engine.sourceforge.net/tutorial_network.php .

Yeah I see there’s a CastleAndroidAssetManager unit which does this.

So UNIX I/O “works” but you can’t read anything without the asset manager? That doesn’t really make sense but either way using the Asset manager in a stream like you did is just fine. I only need to read the contents of images and XML files anyways.

I was using ReadXMLFile to read the XML file but I assume this is just a helper function that wraps something lower level I could use in conjunction with the asset manager. Is that correct? Can’t find your code for this.

>
>>
>> 2) OpenGL functions are not available so how can I call into the Android library? Callback wrappers perhaps?
>
> You use OpenGL ES for rendering (version 2 or 3, for modern devices).
> Pascal bindings are available already in FPC, see unit GLES20. Of
> course engines like CGE  give you more comfortable API for rendering,
> on top of this.

So internally Android just uses the same C libraries we were using in Pascal? I guess that makes sense but I was thinking EVERYTHING on the system was now Java but that’s not the case I guess. The lesson here is that Android is built on Linux so we’re sharing more than I think.


Regards,
        Ryan Joseph

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Android porting ideas

Michalis Kamburelis-3

I see you made a builder tool I’ll have to check out. From the Android tutorials I’ve seen building Android packages seems be pretty forward. You can run/install apps from the terminal also I think based on your documents. That’d be nice to not be in Android Studio if I don’t have to be.


Yes, you can do whole compile + install + run process using castle-engine tool. It's a rather simple wrapper around some Google tools, ant etc. You don't need to download Android Studio at all, you only need basic Android SDK and NDK.

It looks like the majority of the startup code is in castlewindow_android but I was expected to see an OpenGL context (surface view I think in Android). How do you render OpenGL then?


castlewindow_android.inc does

  {$I castlewindow_egl.inc}

and the code that initializes OpenGLES context is inside castlewindow_egl.inc. It's using EGL library, https://www.khronos.org/registry/egl/ , specification blessed by Khronos, available on Android, able to initialize OpenGL ES context.

It can also be used to initialize OpenGL ES context on the desktops (which is useful to test OpenGL ES renderer on Linux, Windows etc.). That's why the code is not inside castlewindow_android.inc --- it's more generally useful.

So UNIX I/O “works” but you can’t read anything without the asset manager? That doesn’t really make sense but either way using the Asset manager in a stream like you did is just fine. I only need to read the contents of images and XML files anyways.


UNIX I/O works for reading files. On disk (device internal memory, sdcard etc.).

It doesn't work to read stuff from the apk. Because the files you put inside apk ("assets" in Android terminology) are not available to you as normal files inside the Android application. You don't know where (and if, at all) they are unpackaged on the Android filesystem. You can only access them (read-only) using AssetManager.

I was using ReadXMLFile to read the XML file but I assume this is just a helper function that wraps something lower level I could use in conjunction with the asset manager. Is that correct? Can’t find your code for this.


See the URLReadXML implementation:

procedure URLReadXML(out Doc: TXMLDocument; const URL: String);
var
 Stream: TStream;
begin
 Doc := nil; // clean "out" param at start, just like ReadXMLFile
 Stream := Download(URL, []);
 try
   ReadXMLFile(Doc, Stream);
 finally FreeAndNil(Stream) end;
end;

So it simply reads URL to a TStream, then uses overloaded ReadXMLFile version that reads XML from a stream:)

So internally Android just uses the same C libraries we were using in Pascal? I guess that makes sense but I was thinking EVERYTHING on the system was now Java but that’s not the case I guess. The lesson here is that Android is built on Linux so we’re sharing more than I think.


There are *some* C libraries on Android, indeed. There is a version of libc, there is OpenGLES (and friends like EGL). But not much else, so don't get your hopes up:) http://developer.android.com/ndk/guides/stable_apis.html has the list of available native libraries.

You can make a working game with them --- you have access to display (through EGL and OpenGL ES) and sound (through OpenMAX). And you have access to input by communicating through JNI with NativeActivity. 

But a lot of other stuff is only available in Java libraries, and you need to write custom Java code to access it. For example, everything that is part of https://en.wikipedia.org/wiki/Google_Play_Services --- integration with Google Maps, Google Analytics, Google Games, etc. Also in-app purchases. Also integration with other apps e.g. to get/make photos. Or take location from GPS. Or even open a WWW browser.

Fortunately, using these Java APIs through a JNI layer is not really hard in my experience. And Castle Game Engine gives you some helpers:) --- CastleMessaging and component system, to make it even easier.

Regards,
Michalis

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Android porting ideas

Ryan Joseph

> On Feb 23, 2016, at 8:30 AM, Michalis Kamburelis <[hidden email]> wrote:
>
> It looks like the majority of the startup code is in castlewindow_android but I was expected to see an OpenGL context (surface view I think in Android). How do you render OpenGL then?
>
>
> castlewindow_android.inc does
>
>   {$I castlewindow_egl.inc}
>
> and the code that initializes OpenGLES context is inside castlewindow_egl.inc. It's using EGL library, https://www.khronos.org/registry/egl/ , specification blessed by Khronos, available on Android, able to initialize OpenGL ES context.
>
> It can also be used to initialize OpenGL ES context on the desktops (which is useful to test OpenGL ES renderer on Linux, Windows etc.). That's why the code is not inside castlewindow_android.inc --- it's more generally useful.
>

I see now! I’m using OpenGLES 1.x but I think I can use the EGL window functions you have translated in your OpenGLES 2 unit.

> So UNIX I/O “works” but you can’t read anything without the asset manager? That doesn’t really make sense but either way using the Asset manager in a stream like you did is just fine. I only need to read the contents of images and XML files anyways.
>
>
> UNIX I/O works for reading files. On disk (device internal memory, sdcard etc.).
>
> It doesn't work to read stuff from the apk. Because the files you put inside apk ("assets" in Android terminology) are not available to you as normal files inside the Android application. You don't know where (and if, at all) they are unpackaged on the Android filesystem. You can only access them (read-only) using AssetManager.

Ok. I need to learn about the file system more then.

>
> I was using ReadXMLFile to read the XML file but I assume this is just a helper function that wraps something lower level I could use in conjunction with the asset manager. Is that correct? Can’t find your code for this.
>
>
> See the URLReadXML implementation:
>
> procedure URLReadXML(out Doc: TXMLDocument; const URL: String);
> var
>  Stream: TStream;
> begin
>  Doc := nil; // clean "out" param at start, just like ReadXMLFile
>  Stream := Download(URL, []);
>  try
>    ReadXMLFile(Doc, Stream);
>  finally FreeAndNil(Stream) end;
> end;
>
> So it simply reads URL to a TStream, then uses overloaded ReadXMLFile version that reads XML from a stream:)
> So internally Android just uses the same C libraries we were using in Pascal? I guess that makes sense but I was thinking EVERYTHING on the system was now Java but that’s not the case I guess. The lesson here is that Android is built on Linux so we’re sharing more than I think.

This will probably make more sense when I learn about the asset manager more. I think I read there is a format like asset:// for file URL’s.

Thanks again!



Regards,
        Ryan Joseph

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
Reply | Threaded
Open this post in threaded view
|

Re: Android porting ideas

Michalis Kamburelis-3

This will probably make more sense when I learn about the asset manager more. I think I read there is a format like asset:// for file URL’s.


Remember that the "asset:/..." URL is just implemented in Castle Game Engine (see http://castle-engine.sourceforge.net/tutorial_network.php ). It's not something recognized by other APIs (Android or not). Although Qt has analogous concept (http://doc.qt.io/qt-5/platform-notes-android.html#assets-file-system).

If you want to do it yourself: the AssetManager functions are part of CastleAndroidInternalAssetManager , which are bindings for C API documented as part of Android NDK here: http://developer.android.com/ndk/reference/group___asset.html . These are the gory details to access assets:)

The unit CastleAndroidInternalAssetStream wraps Android assets into a Pascal TStream, and converts URL (more-or-less strips/adds the "asset:/..." prefix) by URIToAssetPath / AssetPathToURI functions.

Regards,
Michalis

_______________________________________________
fpc-pascal maillist  -  [hidden email]
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal