Android external OES Texture integration (SWIG bindings)

A place for users of OGRE to discuss ideas and experiences of utilitising OGRE in their games / demos / applications.
Post Reply
overtired
Gnoblar
Posts: 14
Joined: Tue Oct 23, 2018 1:32 pm
x 1

Android external OES Texture integration (SWIG bindings)

Post by overtired »

Hello, everybody!
I write an android app with augmented reality using google arcore engine. The problem I faced is that sceneform (render engine, which is recommended to use with arcore) does not support skeletal animations. I would like replace the renderer with Ogre. The main step to achieve the goal is to add supporting GL_TEXTURE_EXTERNAL_OES for ogre. I need to create an GL_TEXTURE_EXTERNAL_OES and provide its id/name to my java code. There is only one method from arcore, which takes the texture id and draw camera capture stream into this.

I saw another topic viewtopic.php?t=93966 and try follow ideas which @paroj offered:
paroj wrote: Fri Feb 02, 2018 11:50 am 3) You would need to setup all the gl state yourself and use the GLID attribute of the Ogre::Texture, as Ogre does not know how to use TEXTURE_EXTERNAL_OES. However if recompiling Ogre is an option, adding TEXTURE_EXTERNAL_OES support is just a matter of adding an enum value here
https://github.com/OGRECave/ogre/blob/v ... ture.h#L99
and correctly handling it here:
https://github.com/OGRECave/ogre/blob/m ... re.cpp#L73

(as well as making sure the limitations of TEXTURE_EXTERNAL_OES are respected). A pull request with this addition would be welcome ;)
I added some changed to ogre source and now I cannot understand how to generate bindings for java using Swig. I added getId() method to Texture class (https://github.com/OGRECave/ogre/blob/m ... eTexture.h), but getId() method still no generates during project build. I also tried to found four *.i in the whole project, but there is no any mention about Texture class. I suppose these files are generated during build process too.

Could anybody help me with the problem?

P.S. Sorry for my English
paroj
OGRE Team Member
OGRE Team Member
Posts: 1994
Joined: Sun Mar 30, 2014 2:51 pm
x 1074
Contact:

Re: Android external OES Texture integration (SWIG bindings)

Post by paroj »

swig often does not notice if a source file was changed. A workaround is to add a white space to the parent .i file or to do a clean build.
To verify if your build correctly picks up swig, take a look at our CI output:
https://travis-ci.org/OGRECave/ogre/jobs/445208934
overtired
Gnoblar
Posts: 14
Joined: Tue Oct 23, 2018 1:32 pm
x 1

Re: Android external OES Texture integration (SWIG bindings)

Post by overtired »

Thank you for quick response. I generated swig bindings. And now I can:
  • Create texture with GL_TEXTURE_EXTERNAL_OES type
  • Place it in front of camera
  • Get texture ID
I traced app using special android debugging tool - GAPID (it lets to wrap android app, show all gl-calls, created textures, drawn object). Changes I did:
  • RenderSystems/GLES2/src/OgreGLES2Texture.cpp: mRenderSystem->_getStateCacheManager()->setTexParameteri() invocation parameters for GL_TEXTURE_EXTERNAL_OES case, saving previous parameters for other texture types
  • RenderSystems/GLES2/src/OgreGLES2RenderSystem.cpp: added return in the beginning of the _setSampler() method in the case of GL_TEXTURE_EXTERNAL_OES (glError occurs otherwise)
What is sampler? What it is used for?

I added a model on scene. I see the model and its texture on the background (instead of my camera image) for several seconds, then it replaces by back screen. Using GAPID I can see correct texture with camera buffer in texture list, I see rect for the texture, but it doesn't appear on the screen.

Could you help me to understand, what am I doing wrong?
paroj
OGRE Team Member
OGRE Team Member
Posts: 1994
Joined: Sun Mar 30, 2014 2:51 pm
x 1074
Contact:

Re: Android external OES Texture integration (SWIG bindings)

Post by paroj »

can you provide a WIP pull request here https://github.com/OGRECave/ogre/pulls so I can see all your changes?

my guess would be that here "added return in the beginning of the _setSampler() method " you should at least activate the texture unit.

Samplers: https://www.khronos.org/opengl/wiki/Sampler_Object (note that for GLES2 ogre does not actually use GL Sampler Objects)
The errors you see are related: https://www.khronos.org/registry/OpenGL ... ternal.txt (note the limitations of sampling)
overtired
Gnoblar
Posts: 14
Joined: Tue Oct 23, 2018 1:32 pm
x 1

Re: Android external OES Texture integration (SWIG bindings)

Post by overtired »

I created pr: https://github.com/OGRECave/ogre/pull/914, see it please. Now I try to understand _setSampler() code
paroj
OGRE Team Member
OGRE Team Member
Posts: 1994
Joined: Sun Mar 30, 2014 2:51 pm
x 1074
Contact:

Re: Android external OES Texture integration (SWIG bindings)

Post by paroj »

another thing that came to my mind: do you use custom shaders for rendering the background? Because you must use samplerExternalOES instead of sampler2D to access the texture.
overtired
Gnoblar
Posts: 14
Joined: Tue Oct 23, 2018 1:32 pm
x 1

Re: Android external OES Texture integration (SWIG bindings)

Post by overtired »

Yeah, I use shaders, but there is samplerExternalOES defined :(

Code: Select all

#extension GL_OES_EGL_image_external : require

precision mediump float;
varying vec2 v_TexCoord;
uniform samplerExternalOES sTexture;


void main() {
    gl_FragColor = texture2D(sTexture, v_TexCoord);
}
overtired
Gnoblar
Posts: 14
Joined: Tue Oct 23, 2018 1:32 pm
x 1

Re: Android external OES Texture integration (SWIG bindings)

Post by overtired »

Summary:
We continued communication with paroj in github and added oes texture support for android in the pr: https://github.com/OGRECave/ogre/pull/914. Sample can be build with OGRE_BUILD_ANDROID_TEXTURE_OES_SAMPLE cmake flag
Post Reply