RTT performance on iOS/Mobile

[JanKleemann]

Ogre Version:1.11.3 (clone from github)
Operating System: iOS
Render System: GLES2

Hi all,

I have a cross platform app using Qt/QML and I need to integrate a 3d scene into it. After some (lots) of tinkering I managed to get QmlOgre and its example app to run with Ogre 1.11 on my iPad 6. QmlOgre renders a scene into a manual texture, which is then used in Qml. In my code I have activated RTSS, since QmlOgre does not work without it on 1.11.

Problem is: Ogre is incredibly slow in the example app (which only contains one ogrehead mesh). It takes several seconds between touch input and an updated image and I have absolutely no clue why. Hopefully someone here can help.

Are there any known performance issues with RTT or RTSS on iOS? Is the QmlOgre approach feasible at all on mobile devices?

Below I have included the apps console output, which shows some warning about missing shader attributes. Could those be the cause of the slowdown? I have searched the forum, but the last mention of these warning was 2009 and said to be fixed...

I have attached the modified QmlOgre project for reference. If you want to try it out, you will have to copy ogrehead.mesh to the example/myresources folder and the Samples folder from Ogre into the example folder. I have removed those files from the zip to reduce it's size.
The .pro-files are specific to my Mac, so you will have to change some paths.

I also had to comment the defines from sdk/include/OGRE/RenderSystems/GLES2/GLES3/glesw.h, because those conflicted with gl.h, which gets included by Qt.

Code: Select all

stale focus object QObject(0x0) , doing manual update
Creating resource group General
Creating resource group OgreInternal
Creating resource group OgreAutodetect
SceneManagerFactory for type 'DefaultSceneManager' registered.
Registering ResourceManager for type Material
Registering ResourceManager for type Mesh
Registering ResourceMana
ger for type Skeleton
MovableObjectFactory for type 'ParticleSystem' registered.
ArchiveFactory for archive type FileSystem registered.
ArchiveFactory for archive type Zip registered.
ArchiveFactory for archive type EmbeddedZip registered.
DDS codec r
egistering
PVRTC codec registering
ETC codec registering
Registering ResourceManager for type HighLevelGpuProgram
Registering ResourceManager for type Compositor
MovableObjectFactory for type 'Entity' registered.
MovableObjectFactory for type 'Light'
 registered.
MovableObjectFactory for type 'BillboardSet' registered.
MovableObjectFactory for type 'ManualObject' registered.
MovableObjectFactory for type 'BillboardChain' registered.
MovableObjectFactory for type 'RibbonTrail' registered.
*-*-* OGR
E Initialising
*-*-* Version 1.11.3 (Rhagorthua)
Installing plugin: OpenGL ES 2.0 RenderSystem
OpenGL ES 2.x Rendering Subsystem created.
Plugin successfully installed
Installing plugin: Octree Scene Manager
Plugin successfully installed
Installing 
plugin: ParticleFX
Particle Emitter Type 'Point' registered
Particle Emitter Type 'Box' registered
Particle Emitter Type 'Ellipsoid' registered
Particle Emitter Type 'Cylinder' registered
Particle Emitter Type 'Ring' registered
Particle Emitter Type 
'HollowEllipsoid' registered
Particle Affector Type 'LinearForce' registered
Particle Affector Type 'ColourFader' registered
Particle Affector Type 'ColourFader2' registered
Particle Affector Type 'ColourImage' registered
Particle Affector Type 'Colou
rInterpolator' registered
Particle Affector Type 'Scaler' registered
Particle Affector Type 'Rotator' registered
Particle Affector Type 'DirectionRandomiser' registered
Particle Affector Type 'DeflectorPlane' registered
Plugin successfully installed
Installing plugin: BSP Scene Manager
Plugin successfully installed
Installing plugin: STB Image Codec
stb_image - v2.19 - public domain JPEG/PNG reader
Supported formats: jpeg,jpg,png,bmp,psd,tga,gif,pic,ppm,pgm,hdr
Plugin successfully installed
Rend
erSystem: OpenGL ES 2.x Rendering Subsystem
CPU Identifier & Features
-------------------------
 *   CPU ID: ARM64v8
 *          VFP: no
 *         NEON: yes
-------------------------
Registering ResourceManager for type Texture
GLES2RenderSystem::
_createRenderWindow "OgreWindow", 1x1 windowed  miscParams: currentGLContext=true externalGLControl=true 
iOS: Window created 1 x 1 with backing store size 1 x 1 using content scaling factor 1.0
GL_VERSION = 2.0.0.0
GL_VENDOR = Apple Inc.
GL_RENDERER = Apple A10 GPU
GL_EXTENSIONS = GL_OES_depth_texture GL_OES_depth_texture_cube_map GL_OES_depth24 GL_OES_element_index_uint GL_OES_fbo_render_mipmap GL_OES_mapbuffer GL_OES_packed_depth_stencil GL_
OES_rgb8_rgba8 GL_OES_standard_derivatives GL_OES_texture_float GL_OES_texture_half_float GL_OES_texture_half_float_linear GL_OES_vertex_array_object GL_EXT_blend_minmax GL_EXT_color_buffer_half_float GL_EXT_debug_label GL_EXT_debug_marker GL_EXT_discard_f
ramebuffer GL_EXT_draw_instanced GL_EXT_instanced_arrays GL_EXT_map_buffer_range GL_EXT_occlusion_query_boolean GL_EXT_pvrtc_sRGB GL_EXT_read_format_bgra GL_EXT_separate_shader_objects GL_EXT_shader_framebuffer_fetch GL_EXT_shader_texture_lod GL_EXT_shadow
_samplers GL_EXT_sRGB GL_EXT_texture_filter_anisotropic GL_EXT_texture_rg GL_EXT_texture_storage GL_APPLE_clip_distance GL_APPLE_color_buffer_packed_float GL_APPLE_copy_texture_levels GL_APPLE_framebuffer_multisample GL_APPLE_rgb_422 GL_APPLE_sync GL_APPLE
_texture_format_BGRA8888 GL_APPLE_texture_max_level GL_APPLE_texture_packed_float GL_IMG_read_format GL_IMG_texture_compression_pvrtc 
**************************************
*** OpenGL ES 2.x Renderer Started ***
**************************************
Shading language version: OpenGL ES GLSL ES 1.00
Registering ResourceManager for type GpuProgram
GL ES 2: Using FBOs for rendering to textures
FBO PF_R5G6B5 depth/stencil support: D0S0 D0S8 D16S0 D24S0 Packed-D24S8 
FBO PF_A4R4G4B4 depth/stencil suppor
t: D0S0 D0S8 D16S0 D24S0 Packed-D24S8 
FBO PF_A1R5G5B5 depth/stencil support: D0S0 D0S8 D16S0 D24S0 Packed-D24S8 
FBO PF_B8G8R8 depth/stencil support: D0S0 D0S8 D16S0 D24S0 Packed-D24S8 
FBO PF_A8B8G8R8 depth/stencil support: D0S0 D0S8 D16S0 D24S0 Packe
d-D24S8 
FBO PF_FLOAT16_RGB depth/stencil support: D0S0 D0S8 D16S0 D24S0 Packed-D24S8 
FBO PF_FLOAT16_RGBA depth/stencil support: D0S0 D0S8 D16S0 D24S0 Packed-D24S8 
FBO PF_X8B8G8R8 depth/stencil support: D0S0 D0S8 D16S0 D24S0 Packed-D24S8 
FBO PF_R8 d
epth/stencil support: D0S0 D0S8 D16S0 D24S0 Packed-D24S8 
FBO PF_RG8 depth/stencil support: D0S0 D0S8 D16S0 D24S0 Packed-D24S8 
[GLES2] : Valid FBO targets PF_R5G6B5 PF_A4R4G4B4 PF_A1R5G5B5 PF_B8G8R8 PF_A8B8G8R8 PF_FLOAT16_RGB PF_FLOAT16_RGBA PF_X8B8G8R8
 PF_R8 PF_RG8 
RenderSystem capabilities
-------------------------
RenderSystem Name: OpenGL ES 2.x Rendering Subsystem
GPU Vendor: apple
Device Name: Apple A10 GPU
Driver Version: 2.0.0.0
 * Fixed function pipeline: no
 * Anisotropic texture filte
ring: yes
 * Cube mapping: yes
 * Hardware stencil buffer: yes
   - Stencil depth: 8
   - Two sided stencil support: yes
   - Wrap stencil values: yes
 * 32-bit index buffers: yes
 * Vertex programs: yes
 * Number of floating-point constants for ve
rtex programs: 128
 * Number of integer constants for vertex programs: 128
 * Number of boolean constants for vertex programs: 128
 * Fragment programs: yes
 * Number of floating-point constants for fragment programs: 64
 * Number of integer constants
 for fragment programs: 64
 * Number of boolean constants for fragment programs: 64
 * Geometry programs: no
 * Number of floating-point constants for geometry programs: 20319
 * Number of integer constants for geometry programs: 21317
 * Number of bo
olean constants for geometry programs: 26207
 * Tessellation Hull programs: no
 * Number of floating-point constants for tessellation hull programs: 24424
 * Number of integer constants for tessellation hull programs: 29811
 * Number of boolean constan
ts for tessellation hull programs: 28261
 * Tessellation Domain programs: no
 * Number of floating-point constants for tessellation domain programs: 26979
 * Number of integer constants for tessellation domain programs: 8300
 * Number of boolean consta
nts for tessellation domain programs: 19527
 * Compute programs: no
 * Number of floating-point constants for compute programs: 20319
 * Number of integer constants for compute programs: 21317
 * Number of boolean constants for compute programs: 29279

 * Supported Shader Profiles: glsles
 * Texture Compression: yes
   - DXT: no
   - VTC: no
   - PVRTC: yes
   - ATC: no
   - ETC1: no
   - ETC2: no
   - BC4/BC5: no
   - BC6H/BC7: no
   - ASTC: no
   - Mipmaps for compressed formats: no
 * Har
dware Occlusion Query: yes
 * User clip planes: no
 * VET_UBYTE4 vertex element type: yes
 * Infinite far plane projection: yes
 * Hardware render-to-texture: yes
 * Floating point textures: yes
 * Non-power-of-two textures: no
 * 1d textures: no
 
* Volume textures: no
 * Multiple Render Targets: 1
   - With different bit depths: no
 * Point Sprites: yes
 * Wide Lines: yes
 * Hardware Gamma: no
 * Extended point parameters: yes
 * Max Point Size: 0
 * Vertex texture fetch: no
 * Number of t
exture units: 8
 * Number of vertex attributes: 16
 * Stencil buffer depth: 8
 * Number of vertex blend matrices: 0
 * Render to Vertex Buffer : no
 * Hardware Atomic Counters: no
 * PBuffer support: no
 * Vertex Array Objects: yes
 * Separate shad
er objects: yes
 * GLSL SSO redeclare interface block: yes
 * Debugging/ profiling events: yes
 * Map buffer storage: yes
DefaultWorkQueue('Root') initialising on thread 0x16f313000.
DefaultWorkQueue('Root')::WorkerFunc - thread 0x16f39f000 starting.

DefaultWorkQueue('Root')::WorkerFunc - thread 0x16f42b000 starting.
Particle Renderer Type 'billboard' registered
SceneManagerFactory for type 'OctreeSceneManager' registered.
SceneManagerFactory for type 'BspSceneManager' registered.
Registering Scen
eLoader Q3BSP
Creating resource group Essential
Added resource location 'myresources/data' of type 'FileSystem' to resource group 'Essential'
Added resource location 'Samples/Media/RTShaderLib/materials' of type 'FileSystem' to resource group 'General'

Added resource location 'Samples/Media/RTShaderLib/GLSL' of type 'FileSystem' to resource group 'General'
Added resource location 'Samples/Media/RTShaderLib/GLSLES' of type 'FileSystem' to resource group 'General'
Added resource location 'Samples/Media/
materials/programs/GLSLES' of type 'FileSystem' to resource group 'General'
Parsing scripts for resource group Essential
Parsing script skybox.material
Parsing script Ogre.material
Finished parsing scripts for resource group Essential
Creating resourc
es for group Essential
All done
Parsing scripts for resource group General
Parsing script TriplanarTexturing.material
Parsing script RTShaderSystem.material
Parsing script HardwareSkinningShadow.material
Parsing script DualQuaternionSkinning_Shadow.material
Finished parsing scripts for resource group General
Creating resources for group General
All done
Parsing scripts for resource group OgreAutodetect
Finished parsing scripts for resource group OgreAutodetect
Creating resources for group OgreAutodetect

All done
Parsing scripts for resource group OgreInternal
Finished parsing scripts for resource group OgreInternal
Creating resources for group OgreInternal
All done
Mesh: Loading ogrehead.mesh.
Texture: GreenSkin.jpg: Loading 1 faces(PF_B8G8R8,256x256x1) with 8 hardware generated mipmaps from Image. Internal format is PF_B8G8R8,256x256x1.
Texture: spheremap.png: Loading 1 faces(PF_B8G8R8,256x256x1) with 8 hardware generated mipmaps from Image. Internal format is PF_B8G8R8,256x256x1.
Texture: tusk.jpg: Loading 1 faces(PF_B8G8R8,128x128x1) with 7 hardware generated mipmaps from Image. Internal format is PF_B8G8R8,128x128x1.
Vertex Program:7b38fc5c6d3487c28139e6cb4e8fc3ea_VS Fragment Program:081d6d81c545d5f6ad1d6345a6930495_FS
GLSL vertex program result : 
WARNING: Could not find vertex shader attribute 'blendWeights' to match BindAttributeLocation request.
WARNING: Could n
ot find vertex shader attribute 'uv5' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'secondary_colour' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv1' to match BindAt
tributeLocation request.
WARNING: Could not find vertex shader attribute 'uv4' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'blendIndices' to match BindAttributeLocation request.
WARNING: Could not find vertex 
shader attribute 'uv6' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv2' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'colour' to match BindAttributeLocation request.

WARNING: Could not find vertex shader attribute 'position' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'tangent' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv3' t
o match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv7' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'binormal' to match BindAttributeLocation request.
WARNING: Could not fi
nd vertex shader attribute 'uv0' to match BindAttributeLocation request.
Vertex Program:0b445b3e78de4c0a99383aed74e0e16b_VS Fragment Program:aa1360982b0c358e0ad4465b35323d18_FS
GLSL vertex program result : 
WARNING: Could not find vertex shader attribut
e 'blendWeights' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv5' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'secondary_colour' to match BindAttributeLocation reque
st.
WARNING: Could not find vertex shader attribute 'uv1' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv4' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'blendIndices
' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv6' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv2' to match BindAttributeLocation request.
WARNING: Could not find
 vertex shader attribute 'colour' to match BindAttributeLocation request.

WARNING: Could not find vertex shader attribute 'position' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'tangent' to match BindAttribute
Location request.
WARNING: Could not find vertex shader attribute 'uv3' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv7' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute
 'binormal' to match BindAttributeLocation request.
Vertex Program:07e05a2631cf5c0ccb8f274e65762407_VS Fragment Program:e35d6c9ca66159669c85f5619dc50a78_FS
GLSL vertex program result : 
WARNING: Could not find vertex shader attribute 'blendWeights' to m
atch BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv5' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'secondary_colour' to match BindAttributeLocation request.
WARNING: Could n
ot find vertex shader attribute 'uv1' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv4' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'blendIndices' to match BindAttrib
uteLocation request.
WARNING: Could not find vertex shader attribute 'uv6' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv2' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attrib
ute 'colour' to match BindAttributeLocation request.

WARNING: Could not find vertex shader attribute 'position' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'tangent' to match BindAttributeLocation request.
WA
RNING: Could not find vertex shader attribute 'uv3' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv7' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'binormal' to match 
BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv0' to match BindAttributeLocation request.
Vertex Program:d337eccfcb29364fb14ccd1c2ad80424_VS Fragment Program:aa1360982b0c358e0ad4465b35323d18_FS
GLSL vertex program resu
lt : 
WARNING: Could not find vertex shader attribute 'blendWeights' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv5' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 's
econdary_colour' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv1' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv4' to match BindAttributeLocation request.
WARNING:
 Could not find vertex shader attribute 'blendIndices' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv6' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv2' to match Bi
ndAttributeLocation request.
WARNING: Could not find vertex shader attribute 'colour' to match BindAttributeLocation request.

WARNING: Could not find vertex shader attribute 'position' to match BindAttributeLocation request.
WARNING: Could not find ver
tex shader attribute 'tangent' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv3' to match BindAttributeLocation request.
WARNING: Could not find vertex shader attribute 'uv7' to match BindAttributeLocation requ
est.
WARNING: Could not find vertex shader attribute 'binormal' to match BindAttributeLocation request.

[paroj]

in OgreNode::updateFBO you are re-creating the FBO for each update which is probably the largest culprit.
Also you are using PF_R8G8B8 which is probably not natively supported by the hardware. Rather use PF_BYTE_RGBA.

In doubt profile your application and use the SampleBrowser as a (performance) reference.

[JanKleemann]

Thank you very much for checking the code.

updateFBO is only called when there is a change in the size of the view. I checked that using a console output because it was suspicious to me, too. At start the method was only called once, so multiple callings can't be the cause of the slowdown.

I changed the color format of the rendertexture to PF_BYTE_RGBA, but it is still slow.

Edit:
I tried profiling the app using Xcode, but I'm not sure how to interpret the results. The CPU Thread looks ok. The only thing that is interesting is the "Metal Application" Instrument. It shows two render passes, each more than 8 seconds long, which probably correspond to the initial rendering and the one time I rotated the ogrehead. I have attached a screenshot of the Instruments window.

Samplebrowser doesn't work properly on my iPad. I can run the app, but its not fullscreen and does not react to touch input. I'll check if I can get it working.

P.S.: Samplebrowser shows the same warnings as QmlOgre ("WARNING: Could not find vertex shader attribute ...")

[paroj]

you can safely ignore the warnings - the iOS driver is overly verbose here.

Unfortunately I do not have access to an ios device myself, so I cannot add more info to your trace than you already know. However I asked a team member who is shipping Ogre on iOS (albeit an older version) and he has no performance issues.

You should continue bringing up the sample browser to narrow down whether the slowdown is in Ogre itself or in the QML bridge code. The sample browser also includes several samples using RTT, which should further help.

[JanKleemann]

Thanks again for your advice and effort.

Unfortunately I have no experience in Objective C or the frameworks. The UIView mGestureView, which seems to be supposed to handle touch events, just doesn't receive any events. I don't have any clue to why or even where to begin debugging. Getting the Samplebrowser to work is not possible for me.

I checked the profiler output again and found something interesting: After those big blocks in the "Metal Application"-instrument (the upper one says "Command Buffer x", the lower one says "Render pass 3/0"), there is a short activity in the shaders, where the render passes are processed. To me it looks like this is where the actual work is done. So the question is: What is happening in those 8 second blocks before?

Out of the 21 seconds runtime, the CPU is only 5.76 seconds active (over several threads, the longest thread using 3.64 seconds). So it is not a slowdown in the C++ code. The whole gui is reacting to inputs at all times, even while Ogre is rendering. I can even move the view around, without any problems.

What I did manage was making the SampleBrowser to load specific Samples, by setting startupSampleTitle manually in loadSamples. So far I have tried several Samples (including Cube Mapping and Fresnel, which use a rendertexture) and they all run at 30 FPS without slowdowns.

So it is indeed a problem in QmlOgre (or with the combination of Qt/QML and Ogre, in general). Maybe it's because both Ogre and QML are using OpenGL.

[paroj]

So it is indeed a problem in QmlOgre (or with the combination of Qt/QML and Ogre, in general). Maybe it's because both Ogre and QML are using OpenGL.

this is likely the case. QmlOgre uses OpenGL context sharing which is a corner feature and only seldomly used on desktop, let alone on mobile. Also externalGLControl does not get much exposure with Ogre. Additionally all GL state is managed by Qt here, which might or might not do the right thing.

Maybe you can do some intermediate steps like embedding Ogre into a QWidget and let Ogre manage its context creation (if thats possible on Qt/iOS). Then you could move to a QGLWidget and use externalGLControl and just then try sharing the FBO between the Ogre and the QML context.

[JanKleemann]

Hmm, I'm not sure if that would work, but it would definitely be quite complicated and hacky. If I do that, the next guy who maintains the code will get very frustrated .

For now I will try out the Emscripten/WebGL route, if I manage to compile Ogre with Emscripten .

Thanks again for your help.

[paroj]

I didnt mean that you leave with QWidget instead of QML, but rather use it to narrow down why the QML is so slow.

Also the situation is probably worse regarding QML via Emscripten..

[JanKleemann]

WebGL runs fine in QML WebView. I managed to get our old 3d viewer (BabylonJS) to run in my QML App on iOS and Android. I have no speed problems. It comes with its own problems, of course. Like the need for a webserver. But for my application this is not a problem.

So for now WebGL looks like the more promising route. Getting Ogre to run natively on iOS and Android in QML is just too much work, as I have limited experience on mobile platforms.