Bug in GLRenderSystem::_createDepthBufferFor?

Minor issues with the Ogre API that can be trivial to fix
Post Reply
arthare
Halfling
Posts: 44
Joined: Fri Mar 29, 2013 5:40 pm
x 4

Bug in GLRenderSystem::_createDepthBufferFor?

Post by arthare »

I've been debugging a crash on a fairly crummy intel card.

Here's the graphical portion of the ogre.log:

Code: Select all

11:14:30: GL_VERSION = 2.1.0 - Build 8.15.10.2086
11:14:30: GL_VENDOR = Intel
11:14:30: GL_RENDERER = Intel(R) HD Graphics
11:14:30: GL_EXTENSIONS = GL_EXT_blend_minmax GL_EXT_blend_subtract GL_EXT_blend_color GL_EXT_abgr GL_EXT_texture3D GL_EXT_clip_volume_hint GL_EXT_compiled_vertex_array GL_SGIS_texture_edge_clamp GL_SGIS_generate_mipmap GL_EXT_draw_range_elements GL_SGIS_texture_lod GL_EXT_rescale_normal GL_EXT_packed_pixels GL_EXT_texture_edge_clamp GL_EXT_separate_specular_color GL_ARB_multitexture GL_EXT_texture_env_combine GL_EXT_bgra GL_EXT_blend_func_separate GL_EXT_secondary_color GL_EXT_fog_coord GL_EXT_texture_env_add GL_ARB_texture_cube_map GL_ARB_transpose_matrix GL_ARB_texture_env_add GL_IBM_texture_mirrored_repeat GL_EXT_multi_draw_arrays GL_NV_blend_square GL_ARB_texture_compression GL_3DFX_texture_compression_FXT1 GL_EXT_texture_filter_anisotropic GL_ARB_texture_border_clamp GL_ARB_point_parameters GL_ARB_texture_env_combine GL_ARB_texture_env_dot3 GL_ARB_texture_env_crossbar GL_EXT_texture_compression_s3tc GL_ARB_shadow GL_ARB_window_pos GL_EXT_shadow_funcs GL_EXT_stencil_wrap GL_ARB_vertex_program GL_EXT_texture_rectangle GL_ARB_fragment_program GL_EXT_stencil_two_side GL_ATI_separate_stencil GL_ARB_vertex_buffer_object GL_EXT_texture_lod_bias GL_ARB_occlusion_query GL_ARB_fragment_shader GL_ARB_shader_objects GL_ARB_shading_language_100 GL_ARB_texture_non_power_of_two GL_ARB_vertex_shader GL_NV_texgen_reflection GL_ARB_point_sprite GL_EXT_blend_equation_separate GL_ARB_depth_texture GL_ARB_texture_rectangle GL_ARB_draw_buffers GL_ARB_color_buffer_float GL_ARB_half_float_pixel GL_ARB_texture_float GL_ARB_pixel_buffer_object GL_EXT_framebuffer_object GL_ARB_draw_instanced GL_ARB_half_float_vertex GL_EXT_draw_buffers2 GL_WIN_swap_hint GL_EXT_texture_sRGB GL_EXT_packed_float GL_EXT_texture_shared_exponent GL_ARB_texture_rg GL_ARB_texture_compression_rgtc GL_NV_conditional_render GL_EXT_texture_swizzle GL_ARB_framebuffer_sRGB GL_EXT_packed_depth_stencil GL_ARB_depth_buffer_float GL_EXT_transform_feedback GL_EXT_framebuffer_blit GL_ARB_vertex_array_object 
11:14:30: Supported WGL extensions: WGL_EXT_depth_float WGL_ARB_buffer_region WGL_ARB_extensions_string WGL_ARB_make_current_read WGL_ARB_pixel_format WGL_ARB_pbuffer WGL_EXT_extensions_string WGL_EXT_swap_control WGL_ARB_pixel_format_float WGL_ARB_framebuffer_sRGB 
11:14:30: ***************************
11:14:30: *** GL Renderer Started ***
11:14:30: ***************************
11:14:30: Registering ResourceManager for type GpuProgram
11:14:30: GLSL support detected
11:14:30: GL: Using GL_EXT_framebuffer_object for rendering to textures (best)
11:14:30: FBO PF_UNKNOWN depth/stencil support: D0S1 D0S4 D0S8 D0S16 D16S0 D24S0 D32S0 
11:14:30: FBO PF_R5G6B5 depth/stencil support: D0S0 D0S1 D0S4 D0S8 D0S16 D16S0 D24S0 D32S0 
11:14:30: FBO PF_B5G6R5 depth/stencil support: D0S0 D0S1 D0S4 D0S8 D0S16 D16S0 D24S0 D32S0 
11:14:30: FBO PF_R8G8B8 depth/stencil support: D0S0 D0S1 D0S4 D0S8 D0S16 D16S0 D24S0 D32S0 
11:14:30: FBO PF_B8G8R8 depth/stencil support: D0S0 D0S1 D0S4 D0S8 D0S16 D16S0 D24S0 D32S0 
11:14:30: FBO PF_A8R8G8B8 depth/stencil support: D0S0 D0S1 D0S4 D0S8 D0S16 D16S0 D24S0 D32S0 
11:14:30: FBO PF_B8G8R8A8 depth/stencil support: D0S0 D0S1 D0S4 D0S8 D0S16 D16S0 D24S0 D32S0 
11:14:30: FBO PF_FLOAT16_RGB depth/stencil support: D0S0 D0S1 D0S4 D0S8 D0S16 D16S0 D24S0 D32S0 
11:14:30: FBO PF_FLOAT16_RGBA depth/stencil support: D0S0 D0S1 D0S4 D0S8 D0S16 D16S0 D24S0 D32S0 
11:14:30: FBO PF_FLOAT32_RGB depth/stencil support: D0S0 D0S1 D0S4 D0S8 D0S16 D16S0 D24S0 D32S0 
11:14:30: FBO PF_FLOAT32_RGBA depth/stencil support: D0S0 D0S1 D0S4 D0S8 D0S16 D16S0 D24S0 D32S0 
11:14:30: FBO PF_X8R8G8B8 depth/stencil support: D0S0 D0S1 D0S4 D0S8 D0S16 D16S0 D24S0 D32S0 
11:14:30: FBO PF_X8B8G8R8 depth/stencil support: D0S0 D0S1 D0S4 D0S8 D0S16 D16S0 D24S0 D32S0 
11:14:30: FBO PF_R3G3B2 depth/stencil support: D0S0 D0S1 D0S4 D0S8 D0S16 D16S0 D24S0 D32S0 
11:14:30: [GL] : Valid FBO targets PF_UNKNOWN PF_R5G6B5 PF_B5G6R5 PF_R8G8B8 PF_B8G8R8 PF_A8R8G8B8 PF_B8G8R8A8 PF_FLOAT16_RGB PF_FLOAT16_RGBA PF_FLOAT32_RGB PF_FLOAT32_RGBA PF_X8R8G8B8 PF_X8B8G8R8 PF_R3G3B2 
11:14:30: RenderSystem capabilities
11:14:30: -------------------------
11:14:30: RenderSystem Name: OpenGL Rendering Subsystem
11:14:30: GPU Vendor: intel
11:14:30: Device Name: Intel(R) HD Graphics
11:14:30: Driver Version: 2.1.0.0
11:14:30:  * Fixed function pipeline: yes
11:14:30:  * Hardware generation of mipmaps: no
11:14:30:  * Texture blending: yes
11:14:30:  * Anisotropic texture filtering: yes
11:14:30:  * Dot product texture operation: yes
11:14:30:  * Cube mapping: yes
11:14:30:  * Hardware stencil buffer: yes
11:14:30:    - Stencil depth: 8
11:14:30:    - Two sided stencil support: yes
11:14:30:    - Wrap stencil values: yes
11:14:30:  * Hardware vertex / index buffers: yes
11:14:30:  * Vertex programs: yes
11:14:30:  * Number of floating-point constants for vertex programs: 256
11:14:30:  * Number of integer constants for vertex programs: 0
11:14:30:  * Number of boolean constants for vertex programs: 0
11:14:30:  * Fragment programs: yes
11:14:30:  * Number of floating-point constants for fragment programs: 256
11:14:30:  * Number of integer constants for fragment programs: 0
11:14:30:  * Number of boolean constants for fragment programs: 0
11:14:30:  * Geometry programs: no
11:14:30:  * Number of floating-point constants for geometry programs: 25956
11:14:30:  * Number of integer constants for geometry programs: 11890
11:14:30:  * Number of boolean constants for geometry programs: 25965
11:14:30:  * Supported Shader Profiles: arbfp1 arbvp1 glsl
11:14:30:  * Texture Compression: yes
11:14:30:    - DXT: yes
11:14:30:    - VTC: no
11:14:30:    - PVRTC: no
11:14:30:  * Scissor Rectangle: yes
11:14:30:  * Hardware Occlusion Query: yes
11:14:30:  * User clip planes: yes
11:14:30:  * VET_UBYTE4 vertex element type: yes
11:14:30:  * Infinite far plane projection: yes
11:14:30:  * Hardware render-to-texture: yes
11:14:30:  * Floating point textures: yes
11:14:30:  * Non-power-of-two textures: yes
11:14:30:  * Volume textures: yes
11:14:30:  * Multiple Render Targets: 8
11:14:30:    - With different bit depths: yes
11:14:30:  * Point Sprites: yes
11:14:30:  * Extended point parameters: yes
11:14:30:  * Max Point Size: 255
11:14:30:  * Vertex texture fetch: yes
11:14:30:  * Number of world matrices: 0
11:14:30:  * Number of texture units: 16
11:14:30:  * Stencil buffer depth: 8
11:14:30:  * Number of vertex blend matrices: 0
11:14:30:    - Max vertex textures: 16
11:14:30:    - Vertex textures shared: yes
11:14:30:  * Render to Vertex Buffer : no
11:14:30:  * GL 1.5 without VBO workaround: no
11:14:30:  * Frame Buffer objects: yes
11:14:30:  * Frame Buffer objects (ARB extension): no
11:14:30:  * Frame Buffer objects (ATI extension): no
11:14:30:  * PBuffer support: yes
11:14:30:  * GL 1.5 without HW-occlusion workaround: no
11:14:30:  * Separate shader objects: no
The main notable part is that unlike most of the other cards I've tested on, it doesn't have GL_DEPTH24_STENCIL8_EXT. So let's look at GLRenderSystem::_createDepthBufferFor. My comments are prefixed with an arthare

Code: Select all

DepthBuffer* GLRenderSystem::_createDepthBufferFor( RenderTarget *renderTarget )
	{
		GLDepthBuffer *retVal = 0;

		//Only FBO & pbuffer support different depth buffers, so everything
		//else creates dummy (empty) containers
		//retVal = mRTTManager->_createDepthBufferFor( renderTarget );
		GLFrameBufferObject *fbo = 0;
        renderTarget->getCustomAttribute(GLRenderTexture::CustomAttributeString_FBO, &fbo);

		if( fbo )
		{
			//Presence of an FBO means the manager is an FBO Manager, that's why it's safe to downcast
			//Find best depth & stencil format suited for the RT's format
			GLuint depthFormat, stencilFormat;
			static_cast<GLFBOManager*>(mRTTManager)->getBestDepthStencil( fbo->getFormat(),
																		&depthFormat, &stencilFormat );

			GLRenderBuffer *depthBuffer = new GLRenderBuffer( depthFormat, fbo->getWidth(),
																fbo->getHeight(), fbo->getFSAA() );

			GLRenderBuffer *stencilBuffer = depthBuffer; // arthare: assumes the stencilbuffer is the same as the depthbuffer.  This seems fine if they are the same, as they are in the GL_DEPTH24_STENCIL8_EXT case.
			if( depthFormat != GL_DEPTH24_STENCIL8_EXT && stencilBuffer ) // arthare: check the 2nd && parameter here: it's checking to see if the stencilBuffer is non-null.  But since we just copied the pointer's value from the depthBuffer (which unless we're out of memory will be non-null), this check is useless.  I think what was meant to be checked here is stencilFormat.
			{
				stencilBuffer = new GLRenderBuffer( stencilFormat, fbo->getWidth(),
													fbo->getHeight(), fbo->getFSAA() );
			}

			//No "custom-quality" multisample for now in GL
			retVal = new GLDepthBuffer( 0, this, mCurrentContext, depthBuffer, stencilBuffer,
										fbo->getWidth(), fbo->getHeight(), fbo->getFSAA(), 0, false );
		}

		return retVal;
	}
On the intel machine that is giving me trouble, GDebugger shows this:
GLRenderBuffer::GLRenderBuffer calls glRenderbufferStorageEXT with a valid format (this is the depth buffer getting initialized)
GLRenderBuffer::GLRenderBuffer calls glRenderbufferStorageEXT with format=0 (this is the stencil buffer attempting to get initialized, but with a bad format)

On my good machine, getBestDepthStencil returns GL_DEPTH24_STENCIL8_EXT for the depth format, which allows it to skip the 2nd GLRenderBuffer call.

Through some more debugging, I determined that getBestDepthStencil was returning a valid GL format for the depth buffer format, but 0x0 for the stencil buffer format. In fact, the way getBestDepthStencil was structured, it looked like it was impossible for it to return a valid depth format AND a valid stencil format. The modes it went through on my intel machine either had a valid stencil format or a valid depth format, but never both.

I can think of a couple possible fixes, but I'm not sure what is the most correct:
1) I'm misinterpreting this and it's all fine
2) The if(... && stencilBuffer) should be changed to if(... && stencilFormat) (in this case, the stencilBuffer pointer continues to be the same as the depth buffer, which I think is giving me some odd results)
3) Change getBestDepthStencil so it can return a valid depth format AND a valid stencil format, in case it can't find a mode running GL_DEPTH24_STENCIL8_EXT

I'm currently running with fix #2. It fixed my crash (I'm guessing the intel drivers didn't gracefully handle Ogre trying to use an invalid stencil buffer), but I've ended up with a big ugly black square in the middle of my screen, which I assume is a bit of corruption from incorrectly using the same GLRenderBuffer for both depth and stencil buffers. I'm going to see if I can implement #3 and what that gets me.

I checked in the repo, and it looks like this code is still present in the latest revision, so I thought I'd ask about it.
User avatar
spacegaier
OGRE Team Member
OGRE Team Member
Posts: 4304
Joined: Mon Feb 04, 2008 2:02 pm
Location: Germany
x 135
Contact:

Re: Bug in GLRenderSystem::_createDepthBufferFor?

Post by spacegaier »

Could you please create a JIRA ticket and link it to this thread. Thanks!
Ogre Admin [Admin, Dev, PR, Finance, Wiki, etc.] | BasicOgreFramework | AdvancedOgreFramework
Don't know what to do in your spare time? Help the Ogre wiki grow! Or squash a bug...
arthare
Halfling
Posts: 44
Joined: Fri Mar 29, 2013 5:40 pm
x 4

Re: Bug in GLRenderSystem::_createDepthBufferFor?

Post by arthare »

Done!

https://ogre3d.atlassian.net/browse/OGRE-213

It felt dirty to use JIRA outside of my real job...
Post Reply