[2.2] Rare crash when copying to D3D11StagingTexture (with a fix that works for me)

[rujialiu]

Hi!

When loading large scenes, we were having rare random crashes with DX11 (no crashes on Metal iOS). Recently we made a debug scene with deterministic texture loading sequence to reproduce the crash so I'm able to take a look. The final cause is: after getStreaming() returns a mapped region of a staging texture, when copying source image to that mapped region, we got write access violation.

The mapped region is a 1x1 TextureBox (it's the lowest mip of an 512x512 texture) at the edge of the staging texture (the whole staging texture is 512x512 and the mapped region is x=y=511, width=height=1) which does not look good.

I simply enforced all free boxes to be at least 2x2, then there was no crash any more! I'm not sure about the REAL cause of the crash, but anyway the following patch fixes my problem (note that the line numbers don't match because I've added some logging codes elsewhere)

Code: Select all

.../Direct3D11/src/OgreD3D11StagingTexture.cpp       | 20 +++++++++++++-------
 1 file changed, 13 insertions(+), 7 deletions(-)

diff --git a/RenderSystems/Direct3D11/src/OgreD3D11StagingTexture.cpp b/RenderSystems/Direct3D11/src/OgreD3D11StagingTexture.cpp
index 1edf14d..038ed0b 100644
--- a/RenderSystems/Direct3D11/src/OgreD3D11StagingTexture.cpp
+++ b/RenderSystems/Direct3D11/src/OgreD3D11StagingTexture.cpp
@@ -273,9 +273,15 @@ namespace Ogre
             newRecord[2].y += newRecord[2].height;
 
             const size_t idx = record - mFreeBoxes[slice].begin();
-            mFreeBoxes[slice].push_back( newRecord[0] );
-            mFreeBoxes[slice].push_back( newRecord[1] );
-            mFreeBoxes[slice].push_back( newRecord[2] );
+            if (newRecord[0].width > 1 && newRecord[0].height > 1) {
+                mFreeBoxes[slice].push_back(newRecord[0]);
+            }
+            if (newRecord[1].width > 1 && newRecord[1].height > 1) {
+                mFreeBoxes[slice].push_back(newRecord[1]);
+            }
+            if (newRecord[2].width > 1 && newRecord[2].height > 1) {
+                mFreeBoxes[slice].push_back(newRecord[2]);
+            }
             record = mFreeBoxes[slice].begin() + idx;
         }

@@ -312,7 +318,7 @@ namespace Ogre
         StagingBox newRecord = originalRecord;
         newRecord.x     = consumedBox.x + consumedBox.width;
         newRecord.width = originalRecord.width - consumedBox.width;
-        if( newRecord.width > 0 )
+        if( newRecord.width > 1 && newRecord.height > 1 )
             mFreeBoxes[slice].push_back( newRecord );
     }
     //-----------------------------------------------------------------------------------
@@ -360,7 +366,7 @@ namespace Ogre
         newRecord.width     = consumedBox.x;
         newRecord.y         = consumedBox.y;
         newRecord.height    = consumedBox.height;
-        if( newRecord.width > 0 )
+        if( newRecord.width > 1 && newRecord.height > 1 )
             mFreeBoxes[slice].push_back( newRecord );
 
         // New slice #2
@@ -369,14 +375,14 @@ namespace Ogre
         newRecord.width     = originalRecord.width - newRecord.x;
         newRecord.y         = consumedBox.y;
         newRecord.height    = consumedBox.height;
-        if( newRecord.width > 0 )
+        if( newRecord.width > 1 && newRecord.height > 1 )
             mFreeBoxes[slice].push_back( newRecord );
 
         // New slice #3
         newRecord = originalRecord;
         newRecord.y         = consumedBox.y + consumedBox.height;
         newRecord.height    = originalRecord.height - newRecord.y;
-        if( newRecord.height > 0 )
+        if( newRecord.width > 1 && newRecord.height > 1 )
             mFreeBoxes[slice].push_back( newRecord );
     }
     //-----------------------------------------------------------------------------------

[dark_sylinc]

I haven't got the time yet to take a look into it; but it looks like most likely it's a off by one error when partitioning the StagingTexture (D3D11 is the only API where the StagingTextures are two-dimensional, the rest of the APIs are more sane in that regard and are one-dimensional)

I think I should hopefully be able to repro it by taking a 512x512 staging texture, then asking to map 511x511, then 511x1, then 1x512, and finally 1x1.

Could you provide the full dump of the TextureBox when it crashes? i.e. not just x, y, width & height, but all of its contents. And is the texture of the RGBA8_* family?

Cheers
Matias

[rujialiu]

Could you provide the full dump of the TextureBox when it crashes? i.e. not just x, y, width & height, but all of its contents. And is the texture of the RGBA8_* family?

I'm afraid I won't have time to reproduce it in a few days, but I can remember that in EVERY crash the troublesome texture is a PNG, initially loaded as RGBA8 but later converted to RG8 via a texture filter as a result of "PrepareForNormalMapping", maybe RGBA8 -> RG8 is the key?

[dark_sylinc]

This crash should be fixed in https://github.com/OGRECave/ogre-next/c ... ce270819f2

Thanks to TaaTT4 for providing an extremely easy repro!

Cheers

[rujialiu]

This crash should be fixed in https://github.com/OGRECave/ogre-next/c ... ce270819f2

Thanks to TaaTT4 for providing an extremely easy repro!

Cheers

That's fantastic. We just published our latest release 2 weeks ago so I guess we won't seriously test this until our next release... but it's nice to see it fixed!