[2.2] Rare crash when copying to D3D11StagingTexture (with a fix that works for me)

Discussion area about developing with Ogre2 branches (2.1, 2.2 and beyond)
Post Reply
rujialiu
Goblin
Posts: 224
Joined: Mon May 09, 2016 8:21 am
x 20

[2.2] Rare crash when copying to D3D11StagingTexture (with a fix that works for me)

Post by rujialiu » Sat Mar 09, 2019 9:49 am

Hi!

When loading large scenes, we were having rare random crashes with DX11 (no crashes on Metal iOS). Recently we made a debug scene with deterministic texture loading sequence to reproduce the crash so I'm able to take a look. The final cause is: after getStreaming() returns a mapped region of a staging texture, when copying source image to that mapped region, we got write access violation.

The mapped region is a 1x1 TextureBox (it's the lowest mip of an 512x512 texture) at the edge of the staging texture (the whole staging texture is 512x512 and the mapped region is x=y=511, width=height=1) which does not look good.

I simply enforced all free boxes to be at least 2x2, then there was no crash any more! I'm not sure about the REAL cause of the crash, but anyway the following patch fixes my problem (note that the line numbers don't match because I've added some logging codes elsewhere)

Code: Select all

.../Direct3D11/src/OgreD3D11StagingTexture.cpp       | 20 +++++++++++++-------
 1 file changed, 13 insertions(+), 7 deletions(-)

diff --git a/RenderSystems/Direct3D11/src/OgreD3D11StagingTexture.cpp b/RenderSystems/Direct3D11/src/OgreD3D11StagingTexture.cpp
index 1edf14d..038ed0b 100644
--- a/RenderSystems/Direct3D11/src/OgreD3D11StagingTexture.cpp
+++ b/RenderSystems/Direct3D11/src/OgreD3D11StagingTexture.cpp
@@ -273,9 +273,15 @@ namespace Ogre
             newRecord[2].y += newRecord[2].height;
 
             const size_t idx = record - mFreeBoxes[slice].begin();
-            mFreeBoxes[slice].push_back( newRecord[0] );
-            mFreeBoxes[slice].push_back( newRecord[1] );
-            mFreeBoxes[slice].push_back( newRecord[2] );
+            if (newRecord[0].width > 1 && newRecord[0].height > 1) {
+                mFreeBoxes[slice].push_back(newRecord[0]);
+            }
+            if (newRecord[1].width > 1 && newRecord[1].height > 1) {
+                mFreeBoxes[slice].push_back(newRecord[1]);
+            }
+            if (newRecord[2].width > 1 && newRecord[2].height > 1) {
+                mFreeBoxes[slice].push_back(newRecord[2]);
+            }
             record = mFreeBoxes[slice].begin() + idx;
         }

@@ -312,7 +318,7 @@ namespace Ogre
         StagingBox newRecord = originalRecord;
         newRecord.x     = consumedBox.x + consumedBox.width;
         newRecord.width = originalRecord.width - consumedBox.width;
-        if( newRecord.width > 0 )
+        if( newRecord.width > 1 && newRecord.height > 1 )
             mFreeBoxes[slice].push_back( newRecord );
     }
     //-----------------------------------------------------------------------------------
@@ -360,7 +366,7 @@ namespace Ogre
         newRecord.width     = consumedBox.x;
         newRecord.y         = consumedBox.y;
         newRecord.height    = consumedBox.height;
-        if( newRecord.width > 0 )
+        if( newRecord.width > 1 && newRecord.height > 1 )
             mFreeBoxes[slice].push_back( newRecord );
 
         // New slice #2
@@ -369,14 +375,14 @@ namespace Ogre
         newRecord.width     = originalRecord.width - newRecord.x;
         newRecord.y         = consumedBox.y;
         newRecord.height    = consumedBox.height;
-        if( newRecord.width > 0 )
+        if( newRecord.width > 1 && newRecord.height > 1 )
             mFreeBoxes[slice].push_back( newRecord );
 
         // New slice #3
         newRecord = originalRecord;
         newRecord.y         = consumedBox.y + consumedBox.height;
         newRecord.height    = originalRecord.height - newRecord.y;
-        if( newRecord.height > 0 )
+        if( newRecord.width > 1 && newRecord.height > 1 )
             mFreeBoxes[slice].push_back( newRecord );
     }
     //-----------------------------------------------------------------------------------
0 x

User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 4077
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 231
Contact:

Re: [2.2] Rare crash when copying to D3D11StagingTexture (with a fix that works for me)

Post by dark_sylinc » Wed Mar 13, 2019 6:42 am

I haven't got the time yet to take a look into it; but it looks like most likely it's a off by one error when partitioning the StagingTexture (D3D11 is the only API where the StagingTextures are two-dimensional, the rest of the APIs are more sane in that regard and are one-dimensional)

I think I should hopefully be able to repro it by taking a 512x512 staging texture, then asking to map 511x511, then 511x1, then 1x512, and finally 1x1.

Could you provide the full dump of the TextureBox when it crashes? i.e. not just x, y, width & height, but all of its contents. And is the texture of the RGBA8_* family?

Cheers
Matias
0 x

rujialiu
Goblin
Posts: 224
Joined: Mon May 09, 2016 8:21 am
x 20

Re: [2.2] Rare crash when copying to D3D11StagingTexture (with a fix that works for me)

Post by rujialiu » Wed Mar 13, 2019 1:45 pm

dark_sylinc wrote:
Wed Mar 13, 2019 6:42 am
Could you provide the full dump of the TextureBox when it crashes? i.e. not just x, y, width & height, but all of its contents. And is the texture of the RGBA8_* family?
I'm afraid I won't have time to reproduce it in a few days, but I can remember that in EVERY crash the troublesome texture is a PNG, initially loaded as RGBA8 but later converted to RG8 via a texture filter as a result of "PrepareForNormalMapping", maybe RGBA8 -> RG8 is the key?
0 x

Post Reply