Background resource loading blocks rendering (Ogre 1.9)

[robert82h]

I'm streaming textures via HTTP into Ogre using a custom Archive implementation which hands off the request to libcurl in my Archive::open() implementation using the blocking curl_easy API. I'm loading the resources in the background using Ogre's background loading support and OGRE_THREAD_SUPPORT = 2. This has been working great, but I noticed the rendering loop on the main thread gets blocked when a download is in progress.

The reason is the locking that goes on in OgreResourceGroupManager::openResource():

In the background threads, both OGRE_LOCK_AUTO_MUTEX and OGRE_LOCK_MUTEX(grp->OGRE_AUTO_MUTEX_NAME) are held while

Code: Select all

                        DataStreamPtr ptr = arch->open(resourceName);

is called. With my archive, this call can take many seconds to complete, depending on network conditions and file size.

On the main thread, locks on the same mutexes are acquired by ResourceBackgroundQueue::handleResponse()

Code: Select all

				ResourceManager *rm = ResourceGroupManager::getSingleton()
					._getResourceManager(req.resourceType);

This blocks the main thread until open() returns. I don't understand how background loading of resources is supposed to work with locking like that in place. After all, isn't background loading supposed to be used in exactly those situations where opening resources can take a while?

I came up with a patch against Ogre that unlocks the mutexes before invoking Archive::open(). This fixes the problem for me, however I had to make my Archive class thread-safe (multiple background threads now calling open() the same time).

I'd appreciate your comments about this issue and my patch. Any side effects maybe that I didn't think about (my understanding of Ogre's internals, esp. with regard to threading, is not that good)?

This is the patch:

Code: Select all

diff -r d4a99f7259f8 OgreMain/src/OgreResourceGroupManager.cpp
--- a/OgreMain/src/OgreResourceGroupManager.cpp    Sun Jan 19 22:02:41 2014 +0100
+++ b/OgreMain/src/OgreResourceGroupManager.cpp    Sun Jun 15 00:38:31 2014 +0200
@@ -688,7 +688,7 @@
                 "ResourceGroupManager::openResource");
         }
 
-        OGRE_LOCK_MUTEX(grp->OGRE_AUTO_MUTEX_NAME); // lock group mutex
+        OGRE_LOCK_MUTEX_NAMED(grp->OGRE_AUTO_MUTEX_NAME, groupLock); // lock group mutex
 
         Archive* pArch = 0;
         ResourceLocationIndex::iterator rit = grp->resourceIndexCaseSensitive.find(resourceName);
@@ -696,6 +696,8 @@
         {
             // Found in the index
             pArch = rit->second;
+            groupLock.unlock();
+            ogreAutoMutexLock.unlock();
             DataStreamPtr stream = pArch->open(resourceName);
             if (mLoadingListener)
                 mLoadingListener->resourceStreamOpened(resourceName, groupName, resourceBeingLoaded, stream);
@@ -711,6 +713,8 @@
             {
                 // Found in the index
                 pArch = rit->second;
+                groupLock.unlock();
+                ogreAutoMutexLock.unlock();
                 DataStreamPtr stream = pArch->open(resourceName);
                 if (mLoadingListener)
                     mLoadingListener->resourceStreamOpened(resourceName, groupName, resourceBeingLoaded, stream);
@@ -726,6 +730,8 @@
                     Archive* arch = (*li)->archive;
                     if (arch->exists(resourceName))
                     {
+                        groupLock.unlock();
+                        ogreAutoMutexLock.unlock();
                         DataStreamPtr ptr = arch->open(resourceName);
                         if (mLoadingListener)
                             mLoadingListener->resourceStreamOpened(resourceName, groupName, resourceBeingLoaded, ptr);