Questions about thread pooling alongside OgreNext

[knn217]

Hello, I want to create my own logic thread since OgreNext won't be the only renderer for my app.

My original idea was to thread pool based on the number of logic cores, and then have one thread use OgreNext to render. However, I noticed that even though I have not set up thread pooling, the program still printed out 16 exit-thread lines:

Code: Select all

The thread 6400 has exited with code 0 (0x0).
The thread 23632 has exited with code 0 (0x0).
The thread 8272 has exited with code 0 (0x0).
The thread 4996 has exited with code 0 (0x0).
The thread 18632 has exited with code 0 (0x0).
The thread 1600 has exited with code 0 (0x0).
The thread 6116 has exited with code 0 (0x0).
The thread 13820 has exited with code 0 (0x0).
The thread 18220 has exited with code 0 (0x0).
The thread 9140 has exited with code 0 (0x0).
The thread 24944 has exited with code 0 (0x0).
The thread 8068 has exited with code 0 (0x0).
The thread 7560 has exited with code 0 (0x0).
The thread 18684 has exited with code 0 (0x0).
The thread 14380 has exited with code 0 (0x0).
The program '[20492] game.exe' has exited with code 0 (0x0).

The number of threads doesn't match my number of cores (12) so OgreNext is probably not thread pooling. My assumtion is that some of these threads only handle 1 task, which normally lay dormant and only get activated sometimes (something like a resource loader, for example).

My questions are:
_What tasks are these threads doing?
_Is it possible these Ogre-threads will affect the performance of my external thread pool?
_If they do, can I control it some how, like putting some of these tasks into the same thread (for example, since my app does a loading screen instead of dynamically load resources during playtime, I can afford longer load time, which mean there's no need for an external thread for that).

[dark_sylinc]

Hi!

We ask you how many threads to use during SceneManager creation:

Code: Select all

#if OGRE_DEBUG_MODE >= OGRE_DEBUG_HIGH
        // Debugging multithreaded code is a PITA, disable it.
        const size_t numThreads = 1;
#else
        // getNumLogicalCores() may return 0 if couldn't detect
        const size_t numThreads = std::max<size_t>( 1, Ogre::PlatformInformation::getNumLogicalCores() );
        // Create the SceneManager, in this case a generic one
#endif
        mSceneManager = mRoot->createSceneManager( Ogre::ST_GENERIC, numThreads, "ExampleSMInstance" );

The TextureGpuManager also creates its own thread for steraming unless OGRE_FORCE_TEXTURE_STREAMING_ON_MAIN_THREAD is defined on build time (which is mainly intended for debugging rather than production).

By default TextureGpuManager::setMultiLoadPool is set to 0 (which means no threads) but if you use this setting, additional threads will be created.

If you're using OgreNext 4.0, we also ask how many threads in HlmsDiskCache::applyTo, but those live shortly.

And that's it.

If you see more threads, those are not ours. However the GPU driver may create threads of their own. If you're using SDL2, they may create their own threads too.

_What tasks are these threads doing?

They do a lof ot things like checking visibility each frame for all objects to submit to GPU, update scene node transformations, calculate AABBs, etc. If you have Forward+ enabled they also populate Forward+ structures with lights (very CPU intensive).

_Is it possible these Ogre-threads will affect the performance of my external thread pool?

Having too many threads is known as "oversubscription" and can indeed affect your performance; however usually that applies to threads that are not sleeping; unless you have a disproportionate amount of threads (e.g. 2000 threads on a 4 core CPU).

_If they do, can I control it some how, like putting some of these tasks into the same thread (for example, since my app does a loading screen instead of dynamically load resources during playtime, I can afford longer load time, which mean there's no need for an external thread for that).

You can use UserScalableTask and call mSceneManager->executeUserScalableTask & mSceneManager->waitForPendingUserScalableTask to run your algorithms in our threads.

For example you can't issue multiple mSceneManager->executeUserScalableTask at the same time. Each task must finish before submitting a new one.

[knn217]

If you see more threads, those are not ours. However the GPU driver may create threads of their own. If you're using SDL2, they may create their own threads too.

I see, when I checked the code:

Code: Select all

#if OGRE_DEBUG_MODE >= OGRE_DEBUG_HIGH
    // Debugging multithreaded code is a PITA, disable it.
    const size_t numThreads = 1;
#else
    // getNumLogicalCores() may return 0 if couldn't detect
    const size_t numThreads = std::max<size_t>(1, Ogre::PlatformInformation::getNumLogicalCores());
#endif

numThreads was set to 1 since I'm building on Debug, and I haven't set up any other thread, so it must be either GPU driver or SDL2.

I'm not sure how to stop the GPU from creating threads, but I actually did try to turn off SDL2, I just wasn't sure I did it the right way.

I turned off SDL2 by redefining it to 0, but the threads still get created and I also got a warning from the compiler so I just left it as 1 for now.
How do you turn off SDL2 properly?

[dark_sylinc]

Hi!

You may want to check out Samples/2.0/Tutorials/Tutorial00_Basic which shows how to initialize OgreNext without SDL2 (but it's also missing a lot of other funcionality and good practices from the OgreCommon framework).

Please note that even libc (e.g. malloc, printf, fopen) may spawn some threads to do background work.

If you are looking for absolute 0 foreign threads you need to look into the same practices that embedded programmers and demosceners use, and that will unlikely work with OgreNext though.

It's good that you want to avoid unnecessary threads spawning, but reducing it to 0 is often a bad idea too. Some background work needs to be done and that is just a part of any program in the last 2 decades (and then there is Android where it's "normal" for a Hello World to have 30 threads ).

I'm not sure how to stop the GPU from creating threads

You can't. Some drivers expose extra functionality to have certain degree of control over this (Mesa may have driconf and env variables, AMD has AGS SDK, NVIDIA has NVAPI), but ultimately you are at their mercy.

Cheers

[knn217]

It's good that you want to avoid unnecessary threads spawning, but reducing it to 0 is often a bad idea too. Some background work needs to be done and that is just a part of any program in the last 2 decades (and then there is Android where it's "normal" for a Hello World to have 30 threads ).

Yeah, that makes sense, I was a bit paranoid since I have little experience with thread pooling, especially when using it alongside other libs so I've never considered that the libs can also spawn threads. I'll rethink my logic design since forcing the renderer to use only one thread is probably too unrealistic.