Vulkan out of memory error

[jwwalker]

I'm getting a VK_ERROR_OUT_OF_DEVICE_MEMORY error from a vkAllocateMemory, having something to do with shadows. Poking around in the debugger, I see that it's dealing with a texture of format PFG_D32_FLOAT, dimensions 2048 x 5120. Maybe this is a case of trying to do too much on a somewhat old and wimpy PC, but there's no error if I do the same stuff using the Direct3D RenderSystem.

Backtrace:

Code: Select all

RenderSystem_Vulkan_d.dll!Ogre::ExceptionFactory::throwException(Ogre::Exception::ExceptionCodes code, int number, const std::string & desc, const std::string & src, const char * file, long line) Line 301	C++
RenderSystem_Vulkan_d.dll!Ogre::VulkanVaoManager::allocateVbo(unsigned __int64 sizeBytes, unsigned __int64 alignment, Ogre::VulkanVaoManager::VboFlag vboFlag, unsigned int textureMemTypeBits, unsigned __int64 & outVboIdx, unsigned __int64 & outBufferOffset) Line 1092	C++
RenderSystem_Vulkan_d.dll!Ogre::VulkanVaoManager::allocateTexture(const VkMemoryRequirements & memReq, unsigned __int64 & outVboIdx, unsigned __int64 & outBufferOffset) Line 1250	C++
RenderSystem_Vulkan_d.dll!Ogre::VulkanTextureGpu::createInternalResourcesImpl() Line 148	C++
OgreMain_d.dll!Ogre::TextureGpu::transitionToResident() Line 462	C++
OgreMain_d.dll!Ogre::TextureGpu::_transitionTo(Ogre::GpuResidency::GpuResidency newResidency, unsigned char * sysRamCopy, bool autoDeleteSysRamCopy) Line 521	C++
OgreMain_d.dll!Ogre::TextureDefinitionBase::setupTexture(Ogre::TextureGpu * tex, const Ogre::TextureDefinitionBase::TextureDefinition & textureDef, const Ogre::TextureGpu * finalTarget) Line 388	C++
OgreMain_d.dll!Ogre::TextureDefinitionBase::createTexture(const Ogre::TextureDefinitionBase::TextureDefinition & textureDef, const std::string & texName, const Ogre::TextureGpu * finalTarget, Ogre::RenderSystem * renderSys) Line 317	C++
OgreMain_d.dll!Ogre::TextureDefinitionBase::createTextures(const std::vector<Ogre::TextureDefinitionBase::TextureDefinition,std::allocator<Ogre::TextureDefinitionBase::TextureDefinition>> & textureDefs, std::vector<Ogre::TextureGpu *,std::allocator<Ogre::TextureGpu *>> & inOutTexContainer, unsigned int id, const Ogre::TextureGpu * finalTarget, Ogre::RenderSystem * renderSys) Line 297	C++
OgreMain_d.dll!Ogre::CompositorNode::CompositorNode(unsigned int id, Ogre::IdString name, const Ogre::CompositorNodeDef * definition, Ogre::CompositorWorkspace * workspace, Ogre::RenderSystem * renderSys, Ogre::TextureGpu * finalTarget) Line 84	C++
OgreMain_d.dll!Ogre::CompositorShadowNode::CompositorShadowNode(unsigned int id, const Ogre::CompositorShadowNodeDef * definition, Ogre::CompositorWorkspace * workspace, Ogre::RenderSystem * renderSys, Ogre::TextureGpu * finalTarget) Line 83	C++
OgreMain_d.dll!Ogre::CompositorWorkspace::findOrCreateShadowNode(Ogre::IdString nodeDefName, bool & bCreated) Line 918	C++
OgreMain_d.dll!Ogre::CompositorPassScene::CompositorPassScene(const Ogre::CompositorPassSceneDef * definition, Ogre::Camera * defaultCamera, const Ogre::RenderTargetViewDef * rtv, Ogre::CompositorNode * parentNode) Line 69	C++
OgreMain_d.dll!Ogre::CompositorNode::createPasses() Line 728	C++
OgreMain_d.dll!Ogre::CompositorWorkspace::connectAllNodes() Line 342	C++
OgreMain_d.dll!Ogre::CompositorWorkspace::recreateAllNodes() Line 645	C++
OgreMain_d.dll!Ogre::CompositorWorkspace::CompositorWorkspace(unsigned int id, const Ogre::CompositorWorkspaceDef * definition, const std::vector<Ogre::TextureGpu *,std::allocator<Ogre::TextureGpu *>> & externalRenderTargets, Ogre::SceneManager * sceneManager, Ogre::Camera * defaultCam, Ogre::RenderSystem * renderSys, bool bEnabled, unsigned char executionMask, unsigned char viewportModifierMask, const Ogre::Vector4 & vpOffsetScale, const std::vector<Ogre::UavBufferPacked *,std::allocator<Ogre::UavBufferPacked *>> * uavBuffers, const Ogre::StdMap<Ogre::GpuTrackedResource *,Ogre::ResourceStatus,std::less<Ogre::GpuTrackedResource *>> * initialLayouts) Line 93	C++
OgreMain_d.dll!Ogre::CompositorManager2::addWorkspace(Ogre::SceneManager * sceneManager, const std::vector<Ogre::TextureGpu *,std::allocator<Ogre::TextureGpu *>> & externalRenderTargets, Ogre::Camera * defaultCam, Ogre::IdString definitionName, bool bEnabled, int position, const std::vector<Ogre::UavBufferPacked *,std::allocator<Ogre::UavBufferPacked *>> * uavBuffers, const Ogre::StdMap<Ogre::GpuTrackedResource *,Ogre::ResourceStatus,std::less<Ogre::GpuTrackedResource *>> * initialLayouts, const Ogre::Vector4 & vpOffsetScale, unsigned char vpModifierMask, unsigned char executionMask) Line 521	C++
OgreMain_d.dll!Ogre::CompositorManager2::addWorkspace(Ogre::SceneManager * sceneManager, Ogre::TextureGpu * finalRenderTarget, Ogre::Camera * defaultCam, Ogre::IdString definitionName, bool bEnabled, int position, const std::vector<Ogre::UavBufferPacked *,std::allocator<Ogre::UavBufferPacked *>> * uavBuffers, const Ogre::StdMap<Ogre::GpuTrackedResource *,Ogre::ResourceStatus,std::less<Ogre::GpuTrackedResource *>> * initialLayouts, const Ogre::Vector4 & vpOffsetScale, unsigned char vpModifierMask, unsigned char executionMask) Line 498	C++

[dark_sylinc]

It could be a legit out of memory error or an OgreNext bug.

It could be an OgreNext bug. For example in this commit I found mUsedHeapMemory was never decreased, so our allocator went through all the blocks and eventually failed all test for this:

Code: Select all

if( mUsedHeapMemory[heapIdx] + sizeBytes < memHeaps[heapIdx].size )

It found no heap was big enough to hold everything we've allocated so far (because deallocations were not being accounted!) and gave up.

Why it could be legit? D3D11 is more flexible about memory; so it will start swapping out of VRAM contents that haven't been in use. You could overall request 3GB on a 2GB GPU and as long as you don't use it all at once in the same frame the driver would do transfer gymnastics to get the frame rendered (at a reduced framerate).

Our Vulkan allocator implementation doesn't do that, so if you request more than what the GPU can physically do, it will error out.

What sounds like the suspect root cause is the resolution: 2048x5120. Sometimes the driver (SPECIALLY in old GPUs) will use obscene alignments (or perhaps we're the ones over-aligning? that is possible) so I wouldn't be surprised if the amount of memory requested in bytes is the same as 2048 x 8192, maybe even a lot more. I wouldn't be surprised if 4096x4096 succeeds (when in theory 4096x4096 > 2048x5120, aka 64MB vs 40MB).

Typically this comes from vkGetImageMemoryRequirements( device->mDevice, mFinalTextureName, &memRequirements ); called before vaoManager->allocateTexture.

Watch out for both alignment and size (the returned memReq.size may be much bigger than 2048x5120x4).

See Samples/2.0/Tutorials/Tutorial_Memory and keep an eye on the memory. There's a chance you're simply running out of memory.

[jwwalker]

After some more testing, I noticed that the error happened after creating a window with a PlanarReflections object allowing a maximum active actor count of 10. When I reduced that to 5, the error went away, at least for now. I guess that makes some sense. If it must allocate a separate set of shadow map textures and such for each potential mirror, that memory could mount up quickly.

[jwwalker]

By the way, in the memory tutorial, if I pick the option to minimize memory, it calls VulkanVaoManager::cleanupEmptyPools, which throws an ERR_NOT_IMPLEMENTED exception.

[dark_sylinc]

Just catch that exception or don't call the function for Vulkan RenderSystem.