Ogre Forums

Posted: **Sun Oct 11, 2020 10:46 am**

Hi!

I'm trying to add ImGui to Ogre. I've seen a few people on this forum implement this on 2.2 with D3D11 but haven't been able to find any examples for Metal.

Basically, I'm trying to use the example imgui implementation for metal. I've created a compositor manager as suggested in one of the links I've provided below.

Initialising seems simple like so:

Code: Select all

renderWindow->getCustomAttribute("MetalDevice", &device);
ImGui_ImplMetal_Init(device->mDevice);

However, unlike the D3D11 implementation provided by ImGui, I need to pass a 'MTLRenderPassDescriptor' to ImGui. Where I can get a MTLRenderPassDescriptor from Ogre?

Code: Select all

void ImGui_ImplMetal_NewFrame(MTLRenderPassDescriptor *renderPassDescriptor)

Additionally, I also need to access to a 'id<MTLCommandBuffer>' and 'id<MTLRenderCommandEncoder>'.

Code: Select all

ImGui_ImplMetal_RenderDrawData(ImDrawData* draw_data, id<MTLCommandBuffer> commandBuffer, id<MTLRenderCommandEncoder> commandEncoder)

Some reading I've done so far, both of these examples are for d3d11:
viewtopic.php?t=95975
viewtopic.php?t=94494

Posted: **Sun Oct 11, 2020 11:01 am**

I've just managed to HALF answer my own question, however using my method I am only able to access the stuff I need inside the CompositorPass's execute.

Ideally, I should be able to access the MTLRenderPassDescriptor outside of this execute function so that I can trigger a new ImGui frame before 'renderOneFrame'?

I've had to cram everything into the 'execute' method for this to work:

Code: Select all

    auto ogreRenderPass = static_cast<Ogre::MetalRenderPassDescriptor*>(ogreRoot->getRenderSystem()->getCurrentPassDescriptor());
    
    MTLRenderPassDescriptor *passDesc = [MTLRenderPassDescriptor renderPassDescriptor];
    ogreRenderPass->performLoadActions( passDesc, false );
    
    ImGui_ImplMetal_NewFrame(passDesc);
    ImGui::NewFrame();
    ImGui::ShowDemoWindow(NULL);
    
    ImGui::Render();
    ImDrawData* drawData = ImGui::GetDrawData();
    ImGui_ImplMetal_RenderDrawData(drawData, device->mCurrentCommandBuffer, device->mRenderEncoder);

Posted: **Sun Oct 11, 2020 4:44 pm**

crjc wrote: Sun Oct 11, 2020 11:01 am Ideally, I should be able to access the MTLRenderPassDescriptor outside of this execute function so that I can trigger a new ImGui frame before 'renderOneFrame'?

I'm afraid that is impossible. In Metal (and Vulkan and D3D12) rendering to a RenderTarget is each encapsulated in render passes. Once closed, no more rendering commands can be issued to that render pass.

By the time renderOneFrame returns the render pass(es) has long been closed.

As you found out CompositorPass::execute is the perfect location to do it.

I suggest you use setRenderPassDescToCurrent() and executeDelayedActions() like all the other passes do. Directly calling performLoadActions may cause unnecessary performance degradation

Posted: **Tue Oct 20, 2020 11:11 am**

dark_sylinc wrote: Sun Oct 11, 2020 4:44 pm I suggest you use setRenderPassDescToCurrent() and executeDelayedActions() like all the other passes do. Directly calling performLoadActions may cause unnecessary performance degradation

I have it all working but in the interest of performance I'd like to use those methods you've mentioned - But how would I get an instance of MTLRenderPassDescriptor that I can pass to ImGui_ImplMetal_NewFrame without using performLoadActions?

A side problem I now have: If I use SetSky, it completely breaks my ImGui setup, looks like ImGui is getting the sky's cube map by mistake, any suggestions on how to fix this?

Code: Select all

validateFunctionArguments:3536: failed assertion `Fragment Function(fragment_main): incorrect type of texture (MTLTextureTypeCube) bound at texture binding at index 0 (expect MTLTextureType2D) for texture[0].'

Thread 1 Queue : com.apple.main-thread (serial)
#0	0x00007fff6f2db33a in __pthread_kill ()
#1	0x00000000003a89bc in pthread_kill ()
#2	0x00007fff6f262808 in abort ()
#3	0x00007fff6f261ac6 in __assert_rtn ()
#4	0x00007fff3a679729 in MTLReportFailure.cold.2 ()
#5	0x00007fff3a66ce88 in MTLReportFailure ()
#6	0x00007fff5bcc54a9 in validateFunctionArguments(id<MTLDevice>, NSString*, NSString, NSArray*, MTLDebugFunctionArgument*, unsigned long, MTLDebugFunctionArgument, unsigned long, MTLDebugFunctionArgument, unsigned long, unsigned long long) ()
#7	0x00007fff5bcc4331 in -[MTLDebugRenderCommandEncoder validateCommonDrawErrors:instanceCount:baseInstance:maxVertexID:] ()
#8	0x00007fff5bcc5d44 in -[MTLDebugRenderCommandEncoder validateDrawIndexedPrimitives:indexCount:indexType:indexBuffer:indexBufferOffset:instanceCount:function:] ()
#9	0x00007fff5bcc5f5c in -[MTLDebugRenderCommandEncoder drawIndexedPrimitives:indexCount:indexType:indexBuffer:indexBufferOffset:] ()
#10	0x00007fff6bf24d77 in ___lldb_unnamed_symbol874$$libMTLCapture.dylib ()
#11	0x000000010001df92 in -[MetalContext renderDrawData:commandBuffer:commandEncoder:] at /3rd/imgui/examples/imgui_impl_metal.mm:528
#12	0x000000010001ba47 in ImGui_ImplMetal_RenderDrawData(ImDrawData*, id<MTLCommandBuffer>, id<MTLRenderCommandEncoder>) at /3rd/imgui/examples/imgui_impl_metal.mm:109
#13	0x0000000100279546 in ImGuiRenderer::render(Ogre::Camera const*) at /src/imguiRenderer/ImGuiRenderer.cpp:101
#14	0x0000000100277aa7 in ImGuiCompositorPass::execute(Ogre::Camera const*) at /src/imguiRenderer/ImGuiCompositorPass.cpp:22

Posted: **Wed Oct 21, 2020 4:57 pm**

crjc wrote: Tue Oct 20, 2020 11:11 am I have it all working but in the interest of performance I'd like to use those methods you've mentioned - But how would I get an instance of MTLRenderPassDescriptor that I can pass to ImGui_ImplMetal_NewFrame without using performLoadActions?

I took a look at Imgui's Metal code, they don't really need the MTLRenderPassDescriptor (that confused me for a while), they just need some of the information inside MTLRenderPassDescriptor to construct their PSOs.

Fortunately! After looking at what performLoadActions is doing, it doesn't actually set anything to the current state (unlike the other backends). It just fills passDesc.

Your code is correct! There is nothing to be done, and in your case don't call setRenderPassDescToCurrent nor executeRenderPassDescriptorDelayedActions; this way rendering the UI will continue after your regular rendering.

Unless you do Imgui at the end after postprocessing, in which case you should call:

Code: Select all

setRenderPassDescToCurrent();
RenderSystem *renderSystem = sceneManager->getDestinationRenderSystem();
renderSystem->executeRenderPassDescriptorDelayedActions();
MTLRenderPassDescriptor *passDesc = [MTLRenderPassDescriptor renderPassDescriptor];
MetalRenderPassDescriptor *ogreMetalRenderPass =
    static_cast<MetalRenderPassDescriptor *>( renderSystem->getCurrentPassDescriptor() );
renderPassDesc->performLoadActions( passDesc, false );

TL;DR: If it works for you (i.e. no validation complains) avoid calling setRenderPassDescToCurrent & executeRenderPassDescriptorDelayedActions();
If you get problems, then do call them.

Edit: To elaborate on this: Ogre passes have load & store actions per pass. For best performance, one should avoid closing a render encoder and opening a new one as much as possible (e.g. like submitting 2 products in the same package instead of submitting 2 packages)

Ogre's setRenderPassDescToCurrent & executeRenderPassDescriptorDelayedActions try their best to merge passes as much as possible if the user specified actions are compatible. But sometimes this breaks in subtle ways because the load/store actions the user requested are incompatible; and the only way to debug this is with RenderDoc (Vulkan backend) or XCode Graphics Debugger (Metal backend).

One easy way to always force passes being merged is by not calling setRenderPassDescToCurrent & executeRenderPassDescriptorDelayedActions at all; thus the render pass desc from the previous compositor pass gets reused. This may be undesirable however, if the previous pass was rendering to a completely unrelated render target to the one we will be rendering now (which is very rare for an UI rendering pass).

A side problem I now have: If I use SetSky, it completely breaks my ImGui setup, looks like ImGui is getting the sky's cube map by mistake, any suggestions on how to fix this?

That's odd. Imgui should set their own textures, overwriting whatever state we set; this implies there may be a bug in Imgui.
Unless you somehow issued Ogre calls in the middle of Imgui calls.

Anyway, the quick solution would be to tell Ogre to unset the texture at that slot:

Code: Select all

renderSystem->_setTexture( 0, nullptr, false );

But there may be something deeper going on since Imgui's code should be preventing this from happening at all.

Posted: **Wed Oct 21, 2020 5:58 pm**

Thanks so much for your informative post!

dark_sylinc wrote: Wed Oct 21, 2020 4:57 pm Anyway, the quick solution would be to tell Ogre to unset the texture at that slot:
Code: Select all
renderSystem->_setTexture( 0, nullptr, false );

This worked for me for now

, I guess I'll try and get around to reporting this as a bug to ImGui later.

Just curious if there's any plans to have ImGui built into Ogre "Next", as I believe this is the case with 1.xx?

Posted: **Tue Dec 22, 2020 11:52 pm**

MSAA has been destroying performance of ImGui.

To try and fix this, I've created a new target pass with no MSAA. ImGui is correctly rendering to this target however I'm not sure what is the correct way to merge together these two target passes, as the ImGui target pass hasn't been plugged in to anything.

How would I get the gui0 target pass to appear on top of the first WindowRT pass?

Code: Select all

nodeDef->setNumLocalTextureDefinitions(1);
    {
        auto textureDef = nodeDef->addTextureDefinition("gui0");
        textureDef->format = Ogre::PixelFormatGpu::PFG_RGBA8_SNORM;
        textureDef->fsaa = "1";
        
        auto rtv = nodeDef->addRenderTextureView( "gui0" );
        Ogre::RenderTargetViewEntry attachment;
        attachment.textureName = "gui0";
        rtv->colourAttachments.push_back( attachment );
        rtv->depthBufferId = Ogre::DepthBuffer::POOL_NO_DEPTH;
    }
    
    nodeDef->setNumTargetPass( 2 );
    
    {
        Ogre::CompositorTargetDef *targetDef = nodeDef->addTargetPass( "WindowRT" );
        ...
    }
    
    {
        Ogre::CompositorTargetDef *targetDef = nodeDef->addTargetPass( "gui0" );
        targetDef->setNumPasses( 1 );
        {
            {
                auto pass = targetDef->addPass( Ogre::PASS_CUSTOM, Ogre::IdString("imgui_pass") );
            }
        }
    }

Posted: **Tue Dec 22, 2020 11:58 pm**

My best guess is that Ogre is resolving the render texture (which 99% of cases it means we're done rendering to this render texture), then ImGui renders more stuff and it gets resolved again.

In terms of Ogre semantics, Store Actions for our Ogre passes should be set to "StoreAction::Store" instead of "MultisampleResolve", "StoreAndMultisampleResolve", or "StoreOrResolve"; because it will be your Imgui pass the one that should be doing the final resolve

Edit: I re-read what I wrote earlier. I forgot that Imgui is simply continuing our open pass without closing it. I have no idea why the giant perf drop. XCode Graphics Debugger may help you see what's going on

Posted: **Wed Dec 23, 2020 12:36 am**

dark_sylinc wrote: Tue Dec 22, 2020 11:58 pm Edit: I re-read what I wrote earlier. I forgot that Imgui is simply continuing our open pass without closing it. I have no idea why the giant perf drop. XCode Graphics Debugger may help you see what's going on

Frame Times from Xcode (when imgui is continuing the open pass):

Code: Select all

With Render Window configured FSAA = 8:

CPU = 33.3 ms
GPU = 10.3 ms

Code: Select all

With Render Window configured without FSAA parameter:

CPU = 16.7 ms
GPU = 1.0 ms

To try and solve this issue, I have created a new separate target pass that renders to a texture that has no MSAA, this is working and there doesn't seem to be a hit on performance, however I'm just not sure how to get this separate target to show on screen? (or is this a bad approach)

Posted: **Wed Dec 23, 2020 12:42 am**

I meant the Graphics Debugger

crjc wrote: Wed Dec 23, 2020 12:36 am To try and solve this issue, I have created a new separate target pass that renders to a texture that has no MSAA, this is working and there doesn't seem to be a hit on performance, however I'm just not sure how to get this separate target to show on screen? (or is this a bad approach)

It's a valid one. Draw your IMGUI onto a different RenderTarget, then use a pass_quad compositor pass with a modified version of the material Ogre/Copy/4xFP32 (Samples/Media/2.0/scripts/materials/Common/Copyback.material). Modify it to use alpha blending.

Note that this pass is not free; and if your resolution is huge (e.g. 4K) the performance impact might be noticeable (I dunno how much)

Posted: **Wed Dec 23, 2020 12:21 pm**

dark_sylinc wrote: Wed Dec 23, 2020 12:42 am I meant the Graphics Debugger

It seems that the first draw call from ImGui takes a full 3.5ms when Ogre's AA is enabled, but all the following calls are fast and take (mostly) microseconds.

When AA is disabled, no single ImGui call lasts milliseconds.

But I doubt this screenshot offers an explanation on the performance drop, if not is there anything in particular I should be probing?

EDIT: root->renderOneFrame() is what takes 33ms of the CPU. Rather worryingly, I've noticed this happening with the alternative approach, although not very often.

Ogre Forums

[2.3] ImGui with Metal

[2.3] ImGui with Metal

Re: [2.3] ImGui with Metal

Re: [2.3] ImGui with Metal

Re: [2.3] ImGui with Metal

Re: [2.3] ImGui with Metal

Re: [2.3] ImGui with Metal

Re: [2.3] ImGui with Metal

Re: [2.3] ImGui with Metal

Re: [2.3] ImGui with Metal

Re: [2.3] ImGui with Metal

Re: [2.3] ImGui with Metal