Imgui with ogre-next

[bri77]

Hi,

I'm trying to get imgui to work with ogre-next on a vulkan samsung galaxy s25 android phone (Imgui does not render on it, but the main ogre next mesh/etc renders fine).
It works fine with vulkan on windows and vulkan on a galaxy s9 android phone.

I'm using the imgui code from: https://github.com/edherbert/ogre-next-imgui

I read in another post that it can be done with creating another compositor pass. I've done this, and I can see through renderdoc that the projection matrix uniform comes through and the inputs looks correct, but the output was all like -1 or 0s.

Before creating another compositor pass the projection matrix uniform was not getting through. I tested this by just assigning the gl_Position to 4 values of the projection matrix in the shader.

After using a custom compositor pass for imgui, the projection matrix seems to be getting through fine now (checked with renderDoc), but the vertices position, uv, color doesn't seem to be making it through.

The main changes I made in the ImguiManager is to split up the rendering into two parts. preRender() and render(Ogre::RenderPassDescriptor *renderPass) (The renderpass is passed in from the compositor pass execute().) preRender() is called before renderOneFrame().

I'm probably not using the mPSOCache cache correctly. I tried making it look like what is in the information in OgrePsoCacheHelper.h.

preRender:

Code: Select all

void ImguiManager::preRender() {
#ifdef __APPLE__
    @autoreleasepool {
#endif
    numberOfDrawsForFrame = 0;
    Ogre::RenderSystem* renderSystem = mSceneMgr->getDestinationRenderSystem();

    if (!renderPassDesc) {
        renderPassDesc = renderSystem->createRenderPassDescriptor();
        renderPassDesc->mColour[0].texture = mCompositor->getFinalTarget();
        renderPassDesc->mColour[0].loadAction = Ogre::LoadAction::Load;
        renderPassDesc->mColour[0].storeAction = Ogre::StoreAction::StoreAndMultisampleResolve;
        renderPassDesc->entriesModified(Ogre::RenderPassDescriptor::All);
    }
    Ogre::Vector4 viewportSize(0, 0, 1, 1);
    Ogre::Vector4 scissors(0, 0, 1, 1);
    // renderSystem->beginRenderPassDescriptor(renderPassDesc, mCompositor->getFinalTarget(), 0, &viewportSize, &scissors, 1, false, false);
    // renderSystem->executeRenderPassDescriptorDelayedActions();

    // Cancel rendering if not necessary
    // or if newFrame didn't got called
    if (mFrameEnded)
        return;
    mFrameEnded = true;

    currentFrame = ImGui::GetFrameCount();
    if (currentFrame == mLastRenderedFrame) {
        LOGE("ImguiManager::render return");
        return;
    }
    mLastRenderedFrame = currentFrame;

    // Tell ImGui to create the buffers
    ImGui::Render();

//   if (currentFrame <= 1) {
        // Lots of stuff can be done only once for the sake of efficiency.

        mPSOCache->clearState();
        const Ogre::HlmsBlendblock* blendblock = mPass->getBlendblock();
        const Ogre::HlmsMacroblock* macroblock = mPass->getMacroblock();
        mPSOCache->setMacroblock(macroblock);
        mPSOCache->setBlendblock(blendblock);
        mPSOCache->setVertexShader(const_cast<Ogre::GpuProgramPtr&>(mPass->getVertexProgram()));
        mPSOCache->setPixelShader(const_cast<Ogre::GpuProgramPtr&>(mPass->getFragmentProgram()));
//   }

    ImDrawData* drawData = ImGui::GetDrawData();
    int numberDraws = 0;

    int vpWidth = mCompositor->getFinalTarget()->getWidth();
    int vpHeight = mCompositor->getFinalTarget()->getHeight();

    // iterate through all lists (at the moment every window has its own)
    for (int n = 0; n < drawData->CmdListsCount; n++) {
        const ImDrawList* drawList = drawData->CmdLists[n];
        const ImDrawVert* vtxBuf = drawList->VtxBuffer.Data;
        const ImDrawIdx* idxBuf = drawList->IdxBuffer.Data;

        unsigned int startIdx = 0;

        for (int i = 0; i < drawList->CmdBuffer.Size; i++) {
            // create renderables if necessary
            // This creates a new one each time it notices the list is bigger than necessary.
//            if (i >= mRenderables.size()) {
                mRenderables.push_back(new ImguiRenderable());
//            }
            int currentRenderableIndex = mRenderables.size() - 1;

            // update their vertex buffers
            const ImDrawCmd* drawCmd = &drawList->CmdBuffer[i];
            mRenderables[currentRenderableIndex]->updateVertexData(vtxBuf, &idxBuf[startIdx], drawList->VtxBuffer.Size, drawCmd->ElemCount);
            mRenderables[currentRenderableIndex]->textureGpu = (Ogre::TextureGpu*)drawCmd->TextureId;


            // renderSystem->beginRenderPassDescriptor(renderPassDesc, mCompositor->getFinalTarget(), 0, &viewportSize, &scissors, 1, false, false);
            // renderSystem->executeRenderPassDescriptorDelayedActions();

            Ogre::v1::RenderOperation renderOp;
            mRenderables[currentRenderableIndex]->getRenderOperation(renderOp, false);
    
            bool enablePrimitiveRestart = true;  // tried both true and false...no change
    
            Ogre::VertexElement2VecVec vertexElements = renderOp.vertexData->vertexDeclaration->convertToV2();
            mPSOCache->setVertexFormat(vertexElements,
                                    renderOp.operationType,
                                    enablePrimitiveRestart);

            mRenderables[currentRenderableIndex]->customSavedValue = mPSOCache->getRenderableHash();                                    
    
            // increase start index of indexbuffer
            startIdx += drawCmd->ElemCount;
            numberDraws++;
            numberOfDrawsForFrame++;
        }
    }


    // delete unused renderables
//     while (mRenderables.size() > numberDraws) {
// //        LOGE("ImGui::Render() here 3");
//         delete mRenderables.back();
//         mRenderables.pop_back();
//     }

#ifdef __APPLE__
    }
#endif
}
````


render:
```c
void ImguiManager::render(Ogre::RenderPassDescriptor *renderPass) {
    int numberDraws = 0;

    if (numberOfDrawsForFrame > 0) {
        mPSOCache->clearState();
        mPSOCache->setRenderTarget(renderPass);  // dark_sylinc's advice on setting rendertarget, which looks like the renderwindow obj)
    }

    for (int i = 0; i < numberOfDrawsForFrame; i++) {

        Ogre::v1::RenderOperation renderOp;
        mRenderables[i]->getRenderOperation(renderOp, false);

        bool enablePrimitiveRestart = true;  // tried both true and false...no change

        Ogre::VertexElement2VecVec vertexElements = renderOp.vertexData->vertexDeclaration->convertToV2();
        mPSOCache->setVertexFormat(vertexElements,
                                renderOp.operationType,
                                enablePrimitiveRestart);
        
        Ogre::HlmsPso* pso = mPSOCache->getPso(mRenderables[i]->customSavedValue);
        mSceneMgr->getDestinationRenderSystem()->_setPipelineStateObject(pso);

        mSceneMgr->getDestinationRenderSystem()->bindGpuProgramParameters(Ogre::GPT_VERTEX_PROGRAM, mPass->getVertexProgramParameters(), Ogre::GPV_ALL);

        Ogre::TextureGpu* texGpu = (Ogre::TextureGpu*) mRenderables[i]->textureGpu; // drawCmd->TextureId;
        mSceneMgr->getDestinationRenderSystem()->_setTexture(0, texGpu, false);
        mSceneMgr->getDestinationRenderSystem()->_setHlmsSamplerblock(0, mSamplerblock);

        Ogre::v1::CbRenderOp op(renderOp);
        mSceneMgr->getDestinationRenderSystem()->_setRenderOperation(&op);
        mSceneMgr->getDestinationRenderSystem()->_render(renderOp);

        numberDraws++;
    }

    // delete unused renderables
    while (mRenderables.size() > 0) {
        delete mRenderables.back();
        mRenderables.pop_back();
    }        
}

[bri77]

I've been using ogre next for about 2-3 weeks now, so I'm not really familiar with the code base yet. Lots of debugging and stepping through the code so far.

I used another vulkan speicfic imgui to render through the new custom compositor pass, and it renders fine on the galaxy s9 and s25.

The render code for this is just:

Code: Select all

void ImGui_Render() {
    Ogre::VulkanRenderSystem *vulkanRenderSystem = (Ogre::VulkanRenderSystem *) Ogre::Root::getSingleton().getRenderSystem();
    Ogre::VulkanQueue &graphicsQueue = vulkanRenderSystem->getVulkanDevice()->mGraphicsQueue;
    VkFence fence = graphicsQueue.acquireCurrentFence();
    ImGui::Render();
    VkCommandBuffer vkcb = graphicsQueue.getCurrentCmdBuffer();
    ImGui_ImplVulkan_RenderDrawData(ImGui::GetDrawData(), vkcb, VK_NULL_HANDLE);
    graphicsQueue.releaseFence(fence);
}

I'm using most of the vulkan parts from the ogre-next render system to feed into imgui. I created a new vulkan render pass (vkCreateRenderPass) that is used by imgui only. This all seems to work some how. (New to vulkan, but have used open gl / d3d a long time ago).

Imgui shows up correctly on the phone screen using this approach, but RenderDoc still shows the pos, uv, color, etc output as in incorrect values. The main ogre-next meshes in RenderDoc all work perfectly (vertice/data input and output).

I was wondering, if a better approach would be just to create a mesh and fill it with the imgui renderables. I would have to figure out how to get this imgui "mesh" to render last. I think this approach would be much more stable.

Looking for comments regarding if the mesh approach would be better than trying to fix the other 2 imgui implementations that I've tried.

Thanks!

[dark_sylinc]

Hi!

That graphicsQueue.acquireCurrentFence() and graphicsQueue.releaseFence(fence) are doing nothing. An acquired fence is supposed to be used somewhere, like e.g. in vkWaitForFences(). Since you acquire and then release, it just does nothing with it.

but RenderDoc still shows the pos, uv, color, etc output as in incorrect values

Is that when connected to the phone, or when debugging your Windows/Linux app? Because RenderDoc displaying wrong data while debugging a phone capture isn't surprising. But if that happens when debugging a Windows/Linux capture, that is more serious.

Have you tried validation layers to see if they have something to say about the Vulkan code?

[bri77]

Thanks for the reply, dark_sylinc,

I probably should read more on the vulkan fences, I thought they were like mutexes. I just wanted to make sure some unaccounted multi threading stuff was happening. I'll remove the acquire/release for the fence.

Regarding the incorrect VS Output in RenderDoc, this is when RenderDoc is connected to the android galaxy s25 phone (The incorrect VS output was only for the imgui rendering, ogre-next scene VS Output looks correct). Everything looks correct on windows.

I have not tried using vulkan validation layers. I probably should've done that from the start instead of spending 4-5 days trying to figure out what was causing it to break. I learned a bit about the ogre next render system from doing that.

I did try the original imgui implementation on a pixel 8 pro, which caused some crash in the render system (I forgot where it was at now, it was 4-6 call stack deep or so). Using the new imgui integration, it's working fine on the pixel 8 pro also.

I'm currently trying to just finish the project as fast as possible. I'll probably come back to the vulkan rendering in the future.

[dark_sylinc]

Regarding the incorrect VS Output in RenderDoc, this is when RenderDoc is connected to the android galaxy s25 phone (The incorrect VS output was only for the imgui rendering, ogre-next scene VS Output looks correct). Everything looks correct on windows.

Oh! Then this is "expected". Vulkan Android driver quality is horrible, and Android itself causes a lot of issues that can make RenderDoc to malfunction or crash. It's a miracle at all that it even works.

This is assuming that Validation Layers are not complaining.