Ray tracing support for Ogre 2.2+

[Hotshot5000]

I would like to mess around a bit with some ray/path tracing on the Vulkan render system. I am curious if there is any work done towards ray tracing support in Ogre so not start from zero or redo work already done.

I will start reading about Vulkan ray tracing extension and in a few days maybe create a fork from Ogre repo and create a branch from the latest Ogre version that supports Vulkan. I think it was integrated in the mainline a while ago but I haven't kept up with what changed in the meantime.

I am not sure Ogre needs ray tracing or wants it but it's just a side project for me. If I come up with anything useful people might want to take a look, I don't know...

I have a 1660 Ti on my laptop so I don't actually have ray tracing hardware support but running the samples it seems to be working fine for simple stuff. Maybe at some point I will buy a 3070 or 3080 but the supply issue means I only get "sold out" messages everywhere.

[dark_sylinc]

0 work done

maybe create a fork from Ogre repo and create a branch from the latest Ogre version that supports Vulkan. I think it was integrated in the mainline a while ago but I haven't kept up with what changed in the meantime.

Yeah, fork from latest since we've improved our Vulkan renderer a lot since then. It's on master branch

Maybe at some point I will buy a 3070 or 3080 but the supply issue means I only get "sold out" messages everywhere

Ditto

[Hotshot5000]

Hmm.. I get

Exception thrown at 0x00007FF89CE60F41 (nvoglv64.dll) in Sample_PbsMaterials.exe: 0xC0000005: Access violation reading location 0xFFFFFFFFFFFFFFFF.

at line 544 in VulkanWindow.cpp (VkResult result = vkQueuePresentKHR( mDevice->mPresentQueue, &present );)

This happens on both CustomRenderable and PbsMaterials sample (I haven't tested the others) when running on the nvidia graphics card (1660 Ti). On the integrated AMD graphics it works fine. I am running the latest nvidia drivers (461.09).

I have the latest build from master from OgreCave.

I will investigate what is going wrong here. I see no validation layer error and the objects passed as params seem ok.

EDIT: Looks like there is a bug in Ogre. I went back to the Ogre 2.2 vulkan branch and it is working there.

[dark_sylinc]

We started having issues with NVIDIA after these bugfixes:

viewtopic.php?p=549502#p549502
and these
viewtopic.php?p=549518#p549518

But we could never truly understand what's going wrong and NVIDIA is literally the only vendor having trouble (works fine on Mesa AMD, Propieteary AMD, Mesa Intel, Propietary Intel, Mali, Adreno).
Since I don't have an NV card with me to repro the issue, I've been unable to fix or workaround this problem.

It could be a driver bug, it could be an Ogre bug.

Perhaps you get better luck? Sorry for pushing this one onto you, but you understand the code and can repro it.

If you're in a rush to get Raytracing you can undo the changes and it should start working again on NV (or use bisect to detect the exact commit, but I think it was one of those I pointed).

[Hotshot5000]

I am in no rush. I will try to get to the bottom of this.

[Hotshot5000]

Yep found the bug. It wasn't related to the semaphores in around the vkQueueSubmit. The problem was that mVkInstance is created before the _createWindow() is called. After which a device is created. Everything vulkan related seem to need to be used only after window creation. At least on nVidia.

I will try to leave the creation of VkInstance before window creation in order to have the cards for enumeration in addConfig() but then I will destroy it and recreate it after the window is up and running. See if that works.

EDIT: Nope it doesn't work. I get the same error if I recreate mVkInstance. It looks like I need to initialize it after creating the window.... But then, how do I get the video card options in the startup dialog?

[dark_sylinc]

Can you post a diff of the hacked version that works for you?

Because it doesn't make sense. We call OGRE_NEW VulkanWin32Window() then initialize mVkInstance (but due to config settings, mVkInstance should be initialized by now).

However VulkanWin32Window's constructor does pretty much nothing.

After creating the VkInstance (if not already created), we create the VkDevice and then call win->_initialize which actually creates the window.

Based on what you said, my best guess is that we perform a buffer overflow enumerating settings for the config dialog, or leave the API in an inconsistent state, but Valgrind says nothing on Linux; thus maybe if I see the sequence that works for you (even if hacked) may shed some light

[Hotshot5000]

Code: Select all

diff --git a/RenderSystems/Vulkan/src/OgreVulkanSupport.cpp b/RenderSystems/Vulkan/src/OgreVulkanSupport.cpp
index be468a291..29f2aaa8b 100644
--- a/RenderSystems/Vulkan/src/OgreVulkanSupport.cpp
+++ b/RenderSystems/Vulkan/src/OgreVulkanSupport.cpp
@@ -85,8 +85,9 @@ namespace Ogre
         if( renderSystem->getVkInstance() )
             return;
 
-        renderSystem->initializeVkInstance();
-        enumerateDevices( renderSystem );
+        // renderSystem->initializeVkInstance();
+        // enumerateDevices( renderSystem );
+        // renderSystem->deinitializeVkInstance();
     }
     //-------------------------------------------------------------------------
     void VulkanSupport::addConfig( VulkanRenderSystem *renderSystem )
@@ -97,6 +98,9 @@ namespace Ogre
 
         optDevices.name = "Device";
 
+        mDevices.clear();
+        mDevices.push_back( "GeForce GTX 1660 Ti #0" );
+
         FastArray<String>::const_iterator itor = mDevices.begin();
         FastArray<String>::const_iterator endt = mDevices.end();
 

I added a deinitializeVkInstance when tried to force deinit before creating the window:

Code: Select all

diff --git a/RenderSystems/Vulkan/src/OgreVulkanRenderSystem.cpp b/RenderSystems/Vulkan/src/OgreVulkanRenderSystem.cpp
index a475c6c7e..145c91f2c 100644
--- a/RenderSystems/Vulkan/src/OgreVulkanRenderSystem.cpp
+++ b/RenderSystems/Vulkan/src/OgreVulkanRenderSystem.cpp
@@ -228,11 +228,7 @@ namespace Ogre
             mDebugReportCallback = 0;
         }
 
-        if( mVkInstance )
-        {
-            vkDestroyInstance( mVkInstance, 0 );
-            mVkInstance = 0;
-        }
+        deinitializeVkInstance();
     }
     //-------------------------------------------------------------------------
     void VulkanRenderSystem::shutdown( void )
@@ -823,6 +819,15 @@ namespace Ogre
 #endif
     }
     //-------------------------------------------------------------------------
+    void VulkanRenderSystem::deinitializeVkInstance( void )
+    {
+        if( mVkInstance )
+        {
+            vkDestroyInstance( mVkInstance, 0 );
+            mVkInstance = 0;
+        }
+    }
+    //-------------------------------------------------------------------------

[dark_sylinc]

Ok you posted something very interesting:

You commented the call to initializeVkInstance. But this is called later and the order of operations is pretty much the same.

However you also commented out enumerateDevices, which is never called again. What happens if you only comment out enumerateDevices and leave initializeVkInstance? (in VulkanSupport::initialize)

Edit: How many and which devices are listed? Is your NV card the only one? Or is there an integrated GPU too?

Edit 2: What is the return value of VulkanSupport::getSelectedDeviceIdx when VkInstance is initialized early, and what is the returned value in the hacked version?

[Hotshot5000]

Ok you posted something very interesting:

You commented the call to initializeVkInstance. But this is called later and the order of operations is pretty much the same.

However you also commented out enumerateDevices, which is never called again. What happens if you only comment out enumerateDevices and leave initializeVkInstance? (in VulkanSupport::initialize)

Edit: How many and which devices are listed? Is your NV card the only one? Or is there an integrated GPU too?

Code: Select all

renderSystem->initializeVkInstance();
        // enumerateDevices( renderSystem );
        // renderSystem->deinitializeVkInstance();

This gets me the error again. I don't think it was related to device enumeration.

[dark_sylinc]

Then I'm clueless.

If you run out of ideas, I would try to repro in a simple Vulkan application.

More particularly I could suspect of a race condition (i.e. most apps create the instance and almost immediately afterwards the window).

Thus if that theory is correct, a sample which does:

Code: Select all

createVkInstance();
for( size_t i=0;i<2; ++i ) //Increase count to wake up this thread then go to sleep again
    Ogre::Threads::Sleep( 1000 );
performRestOfInit();

And see if that fleshes out the bug. Now that I think of it, you could do this in Ogre itself. Use your hacked version which is known to work, and add the sleep code to see if the bug appears.

Another possibility is setting GraphicsSystem::mAlwaysAskForConfig = false so that the config dialog is never created (if a valid ogre.cfg exists). Perhaps the bug/corruption is created by the config dialog itself.

Edit: Try commenting out the GL3+ and D3D11 plugins from plugins.cfg
I've hit an AMD bug on Windows (already fixed after I reported it) where the system would TDR if trying to profile using Radeon GPU profiler and all 3 plugins were loaded. Vulkan is not the only RenderSystem performing a "quick poke" to determine HW capabilities.

[Hotshot5000]

I modified the vk_raytracing_tutorial/ray_tracing_simple from nvpro-samples to create the vkInstance before the window creation. And it works there.

Code: Select all

// Requesting Vulkan extensions and layers
  nvvk::ContextCreateInfo contextInfo(true);
  contextInfo.setVersion(1, 2);
  contextInfo.addInstanceLayer("VK_LAYER_LUNARG_monitor", true);
  contextInfo.addInstanceExtension(VK_EXT_DEBUG_UTILS_EXTENSION_NAME, true);
  contextInfo.addInstanceExtension(VK_KHR_SURFACE_EXTENSION_NAME);
#ifdef WIN32
  contextInfo.addInstanceExtension(VK_KHR_WIN32_SURFACE_EXTENSION_NAME);
#else
  contextInfo.addInstanceExtension(VK_KHR_XLIB_SURFACE_EXTENSION_NAME);
  contextInfo.addInstanceExtension(VK_KHR_XCB_SURFACE_EXTENSION_NAME);
#endif
  contextInfo.addInstanceExtension(VK_KHR_GET_PHYSICAL_DEVICE_PROPERTIES_2_EXTENSION_NAME);
  contextInfo.addDeviceExtension(VK_KHR_SWAPCHAIN_EXTENSION_NAME);
  contextInfo.addDeviceExtension(VK_KHR_DEDICATED_ALLOCATION_EXTENSION_NAME);
  contextInfo.addDeviceExtension(VK_KHR_GET_MEMORY_REQUIREMENTS_2_EXTENSION_NAME);
  // #VKRay: Activate the ray tracing extension
  vk::PhysicalDeviceAccelerationStructureFeaturesKHR accelFeature;
  contextInfo.addDeviceExtension(VK_KHR_ACCELERATION_STRUCTURE_EXTENSION_NAME, false,
                                 &accelFeature);
  vk::PhysicalDeviceRayTracingPipelineFeaturesKHR rtPipelineFeature;
  contextInfo.addDeviceExtension(VK_KHR_RAY_TRACING_PIPELINE_EXTENSION_NAME, false,
                                 &rtPipelineFeature);
  contextInfo.addDeviceExtension(VK_KHR_MAINTENANCE3_EXTENSION_NAME);
  contextInfo.addDeviceExtension(VK_KHR_PIPELINE_LIBRARY_EXTENSION_NAME);
  contextInfo.addDeviceExtension(VK_KHR_DEFERRED_HOST_OPERATIONS_EXTENSION_NAME);
  contextInfo.addDeviceExtension(VK_KHR_BUFFER_DEVICE_ADDRESS_EXTENSION_NAME);

	// Creating Vulkan base application
  nvvk::Context vkctx{};
  vkctx.initInstance(contextInfo);

  // Setup GLFW window
  glfwSetErrorCallback(onErrorCallback);
  if(!glfwInit())
  {
    return 1;
  }
  glfwWindowHint(GLFW_CLIENT_API, GLFW_NO_API);
  GLFWwindow* window =
      glfwCreateWindow(SAMPLE_WIDTH, SAMPLE_HEIGHT, PROJECT_NAME, nullptr, nullptr);

  // Setup camera
  CameraManip.setWindowSize(SAMPLE_WIDTH, SAMPLE_HEIGHT);
  CameraManip.setLookat(nvmath::vec3f(5, 4, -4), nvmath::vec3f(0, 1, 0), nvmath::vec3f(0, 1, 0));

I've also modified mAlwaysAskForConfig to false, still get the same error.

EDIT: I've disabled the other render systems from plugins.cfg like you said and the problem went away. Now it's working. I don't even know what to make of this. Is it a driver bug? Is it a windows bug?

[Hotshot5000]

Anyway, I've found a solution.

Code: Select all

# Defines plugins to load

# Define plugin folder
PluginFolder=.

# Define plugins
 
 Plugin=RenderSystem_Direct3D11_d
 Plugin=RenderSystem_GL3Plus_d
 Plugin=RenderSystem_Vulkan_d
# Plugin=RenderSystem_GLES_d
# Plugin=RenderSystem_GLES2_d
# Plugin=RenderSystem_Metal_d
 Plugin=Plugin_ParticleFX_d

Just move the Vulkan render system to be the last one loaded from the active render systems. It was the first one previously and it seems it doesn't like when other APIs are called after calling something vk*().

So changing the CMake templates for plugins.cfg.in and plugins_d.cfg.in should do the trick.

Commit

[dark_sylinc]

Awesome!

Feel free to submit it as a PR; or I can make the commit myself

[Hotshot5000]

Done with the PR.

[dark_sylinc]

Thanks for your good work!

I tweeted about the issue, perhaps someone from NV sees it and picks it up.

[Hotshot5000]

My first idea for ray tracing in Ogre would be to have ray traced shadows. So everything is rendered normally with rasterization and then switch to the RT pipeline where the backface culling is disabled. Then cast rays from the camera and if it's not a miss then trace another ray to, say, the closest light source. If it hits anything in its path it means that that point is in shadow (ignoring translucent materials for now).

I started looking into PbsMaterials.compositor and see that there is a shadow node but this is using shadow maps and it is used before the actual rendering from user camera perspective to render to the shadow texture.

So, right now with my limited understanding of the compositor layer of Ogre, I am thinking of doing the following:

1. Render the scene normally. Disable any shadow maps.
2. Build/rebuild/update top and bottom level acceleration structures (tlas and blas). Frustum culling would generate weird results but I am not sure that adding all objects in the world in the AS is a good idea. For now I will add everything.
3. Using a new compositor pass that runs after the scene rendering but before post processing, set the RT pipeline, update descriptor sets etc. and trace the rays to check if a point is in shadow or not.

Would this be a good approach? I am trying to integrate this RT stuff with ideally zero disruption to anything outside the Vulkan render system code but I don't that would be possible.

[dark_sylinc]

Your approach requires 2 rays per pixel: 1 from camera to pixel + 1 from pixel to light source.

That is wasteul: You already know the pixel's location from the rasterization pass (see inPs.pos which is in view space).

You can:

1. Write the position to a GBuffer/MRT
2. Bind an UAV to the pixel shader to write this position (not recommended, it's pointless since a GBuffer is better suited here)
3. Launch the ray query from the pixel shader (I believe this is possible but I'm not perfectly familiar with RT APIs, I believe it was added later)

I'll ask a friend who's been playing with RT if he has anything to add.

2. Build/rebuild/update top and bottom level acceleration structures (tlas and blas). Frustum culling would generate weird results but I am not sure that adding all objects in the world in the AS is a good idea. For now I will add everything.

Our 'shadow mapping cameras' position the camera and setup a frustum that will encompass all the objects that may cast shadow on the scene (has no false negatives, does have false positives).

However these acceleration structures are optimized for static content and have O( log N ) traversal, which means it is preferred to add all static content (i.e. no frustum culling at all), and apply frustum culling only to dynamic content.

[Hotshot5000]

It looks like I need to use the ray generation shader stage and trace a ray out of a camera. It might not be feasible to reuse the gbuffer with positions from the rasterization stage:

Code: Select all

VUID-VkRayTracingPipelineCreateInfoKHR-stage-03425
If flags does not include VK_PIPELINE_CREATE_LIBRARY_BIT_KHR, the stage member of at least one element of pStages, including those implicitly added by pLibraryInfo, must be VK_SHADER_STAGE_RAYGEN_BIT_KHR

It also seems that there is no requirement that you must trace one ray through each pixel of the screen and also a ray has no thickness so you can launch it from the center of the pixel or from wherever you want. Which means the pixel position from the gbuffer might not be exactly the position that the ray will hit.

Code: Select all

Raygen is a shader responsible for generating rays and invoking traces (a traceNVX function in GLSL). Required.

So a traceNVX call is needed from the shader anyway.. I cannot just read the data from a texture

It probably has something to do with the acceleration structure. It needs the camera position to have a cluster of rays coming from the same direction or something...

[dark_sylinc]

It looks like I need to use the ray generation shader stage and trace a ray out of a camera.

That's wrong. Raytracing asks you for an origin, direction, tmin and tmax.

Origin can be the same for all (e.g. camera position) or sourced from another location, such as a GBuffer.

You can't 'sample' from the GBuffer, but you can use texelFetch and index it via gl_LaunchIDNVX.

It might not be feasible to reuse the gbuffer with positions from the rasterization stage:

Code: Select all

VUID-VkRayTracingPipelineCreateInfoKHR-stage-03425
If flags does not include VK_PIPELINE_CREATE_LIBRARY_BIT_KHR, the stage member of at least one element of pStages, including those implicitly added by pLibraryInfo, must be VK_SHADER_STAGE_RAYGEN_BIT_KHR

I fail to see how that error is related with being unable to use GBuffers

It also seems that there is no requirement that you must trace one ray through each pixel of the screen

That's right. RT is generic. Number of rays to shoot is arbitrary (could be one, could be a billion), and you need to provide data for each ray (origin, direction, tmin, tmax, etc)

In fact you can use RT for non-rendering applications, such as physics or AI.

Which means the pixel position from the gbuffer might not be exactly the position that the ray will hit.

Regular rasterization is defined by spec to be at the center of each pixel. However this is not true for MSAA, where the subpixel locations can either be GPU-defined or spec-defined.
The function TextureGpu::getSubsampleLocations was supposed to return these values but it was left mostly unimplemented.

So a traceNVX call is needed from the shader anyway.. I cannot just read the data from a texture

Yes, you can

[dark_sylinc]

It seems you're using this sample as a base, which shows how to render using raytracing completely (i.e. no rasterization).

But you can source data from a rasterization GBuffer stage:

Code: Select all

#version 450 core
#extension GL_NV_ray_tracing : require
layout(set = 0, binding = 0, rgba8) uniform image2D image;
layout(set = 0, binding = 1) uniform accelerationStructureNV as;
layout(set = 0, binding = 2) uniform texture2D positionData; // GBuffer
layout(location = 0) rayPayloadNV float payload;

void main()
{
   vec4 col = vec4(0, 0, 0, 1);

   vec3 origin = texelFetch( positionData, gl_LaunchIDNV.xy );
   vec3 dir = vec3(0.0, 0.0, -1.0); // light direction

   traceNV(as, 0, 0xff, 0, 1, 0, origin, 0.0, dir, 1000.0, 0);

   col.y = payload;

   imageStore(image, ivec2(gl_LaunchIDNV.xy), col);
}

Granted... if we want to render with a single forward pass, we run into a chicken and egg problem (we need the raster pass to get the positions for the shadows, we need the shadows to perform the raster pass) which can be solved by shooting two rays (one from camera to triangle, another from triangle to light)

My solution of using forward raster and only 1 rpp requires a depth pre-pass, then raytracing, then final shading pass.

[Hotshot5000]

http://cwyman.org/code/dxrTutors/tutors ... 05.md.html

Yes you are right. It looks like I can trace directly from gBuffer. It's easier to read the DXR tutorials to get a better understanding because the Vulkan ray tracing stuff documentation is abysmal.

EDIT: Totally forgot about the depth pre-pass.... I have a lot to learn. But that's why I like this

[Hotshot5000]

Then I will do the pre-pass first. When I have the xyz it means I have all I need to start the ray tracing directly from the point and don't bother with the rays from the camera.

At that point I have the shadows and can do the rasterized pass which renders the scene.

What would the difference be between doing the RT pass first then the rasterized pass or the other way around? Shouldn't I get the same results?

[dark_sylinc]

When I have the xyz it means I have all I need to start the ray tracing directly from the point and don't bother with the rays from the camera.

Yes. Actually just the depth buffer is enough. You can check Samples/2.0/Tutorials/Tutorial_ReconstructPosFromDepth sample which reconstructs XYZ from just depth (and knowing the projection matrix parameters). The sample includes URL links to an article explaining the math behind it.

It's simple, but if you find it gets in the way, it may be a good idea to first write the 3D positions to a big fat RGBA32_FLOAT MRT and once you got it working, implement reconstruct pos from depth optimization to get rid of that fat gbuffer.

Btw we already have pre-pass support. The SSR (Screen Space Reflections) sample uses it. There may be more samples using it.

Although it's not heavily maintained, and we write roughness and IIRC shadow mapping data too to a GBuffer in the pre pass, in order to spread VGPR shader occupancy between the two passes.

I a well intended idea but made the shader more complicated with dubious gains.
I can't remember well now what data did we generate in the prepass.

What would the difference be between doing the RT pass first then the rasterized pass or the other way around? Shouldn't I get the same results?

There's two ways:

Way I

1. Raster: Pre-pass to generate at least a depth buffer to get the XYZ positions
2. Raytrace from XYZ positions to light
3. Raster: Final forward pass that uses the shadow information generated by raytrace in step 2.

Way II

1. Raytrace from camera to polygon
2. Raytrace from polygon (XYZ positions) to light
3. Raster: Final forward pass that uses the shadow information generated by raytrace in step 2.

Both ways should generate similar results. I say similar because there may be numerical differences, and objects with depth writes off (and/or transparents) may cause different results depending on how it's being handled in Way II.
But yeah, they should be very close.

[Hotshot5000]

Before tracing the rays through the scene I need to build the TLAS and BLAS. I thought about adding this AS creation/update in MetalRenderSystem::_beginFrameOnce() but then I realised I would need the scene node list from SceneManager in order to get the active/visible Items which contain info on SubMeshes which contain the vertex data to use in BLAS and transforms for TLAS. Where can I get this mesh vertex data + instance data in the backend part of the renderer?
It doesn't look like calling anything SceneManager related at this point in the pipeline is a good idea.