Ray tracing support for Ogre 2.2+
-
- OGRE Contributor
- Posts: 233
- Joined: Thu Oct 14, 2010 12:30 pm
- x 58
Ray tracing support for Ogre 2.2+
I will start reading about Vulkan ray tracing extension and in a few days maybe create a fork from Ogre repo and create a branch from the latest Ogre version that supports Vulkan. I think it was integrated in the mainline a while ago but I haven't kept up with what changed in the meantime.
I am not sure Ogre needs ray tracing or wants it but it's just a side project for me. If I come up with anything useful people might want to take a look, I don't know...
I have a 1660 Ti on my laptop so I don't actually have ray tracing hardware support but running the samples it seems to be working fine for simple stuff. Maybe at some point I will buy a 3070 or 3080 but the supply issue means I only get "sold out" messages everywhere.
-
- OGRE Team Member
- Posts: 5434
- Joined: Sat Jul 21, 2007 4:55 pm
- Location: Buenos Aires, Argentina
- x 1342
Re: Ray tracing support for Ogre 2.2+
Yeah, fork from latest since we've improved our Vulkan renderer a lot since then. It's on master branchmaybe create a fork from Ogre repo and create a branch from the latest Ogre version that supports Vulkan. I think it was integrated in the mainline a while ago but I haven't kept up with what changed in the meantime.
DittoMaybe at some point I will buy a 3070 or 3080 but the supply issue means I only get "sold out" messages everywhere
-
- OGRE Contributor
- Posts: 233
- Joined: Thu Oct 14, 2010 12:30 pm
- x 58
Re: Ray tracing support for Ogre 2.2+
Exception thrown at 0x00007FF89CE60F41 (nvoglv64.dll) in Sample_PbsMaterials.exe: 0xC0000005: Access violation reading location 0xFFFFFFFFFFFFFFFF.
at line 544 in VulkanWindow.cpp (VkResult result = vkQueuePresentKHR( mDevice->mPresentQueue, &present );)
This happens on both CustomRenderable and PbsMaterials sample (I haven't tested the others) when running on the nvidia graphics card (1660 Ti). On the integrated AMD graphics it works fine. I am running the latest nvidia drivers (461.09).
I have the latest build from master from OgreCave.
I will investigate what is going wrong here. I see no validation layer error and the objects passed as params seem ok.
EDIT: Looks like there is a bug in Ogre. I went back to the Ogre 2.2 vulkan branch and it is working there.
-
- OGRE Team Member
- Posts: 5434
- Joined: Sat Jul 21, 2007 4:55 pm
- Location: Buenos Aires, Argentina
- x 1342
Re: Ray tracing support for Ogre 2.2+
viewtopic.php?p=549502#p549502
and these
viewtopic.php?p=549518#p549518
But we could never truly understand what's going wrong and NVIDIA is literally the only vendor having trouble (works fine on Mesa AMD, Propieteary AMD, Mesa Intel, Propietary Intel, Mali, Adreno).
Since I don't have an NV card with me to repro the issue, I've been unable to fix or workaround this problem.
It could be a driver bug, it could be an Ogre bug.
Perhaps you get better luck? Sorry for pushing this one onto you, but you understand the code and can repro it.
If you're in a rush to get Raytracing you can undo the changes and it should start working again on NV (or use bisect to detect the exact commit, but I think it was one of those I pointed).
-
- OGRE Contributor
- Posts: 233
- Joined: Thu Oct 14, 2010 12:30 pm
- x 58
Re: Ray tracing support for Ogre 2.2+
-
- OGRE Contributor
- Posts: 233
- Joined: Thu Oct 14, 2010 12:30 pm
- x 58
Re: Ray tracing support for Ogre 2.2+
I will try to leave the creation of VkInstance before window creation in order to have the cards for enumeration in addConfig() but then I will destroy it and recreate it after the window is up and running. See if that works.
EDIT: Nope it doesn't work. I get the same error if I recreate mVkInstance. It looks like I need to initialize it after creating the window.... But then, how do I get the video card options in the startup dialog?
-
- OGRE Team Member
- Posts: 5434
- Joined: Sat Jul 21, 2007 4:55 pm
- Location: Buenos Aires, Argentina
- x 1342
Re: Ray tracing support for Ogre 2.2+
Because it doesn't make sense. We call OGRE_NEW VulkanWin32Window() then initialize mVkInstance (but due to config settings, mVkInstance should be initialized by now).
However VulkanWin32Window's constructor does pretty much nothing.
After creating the VkInstance (if not already created), we create the VkDevice and then call win->_initialize which actually creates the window.
Based on what you said, my best guess is that we perform a buffer overflow enumerating settings for the config dialog, or leave the API in an inconsistent state, but Valgrind says nothing on Linux; thus maybe if I see the sequence that works for you (even if hacked) may shed some light
-
- OGRE Contributor
- Posts: 233
- Joined: Thu Oct 14, 2010 12:30 pm
- x 58
Re: Ray tracing support for Ogre 2.2+
Code: Select all
diff --git a/RenderSystems/Vulkan/src/OgreVulkanSupport.cpp b/RenderSystems/Vulkan/src/OgreVulkanSupport.cpp
index be468a291..29f2aaa8b 100644
--- a/RenderSystems/Vulkan/src/OgreVulkanSupport.cpp
+++ b/RenderSystems/Vulkan/src/OgreVulkanSupport.cpp
@@ -85,8 +85,9 @@ namespace Ogre
if( renderSystem->getVkInstance() )
return;
- renderSystem->initializeVkInstance();
- enumerateDevices( renderSystem );
+ // renderSystem->initializeVkInstance();
+ // enumerateDevices( renderSystem );
+ // renderSystem->deinitializeVkInstance();
}
//-------------------------------------------------------------------------
void VulkanSupport::addConfig( VulkanRenderSystem *renderSystem )
@@ -97,6 +98,9 @@ namespace Ogre
optDevices.name = "Device";
+ mDevices.clear();
+ mDevices.push_back( "GeForce GTX 1660 Ti #0" );
+
FastArray<String>::const_iterator itor = mDevices.begin();
FastArray<String>::const_iterator endt = mDevices.end();
Code: Select all
diff --git a/RenderSystems/Vulkan/src/OgreVulkanRenderSystem.cpp b/RenderSystems/Vulkan/src/OgreVulkanRenderSystem.cpp
index a475c6c7e..145c91f2c 100644
--- a/RenderSystems/Vulkan/src/OgreVulkanRenderSystem.cpp
+++ b/RenderSystems/Vulkan/src/OgreVulkanRenderSystem.cpp
@@ -228,11 +228,7 @@ namespace Ogre
mDebugReportCallback = 0;
}
- if( mVkInstance )
- {
- vkDestroyInstance( mVkInstance, 0 );
- mVkInstance = 0;
- }
+ deinitializeVkInstance();
}
//-------------------------------------------------------------------------
void VulkanRenderSystem::shutdown( void )
@@ -823,6 +819,15 @@ namespace Ogre
#endif
}
//-------------------------------------------------------------------------
+ void VulkanRenderSystem::deinitializeVkInstance( void )
+ {
+ if( mVkInstance )
+ {
+ vkDestroyInstance( mVkInstance, 0 );
+ mVkInstance = 0;
+ }
+ }
+ //-------------------------------------------------------------------------
-
- OGRE Team Member
- Posts: 5434
- Joined: Sat Jul 21, 2007 4:55 pm
- Location: Buenos Aires, Argentina
- x 1342
Re: Ray tracing support for Ogre 2.2+
You commented the call to initializeVkInstance. But this is called later and the order of operations is pretty much the same.
However you also commented out enumerateDevices, which is never called again. What happens if you only comment out enumerateDevices and leave initializeVkInstance? (in VulkanSupport::initialize)
Edit: How many and which devices are listed? Is your NV card the only one? Or is there an integrated GPU too?
Edit 2: What is the return value of VulkanSupport::getSelectedDeviceIdx when VkInstance is initialized early, and what is the returned value in the hacked version?
-
- OGRE Contributor
- Posts: 233
- Joined: Thu Oct 14, 2010 12:30 pm
- x 58
Re: Ray tracing support for Ogre 2.2+
dark_sylinc wrote: ↑Tue Jan 19, 2021 7:24 pm Ok you posted something very interesting:
You commented the call to initializeVkInstance. But this is called later and the order of operations is pretty much the same.
However you also commented out enumerateDevices, which is never called again. What happens if you only comment out enumerateDevices and leave initializeVkInstance? (in VulkanSupport::initialize)
Edit: How many and which devices are listed? Is your NV card the only one? Or is there an integrated GPU too?
Code: Select all
renderSystem->initializeVkInstance();
// enumerateDevices( renderSystem );
// renderSystem->deinitializeVkInstance();
-
- OGRE Team Member
- Posts: 5434
- Joined: Sat Jul 21, 2007 4:55 pm
- Location: Buenos Aires, Argentina
- x 1342
Re: Ray tracing support for Ogre 2.2+
If you run out of ideas, I would try to repro in a simple Vulkan application.
More particularly I could suspect of a race condition (i.e. most apps create the instance and almost immediately afterwards the window).
Thus if that theory is correct, a sample which does:
Code: Select all
createVkInstance();
for( size_t i=0;i<2; ++i ) //Increase count to wake up this thread then go to sleep again
Ogre::Threads::Sleep( 1000 );
performRestOfInit();
Another possibility is setting GraphicsSystem::mAlwaysAskForConfig = false so that the config dialog is never created (if a valid ogre.cfg exists). Perhaps the bug/corruption is created by the config dialog itself.
Edit: Try commenting out the GL3+ and D3D11 plugins from plugins.cfg
I've hit an AMD bug on Windows (already fixed after I reported it) where the system would TDR if trying to profile using Radeon GPU profiler and all 3 plugins were loaded. Vulkan is not the only RenderSystem performing a "quick poke" to determine HW capabilities.
-
- OGRE Contributor
- Posts: 233
- Joined: Thu Oct 14, 2010 12:30 pm
- x 58
Re: Ray tracing support for Ogre 2.2+
Code: Select all
// Requesting Vulkan extensions and layers
nvvk::ContextCreateInfo contextInfo(true);
contextInfo.setVersion(1, 2);
contextInfo.addInstanceLayer("VK_LAYER_LUNARG_monitor", true);
contextInfo.addInstanceExtension(VK_EXT_DEBUG_UTILS_EXTENSION_NAME, true);
contextInfo.addInstanceExtension(VK_KHR_SURFACE_EXTENSION_NAME);
#ifdef WIN32
contextInfo.addInstanceExtension(VK_KHR_WIN32_SURFACE_EXTENSION_NAME);
#else
contextInfo.addInstanceExtension(VK_KHR_XLIB_SURFACE_EXTENSION_NAME);
contextInfo.addInstanceExtension(VK_KHR_XCB_SURFACE_EXTENSION_NAME);
#endif
contextInfo.addInstanceExtension(VK_KHR_GET_PHYSICAL_DEVICE_PROPERTIES_2_EXTENSION_NAME);
contextInfo.addDeviceExtension(VK_KHR_SWAPCHAIN_EXTENSION_NAME);
contextInfo.addDeviceExtension(VK_KHR_DEDICATED_ALLOCATION_EXTENSION_NAME);
contextInfo.addDeviceExtension(VK_KHR_GET_MEMORY_REQUIREMENTS_2_EXTENSION_NAME);
// #VKRay: Activate the ray tracing extension
vk::PhysicalDeviceAccelerationStructureFeaturesKHR accelFeature;
contextInfo.addDeviceExtension(VK_KHR_ACCELERATION_STRUCTURE_EXTENSION_NAME, false,
&accelFeature);
vk::PhysicalDeviceRayTracingPipelineFeaturesKHR rtPipelineFeature;
contextInfo.addDeviceExtension(VK_KHR_RAY_TRACING_PIPELINE_EXTENSION_NAME, false,
&rtPipelineFeature);
contextInfo.addDeviceExtension(VK_KHR_MAINTENANCE3_EXTENSION_NAME);
contextInfo.addDeviceExtension(VK_KHR_PIPELINE_LIBRARY_EXTENSION_NAME);
contextInfo.addDeviceExtension(VK_KHR_DEFERRED_HOST_OPERATIONS_EXTENSION_NAME);
contextInfo.addDeviceExtension(VK_KHR_BUFFER_DEVICE_ADDRESS_EXTENSION_NAME);
// Creating Vulkan base application
nvvk::Context vkctx{};
vkctx.initInstance(contextInfo);
// Setup GLFW window
glfwSetErrorCallback(onErrorCallback);
if(!glfwInit())
{
return 1;
}
glfwWindowHint(GLFW_CLIENT_API, GLFW_NO_API);
GLFWwindow* window =
glfwCreateWindow(SAMPLE_WIDTH, SAMPLE_HEIGHT, PROJECT_NAME, nullptr, nullptr);
// Setup camera
CameraManip.setWindowSize(SAMPLE_WIDTH, SAMPLE_HEIGHT);
CameraManip.setLookat(nvmath::vec3f(5, 4, -4), nvmath::vec3f(0, 1, 0), nvmath::vec3f(0, 1, 0));
EDIT: I've disabled the other render systems from plugins.cfg like you said and the problem went away. Now it's working. I don't even know what to make of this. Is it a driver bug? Is it a windows bug?
-
- OGRE Contributor
- Posts: 233
- Joined: Thu Oct 14, 2010 12:30 pm
- x 58
Re: Ray tracing support for Ogre 2.2+
Code: Select all
# Defines plugins to load
# Define plugin folder
PluginFolder=.
# Define plugins
Plugin=RenderSystem_Direct3D11_d
Plugin=RenderSystem_GL3Plus_d
Plugin=RenderSystem_Vulkan_d
# Plugin=RenderSystem_GLES_d
# Plugin=RenderSystem_GLES2_d
# Plugin=RenderSystem_Metal_d
Plugin=Plugin_ParticleFX_d
So changing the CMake templates for plugins.cfg.in and plugins_d.cfg.in should do the trick.
Commit
-
- OGRE Team Member
- Posts: 5434
- Joined: Sat Jul 21, 2007 4:55 pm
- Location: Buenos Aires, Argentina
- x 1342
Re: Ray tracing support for Ogre 2.2+
Feel free to submit it as a PR; or I can make the commit myself
-
- OGRE Contributor
- Posts: 233
- Joined: Thu Oct 14, 2010 12:30 pm
- x 58
Re: Ray tracing support for Ogre 2.2+
-
- OGRE Team Member
- Posts: 5434
- Joined: Sat Jul 21, 2007 4:55 pm
- Location: Buenos Aires, Argentina
- x 1342
Re: Ray tracing support for Ogre 2.2+
I tweeted about the issue, perhaps someone from NV sees it and picks it up.
-
- OGRE Contributor
- Posts: 233
- Joined: Thu Oct 14, 2010 12:30 pm
- x 58
Re: Ray tracing support for Ogre 2.2+
I started looking into PbsMaterials.compositor and see that there is a shadow node but this is using shadow maps and it is used before the actual rendering from user camera perspective to render to the shadow texture.
So, right now with my limited understanding of the compositor layer of Ogre, I am thinking of doing the following:
1. Render the scene normally. Disable any shadow maps.
2. Build/rebuild/update top and bottom level acceleration structures (tlas and blas). Frustum culling would generate weird results but I am not sure that adding all objects in the world in the AS is a good idea. For now I will add everything.
3. Using a new compositor pass that runs after the scene rendering but before post processing, set the RT pipeline, update descriptor sets etc. and trace the rays to check if a point is in shadow or not.
Would this be a good approach? I am trying to integrate this RT stuff with ideally zero disruption to anything outside the Vulkan render system code but I don't that would be possible.
-
- OGRE Team Member
- Posts: 5434
- Joined: Sat Jul 21, 2007 4:55 pm
- Location: Buenos Aires, Argentina
- x 1342
Re: Ray tracing support for Ogre 2.2+
That is wasteul: You already know the pixel's location from the rasterization pass (see inPs.pos which is in view space).
You can:
- Write the position to a GBuffer/MRT
- Bind an UAV to the pixel shader to write this position (not recommended, it's pointless since a GBuffer is better suited here)
- Launch the ray query from the pixel shader (I believe this is possible but I'm not perfectly familiar with RT APIs, I believe it was added later)
Our 'shadow mapping cameras' position the camera and setup a frustum that will encompass all the objects that may cast shadow on the scene (has no false negatives, does have false positives).2. Build/rebuild/update top and bottom level acceleration structures (tlas and blas). Frustum culling would generate weird results but I am not sure that adding all objects in the world in the AS is a good idea. For now I will add everything.
However these acceleration structures are optimized for static content and have O( log N ) traversal, which means it is preferred to add all static content (i.e. no frustum culling at all), and apply frustum culling only to dynamic content.
-
- OGRE Contributor
- Posts: 233
- Joined: Thu Oct 14, 2010 12:30 pm
- x 58
Re: Ray tracing support for Ogre 2.2+
Code: Select all
VUID-VkRayTracingPipelineCreateInfoKHR-stage-03425
If flags does not include VK_PIPELINE_CREATE_LIBRARY_BIT_KHR, the stage member of at least one element of pStages, including those implicitly added by pLibraryInfo, must be VK_SHADER_STAGE_RAYGEN_BIT_KHR
Code: Select all
Raygen is a shader responsible for generating rays and invoking traces (a traceNVX function in GLSL). Required.
It probably has something to do with the acceleration structure. It needs the camera position to have a cluster of rays coming from the same direction or something...
-
- OGRE Team Member
- Posts: 5434
- Joined: Sat Jul 21, 2007 4:55 pm
- Location: Buenos Aires, Argentina
- x 1342
Re: Ray tracing support for Ogre 2.2+
That's wrong. Raytracing asks you for an origin, direction, tmin and tmax.Hotshot5000 wrote: ↑Wed Jan 27, 2021 8:01 pm It looks like I need to use the ray generation shader stage and trace a ray out of a camera.
Origin can be the same for all (e.g. camera position) or sourced from another location, such as a GBuffer.
You can't 'sample' from the GBuffer, but you can use texelFetch and index it via gl_LaunchIDNVX.
I fail to see how that error is related with being unable to use GBuffersIt might not be feasible to reuse the gbuffer with positions from the rasterization stage:
Code: Select all
VUID-VkRayTracingPipelineCreateInfoKHR-stage-03425 If flags does not include VK_PIPELINE_CREATE_LIBRARY_BIT_KHR, the stage member of at least one element of pStages, including those implicitly added by pLibraryInfo, must be VK_SHADER_STAGE_RAYGEN_BIT_KHR
That's right. RT is generic. Number of rays to shoot is arbitrary (could be one, could be a billion), and you need to provide data for each ray (origin, direction, tmin, tmax, etc)It also seems that there is no requirement that you must trace one ray through each pixel of the screen
In fact you can use RT for non-rendering applications, such as physics or AI.
Regular rasterization is defined by spec to be at the center of each pixel. However this is not true for MSAA, where the subpixel locations can either be GPU-defined or spec-defined.Which means the pixel position from the gbuffer might not be exactly the position that the ray will hit.
The function TextureGpu::getSubsampleLocations was supposed to return these values but it was left mostly unimplemented.
Yes, you canSo a traceNVX call is needed from the shader anyway.. I cannot just read the data from a texture
-
- OGRE Team Member
- Posts: 5434
- Joined: Sat Jul 21, 2007 4:55 pm
- Location: Buenos Aires, Argentina
- x 1342
Re: Ray tracing support for Ogre 2.2+
But you can source data from a rasterization GBuffer stage:
Code: Select all
#version 450 core
#extension GL_NV_ray_tracing : require
layout(set = 0, binding = 0, rgba8) uniform image2D image;
layout(set = 0, binding = 1) uniform accelerationStructureNV as;
layout(set = 0, binding = 2) uniform texture2D positionData; // GBuffer
layout(location = 0) rayPayloadNV float payload;
void main()
{
vec4 col = vec4(0, 0, 0, 1);
vec3 origin = texelFetch( positionData, gl_LaunchIDNV.xy );
vec3 dir = vec3(0.0, 0.0, -1.0); // light direction
traceNV(as, 0, 0xff, 0, 1, 0, origin, 0.0, dir, 1000.0, 0);
col.y = payload;
imageStore(image, ivec2(gl_LaunchIDNV.xy), col);
}
My solution of using forward raster and only 1 rpp requires a depth pre-pass, then raytracing, then final shading pass.
-
- OGRE Contributor
- Posts: 233
- Joined: Thu Oct 14, 2010 12:30 pm
- x 58
Re: Ray tracing support for Ogre 2.2+
Yes you are right. It looks like I can trace directly from gBuffer. It's easier to read the DXR tutorials to get a better understanding because the Vulkan ray tracing stuff documentation is abysmal.
EDIT: Totally forgot about the depth pre-pass.... I have a lot to learn. But that's why I like this
-
- OGRE Contributor
- Posts: 233
- Joined: Thu Oct 14, 2010 12:30 pm
- x 58
Re: Ray tracing support for Ogre 2.2+
At that point I have the shadows and can do the rasterized pass which renders the scene.
What would the difference be between doing the RT pass first then the rasterized pass or the other way around? Shouldn't I get the same results?
-
- OGRE Team Member
- Posts: 5434
- Joined: Sat Jul 21, 2007 4:55 pm
- Location: Buenos Aires, Argentina
- x 1342
Re: Ray tracing support for Ogre 2.2+
Yes. Actually just the depth buffer is enough. You can check Samples/2.0/Tutorials/Tutorial_ReconstructPosFromDepth sample which reconstructs XYZ from just depth (and knowing the projection matrix parameters). The sample includes URL links to an article explaining the math behind it.When I have the xyz it means I have all I need to start the ray tracing directly from the point and don't bother with the rays from the camera.
It's simple, but if you find it gets in the way, it may be a good idea to first write the 3D positions to a big fat RGBA32_FLOAT MRT and once you got it working, implement reconstruct pos from depth optimization to get rid of that fat gbuffer.
Btw we already have pre-pass support. The SSR (Screen Space Reflections) sample uses it. There may be more samples using it.
Although it's not heavily maintained, and we write roughness and IIRC shadow mapping data too to a GBuffer in the pre pass, in order to spread VGPR shader occupancy between the two passes.
I a well intended idea but made the shader more complicated with dubious gains.
I can't remember well now what data did we generate in the prepass.
There's two ways:What would the difference be between doing the RT pass first then the rasterized pass or the other way around? Shouldn't I get the same results?
Way I
- Raster: Pre-pass to generate at least a depth buffer to get the XYZ positions
- Raytrace from XYZ positions to light
- Raster: Final forward pass that uses the shadow information generated by raytrace in step 2.
- Raytrace from camera to polygon
- Raytrace from polygon (XYZ positions) to light
- Raster: Final forward pass that uses the shadow information generated by raytrace in step 2.
But yeah, they should be very close.
-
- OGRE Contributor
- Posts: 233
- Joined: Thu Oct 14, 2010 12:30 pm
- x 58
Re: Ray tracing support for Ogre 2.2+
Before tracing the rays through the scene I need to build the TLAS and BLAS. I thought about adding this AS creation/update in MetalRenderSystem::_beginFrameOnce() but then I realised I would need the scene node list from SceneManager in order to get the active/visible Items which contain info on SubMeshes which contain the vertex data to use in BLAS and transforms for TLAS. Where can I get this mesh vertex data + instance data in the backend part of the renderer?
It doesn't look like calling anything SceneManager related at this point in the pipeline is a good idea.