Ogre3D + Optix (partially solved)

Discussion area about developing or extending OGRE, adding plugins for it or building applications on it. No newbie questions please, use the Help forum for that.
Post Reply
Posts: 6
Joined: Tue Jan 25, 2011 1:38 pm

Ogre3D + Optix (partially solved)

Post by DefiniteIntegral » Fri Jan 18, 2013 7:54 am


I am presently trying to integrate Nvidia Optix into Ogre3D as part of my application framework. Mostly so I can play around with raytracing, where the output can be rendered to any object via an Ogre::Material. The intention is that Optix renders directly into the hardware pixel buffer of the texture associated with the material, by using CUDA-OpenGL interop. Then the material can be attached to anything; overlays, billboards, geometry etc.

I am able to create the Optix context, create an optix rendering output buffer using the textures GL tex ID, load PTX programs, validate and compile etc, all without problems. However launching causes a crash in the OpenGL video driver, returning "Error code 3." If I had to guess probably this is something to do with how Ogre creates the hardware pixel buffer for the texture that isn't playing well with Optix.

So I am wondering if anyone has experience with Ogre3D + Optix/CUDA and might know about how to deal with buffers.

I am creating the Ogre::Texture like this:

Code: Select all

mpRenderSystemTexture = Ogre::TextureManager::getSingleton().createManual(	mRenderSystemTextureName, 
															LIBAPP_OGRE_RESOURCE_GROUP_NAME,			// use app base resource group
															Ogre::TEX_TYPE_2D,							// 2d texture
															mPanelSize.getX(),							// texture size = GUI panel size
															0,										// no mip mapping for interface texture
															Ogre::PF_FLOAT32_RGBA,						// FLOAT32 RGBA buffer format (Optix FLOAT4)
															// set as a render-target, since rendering to texture.

In addition to TU_RENDERTARGET, I have tried HardwareBuffer::HBU_DYNAMIC and HardwareBuffer::HBU_STATIC, all with the same result. When launching the Optix render, I have tried locking the pixel buffer with HardwareBuffer::HBL_NORMAL and HardwareBuffer::HBL_READ_ONLY (depending on the buffer usage mode at creation time), as well as not locking the pixel buffer at all. Since the Optix render writes to the buffer GPU side, I would have thought using a static read-only buffer would work since the CPU doesn't write to it at all.

The Ogre hardware pixel buffer is associated with Optix basically like this:

Code: Select all

unsigned int texGL_ID = static_cast<Ogre::GLTexturePtr>(mpRenderSystemTexture )->getGLID();
mOptixOutputBuffer = mOptixContext->createBufferFromGLBO(RT_BUFFER_OUTPUT, texGL_ID);   // using RT_BUFFER_OUTPUT as GPU is writing to the output buffer only
mOptixOutputBuffer->setFormat(RT_FORMAT_FLOAT4);  // RT_FORMAT_FLOAT4 to match Ogre::PF_FLOAT32_RGBA for texture format. this should automatically set element size to 128 bits.
mOptixOutputBuffer->setSize(mpRenderSystemTexture->getWidth(), mpRenderSystemTexture->getHeight());  // set buffer to same size as texture
This works without error. However upon launching the Optix render, the Nvidia OpenGL driver crashes and CUDA returns this awesomely descriptive error:

Code: Select all

Unknown error (Details: Function "_rtContextLaunch2D" caught exception: Encountered a CUDA error: Kernel launch returned (999): Unknown, [6619200])
I have been able to get Optix raytracing working with straight OpenGL easily by creating a HBO manually, but Ogre3D is.... proving to be a bit more of a challenge. Is there perhaps some way I can create an OpenGL HBO manually, then pass this into the HardwareBuffer used by Ogre::Texture?

At this point I am pretty stumped. I figure I need to know more about Ogre hardware buffers to get this working.

So if anyone has any ideas, please do tell.
Last edited by DefiniteIntegral on Mon Jan 21, 2013 3:26 am, edited 1 time in total.
0 x

Posts: 6
Joined: Tue Jan 25, 2011 1:38 pm

Re: Ogre3D + Optix integration headaches

Post by DefiniteIntegral » Sun Jan 20, 2013 1:03 pm

After looking at my old source code today I realized the crash was due to using a pixel buffer for rendering, which I had forgotten Optix does not support.

So I am able to get the Optix program to launch successfully by using a vertex buffer to store the Optix render output. But I have no idea how to display a vbo as a texture in Ogre3D. In OpenGL I just did something like this when rendering:

Code: Select all

glBindTexture(GL_TEXTURE_2D, texID);
glPixelStorei(GL_UNPACK_ALIGNMENT, 4);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA8, width, height, 0, GL_BGRA, GL_UNSIGNED_BYTE, 0);
Is there a way to do something similar in Ogre3D? Like create a texture or material to allow using vertex buffer to render, or a way to do a device->device buffer copy from vertex->pixel buffer? I am only using OpenGL so even if I need to manually create a buffer in OpenGL etc is not a problem.

I have tried buffer copy using HardwareBuffer::copyData() but that does not currently work as HardwarePixelBuffer::writeData() is not yet implemented. Also I have no idea if that is eventually intended to do a device->device copy or only for host<->device copies. The blit functions also are no good since they only seem to support pixel->pixel buffer copy, not vertex->pixel, also it looks like they only do host<->device copies and not device->device.

As a last resort I can manually do device->host->device buffer copy but that would be super slow and kind of absurd.

The only forum threads really I found on this were several years old and went nowhere. So if anyone knows of some good threads or a solution please let me know.
0 x

Posts: 6
Joined: Tue Jan 25, 2011 1:38 pm

Re: Ogre3D + Optix (partially solved)

Post by DefiniteIntegral » Mon Jan 21, 2013 8:47 am

Well I did manage to get Optix rendering in Ogre3D working. However, I had to do a vertex->pixel buffer copy using Ogre::HardwareVertexBuffer::readData() which is quite slow.

Actually it is barely any faster than the CPU-only version of the same rendering (where rendering done by CPU to a raw texture array on the heap then copied into Ogre::HardwarePixelBuffer).

Ideally I'd like to use something similar to the GL code in the post above, but it seems that would require modifying Ogre::TextureManager, Ogre::Texture, Ogre::GLTexture, etc.
0 x

Posts: 170
Joined: Tue Nov 25, 2008 10:58 am
Location: Kristiansand, Norway
x 10

Re: Ogre3D + Optix (partially solved)

Post by hedphelym » Mon Jan 21, 2013 8:56 am

I cannot help with your issue, but I got very curious when I saw your post.
I do not want to highjack your topic or anything, but
I'm wondering if you could share some more details on what you are doing?

I've worked to some degree with optix and the photon sample code,
and currently we're working on baking ambient occlusion in our ogre based engine, but when I saw
your post here it got me thinking - that it could be used to calculate and bake more advanced lighting too.
0 x

Posts: 6
Joined: Tue Jan 25, 2011 1:38 pm

Re: Ogre3D + Optix (partially solved)

Post by DefiniteIntegral » Mon Jan 21, 2013 9:27 am

Yeah I also had ideas about using Optix for lighting, but haven't looked into it yet. Even just generating high quality static lightmaps would be pretty sweet, since can use photon mapping, path tracing etc and do as many passes as you want to get best quality and still way faster and easier to do than on CPU.

I have only tried basic stuff so far (procedural geometry) so haven't looked into anything like binding the depth buffer, or vertex buffers from an Ogre entity etc for use with TriangleKdTree etc. So far I could only tell you how to get the render up and running.

I still have some things to work out, but when I am finished I'll try to write up a wiki article about it when I find the time.

For now I can tell you I'm doing basically this:

-Create Optix context. Set ray type count, entry point count, stack size.
-Create root group with NoAccel structure (then I attach accel groups with geometry to this)
-Create an Ogre::Texture using Ogre::PF_FLOAT32_RGBA format and Ogre::TU_DYNAMIC_WRITE_ONLY_DISCARDABLE usage mode. Associate this texture with a material pass.
-Create an Ogre::HardwareVertexBuffer in HBU_STATIC usage mode. Vertex size 16 for FLOAT4 of course. Store the full buffer length (width * height * 16).
-Use createBufferFromGLBO to create optix interop buffer with the vertex buffer GL ID. Method depends on RenderSystem, eg for OpenGL: vboID = static_cast<Ogre::GLHardwareVertexBuffer*>(vertexBufferPtr.get())->getGLBufferId();
-Set optix buffer format to RT_FORMAT_FLOAT4, set size, no need to set element size as automatically done
-Bind the optix buffer to the context with appropriate buffer name from your .cu files
-Load exception, raygen, miss programs.
-Create and associate any custom variables or buffers with the optix programs, eg light buffer etc. Need to do an upload once before compile to initialize a custom buffer.
-Validate and compile context.
-Do test context launch to make sure programs will run.

-Update custom buffers.
-Launch context
-Lock output texture pixel buffer
-Copy data from vertex buffer into pixel buffer using pixel box: vertexBuffer->readData(0, bufferLength, pixelBox.data);
-Unlock output texture pixel buffer

This method for Optix rendering in Ogre works but is quite slow because of the vertex->pixel buffer copy. I get about 130 FPS for a 512x512 texture on my rather average computer.

You can attach these materials to any Ogre renderable so it pretty useful, just need to find a way to make it faster.
0 x

Post Reply