DirectX 11 render system - work-in-progress

[Assaf Raman]

@sinbad: This will mean sending more paramters then I need in meny cases. I was thinking more in the line of saving and loading a state cache for your program, this way you will get the best results. Each time the program closes - it saves all the fixedFunc states that were in use, next time it will load and compile them at start.

[Assaf Raman]

Got the stencil working, I tested it with the Stencil Glow Demo.

Here is a screenshot:

The latest code is in the CVS head.

Here is the material file I used with the demo and DX10.

[Assaf Raman]

got dynamic texture sort of working:

As you can see - it seems I have a problem with the color.

Latest source in the CVS head.

[Assaf Raman]

I get really nice FPS - almost the same as DX9 (and DX9 runs without any shaders for the demos I check).
I don't know what to do next - clean the code or make other demos work or start on DX10 only subjects....

What do you think?

[sinbad]

Thanks for all your work on this.

[PolyVox]

Fantastic work! I'm particularly interested in making use of texture arrays for my project (to remove ugly texture atlas hacks) so when they are available I'll be pleased to test for you

[Assaf Raman]

I am going to work tomorrow on the render system, I was thinking of adding texture arrays support for PolyVox, doing more work on the shader generator, add support for cube maps and perhaps adding geometric shader support, anyone needs anything? - I can still change my plan.

[PolyVox]

Work on geometry shaders would be interesting and it seems it could have a significant impact on the design of the DirectX 10 render system. I breifly raised the issue here:

http://www.ogre3d.org/phpBB2/viewtopic.php?t=37197

Note that I'm no longer intending to work with them anytime soon (so I can't test them for you...) but it seems it would be worth adding some preliminary support so you know what effect it will have.

[Assaf Raman]

I worked today on the cube map support.
Almost got it.


It seems that I don't load the cube map correctly or something.
It seems that you can see only one pic of the 6.

[Zeux]

You know, this is sooo much open-source...

I recommend you stop working on D3D10 RenderSystem for a week or so, read through the entire D3D10 documentation section (excluding reference), read through all tutorials, read through all reference sections related to the code you're using.

Because it seems to me that you don't quite understand what is D3D10, what it can/can't do, what is implemented and what is not, and what different parts mean.

This is not to say that your work is not great - it's great simply because you put effort to it, I know how hard it is to work in your spare time. But the quality could have been better.

Two small examples.

Why do you use, er, R32_UINT as a color type in input layout? There are a lot of wonderful types out there. How about R8G8B8A8_UNORM?

The talk about Cg shaders up there shows that you don't know about the backward compatibility switch (/Gec in fxc.exe, D3D10_SHADER_ENABLE_BACKWARDS_COMPATIBILITY in D3D10CompileShader).

[Assaf Raman]

@Zeux: Well, all that I can say is that I think I know the DX10 documentation very well, how do you think I write this code? Regarding you question why to use of R32_UINT, R8G8B8A8_UNORM and the D3D10_SHADER_ENABLE_BACKWARDS_COMPATIBILITY – well – if you read this thread carefully you will see that I am well familiar with your ideas, tested them already and that they didn’t work out for me.

Guys – I will ask you not to attack Zeux, I think it will hijack this thread to directions that are not good.

Zeux – I will be happy for any patch you will submit that will improve the render system.

[Zeux]

My last post here, as it's pointless to take it any further.

If R8G8B8A8_UNORM does not work for you, you must be doing something wrong. There's absolutely no change in shader for R32G32B32A32_FLOAT -> R8G8B8A8_UNORM transition, so it's exactly the format you want for color.

Regarding backwards compatibility macro, well, any Cg/HLSL shader (most of them, anyway) can be compiled with this flag to vs_4_0/ps_4_0 profile, getting both working bytecode and reflection-related information.

As for patches - thanks, but I've got a job, some occasional one-time paying opportunities and two pet-projects to maintain and develop.

[Assaf Raman]

@Zeux: What is the point of your posts?

[DDd]

Sorry i just have to chime in...

You know, this is sooo much open-source...

Yeah it is! Such is the nature of open source projects and that is what is great about them.

People with different levels of experience and expertise contributing to the major good of a whole community. People come and go, and freely contribute their time and skills to a project.

Assaf is doing an amazing work (hard work, implementing a new rendering system is not a couple of lines long), however i have no doubt that improvements can be done upon his code, like all code, as we all get more experienced with the DX10 API and it's pitfalls, those that can improve the base code should improve it by submitting patches.

That is the meaning of the word open in open source, open to all contributions.

Let Assaf complete the support for DX10, or better yet help him! It is up to all of us to optimize the code and make it run smoother, but the way i see it Assaf is the lead developer in the DX10 API implementation. If you want to contribute, do so in a more constructive manner, by using code or pointing out implementation flaws using documentation.

[Assaf Raman]

I wanted to get the TEXCALC_PROJECTIVE_TEXTURE working.
I added

Code: Select all

texUnit->setProjectiveTexturing(true, mCamera); 

to one of the samples.

This is how it looks in DX9:



I sort of got it working, it looks good from if the camera is far but not if the camera is near, I guess I didn't write the shader right.
Here is how the same sample looks in DX10 now:



If someone can post a replay with an hlsl or cg program with an OGRE material to do this kind of projection - I will thank him\her.

[Assaf Raman]

Almost got is:

[CaseyB]

@Assaf Raman - Amazing work! This is really coming along! It's a good thing you said to ignore Zeux because it's all I can to not to point out the flaws in his logic!

[Assaf Raman]

OK, I did my best for now, I will love some help with this.
Here is how it looks now:

[Assaf Raman]

I made a big performance update - I worked on it all night with a performance tool and fixed issues.
Turns out that if I add a data member to Renderable - I will be able to get better performance because I can save the D3D10 state for each Renderable (ID3D10BlendState, ID3D10RasterizerState, ID3D10DepthStencilState, ID3D10SamplerState). There is still work to do on this – invalidate the data member if the fixed func state or the material changes.
This is the data member I need to add to Renderable look like this for now:

Code: Select all

		// this should be used only by a render system for internal use, this can't be an array
		mutable void * renderSystemData;

It also needs an init to 0 in the constructor and be deleted if it is not 0 in the destructor.

For now I have a #define named ADDED_A_DATA_MEMBER_TO_RENDERABLE, when the data member will be added – 1 value will enable the use of the new data member, until then it is 0 so you will be able to compile.

[Game_Ender]

Code: Select all

		// this should be used only by a render system for internal use, this can't be an array
		mutable void * renderSystemData;

It also needs an init to 0 in the constructor and be deleted if it is not 0 in the destructor.

This code is valid:

Code: Select all

char* ptr = 0;
delete ptr;

Its valid to "delete 0", so you don't need a check.

[syedhs]

Nice performance update.

Just a thought Assaf: if the new member data is added only for DX10, then what happen if people want to use Shoggoth without DX10? Do they have to turn off the ADDED_A_DATA_MEMBER_TO_RENDERABLE when compiling Ogre because it is a memory waste. And then the Ogre download package may have two different version, one is DX10 ready, the other is not. So it is quite complicated.

So what I was thinking is
1) No preprocessor definition like 'ADDED_A_DATA_MEMBER_TO_RENDERABLE'. Instead the member data will always be there whether you want DX10 or not. The memory usage will be very minimal since it is not used.

2) Self-contained DX10 member variable - but I have no idea on how it can be done. But the idea is the variable is placed within DX10 plugin, not in Ogremain.dll.

Just my 2 cents

[Berserker]

Nice work Assaf

A little suggestion anyway:

It also needs an init to 0 in the constructor and be deleted if it is not 0 in the destructor.

Delete a void pointer is not a good idea, consider this sample:

Code: Select all

class RenderableData
{
public:
   RenderableData() { }
   virtual ~RenderableData() { }
};

void foo()
{
   // This is what is done in the plugin
   RenderableData *data = new RenderableData();
   // This is what is done in the core
   delete reinterpret_cast<void *>(data);
}

Note that the destructor of RenderableData is never called casting to void *, what you need is a callback in the plugin that make the correct cast and call the delete operator, something like this:

Code: Select all

Renderable::~Renderable()
{
	if(renderSystemDataDeleteCallback)
		renderSystemDataDeleteCallback(renderSystemData)
}

A more "elegant" solution could be a shared_ptr with a custom deleter:

Code: Select all

// Plugin code

class RenderableData
{
public:
	RenderableData() { }
	virtual ~RenderableData() { }
};

void finalizeRenderData(byte *data)
{
	delete reinterpret_cast<RenderableData *>(data);
}

boost::shared_ptr<byte> createRenderData()
{
	return boost::shared_ptr<byte>(reinterpret_cast<byte *>(new RenderableData()), &finalizeRenderData);
}

// Core code

class Renderable
{
private:
   boost::shared_ptr<byte> mRenderSystemData;
}

No additional code is required in the destructor (just a get/set of mRenderSystemData obviously)

[sinbad]

Code: Select all

		// this should be used only by a render system for internal use, this can't be an array
		mutable void * renderSystemData;

It also needs an init to 0 in the constructor and be deleted if it is not 0 in the destructor.

For now I have a #define named ADDED_A_DATA_MEMBER_TO_RENDERABLE, when the data member will be added – 1 value will enable the use of the new data member, until then it is 0 so you will be able to compile.

I've promoted your CVS access so you can make changes like this yourself in future.

However, please don't use void pointers at all - Berserker has a point but even more so, in general they're nasty - our coding standards doc says not to use them. Use Ogre::Any instead, it's typesafe (use Ogre::any_cast<> to convert) and you can add things by value to it which means you don't have to worry about memory management (internally it's held as a pointer so it's not expensive) - although in this case if you want to share information across renderables you'd still want to be using a pointer. It can be a SharedPtr if you want to still have automatic memory management.

I had actually thought you would need to start adding things like this to Pass, for example keeping compiled state blocks which Dx10 likes. But I guess if you need the vertex data too then Renderable is the only place you can do it. I would just be concerned about making sure as much information is shared between renderables that use the same combination of data as possible, since there are a lot of Renderables and many of them shoudl actually use the same render system data (vertex formats, state blocks etc).

[Wolfmanfx]

Hoi,
I got problems to use the memory leak tracker together with the DX10 plugin.
The bug is in OgreD3D10Driver.cpp line 103

delete str: -> have to delete[] str; and the memory leak tracker will not assert.

[edit]
Is there a way to tell the DX 10 rendersystem not to except when assembly shaders not supported cause it may suck to remove shaders that are not needed but try to compile.

[Assaf Raman]

@syedhs+sinbad+Berserker: I know that void * is bad, this is one of the reasons I didn’t suggest a patch and post the post I did, I wasn’t sure of the best way to do this and wanted the community help with this. I mostly check-in the code as a backup so I won’t lose it if my computer dies – because I did do lots of work on performance.

I hope it is obvious I wasn’t going to leave this kind of a data member – one that hasn’t got a “get\setâ€