DirectX 11 render system - work-in-progress

Discussion area about developing or extending OGRE, adding plugins for it or building applications on it. No newbie questions please, use the Help forum for that.
User avatar
Assaf Raman
OGRE Team Member
OGRE Team Member
Posts: 3092
Joined: Tue Apr 11, 2006 3:58 pm
Location: TLV, Israel
x 76

Post by Assaf Raman »

haffax wrote:Is Cg so different from SM 4.0 that it can't be compiled to it?
Yes, also hlsl that is under shader model 4.
One of the main points is the use of SV_POSITION instead of POSITION.
SV_POSITION, is a pre-defined semantics with special meaning. This semantics tells the graphics pipeline that the data associated with the semantics defines the clip-space position. This position is needed by the GPU in order to drawn pixels on the screen.
The help file says that:
For backwards compatibility, the semantic POSITION0 is also treated as SV_Position.
But in many shaders the name is POSITION and not POSITION0 and it is hard to replace, also I am not sure that the quote is correct.
Watch out for my OGRE related tweets here.
User avatar
Assaf Raman
OGRE Team Member
OGRE Team Member
Posts: 3092
Joined: Tue Apr 11, 2006 3:58 pm
Location: TLV, Israel
x 76

Post by Assaf Raman »

I found this nice ATI demo for DX9 called " A Fixed Function Shader in HLSL", you can download it from here: http://ati.amd.com/developer/samples/dx ... hader.html

When you run it you will notice that the real fixed func is faster then the programmable one, this might explain why DX9 runs better then DX10.

I will be able to use this code when creating the lighting part of the fixedFunc emulation.

I also found this code – " ShaderGen - Fixed Functionality Shader Generation Tool"
The purpose of this application is to show that programmable hardware can be used to obtain the same rendering results as the OpenGL fixed function path. This can be useful as a starting point if you want to use the OpenGL Shading Language to create an effect that is similar to what can be done with fixed functionality.
That can be downloaded from here:
http://developer.3dlabs.com/downloads/shadergen/
I will review the code and see if it can be used. The shaders of this tool are GLSL.
It has a good license - we can use the code as part as OGRE with no problem.

I wander if we should take a deeper step with the fixed func – add an option to the base render system – "useProgrammableFixFunc" – and have a new class named "fixedFuncManager" – that will create and hash the relevant shaders. This is the best option – then my work won't be restricted to DX10. Each render system will implement the relevant code in it's own language (HLSL3, GLSL, HLSL4).
Last edited by Assaf Raman on Mon Feb 18, 2008 3:24 pm, edited 1 time in total.
Watch out for my OGRE related tweets here.
User avatar
Wolfmanfx
OGRE Team Member
OGRE Team Member
Posts: 1525
Joined: Fri Feb 03, 2006 10:37 pm
Location: Austria - Leoben
x 99

Post by Wolfmanfx »

I think thats a nice option for the xbox360 there something like dx9+ i heard there is no fixfunction.
User avatar
Assaf Raman
OGRE Team Member
OGRE Team Member
Posts: 3092
Joined: Tue Apr 11, 2006 3:58 pm
Location: TLV, Israel
x 76

Post by Assaf Raman »

I am working today on the fixedFuncEmu. I am trying to create a general solution.
My end result will be a fixed function shader generator that will be able to create fixed function emulation shaders for glsl, hlsl and hlsl4. I decided that to write d3d10 only generator is almost the same work as for the 3. Lots of work – but in the end we will have something that can really help anyone that writes a shader, and can also be a base for larger ideas of a mixture of auto-generated shaders with user created shader code.
Watch out for my OGRE related tweets here.
User avatar
Assaf Raman
OGRE Team Member
OGRE Team Member
Posts: 3092
Joined: Tue Apr 11, 2006 3:58 pm
Location: TLV, Israel
x 76

Post by Assaf Raman »

I will be happy to get any new links for any existing fixedFunc shader that I didn't find.

I know about the following for now:
1. ATI (for DX 9)
2. 3dLabs (for GL)
3. DX10 sdk fixedFuncEmu sample (for DX10)
Watch out for my OGRE related tweets here.
DDd
Kobold
Posts: 30
Joined: Thu Feb 28, 2008 9:04 pm

Post by DDd »

Two questions:

Is the CVS version up to date?

From what i can gather reading this thread, now, you are working on a shader generator, to feed to all the different render systems. what will you be using as the source shader?

Looks like DX10 is a real pain in the buttocks, very restrictive, I'm downloading the latest code it should be interesting reading.
User avatar
Assaf Raman
OGRE Team Member
OGRE Team Member
Posts: 3092
Joined: Tue Apr 11, 2006 3:58 pm
Location: TLV, Israel
x 76

Post by Assaf Raman »

DDd wrote:Is the CVS version up to date?
Yes. (get the CVS head)
DDd wrote:what will you be using as the source shader?
I will generate the shader by a new fixedFunc state class that I have created and the vertex declaration (you can read in the thread that the vertex declaration in the shader must be the same as in the declaration in the vertex buffer). I haven't committed this general shader generator to the CVS yet and the current CVS code only have the DX10 specific generator as part of the DX10 vertex declaration.
Watch out for my OGRE related tweets here.
User avatar
Assaf Raman
OGRE Team Member
OGRE Team Member
Posts: 3092
Joined: Tue Apr 11, 2006 3:58 pm
Location: TLV, Israel
x 76

Post by Assaf Raman »

Seems that 3dlabs have removed their ShaderGen 3.0 page from some reason in the last few days after my post, I don't know if the two are connected.
Here is the source that I got when the site was up (ShaderGen-3.0-src-data.zip):
http://assafogre3dforumfiles2.googlepag ... .part1.rar
http://assafogre3dforumfiles2.googlepag ... .part2.rar
http://assafogre3dforumfiles2.googlepag ... .part3.rar
http://assafogre3dforumfiles2.googlepag ... .part4.rar

It has a free license so I guess that I can post the source here for any one that will want it.

Code: Select all

/************************************************************************
*                                                                       *
*               Copyright (C) 2002-2005  3Dlabs Inc. Ltd.               *
*                                                                       *
*                        All rights reserved.                           *
*                                                                       *
* Redistribution and use in source and binary forms, with or without    *
* modification, are permitted provided that the following conditions    *
* are met:                                                              *
*                                                                       *
*     Redistributions of source code must retain the above copyright    *
*     notice, this list of conditions and the following disclaimer.     *
*                                                                       *
*     Redistributions in binary form must reproduce the above           *
*     copyright notice, this list of conditions and the following       *
*     disclaimer in the documentation and/or other materials provided   *
*     with the distribution.                                            *
*                                                                       *
*     Neither the name of 3Dlabs Inc. Ltd. nor the names of its         *
*     contributors may be used to endorse or promote products derived   *
*     from this software without specific prior written permission.     *
*                                                                       *
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS   *
* "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT     *
* LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS     *
* FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE        *
* COPYRIGHT HOLDERS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, *
* INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,  *
* BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;      *
* LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER      *
* CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT    *
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN     *
* ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE       *
* POSSIBILITY OF SUCH DAMAGE.                                           *
*                                                                       *
************************************************************************/
Here is the missing page:
http://developer.3dlabs.com/downloads/s ... /index.htm
And here is its content from web.archive.org:
http://web.archive.org/web/200708290519 ... /index.htm

Still, it is hard to believe that someone saw my post and decided he didn't want to share the file anymore, I guess this is a coincidence.
Watch out for my OGRE related tweets here.
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19269
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66

Post by sinbad »

Interesting - I'd actually planned to skip emu and only allow shaders, I'd assumed that anyone wanting to run the Dx10 rendersystem would be using shaders all the time anyway. The only downside is that some of the builtin materials built in code would have to be altered to cope with shader-only rendersystems - that's why I added the RSC_FIXED_FUNCTION capability to RenderSystemCapabilities, so it could be picked up and dealt with.

I wasn't expecting to find a solution to complete fixed-function emulation without some really complex conditional compilation or something like that. But if you find an elegant way to do it, that's great too :)
User avatar
Assaf Raman
OGRE Team Member
OGRE Team Member
Posts: 3092
Joined: Tue Apr 11, 2006 3:58 pm
Location: TLV, Israel
x 76

Post by Assaf Raman »

@sinbad: I am at a stage where I have the infrastructure for a general fixed function shdader generator, currently I have it as a part of the DX10 render system, but I think that the generator is not DX10 specific and should be committed as part of the OGRE main project and not as part of the DX10 render system. Where should I commit? I will be waiting for your response before I commit.
Watch out for my OGRE related tweets here.
techguyx
Gnoblar
Posts: 3
Joined: Sat Mar 01, 2008 7:00 am

Post by techguyx »

Hmmm, definitely interesting to hear about the shader requirements from DX10. I hadn't known about that previously... I'm going to try to download / compile all the OGRE stuff as well as this project from the CVS, but I do have one question out of curiosity - you mentioned in the other thread that a primary goal is to get the OGRE demos and such all working under DX10.

This may be a stupid question - apologies if it is - but what exactly would you say that all entails as far as what has to be done once the shader issue is resolved? Is it just a matter of resolving case-by-case issues, or are there broader changes to be made?
User avatar
Assaf Raman
OGRE Team Member
OGRE Team Member
Posts: 3092
Joined: Tue Apr 11, 2006 3:58 pm
Location: TLV, Israel
x 76

Post by Assaf Raman »

techguyx wrote:what exactly would you say that all entails as far as what has to be done once the shader issue is resolved? Is it just a matter of resolving case-by-case issues, or are there broader changes to be made?
Well, the demos are only use samples; I try to solve the gap from the other render systems unrelated to the demos.
What I am working on now – the fixed function shader generator will enable most of the demos that don't use shaders to work and more over – it will enable ppl that aren't experienced with shaders to start to work with DX10 because the render system will create the simple shaders for them during runtime.
If I didn't answer your question please be more specific.
Watch out for my OGRE related tweets here.
User avatar
Assaf Raman
OGRE Team Member
OGRE Team Member
Posts: 3092
Joined: Tue Apr 11, 2006 3:58 pm
Location: TLV, Israel
x 76

Post by Assaf Raman »

Here the work I have done today, lots of changes, started to work on the fog and more. This is a backup until I will commit.
http://assafogre3dforumfiles2.googlepag ... backup.zip
Watch out for my OGRE related tweets here.
User avatar
Assaf Raman
OGRE Team Member
OGRE Team Member
Posts: 3092
Joined: Tue Apr 11, 2006 3:58 pm
Location: TLV, Israel
x 76

Post by Assaf Raman »

I just found this nice pdf on the topic "DirectX 10 Performance":
http://developer.download.nvidia.com/pr ... rmance.pdf

Worth a look.
Watch out for my OGRE related tweets here.
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19269
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66

Post by sinbad »

Assaf Raman wrote:@sinbad: I am at a stage where I have the infrastructure for a general fixed function shdader generator, currently I have it as a part of the DX10 render system, but I think that the generator is not DX10 specific and should be committed as part of the OGRE main project and not as part of the DX10 render system. Where should I commit? I will be waiting for your response before I commit.
Although it's in theory something that could be made more general, it's only actually of use (right now) in Dx10 since all the other rendersystems support fixed-function directly, so I would still place it in the Dx10 area for now. That way we don't have to start implementing it for GLSL or Cg where it's not strictly needed. Later on if we need it for something else (e.g. 360 port ;)) then we can always refactor it out.
User avatar
Assaf Raman
OGRE Team Member
OGRE Team Member
Posts: 3092
Joined: Tue Apr 11, 2006 3:58 pm
Location: TLV, Israel
x 76

Post by Assaf Raman »

Agreed.
Watch out for my OGRE related tweets here.
User avatar
Assaf Raman
OGRE Team Member
OGRE Team Member
Posts: 3092
Joined: Tue Apr 11, 2006 3:58 pm
Location: TLV, Israel
x 76

Post by Assaf Raman »

Created the basic fixedFuncEmu shader generator and also added basic fog.
Next I will add lights and full texture layer blend operation and matrix support then I will organize the generator code to functions.

The latest code is in the CVS.

Here is a screenshot of the fog:

Code: Select all

mSceneMgr->setFog(FOG_LINEAR, ColourValue::Blue,0.01,5,1000);
Image
Watch out for my OGRE related tweets here.
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19269
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66

Post by sinbad »

Fantastic work Assaf, you rock.
User avatar
Assaf Raman
OGRE Team Member
OGRE Team Member
Posts: 3092
Joined: Tue Apr 11, 2006 3:58 pm
Location: TLV, Israel
x 76

Post by Assaf Raman »

Started to work on the lights - now we have the basic infrastructure.
Also fixed some bugs in the new generator.
Latest code is in the CVS.

Image
Watch out for my OGRE related tweets here.
scroogie
Halfling
Posts: 45
Joined: Sat Mar 11, 2006 8:43 pm

Post by scroogie »

Its fantastic to see your progress documented like that. I don't have any concrete use for the DirectX10 support, but its very interesting to follow your discussions and see your progress. kudos to you!
User avatar
Assaf Raman
OGRE Team Member
OGRE Team Member
Posts: 3092
Joined: Tue Apr 11, 2006 3:58 pm
Location: TLV, Israel
x 76

Post by Assaf Raman »

@scroogie: Thanks! :D :D :D
Watch out for my OGRE related tweets here.
Vectrex
Ogre Magi
Posts: 1266
Joined: Tue Aug 12, 2003 1:53 am
Location: Melbourne, Australia
x 1

Post by Vectrex »

yeah agreed. It's like a little christmas present :)
User avatar
Assaf Raman
OGRE Team Member
OGRE Team Member
Posts: 3092
Joined: Tue Apr 11, 2006 3:58 pm
Location: TLV, Israel
x 76

Post by Assaf Raman »

Made same order in the code.
I see there is a problem with 1d texture.
Got the working in I change the Examples/LightRibbonTrail material to:

Code: Select all

material Examples/LightRibbonTrail
{
	technique
	{
		pass
		{
			lighting off
			scene_blend add
			depth_write off

			texture_unit
			{
				texture ribbonband.png
				//texture ribbonband.png 1d
				//tex_address_mode clamp
				//filtering none
			}
		}
	}
}
Latest code in the CVS.

here is a screenshot:
Image

Help needed:
It seems that I have a problem with the shader generator idea – whenever the light state changes (a new state of the lights that we didn't have before) we lose a few frames because a new fixedFunc shader compiles.
I have an idea how to solve this, but I want your ideas before I tell you mine – perhaps someone will think of a better idea.
Remember – I want to have the minimum shader commands – we want performance.
Watch out for my OGRE related tweets here.
User avatar
Assaf Raman
OGRE Team Member
OGRE Team Member
Posts: 3092
Joined: Tue Apr 11, 2006 3:58 pm
Location: TLV, Israel
x 76

Post by Assaf Raman »

Worked on the textures in the fixedFunc shader generator.
I am not done, but getting there.
Here is a screenshot of the envMapping demo:
Image
Watch out for my OGRE related tweets here.
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19269
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66

Post by sinbad »

Assaf Raman wrote: It seems that I have a problem with the shader generator idea – whenever the light state changes (a new state of the lights that we didn't have before) we lose a few frames because a new fixedFunc shader compiles.
I have an idea how to solve this, but I want your ideas before I tell you mine – perhaps someone will think of a better idea.
Remember – I want to have the minimum shader commands – we want performance.
Branching and iteration? You could use light code that doesn't care about light type, since params are initialised to known values when dealing with properties that are not relevant to given types. Since we know this is SM4 you can use branching to avoid code that's not needed.