[Accepted] Graphics Card Emulation

[juozasg]

I'm looking for any questions, criticism and suggestions on my soon-to-be-submitted project proposal for GSoC 2007.


Abstract:

Most 3D applications are expected to work on a large variety of machines with widely different hardware capabilities. OGRE is designed with this expectation in mind and includes a simple, but effective mechanism for automatic fallback from advanced to simple graphics techniques (depending on the capabilities of user's graphics card). Although this mechanism is a gift for the end users, it makes things very difficult for the developers of OGRE applications. That is because testing both the advanced and simple techniques at the same time is very hard - there are two options for doing it: modifying the code/material definitions to trigger the use of each "level" of techniques and running the same program on several different machines. The second option may not be available to many users of OGRE, while the first option is time consuming and error prone. A better solution is to add a new option to OGRE that lets developers force the GPU fallback mechanism by emulating the specs of less advanced GPUs. This proposal suggests how to implement this feature in three months. The proposal is split into four parts: interface, which defines how this feature can be used in OGRE applications, implementation, which suggests how this feature could be programmed, a preliminary test plan and the timeline/deliverables for implementing this feature.


Interface:

Usage:
Asking OGRE to pretend it is running on different hardware should be no more difficult than asking it to use a different render system. Once the GPU database is complete, the developer should be able to call write 'renderSystem->emulateCapabilities("GeForce3")' and see what his/her program will look like on GeForce3 video card. Furthermore, if the developer relies on the Root object to call create the default window, setting the "EmulateCapabilities" field in ogre.cfg will have the same effect.

Database:
GPU capabilities will be stored in .rendercaps files. Each capability will be defined as in this example:

render_system_capability GeForceSuperMaxExtremeG77
{
pixel_syntax "ps_1_1"
vertex_syntax "vs_3_0"
blending "true"
...
}

In the standard distribution of Ogre, .rendercaps files will be stored in TestAssets.zip pack. Each .rendercaps file in the pack will contains render_system_capability entries for all cards based on a single chipset.


Implementation:

Instead of suggested emulateLesserHardware method that takes RenderSystemCapabilities and a list of shader syntaxes I believe it would be more consistent to abstract the whole video card in a RenderSystemCapabilities object, including the supported shader syntaxes. Once that is done, emulating hardware becomes the question of substituting one RenderSystemCapabilities for another. Serializing RenderSystemCapabilities then makes it very simple to create a database of video cards. The following classes and methods will have to be modified to support this approach.

RenderSystemCapabilities updates:
RenderSystemCapabilities will be extended include information about supported shader syntaxes. Serialization support and isSubset() method will be added to RenderSystemCapabilities. To account for differences in various shader versioning schemes a isSyntaxSupported(String syntax) method will be added to RSC class.

RenderSystem updates:
The initCapabalities (D3D9) and initGL (OpenGL) methods will be refactored into the following: queryRealCapabilities() and loadCapabilities(RenderSystemCapabilities caps). The first method will create a RenderSystemCapabilities object specifying the highest possible capabilities. The second method will take a RenderSystemCapabilities object, which may or may not be the best possible. The invoking code will use isSubset method to determine if requested capabilities can be emulated on user's hardware. New member attributes mRealCapabilities and mCurrrentCapabilities will be added to RenderSystem to distinguish between the two.

emulateCapabilities(String capabilitiesProfile) method will be added to the RenderSystem class. To preserve the current interface (and not break existing Ogre code) this method will only set several flags that will come into action the first time RenderSystem::createWindow is called. To make emulating hardware as simple as changing the ogre.cfg file, a new option will be added to RenderSystem::setConfigOption: "EmulateCapabilities. If there is time, emulateCapabilities() method will be made dynamic, allowing the developer to reset capabilities during run-time and not only during initialization.

Serialization:
A new subclass of ScriptLoader, RenderSystemCapabilitiesLoader will be added and used by RenderSystem during initialization to load create requested RenderSystemCapabilities objects.

Building the database:
A small utility called "rcapdump" (dump render capabilities) will be created. rcapdump will start Ogre, query the default RenderSystemCapabilities and will serialize the capabilities to a .rendercaps file. With the help of community and rcapdump we can quickly build a database of video cards.


Testing:

A new test suite will be created. It will be called EmulateCapabilitiesTests. It will test two items: that emulating a higher end video card throws an exception and that emulated video card has all the features specified in .rendercaps file. I will consult with Ogre team and my mentor to decide the exact scope of these test. Although at the minimum the second test will query the RenderSystem for it's capabilities and compare it to expected capabilities from the test .rendercaps file. Each of the two tests will be repeated with both emulateCapabilities() method call and "EmulateCapabilities" option in ogre.cfg.


Deliverables and Schedule:

While working on this project I would like to meet 3 times a week with my mentor for a brief Scrum-like discussion. Every Sunday I will update a wiki page describing my progress. Every other week I would like to meet with my Mentor for a Sprint-like evaluation and readjustment of plans. I will keep a development blog if there is enough demand.

4/12 - 5/28
Discuss proposed changes to RenderSystem (D3D9 and GL) class with mentor.
Study OverlayManager and other script loaders.
Learn about existing Ogre testing.
Develop a reasonable test plan with help of experience Ogre testers.
Learn how difficult it would be to permit dynamic (run-time) adjustment of capabilities.

5/28 - 6/11
Add support for shader information to RenderSystemCapabilities. Implement isSyntaxSupported() and translation from OpenGL to DirectX shader syntax versioning. Implement isSubset method. Write the test script for isSubset.
Deliverable: a test script for isSubset that checks shader versioning and passes.

6/11 - 6/25
Add new methods to RenderSystem interface. Refactor D3D9RenderSystem and GLRenderSystem to use queryRealCapabilities() and loadCapabilities() instead of their current disparate init methods. Add emulateCapabilities method and config option.
Deliverable: demonstration that emulateCapabilities method creates a dummy RenderSystemCapabilities object and that RenderSystem uses it during initialization.

6/25 - 7/9
Create a RenderSystemCapabilitiesLoader and implement loading of RenderSystemCapabilities from files. Update RenderSystem::emulateCapabilities to use the loader.
Deliverable: demonstration that setting the EmulateCapabilities option in ogre.cfg results in a render system being initialized with emulated capabilities specified in a file.

7/9 - 7/16
Buffer time for completing the previous goals

7/16 - 7/30
Work on rcapdump and figure out how to involve the community in building the database. Implement RenderSystemCapabilities dump to file. Create cross-platform build scripts for rcapdump.
Deliverable: rcadump is finished and ready for distribution.

7/30 - 8/13
Test scripting. Will be defined in more detail later.
Deliverable: No less than 4 scripts covering isSubset and functionality of GPU emulation via EmulateCapabilities config option and emulateCapabilities method call.

8/13 - 8/20
Buffer time for previous tasks and other ideas

Additional possibilities:
Expand configDialog to include a selection from all available EmulateCapabilities options.

Questions:
Do students get commit access to Ogre SVN? If that's not the case, I will use my own SVN repository. Depending on your answer I might need to schedule some time for integrating my changes.

Bit about me:
I'm an undergraduate student at Rochester Institute of Technology. I have finished all my classes; however I can not graduate until I complete a certain amount of internship work. This summer would count as my last internship block. Until the summer (middle of May to be exact) I'm working as a test developer intern for an internationally reviled software company while residing in Denmark. I plan to move back to USA some time in May. I use Linux as my primary development platform, but I have the means to test my changes on Windows. Finally, I have been using Ogre for a hobby project for about a year.

[Aladrin]

As a newcomer and not-very-knowledgeable-in-3d person, I don't have a lot of say here (if any ), but I love the sound of this project. Testing on multiple hardwares has always been a pain in the rear, and having it all on 1 PC is -so- much easier than trying to work with multiple, not to mention cheaper. You'd still need the physical hardware to be -sure-, but quick testing and finding specific issues would be a lot quicker, I'm sure.

1 more note... You said 'internationally reviled software company' ...

A) Are they really that hated?
and
B) Why would you say that while still working for them?

[xavier]

Ogre uses CVS, not SVN.

A better solution is to add a new option to OGRE that lets developers force the GPU fallback mechanism by emulating the specs of less advanced GPUs.

You might not be aware that this is already implemented in Ogre, by allowing various rendersystem capabilities to be turned off (good for checking technique fallback). Also, nVidia has an emulation layer for their GPUs:

http://developer.nvidia.com/object/nvemulate.html

I would also argue that you can not really tell what a GeF3 would look like when you have an 8800GT, because of the many more pixel pipeline available to you (which you cannot turn off from the application level that I am aware of). In other words, you can't emulate framerate of lesser hardware without actually using that hardware.

[ynaig]

(OOps my mistake, didn't know that Delphi3D database just contains the OpenGL capabilities, not DirectX)

You can find a full database of OpenGL capabilities for any video cards at Delphi3D or just download the GLView utility from Realtech VR website

Ynaig

[Kencho]

I must argue about the idea of scripting the capabilities... This data is likely to stay static for a long while (if not forever), though you might store a lot of different brands and models there. What about using a light-weight database instead?

[juozasg]

Ogre uses CVS, not SVN.

You're correct. Thanks for spotting the mistake.

A better solution is to add a new option to OGRE that lets developers force the GPU fallback mechanism by emulating the specs of less advanced GPUs.

You might not be aware that this is already implemented in Ogre, by allowing various rendersystem capabilities to be turned off (good for checking technique fallback). Also, nVidia has an emulation layer for their GPUs:

http://developer.nvidia.com/object/nvemulate.html

I would also argue that you can not really tell what a GeF3 would look like when you have an 8800GT, because of the many more pixel pipeline available to you (which you cannot turn off from the application level that I am aware of). In other words, you can't emulate framerate of lesser hardware without actually using that hardware.

I was not aware that turning off capabilities is possible other than calling getRenderSystemCapabilities and adjusting the resulting object. Whatever materials are loaded after that will use the adjusted capabilities. But this approach leaves the developer responsible for keeping track of what cards support what. The benefit of this project is automating that headache away.


NVemulate is very nice, but it's software based and very slow. It works for testing fallback techniques, but is badly suited for testing anything animated.

Finally, emulating framerate of lesser hardware is in no way a goal of this project, nor is it possible without downclocking the GPU and other nasty tricks. We want to emulate the "look" of lesser hardware (while running Ogre at blazing fast speeds). That's possible because lesser hardware (with fewer pixel pipelines) will automatically do more passes. I will be more explicit in the new draft of my proposal.


Thank you for comments.


EDIT: Bad quote formating.

[juozasg]

I must argue about the idea of scripting the capabilities... This data is likely to stay static for a long while (if not forever), though you might store a lot of different brands and models there. What about using a light-weight database instead?

I was thinking less in terms of scripting and more in terms of plain text that is easy for Ogre to parse. I also thought about something like SQLite, but I hate adding dependencies. In the end, with plain text, the user will still receive a single file storing all of the card capabilities info. And simple plain text ages very gracefully. I also thought about XML, but Ogre core doesn't seem to be using XML and I wanted to stay consistent.

[syedhs]

I was thinking less in terms of scripting and more in terms of plain text that is easy for Ogre to parse. I also thought about something like SQLite, but I hate adding dependencies

That is a good point, dependencies should be lessened where possible. However, may I suggest that instead of hard-coding the code so that it reads only text file, you may introduce a class which abstracts the I/O operation. That way, it is easier should the user need to have it read from database instead of text file or script. Just a thought.

[spookyboo]

The database could be stored on the Ogre server and maintained by the moderators (if they don't have inspiration to code)

[sinbad]

I think a simple text file to store a database of capabilities would be just fine. I agree that adding a dependency (either XML parsing or SQLLite) is unnecessary. What's important though is that these .rendercaps files are only parsed the first time a user specifically asks for an emulation mode so that startup times aren't impacted when no emulation is being used.


The idea of this feature, which I think the OP has grasped, is that we'd like to automate as much as possible the testing of the 'what would this do on a GeForce4MX' problem. It'll never be foolproof but it will allow simpler testing for which right now I just hack about in the initCapabilities() method on a case-by-case basis.

I think being able to serialize RenderSystemCapabilities automatically to one of these scripts is a great idea. As you say it will allow simple collection of information about cards from a distributed audience.

[Lioric]

Yes, the target of this project should be testing an application on different hardware caps, to correctly test and implement the code paths on different machines, as what to do when a certain hardware support is not available, not to test performance

Are you aware of similar (seems to, i have not used it) application, to configure the driver hardware reporting features, including shader model version:

http://www.tommti-systems.com/main-Dateien/files.html

It uses a batch file for storing hardware specific specs, and uses a the Vendor and Device ID's to set the proper configuration

[juozasg]

Yes, the target of this project should be testing an application on different hardware caps, to correctly test and implement the code paths on different machines, as what to do when a certain hardware support is not available, not to test performance

Are you aware of similar (seems to, i have not used it) application, to configure the driver hardware reporting features, including shader model version:

http://www.tommti-systems.com/main-Dateien/files.html

It uses a batch file for storing hardware specific specs, and uses a the Vendor and Device ID's to set the proper configuration

That applications sounds interesting. I'll have to test it out when I get back to my desktop computer with it's fancy nvidia gpu and windows xp.

[haffax]

This project has been accepted, congratulation to juozasg for the successful application.
Since I take the mentorship, I want to share my impression so far here and write a few comments and questions I have regarding the nature of this project. Everyone feel free to comment.
So without further ado:

Overall I like the idea itself and the way you propose to implement it. There are just a few details to be refined and unanswered questions to investigate But that's what the time till June is for.

First thing on the list is the nature of the rendercap files. They are not resources, but what are they? This is important to determine in order to load the file and for other design decisions. Resources are found in the resource path, Ogre finds its config files from the Path given to the Root constructor. But I am not sure what is feasible for the rendercap files. Something to discuss.

What happens, if the emulated RenderSystemCapabilities are not a subset of the real RenderSystemCapabilities?

The rendercap file doesn't yet mention rendersystem specific features. For instance shader profiles are different in OpenGL and DirectX and the solution has to work with Cg too. Not all checks are currently written back into the RenderSystemCapabilities (famous example: D3DPMISCCAPS_PERSTAGECONSTANT)
This is also an example for a setting, that only applies to DirectX and not to OpenGL. How are these handled? What comes to my mind are either separate subsections in the rendercap or abstracting them and just default them to either on or off for the RenderSystem they don't apply to. Is there another sensible way?

How and where is the database stored? Options are zip and multiple/single file. Each has advantages and disadvantages. Deciding where to place it is also not trivial. (contrary to how it sounds at first)

Another question, and this is something I'm more asking the community than juozasg, is the naming of it all. The project has been proposed by Sinbad as a solution for emulation of "lesser" systems. But it doesn't really emulate, it rather constraints the used feature set to a subset of what is possible. It is useful for other things than emulation too. Namely troubleshooting. Often drivers don't really support all the features they purport to support. Being able to selectively disable features, even on the client's computer, is a way to enhance compatibility. But this has nothing to do with emulation.
So I'm not content with calling it emulation really. It doesn't really do what the name suggests and it also does more.

The time table for the deliverables looks sensible imho. The time you reserve for each work item should suffice. Only thing I wonder about is why test scripting is last. What testing strategy do you implement? I am in no way an expert here, but I write my tests and test cases before and during development. (if at all )

I have ideas myself on most of these problems already, but I want to first hear what others have to say about it.

I really look forward to the summer and am sure, that I too will learn much in the process. Not the most in Ogre, but more about how you go at it. For instance, I've never seen SCRUM implemented before and am curious about it.

Godspeed!

[Kencho]

Totally agree with you.

Regarding the project name, I think "Render capabilities tweaking" would fit better according to this purpose. What do you think?

I'm not an expert on Scrum, XP or testing, but have certain experience there. I wrote a small guide for the SoC some days ago where I give some hints based precissely on that experience, in case you find them useful. It's in the wiki in case you want to have a boring reading

[jacmoe]

I don't agree with you two!

Unless I am misunderstanding something, this SoC project is about emulation of lesser cards, right?
Done by modifying the render caps (after querying them).
Which means that we can test our application on a wide range of (fake) graphics cards - all of them less capable than the one we're running the GPU emulator on.

[Kencho]

As I've understood it, you can tweak the render capabilities manually (of course not enabling something unavailable through hardware). In addition, there can be a database of existing cards with their capabilities, so you have some "presets" instead of having to manually change each cap.

[haffax]

Unless I am misunderstanding something, this SoC project is about emulation of lesser cards, right?
Done by modifying the render caps (after querying them).
Which means that we can test our application on a wide range of (fake) graphics cards - all of them less capable than the one we're running the GPU emulator on.

That's unnecessarily narrowing down the capability of this project and pretending something that's not the case either.
You can't emulate an old card on a new card really. You can only limit the abilities of a new card, which is not the same in the end. Old cards are driven by old drivers. Support for older cards eventually stops in new drivers don't improve visuals on older cards, but on newer ones. Thus the image you get will be different from the image a user with the old card will get. The name pretends something that can't be achieved. There also is no *emulation* taking place at all. Only imposing a restriction on feature usage. It is like emulating a compact car with a Ferrari by throttling it to 80 HP.
Also, that you have "presets" for the caps doesn't mean, that you cannot change them individually. The API should contain this feature. After having implemented everything else, this is easy to achieve and a nice helper. All you have to do is to change the RenderSystemCapabilites manually instead of loading it from a file. Not allowing it would again be an unnecessary self-imposed limitation.

Also you're missing the compatibility aspect. There are many cases documented in the forum, where a driver purports to support a feature, when it really doesn't. Disabling it for compatibility reason has nothing at all to do with "emulation".

Instead of looking at what it originally was proposed to do, look at what it is able to do.

@Kencho, your name is better than what I had in mind, but still... Something is missing.

[jacmoe]

It is like emulating a compact car with a Ferrari by throttling it to 80 HP.

Yes, it is clear to me now!

How about calling it "RSC Wrangler" ?
Interesting project!

[juozasg]

This project has been accepted, congratulation to

First thing on the list is the nature of the rendercap files. They are not resources, but what are they? This is important to determine in order to load the file and for other design decisions. Resources are found in the resource path, Ogre finds its config files from the Path given to the Root constructor. But I am not sure what is feasible for the rendercap files. Something to discuss.

I've been rooting in the ResourceGroupManager, ResourceManager and other related files and I've come up with the following idea: let's add another "special" resource group (like "General") called "OgreTestingResources"

Then a single switch in ogre.cfg and some special case code in ResourceGroupManager can be used to decide if these files should be parsed. We can even use #define to exclude OgreTestingResources special case handling. That way we get full start-up performance in production builds of Ogre and no fuss about special paths and loading for rendercaps files. They will be just like regular resources. I'll add a new ResourceManager subclass RenderCapabilitiesManager to OgreMain/src folder and load .rendercaps files recycling the techniques from FontsManager. Finally, RenderCapabilites will become an instance of Resource.

What do people think? Are these changes too intrusive? One option is to add a RenderCapabilitiesDefinition, which would be a real resource, but leave RenderCapabilities unchanged - references to RenderCapabilitiesDefinition are easier to exlude with #define to keep Ogre production builds leaner.

Additionally, reading .rendercaps files should be done using standard Resource handling, but writing them will be a custom deal using something like RenderCapabilities::writeToStream(DataSteam& target)



EDIT: grammar is good

[Kencho]

I think this quite overkill, from my point of view. In final releases (production bundles) you are not likely to include testing resources of any kind, including graphics cards "emulation" scripts. You simply use them for internal development and remove them from the final release.
Thus, investing such amount of work (and perform the core changes) needed to be able to exclude scripts parsing is redundant. Indeed, it's not done even for the other resources. In Ogre, if you don't want a resource group to be loaded, you don't tell the res manager to load that group, or even include it in the config files.

I would better focus on learning about the Ogre render systems core first rather than the scripts and resource systems. You can do script parsers having the render system caps "emulation", but not in the reverse order, so definitely implementing the functionality so that programmers can hard-code the emulation themselves has a much higher priority than implementing a scripting system

[sinbad]

I agree. Best to put the scripts in a clearly marked location and add that to the resource path, which will of course be altered by any production application anyway.

[juozasg]

I agree. Best to put the scripts in a clearly marked location and add that to the resource path, which will of course be altered by any production application anyway.

I assume this means that it is a good design for rendercaps scripts to be another type of resource. In that case, we can also agree that for now rendercaps files should be stored in a resource folder, but later moved to a zip archive.


On to the next question of render systems. I'm still investigating rendersystems to decide what how to handle API specific caps. Recently I have been a bit too busy (I'm employed full time until May 18th with weekends and few days of vacation until then) to do it more seriously, but right now I'm reading various documents on HLSL, GLSL, Cg, OpenGL and DirectX to figure out how various shader profiles and capabilities relate.

Can anyone recommend a good overview, either in a book or in an article?

[haffax]

I don't know a single resource that handles all, but there are a few sites/books that might help.

First is the Ogre manual section 3.1.4 it lists all supported shader syntaxes and roughly how they correlate. Though there is no 1:1 mapping from OpenGL to DX syntaxes.

As for the capabilities in DirectX and extensions in OpenGL. All DirectX capabilities are neatly organised in the monster struct D3DCAPS9.
There is no such thing in OpenGL, here capabilites are exposed by core ogl and extensions separately. Mostly you can query support by using the glGet functions. Ogre uses GLEW to identify supported extensions, which makes the process easier and better readable.
The OpenGL extension registry lists all available extensions, but only a fraction of those are used in Ogre.

HTH

[juozasg]

My mentor, haffax, and I have discussed how RenderSystemCapabiilities should be stored on disk. We came up with some ideas and the next step is getting your feedback.

I have decided to store RenderSystemCapabilities as a script (similar to .material script) because that will not stand out from the rest of Ogre and lets me reuse more code.

Here is an example script that stores a RenderSystemCapabilities object.

Code: Select all

render_system_capabilities "ATI Mobility Radeon X1600"
{
  gl_driver "8.34.8"
  gl_vendor "ATI Technologies Inc."
  gl_renderer "ATI Mobility Radeon X1600"


  dx9_driver "9.999"
  dx9_vendor "ATI Technologies Inc."
  dx9_renderer "ATI Mobility Radeon X1600"

  num_world_matrices 10
  num_texture_units 10
  stencil_buffer_bit_depth 8
  num_vertex_blend_matrices 5
  vertex_program_constant_float_count 10           
  vertex_program_constant_int_count 10           
  vertex_program_constant_bool_count 10           
  fragment_program_constant_float_count 10           
  fragment_program_constant_int_count 12           
  fragment_program_constant_bool_count 12
  num_multi_render_targets 6
  max_point_size 128
  non_POW2_textures_limited false
  num_vertex_texture_units 8
  vertex_texture_units_shared 6  


  automipmap				true
  blending				      true
  anisotropy				     true
  dot3						  true
  cubemapping				true
  hwstencil				      true
  vbo						   true
  vertex_program			true
  fragment_program		    true
  texture_compression		   true
  texture_compression_DXT     false
  texture_compression_vtc	true
  scissor_test				     true
  two_sided_stencil			true
  stencil_wrap				    false
  hwocclusion				   false
  user_clip_planes			 false
  vertex_format_ubyte4		   true
  infinite_far_plane			 true
  hwrender_to_texture		    true
  texture_float				    true
  non_power_of_2_textures      true
  texture_3d				     false
  point_sprites				    true
  point_extender_parameters  true
  vertex_texture_fetch		      true
  mipmap_lod_bias		      true



  per_stage_constants                true
  fbo          true
  pbuffer   true
  draw_buffers_ati    false
  draw_buffers_arb true


  shader_profile ps_1
  shader_profile vs_2
  shader_profile ps_1_x

  shader_profile fp_20
  shader_profile vp_20
  shader_profile vp_30
}

There are two additions to the standard RSC here: shader syntaxes and system specific capabilities. RSC doesn't care which shader profiles are used by whom, so that is an easy case. For system specific capabilites (per_stage_constant, fbo, pbuffer, draw_buffers_ati, draw_buffers_arb) there is an option to prepend a gl_ or dx9_ prefix to show whrich RenderSystem uses that capability.

The existance of dx9_driver and gl_driver fields shows that this RSC is guaranteed to work on DirecX9 and OpenGL systems respectively. This is a fairly simple and robust way of handling special case capabilities in both systems in a single script without nesting sections imo. One flawis that sometimes DX9 and GL drivers don't have the same capabilities. If this is rare, then we can simply split the capability into two: "ATI Mobility X1600 (DX9)" and "ATI Mobility X1600(OpenGL)". Detecting when splits are necessary will be the responsibility if the rcapdump tool.

Please post your comments and concerns.

[tuan kuranes]

I'm not sure if it's in the scope of the subject, but I'd really like new info/emulation caps :

max texture size
max render texture size
max texture unit usable in a pixel shader (might not be the sames as in Fixed Function case)
max texture unit vertex shader (VTF caps)