null renderer

Discussion area about developing or extending OGRE, adding plugins for it or building applications on it. No newbie questions please, use the Help forum for that.
User avatar
xavier
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 9481
Joined: Fri Feb 18, 2005 2:03 am
Location: Dublin, CA, US
x 22

Re: null renderer

Post by xavier »

Praetor wrote:
Tiser wrote:So, you say it would be welcome, does that mean that if somebody finished the null renderer linked above, would you guys add it to the trunk and go through the trouble of keeping it updated? i.e: official "support". Is there enough interest inside the team? That's all I want to know.
Some Ogre addons have great support and maintenance.
And the flipside, others have none. It all depends on who owns the addon. It sounds like this would be one of those that gets tossed up and forgotten (not maintained) and we'd be right back answering the same question after the next API-breaking Ogre release.
Do you need help? What have you tried?

Image

Angels can fly because they take themselves lightly.
Tiser
Gnoblar
Posts: 7
Joined: Thu Sep 10, 2009 4:08 am

Re: null renderer

Post by Tiser »

xavier wrote:
Tiser wrote: PS: Just noticed that both Irrlicht and Crystalspace have null renderers (The fact that others do it doesn't mean a thing, I'm just saying, it would be nice for people coming from those engines also.)
That's because both of them have software renderers -- Ogre is 100% hardware-accelerated.
False. Correlation doesn't imply causality. Ogre 1.4 had a NullRenderer floating around and it didn't need a software render ;-) Now, how do you explain that?
DanielSefton wrote:
xavier wrote:Try to create/load a texture without initializing a render system.
Why would you want to do that on a server?
http://www.heroengine.com/world-building (no need to watch it, just a collaborative terrain editing video)

Dynamic content creating + streaming = server needs to be able to create lightmaps, provide thumbnails of textures that the artist still hasnt synced locally, pre-rendered thumbnail previews of shaders, etc. Not to mention animations... Just because what currently exists is enough for you it doesn't mean it's enough for everybody. Even if I could extend that and implement it, just plugging a NullRenderer is much faster than extending things, the dependencies are everywhere in Ogre. Yes, it's a feasible amount of work, but why would I do that if it's 20x faster to just plug a Null Renderer, (or what I do now, not rendering + mesa3d + virtual X server.) And to think that you still call that massive waste of time the only true good design... And no, don't tell me that it will make it hard to mantain, its the absolute opposite, because if I don't have to do stuff inside Ogre I dont have to update anything as Ogre changes.
User avatar
xavier
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 9481
Joined: Fri Feb 18, 2005 2:03 am
Location: Dublin, CA, US
x 22

Re: null renderer

Post by xavier »

Tiser wrote:
xavier wrote:
Tiser wrote: PS: Just noticed that both Irrlicht and Crystalspace have null renderers (The fact that others do it doesn't mean a thing, I'm just saying, it would be nice for people coming from those engines also.)
That's because both of them have software renderers -- Ogre is 100% hardware-accelerated.
False. Correlation doesn't imply causality. Ogre 1.4 had a NullRenderer floating around and it didn't need a software render ;-) Now, how do you explain that?
Correction -- it had a "null renderer" that didn't work because of the resource dependency issue I keep trying to explain.
Dynamic content creating + streaming = server needs to be able to create lightmaps,
Software lightmap generation, or rendering hardware on the server -- assuming that lightmaps are created on the server and not cached on the client when they change (the former would be an incredible bandwidth hog, regardless whether said maps come from a content server)
provide thumbnails of textures that the artist still hasnt synced locally
Freeimage/gd/your-fav-2D-image-package-here on the server -- a render device texture object never has to be created
pre-rendered thumbnail previews of shaders
That are not generated at runtime on the server, and if they are, there is rendering hardware on the server side -- again, more likely done client-side on the fly and cached
Not to mention animations
Which I do on my server entirely without any rendering API or support at all.
... Just because what currently exists is enough for you it doesn't mean it's enough for everybody. Even if I could extend that and implement it, just plugging a NullRenderer is much faster than extending things, the dependencies are everywhere in Ogre. Yes, it's a feasible amount of work, but why would I do that if it's 20x faster to just plug a Null Renderer, (or what I do now, not rendering + mesa3d + virtual X server.) And to think that you still call that massive waste of time the only true good design... And no, don't tell me that it will make it hard to mantain, its the absolute opposite, because if I don't have to do stuff inside Ogre I dont have to update anything as Ogre changes.
You are simply trying to rationalize your bad decision, sorry. You made a conscious choice up front to require -- yes, absolutely require -- accelerated rendering support on your server when you decided to tie your code inextricably to Ogre. The bad decision was making your rendering API your platform -- your engine should not depend on a single library for functionality unrelated to that library. I'm going to take a wild shot in the dark here -- you probably use Ogre::Vector3 all over your non-rendering-related code too, don't you?

Given that I am starting to repeat myself, and you clearly are not going to accept my arguments, I'm probably done making them. You made your bed, sleep in it.
Do you need help? What have you tried?

Image

Angels can fly because they take themselves lightly.
Vectrex
Ogre Magi
Posts: 1266
Joined: Tue Aug 12, 2003 1:53 am
Location: Melbourne, Australia
x 1
Contact:

Re: null renderer

Post by Vectrex »

xavier wrote:you probably use Ogre::Vector3 all over your non-rendering-related code too, don't you?
At the risk of dousing this thread with petrol, I never understood what's so wrong with using the Ogre::Vector3 etc classes ;) I mean, you need vec3/quat etc, ogre's is there, it's totally independent of ogre, it's even just copied from some other lib. If I copied the classes and rename Ogre->MyGame would it be ok then? :twisted:
User avatar
sparkprime
Ogre Magi
Posts: 1137
Joined: Mon May 07, 2007 3:43 am
Location: Ossining, New York
x 13
Contact:

Re: null renderer

Post by sparkprime »

Should be trivial to extract the 'utility' parts of OGRE like these math classes and compile a server binary that has no need for graphics hardware. Would be even easier if OGRE just had a separate maths/utility library.
User avatar
xavier
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 9481
Joined: Fri Feb 18, 2005 2:03 am
Location: Dublin, CA, US
x 22

Re: null renderer

Post by xavier »

Vectrex wrote:
xavier wrote:you probably use Ogre::Vector3 all over your non-rendering-related code too, don't you?
At the risk of dousing this thread with petrol, I never understood what's so wrong with using the Ogre::Vector3 etc classes ;)
It depends on how close you want to couple your application to Ogre. For a 100% standalone application, with no chance ever of using another rendering library, it's probably fine. If you have the discipline to resist the very common temptation to continue using more and more of Ogre "because it's already there", you'll probably be fine. Without that discipline, especially when you intend up front to have a server component on which you want no rendering to occur, you will likely end up one day starting threads like this one, because you let Ogre become your application's platform, rather than a just another library that your application uses.
Do you need help? What have you tried?

Image

Angels can fly because they take themselves lightly.
User avatar
xadhoom
Minaton
Posts: 973
Joined: Fri Dec 28, 2007 4:35 pm
Location: Germany
x 1

Re: null renderer

Post by xadhoom »

@xavier:
Hmm, interesting points you raise here. So, does that mean that you would do e.g. all animation keyframe calculations in your own custom class and provide Ogre with the results? Having custom representations of all bones for example which you synchronise with the Ogre bones for final rendering (if you are not a server)?
I´m pretty sure that you would not use the *.mesh format either then, right? But providing a custom format from where you convert everything from your own mesh representation to the Ogre::Mesh class finally?! How do you avoid using two times of the memory in these cases?
Ok, one last question because I´m really curious: Do you have a clue how to wrap the upcoming Ogre::Paging component?

By reading this again it sounds a bit as if I would mock you but thats not true, I´m really curious. :wink:

xad
User avatar
xavier
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 9481
Joined: Fri Feb 18, 2005 2:03 am
Location: Dublin, CA, US
x 22

Re: null renderer

Post by xavier »

xadhoom wrote:@xavier:
Hmm, interesting points you raise here. So, does that mean that you would do e.g. all animation keyframe calculations in your own custom class and provide Ogre with the results?
I don't personally -- I let Havok do that.
I´m pretty sure that you would not use the *.mesh format either then, right? But providing a custom format from where you convert everything from your own mesh representation to the Ogre::Mesh class finally?!
Actually, yes. that's pretty much what I do. But if you absolutely *must* use .mesh on a server, then you are perfectly able to do so -- the command line tools run just fine without a render window.
How do you avoid using two times of the memory in these cases?
What you do is avoid using the D3DPOOL_MANAGED, if your mesh resource data needs to remain in memory after the object has been created. This isn't an option with Ogre, unfortunately; it forces everything into D3DPOOL_MANAGED (except for those resource types that cannot be MANAGED, such as dynamic textures or render targets). That said, there are few cases where this is needed; on a server, if you for some reason are creating your physics bodies from graphics representations at runtime instead of offline, then you have the choice of (a) using twice the memory, or (b) loading/creating and then locking the Mesh data.

Ok, one last question because I´m really curious: Do you have a clue how to wrap the upcoming Ogre::Paging component?
No because I'm not using it, so it's not my concern.
Do you need help? What have you tried?

Image

Angels can fly because they take themselves lightly.
taesoo
Halfling
Posts: 44
Joined: Sun Jan 02, 2005 12:55 am
Location: Daejeon, Korea
Contact:

Re: null renderer

Post by taesoo »

I have a good example where a null renderer is needed.
I am a graphics researcher who is implementing a multi-processor version of a kind of dynamics simulation using my 8 core desktop.
Currently, the visualization relies heavily on Ogre, but takes less than 0.01% of the running time. (The rendering is done only once for every 5000 simulations.)
The visualization part mainly draws debugging messages (that are hard to trace using a debugger or printf..) and virtual characters.
Now, I want to run the same program on a super computer with hundreds of nodes without opengl support.

I have two options,
1. detaching all the debug visualizations from my code, and maintain two versions of program.
2. simply use a null renderer.

I think that a null renderer is the right choice here because I am not writing a commercial program.
(Actually, I think there are many useful cases of a null renderer including unit testing.)
User avatar
xavier
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 9481
Joined: Fri Feb 18, 2005 2:03 am
Location: Dublin, CA, US
x 22

Re: null renderer

Post by xavier »

Option 1 does not require two versions of the program -- it simply requires your code to detect if graphics hardware is available and disable the visualization code if none is present.
Do you need help? What have you tried?

Image

Angels can fly because they take themselves lightly.
taesoo
Halfling
Posts: 44
Joined: Sun Jan 02, 2005 12:55 am
Location: Daejeon, Korea
Contact:

Re: null renderer

Post by taesoo »

Maintaining series of #ifdef/#endifs can be tedious and can produce subtle bugs in an extreme programming environment (i.e. research).
User avatar
xavier
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 9481
Joined: Fri Feb 18, 2005 2:03 am
Location: Dublin, CA, US
x 22

Re: null renderer

Post by xavier »

Again, it's nothing to do with #ifdefs (which presume two versions of the program). It could be as simple as "is the pointer to my renderer valid? If so, use it, if not, skip this block of graphics-rendering-related code". I use it quite well in my application, and my server needs no renderer of any sort, null or otherwise.
Do you need help? What have you tried?

Image

Angels can fly because they take themselves lightly.
User avatar
jacmoe
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 20570
Joined: Thu Jan 22, 2004 10:13 am
Location: Denmark
x 179
Contact:

Re: null renderer

Post by jacmoe »

taesoo wrote:Maintaining series of #ifdef/#endifs can be tedious and can produce subtle bugs in an extreme programming environment (i.e. research).
Take a look at this topic, where Wally actually managed to mock Ogre:
http://www.ogre3d.org/forums/viewtopic. ... ck#p342900

Mocking seems to be way in extreme programming?
/* Less noise. More signal. */
Ogitor Scenebuilder - powered by Ogre, presented by Qt, fueled by Passion.
OgreAddons - the Ogre code suppository.
User avatar
Praetor
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 3335
Joined: Tue Jun 21, 2005 8:26 pm
Location: Rochester, New York, US
x 3
Contact:

Re: null renderer

Post by Praetor »

I feel like you could certainly use a null renderer if it is easy. But keep in mind "easy" here means implementing it. My response here is that I have so much other functionality that actually involves rendering to work on, that maintaining a null renderer isn't a priority. Perhaps it could end up in ogre addons.
Game Development, Engine Development, Porting
http://www.darkwindmedia.com
taesoo
Halfling
Posts: 44
Joined: Sun Jan 02, 2005 12:55 am
Location: Daejeon, Korea
Contact:

Re: null renderer

Post by taesoo »

I agreed. Mocking my ogre wrapper instead of all render-system specific ogre APIs seems to be the way to go.
User avatar
madmarx
OGRE Expert User
OGRE Expert User
Posts: 1671
Joined: Mon Jan 21, 2008 10:26 pm
x 50

Re: null renderer

Post by madmarx »

Hi, I have a question (mainly for xavier) concerning the following :
I´m pretty sure that you would not use the *.mesh format either then, right? But providing a custom format from where you convert everything from your own mesh representation to the Ogre::Mesh class finally?!
answer from xavier :
Actually, yes. that's pretty much what I do.
I find it very interesting to read that, because I was wondering myself if I should do that for my own need, or if it was too extrem. (I needed it for background loading within ogre, because mesh stores material name, and I can't know it without loading ... in the end I use a multi process system that launch a second ogre which use the default software mesh loader to get the information, and transmit them to the main process....).

My question is the following :
if you got your own format, how do you export from your modeling tool ?
I mean, do you have a post process that load everything that was exported by a traditionnal ogre exporter to re-transform it to your chosen format, or did you wrote a custom exporting system directly integrated in the modeling tool ?
Do you know a good 'base' exporter? I mean not for the ogre file format, just some base maxscript or python done only for exporting data from content tool?
Because from my experience, exporting non moving geometry is pretty easy, but exporting animations (skeletal, morphers, etc...) is close to hell in 3dsmax for example.

Pierre
Tutorials + Ogre searchable API + more for Ogre1.7 : http://sourceforge.net/projects/so3dtools/
Corresponding thread : http://www.ogre3d.org/forums/viewtopic. ... 93&start=0
User avatar
xavier
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 9481
Joined: Fri Feb 18, 2005 2:03 am
Location: Dublin, CA, US
x 22

Re: null renderer

Post by xavier »

I use the Havok exporter to export mesh/material/animation/physics, and extract the mesh/material information offline with a command-line conditioning tool that produces the memory image that my engine runtime can load directly. The command-line tool is executed as part of a larger build system.

Unfortunately, it doesn't make a ton of sense to do this with Ogre because it will want to create gobs of heap-allocated objects anyway when you make a Mesh instance. It will also copy your vertex data into an additional memory-resident buffer by default (since it uses D3DPOOL_MANAGED), so any gains you get from having a memory image of a resource are completely lost with the Ogre resource class design.

What I've done in the past for Ogre resources is pack them, and all of their materials and dependent textures, into a binary file (this was for scene loading) so that all of the dependencies could be handled offline and when it came time to load the data, ManualResourceLoaders took care of creating the instances from the packfile data. The objects I packed into the binary world file, however, were just Ogre .mesh and .material files (plus the textures) -- a ZIP file, essentially (the reason I won't use the Zip archive type in Ogre is a massive memory leak I discovered in the zziplib internal seek functions -- not sure if that's been fixed since then but I've not really cared to find out since that discovery).
Do you need help? What have you tried?

Image

Angels can fly because they take themselves lightly.
User avatar
madmarx
OGRE Expert User
OGRE Expert User
Posts: 1671
Joined: Mon Jan 21, 2008 10:26 pm
x 50

Re: null renderer

Post by madmarx »

Ok, thank you very much for the infos :D !
Tutorials + Ogre searchable API + more for Ogre1.7 : http://sourceforge.net/projects/so3dtools/
Corresponding thread : http://www.ogre3d.org/forums/viewtopic. ... 93&start=0
Mohican
Gnoblar
Posts: 2
Joined: Sun May 01, 2011 5:06 pm

Re: null renderer

Post by Mohican »

I picked up 2 versions from this thread, but neither will compile against 1.7.2 (cannot instantiate abstract class errors).
As anyone done anymore work on the NULL RenderSystem lately?

I plan to use the NULL RenderSystem with Python-Ogre, to run dedicated servers with the same code as the clients.
I can remake a 1.7.2 version myself, but it would be nice if I don't need to!
:wink:
Pforce
Kobold
Posts: 27
Joined: Wed Apr 07, 2010 12:45 am

Re: null renderer

Post by Pforce »

I didn't seem a link to this yet here: https://bitbucket.org/rride_a/ogre-null-rendersystem/ So someone is working on it, maybe even from this thread? :) Anyways, I'd also be interested in this. There has been some work in the repo 5 months ago but I don't know if it actually builds against 1.7.x as haven't tried yet.

We kind of have a similar thing here in out application, I can't say I have investigated i too much but. We have a client and server, both can be ran in headless mode. We use Bullet for physics and as I have understood we need the mesh geometry from ogre for certain collision types (trimesh, convex hull afaik). Running in headless mode in your desktop is not a problem really, we can even run it in linux without X with the also here mentioned DefaultHardwareBufferManager and loading opengl rendering plugin, but without creating a window. We also have a Qt integration where ogre actually renders to a QWidgets winId, so this might be different from other apps. Anyways the problem we have is windows servers, if you don't have the remote desktop open to it when you run the headless server, it will crash on loading directx plugin. I'm no expert in it so cant comment too much whats going in there but I believe this null thing could do the trick. I think I also tried opengl also crashing the app.

Never the less I will give a shot at this null renderer to build and run steadily against 1.7.2, but no promises :) We should really share a repo or something if others are interested in this also, or we could contact that hg guy for commit rights so the work does not go to waste.

I think I read valid points (over the years, a looong thread if you measure by time) from both sides. I understand the ogre admins points, they do have a lot more that is more important to make the actual rendering and its perf to be better. But seems like many people could use this if someone would make it work.
rride
Kobold
Posts: 30
Joined: Thu May 13, 2010 1:14 pm
Location: Minsk, Belarus
x 1

Re: null renderer

Post by rride »

Pforce wrote: Never the less I will give a shot at this null renderer to build and run steadily against 1.7.2, but no promises :) We should really share a repo or something if others are interested in this also, or we could contact that hg guy for commit rights so the work does not go to waste.
Hey, that's me! I've started this implementation but lost need in it due to changes in my plans. It should be already compilable with Ogre 1.7.2 but some simple operations such as texture loading or whatever may cause an exception. I could give commit rights to anybody who cares about it=)
olbr
Kobold
Posts: 30
Joined: Sat Mar 08, 2014 12:47 am

Re: null renderer

Post by olbr »

Hey Guys

I really do need to use null renderer to make Ogre work on a server, have anybody tried it out recently with latest version of Ogre 1.8.1 ?
Can anybody give me a hint on how to make it work ?

Thanks
User avatar
TheOnlyJoey
Halfling
Posts: 53
Joined: Sun Apr 10, 2011 12:05 pm
Location: The Netherlands
x 6
Contact:

Re: null renderer

Post by TheOnlyJoey »

olbr wrote:Hey Guys

I really do need to use null renderer to make Ogre work on a server, have anybody tried it out recently with latest version of Ogre 1.8.1 ?
Can anybody give me a hint on how to make it work ?

Thanks
I am actually working on porting a Null Renderer I had created a year ago to my new codebase. (also Ogre 1.8.1 for now, but needs to work for 1.9 as well)
If i have everything ready, I can send it to you, and maybe if there is more interests from other developers, create a patch for Ogre mainline?

Please PM me your email so we can test this more easily together.
User avatar
Herb
Orc
Posts: 412
Joined: Thu Jun 04, 2009 3:21 am
Location: Kalamazoo,MI
x 38

Re: null renderer

Post by Herb »

@TheOnlyJoey - Myself (and I'm sure others) would have interest in this. If you could put this work in progress in a bitbucket or github repo and share the link on this thread, that'd be awesome.
Pforce
Kobold
Posts: 27
Joined: Wed Apr 07, 2010 12:45 am

Re: null renderer

Post by Pforce »

@Herb and @TheOnlyJoey Sorry I haven't posted here earlier! I actually made a fresh implementation of my own HeadlessRenderer. It works with 1.8 and 1.9 and essentially does what rrides thing did with minor differences (fixes in my opinion).

Check the readme for quite detailed description what you can or cant do with it. Its a pretty slim use case that this renderer will be useful under, but maybe its helpful to you https://github.com/jonnenauha/ogre-headless-renderer

We use this thing on our production servers in http://meshmoon.com/ for https://github.com/Adminotech/tundra on Windows where DirectX or a good graphics card is not available. I haven't tested Linux/OSX but I would expect it to function the same there.

Edit: Licensed under liberal Apache 2 so you can use this in commercial applications. Maybe give a star in github if you find it useful :) Use github to report bugs or send me pull requests from forks. I'd love to make this thing better with other people.

Edit2: I was already in contact with Ogre devs if they would be interested in having this in upstream Ogre. The vibe I got was they dont want another renderer to maintain, especially when the use case for it is very unique and rare (I suppose). After all Ogre is a rendering engine, what we are doing here is just using it for its scene graphs etc. without the rendering bits. So it might be counter intuitive to have it in Ogre itself. Maybe we could make a Ogre wiki page to let people know this exists.
Post Reply