DERGO 3D: An Ogre3D plugin for Blender (WIP/Alpha)

[dark_sylinc]

Hi everyone!

It's been a long time since I don't post a topic in this subforum.

I've been very tired of the Ogre 2.1 branch, so I decided to work in a side project last two weekends that is looking more and more promising.
I present you: DERGO. (That's Ogre3D backwards!)

It's a Blender plugin that renders the 3D output of Blender in real time inside of Ogre. It requires Blender 2.76; Enough talking, video time!


What's the point of it? I have two main goals:

• Use it as a fully-fledged renderer. Similar to what Crytek wants to do with Cinebox. Use regular game rendering techniques to animate movies (except that Cinebox tries to move your workflow to its own UI environment, while Dergo relies on Blender to do all the animation, camera, keyframing, etc).
• Use it as a real time material / mesh / scene editor.

I'm personally more interested in the first point, but certainly the rest of the community will benefit more or find more useful the second goal.
Exporting the whole scene should be possible. The scene data is already there, and the materials too. We also have mesh data, but we let Blender skin the meshes with skeletons, so we wouldn't be able to export animated meshes (not at least without additional code).
I expect artists already familiar with Blender to love it, and those that are not familiar, can get used to it and have a UI to tweak their materials in real time. Not to mention they get to learn live how their changes translate to Ogre's parameters.
Granted, Blender's UI isn't the most user-friendly for the newcomers (quite a difficult learning curve), but editing materials will be quite easy. You don't need to know Blender in depth.

In its current state, it is not exporting materials. That's why the video everything is grey (coloured lights work though!). Soon I'll be implementing the material export and UI.
It is very WIP right now, background colour can't be changed

Once I'm done with that, I'll probably focus on Metallic workflow and fresnel textures which are missing in Ogre 2.1; and will greatly benefit this project (and some users that have been requesting this feature for ages!).

How does it work? the plugin gathers all mesh & material changes, and sends it through TCP to the server. The server can also send the rendered results back via TCP.

I don't know about you guys but I'm really excited about this project.

The repository can be found at:
https://bitbucket.org/dark_sylinc/dergo-blender

There are build instructions but I won't be providing assistance on building the source code. Not at least yet. It's very WIP and constantly changing.
It's being linked against the latest commit of the 2.1-pso branch of Ogre (this is very important: branch 2.1-pso, not 2.1).
If you want to check it out right now, you can download the already compiled plugin. The server must be running at localhost at port 9995 before Blender is launched. Make sure you follow the same steps I do in the video. If the server crashes, reestablishing a connection is hard right now. Probably it will be easier if you just restart the server and Blender alltogether.
If it doesn't work for you, bummer. It's in alpha. I don't care. It's not yet the time to focus on "it doesn't work on my machine requests".

Any feedback?

[xrgo]

Amazing!!!
I am highly interested in both goals
but mainly in the second one, our engine (using Ogre 2.1) already uses Blender as material/mesh/scene editor, it loads blend files alla gamekit, but we can't see how its looking, we have to run the game to see the result, and sometimes we have to do it several times. So a tool like this would be very useful.

I'll probably focus on Metallic workflow and fresnel textures which are missing in Ogre 2.1; and will greatly benefit this project (and some users that have been requesting this feature for ages!).

really? who?

btw I like the name... dergo... xrgo

[al2950]

Awesome

I normally use 3D Max but this gives me a good excuse to check out blender.

Now you just need to add global illumination, volumetric lighting & raytraced shadows

[Germanunkol]

Really cool project. I had been hoping for something along these lines for a long time, my scenes always looked so different in Ogre, when compared to blender. Made light-tweaking really painful...

Keep it up!

[spookyboo]

Good work.

[Kojack]

Cool stuff!

[Zonder]

very nice

[frostbyte]

logged in just to say...GREAT, thanks man,
thinking of porting this to 1.10 branch...its the same mesh/vbo format, right?
on same notes...once stable/usful i suggest adding it to ogre-tools in the main repo...

[dark_sylinc]

logged in just to say...GREAT, thanks man,
thinking of porting this to 1.10 branch...its the same mesh/vbo format, right?

Well, I'm populating a 2.1 Mesh format (Items) while a port to 1.10 would need to use the v1 interfaces, which are clunkier. But definitely not impossible.

I don't see much of a point though. The main advantage of 2.1 that makes this tool useful is that you get to preview the mesh using the Hlms shaders (e.g. how it looks with the default PBS materials).
Ogre 1.10 used fixed function pipeline, which looks the same for everyone and not too different from watching the live preview in Blender using the "Multitexture" or the "GLSL" shading modes (aka "Textured" mode).
Unless you're planning on using it with your own material system. But if you don't already have an existing one written, it's a ton of work.

on same notes...once stable/usful i suggest adding it to ogre-tools in the main repo...

And then it would be lost and abandoned like most of those tools in the repo . Not to mention it clutters the log history.
I actually plan on working on it actively, so it will stay in a separate repo.

[frostbyte]

was thinking of using hlms 1.10 back-port, anyway i want it as a mesh editor...don't care much for mat...guess every one has their uses for this...
global ilum and light-maps...yummy....
anyway, its too usful to be abandoned, thanks again and again...

[TheOnlyJoey]

Looks very nice!

We were actually looking into something like this ourselves, since our engine is based around Blender for 100% loading blendfiles and such.

I actually showed this to some Blender developers (since I am at the Blender Conference right now) and there might be some optimizations that can be made to speed up the 'in viewport' rendering.
Since the idea is to upgrade the Blender viewport to later OpenGL, some extensions might help out in this process as well.

[dark_sylinc]

That's great!!!

Well then... if there's a chance Blender devs can see this, then some dev feedback:

1. Right now I need to iterate through every object, material and texture to check if they've been updated and resync them. While it should be fast, I don't see this scaling too well for very, very large scenes. A far more efficient approach would be to hand out a list of objects, materials and textures that have been modified during this view_update call.

2. Similarly, the only way to figgure out which objects have been removed between two view_update calls is by comparing a list with a all objects from previous call with a list of all objects of the current call. This stupidly unnecessary and doesn't scale well. Would be far more productive if I get handed out a list of objects that are being removed. Also another problem is that the objects are already dead by the next view_update call, which means I can't hold on to a pointer to it (dangling pointer)

3. There is no way to check if a texture_slot in a material has been changed. Right now I'm 'hacking' to see if the user is in the Material or in the Texture panels; and if he is, then resend all texture slots of the active material on every view_update. I only send all texture slots of all materials at startup or on a re-sync (last case scenario when the server needs to be reset).

4. Add a setting so that Properties are set to default when object is cloned. Right now Blender supports adding IntProperty, FloatProperty, StringProperty, etc. Aside from using them for user settings and internal parameters, I use them to know which objects have been sync'ed. For example "object.dergo.in_sync = True". The problem with this approach is that when a user clones the object (Shift+D / Alt+D), all these properties are cloned too. This means that there is a new object in scene that says in_sync = True, when this is not true. Right now I'm workarounding this issue by also storing the name of the object inside a StringProperty. If the names don't match, then it means the user either changed the name or cloned the object, so I just resync all objects that had the same internal ID (internal ID also has the same problem). This would be far easier if Blender had an option for properties that when toggled, it goes back to default when the user duplicates the object.

5. I don't understand how on earth to get the tangents from a tessface. I'm not sure it's even possible. Documentation says it should be in split_normals member, but it has 12 floats (no explanation what these 12 floats mean, I suspect they're 4 vector3, but that's all) and don't seem to be affected at all by calc_tangent calls followed by calc_tessface calls (bug?). Not a big of an issue though, I just wrote the code to calculate the tangents server side.
I look on Cycles as a reference implementation, and for the live preview it relies on glDrawPixels. THIS FUNCTION IS INCREDIBLY SLOW. Seriously. It's S.L.O.W.

As for viewport optimization, yeah Blender uses incredibly old GL functionality. A GL4 pipeline could even do dynamic vertex pulling, thus reading directly from Blender structures without having to preprocess anything except for converting NGons to tris (e.g. read 3 indices from each triangle, then fetch the vertex, normal and UV data on the fly). I don't know how fast something like that would be. Would certainly remove a lot of CPU processing (used to convert to GPU friendly formats), but at the cost of having 3 vertices per triangle to process in the vertex shader without a chance to reuse (tradeoff less CPU processing for more GPU processing).
If Blender could expose a way to grab the HWND/OS-specific handle, I might have chosen another route and embed the Ogre server as a DLL instead of relying on TCP (would've been definitely faster). I found out too late someone was already looking into this, but it feels too hacky (hey, look, it's from the Blender2Ogre dev!)

Well, hope this is useful to Blender devs. Make sure someone reads it!

Cheers
Matias

[xrgo]

I just wrote the code to calculate the tangents server side.

would this help this problem?: http://www.ogre3d.org/forums/viewtopic. ... 99#p519934
can that piece of code be integrated into Ogre? so we can calculate tangents for v2 meshes

thanks!

[dark_sylinc]

Yes and no.

The code from v1 meshes is more robust and has lots more options (specifically for dealing with discontinuities).
My code just doesn't deal with discontinuities at all, which is the reason I didn't write it inside Ogre 2.1, but rather in the app's code instead. First, this makes it super fast (also threadable) and second, the rationale is that if you can live edit the mesh in real time; the artist can easily duplicate the offending triangle or alter or detach the UV map face(s) until it looks fine.

Technically speaking discontinuities happen when two triangles share the same vertex, but the tangent is radically different (e.g. 180° turn); or the parity signs of their bitangents are opposites.
Artistically speaking, this often happens when UVs are mirrored. For example model half of a human body, and the other half uses the exact same UVs (e.g. the textures only contain half of the torso or face to save space).

Interestingly back in 2007 when Steve wrote the original code he deduced that code that deals with discontinuities isn't magical as there still will be a lighting discontinuity which cannot be solved. The tool can only guarantee the seam will look correct (which won't be with my tangent gen code), but it can't guarantee it will look good.

[dark_sylinc]

However yes, you can use the code. It's quite simple to use (only tri lists are supported).

See VertexUtils.cpp

[xrgo]

thanks! I'll look at it =)

[dark_sylinc]

I have barely got the time to work on this, but I forgot to show some progress!!!

Textures are finally supported! Diffuse, normal maps, specular, roughness, detail maps (diffuse and normals).
The UI widgets for detail maps are not working yet though (but soon will be). At the moment the server assumes the textures are at the same path location as in the client (thus, running from localhost is highly recommended). At some point it might support sending the textures over TCP (should be slow, but I'm interested to see if it's feasible for real time updates when painting)

These pictures are just some random diffuse & normal maps I had on my drive:

[frostbyte]

hi, just in case you did'nt notice jme3 have a similiar project going on
hub.jmonkeyengine.org/t/xbuf-format-a-d ... or-3d-data,
the guy is using protobuf to make communication speeder/lighter( he also wanted to make it work with ogre...but got cold showers from community members... )

don't think updating the client should be ultra fast or per frame...it doesn't realy matter...
i understand where the erge to optimize performance is comming from( smily ) but some-times it's just not needed/worth it...
i think it better to stick with the client/server arch for two main reasons...
1) blender artist / ogre developer remotly working together online scenerio
2) integrating the DERGO client with the developers game/game engine( as opposed to using DERGO as a stand-alone viewer ) can benefit developer with live-editing of his running game/engine

also integrating Ogre hwnd with another gui can be easy or it can be a major PITA - you never know until you try, not to mention its not cross platform...
benefits are application performance and easy coding, but i don't think max performance is realy needed here.
P.S maybe its better to ask blender Q in blender devs forums?
thanks for reading...

[dark_sylinc]

don't think updating the client should be ultra fast or per frame...it doesn't realy matter...

Well yes, it does. When it gets compared to Cycles, LuxRender or Mitsuba (which look beautiful out of the box with its GI and caustics and stuff, they're all path-/ray-tracers), the only advantage we have is speed which translates to productivity.

i think it better to stick with the client/server arch for two main reasons...

We're not going to change now. I just said if things had been different, I may have gone another route.

hi, just in case you did'nt notice jme3 have a similiar project going on
hub.jmonkeyengine.org/t/xbuf-format-a-d ... or-3d-data,
the guy is using protobuf to make communication speeder/lighter( he also wanted to make it work with ogre...but got cold showers from community members... )

Interesting. It's the 1st time I see it.
Anyway too late now. I've got almost everything working now. It wasn't a hard job at all considering the little time spent. I don't see the point in using protobuf or any similar extra layer of abstraction for this project when you can send the raw data in binary.

[dark_sylinc]

Updated download link with latest version, which added Metallic workflow support.

[xrgo]

looking great! is it integrated in a blender viewport?

[dark_sylinc]

Yep, it always has been. I just happened to show the async features (which are waaaaay faster), but you can also run it inside Blender if you want.

The only thing not working yet is the final Render (when you hit F12)

[dark_sylinc]

Reuploaded plugin.

I changed how coloured fresnel and metallic behave. THIS is why I wanted a tool like Dergo. I can test all settings interactively. It became very obvious that it was.
Now metallic workflow behaves exactly (or almost exactly) like other PBRs.

The only missing feature to Ogre is coloured fresnel texture, and thus specular workflow could be easily imported from other PBR apps.

[only_a_ptr]

This project is wicked!

Questions: Since the server has the entire Blender scene (incl. materials) in OGRE2.1 for display, it should be easy to add export logic to it, making DERGO an interactive exporter. Is there some technical showstopper to this? Is it already planned?

I'm planning to create a vehicle editor for an open-source (GPLv3) simulator game (http://rigsofrods.github.io/en/). I'm counting with use of OGRE2.1, and I want to provide users with interactive material editor + convenient way to switch between editor and simulation for tuning. This was my original workflow draft:
1. Modelling in Blender exclusively
2. UV mapping in Blender
3. Blender 2 OGRE export. -> import to the editor.
4. Material editing in a built-in editor (HLMS based with interactive preview).
5. Physics rigging and setup in the editor.
6. Switching between editor/simulation
7. Export of materials [meshes?] from the editor.

DERGO appears to change the rules completely. Assuming I can hijack DERGO the way I imagine, this is the future workflow:
1. Modelling in Blender
2. Full HLMS material editing in Blender with DERGO preview.
3. Physics rigging and setup in Blender
4. One-click export through Blender scripts + modified DERGO.
5. Testing in simulation.

I haven't really tried DERGO yet. When I start, I'll be back with more details. Also, I'll try to make useful contributions to DERGO where mutually beneficial.

Kudos to dark_sylinc for out-of-the-box thinking and creativity!

[dark_sylinc]

Questions: Since the server has the entire Blender scene (incl. materials) in OGRE2.1 for display, it should be easy to add export logic to it, making DERGO an interactive exporter. Is there some technical showstopper to this? Is it already planned?

Yes. And this is indeed planned. There is no technical showstopper to this (other than available time )

Particularly exporting of materials is what I'm interested in. Although meshes and other scene stuff could be exported as well. The idea is to get Blender to select the download location, and the client asks the server to send the data as a binary blob via TCP and then gets saved to disk (since the server could be in another machine). This way I avoid duplicating work by rewriting the exporter in Python.
Of course the server could write to disk directly, but I don't know how that would work with a Blender UI to select the download location (which is relative to client) and a server (note that right now the server should be in the same machine as the client since we're opening the texture files directly; I took a little shortcut there... but the long term plan is that the textures would be sent via TCP which would be slooooooow or have the server cache a folder structure for where to look for textures)

Note that Dergo is using the skeleton animation from Blender (i.e. Blender skins the model, animates it, and sends the vertices to us); so we don't have skeleton or animation information to export, at least in its current state.