What do YOU want in a BSP compiler?

[qsilver]

Hi guys,

I am working on an open-source MAP->BSP compiler and lightmapping tool, designed for use with Ogre. A preliminary version should be available some time in March. This is a serious project, and I intend to provide a commercial-quality release.

Anyway, I'm taking requests for features! Here is a short list of interesting things which I am hoping to implement:

- full global illumination via photon mapping
- option: "ASAP" mode, trade accuracy for speed
- option: convert output to Y-up coordinate system
- automatic portal generation

However, I'm not a level designer, and I don't know what you guys really want. What's a cool feature that you'd like to see? What's a common problem that you'd like me to address? Tell me!

[jacmoe]

Would be nice to have userdefined objects in the map, like entities, triggers, etc.
And it would be cool to have it integrate with OgreODE, like setting physics properties (slippery floors, etc.)
And maybe funcs, like doors, buttons, etc. (Entities with scripts/callbacks attatched)
Shaders ...

Great news, qsilver!

[Fredz]

A similar project does exist, it's called OpenBSP : http://www.osmanturan.com/ . It supports Quake3 Brush Primitives, Classic Quake and Doom3 map files but I don't know if it supports real lightmapping yet. Judging by the screenshot on their page, it seems to support bump mapped lightmapping though. An other project in the same vein was started but then abandonned some years ago : http://map3bspc.sourceforge.net/ .

Personnaly, I would be more interested in an octree map compiler - maybe based on .scene - since BSP are of limited use on today's hardware and adapted to closed indoors only.

I don't see the point of using full GI for lightmaps though, since they should be view independent anyway. In this case, the first pass of photon mapping would be enough. And I've heard that for pure diffuse solutions, radiosity was better than photon mapping, but I can't find a link about that for now.

[qsilver]

Yes. I'm aware of OpenBSP ... there are two problems with it: First of all, the lighting computation and lightmap packing are far from ideal. Secondly, nobody knows what the license is. Can you use OpenBSP in a commercial product? Can you incorporate the source into another program? It's hard to tell, and the author has not replied to any of my emails.

As for outdoor scenes, BSPs are perfectly viable for those areas. The Unreal Engine uses BSPs exclusively. The culling benefits are greatly reduced, but that doesn't make it worthless. After all, an octree is little more than a BSP with axis-oriented splitting planes.

As for lighting, I'm not really doing "full" GI, but I am doing a much better approximation than radiosity. For example, a light next to a mirror will reflect a bright patch back into the room. Photon mapping also handles caustics, like what you see in the shadow of a glass cup. I am even experimenting with realtime hybrid techniques like baked occlusion data and reflective/specular cubemaps ... no promises, since that's not my priority right now, but I hope to exceed the lighting capabilities of Valve's Source engine.

[sinbad]

Awesome qsilver; if you pull this off it'll be legendary

[qsilver]

Would be nice to have userdefined objects in the map, like entities, triggers, etc.
And it would be cool to have it integrate with OgreODE, like setting physics properties (slippery floors, etc.)
And maybe funcs, like doors, buttons, etc. (Entities with scripts/callbacks attatched)
Shaders ...

Great news, qsilver!

A lot of this is application-level, while I'm working on the backend and the BSP scene manager, but I'll make sure to support this kind of stuff! You can already define custom properties for non-brush entities in the MAP file, and check those properties at runtime with the BSP scene manager.

[jacmoe]

Cool!

[bana]

Sounds ambitious and awesome

So would this only be a backend converter? Would you have recommended frontends?

One thing I really liked about Hammer (HL2 source editor) is how they had Materials rather than textures, and a material would be inclusive of the standard color map, normal map, reflectance, surface properties (footstep noise etc..) I don't know if this is in the domain of your project but just thought I would would spout my ideas

Good luck and keep us posted

[qsilver]

Yes, this is only a backend that compiles the standard .MAP format into a Quake3-like BSP format. I am also creating a modified BSPSceneManager for Ogre, which will read and manage the new BSP format.

For front-ends, I will probably recommend Quark since it is the only one I know that does not have non-commercial use restrictions.

If possible, I will try to set it so that "texture names" in Quark correspond to Ogre material scripts. I'm also trying to convince Sinbad to extend material support to include reflectivity and/or gloss mapping. However, I will make sure that whatever properties are not supported by Ogre's native materials (eg, sound) can be defined in a separate "bsp-materials" file.

Making a full material editor to deal with this is out of my scope, but I will definately make the file format simple enough that a non-programmer could edit materials using a text editor.

[IoN_PuLse]

If it's open-source and it works...then it works for me

[dennis]

I would like optimizing from polygon soup please ...

[qsilver]

I would like optimizing from polygon soup please ...

Sorry, I'm not sure what you mean. Do you mean that you want to turn any polygon mesh (like from 3DSMax or Maya) into a level map? Because right now, the plan is only to support maps created in a specialized .MAP editor such as Quark.

Dealing with "polygon soup" from a triangle-based mesh is too hard for me to support, at least for now...

[bana]

Oh now that I think about it I do have a feature request: I have heard that integrating the current .bsp maps with physics is quite difficult (to get the mesh and turn it into collideable geoms). Perhaps your BSPSceneManager could have an easier method of going about this and putting all the geometry into a nice easily read buffer readable by Ode?

Or perhaps what would be easier is when compiling the .Map file, you could take the Q3 Brushes and save them as Ode primitives.

Or perhaps this is already easily enough done as it is?

[jacmoe]

Dealing with "polygon soup" from a triangle-based mesh is too hard for me to support, at least for now...

Well, I have a copy of an article from the now almost gone Dr.GameMaker site, about a solid-leaf bsp compiler - pretty interesting.
It can be used as a starting point for turning mesh-files into bsp-files...
(I thought of hacking away with it myself..) Send me a PM if you are interested

[qsilver]

I'd love any and all reference material about BSP trees, constructive solid geometry, or even Quark tutorials You can never have enough information!

Bana, I will see what can be done. I'm pretty sure that there are efficient ways to integrate BSPs with ODE; simple collision detection is one of the reasons BSPs ever became popular. It's possible that the current BSPSceneManager doesn't expose the right kind of information, or that ODE (and also OgreOde) doesn't have any documentation on the topic. Physics is not a priority for my first release, but I will definately keep it in mind and try to find an elegant solution.

[spookyboo]

qsilver: It sounds interesting. What is the schedule for this compiler?

[makeshiftwings]

The MOST important thing is a completely legal license. Are you sure that id can not claim copyright for converting map to bsp? I remember reading on the OpenBSP forums that they were still unsure what id's stance was on the whole thing. Obviously the main reason for a component like this is to use all the great tools that are available for building Quake maps and use them for free instead of paying id, and I can't see them being particularly happy about it. I'd say getting legal advice and confirmation from id that they won't try to sue you needs to be done before you even start designing it.

[qsilver]

I have done a little research on this. Id does not have any claim on the idea of BSP trees in general. Nor have they patented their .BSP file format. John Carmack has said that the file format is free of restrictions, but the tools are not.

I have never seen or downloaded the source code for the official Quake tools, nor have I ever tried to reverse-engineer the compiled executables. My implementation is done completely from scratch, with no external libraries or source code. I've written every line in the program myself. So yes, I am quite sure that this project is legal!

If id tries to sue and I can't find a lawyer to represent me, I may have to drop Quake compatibility and create my own file format (and a matching Ogre loader). But even that won't shut down the project.

HOWEVER ... actually using maps from my compiler in the official Quake 3 engine may be illegal. That depends on the EULA, which I have never read since I don't own a copy of Quake 3! The burden falls on the user in that case, not me.

[qsilver]

qsilver: It sounds interesting. What is the schedule for this compiler?

I don't want to commit myself to any hard deadlines, since I do have other things to do in my life, but I'm aiming for a 1.0 release (with source) around the beginning of April. I may release preliminary 0.xx versions as major milestones are completed, but I'm not sure about that. I don't want people to see an early release and dismiss the project due to poor output quality.

[:wumpus:]

One of the great drawbacks of the quake format as-is I've been told is that lightmaps are prebaked so that dynamic lighting is impossible. Will it be possible to just use the BSP structure for collision and per-zone visibility culling, but not for the actual rendering (as stated before, using BSP for per-polygon visibilty actually slows down current hardware)? This would also allow to provide a low res version of the level to the BSP compiler which is probably nice and fast.

[sinbad]

Quake does dynamic lighting by adding additional brightening passes (kinda like decals) in the areas affected. I've also seen someone perform an additive second pass of the entire world geometry using stock per-vertex lighting, although that's extremely limited unless you use pixel shaders because of the size of the polys.

The downside of brightening passes is that it washes out the original colours the darker the original background light. You can make that better by combining the dynamic brightness map and the lightmap, but this really requires pulling the lightmap out of the multitexture and into it's own pass, otherwise you're prone to run out of texture units on more complex materials.

[qsilver]

As I understand it, in Quake 1 and Quake 3, you cannot light up a black spot no matter how many lights you throw on it.

Quake 2 uses a more realistic lighting model, but id switched back to the Q1 method because artists claimed that they couldn't get enough contrast in their textures.

I will do some more research on this and see what I can come up with.

[Chaster]

*bump* How's it going on this project qSilver?

[qsilver]

Sorry for the long delay! I got discouraged by the limitations of the Quake 3 BSP format, and then other real-life matters interfered with my concentration...

After considering the options, I've decided to change the focus of this project.

It no longer compiles to a Quake3-style BSP, but instead to a combination of individual meshes, lightmap textures, and a custom information file.

That's right, I'm working on automatic portalization of any indoor scene, with one individual mesh file per zone. But what good would that be without an indoor portal-based scene manager?

Further updates to come within a week or so.

[haffax]

Now this sound exciting.
Are there any plans to make it work efficiently with mixed indoor/outdoor scenes too? Like a city where you got the outdoor part with doors/windows into the seperate buildings as portals but a rather big cell for the city as a whole. How does each cell organize its immediate content? My knowledge about portals fits on three pages (iirc the part in the real-time rendering book on portal culling was this long ) but I am eager to learn how it is done.