What do YOU want in a BSP compiler?

[jacmoe]

I was talking about leveraging all the exsisting levels, tools, etc. for bsp maps. There is a lot of modders out there who are eager to make content for the Ogre.
Decent support for the bsp (albeit old tech) would be a nice thing to have, IMO.

Until a full-featured Ogre Level Designer Tool appears (if ever) - this route is a good thing.

((Some tools can decompile/export bsp's to other formats..))

[:wumpus:]

qsilver already stated that he isn't going to make his BSP compiler produce quake3 bsp files, or openbsp for that matter. We can expect he's going for a modern implementation of BSP, and not the years old format which has the disadvantages Lloric mentions.

[jacmoe]

Exactly!

This is sort of two topics in one.

I am not adressing qsilvers scenemanager project, but answering some questions about the Ogre BSP scenemanager.

Would be nice to see what he could come up with.

In the mean-time it really can't hurt to give some pointers on how to improve the exsisting (old) BSPScenemanager.

[DarkSeraph]

Yes, sorry for getting off topic, but I figured since we might be waiting for a while for qsilver's project, making our own might be feasible, since there is more than enough interest.

A redone BSP format would be nice, I really can't wait to see it, but as always I'm impatient I like to get my work done now, but it does seem that the best option is to wait for a little bit.

As for qsilver's project, I never really got an answer: So it will take Hammer/Worldcraft .maps?

[jacmoe]

My guess would be: .scene, or .mesh even.

[DarkSeraph]

My guess would be: .scene, or .mesh even.

Now i'm confused, it's a MAP->BSP compiler, and only takes .scene or .mesh? so lost...

I'm just wondering, since I'm pretty sure there are different formats of MAP(the way the text is set up), what MAP type it will be set up to take in.

[haffax]

DarkSeraph, no. qsilver cancel this bspcompiler project in favor of a full replacement of the bsp scene manager for indoor scenes. It won't be using BSP anymore iirc, but it uses a cell based portal approach. Just read the details further up this thread.

[jacmoe]

Now i'm confused, it's a MAP->BSP compiler, and only takes .scene or .mesh? so lost...

Don't worry!
If you could get hold of that Keith, you'll have a bsp scenemanager for all formats which OpenBSP supports.
If it doesn't support vis calculations, who cares? The hardware can handle it.

[Lioric]

...but it's better than creating the same texture 50 different times because it has a different lightmap applied to it

Its not needed to make new textures because every wall/geom has different light, that is why multi-pass rendering is for, you use the same texture in all walls, but you add another pass with the light from the lightmap, and even the lightmaps are packed in a few textures, typically 2 or 3 lightmap textures per level

Probably you know this already, but lighmaps are different textures than detail/color textures, you dont add any of this textures because of the light pre-procces, light are packed in different textures, as i said all the level light in 2 or 3 textures usually

Lioric

[DarkSeraph]

*feels dumb* ahh ok, don't know why I didn't get that the first time I read through this thread, sorry. I realized I should've re-read the thread before asking some of the questions I've asked.

Like I said jacmoe, compile a HL2 map w/o any vis, especially if it has special lighting instead of just outdoor. Even on high-end systems you'd get reasonably low FPS, and on mediocre systems like mine I'd be lucky to get 5-15fps. Then again I did have to turn off rain in HL2 cuz it killed fps.

Ok, here's the next dilemma I see thuogh, with anything that exports .mesh or .scene it's at least pretty hard, if not impossible, to add in your own entities, such as light, start points, not to mention things that can be game specific such as monster start points. I haven't had much time to research this, but I know how it's done in mapeditors for BSPs, but 3dsmax? It would be... difficult, I think.

[edit]Lioric, my understanding from the other thread about OpenBSP is that it would apply a lightmap over the texture to create whatever new texture will be applied to the wall, being texture+lightmap. Am I wrong? or did he fix this?

[Lioric]

Not difficult at all, but very game/application specific, that is why bsp foramt stores the entities as text, then you application can parse the entity chunk and do whatever you want to do with what the editors have put on it, like instancing monsters, geometric entities, apply material fxs, and in 3dsmax is not too different, you can attach infoamrtion via tracks notes, or text data, or other users make some naming conventions so the engine can act based on the name, like appending some id name when the material needs some fx

I dont saw the post you mention about OpenBsp, but i dont think creating different textures for every single geom element makes sense, its called lightmap because its a map that contains the light information, if this light is baked on the texture, then they cant be called lightmaps anymore, and if you have already the lightmap, why you would like to bake to every texture, creating hundred of different textures with different sizes, this is going to introduce several glitches and artifacts at seams because of the difference of resolution
If you have the lightmaps, you use them on the cards that supports multi-tex, and if not you simply use vertex lighting, but we are far beyond those times by now
This tool, OpenBsp, even if its in early stages, if it supports light geenration, has to support lighmaps via multi-tex

Lioric

[DarkSeraph]

Here it is: OpenBSP Scene Manager

You have to get hold of that Keith guy, though. He holds the code.

I agree with you Lioric, it makes no sense, but that's just what I got out of what I read.

Another question would be how the 3dsmax exporter would handle the entities. *shrug* again I have no idea, but it's somethign I'll talk over with my team, whether we want to continue creating a better BSPSceneManager, or abandon that for other things. Personally I don't udnerstand how BSP is so bad, if the most modern FPS games use them. I do understand there are better ways, but for our purposes I think there is no reason why it doesn't fit nicely, but I'm sure that's just personal preference. This compiler would be nice... I guess I'll see how things go. I'm just trying to get a feeling for what I should be looking at on how to do things before I get too deep into the project, or before we spend too much time making a better BSPSceneManager. I do have to wonder what limitations qsilver ran into with the Q3 BSP that made him abandon the original project.

[jacmoe]

You guys are too serious!
What I meant was: see this guy - he made Ogre load and run OpenBSP maps - and what does that mean? This means that we have a way of moving ourselves from proprietary ID compilers.
Quite obviously, he needs some help in some areas, such as texture management, but that's another matter.

It wouldn't be difficult to improve upon the exsisting bsp scenemanager, though, if you browsed around the code. The entity loading code is there, only that it looks for deathmatch_info_playerstart and then exits.
It wouldn't be too difficult to support different formats of bsp's either, just a bit of branching.

What the heck ...

[:wumpus:]

I do have to wonder what limitations qsilver ran into with the Q3 BSP that made him abandon the original project.

Simple, read LLorics post above. He describes those disadvantages already.

BSP is too old technology for todays GPUS, its that old that now its not a speedup or some good geometry optimization, but a bottleneck in the graphics pipeline

For example, now you can throw a general Q3 lelvel geometry in the engine and render it without using BSP algorithm and still get playable fps

BSP was made when software renderers or primitive hardware acceleration existed, so an extra poly face, made a difference in the fps

But now the gpus are hungry for polygons, a batch beasts like Sinbad said (if i remember well).
What can affect the geometry performance are the render state changes, and the screen pixel resolution for the fragments

Its still good for collision detection or level lights optimization, but in this case a low resolution bsp version in better

There are much efficient culling algorithms now, even convex/portal kd-trees can easily beat bsps, and the memory use is good

Lioric

And no modern game is using BSP as-is the old .bsp format. That doesn't mean they don't use Binary Space Partitioning at all, but in a different way. for example, for collision, very rough culling, detecting which lights influence a certain spot, ... for the level, not the geometry

[DarkSeraph]

not too serious, just curious. We're already working about getting entities into Ogre the same way as info_player_deathmatch, and then getting a class to read them and render them into the game. Just none of us know crap about how to deal with geometry or textures, we only know what's built into BSPSceneManager. I think we're all semi-new to 3d rendering, I have no experience, and most of our programmers have minimal.

As for OpenBSP VIS, can't he just pretty much copy the code from HLLib? I haven't gotten a chance to look at HLLib yet, but VIS should be in there; it's a very important part of BSPs, and is what usually requires the most compile time. I will take a look @ OpenBSP though, I'd like to be able to use Worldcraft/Hammer to make maps, since I learned level design for HL based games The problem is the areas where OpenBSP falls short I have no experience whatsoever. If it could do VIS, it would be very cool to just take out the RAD (lightmapping) part, so we can just compile it and use Ogre lights for dynamic lighting, instead of lightmaps. *shrug* there's a LOT of possibilities, and I'm still up in the air about what I should do ty for your help jacmoe

Wumpus, thank you, that does clear it up. I was thinking it was the file type BSP that was bad, not the organization of it and how it is actually processed by the GPU. again my knowledge of 3d rendering falls terribly short. But if we were to manage to get a BSP to process more like... HL2's for example... then it would be more efficient?

[jacmoe]

That doesn't mean they don't use Binary Space Partitioning at all, but in a different way. for example, for collision, very rough culling, detecting which lights influence a certain spot, ... for the level, not the geometry

My thoughts exactly!
That's why I don't care about vis calculation - just feed the polygons to the hungry Ogre!

Or - feed them into the DotsceneOctree compiler.

I quite like the niceties of a bsp when it comes to searching, branching, organization, etc.

[Lioric]

...it makes no sense, but that's just what I got out of what I read.

What the author is saying is about materials, not about textures, textures are not added or modified in any way, he still uses multi-tex to add the light stage/pass

Material and textures are very different, kind of entity and mesh

Modifying the BSPManager for your own use its a piece of cake, the hard part is already done, the clusters, the pvs info, the brushes, the bsp nodes

BSP structure has very little to do with 3d Rendering per se, but more in the space partitioning area, so if you are programmers with some experience, you will not have too much problem with it, but remember that is a very complex spatial structure

You want to know another bsp problem, well you asked, so how about dynamic worlds

Lioric

[BlasterN]

So here I post some ideas:

OGREBSPFORMAT ( OBF? )
- Use a specific .fgd (this is the collection of entities)
- Don't use Q3 SHADERS! instead of that, use Ogre materials (we can discuss this in future)
- Geometry in BSP. (i don't want to say anything but HL2 source code )
- High geometry in other way... & use some type of culling.

--
To do this we only need two things
OBFSceneManger & OBFEditor (could be gtkRadian modified or quArk IMO: Worldcraft, easy & powerfull )
It will be too cool if simbad help to do a OBF (or whatever you want to call it) specification.

--
Discuss pros & cons it's important to begin with the right leg.

[Lioric]

Forget about qRadiant (or gtkRadiant if you want) its under a restrictive license, it cant be used for anything outside Q3 based engines, i dont know about worldcraft but probably its the same, and forget about bsp, belive me, why spend time in this when we can create some better according to the current technology, i.e GPU friendly

Its really a big amount of work to re-invent a bsp compiler, a visibility processor and a light processor, and there is nothing out there to start as a base, all has to be done from the ground. I saw some projects (some open source) that tried to re implement bsp compilers without the license issues, but the amount of work is not worth for the outdated technology, so they were dropped

Its better to implement a scenemanager for dynamic geometry indoors

Lioric

[DarkSeraph]

Implementing a scenemanager isn't the creation. The advantage of BSP is there are reasonably easy to use editors out there that can easily assist in the creation of a BSP.

So it is unreasonable to think of going in this direction and still having a GPU friendly map? I just want to be able to use one of these editors, or a created one, that can output a filetype that we can use, and we can create a scenemanager to use that will also be GPU friendly.

[haffax]

DarkSeraph, Radiant/Hammer/Worldcraft maps can be easily converted to Ogre meshes. We developed a converter for our project too for instance. Search the forum for links.

The afore mentioned editors are rather limited in how freely you can design your world though. You have much more freedom with modellers like Blender. Blender can export to .dotscene, has superb Ogre support and has no licensing difficulties, making it a good option for mapping.

In our project now all mappers switched to Blender, making the Map2Mesh converter obsolete. And no one is unhappy about it here.

[BlasterN]

I make a little more research about how to create a optimized BSP.

- Control VIS manualy, using Hints brushes & Occluders. The compiler require this information. For example if you hace 2 rooms, you create a hint brush in the middle of both. Is hard to mappers try to divide all the maps but it's easy to programme.
- Lightmaps, I really think that this is the main part of a BSP compiler, the compiler (& the editor) must set a variable size of lightmap for each map zone (another optimization maybe is per each hint brush)


--
More & more information to came... I expect qSilver back one day...

[DarkSeraph]

Blaster, I'm assuming you haven't done any serious mapping for any Half-Life games? This is why I was saying VIS is terribly important, I'm used to using hint brushes to cut up vis leafs, thus optimizing my map. I've heard Occluders can be useful, but you have to be very careful how you use them, or they end up causing just as many computations as if you hadn't used it at all.

I'm not sure what you're saying about lightmaps. Most lights come with a variable that holds the max radius of the light anyways.

[jacmoe]

Dear BlasterN - I don´t think qsilver is gonna make any bsp compiler.

[qsilver]

There seems to be some confusion here ...

Quark/Hammer/etc are all .MAP editors. They create static geometry using "brushes" and place "entities" around the world. Some people call them .BSP editors even though they do not read or write .BSP files.

Binary Space Partitioning, or BSP, is a technique for breaking down and organizing a scene. The result is a "BSP tree", which is a set of planes that divide the scene into regions. Every face of the geometry lies on one of those planes, and regions are either inside the geometry (solid) or outside (empty space).

The .BSP format, created by id and expanded by Valve, contains all the information for a level -- much more than just a BSP tree. The format includes indexed geometry which can be sent straight to the video card. It also contains lightmap textures and visibility information, as well as a complete list of the level's entities.

Here's the thing: the .BSP format sucks. It combines lots of useful information into a compact, unreadable, hard-to-edit file. There are no tools to edit the lightmaps inside the .BSP file, nor are there tools to tweak the geometry without recompiling everything. It is a closed and unnecessarily obscure format.

Rather than pack everything into an useless "Ogre .BSP" format, I've decided to leave the components of the level in standard, readable formats. My tools can load a .MAP file and output a series of files representing geometry (.mesh or .obj), lightmaps (.tga or .jpg), and level information (.level -- a new format).

(Side note: I'm using some new techniques described in a 2003 academic paper to turn a complete BSP tree into a reasonable cell-and-portal system. Each cell is a .mesh, and portals are used to determine visibility. The algorithm does a decent job without any hint/occlusion brushes, but manual hints in the .MAP file are always better.)

I am NOT writing a tool to create Quake 3 or Half-Life .BSP files. The information in the .mesh + .tga + .level is equivalent to the information ina .BSP, so someone else could write such a tool -- but why would you want to?


And again, a reminder: because I am busy moving from Illinois to a new job in California, it might be over a month before I can release a working version.