'Tindalos' (Ogre v2.0) SceneManager redesign

[sinbad]

[edit]Split from Chaster's portal thread[/edit]
What you see on the wiki and what I've said in the forum so far is my current stage on Shoggoth - all musing and no code yet

I can't imagine that any of this is going to be completely obsolete, but there may be some repositioning required. In Shoggoth I'm going to try to solve a range of connected issues in one design, namely:

1. Large world coordinate precision
2. Combining scene managers
3. Paging

That means there will be a refactoring of the SceneManager to split scene organisation at the top level into 'Localities'. These will represent a chunk of the world expressed completely relative to a local origin, which may not be the 'real' origin. They will be the unit of world precision and also the unit of paging.

Clearly localities have to be connected to each other. Portals is one way (and I'm leaning towards calling them 'connectors' instead to differentiate them from internal lower-level portals), simple distance grids is another (e.g. an open world). The important thing is that these localilties are very coarse-grained and should not be considered similar to a cell in a CPG. There should be no more than a small handful in memory at one time. And they may interleave - for example if you have a portal based set of interiors for houses and an octree-based external terrain, there could be only 2 localities there, with all the houses in one and all the terrain in another, occupying the same physical space with connectors between them.

There are 2 things to consider when deciding which localities are important - visibility and loading. They won't use the same heuristics although they will use some of the same underlying information. For example loading is going to be entirely based on proximity - and that proximity may have to be measured via connectors. It really doesn't matter if you can see it or not, what matters is whether it's likely you'll be able to see it soon. So not a visibiliy calculation.

Visibility at the macro level is going to be based on either AABB (localities connected via grid distances) or connector visibility. The latter is similar to a portal obviously but again I want to differentiate them because they're major world segment boundaries and not detail portals.

Localities are owned and run by a single SceneManager, which will be delegated to once the collection of localities to be rendered (or loaded / unloaded) is known.

Transition of nodes and objects between localities and border conditions are very important concepts. I'm going to have to come up with something completely new here to manage transition from one locality to another and cross-locality queries especially when the two are using different scene managers. There will definitely be some SceneNode / MovableObject etc changes here.

I would imagine that your PCZ manager could be adapted just like the others to be a locality manager once the Shoggoth framework is in place. Internally the SceneManager probably won't operate fundamentally differently to how it does now, but there will be some additional protocols required and some adjustments in responsibilities to correctly manage itself under the new system.

So if you're happy with that, I say go ahead with an Eihort-compatible version and Shoggoth compatibility can come later.

[Chaster]

Hmm, there seems to be a fair amount of overlap between your musings and my implementations for my PCZ manager... If you don't mind Sinbad, perhaps we could expound a bit and do some constructive comparisons.. I think a discussion would be helpful to both our efforts...Perhaps I can my code can even be helpful for your work in Shoggoth...

First, I want to say that ANY implementation which is INTEGRAL to Ogre is going to be FAR superior than my implementation within the current SceneManager framework. I am doing my implementation because it's needed, and Ogre doesn't have it yet. Plain and Simple.

My manager is based on the concept of "zones" which would seem to be analogous to your "localities". Zones are simply volumes which contain scene nodes (and all the things associated with scene nodes) and are defined by bounding volumes (currently AABB but I see conditions which WILL call for "tighter" BV's like OBB or SB). The Zone does not impose any particular partioning scheme on the scene contained within it (i.e. could be Octree, could be BSP, could be non-partitioned, etc.). Zones can be associated with scene nodes - i.e. the zone can be "positioned" and oriented (rotated). Zones can contain other zones. So, for example, you could have a terrain zone which contains a building and within the building, each room is a zone. The building can be MOVING also.

However, my current implementation does not allow zones to cross into other zones (in other words, a subzone must be fully contained within its' parent zone). I have an idea which would allow zones to cross into other zones, but I will not attempt it until I have perfected the current implementation (with non-crossing zones). Note that neither of these implementations currently allows "Tardis-like" zone structure (where a subzone is bigger than it appears to its' parent zone - like Dr. Who's Tardis, for those geeks who are familiar with Dr. Who...)

The zones are connected by, yep, Portals. Portals are merely specialized MovableObjects modeled as planar quads (4 points and a normal) which have a reference to the connected zone. During visibility determination, when a Portal is found to be visible (currently, they use a spherical bounding volume check coupled with a directional comparison between the portal normal and the camera direction) then the connected zone is recursed into. During such visibility recursions, extra culling planes created from the edges of the portal and the origin of the camera are intelligently added to the modified Camera which utilizes these culling planes to do extra culling. These extra planes are removed when the visibility routine returns from the zone recursively, so things like two portals to the same zone being visible at the same time works correctly.

Object traversal in zones is taken care of with the following rules. Objects (or more accurately, the scene nodes they are attached to) have a home zone which they are associated with. The home zone is determined by the center point of the object (as determined by its' BV in derived space). The SM finds the lowest level (i.e. smallest) zone which contains the object's center point and makes that the home zone of the object. Since objects can straddle multiple zones, they can also be "visitors" to other zones. However, any given object can only have ONE home zone (which can change of course if the object moves OR the zone moves OR both move). Basically, if an object is touching a portal's bouding sphere, it is assumed that the object is visiting the connected zone and is added to that zone's list of objects as a visitor.

That's the basic gist of my scene manager (lots of details left out of course). Comparison to Shoggoth targets:

1) My manager does NOT address large world coordinate precision. My system currently depends on everything being physically coherent (because the system uses a straight forward position check to figure out which zone(s) an object is in. I *could* use a system which is dependent entirely on portal crossings to determine zone (thereby removing the physical coherency requirement), but I have done that in the past and found it to be very problematic.... (but perhaps my previous implementations were just not robust enough).

2) My system should allow for different partitioning schemes for each zone, but it will require reimplementation within the zone (probably as a derivation of the zone class). Not because I prefer it that way, but because my manager is a regular Ogre scene manager, not "integral" as the system would be in Shoggoth (which if you look waaaay back in this thread, I argued strongly for in the first place as the most desirable solution... heh heh.. Glad to hear all my ranting didn't go for nought! )

3) My system should be amenable to paging. Each Zone could be written to page in and out, and some sort of metric could be created based on Portal visibility probability (would most likely be the same or similar to any system implemented for Shoggoth).

BTW, I don't really care WHAT you call the object which connects zones (or "localities") together. Portals are just a term. People seem to make a lot of assumptions based on other implementations. True, my implementation does extra culling using the portal edges, but that is hardly a REQUIREMENT and it fits nicely with the standard view frustum culling done in Ogre anyway. Call 'em connectors or bridges or worm holes... Just get 'em in there!

Chaster
(Please excuse the rambling cohesiveness of my post.. it was written very non-linearly...)

[sinbad]

Yeah, you were clearly ahead of the curve there. For me personally, I don't like to leap too fast to design conclusions - I prefer to see what people do with things in the real world and try to abstract it all. I don't think I've had a good handle on a large enough breadth of apps until fairly recently to see how it all might click together, and that's why it's only in Shoggoth that I'm tackling these things.

I personally think separating the concept of localities and lower-level cell/portal constructs like your zone is important. That's partly because of the large world coordinate and paging issues (no point propagating all that logic down to every low-level zone for something that just doesn't fit finer-grained concepts), and partly because it's much clearer about responsibility. Localities, being a macro-level structure, are far less interested in visibility. It's a factor, but it's the most minor one in many ways - it's far more important to manage loading and paging. What is contained within the locality is far, far more interested in visibility and also other things like collision etc.

I personally think this separation helps to both add greater definition to the responsibilities of each domain and the restrictions help to make implementations more practical, particularly when contained localities may not be portal-based. Making the portal the 'king' of the organisation structure from macro-scale to micro-scale isn't something I personally want to do, because macro-scale there are other more suitable alternatives (e.g. I don't really want to represent outer space as cube-shaped cells with 6 portals in them, even though I could).

As I said, I think your approach will fit just fine into a new scene manager and the portal implementation may well influence my connector approach (for localities that are linked that way). For design reasons though I do want to keep a stronger dividing line between macro and micro scale structures and not try to use a portal structure to cover it all, I don't think it's a good fit for the complete range I want to cover, although it will be a good fit for some structures and will no doubt be one locality connector option.

Hope that makes sense. I'm not going to be able to get in to much more of a detailed design discussion here yet since Eihort is still the #1 focus and Shoggoth still in what I'd call 'arm waving stage'. Soon though.

[jwatte]

If scene managers overlap (portal manager plus octree manager), how will you deal with early-Z passes, or (even worse) transparency sorting? You'd have to sort individual draw calls between the different scene managers.

Anyway, if you refactor scene manager, I sincerely hope you break the various roles of the scene manager into separate components.

[Chaster]

Yeah, you were clearly ahead of the curve there. For me personally, I don't like to leap too fast to design conclusions - I prefer to see what people do with things in the real world and try to abstract it all. I don't think I've had a good handle on a large enough breadth of apps until fairly recently to see how it all might click together, and that's why it's only in Shoggoth that I'm tackling these things.

Sounds like a smart approach for Ogre to me. No complaints here.. =)

I personally think separating the concept of localities and lower-level cell/portal constructs like your zone is important. That's partly because of the large world coordinate and paging issues (no point propagating all that logic down to every low-level zone for something that just doesn't fit finer-grained concepts), and partly because it's much clearer about responsibility. Localities, being a macro-level structure, are far less interested in visibility. It's a factor, but it's the most minor one in many ways - it's far more important to manage loading and paging. What is contained within the locality is far, far more interested in visibility and also other things like collision etc.

After some thought, I agree. I was thinking about the stated goals of Shoggoth's system, and what you say makes a lot of sense. Large world coordinate accuracy and paging really aren't really related to local scene visibility determination (except in a very basic way) and the delineation between the management at the scene level and management at the "locality" level makes sense.

I personally think this separation helps to both add greater definition to the responsibilities of each domain and the restrictions help to make implementations more practical, particularly when contained localities may not be portal-based.

Agreed - in theory. We'll see how it all works out..

Making the portal the 'king' of the organisation structure from macro-scale to micro-scale isn't something I personally want to do, because macro-scale there are other more suitable alternatives (e.g. I don't really want to represent outer space as cube-shaped cells with 6 portals in them, even though I could).

Oh, sorry, my previous post wasn't written very well. I didn't mean to imply that portals should be "The Way" to connect multiple scenes. I just meant that there needs to be SOMETHING to connect them, and I didn't really care what they were named, just so long as they were there. =)

As I said, I think your approach will fit just fine into a new scene manager and the portal implementation may well influence my connector approach (for localities that are linked that way). For design reasons though I do want to keep a stronger dividing line between macro and micro scale structures and not try to use a portal structure to cover it all, I don't think it's a good fit for the complete range I want to cover, although it will be a good fit for some structures and will no doubt be one locality connector option.

Agreed.

Hope that makes sense. I'm not going to be able to get in to much more of a detailed design discussion here yet since Eihort is still the #1 focus and Shoggoth still in what I'd call 'arm waving stage'. Soon though.

Yes, it makes sense, but it still seems like Shoggoth's functionality will supercede the functionality of my scene manager...

[sinbad]

If scene managers overlap (portal manager plus octree manager), how will you deal with early-Z passes, or (even worse) transparency sorting? You'd have to sort individual draw calls between the different scene managers.

Yes, this is one of the challenges. At the most basic level it could be just that each SM contributes to an external render queue system which does a global sort as is done inside a single SM. Another option is to allow chunked sorting so that each SM / locality combination only sorts within itself, assuming a 2-pass top-level strategy (all solids then all transparencies), with a forward traversal sort between them for solids and a global reverse depth sort for transparencies. Configurability will be important but I don't see the strategies being hugely different to now, they don't have to be if you separate solids and transparencies.

Anyway, if you refactor scene manager, I sincerely hope you break the various roles of the scene manager into separate components.

It has crossed my mind - what separable roles would you personally define?

[Chaster]

If scene managers overlap (portal manager plus octree manager), how will you deal with early-Z passes, or (even worse) transparency sorting? You'd have to sort individual draw calls between the different scene managers.

Yes, this is one of the challenges. At the most basic level it could be just that each SM contributes to an external render queue system which does a global sort as is done inside a single SM. Another option is to allow chunked sorting so that each SM / locality combination only sorts within itself, assuming a 2-pass top-level strategy (all solids then all transparencies), with a forward traversal sort between them for solids and a global reverse depth sort for transparencies. Configurability will be important but I don't see the strategies being hugely different to now, they don't have to be if you separate solids and transparencies.

Those are the first 2 ideas that came to my mind... Not sure what specifically you are referring to with regards to early z-pass (many different things I can think of that utilize an early z-pass..)

Anyway, if you refactor scene manager, I sincerely hope you break the various roles of the scene manager into separate components.

It has crossed my mind - what separable roles would you personally define?

Visibility and rendering should always be completely separate passes. Octree Scene Manager (er, at least the version I last looked at) seems to meld these two..

[sinbad]

Visibility and rendering should always be completely separate passes. Octree Scene Manager (er, at least the version I last looked at) seems to meld these two..

Not really. It uses findVisibleObjects to populate the render queue and then the render queue is parsed out to the renderer separately, just like all the others. Perhaps you're thinking of BSPSceneManager which is the only one which does some manual rendering just based on the typical Q3A approach.

[Chaster]

Visibility and rendering should always be completely separate passes. Octree Scene Manager (er, at least the version I last looked at) seems to meld these two..

Not really. It uses findVisibleObjects to populate the render queue and then the render queue is parsed out to the renderer separately, just like all the others. Perhaps you're thinking of BSPSceneManager which is the only one which does some manual rendering just based on the typical Q3A approach.

Hm, to me, that is melding the two. But I see what you mean. For my scene manager, I have purposely separated determining visibility of objects in a scene and the process of adding them to the render queue in order to avoid certain issues which I anticipate when I implement the different zone managers...

[jwatte]

It has crossed my mind - what separable roles would you personally define?

Well, my post from 2005 (further up this thread :-) list the following:

1) manage mesh->entity objects
2) implement spatial hierarchy
3) do visibility culling
4) do state management/sorting
5) do pass management/sorting
6) load terrain data
7) generate terrain/heightmaps
8) manage sets of (lights,cameras,billboards,animations,entities)
9) do sky rendering

I could see all of those being separate roles, which got combined into a scene manager. Either by defining virtual interfaces and instantiating components, by using template inheritance from the different base classes, or some other composition method.

Ideally, you'd probably not even collect these into a "scene manager" grand class, but instead provide them as separate services (some of which depend on others). One strategy for doing that is the service factor or execution environment approach.

[jwatte]

If you separate "render system" from "scene manager" then you can move to a fully deferred render system, where the scene managers first issue geometry, and then the render system sorts it. Thus, transparency from two different scene managers are both in the same deferred queue, and sort together. You have to calculate the sort distance as part of issuing, or look it up in the transform matrix.

[xavier]

That's how Ogre works currently.

[jwatte]

Yes, that's what the render queue does. I was just suggesting to staying with a single queue when moving to multiple scene managers, rather than trying to interleave multiple queues.

[Chaster]

I favor the idea of a single shared render queue as well. However, I am wondering if that will somehow cause problems with multiple scene managers who utilize the queue differently. (I don't have any particular example in mind, I just worry about it...)

[sinbad]

Yes, this definitely works for transparent objects. However my point was that in the case where you want to do an early-Z pass (for solids), there can be a significant benefit to sorting by connectivity - this is particularly true in portal type systems. Therefore I was suggesting that whilst a shared render queue which operates just like the standard mode now would be fine, it would be worth having the option of being able to render solid objects in ascending order of connection traversal from the camera - both between localities and within localities that can benefit from that. Transparencies would be done after that in the traditional way.

For example, you could forward-connection sort solids at a top level, and then within each of these, render-state sort. This will give you good early-Z out results but still do good pass sorting within localities (and potentially within cells within a locality if that's a portal-based locality). So if you're inside a long thin building with windows on all sides for example, where just camera depth might mean that something outside the building is closer than something inside, solids are still sorted such that everything inside the building is rendered first to seed the depth buffer, then everything outside afterwards which will result in potentially more depth rejection than a simple pass sort or depth sort in edge cases.

[Chaster]

Makes sense. One question - would you want to do this 'split' renderqueue (one for solid, one for transparent) "behind the scenes" in the super manager or provide direct access to both queues to each locality manager?

I am thinking that some manager authors would prefer direct access, while others would be perfectly happy with having the super manager direct traffic behind the scenes...

Chaster

[sinbad]

My gut feeling says this sequence is within each queue group, as is the case now for all sorting. Each render queue group right now does a separate solids and transparencies sequence. Render queue groups are there to put 'firebreaks' between the automatic sorting (e.g. to do UIs last) and that should almost certainly continue.

In my experience with people writing custom scene managers, almost nobody drives the sequence direct. I allowed that as an option in the current design, basically because I didn't want to limit SceneManager writers at all, but pretty much nobody used it, they pretty much all just plugged into the standard render queue system. And the problem was that possibility that a SM might completely defy all conventions actually damaged extensibility, in hindsight, because there were things that you couldn't rely on - the protocol was too vaguely defined. In retrospect some additional limitations on what Sm writers could do would have been better, but that genie couldn't be put back in the bottle afterwards. Until now.

My goal with the new version is to have a flexible core management approach which can be configured, extended and plugged into, which can cover the major sequence variations but that there are key principles that always, always apply. This bedrock will lead to better future extensions, and probably better tool support, IMO. Provided I get it right The existing SM system didn't have a clear enough distinction between skeleton and meat, I plan to resolve that - part of that skeleton is the locality system and how to let SMs interoperate by setting clear protocols.

In my desperation to get away from one-scene-type systems I'd used / studied in the past, I went to the completely opposite extreme with the original SceneManager design - hey, do whatever you want! I realise now that somewhere in between is the correct place to be, and layering that using macro/micro scale designs and clear border controls is I think a key factor.

[Praetor]

It all sounds so complicated and wonderful. I'm getting visions of PLSM paging landscape in and out, and the super scene manager paging individual PLSMs in and out at the same time! Holy recursive paging batman. Switching from indoors to outdoors without loading screens, space sims with proper units even in galaxy-sized scenes.

I suppose the new structure would be a new kind of manager (we'll just call it UniverseManager for now) which is created from Root much like current scene managers are created from Root now. The difference being that there is only the one kind of UniverseManager. From UniverseManager you can create individual SceneManagers, which are now known as Localities. So, the Locality itself will need to be extensible, for something like Quadtree and Portal to sneak in there. There is a notion of a Connector I think this will need to be a class as well. Somehow the strategy of connecting Localities needs to be encapsulated and extended since you seem to suggest there should be multiple methods for doing it. That way you can do a simple grid of connectors with multiple types of portal scene managers, or portals connecting octree Localities, or whatever. So, now you have something of a super scene graph, with the vertices as Localities, and the edges as connectors. When rendering you traverse out from the current Locality and you'll have actually pass traversals through the edges themselves, so Connector sub-classes can work there individual magic on the traversal algorithm, then into the vertices for special processing.

How's that for a ramble? I just wanted to poke my head in and stir the pot of design discussion.

[Zbyl]

I've been thinking about this UniverseManager. The idea of localities and connectors can't be just layering one spatial algorithm (for UM) on the other (for SM's). It would be just a catch-22, not a solution.

Let's try to figure out what we would like the engine to do in the "corridor with windows" use-case Sinbad mentioned:

• - when we are inside the corridor, we don't want to traverse some UM's arbitrary spatial structure, but instead we want the corridor's PortalSM to "be in charge",
  - PSM "draws" the corridor, and then gives the command to whatever it is visible via the portals,
  - it is the terrain - a QuadtreeSM - and it "draws" the quads as it pleases,
  - on one of the quads there's another building, with another (or the same as before) PortalSM - what should we do now?

The generic QSM shouldn't have to be aware of the existence of anything like PSM, obviously. One possible solution would be to come up with an abstraction of all spatial structures, arbitrary nested, so that we can traverse all QuadZones, PortalZones, BSPZones etc. nested in each other in one uniform algorithm (i.e start a Dijkstra's seach on these objects). Nice idea, but that doesn't sound like a real-world, practical solution.

My idea is to not be that generic. But first let me quote Ogre's manual:

The application using OGRE does not have to know which SM subclasses are available. The application simply calls Root::createSceneManager(..) passing ST_GENERIC, ST_INTERIOR etc. OGRE will automatically use the best SceneManager subclass available for that scene type. This allows the developers of OGRE to add new scene specialisations later and thus optimise previously unoptimised scene types without the user applications having to change any code.

The last sentence is, in my opinion, not a real-world thing either. While creating SM we know exactly weather we want Octree, PSM or BSP. We care about what performance we can get now, and it is seriously doubtful that a scene optimised for the particular SM can do better on any other, even more modern, SM. But even if so - we don't care.

I think it should be made less generic. Let the SM's have custom interfaces for the spatial traversal, and let the Connectors be objects for communication between some two particular SM's, in some particular conditions.
Returning to the corridor scene:

• - on one of the quads there's another building, with another (or the same as before) PortalSM,
  - this particular quad instance knows how to interact with the PSM, so it tells it to "draw it's contents",
  - the quadtree traversal continues.

In this example that particular quad is a Connector. This Connector knows how to connect only QSM and PSM, and only in one direction (QSM->PSM).
Even more specific: this Connector knows how to connect a terrain quad with a building. Connecting QSM->PSM in other conditions could be different.
Moreover - the interface of the Connector is specific for a particular "home" SM. The example above would be a QuadtreeConnector. But QuadtreeConnector and PortalConnector wouldn't have a common base class. So "the Connector" becomes only a "concept", which is ok, since no one else but the "home" SM need to access the connector.

It is a flexible solution as in "it is possible to do all the stuff we want, and do it fast", but not as in "we can change one SM to another and it'll just work". But we really don't need the second thing.

What do you think about this?

(As a side note: please consider dropping the "ST_GENERIC, ST_INTERIOR..." idea.)

[jwatte]

It seems to me that you can still keep a bit of genericity. However, you will have to change all existing scene managers slightly.

If you can query each scene manager for "objects visible in the given frustum," and then also query for "other scene connectors visible in the given frustum" then you can tie scene managers arbitrarily together, as you'd just do a breadth-first search through whatever the current "main" scene manager is.

The draw-back is that this, by necessity, needs to look a little bit like a portal manager built-into each scene manager that wants to support connections to other scene managers. However, querying for objects-in-frustum is already an implemented function, and querying for objects-of-special-kind is also already an implemented function, so it's not that foreign.

Each camera then needs to know which scene manager its current position is "in," and rendering would start there.

Further, we would need to be able to move entities between scene managers, somehow. Re-creating a player each time he walks through a door does not seem feasible. One way of accomplishing this is to separate spatial management from object ownership, and make entities, meshes, etc be owned by an entity manager, who is a peer to the spatial scene managers.

[Praetor]

@Zbyl I get your point about questioning the need for SceneManager's being generic, but I don't quite see how your solution buys us anything. What I noticed is you've switched the paradigm from UniverseManager > Connectors > SceneManagers to UniverseManager > SceneManagers > Connectors. So now scene managers seem to hold specific connectors to other scene managers. But then, we'll still need an abstracted interface for the UniverseManager to interact with the SceneManager won't we?

[Zbyl]

But then, we'll still need an abstracted interface for the UniverseManager to interact with the SceneManager won't we?

In my idea the UniverseManager would be a manager for the SM's (their existence and transformation) and would act as a service for the Connectors.
The connectors would actually do the job transfering movables and cameras between SM's and transfering the command during spatial traversals. Why connectors? Because only they are close enough to the connected SM's to know how to do it - the generic UM would be too generic to be useful (probably...).

Now when it comes to transfering the movables between the SM's - I think it woudn't be that ineficient to just do a "shallow destruction" of it in one SM and then recreate it in the other SM. Naturally the SM's would need to be adapted to support this.

[Praetor]

So would the UM own movable objects/scene nodes, or would specific scene managers still own them?

[Zbyl]

The SM's would own them. The Connectors would destroy them in one SM and recreate in the other. The information how to do it would be in the UM (that is - the SM's transformation). But maybe this thing could be done in the UM like this: UM::moveMovable() -> SM::moveMovable() -> Connector::triggered() -> Conector::transferMovable() -> OtherSM::moveMovable().
The important thing here is that the Connector has to be native for the first SM. It could be a connector on the seam of two Quadtrees, or a connector between a terrain and a house. The interface exposed by the Connector to the first SM can't be generic, cause the possible triggers are not trivial and tightly bound to the SM's nature.
The interface UM->SM must be generic, obviously.
The interface Connector->OtherSM maybe could be generic, but if it would, some of the information would be lost.
How about a setup SM_0<->Connector_0<->Connector_1<->SM_1? In which only the middle interface would be generic? The information lost by the genericity can be partialy regained by customizing the second connector.

(PS.: maybe this should be a separate thread?)

[sinbad]

Ok, a number of ideas here. Firstly @Zbyl, I don't think we're completely at crossed purposes at all, but there's a significant thing that I'm not sure you've registered - I'm talking about 2 different topics here: loading and visibility. They're related, but they are not the same thing, and I probably haven't been clear enough about this.

The spatial organisation of the UM is mostly about paging, not visibility. It's simplistic and coarse-grained. Containment (ie which locality is an object in) is an issue, as is handover of that containment at boundaries. But all of that is more about macro world organisation and loading and not so much about visibility.

When you come to visibility, you of course have to start with the locality (and hence the SceneManager) which the camera is in. Visibility for that locality would have to be applied, and one of the core features of the new SceneManager design will be detecting external connections to other localities (which may be managed by different SMs or the same one) as part of that visibility pass, not just renderable objects. Clearly when passign to another locality you will have potentially 1 or 2 issues:

1. A different origin
2. A different scene management approach

The idea is that you cascade through the localities (and in practice it very unlikely for there to ever be more than a very small handful in visibility terms, and most often just one or two) using the SM you're exiting from to identify the need to link or not. This doesn't put any single SceneManager in charge, but lets each use it's own visibility terms to determine exiting or not (for portal based systems this will be a portal, for terrain it will probably be a simple range test or with occlusion maps).

The UM is really just there as an arbiter, a connection resolver and a paged loading system.

Objects will not be owned by the SM anymore. They can't be, because ownership has to be seamlessly passed between localities, each of which may be managed by a different Scene Manager. Paging of objects should be by containing locality and not by scene manager so the ownership issue has to be resolved. My current plan is to make scene objects reference counted and references passed between localities, such that when a locality is paged out, so are all the objects inside it. We have a small object allocator and more advanced memory pools planned for Shoggoth too which will mitigate any reallocation issues.

I'm not giving up on a simultaneously generic and specific scene management system yet. The key is to better graduate the roles involved such that generalisation / specialisation is better targetted. I think it's doable, I really do - I've learned a lot over the last few years since I set the generalisation as my goal and I'm ready for a second crack at it.

But yeah, the scene type enum can go

[edit]BTW a Locality != SceneManager instance. A Scenemanager can manage more than one Locality, the SM is the strategy, and the Locality is a world region instance - SM controls a Locality but they're not the same.