HLMS and storing additional data per mesh

[bishopnator]

I am trying to figure out how to use some additional data per instance. The current implementation uses only world matrix as instance data which is accessed in VS using drawId to reference it within the buffer. Let consider that i would like to store additional data for each mesh, but the size of the data is not constant - e.g. it contains a float4 for each triangle. In the GS I can use PrimitiveID to access the data for a particular triangle.

I am considering following management in HLMS:

• buffer for storing all custom instance data - like 10MB (read-only buffer) - BufferX (just working name to be able to refer to it later within the text)
• per instance, there is world matrix + offset into BufferX where the data for the instance begins

In fillBuffersForV2:

• fill world matrix + currently stored size of BufferX in the instance data buffer
• add data to BufferX from the renderable

In GS or PS, I can use:

Code: Select all

// 1. using drawId to access the offset in the buffer
// 2. using primitiveId (generated automaticall by IA) to access myValue within the buffer after applying bufferXOffset
const float4 primitiveData = bufferX[instanceData[drawId].bufferXOffset + primitiveId].myValue;

I see however that it is probably not optimal approach. On one side there will be huge fill every frame to prepare BufferX and actually the data stored in BufferX are same for all same meshes so instancing seems like a good idea. I will have VB+IB+BufferX for a single mesh, but how should I pack it together for multiple visible instances? The BufferX has nothing to do with number of stored vertices in VB, it is just a buffer which is accessed and somehow processed by GS or PS (according to PrimitiveID).

[bishopnator]

I have strong feeling, that it should work somehow straightforward. If I have a 3 different meshed (like box, cylinder and sphere), Ogre can store them in same VB + IB. It uses its internal magic to upload proper drawId (there will be internally an offset stored for box=0, cylinder=1, sphere=2 which is then used for converting InstanceID to drawId I think). If then also those additional data are stored in same buffer in the same order (box data, followed by cylinder data and then finally followed by sphere data), I should be able to just bind that buffer to GS and/or PS and as IA restarts PrimitveID to 0 for each instance, then it actually PrimitiveID will refer properly the correct data for a triangle without even using drawId.

If above is true, then how can I ensure that Ogre will store the data internally in the same buffer? From the app perspective, I create separate VB+IB+BufferX and it looks to the app as 3 separate stand-alone buffers. For VB+IB the Ogre takes special care, that they are properly packed together. But I think for the 3rd buffer, there is nothing - it can be created as sub-buffer within bigger buffer or not, depending on its size.

[dark_sylinc]

What you're asking sounds a lot like what we already do for skeletal animations.

Skeletal animations have the following properties:

1. Each submesh has a different number of bones that affect it
2. All sub-instances (Renderable) of the same sub-mesh share the same properties, but not the same data
3. When rendering all Items, we have to deal with a MovableObject that has multiple subinstances/submeshes; each with their own number of bones.
4. The RenderQueue should've sorted them by renderables (not by MovableObjects) so that all sub-instances are rendered together, but this is not guaranteed.
5. The world matrix data of all skeletons from all sub-meshes is stored in the same buffer. We don't have one buffer per skeleton or something like that. We may have more than one buffer if we run out of space; but not because we're putting them in categories or something.

HlmsPbs went for your idea of storing the offset to the start of the bone matrices and assumes 23 bits is enough for the offset:

Code: Select all

size_t distToWorldMatStart = static_cast<size_t>( mCurrentMappedTexBuffer - mStartMappedTexBuffer );
distToWorldMatStart >>= 2; // This is the same as distToWorldMatStart * 4; because we have float4, I think (either that, or it's sizeof(float))
*currentMappedConstBuffer = uint32( ( distToWorldMatStart << 9 ) | ( datablock->getAssignedSlot() & 0x1FF ) );

With that, it becomes possible to retrieve the offset in shader with:

Code: Select all

	uint matStart = worldMaterialIdx[inVs_drawId].x >> 9u;
	float4 worldMat[3];
	worldMat[0] = readOnlyFetch( worldMatBuf, int(matStart + _idx + 0u) );
	worldMat[1] = readOnlyFetch( worldMatBuf, int(matStart + _idx + 1u) );
	worldMat[2] = readOnlyFetch( worldMatBuf, int(matStart + _idx + 2u) );

You'd have to add the primitiveId, but the idea is similar.

However what remains is how to fill that buffer.

If you fill outside the draw loop, you have no guarantee the data will be in the order that OgreNext wants to draw it. You could disable renderqueue sorting, but there's still many things that could go wrong.
However you could leverage getCustomParameter and do:

Code: Select all

// Outside OgreNext's render loop:
size_t startOffset = fillData( buffer, subItem );
subItem->setCustomParameter( uniqueKey, Ogre::Vector4( startOffset, 0, 0, 0 ) ); // Beware conversion to float

// Inside HlmsMyPbs:
void HlmsMyPbs::fillBuffersFor( ... )
{
// I'm skipping the part where you check if the parameter exists
Vector4 offset = queuedRenderable.renderable->getCustomParameter( uniqueKey );
worldMaterialIdx[inVs_drawId].w = offset;
}

Your other choice is to fill the buffer every frame inside fillBuffersFor in the order they arrive (that's what skeleton animations does, but skeleton data is different for each sub item), but sounds like it'd be expensive for your case since I believe your data stays always the same for the same submesh.

Note: getCustomParameter is usually not the most efficient way of doing things. But most of OgreNext patterns are optimized to render a large number of instances of the same object, while you seem to be appear to require much more specialization per object; thus it sounds like performance hits of using getCustomParameter are the least of your concerns.

[dark_sylinc]

I have strong feeling, that it should work somehow straightforward. If I have a 3 different meshed (like box, cylinder and sphere), Ogre can store them in same VB + IB. It uses its internal magic to upload proper drawId (there will be internally an offset stored for box=0, cylinder=1, sphere=2 which is then used for converting InstanceID to drawId I think). If then also those additional data are stored in same buffer in the same order (box data, followed by cylinder data and then finally followed by sphere data), I should be able to just bind that buffer to GS and/or PS and as IA restarts PrimitveID to 0 for each instance, then it actually PrimitiveID will refer properly the correct data for a triangle without even using drawId.

If above is true, then how can I ensure that Ogre will store the data internally in the same buffer? From the app perspective, I create separate VB+IB+BufferX and it looks to the app as 3 separate stand-alone buffers. For VB+IB the Ogre takes special care, that they are properly packed together. But I think for the 3rd buffer, there is nothing - it can be created as sub-buffer within bigger buffer or not, depending on its size.

I forgot to comment in this: I simply didn't understand what you meant or what you have in mind.

From the app perspective, I create separate VB+IB+BufferX and it looks to the app as 3 separate stand-alone buffers. For VB+IB the Ogre takes special care, that they are properly packed together.

That is correct in principle, but memory blocks can be fragmented and reused, or maybe we ran out of space in a block and decided to allocate a new block.

Therefore it's possible that sphere=0 because memory got recycled, or sphere is in a different internal buffer because we ran out of space.

[bishopnator]

Let's assume that I want to prepare a buffer which holds a struct per triangle (for simplicity, let it be a float4 per triangle). The buffer will be created per combination VB+IB so in ideal case, I could store such buffer directly in VAO or somewhere next to the VAO. Without instancing, I would bind VB, IB and that new buffer and render a single object. In GS or PS I will be able to read the data using PrimitiveID as it corresponds to the triangles stored in the IB.

How does it change with instancing? If there is a single object type, I can do actually the same - the data accessed per triangle are accessed actually without using drawId as the PrimitiveID is always reset to 0 when new instance is drawn. In HLMS I can just track which buffer is currently bound and if input renderable use different buffer, I will just bind it. In RenderQueue I see that the instancing is broken if new Renderable has different VAO from the previously bound VAO so it could work, couldn't it? That way I can save copying of the data and preparing of the buffer(s). Do you see any flaws with such approach if there will be a lot of different objects?

note: I checked where it is possible to save such buffer and I didn't find a proper place. VertexArrayObject doesn't support to attach any custom data, nor Mesh/SubMesh. The best way seems to create a singleton "manager" where I can map VAO pointer to my buffer, but then there is a problem to properly clear the singleton - it should remove an entry from map when VAO is destroyed, but there is no listener for it in VAO manager.