game character customization, skeletal morph target

[slapin]

Hi, all!
For my games I need a character customization system. It should serve 2 purposes.

1. Cheaply make different characters different by not modelling every model manually and still have difference feel, not (I've seen another copy) feel.
2. Save on animations and not animate every character separately.

The most cheap approach is of course skeletal-based approach where we add additional leaf scale-only bones to the body
which in addition to a few mesh morphs will change the look. As we're careful enough that system allows to do much less modelling as
the bone scales change all meshes simultaneously which allows layered clothing on top. The problem with this approach as it makes extra bones which might affect both performance and limitations, expecially with embedded GPUs, but is really easy to implement from both code and art perspective.
Would be nice is if after the change is applied, the character would be frozen in more optimal state with extra bones removed.

Another approach I seen implemented was special morph targets... The idea is you have base character mesh which has also robe-like cloth added
and all your other mesh vertices are indexed by base mesh (each mesh vertex knows which vertex of base mesh it references). Same applies to skeleton,
which is converted to a set of joints and behaves like additional set of geometry in this approach. This way we create morph targets / blend shapes / shape keys for the base mesh and propagate these shapes to all other meshes allowing changes like baby - adult et al. Actually we produce mesh+skeleton morph targets here, which can be done in various ways. There are varying vetrtex indexing methods (UV-based, region-based, manual, etc. it is irrelevant) and as clothes are controlled by the same shapes it avoids manual creation of individual morph targets, which is desired.

For whatever approach I choose I need help with understanding how I can manipulate character meshes and skeleton in quite intrusive ways.

Would appreciate any pointers.

[slapin]

Now I can merge multiple animated meshes on submesh basis and get animations from multiple files. Then I need to understand how to change mesh itself...

[slapin]

Looks like the core of the system looks like

Code: Select all

static void getSubmeshNormals(const Ogre::Mesh *mesh,
			      const Ogre::SubMesh *submesh,
			      std::vector<Ogre::Vector3> &normals)
{
	int j;
	float *pReal;
	int vertex_count = 0;
	if (submesh->useSharedVertices)
		vertex_count += mesh->sharedVertexData->vertexCount;
	else
		vertex_count += submesh->vertexData->vertexCount;
	Ogre::HardwareVertexBufferSharedPtr vbuf;
	Ogre::VertexData *vertex_data = submesh->useSharedVertices ?
						mesh->sharedVertexData :
						submesh->vertexData;
	const Ogre::VertexElement *normalsElem =
		vertex_data->vertexDeclaration->findElementBySemantic(
			Ogre::VES_NORMAL);
	if (!normalsElem)
		return;
	OgreAssert(normals.size() == 0 || normals.size() == vertex_count,
		   "bad vertex count");
	normals.resize(vertex_count);
	vbuf = vertex_data->vertexBufferBinding->getBuffer(
		normalsElem->getSource());
	unsigned char *vertex = static_cast<unsigned char *>(
		vbuf->lock(Ogre::HardwareBuffer::HBL_READ_ONLY));
	for (j = 0; j < vertex_data->vertexCount;
	     ++j, vertex += vbuf->getVertexSize()) {
		normalsElem->baseVertexPointerToElement(vertex, &pReal);
		normals[j] = Ogre::Vector3(pReal[0], pReal[1], pReal[2]);
	}
	vbuf->unlock();
}
static void getSubmeshUVs(const Ogre::Mesh *mesh, const Ogre::SubMesh *submesh,
			  std::vector<Ogre::Vector2> &uvs, int index)
{
	int j;
	float *pReal;
	Ogre::HardwareVertexBufferSharedPtr vbuf;
	Ogre::VertexData *vertex_data = submesh->useSharedVertices ?
						mesh->sharedVertexData :
						submesh->vertexData;
	const Ogre::VertexElement *uvElem =
		vertex_data->vertexDeclaration->findElementBySemantic(
			Ogre::VES_TEXTURE_COORDINATES, index);
	int vertex_count = 0;
	if (submesh->useSharedVertices)
		vertex_count += mesh->sharedVertexData->vertexCount;
	else
		vertex_count += submesh->vertexData->vertexCount;
	if (!uvElem)
		return;
	OgreAssert(uvs.size() == 0 || uvs.size() == vertex_count,
		   "bad vertex count");
	uvs.resize(vertex_count);
	vbuf = vertex_data->vertexBufferBinding->getBuffer(uvElem->getSource());
	unsigned char *uv = static_cast<unsigned char *>(
		vbuf->lock(Ogre::HardwareBuffer::HBL_READ_ONLY));
	for (j = 0; j < vertex_data->vertexCount; ++j) {
		uvElem->baseVertexPointerToElement(uv, &pReal);
		uvs[j] = Ogre::Vector2(pReal[0], pReal[1]);
		uv += vbuf->getVertexSize();
	}
	vbuf->unlock();
}
static void getSubmeshVertices(const Ogre::Mesh *mesh,
			       const Ogre::SubMesh *submesh,
			       std::vector<Ogre::Vector3> &vertices)
{
	int j;
	float *pReal;
	int vertex_count = 0;
	if (submesh->useSharedVertices)
		vertex_count += mesh->sharedVertexData->vertexCount;
	else
		vertex_count += submesh->vertexData->vertexCount;
	Ogre::HardwareVertexBufferSharedPtr vbuf;
	Ogre::VertexData *vertex_data = submesh->useSharedVertices ?
						mesh->sharedVertexData :
						submesh->vertexData;
	const Ogre::VertexElement *posElem =
		vertex_data->vertexDeclaration->findElementBySemantic(
			Ogre::VES_POSITION);
	if (!posElem)
		return;
	OgreAssert(vertices.size() == 0 || vertices.size() == vertex_count,
		   "bad vertex count");
	vertices.resize(vertex_count);
	vbuf = vertex_data->vertexBufferBinding->getBuffer(
		posElem->getSource());
	unsigned char *vertex = static_cast<unsigned char *>(
		vbuf->lock(Ogre::HardwareBuffer::HBL_READ_ONLY));
	for (j = 0; j < vertex_data->vertexCount;
	     ++j, vertex += vbuf->getVertexSize()) {
		posElem->baseVertexPointerToElement(vertex, &pReal);
		vertices[j] = Ogre::Vector3(pReal[0], pReal[1], pReal[2]);
	}
	vbuf->unlock();
}
static void getSubmeshIndices(const Ogre::Mesh *mesh,
			      const Ogre::SubMesh *submesh,
			      std::vector<unsigned long> &indices)
{
	int index_count = 0;
	index_count += submesh->indexData->indexCount;
	int index_offset = 0;
	indices.resize(index_count);
	Ogre::IndexData *index_data = submesh->indexData;
	size_t numTris = index_data->indexCount / 3;
	Ogre::HardwareIndexBufferSharedPtr ibuf = index_data->indexBuffer;

bool use32bitindexes =
	(ibuf->getType() == Ogre::HardwareIndexBuffer::IT_32BIT);

unsigned long *pLong = static_cast<unsigned long *>(
	ibuf->lock(Ogre::HardwareBuffer::HBL_READ_ONLY));
unsigned short *pShort = reinterpret_cast<unsigned short *>(pLong);

size_t offset = 0;

if (use32bitindexes) {
	for (size_t k = 0; k < numTris * 3; ++k) {
		indices[index_offset++] =
			pLong[k] + static_cast<unsigned long>(offset);
	}
} else {
	for (size_t k = 0; k < numTris * 3; ++k) {
		indices[index_offset++] =
			static_cast<unsigned long>(pShort[k]) +
			static_cast<unsigned long>(offset);
	}
}

ibuf->unlock();
}
struct SubMeshInformation {
	Ogre::String materialName;
	bool sharedVertices;
	std::vector<Ogre::Vector3> vertices;
	std::vector<Ogre::Vector3> normals;
	std::vector<Ogre::Vector2> uvs;
	std::vector<Ogre::Vector2> uv2s;
	std::vector<unsigned long> indices;
	Ogre::SubMesh::LODFaceList lodFaceList;
	Ogre::Mesh::VertexBoneAssignmentList boneList;
	void createSubmesh(Ogre::Mesh *mesh)
	{
		int i;
		Ogre::SubMesh *out = mesh->createSubMesh();
		out->useSharedVertices = false;
		out->vertexData = new Ogre::VertexData();
		size_t currOffset = 0;
		Ogre::VertexDeclaration *vertexDecl =
			out->vertexData->vertexDeclaration;
		vertexDecl->addElement(0, currOffset, Ogre::VET_FLOAT3,
				       Ogre::VES_POSITION);
		currOffset +=
			Ogre::VertexElement::getTypeSize(Ogre::VET_FLOAT3);
		vertexDecl->addElement(0, currOffset, Ogre::VET_FLOAT3,
				       Ogre::VES_NORMAL);
		currOffset +=
			Ogre::VertexElement::getTypeSize(Ogre::VET_FLOAT3);
		vertexDecl->addElement(0, currOffset, Ogre::VET_FLOAT2,
				       Ogre::VES_TEXTURE_COORDINATES, 0);
		currOffset +=
			Ogre::VertexElement::getTypeSize(Ogre::VET_FLOAT2);
		out->vertexData->vertexCount = vertices.size();
		Ogre::HardwareVertexBufferSharedPtr vbuf =
			Ogre::HardwareBufferManager::getSingleton()
				.createVertexBuffer(
					vertexDecl->getVertexSize(0),
					out->vertexData->vertexCount,
					Ogre::HardwareBuffer::
						HBU_STATIC_WRITE_ONLY, // only GPU side
					false);
		Ogre::VertexBufferBinding *binding =
			out->vertexData->vertexBufferBinding;
		binding->setBinding(0, vbuf);
		float *pvertex = static_cast<float *>(
			vbuf->lock(Ogre::HardwareBuffer::HBL_DISCARD));
		out->indexData->indexCount = indices.size();
		out->indexData->indexBuffer =
			Ogre::HardwareBufferManager::getSingleton().createIndexBuffer(
				vertices.size() < 32768 ?
					Ogre::HardwareIndexBuffer::IT_16BIT :
					Ogre::HardwareIndexBuffer::IT_32BIT,
				out->indexData->indexCount,
				Ogre::HardwareBuffer::HBU_STATIC_WRITE_ONLY,
				false);
		OgreAssert(normals.size() == vertices.size(), "bad normals");
		for (i = 0; i < vertices.size(); i++) {
			*pvertex++ = vertices[i].x;
			*pvertex++ = vertices[i].y;
			*pvertex++ = vertices[i].z;
			if (normals.size() > 0) {
				*pvertex++ = normals[i].x;
				*pvertex++ = normals[i].y;
				*pvertex++ = normals[i].z;
			} else {
				*pvertex++ = 0.0f;
				*pvertex++ = 1.0f;
				*pvertex++ = 0.0f;
			}
			if (uvs.size() > 0) {
				*pvertex++ = uvs[i].x;
				*pvertex++ = uvs[i].y;
			} else {
				*pvertex++ = 0.0f;
				*pvertex++ = 1.0f;
			}
		}
		vbuf->unlock();
		Ogre::HardwareIndexBufferSharedPtr ibuf =
			out->indexData->indexBuffer;
		if (vertices.size() < 32768) {
			unsigned short *pindices =
				static_cast<unsigned short *>(ibuf->lock(
					Ogre::HardwareBuffer::HBL_DISCARD));
			for (i = 0; i < indices.size(); i++)
				*pindices++ = (unsigned short)indices[i];
			ibuf->unlock();
		} else {
			unsigned long *pindices = static_cast<unsigned long *>(
				ibuf->lock(Ogre::HardwareBuffer::HBL_DISCARD));
			for (i = 0; i < indices.size(); i++)
				*pindices++ = indices[i];
			ibuf->unlock();
		}
		out->clearBoneAssignments();
		auto vbass_it = boneList.begin();
		while (vbass_it != boneList.end()) {
			out->addBoneAssignment(vbass_it->second);
			vbass_it++;
		}
		out->setMaterialName(materialName);
	}
};

static void getSubMeshInformation(const Ogre::Mesh *mesh,
				  const Ogre::SubMesh *submesh,
				  struct SubMeshInformation *info)
{
	info->materialName = submesh->getMaterialName();
	info->sharedVertices = submesh->useSharedVertices;
	info->lodFaceList = submesh->mLodFaceList;
	info->boneList = submesh->getBoneAssignments();
	getSubmeshIndices(mesh, submesh, info->indices);
	getSubmeshVertices(mesh, submesh, info->vertices);
	getSubmeshNormals(mesh, submesh, info->normals);
	getSubmeshUVs(mesh, submesh, info->uvs, 0);
	getSubmeshUVs(mesh, submesh, info->uv2s, 1);
}

struct MeshInformation {
	std::vector<struct SubMeshInformation> submesh;
	Ogre::SkeletonPtr skelp;
	Ogre::String skelName;
	MeshInformation(Ogre::Mesh *prototype)
	{
		int i;
		skelName = prototype->getSkeletonName();
		skelp = Ogre::SkeletonManager::getSingleton().getByName(
			prototype->getSkeletonName(), "Characters");
		OgreAssert(skelp, "Could not load skeleton " +
					  prototype->getSkeletonName());
		submesh.resize(prototype->getNumSubMeshes());
		for (i = 0; i < submesh.size(); i++) {
			getSubMeshInformation(prototype,
					      prototype->getSubMesh(i),
					      &submesh[i]);
			std::cout << i << " " << "mesh material name: "
				  << submesh[i].materialName << std::endl;
			std::cout << i << " "
				  << "shared: " << submesh[i].sharedVertices
				  << std::endl;
			std::cout << i << " " << "vertex_count: "
				  << submesh[i].vertices.size() << std::endl;
			std::cout << i << " " << "index_count: "
				  << submesh[i].indices.size() << std::endl;
		}
	}
	Ogre::MeshPtr create(const Ogre::String &meshName)
	{
		int j;
		Ogre::MeshPtr baseMesh =
			Ogre::MeshManager::getSingleton().createManual(
				meshName, Ogre::ResourceGroupManager::
						  DEFAULT_RESOURCE_GROUP_NAME);
		for (j = 0; j < submesh.size(); j++)
			submesh[j].createSubmesh(baseMesh.get());
		//define a extreme boundary values
		Ogre::Real max_x = -1e+8;
		Ogre::Real min_x = 1e+8;
		Ogre::Real max_y = -1e+8;
		Ogre::Real min_y = 1e+8;
		Ogre::Real max_z = -1e+8;
		Ogre::Real min_z = +1e+8;
		// Setting bounding box
		Ogre::Mesh::SubMeshList mlist = baseMesh->getSubMeshes();
		for (j = 0; j < mlist.size(); j++) {
			Ogre::SubMesh *in = mlist[j];
			Ogre::VertexData *vertex_data = in->vertexData;
			const Ogre::VertexElement *posElem =
				vertex_data->vertexDeclaration
					->findElementBySemantic(
						Ogre::VES_POSITION);
			Ogre::HardwareVertexBufferSharedPtr hwvb =
				in->vertexData->vertexBufferBinding->getBuffer(
					posElem->getSource());
			unsigned char *hbuff =
				static_cast<unsigned char *>(hwvb->lock(
					Ogre::HardwareBuffer::HBL_READ_ONLY));
			Ogre::Real *pValue;
			Ogre::Real value;

		for (size_t idx = 0; idx < vertex_data->vertexCount;
		     ++idx, hbuff += hwvb->getVertexSize()) {
			posElem->baseVertexPointerToElement(hbuff,
							    &pValue);
			value = (*pValue++);
			if (value > max_x)
				max_x = value;
			if (value < min_x)
				min_x = value;
			value = (*pValue++);

			if (value > max_y)
				max_y = value;
			if (value < min_y)
				min_y = value;
			value = (*pValue++);

			if (value > max_z)
				max_z = value;
			if (value < min_z)
				min_z = value;
		}
		hwvb->unlock();
	}
	baseMesh->setSkeletonName(skelName);
	baseMesh->_setBounds(Ogre::AxisAlignedBox(min_x, min_y, min_z,
						  max_x, max_y, max_z));
	return baseMesh;
}
};
[/code[

which can be used like
[code]
		Ogre::MeshPtr base = Ogre::MeshManager::getSingleton().load(
			"male/normal-male.glb", "Characters");
		MeshInformation meshinfo(base.get());
		Ogre::MeshPtr baseMesh = meshinfo.create("UpdatedCharacter");
		Ogre::Entity *ent =
			mScnMgr->createEntity("entName", m_BaseMesh->getName());
		Ogre::SceneNode *sceneNode =
			mScnMgr->getRootSceneNode()->createChildSceneNode();
		sceneNode->attachObject(ent);

The character has all buffers completely re-created and animations work. Next I need vertices modification...

[slapin]

Made vertex morph work even though it is slow (will handle it later), now I need to make skeleton morph, i.e. I need to be able to edit skeleton rest pose...

[slapin]

Made vertex-order independent geometry morph work.Time to handle skeleton...