Forcing (procedural) terrain update

[slapin]

Hi, all!
I have procedural paged terrain with custom definer which works perfectly well except for 2 things:

1. If I teleport quickly with camera (in custom editor) I see some chunks looking strange. That is fixed by terrainGroup->updateGeometry() several times. I don't think that this fix is correct...
2. If I edit my height map and update image (where height map data are stored) and then call

   Code: Select all

   terrainGroup->updateDerivedData();
   terrainGroup->updateGeometry();
   terrainGroup->update();
   

   nothing happens. However if I teleport somewhere else and return back, the modification is visible. How can I handle this better?
   The best of course would be to see visual feedback from terrain editing as quickly and as smooth as possible.

The code I use to update height map:

Code: Select all

	void updateHeightmap()
	{
		/* this iust updates image data in definer, Ogre::Image worldMapImage  */
		TerrainModule::update_heightmap(worldMapImage);
		auto slots =
			ECS::get<Terrain>().mTerrainGroup->getTerrainSlots();
		/* trying to dirty terrains and not crash */
		for (auto slot : slots) {
			if (slot.second && slot.second->instance &&
			    slot.second->instance->isLoaded())
				slot.second->instance->dirty();
		}

	ECS::get<Terrain>().mTerrainGroup->updateDerivedData();
	ECS::get<Terrain>().mTerrainGroup->updateGeometry();
	ECS::get<Terrain>().mTerrainGroup->update();
}

[slapin]

@paroj sorry for the ping but I would appreciate the hint as it is very important to me...

[paroj]

try to place a breakpoint here:
https://ogrecave.github.io/ogre/api/lat ... 7d7d325f68

and see when/ if it is called, depending on your two scenarios.

[slapin]

dirty() itself runs fine every time, probably some process somewhere does not run.
if I do update(true) from ImGui button, several (2-3) presses to the button fix the terrain.

[slapin]

Looks like LODs are never updated in my case for some reason unless I run mTerrainGroup->update(true) periodically.
Added that to once per second. Periodic running seems to always fix the problem as single runs after teleportation do not work.
Looks like autimagic is broken for me and I broke my brain figuring the values out. It just never update unless I run update()

[slapin]

Also it looks like directly updating terrain once loaded is impossible and full unload and moving out, update, going back, update are required in sequence

[slapin]

Well, there is one way - to manually define terrain via paged world terrain definer, but it is crashing later.

[slapin]

Well, the crash is due to conflict with background thread. It looks like it tries to do something in this case, but not what I need...

[slapin]

Looks like the only way to go is to re-define terrain somehow via PagedWorld as otherwise nothing works at all.
Running definer from outside produces zillion of background tasks and crash so I guess it must be done in order via PagedWorld somehow...

[paroj]

Looks like LODs are never updated in my case for some reason unless I run mTerrainGroup->update(true) periodically.
Added that to once per second. Periodic running seems to always fix the problem as single runs after teleportation do not work.
Looks like autimagic is broken for me and I broke my brain figuring the values out. It just never update unless I run update()

note that the composite map is updated only after 2s by default:
https://ogrecave.github.io/ogre/api/lat ... 92f81ede4a

and updateDerivedData is async
https://ogrecave.github.io/ogre/api/lat ... a66673ef00

[slapin]

well, unloading page then loading page leads to the same crash in definer

Code: Select all

		for (auto &slot :
		     ECS::get<Terrain>().mTerrainGroup->getTerrainSlots()) {
			Ogre::uint32 page =
				ECS::get<Terrain>().mTerrainGroup->packIndex(
					slot.second->x, slot.second->y);
			ECS::get<Terrain>()
				.mTerrainPagedWorldSection->unloadPage(page,
								       true);
			ECS::get<Terrain>().mTerrainPagedWorldSection->loadPage(
				page, true);
		}
		ECS::get<Terrain>().mTerrainGroup->loadAllTerrains();
		ECS::get<Terrain>().mTerrainGroup->update(true);

Code: Select all

__memmove_avx_unaligned_erms () at ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:708
708	../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S: No such file or directory.
(gdb) bt
#0  __memmove_avx_unaligned_erms () at ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:708
#1  0x00007ffff7da179e in memcpy (__len=<optimized out>, __src=0x7fffe8063420, __dest=<optimized out>) at /usr/include/x86_64-linux-gnu/bits/string_fortified.h:29
#2  Ogre::TerrainGroup::defineTerrain (this=0x555556981b00, x=<optimized out>, y=<optimized out>, pFloat=0x7fffe8063420, layers=0x0)
    at /media/slapin/library/ogre3/ogre/Components/Terrain/src/OgreTerrainGroup.cpp:204
#3  0x000055555562b58d in ECS::FlatTerrainDefiner::define (this=0x55555714c1e0, terrainGroup=0x555556981b00, x=21, y=51) at /media/slapin/library/ogre3/ogre-projects/world2/src/gamedata/TerrainModule.cpp:310
#4  0x00007ffff7dbdae3 in Ogre::TerrainPagedWorldSection::handleRequest (req=<optimized out>, srcQ=<optimized out>, this=0x555556cb7b90)
    at /media/slapin/library/ogre3/ogre/Components/Terrain/src/OgreTerrainPagedWorldSection.cpp:302
#5  Ogre::TerrainPagedWorldSection::handleRequest (this=0x555556cb7b90, req=<optimized out>, srcQ=<optimized out>) at /media/slapin/library/ogre3/ogre/Components/Terrain/src/OgreTerrainPagedWorldSection.cpp:278
#6  0x00007ffff7dbdc0b in operator() (__closure=<optimized out>) at /media/slapin/library/ogre3/ogre/Components/Terrain/src/OgreTerrainPagedWorldSection.cpp:326
#7  std::__invoke_impl<void, Ogre::TerrainPagedWorldSection::handleResponse(const Ogre::WorkQueue::Response*, const Ogre::WorkQueue*)::<lambda()>&> (__f=...) at /usr/include/c++/11/bits/invoke.h:61
#8  std::__invoke_r<void, Ogre::TerrainPagedWorldSection::handleResponse(const Ogre::WorkQueue::Response*, const Ogre::WorkQueue*)::<lambda()>&> (__fn=...) at /usr/include/c++/11/bits/invoke.h:154
#9  std::_Function_handler<void(), Ogre::TerrainPagedWorldSection::handleResponse(const Ogre::WorkQueue::Response*, const Ogre::WorkQueue*)::<lambda()> >::_M_invoke(const std::_Any_data &) (__functor=...)
    at /usr/include/c++/11/bits/std_function.h:290
#10 0x00007ffff7752b53 in std::function<void ()>::operator()() const (this=0x7fffefba69b0) at /usr/include/c++/11/bits/std_function.h:590
#11 Ogre::DefaultWorkQueueBase::_processNextRequest (this=0x5555562227b0) at /media/slapin/library/ogre3/ogre/OgreMain/src/OgreWorkQueue.cpp:157
#12 0x00007ffff7569b65 in Ogre::DefaultWorkQueue::_threadMain (this=0x5555562227b0) at /media/slapin/library/ogre3/ogre/OgreMain/src/OgreDefaultWorkQueueStandard.cpp:169
#13 0x00007ffff70e62b3 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#14 0x00007ffff6c94ac3 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#15 0x00007ffff6d268c0 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

but it looks like it might work eventually
At least I see something happenning...

[slapin]

Code: Select all

#0  0x00007ffff6a8ff29 in ?? () from /lib/x86_64-linux-gnu/libGLdispatch.so.0
#1  0x00007ffff4900cb5 in Ogre::GLES2Texture::freeInternalResourcesImpl (this=0x5555682dce20) at /media/slapin/library/ogre3/ogre/RenderSystems/GLES2/src/OgreGLES2Texture.cpp:314
#2  0x00007ffff7733965 in Ogre::Texture::freeInternalResources (this=0x5555682dce20) at /media/slapin/library/ogre3/ogre/OgreMain/src/OgreTexture.cpp:343
#3  Ogre::Texture::freeInternalResources (this=0x5555682dce20) at /media/slapin/library/ogre3/ogre/OgreMain/src/OgreTexture.cpp:338
#4  0x00007ffff7677e4b in Ogre::Resource::unload (this=0x5555682dce20) at /media/slapin/library/ogre3/ogre/OgreMain/src/OgreResource.cpp:348
#5  0x00007ffff4900e42 in Ogre::GLES2Texture::~GLES2Texture (this=0x5555682dce20, __in_chrg=<optimized out>) at /media/slapin/library/ogre3/ogre/RenderSystems/GLES2/src/OgreGLES2Texture.cpp:51
#6  0x00007ffff490101d in Ogre::GLES2Texture::~GLES2Texture (this=0x5555682dce20, __in_chrg=<optimized out>) at /media/slapin/library/ogre3/ogre/RenderSystems/GLES2/src/OgreGLES2Texture.cpp:52
#7  0x00007ffff7743020 in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release (this=0x55556d44f660) at /usr/include/c++/11/bits/shared_ptr_base.h:168
#8  std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count (this=0x5555691af708, __in_chrg=<optimized out>) at /usr/include/c++/11/bits/shared_ptr_base.h:705
#9  std::__shared_ptr<Ogre::Texture, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr (this=0x5555691af700, __in_chrg=<optimized out>) at /usr/include/c++/11/bits/shared_ptr_base.h:1154
#10 std::shared_ptr<Ogre::Texture>::~shared_ptr (this=0x5555691af700, __in_chrg=<optimized out>) at /usr/include/c++/11/bits/shared_ptr.h:122
#11 Ogre::SharedPtr<Ogre::Texture>::~SharedPtr (this=0x5555691af700, __in_chrg=<optimized out>) at /media/slapin/library/ogre3/ogre/OgreMain/include/OgreSharedPtr.h:44
#12 std::_Destroy<Ogre::SharedPtr<Ogre::Texture> > (__pointer=0x5555691af700) at /usr/include/c++/11/bits/stl_construct.h:151
#13 std::_Destroy_aux<false>::__destroy<Ogre::SharedPtr<Ogre::Texture>*> (__last=0x5555691af710, __first=0x5555691af700) at /usr/include/c++/11/bits/stl_construct.h:163
#14 std::_Destroy<Ogre::SharedPtr<Ogre::Texture>*> (__last=0x5555691af710, __first=<optimized out>) at /usr/include/c++/11/bits/stl_construct.h:196
#15 std::_Destroy<Ogre::SharedPtr<Ogre::Texture>*, Ogre::SharedPtr<Ogre::Texture> > (__last=0x5555691af710, __first=<optimized out>) at /usr/include/c++/11/bits/alloc_traits.h:848
#16 std::vector<Ogre::SharedPtr<Ogre::Texture>, std::allocator<Ogre::SharedPtr<Ogre::Texture> > >::~vector (this=0x5555708e6fe0, __in_chrg=<optimized out>) at /usr/include/c++/11/bits/stl_vector.h:680
#17 Ogre::TextureUnitState::~TextureUnitState (this=0x5555708e6ef0, __in_chrg=<optimized out>) at /media/slapin/library/ogre3/ogre/OgreMain/src/OgreTextureUnitState.cpp:174
#18 0x00007ffff76324d8 in Ogre::Pass::removeAllTextureUnitStates (this=this@entry=0x555557166ce0) at /media/slapin/library/ogre3/ogre/OgreMain/src/OgrePass.cpp:465
#19 0x00007ffff763256f in Ogre::Pass::queueForDeletion (this=0x555557166ce0) at /media/slapin/library/ogre3/ogre/OgreMain/src/OgrePass.cpp:1049
#20 0x00007ffff772ddac in Ogre::Technique::removeAllPasses (this=this@entry=0x55556b7013b0) at /media/slapin/library/ogre3/ogre/OgreMain/src/OgreTechnique.cpp:344
#21 0x00007ffff772e8e0 in Ogre::Technique::~Technique (this=0x55556b7013b0, __in_chrg=<optimized out>) at /media/slapin/library/ogre3/ogre/OgreMain/src/OgreTechnique.cpp:49
#22 0x00007ffff75da2b8 in Ogre::Material::removeAllTechniques (this=this@entry=0x555569a3e290) at /media/slapin/library/ogre3/ogre/OgreMain/src/OgreMaterial.cpp:368
#23 0x00007ffff75da31b in Ogre::Material::~Material (this=0x555569a3e290, __in_chrg=<optimized out>) at /media/slapin/library/ogre3/ogre/OgreMain/src/OgreMaterial.cpp:69
#24 0x00007ffff75da3dd in Ogre::Material::~Material (this=0x555569a3e290, __in_chrg=<optimized out>) at /media/slapin/library/ogre3/ogre/OgreMain/src/OgreMaterial.cpp:73
#25 0x00007ffff7d9b08a in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release (this=0x55555ad7d8d0) at /usr/include/c++/11/bits/shared_ptr_base.h:168
#26 0x00007ffff7d9094c in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count (this=<optimized out>, __in_chrg=<optimized out>) at /usr/include/c++/11/bits/shared_ptr_base.h:705
#27 std::__shared_ptr<Ogre::Material, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr (this=<optimized out>, __in_chrg=<optimized out>) at /usr/include/c++/11/bits/shared_ptr_base.h:1154
#28 std::__shared_ptr<Ogre::Material, (__gnu_cxx::_Lock_policy)2>::reset (this=0x55555e376b28) at /usr/include/c++/11/bits/shared_ptr_base.h:1272
#29 Ogre::Terrain::freeGPUResources (this=this@entry=0x55555e3769e0) at /media/slapin/library/ogre3/ogre/Components/Terrain/src/OgreTerrain.cpp:1986
#30 0x00007ffff7d96b69 in Ogre::Terrain::~Terrain (this=0x55555e3769e0, __in_chrg=<optimized out>) at /media/slapin/library/ogre3/ogre/Components/Terrain/src/OgreTerrain.cpp:215
#31 0x00007ffff7d9709d in Ogre::Terrain::~Terrain (this=0x55555e3769e0, __in_chrg=<optimized out>) at /media/slapin/library/ogre3/ogre/Components/Terrain/src/OgreTerrain.cpp:222
#32 0x00007ffff7d9fb67 in Ogre::TerrainGroup::TerrainSlot::freeInstance (this=0x7fffe003d8c0) at /media/slapin/library/ogre3/ogre/Components/Terrain/src/OgreTerrainGroup.cpp:1081
#33 0x00007ffff7d9fd02 in Ogre::TerrainGroup::freeTerrainSlotInstance (this=this@entry=0x555556981d70, slot=slot@entry=0x7fffe003d8c0)
    at /media/slapin/library/ogre3/ogre/Components/Terrain/src/OgreTerrainGroup.cpp:883
#34 0x00007ffff7da1663 in Ogre::TerrainGroup::defineTerrain (this=0x555556981d70, x=<optimized out>, y=<optimized out>, pFloat=0x7fffe80420f0, layers=0x0)
    at /media/slapin/library/ogre3/ogre/Components/Terrain/src/OgreTerrainGroup.cpp:194
#35 0x000055555562b58d in ECS::FlatTerrainDefiner::define (this=0x55555714c870, terrainGroup=0x555556981d70, x=-20, y=52) at /media/slapin/library/ogre3/ogre-projects/world2/src/gamedata/TerrainModule.cpp:310
#36 0x00007ffff7dbdae3 in Ogre::TerrainPagedWorldSection::handleRequest (req=<optimized out>, srcQ=<optimized out>, this=0x555556cb8540)
    at /media/slapin/library/ogre3/ogre/Components/Terrain/src/OgreTerrainPagedWorldSection.cpp:302
#37 Ogre::TerrainPagedWorldSection::handleRequest (this=0x555556cb8540, req=<optimized out>, srcQ=<optimized out>) at /media/slapin/library/ogre3/ogre/Components/Terrain/src/OgreTerrainPagedWorldSection.cpp:278
#38 0x00007ffff7dbdc0b in operator() (__closure=<optimized out>) at /media/slapin/library/ogre3/ogre/Components/Terrain/src/OgreTerrainPagedWorldSection.cpp:326
#39 std::__invoke_impl<void, Ogre::TerrainPagedWorldSection::handleResponse(const Ogre::WorkQueue::Response*, const Ogre::WorkQueue*)::<lambda()>&> (__f=...) at /usr/include/c++/11/bits/invoke.h:61
#40 std::__invoke_r<void, Ogre::TerrainPagedWorldSection::handleResponse(const Ogre::WorkQueue::Response*, const Ogre::WorkQueue*)::<lambda()>&> (__fn=...) at /usr/include/c++/11/bits/invoke.h:154
#41 std::_Function_handler<void(), Ogre::TerrainPagedWorldSection::handleResponse(const Ogre::WorkQueue::Response*, const Ogre::WorkQueue*)::<lambda()> >::_M_invoke(const std::_Any_data &) (__functor=...)
    at /usr/include/c++/11/bits/std_function.h:290
#42 0x00007ffff7752b53 in std::function<void ()>::operator()() const (this=0x7fffefba69b0) at /usr/include/c++/11/bits/std_function.h:590
#43 Ogre::DefaultWorkQueueBase::_processNextRequest (this=0x5555562227b0) at /media/slapin/library/ogre3/ogre/OgreMain/src/OgreWorkQueue.cpp:157
#44 0x00007ffff7569b65 in Ogre::DefaultWorkQueue::_threadMain (this=0x5555562227b0) at /media/slapin/library/ogre3/ogre/OgreMain/src/OgreDefaultWorkQueueStandard.cpp:169
#45 0x00007ffff70e62b3 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#46 0x00007ffff6c94ac3 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#47 0x00007ffff6d268c0 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Another crash in the same circumstances.

[slapin]

Reduced updates to once per 2 seconds, that made crashes rare and terrain updates without jumping around work now. So having main issue resolved I need to find proper solution without risk of crashing everything. But I still wonder why LOD updates do not work without constant update() runs...

[slapin]

Code: Select all

#0  0x00007ffff7d4d689 in Ogre::Page::handleResponse (this=0x5555e6f1d920, res=0x7fffe8034110, srcQ=<optimized out>) at /media/slapin/library/ogre3/ogre/Components/Paging/src/OgrePage.cpp:233
#1  0x00007ffff7d4d84b in operator() (__closure=0x7fffffffd890) at /media/slapin/library/ogre3/ogre/Components/Paging/src/OgrePage.cpp:186
#2  std::__invoke_impl<void, Ogre::Page::load(bool)::<lambda()>::<lambda()>&> (__f=...) at /usr/include/c++/11/bits/invoke.h:61
#3  std::__invoke_r<void, Ogre::Page::load(bool)::<lambda()>::<lambda()>&> (__fn=...) at /usr/include/c++/11/bits/invoke.h:154
#4  std::_Function_handler<void(), Ogre::Page::load(bool)::<lambda()>::<lambda()> >::_M_invoke(const std::_Any_data &) (__functor=...) at /usr/include/c++/11/bits/std_function.h:290
#5  0x00007ffff7752d1c in std::function<void ()>::operator()() const (this=0x7fffffffd890) at /usr/include/c++/11/bits/std_function.h:590
#6  Ogre::DefaultWorkQueueBase::processMainThreadTasks (this=0x5555562227b0) at /media/slapin/library/ogre3/ogre/OgreMain/src/OgreWorkQueue.cpp:176
#7  0x00007ffff76987bc in Ogre::Root::_fireFrameEnded (this=this@entry=0x55555621e7a0, evt=...) at /media/slapin/library/ogre3/ogre/OgreMain/src/OgreRoot.cpp:685
#8  0x00007ffff769c0e9 in Ogre::Root::_fireFrameEnded (this=0x55555621e7a0) at /media/slapin/library/ogre3/ogre/OgreMain/src/OgreRoot.cpp:712
#9  0x00007ffff769c188 in Ogre::Root::startRendering (this=0x55555621e7a0) at /media/slapin/library/ogre3/ogre/OgreMain/src/OgreRoot.cpp:779
#10 0x000055555559d5b2 in main () at /media/slapin/library/ogre3/ogre-projects/world2/src/editor/main.cpp:793

Another crash. I guess this code requires to be protected by some mutex to prevent conflicts with worker thread, however the thread itself is unprotected so
there is no safe way to reload the chunk without risk of crash.

[slapin]

After removal of page loading keeping only unloading and adding a mutex to definer I was not able to reproduce the problem (for now).
Finally the code to reload terrain looks like this:

Code: Select all

		for (auto &slot :
		     ECS::get<Terrain>().mTerrainGroup->getTerrainSlots()) {
			Ogre::uint32 page =
				ECS::get<Terrain>().mTerrainGroup->packIndex(
					slot.second->x, slot.second->y);
			ECS::get<Terrain>()
				.mTerrainPagedWorldSection->unloadPage(page,
								       false);
		}

Maype I miss some optimizations but I'm afraid it will crumble...