New Terrain Early Shots

A place to show off your latest screenshots and for people to comment on them. Only start a new thread here if you have some nice images to show off!
Post Reply
User avatar
joew
Greenskin
Posts: 113
Joined: Fri Nov 03, 2006 6:03 pm

Re: New Terrain Early Shots

Post by joew »

I much prefer the way the lower detail is handled by sampling in a second thread right now over having separate low detail data saved. In our project we stream and page a lot of data from disk and so whenever we can efficiently get away with not touching the disk (such as in this case) it is very preferable compared to racing to load assets (even if it is in a p-queue it is likely to content with other assets much of the time).
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19269
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66
Contact:

Re: New Terrain Early Shots

Post by sinbad »

joew wrote:I much prefer the way the lower detail is handled by sampling in a second thread right now over having separate low detail data saved. In our project we stream and page a lot of data from disk and so whenever we can efficiently get away with not touching the disk (such as in this case) it is very preferable compared to racing to load assets (even if it is in a p-queue it is likely to content with other assets much of the time).
Having the lower detail data available can actually reduce your disk overhead, assuming that you see more distant pages than you actually visit. Of course, if you load the lower detail version and then subsequently visit it, you've overall incurred some additional loading (the low-detail version), but that's likely to be outweighed by the other pages that you've loaded only the low-detail version of and never got close enough to require that the higher detail version was paged in. This is assuming a large terrain with large draw distances though.

In any case, and low/high external LOD will be an option rather than a requirement (once it's added - the core paging of just the standard terrain tiles comes first) so you can pick what works best for you.
User avatar
joew
Greenskin
Posts: 113
Joined: Fri Nov 03, 2006 6:03 pm

Re: New Terrain Early Shots

Post by joew »

sinbad wrote:Having the lower detail data available can actually reduce your disk overhead, assuming that you see more distant pages than you actually visit. Of course, if you load the lower detail version and then subsequently visit it, you've overall incurred some additional loading (the low-detail version), but that's likely to be outweighed by the other pages that you've loaded only the low-detail version of and never got close enough to require that the higher detail version was paged in. This is assuming a large terrain with large draw distances though.
Ah yes I wasn't thinking about that initially as I am keeping a smaller draw distance right now so I was only thinking about my situation :D It would definitely be helpful on the disk if you had a far draw distance and were loading lower detail data where a fairly high percentage will likely not be visited in high detail.
User avatar
Falagard
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 2060
Joined: Thu Feb 26, 2004 12:11 am
Location: Toronto, Canada
x 3
Contact:

Re: New Terrain Early Shots

Post by Falagard »

I'm only saving the high detail data, the hierarchy (which is a quadtree) is built from that, and it's very fast to do so because of the regular grid and that all the other data is already determined or an undersampled version of the same data (plus, it's all done in the background thread - I build CPU versions of all the buffer structures and vertex declarations etc and just clone them across to the GPU in the render thread when they're ready).

My intention is to continue to do that, and that the low-detail version of the terrain will just appear to be a regular terrain with less base resolution from an internal standpoint. This keeps things simple, as well as decoupling any data dependence between the load levels, meaning they can be processed entirely separately (important when you're bringing in and processing new data in the background). Essentially then transition between loaded levels is just a case of switching out one Terrain instance for another. I favour this separable approach over trying to make the terrain instance itself aware of high & low detail loaded levels, because I think it overcomplicates things to have internal re-loading going on. That's why I made a distinction between 'internal' and 'external' LOD.
So to clarify, does that mean that for distant terrain pages (when pages are eventually implemented by yourself or someone else), the highest level of detail data needs to be loaded, then the lower levels of detail are generated and the higher levels of detail can be discarded until they are later needed (if the camera moves towards them, for example)? I may need to look deeper into the implementation of external and internal LODs, because I don't grok that part of the architecture yet.
This is in theory possible, but I have no intention of supporting it myself. One of the reasons is that you can make lots of simplifying assumptions if you know you're always dealing with a regular grid, including about how much data you need to hold (positions only need one value - height- rather than being explicit), how you transition between LODs (one delta factor instead of 3 for full 3D morphing), how you deal with collision & ray queries (you can use a fast Bresnhams algorithm), and also even that your vertex shader calculation resolution is regular and predictable, meaning it's easier to avoid artefacts due to irregular tesselation & interpolation. I don't believe that the vertex overhead you save with an irregular terrain in the areas where you don't need the resolution is necessarily worth it. I'll be adding extra vertex compression later on which will exploit the regular grid too.
Yeah, I'm not sure if the simplification of the terrain is worth it in practice. I know that you can reduce the number of vertices by an average of 30% when decimating the terrain, and can make that even higher at risk of visual artifacts, but I don't know if the savings are worth it when considering LOD transitions and collision tests. Anyhow, thanks for the info.
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19269
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66
Contact:

Re: New Terrain Early Shots

Post by sinbad »

Falagard wrote:So to clarify, does that mean that for distant terrain pages (when pages are eventually implemented by yourself or someone else), the highest level of detail data needs to be loaded, then the lower levels of detail are generated and the higher levels of detail can be discarded until they are later needed (if the camera moves towards them, for example)?
Yes, although the generation of the lower detail version would occur at save time when you've finished editing the terrain, not at runtime (although you could make it do it that way too), so that the runtime would only need to load the low detail level version in the first instance.
I may need to look deeper into the implementation of external and internal LODs, because I don't grok that part of the architecture yet.
Probably because there's only a generalised structure to hold it right now (PageContentCollection) - it's deliberately abstract because I want to give people options for how they might use dynamic switch-out of PageContent instances; external LOD is the most obvious but I didn't want to bake that concept in there, I'm sure there are other reasons why someone might want to have multiple representations of a PageContent and switch them out based on some criterion.
User avatar
Xplodwild
Goblin
Posts: 231
Joined: Thu Feb 12, 2009 3:49 pm
Location: France
x 13
Contact:

Re: New Terrain Early Shots

Post by Xplodwild »

I nearly fixed all my crashes, even if some still remains. The main problem now is a lod/lighting issue when I have four different terrains (each 512x512, so a total of 2048x2048).

Here's a screenshot :
http://img190.imageshack.us/img190/3996 ... gissue.jpg

I intentionally done a "leak" in my map to see the separation between the two terrains
As you can see on the left of the rendering widget, there's a LOD issue that never disappears, even when you're near it. My maxpixelerror is set to 8. The second problem is the lighting of that terrain (not updated). I first though about a threading issue, which leads me to try with Ogre compiled with no threading support, but the issue is still here (the sshot was taken with Ogre with no threading support). It doesn't bugs immediately but after some deformations.
Also, as soon as I notice the LOD issue, I get a freeze when I try to save the terrain. Here's the callstack :

Code: Select all

 	ntdll.dll!7c90e514() 	
 	[Les frames ci-dessous sont peut-être incorrects et/ou manquants, aucun symbole chargé pour ntdll.dll]	
 	ntdll.dll!7c90dcba() 	
 	kernel32.dll!7c82fabd() 	
 	OgreMain.dll!Ogre::Timer::getMilliseconds()  Ligne 134	C++
>	OgreMain.dll!Ogre::DefaultWorkQueueBase::processResponses()  Ligne 374	C++
 	OgreTerrain.dll!Ogre::Terrain::waitForDerivedProcesses()  Ligne 1646 + 0xf octets	C++
 	OgreTerrain.dll!Ogre::Terrain::save(Ogre::StreamSerialiser & stream={...})  Ligne 234	C++
 	OgreTerrain.dll!Ogre::Terrain::save(const std::basic_string<char,std::char_traits<char>,std::allocator<char> > & filename="Zones/test1024/test1024_2.adt")  Ligne 227	C++
 	ZoneCreator.exe!Zone::saveToFile(const char * loc=0x0012b9b0)  Ligne 36 + 0x18a octets	C++
 	ZoneCreator.exe!ZoneManager::saveZone(const char * loc=0x0012b9b0)  Ligne 62	C++
 	ZoneCreator.exe!MainWindow::saveZone()  Ligne 973 + 0x7b octets	C++
 	ZoneCreator.exe!MainWindow::qt_metacall(QMetaObject::Call _c=InvokeMetaMethod, int _id=8, void * * _a=0x0012bb08)  Ligne 166 + 0x8 octets	C++
 	QtCore4.dll!670f58fd() 	
 	QtCore4.dll!670f5bf0() 	
 	QtCore4.dll!670f5b80() 	
 	QtGui4.dll!6500cf91() 	
 	QtGui4.dll!653534b5() 	
 	QtGui4.dll!652b0c30() 	
 	QtGui4.dll!6535280c() 	
 	QtGui4.dll!652e07e2() 	
 	msvcr90.dll!free(void * pBlock=0x9e7c93d5)  Ligne 110	C
 	95ffffff()	
It seems that Ogre waits for an operation to finish, but it never finishes. I remember you that threading is disabled! Also, I tried to set a time limit to 5s to the defaultworkqueue, but I tried to save like 20 minutes ago and it still didn't move.

I know, I really have strange bugs :p But since I didn't have any bugs while using ETM I really find that strange. I'm going to try to reproduce the issue in PlayPen.
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19269
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66
Contact:

Re: New Terrain Early Shots

Post by sinbad »

Lighting between terrain instances isn't supported yet. I've tested 2 terrains next to each other and they seemed ok with their lighting, except that they didn't cast shadows on each other (known issue).

I'm afraid I'm outta here for 2.5 weeks now so I won't be working on this anymore until my return.
User avatar
Xplodwild
Goblin
Posts: 231
Joined: Thu Feb 12, 2009 3:49 pm
Location: France
x 13
Contact:

Re: New Terrain Early Shots

Post by Xplodwild »

I know that lighting between terrain instances isn't supported, but you can see that there's NO lighting at all on this terrain. It doesn't happens all the time but still happens, and makes me unable to save (freeze on the WorkQueue).
User avatar
stealth977
Gnoll
Posts: 638
Joined: Mon Dec 15, 2008 6:14 pm
Location: Istanbul, Turkey
x 42

Re: New Terrain Early Shots

Post by stealth977 »

@sinbad:
I know this is a bit offtopic, but since you mentioned it in this topic, i want to ask a question:
How can having a 1x1 primary window prevent device losts during resize? When the resize happens, i call renderWindowMovedOrResized() which in turn changes the windows size, then during rendering D3D9RenderSystem checks if the device is valid, while doing that, it checks if the render window size is equal to presentParameters and if not it invalidates the device, which causes a reset.

So, resizing any render window causes a device reset. What am i doing wrong here?

ismail,
Ismail TARIM
Ogitor - Ogre Scene Editor
WWW:http://www.ogitor.org
Repository: https://bitbucket.org/ogitor
User avatar
jacmoe
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 20570
Joined: Thu Jan 22, 2004 10:13 am
Location: Denmark
x 179
Contact:

Re: New Terrain Early Shots

Post by jacmoe »

I am afraid you have to wait 2 1/2 weeks for the answer - he's on holiday. :)
/* Less noise. More signal. */
Ogitor Scenebuilder - powered by Ogre, presented by Qt, fueled by Passion.
OgreAddons - the Ogre code suppository.
User avatar
KungFooMasta
OGRE Contributor
OGRE Contributor
Posts: 2087
Joined: Thu Mar 03, 2005 7:11 am
Location: WA, USA
x 16
Contact:

Re: New Terrain Early Shots

Post by KungFooMasta »

I know Sinbad's on holiday, but I wanted to post an issue I'm seeing, in case somebody knows the answer, or for when Sinbad gets back.

In my app I use multiple windows, each with their own SceneManager's, camera's, etc. However I'm finding that when I use the terrain in one window, it somehow affects other windows. In my scenario I have a main window with my main Scene, including the terrain. After creation of this window I create a second window, my MeshViewer.

If I comment out creation of Terrain, second window looks like this:
Image

If Terrain is created:
Image

The point at which the problem occurs is during the last 2 lines of the following code snippet, which I've pretty much taken from the PlayPen and adapted to my scenario:

Code: Select all

		Image img;
		img.load("terrain.png", ResourceGroupManager::DEFAULT_RESOURCE_GROUP_NAME);
		//img.load("terrain_flattened.png", ResourceGroupManager::DEFAULT_RESOURCE_GROUP_NAME);

		Terrain::ImportData imp;
		imp.inputImage = &img;
		imp.terrainSize = img.getWidth();
		imp.worldSize = 1000;
		//imp.inputScale = 600;
		imp.inputScale = 1;
		imp.minBatchSize = 33;
		imp.maxBatchSize = 65;
		imp.pos = Vector3::ZERO;
		// textures
		imp.layerList.resize(2);
		imp.layerList[0].worldSize = 20;
		imp.layerList[0].textureNames.push_back("grass_green-01_diffusespecular.dds");
		imp.layerList[0].textureNames.push_back("grass_green-01_normalheight.dds");
		imp.layerList[1].worldSize = 100;
		imp.layerList[1].textureNames.push_back("dirt_grayrocky_diffusespecular.dds");
		imp.layerList[1].textureNames.push_back("dirt_grayrocky_normalheight.dds");
		mTerrain->prepare(imp);
		mTerrain->load();
I've tried setting the shadow method to NONE in my second window, but that didn't affect the outcome. I'm not sure why the Ogre Terrain is altering properties in other SceneManagers.. or maybe its some rendering setting being set?

Can anybody repro this?

1. Create a window with its own SceneManager, Camera, viewport, and Ogre Terrain
2. Create a second window with its own SceneManager, Camera, viewport and add a mesh or 2.

[Edit] I only see this problem when I use OpenGL as my rendering system {/Edit]
Creator of QuickGUI!
User avatar
Xplodwild
Goblin
Posts: 231
Joined: Thu Feb 12, 2009 3:49 pm
Location: France
x 13
Contact:

Re: New Terrain Early Shots

Post by Xplodwild »

My lead artist made a little video of a small terrain made in our editor :
http://www.vimeo.com/6436900

We're using and showing all the aspects of OgreTerrain (specular, splatting, etc...) and in high-definition it's really beautiful :)
The only thing i'm waiting for now is the objects shadow support and paging :) I'd like to help, but I'm not enough fluent in 3D engine programming to be able to do things like that.
User avatar
lonwolf
Halfling
Posts: 79
Joined: Sat Dec 24, 2005 9:31 pm
Location: Bucharest
Contact:

Re: New Terrain Early Shots

Post by lonwolf »

Wow looks very nice.
One small question tho, i see the ETM is a dead-end and this nifty add-on is something to replace it, however i can't see any deformation tools around in the source code. I've searched the wiki and other forum topics about this new SM but couldn't find more specifications. Any ETA for a general release or someone could direct me to a solution untill then?.
the goal is just to have a terrain and to be able to rise and lower patches / vertices / etc deforming it on runtime. besides ETM which is kinda old now, i can't find anything to help. (moving vertices by accessing the mesh data isn't something i've done before and seems rather harsh to newbies since there are no complete tutorials / articles on how to properly do this)
suggestions / ideas ?
Check out my tutorial for Content Creating in Ogre: (also available on wiki)
Creating realistic low poly 3D trees, under 500 polys
http://files.filefront.com/tutorialsrar ... einfo.html
User avatar
alberts
Gremlin
Posts: 177
Joined: Fri Mar 31, 2006 8:43 am
Location: Granada-Cádiz-Jaén, Spain
x 20

Re: New Terrain Early Shots

Post by alberts »

lonwolf wrote:Wow looks very nice.
One small question tho, i see the ETM is a dead-end and this nifty add-on is something to replace it, however i can't see any deformation tools around in the source code. I've searched the wiki and other forum topics about this new SM but couldn't find more specifications. Any ETA for a general release or someone could direct me to a solution untill then?.
the goal is just to have a terrain and to be able to rise and lower patches / vertices / etc deforming it on runtime. besides ETM which is kinda old now, i can't find anything to help. (moving vertices by accessing the mesh data isn't something i've done before and seems rather harsh to newbies since there are no complete tutorials / articles on how to properly do this)
suggestions / ideas ?
Maybe you can take a look to http://www.ogitor.org/. It think it uses the newest terrain.
User avatar
lonwolf
Halfling
Posts: 79
Joined: Sat Dec 24, 2005 9:31 pm
Location: Bucharest
Contact:

Re: New Terrain Early Shots

Post by lonwolf »

yep i checked it, it's very nice looking and all, but seems it's a stand-alone editor. I guess i could get the code i needed from it but as always, without guidelines it a really time consuming job. I think i'll stick to ETM (the agony searching for something to just add a teraforming tool in a game) until something official pops up from Ogre3d team as in a stable release and some articles about using it properly.

Good job so far, looking forward to it :wink:
Check out my tutorial for Content Creating in Ogre: (also available on wiki)
Creating realistic low poly 3D trees, under 500 polys
http://files.filefront.com/tutorialsrar ... einfo.html
User avatar
jacmoe
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 20570
Joined: Thu Jan 22, 2004 10:13 am
Location: Denmark
x 179
Contact:

Re: New Terrain Early Shots

Post by jacmoe »

Just take a look at Ogitor - it uses raw Ogre.
/* Less noise. More signal. */
Ogitor Scenebuilder - powered by Ogre, presented by Qt, fueled by Passion.
OgreAddons - the Ogre code suppository.
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19269
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66
Contact:

Re: New Terrain Early Shots

Post by sinbad »

jacmoe wrote:Just take a look at Ogitor - it uses raw Ogre.
Or indeed the Demo_Terrain in trunk which gives you deforming and painting tools - albeit rather simple ones but they show you how it's done.

@KungFooMasta: weird, the only thing being shared is the material list and the render system. I'd need a repro case.
stealth977 wrote:know this is a bit offtopic, but since you mentioned it in this topic, i want to ask a question:
How can having a 1x1 primary window prevent device losts during resize? When the resize happens, i call renderWindowMovedOrResized() which in turn changes the windows size, then during rendering D3D9RenderSystem checks if the device is valid, while doing that, it checks if the render window size is equal to presentParameters and if not it invalidates the device, which causes a reset.

So, resizing any render window causes a device reset. What am i doing wrong here?
You never call any resize method on the primary window. You create it at 1x1, you hide it somewhere (behind a toolbar or something), and you never touch it again. You only call the resize on your secondary windows, which are the ones you actually use for rendering. This way, device lost conditions never happen when you resize these secondary windows. [edit]Unless this has changed in trunk with Nir's windowing changes - I haven't tried it on the most recent code...

I'll be coming back to the terrain in a couple of weeks, I have some other deadlines in between.
User avatar
jacmoe
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 20570
Joined: Thu Jan 22, 2004 10:13 am
Location: Denmark
x 179
Contact:

Re: New Terrain Early Shots

Post by jacmoe »

Small comment: Device lost conditions definitely happens on resizes, even if we're using the 1x1 primary window trick.
And I even checked with an unparented primary window, which definitely is *not* being resized in any way whatsoever.
It could be Qt, though I doubt it.
/* Less noise. More signal. */
Ogitor Scenebuilder - powered by Ogre, presented by Qt, fueled by Passion.
OgreAddons - the Ogre code suppository.
User avatar
stealth977
Gnoll
Posts: 638
Joined: Mon Dec 15, 2008 6:14 pm
Location: Istanbul, Turkey
x 42

Re: New Terrain Early Shots

Post by stealth977 »

Actually, it seems to be solved now, since now i dont get device lost messages anymore during resize..
Ismail TARIM
Ogitor - Ogre Scene Editor
WWW:http://www.ogitor.org
Repository: https://bitbucket.org/ogitor
User avatar
jacmoe
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 20570
Joined: Thu Jan 22, 2004 10:13 am
Location: Denmark
x 179
Contact:

Re: New Terrain Early Shots

Post by jacmoe »

Heh - that's good news! :)
/* Less noise. More signal. */
Ogitor Scenebuilder - powered by Ogre, presented by Qt, fueled by Passion.
OgreAddons - the Ogre code suppository.
User avatar
Xplodwild
Goblin
Posts: 231
Joined: Thu Feb 12, 2009 3:49 pm
Location: France
x 13
Contact:

Re: New Terrain Early Shots

Post by Xplodwild »

I'm encountering a bug AND i've a GREAT repro case :p

When I save my terrain at time x, blendmaps are okay. x+1, blendmap 2 and 4 are kind of mixed randomly, but if you save again (so x+2) they will be okay again ! (and so on, x+3 will mix blendmaps, saving again x+4 will make them right again, etc...)

Here's the archive :
http://filebeam.com/c4a54473f06ba894e7ab93889ef39b94

Put testTerrain.dat into Demo_Terrain.exe folder, and textures in textures folder in media.
When you open demo_terrain, you can see in the little "island" in the middle of the map that rock1 texture is splatted under the little road and, well, at places that splatting2 textures uses normally. Do Ctrl + S, quit, and open again. Textures are splatted now as they should be (so there's grass under the road and on the little "island"). Hit Ctrl + S again, quit, and open again. Rock1 has again stole blendmaps of splatting1. Hit Ctrl + S again, quit, open : blendmaps are right again !

What's going wrong ?
(Also, why is my terrain more laggy than stock terrain ? (It's 512x512 terrain, but it make my fps drop down to 16fps instead of 90fps for the stock terrain))
(And I've another bug while editing multiple terrains, randomly an RenderingAPIException occurs while renderOneFrame. If I catch it with a try { }, the rendering continues but lighting/lightmap generation seems freezed, and if I save it freezes on waitForDerivedTasks or something like that (WorkQueue). I'll try to investigate a little bit more).
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19269
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66
Contact:

Re: New Terrain Early Shots

Post by sinbad »

Can you log bugs separately in the tracker please? There's a specific terrain category. Like I say I have other deadlines that are not going to leave me any time for the terrain for a couple of weeks so I need a structured place to queue these things.

I have never been able to recreate your 'random' exception, so it's tough to investigate it - more detail definitely required on that one.

As to why your terrain is slower than the example, that depends. Maybe it's more irregular so the vertex count is higher, or maybe the textures are larger (mine are DXT5 compressed too), or you have more layers.
User avatar
Xplodwild
Goblin
Posts: 231
Joined: Thu Feb 12, 2009 3:49 pm
Location: France
x 13
Contact:

Re: New Terrain Early Shots

Post by Xplodwild »

Bug reported to the tracker.
For the random exception, it's very strange because I get an RenderingAPIException, but D3D9 in Debug mode don't return anything special.

Also, I took a look at Ogitor's source code, and saw that they just do "updateGeometry()", but I can't find any update() anywhere. If I only do updateGeometry(), lod isn't updated and it doesn't work as intended, so what am I missing?
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19269
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66
Contact:

Re: New Terrain Early Shots

Post by sinbad »

If you call updateGeometry() separately, then you also call updateDerivedData() separately, not update().
Zero
Halfling
Posts: 50
Joined: Mon Mar 10, 2008 12:08 am
Location: Stuttgart|Germany
x 1
Contact:

Re: New Terrain Early Shots

Post by Zero »

I think, i know how to reproduce this bug with "OGRE EXCEPTION(3:RenderingAPIException): D3DXLoadSurfaceFromMemory failed in D3D9HardwarePixelBuffer::blitFromMemory". You have to use a very big brush and a very big brush strength and then create very high mountain`s and edit a little bit at the borders. And then (not everytime^^) you get this error. This Error wont let crash my editor, but when I save the Terrain. I got an infinite loop and my terrain doesn`t save.

I hope this help to reproduce the bug.
Image
Post Reply