[GSoC 2011 - Accepted] Terrain paging improvements

Threads related to Google Summer of Code
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19269
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66
Contact:

Re: [GSoC 2011 - Accepted] Terrain paging improvements

Post by sinbad »

Ahhh, I see. So when that option is off, the data is just frozen in its currently loaded state. You're right, checking it fixed everything immediately.

I think it would be a good idea to default that option to 'on' instead :)
kuxv
Google Summer of Code Student
Google Summer of Code Student
Posts: 53
Joined: Wed Jan 09, 2008 7:51 pm
Location: Czech Republic
x 16

Re: [GSoC 2011 - Accepted] Terrain paging improvements

Post by kuxv »

Yes that is a good idea. It can save a lot of headaches ;)
My Google summer of code 2011 topic: Terrain paging improvements
My Google summer of code thread
My Google summer of code wiki page
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19269
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66
Contact:

Re: [GSoC 2011 - Accepted] Terrain paging improvements

Post by sinbad »

Cool - I'll make that little tweak :)

I've pulled in the latest changes from the upstream repository into my local copy and I've merged - no major issues yet but it's still building ;) If it all goes well, do you mind if I push this back to your fork? That way, your fork is prepared for merging back into the upstream provided others agree.

By the way, feel free to continue to commit new changes if you want. I'm only supposed to review for GSoC based on what you did up to the deadline, but that's easy for me to do just based on the commit timeline, you don't have to stop if you want to do other things.
kuxv
Google Summer of Code Student
Google Summer of Code Student
Posts: 53
Joined: Wed Jan 09, 2008 7:51 pm
Location: Czech Republic
x 16

Re: [GSoC 2011 - Accepted] Terrain paging improvements

Post by kuxv »

Cool - I'll make that little tweak
Sweet, thank you ;)
If it all goes well, do you mind if I push this back to your fork?
That would be great! Please go ahead. Do you need anything to do for that from my side?
My Google summer of code 2011 topic: Terrain paging improvements
My Google summer of code thread
My Google summer of code wiki page
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19269
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66
Contact:

Re: [GSoC 2011 - Accepted] Terrain paging improvements

Post by sinbad »

kuxv wrote:Please go ahead. Do you need anything to do for that from my side?
Nope :) It's done, everything seems fine in the merged version so I've pushed all the changes back to your fork. So this means that in your fork, you're up to date with the upstream as of now and the 2 lines of development have been re-integrated. Obviously it might temporarily diverge again for a while until these changes are pushed to upstream, but at least it proves that everything slots together.

We'll need someone on Windows to run some tests too to make sure everything is ok there. Any volunteers?
al2950
OGRE Expert User
OGRE Expert User
Posts: 1227
Joined: Thu Dec 11, 2008 7:56 pm
Location: Bristol, UK
x 157

Re: [GSoC 2011 - Accepted] Terrain paging improvements

Post by al2950 »

Just set up new development machine so these need to be checked, however;
  • 1.8 Unstable, VS2010, Win7, Release - Builds and runs fine
  • TPI, VS2010, Win7, Release - Linker crashes! with

    Code: Select all

    3>  OgreTerrainPagedWorldSection.cpp
    3>  Generating Code...
    3>c:\users\angus\documents\projects\ogre\gsoc_2011_tpi\components\terrain\src\ogreterrain.cpp(1399): fatal error C1001: An internal error has occurred in the compiler.
    3>  (compiler file 'f:\dd\vctools\compiler\utc\src\p2\main.c[0x6C6618C0:0x00000004]', line 183)
    3>   To work around this problem, try simplifying or changing the program near the locations listed above.
    3>  Please choose the Technical Support command on the Visual C++ 
    
    LOL at the "To work around this problem" message from Microsoft! :)
  • TPI, VS2010, Win7, Debug- Compile fails with;

    Code: Select all

    1>------ Build started: Project: OgreTerrain, Configuration: Debug Win32 ------
    1>  OgreTerrain.cpp
    1>C:\Users\Angus\Documents\Projects\Ogre\GSOC_2011_TPI\Components\Terrain\src\OgreTerrain.cpp(5038): error C2665: 'operator new' : none of the 5 overloads could convert all the argument types
    1>          C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\include\crtdbg.h(1080): could be 'void *operator new(size_t,int,const char *,int)'
    1>          C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\include\xdebug(33): or       'void *operator new(size_t,const std::_DebugHeapTag_t &,char *,int) throw(...)'
    1>          while trying to match the argument list '(unsigned int, const char [92], int, const char [64])'
    ========== Build: 0 succeeded, 1 failed, 2 up-to-date, 0 skipped ==========
    
I used the default settings for CMake.

I dont have time to look at this tonight I am afraid but will have a closer look at the weekend. However as I said this is a new dev machine so I may have messed something up!

**EDIT** Built as a 32bit app on a 64 bit OS
kuxv
Google Summer of Code Student
Google Summer of Code Student
Posts: 53
Joined: Wed Jan 09, 2008 7:51 pm
Location: Czech Republic
x 16

Re: [GSoC 2011 - Accepted] Terrain paging improvements

Post by kuxv »

Hi, thank you for test. Problem is with Ogre::NameGenerator not inheriting from AllocatedObject. I'm using NameGenerator to create names for textures in DefaultTextureAllocator. I didn't want to make it static member so I need to create it in constructor. That is where OGRE_NEW NameGenerator("...") is called and that's place of error. So solution is to make NameGenerator inherit from GeneralAllocatedObject. I can fix it and commit if you guys can tell me it is best solution here as I'm not extra familiar with those internals. I got this idea from Kulik on IRC. I tried to compile and run it thou and didn't see any errors.
My Google summer of code 2011 topic: Terrain paging improvements
My Google summer of code thread
My Google summer of code wiki page
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19269
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66
Contact:

Re: [GSoC 2011 - Accepted] Terrain paging improvements

Post by sinbad »

Doh, email notification fail again.

All the allocators are typedefed, so please make it a subclass of 'UtilityAlloc'. This actually is GeneralAllocatedObject, but it should retain the option to be redefinable.
User avatar
amigoface
Greenskin
Posts: 132
Joined: Mon Oct 13, 2008 3:01 pm

Re: [GSoC 2011 - Accepted] Terrain paging improvements

Post by amigoface »

hi,
any news on the documentation for the new terrain paging system ?

even incomplete it could be very helpfull !
noorus
Halfling
Posts: 75
Joined: Wed Apr 20, 2011 9:55 pm
Location: Helsinki, Finland
x 3

Re: [GSoC 2011 - Accepted] Terrain paging improvements

Post by noorus »

I'd also like to know where this project stands now. Will you continue working on this?
Creator of Nice Input Library, for your advanced input needs.
Image
User avatar
amigoface
Greenskin
Posts: 132
Joined: Mon Oct 13, 2008 3:01 pm

Re: [GSoC 2011 - Accepted] Terrain paging improvements

Post by amigoface »

@noorus :

it stll there :
https://bitbucket.org/kuxv/ogre_soc_tpi/overview

and according to the roadmap every thing could be finnisehd including the documentation for the new terrain system


@kuxv : are you alive ? :|
noorus
Halfling
Posts: 75
Joined: Wed Apr 20, 2011 9:55 pm
Location: Helsinki, Finland
x 3

Re: [GSoC 2011 - Accepted] Terrain paging improvements

Post by noorus »

Will this be in Ogre 1.8 final?
I do compile my own Ogre from trunk, but I wouldn't want to merge separate forks locally, in case it breaks now or in the future.
Creator of Nice Input Library, for your advanced input needs.
Image
kuxv
Google Summer of Code Student
Google Summer of Code Student
Posts: 53
Joined: Wed Jan 09, 2008 7:51 pm
Location: Czech Republic
x 16

Re: [GSoC 2011 - Accepted] Terrain paging improvements

Post by kuxv »

Hi guys,
sorry for the delay but mail notifications failed me AGAIN... I was really busy with school since my last post but I will look into that next week and finish what is left to do.
My Google summer of code 2011 topic: Terrain paging improvements
My Google summer of code thread
My Google summer of code wiki page
kuxv
Google Summer of Code Student
Google Summer of Code Student
Posts: 53
Joined: Wed Jan 09, 2008 7:51 pm
Location: Czech Republic
x 16

Re: [GSoC 2011 - Accepted] Terrain paging improvements

Post by kuxv »

Hi,
I commited 3 little changes to my repository:
1) delete texture name generator
2) remove paging debug mode
3) make NameGenerator inherit from UtilityAlloc(that should fix that compilation error al2950 has)

Regarding documentation I'm not sure what everything you guys need. I want to add documentation for new terrain format (version 2) but otherwise I think it is documented quite well(altogether with wiki). Let me know your suggestions.
My Google summer of code 2011 topic: Terrain paging improvements
My Google summer of code thread
My Google summer of code wiki page
User avatar
jacmoe
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 20570
Joined: Thu Jan 22, 2004 10:13 am
Location: Denmark
x 179
Contact:

Re: [GSoC 2011 - Accepted] Terrain paging improvements

Post by jacmoe »

We can't have too much documentation on the terrain component. :)
/* Less noise. More signal. */
Ogitor Scenebuilder - powered by Ogre, presented by Qt, fueled by Passion.
OgreAddons - the Ogre code suppository.
User avatar
Mikachu
Gnoll
Posts: 603
Joined: Thu Jul 28, 2005 4:11 pm
Location: Nice, France
x 35

Re: [GSoC 2011 - Accepted] Terrain paging improvements

Post by Mikachu »

jacmoe wrote:We can't have too much documentation on the terrain component. :)
Sorry if that's a bit off-topic, but I'd really like to see documentation about Ogre components (new terrain, RTSS, etc.) included into the Manual...
Since these components ship together with Ogre, the wiki might not be enough, IMHO.
OgreProcedural - Procedural Geometry for Ogre3D
User avatar
spacegaier
OGRE Team Member
OGRE Team Member
Posts: 4304
Joined: Mon Feb 04, 2008 2:02 pm
Location: Germany
x 135
Contact:

Re: [GSoC 2011 - Accepted] Terrain paging improvements

Post by spacegaier »

Mikachu wrote:
jacmoe wrote:We can't have too much documentation on the terrain component. :)
Sorry if that's a bit off-topic, but I'd really like to see documentation about Ogre components (new terrain, RTSS, etc.) included into the Manual...
Since these components ship together with Ogre, the wiki might not be enough, IMHO.
The problem is: Someone has to volunteer to write that documentation first in a way that can be included in the manual.
Ogre Admin [Admin, Dev, PR, Finance, Wiki, etc.] | BasicOgreFramework | AdvancedOgreFramework
Don't know what to do in your spare time? Help the Ogre wiki grow! Or squash a bug...
User avatar
Mikachu
Gnoll
Posts: 603
Joined: Thu Jul 28, 2005 4:11 pm
Location: Nice, France
x 35

Re: [GSoC 2011 - Accepted] Terrain paging improvements

Post by Mikachu »

spacegaier wrote:
Mikachu wrote:
jacmoe wrote:We can't have too much documentation on the terrain component. :)
Sorry if that's a bit off-topic, but I'd really like to see documentation about Ogre components (new terrain, RTSS, etc.) included into the Manual...
Since these components ship together with Ogre, the wiki might not be enough, IMHO.
The problem is: Someone has to volunteer to write that documentation first in a way that can be included in the manual.
Oops, I didn't realize that the sources for Ogre manual are actually in the repository... :oops:
So indeed, anyone can contribute just as they would do with classic patches.
Of course, the best would be that whenever someone makes a big patch, he'd write the doc at the same time (that's for theory, I know by experience that writing docs is not a very entertaining task ;) )
OgreProcedural - Procedural Geometry for Ogre3D
PacoRG
Kobold
Posts: 26
Joined: Fri May 23, 2008 10:46 pm
Location: Spain
x 1

Re: [GSoC 2011 - Accepted] Terrain paging improvements

Post by PacoRG »

HI,

I am testing the last terrain revision in the "ogre_soc_tpi" repository (revision 6ae90cbaff56). I compiled this revision in Visual C++ 2008.

The paging terrain example work fine when the paging mode is disabled. However, I have found some problems related to the "autoUpdateLod" mechanism. I have been following the instructions and I have modified the paging terrain example in order to enable the paging system. I created 5x5 flat pages (blankTerrain = true), and, after that, I uncommented "#define PAGING".

When the new terrain is visible I move the camera and all seem work correctly. However, some pages show a incongruous LOD according to the distance from camera and with its neighbor pages. When a terrain page is loaded after it has been unload the problem is a lot more frequent, even, the example sometimes crashed. In the next screenshots, the problems are showed:

Image

Image

In debug mode I get this assertion very frequently. I guess they are related to the same problem observed in release mode.

In adition, the example sometime crashed here in debug mode

All these issues occur too once I have integrated the new paging terrain in my application.

Does someone have observed this issue in paging mode?
Is there some approach in order to solve this?

Thanks in advance,
Paco

Windows Vista 32, AMD athlon 64 X2 5200+, 4GB
NVIDIA GeForce GTX 560 TI
Last edited by spacegaier on Sat Dec 17, 2011 10:33 pm, edited 1 time in total.
Reason: inlined the main images
Paco, Spain
Yacoby
Halfling
Posts: 85
Joined: Sun Sep 23, 2007 7:58 pm

Re: [GSoC 2011 - Accepted] Terrain paging improvements

Post by Yacoby »

What are the plans with regard to merging this into the main 1.8 branch?
CABAListic
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 2903
Joined: Thu Jan 18, 2007 2:48 pm
x 58
Contact:

Re: [GSoC 2011 - Accepted] Terrain paging improvements

Post by CABAListic »

If it's not in there yet, it's not going to be in 1.8. That version is very close to release; adding features at the last minute is asking for trouble. I'm afraid it'll have to wait for 1.9.
Yacoby
Halfling
Posts: 85
Joined: Sun Sep 23, 2007 7:58 pm

Re: [GSoC 2011 - Accepted] Terrain paging improvements

Post by Yacoby »

Ok. Cheers.
kuxv
Google Summer of Code Student
Google Summer of Code Student
Posts: 53
Joined: Wed Jan 09, 2008 7:51 pm
Location: Czech Republic
x 16

Re: [GSoC 2011 - Accepted] Terrain paging improvements

Post by kuxv »

Hi,

@PacoRG: I confirm the bug you found. It is problem with wrong LOD calculation but I hasn't find solution yet. I will report a bug and look on it. Thank you for all the info you posted.

@CABAListic: I agree that adding features on last time is bad practise but those last 3 changes very pretty small and couldn't cause bugs PacoRG found. Nevertheless until the auto LOD bug is solved it is not ready to be merged into mainline :/
My Google summer of code 2011 topic: Terrain paging improvements
My Google summer of code thread
My Google summer of code wiki page
Yacoby
Halfling
Posts: 85
Joined: Sun Sep 23, 2007 7:58 pm

Re: [GSoC 2011 - Accepted] Terrain paging improvements

Post by Yacoby »

This is just me being hopeful, but my problem is this:
For performance reasons I have relatively small terrain blocks, however, at a distance I would prefer to batch up these blocks into a larger terrain with a composite texture. From what I gathered your improvments are more geared towards better dealing with LOD using single terrain objects, rather than a higher level approach?
kuxv
Google Summer of Code Student
Google Summer of Code Student
Posts: 53
Joined: Wed Jan 09, 2008 7:51 pm
Location: Czech Republic
x 16

Re: [GSoC 2011 - Accepted] Terrain paging improvements

Post by kuxv »

I aimed at partial terrain loading so data which are not needed (like higher LOD geometry data) are not loaded nor processed. So what you can do is to split your terrain into chunks and let distant terrain chunks load only lowest level which could be just 4 vertices. Together with the use of composite map which is already used in terrain that sounds to me like what you are looking for.
My Google summer of code 2011 topic: Terrain paging improvements
My Google summer of code thread
My Google summer of code wiki page
Post Reply