[GSoC 2012 - Accepted] Improve and Demo the Terrain System
-
- Google Summer of Code Student
- Posts: 102
- Joined: Tue Mar 20, 2012 3:20 am
- Location: China
- x 13
Re: [GSoC 2012 - Accepted] Improve and Demo the Terrain Syst
Weekly Progress
Make the new terrain system work with paging ON
Fix the init problem, the sample can be restarted, now
Make AutoUpdateLodStrategy as singleton, and user can register his own strategy
Issue
With paging on, flicker occurs when loading a new page. I haven't found the main reason, yet.
Make the new terrain system work with paging ON
Fix the init problem, the sample can be restarted, now
Make AutoUpdateLodStrategy as singleton, and user can register his own strategy
Issue
With paging on, flicker occurs when loading a new page. I haven't found the main reason, yet.
Google Summer of Code 2012 Student
Topic: "Improve and Demo the Terrain System"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: masterfalcon
Topic: "Improve and Demo the Terrain System"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: masterfalcon
-
- Google Summer of Code Student
- Posts: 102
- Joined: Tue Mar 20, 2012 3:20 am
- Location: China
- x 13
Re: [GSoC 2012 - Accepted] Improve and Demo the Terrain Syst
I have some thoughts about the second half of the program.
Though I have renamed the TerrainPaging sample to EndlessWorld and started the process of making the terrain generator as planed in my proposal, I think it's not good to slow down the key work, making the new terrain system stable enough to be merged into trunk.
It has been much stabler than before, but still far from perfect. For example,
And, of course, welcome to testing the newest code. I haven't met previously known bugs myself for several days.
Though I have renamed the TerrainPaging sample to EndlessWorld and started the process of making the terrain generator as planed in my proposal, I think it's not good to slow down the key work, making the new terrain system stable enough to be merged into trunk.
It has been much stabler than before, but still far from perfect. For example,
- The new format of terrain file is 3 times larger than before, as it is not well compressed
The flicker when loading new terrain pages is still mysterious.
Still lack of documentation
And, of course, welcome to testing the newest code. I haven't met previously known bugs myself for several days.
Google Summer of Code 2012 Student
Topic: "Improve and Demo the Terrain System"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: masterfalcon
Topic: "Improve and Demo the Terrain System"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: masterfalcon
-
- Kobold
- Posts: 26
- Joined: Fri May 23, 2008 10:46 pm
- Location: Spain
- x 1
Re: [GSoC 2012 - Accepted] Improve and Demo the Terrain Syst
Hi,
I just compiled the last revision.
I get again the CMake error: "Error in configuration process, project files may be invalid" (see post). The bug can be fixed by modifying the "ogre_soc_tpi/Samples/CMakeLists.txt" file replacing the line:
with this other:
I do not observe any problem except the flicker when a new terrain page becomes visible. Related to this I have observed that when the camera has no terrain under it the flicker seems a huge jump back of the camera, however, if the camera is over the terrain, I cannot be sure whether the camera jump but the flicker occurs.
IMHO I agree with you regarding your proposal of key work.
I just compiled the last revision.
I get again the CMake error: "Error in configuration process, project files may be invalid" (see post). The bug can be fixed by modifying the "ogre_soc_tpi/Samples/CMakeLists.txt" file replacing the line:
Code: Select all
set(SAMPLE_DEPENDENCIES ${SAMPLE_DEPENDENCIES} OgreEndlessWorld)
with this other:
Code: Select all
set(SAMPLE_DEPENDENCIES ${SAMPLE_DEPENDENCIES} OgrePaging)
IMHO I agree with you regarding your proposal of key work.
Paco, Spain
-
- Google Summer of Code Student
- Posts: 102
- Joined: Tue Mar 20, 2012 3:20 am
- Location: China
- x 13
Re: [GSoC 2012 - Accepted] Improve and Demo the Terrain Syst
Thank you!PacoRG wrote:Hi,
I just compiled the last revision.
I incorrectly renamed the dependency setting, fixed now. So strange that no compile error occurs on my computer
I'll keep looking into the flicker problem. It seems a little troublesome.
Google Summer of Code 2012 Student
Topic: "Improve and Demo the Terrain System"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: masterfalcon
Topic: "Improve and Demo the Terrain System"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: masterfalcon
-
- Kobold
- Posts: 26
- Joined: Fri May 23, 2008 10:46 pm
- Location: Spain
- x 1
Re: [GSoC 2012 - Accepted] Improve and Demo the Terrain Syst
Yes, I did not get compile errors in my computer neither, it only was a CMAKE error message during the configuration process.xiaoxiangquan wrote: I incorrectly renamed the dependency setting, fixed now. So strange that no compile error occurs on my computer
Paco, Spain
-
- Google Summer of Code Student
- Posts: 102
- Joined: Tue Mar 20, 2012 3:20 am
- Location: China
- x 13
Re: [GSoC 2012 - Accepted] Improve and Demo the Terrain Syst
Weekly Progress:
- Extend StreamSerialiser to support Part-by-Part compression, apply it to terrain file, saves 65% disk space.
Fix some bugs, such as camera-jump-back.
PG-UP/DOWN to inc/dec LOD manually (revert, and enhance).
Update wiki.
- Delay terrain generation work.
Keep on looking into the flicker problem when loading new terrain pages
Google Summer of Code 2012 Student
Topic: "Improve and Demo the Terrain System"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: masterfalcon
Topic: "Improve and Demo the Terrain System"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: masterfalcon
-
- OGRE Team Member
- Posts: 4270
- Joined: Sun Feb 25, 2007 4:56 am
- Location: Bloomington, MN
- x 126
Re: [GSoC 2012 - Accepted] Improve and Demo the Terrain Syst
Great work! I've tried to reproduce the flicker but I can't see it. Though I have vsync enabled, perhaps it only happens at much higher frame rates?
-
- Kobold
- Posts: 26
- Joined: Fri May 23, 2008 10:46 pm
- Location: Spain
- x 1
Re: [GSoC 2012 - Accepted] Improve and Demo the Terrain Syst
I have enabled the vSync and the flicker is observable too. It seems independent on the frame rate, the screen resolution and whether the fullscreen mode is enabled.masterfalcon wrote:I've tried to reproduce the flicker but I can't see it. Though I have vsync enabled, perhaps it only happens at much higher frame rates?
When I have tried to test the EndLess World example in OpenGL I get a crash when the example try to destroy an overlay not found.
Paco, Spain
-
- Google Summer of Code Student
- Posts: 102
- Joined: Tue Mar 20, 2012 3:20 am
- Location: China
- x 13
Re: [GSoC 2012 - Accepted] Improve and Demo the Terrain Syst
According to my profiler, when a new terrain page loaded, some costly procedures will execute synchronously, such as:
So a new terrain page will cause at least 0.3s (on my computer, debug mode). That's why the flicker occurs.
And if more pages are loaded (nearly) at once, the result will be accumulated.
I have tried to move some of the procedures to background, but it seems difficult, as they all do GPU operations.
Anyone has some ideas? Or just tolerate it?
Code: Select all
Terrain::load()
{
...
//these take 0.15s
checkLayers(true);
createOrDestroyGPUColourMap();
createOrDestroyGPUNormalMap();
createOrDestroyGPULightmap();
createOrDestroyGPUCompositeMap();
...
}
...
// the material is null when the page just loaded, so generation procedure will be called
Terrain::getMaterial()
{
...
// these take more than 0.15s
mMaterial = mMaterialGenerator->generate(this);
...
mCompositeMapMaterial = mMaterialGenerator->generateForCompositeMap(this);
...
}
And if more pages are loaded (nearly) at once, the result will be accumulated.
I have tried to move some of the procedures to background, but it seems difficult, as they all do GPU operations.
Anyone has some ideas? Or just tolerate it?
Google Summer of Code 2012 Student
Topic: "Improve and Demo the Terrain System"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: masterfalcon
Topic: "Improve and Demo the Terrain System"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: masterfalcon
-
- OGRE Team Member
- Posts: 4270
- Joined: Sun Feb 25, 2007 4:56 am
- Location: Bloomington, MN
- x 126
Re: [GSoC 2012 - Accepted] Improve and Demo the Terrain Syst
Hmm, looks like a lot of GPU resources being created or modified during that time. Have you narrowed it down further within those functions? Is it the buffer write, texture allocation, texture blit?
-
- Google Summer of Code Student
- Posts: 102
- Joined: Tue Mar 20, 2012 3:20 am
- Location: China
- x 13
Re: [GSoC 2012 - Accepted] Improve and Demo the Terrain Syst
Not yet, do it right now.masterfalcon wrote:Hmm, looks like a lot of GPU resources being created or modified during that time. Have you narrowed it down further within those functions? Is it the buffer write, texture allocation, texture blit?
Google Summer of Code 2012 Student
Topic: "Improve and Demo the Terrain System"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: masterfalcon
Topic: "Improve and Demo the Terrain System"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: masterfalcon
-
- Google Summer of Code Student
- Posts: 102
- Joined: Tue Mar 20, 2012 3:20 am
- Location: China
- x 13
Re: [GSoC 2012 - Accepted] Improve and Demo the Terrain Syst
masterfalcon wrote:Hmm, looks like a lot of GPU resources being created or modified during that time. Have you narrowed it down further within those functions? Is it the buffer write, texture allocation, texture blit?
Code: Select all
Root::renderOneFrame ->
...
SceneManager::firePreFindVisibleObjects ->
...
Terrain::getMaterial ->
TerrainMaterialGeneratorA::addTechnique * 3 times ->
generateVertexProgram and generateFragmentProgram
...
Resource::load(bool background) * 6 times
Code: Select all
Root::renderOneFrame ->
...
Terrain::load ->
Terrain::createGPUBlendTextures and Terrain::createOrDestroyGPUCompositeMap ->
...
D3D9HardwarePixelBuffer::blitFromMemory * 2 times
So loading a new terrain page cost 0.2s+ on these two operations.
Google Summer of Code 2012 Student
Topic: "Improve and Demo the Terrain System"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: masterfalcon
Topic: "Improve and Demo the Terrain System"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: masterfalcon
-
- Google Summer of Code Student
- Posts: 102
- Joined: Tue Mar 20, 2012 3:20 am
- Location: China
- x 13
Re: [GSoC 2012 - Accepted] Improve and Demo the Terrain Syst
Eh, I can not reproduce the crash. Both OpenGL and DX seem to work fine.PacoRG wrote: I have enabled the vSync and the flicker is observable too. It seems independent on the frame rate, the screen resolution and whether the fullscreen mode is enabled.
When I have tried to test the EndLess World example in OpenGL I get a crash when the example try to destroy an overlay not found.
Google Summer of Code 2012 Student
Topic: "Improve and Demo the Terrain System"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: masterfalcon
Topic: "Improve and Demo the Terrain System"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: masterfalcon
-
- Silver Sponsor
- Posts: 2703
- Joined: Mon Aug 29, 2005 3:24 pm
- Location: Kuala Lumpur, Malaysia
- x 51
Re: [GSoC 2012 - Accepted] Improve and Demo the Terrain Syst
Is the flicker just concern the display only (but FPS remains smooth) or is it actually jitter in frame rate that cause it to momentarily stop for a fraction of seconds..? The latter is the issue that Sinbad faces before, and I believe it was the main reason why he was not releasing Terrain+Paging for production uses..
A willow deeply scarred, somebody's broken heart
And a washed-out dream
They follow the pattern of the wind, ya' see
Cause they got no place to be
That's why I'm starting with me
And a washed-out dream
They follow the pattern of the wind, ya' see
Cause they got no place to be
That's why I'm starting with me
-
- Google Summer of Code Student
- Posts: 102
- Joined: Tue Mar 20, 2012 3:20 am
- Location: China
- x 13
Re: [GSoC 2012 - Accepted] Improve and Demo the Terrain Syst
It's the latter. A single frame takes about 0.3s for a new terrain page to do initialization. The following frames will be smooth.syedhs wrote:Is the flicker just concern the display only (but FPS remains smooth) or is it actually jitter in frame rate that cause it to momentarily stop for a fraction of seconds..? The latter is the issue that Sinbad faces before, and I believe it was the main reason why he was not releasing Terrain+Paging for production uses..
Google Summer of Code 2012 Student
Topic: "Improve and Demo the Terrain System"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: masterfalcon
Topic: "Improve and Demo the Terrain System"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: masterfalcon
-
- Gnoblar
- Posts: 3
- Joined: Sun Jul 22, 2012 6:30 pm
Re: [GSoC 2012 - Accepted] Improve and Demo the Terrain Syst
Does a single page loading causes the jitter.. or when it loads 2-3?
Can you split the loading phase over frames? Lets say.. one load/blit operation and then put a mark to pass one frame.
(I know, can be tricky... but should be possible )
Can you split the loading phase over frames? Lets say.. one load/blit operation and then put a mark to pass one frame.
(I know, can be tricky... but should be possible )
-
- Google Summer of Code Student
- Posts: 102
- Joined: Tue Mar 20, 2012 3:20 am
- Location: China
- x 13
Re: [GSoC 2012 - Accepted] Improve and Demo the Terrain Syst
Yes, a single page does. To load 2 or 3 pages, the jitter effect will add up.Sadahar wrote:Does a single page loading causes the jitter.. or when it loads 2-3?
Can you split the loading phase over frames? Lets say.. one load/blit operation and then put a mark to pass one frame.
(I know, can be tricky... but should be possible )
Splitting loading will be a little difficult, especially when several pages are loaded closely.
Google Summer of Code 2012 Student
Topic: "Improve and Demo the Terrain System"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: masterfalcon
Topic: "Improve and Demo the Terrain System"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: masterfalcon
-
- Kobold
- Posts: 26
- Joined: Fri May 23, 2008 10:46 pm
- Location: Spain
- x 1
Re: [GSoC 2012 - Accepted] Improve and Demo the Terrain Syst
OK, I get the crash with all the examples in OpenGL. It is posible an incorrect installation in my system. I must revise it.xiaoxiangquan wrote:Eh, I can not reproduce the crash. Both OpenGL and DX seem to work fine.PacoRG wrote: I have enabled the vSync and the flicker is observable too. It seems independent on the frame rate, the screen resolution and whether the fullscreen mode is enabled.
When I have tried to test the EndLess World example in OpenGL I get a crash when the example try to destroy an overlay not found.
It will always be posible to separate the load of two or more pages by enqueing them and then applying a delay of 1 or 2 second between each load because the multiple load page coincidence will only occur in a disperse way along the time.xiaoxiangquan wrote:Yes, a single page does. To load 2 or 3 pages, the jitter effect will add up.Sadahar wrote:Does a single page loading causes the jitter.. or when it loads 2-3?
Can you split the loading phase over frames? Lets say.. one load/blit operation and then put a mark to pass one frame.
(I know, can be tricky... but should be possible )
Splitting loading will be a little difficult, especially when several pages are loaded closely.
Paco, Spain
-
- Google Summer of Code Student
- Posts: 102
- Joined: Tue Mar 20, 2012 3:20 am
- Location: China
- x 13
Re: [GSoC 2012 - Accepted] Improve and Demo the Terrain Syst
Thanks, Let me have a try to separate them.PacoRG wrote: It will always be posible to separate the load of two or more pages by enqueing them and then applying a delay of 1 or 2 second between each load because the multiple load page coincidence will only occur in a disperse way along the time.
Google Summer of Code 2012 Student
Topic: "Improve and Demo the Terrain System"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: masterfalcon
Topic: "Improve and Demo the Terrain System"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: masterfalcon
-
- Silver Sponsor
- Posts: 2703
- Joined: Mon Aug 29, 2005 3:24 pm
- Location: Kuala Lumpur, Malaysia
- x 51
Re: [GSoC 2012 - Accepted] Improve and Demo the Terrain Syst
I hope you can really solve this jitter for good, because previous GSOC project had done lots of code factoring to solve it but apparently, it is still there.
A willow deeply scarred, somebody's broken heart
And a washed-out dream
They follow the pattern of the wind, ya' see
Cause they got no place to be
That's why I'm starting with me
And a washed-out dream
They follow the pattern of the wind, ya' see
Cause they got no place to be
That's why I'm starting with me
-
- Google Summer of Code Student
- Posts: 102
- Joined: Tue Mar 20, 2012 3:20 am
- Location: China
- x 13
Re: [GSoC 2012 - Accepted] Improve and Demo the Terrain Syst
I've committed two new revs to queue the page-loading, move material-generation to loading time and insert some time intervals to the process.
Now the page-loading seems a little smoother, especially when under release mode.
Now the page-loading seems a little smoother, especially when under release mode.
Google Summer of Code 2012 Student
Topic: "Improve and Demo the Terrain System"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: masterfalcon
Topic: "Improve and Demo the Terrain System"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: masterfalcon
-
- Google Summer of Code Student
- Posts: 102
- Joined: Tue Mar 20, 2012 3:20 am
- Location: China
- x 13
Re: [GSoC 2012 - Accepted] Improve and Demo the Terrain Syst
Weekly Progress:
Tried to make loading page smoother: queueing page-loadings, and divide material generation procedure.
To my knowledge, most of the loading time is spent on generating material. Looking the calling tree below:
Terrain::getMaterial
I have also tried to divide all generateVertexPrograms and generateFragmentPrograms(either of them will be called 3 times in a generation process). Surely it becomes smoother, but the code complexity increases a lot and becomes difficult to read.
Please have a try if you have time, and tell me whether the current effect is acceptable.
Next week I'll startup the terrain generation work. I'm afraid the "Endless World" sample cannot be finished during the gsoc program, but surely I'll keep on and make it complete.
Tried to make loading page smoother: queueing page-loadings, and divide material generation procedure.
To my knowledge, most of the loading time is spent on generating material. Looking the calling tree below:
Terrain::getMaterial
- generate
- addTechnique
- generateVertexProgram
generateFragmentProgram
- generateVertexProgram
generateFragmentProgram
- generateVertexProgram
- addTechnique
- generateVertexProgram
generateFragmentProgram
- generateVertexProgram
- addTechnique
I have also tried to divide all generateVertexPrograms and generateFragmentPrograms(either of them will be called 3 times in a generation process). Surely it becomes smoother, but the code complexity increases a lot and becomes difficult to read.
Please have a try if you have time, and tell me whether the current effect is acceptable.
Next week I'll startup the terrain generation work. I'm afraid the "Endless World" sample cannot be finished during the gsoc program, but surely I'll keep on and make it complete.
Google Summer of Code 2012 Student
Topic: "Improve and Demo the Terrain System"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: masterfalcon
Topic: "Improve and Demo the Terrain System"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: masterfalcon
-
- Google Summer of Code Student
- Posts: 102
- Joined: Tue Mar 20, 2012 3:20 am
- Location: China
- x 13
Re: [GSoC 2012 - Accepted] Improve and Demo the Terrain Syst
See How_to_Testing in wiki.
Google Summer of Code 2012 Student
Topic: "Improve and Demo the Terrain System"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: masterfalcon
Topic: "Improve and Demo the Terrain System"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: masterfalcon
-
- OGRE Team Member
- Posts: 4270
- Joined: Sun Feb 25, 2007 4:56 am
- Location: Bloomington, MN
- x 126
Re: [GSoC 2012 - Accepted] Improve and Demo the Terrain Syst
I need to test it out here but that sounds like a good plan. Keeping the code readable will be very important for maintenance. How much of the Endless World sample do you think you will be able to get finished?
-
- Google Summer of Code Student
- Posts: 102
- Joined: Tue Mar 20, 2012 3:20 am
- Location: China
- x 13
Re: [GSoC 2012 - Accepted] Improve and Demo the Terrain Syst
I hope I can finish the alpha version in two weeks, just before the official Pencil-Down. And then make it better in another two weeks.masterfalcon wrote:I need to test it out here but that sounds like a good plan. Keeping the code readable will be very important for maintenance. How much of the Endless World sample do you think you will be able to get finished?
Google Summer of Code 2012 Student
Topic: "Improve and Demo the Terrain System"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: masterfalcon
Topic: "Improve and Demo the Terrain System"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: masterfalcon