[GSoC 2011] Texture streaming

[red[]]

Hi everyone,

my name is Marco, I'm an Italian student, currently attending a master degree in Information Science. It's been some years since I've decided to do the GSoC, but I always had something to do, like my bachelor's degree thesis or other works... but, this year it's finally I hope I can attend the GSoC!

I've always been interested in game development, though I've never done anything in this field. I've quite an experience in programming though: I programmed Java, C, C#, C++, C++/CLI, and many more. The major projects I've worked on are an interpreter-compiler for a stochastic programming language for simulation of biological structures (Monte Carlo simulations, Gillespie's algorithm, like that), developed in C, Flex and Bison, and a surveillance suite based on proprietary libraries, entirely in C++/CLI, for a company in my city. These works gave me a strong background in object programming, thread management, and (ugh!) pointers

I'm actually planning to develop a game in OGRE / MOGRE with some of my friend, so I thought it could be very interesting give some help to OGRE so I examined the Help Requested Section and I found the Texture Streaming request, which really turned my interest on. I'd really like to implement the requests in the page, especially the first three steps seem really interesting and challenging to me

I don't know if the work is too big (or either too small, but I really don't think so ) for doing it in a GSoC, but I'd really like to give my help to Ogre and participating in the GSoC this year!

Cheers

Red

[so0os]

This is the best idea so far, imo!

Good luck with that!

[red[]]

This is the best idea so far, imo!

Good luck with that!

Thanks I hope to find a mentor for this!

Is there a place where I can apply for the GSoC with OGRE?

[spacegaier]

Noman is doing the organisation for GSoC this year. So best to directly contact him via PM:
http://www.ogre3d.org/forums/memberlist ... ile&u=3067

[red[]]

Noman is doing the organisation for GSoC this year. So best to directly contact him via PM:
http://www.ogre3d.org/forums/memberlist ... ile&u=3067

Thank you, I'll contact him right now.

[Noman]

Answered your PM.

As for the idea, its always nice to have someone pick up an item from the help requested list.
If you want to get serious about it, I would suggest reading a bit about it (perhaps find papers/demos about it) and establishing a more detailed plan of attack than the one listed in the help requested wiki page.

[red[]]

Answered your PM.

As for the idea, its always nice to have someone pick up an item from the help requested list.
If you want to get serious about it, I would suggest reading a bit about it (perhaps find papers/demos about it) and establishing a more detailed plan of attack than the one listed in the help requested wiki page.

Thanks, I'll definitely do it. Here in Italy it's quite late, but tomorrow I'll begin to work on it

[red[]]

Answered your PM.

As for the idea, its always nice to have someone pick up an item from the help requested list.
If you want to get serious about it, I would suggest reading a bit about it (perhaps find papers/demos about it) and establishing a more detailed plan of attack than the one listed in the help requested wiki page.

Actually my early plans are:

- make a demo in ogre to take familiarity with the environment
- study texture theory on a book like Real-Time Rendering, by Tomas Akenine-Möller and Eric Haines, which I can get from a friend of mine
- study the structure of OGRE and understand how does it implement Texture now

About the third point, I had a look in the OGRE Manual and Wiki, but there are not so many details about the TextureManager class in OGRE. How can I find detailed documentation about this?

I actually develop on Windows and Visual Studio, I hope it's not a problem for you.

Thanks,

Marco

[icepick]

Dang, that's the task I was going to look into. : ) Let me know if you decide not to do it or want help.

[red[]]

Dang, that's the task I was going to look into. : ) Let me know if you decide not to do it or want help.

Well, I don't think the thing is easy to do and neither small, but I'd like some OGRE developer confirm this if the work is big, and the OGRE team accepts us both, it would be really nice to work in team!

[red[]]

OK, I started to work for this (hopefully) GSoC project. I'm developing a small project to understand the OGRE logic and texturing and Thursday I'll start the 3D graphics university course. I'm trying to understand how Ogre manages LOD and how could I implement the texture streaming management in it; I have not downloaded the full OGRE source code yet, because I think is no use to analyze some code without knowing how it works but I plan to do it ASAP.

The only question that I have is, when should the low LOD texture fade to the high LOD texture? I have tree options in my mind:

- set with hardcoded fixed distance,
- set by the user on the camera, i.e. like the ClippingDistance
- most complex, set the switching distance for every texture at creation

Naturally I think that the third option is the best, but I'd like to have some feedback for this

I'd really like to talk to a OGRE developer in the IRC channel to know more about the OGRE structure, and how the texture streaming could be implemented. When can I find someone to talk with? I have the exactly same nick in #ogre3d, you can find me mostly there at Italian evening (here is +1GMT; look at the timestamp of this post, now it's 10.30 pm here )

[Noman]

At first glance, texture streaming is somewhat parallel to mesh LOD settings. There has been work on level of detail control in the past (with different LOD strategies, etc). Perhaps the two should be connected in a way (LOD strategy would also answer texture queries).
If you were to create such a connection, you would automatically get all of the implemented strategies (distance from camera, on-screen pixel count, and any custom extensions) for free. I would start investigating the current LOD system to see if/how such a connection could work.

But this can be more than just LOD work for performance reasons. Consider an application that has too many textures to load them all at startup. When a texture is requested, you might want the mipmaps to be loaded gradually to enable to instantly display the textured object, with the visual quality increasing with time (more mipmaps finished loading) rather than camera movement.

[reptor]

You may look at the PC game Armed Assault by Bohemia Interactive as an example of how terrible it will look when textures are loading gradually and when it's not quite working as it should. I had a below-minimum spec computer to run it on so I can't really complain on my part about it but other people with much better computers also had the same problem with it. I don't know what exactly was their mistake but try to avoid it! I just throw it in here as a bad example of gradual texture loading, that's all...

[red[]]

At first glance, texture streaming is somewhat parallel to mesh LOD settings. There has been work on level of detail control in the past (with different LOD strategies, etc). Perhaps the two should be connected in a way (LOD strategy would also answer texture queries).
If you were to create such a connection, you would automatically get all of the implemented strategies (distance from camera, on-screen pixel count, and any custom extensions) for free. I would start investigating the current LOD system to see if/how such a connection could work.

Thanks, I'll give a look to the LOD system ASAP.

But this can be more than just LOD work for performance reasons. Consider an application that has too many textures to load them all at startup. When a texture is requested, you might want the mipmaps to be loaded gradually to enable to instantly display the textured object, with the visual quality increasing with time (more mipmaps finished loading) rather than camera movement.

Do you mean, like how texture are managed in Bioshock for example?

You may look at the PC game Armed Assault by Bohemia Interactive as an example of how terrible it will look when textures are loading gradually and when it's not quite working as it should. I had a below-minimum spec computer to run it on so I can't really complain on my part about it but other people with much better computers also had the same problem with it. I don't know what exactly was their mistake but try to avoid it! I just throw it in here as a bad example of gradual texture loading, that's all...

Never played the title, but I'll definitely give it a try. Knowing what NOT to do is a good way to start is there a demo of the game freely available?

[reptor]

http://www.armedassault.com/dwnl_demo.html

I have to say, they did improve it quite a bit since the first release but the amount of patching was ridiculous and I don't think it ever really came out like it should have been to begin with. I am sure ArmA 2 is much better (I played the demo and it seems much better).

[fzh]

Another example is The Last Remnant by squaresoft, they are using unreal engine, but the way they do it is that an ugly pixellelated texture is first loaded before the higher detailed textured are loaded, it feels quite bad.

From what I can understand, I think what you meant is degrading the texture depending on distance from the camera, and loaded when camera moves, some what like a "fog". There would probably need for some user control for LOD texture, texture quality at different distance from the camera should be able to be specified so that the user can actually create a layered feel to the graphics, which would mean that the distance at which LOD change would be rather user-specific depending on the scale of the game.

Didn't play armed assault so not sure how bad it would it be, probably going to try it out sometime.

[red[]]

Another example is The Last Remnant by squaresoft, they are using unreal engine, but the way they do it is that an ugly pixellelated texture is first loaded before the higher detailed textured are loaded, it feels quite bad.

This is exactly the same look and feel of Bioshock, which uses Unreal Engine too. I recently played Bioshock 2 and, until the HQ texture are loaded, it displays a extremely ugly, low resolution, low detail texture. I'll surely try to avoid this effect

[Praetor]

Controlling how low the low-res version of the texture will be can be something left to the application developer. I'm going to just do a brain dump here on this topic, both questions to think about and comments:

• -Can the system be made not to use extra texture memory? Will both the high res and low res version have to be in texture memory at the same time at one point? Can you exploit mipmaps somehow (assaf demonstrated the ability to selectively load mipmaps I think in DX)?

• - How will "streamed" textures to used in Ogre's resource system? Is nothing changed and the resource system just views it as 1 texture, or do you think a new "streamed texture" resource type is needed which provides resource-level loading options. Will this system have hooks in the scripting system?

• - How is the switch from low res to high res done? What are the strategies and how do we design a system so new strategies can be added or combined?
  • * Do you just switch the high res texture in the instant it is ready?
  • * Do you prioritize switching textures outside or at the edge of the camera's view so the user doesn't see the switch?
  • * Cross-fading? Can this be done by using mipmap bias to slowly fade in the higher mip level, or do we need the two-texture approach and blend between them which uses more texture memory and need custom material and/or shader support (for platforms which have no fixed function)?

[red[]]

Uh, no luck for me. I broke my glasses yesterday and I'm forced to use the old ones for a week, which cause my heavy headache when reading on pc. Anyway, I'm not going to stop my work

/*
* forget this. restarting thinking from scratch
*/

Sorry if I am not clear in what I write of if I made some English mistakes but, as I said before, I have tremendous headache so I'm not totally sure of what I'm writing right now

[red[]]

Back and now fully operational. Happy to see OGRE3D in the GsOC 2011 list, and I hope to work for OGRE too

I don't have any significant update about the topic, but I'm doing some C++ (re)practice and I'm developing in OpenGL to take familiarity with 3D programming. Still my 3D graphics course at uni is doing well.

I have just a question: can I have my (eventual) GsOC 2011 activity recognized by the University, i.e. as a laboratory? I'd really like to because this would be a nice way to have some help from a professor. The professor that teaches at my 3D graphics course seems quite (a lot) skilled in the topic, so if I can have some help from him and get this help recognized would be great

[Assaf Raman]

Have a look at this thread - it can help you load dds files without its better quality level (top mipmap).

[spacegaier]

I have just a question: can I have my (eventual) GsOC 2011 activity recognized by the University, i.e. as a laboratory? I'd really like to because this would be a nice way to have some help from a professor.

http://www.google-melange.com/document/ ... rse_credit

[red[]]

Have a look at this thread - it can help you load dds files without its better quality level (top mipmap).

Thanks, very interesting reading!

I have just a question: can I have my (eventual) GsOC 2011 activity recognized by the University, i.e. as a laboratory? I'd really like to because this would be a nice way to have some help from a professor.

http://www.google-melange.com/document/ ... rse_credit

Actually, I'm quite an idiot, because I've read this page about a thousand times and always skipped this.

[red[]]

Ok, I think I need to start again from scratch. I went deeper in understanding various how Texture Streaming has been implemented by now and I really need to understand *what* do you want to be implemented in the OGRE engine.

I assume you'd like something like this: http://www.youtube.com/watch?v=i78RQPg7dzE , watch at 00:27 .

If this is what you can see in the video, I think memory management is something I have to think a lot. As I can understand by now, this is the most challenging point. Having to load/unload lots of textures may be very stressing for the system. Many strategies could be used:

• Loading all the lowest details MIPS of the scene to have something ready to display, and load dynamically the higher mips basing the decision on the distance from the camera and what is on/off screen

• Load the lowest detail MIPS, and then replace them all with the higher detail mipmaps asap, disregarding of whether these MIPS are actually needed or not

I assume that we choose the first options; this causes memory management problem, basically on about *what* texture do we have to load and *when* can we unload a texture. The problems are:

• Do we always have to keep the lower MIPS in memory, or do we have do discard it when they're no more needed?

• What is the priority of loading the higher level mips? I think that the on-screen objects should have priority, even if the "texture popping" is annoying

• When can I delete a MIP? More specifically: do I have to discard it as soon it no more needed to minimize memory occupation, or do I have to keep a "garbage collector" of the recently used MIPS and reuse them if needed, in order to minimize disk access?

I think that we need to keep the lower MIPS and, I will hate this words if I have to implement it, an efficient garbage collector has to be implemented. Wasting a texture that could be requested 10 ms later is no sense - a good garbage collector is a must to do texture streaming the right way. So, I think that this could be the most difficult part of the feature - making quick garbage collection is not easy, and selecting which MIPS are to be deleted first doesn't seem very easy too.

About the OGRE Resource system questions, I think I have to think more. Let me analyse the OGRE code and documentation deeper and I can give you an answer.

I personally think that texture must be displayed on screen ASAP, prioritizing texture inside the camera view to give the user the better detail possible; "texture popping", as some user describes it, may be annoying if is present all throughout a game, but I think if it's limited to the first 10-15 second of the game experience it is acceptable. Popping could be made less annoying using shaders or fading to make the changes less evident to the eye of the user.

Last but not least, I asked my 3D graphics professor for support for my eventual Google Summer Of Code and he said he can help me, on whatever I need. He already known well the OGRE3D engine and he seemed interested to the whole thing.

Can I ask you another thing: How do I have to make my proposal to OGRE3d? Can I see some document from older GSoCs to see how things went in the past years? I've already read the old threads but I always hope to see something more

Thanks,

Red

[Noman]

Some of this year's other applications are already in the correct format, so you can see how they did it.
Eventually, you will have to submit yours at the google-melange.com web site.

Good luck!