[2.2] How to not load first(s) mipmaps on textures

Discussion area about developing with Ogre2 branches (2.1, 2.2 and beyond)
Post Reply
xrgo
OGRE Expert User
OGRE Expert User
Posts: 1046
Joined: Sat Jul 06, 2013 10:59 pm
Location: Chile
x 74

[2.2] How to not load first(s) mipmaps on textures

Post by xrgo » Thu Mar 14, 2019 3:52 am

Hello! what would be the correct 2.2 way to load lower resolution versions of textures (dds that have generated mipmaps) that actually use less vram?
like this:
viewtopic.php?f=25&t=83037&p=518003&#p518003

I tried this:

Code: Select all

                Ogre::Image2 img;
                std::ifstream ifs(texture->mTextureData.mFileName.c_str(), std::ios::binary|std::ios::in);
                //create image from texture buffer
                Ogre::DataStreamPtr data_stream(new Ogre::FileStreamDataStream(texture->mTextureData.mFileName, &ifs, false));
                img.load( data_stream );

                img.uploadTo(ogreTexture, 0, 0); //mip 0 just to test
but I get a crash on uploadTo...

Also related:
I will going to need to use Ogre::Image2 later because we are working on encripting the texture files... so using Ogre::Image2 for the mip thing would be ideal, but the (second) question is: If I use img.uploadTo(ogreTexture... is there a way to AutomaticBatching these textures?

thanks!
0 x

xrgo
OGRE Expert User
OGRE Expert User
Posts: 1046
Joined: Sat Jul 06, 2013 10:59 pm
Location: Chile
x 74

Re: [2.2] How to not load first(s) mipmaps on textures

Post by xrgo » Fri Mar 15, 2019 4:19 pm

I have this code:

Code: Select all

                    Ogre::Image2 img;
                    std::ifstream ifs(texture->mTextureData.mFileName.c_str(), std::ios::binary|std::ios::in);
                    //create image from texture buffer
                    Ogre::DataStreamPtr data_stream(new Ogre::FileStreamDataStream(texture->mTextureData.mFileName, &ifs, false));
                    img.load( data_stream );

                    ogreTexture = mTextureManager->createTexture( texture->mTextureData.mName,
                                                                  Ogre::GpuPageOutStrategy::Discard,
                                                                  Ogre::TextureFlags::ManualTexture,
                                                                  Ogre::TextureTypes::Type2DArray );

                    int initialDesiredMip = 0;
                    int initialMip = std::min( initialDesiredMip, img.getNumMipmaps()-1 );

                    ogreTexture->setResolution( img.getWidth(), img.getHeight() );
                    ogreTexture->setNumMipmaps( img.getNumMipmaps() - initialMip );
                    ogreTexture->setPixelFormat( img.getPixelFormat() );
                    ogreTexture->_transitionTo( Ogre::GpuResidency::Resident, (uint8*)0 );

                    img.uploadTo(ogreTexture, initialMip, img.getNumMipmaps()-1);
that its kind of working (on debug)... colors/normals looks weird, but I am guessing that is because I am not considering gamma stuffs yet... but! its crashing on release when the texture its actually being setted (datablock->settexture)

also, I tried to use other than 0 initialDesiredMip (to achieve what I really need), and it crashes (also on debug) it says "Texture and Image must have matching resolution and format!"
0 x

User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 4077
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 231
Contact:

Re: [2.2] How to not load first(s) mipmaps on textures

Post by dark_sylinc » Fri Mar 15, 2019 7:27 pm

You're possibly missing a call to ogreTexture->notifyDataIsReady and a call to ogreTexture->_setNextResidencyStatus (which is done for you if isManualTexture returns true; but not otherwise)
(also on debug) it says "Texture and Image must have matching resolution and format!"
This error is self explanatory. Step in to check what is exactly failing.
0 x

xrgo
OGRE Expert User
OGRE Expert User
Posts: 1046
Joined: Sat Jul 06, 2013 10:59 pm
Location: Chile
x 74

Re: [2.2] How to not load first(s) mipmaps on textures

Post by xrgo » Fri Mar 15, 2019 7:57 pm

I tried ogreTexture->notifyDataIsReady and its the same, keep crashing in release... also the texture is returning true on isManualTexture
dark_sylinc wrote:
Fri Mar 15, 2019 7:27 pm
This error is self explanatory. Step in to check what is exactly failing.
yeah but I am actually copying the values...

Code: Select all

                    ogreTexture->setResolution( img.getWidth(), img.getHeight() );
                    ogreTexture->setNumMipmaps( img.getNumMipmaps() - initialMip );
                    ogreTexture->setPixelFormat( img.getPixelFormat() );
except the NumMipmaps! so I tried using the same number, and in uploadTo just don't upload the first X ones:

Code: Select all

ogreTexture->setNumMipmaps( img.getNumMipmaps() );// - initialMip );
img.uploadTo(ogreTexture, initialMip, img.getNumMipmaps()-1);
and it doesn't crash (in debug) but the texture looks black when I am near it (obviously since I didn't uploaded the firsts mipmaps), so its not what I am looking for... I need the texture to actually have less mipmaps, completely omit the firsts X ones so it uses less vram... is this even the correct approach to achieve this?
0 x

dermont
Orc Shaman
Posts: 782
Joined: Thu Dec 09, 2004 2:51 am
x 9

Re: [2.2] How to not load first(s) mipmaps on textures

Post by dermont » Fri Mar 15, 2019 8:48 pm

You could try checking the assert isn't caused by the image/texture having different texture types and different DepthOrSlices e.g (6/1) .

You can test by loading the texture with TextureType img.getTextureType().
0 x

xrgo
OGRE Expert User
OGRE Expert User
Posts: 1046
Joined: Sat Jul 06, 2013 10:59 pm
Location: Chile
x 74

Re: [2.2] How to not load first(s) mipmaps on textures

Post by xrgo » Sat Mar 16, 2019 3:38 am

thanks, but If I use img.getTextureType() on createTexture (on debug so it wont crash) I don' t see any texture, that's because in order to work on pbs it has to be a 2DArray
I also checked and both texture and img has only 1 slice

but independent of the crash, I still see the object with weird colors/normals (I am using normal maps), and now I made sure I was using the correct gamma... plus I believe I will lose texture batching (hence performance)(?), so I am still wondering if this is the correct approach, lets forget that I mentioned that I would like to use image2 because I'll later want to encrypt my files... that' s a future (low priority) topic.

I was trying to implement the solution I used in 2.1 (viewtopic.php?f=25&t=83037&p=518003&#p518003) but I am bit lost with the new textureGpuManager :P
0 x

xrgo
OGRE Expert User
OGRE Expert User
Posts: 1046
Joined: Sat Jul 06, 2013 10:59 pm
Location: Chile
x 74

Re: [2.2] How to not load first(s) mipmaps on textures

Post by xrgo » Sat Mar 16, 2019 4:34 am

this code I found on Terra seems to be close:

Code: Select all

                    Ogre::Image2 image;
                    std::ifstream ifs(texture->mTextureData.mFileName.c_str(), std::ios::binary|std::ios::in);
                    //create image from texture buffer
                    Ogre::DataStreamPtr data_stream(new Ogre::FileStreamDataStream(texture->mTextureData.mFileName, &ifs, false));
                    image.load( data_stream );

                    ogreTexture = mTextureManager->createTexture( texture->mTextureData.mName,
                                                                  Ogre::GpuPageOutStrategy::Discard,
                                                                  Ogre::TextureFlags::ManualTexture,
                                                                  Ogre::TextureTypes::Type2DArray );

                    const uint8 numMipmaps = 1;//image.getNumMipmaps();

                    ogreTexture->setResolution( image.getWidth(), image.getHeight() );
                    ogreTexture->setNumMipmaps( numMipmaps );
                    ogreTexture->setPixelFormat( image.getPixelFormat() );
                    ogreTexture->scheduleTransitionTo( Ogre::GpuResidency::Resident );


                    Ogre::StagingTexture *stagingTexture = mTextureManager->getStagingTexture( image.getWidth(),
                                                                                        image.getHeight(),
                                                                                        1u, 1u,
                                                                                        image.getPixelFormat() );
                    stagingTexture->startMapRegion();
                    Ogre::TextureBox texBox = stagingTexture->mapRegion( image.getWidth(), image.getHeight(), 1u, 1u,
                                                                   image.getPixelFormat() );

                    for( uint8 mip=0; mip<numMipmaps; ++mip ){
                        texBox.copyFrom( image.getData( mip ) );
                        stagingTexture->upload( texBox, ogreTexture, mip, 0 );
                    }
                    stagingTexture->stopMapRegion();
                    mTextureManager->removeStagingTexture( stagingTexture );
                    stagingTexture = 0;

                    ogreTexture->notifyDataIsReady();
with this (I think) I can map the mipmaps how I want... but If I use numMipmaps = image.getNumMipmaps(); I have a crash on copyFrom, also, even with numMipmaps = 1 I have a crash in release. plus the batching thingy

cheers!!
0 x

User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 4077
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 231
Contact:

Re: [2.2] How to not load first(s) mipmaps on textures

Post by dark_sylinc » Sat Mar 16, 2019 7:26 am

xrgo wrote:
Sat Mar 16, 2019 3:38 am
plus I believe I will lose texture batching (hence performance)(?),
You don't lose performance. Automatic Batching means we take care of it and pretend the texture arrays are separate 2D textures. But if you batch them yourself by hand, that works too.

Btw I spotted this:
int initialMip = std::min( initialDesiredMip, img.getNumMipmaps()-1 );

This is wrong. It will always be 0. I don't know what you're trying to do there. Maybe you wanted to set i itialDesiredMip to 1 instead of 0?
Btw downgrading texture resolution is planned. This is something Ogre should do behind the scenes.

Edit: nevermind, you said you set to 1 and crashes, and 0 glitches. I misunderstood

Edit 2: there may be a bug with uploadTo.
0 x

xrgo
OGRE Expert User
OGRE Expert User
Posts: 1046
Joined: Sat Jul 06, 2013 10:59 pm
Location: Chile
x 74

Re: [2.2] How to not load first(s) mipmaps on textures

Post by xrgo » Sat Mar 16, 2019 12:11 pm

dark_sylinc wrote:
Sat Mar 16, 2019 7:26 am
But if you batch them yourself by hand, that works too.
Yeah! I just thought that I could use the automatic one
dark_sylinc wrote:
Sat Mar 16, 2019 7:26 am
Btw downgrading texture resolution is planned. This is something Ogre should do behind the scenes.
Yuhuuu! Any ETA? I have a client with a low vram PC waiting... not that I don't have any other option (resizing the textures) but I prefer the final solution at once.

Thank you so much!
0 x

User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 4077
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 231
Contact:

Re: [2.2] How to not load first(s) mipmaps on textures

Post by dark_sylinc » Mon Mar 18, 2019 2:22 am

A bug has been fixed when 2D arrays are used with uploadTo, so please try again. Perhaps your crashes/glitches were just a manifestation of this bug.
xrgo wrote:
Sat Mar 16, 2019 12:11 pm
dark_sylinc wrote:
Sat Mar 16, 2019 7:26 am
Btw downgrading texture resolution is planned. This is something Ogre should do behind the scenes.
Yuhuuu! Any ETA? I have a client with a low vram PC waiting... not that I don't have any other option (resizing the textures) but I prefer the final solution at once.
I can't give you one, but it isn't too hard to implement (but not 2 lines of code either):
1. There needs to be an API to indicate you want lower MIPs. Maybe create a TextureGpu::mLowestMip plus a global default value in TextureGpuManager?
We also have GpuResource::mRank which is currently not being used but it is documented what it should do. I don't know yet how mRank would interact with this

2. TextureGpuManager::processLoadRequest and TextureGpuManager::processQueuedImage are the main places that would need modifications, so that they ignore the mips lower than TextureGpu::mLowestMip that are present in Image2.
Perhaps instead of ignoring, it would MUCH easier to clone the Image2 pointer with fewer mips. We already support cloning the data due to the needsMultipleImages && mustKeepSysRamPtr path.
1 x

xrgo
OGRE Expert User
OGRE Expert User
Posts: 1046
Joined: Sat Jul 06, 2013 10:59 pm
Location: Chile
x 74

Re: [2.2] How to not load first(s) mipmaps on textures

Post by xrgo » Mon Mar 18, 2019 3:30 am

Thank you I am going to check all that!!
0 x

xrgo
OGRE Expert User
OGRE Expert User
Posts: 1046
Joined: Sat Jul 06, 2013 10:59 pm
Location: Chile
x 74

Re: [2.2] How to not load first(s) mipmaps on textures

Post by xrgo » Mon Mar 18, 2019 9:22 pm

yay its working!!

I have used this code:

Code: Select all

		if (mSkipMip && img->getTextureType() == TextureTypes::Type2D ){

			const size_t sizeBytes = img->getSizeBytes();
			void *sysRamCopy2 = reinterpret_cast<uint8*>(
						OGRE_MALLOC_SIMD(sizeBytes, MEMCATEGORY_RESOURCE));

			Image2* imgDst = OGRE_NEW Image2();

			int numMips2 = img->getNumMipmaps();
			int mipsToSkip = std::min(1, numMips2 - 1);
			float scaleFactor = 1.0;
			if (mipsToSkip == 1) scaleFactor = 0.5;

			imgDst->loadDynamicImage(sysRamCopy2, img->getWidth()*scaleFactor, img->getHeight()*scaleFactor,
				img->getNumSlices(),
				img->getTextureType(), img->getPixelFormat(),
				true, numMips2 - mipsToSkip);


			for (uint8 mip = 0; mip < numMips2 - mipsToSkip; ++mip)
			{
				TextureBox srcBox = img->getData(mip + mipsToSkip);
				TextureBox dstBox = imgDst->getData(mip);
				dstBox.copyFrom(srcBox);
			}
			img = imgDst;
		}
right after

Code: Select all

		//Load the image from file into system RAM
		Image2 imgStack;
		Image2 *img = loadRequest.image;

		if (!img)
		{
.............
		}
around line 2518 in OgreTextureGpuManager.cpp

its a bit dirty, it uses a global bool that just enabled skip the first mip, but that's enough for my usage, and it does uses less ram. I hope this functionality gets in to ogre officially soon!

thanks for everything!
Last edited by xrgo on Mon May 27, 2019 8:11 pm, edited 1 time in total.
0 x

User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 4077
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 231
Contact:

Re: [2.2] How to not load first(s) mipmaps on textures

Post by dark_sylinc » Tue Mar 19, 2019 1:02 am

A crude solution. But simple enough to perhaps include it in Ogre with little work.
1 x

Post Reply