[iOS] Touch coordinates wrong on retina iPad

Discussion of issues specific to mobile platforms such as iOS, Android, Symbian and Meego.
Post Reply
simedj
Goblin
Posts: 262
Joined: Fri Nov 18, 2011 6:50 pm
x 3

[iOS] Touch coordinates wrong on retina iPad

Post by simedj »

I have a feeling of deja vu typing this but can't find a similar thread...

My app is targeting iPad only, not iPhone. On non-retina iPad it works great, but in the retina simulator I have input problems - it renders fine but touches are reporting coordinates based on the view size (in 2048x1536 resolution) which messes up since Ogre thinks the display is 1024x768.

Here's my utility code, based on some Ogre sample code IIRC:

Code: Select all

	OIS::MouseState MainView::ConvertMultiTouch(const OIS::MultiTouchState &in)
	{
		Ogre::Viewport *vp = m_renderTarget.getViewport(0);
		OIS::MultiTouchState state = in;

		int w = vp->getActualWidth();
		int h = vp->getActualHeight();
		int absX = state.X.abs;
		int absY = state.Y.abs;
		int relX = state.X.rel;
		int relY = state.Y.rel;

		UIInterfaceOrientation interfaceOrientation = [UIApplication sharedApplication].statusBarOrientation;
		switch (interfaceOrientation)
		{
		case UIInterfaceOrientationPortrait:
			break;
		case UIInterfaceOrientationLandscapeLeft:
			state.X.abs = w - absY;
			state.Y.abs = absX;
			state.X.rel = -relY;
			state.Y.rel = relX;
			break;
		case UIInterfaceOrientationPortraitUpsideDown:
			state.X.abs = w - absX;
			state.Y.abs = h - absY;
			state.X.rel = -relX;
			state.Y.rel = -relY;
			break;
		case UIInterfaceOrientationLandscapeRight:
			state.X.abs = absY;
			state.Y.abs = h - absX;
			state.X.rel = relY;
			state.Y.rel = -relX;
			break;
		}
		OIS::MouseState s;
		s.X = state.X;
		s.Y = state.Y;
I'm using Ogre 1.8 with pre-compiled iOS dependencies including OIS.

I wondered what the correct way is to get touch coordinates into viewport space safely? This works great on non-retina but it seems to me my code would ideally not have to check if I'm on a retina device! And also I have a feeling this might only work when my renderwindow is fullscreen.

I also wondered, in Ogre 1.9 and latest version of dependencies, if there are fixes relevant here.
Looking to find experienced Ogre & shader developers/artists. PM me with a contact email address if interested.
User avatar
masterfalcon
OGRE Team Member
OGRE Team Member
Posts: 4270
Joined: Sun Feb 25, 2007 4:56 am
Location: Bloomington, MN
x 126
Contact:

Re: [iOS] Touch coordinates wrong on retina iPad

Post by masterfalcon »

1.9 Definitely works great on retina devices here.

I would recommend using your own input handling rather than OIS. There are several threads and wiki pages about using a separate UIView just for input handling.
simedj
Goblin
Posts: 262
Joined: Fri Nov 18, 2011 6:50 pm
x 3

Re: [iOS] Touch coordinates wrong on retina iPad

Post by simedj »

But did you ever see this problem in 1.8 with custom UIView... otherwise it doesn't answer if the issue is OIS or Ogre. Since OIS simply wraps a UIView anyway, how would using your own make a difference in this specific case where coords are reported in a different resolution to viewport?

I should note I am not rendering at retina resolution, if I was I probably wouldn't have a problem.
Looking to find experienced Ogre & shader developers/artists. PM me with a contact email address if interested.
User avatar
masterfalcon
OGRE Team Member
OGRE Team Member
Posts: 4270
Joined: Sun Feb 25, 2007 4:56 am
Location: Bloomington, MN
x 126
Contact:

Re: [iOS] Touch coordinates wrong on retina iPad

Post by masterfalcon »

I have not. All input handling is done in OIS so…that question is answered. I wrote the multitouch support for OIS but I'm not very proud of it. Which is why I recommend rolling your own, it's actually less complex than implementing it with OIS.
simedj
Goblin
Posts: 262
Joined: Fri Nov 18, 2011 6:50 pm
x 3

Re: [iOS] Touch coordinates wrong on retina iPad

Post by simedj »

I actually thought I'd seen some more recent updates to OIS, not sure what they were about.

But on the core issue, on a retina iPad will UIView always report size of the actual resolution used, or what? I know Ogre has this scale factor functionality but does that mirror iOS? I'm just wondering if I run on an iPad 3, if my UIView should be 2048x1536 or should be 1024x768 if my app is non-retina?
Looking to find experienced Ogre & shader developers/artists. PM me with a contact email address if interested.
User avatar
masterfalcon
OGRE Team Member
OGRE Team Member
Posts: 4270
Joined: Sun Feb 25, 2007 4:56 am
Location: Bloomington, MN
x 126
Contact:

Re: [iOS] Touch coordinates wrong on retina iPad

Post by masterfalcon »

The UIView is 1024x768 but the scaling factor is 2.0. It's a weird topic but Apple does a fairly good job as describing it.
simedj
Goblin
Posts: 262
Joined: Fri Nov 18, 2011 6:50 pm
x 3

Re: [iOS] Touch coordinates wrong on retina iPad

Post by simedj »

OK, scaling is actually a feature of iOS, that's something to read up on.

Thanks!
Looking to find experienced Ogre & shader developers/artists. PM me with a contact email address if interested.
simedj
Goblin
Posts: 262
Joined: Fri Nov 18, 2011 6:50 pm
x 3

Re: [iOS] Touch coordinates wrong on retina iPad

Post by simedj »

Now I have a full build-from source on iOS, I've been able to actually step through. I've built with ogre 1.9 and the dependencies from source and the issue still exists, touches are reported in retina resolution.

It seems the problem is caused here:

Code: Select all

void iPhoneMultiTouch::_touchBegan(UITouch *touch)
{
    UIView *touchView = static_cast<iPhoneInputManager*>(mCreator)->_getDelegate();
    CGPoint location = [touch locationInView:touchView];
    CGFloat contentScale = 1.0;
#if __IPHONE_4_0
    if([touchView respondsToSelector:@selector(contentScaleFactor)])
        contentScale = [[UIScreen mainScreen] scale];
#endif

    MultiTouchState newState;
    newState.X.abs = location.x * contentScale;
    newState.Y.abs = location.y * contentScale;
    newState.touchType |= 1 << MT_Pressed;

    if( mListener && mBuffered )
    {
        mListener->touchPressed(MultiTouchEvent(this, newState));
    }
    else
    {
        mStates.push_back(newState);
    }
}
Specifically, contentScale = [[UIScreen mainScreen] scale]; returns 2.0.

Now I know all the advice is "don't use IOS" but still, apart from this one issue you can write cross-platform input functionality very easily indeed... sure you'd want proper touch support but for prototyping, etc, it's very useful. So I would appreciate any help here - my OBJ-C is very patchy so I can trace through the code but I don't know why the scale is 2. And I'm pretty sure it shouldn't be. I don't actually know what UIView OIS is attaching itself to, but since Ogre supports scale-factor, shouldn't Ogre be setting it to 1 here?

Aside from all that... is there a simple fix I can do to my code which will make this work, as a band-aid, or is it only possible to fix Ogre/OIS (I'm not sure which one is at fault)?

Thanks for any help - and hopefully this investigation might inspire a fix and/or help other people.
Looking to find experienced Ogre & shader developers/artists. PM me with a contact email address if interested.
User avatar
masterfalcon
OGRE Team Member
OGRE Team Member
Posts: 4270
Joined: Sun Feb 25, 2007 4:56 am
Location: Bloomington, MN
x 126
Contact:

Re: [iOS] Touch coordinates wrong on retina iPad

Post by masterfalcon »

Yeah, you're right. OIS should be using the scaling factor from the view that it is being attached to. I'll take a look at this relatively soon. In the meantime you can just comment out the two lines in the __IPHONE_4_0 ifdef.
simedj
Goblin
Posts: 262
Joined: Fri Nov 18, 2011 6:50 pm
x 3

Re: [iOS] Touch coordinates wrong on retina iPad

Post by simedj »

Many thanks. I can't imagine I'll be wanting to render at 2048x1536 any time soon :)
Looking to find experienced Ogre & shader developers/artists. PM me with a contact email address if interested.
simedj
Goblin
Posts: 262
Joined: Fri Nov 18, 2011 6:50 pm
x 3

Re: [iOS] Touch coordinates wrong on retina iPad

Post by simedj »

I was doing some quick testing, seeing if touchView.contentScaleFactor would work, but that has the opposite problem - it's always 1.0 even if Ogre is using 2.0. I guess some communication between Ogre/IOS is needed, and that goes above my understanding or I'd submit a patch.

I DO plan to modify OIS locally so all that stuff gets pulled into a method rather than being repeated in 4 places. I don't know how I might submit it for inclusion, maybe I'll just post a patch here in case an existing contributor wants to...
Looking to find experienced Ogre & shader developers/artists. PM me with a contact email address if interested.
User avatar
masterfalcon
OGRE Team Member
OGRE Team Member
Posts: 4270
Joined: Sun Feb 25, 2007 4:56 am
Location: Bloomington, MN
x 126
Contact:

Re: [iOS] Touch coordinates wrong on retina iPad

Post by masterfalcon »

I have a fix for this all lined up. The next release of the dependencies will include part of the fix, the other part involves passing your view pointer to OIS so that it can get the contentScaleFactor from it. Like this:

Code: Select all

			std::ostringstream viewHandleStr;
			size_t viewHandle = 0;
			mWindow->getCustomAttribute("VIEW", &viewHandle);
			viewHandleStr << viewHandle;
            pl.insert(std::make_pair("VIEW", viewHandleStr.str()));
Post Reply