bidi support for right to left languages

What it says on the tin: a place to discuss proposed new features.
Post Reply
dr_amr
Halfling
Posts: 65
Joined: Wed Mar 02, 2005 2:22 am

bidi support for right to left languages

Post by dr_amr »

hello,
it would be nice if we had support for rtl scripts like hebrew and arabic.
this would include bidirectional text support and right allignment for text boxes and the like, (I understand unicode support is already done).
I know that this might belong more to gui libs , but ogre has font support already and 1.7 has gui that doesn't depend on external libs, we'd like that to have complete support for all languages, including rtl languages.

notice that arabic script is not necessarily for arabic language, it's used for writing many other languages like Persian, Dari, Punjabi, Pashto, Urdu, Kashmiri, Sindhi, and Uyghur.
refer to wikipedia's article on arabic script : http://en.wikipedia.org/wiki/Arabic_alp ... c_alphabet

below is a map of areas where arabic script is used, dark green indicates exclusive arabic script use, while light green indicates arabic script use among others:
Attachments
Arabic_alphabet_world_distribution.png
Arabic_alphabet_world_distribution.png (18.66 KiB) Viewed 6093 times
I love OGRE !
User avatar
jacmoe
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 20570
Joined: Thu Jan 22, 2004 10:13 am
Location: Denmark
x 179
Contact:

Re: bidi support for right to left languages

Post by jacmoe »

Use CEGUI :)
Assaf Raman did bidi support for that (IIRC, using freeBidi).

http://www.ogre3d.org/forums/viewtopic.php?f=11&t=47479
/* Less noise. More signal. */
Ogitor Scenebuilder - powered by Ogre, presented by Qt, fueled by Passion.
OgreAddons - the Ogre code suppository.
dr_amr
Halfling
Posts: 65
Joined: Wed Mar 02, 2005 2:22 am

Re: bidi support for right to left languages

Post by dr_amr »

thanks jacmoe,
I'm aware of assaf's work, and I was wondering if it could be integrated into ogre's own font system, which is more convenient.
I love OGRE !
dr_amr
Halfling
Posts: 65
Joined: Wed Mar 02, 2005 2:22 am

Re: bidi support for right to left languages

Post by dr_amr »

I have downloaded fribidi and got it compiled on windows, and it's very simple; it's actually just one function that does the reordering and that's it.
the problem is that I don't know where that would fit into ogre, and I think it would be easy to integrate for someone who knows more about the internals of the engine, it won't take more than 15 minutes if you know what you're doing.
I love OGRE !
dr_amr
Halfling
Posts: 65
Joined: Wed Mar 02, 2005 2:22 am

Re: bidi support for right to left languages

Post by dr_amr »

no one ?
I love OGRE !
User avatar
n9ine
Halfling
Posts: 55
Joined: Mon Aug 25, 2008 4:24 pm

Re: bidi support for right to left languages

Post by n9ine »

Image
waw big arabia! :D
I could be helpful, please describe describe more your needs.
reptor
Ogre Magi
Posts: 1120
Joined: Wed Nov 15, 2006 7:41 pm
Location: Finland
x 5

Re: bidi support for right to left languages

Post by reptor »

I did some research on "global coverage" of languages as a curiosity and I came to the conclusion that if one masters English, French, Spanish and Arabic then a whole lot of the World is covered :) It was in fact quite eye-opening for me to notice how wide-spread languages French and Arabic are. I don't mean as a total number of speakers world-wide but as number of countries where the language has a significant status and where you therefore should be able to get by if you manage those languages. That would be more important for a traveller than the total number of speakers.

As a result of my research my will to learn French increased. It's pretty difficult for a Finn whose native language is pretty isolated - the only real language buddy we have is Estonia,which is not much to be honest. I'll try to learn French any ways I suppose it's not impossible as many others can master it so can I.



Yeah, OK, some on-topic discussion... I was wondering should this thing really go into the Ogre3D library. I am all for increasing support for many languages but another question is whether the Ogre3D library is the right place for it. There is the possibility that it would be implemented as a "Sample" or "Demo" instead of putting it directly into library. This is not for me to decide and I am just pondering - not having made up my own mind about this yet. I don't see Ogre3D as a GUI library and I would imagine the project leader has no intention for it to become one. I don't want to discourage you but I think that is a realistic view on this. Personally I like the idea of supporting more languages/cultures but that is irrelevant to this discussion I think.
User avatar
jacmoe
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 20570
Joined: Thu Jan 22, 2004 10:13 am
Location: Denmark
x 179
Contact:

Re: bidi support for right to left languages

Post by jacmoe »

Which is why I said: use CEGUI - it's excellent! :)
Ogre has utf8 (IIRC) already, so with CEGUI you can get exactly what you want.
/* Less noise. More signal. */
Ogitor Scenebuilder - powered by Ogre, presented by Qt, fueled by Passion.
OgreAddons - the Ogre code suppository.
dr_amr
Halfling
Posts: 65
Joined: Wed Mar 02, 2005 2:22 am

Re: bidi support for right to left languages

Post by dr_amr »

@n9ine : it's not really all arabia, it's about countries that have languages that are written in the arabic script, which are many languages not just arabic.
what is needed is to find the place where the reordering function should correct the order of bidirectional text before being sent to the font system to print out. in the current version of ogre's text renderer it would treat all languages as if they go from left to right, which mangles up texts in arabic and hebrew.

examples for bidirectional text :

(arabic first, english second, hebrew third), notice that arabic and hebrew go from right to left, while english goes from left to right
עִבְרִית עִבְרִית english english عربي عربي

(english first, hebrew second, arabic last)
english english עִבְרִית עִבְרִית عربي عربي

@reptor : you ruled out chinese and russian, which both have great coverage in both number of speakers and geographical area. in fact chinese has the largest number of native speakers in the world, almost triple its runner up, spanish. but yes, china is just one country.

@jacmoe, I do use CEGUI, and it's fine for my needs. I just don't see why ogre's own font system should be lacking, specially when the mod is very simple, I recall that assaaf implemented bidi into CEGUI in just one day, using 3 different bidi libraries.

it's nothing about gui here, it's pure text rendering. which is a functionality other ogre-based gui systems should be able to use not rewrite.
I love OGRE !
User avatar
jacmoe
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 20570
Joined: Thu Jan 22, 2004 10:13 am
Location: Denmark
x 179
Contact:

Re: bidi support for right to left languages

Post by jacmoe »

The only way of (hoping to) getting this into to Ogre is to submit a patch. :wink:
/* Less noise. More signal. */
Ogitor Scenebuilder - powered by Ogre, presented by Qt, fueled by Passion.
OgreAddons - the Ogre code suppository.
reptor
Ogre Magi
Posts: 1120
Joined: Wed Nov 15, 2006 7:41 pm
Location: Finland
x 5

Re: bidi support for right to left languages

Post by reptor »

dr_amr wrote:
@reptor : you ruled out chinese and russian, which both have great coverage in both number of speakers and geographical area. in fact chinese has the largest number of native speakers in the world, almost triple its runner up, spanish. but yes, china is just one country.
I don't think I ruled out anything. I just didn't mention them.

I mean I was indeed talking of comparing in how many countries does a language have a significant status.

Here seems to be a good summary of what I was referring to: http://en.wikipedia.org/wiki/List_of_of ... l_language

English, French, Arabic and Spanish are clearly bigger than the others in this respect.


As a native speaker of a pretty isolated language I am practically forced to learn a foreign language to be able to converse with foreigners. It is something for example the British or Americans (USA) don't have to do in general. I guess the battle ships and explorations made by the British earned them this. I know an English language Professor here at a local University and he has been in the country since the 1970's and still doesn't speak our language more than a few words. I don't understand that. Maybe we are too nice to the foreigners - we should be more like the French and demand they learn our language if they want to live and work here. I am a proponent of learning many languages but in this issue I think we need to oppose the foreign languages - don't let them conquer our country with their language like that.

Sorry for the off-topic...


I too think if you want to get it into Ogre3D then you have to develop it by yourself and submit a patch. I think I know that Sinbad is not going to be excited about working on this and it would be rude to expect him to :) Sinbad is ultimately working on a graphics engine and text handling tools fall out of that I think. Moreover, we would prefer someone working on it who can actually test that it works correctly. You are probably more qualified to do that than Sinbad the Brit whose nation has conquered a big part of the World with their language :lol:
dr_amr
Halfling
Posts: 65
Joined: Wed Mar 02, 2005 2:22 am

Re: bidi support for right to left languages

Post by dr_amr »

well, I managed to get fribidi working with ogre, but still I'm having trouble displaying arabic fonts.

any help on getting arabic fonts to work is appreciated, and if you see any mistakes in the code please point them out.

anyway, here's the code, I just adjusted one function : TextAreaOverlayElement::updatePositionGeometry()

Code: Select all

    void TextAreaOverlayElement::updatePositionGeometry()
    {
		float *pVert;

		if (mpFont.isNull())
		{
			// not initialised yet, probably due to the order of creation in a template
			return;
		}

		size_t charlen = mCaption.size();
		checkMemoryAllocation( charlen );

		mRenderOp.vertexData->vertexCount = charlen * 6;
		// Get position / texcoord buffer
		const HardwareVertexBufferSharedPtr& vbuf = 
			mRenderOp.vertexData->vertexBufferBinding->getBuffer(POS_TEX_BINDING);
		pVert = static_cast<float*>(
			vbuf->lock(HardwareBuffer::HBL_DISCARD) );

		float largestWidth = 0;
		float left = _getDerivedLeft() * 2.0f - 1.0f;
		float top = -( (_getDerivedTop() * 2.0f ) - 1.0f );

		// Derive space with from a number 0
		if (mSpaceWidth == 0)
		{
			mSpaceWidth = mpFont->getGlyphAspectRatio(UNICODE_ZERO) * mCharHeight * 2.0f * mViewportAspectCoef;
		}

		bool newLine = true;
		
		FriBidiCharType pbase_dir = FRIBIDI_TYPE_ON;
		size_t length = mCaption.asUTF32().length();
		Ogre::UTFString::unicode_char *reordered_c_str = new Ogre::UTFString::unicode_char[length + 1];
		fribidi_log2vis(mCaption.asUTF32_c_str(), length, &pbase_dir, reordered_c_str, 0, 0, 0);

		for( size_t i = 0; i < length; ++i )
		{
			if( newLine )
			{
				Real len = 0.0f;
				for( size_t j = i; j < length; j++ )
				{
					Font::CodePoint character = reordered_c_str[j];
					if (character == UNICODE_CR
						|| character == UNICODE_NEL
						|| character == UNICODE_LF) 
					{
						break;
					}
					else if (character == UNICODE_SPACE) // space
					{
						len += mSpaceWidth;
					}
					else 
					{
						len += mpFont->getGlyphAspectRatio(character) * mCharHeight * 2.0f * mViewportAspectCoef;
					}
				}

				if( mAlignment == Right )
					left -= len;
				else if( mAlignment == Center )
					left -= len * 0.5f;

				newLine = false;
			}

			Font::CodePoint character = reordered_c_str[i];
			if (character == UNICODE_CR
				|| character == UNICODE_NEL
				|| character == UNICODE_LF)
			{
				left = _getDerivedLeft() * 2.0f - 1.0f;
				top -= mCharHeight * 2.0f;
				newLine = true;
				// Also reduce tri count
				mRenderOp.vertexData->vertexCount -= 6;

				// consume CR/LF in one
				if (character == UNICODE_CR)
				{
					size_t peeki = i;
					peeki++;
					if (peeki < length && reordered_c_str[peeki] == UNICODE_LF)
					{
						i = peeki; // skip both as one newline
						// Also reduce tri count
						mRenderOp.vertexData->vertexCount -= 6;
					}

				}
				continue;
			}
			else if (character == UNICODE_SPACE) // space
			{
				// Just leave a gap, no tris
				left += mSpaceWidth;
				// Also reduce tri count
				mRenderOp.vertexData->vertexCount -= 6;
				continue;
			}

			Real horiz_height = mpFont->getGlyphAspectRatio(character) * mViewportAspectCoef ;
			const Font::UVRect& uvRect = mpFont->getGlyphTexCoords(character);

			// each vert is (x, y, z, u, v)
			//-------------------------------------------------------------------------------------
			// First tri
			//
			// Upper left
			*pVert++ = left;
			*pVert++ = top;
			*pVert++ = -1.0;
			*pVert++ = uvRect.left;
			*pVert++ = uvRect.top;

			top -= mCharHeight * 2.0f;

			// Bottom left
			*pVert++ = left;
			*pVert++ = top;
			*pVert++ = -1.0;
			*pVert++ = uvRect.left;
			*pVert++ = uvRect.bottom;

			top += mCharHeight * 2.0f;
			left += horiz_height * mCharHeight * 2.0f;

			// Top right
			*pVert++ = left;
			*pVert++ = top;
			*pVert++ = -1.0;
			*pVert++ = uvRect.right;
			*pVert++ = uvRect.top;
			//-------------------------------------------------------------------------------------

			//-------------------------------------------------------------------------------------
			// Second tri
			//
			// Top right (again)
			*pVert++ = left;
			*pVert++ = top;
			*pVert++ = -1.0;
			*pVert++ = uvRect.right;
			*pVert++ = uvRect.top;

			top -= mCharHeight * 2.0f;
			left -= horiz_height  * mCharHeight * 2.0f;

			// Bottom left (again)
			*pVert++ = left;
			*pVert++ = top;
			*pVert++ = -1.0;
			*pVert++ = uvRect.left;
			*pVert++ = uvRect.bottom;

			left += horiz_height  * mCharHeight * 2.0f;

			// Bottom right
			*pVert++ = left;
			*pVert++ = top;
			*pVert++ = -1.0;
			*pVert++ = uvRect.right;
			*pVert++ = uvRect.bottom;
			//-------------------------------------------------------------------------------------

			// Go back up with top
			top += mCharHeight * 2.0f;

			float currentWidth = (left + 1)/2 - _getDerivedLeft();
			if (currentWidth > largestWidth)
			{
				largestWidth = currentWidth;
			}
		}

		delete[] reordered_c_str;

		// Unlock vertex buffer
		vbuf->unlock();

		if (mMetricsMode == GMM_PIXELS)
		{
			// Derive parametric version of dimensions
			Real vpWidth;
			vpWidth = (Real) (OverlayManager::getSingleton().getViewportWidth());

			largestWidth *= vpWidth;
		};

		if (getWidth() < largestWidth)
			setWidth(largestWidth);
    }
I love OGRE !
dr_amr
Halfling
Posts: 65
Joined: Wed Mar 02, 2005 2:22 am

Re: bidi support for right to left languages

Post by dr_amr »

@reptor : you're probably right :)
I love OGRE !
dr_amr
Halfling
Posts: 65
Joined: Wed Mar 02, 2005 2:22 am

Re: bidi support for right to left languages

Post by dr_amr »

and here's the arabic font I'm trying to get working,

my tries have been fruitless :(
Attachments
arabic_font.7z
an arabic font for testing
(62.96 KiB) Downloaded 140 times
I love OGRE !
_monkey_
Gnoblar
Posts: 4
Joined: Fri Feb 26, 2010 3:51 pm

Re: bidi support for right to left languages

Post by _monkey_ »

If the size of your application does not matter too much, you could give Qt a try.
Here's a quick and dirty example:

Code: Select all


void MyClass::renderOverlay()
{
	QPicture	picture;
	QPainter	painter;
	QRect		rect;
	QRect		area(0, 0, 256, 256);
	QPen		penText (QColor(255,255,255));

	painter.begin(&picture);           // paint in picture

	// example: converting MS WCHAR to QString
	wchar_t text [] = L"新田 賢福岡支店新田 賢";
	ushort text1[100];
	// fill with zero, just to ensure termination bytes in this sample
	memset(text1, 0, sizeof(text1));
	for(int i = 0; i < _countof(text); i++)
		text1[i] = (ushort)qFromLittleEndian ((uchar*)text[i]);
	text[i] = 0; text[++i] = 0;

	rect.setCoords(0,0,0,0);
	painter.setPen(penText);
	painter.drawText(area, Qt::AlignHCenter | Qt::TextWordWrap | Qt::TextDontClip, QString::fromUtf16(text1), &rect);

	painter.end();                     // painting done

    // draw a rounded rect border
	myImage = new QImage(rect.width() + 6, rect.height() + 6, QImage::Format_ARGB32);
	myImage->fill(0);
	painter.begin(myImage);
	painter.drawPicture(-rect.left() + 4, -rect.top() + 3, picture);   // draw the picture at text offset
	painter.setPen(penText);
	QRectF rectangle(1.0, 1.0, rect.width() + 4, rect.height() + 4);
	painter.drawRoundedRect(rectangle, 4.0, 4.0);
	painter.end();

	// you can also save the image here
	// myImage->save("QtText.png", "png");

	// create the texture and the material
	Ogre::DataStreamPtr imgstream(new Ogre::MemoryDataStream(myImage->bits(), myImage->byteCount()));
	Ogre::TextureManager::getSingleton().loadRawData("Qt/myImage",
		Ogre::ResourceGroupManager::DEFAULT_RESOURCE_GROUP_NAME,
		imgstream, myImage->width(), myImage->height(), Ogre::PF_A8R8G8B8, Ogre::TEX_TYPE_2D, 0);
	Ogre::MaterialPtr material =
		Ogre::MaterialManager::getSingleton().create( "Qt/myImage",
		Ogre::ResourceGroupManager::DEFAULT_RESOURCE_GROUP_NAME);

	Ogre::TextureUnitState *texLayer = material->getTechnique(0)->getPass(0)->createTextureUnitState( "Qt/myImage" );
	texLayer->setTextureAddressingMode( Ogre::TextureUnitState::TAM_CLAMP );
	material->setSceneBlending( Ogre::SBT_ADD );
	material->setDepthWriteEnabled( false ) ;
	material->load(false);

	// create overlay element and it to the "panel"
	if (!Ogre::OverlayManager::getSingleton().hasOverlayElement("Qt/myImage", false))
	{
		texOverlayElement = Ogre::OverlayManager::getSingleton().createOverlayElement("Panel", "Qt/myImage", false);
		texOverlayElement->setMetricsMode(Ogre::GMM_PIXELS);
		texOverlayElement->setWidth(myImage->width());
		texOverlayElement->setHeight(myImage->height());
		texOverlayElement->setMaterialName("Qt/myImage");

		Ogre::OverlayContainer* panel = static_cast<Ogre::OverlayContainer*>(Ogre::OverlayManager::getSingleton().getOverlayElement("MyTextPanel"));
		panel->addChild(texOverlayElement);
	}

}

Post Reply