Thoughts on multi-monitor rendering in windows

[Mattan Furst]

The following paragraphs summaries the attempts at my company to develop a render engine that supports rendering a multi-monitor environment with high framerate on a single PC. This summary comes in the hope that it may benefit other people in the future. I am unable to give the code concerning this post as it is proprietary code that does not involve Ogre directly. however you may still be able to benefit from the ideas in this post.

I work at a company that, among other things, develops PC based simulators (mostly for the aeronautics industry). The graphic engine we use is based on Ogre. One of the features that our customers continuously request is the ability to run on multiple windows / monitors. Our program is CPU bounded. Relative to rendering 1 monitor, rendering on 4 different monitors will reduce our framerate by around 25% percent. For marketing considerations it was initially important we use only one computer to render all the monitors we needed

Using ATI EyeFinity and complex viewport shaders were considered but latter dismissed, for various reasons, as ineffective.

Latter we tried to extend Ogre so that the rendering process could be multi threaded. Initial research was promising, however we couldn't manage to make it work under the Ogre code. And after a few weeks worth of work we abandoned this line of development (in hindsight we might not have had the proper computer for testing this development, but the newer solution we ended up with should be better than anything we could have gotten from this approach).

In the end we decided to forgo the idea of running everything from a single computer and instead render through a network of computers. That meant running multiple programs on multiple computers each responsible for 1 or 2 monitors. We have a simple interface that connects the simulation engine to the graphic engine. What we did is create a simple client program containing only the graphic engine and a small network component. Then we basically streamed all the calls made to the graphic engine on the simulation side over a network to the client programs*. The main simulation process still rendered 1 or 2 monitors but the client programs helped with the rest.

Now for the twist in the plot.

Assaf Raman (My boss at work and Ogre team member) had come to display the solution to one of our clients. As the development was still in beta testing and we were short on time, he decided that instead of using multiple computers he will place 4 virtual machines on a single computer and run the network rendering solution through that. Even through the VM usage, this showed a 60% increase in rendering speed.

Running multiple screens with high framerate on a single computer become a real possibility. However there were several problems.

We usually run our simulation in windowed mode, but using this solution meant that we will loose the windows interface on the windows running from the secondary processes (context menus, various dialogs, etc..). Exclusive mode presented its own problems with windows from secondary processes minimizing when loosing focus. There was also GPU memory allocation to consider, we needed each client to only allocate memory on the device assigned to the window it was rendering on. That meant we couldn't keep using the virtual machines. Another problem was that running high CPU consuming processes on a single computer created framerate "jittering".

(For the recored the following was Assaf's idea and not mine) It turns out that if you give Ogre a handle of a window to render on, even if that window was not created by your program, Ogre renders on it just fine. That meant that we could create the windows on the main simulation program, and render them in the client processes. Assigning different CPU core affinity masks removed 99% of the jittering problem. And assigning specific set of monitors and devices to each process solved the GPU memory concerns.

We are currently in the testing phase of this development but initial results look very promising.


* - This is a bit of a simplistic explanation. Not everything was mirrored. Getter functions were thrown out. Special implementation were given to specific calls such as API calls for creating cameras. But it was not a hard development.

[Klaim]

Thanks for that report, I was planning to add such support to my game.

If I read correctly what you say, the best way to exploit multiple monitors would be to have separate processes for each monitor, each process running an Ogre instance? Or did I misread?

[Mattan Furst]

No you didn't misunderstand.

I don't know if its the best. but its a definite possibility. And from my expirience relative simple to implement. However this method will only be useful if you are CPU bound. Not GPU or bus throughput bound. It will also create problems on low end computers that don't have enough CPU cores. Were you don't have enough cores to assign to the different processes.

Also important to note that when you create the socket that sends the information to the client. The server must not wait for information back from the client on the same socket otherwise it will tremendously slow down the server. We ended up creating a secondary socket for synchronization purposes where the client can send information back to the server.

Edit:
There are also a bunch of other small stuff such as replacing the random value generator in Ogre with something that will be consistent amoung all computers. syncroniaing the timing values between frames. stuff like that.

[Mattan Furst]

Before you get ahead of yourself you need to check whether you need such a thing. This solution is only if you need 3 or more monitors (if you are CPU bound with 2 monitors then you might want to take a second look at your code). you can also check whether for your purpuses you can simply use things like stretch mode or EyeFinity (I don't remember the NVidia equivalent but there is one).

The reason we couldn't use those was because we need to have dome arrays. meaning we needed to cover more than 180 degrees view or different perspectives all together.

[Kojack]

EyeFinity (I don't remember the NVidia equivalent but there is one).
The reason we couldn't use those was because we need to have dome arrays. meaning we needed to cover more than 180 degrees view or different perspectives all together.

Eyefinity works great with ogre and easily handles wide fovs.
Well, not the way most games do eyefinity, just treating it as one ultra wide screen with wide fov, that looks crap.
What I do is either:
- Make 3 or more viewports across the eyefinity display. Each has a camera at a different direction (more cameras/viewports makes it more cylindrical and can handle wider fov without distortion, but more time is taken in the scenegraph code).
- Make 3-6 cameras attached to the faces of a cubic environment map render target. Then use a compositor to render the screen using a cylindrical or spherical lookup of the cubic map. This gives per pixel cylindrical display. (You only need 3 cameras if the horizontal fov doesn't go over 270 degrees and the vertical doesn't go over 90 degrees)

Although this doesn't really help since you say you typically run in windowed mode, although windowed mode for each display is going to look odd on a dome projector.

[Mattan Furst]

There something I don't get or maybe I haven't made myself clear.
The whole point of the project was to be able to display multi-monitor scenarios with greater performance. If you render 3 cameras for the cube map and display them on 3 monitors where do you get the extra performance from.

although windowed mode for each display is going to look odd on a dome projector.

The dome display is actually done in full screen mode and the output is handed to a third party component responsible from distorting and blending the outputs together

[LarsW]

I also tried to distribute the rendering across multiple computers to use a 5 sided cave including stereo rendering with my application.
For that approach i used the zoidcom library to replicate Ogre scenenodes over the network. This worked quite well but only on a limited subset (position, orientation and a color value for metadata and serialized meshes for the geometry on node creation), cause the overhead would be to large to transmit all node update changes, especially when they are traversing the whole scenetree.

Later on the feature has been declared unimportant and the connection was not necessary anymore. But i was hoping that there would be a network synchronized scene manager for Ogre sometime. But if i understood your setup correctly you are not synchronising on a scene node level, but more on a abstract Object level?

greetings
Lars

[syedhs]

We have done multi-monitor rendering too, and we use Matrox 3H ToGo - have you considered that? Basically, all the signals from 3 monitors are merged by Matrox hardware, and all our Ogre-based simulator sees is a very wide screen eg (1920x1080x3 => 3760x1080). And then we still need to create 3 viewports on that very-wide screen so that we can adjust the camera angle and position for each viewport. Obviously, you need a good GPU card (instead of CPU-bound, it is GPU-bound) to make the frames rendered at at least 60hz.

Previously, we used 3 computers to render to 3 monitors (one computer render to one monitor) and they all are linked by network - but it is more hassle than necessary as we have to install onto 3x computers (not to mention support).

[Mattan Furst]

@LarsW

No that's no what I'm doing.

It would have made the implementation much more complicated. Plus communicating on that API level would mean a huge bandwidth usage. On that level every change in images or buffers would need to be transmitted over the network.

[Mattan Furst]

@syedhs

The Matrox 3H ToGo wasn't a solution I was aware of. but again the purpose of the original project wasn't to render more monitors or combine them in a different way. It was about squeezing more performance out of the monitors we already rendered on.

[Kojack]

There something I don't get or maybe I haven't made myself clear.
The whole point of the project was to be able to display multi-monitor scenarios with greater performance. If you render 3 cameras for the cube map and display them on 3 monitors where do you get the extra performance from.

True, there might not be a performance boost. But it removes the gpu memory hassle you mentioned. No need to allocate on the correct device, with eyefinity all monitors share the same memory.

[Assaf Raman]

We are going to buy a good computer - like the one I had the VM benefit on, I will post more information when we do.
My biggest issue still - how to synchronize the different processes.