Battlepods Development - Zarth's Networked Physics Rant

Anything and everything that's related to OGRE or the wider graphics field that doesn't fit into the other forums.
Post Reply
User avatar
zarthrag
Greenskin
Posts: 128
Joined: Sat Jul 24, 2004 9:07 am
Location: Tulsa, Oklahoma

Battlepods Development - Zarth's Networked Physics Rant

Post by zarthrag »

Just a note, if you're gonna use networking and physics - prepare for a rather wild ride. (I'm currently bleching on one of those loop-d-loos). I'm trying to use ODE in a deterministic environment.

Dealing with syncronization is getting to be quite a challenge. Not only do I have to keep primitives sync'd across the network, but I have to ensure that latency is hidden and everything! Sure, I could only run the physics on the server, and push out the results, but that would be expecting too much out of the network unless the game is locked at ~30 fps.

The game is pretty fast paced vechicle sim with lots of flying, shooting, driving, and mech-fighting. We were hoping to create some awfully lush and unbelievably-large environments using PLSM2. But everying seems to be breaking at the physics level.

I'm clueless on how my ODE implementation will work across the network (our team chose raknet) or how it will scale, only the alpha test (awhile from now) will show.

So far, there are 5 different vehicles to be implemented, but the environment is still in question. I've looked at opal, but the speed we require isn't quite there. Newton is stable but, once again, speed usurps accuracy. (Things smash into buildings which, in turn, explode. Falling debris can be as hazardous as expected, and blah blah blah. Read: utter mayhem)

Note that the client is win32/linux/mac. So raknet was pretty much the choice, short of OpenTNL (Our budget is already over several hundred thanks to our taste in modeling "accessories." ...Let's not mention the Macs!)

...Of course, dropping multiplayer entirely would "fix" all of this. The game is largely cooperative, and I've little ambition of hosting a master-server/matching service, simple LAN support would do - so anti-cheat issues are far from my mind. But if an API were to keep ODE deterministic and synch'd, I'd be cool. I guess I'm at somewhat of a confusing crossroads, is it worth it?

User avatar
Phantom
Greenskin
Posts: 106
Joined: Mon Aug 02, 2004 10:28 pm
Location: Helsinki, Finland

Post by Phantom »

(Internet)Multiplayer is allways worth it. (IMO)

Atleast if you want the game to be popular/fun/competetive.

User avatar
tuan kuranes
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 2653
Joined: Wed Sep 24, 2003 8:07 am
Location: Haute Garonne, France
x 4
Contact:

Post by tuan kuranes »

The must read is the networked physic demo+slides+sources in wiki web links, it comes with a nice demo that explain everything (history,important moves, latency, packet loss, proxy, etc).

You'll be fixed on many network-physics aspects.

(author of tribes physic and networking )

perhaps you can use Ode on server, and use simplistic physics client-side for dead reckoning (that is sort of simple interpolation between physics stats).

User avatar
Olex
Hobgoblin
Posts: 593
Joined: Fri Apr 08, 2005 6:08 pm
Location: WA, USA

Post by Olex »

@zarthrag: I'll be in your shoes very soon.
Most likely, it will be ODE and RakNet as well. Good luck with your design. Maybe we will be able to compare them in the near future. :wink:

In a coder-to-coder style, could you share some of your challenges and thoughts that you have about crossing physics and network?

@tuan kuranes: wiki gives some help, but the material is not geared towards dynamic physical environment.

User avatar
regress
Halfling
Posts: 78
Joined: Sat Mar 26, 2005 8:39 pm

Post by regress »

Our team chose to go with OpenTnL as the cost really is minor in the long run, and it provides a few features we found to be helpful.
That being said, the documentation and support for Raknet is quite a bit better, at least from our initial testing.

Networked physics is always a challenging area - have you completed anything on a smaller level? Starting out so large may cause headache when trying to implement various techniques (server-side physics with client dead-reckoning, physics simulation on all sides, simple physics on client side, authoratative on server, etc.).

Keep us up to date, it sounds like an amazing project!

edit: How do you plan on breaking the buildings? I come back to the problem every now and then .. . . do you dynamically calculate a smaller set of meshes, or do you use pre-made models a la XSI's cutting tool? Just curious . . .
regards

User avatar
zarthrag
Greenskin
Posts: 128
Joined: Sat Jul 24, 2004 9:07 am
Location: Tulsa, Oklahoma

Post by zarthrag »

I just read through alot of those papers, while I've regained a little more hope, many were written circa 1997, before the "great penetration" of broadband. Also, FPS was (and, sadly, still is) all the rage.

I've done smaller projects before, but their requirements weren't nearly as stringent as this is. Battlepods is a whole new animal to me, my biggest to date.

This being a low-budget vehicle sim, I can enjoy at least a "little" tolerance for latency - due to the (relatively) slow changes in input params from the players. I've no intention of supporting modem play, so in admission that no implementation will be "perfect", I'm currently leaning towards this:

* The (listen) server runs the "authoritative" physics simulation in ODE. Each client also runs it's own physics simulation in step with the server.

* Timestep is based on the system speed. If your system can't keep-up, you're kicked.

* Each frame, clients send their input/control values to the server. The server then validates those inputs and resends to everyone else. That is as far as anticheat efforts will go.

* If the client's timestep is missed, then it's assumed that no control states have changed. (Dead-reckoning). This is corrected later by adding a force to correct the position. (Possibly obtain the corrective force by request from the server?)

* Gamespeed will be tied to the simulation speed. I'm not sure how ODE will act at multiples of the timestep. I will probably shoot for a 30Hz server speed, with the clients allowed to draw frames at any multiple ("mini" timestep ODE without performing changes/collision until the next server timestep.)

I have a few problems to surmount with this approach:

* ODE's usage of random numbers. I expect allowing fast systems to run at higher framerates to diverge because of the loss of random number sync. Would adding a separate random number generator for "extra" timesteps in ODE work?

* With "snapping" and divergence eliminated (...or so I hope), latency is everything. When playing CS, my ping varies from ~30 upto 150 before I feel like quitting. 30Hz equates to what's "acceptable".

* I'm praying really hard that *several* operating systems will keep ODE's random usage consistant.

* Missed frames/packets suck. Network problems/inconsistancies are resolved by the server. ODE doesn't handle "snapping" well at all, from my experience (adds angular velocity and other crap). Can this be changed??? I'd love to be able to simply set values on objects and continue simulation!!!!! This is probably my biggest problem, next to deterministics.

* If newton can handle this better, I'm all ears.

There may be a better/more-complex solution, but it's meant for only 2-4 players working together - preferably on a LAN. The feature is important, but still secondary. Also note that raknet's voip support is in-the-mix too.

User avatar
zarthrag
Greenskin
Posts: 128
Joined: Sat Jul 24, 2004 9:07 am
Location: Tulsa, Oklahoma

Post by zarthrag »

Upon reflection, I'm having second thoughts. Maybe, instead of propagating the physics, I'd be better-off holding the ODE physics on the server machine, then using distributed-objects in raknet to do the rest. Gawd-knows it's *easier* to implement, and acceptable for LAN play. Also, it removes alot of other programming issues mentioned above.

The only problem is the physics-based explosions and such, a dedicated server option (and/or a slight reduction of special effects in multiplayer) may be necessary. Using larger/fewer pieces of debris (customizable), and fewer/more predicable objects in general.

As a result, bandwidth will be wasted. I wasn't expecting very good performance on slow systems/connections anyway. But, hopefully, it'll work fine. Someday, when I have more time/help, I can produce a more mature networked-physics implementaton on top of ODE. Possibly a fork/patch that strips out ODE's dependance upon random numbers (or at least makes it managable/tolerant) allow for variable timesteps that maintain determinancy, and also allows for *easy* syncronization of objects, and resetting of positions. Also easy state-freezing....someday.

User avatar
tuan kuranes
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 2653
Joined: Wed Sep 24, 2003 8:07 am
Location: Haute Garonne, France
x 4
Contact:

Post by tuan kuranes »

@Olex :

http://69.55.229.29/articles/

and specially

http://69.55.229.29/articles/NetworkedPhysics.html

and the demo/slides with source that explain all

http://www.gaffer.org/downloads/NetworkedPhysics.zip

this is the MUST READ/TRY/SOURCE lookup

@zarthrag :

-> if you didn't read/try this one upside. this is the one. (read also the previous ones, like 'fix your timestep')

->written circa 1997 : nothing really changed and won't likely change, latency still sucks, dropped packets still exist, network stack still takes imes. larger download band could help, but still blocked by the server upload band (divided by player and packet/s you get very fast bad results.)
Only new things is p2p, for voice and chat...

-> server then validates those inputs and sends physics states. This is the industry way, you can innovate, but that won't be free and you're not sure of success. (timing differencies, physics perfect sync doesn't exists.) Player shooting needs to be decided by server.
Which vehicle crossed the line first needs to be decided by server.

-> This is corrected later by adding a force to correct the position. =>
again, most complex choice : use history to rewind to before problem pos and re-runs physics using server data.
(corrective force will drop you in the land of math&physics hell, even worst than re-coding Ode.)

-> on a LAN, the server does physics and send physics states. client just sends input and receive physics states, no Ode. You won't get any problem as latency is very, very small.

-> Timestep should be fixed and same for all client. no other way

-> Same timestep and use same srand init call. no other choice.

-> voip on net or lan is not a problem if p2p and not on server, and voip packet has lower prio than others.

-> Read ODE's mailing list for 'deterministic behaviors' posts : amongst other info . you get info on Float Consistency Checks (on Win32 : run Ogre d3d with float coherent, and compile with float consistency, otherwise you won't get anything.)

Post Reply