Torchlight 2 - The MMO Project (Unofficial)

A place to show off your latest screenshots and for people to comment on them. Only start a new thread here if you have some nice images to show off!
User avatar
Henrik
Gnoblar
Posts: 11
Joined: Sun May 13, 2012 8:03 pm

Torchlight 2 - The MMO Project (Unofficial)

Post by Henrik »

Greetings!

We’re a team of two developers that some of you might recognize from before. We previously worked on a project that set out to recreate the entire Torchlight client from scratch. Runic approved it and receiving their blessing ment a lot to us. You can read more about that project in the original topic and the topic at Ogre3D.

We're now creating an MMO version of Torchlight 2. We’ve been working on it for more than a year. We started by recreating the Torchlight 2 client and then spent the majority of the time creating a server for it. The server architecture has been stressed and proven to support a great amount of players. You can read more about this project in the topic at Runic

By the fans for the fans.

Image

Footage:

[youtube]1mI33z3x1wA[/youtube]

Screenshots:

Image Image
User avatar
mkultra333
Gold Sponsor
Gold Sponsor
Posts: 1894
Joined: Sun Mar 08, 2009 5:25 am
x 116

Re: Torchlight 2 - The MMO Project (Unofficial)

Post by mkultra333 »

That sounds excellent, will have to give that a try.
"In theory there is no difference between practice and theory. In practice, there is." - Psychology Textbook.
User avatar
Klaim
Old One
Posts: 2565
Joined: Sun Sep 11, 2005 1:04 am
Location: Paris, France
x 56

Re: Torchlight 2 - The MMO Project (Unofficial)

Post by Klaim »

Wow that's quite a project! Interesting! Good luck! :D
User avatar
spacegaier
OGRE Team Member
OGRE Team Member
Posts: 4304
Joined: Mon Feb 04, 2008 2:02 pm
Location: Germany
x 136

Re: Torchlight 2 - The MMO Project (Unofficial)

Post by spacegaier »

Big task. Curious to see how it works out. All the best!
Ogre Admin [Admin, Dev, PR, Finance, Wiki, etc.] | BasicOgreFramework | AdvancedOgreFramework
Don't know what to do in your spare time? Help the Ogre wiki grow! Or squash a bug...
Treon
Kobold
Posts: 37
Joined: Mon Mar 07, 2011 2:30 pm

Re: Torchlight 2 - The MMO Project (Unofficial)

Post by Treon »

Amazing project, Love to see it done !
Magnus
Gnoblar
Posts: 4
Joined: Wed Oct 30, 2013 12:43 pm

Re: Torchlight 2 - The MMO Project (Unofficial)

Post by Magnus »

Nice project, hope that will be released.
User avatar
Herb
Orc
Posts: 412
Joined: Thu Jun 04, 2009 3:21 am
Location: Kalamazoo,MI
x 38

Re: Torchlight 2 - The MMO Project (Unofficial)

Post by Herb »

The video looks great. I take its it's closed source code as I don't see any details on a source repo, but could you share what libraries you're using for this? I'm curious if you're using CEGUI like Runic did and what network library and protocol (TCP or UDP) you're using, physics library, etc. Any details you can share would be great.

I'll be keeping my eyes on this project, as I'm a big Torchlight fan! :)
Magnus
Gnoblar
Posts: 4
Joined: Wed Oct 30, 2013 12:43 pm

Re: Torchlight 2 - The MMO Project (Unofficial)

Post by Magnus »

Nice project man!
Herb wrote:The video looks great. I take its it's closed source code as I don't see any details on a source repo, but could you share what libraries you're using for this? I'm curious if you're using CEGUI like Runic did and what network library and protocol (TCP or UDP) you're using, physics library, etc. Any details you can share would be great.

I'll be keeping my eyes on this project, as I'm a big Torchlight fan! :)
I'm curious about this too.
User avatar
Henrik
Gnoblar
Posts: 11
Joined: Sun May 13, 2012 8:03 pm

Re: Torchlight 2 - The MMO Project (Unofficial)

Post by Henrik »

Image

Version 0.0.4 - Preview

In this update we're previewing pre-alpha version 0.0.4. Check out the changelog below to see its full content.

Changelog:

• Chat system overall imrovements including colors and line-fading.
• Support for animated generic models.
• Layout loading on login scene.
• Improved pathfinding.
• World vegetation.
• Lights.
• Client support for Mac OS X.
• Overall improvements and changes (Client/Server).

Footage:

[youtube]qSLUHvPdILc[/youtube]

Screenshots:

Image Image
Image Image

Thanks for all the encouraging feedback, it really helps us stay motivated.
Herb wrote:The video looks great. I take its it's closed source code as I don't see any details on a source repo, but could you share what libraries you're using for this? I'm curious if you're using CEGUI like Runic did and what network library and protocol (TCP or UDP) you're using, physics library, etc. Any details you can share would be great.

I'll be keeping my eyes on this project, as I'm a big Torchlight fan! :)
Ogre3D, CEGUI, Boost::asio (TCP), MSSQL, RapidXML, OpenSSL - Cryptography, Recastnavigation
User avatar
c6burns
Beholder
Posts: 1512
Joined: Fri Feb 22, 2013 4:44 am
Location: Deep behind enemy lines
x 139

Re: Torchlight 2 - The MMO Project (Unofficial)

Post by c6burns »

Henrik wrote:MSSQL
:shock:
User avatar
Zonder
Ogre Magi
Posts: 1172
Joined: Mon Aug 04, 2008 7:51 pm
Location: Manchester - England
x 76

Re: Torchlight 2 - The MMO Project (Unofficial)

Post by Zonder »

c6burns wrote:
Henrik wrote:MSSQL
:shock:
:shock:

You need to use something that can scale SQL doesn't very well or cheaply. Also you need to rethink using relational model property stores are probably more suited.
Last edited by Zonder on Thu Dec 12, 2013 5:22 pm, edited 1 time in total.
There are 10 types of people in the world: Those who understand binary, and those who don't...
User avatar
Klaim
Old One
Posts: 2565
Joined: Sun Sep 11, 2005 1:04 am
Location: Paris, France
x 56

Re: Torchlight 2 - The MMO Project (Unofficial)

Post by Klaim »

I agree that MSSQL is really not a good idea.

However, they do need a SQL database, you can't make a MMO with most NoSQL schemes (because they assume that it's ok to not have the information distributed immediately, which can't work in a MMO).
Nikitron
Gnoblar
Posts: 6
Joined: Wed Mar 21, 2012 9:48 am

Re: Torchlight 2 - The MMO Project (Unofficial)

Post by Nikitron »

Looks great.
Klaim wrote:I agree that MSSQL is really not a good idea.

However, they do need a SQL database, you can't make a MMO with most NoSQL schemes (because they assume that it's ok to not have the information distributed immediately, which can't work in a MMO).
I think it just works very well in the MMO. You place the information in a convenient location and fill it out as they become available. For example in the user table is stored its inventory. It's a matter of organizing information and nothing more.
User avatar
Klaim
Old One
Posts: 2565
Joined: Sun Sep 11, 2005 1:04 am
Location: Paris, France
x 56

Re: Torchlight 2 - The MMO Project (Unofficial)

Post by Klaim »

Depends on the information but most information in a MMO have to be the same for everybody at the same time, otherwise you can't even have local interactions as different clients will get different informations. Not all NoSQL database have this philosopy of not guaranteeing changes to be propagated to all the nodes immediately, but most do (I don't know one that doesn't, as this philisophy is basically about making sure the data is not centralized), which exclude them from being used in MMOS or any soft real time kind of games (even turn by turn as you have no guarantee about the time for propagating information through nodes).

For example, Eve Online uses grapes of MySQL (maybe now MariaDB) based servers which each one manage one solar system. So, inside one system you have one unique source of information about the game state, but having the world cut in these different systems makes easy to manage a very big world. Also it makes possible to optimize some systems/node because there is a lot of people getting there often.

It would actually work if you have something like a unique MongoDB database, or one unique such db for each part of the world, but you wouldn't benefit from any kind of node scalability, which is almost not possible if you don't scale the world itself the same way.

For a small online game, going with a usual SQL db is enough (except maybe it's hard to make evolve). For a bigger world, I guess the strategy of Eve Online is certainly the best currently.
User avatar
c6burns
Beholder
Posts: 1512
Joined: Fri Feb 22, 2013 4:44 am
Location: Deep behind enemy lines
x 139

Re: Torchlight 2 - The MMO Project (Unofficial)

Post by c6burns »

I've had this discussion many times (sql vs nosql, relational vs heirarchical data model, etc). I've been outright told that SQL solutions aren't powerful enough for the type of real-time distributed app that an MMO game is, but I just don't see it that way. The type of data you are storing falls neatly into relationships, and you can even scale by frontending your SQL solution with a NoSQL solution. For example, memcached (simple key->value nosql solution) in front of a mysql or postgres cluster is very scaleable and you get to retain your easy relationship management.

I was surprised to see MSSQL as the choice, but one thing I often recommend is to go with what you know, as (in my opinion) it is better to become a victim of your own success than to not get your project off the ground. Especially in a team of two ... if things get off the ground more will join the team and some brave soul can take over the data situation. Switching away from MSSQL would not be a difficult task down the road, as long as you don't use anything specific to TSQL or whatever their implementation is.
User avatar
Klaim
Old One
Posts: 2565
Joined: Sun Sep 11, 2005 1:04 am
Location: Paris, France
x 56

Re: Torchlight 2 - The MMO Project (Unofficial)

Post by Klaim »

c6burns wrote:I've had this discussion many times (sql vs nosql, relational vs heirarchical data model, etc). I've been outright told that SQL solutions aren't powerful enough for the type of real-time distributed app that an MMO game is, but I just don't see it that way. The type of data you are storing falls neatly into relationships, and you can even scale by frontending your SQL solution with a NoSQL solution. For example, memcached (simple key->value nosql solution) in front of a mysql or postgres cluster is very scaleable and you get to retain your easy relationship management.
Yeah, the main problem in these kind of discussions is that most people used to work with dbs are used to do online services, which have to be efficient at the cost of users not always having the same data; while for a MMO, there have to be a unique "reality" state of the world which keeps everything in sync so that representations are equivalent for all users.
This is important (both for fairness and being able to have coherent interactions between game objects) but also unique compared to most online services.

I have discussed with a NoSQL specialist about how to use something like MongoDB for a game state, he first thought I was wanting to make a game website service. Once he realized it had to be the full state of a game, shared among users, he stopped and said, just use MySQL/MariaDB. The advantage of most NoSQL db is in their easy node disptach.
However, it's still interesting to use some of these dbs for their high speed of access (depending on the kind of games) or for being able to change the model easily (in document dbs for example).

I've made several "small mmo" games in Flash on the client side and java on the server side that used memcached so I see what you mean. The only important thing to understand in the case of mmo is that there have to be a unique truth of the state of the world, there is no way you can duplicate it without killing performance. However, you can split that world into several databases, which works well until there is a place that everybody wants to go to.
c6burns wrote: I was surprised to see MSSQL as the choice, but one thing I often recommend is to go with what you know, as (in my opinion) it is better to become a victim of your own success than to not get your project off the ground. Especially in a team of two ... if things get off the ground more will join the team and some brave soul can take over the data situation. Switching away from MSSQL would not be a difficult task down the road, as long as you don't use anything specific to TSQL or whatever their implementation is.
I agree. Using a SQL database have at least the advantage that if you don't use it in a too db-engine-specific way you can migrate to something else without much trouble.


Anyway, well done guys! (forgot to say that)
User avatar
c6burns
Beholder
Posts: 1512
Joined: Fri Feb 22, 2013 4:44 am
Location: Deep behind enemy lines
x 139

Re: Torchlight 2 - The MMO Project (Unofficial)

Post by c6burns »

Klaim wrote:I've made several "small mmo" games in Flash on the client side and java on the server side that used memcached so I see what you mean. The only important thing to understand in the case of mmo is that there have to be a unique truth of the state of the world, there is no way you can duplicate it without killing performance. However, you can split that world into several databases, which works well until there is a place that everybody wants to go to.
Of course, well put. You are making a very valid point about authoritative game state.

I don't really have previous game experience - I just started in this field and will ship my 1st title next year! - but I have a lot of experience with real time systems (like VoIP) and scaling data services. In phone systems you have the same issue ... there needs to be an authority on the state of a given endpoint, or voice bridge, or conference room, etc. For us memcached was extremely effective as a frontend to our existing SQL solution, which is why my opinion now when designing data services is: go for SQL if data is relational, throw memory at the problem while scaling, host a giant party when you scale beyond that solution because you now have hundreds of thousands of customers :)
Klaim wrote:Anyway, well done guys! (forgot to say that)
Haha sorry to derail this thread. I also forgot to say this project looks great and good luck! I am envious ... I hope very much to be able to work on multiplayer games in the future.
Nikitron
Gnoblar
Posts: 6
Joined: Wed Mar 21, 2012 9:48 am

Re: Torchlight 2 - The MMO Project (Unofficial)

Post by Nikitron »

Klaim wrote:Depends on the information but most information in a MMO have to be the same for everybody at the same time, otherwise you can't even have local interactions as different clients will get different informations. Not all NoSQL database have this philosopy of not guaranteeing changes to be propagated to all the nodes immediately, but most do (I don't know one that doesn't, as this philisophy is basically about making sure the data is not centralized), which exclude them from being used in MMOS or any soft real time kind of games (even turn by turn as you have no guarantee about the time for propagating information through nodes).

For example, Eve Online uses grapes of MySQL (maybe now MariaDB) based servers which each one manage one solar system. So, inside one system you have one unique source of information about the game state, but having the world cut in these different systems makes easy to manage a very big world. Also it makes possible to optimize some systems/node because there is a lot of people getting there often.

It would actually work if you have something like a unique MongoDB database, or one unique such db for each part of the world, but you wouldn't benefit from any kind of node scalability, which is almost not possible if you don't scale the world itself the same way.

For a small online game, going with a usual SQL db is enough (except maybe it's hard to make evolve). For a bigger world, I guess the strategy of Eve Online is certainly the best currently.
I think you are mistaken. From an SQL select or key-value model do not depend. Both solutions are scaled. The client application is not important that you use as it does not work with the database directly. All these disputes are based on nothing.
As I said, the main question - "how do you implement the data model". And depending on your preference and experience this solution will be scalable adn convenient, or not!...
I think the model of key-value is much more attractive and perfect for creating "human-understandable" data models ;)

P.S. As far as I know EVE Online uses MSSQL - https://wiki.eveonline.com/en/wiki/Tran ... ase_Server

Added:
Klaim wrote:This is important (both for fairness and being able to have coherent interactions between game objects) but also unique compared to most online services.

I have discussed with a NoSQL specialist about how to use something like MongoDB for a game state, he first thought I was wanting to make a game website service. Once he realized it had to be the full state of a game, shared among users, he stopped and said, just use MySQL/MariaDB. The advantage of most NoSQL db is in their easy node disptach.
However, it's still interesting to use some of these dbs for their high speed of access (depending on the kind of games) or for being able to change the model easily (in document dbs for example).
Coherence of data in the database should be achieved logic of your server and not used model. As I said, it all depends on your experience. There are no physical limitations to use NoSQL in MMO, service or in any other application. All this psychological simtomy.
User avatar
c6burns
Beholder
Posts: 1512
Joined: Fri Feb 22, 2013 4:44 am
Location: Deep behind enemy lines
x 139

Re: Torchlight 2 - The MMO Project (Unofficial)

Post by c6burns »

My surprise at the use of MSSQL had nothing to do with anything except windows servers cost more to run, windows itself costs money to license, and eventually MSSQL costs money to license (though I am aware you can go quite a ways without a paid license). Maybe I'm just a giant cheapo who surrounded himself with giant cheapo colleagues and thus my surprise at a small project choosing MSSQL ... no judgement other than that :)
User avatar
Klaim
Old One
Posts: 2565
Joined: Sun Sep 11, 2005 1:04 am
Location: Paris, France
x 56

Re: Torchlight 2 - The MMO Project (Unofficial)

Post by Klaim »

Nikitron, I think you misunderstood what I said. I pointed that you can use any kind of DB for an mmo, but you just can't use the replication features of the databases themselves if they are guaranteed NOT to be in sync immediately, which most NoSQL databases guarantee (to not be in sync), which make them less interesting if replication was the strong selling point. If you use MongoDB for example, you might do it for document schemes, not for replication features.
We are not arguing against each others here.
I used the Eve Online example to point this too.

By the way, Eve Online using MSSQL don't mean that it's a good idea for any game or team. But yeah, why not (if you can pay it).
User avatar
Klaim
Old One
Posts: 2565
Joined: Sun Sep 11, 2005 1:04 am
Location: Paris, France
x 56

Re: Torchlight 2 - The MMO Project (Unofficial)

Post by Klaim »

By the way, surprisingly, the Eve Online wiki page you pointed don't explain the database layout which is explained there: http://www.gamasutra.com/view/feature/1 ... hp?print=1

So the info isn't complete and is actually misleading: there is one database "in the end", which contain the merged data of all solar-systems or nodes, but there are actually multiple database servers running, allocated to nodes, which do the local storing and then are merged back to the big one.

These subjects are always fascinating. :o
bstone
OGRE Expert User
OGRE Expert User
Posts: 1920
Joined: Sun Feb 19, 2012 9:24 pm
Location: Russia
x 201

Re: Torchlight 2 - The MMO Project (Unofficial)

Post by bstone »

Being a guy who consulted a few startups about how to save their lifes after sudden exposure to vast userbases, I thought I would share my observations here about OO/hierarchical vs relational DB. The two of the startups have been stuck with MongoDB as their poison of choice, btw.

The general idea is that if you are a very experienced developer and have done your homework well then it doesn't really matter if you go for OODB or RDBMS. Both have their advantages and disadvantages. You just have to know how to play the good and bad cards beforehand. But... if you are not that experienced then causing a fatal wound to your foot is so much easier with OODB. It seems to be much more flexible at planning stages. You don't have to bend it that much around your fascinating OOP designs. It only makes sense in the OOP world after all. But while OOP designs make refactoring and evolving the codebase a bliss, evolving an OODB schema with some radical changes and your game having thousands of users online... the whole other story.

RDBMS at least puts you into a rather strict set of rules right from the start and the relational nature itself makes adding/dropping relations something very natural. You can massage your schema any way you need, like getting something from the deeps of one hierarchy and burying it in the middle of another without that much of a change to your code and often with zero changes to your schema. Try that with OO/document oriented DB...

And on top of that, in 50% of the cases where you suddenly need to scale, you will find yourself switching to the simplest master/slave deployment and calling it a day. The other half requiring sharding or switching to partitioned clusters benefit a lot from careful planning at the early stages but is still doable even when disaster is already knocking into your office.

OODB is awesome for prototyping but if you plan for any success at all, well... you get the rough picture. Or just check how many OODB implementations survived through the last decade. There's a reason for most of them getting ditched.

Yeah, and you can create very flexible OO abstractions around your relational databases. You just have to embrace the relational nature of your data and use OOP as a tool and not as an excuse to warp everything around till it fits your 1:1 domain-to-classes mapping.