Archive:Update system

Update System

The Mana World plans to be an constantly expanding game, with new features, maps and other game media being added all the time. This constant addition requires some way to distribute new material. The Mana World up to this point has been using a monthly release cycle to update the game, but this distribution method suffers in the following ways:

• Data redundancy - This is a major concern for those who have slower connections to the Internet.
• Releases aren't frequent enough - Players want new material as fast as they can get it. Players like a fast updated, dynamically changing game world in which they can immerse themselves - any stagnation can lead to player loss.
• Bad releases hard to update - Once a package has been released, soon after its release some major problems may be found, which may require immediate attention. The old package must then be removed and replaced with a new fixed package, by this time many people would have downloaded it and would probably not know about the new updated release. This is a major problem in itself.

Regarding the last topic, we discussed it a bit, and thought that first of all, we need a version checking, so that everyone is informed about a new release. In windows we were evaluating the possibility of directly replacing the binary, while this won't be possible in GNU/linux. I guess we will continue with monthly releases, and will discuss in this article only static data updates. --ElvenProgrammer 02:08, 4 Jul 2005 (PDT)

The Mana World wishes to address this by using an in-built update system which will automatically download the latest game media. By doing this, the second and third points noted above are eliminated. This update system should provide the following features:

• Smart updating - the updater should be able to find files which need to be updated and update them, without trying to update files which are already up to date.
• Cumulative patches (Optional)
• Media updates - should be able to fetch new and updated game media.
• Executable updates - should be able to update the binary executables appropriately.

The option of using a built-in updater is meaningful if using bitTorrent protocol (even if not necessary). If using HTTP/FTP, it could be valuable using an external application to avoid increasing the number of dependencies/size of executable in the client. About executable updates, see comment above. --ElvenProgrammer 02:08, 4 Jul 2005 (PDT)

Update Format

The patch format must be easy to use and flexible, and should support the required features mentioned above.

• The format will be added here.

The patches will be released using compressed archives. Probably using bzip compression is a wise choice to reduce downloads size. I suggest that every client keeps a local copy of the version of each file: /data/items.xml 0.12 /data/music/Faith.ogg 0.3 and every patch file will have a list of the changes, let's say changes.txt: /data/items.xml U 0.13 means it was updated, check version anyway. /data/music/desert.ogg R the file should be removed, no longer used /data/graphics/avatars/hammerbearcaricature.png A 0.1 new file, needs to be added to local list Using those infos, the previous data packages could be easily merged with the patch. Note: this is a quick suggestion, we could think of a lot of better ways. --ElvenProgrammer 02:08, 4 Jul 2005 (PDT)

---

This could be changed to:

Updates come in compressed archives, probably .zip files as they are easy to create and allow individual files to be read directly from them (this is already being abstracted using PhysFS, so there is no need to inflate the archives after downloading). Initially, the client will retrieve an ordered list of archives that is should load, for example:

tmw-base-1.0.zip
tmw-base-patch-1.1.zip
tmw-tonori-region-1.0.zip

The client will check whether all the listed zip files are present in its data directory, and attempt to download any missing ones (see update distribution below). Once downloaded, it will put each archive in the virtual file system provided by PhysFS. Once a data file is loaded, the archives are search bottom to top so that the file will be loaded from the last archive that provided it. This allows us to patch big data archives with a small archive that contains just the modified files.

Correct me if I'm wrong, but this won't keep all the old data? Or maybe old packages will be overwritten by new ones? If not just think that having 3 different versions of a song would lead for example to not required 6-9 MBs. --ElvenProgrammer 12:55, 4 Jul 2005 (PDT)

This ordered list could should contain protocol information for file retrieval and information concerning the platform which the patch can be used. Such a list could look like:

win32 http://themanaworld.org/updates/tmw-base-binary-win32-1.1.zip
osx http://themanaworld.org/updates/tmw-base-binary-osx-1.1.zip
all ftp://themanaworld.org/updates/tmw-maps-tonori-1.3.zip
all http://themanaworld.org/updates/tmw-gfx-items-1.1.zip

It is also suggested that the updates can be used in the configuration director (~/.tmw) on UNIX based platforms, as root priveleges would be required to install the updates correctly otherwise. Nym 23:56, 4 Jul 2005 (PDT)

Update Distribution

Updates must be distributed fast and effectively, providing a decent throughput even when the update server is under a high load. The Mana World development team has decided that the Bit Torrent protocol will be utilised for update distribution. BitTorrent is a distributed peer-to-peer protocol which utilises the bandwidth of all users. For more information about Bit Torrent, see the Bit Torrent Introduction.

To easily utilize to the Bit Torrent protocol, one of the following libraries can be used:

• libbt - C
• libtorrent - C++

Yeah I know we already choosed for bitTorrent, anyway we shouldn't discard HTTP/FTP both because some people don't want to be forced to share their bandwidth or if they're having problems. Someone also asked for the possibility to stop sharing patches (in case updater is built-in). A noticeable example is WoW updater which both allows for bitTorrent or HTTP downloads. And the patch sharing is limited to the time you download the patch (until you don't press "Play game!" I guess). A deeper evaluation of band usage and lag problems should be provided.--ElvenProgrammer 02:08, 4 Jul 2005 (PDT)

Because both libbt and libtorrent still seem immature and have annoying dependencies, I would suggest to hold off from BitTorrent for now and use libcurl to retrieve the updated data files through HTTP or FTP. --Bjørn 12:06, 4 Jul 2005 (PDT)

I agree, libcurl is at least used by libbt so if we're going to use it, it will be required anyway. I'd like also to say that libcurl it's really easy and could help us to create the HTTP/FTP updater in no time.--ElvenProgrammer 12:55, 4 Jul 2005 (PDT)