Sign in to follow this  
Gavin

Please make the pack updates faster - CDN, multiple download locations etc

Recommended Posts

Lately Wurm has been releasing a new graphics update every few days, whilst these new graphics are cool something that frustrates me is the ridiculous amount of time it takes to download these large files. Often I'll finish work after a long shift and have perhaps 3-4 hours to play yet I spend 2 hours downloading the graphics file before I can even get in because everyone and their dog is hammering the servers

 

First up, nothing wrong with my internet connection:

4945432112.png

My connectivity to the Hetzner speedtest server (same datacenter as Wurm) is fine too

5081541742.png

(this shows I should be able to get 2.4MB/s with a single thread, yet I can't break 150KB/s with a single thread - the problem isn't the datacenter, it's the server)

 

It's just my speeds to Wurm servers are always terrible when a pack is released, I assume because the servers are being hammered by hundreds of people trying to download at the same time congesting the servers connectivity

 

Even if I use a download accelerator with 20 or so download threads I can't break more than 500KB/s average from the Wurm server. So downloading the 1GB of graphics files takes quite a long time. It takes literally hours if I download it using the launcher because it stops and checks every individual file.

 

I have a few suggestions on how this could be fixed

 

1) Instead of putting the packs on a single server, use a CDN

 

Putting the files on a content delivery network would be awesome, this way the files will be delivered locally to everyone quickly - this isn't the cheapest option but it would be the deluxe and best option

Something like Cachefly would work fine: https://www.cachefly.com/plans-pricing/pay-as-you-go-cdn/

One of the cheapest CDN's and they have servers in each continent and close to all major developed cities, $0.1 - $0.2 per GB

 

~3,500 people (current prem players, ignoring alts who would only download once) downloading 1GB per month would cost $350 - $700 - whilst it's not cheap, this is the deluxe option and would have everyone saturating their internet connections capacity so the packs would take a few seconds to download for people on good connections.

 

2) Mirror the file in other locations

The main problem is that the Wurm server gets overloaded and can't distribute the content fast enough, even something simple like hosting the update packs on multiple servers would allow the people who use download accelerators to download from multiple servers at once.

Do the game servers have unlimited or lots of traffic? If so, you already have 10 servers you could mirror the files on.

 

If this is done integrate it into the Wurm client that it can download from multiple servers at the same time

 

3) Split the graphics up into smaller packages

As mentioned earlier my #1 problem is that the client takes too long to go through each file individually and verify the filesize, so I (and I assume many) end up turning to a download accelerator which uses multiple threads to manually pull the file from the Wurm servers as fast as possible, this in turn probably overloads the crap out of the pack update server but it's the only way I can get ingame quick enough. But even then it's very slow.

 

The problem is when doing this I need to download the entire 1GB pack update even if only 20MB of files have been changed.

 

Instead, why not split the graphics files up a bit? For example graphics_npc.jar graphics_textures.jar graphics_furniture.jar graphics_buildings.jar and so on and so forth, that way the client will verify the file sizes and tell you which of the .jar's have actually changed in size, and instead of downloading the entire 1GB of files with a download accelerator you can just download the smaller 100-200MB single pack that has changed - this would take lots of stress away from the Wurm servers.

 

4) Lastly, perhaps the best suggestion - make it possible to connect with old graphics files!

Why not make it so when you attempt to login and the packs have been updated, instead of the website only hosting /packs/ have packs/previous/ on the server - this way the client can still verify that the size of the packs matches the previous packs to ensure you haven't messed with the graphics files, but so long as it matches the old files it allows you to play the game using the old graphics whilst it downloads the new graphics in the background, so you can still play the game whilst the packs download slowly in the background ready for the next time you login.

 

 

 

 

Whatever happens as Wurm grows this problem really must be addressed, I'm sure I made a post about this a year or so ago when the pack server crashed and noone could get ingame because there was no redundancy, but it's just frustrating... I turned my PC on at 5am today and spent a hour downloading the packs only to find that when it was 95% some update had been released and I had to start from scratch downloading the thing again, it's now 7:30 - two and a half hours later and now I can finally login. Surely if I was a new player I would have well and truly given up by now.

 

 

 

 

Edited by Gavin
  • Like 8

Share this post


Link to post
Share on other sites

Oh great finished work, got a few hours to play toni.... Oh, gotta spend the next 6 hours downloading a pack.

 

7ec72cad20.png

 

(ie bump)

Edited by Gavin
  • Like 1

Share this post


Link to post
Share on other sites

Thanks for linking this thread in the other relevant thread Gavin, looks like you've put a decent amount of work into your OP. I really like #4 - but anything besides it and I would still say do not force client updates at non-maintenance restarts.
For people in PvP situations, if you're even taking 5 minutes to update the client during regular play hours, then that's a really crappy thing to have to deal with along side the constant memory leaks and crashes.

+1
This thread needs more attention - posted almost a week ago and there's no posts? Come on. At least acknowledge this kind of thing. Gavin gives a great example of where his internet is well above par, and he's still getting crap speeds.
There's been at least two other similar posts in the past couple of days about a similar thing, and another post asking if anyone relevant even reads these suggestion posts anymore. Please pick it up, development team. I love the new quality of life changes, but sweeping issues under the rug kinda sucks.

Share this post


Link to post
Share on other sites

Just because I doubt this is going to get fixed, here is something that might help: There is a great download accelerator called Internet Download Manager, if you use a cracked/paid copy of it you can download with up to 32 threads (basically splits the file into 32 parts and downloads 32 times at once from the server and puts it together at the end)

 

Probably gonna overload the server but hey it can't get any worse anyways, maybe it will encourage a fix.

 

98ef72a3eb.png

 

Still slow but 25 minutes is better than 4 hours.

 

  • Like 1

Share this post


Link to post
Share on other sites

i have very fast throughput, the fastest available in the region currently  and faster would require fiber to my box and i live in a major city and it is a large telecom hub.  ...the problem is not the download speed it is the patching process and someone has said that it had been that way for a while.  The last few times i load up and see jar download, i kill wurm go delete the jar, start back up and it downloads fine.  

 

Really do not need to get crazy with the download formats/options there is only like maybe 1000 people in the *entire world* who play Wurm regularly; a laptop at Starbucks could handle that kind of temporary serving, which should make it clear that the problem is not in the throughput and if it is then it is your connection not CC's issue.  I would prefer they work on the game and keep adding things personally than go waste time on hosting and split channel patch utility when the problem is not even the the throughput to begin with.

 

Easy fix...patch utility renames the exiting jar then loads the jar, not try to patch and verify each and every image in 1gb jar, seriously that is just wrong.

Edited by ClericGunem
  • Like 1

Share this post


Link to post
Share on other sites
1 hour ago, ClericGunem said:

i have very fast throughput, the fastest available in the region currently  and faster would require fiber to my box and i live in a major city and it is a large telecom hub.  ...the problem is not the download speed it is the patching process and someone has said that it had been that way for a while.  The last few times i load up and see jar download, i kill wurm go delete the jar, start back up and it downloads fine.  

 

Really do not need to get crazy with the download formats/options there is only like maybe 1000 people in the *entire world* who play Wurm regularly; a laptop at Starbucks could handle that kind of temporary serving, which should make it clear that the problem is not in the throughput and if it is then it is your connection not CC's issue.  I would prefer they work on the game and keep adding things personally than go waste time on hosting and split channel patch utility when the problem is not even the the throughput to begin with.

 

Easy fix...patch utility renames the exiting jar then loads the jar, not try to patch and verify each and every image in 1gb jar, seriously that is just wrong.

Why are you deleting the jar after you kill wurm? You should only have to close the limbo wurm process, and it should allow you to modify the jar with the files you just downloaded, with only another filecheck instead of having to redownload the entire jar.
At least that's always fixed my issue.

(Bump)

Share this post


Link to post
Share on other sites
9 minutes ago, Stik said:

Why are you deleting the jar after you kill wurm? You should only have to close the limbo wurm process, and it should allow you to modify the jar with the files you just downloaded, with only another filecheck instead of having to redownload the entire jar.
At least that's always fixed my issue.

(Bump)

 

hmmm, not certain what you mean.  When i see he jar downloading, flip to the packages screen i see it going though files/images whatever (i honestly dont care what they are), it is not simply downloading.  I have to close the client as with it open it does not remove the existing jar, but makes a temporary file which is where the new jar is being built/downloaded. Jar is just a zip file so if it doing it file-by-file then holy crap it is going to be slow by the simple fact of all the 3wh and rst involved, not to mention compression.  No matter, quit, delete the jar and the temp and the new jar downloads as a single file in, i think the slowest i have seen just after a release, in about 1 minute.

Edited by ClericGunem

Share this post


Link to post
Share on other sites
3 minutes ago, ClericGunem said:

 

hmmm, not certain what you mean.  When i see he jar downloading, flip to the packages screen i see it going though files/images whatever (i honestly dont care what they are), it is not simply downloading.  I have to close the client as with it open it does not remove the existing jar, but makes a temporary file which is where the new jar is being built/downloaded. Jar is just a zip file so if it doing it file-by-file then holy crap it is going to be slow by the simple fact of all the 3wh and rst involved, not to mention compression.  No matter, quit, delete the jar and the temp and the new jar downloads as a single file in, i think the slowest i have seen just after a release, in about 1 minute.

Ah my mistake, I just misunderstood you then.
A lot of people are having that issue where they have an invisible client open and they just need to close it through taskmanager and relaunch the game to start the download again. People in the suggestion threads kept telling them to delete their jars and that fixes it, when there's a much easier fix that has always worked for me.

Share this post


Link to post
Share on other sites
6 minutes ago, Stik said:

Ah my mistake, I just misunderstood you then.
A lot of people are having that issue where they have an invisible client open and they just need to close it through taskmanager and relaunch the game to start the download again. People in the suggestion threads kept telling them to delete their jars and that fixes it, when there's a much easier fix that has always worked for me.

 

Ah yes, i have had that one too, since i always have multiple clients i can tell pretty quick if that is the issue and have to choose to restart all or ignore update for the time.  No days it is more enforced as a client crashes (I never used to have crashes) i start back up and there is a new package, but when one client crashes they tend all crash over the next few minutes, so i go ahead and close all, kill any hung, then do the whole delete and then start a single client.  

 

One more thing i do is once the update is complete I go ahead and quit and start back up as seems some times there is some final commit that does not complete until the client is restarted

 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this