Retrograde

W. S. A. Independence lag

Recommended Posts

Hi everyone, 

 

As you may have seen independence is currently experiencing some significant lag. Our investigations are pointing towards hardware issues, which mean swapping independence to different hardware. 

 

We aim to have an interim solution up tomorrow and will work towards a longer term solution over the coming days. 

 

Edit:

In 7 hours we'll be starting maintenance to transition Indy to the new hardware. Downtime likely to be 1-3 hours and will apply to ALL servers. 

 

Edit (Keenan): Down time updated to reflect the data transfer speed (or lack thereof). Sorry folks!

 

Share this post


Link to post
Share on other sites

Chaos as well, my horse, same for others is always 10 tiles behind. 

Share this post


Link to post
Share on other sites

thank you this has affected me greatly

 

where do i send you my paypal info for compensation

 

 

on a serious note tho, my backup horse is also always like 5 to 10 tiles behind me and to the left, so far away i usually think its killed in pvp and dont even use it

Edited by platinumteef

Share this post


Link to post
Share on other sites

If Inde gets brand new hardware for a bit of lag then what should Xanadu get moved to? The hadron collider?:)

 

On a serious note: thank you for fixing it guys. We all await sleep bonus now :)

Share this post


Link to post
Share on other sites

I am excited the lag is getting fixed! Thank you!

Share this post


Link to post
Share on other sites

I hope the lag gets fixed with this new hardware. Thank you for your support and your work!

Share this post


Link to post
Share on other sites

While everything is possible, from my limited view it is not likely that hardware issues can cause something like that, especially when the web server and icmp request on the same machine are handled without any noticeable delays.

Share this post


Link to post
Share on other sites
3 minutes ago, Sklo:D said:

While everything is possible, from my limited view it is not likely that hardware issues can cause something like that, especially when the web server and icmp request on the same machine are handled without any noticeable delays.

You have to realize they are expert investigators clearly there was something hardware wise that was failing ;)

Anyway any chance we could hear some more info on it for those curious as to what was going on as id be interested in knowing and im sure a few others are as well

Share this post


Link to post
Share on other sites

I'm interested in what broke, but also what is being changed.

 

From the sounds could be a simple hardware swap to a whole system swap.

Share this post


Link to post
Share on other sites
19 minutes ago, Kelody said:

I'm interested in what broke, but also what is being changed.

 

From the sounds could be a simple hardware swap to a whole system swap.

 

For the record I would like to know that too since Sklotopolis runs on the same machines at hetzner :P

Share this post


Link to post
Share on other sites

tyvm for your hard work, hope it'll help

Edited by ShloEmi

Share this post


Link to post
Share on other sites

Thank you, sounds like the lag torture is gonna end soon!

Share this post


Link to post
Share on other sites

Drank coffee.

Prepped new hardware.

Got heckled by Redi. :P

I think we're ... Redi.

Share this post


Link to post
Share on other sites

Kicked off the usual 30 minute shutdown for all servers. The reason all servers need to come down for this is that we must update their references to Independence.

 

For those who have asked, I will be posting a postmortem once we confirm that new hardware has addressed the issue.

 

 

Share this post


Link to post
Share on other sites
3 minutes ago, Keenan said:

Kicked off the usual 30 minute shutdown for all servers. The reason all servers need to come down for this is that we must update their references to Independence.

 

For those who have asked, I will be posting a postmortem once we confirm that new hardware has addressed the issue.

 

 

 

Thumbs up and GL on the fix Keenan! :)

 

Share this post


Link to post
Share on other sites

Thanks for the information and updates team.

 

I know it is a real pain for people, however, looking at who amongst the management is on-line anyone can see that it is 'all hands to the pump'. They are obviously doing their best.

 

Good luck with the medicine

Share this post


Link to post
Share on other sites

Good luck for the switch thanks for improving our game even more 👍

Share this post


Link to post
Share on other sites

'Wurmageddon 2' says the little red hen.

(me pessimist? never)

Share this post


Link to post
Share on other sites

Hopefully the restart wont kill more animals. buddy had a tamed champ die on restart. Goodluck on the transfer and long term goals about this matter.

Share this post


Link to post
Share on other sites
Just now, ZeldaGold said:

Hopefully the restart wont kill more animals. buddy had a tamed champ die on restart. Goodluck on the transfer and long term goals about this matter.

i read this the first time as "Tamed Chimp"

 

wheres my glasses!

Share this post


Link to post
Share on other sites
54 minutes ago, Keenan said:

Kicked off the usual 30 minute shutdown for all servers. The reason all servers need to come down for this is that we must update their references to Independence.

 

For those who have asked, I will be posting a postmortem once we confirm that new hardware has addressed the issue.

 

 

 

tyvm, this lag drove me crazy, barely able to play. GL!!

Share this post


Link to post
Share on other sites

UPDATE

The database has been copied over to the new hardware and verified.

 

I can give a little more explanation as it will help to understand why this is taking so long. Essentially the read performance of Independence's drives took a tank. The data integrity is fine, but it's taking an insane amount of time to actually read the data. Which makes transferring said data to new hardware take longer as well.

 

In the interest of full transparency: we heavily suspect the hardware is the culprit, but we have not ruled out that there's a software component causing this. Diagnostics on the drives are fine, hence the data integrity being good. I always believe that a good system administrator remains a skeptic to the very end!

 

Another note to mention is that the lag graph will not be updated to the new hardware right away. The primary focus is getting back up. Besides, I think it'll be obvious if this doesn't rectify the situation. The lag wasn't exactly difficult to notice.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now