Keenan

Developer
  • Content Count

    2070
  • Joined

  • Last visited

  • Days Won

    36

Keenan last won the day on April 23

Keenan had the most liked content!

Community Reputation

3101 Rare

About Keenan

  • Rank
    Villager
  • Birthday 07/02/1981

Profile Information

  • Gender
    Male
  • Location
    (final String _locale)

Accounts

  • Chaos
    Keenan
  • Exodus
    Xorith

Recent Profile Visitors

6255 profile views
  1. It was a crash. The cause was identified as an edge case with tower influence and will be included in the update coming later this week. We don't expect it to be an issue again until then. So no, this isn't what can be expected.
  2. I was saving the bigger announcement for our hosting status update, but yes - AWS is not a viable solution for us. While in testing it worked fine, once we put more load on the system we ran into bottlenecks on the disk I/O that not even raising IOPS would resolve to a satisfactory condition. I'll give more information in another post over the next few days. I will say that I'm in the middle of testing one provider and in talks with a second.
  3. Hello all. We were made aware of Desertion's issues a few weeks ago, and at the time the plan was to roll it into AWS after Celebration's optimizations were complete. Today we had found that Desertion had degraded too far to continue waiting for our hosting solution to be ready and we decided to move it to a spare within Hetzner during maintenance. After moving the server it was unable to connect with other servers. This outage lasted about an hour until we found the configuration issue and corrected it. We will be posting an update on our hosting situation in the near future as we are currently in talks with other providers to see what fits us best. We've learned many lessons in our attempted move to AWS and we plan on using those lessons as we move forward. - Happy Wurming!
  4. A quick update. We rolled out the optimizations a day early due to needing to restart for Easter eggs to be available for folks. Unfortunately the database optimizations did not make enough of an impact on lag, so we are currently looking at the next steps to address this situation. I'll post a more meaty update when I have more information to share. Just know that we're still working on this and appreciate the patience as we work through such a huge change in server infrastructure.
  5. I've been aware of it for about a week. It's the same situation we had on Celebration where we suspect hardware but there's nothing definite showing up in our diagnostics (I just checked again today). Given the current state of the Celebration AWS uplift, we're waiting on that before we can really take action with Desertion. Should our optimizations on Monday go well, Desertion will be the 2nd server for uplift due to the situation. Update post:
  6. I can't speak for compensation, but I do want to discuss internally. The main issue with compensation is we cannot really do a single-server level of compensation. I mean we can do sleep bonus per-server, but that doesn't help after the fact since people may have gone to stay at a friend's deed because of the lag. Those people would miss out. And if we announced it beforehand and said "Hey! Come home so you can get free stuff!" then we'd have people flock to Celebration for it anyway. So anything we decide upon will likely be after we're done with all this host-moving and be applied globally. So it's not something I can promise, but it is something on my mind due to all this.
  7. Hi all, It's been a quiet week from us so I thought I'd give an update on what we're doing with Celebration, AWS, and future plans. On Monday we will be taking Celebration down to make some optimizations to the database. We are hopeful that this addresses the lag and that we can right-size the volume that hosts Celebration's data after a period of reviewing metrics. We can then continue with the AWS migration. I've been working on a back-up plan should the above fail, which is to convert the current infrastructure I've built into something that can be deployed anywhere - not just AWS. I don't think I've really said enough about how radically different the server configuration is in this new infrastructure, and how much easier it is to maintain. I didn't want to lose any of that in the event AWS doesn't work out for us. So for those tech-savvy people: I've been using Ansible to provision a VM locally to the same specifications as the instances I built in AWS. There's still some AWS dependency for now, but this gives us the freedom to look at other hosting options should we be unable to optimize the instance and database sufficiently to support Wurm. Just to be clear, the presence of this back-up plan is just that - as a fall back plan in the event that AWS cannot be optimized enough. We have not abandoned AWS at this time and we want to exhaust all of our options before we make that call. We just don't want to keep all our eggs in the same cloud. So that sums things up! Happy Wurming!
  8. As Alectrys said, it's more that the link has changed for servers in AWS. I've not had the time to set up a dummy nginx host to bounce people to the new link instead with a 301
  9. I keep asking to host a podcast or stream called "You think you do, but you really don't." - where I'd cover ideas that people bring up and say "No" to them in various ways. But alas, I've been denied my dream.
  10. Yeah I recalled the feature after it had been discussed. I don't think anyone really knew ignores counted, even though it was in the patch notes. As for my 5 year mistake, it's to do with how far back the history goes in that particular place I was looking. It goes back further elsewhere. So @MrGARY- I stand corrected.
  11. According to the code, it's been in for 5 years. Was there a previous version? As far as how easy it is to abuse, it's been out in WU therefore not hard to code-dive and find out how to abuse.
  12. Around 11am CEST the Celebration server experienced an outage. The cause: MySQL 8 has a default of 30 days retention on binary log files, which in our case are of a substantial size. They filled up the data drive that stores Celebration's game data and resulted in errors from both the map saving and the database. The server seems to have shutdown cleanly as soon as this occurred. The resolution: We've reduced the default and increased the size of the drive. We do not need 30 day retention of binary log files due to our server backup method. Other notes: Max was unable to restore services due to the nature of the outage and I was unable to be reached prior to 8am Eastern US time. I will be working with Max on a direct communication method that will cut through my phone's Do Not Disturb time so that I can be reached. We enabled sleep bonus for Celebration only upon the restart due to the duration of the outage.
  13. Celebration is coming back up with sleep bonus enabled.
  14. What is this?

    Like I said in my post above. It's a 5 year old feature. I'm not even sure anyone on the current dev team was aware it was in there since it was written by a former developer - 5 years ago. I had to check out of curiosity. Whether or not it stays in will be up to @Samool
  15. What is this?

    You've found a 5 year old feature that I'm not entirely sure anyone else has ever encountered before in Wurm Online.