• Content Count

  • Joined

  • Last visited

  • Days Won


Keenan last won the day on January 20

Keenan had the most liked content!

Community Reputation

2893 Rare

About Keenan

  • Rank
  • Birthday 07/02/1981

Profile Information

  • Gender
  • Location
    (final String _locale)


  • Chaos
  • Exodus

Recent Profile Visitors

5858 profile views
  1. Sorry for having left the edit and header up as long as I did, but I needed some sleep! Hetzner eventually marked the issue resolved around 05:30 server time. So we've been in the clear for a while now. Happy Wurming!
  2. We are currently experiencing an all-server outage. Due to the nature of the outage we suspect Hetzner has had an internal issue that has made all of our servers inaccessible. Even our build infrastructure is unavailable at this time. Only the services migrated to AWS and the main website are accessible, with the main website being a different type of Hetzner service than the rest of the servers. We've lodged a support ticket with our provider and are awaiting a response. There is currently no ETA. We will update this post once we have more information. Thank you for your patience. Edit: 04:30 (server time) - Servers appear to be online and accessible once again. Edit: 04:35 (server time) - Hetzner has confirmed a network outage and are working on the problem. Expect connectivity to be degraded or intermittent while they work to correct the issue. Edit: The issue has been resolved.
  3. At 03:50 server time an issue occurred that affected the skills tables for both creatures and players. This caused an issue with reading skill data and possibly saving skill data. The issue was brought to my attention at 04:07 server time when players complained about having no skill and being unable to complete actions as a result. I had first attempted to repair the table while the server was running in order to preserve any data from skill gains that I could, but this only made the problem worse - the table seemed to have vanished due to the repair failing. Xanadu was brought down at 04:20 server time. At this point I was able to save the binary data of the skills table in order to preserve it and initiated a MySQL restart. When the database server came back online the table was restored and repaired. I verified that skill data seemed intact by picking a few players who had complained originally and finding their skills to have appropriate values. Xanadu was brought back online and available for connections at 04:50 server time. Due to this issue there may have been a period of about 30 minutes where skill ticks were not saved. I will be meeting with Samool in the morning to discuss compensation for the downtime and possible 30 minute skill loss. Most likely we will be doing 5 hours of sleep bonus on Thursday's restart. We will update to confirm the decision. Edit: Confirmed - 5 hours of sleep bonus on Jan 23 restart. Thank you for your patience.
  4. Server down

    I'll be putting up a postmortem once the issue is resolved. Xanadu is starting back up now.
  5. I'm glad. Happy Wurming!
  6. I tried so hard, and got so far. But in the end, it didn't even matter. I'm still going to finish this install. I can't stop now.
  7. Part of me hopes this works. Part of me wants to cry if it does. I've spent all morning installing Gentoo on a VM to test this issue.
  8. Data privacy is problematic on Amazon? Maybe for misconfigured systems... You're stepping into my professional wheelhouse here. I know all about AWS security, data privacy, etc. I'm responsible for ensuring PII is well-kept at work, which far exceeds what Wurm's data is. You can't attribute the debacles with Ring and what they do with Amazon's shopping front with AWS. They may be owned by the same person but they're entirely different organizations with AWS being vetted by regulators.
  9. You can use -c for choose client or -test for the test client directly.
  10. I have had the same problems, and unfortunately I've also had issues with the Frankfurt AWS zone. I think this issue is wider spread than just Hetzner this time.
  11. Hi all. After getting the test servers running in AWS, I turned my attention to our build servers. While doing so I identified several areas where security could be improved and have been working on that. I've also been struggling to find ways to export our current Jenkins pipeline into a format that can be safely committed to source control and imported back without issue. I'm tempted to forego this in the interest of furthering our on-boarding to AWS though. If anyone has had experience with Jenkins jobs in SCM, please feel free to comment! Finally, I've been doing more research into some best practices regarding managing the Wurm server containers. One thing that test has shown me is that my current set up is harder to manage than I'd like. While it's better than what we had before on the test side of things, it's still not where I want it to be for us to go live. I'd like to build monitoring and alerts into this infrastructure as well. I look forward to testing the new infrastructure changes I've made in the coming weeks. Thanks for the patience and happy Wurming!
  12. Are you able to open ? The client's files are hosted on the same server. If you can't then there's not much we can do as there's something blocking it from outside of our control.
  13. I also think it's rather toxic to be discussing the removal of other staff.
  14. Thank you. I couldn't have said it better myself.