Sign in to follow this  
Chakron

Niarja - Go use WurmNode!

Recommended Posts

Thanks Chakron, though that was truly bad news.

 

Just an idea: Niarja has proven a nearly indispensible tool for Wurmians, and its loss will be outright terrible. And it will be a loss for the attractivity of the game altogether. What about a takeover of the software by the company?

  • Like 6

Share this post


Link to post
Share on other sites
7 hours ago, Chakron said:

Well, it's been fun.

 

Niarja has been around since...at least 2015? I think a simplified version existed before then, but it's hard to remember now. I played Wurm at a very different time in my life. That's over now - I have no interest in the game, have no idea what's going on, and probably couldn't name a single developer if I tried. Niarja is very cheap to run (free) but there are still responsibilities such as performing security and backend upgrades on a regular basis. I've kept it running for 4-5 years past my "don't care" point, but I'm done now.

 

I'm still very impressed by how many people have enjoyed the tool and talk about it on the forums or in the official Wurm Discord. It's going away, but I'm confident someone will be able to create a new and better version that has all the features I never bothered adding.

 

All aspects of Niarja.com will shut down on October 1st, 2022. If you've used Niarja Skill Compare before, your "Manage Characters" page now contains a link to download all of your skill uploads into a single CSV file. Whatever tool exists in the future should be able to consume that file and get you caught up.

 

Thank you for everything you've done, Chakron - Niarja is a fantastic tool and it's really a big credit to you that you've kept it running this long, even well past the point at which Wurm still interests you.

 

 

I have to ask, would you consider open sourcing the code behind the platform? The site is extensively used by so many people and I'm sure many (myself included) would be willing to jump in and start hosting a replacement. However, the biggest issue to doing that is simply the amount of time it would take to develop a copy of Niarja from scratch.

 

If you're worried about handing over the project, I'd say open sourcing it would be a really good way to put the project to bed but still preserve the legacy you deserve as a part of the Wurm community and developer of what's arguably the most popular and useful fansite Wurm has ever had. By open sourcing the code, it would still be the tool you developed - except it would live on in another form and preserve the legacy of all the work you put into it.

 

In addition, there is so much other data that Niarja has - such as historical deed founding dates, slaying dates, etc - would you also be willing to do a single release of all of that data? If you're worried about hosting a file that big, then you could give it to just one person (I'd be willing!) who could then distribute it to others as and when needed. I have the hosting capacity available right now to put upto 1 TB of data up for anyone to download, and if it's bigger than that I could make it work any other way. Just the worst outcome would be if we lost all of that data forever with no straightforward way of replacing it (with how Twitter rate-limits it isn't straightforward to datamine old tweets, so would be very challenging to recreate from scratch.)

 

Again, thank you so much for all that you've done for Wurm over the years. I totally understand if you want to be rid of the tool and wouldn't even want to publicly release any of the data or code that runs it. But I think it would be a huge shame for Wurm to lose the tool and all of the data it has - and so if you're willing, it would be extremely useful if the code and data could be released in some larger (preferably open source) way. I'm here if you want to shoot a PM over to chat about how that could happen. 😃

Edited by Docterchese
  • Like 19
  • Cat 2

Share this post


Link to post
Share on other sites
2 hours ago, Ekcin said:

What about a takeover of the software by the company?

My thoughts exactly. Niarja is such a woderfull tool, was essential part of my wurm life since I discoverd it. I'm sure it would be valuable official addition to Wurm in some form. @Keenannot sure if team gave some thougts on it? Might be integrated into longly planned bigger system with account managment and stuff like that..

  • Like 2

Share this post


Link to post
Share on other sites

We never know how good we had things, until what we had is gone.

 

Thank you for sticking around so long. Niarja was one of the best parts of Wurm - and it wasn't even official.

  • Like 4

Share this post


Link to post
Share on other sites

Thank you so much for running niarja for so long, especially doing so long after you lost interest in the game. As indispensable as niarja is, I echo the sentiments mentioned in posts above and sincerely hope Wurm can produce a similar utility on an official level.

 

Thanks again Chakron 

  • Like 1

Share this post


Link to post
Share on other sites

i am sad i look at this tool everyday but i understand your point of view because when i am done with something i usually dont feel like keeping on doing it

  • Like 2

Share this post


Link to post
Share on other sites

Thanks, also mad respect for keeping it alife this long after being done with Wurm. 👍

Share this post


Link to post
Share on other sites

Wow thanks Chakron for running Niarja for so long, I had no idea it was one person behind it all, amazing.

 

The skill tracker/compare have been one of the main reasons I keep playing wurm. Really nice to see your rank for each skill and the graph to see how fast you've been training etc. I'm grateful to have had this tool over the years but It will be missed greatly. The Wurm Team or someone else needs to get on this 😃

Share this post


Link to post
Share on other sites
11 hours ago, Chakron said:

Well, it's been fun.

 

Niarja has been around since...at least 2015? I think a simplified version existed before then, but it's hard to remember now. I played Wurm at a very different time in my life. That's over now - I have no interest in the game, have no idea what's going on, and probably couldn't name a single developer if I tried. Niarja is very cheap to run (free) but there are still responsibilities such as performing security and backend upgrades on a regular basis. I've kept it running for 4-5 years past my "don't care" point, but I'm done now.

 

I'm still very impressed by how many people have enjoyed the tool and talk about it on the forums or in the official Wurm Discord. It's going away, but I'm confident someone will be able to create a new and better version that has all the features I never bothered adding.

 

All aspects of Niarja.com will shut down on October 1st, 2022. If you've used Niarja Skill Compare before, your "Manage Characters" page now contains a link to download all of your skill uploads into a single CSV file. Whatever tool exists in the future should be able to consume that file and get you caught up.

 

I can offer you to continue running Niarja. I have the server infrastructure and the skill to do so, in case you want to open source it or search someone taking up the project.

 

It is a great service used a lot on both Wurm Unlimited and Wurm Online. Would hate to see it go, especially because my players are also using it.

Edited by Sklo:D
  • Like 21
  • Cat 2

Share this post


Link to post
Share on other sites

First off, Thank you Chakron for making this valuable tool and keeping it up and running for us Wurmians, even though you no longer identify as such :) It's an incredible resource for us, thanks you so much.  Second, I echo the sentiments of those above in hoping that somehow we can preserve the data contained in the program and replicate it on another site, whether officially by the company or by another Wurmian. Thanks again Chakron, much appreciated :)

Share this post


Link to post
Share on other sites

Thank you so much for all this work over the years.  It really meant a lot to me. I'm amazed that anyone can make things like this. It is extremely valuable. If there is any way to keep it going, I think a lot of us would love that. Hell, I'd even be willing to pay a small subscription fee.

Share this post


Link to post
Share on other sites

Defnitely sad to see the service go. Thank you for all that you've done over the years!

 

I definitely blame the Triglavians.

Edited by Karthannar

Share this post


Link to post
Share on other sites

I've received almost a dozen messages from different people asking to take Niarja over. I'm not against the idea, but it may not be as straightforward as some people assume and I want to make sure they understand what they'd be taking on. Here's an explanation of the tech which you can ignore if you're not into that kind of thing.

 

Niarja isn't a single host that can be handed over. It runs on a variety of different AWS offerings. The Niarja website itself is Ruby on Rails and runs on EC2 instance(s) (there is a Beta instance for manual testing), and it doesn't do much more than render views. Niarja's database uses RDS (MySQL) and contains all of the processed tweet data, population data, and character skill data. The rest is "serverless" - all processing of tweets, population snapshots, and character data happens through AWS Lambdas written in Java, and fronted by API Gateway. Raw skill dump files live in S3. Both the backend and frontend packages push to CodeCommit. The frontend uses CodePipeline to deploy through the Beta and Production environment stages. The backend uses serverless (because this was pre-the AWS CDK being any good) to perform infrastructure updates via CloudFormation and while the tweet-consuming process contains decent unit tests it's otherwise very manual. You'll also need to know a little bit about VPCs and communicating in and out of private subnets. Oh right, the frontend EC2 instances are actually set up via Elastic Beanstalk for some reason I can't remember.

 

If you know what at least 80% of those words mean, you're probably in a decent position to take over Niarja. But, you will have to deal with these problems:

- I use the overall AWS account for other projects (best practices be damned) and can't simply hand it over to you. This isn't that big of a deal, but just FYI.

- Everything is within limits of AWS's free tiers, except any EC2 instances I'm currently running & the MySQL database. Those end up net-free for me for various reasons, but they wouldn't be free for you.

- I created Niarja to learn more about AWS, back before I started using it in my professional life. It's scrappy, poorly tested, and doesn't follow best practices. Ruby on Rails is an opinioned framework and I...kinda followed that? I guess Niarja would be fun to own if you enjoy refactoring bad code that other people wrote. Niarja was my testing ground and the next thing I would have done would be converting the frontend to React, and the backend to DynamoDB.

- I also wouldn't want to give you any character skill data, because that's all linked to existing email addresses of registered users. I'd never expose those to another owner, those will die with "Niarja 1.0" and will need to be re-created.

 

So that's what you'd be in for. This isn't very complicated if you've ever managed cloud-based infrastructure before, but if you haven't then this would probably be discouraging.

 

----

 

If you didn't feel like dealing with that, I could also just upload stuff to GitHub. The logic for parsing tweets and skill dump files would be useful. I also have a full archive of "unprocessed" and "processed" tweets, though you'd have to keep in mind that manual corrections were often necessary so replaying the "unprocessed" tweets has a lot of edge cases.

 

Just let me know 🤷‍♂️

  • Like 8
  • Cat 1

Share this post


Link to post
Share on other sites
15 minutes ago, Karthannar said:

I definitely blame the Triglavians.

 

No one has ever asked me what that weird Niarja word means. Yes, I bought the domain a long time ago to track ganking activity in Niarja. Never ended up making an app, but still had the domain so I reused it. Some good times were had in Niarja a very long time ago.

  • Like 3

Share this post


Link to post
Share on other sites

Thank you Chakron, for a wonderful indispensable resource and for all the help over the years.  Please try to hand the baton over to someone else to carry if possible.  Wishing you all the best - please drop in to the forums and say "Hi!" from time to time. :) 

Share this post


Link to post
Share on other sites
22 minutes ago, Chakron said:

I've received almost a dozen messages from different people asking to take Niarja over. I'm not against the idea, but it may not be as straightforward as some people assume and I want to make sure they understand what they'd be taking on. Here's an explanation of the tech which you can ignore if you're not into that kind of thing.

 

Niarja isn't a single host that can be handed over. It runs on a variety of different AWS offerings. The Niarja website itself is Ruby on Rails and runs on EC2 instance(s) (there is a Beta instance for manual testing), and it doesn't do much more than render views. Niarja's database uses RDS (MySQL) and contains all of the processed tweet data, population data, and character skill data. The rest is "serverless" - all processing of tweets, population snapshots, and character data happens through AWS Lambdas written in Java, and fronted by API Gateway. Raw skill dump files live in S3. Both the backend and frontend packages push to CodeCommit. The frontend uses CodePipeline to deploy through the Beta and Production environment stages. The backend uses serverless (because this was pre-the AWS CDK being any good) to perform infrastructure updates via CloudFormation and while the tweet-consuming process contains decent unit tests it's otherwise very manual. You'll also need to know a little bit about VPCs and communicating in and out of private subnets. Oh right, the frontend EC2 instances are actually set up via Elastic Beanstalk for some reason I can't remember.

 

If you know what at least 80% of those words mean, you're probably in a decent position to take over Niarja. But, you will have to deal with these problems:

- I use the overall AWS account for other projects (best practices be damned) and can't simply hand it over to you. This isn't that big of a deal, but just FYI.

- Everything is within limits of AWS's free tiers, except any EC2 instances I'm currently running & the MySQL database. Those end up net-free for me for various reasons, but they wouldn't be free for you.

- I created Niarja to learn more about AWS, back before I started using it in my professional life. It's scrappy, poorly tested, and doesn't follow best practices. Ruby on Rails is an opinioned framework and I...kinda followed that? I guess Niarja would be fun to own if you enjoy refactoring bad code that other people wrote. Niarja was my testing ground and the next thing I would have done would be converting the frontend to React, and the backend to DynamoDB.

- I also wouldn't want to give you any character skill data, because that's all linked to existing email addresses of registered users. I'd never expose those to another owner, those will die with "Niarja 1.0" and will need to be re-created.

 

So that's what you'd be in for. This isn't very complicated if you've ever managed cloud-based infrastructure before, but if you haven't then this would probably be discouraging.

 

----

 

If you didn't feel like dealing with that, I could also just upload stuff to GitHub. The logic for parsing tweets and skill dump files would be useful. I also have a full archive of "unprocessed" and "processed" tweets, though you'd have to keep in mind that manual corrections were often necessary so replaying the "unprocessed" tweets has a lot of edge cases.

 

Just let me know 🤷‍♂️

 

Well that is quite a technology stack. I know most of that stuff and also dealt with Ruby on Rails in the past. But the time needed to set this all up and become familiar with the setup on AWS is huge. I still have like 6 weeks off until the end of the year so I could come up with the time. But honestly, with the huge number of people offering to become the hoster of niarja, I think the fairest option would be to upload everything to GitHub and whoever can host it stable first should be the person who continues running Niarja.

Edited by Sklo:D
  • Like 2

Share this post


Link to post
Share on other sites
1 minute ago, Sklo:D said:

Well that is quite a technology stack.

 

Yeah, this. At least it runs on a MySQL database, though; it wouldn't be so difficult to at least use the existing code as a guide and write new backend components, most languages and frameworks have ways to interact with MySQL databases. Tweet scraping is actually super easy in e.g. Python using Tweepy, for instance (the point being that whoever takes it on could rewrite backend components in tech of their choosing that could also work in a non-AWS context, as AWS lambdas can obviously only run on AWS.)

 

 

18 minutes ago, Chakron said:

- I also wouldn't want to give you any character skill data, because that's all linked to existing email addresses of registered users. I'd never expose those to another owner, those will die with "Niarja 1.0" and will need to be re-created.

 

It does make sense to keep things private, but I do wonder about the merit of not giving over any player data. A lot of basics (like STP leaderboards) are archived at the Wayback Machine and the data will be there forever.  Hell, many accounts (like Nineol) have had at least a few snapshots over the years, so I would argue much of that data is already out in public. Maybe it would at least be possible to transfer over a small amount? Like just the latest snapshots on all accounts (but not including email data), so that the leaderboard aspect of whatever comes after Niarja can still work (and uses data that's realistically publicly available anyway.)

 

Share this post


Link to post
Share on other sites
1 minute ago, Docterchese said:

 

Yeah, this. At least it runs on a MySQL database, though; it wouldn't be so difficult to at least use the existing code as a guide and write new backend components, most languages and frameworks have ways to interact with MySQL databases. Tweet scraping is actually super easy in e.g. Python using Tweepy, for instance (the point being that whoever takes it on could rewrite backend components in tech of their choosing that could also work in a non-AWS context, as AWS lambdas can obviously only run on AWS.)

 

 

I could hand over all the Lambda-executing code, the Lambda specific portions are actually very small. It could run anywhere with very minimal effort. When Niarja first launched all the processing logic was written in Ruby and in the Rails application - that was a particularly bad, monolithic approach. Wouldn't recommend that.

 

1 minute ago, Docterchese said:

It does make sense to keep things private, but I do wonder about the merit of not giving over any player data. A lot of basics (like STP leaderboards) are archived at the Wayback Machine and the data will be there forever.  Hell, many accounts (like Nineol) have had at least a few snapshots over the years, so I would argue much of that data is already out in public. Maybe it would at least be possible to transfer over a small amount? Like just the latest snapshots on all accounts (but not including email data), so that the leaderboard aspect of whatever comes after Niarja can still work (and uses data that's realistically publicly available anyway.)

 

 

Yeah sorry I should clarify - no issue with handing over the public character data, just not the user data. The only problem is that users have permission to create/edit/delete characters and character data. You'd have a bunch of orphaned characters and would need to figure out a good way to get them back to their owners (or into an owner-less state). Not my problem, but it's worth pointing out the work that'd need to be done!

  • Like 4

Share this post


Link to post
Share on other sites
3 minutes ago, Chakron said:

 

I could hand over all the Lambda-executing code, the Lambda specific portions are actually very small. It could run anywhere with very minimal effort. When Niarja first launched all the processing logic was written in Ruby and in the Rails application - that was a particularly bad, monolithic approach. Wouldn't recommend that.

 

Yeah sorry I should clarify - no issue with handing over the public character data, just not the user data. The only problem is that users have permission to create/edit/delete characters and character data. You'd have a bunch of orphaned characters and would need to figure out a good way to get them back to their owners (or into an owner-less state). Not my problem, but it's worth pointing out the work that'd need to be done!

 

 

Ok, this sounds great! And it would be awesome to have all of the player data. Dealing with the orphaned accounts would be a pain, but would be worth it.

 

Thank you so much for agreeing to facilitate handing the project over! It sounds like it will take a bit of work, but we'll all appreciate it a lot. 😊

Share this post


Link to post
Share on other sites
1 hour ago, Chakron said:

 

No one has ever asked me what that weird Niarja word means. Yes, I bought the domain a long time ago to track ganking activity in Niarja. Never ended up making an app, but still had the domain so I reused it. Some good times were had in Niarja a very long time ago.

wow i always wondered if the name has something to do with Eve online, every time I head to the site i think about the old days.... Anyway, thank you so much for continuing to host this service for so long, its been incredibly useful over the years. Thank you so much for your efforts, i really hope you can manage to transfer this project to someone else will will maintain, its a core part of wurm even if code club has nothing to do with it.

Share this post


Link to post
Share on other sites

Thank you for hosting the site for so long :) it came in handy over the years. I'd love to get my hands on the stp calculation code, it was a great way to track progress.

Share this post


Link to post
Share on other sites
8 hours ago, Chakron said:

I've received almost a dozen messages from different people asking to take Niarja over. I'm not against the idea, but it may not be as straightforward as some people assume and I want to make sure they understand what they'd be taking on. Here's an explanation of the tech which you can ignore if you're not into that kind of thing.

 

Let us know how you decided. I should have the time to move it away from AWS to make sure it can be hosted easier starting from mid October. Definitly willing to try if you have no one who is fitting better. 

Edited by Sklo:D
  • Like 4

Share this post


Link to post
Share on other sites

Oi, Chakron.

 

Thank you for creating and running Niarja all these years.  It's been everything from dead useful (Rifts), to interesting (casts, skill comparisons, player numbers).  You deserved to be thanked before, but I won't miss the opportunity now.

 

Also, sometimes people come back to Wurm, but I wish you well whether or no.

 

Cheers.

  • Like 1

Share this post


Link to post
Share on other sites

Question: Given that someone like Sklo is ready to migrate/reverse engineer Niarja service, would it be possible to postpone the closure past Oct 1 which seems to be a very strict deadline for a new service? If this is a matter of cost, would it be a possibility to set up a collection to cover them if necessary?

Share this post


Link to post
Share on other sites
3 hours ago, Teggs said:

Oi, Chakron.

 

Thank you for creating and running Niarja all these years.  It's been everything from dead useful (Rifts), to interesting (casts, skill comparisons, player numbers).  You deserved to be thanked before, but I won't miss the opportunity now.

 

Also, sometimes people come back to Wurm, but I wish you well whether or no.

 

Cheers.

that one, there

shoutout for @Chakron

 

Won't throw rocks and such at rifts.. but that simple functionality helped many people a lot with the timezones and start time and sharing it around to lower frustraction(even today people travel a day or two(IRL) earlier to reach the spots and be on time for the event when it happens; these never changed.. only QoL to rifts was Niarja)

Niarja turned into only place to brag for skill ticks and such, to the point some lost their minds after stp ladder.🤣(job well done there, you created a game inside the game, outside of it :))

No doubt best visualization for skills, even if there was room to expand functionality, this was plenty and best part .. was timeline progression tracking.(for fun and bragging)

Best sneak tool to whisper a top skilled player for tips or find some odd hidden deed whispering the main on other server.. to do a mission and such.(not great for privacy, but c'mon they are in mmo, they enjoyed being a little bit more social for a moment)

Since we lack any kind of official tracking who did what and so on.. this was the only dArgon or w/e unique kill tracker, epic construction, village creation history, etc.. rounded up in 1 profile page - A- ***** -mazing right there.

 

A lot of really good things, yes.. also quite useless most of the time, but when needed - all summarized at one reliable place for so long.

Few fellas gone wild with the power of deed R.I.Ps.. and tracked these events etc.. made a bank..

Bunch of stories created around Niarja's features. 

  • Like 1

Share this post


Link to post
Share on other sites
Guest
This topic is now closed to further replies.
Sign in to follow this