Sign in to follow this  
madnezz

Extensive Testing of changes before going live

Recommended Posts

It's great that we have developers that want to add new things and work on the game in a volunteer capacity but its quite obvious they have no intention of thoroughly testing their changes before forcing us, the paying customers, to endure their mistakes and test for them whether we want to or not.  

 

In other games where development is not a hobby or a game in and of itself, for instance Ultima Online, they at least follow these steps:

 

1.  Develop ideas and test them themselves.

2.  Announce changes and document them thoroughly for player consumption, every change no matter how significant.  

3.  Make the changes available to test on a test server that actually promotes player testing, maybe a week or more.

4.  Make the changes available to test on 2 or 3 out of 15+ production servers for further testing, usually for a week.

5.  Release the changes on all live servers and update the patch notes with every new change as a result of the testing.

 

Here are a couple of easy things that can be implemented immediately and later improved:

 

1.  Encourage Wurm's paying customers to participate in testing changes on a test server by rewarding players with sleep bonus on their main accounts.  Easy and free.  Have the changes available to test for at least 48 hours, the weekends would be a good time for this.  

 

2.  Document, document, document every single, I repeat, every single change you make to the code and make those changes available on the public forums before the changes go live and recap the extent of those changes and tweaks to them after they go live.  

 

 

 

 

 

 

 

 

 

Edited by madnezz
  • Like 8

Share this post


Link to post
Share on other sites

CC does not have the staff to do extensive testing, nor does it have the player base to help out too on the test server.

 

 

 

  • Like 3

Share this post


Link to post
Share on other sites
1 hour ago, bigsteve said:

CC does not have the staff to do extensive testing, nor does it have the player base to help out too on the test server.

 

 

 

 

Virtually all of this can be automated. Changes need much more proper and extensive testing, especially testing on large & populous servers. And yes, that last bit can be automated/simulated. This is well within the bounds of what would be considered normal testing.

 

Testing a feature on a rather small server without many people on it and just assuming it'll work on a server several times the size and with dozens of times more animals, players, items, etc. is just not sustainable and results in the exact problems we were (are?) seeing on Xanadu after the movement changes were implemented.

Share this post


Link to post
Share on other sites

While the principle is sound Ostentatio madnezz, the fact is even those " In other games where development is not a hobby or a game " screw up as bad and significantly worse even with far more extensive testing and highly advance PTS systems.  You want extensive testing donate a 100k, i.e. put up or shut up.  What you are proposing to CC is "stop doing anything useful" and there are less people on overall right now than were on Xanadu alone 4 months ago and from what i have seen a good portion of those are looter alts, a blip to make things better and hopefully make some appealing changes to bring people back.  Because right now the release of new things is not fast enough to retain people and there is a ways to go before they can compete and pull people away from some of the new sandbox mmos coming out

 

What may be reasonably done, assuming they do not already, is for them to spend some time (to Ostentatio's point) adding in some automation on the test servers to mimic a few hundred of the common actions that happen.  You assume they do not test or that their testing is sub par and i presume that you know that what "assume" means.  Unless you happen to own a dev house of any kind try offering to help not telling them how to run things.

 

Edited by ClericGunem
addressing wrong person

Share this post


Link to post
Share on other sites

If they do not have the capability/man hours to test their changes then maybe they should make much smaller changes where they can test them.  It's preposterous to say they can't afford to test, they can't afford not to.

 

If they aren't going to test sufficiently and they are unwilling to reward the player-base with free sleep bonus for testing then the least they can do is an entire pvp ban for 24 to 48 hours after each change so we can test for them without losing in pvp to bugs.

 

This is the 3rd time recently when a major change has put accounts in jeopardy, its well past time to correct this problem that is so simple to fix.  Bridges, permissions, and now movement.

 

Edited by madnezz

Share this post


Link to post
Share on other sites

wasn't a problem for you before, why change your mind now?

 

hopefully you're one of the first people to test things, but i really doubt you would be.

 

+1 either way.

 

 

Share this post


Link to post
Share on other sites
2 hours ago, ClericGunem said:

While the principle is sound Ostentatio, the fact is even those " In other games where development is not a hobby or a game " screw up as bad and significantly worse even with far more extensive testing and highly advance PTS systems.  You want extensive testing donate a 100k, i.e. put up or shut up.  What you are proposing to CC is "stop doing anything useful" and there are less people on overall right now than were on Xanadu alone 4 months ago and from what i have seen a good portion of those are looter alts, a blip to make things better and hopefully make some appealing changes to bring people back.  Because right now the release of new things is not fast enough to retain people and there is a ways to go before they can compete and pull people away from some of the new sandbox mmos coming out

 

What may be reasonably done, assuming they do not already, is for them to spend some time adding in some automation on the test servers to mimic a few hundred of the common actions that happen.  You assume they do not test or that their testing is sub par and i presume that you know that what "assume" means.  Unless you happen to own a dev house of any kind try offering to help not telling them how to run things.

 

 

I am, at this point, from speaking with staff and people actually involved, about 90% certain that Wurm Online's developers do not actually have a means set up by which to simulate the player/map environment on Xanadu for testing purposes. If I'm wrong, I'm wrong, but the problems were so incredibly self-evident this time around that I don't see how they possibly could have tested it in such an environment without noticing.

Share this post


Link to post
Share on other sites

-1 quit crying

its just a game, they do live testing because NOBODY ever uses the test server... if you want it fixed, start using the test server

Share this post


Link to post
Share on other sites

The test server is a very limited testing environment. It's very useful for a lot of things, but does not accomplish the kind of testing I was just talking about. The test server is never, ever going to have as many active players at once as Xanadu, for example, no matter how many people genuinely want to help out, nor would this be a very efficient way of testing things.

 

If you're going to keep talking about this, please put some thought into what you're saying and replying to, beyond just unwarranted aggression. People have a reason to be annoyed when stuff like this happens.

Share this post


Link to post
Share on other sites

-5000 this would greatly delay new content... 

 

all i see is cry cry cry... a new patch made it laggy for a couple days... take a chill pill they are working on it

Share this post


Link to post
Share on other sites
2 minutes ago, Evilreaper said:

-5000 this would greatly delay new content...

 

It would not in any meaningful sense. Besides, working out bugs after the fact involves similar effort anyway, plus the damage control necessitated by players getting aggravated by it, not to mention dealing with things like abusable bugs in PvP and things adversely affecting players in a significant enough manner that they're all clogging up the support system with it.

Share this post


Link to post
Share on other sites
5 minutes ago, Evilreaper said:

-5000 this would greatly delay new content... 

 

all i see is cry cry cry... a new patch made it laggy for a couple days... take a chill pill they are working on it

i'd rather see an update delayed for a week then deal with broken stuff messing the gameplay up.

 

People would rather play the game as is then get new shiny things that are broken or break other working things.

Edited by TradingAlt

Share this post


Link to post
Share on other sites
6 minutes ago, Evilreaper said:

-5000 this would greatly delay new content... 

 

all i see is cry cry cry... a new patch made it laggy for a couple days... take a chill pill they are working on it

 

I love new content, but would rather wait for it than see broken content go live.

Share this post


Link to post
Share on other sites

never mind, thread appears to be going direction i do not want to be part of

Edited by ClericGunem

Share this post


Link to post
Share on other sites

+1 please. Most people have been asking for better patch notes, testing and overall professionalism for literally the last decade. If Rolf wants to know what will get people to play this game for longer and stick around. This would be a bloody great place to start! 

  • Like 1

Share this post


Link to post
Share on other sites

-1

 

While the idea behind this is sound it is simply not feasible.  As previously mentioned wurm's community is now too small to rely on a dedicated group to do the testing and CC does not have the staff to do it inhouse.  The test server reward idea was a nice one, but in reality it would just be used by people to farm sb (log a lot of alts on, accrue sb, log out) and doesn't encourage bug testing.

 

Something more workable would be a tangible reward for being the first to report a (verified) bug on the test server (pay us in silver, premium time or lovely items we currently cannot get!), then you would see people taking up the role of testers more readily.  Of course then you'd have the issue of people reporting duplicate bugs, but thats what forum staff are for...

Share this post


Link to post
Share on other sites
4 hours ago, Etherdrifter said:

-1

 

While the idea behind this is sound it is simply not feasible.  As previously mentioned wurm's community is now too small to rely on a dedicated group to do the testing and CC does not have the staff to do it inhouse.  The test server reward idea was a nice one, but in reality it would just be used by people to farm sb (log a lot of alts on, accrue sb, log out) and doesn't encourage bug testing.

 

Something more workable would be a tangible reward for being the first to report a (verified) bug on the test server (pay us in silver, premium time or lovely items we currently cannot get!), then you would see people taking up the role of testers more readily.  Of course then you'd have the issue of people reporting duplicate bugs, but thats what forum staff are for...

 

Regarding the first point: You don't need a lot of staff to simulate a large, populous server. You just need to set the thing up.

 

Regarding the second: Incentivizing people to report bugs quickly does nothing in cases like what just happened, where it affects entire servers in extremely obvious ways; that stuff gets reported pretty much immediately anyway.

Share this post


Link to post
Share on other sites

This shouldnt even need to be a suggestion! It is becoming ridiculous how much non tested patches are affecting this game especially the PVP servers. But even the PVE servers are affected negatively. 

 

Its fine if this project is nothing other then a play toy for the devs (because thats what it feels like) but if thats the case then with every new patch put a 24h raid ban so bugs can be found.. And when bugs are found extend the ban accordingly... Its not that hard. 

 

Also why are their not tests run on patches... If the answer is not enough people to test i can bet you there are a million ways to get enough people to test which wont cost wurm any real money. Creat test scenarios and thise that complete it recieve something thats not OP.. A sleep powder non transferable.

Share this post


Link to post
Share on other sites

-1 im perfectly happy with the way things are done.
adds personality to the game and makes us feel like part of the process.

That said, im not a pvp'er, if i was.. others discovering exploits before me would probably annoy me quite a bit.
But im not exploitative so i don't care about that.
 

Edited by Steveleeb

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this