Posted March 7, 2016 (edited) This is a thread which goes especially to the teams of bigger servers. Servers are growing and lag is turning up from time to time. On Sklotopolis we are having some troubles with lag lately since we turned creatures up to 60k, so creatures seem to have a mayor impact beause of the different Zone poller methods. The method pollNextZones takes up to 16 seconds sometimes so more than 16x the time of a normal loop, which is bad of course but not happening more often than twice a day... Our first guess is to adjust the following values in the next days and see how things turn out: NUMBER_OF_DB_ITEM_DAMAGES_TO_UPDATE_EACH_TIME => lower NUMBER_OF_DB_CREATURE_POSITIONS_TO_UPDATE_EACH_TIME => lower NUMBER_OF_DB_PLAYER_POSITIONS_TO_UPDATE_EACH_TIME => lower USE_SCHEDULED_EXECUTOR_TO_UPDATE_CREATURE_POSITION_IN_DATABASE = true??? Does it increase performance? USE_SCHEDULED_EXECUTOR_FOR_SERVER = true??? Does it increase performance? What are you guys out there using to increase performance? Do you think our ideas will help? Edited March 7, 2016 by Sklo:D Share this post Link to post Share on other sites
Posted March 7, 2016 I don't know much about the code, but when I did a test 2048 server with 1mil critters set, I found after a few days, the lag hit. Quick server reboot and it was back to normal. Do you do server reboots regularly? Sorry if this doesn't apply to your request appropriately. Share this post Link to post Share on other sites
Posted March 7, 2016 It is a good point, so server restarts can matter of course and reduce lag. Do you really mean 1 000 000 creatures? Don't think that so many creatures will spawn, the server has something like a limit per Zone? or something like that. Share this post Link to post Share on other sites
Posted March 7, 2016 What size is your map? Shoot with 80k creatures set they're almost stacked on top of each other on our server Share this post Link to post Share on other sites
Posted March 7, 2016 25 minutes ago, Sklo:D said: It is a good point, so server restarts can matter of course and reduce lag. Do you really mean 1 000 000 creatures? Don't think that so many creatures will spawn, the server has something like a limit per Zone? or something like that. yes, I think it was cutting me off at some max number, but it still got pretty crowded. Below are some of the images.. http://imgur.com/a/dnpjZ Share this post Link to post Share on other sites
Posted March 8, 2016 We run daily morning maintenance on all of our dedicated boxes, which includes a restart. We have had no noticeable lag with 10+ concurrent users on PvP and 70k animals. Share this post Link to post Share on other sites
Posted March 8, 2016 You definitely want USE_SCHEDULED_EXECUTOR_TO_UPDATE_CREATURE_POSITION_IN_DATABASE set to true - otherwise it will perform position updates from the server main loop and it's pretty slow (especially with more creatures) 1 Share this post Link to post Share on other sites
Posted March 8, 2016 1 hour ago, bdew said: You definitely want USE_SCHEDULED_EXECUTOR_TO_UPDATE_CREATURE_POSITION_IN_DATABASE set to true - otherwise it will perform position updates from the server main loop and it's pretty slow (especially with more creatures) Thanks for the tip. I'll put that change into my server. As a large server (I don't know if there are any other 16k servers), I've noticed MASSIVE lag for about an hour after startup. I've assumed it's because of 400k-600k creatures suddenly "waking up" and wanting to do things. Are there any other contributors to massive startup lag? Share this post Link to post Share on other sites
Posted March 8, 2016 To be honest, your server is pretty unique in terms of size and creature counts. I don't think anyone (including the authors) has any idea what the exact performance implications are. A good starting point would be grabbing VisualVM or some other profiling system, attaching it to the server and looking at what it's doing. Share this post Link to post Share on other sites
Posted March 8, 2016 (edited) Short update: Restart and these changes... NUMBER_OF_DB_ITEM_DAMAGES_TO_UPDATE_EACH_TIME = 250 NUMBER_OF_DB_CREATURE_POSITIONS_TO_UPDATE_EACH_TIME = 250 instead of 500 for both took down the lag from 16+ seconds to a max of 6 seconds, which is a great performance increase. Next thing I will try to add is: USE_SCHEDULED_EXECUTOR_TO_UPDATE_CREATURE_POSITION_IN_DATABASE = true Thanks for that hint bdew. Edited March 8, 2016 by Sklo:D Share this post Link to post Share on other sites
Posted March 8, 2016 another thing to note from Lichbanes server was the tree spread causing much lag Share this post Link to post Share on other sites
Posted March 8, 2016 2 hours ago, Aum said: another thing to note from Lichbanes server was the tree spread causing much lag Yeah. The server does have massive forests, so the lag when tree spread is on is through the roof. I am considering setting the parameter to around 48 or 52, which will do a spread every 2 or 3 days. I'm thinking that SOME spread is better than none at all. But as bdew said, the server is pretty unique and noone else has really pushed the edge that far. I thank "The Three" that I couldn't go ahead with 32k map. I guess that would REALLY push the limits of what the engine can handle. Share this post Link to post Share on other sites
Posted March 9, 2016 Be careful what you tune and I highly recommend you do this on a test server. I was messing with optimization settings and corrupted the test server over a period of time. Share this post Link to post Share on other sites
Posted March 14, 2016 USE_SCHEDULED_EXECUTOR_TO_UPDATE_CREATURE_POSITION_IN_DATABASE = true This helped quite much but still not perfect yet. Share this post Link to post Share on other sites
Posted March 21, 2016 What about this value? USE_SCHEDULED_EXECUTOR_FOR_SERVER=false Does it help with lag etc to change it to true? Not sure what this one does to be honest. Share this post Link to post Share on other sites