Posted October 22, 2017 Hello friends We found out now, always if the servers are restarting (with a cronjob in the morning) there is a rollback with 15 minutes. Have anyone of you a solution for this odd problem? Eject Share this post Link to post Share on other sites
Posted October 22, 2017 2 hours ago, Eject said: Hello friends We found out now, always if the servers are restarting (with a cronjob in the morning) there is a rollback with 15 minutes. Have anyone of you a solution for this odd problem? Eject What is rolled back? Share this post Link to post Share on other sites
Posted October 22, 2017 32 minutes ago, Sklo:D said: What is rolled back? A rollback is when the saved data is earlier than when the server goes down, so that players in effect lose 15 minutes of the time they spent and all of their progress is rolled back by 15 minutes. Share this post Link to post Share on other sites
Posted October 22, 2017 (edited) 4 minutes ago, Brash_Endeavors said: A rollback is when the saved data is earlier than when the server goes down, so that players in effect lose 15 minutes of the time they spent and all of their progress is rolled back by 15 minutes. Complete progress is unlikely, he should check that again. Edited October 22, 2017 by Sklo:D Share this post Link to post Share on other sites
Posted October 22, 2017 Does the 15 minute roll back happen if Wurm service is just restarted by itself without server reboot? If not then turn off all services before reboot within your cron job. Wurm Unlimited and MariaDB services? I don't have a clue how WU works though. Never tried it, only WO. Assuming it uses 2 systemd services. If not then make a systemd script for WU. Systemd scripts simple to write. Share this post Link to post Share on other sites
Posted October 22, 2017 I wonder if the server is caching stuff in memory and only writing it to disk periodically. This would match the observed behavior and could be caused by a cron job that "kill -9" a process not allowing that process to properly shutdown. 1 Share this post Link to post Share on other sites
Posted October 25, 2017 On 22.10.2017 at 7:09 PM, Nappy said: I wonder if the server is caching stuff in memory and only writing it to disk periodically. This would match the observed behavior and could be caused by a cron job that "kill -9" a process not allowing that process to properly shutdown. The map is stored periodically like every 1-2 minutes, but items are stored almost instantly. So a 15 minutes rollback sounds insane. Share this post Link to post Share on other sites
Posted November 12, 2017 The databases are SQLite in WU, but we don't know how much caching the SQLite driver WU is instructing. If you're restarting via a crontab it's very important that you perform graceful restarts of your server, which as @Nappymentioned, is required to that all cached file writes (databases and binary maps) are flushed to disk correctly. If you're running them within a "screen" or "tmux" session you could send ^C (ctrl + c) to that terminal session then wait for the process to close. (This typically completes within 30 seconds.) -- This also works for Windows servers using FireDaemon, for 1 other person than me that probably does this. If however your server is not running in "screen", "tmux", FireDaemon, or any other kind of service wrapper whereby you couldn't send a text input to the process' console, your only option for a graceful shutdown would be to use an RMI tool to send the "shutdown" command. (This will work on any platform capable of running Java.) This would work similar to the ^C input method, but has the additional benefit of a custom shutdown message and time. It doesn't change that you still need to wait approximately 30 seconds (plus the specified shutdown delay) for the WU server to gracefully close down. Therefore any cron you write to do the restart would need to be able to deal with that. Share this post Link to post Share on other sites
Posted November 13, 2017 I believe the shut down in game via the GM wand is the most gracful way. Share this post Link to post Share on other sites
Posted November 14, 2017 (edited) 14 hours ago, bigsteve said: I believe the shut down in game via the GM wand is the most gracful way. That and shutdown via shutdown console command. Never do Ctrl C it will kill the server. Edited November 14, 2017 by Sklo:D Share this post Link to post Share on other sites
Posted November 14, 2017 10 hours ago, Sklo:D said: That and shutdown via shutdown console command. Never do Ctrl C it will kill the server. Unless it's different on Linux I can assure you that sending CTRL+C to the server console on Windows triggers a graceful exit. The process doesn't get killed immediately, it definitely goes through the process of flushing all pending save data to disk first. I wouldn't expect this to be different on Linux as they both use the same Java binary. Share this post Link to post Share on other sites
Posted November 15, 2017 (edited) 16 hours ago, Adambean said: Unless it's different on Linux I can assure you that sending CTRL+C to the server console on Windows triggers a graceful exit. The process doesn't get killed immediately, it definitely goes through the process of flushing all pending save data to disk first. I wouldn't expect this to be different on Linux as they both use the same Java binary. CTRL+C is killing the process it is NOT running Wurm shutdown hooks long enough on neither linux or windows. The Wurm way of handling shutdown hooks is not good enough to initiate a graceful terminate via CTRL+C only the command "shutdown" will gracefully shut down a server. You will se something like this when pressing CTRL+C on windows: Assertion Failed: Illegal termination of worker thread 'Thread(0x000000005D42B590/0x000' Assert( Assertion Failed: Illegal termination of worker thread 'Thread(0x000000005D42B590/0x000' ):threadtools.cpp:3294 NEVER use CTRL+C on any operating system, it will cause data loss. Edited November 15, 2017 by Sklo:D 1 Share this post Link to post Share on other sites
Posted November 15, 2017 6 hours ago, Sklo:D said: CTRL+C is killing the process it is NOT running Wurm shutdown hooks long enough on neither linux or windows. The Wurm way of handling shutdown hooks is not good enough to initiate a graceful terminate via CTRL+C only the command "shutdown" will gracefully shut down a server. You will se something like this when pressing CTRL+C on windows: Assertion Failed: Illegal termination of worker thread 'Thread(0x000000005D42B590/0x000' Assert( Assertion Failed: Illegal termination of worker thread 'Thread(0x000000005D42B590/0x000' ):threadtools.cpp:3294 NEVER use CTRL+C on any operating system, it will cause data loss. I don't know what you're doing to cause that. Maybe there is a difference if you don't run it in headless mode, or via the EXE. I run the JAR directly in headless mode, which when CTRL+C is picked up, shows this: blah blah blah [05:40:18 PM] INFO com.wurmonline.server.Server: Shutting down - Saving Cave Mesh [05:40:20 PM] INFO com.wurmonline.server.Server: Shutting down - Saving Resource Mesh [05:40:21 PM] INFO com.wurmonline.server.Server: Shutting down - Saving Flags Mesh [05:40:23 PM] INFO com.wurmonline.server.Server: Shutting down - Saving Constants [05:40:23 PM] INFO com.wurmonline.server.Server: Shutting down - Saving WurmID Numbers [05:40:23 PM] INFO com.wurmonline.server.Server: Shutting down - Closing Database Connections [05:40:23 PM] INFO com.wurmonline.server.DbConnector: Starting to close all Database Connections. [05:40:24 PM] INFO com.wurmonline.server.DbConnector: Finished closing all Database Connections. "Illegal termination of worker thread" sounds like the terminal you use is sending an unnecessarily heavy kill signal. Anyway, @Eject if CTRL+C does throw "Illegal termination of worker thread" and you still want to be able to shut down your server without having to be in game, ensure RMI is enabled and either use the RMI tool if you want to do it from anywhere capable of running Java or consider a web UI such as this or this. Share this post Link to post Share on other sites
Posted November 16, 2017 23 hours ago, Adambean said: I don't know what you're doing to cause that. Maybe there is a difference if you don't run it in headless mode, or via the EXE. I run the JAR directly in headless mode, which when CTRL+C is picked up, shows this: blah blah blah [05:40:18 PM] INFO com.wurmonline.server.Server: Shutting down - Saving Cave Mesh [05:40:20 PM] INFO com.wurmonline.server.Server: Shutting down - Saving Resource Mesh [05:40:21 PM] INFO com.wurmonline.server.Server: Shutting down - Saving Flags Mesh [05:40:23 PM] INFO com.wurmonline.server.Server: Shutting down - Saving Constants [05:40:23 PM] INFO com.wurmonline.server.Server: Shutting down - Saving WurmID Numbers [05:40:23 PM] INFO com.wurmonline.server.Server: Shutting down - Closing Database Connections [05:40:23 PM] INFO com.wurmonline.server.DbConnector: Starting to close all Database Connections. [05:40:24 PM] INFO com.wurmonline.server.DbConnector: Finished closing all Database Connections. "Illegal termination of worker thread" sounds like the terminal you use is sending an unnecessarily heavy kill signal. Anyway, @Eject if CTRL+C does throw "Illegal termination of worker thread" and you still want to be able to shut down your server without having to be in game, ensure RMI is enabled and either use the RMI tool if you want to do it from anywhere capable of running Java or consider a web UI such as this or this. Well it matters indeed how you are running the java application because there is a difference between killing the JVM itself or killing Wurm gracefully. A terminal session with the modlauncher will always kill the JVM not gracefully, but you are right it could work if you use the jar itself. But then you can't use the modloader, I guess. I would recommend the use of RMI tool too. Share this post Link to post Share on other sites
