Posted April 14, 2015 So, I wanted to have a list of my character's skills and the normal way to do this would be using the skill dump. However, that would require logging in and I'm not feeling like doing that, so I had to do it another way. Posting my method here in case anyone finds it interesting. cat ~/wurm/players/muzzy/logs/_Skills.* | perl -e 'while(<>) { /[^]]*\] (.*) increased .* to (.*)/ and do { if ($2>$skills{$1}) { $skills{$1} = $2; } }}; @ss = reverse sort {$skills{$a} <=> $skills{$b}} keys %skills; foreach my $skill (@ss) { print "$skills{$skill} $skill\n"; }'This will print out in a nice list of skills, sorted by highest skill first. It will list the highest values the skills have ever been, so in case of lost skills it will not necessarily be the current value. The output will look like this (only pasting the first few lines because the full dump is kinda long)72.0259 Carpentry62.0083 Digging56.9122 Blacksmithing56.8352 Miscellaneous items56.548 Fighting56.4393 PickaxeSince there are timestamps in those logs it would be easy to make some graphs of skill progress too based on the logs, I might get into that later today if I get too bored in the evening. The existing skill graphing solutions seem to require the skill dumps, at least I couldn't find anything that would make graphs from the raw log files. 1 Share this post Link to post Share on other sites
Posted April 14, 2015 (edited) Nice one but shouldnt you post this in suggestions thread instead other then that. +1 Edited April 14, 2015 by lpleroylp Share this post Link to post Share on other sites
Posted April 14, 2015 Its not entirely a suggestion really. More or less showing some brilliant skillz in coding Share this post Link to post Share on other sites
Posted April 14, 2015 Okay, so I got bored quite fast and decided to tinker with the graph generation. After checking out various plotting solutions, I realized I already had a python library for this purpose installed so I used python and matplotlib for the job. Trying to decide which skills to put in the graph made me realize how pointless all the skills really are. Which ones of them are important? Trying to fit all the skills in there ended up with a horrible mess so I picked a few random ones. Here's the full source code to the python script I used. With hardcoded paths because I'm not writing this as a tool for other people, I just wanted some graphs to reflect on the past few months of my life. import matplotlib.pyplot as plt from matplotlib.dates import date2num from glob import glob import datetime import re def process_skill_log(file): data = {}; previoustime = None for line in [x.strip() for x in file]: if line.startswith("Logging started"): # Assume YYYY-mm-dd at exact offsets day = datetime.date(int(line[16:20]), int(line[21:23]), int(line[24:26])) m = re.match('[^]]*\] (.*) increased .* to (.*)', line) if m: # Assume [hh:mm:ss] at beginning of line time = datetime.datetime.combine(day, datetime.time(int(line[1:3]), int(line[4:6]), int(line[7:9]))) # Handle timestamp wrapping if (previoustime != None and time < previoustime): time += datetime.timedelta(days=1) previoustime = time skillname = m.group(1) skillvalue = float(m.group(2)) if not skillname in data: data[skillname] = {'dates': [], 'values': []}; data[skillname]['dates'].append(date2num(time)) data[skillname]['values'].append(skillvalue) return data skilldata = {} for x in glob('/Users/muzzy/wurm/players/muzzy/logs/_Skills.*'): newdata = process_skill_log(open(x)) for skill in newdata: if not skill in skilldata: skilldata[skill] = newdata[skill] else: skilldata[skill]['dates'].extend(newdata[skill]['dates']) skilldata[skill]['values'].extend(newdata[skill]['values']) for skill in ['Carpentry', 'Fighting', 'Blacksmithing', 'Cloth tailoring', 'Leatherworking', 'Woodcutting', 'Weapon smithing']: plt.plot_date(skilldata[skill]['dates'], skilldata[skill]['values'], '-', label=skill, xdate=True, linewidth=2) plt.legend(loc="upper left") plt.grid(True) plt.show() Share this post Link to post Share on other sites
Posted April 14, 2015 I hope someone does take this and code it into something like the wurm assistant thing -- it's a neat little tool! Share this post Link to post Share on other sites
Posted April 14, 2015 I hope someone does take this and code it into something like the wurm assistant thing -- it's a neat little tool!Even better - Wurm Assistant mixed with Wurm Online Skillcompare. 2 Share this post Link to post Share on other sites
Posted April 14, 2015 Very nice commands, will be bookmarking this and using the cat command from time to time. But is it possible to let the code work with skills that have comma seperator instead of periods. All my skills are saved and showed with 10,5 not 10.5. Would be very nice if this is some easy fix. Share this post Link to post Share on other sites
Posted April 14, 2015 That graph looked awesome, if I only knew what the hell that was and how to use it I guess google when I get back from work will do the trick as usual. Share this post Link to post Share on other sites
Posted April 14, 2015 Well yeah, it's going to be difficult to use for non-programmers since I'm not really releasing user friendly tools, just dumping what I made so that other programmers can take it and run with it. I should've also added a license there or something, let's just say it's public domain for now, use as you wish and no attribution required. @Stimrol, the skills in your skill log have different format? Well, uh, wow. If that's really the case, then this line:skillvalue = float(m.group(2))needs to be changed like this:skillvalue = float(m.group(2).replace(",","."))I obviously only tested my script with my own log files, and I have no idea how it's going to work with other people. There might be some other surprises as well for some people, i.e. if the number formats change based on locale then the datestamps might be messed too, who knows. I'm just dropping the code here because it worked for me. I'm not really planning to write a tool that others could use or maintain the code, sorry. Share this post Link to post Share on other sites
Posted April 14, 2015 (edited) Here's an alternative shellscript, uses unix programs rather than perl so it should work in different locales (like on Stimrol's system):cat wurm/players/Asdf/logs/_Skills.20* | perl -ne '/[^]]*\] (.*) increased .* to (.*)/ and print "$2 $1\n";' | sort -k 2,3 -k 1nr | uniq -f 1 |sort -rnStill uses perl for the regexp because sed's regexps are way slow As for the python code, the way to make it work in other locales is to change theskillvalue = float(m.group(2))line to say this instead:skillvalue = locale.atof(m.group(2))and also add this at the top of the file:import locale(this will work on any locale, with both . and , and also others if they're used on some systems). Also the python script will break if you have more than a year worth of logs, because the input files are never sorted, so it'll first add -01.txt to the graph, then -10.txt, etc, borking the graph. Need to sort the files first.Change this line:for x in glob('/Users/muzzy/wurm/players/muzzy/logs/_Skills.*'):to say:def sortfilename(filename): year, month = re.search("_Skills.(\d+)-(\d+)", filename).groups() return int(year), int(month)for x in sorted(glob('wurm/players/Asdf/logs/_Skills.*'), key=sortfilename):Complete code with both changes:import matplotlib.pyplot as pltfrom matplotlib.dates import date2numfrom glob import globimport datetimeimport localeimport redef process_skill_log(file): data = {}; previoustime = None for line in [x.strip() for x in file]: if line.startswith("Logging started"): # Assume YYYY-mm-dd at exact offsets day = datetime.date(int(line[16:20]), int(line[21:23]), int(line[24:26])) m = re.match('[^]]*\] (.*) increased .* to (.*)', line) if m: # Assume [hh:mm:ss] at beginning of line time = datetime.datetime.combine(day, datetime.time(int(line[1:3]), int(line[4:6]), int(line[7:9]))) # Handle timestamp wrapping if (previoustime != None and time < previoustime): time += datetime.timedelta(days=1) previoustime = time skillname = m.group(1) skillvalue = locale.atof(m.group(2)) if not skillname in data: data[skillname] = {'dates': [], 'values': []}; data[skillname]['dates'].append(date2num(time)) data[skillname]['values'].append(skillvalue) return dataskilldata = {}def sortfilename(filename): year, month = re.search("_Skills.(\d+)-(\d+)", filename).groups() return int(year), int(month)for x in sorted(glob('wurm/players/Asdf/logs/_Skills.*'), key=sortfilename): newdata = process_skill_log(open(x)) for skill in newdata: if not skill in skilldata: skilldata[skill] = newdata[skill] else: skilldata[skill]['dates'].extend(newdata[skill]['dates']) skilldata[skill]['values'].extend(newdata[skill]['values'])for skill in ['Carpentry', 'Fighting', 'Blacksmithing', 'Cloth tailoring', 'Leatherworking', 'Woodcutting', 'Weapon smithing']: plt.plot_date(skilldata[skill]['dates'], skilldata[skill]['values'], '-', label=skill, xdate=True, linewidth=2)plt.legend(loc="upper left")plt.grid(True)plt.show() Edited April 14, 2015 by asdf 1 Share this post Link to post Share on other sites
Posted April 14, 2015 @asdf, Nice! I wondered about the filename sorting, but since it didn't pose a problem for me I didn't think into it too deeply. Also I completely forgot about uniq -f, I totally would have done it that way myself too if only I had realized to man uniq. Your version is definitely better than my ugly perl hack Share this post Link to post Share on other sites
Posted April 14, 2015 Thanks both for the help, yes the files are written with dots in my local settings, becaue we use . as 1000 seperator and , as extra letters. I am so thankfull for this Muzzy because I have not been able to use this wurmassistant tool because of running linux based system. This explains why my result with the replace of , to . where all messy in the output.Also the python script will break if you have more than a year worth of logs, because the input files are never sorted, so it'll first add -01.txt to the graph, then -10.txt, etc, borking the graph. Need to sort the files first.Change this line:for x in glob('/Users/muzzy/wurm/players/muzzy/logs/_Skills.*'): Share this post Link to post Share on other sites
Posted April 14, 2015 (edited) I tried again and still got a error: Traceback (most recent call last): File "skill_graph.py", line 41, in <module> newdata = process_skill_log(open(x)) File "skill_graph.py", line 25, in process_skill_log skillvalue = locale.atof(m.group(2)) File "/usr/lib/python2.7/locale.py", line 316, in atof return func(string) ValueError: invalid literal for float(): 2,000I then changed the skillvalue line to this: skillvalue = float(m.group(2).replace(',', '.'))and then this works. Thanks again Edited April 14, 2015 by Stimrol Share this post Link to post Share on other sites
Posted April 14, 2015 Eh, I dunno then, probably some interesting combination of locales where wurm thinks it's in one that uses a comma, and the script thinks it should use the dot. String methods are fine Share this post Link to post Share on other sites
Posted April 14, 2015 Locales are always massive pain Share this post Link to post Share on other sites
Posted April 14, 2015 Next is of course to add check marks to all the available skills so there would be no need to save the file all the time. Feel little bad always asking for more hehe. Share this post Link to post Share on other sites
Posted April 14, 2015 @Stimrol, Instead of specifying an array of skills, just use this instead and cry in horror: for skill in skilldata.keys():So, check marks, eh? I suppose I'll check the documentation to see if it's easy to make an UI for that, but unless it's super easy to do I'm not going to bother with it. 1 Share this post Link to post Share on other sites
Posted April 14, 2015 It looks like UI would be too much trouble. I mean, I managed to make the checkmarks sort of fine, but there are just too many skills to fit in the screen at once and making a scrollable list of checkmarks would require more than the matplotlib widgets allows for. Embedding the whole thing inside a gtk application and making a scrollable list for skills would work, but at the same time it would also require way too much effort because it would mean losing the built-in UI with the save button etc (at least I think, didn't try embedding). The second best alternative to editing the source code would be taking the list of skills from command line arguments. But I'm not going to bother making the commandline parsing either because it doesn't significantly improve the tool by itself, I'd just end up being consumed by feature creep since things like player name should be made a commandline argument too and maybe date range and perhaps the skill selection should happen automatically based on threshold (e.g. display all skills that have ended up above 50), maybe an option for logarithmic scale, etc etc etc... I'll let someone else tinker with it, I'm done with the featureset unless I can think of something new that I want to see about my own data 1 Share this post Link to post Share on other sites