Sign in to follow this  
Muzzy

Dumping skills without skill dump setting :)

Recommended Posts

So, I wanted to have a list of my character's skills and the normal way to do this would be using the skill dump. However, that would require logging in and I'm not feeling like doing that, so I had to do it another way. Posting my method here in case anyone finds it interesting.
 

cat ~/wurm/players/muzzy/logs/_Skills.* | perl -e 'while(<>) { /[^]]*\] (.*) increased .* to (.*)/ and do { if ($2>$skills{$1}) { $skills{$1} = $2; } }}; @ss = reverse sort {$skills{$a} <=> $skills{$b}} keys %skills; foreach my $skill (@ss) { print "$skills{$skill} $skill\n"; }'

This will print out in a nice list of skills, sorted by highest skill first. It will list the highest values the skills have ever been, so in case of lost skills it will not necessarily be the current value.
 
The output will look like this (only pasting the first few lines because the full dump is kinda long)

72.0259 Carpentry62.0083 Digging56.9122 Blacksmithing56.8352 Miscellaneous items56.548 Fighting56.4393 Pickaxe

Since there are timestamps in those logs it would be easy to make some graphs of skill progress too based on the logs, I might get into that later today if I get too bored in the evening. The existing skill graphing solutions seem to require the skill dumps, at least I couldn't find anything that would make graphs from the raw log files.

  • Like 1

Share this post


Link to post
Share on other sites

Nice one but shouldnt you post this in suggestions thread instead other then that.


 


+1


Edited by lpleroylp

Share this post


Link to post
Share on other sites

Its not entirely a suggestion really. More or less showing some brilliant skillz in coding :P

Share this post


Link to post
Share on other sites

Okay, so I got bored quite fast and decided to tinker with the graph generation. After checking out various plotting solutions, I realized I already had a python library for this purpose installed so I used python and matplotlib for the job.


 


wurm-random-skillgraph-800px.png


Trying to decide which skills to put in the graph made me realize how pointless all the skills really are. Which ones of them are important? Trying to fit all the skills in there ended up with a horrible mess so I picked a few random ones.


 


Here's the full source code to the python script I used. With hardcoded paths because I'm not writing this as a tool for other people, I just wanted some graphs to reflect on the past few months of my life.



import matplotlib.pyplot as plt
from matplotlib.dates import date2num
from glob import glob
import datetime
import re

def process_skill_log(file):
data = {};
previoustime = None
for line in [x.strip() for x in file]:
if line.startswith("Logging started"):
# Assume YYYY-mm-dd at exact offsets
day = datetime.date(int(line[16:20]), int(line[21:23]), int(line[24:26]))
m = re.match('[^]]*\] (.*) increased .* to (.*)', line)
if m:
# Assume [hh:mm:ss] at beginning of line
time = datetime.datetime.combine(day, datetime.time(int(line[1:3]), int(line[4:6]), int(line[7:9])))
# Handle timestamp wrapping
if (previoustime != None and time < previoustime):
time += datetime.timedelta(days=1)
previoustime = time

skillname = m.group(1)
skillvalue = float(m.group(2))

if not skillname in data:
data[skillname] = {'dates': [], 'values': []};
data[skillname]['dates'].append(date2num(time))
data[skillname]['values'].append(skillvalue)

return data

skilldata = {}

for x in glob('/Users/muzzy/wurm/players/muzzy/logs/_Skills.*'):
newdata = process_skill_log(open(x))
for skill in newdata:
if not skill in skilldata:
skilldata[skill] = newdata[skill]
else:
skilldata[skill]['dates'].extend(newdata[skill]['dates'])
skilldata[skill]['values'].extend(newdata[skill]['values'])

for skill in ['Carpentry', 'Fighting', 'Blacksmithing', 'Cloth tailoring', 'Leatherworking', 'Woodcutting', 'Weapon smithing']:
plt.plot_date(skilldata[skill]['dates'], skilldata[skill]['values'], '-', label=skill, xdate=True, linewidth=2)

plt.legend(loc="upper left")
plt.grid(True)

plt.show()

Share this post


Link to post
Share on other sites

I hope someone does take this and code it into something like the wurm assistant thing -- it's a neat little tool! 


Share this post


Link to post
Share on other sites

I hope someone does take this and code it into something like the wurm assistant thing -- it's a neat little tool!

Even better - Wurm Assistant mixed with Wurm Online Skillcompare. :)

  • Like 2

Share this post


Link to post
Share on other sites

Very nice commands, will be bookmarking this and using the cat command from time to time. But is it possible to let the code work with skills that have comma seperator instead of periods. All my skills are saved and showed with 10,5 not 10.5. Would be very nice if this is some easy fix.


Share this post


Link to post
Share on other sites

That graph looked awesome, if I only knew what the hell that was and how to use it :)


I guess google when I get back from work will do the trick as usual.


Share this post


Link to post
Share on other sites

Well yeah, it's going to be difficult to use for non-programmers since I'm not really releasing user friendly tools, just dumping what I made so that other programmers can take it and run with it. I should've also added a license there or something, let's just say it's public domain for now, use as you wish and no attribution required.


 


@Stimrol, the skills in your skill log have different format? Well, uh, wow. If that's really the case, then this line:



skillvalue = float(m.group(2))

needs to be changed like this:



skillvalue = float(m.group(2).replace(",","."))

I obviously only tested my script with my own log files, and I have no idea how it's going to work with other people. There might be some other surprises as well for some people, i.e. if the number formats change based on locale then the datestamps might be messed too, who knows. I'm just dropping the code here because it worked for me. I'm not really planning to write a tool that others could use or maintain the code, sorry.


Share this post


Link to post
Share on other sites

Here's an alternative shellscript, uses unix programs rather than perl so it should work in different locales (like on Stimrol's system):



cat wurm/players/Asdf/logs/_Skills.20* |  perl -ne '/[^]]*\] (.*) increased .* to (.*)/ and print "$2 $1\n";' | sort -k 2,3 -k 1nr | uniq -f 1 |sort -rn


Still uses perl for the regexp because sed's regexps are way slow :D


 


As for the python code, the way to make it work in other locales is to change the



skillvalue = float(m.group(2))

line to say this instead:



skillvalue = locale.atof(m.group(2))

and also add this at the top of the file:



import locale

(this will work on any locale, with both . and , and also others if they're used on some systems).


 


Also the python script will break if you have more than a year worth of logs, because the input files are never sorted, so it'll first add -01.txt to the graph, then -10.txt, etc, borking the graph. Need to sort the files first.


Change this line:



for x in glob('/Users/muzzy/wurm/players/muzzy/logs/_Skills.*'):

to say:



def sortfilename(filename):
    year, month = re.search("_Skills.(\d+)-(\d+)", filename).groups()
    return int(year), int(month)

for x in sorted(glob('wurm/players/Asdf/logs/_Skills.*'), key=sortfilename):

Complete code with both changes:





import matplotlib.pyplot as plt
from matplotlib.dates import date2num
from glob import glob
import datetime
import locale
import re

def process_skill_log(file):
    data = {};
    previoustime = None
    for line in [x.strip() for x in file]:
        if line.startswith("Logging started"):
            # Assume YYYY-mm-dd at exact offsets
            day = datetime.date(int(line[16:20]), int(line[21:23]), int(line[24:26]))
        m = re.match('[^]]*\] (.*) increased .* to (.*)', line)
        if m:
            # Assume [hh:mm:ss] at beginning of line
            time = datetime.datetime.combine(day, datetime.time(int(line[1:3]), int(line[4:6]), int(line[7:9])))
            # Handle timestamp wrapping
            if (previoustime != None and time < previoustime):
                time += datetime.timedelta(days=1)
            previoustime = time

            skillname = m.group(1)
            skillvalue = locale.atof(m.group(2))

            if not skillname in data:
                data[skillname] = {'dates': [], 'values': []};
            data[skillname]['dates'].append(date2num(time))
            data[skillname]['values'].append(skillvalue)

    return data

skilldata = {}

def sortfilename(filename):
    year, month = re.search("_Skills.(\d+)-(\d+)", filename).groups()
    return int(year), int(month)

for x in sorted(glob('wurm/players/Asdf/logs/_Skills.*'), key=sortfilename):
    newdata = process_skill_log(open(x))
    for skill in newdata:
        if not skill in skilldata:
            skilldata[skill] = newdata[skill]
        else:
            skilldata[skill]['dates'].extend(newdata[skill]['dates'])
            skilldata[skill]['values'].extend(newdata[skill]['values'])

for skill in ['Carpentry', 'Fighting', 'Blacksmithing', 'Cloth tailoring', 'Leatherworking', 'Woodcutting', 'Weapon smithing']:
    plt.plot_date(skilldata[skill]['dates'], skilldata[skill]['values'], '-', label=skill, xdate=True, linewidth=2)

plt.legend(loc="upper left")
plt.grid(True)

plt.show()


Edited by asdf
  • Like 1

Share this post


Link to post
Share on other sites

@asdf, Nice! I wondered about the filename sorting, but since it didn't pose a problem for me I didn't think into it too deeply.


 


Also I completely forgot about uniq -f, I totally would have done it that way myself too if only I had realized to man uniq. Your version is definitely better than my ugly perl hack :D


Share this post


Link to post
Share on other sites

Thanks both for the help, yes the files are written with dots in my local settings, becaue we use . as 1000 seperator and , as extra letters.

 

I am so thankfull for this Muzzy because I have not been able to use this wurmassistant tool because of running linux based system.

 

 

This explains why my result with the replace of , to . where all messy in the output.


Also the python script will break if you have more than a year worth of logs, because the input files are never sorted, so it'll first add -01.txt to the graph, then -10.txt, etc, borking the graph. Need to sort the files first.

Change this line:

for x in glob('/Users/muzzy/wurm/players/muzzy/logs/_Skills.*'):

Share this post


Link to post
Share on other sites

I tried again and still got a error:



Traceback (most recent call last):
  File "skill_graph.py", line 41, in <module>
    newdata = process_skill_log(open(x))
  File "skill_graph.py", line 25, in process_skill_log
    skillvalue = locale.atof(m.group(2))
  File "/usr/lib/python2.7/locale.py", line 316, in atof
    return func(string)
ValueError: invalid literal for float(): 2,000

I then changed the skillvalue line to this:



skillvalue = float(m.group(2).replace(',', '.'))

and then this works.


 


 


Thanks again :)


Edited by Stimrol

Share this post


Link to post
Share on other sites

Eh, I dunno then, probably some interesting combination of locales where wurm thinks it's in one that uses a comma, and the script thinks it should use the dot. String methods are fine :D


Share this post


Link to post
Share on other sites

Next is of course to add check marks to all the available skills so there would be no need to save the file all the time. :D


 


Feel little bad always asking for more hehe.


Share this post


Link to post
Share on other sites

@Stimrol, Instead of specifying an array of skills, just use this instead and cry in horror:



for skill in skilldata.keys():

So, check marks, eh? I suppose I'll check the documentation to see if it's easy to make an UI for that, but unless it's super easy to do I'm not going to bother with it.


  • Like 1

Share this post


Link to post
Share on other sites

It looks like UI would be too much trouble. I mean, I managed to make the checkmarks sort of fine, but there are just too many skills to fit in the screen at once and making a scrollable list of checkmarks would require more than the matplotlib widgets allows for. Embedding the whole thing inside a gtk application and making a scrollable list for skills would work, but at the same time it would also require way too much effort because it would mean losing the built-in UI with the save button etc (at least I think, didn't try embedding).


 


The second best alternative to editing the source code would be taking the list of skills from command line arguments. But I'm not going to bother making the commandline parsing either because it doesn't significantly improve the tool by itself, I'd just end up being consumed by feature creep since things like player name should be made a commandline argument too and maybe date range and perhaps the skill selection should happen automatically based on threshold (e.g. display all skills that have ended up above 50), maybe an option for logarithmic scale, etc etc etc...


 


I'll let someone else tinker with it, I'm done with the featureset unless I can think of something new that I want to see about my own data :P


  • Like 1

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Sign in to follow this