Classic talk:Item Reference

Continuing script maintenance

Script failed attempting to resurrect, but I'm not sure how I feel about the item categories created.

Multiple Links to same page

It appears from here that this page links to the Legacy Items with the same information on it. I don't know if it is necessary or not. The same goes from the Legacy Items page, in that it links to itself in its list. DeadManWalking (talk) 17:06, 19 July 2017 (EDT)

Archives

Archives of outdated information from this talk page are here:

• Archive #1

Problems

itemdbtowiki does not take enough arguments

In the future, itemdbtowiki should be modified to take some extra arguments from the command line, such as 'image_path', 'checkimagesonwiki', and perhaps the username and password.

[FIXED] Cannot run script

Output:

Item-ID -1: Cannot convert integer attribute type to an integer. Value: 'hairsprite'
.
.
.
Item-ID 1213: Cannot convert integer attribute type to an integer. Value: 'generic'
Traceback (most recent call last):
  File "./itemdbtowiki.py", line 531, in <module>
    citems = parsexmlitems(f);
  File "./itemdbtowiki.py", line 252, in parsexmlitems
    items[curitem['id']] = curitem
KeyError: 'id'

I have tried to understand the script so I can solve it myself, but I do not really understand the logic of what the code is trying to do or the logic of python’s error message. It seems I need to learn more scripting and python as well. ✎ Kess^☽ 10:48, 7 November 2008 (CET)

I have solved this part, just needed to take the time to understand the script and update/fix the relevant parts. I will update the item pages in a few days time when I have ironed out some ‘small’ issues like dyed items and rarity. At that time I think I’ll make another archive of this talk page as well. ✎ Kess^☽ 19:50, 10 November 2008 (CET)

I do not have this time currently. The latest state of the script is available here: User talk:Kess/Sandbox, with output as shown here: User:Kess/Sandbox. Feel free to use it or skip it. I might get back to this next year. Depends on where my priorities will lie. ✎ Kess^☽ 15:26, 18 November 2008 (CET)

How to create this page

I have completed the new new script to recreate this page, please leave me a note if you find any problems. It is now capable of uploading new images to the wiki! --Hype0 02:42, 3 September 2007 (CEST)

You can run itemdbtowiki to generate the wiki page. The usage is:

itemdbtowiki <item_db.txt> <items.xml> > wikipage.txt

If run without any arguments, it'll look for item_db.txt and items.xml in the current directory.

To upload new images to the wiki automatically, 'checkimagesonwiki' in the file to 1 and set a corresponding valid 'wikiuser' and 'wikipassword' as well as set the 'image_path' to be correct (these are all parameters at the start of the script.)

Warning: if you choose to upload images, authentication will be done in plaintext, which is exactly what happens if you log in over http:// anyway... just thought you might like a reminder.

Python-Tools

itemdbtowiki

#!/usr/bin/python
#Licensed under GNU General Public License

import sys, os, popen2;
import xml.parsers.expat

debug=0
checkimagesonwiki=0
#wikiuser="Hype0"
#wikipassword="password"
image_path="/usr/home/dex/src/themanaworld/tmw/trunk/data/graphics/items"

health_titles = ["Image","Name","ID","HP Bonus","SP Bonus",
                 "Price<BR />BUY/Sell","Weight","Description"]
status_titles = ["Image","Name","ID","Spell name","Parameter 1","Parameter 2",
                 "Price<BR />BUY/Sell","Weight","Description"]
weapon_titles = [" Image","Name","ID","Damage<BR />(Range)",
                 "Price<BR />BUY/Sell","Weight","Description"]
armor_titles = ["Image", "Name", "ID", "Defense", 
              "Price<BR />BUY/Sell", "Weight","Description"]
other_titles = ["Image","Name","ID","Price<BR />BUY/Sell",
                "Weight","Description "]


imageurls = [
# The index represents the imageID from the items.xml of the client.
    "",  #    0
    "http://img52.imageshack.us/img52/3072/423xc.png",         #   42
    "http://img187.imageshack.us/img187/2963/434dc.png",       #   43
    "http://img125.imageshack.us/img125/4331/442fm.png",       #   44
    "http://img260.imageshack.us/img260/6627/456rj.png",       #   45
    "http://img211.imageshack.us/img211/9089/469fy.png",       #   46
    "http://img219.imageshack.us/img219/1188/479yz.png",       #   47
    "http://img232.imageshack.us/img232/3517/489oa.png",       #   48
    "http://img291.imageshack.us/img291/8643/491eg.png",       #   49
    "http://img157.imageshack.us/img157/1425/502hg.png",       #   50
    "http://img293.imageshack.us/img293/4524/514dn.png",       #   51
    "http://img42.imageshack.us/img42/7271/523hc.png",         #   52
    "http://img187.imageshack.us/img187/5241/535fk.png",       #   53
    "http://img125.imageshack.us/img125/8093/543yr.png",       #   54
    "http://img260.imageshack.us/img260/1315/554bt.png",       #   55
    "http://img211.imageshack.us/img211/9687/564wu.png",       #   56
    "http://img219.imageshack.us/img219/6842/571wz.png",       #   57
    "http://img232.imageshack.us/img232/9236/582ue.png",       #   58
    "http://img291.imageshack.us/img291/435/598ni.png",        #   59
    "http://img157.imageshack.us/img157/1323/605pt.png",       #   60
    "http://img293.imageshack.us/img293/1371/618fu.png",       #   61
    "http://img42.imageshack.us/img42/69/625mj.png",           #   62
    "http://img187.imageshack.us/img187/6240/637ju.png",       #   63
    "http://img275.imageshack.us/img275/9440/648zq.png",       #   64
]

imagesused = set()

class whatever: pass

log = []


# parseitems(file)
## Returns list with items from eathena item_db file.

def saveint(string, altval = 0):
    a = 0
    try:
        a = int(string)
    except:
        a = altval
    return a
        
def parsescript(s):
    # Assumes that there's only one call of each method, otherwise it would need to know
    # how to combine those function calls. In practice, the latter call would prevail.
    script = {}
    scriptentry = ""
    parentry = ""
    mode = 0
    for a in s:
        if mode == 0: # looking for method
            if a.isalpha(): 
                mode = 1
                scriptentry += a
            elif a == '}': mode = 9
        elif mode == 1: # reading method name
            if a in " ;}":
                if a == " ": mode = 2
                elif a == ";": mode = 1
                elif a == "}": mode = 9
                parentry = ""
                script[scriptentry] = []
            else: scriptentry += a
        elif mode == 2: #looking for param
            if a == " ": pass
            elif a == ";": 
                mode = 0
                scriptentry = ""
            else:
                mode = 3
                parentry = a
        elif mode == 3: #reading param
            if a in (" ", ",", ";"):
                script[scriptentry].append(parentry)
                parentry = ""
                if a == ';': 
                    mode = 0
                    scriptentry = ""
                else: mode = 2
            else: 
                parentry += a
        elif mode == 9: #finished
            pass

    # Convert all possible parameters to integers
    for i in script.keys():
        for j in range(len(script[i])):
            try:
                script[i][j] = int(script[i][j])
            except:
                #print script[i][j]
                pass
    return script
    

def parseitems(file):
    objects = []
    for line in file:
        s = line[:line.find('//')].strip().replace('\t','')
        if s:
            #Replace commas inbetween {} with |, so we can use split
            mode = 0
            sout = ""
            for a in s:
                if mode == 0: #Out of {}
                    if a == '{': mode = 1
                    sout += a
                elif mode == 1: #Inside {}
                    if a == ',': sout += '|'
                    else:
                        sout += a
                        if a == '}': mode = 0

            values = sout.split(',')
            if line[0] == '#':
                if debug:
                    log.append("FOUND COMMENT LINE: %s" % str(values))
                continue
            if len(values) != 20:
                log.append("item_db: Warning, item-line with ID %s has %d values instead of 19" % (values[0], len(values)))
                if debug:
                    log.append("  line was %s" % str(values))
                while len(values) < 20:
                    values.append('')
                while len(values) > 20:
                    values.pop()
            o = whatever()
            o.id        = int(values[0])
            o.name      = values[1]
            o.jname     = values[2]
            o.type      = saveint(values[3])
            o.price     = saveint(values[4])
            o.sell      = saveint(values[5])
            o.weight    = saveint(values[6])
            o.atk       = saveint(values[7])
            o.defense   = saveint(values[8])
            o.range     = saveint(values[9])
            o.mbonus    = saveint(values[10])
            o.slot      = saveint(values[11],-1)
            o.job       = saveint(values[12],-1)
            o.gender    = saveint(values[13],-1)
            o.loc       = saveint(values[14],-1)
            o.wlv       = saveint(values[15])
            o.elv       = saveint(values[16])
            o.view      = saveint(values[17],-1)
            o.usescript = parsescript(values[18].replace('|',','))
            o.equipscript = parsescript(values[19].replace('|',','))


            objects.append(o)

    return objects


# parsexmlitems(file)
## Creates a dictionary containing the values of a client items.xml
## Yeah, there are XML parsers in the standard python libraries, but they're too object
## oriented and thus don't fit the style of this program.
def parsexmlitems(file):
    items = {}
    pre = "<item "
    term = "/>"
    attrs = ["id", "image", "art", "name", "description", "type", "weight", "slot"]
    intattrs = ["id", "art", "type", "weight", "slot"]
    s = file.read()
    index = 0
    debug = 0
    while pre in s[index:]:
        index += s[index:].find(pre) + len(pre)
        curitem = {}
        termstart = index + s[index:].find(term) + len(term)

        for attr in attrs:
            found = s[index:].find(attr+'="')
            if found >= 0:
                start = index + found + len(attr+'="')
                end= start + s[start:].find('"')
            else:
                start = termstart

            if start < termstart:
                curitem[attr] = s[start:end]

        for a in intattrs:
            try:
                if a in curitem: curitem[a] = int(curitem[a])
            except:
                log.append("Item-ID %s: Cannot convert integer attribute %s to an integer. Value: '%s'" % (curitem["id"], a, curitem[a]))

        items[curitem['id']] = curitem


    return items
        

# addclientinformation(items, citems)
## Entends the item data with the data collected from the client items.xml. Adding imageurls,
## client-name and -description

def addclientinformation(items,citems):
    for i in items:
        if i.id in citems:
            i.imagename=citems[i.id]["image"]
            url=i.imagename[0].upper() + i.imagename[1:]
            i.imgurl = "[[Image:"+ url + "]]"
            imagesused.add( citems[i.id]["image"])
            i.description = citems[i.id]["description"]
            i.clientname = citems[i.id]["name"]
        else:
            i.imgurl = ''
            i.description = ''
            i.clientname = ''



# gettypedir (items)
## Returns sorted lists of items by itemtype
def gettypedir(items):
    items.sort(key=lambda x: x.price+x.sell)

    typedir = whatever()
    typedir.healthy = []
    typedir.status = []
    typedir.inspiring = []
    typedir.weapons = []
    typedir.combos = []
    typedir.armor = []
    typedir.other = []
    for item in items:
        if item.imgurl.strip() or item.clientname.strip():
            #if item.id == 537: log.append('"%s", "%s"' % (item.imgurl, item.name)
            if item.atk > 0:
                if item.defense == 0: typedir.weapons.append(item)
                else: typedir.combos.append(item)
            elif item.defense > 0: typedir.armor.append(item)
            elif "itemheal" in item.usescript:
                if item.usescript["itemheal"][0] > item.usescript["itemheal"][1]:
                    typedir.healthy.append(item)
                else:
                    typedir.inspiring.append(item)
            elif "sc_start" in item.usescript:
                typedir.status.append(item)
            else: typedir.other.append(item)

    typedir.weapons.sort(key=lambda x: x.atk)
    typedir.armor.sort(key=lambda x: x.defense)
    typedir.combos.sort(key=lambda x: x.defense+x.atk)

    typedir.healthy.sort(key=lambda x: int(x.usescript["itemheal"][0]))
    typedir.inspiring.sort(key=lambda x: int(x.usescript["itemheal"][1]))
    #typedir.other.sort(key=lambda x: x.price+x.sell)
    typedir.status.sort(key=lambda x: x.price+x.sell)

    return typedir
    

            
# printlog()
## Prints the global variable log to stdout
def printlog():
    global log
    if len(log) > 0:
        print '\n---------------------------------------'
    for line in log:
        print line


# print<>items(items, title)
## Creates the table in wikicode, depending on what kind of item is being printed

def getmoneystring(buy, sell):
    return '| align="right" | %d GP<br>%d gp\n' % (buy,sell)
def getidstring(id):
    return '| align="center" | [%d]\n' % id
    
def printhealitems(items,title):
    print '==%s==' % title
    print '{| border="1" cellspacing="0" cellpadding="5" width="100%" align="center"'
    # Print labels
    for title in health_titles:
        print '! style="background:#efdead;" | %s' % title
        
    for i in items:
        print '|-'
        print '| align="center" | %s' % i.imgurl
        #print '| %s' % i.jname.replace('_',' ')
        print '| %s' % i.clientname
        sys.stdout.write( getidstring(i.id) )
        print '| align="center" | %d' % i.usescript["itemheal"][0]
        print '| align="center" | %d' % i.usescript["itemheal"][1]
        sys.stdout.write( getmoneystring(i.price,i.sell) )
        print '| align="center!" | %d' % i.weight
        print '| %s' % i.description
    print '|}\n'
            
def printstatusitems(items,title):
    print '==%s==\n' % title)
    print '{| border="1" cellspacing="0" cellpadding="5" width="100%" align="center"\n')
    # Print labels
    for title in status_titles:
        print '! style="background:#efdead;" | %s' % title
        
    for i in items:
        print '|-'
        print '| align="center" | %s' % i.imgurl
        #print '| %s' % i.jname.replace('_',' ')
        print '| %s' % i.clientname
        sys.stdout.write( getidstring(i.id) )
        print '| align="center" | %s' % i.usescript["sc_start"][0]
        print '| align="center" | %s' % i.usescript["sc_start"][1]
        print '| align="center" | %s' % i.usescript["sc_start"][2]
        sys.stdout.write( getmoneystring(i.price,i.sell) )
        print '| align="center!" | %d' % i.weight
        print '| %s' % i.description
    print '|}\n'
            

def printweaponitems(items, title):
    print '==%s==\n' % title)
    print '{| border="1" cellspacing="0" cellpadding="5" width="100%" align="center"\n')
    # Print labels
    for title in weapon_titles:
        print '! style="background:#efdead;" | %s\n' % title)

    for i in items:
        print '|-'
        print '| align="center" | %s' % i.imgurl
        #print '| %s' % i.jname.replace('_',' ')
        print '| %s' % i.clientname
        sys.stdout.write( getidstring(i.id) )
        print '| align="center" | %d (%d)' % (i.atk,i.range)
        sys.stdout.write( getmoneystring(i.price,i.sell) )
        print '| align="center!" | %d' % i.weight
        print '| %s' % i.description
    print '|}\n'
    
def printarmoritems(items, title):
    print '==%s==' % title
    print '{| border="1" cellspacing="0" cellpadding="5" width="100%" align="center"'
    print '! style="background:#efdead;" | Image'
    print '! style="background:#efdead;" | Name'
    print '! style="background:#efdead;" | ID'
    print '! style="background:#efdead;" | Defense'
    print '! style="background:#efdead;" | Price<br>BUY/Sell'
    print '! style="background:#efdead;" | Description'
    for i in items:
        print '|-'
        print '| align="center" | %s' % i.imgurl
        #print '| %s' % i.jname.replace('_',' ')
        print '| %s' % i.clientname
        sys.stdout.write( getidstring(i.id) )
        print '| align="center" | %d' % i.defense
        sys.stdout.write( getmoneystring(i.price,i.sell) )
        print '| %s' % i.description
    print '|}\n'

def printcomboitems(items, title):
    print '==%s==' % title
    print '{| border="1" cellspacing="0" cellpadding="5" width="100%" align="center"'
    print '! style="background:#efdead;" | Image'
    print '! style="background:#efdead;" | Name'
    print '! style="background:#efdead;" | ID'
    print '! style="background:#efdead;" | Damage<br>(Range)'
    print '! style="background:#efdead;" | Defense'
    print '! style="background:#efdead;" | Price<br>BUY/Sell'
    print '! style="background:#efdead;" | Description'
    for i in items:
        print '|-'
        print '| align="center" | %s' % i.imgurl
        #print '| %s' % i.jname.replace('_',' ')
        print '| %s' % i.clientname
        sys.stdout.write( getidstring(i.id) )
        print '| align="center" | %d (%d)' % (i.atk,i.range)
        print '| align="center" | %d' % i.defense
        sys.stdout.write( getmoneystring(i.price,i.sell) )
        print '| %s' % i.description
    print '|}\n'

def getpropertystring(item):
    s = ""
    s += "Weight: %d, " % item.weight
    s += "Slot: %d, " % item.slot
    s += "Job: %d, " % item.job
    s += "Gender: %d, " % item.gender
    s += "Loc: %d, " % item.loc
    s += "wLV: %d, " % item.wlv
    s += "eLV: %d, " % item.wlv
    s += "View: %d " % item.view
    return s


def printotheritems(items, title):
    print '==%s==' % title
    print '{| border="1" cellspacing="0" cellpadding="5" width="100%" align="center"'
    print '! style="background:#efdead;" | Image'
    print '! style="background:#efdead;" | Name'
    print '! style="background:#efdead;" | ID'
    #print '! style="background:#efdead;" | Type'
    #print '! style="background:#efdead;" | Properties'
    print '! style="background:#efdead;" | Price<br>BUY/Sell'
    print '! style="background:#efdead;" | Description'
    for i in items:
        print '|-'
        print '| align="center" | %s' % i.imgurl
        #print '| %s' % i.jname.replace('_',' ')
        print '| %s' % i.clientname
        sys.stdout.write( getidstring(i.id) )
        #print '| align="center" | %d' % i.type
        #print '| align="center" | %s' % getpropertystring(i)
        sys.stdout.write( getmoneystring(i.price,i.sell) )
        print '| %s' % i.description
    print '|}\n'



def printunuseditems(title):
    ids = []
    for i in range(1,len(imageurls)):
        if i not in imagesused:
            ids.append(i)
    if len(ids):
        print '==%s==' % title
        print '{| border="1" cellspacing="0" cellpadding="5" width="100%" align="center"'
        print '|',
        for i in ids:
            print imageurls[i],
        print '\n|}\n'

    

#####################################################################
#  MAIN
#####################################################################

try:
    if len(sys.argv) == 1:
        item_db = "item_db.txt"
        item_xml = "items.xml"
    elif len(sys.argv) == 3:
        item_db = sys.argv[1]
        item_xml = sys.argv[2]
    else: 
        item_db = ''
        item_xml = ''
        print "Wrong number of arguments"

    if item_db and not os.path.isfile(item_db):
        print "File does not exist: %s" % item_db
        item_db = ''
    if item_xml and not os.path.isfile(item_xml):
        print "File does not exist: %s" % item_xml
        item_db = ''
    
    if not (item_db and item_xml):
        print "\nUSAGE:"
        print "dbtowiki without any arguments will use item_db.txt and items.xml in the current directory."
        print "to specify custom files, call: dbtowiki <item_db> <item_xml>"
        exit(-1);
    else:
        log.append("Item-list [item_db] = %s" % item_db)
        log.append("Item-list [item_xml] = %s" % item_xml)
        f = open(item_db)
        items = parseitems(f);
        f = open(item_xml)
        citems = parsexmlitems(f);

        addclientinformation(items, citems)

        typedir = gettypedir(items)
        if len(typedir.healthy) > 0: printhealitems(typedir.healthy, "Health")
        if len(typedir.status) > 0: printstatusitems(typedir.status, "Status")
        if len(typedir.inspiring) > 0: printhealitems(typedir.inspiring, "Mana")
        if len(typedir.weapons) > 0: printweaponitems(typedir.weapons, "Weapons")
        if len(typedir.armor) > 0: printarmoritems(typedir.armor, "Armor")
        if len(typedir.combos) > 0: printcomboitems(typedir.combos, "Combos")
        if len(typedir.other) > 0: printotheritems(typedir.other, "Other")

        if checkimagesonwiki:
            print "Checking for images on wiki: "
            images = []
            for item in typedir.healthy:
                images.append( (item.imagename, item.description))
            for item in typedir.inspiring:
                images.append( (item.imagename, item.description))
            for item in typedir.weapons:
                images.append( (item.imagename, item.description))
            for item in typedir.armor:
                images.append( (item.imagename, item.description)) 
            for item in typedir.combos:
                images.append( (item.imagename, item.description))
            for item in typedir.other:
                images.append( (item.imagename, item.description))

            need_to_upload=[]

            for imagename, description in images:
                fixedimagename = imagename[0].upper() + imagename[1:]
                curlstring='curl "http://wiki.themanaworld.org/index.php/' \
                    'Image:%s"' % fixedimagename
                if debug:
                    print "CHECKING URL: %s" % curlstring
                p=popen2.Popen4(curlstring) 
                # this could hang if there is a lot of output!
                returncode = p.wait() # wait for the process to finish
                if debug:
                    print "Process finished with %d" % returncode
                page = p.fromchild.read()
                
                if debug >= 3:
                    print "Read: %s" % page

                if "No file by this name exists" in page:
                    need_to_upload.append((imagename, description))
                    if debug:
                        print "NEED TO UPLOAD: %s" % need_to_upload[-1]
            
            

            if len(need_to_upload):
                curlstring = 'curl -d "wpName=%s&wpPassword=%s&wpLoginattempt=Log+in" -c cookie-jar "http://wiki.themanaworld.org/index.php?title=Special:Userlogin&action=submitlogin&type=login"'% (wikiuser, wikipassword)

                print "Logging in as %s, your password can be seen " \
                                 "in `ps ax`, also this is using http://... " \
                                 "so be warned!" % wikiuser
                p=popen2.Popen4(curlstring) 
                # this could hang if there is a lot of output!
                returncode = p.wait() # wait for the process to finish
                data = p.fromchild.read()
                if debug >= 1:
                    print "Read: %s" % data

                if "Login error" in data:
                    print "Login failed"

                if debug:
                    print "Process finished with %d" % returncode

                sys.stdout.write("Now it is time to upload: %s" % 
                                 need_to_upload)

                for imagename, description in need_to_upload:
                    if debug:
                        print "Uploading image: %s" % imagename

                    curlstring='curl -b cookie-jar -F "wpUploadFile=@%s/%s" -F "filename=%s" -F "wpDestFile=%s" -F "wpUploadDescription=%s" -F "wpUpload=Upload file" "http://wiki.themanaworld.org/index.php?title=Special:Upload"' % (image_path, imagename, imagename, imagename, description)
                    p=popen2.Popen4(curlstring) 
                    # this could hang if there is a lot of output!
                    returncode = p.wait() # wait for the process to finish
                    data = p.fromchild.read()
                    if debug >= 1:
                        print "Read: %s" % data

                    if debug:
                        print "Process finished with %d" % returncode

                    print "Uploaded: %s, %s" % (imagename, description))
                    

        printunuseditems("Still unknown")

        print "\n"

finally:
    printlog()