模拟Matlab工作空间的Python脚本
最近做Adaboost人脸检测训练,感觉为每一个功能都写一个命令行入口太麻烦,比如,三种训练算法,其Python实现的入口分别是train_1, train_2, train_3。通常需要为每一个训练算法写一个入口脚本,接受命令行参数,完成对训练算法的调用。但是这样太繁琐了,工作目录下一下子多出好多文件,而且要保持训练算法和对应入口脚本的一致性。我编写了一个从shell直接调用python函数的小工具,省去了编写入口脚本的麻烦。比如如下定义的Python函数(foo.py)
def foofunc(arg1, arg2, arg3): ... return retval
可通过如下命令调用:
$ ./script foofunc@foo arg1 arg2 arg3
为了能够向函数传递对象作为实参,引入了workspace的概念:workspace是一个路径,下边存放pickle格式的串行化对象。对于传递给函数的实参arg_n,script脚本首先检查workspace下是否存在同名文件,若存在就从该文件读取对象,若不存在,script脚本尝试把arg_n作为Python表达式来求值,并把该值作为函数实参,若求值失败,则arg_n作为字符串处理。可以通过等号操作向workspace内的变量赋值(实际上是把值保存到workspace下的文件)
./script foofunc@foo argv1 argv2 argv3 = retval
小工具由两个短文件实现:
script脚本,python函数的命令行入口
#!/usr/bin/env python import sys import cPickle as pickle from utils import workspace as ws sargv = sys.argv[1:] argv = [] outfile = '' try: eqind = sargv.index('=') argv = sargv[:eqind] outfile = sargv[eqind+1] except: argv = sargv outfile = '' re = ws.command(argv) if not outfile: sys.stdout.write(pickle.dumps(re)) else: ws.save(re, outfile)
workspace支持workspace.py
import cPickle as pickle import os from utils import tlog LOG = tlog.get(__name__) def save(value, name): with open(os.path.join('workspace',name), 'w') as f: pickle.dump(value, f) LOG.info("Saving workspace variable: %s", name) def load(name): with open(os.path.join('workspace',name)) as f: value = pickle.load(f) LOG.info("Loading workspace variable: %s",name) return value def wseval(x): if os.path.isfile(os.path.join('workspace', x)): return load(x) try: re = eval(x) LOG.info("Eval code: %s",x) return re except: LOG.info("String argument: %s",x) return x def command(argv): funname, modname = argv[0].split('@') mod = __import__(modname, fromlist=[argv[0]]) fun = eval('mod.'+funname) try: nkw = filter(lambda x:not x.startswith('--'), argv[1:]) nkw = map(wseval, nkw) except: nkw = [] try: kws = filter(lambda x: x.startswith('--'), argv[1:]) kws = map(lambda x: x.replace('--', '').split('='), kws) kws = dict( map(lambda x:tuple(x[0],wsevalx[1]), kws) ) except: kws = {} if callable(fun): LOG.info("Excuting %s",argv) re = apply(fun, nkw, kws) LOG.info("Finish excute %s", argv) return re else: return fun