For the unfamiliar, Pypy is an alternative just in time (JIT) compiler runtime for the python language. Normally, when you run a python file, the ordinary Python runtime (cpython) interprets the file into an intermediate opcode representation, and then runs those opcodes from inside its own virtual machine. Pypy takes this a step further, turning the opcodes into native instructions and running those. JIT compilers are very popular for interpreted languages for performance and memory optimization. Without JITs like Google's V8 JavaScript engine, Javascrip/Node would not be as feasible as a server side language. Likewise, Facebook's HHVM can dramatically speed up everything from Wordpress to Magento. So what does Pypy bring to the table for speeding up Odoo?
Getting Odoo Running
There is surprisingly little that needs to change to get Odoo running on Pypy, and this is likely not a comprehensive list as my usability testing didn't include all of the apps. (mostly just looking through Sales and Website functionality)
1) a simple cast of an argument to an integer in /service/server.py @ ~line 685
# original
resource.setrlimit(resource.RLIMIT_CPU, (cpu_time + config['limit_time_cpu'], hard))
# becomes
resource.setrlimit(resource.RLIMIT_CPU, (int(cpu_time + config['limit_time_cpu']), hard))
2) Additional opcodes for Pypy in safe eval inside /tools/safe_eval.py @ ~line 66
_SAFE_OPCODES = _EXPR_OPCODES.union(set(opmap[x] for x in [
'LOAD_NAME', 'CALL_FUNCTION', 'COMPARE_OP', 'LOAD_ATTR',
'STORE_NAME', 'GET_ITER', 'FOR_ITER', 'LIST_APPEND', 'DELETE_NAME',
'JUMP_FORWARD', 'JUMP_IF_TRUE', 'JUMP_IF_FALSE', 'JUMP_ABSOLUTE',
'MAKE_FUNCTION', 'SLICE+0', 'SLICE+1', 'SLICE+2', 'SLICE+3', 'BREAK_LOOP',
'CONTINUE_LOOP', 'RAISE_VARARGS', 'YIELD_VALUE',
# New in Python 2.7 - http://bugs.python.org/issue4715 :
'JUMP_IF_FALSE_OR_POP', 'JUMP_IF_TRUE_OR_POP', 'POP_JUMP_IF_FALSE',
'POP_JUMP_IF_TRUE', 'SETUP_EXCEPT', 'END_FINALLY',
'LOAD_FAST', 'STORE_FAST', 'DELETE_FAST', 'UNPACK_SEQUENCE',
'LOAD_GLOBAL', # Only allows access to restricted globals
] if x in opmap))
# new
if 'LOOKUP_METHOD' in opmap:
_SAFE_OPCODES.add(opmap['LOOKUP_METHOD'])
_SAFE_OPCODES.add(opmap['CALL_METHOD'])
_logger = logging.getLogger(__name__)Both 1 and 2 should be safe in the core without having any real side effects in cpython.
3) replace psycop2 with psycopg2cffi
This one would be harder to maintain in the core, but it would be possible with careful imports. However the setup.py requirements might need some sort of runtime environment detection.
Additionally, I found a bug in psycopg2cffi that has since been fixed, but undoubtably these types of differences would be harder to deal with. This sort of difference may also explain some of the performance problems I experienced. (e.g. Anything relying on a list or dict as a default to be initialized only once to use as a cache.)
Want to try it out?
I have compiled up all of the above changed into a Dockerfile available on my Github account, and you can pull it directly from Docker Hub via `hibou/odoo:9.0-pypy`. For my existing projects already using Docker, this is a drop in replacement (just beware that the location of installed source code is different).
Initial Performance
All of this loads the backend and frontend (website) just fine for me, however after doing some initial benchmarks with ApacheBench I was disappointed with the performance. Odoo on Pypy is about 10% slower than on cpython. I will continue to work with it occasionally to try to spot any obvious tweaks or bugs that impair the performance I'm looking for, but for now Odoo Pypy should be considered extremely experimental.