November 11, 2011
Java dependencies

A little python snippet for creating dot-representation of the dependencies in a java app.

Let this little baby chew on your target/classes dir and you will get a dot-file on stdout.

 #!/usr/bin/env python

import re
import os
import sys
from os.path import join, basename

LIB_REG_EXP = re.compile('[,+L#][a-z][A-Za-z0-9/]+')
PACKAGE_FILTER = ''
CLASS_PREFIX_REMOVE = ''

if len(sys.argv) > 2:
    PACKAGE_FILTER = sys.argv[2]

if len(sys.argv) > 3:
    CLASS_PREFIX_REMOVE = sys.argv[3]

def get_class_dependecies(class_file_contents):
    libraries = set(map(lambda x: x.replace('/','.'), [l[1:] for l in LIB_REG_EXP.findall(class_file_contents)]))
    return filter(lambda x: PACKAGE_FILTER in x, libraries)

class_dependecies = {}
for root, dirs, files in os.walk(sys.argv[1]):
    if '.svn' in dirs:
        dirs.remove('.svn')
    class_files = [f for f in files if f.endswith('.class') and '$' not in f]
    for class_file in class_files:
        class_content = open(join(root, class_file)).read()
        class_name = join(root, class_file).replace('/','.')[len(CLASS_PREFIX_REMOVE):-6]
        if PACKAGE_FILTER not in class_name:
            continue
        class_dependecies[class_name] = []
        for dep in get_class_dependecies(class_content):
            if dep == class_name:
                continue
            class_dependecies[class_name].append(dep)

print "digraph G {"
for key, deps in class_dependecies.items():
    for dep in deps:
        print '"%s" -> "%s";' % (key, dep)
print "}"

The second argument creates a basic filter for the class-names you want. The third parameter is used for removing the folder-names (target.classes. in usual cases)

Example:

python dotclass.py target/classes us.klette.myproject target.classes. > deps.dot

Have fun

12:08am  |   URL: http://tmblr.co/Z0E3fxBlyMwk
Filed under: java python tech 
March 31, 2011
apache2-mpm-itk and libapache2-mod-wsgi

If you are running Apache 2 with mpm-itk and mod_wsgi on a debian system, you’re a bit out of luck performance wise.

From the MPM-ITK documentation:

mpm-itk is based on the traditional prefork MPM, which means it’s non-threaded; in short, this means you can run non-thread-aware code (like many PHP extensions) without problems. On the other hand, you lose out to any performance benefit you’d get with threads, of course; you’d have to decide for yourself if that’s worth it or not. You will also take an additional performance hit over prefork, since there’s an extra fork per request.

So mod_wsgi in embedded mode will make apache load your whole python project each request. That seems quite inefficient, so we’ll run mod_wsgi’s daemon mode.

mod_wsgi’s daemon mode launces background deamons that caches your code, and even have have built-in support for running under mpm-itk. The only problem is that for this support to be enables mod_wsgi must be compiled against an mpm-itk patched apache 2. In Debian this is not the case, so the daemon sockets will be owned by www-data and not the user that you assigned to the vhost.

You will see something like this in your apache error log:


[Fri Mar 18 13:11:40 2011] [error] [client 2001:700:300:9::188] (13)Permission denied: mod_wsgi (pid=22237): Unable to connect to WSGI daemon process 'myuser' on '/var/run/wsgi/wsgi.22197.0.1.sock' after multiple attempts.,

I’ve filed a bug with debian on this isses (#619252), but no responses yet.

Luckely it’s not very hard to build your own mod_wsgi package to support daemon mode under mpm-itk.

Here is a quick how-to on the matter:


$ apt-get source libapache2-mod-wsgi
$ cd mod-wsgi-3.3
$ mkdir debian/patches

$ cat - > debian/patches/series
fix-mod-wsgi-for-itk.patch
^D

$ cat - > debian/patches/fix-mod-wsgi-for-itk.patch
Remove #ifdefined for mpm-itk support, and only support mpm-itk, as we only use mpm-itk
--- a/mod_wsgi.c
+++ b/mod_wsgi.c
@@ -10037,11 +10037,7 @@
*/

 if (!geteuid()) {
-#if defined(MPM_ITK)
 if (chown(process->socket, process-&gte;uid, -1) < 0) {
-#else
-if (chown(process->socket, ap_unixd_config.user_id, -1) < 0) {
-#endif
 ap_log_error(APLOG_MARK, WSGI_LOG_ALERT(errno), wsgi_server,
"mod_wsgi (pid=%d): Couldn't change owner of unix "
"domain socket '%s'.", getpid(),
^D

$ debchange -i
$ dpkg-buildpackage

the resulting package will work as expected. I haven’t tested if it works on any non-mpm-itk enabled apache, but I wouldn’t bet on it.

Now, go forth and hack some python web apps!

Liked posts on Tumblr: More liked posts »