Nginx and Metalog

Metalog is an easily configurable system logger daemon which can be substituted for standard syslogd and klogd. It has one limitation though, that you cannot log to remote machines. It’s very easy to configure Nginx to use Syslog (Metalog, in my case).

All you need to do is compile nginx with this syslog plugin. If you’re on Gentoo, you can compile nginx with the syslog useflag.
Configure nginx as follows:

# global section

syslog local0;
error_log syslog;

# http

http {
   access_log syslog:info <format>;
}

And then, metalog configuration:

Nginx Access:
    facility = "local0"
    logdir = "/var/log/nginx/access"    
    minimum = 6
    break = 1

Nginx Errors:
    facility = "local0"
    logdir = "/var/log/nginx/errors"    
    maximum = 4
    break = 1

This configuration must be placed before catch-all rule otherwise, you’ll have all errors and access logs in your /var/log/everything/current which is quite disturbing.

With this configuration, you can find nginx logs at /var/log/nginx/errors/current  and /var/log/nginx/access/current .

The advantage? Metalog performs automatic log rotation. Yes, I know I can configure a cron job for that, but I prefer this.

A handy timezone converter for all Linux users

Linux has a in built timezone converter and I have been using various web services ever since.

Timezone is controlled by the TZ environment variable and you can set it to the desired timezone before executing a command, which will make the command think that you are in that particular timezone.

Here’s how to use it:

$ date
Fri Mar  1 22:55:56 IST 2013
$ TZ=UTC date
Fri Mar  1 17:26:02 UTC 2013
$ TZ=PST date
Fri Mar  1 17:26:08 PST 2013
$ TZ=CET date
Fri Mar  1 18:26:11 CET 2013

Accidental discovery at it’s best 😀

Configuring Roboto font in Linux

Roboto font comes from Android and Google designed it really well. It looks awesome on desktops and laptops. But getting it working as a default system font in Linux is quite a task because the font package of Roboto available has multiple font forms declaring same type of text style.

The standard Roboto package consists of following fonts:

  • Roboto-BlackItalic
  • Roboto-Black
  • Roboto-BoldCondensedItalic
  • Roboto-BoldCondensed
  • Roboto-BoldItalic
  • Roboto-Bold
  • Roboto-CondensedItalic
  • Roboto-Condensed
  • Roboto-Italic
  • Roboto-LightItalic
  • Roboto-Light
  • Roboto-MediumItalic
  • Roboto-Medium
  • Roboto-Regular
  • Roboto-ThinItalic
  • Roboto-Thin

Now the problem is, Roboto Black, Roboto Regular and Roboto Medium, are selected as candidates for the standard font. See the output below:

$ fc-query /usr/share/fonts/roboto/Roboto-{Black,Regular,Medium}.ttf
Pattern has 20 elts (size 32)
        family: "Roboto"(s) "Roboto Bk"(s)
        familylang: "en"(s) "en"(s)
        style: "Black"(s) "Bold"(s)
        stylelang: "en"(s) "en"(s)
        fullname: "Roboto Black"(s)
        fullnamelang: "en"(s)
        slant: 0(i)(s)
        weight: 80(i)(s)
        width: 100(i)(s)
        foundry: "unknown"(s)
        file: "/usr/share/fonts/roboto/Roboto-Black.ttf"(s)
        index: 0(i)(s)
        outline: FcTrue(s)
        scalable: FcTrue(s)
        charset: 
        0000: 00000000 ffffffff ffffffff 7fffffff 00000000 ffffffff ffffffff ffffffff
        0001: ffffffff ffffffff ffffffff ffffffff 00040000 00018003 00000000 fc010000
        0002: 03000000 00800000 00000000 00000000 00000000 10000000 3f0002c0 00080000
        0003: 0000820b 00000008 00000000 00000000 ffffd7f0 fffffffb 00467fff 00000000
        0004: ffffffff ffffffff ffffffff ffffffff ffffff7f ffffffff ffffffff ffffffff
        0005: 000fffff 00000000 00000000 00000000 00000000 00000000 00000000 00000000
        001e: 00000003 c0000000 00000000 00000000 0000003f ffffffff ffffffff 03ffffff
        001f: 00000000 00000000 00002000 00000000 00000000 00000000 00000000 00000000
        0020: 7fb80fff 160d0067 00000010 80100000 00000000 00001898 00000000 00000000
        0021: 00480020 00004044 78000000 00000000 00000000 00000000 00000000 00000000
        0022: 44068044 00000800 00000100 00000031 00000000 00000000 00000000 00000000
        0025: 00000000 00000000 00000000 00000000 00000000 00000000 00000400 00000000
        00f6: 00000000 00000000 00000000 00000000 00000000 00000000 00000008 00000000
        00fb: 0000001e 00000000 00000000 00000000 00000000 00000000 00000000 00000000
        00fe: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 80000000
        00ff: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 30000000
(s)
        lang: aa|ab|af|av|ay|ba|be|bg|bi|bin|br|bs|bua|ca|ce|ch|chm|co|cs|cu|cv|cy|da|de|el|en|eo|es|et|eu|fi|fj|fo|fr|fur|fy|gd|gl|gn|gv|ho|hr|hu|ia|ig|id|ie|ik|io|is|it|kaa|ki|kk|kl|kum|kv|ky|la|lb|lez|lt|lv|mg|mh|mk|mt|nb|nds|nl|nn|no|nr|nso|ny|oc|om|os|pl|pt|rm|ru|sah|se|sel|sh|sk|sl|sma|smj|smn|so|sq|sr|ss|st|sv|sw|tg|tk|tl|tn|tr|ts|tt|tyv|uk|uz|vo|vot|wa|wen|wo|xh|yap|zu|an|crh|csb|fil|hsb|ht|jv|kj|ku-tr|kwm|lg|li|mn-mn|ms|na|ng|pap-an|pap-aw|rn|rw|sc|sg|sn|su|ty|za(s)
        fontversion: 65536(i)(s)
        capability: "otlayout:DFLT"(s)
        fontformat: "TrueType"(s)
        decorative: FcFalse(s)

Pattern has 20 elts (size 32)
        family: "Roboto"(s)
        familylang: "en"(s)
        style: "Regular"(s)
        stylelang: "en"(s)
        fullname: "Roboto Regular"(s)
        fullnamelang: "en"(s)
        slant: 0(i)(s)
        weight: 80(i)(s)
        width: 100(i)(s)
        foundry: "unknown"(s)
        file: "/usr/share/fonts/roboto/Roboto-Regular.ttf"(s)
        index: 0(i)(s)
        outline: FcTrue(s)
        scalable: FcTrue(s)
        charset: 
        0000: 00000000 ffffffff ffffffff 7fffffff 00000000 ffffffff ffffffff ffffffff
        0001: ffffffff ffffffff ffffffff ffffffff 00040000 00018003 00000000 fc010000
        0002: 03000000 00800000 00000000 00000000 00000000 10000000 3f0002c0 00080000
        0003: 0000820b 00000008 00000000 00000000 ffffd7f0 fffffffb 00467fff 00000000
        0004: ffffffff ffffffff ffffffff ffffffff ffffff7f ffffffff ffffffff ffffffff
        0005: 000fffff 00000000 00000000 00000000 00000000 00000000 00000000 00000000
        001e: 00000003 c0000000 00000000 00000000 0000003f ffffffff ffffffff 03ffffff
        001f: 00000000 00000000 00002000 00000000 00000000 00000000 00000000 00000000
        0020: 7fb80fff 160d0067 00000010 80100000 00000000 00001898 00000000 00000000
        0021: 00480020 00004044 78000000 00000000 00000000 00000000 00000000 00000000
        0022: 44068044 00000800 00000100 00000031 00000000 00000000 00000000 00000000
        0025: 00000000 00000000 00000000 00000000 00000000 00000000 00000400 00000000
        00f6: 00000000 00000000 00000000 00000000 00000000 00000000 00000008 00000000
        00fb: 0000001e 00000000 00000000 00000000 00000000 00000000 00000000 00000000
        00fe: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 80000000
        00ff: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 30000000
(s)
        lang: aa|ab|af|av|ay|ba|be|bg|bi|bin|br|bs|bua|ca|ce|ch|chm|co|cs|cu|cv|cy|da|de|el|en|eo|es|et|eu|fi|fj|fo|fr|fur|fy|gd|gl|gn|gv|ho|hr|hu|ia|ig|id|ie|ik|io|is|it|kaa|ki|kk|kl|kum|kv|ky|la|lb|lez|lt|lv|mg|mh|mk|mt|nb|nds|nl|nn|no|nr|nso|ny|oc|om|os|pl|pt|rm|ru|sah|se|sel|sh|sk|sl|sma|smj|smn|so|sq|sr|ss|st|sv|sw|tg|tk|tl|tn|tr|ts|tt|tyv|uk|uz|vo|vot|wa|wen|wo|xh|yap|zu|an|crh|csb|fil|hsb|ht|jv|kj|ku-tr|kwm|lg|li|mn-mn|ms|na|ng|pap-an|pap-aw|rn|rw|sc|sg|sn|su|ty|za(s)
        fontversion: 65536(i)(s)
        capability: "otlayout:DFLT"(s)
        fontformat: "TrueType"(s)
        decorative: FcFalse(s)

Pattern has 20 elts (size 32)
        family: "Roboto"(s) "Roboto Lt"(s)
        familylang: "en"(s) "en"(s)
        style: "Medium"(s) "Regular"(s)
        stylelang: "en"(s) "en"(s)
        fullname: "Roboto Medium"(s)
        fullnamelang: "en"(s)
        slant: 0(i)(s)
        weight: 100(i)(s)
        width: 100(i)(s)
        foundry: "unknown"(s)
        file: "/usr/share/fonts/roboto/Roboto-Medium.ttf"(s)
        index: 0(i)(s)
        outline: FcTrue(s)
        scalable: FcTrue(s)
        charset: 
        0000: 00000000 ffffffff ffffffff 7fffffff 00000000 ffffffff ffffffff ffffffff
        0001: ffffffff ffffffff ffffffff ffffffff 00040000 00018003 00000000 fc010000
        0002: 03000000 00800000 00000000 00000000 00000000 10000000 3f0002c0 00080000
        0003: 0000820b 00000008 00000000 00000000 ffffd7f0 fffffffb 00467fff 00000000
        0004: ffffffff ffffffff ffffffff ffffffff ffffff7f ffffffff ffffffff ffffffff
        0005: 000fffff 00000000 00000000 00000000 00000000 00000000 00000000 00000000
        001e: 00000003 c0000000 00000000 00000000 0000003f ffffffff ffffffff 03ffffff
        001f: 00000000 00000000 00002000 00000000 00000000 00000000 00000000 00000000
        0020: 7fb80fff 160d0067 00000010 80100000 00000000 00001898 00000000 00000000
        0021: 00480020 00004044 78000000 00000000 00000000 00000000 00000000 00000000
        0022: 44068044 00000800 00000100 00000031 00000000 00000000 00000000 00000000
        0025: 00000000 00000000 00000000 00000000 00000000 00000000 00000400 00000000
        00f6: 00000000 00000000 00000000 00000000 00000000 00000000 00000008 00000000
        00fb: 0000001e 00000000 00000000 00000000 00000000 00000000 00000000 00000000
        00fe: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 80000000
        00ff: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 30000000
(s)
        lang: aa|ab|af|av|ay|ba|be|bg|bi|bin|br|bs|bua|ca|ce|ch|chm|co|cs|cu|cv|cy|da|de|el|en|eo|es|et|eu|fi|fj|fo|fr|fur|fy|gd|gl|gn|gv|ho|hr|hu|ia|ig|id|ie|ik|io|is|it|kaa|ki|kk|kl|kum|kv|ky|la|lb|lez|lt|lv|mg|mh|mk|mt|nb|nds|nl|nn|no|nr|nso|ny|oc|om|os|pl|pt|rm|ru|sah|se|sel|sh|sk|sl|sma|smj|smn|so|sq|sr|ss|st|sv|sw|tg|tk|tl|tn|tr|ts|tt|tyv|uk|uz|vo|vot|wa|wen|wo|xh|yap|zu|an|crh|csb|fil|hsb|ht|jv|kj|ku-tr|kwm|lg|li|mn-mn|ms|na|ng|pap-an|pap-aw|rn|rw|sc|sg|sn|su|ty|za(s)
        fontversion: 65536(i)(s)
        capability: "otlayout:DFLT"(s)
        fontformat: "TrueType"(s)
        decorative: FcFalse(s)

Read the style field of each font in the above output, you’ll see “Regular” in the field for Roboto-Regular.ttf and Roboto-Medium.ttf. But note another parameter, the weight of font. Robot-Black and Regular declare it as 80 while Medium declares it as 100.

Font selection happens in the alphabetical order first matching weight and then style. The weight for “Regular” size is 80. So the first font selected would be Roboto-Black, since it declares weight as 80 and appears before Roboto-Regular and Roboto-Medium in alphabetical order.
If you delete Roboto-Black, it will match Roboto-Medium which declares a weight of 100, but has “Regular” in the style specification.

The problem is, both these fonts Black and Medium are just too bold for daily use. You just can’t distinguish between bold and normal text and this causes ridiculous amount of confusion in applications like Thunderbird which use bold text to mark unread message and normal text otherwise.

To deal with this, a slight modification is needed in your font configuration file as follows –

<selectfont>
        <rejectfont>
                <pattern>
                        <patelt name="family"><string>Roboto Lt</string></patelt>
                        <patelt name="weight"><int>100</int></patelt>
                </pattern>
                <pattern>
                        <patelt name="family"><string>Roboto Bk</string></patelt>
                </pattern>
        </rejectfont>
</selectfont>

If you add the above configuration to your fontconfig, it will blacklist Roboto Lt font with weight 100 (which is Medium) and Roboto Bk font which is Roboto Black. See the earlier fc-query output if you’re interested in how I got these values.

And here’s my /etc/fonts/local.conf:

<?xml version="1.0"?>
<!DOCTYPE fontconfig SYSTEM "fonts.dtd">
<fontconfig>
        <match target="font">
                <edit mode="assign" name="hinting">
                        <bool>true</bool>
                </edit>
        </match>
        <match target="font">
                <edit mode="assign" name="hintstyle">
                        <const>hintslight</const>
                </edit>
        </match>
        <match target="font">
                <edit mode="assign" name="antialias">
                        <bool>true</bool>
                </edit>
        </match>
        <!--match target="font">
                <test qual="any" name="size" compare="less">
                        <double>9.0</double>
                </test>
                <edit name="size" mode="assign">
                        <double>9.0</double>
                </edit>
        </match-->
        <match target="font">
                <test qual="any" name="size" compare="less">
                        <double>10.0</double>
                </test>
                <test qual="any" name="family">
                        <string>Monaco</string>
                </test>
                <edit name="size" mode="assign">
                        <double>10.0</double>
                </edit>
        </match>
        <alias>
                <family>serif</family>
                <default>
                        <family>Roboto</family>
                </default>
        </alias>
        <alias>
                <family>sans-serif</family>
                <default>
                        <family>Roboto</family>
                </default>
        </alias>
        <alias>
                <family>monospace</family>
                <default>
                        <family>Monaco</family>
                </default>
        </alias>
        <selectfont>
                <rejectfont>
                        <pattern>
                                <patelt name="family"><string>Roboto Lt</string></patelt>
                                <patelt name="weight"><int>100</int></patelt>
                        </pattern>
                        <pattern>
                                <patelt name="family"><string>Roboto Bk</string></patelt>
                        </pattern>
                </rejectfont>
        </selectfont>
</fontconfig>

Just ensure that this configuration is not overridden in any file /etc/fonts/conf.d. A simple way would be to clean up /etc/fonts/conf.d and keep the bare minimum you need.

Some screenshots for Roboto font as requested by Zachary Potter:

Process Monitor
roboto_thunderbird
roboto_dolphin

NFS cannot allocate memory and No such file or directory errors

NFS cannot allocate memory seems to be common issue a lot of people face while mounting NFSv4 shares, as revealed from search results on Google. Even I faced it when trying to mount a NFS share exported from my desktop on my Raspberry Pi. It didn’t give the error on the Pi, though. I discovered this issue when debugging this problem and tried to mount on localhost.

Initially it appears to be a kernel issue, but it turned out not to be one. After researching a lot about the problem I understood that it is a error in exports configuration!

Basically, in NFSv4, there is a / (root) under which all shares are exported and this must have the fsid=0 parameter specified in the configuration.
At the client, while mounting you are not supposed to specify the full path to the resource, instead it must be relative to the NFS Root.

The following example would clear it up –

I’ll try to export /home and mount it on localhost with following configuration:

# /etc/exports: NFS file systems being exported.  See exports(5).
/home *(rw,no_root_squash,no_all_squash,no_subtree_check)

And when I run the mount command:

Linux ~ # mount localhost:/home /mnt/nfs -v
mount.nfs: timeout set for Tue Jan 29 11:59:27 2013
mount.nfs: trying text-based options 'vers=4,addr=::1,clientaddr=::1'
mount.nfs: mount(2): Cannot allocate memory
mount.nfs: Cannot allocate memory

Now I’ll change the exportfs configuration to:

Linux ~ # cat /etc/exports 
# /etc/exports: NFS file systems being exported.  See exports(5).
/home *(fsid=rw,no_root_squash,no_all_squash,no_subtree_check)

Running the mount command again,

Linux ~ # exportfs -rv
exporting *:/home
Linux ~ # mount localhost:/home /mnt/nfs -v
mount.nfs: timeout set for Tue Jan 29 12:01:58 2013
mount.nfs: trying text-based options 'vers=4,addr=::1,clientaddr=::1'
mount.nfs: mount(2): No such file or directory
mount.nfs: trying text-based options 'addr=::1'
mount.nfs: prog 100003, trying vers=3, prot=6
mount.nfs: trying ::1 prog 100003 vers 3 prot TCP port 2049
mount.nfs: prog 100005, trying vers=3, prot=17
mount.nfs: trying ::1 prog 100005 vers 3 prot UDP port 60646
localhost:/home on /mnt/nfs type nfs (rw)

Now, in the above output, you can see that while it failed to mount with NFSv4, but it succeeded in mounting NFSv3. This comes from the newer way of exports which is partially explained in the manual exports(5).

The solution to the No such file or directory is simple, you have to use / as the path to mount the root and if you are exporting some subdirectories, they need to be relative to / in the mount command instead of the full path.

Now modifying the mount command accordingly:

Linux ~ # mount localhost:/ /mnt/nfs -v
mount.nfs: timeout set for Tue Jan 29 12:15:11 2013
mount.nfs: trying text-based options 'vers=4,addr=::1,clientaddr=::1'
localhost:/ on /mnt/nfs type nfs (rw)

Now if I export a subdirectory inside /home, say my home folder /home/nilesh then it would be available via NFSv4 to clients as /nilesh and not /home/nilesh.

There seems to be a big problem controlling access in this model, you can’t have different permissions for /home and /home/nilesh. If you export /home as read-only, /home/nilesh will also get mounted read-only no matter what.

If you export /home as read-write and /home/nilesh as read-only, then clients would be able to write into all directories on the share, when the share root is mounted, i.e. /, but if the client mounts /home/nilesh it will be read-only.

I’m not sure about how to control access in NFSv4 for subdirectories, since you cannot have more than one entry for fsid=0. If you know the answer, do comment.

Routing for multiple ISP links on Linux

Many of us have multiple ISP uplinks for different reasons like reliability, download limit, etc. but there is no clear guide [at least, I couldn’t find any] on routing for multiple links.

I came across the same problem — I got a new fiber connection, but it is a capped one and I have a slow ADSL link which is not capped. So I obviously would want to use the ADSL link for low priority downloads while fiber for browsing.
After a lot of researching around, I found the solution. Probably a specific case, since my computer has just one Ethernet card and I want to use both the links simultaneously on the same computer.

The network setup scenario:

  • There’s a main router (NAT, DHCP Enabled), which has WAN port, WiFi and LAN ports with IP Address 192.168.0.1
  • There’s another WiFi router (which is used in WDS Bridge mode) with IP Address 192.168.0.2
  • The ADSL Router (NAT Enabled, DHCP disabled) has IP Address 192.168.0.3  and is connected to main router’s LAN port, since fiber must be connected to WAN port for PPPoE connection.
  • An Ethernet Switch is wired to the second WiFi router.
  • The Linux box is connected to the Ethernet Switch.

I setup two IP Addresses on the Linux box: 192.168.0.4 and 192.168.0.5 as follows:

# ifconfig eth0 192.168.0.4
# ifconfig eth0:1 192.168.0.5

Now, I want to route all traffic originating from 192.168.0.4 (the default IP Address for eth0) via fiber, and all traffic originating from 192.168.0.5 via ADSL Router.
To accomplish this, I use policy based routing (CONFIG_ADVANCED_ROUTER must be enabled in kernel configuration) using iproute2.

Routing:

We need to create a new table which will be looked up if traffic is from the second IP and otherwise use the default route.
And then the policy routing rule which specifies that if traffic is from second IP, lookup the other table.

# echo 10 adsl >> /etc/iproute2/rt_tables
# ip route add default via 192.168.0.1
# ip route add default via 192.168.0.3 table adsl
# ip rule add from 192.168.0.5 table adsl

And you’re done. Traffic from applications using 192.168.0.4 (it will be used as default, if you don’t specify anything) will get routed through 192.168.0.1 and those who use 192.168.0.5 will get routed through 192.168.0.3.
As a simple test, you can do this:

$ curl text.whatisyourip.org
$ curl --interface 192.168.0.5 text.whatisyourip.org

That should give you two different IP Address, the first one your Fiber’s public address and the second one your ADSL’s public address.
If that works, you can add the routing code to run at startup or may be even in system network configuration. Every Linux distribution has it’s own way of doing that, so I won’t cover that. If you’re a Gentoo user, you can add it in /etc/conf.d/net as follows:

config_eth0="192.168.0.4/24 192.168.0.5/24"
dns_servers_eth0="192.168.0.1"
routes_eth0="default via 192.168.0.1
default via 192.168.0.3 table adsl"

postup() {
        if [ "${IFACE}" == "eth0" ]; then
                ip rule | grep 192.168.0.5 &> /dev/null
                [ $? -ne 0 ] && ip rule add from 192.168.0.5 table adsl
        fi
}

Now, many applications like Firefox, Chrome, etc. don’t support listening/using to a specific interface/IP Address. There’s a simple solution for the problem I found on Daniel Lange’s Blog.
All you have to do is, download this libc wrapper and compile it as follows:

$ gcc -nostartfiles -fpic -shared bind.c -o bind.so -ldl -D_GNU_SOURCE

Once you have done that, you can use it as follows:

$ LD_PRELOAD=bind.so BIND_ADDR=192.168.0.5 firefox

If you visit text.whatisyourip.org from there, you should see your ADSL public IP. For applications which support binding to a specific address, just configure it do so.

Blog converted to WordPress

There’s no doubt that WordPress is the most popular blogging platform. I used to be a WordPress hater since I had a disaster with the same in 2007 when I had jumped into the blogging bandwagon (that blog does not exist anymore).
And after that incident, I always used Drupal, and in my opinion, Drupal is a very powerful and great Content Management System, suitable for nearly any and every task.

But, when I started my new personal blog, I was in no mood to use WordPress and Drupal was just too heavy for a simple personal blog. So I started out with Octopress.
While most people move from WordPress to Octopress, I did the opposite and I have reasons for it.

Read More

The Minimum Attendance Requirement

One day, I was studying for one of my exams, and this thought came to my mind – Why do colleges keep the minimum required attendance level so high? The first answer that comes to anyone’s mind is: Because students bunk classes.

Okay, but why do students bunk classes?

The reason is simple: Because the lectures are boring. The hard fact is, very few people take up teaching out of dedication. Most of the people are those who for some reasons, could not get a job in their industry and are in the teaching job.

My college specifies a minimum attendance requirement of 75%. And this is almost same across major colleges in India. I don’t know what’s the case in IITs, NITs, etc. I have a few friends from those but nobody ever complained about attendance.

Read More

The Flawed Evaluation System

It’s semester end again at college and we’ve got hell lots of journal-writing work. The journal-writing work is a indespsible part of every semester in Engineering, at least in my University and possibly every other University out there.

The general trend in my college, a lot different from others is – A student (usually the topper) is given the format and sample by the professors. They are then instructed to write it (we do it by hand, unfortunately), distribute their copies to friends, and the chain goes on. So basically, every student has the same journal, differing by some extent due to unavoidable errors that creep in the copy-chain and laziness.

The job of writing journals isn’t very hard that one cannot write it by using their brain, but it is a very boring task to do. Hence everyone (that includes me) refers to be somewhere in the middle of the copy chain, and never in the front. Everyone mindlessly copies the things without even thinking what they are writing in their journals – this means they even copy nonsense. But there’s one thing for sure – nobody would try (exceptions do exist, and they are of the order of 0.01%) to even understand or alter the code which was originally written.

Read More

Getting CherryPy Working With uWSGI

Since Python 3 is the future, I directly started with Python 3 for my projects. While, There are some frameworks for Python 3, but I’m not a fan of frameworks. I prefer to glue the components myself and write the application – even if it involves more work and boilerplate code, and that’s because of freedom.

When you use a particular framework, you’re bound to a few rules and and modifying the behavior of the rules gets quite difficult, unless you know the framework you’re using from head to tail completely in detail. Half knowledge is very bad.

I knew about CherryPy since Python 2, and when I studied how to write applications using that, it became my favorite framework. CherryPy is basically a minimal framework, or more specifically a server which handles the common headaches that are required to do when writing a web application in Python. The common headaches are like mapping a request into Controller/Action, handling HTTP errors, authentication, etc. No, these tasks are not difficult, but are rather time consuming because of the size of the code involved.

So, why not use a pre-built framework? I’m negating my own statement, eh? Yes. In this case, freedom is not a strict requirement as much as it is required in the application logic, and CherryPy supports Python 3 – Awesomeness.

CherryPy can be deployed by various methods – It has an inbuilt HTTP server, it can be used as a FastCGI, SCGI and CGI server as well. The point is, HTTP parsing is slow, and doing a slow task using an interpreted language – doesn’t get me. I could very well use CherryPy’s HTTP server as the server for website and stay quiet. That doesn’t work because we’ll be running a relatively slower code (as compared to C/C++, in which web servers are usually written) even for static resources! This is a big waste of resources.

CherryPy uses the Flup1 module for FastCGI/SCGI/CGI implementation and unfortunately, there’s isn’t a working release of the same for Python 3. It’s development seems to have stalled. I was able to install and use the Flup module in Python 3. It installed successfully, but it seems some of the code in it wasn’t ported to Python 3, running 2to3 fixed it. What about any Bugs in it? – The biggest problem. It doesn’t have a public release! If the author had released a version, I would have just used it.

uWSGI is a application container written in pure C. It has a lot of features like multiple protocol support, process management, easy configuration and a lot. You can learn more about it at the official website.

As per the Python PEP-33332, the WSGI application (a callable) should be named as application, should accept two parameters and return bytes after calling the function that comes as second parameter (usually named as start_response).

The same specification is used by uWSGI. But the problem here is, how to get CherryPy running with uWSGI, since the default method is to spawn a HTTP server if you use cherrypy.quickstart() or, use the cherryd command. You can spawn a FastCGI/SCGI/CGI server with the cherryd command (which requires the Flup module).

In the uWSGI case, the server processes are handled by uWSGI, so you need not spawn any processes in your application. After a lot of searching around, reading manuals and experimenting I finally found a workaround to get this thing working.

Here’s a simple code which deploys two apps (basically two classes) using cherrypy:

#!/usr/bin/python3

# Script written by NileshGR (http://nileshgr.com)
# This script is under BSD license

import cherrypy

class One:
  @cherrypy.expose
  def index(self):
    r = cherrypy.response
    r.headers['Content-Type'] = 'text/plain'
    return "Hello World"

class Two:
  @cherrypy.expose
  def default(self, *args, **kwargs):
    r = cherrypy.response
    r.headers['Content-Type'] = 'text/plain'
    content = "Positional arguments\n\n"
    for k in args:
      content += k + "\n"
    content += "\nKeyword arguments\n\n"
    for k in kwargs:
      content += k + ": " + kwargs[k] + "\n"
    return content

def application(environ, start_response):
  cherrypy.tree.mount(One(), '/', None)
  cherrypy.tree.mount(Two(), '/par', None)
  return cherrypy.tree(environ, start_response)

The first two classes are CherryPy applications. Notice the last part, after line #28, we’re defining a function named application(environ, start_response) as specified by PEP-33332.

In the function, we mount the first application at mount point / and the second application at mount point /par and finally return cherrypy.tree(environ, start_response which is transfers control to CherryPy. The secret here is in the fact that, cherrypy.tree is a WSGI compatible application and which is why this works!

Quoting the text from cherrypy.tree doc page (pydoc):

cherrypy.tree = class Tree(builtins.object)
 |  A registry of CherryPy applications, mounted at diverse points.
 |  
 |  An instance of this class may also be used as a WSGI callable
 |  (WSGI application object), in which case it dispatches to all
 |  mounted apps.

Starting the uWSGI server to run our CherryPy application:

uwsgi --http :8080 --wsgi-file cherrypy_uwsgi.py

I passed --http to uWSGI because I don’t have a WSGI capable HTTP server on my PC where I do all this development and test work. Anyway, if HTTP with uWSGI works, then the other protocols like WSGI, FastCGI, etc. that uWSGI implements should also work (isn’t that obvious?).

See uwsgi --help for more information on how to spawn a server using the protocol you need.

Screenshot of the app running in my browser:

cherrypy_uwsgi

The credit to what made me try this out and get it working goes to @unbit who helped me out on Twitter and pointed me to this article on CherryPy wiki.

Trac and Git: The Right Way

This post is about configuring Trac and Git to work together properly. As you might be knowing, Git is a DVCS – Distributed Version Control System and Trac is a project management system. By default, Trac supports SVN, but there are plugins for Git and Mercurial. I don’t know if there are plugins for other repositories like Bazaar, etc.

In the default configuration in Trac’s Git plugin, the repository is not cached and setting repository_sync_per_request to blank doesn’t stop Trac from syncing repositories on each request. This is a big performance disadvantage and can big a real big trouble if you have lots of repositories with quite a lot of commits.

Read More