Monitoring your internet connections with OpenWRT and a Telegram Bot

For the past 5 years or so, I have been using a single ISP at home and mobile data for backup when it went down. But since last few months, the ISP service became a bit unreliable – this is more related to the rainy season. Mobile data doesn’t give fiber like constant speeds I get on the wire. It’s very annoying to browse at < 10 Mbps on mobile data when you are used to 100 Mbps on the wire.

I decided to get another fiber pipe from a local ISP. One needs to be very unlucky to have both going down at the same time – I hope that never happens. Now the question is how to monitor the two connections: Why do I need monitoring? – so that I can inform the ISP when it goes down, with the fail-over happening automatically thanks to OpenWRT’s mwan3 package, I won’t ever know when I am using which ISP (unless I am checking the public IP address, of course).

The solution: A custom API and a Telegram bot. For those not aware about Telegram, it is an amazing messaging app just like Whatsapp with way more features (bots, channels), and does away with some idiosyncrasies of Whatsapp such as restricting you to always have the phone connected.

A Telegram bot is fairly simple to write, you just have to use their API. Now this bot is just going to send me messages, I am never going to send any to it, so implementing my WAN monitor bot was very easy.

My router is a TP Link WR740N which has 4 MB flash – so it is not possible to have curl with SSL support which is required by the API. I wrote a custom script which can be called over HTTP and plays well with the default wget. The script is present on a cloud server which can, obviously, do the SSL stuff.

A custom wrapper to Telegram API to send message in PHP:

The <your chat id>  part needs to be discovered once you send a /start command to your bot and use Telegram’s getUpdates method. You will get it in API’s response JSON. $key  is just a security check to prevent external attacks on the script.

And this script is called on interface events by mwan3 (/etc/mwan3.user ):

Shell script to monitor connections by cron directly from the server:

The above script uses netcat to do the link test using a TCP connection to a port number which is port forwarded to a server because I found ping was doing some false positives. I couldn’t reproduce it when I was trying it manually but I used to get DOWN messages even though the connection was working.

One must wonder though, how will the message reach me via Telegram when both ISPs go down at the same time – well I leave that job to Android and mobile data. Android switches to mobile data as soon as it finds WiFi doesn’t have internet access.

Advertisements

Asterisk PBX with Reliance PRI Line using Digium TE131F

So I got an opportunity to set up Asterisk PBX with a Reliance Communications E1 line. I have worked with Asterisk PBX, but without PSTN interfacing. This post is about what all stuff I have done to get a Reliance E1 line with Digium TE131F card.

Having explored a lot of other distributions like Fedora, Arch, Gentoo, Sabayon, etc. since I ventured into Linux world and learning the internals of Linux and how different components are stitched together I settled on Ubuntu. It’s my favorite these days because  everything seems to work out of the box… except when it doesn’t, then you have PPAs. 😛 For this project I have installed Ubuntu 16.04 server edition.

Continue reading “Asterisk PBX with Reliance PRI Line using Digium TE131F”

ZFS convert stripe to striped-mirror

OpenZFS LogoI’m a huge fan of ZFS because of its performance and other features like snapshots, transparent compression. In fact I had switched to FreeBSD for servers just because it had native ZFS support. But as of Ubuntu 16.04, ZFS is officially supported for non-root partitions.

Now I’m migrating a FreeBSD server to Ubuntu 16.04 with ZFS for data storage – this is happening because I need support for some special hardware which has drivers only for Linux and I do not have a spare server machine of same capacity in terms of memory/disk/processor.

My case –
Here’s the zpool layout on my existing FreeBSD server:

Each of those disks are 1TB in size and the layout here is something known as RAID 10, or striped mirroring. Striped mirroring can be extended to more than four disks but in my case, I have two pairs of disks. Each pair is mirrored and the each such mirror is striped, illustrated as in the image below:

Image taken from techtarget.com, their trademark/copyright holds.

The advantage of this layout is that you get read speed of four disks, and write speed of two disks and a failure tolerance of two disks (but in different mirrors) at the same time.

I have a spare 1TB disk which I can use for preparing a new server using a low-end machine for migration. I remove one of the disks from the live server so the pool there runs in a degraded state. The removed disk is used in the new server. So I create this zpool in Ubuntu:

The pool created here is a plain simple stripe. To convert this into a striped-mirror, the zpool attach command has to be used:

With this, the pool now becomes a striped mirror:

Perfect! 😀

 

Group based HTTP basic authentication using Nginx and MySQL with help of Lua

Recently I moved from Apache to Nginx on one of my servers due to increase in traffic. But I was using HTTP Basic authentication with group based authorization on Apache in this manner:

However, there’s no AuthGroupFile  in nginx. But LUA, a programming language is supported in nginx. So here’s how I used LUA and MySQL for achieving this:

Now the real magic comes in the authenticate.lua  script, I’m posting the code below which is available in Github as well:

The group authentication script looks for users and groups in a table called http_users. Since this is a script you can modify the way users are searched for in the database or change the database altogether!
The lua modules required to run this script are: resty.mysql, resty.session, resty.string and cjson. Though the passwords are stored in the database as a SHA224 hash, the comparison of the password is done by the database itself. I did not convert the password to hash before sending it to database, so you may want to review this in case you are using remote database. I’m using local database over Unix socket so it doesn’t matter much.

The table and triggers I have for the same:

The triggers are required to convert the INSERT  or UPDATE statements into SHA224. I’m using MySQL’s SET data type to ensure that the group value is fixed. The same values can be used by Nginx in $user_group  variable before specifying the access_by_lua_file  directive.

CloudFlare Dynamic DNS using OpenWRT

I use dynamic DNS for my home internet connection so that I can access the machines from anywhere on the internet. And I use OpenWRT on my router. Earlier I was using Namecheap for managing DNS but I switched to CloudFlare for performance and security reasons of the website.

Unfortunately CloudFlare doesn’t support updating IP via shell script — well, it sort of does but the JSON stuff gets very messy with quoting in shell scripts, so I wrote a Lua script to update my IP whenever my PPPoE connection starts up; I have dropped the script in /etc/ppp/ip-up.d  so it gets executed by pppd whenever my connection comes up. You can run this script via cron or put it /etc/hotplug  if you wish to. This script uses LuaSocket, LuaSec, JSON4Lua and libubus-lua libraries that are easily installable on an OpenWRT router with 4 MB flash memory.

Now I can have the benefits of CloudFlare without losing out on DDNS :D. Here’s the code:

Suggestions? Post in comments or fork on GitHub.