ZFS convert stripe to striped-mirror

OpenZFS LogoI’m a huge fan of ZFS because of its performance and other features like snapshots, transparent compression. In fact I had switched to FreeBSD for servers just because it had native ZFS support. But as of Ubuntu 16.04, ZFS is officially supported for non-root partitions.

Now I’m migrating a FreeBSD server to Ubuntu 16.04 with ZFS for data storage – this is happening because I need support for some special hardware which has drivers only for Linux and I do not have a spare server machine of same capacity in terms of memory/disk/processor.

My case –
Here’s the zpool layout on my existing FreeBSD server:

Each of those disks are 1TB in size and the layout here is something known as RAID 10, or striped mirroring. Striped mirroring can be extended to more than four disks but in my case, I have two pairs of disks. Each pair is mirrored and the each such mirror is striped, illustrated as in the image below:

Image taken from techtarget.com, their trademark/copyright holds.

The advantage of this layout is that you get read speed of four disks, and write speed of two disks and a failure tolerance of two disks (but in different mirrors) at the same time.

I have a spare 1TB disk which I can use for preparing a new server using a low-end machine for migration. I remove one of the disks from the live server so the pool there runs in a degraded state. The removed disk is used in the new server. So I create this zpool in Ubuntu:

The pool created here is a plain simple stripe. To convert this into a striped-mirror, the zpool attach command has to be used:

With this, the pool now becomes a striped mirror:

Perfect! 😀

 

Advertisements

SystemD FastCGI multiple processes

Of late, many mainstream distributions have been switching to SystemD as their init system. This includes Debian (since Debian 8) and Ubuntu (since Ubuntu 15.04). In the traditional SysV init system we used to have stuff like spawn-fcgi or custom scripts for starting a FastCGI process and having the web server connect to it over Unix or TCP sockets. Such kind of usage decreased when PHP FPM was introduced since it’s safe enough to assume that 90% (probably more) of the FastCGI deployments are just launching PHP interpreters using whatever mechanism is there (spawn-fcgi or custom scripts). PHP FPM does this for you now and it’s pretty good at it.

FastCGI is just a protocol, it can be used by any application. For custom applications which do not support starting their own FastCGI processes and listening on a socket we have to use external mechanisms. SystemD has a couple of good features which can help reduce the amount of custom work needed in terms of process monitoring, socket paths, file ownership, etc.

Continue reading “SystemD FastCGI multiple processes”

99% problems in electronics are related to power supply

My dad, who has worked a lot in electronics always says this one thing:

99% of problems in electronics are because of a bad power supply

Today, a real experience of this: In 2014 I bought a GSM signal booster to solve the call drop problem at home. It was working fine for all these 2 years, but since a few weeks the calls started dropping again. The booster had come with an 5V 1A adapter which powered it and I never switched off the booster. When I went to check it, the small screen on it showing the signal strength was blank and even after restarting it a couple of times it didn’t turn up again.

So I replaced the power adapter with my old Nexus 4 charger which can supply 1.2A at 5V. And boom, it works. Signal strength improved and calls are clear again. So always need to keep this in mind, whenever there’s a problem in electronics, first check the power supply.

Results of an online survey about preferred tyres

I am facing some instability on my car rides and after showing it to numerous mechanics, wheel alignment centers everyone said the same: The tyres have gone bad and they need replacement. The car hasn’t had that much running to justify a tyre replacement so I started looking online for solutions. Finally I have to conclude after much online research that the tyres have gone bad.

With so many tyre brands available in the market at different price points it’s difficult to make a good selection by just reading stuff online and knowing about experiences of friends / mechanics. In a commission driven business model like India where every tyre vendor sells tyres of every company they would obviously try to sell the tyre that will fetch them most profits (exceptions exist, I know!). So to make a good choice I decided to run a survey on Twitter and Reddit. Posting the results of the same. I received total 41 responses.

Continue reading “Results of an online survey about preferred tyres”

A simple graph I made using spreadsheet to compare call rate in per second vs per minute. There was a similar graph somewhere around, but I lost the link and hence I made my own graph 😛

call_rate

1.5 paisa/second and 45p/minute are common rates in per second and per minute plans offered across India, so used that as base for creating my graph. If you are interested in downloading the spreadsheet I used for this calculation, you can do so here. It’s an OpenDocument Spreadsheet. I don’t use Windows nor MS Office. 🙂

Group based HTTP basic authentication using Nginx and MySQL with help of Lua

Recently I moved from Apache to Nginx on one of my servers due to increase in traffic. But I was using HTTP Basic authentication with group based authorization on Apache in this manner:

However, there’s no AuthGroupFile  in nginx. But LUA, a programming language is supported in nginx. So here’s how I used LUA and MySQL for achieving this:

Now the real magic comes in the authenticate.lua  script, I’m posting the code below which is available in Github as well:

The group authentication script looks for users and groups in a table called http_users. Since this is a script you can modify the way users are searched for in the database or change the database altogether!
The lua modules required to run this script are: resty.mysql, resty.session, resty.string and cjson. Though the passwords are stored in the database as a SHA224 hash, the comparison of the password is done by the database itself. I did not convert the password to hash before sending it to database, so you may want to review this in case you are using remote database. I’m using local database over Unix socket so it doesn’t matter much.

The table and triggers I have for the same:

The triggers are required to convert the INSERT  or UPDATE statements into SHA224. I’m using MySQL’s SET data type to ensure that the group value is fixed. The same values can be used by Nginx in $user_group  variable before specifying the access_by_lua_file  directive.

Linode, you need a spam filter

I got this usual money scam email not directly, but via Linode Forum! Who even deploys Web applications without basic spam protection?!

Hello nileshgr,

The following is an email sent to you by joy kone via your account on
“Linode Forum”. If this message is spam, contains abusive or other comments
you find offensive please contact the webmaster of the board at the
following address:

https://forum.linode.com/memberlist.php?mode=contactadmin

Include this full email (particularly the headers). Please note that the
reply address to this email has been set to that of joy kone.

Message sent to you follows
~~~~~~~~~~~~~~~~~~~~~~~~~~~

From: Joy Kone

I have sent you this e-mail because of the need to open discussions with
you. I don’t want you to misunderstand this offer in any aspect…if it is
okay with you, I ask for your full cooperation. I have contacted you base
on trust to handle an investment in your country/company on my behalf as a
prospective partner.

My name is Joy Kone. a citizen but resides here . It might interest you to
know that I have US$10.500,000.00 deposited with a financial institution to
be invested in your country/company. It is pertinent to let me know if you
can handle this fund/investment with you in your country so as to furnish
you with all the necessary details about the financial institution for more
information. Meanwhile, i am very honest in my dealings with people and I
also demand the same from you as a Partner to be. Can I trust you with this
fund?

I want you to note that this is a mutual business venture as t here is a
reward for your assistance. I shall let you know your benefit for your
assistance as we proceed. For a more comprehensive details and source of
fund, please contact me as soon as possible. If you find this letter
offensive, please ignore it and accept my apologies.

Regards,
Joy Kone

Okay I received a reply from someone working at Linode, apparently it is possible to turn off the ability of others to send email to you via the forum:

A networking insight into the past

Back in 2007-2008, when I was just starting out with Linux geekery I had an ISP connection which was working fine for almost a year. Previously I was a Windows user and the hardware I used to run Windows XP was a Pentium 3 with 384 MB RAM.

Now I don’t know whether it was the hardware or issues with Windows itself that caused me so much frustration sufficient to make me move to Linux. After switching to Linux, things had become smooth so it was probably not the hardware. Perhaps the hardware was insufficient to run XP although it was slightly better than the recommended hardware specification at that time.

So yeah, this ISP had installed a telephone wire into my house and provided me an ADSL modem. In those days, it was rather uncommon to have multiple devices at home at least in India, so they had a policy of allowing only one PC at a time to use the Internet. My networking knowledge was pretty limited at that time so I never thought about how or why it was like that.

Then dad’s workplace assigned him a laptop and that’s when there were two devices at home that required Internet. Again due to limited knowledge of networking and Linux, I got a long LAN cable so that dad’s laptop could be wired to the modem having a single Ethernet port. A couple of times this plugging/unplugging; we got tired of it and then bought an unmanaged switch. That helped solve the cabling issue, but still two computers couldn’t use the Internet at the same time. I used to log out when my dad wanted to use Internet and vice versa. The switch model worked because the modem provided by the ISP was in bridge mode as per the settings I found out by poking around.

Both the machines were Windows XP initially. Then I switched to Linux and configured a simple DHCP based Ethernet connection to use Internet and it worked fine. The wizards helped me and the authentication mechanism was to login to the ISP using a Web page you got redirected to once you opened some site after acquiring a lease. But one day, all of a sudden the Internet stopped working on my Linux box. A quick observation was that it worked fine in Windows. Numerous calls to the ISP’s call center and as usual a clueless response by them (this continues even today to some extent) but they eventually sent their technician who couldn’t solve the problem either. Then we switched ISP.

Over the seven years after this I have learned a lot of Linux and Networking and worked in real life scenarios. But today while chatting with my friend Nikhil about ISPs and their reviews, I recalled this issue and now I can make sense why it wasn’t working. Reason is simple, routers generally contain Embedded Linux. In DHCP there’s a field mentioning what OS or which client is it (like a Web browser sends user agent to every website). This cunning ISP wanted to make money by selling their own routers and charging more for allowing multiple computers to use Internet at the same time so they decided to block all Linux DHCP clients (probably excluding their own) because every Linux box is a potential NAT box! It is possible to use Windows as a NAT box as well, but then they had no choice. If they blocked windows nobody would use their services 😂😂

CloudFlare Dynamic DNS using OpenWRT

I use dynamic DNS for my home internet connection so that I can access the machines from anywhere on the internet. And I use OpenWRT on my router. Earlier I was using Namecheap for managing DNS but I switched to CloudFlare for performance and security reasons of the website.

Unfortunately CloudFlare doesn’t support updating IP via shell script — well, it sort of does but the JSON stuff gets very messy with quoting in shell scripts, so I wrote a Lua script to update my IP whenever my PPPoE connection starts up; I have dropped the script in /etc/ppp/ip-up.d  so it gets executed by pppd whenever my connection comes up. You can run this script via cron or put it /etc/hotplug  if you wish to. This script uses LuaSocket, LuaSec, JSON4Lua and libubus-lua libraries that are easily installable on an OpenWRT router with 4 MB flash memory.

Now I can have the benefits of CloudFlare without losing out on DDNS :D. Here’s the code:

Suggestions? Post in comments or fork on GitHub.

Proudly powered by WordPress | Theme: Baskerville 2 by Anders Noren.

Up ↑

%d bloggers like this: