Downloading MySQL rpms with a Linux one liner

I love linux. It’s almost one year now since I switched my main work desktop machine to Linux from Windows XP, and I’ve not looked back. Windows was slow, unreliable (regular blue screens) and lacked many of the advanced features that linux has out of the box (or should I say off of the web).

Linux’s features are too numerous to list, but every now and then I use one that just reminds me how superior it is to it’s proprietary rival. In this case I needed to download the latest MySQL 5.1.23 rpms for installation on an ndb cluster.

Normally I would go to MySQL’s download page, and manually right click to save them individually, but since I have been doing it so frequently recently, and that I was likely to need to do it again in the future, I thought there must be a better way.

The answer lay with a few bash commands strung together with pipes:

wget -O - http://mirror.fubra.com/www.mysql.com/Downloads/MySQL-5.1/ | grep -o -P 'href=".+5.1.23-0.glibc23.x86_64.rpm"' | grep -o -P 'MySQL[^"]+' | xargs -I {} wget http://mirror.fubra.com/www.mysql.com/Downloads/MySQL-5.1/{}

The command above (which should be all on one line) does the following:

• First we use wget to download a directory listing of all MySQL 5.1 downloads from our local MySQL mirror (but this could be any mirror). We invoke the -O – option to direct the output of the webpage to STDOUT rather than a file.
• The output from wget is piped to grep which does a perl regular expression to look for links to all rpms from the particular version of MySQL we want, in this case generic 5.1.23 for x86_64 machines. This is returned as a list of every link from the html source containing a link to one of these files.
• The output from grep is piped to another instance of grep. The reason for this is that we want to cut out the href=”” from the links so we are left with just the file name.
• The tidied list of filenames is piped to xargs which runs wget for each one, pre-pending the full path to the beginning of the filename

And that’s it. We end up with each rpm being downloaded to the current working directory.

I know that it is technically possible to do things like this with Windows Power Shell, and cygwin, but they are not native solutions that are available to every machine by default, as they are on all *nix machines.