Category Archives: Geeky

Technical stuff about Perl, Linux and computing & technology in general.

Favourite new Perl features

I’ve been starting to make use of the new features introduced in perl 5.10 recently (after being constrained by my main dev environments still running perl 5.8.8, and not having the time to upgrade).

My favourite features so far are:

The smart match operator

The new smart-match operator, ~~, is a great example of DWIM.

A few examples:

if (@a ~~ 'foo')  # list contains at least one item equalling 'foo'
if (@a ~~ /foo/) # list contains at least one item matching /fo+/
if (@a ~~ @b)   # lists contain same values

That’s just a brief overview; there’s plenty more documentation

say

Not a big change, but the new say keyword acts just like print, but adds an implicit newline to the end – so say 'Hello'; is just the same as print "Hello\n";

It’s more useful in cases where you would have had to add parenthesis to get correct precedence – something like: print join(';', @foo) . "\n"; can now be written more concisely as just say join ';', @foo;.

Switch (given) statement

given ($foo) {
    when (/^abc/) { abc(); }
    when (/^def/) { def(); }
    when (/^xyz/) { xyz(); }
    default { die "Unrecognised foo"; }
}

Defined-or

// is now the defined-or operator.

It’s pretty common to use conditional assignments like: $a ||= $b to assign to $a unless $a already has a value. Now you can use $a //= $b to test for definedness rather than truthiness.

Likewise, if ($hash{foo} // $hash{bar}) will be true if either of them is defined (even if they’re defined but have a false value).

Named regex captures

Parenthesised sub-expressions in regular expressions can now be given a name, and accessed via the special %+ hash:

if ($foo =~ m{ (? \d{4} ) - (? \d{2}) - (? \d{2}) }xms) {
    say "Year: $+{year}";
}

The features above are my own personal favourites, in no particular order. The full (large) set of changes can be found in the perldelta for 5.10.0.

Pastebin Firefox extension

My friend James Ronan has just released a Pastebin Firefox extension, making it even easier to paste code etc to pastebin.com.

As the code by Paul Dixon which powers pastebin.com is Open Source and can be installed on your own server, the extension allows you to provide the URL of your own private pastebin install if you have one – this is ideal for me, as we have a private pastebin setup at work which is often used.

Using the extension is as simple as right clicking and chosing “pastebin my clipboard”, which submits the contents of your clipboard (or highlight buffer) to pastebin, and copies the resulting URL to the clipboard, ready for you to paste on IRC / IM / whatever.
Continue reading Pastebin Firefox extension

Quick Fibonacci calculations are nothing new

Just read this post by Ben Newman (found via Reddit).

Now, the use of C++ templates to calculate the value at compile time rather than runtime is midly clever and amusing (if also impractical and convoluted) but the fact that it can calculate a Fibonacci number quickly is nothing new; it’s solely down to remembering the values you’ve already calculated, and not calculating them again needlessly.
Continue reading Quick Fibonacci calculations are nothing new

Preventing SSH brute-force attacks with iptables

I wanted to do this tonight and couldn’t remember the exact iptables incantation, and I know I’ll want it again, so sharing it here for me and for anyone else it may be useful to.

If you need SSH to be world-accessible, but don’t want to be plagued by SSH brute-force login attempts, the following ought to do the trick:


iptables -A INPUT -p tcp --dport 22 -m state --state NEW -m limit --limit 3/minute -j ACCEPT

That will allow inbound SSH connections, but only 3 per minute (averaged) – this should be more than a user would normally need, but isn’t sufficient for a brute-force login attack. If someone tries a brute-force attack against you, after a few connections they’ll be ignored.

This is assuming that your default INPUT policy is to drop or reject packets, as it should be. For it to work, it also assumes that you have a rule to allow inbound connections which are part of an established connection – something like:


iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT

Of course, if you’re going to open SSH to the world, even with the above protection, you’ll still want to ensure passwords are secure (or disable password-based authentication totally, using SSH keys instead), and I’d recommend disabling root logins via SSH (the PermitRootLogin setting in /etc/ssh/sshd_config)

VPS.net beta testers wanted for new VPS products

VPS.net (a brand of the UK2 group) is looking for VPS beta testers to receive a free VPS for testing.

There’s only 50 places available so, if you want to take part in the beta, go sign up now! We’re looking for users with some experience of VPS/servers, who can provide useful feedback on their experiences with the service.

See also the description of what a VPS cloud is, and how the design provides scalability and reliability for VPS users.

Spam blacklisting is massively useful

I use SpamAssassin to filter spam out of my incoming mail, and it does its job admirably. I didn’t bother using DNS blacklists, since I didn’t really want to hand over control of what mail I accept to a third party, and was worried about false positives. However, with an average of about 3,000 spam mails per day getting shoved down to my home server for it to run spamassassin on, it was quite a waste of resources.

Recently I configured Postfix on my public-facing server to check against a couple of DNS blacklists – the most effective of which seems to be zen.spamhaus.org.

Here’s the number of spam mails which reached my home server each day:


[davidp@supernova:spam]$ grep -c '^From ' spam-2008-11-{27,28,29,30} spam-2008-12-*
spam-2008-11-27:3048
spam-2008-11-28:2759
spam-2008-11-29:3439
spam-2008-11-30:2853
spam-2008-12-01:3113
spam-2008-12-02:1231
spam-2008-12-03:155
spam-2008-12-04:193
spam-2008-12-05:78
spam-2008-12-06:92
spam-2008-12-07:61

See if you can guess which day the blacklisting was put in place :)

Debian Xen guests without /dev/pts – no SSH

I’ve been playing with Xen on a Debian host machine. I created a couple of guest machines using xen-create-image (set to use debootstrap to automatically install + configure a basic Debian install on the guest), and was unable to SSH to the guests – I got:


[davidp@masterplan ~]$ ssh 10.1.1.20 -l root
root@10.1.1.20's password:
PTY allocation request failed on channel 0
stdin: is not a tty

I struggled with this for a while, for some reason accessing the guest’s console with xm console didn’t seem to work either.

However, I now managed to get console access to one of them using xm console domain, and found that /dev/pts wasn’t mounted – it wasn’t listed in the fstab file that had been created.

I added the following to /etc/fstab :


none /dev/pts devpts defaults 0 0

Now I can SSH to the guests. :)

Labelling FAT/FAT32 partitions in Linux

Mostly a post for my future refererence, as it took some Googling to find this, but it might be useful to others.

I wanted to view/change the “drive label” for a FAT partition. This is done with the mlabel tool from the mtools package, but it has a strange insistence on setting up “drive letter” mappings, in /etc/mtools.conf or ~/.mtoolsrc, so that you can use it as, for example:


mlabel e:

Um, no thanks. I don’t want to map a Windows-like notion of drive letters to a partition which may appear at a different device each time (it’s a USB device; the point of having the label is to not have to know or care what device node it’s been assigned this time – if USB devices are connected in a different order, it might not be /dev/sdf next time).

The option needed is -i which doesn’t appear to be documented in the mlabel manpage, used along with the fake drive letter ::, for example:


# setting:
[dave@ruthenium ~]$ sudo mlabel -i /dev/sdf1 ::DAVEBLACKBERRY
# viewing:
[dave@ruthenium ~]$ sudo mlabel -s -i /dev/sdf1 ::
Volume label is DAVEBLACKBERRY

With that done, I can then add an entry to /etc/fstab which identifies the device by its label:


[dave@ruthenium ~]$ grep BLACKBERRY /etc/fstab
/dev/disk/by-label/DAVEBLACKBERRY /mnt/blackberry vfat defaults,uid=dave,gid=users 0 0

I should have been able to use LABEL=DAVEBLACKBERRY rather than the long /dev/disk/by-label/... notation, but LABEL= didn’t work, and I didn’t have tiem to figure out why :)

Creating HTML tables from database queries with HTML::Table::FromDatabase

A task I find myself doing reasonably often when programming is producing a HTML table based on the result of a database query.

This often ends up with the same kind of boring code being written again and again, which get tedious.

For example:

print <
idfoobar
TABLESTART

my $sth = $dbh->prepare(
    "select id, foo, bar from mytable where something = 'somethingelse'"
);
$sth->execute() or die "Failed to query";

while (my $row = $sth->fetchrow_hashref) {
    print '';
    print join '', @$row{qw(id foo bar)};
    print "\n";
}
print "\n";
$sth->finish;

Not hard, but it does get tedious.

HTML::Table makes things better by taking out most of the HTML drudgery, but you still need to loop through adding rows to your table.

This is where my HTML::Table::FromDatabase comes in – it’s a subclass of HTML::Table which accepts an executed DBI statement handle, and automatically produces the table for you.

For instance:

my $sth = $dbh->prepare(
    "select id, foo, bar from mytable where something = 'somethingelse'"
);
$sth->execute() or die "Failed to query";

my $table = HTML::Table::FromDatabase->new( -sth => $sth );
$table->print;

Much easier, and HTML::Table::FromDatabase does all the tedious work.

Sometimes that won’t be quite flexible enough though; you might have something you want to do to certain columns or values before outputting them.

That’s where HTML::Table::FromDatabase’s callbacks come in handy. For a basic example, let’s say that one of the columns you’re fetching contains URLs, and you want to wrap them in anchor tags to make them clickable links. Simply done with:

 my $table = HTML::Table::FromDatabase->new(
    -sth => $sth,
    -callbacks => [
        {
            column => 'url',
            transform => sub { $_ = shift; qq[$_]; },
        },
    ],
 );

Another example – looking for all cells whose value is a number, and formatting them to two decimal places:

 my $table = HTML::Table::FromDatabase->new(
    -sth => $sth,
    -callbacks => [
        {
            value => qr/\d+/,
            transform => sub { return sprintf '%.2f', shift },
        },
    ],
 );

You can apply as many callbacks as you need.

As HTML::Table::FromDatabase is a subclass of HTML::Table, all of HTML::Table’s options can still be used to control how the generated table appears, for example:

  • -class => 'classname' to give the table a specific class to help you apply CSS styling
  • -border => 1 to apply borders, -padding => 3 to set cell padding
  • -evenrowclass and -oddrowclass if you want to have different styling for even and odd rows (e.g. alternating row backgrounds).

The full list of options can be found in the HTML::Table documentation, I’m not going to duplicate it all here.

Currently, the row headings used in the generated table are taken from the column names in the query, but I plan to release a new version sometime soon which allows you to alias them, if you want to do so.

(The code samples in this post are intentionally kept relatively simple, omitting obvious things like connecting to the database first, error checking etc).

(This post also appears on Perlbuzz)