Perl, Uncategorized

Catalyst auto HTML/XSS scrubbing

Leave a comment

At work, we needed to implement some HTML scrubbing and XSS protection across a Perl Catalyst-powered API, so went looking for existing solutions. We found Catalyst::Plugin::HTML::Scrubber which did some of what we needed, but did not scrub within encoded PUT/POST bodies e.g. POSTed JSON.

I implemented some improvements to provide this, but sadly the original author could not be reached – it seems he hasn’t been active in the Perl community for quite some time. With a little help from the CPAN admins (thanks!) I obtained maintainership of it, and have since got a couple of releases out which add the features we needed:

  • scrubbing HTML/XSS attempts within both normal parameters (querystring / POSTed forms) and also recursively within PUTted/POSTed JSON etc
  • the ability to whitelist certain parameters by name or regex to exclude them from scrubbing – we have some admin-only areas where staff can enter “message of the day” content which is allowed to contain HTML
  • a “no encode HTML entities” option to undo HTML::Scrubber‘s automatic HTML entity encoding of e.g. angle brackets – whilst content destined for the browser wants to be HTML-encoded, inbound parameters don’t want that, we just want HTML /XSS attempts stripped, a parameter value like >= 5 should be left alone

The amended version can be found on CPAN – Catalyst::Plugin::HTML::Scrubber.


(Aside: yes, it has been, er, quite some time since I posted anything on this blog.)

Perl, Programming, System Administration

DBI reading MySQL connection details from .my.cnf

Useful trick: I often have my MySQL account credentials stored in .my.cnf so the mysql command-line client can use them. I also often have Perl scripts which want to connect to the database, and want them to use that file, not have to put the params into the script or have the script read its own config file with the credentials duplicated there.

The answer:

my $dsn = "DBI:mysql:database_name;mysql_read_default_file=$ENV{HOME}/.my.cnf";
my $dbh = DBI->connect($dsn,undef,undef,{RaiseError => 1}) 
    or die "Failed to connect to DB!";

Easy!

Wibbles

How to lose a customer, ASDA

1 Comment

I’ve been doing my grocery shopping online for quite some time using Tesco, and have been very happy. I decided to try out ASDA’s offering, prompted somewhat by an attractive voucher offer – so I placed an order for delivery today, applied the voucher (it was successfully accepted and applied by their website), and got the confirmation email.

An hour after the delivery slot selected, no delivery, and no contact from ASDA. I phoned their customer services, and was told the order had been cancelled because I used an “invalid voucher”. This would be the voucher they created and distributed, which their website confirmed was acceptable and applied to my order. Not only that, they didn’t bother to contact me at all to inform me they were cancelling the order – so I would have been sitting waiting all afternoon for a delivery which was never to show up.

Not what I’d consider a decent first impression. Back to Tesco I go – I shan’t be using ASDA again. Now if I could just find an option to delete my ASDA account, or at least remove the card details from it (which I don’t recall authorising them to store)…

Perl, Programming

Using SSL client certs with Perl’s LWP::UserAgent

11 Comments

I recently needed to authenticate to a remote API using an SSL client certificate, and had a bit of trouble getting LWP::UserAgent to work with it.

The examples I found which looked like they should work involved e.g.:

use LWP::UserAgent;

my $ua = LWP::UserAgent->new(
    ssl_opts => {
        SSL_use_cert => 1,
        SSL_cert_file   => "/path/to/clientcert.crt",
        SSL_key_file    => "/path/to/privatekey.key",
    },
);

However, that didn’t work; changing the paths to the cert/key to non-existent files didn’t cause any difference, so I suspected that those options were actually being ignored.

After a fair bit of digging, the option I found that actually worked was loading Net::SSL first, to make LWP use Net::SSLeay, and setting env vars to the client cert to use:

use Net::SSL;
use LWP::UserAgent;

$ENV{HTTPS_CERT_FILE} = "/path/to/clientcert.crt";
$ENV{HTTPS_KEY_FILE}  = "/path/to/privatekey.key";
my $ua = LWP::UserAgent->new();

This, to me, is pretty icky – I’d much rather pass config to affect just that single LWP object. However, it gets it working.

Wibbles

Vodafone, why you no activate new contract SIM?

2 Comments

Vodafone have always been really good for me – a solid, reliable network and good customer service. That’s why both my personal phone and my wife’s phone are on Vodafone contracts.

Over a week ago, though, my wife’s phone stopped connecting to the network. I spoke to Vodafone, and they explained her SIM card was “too old to be updated” (granted, it was many years old), and manually did something to get it to reconnect, but said it needed replacing, and sent out a replacement.

When I received the new SIM I phoned to activate it – that seemed to go OK, but it never actually worked, and the phone could not connect with the new SIM. I phoned back 24 hours later, and was told the activation failed, and they went through the process again; it failed that time, too.

Continue reading Vodafone, why you no activate new contract SIM? →

Wibbles

We shouldn’t arrest criminals anymore?

What the fuck?

From a BBC news story, the grandfather of a murdered youth argues that anti-social youths should not have been arrested:

The grandfather of a Luton cyclist killed in a “targeted attack” has said arrests at the scene of the death will not ease growing tensions.

Seven youths were held for public order and drugs offences at a temporary memorial for Delaney Brown in Vincent Road, Luton on Wednesday night.
[…]
Delaney Brown Senior said the men “need counselling, not arresting” but police said “appropriate action” was required.
[…]
At a community meeting on Thursday, Mr Brown’s grandfather said police should have called community leaders to the scene.

I’m sorry – these youths were hanging around, drinking, possessing drugs (allegedly) and acting in an anti-social manner, and hurling rocks and bottles at police, and this man thinks they should not be arrested, but should be “counselled” instead? What the actual fuck? If you act in that way, you should, nay, must be arrested. Emotional tension after a gang member was murdered (apparently in a dispute between two families, if previous stories are to be believed) does not excuse such behaviour.

Geeky

Testing Samsung auto-dial shortcode exploit on Galaxy Note

1 Comment

It’s been widely reported today that the Samsung Galaxy S3 and other Samsung Galaxy phones have a glaring vulnerability: a webpage can trigger the Samsung dialer to dial a code which wipes the phone.

Example reports:

Gaping Hole in TouchWiz UI is Wiping Samsung Androids Clean (dailytech.com)


Samsung Galaxy S3 can be wiped and hard-reset with a single line of HTML (reddit.com)

Security Bug Can Wipe Out Your Android Phone By Visiting a Web Page (gizmodo.com)

I wanted to see if this was really true, so I thought I’d knock up a proof of concept using a much safer short-code, *#*#4636#*#*, which, if entered in the dialler, will take you to a testing / control menu where you can change various device settings (it’s useful to know about that hidden trick – but don’t change things unless you know what you’re doing).

So, I created a testmenu.html containing a frame which attempts to load tel:*#*#4636#*#*; it does indeed call up the dialler, but does not appear to actually trigger the test menu. (This debug code doesn’t require you to push send to confirm it – when you type the last “#”, the menu pops up – this is one reason I chose it as a test, as I assume that the problem with the wipe code (which I’m not testing out on my phone!) is that it works the same way (that, or people push to dial it, not knowing what will happen.)

Next, I decided to try a code that does need you to push send – *#100#. which typically returns your phone number. I created ownnumber.html. Loading that page in my Galaxy Note’s stock browser does indeed launch the dialler again – this time, though, the code *#100# is displayed, ready to be “dialled” if the user desires. This is the behaviour I’d expect from anything that links to tel:$number – the user to be asked for confirmation before placing a “call”.

Is it just that the Galaxy Note doesn’t have the same problem, or is there something special about the “wipe device” code?

Is, perhaps, the test menu code “executing”, but invisibly?

I am somewhat disturbed that Samsung modified the stock browser to support tel: URLs; I could see them being mildly useful for actual links to click to bring up the dialler, but using them as the source for a frame / image would never make sense.

(For what it’s worth, this is a Samsung Galaxy Note (GT-N7000) running Android 4.0.4, on UK Vodafone.)

Linux, Programming, System Administration

Extract part of a Subversion repo into a Git repo

A few times I’ve wanted to extract part of a large monolithic Subversion repository out into a seperate Git repo, but maintain the commit history.

Here’s how I do it.

First, I set up a mapping of Subversion user => Username in a file, so that the committer can match up easily via GitHub etc – each committer should have an entry like the below, one per line:


davidp = David Precious

Now, I clone the entire Subversion repo via git svn into a new git repository:


# Clone the Subversion repo into a new Git repo:
# (~/subversion_authors.txt is file mentioned above)
git svn clone file:///shared/svn/scripts --no-metadata -A ~/subversion_authors.txt tmp/scripts-repo-tmp

Some tags get added during this process, I believe; I don’t need/want to preserve them, so I remove any and all tags:


# remove tags - we don't need them
git tag -l | xargs git tag -d

Now, the clever part; using git filter-branch to select the path within the repo I want to preserve, and remove everything else, promoting the desired path to the “root” of the repository:


# remove all except a given path:
git filter-branch --tag-name-filter cat --prune-empty --subdirectory-filter path/to/desired/dir HEAD

In the above, path/to/desired/dir is the path within the repo that I want to move to the root of the repo; everything else will be discarded.

At this point, I can add a GitHub repository via git remove add origin $url, and push the new repository.

I *think*, because I pushed to GitHub, then deleted my temporary repo and cloned back down, that unrelated previous commits were automatically removed. In case that’s not true, though, the following ought to purge unrelated commits from the new Git repo:


git reset --hard
git for-each-ref --format="%(refname)" refs/original/ | xargs -n 1 git update-ref -d
git reflog expire --expire=now --all
git gc --aggressive --prune=now

Humour

Olympic Goatse ftw.

1 Comment

Excellent logo suggestion for the London 2012 Olympics:

It would certainly, ahem, stretch the boundaries of the games and represent the great opening available… no, I’ll stop there.

I’m not sure whether an Olympic Goatse would be better or worse than the chosen “Lisa Simpson giving a blowjob” logo:

See also the idiotic terms of use of their website, which remind me of the Irish Cancer Society’s link request form from a few years ago. (For any eager solicitors reading this, I followed a link to the terms of use on their site, read enough to decide I disagree with them, and discontinued use of their site. So nur.)

Wibbles

US wants to extradite UK student Richard O’Dwyer on bogus claims

Richard O’Dwyer, a 24-year old UK student, started a website when he was 22 which carried links to external sources where people could watch U.S. TV and movies online. The website in question, TVShack, acted as a search engine, allowing people to search user-submitted links. None of the allegedly copyright-infringing content was hosted or uploaded by the site or by Richard.

Now he is being targeted for extradition by U.S. Immigration and Customs Enforcement, which has decided to make O’Dwyer its prime target in its battle against digital copyright infringement. O’Dwyer has been charged with criminal infringement of copyright, and conspiracy to commit criminal infringement of copyright. Each carries a maximum five-year prison sentence.

Richard is a UK citizen, his website was not hosted in the US and the alleged offenses were not comitted on US soil, so what fucking right do the US have to try to demand he be extradited?

In general, search engines are not responsible for the content of external sites they link to; sites which carefully moderate al user-submitted content can sometimes be considered more responsible for the content which remains, as leaving it there could be construed as an approval of it. However, the “content” in this case was simply links to material elsewhere; there is no clear direct infringement case to answer as far as I can see.

Even if it was agreed that Richard’s website aided the distribution of copyright-infringing material, hauling him to a country irrelevant to the actions in question to face charges which could lead to up to ten years in jail? Seriously, what the fuck?

The Guardian reports:

However, the US authorities became concerned about a site linking to content often still within copyright. To sell a counterfeit CD or DVD of a copyrighted work is an offence, as is deliberately uploading such a work to the internet.

American customs officials, after campaigning from industry bodies, contended that linking to such items on other sites (as search engines and others automatically do) would also be covered by such laws.

This is a contentious interpretation of the law, even in the US, where linking has in some court cases been regarded as protected speech under the first amendment. Part of the reason for the huge backlash against proposed copyright laws, the Stop Online Piracy Act (Sopa) and the Protect [Intellectual Property] Act (Pipa) was that this provision would come under attack.

O’Dwyer says he hadn’t really considered the legality of his site – he didn’t know much about copyright, and knew he was only posting users’ links to material hosted elsewhere – but did comply with legal notices from publishers asking him to remove links, on the few occasions he received them.

So, for a minor transgression, the US want to be able to haul him out of his own country to the US, and face potential way-over-the-top prison time? (He was already held in Wandsworth prison whilst awaiting bail.) Long-term prison time for a minor offense committed by a young student? Really, US, really?

I really hope the UK doesn’t allow this to happen. The US ICE need a quite simple response to be provided: Fuck Off. However, apparently home secretary Theresa May, who must clear all UK/US extraditions, has already given her approval to it. His appeal therefore now depends on a high court hearing, due later this year.

There is a petition to the UK home office to stop this extradition.

David Precious – professional Perl developer, motorcyclist and beer drinker