To Horms' Home Page

This page is a collection of news feeds from friends of mine and myself. Its not, as the name might imply, a planet of Horms, or for the most part even stuff written by me. For that, please go here.

Nearby Planets: Planet SLUG, Planet Linux Australia, Planet FP-Syd
Subscriptions: Adrian Chadd Alan Robertson Alexander Reeder Amanda Reid Anand Kumria Andrew Cowie Andy Fitzsimon Benno Chizuko Horman Chris DiBona Chris Yeoh Craige McWhirter Dave Miller Dave Ruys David Luyer Erik de Castro Lopo Horms Hugh Blemmings James Morris Jan Schmidt Jaq Jeff Waugh Jeremy Kerr John Ferlito Joseph Arruda Kfish Mark Greenaway Martin Pool Mary Gardiner Matt Palmer Mikey Neuling Nick Jenkins Ozone Pete Ryland Peter Hardy Peter Nixon Pia Waugh Raster Raz Real World Haskell Robert Collins Roger Barnes Russell Coker Rusty Sam Johnston Silvia Pfeiffer Ted T'so Tom Davies Tong Master Tractorgen - Commits Tridge Wichert Akkerman fusion94 あや

-----

February 05, 2012

Russell Coker

Reliability of RAID

ZDNet has an insightful article by Robin Harris predicting the demise of RAID-6 due to the probability of read errors [1]. Basically as drives get larger the probability of hitting a read error during reconstruction increases and therefore you need to have more redundancy to deal with this. He suggests that as of 2009 drives were too big for a reasonable person to rely on correct reads from all remaining drives after one drive failed (in the case of RAID-5) and that in 2019 there will be a similar issue with RAID-6.

Of course most systems in the field aren’t using even RAID-6. All the most economical hosting options involve just RAID-1 and RAID-5 is still fairly popular with small servers. With RAID-1 and RAID-5 you have a serious problem when (not if) a disk returns random or outdated data and says that it is correct, you have no way of knowing which of the disks in the set has good data and which has bad data. For RAID-5 it will be theoretically possible to reconstruct the data in some situations by determining which disk should have it’s data discarded to give a result that passes higher level checks (EG fsck or application data consistency), but this is probably only viable in extreme cases (EG one disk returns only corrupt data for all reads).

For the common case of a RAID-1 array if one disk returns a few bad sectors then probably most people will just hope that it doesn’t hit something important. The case of Linux software RAID-1 is of interest to me because that is used by many of my servers.

Robin has also written about some NetApp research into the incidence of read errors which indicates that 8.5% of “consumer” disks had such errors during the 32 month study period [2]. This is a concern as I run enough RAID-1 systems with “consumer” disks that it is very improbable that I’m not getting such errors. So the question is, how can I discover such errors and fix them?

In Debian the mdadm package does a monthly scan of all software RAID devices to try and find such inconsistencies, but it doesn’t send an email to alert the sysadmin! I have filed Debian bug #658701 with a patch to make mdadm send email about this. But this really isn’t going to help a lot as the email will be sent AFTER the kernel has synchronised the data with a 50% chance of overwriting the last copy of good data with the bad data! Also the kernel code doesn’t seem to tell userspace which disk had the wrong data in a 3-disk mirror (and presumably a RAID-6 works in the same way) so even if the data can be corrected I won’t know which disk is failing.

Another problem with RAID checking is the fact that it will inherently take a long time and in practice can take a lot longer than necessary. For example I run some systems with LVM on RAID-1 on which only a fraction of the VG capacity is used, in one case the kernel will check 2.7TB of RAID even when there’s only 470G in use!

The BTRFS Filesystem

The btrfs Wiki is currently at btrfs.ipv5.de as the kernel.org wikis are apparently still read-only since the compromise [3]. BTRFS is noteworthy for doing checksums on data and metadata and for having internal support for RAID. So if two disks in a BTRFS RAID-1 disagree then the one with valid checksums will be taken as correct!

I’ve just done a quick test of this. I created a filesystem with the command “mkfs.btrfs -m raid1 -d raid1 /dev/vg0/raid?” and copied /dev/urandom to it until it was full. I then used dd to copy /dev/urandom to some parts of /dev/vg0/raidb while reading files from the mounted filesystem – that worked correctly although I was disappointed that it didn’t report any errors, I had hoped that it would read half the data from each device and fix some errors on the fly. Then I ran the command “btrfs scrub start .” and it gave lots of verbose errors in the kernel message log telling me which device had errors and where the errors are. I was a little disappointed that the command “btrfs scrub status .” just gave me a count of the corrected errors and didn’t mention which device had the errors.

It seems to me that BTRFS is going to be a much better option than Linux software RAID once it is stable enough to use in production. I am considering upgrading one of my less important servers to Debian/Unstable to test out BTRFS in this configuration.

BTRFS is rumored to have performance problems, I will test this but don’t have time to do so right now. Anyway I’m not always particularly concerned about performance, I have some systems where reliability is important enough to justify a performance loss.

BTRFS and Xen

The system with the 2.7TB RAID-1 is a Xen server and LVM volumes on that RAID are used for the block devices of the Xen DomUs. It seems obvious that I could create a single BTRFS filesystem for such a machine that uses both disks in a RAID-1 configuration and then use files on the BTRFS filesystem for Xen block devices. But that would give a lot of overhead of having a filesystem within a filesystem. So I am considering using two LVM volume groups, one for each disk. Then for each DomU which does anything disk intensive I can export two LVs, one from each physical disk and then run BTRFS inside the DomU. The down-side of this is that each DomU will need to scrub the devices and monitor the kernel log for checksum errors. Among other things I will have to back-port the BTRFS tools to CentOS 4.

This will be more difficult to manage than just having an LVM VG running on a RAID-1 array and giving each DomU a couple of LVs for storage.

BTRFS and DRBD

The combination of BTRFS RAID-1 and DRBD is going to be a difficult one. The obvious way of doing it would be to run DRBD over loopback devices that use large files on a BTRFS filesystem. That gives the overhead of a filesystem in a filesystem as well as the DRBD overhead.

It would be nice if BTRFS supported more than two copies of mirrored data. Then instead of DRBD over RAID-1 I could have two servers that each have two devices exported via NBD and BTRFS could store the data on all four devices. With that configuration I could lose an entire server and get a read error without losing any data!

Comparing Risks

I don’t want to use BTRFS in production now because of the risk of bugs. While it’s unlikely to have really serious bugs it’s theoretically possible that as bug could deny access to data until kernel code is fixed and it’s also possible (although less likely) that a bug could result in data being overwritten such that it can never be recovered. But for the current configuration (Ext4 on Linux software RAID-1) it’s almost certain that I will lose small amounts of data and it’s most probable that I have silently lost data on many occasions without realising.

Related posts:

  1. Some RAID Issues I just read an interesting paper titled An Analysis of...
  2. ECC RAM is more useful than RAID A common myth in the computer industry seems to be...
  3. Software vs Hardware RAID Should you use software or hardware RAID? Many people claim...


February 05, 2012 02:46 PM

Mary Gardiner

Like advice columns? Check out Captain Awkward

I keep meaning to send the link to individual people I know, but then encountering a crucial etiquette problem, being that one cannot say “here’s an advice column you might like” without being heard as “here’s an advice column YOU NEED TO LISTEN TO BECAUSE YOU OUGHT TO FIX YOUR LIFE DO YOU HEAR ME?”

A broadcast medium is obviously the solution. Captain Awkward. Blogger gives advice, mostly about boundries.


February 05, 2012 11:35 AM

Dave Ruys

rainbowhill: gotitforcheap: shortformblog: Fun guy chillin’...



rainbowhill:

gotitforcheap:

shortformblog:

Fun guy chillin’ in South American rainforest finds plastic-eating fungi

Seriously, though this is kind of a big deal. Know that big problem we have? You know, the one involving a crapload of used plastic hanging around in landfills with nowhere to biodegrade for a couple million years? Well, Jonathan Russell might’ve solved that problem. See, Russell and his fellow Yale students went to Ecuador, where they found a new kind of fungus they’re calling Pestalotiopsis microspora. Big deal, you’re thinking. Anyone can find fungus anywhere! Well, something his fellow students found out after the fact is that this fungus can live on a diet of polyurethane alone — and even crazier, it doesn’t even need air to do so! In other words, we could potentially put it at the bottom of a landfill and cover it with plastic, and it would do the rest of the work. This might be game-changing if it works as advertised. (photo via Flickr user dbutt; EDIT: Updated with link to research abstract) source

Follow ShortFormBlog

SCIENCE IN THIS MUTHAFUCK 

I wonder what the by-products are. Yoga mats and water bottles?

This is big stuff! The more plastic-consuming organisms we can find, the better.

Mental note: don’t go downhill skateboarding through this part of South America.

“Dude, your wheels are lunched! What happened?!”

“Ran over a mushroom…”


February 05, 2012 08:05 AM

-----

February 04, 2012

Pia Waugh

Twitter Weekly Updates for 2012-02-05


February 04, 2012 11:00 PM

Twitter Weekly Updates for 2012-02-05


February 04, 2012 11:00 PM

-----

February 03, 2012

Russell Coker

A Computer Conference on a Cruise Ship

After LCA [1] there was a discussion about possible locations for future conferences, most of the messages in the discussion were jokes or suggestions that don’t seriously apply to LCA. So I’ll add my suggestion for conferences other than LCA.

I’ve previously written generally about the issue of conferences at sea [2]. I don’t think that LCA would be suitable for running at sea because delegates have specific expectations for LCA which are quite different to what a cruise ship can offer, so I don’t think it makes sense to change LCA which is working well as it is. However there are lots of other possible computer conferences which could suite a cruise ship.

Price

Price is a major factor in running a conference, so obviously getting a cheap cruise price is very important. Here is a link for Vacations To Go which shows cruises from the Australia/NZ region which are of at least 5 nights and cost no more than $800 [3]. The cheapest entry at this moment is $609 for 5 nights and the cheapest on a per-night basis is an 8 night cruise for $779. The cheapest cruise currently on offer which allows a conference similar to LCA is 7 nights for $699. The prices should be regarded as rough approximations as some cruises have some mandatory extra fees and the prices are quoted in US dollars and subject to currency fluctuations. Note that those prices are for dual-occupancy cabins, this can be a “double” or a “twin” configuration. Some cruise ships have cabins for 3 or 4 people that are cheaper, but if you have a cabin for a single person then the rate is almost the same as for having two people.

The price for LCA accommodation including breakfast was $78 per night for a single room or $92 for a double room. Then lunch cost a minimum of $10 and for dinner there was $80 for the penguin dinner and probably about $20 for dinner every other night. That gave an overall cost for a 6 night stay (which is probably the minimum for someone who lives further away than Melbourne) in Ballarat of 6*78+6*10+5*20+80==$708. For a double room that would be 6*92+6*10+5*20+2*80==$872.

Even if we don’t count the fact that the Australian dollar is worth more than the US dollar it is obvious that on the basis of accommodation and food two people sharing a twin cabin on a cruise ship could pay LESS than two people in single rooms at the Ballarat University dorms! Now sharing a cabin isn’t so great, but the upside is that cruise ships have excellent food and lots of other entertainment options. I previously reviewed the food on the Dawn Princess and determined that it’s better than the food I would expect to get if I spent the cost of the cruise on dinner at land based restaurants [4].

I have been led to believe that the use of ship conference facilities is typically free for any organisation that books a sufficient number of cabins. So there’s no reason why the conference admission fees should be any greater than for a land based conference.

Advantages

A common problem with conferences is finding suitable dining options. Most people want to eat with other delegates but finding restaurants that have sufficient space and which are conveniently located is difficult at best and often impossible. On a cruise ship everything is within a short walk and the restaurants are big, usually be at least one restaurant will hold 500 people. The fact that you have to reserve times for the “Main Dining Room” makes it more difficult to miss one’s colleagues.

Everything on a cruise ship is luxurious.

There are lots of good locations for BoFs, pools, cafes, restaurants, and bars. Basically the ship is filled with comfortable places for groups of people to sit down.

A cruise ship typically has a main theater with more than 700 seats – more than large enough for most conferences I’ve attended. It’s common for the size of a conference to be limited to the size of the main theater that is used, for a cruise ship this will probably be less of a problem than for most other conference venues.

Disadvantages

The first disadvantage of running a computer conference on a cruise ship is the almost total lack of net access. The costs for net access are more expensive than most delegates will pay. Probably many delegates would check their email but it wouldn’t be practical for people to download source code, browse Wikipedia, and use the Internet in other ways related to the conference. It would be practical to have mirrors of Wikipedia, the source of several distributions of Linux, and other big things of common interest.

Another possible problem is the fact that you need to book it well in advance to avoid the risk of selling out (there is no option to stay at a different hotel). An established conference with financial backing could just pay to reserve the cabins. But when starting a new conference this could be a problem.

Alcohol is rather expensive on cruise ships. But getting really drunk isn’t compatible with learning about computer science anyway.

Finally the requirement to have at least two people in a cabin for good rates is a serious issue. The upside of this is that people travelling with their SO would find that it works really well (regardless of whether the SO is a delegate or not). But anyone who’s not travelling with their SO and doesn’t want to share with a friend will have to either pay a lot more or skip the conference.

Conclusion

I think that there is a good potential for running a computer conference around the Australia/NZ region on a cruise ship. It won’t be overly expensive for delegates and the facilities that are provided are good. The trade-off for solitary travelers of having to share a cabin (or pay more) for getting much better food and leisure facilities will be appreciated by many people (and admittedly hated by some).

Some people won’t appreciate the option of swimming, but even if you consider the cruise ship to be just a floating collection of restaurants and cabins it’s still fairly luxurious and beats the heck out of most conferences I’ve attended.

If you are considering the possibility of running a conference then I think that a cruise ship should be considered. VacationsToGo.com is the best site I’ve found for cheap cruise prices, their large group department has experience handling groups of more than 500 people so I think that anyone who wants to run a new conference in/around Australia should give them a call.

Also cruise ships travel around the world, so the same thing can be done in other countries but at a different time of year. The economic factors will differ by country though. Cruise ships probably aren’t a cheap option for a conference in some other countries.

Related posts:

  1. My First Cruise A few weeks ago I went on my first cruise,...
  2. Cruises It seems that in theory cruises can make for quite...
  3. Creating a Micro Conference The TEDxVolcano The TED conference franchise has been extended to...


February 03, 2012 11:17 AM

-----

February 01, 2012

Raz

Using pulseaudio remotely via ssh

It’s a little untidy and socat fails to exit when ssh does, but: UPDATE 2012-02-04: I’d left out the -L option…


February 01, 2012 05:15 AM

-----

January 30, 2012

Andrew Cowie

My sound hardware didn’t vanish, honest

I’ve been having intermittent problems with sound not working. Usually restarting (ie, killing) PulseAudio has done the trick but today it was even worse; the sound hardware mysteriously vanished from the Sound Settings capplet. Bog knows what’s up with that, but buried in “Sound Troubleshooting” I found “Getting ALSA to work after suspend / hibernate” which contains this nugget:

The alsa “force-reload” command will kill all running programs using the sound driver so the driver itself is able to be restarted.

Huh. Didn’t know about that one. But seems reasonable, and sure enough,

$ /sbin/alsa force-reload

did the trick.

That wiki page goes on to detail adding a script to /etc/pm/sleep.d to carry this out after every resume. That seems excessive; I know that sometimes drivers don’t work or hardware doesn’t reset after the computer has been suspended or hibernated, but in my case the behaviour is only intermittent, and seems related to having docked (or not), having used an external USB headphone (or not), and having played something with Flash (which seems to circumvent PulseAudio. Bad). Anyway, one certainly doesn’t want to kill all one’s audio-using programs just because you suspended! But as a workaround for whatever it is that’s wrong today, nice.

AfC


January 30, 2012 07:10 AM

-----

January 28, 2012

Pia Waugh

Twitter Weekly Updates for 2012-01-29


January 28, 2012 11:00 PM

Twitter Weekly Updates for 2012-01-29


January 28, 2012 11:00 PM

Rusty

Why Everyone Must Oppose The Merging of /usr and /

As co-editor of the last edition of the File Hierarchy Standard before it merged into the Linux Standard Base, I’ve been following the discussion about combining the directories  /bin, /sbin and /lib into /usr/bin, /usr/sbin and /usr/lib respectively.  You can follow it too, via the LWN discussion.

To summarize, there are two sides to the debate.  The “pro” side points out:

  1. Nothing will really change for users, as symlinks will make old stuff still work.
  2. There are precedents in Solaris and Fedora.
  3. The weak reasonings used previously to separate / and /usr no longer apply.
  4. Separate /usr has become increasingly unsupported anyway.
  5. Moving to /usr will enable genuine R/O root filesystem sharing.

The “anti” side, however, raises very salient points:

  1. Lennart Poettering supports it.
  2. Lennart Poettering is an asshole.

Fellow Anti-mergers, I understand the pain and anguish that systemd has caused you personally, and your families.  Your hopes and dreams crushed, by someone with all the charm of a cheese grater across the knuckles.  Your remaining life tainted by this putrescent subhuman who forced himself upon your internet.

Despite the privation we have all endured, please find strength to stop this nightmarish ravaging of our once-pure filesystems.  For if he’s not stopped now, what hope for  /usr/sbin vs /usr/bin?


January 28, 2012 05:04 AM

-----

January 26, 2012

Jeff Waugh

QotD: Jon Corbet on linux.conf.au and Linux Australia

In summary, LCA remains unique in its combination of strongly technical talks, freedom-oriented and hands-on orientation, wide variety of topics covered, and infectious Australian humor. There is a reason some of us seem to end up there every year despite the painful air-travel experiences required. Linux Australia has put together a structure that allows the conference to be handed off to a new team in a new city every year, bringing a fresh view while upholding the standards set in the previous years.

– LWN’s Jon Corbet on linux.conf.au, An LCA 2012 Summary


January 26, 2012 05:52 AM

Russell Coker

Links January 2012

Cops in Tennessee routinely steal cash from citizens [1]. They are ordered to do so and in some cases their salary is paid from the cash that they take. So they have a good reason to imagine that any large sum of money is drug money and take it.

David Frum wrote an insightful article for NY Mag about the problems with the US Republican Party [2].

TreeHugger.com has an interesting article about eco-friendly features on some modern cruise ships [3].

Dan Walsh describes how to get the RSA SecureID PAM module working on a SE Linux system [4]. It’s interesting that RSA was telling everyone to turn off SE Linux and shipping a program that was falsely marked as needing an executable stack and which uses netstat instead of /dev/urandom for entropy. Really the only way RSA could do worse could be to fall victim to an Advanced Persistent Attack… :-#

The Long Now has an interesting summary of a presentation about archive.org [5]. I never realised the range of things that archive.org stores, I will have to explore that if I find some spare time!

Jonah Lehrer wrote a detailed and informative article about the way that American high school students receive head injuries playing football[6]. He suggests that it might eventually be the end of the game as we know it.

François Marier wrote an informative article about optimising PNG files [7], optipng is apparently the best option at the moment but it doesn’t do everything you might want.

Helen Keeble wrote an interesting review of Twilight [8]. The most noteworthy thing about it IMHO is that she tries to understand teenage girls who like the books and movies. Trying to understand young people is quite rare.

Jon Masters wrote a critique of the concept of citizen journalism and described how he has two subscriptions to the NYT as a way of donating to support quality journalism [9]. The only comment on his post indicates a desire for biased news (such as Fox) which shows the reason why most US media is failing at journalism.

Luis von Ahn gave an interesting TED talk about crowd-sourced translation [10]. He starts by describing CAPTCHAs and the way that his company ReCAPTCHA provides the CAPTCHA service while also using people’s time to digitise books. Then he describes his online translation service and language education system DuoLingo which allows people to learn a second language for free while translating text between languages [11]. One of the benefits of this is that people don’t have to pay to learn a new language and thus poor people can learn other languages – great for people in developing countries that want to learn first-world languages! DuoLingo is in a beta phase at the moment but they are taking some volunteers.

Cory Doctorow wrote an insightful article for the Publishers Weekly titles “Copyrights vs Human Rights” [12] which is primarily about SOPA.

Naomi Wolf wrote an insightful article for The Guardian about the “Occupy” movement, among other things the highest levels of the US government are using the DHS as part of the crackdown [13]. Naomi’s claim is that the right-wing and government attacks on the Occupy movement are due to the fact that they want to reform the political process and prevent corruption.

John Bohannon gave an interesting and entertaining TED talk about using dance as part of a presentation [14]. He gave an example of using dancerts to illustrate some concepts related to physics and then spoke about the waste of PowerPoint.

Joe Sabia gave an amusing and inspiring TED talk about the technology of storytelling [15]. He gave the presentation with live actions on his iPad to match his words, a difficult task to perform successfully.

Thomas Koch wrote an informative post about some of the issues related to binary distribution of software [16]. I think the problem is evenm worse than Thomas describes.

Related posts:

  1. Links January 2011 Halla Tomasdottir gave an interesting TED talk about her financial...
  2. Links January 2010 Magnus Larsson gave an interesting TED talk about using bacteria...
  3. Links January 2009 Jennifer 8 Lee gave an interesting TED talk about the...


January 26, 2012 01:49 AM

-----

January 25, 2012

Russell Coker

SE Linux Status in Debian 2012-01

Since my last SE Linux in Debian status report [1] there have been some significant changes.

Policy

Last year I reported that the policy wasn’t very usable, on the 18th of January I uploaded version 2:2.20110726-2 of the policy packages that fixes many bugs. The policy should now be usable by most people for desktop operations and as a server. Part of the delay was that I wanted to include support for systemd, but as my work on systemd proceeded slowly and others didn’t contribute policy I could use I gave up and just released it. Systemd is still a priority for me and I plan to use it on all my systems when Wheezy is released.

Kernel

Some time between Debian kernel 3.0.0-2 and 3.1.0-1 support for an upstream change to the security module configuration was incorporated. Instead of using selinux=1 on the kernel command line to enable SE Linux support the kernel option is security=selinux. This change allows people to boot with security=tomoyo or security=apparmor if they wish. No support for Smack though.

As the kernel silently ignores command line parameters that it doesn’t understand so there is no harm in having both selinux=1 and security=selinux on both older and newer kernels. So version 0.5.0 of selinux-basics now adds both kernel command-line options to GRUB configuration when selinux-activate is run. Also when the package is upgraded it will search for selinux=1 in the GRUB configuration and if it’s there it will add security=selinux. This will give users the functionality that they expect, systems which have SE Linux activated will keep running SE Linux after a kernel upgrade or downgrade! Prior to updating selinux-basics systems running Debian/Unstable won’t work with SE Linux.

As an aside the postinst file for selinux-basics was last changed in 2006 (thanks Erich Schubert). This package is part of the new design of SE Linux in Debian and some bits of it haven’t needed to be changed for 6 years! SE Linux isn’t a new thing, it’s been in production for a long time.

Audit

While the audit daemon isn’t strictly a part of SE Linux (each can be used without the other) it seems that most of the time they are used together (in Debian at least). I have prepared a NMU of the new upstream version of audit and uploaded it to delayed/7. I want to get everything related to SE Linux up to date or at least with comparable versions to Fedora. Also I sent some of the Debian patches for the auditd upstream which should reduce the maintenance effort in future.

Libraries

There have been some NMUs of libraries that are part of SE Linux. Due to a combination of having confidence in the people doing the NMUs and not having much spare time I have let them go through without review. I’m sure that I will notice soon enough if they don’t work, my test systems exercise enough SE Linux functionality that it would be difficult to break things without me noticing.

Play Machine

I am now preparing a new SE Linux “Play Machine” running Debian/Unstable. I wore my Play Machine shirt at LCA so I’ve got to get one going again soon. This is a good exercise of the strict features of SE Linux policy, I’ve found some bugs which need to be fixed. Running Play Machines really helps improve the overall quality of SE Linux.

Related posts:

  1. Status of SE Linux in Debian LCA 2009 This morning I gave a talk at the Security mini-conf...
  2. SE Linux in Debian I have now got a Debian Xen domU running the...
  3. Debian SE Linux Status At the moment I’ve got more time to work on...


January 25, 2012 11:36 AM

Pia Waugh

Moving on – the journey continues

Today is a complicated day. I’m both sad and excited in equal measure about what this year may hold in store.

I’m sad because this is my last day working for Senator Kate Lundy as her IT Policy Advisor and inhouse geek. Kate headhunted me almost 3 years ago at BarCamp Canberra, though we had known each other for a few years beforehand from when she was the Shadow Minister for IT. I was quite wary of going to work in a political office, but my curiosity about how the machine works combined with a desire to help make good tech policy and an immense amount of respect for Kate brought me into one of the most interesting, fun and challenging jobs I’ve ever had.

I particularly wanted to better understand the legislative and executive arms of government. How ideas turn into policy and policy into implementation. As a result, along with doing my job I’ve spent time researching the history of democracy, of Australian politics, of the ideological and historical premise of all the major Australian parties and the interaction between party politics and democracy over the years.  I’ve also spent time coming to understand some of the layout, responsibilities and challenges of a multi-tiered system of government.

I have learnt a great deal in this job about government, but also about human nature. Working in an electoral office gives one some insight to the difficulties faced by many, but also some insight to the challenge in maintaining a constructive and respectful dialogue. I think it is human nature to try to boil issues down to black and white. But we are essentially grey creatures with enormous complexity, and I think democracy is about finding ways to have a transparent, informed, respectful and constructive dialogue with all the people on complex policies and implementation, so governments can best implement the best policies for the communities they serve.

I have been lucky to work for a politician who is passionate and knowledgeable about technology and good policy. She has been a valuable teacher and mentor. I shall always be thankful for the wisdom, patience, compassion, critical thinking, strategy and policy development I have learnt in this role and from Kate. I’m sure these skills will continue to serve me well.

My work on Kate’s website, the Public Spheres, Open Government, assisting Kate in linking together different tech policies across a variety of portfolios are all things I am proud of. I also feel very lucky to have met and worked with such inspirational people from many different walks of life through this role and in Kate’s office.

Meanwhile, having developed some understanding of the legislative and executive arms of government, I realised that I wanted to have more experience in the administrative arm of government. I had done some tech work in a previous life within departments but always as the outsourced person. I knew I wanted to really get in and contribute to the public service, as well as learn more about the implementation of policy and the delivery of government services to citizens.

As such, I’m excited to say I am hopefully moving into a role in the APS in the coming weeks and I hope my efforts there will be broadly useful to others in the APS. I can’t say more at this stage as it is being finalised at the moment, but I’ll update this post in the weeks ahead with more information.

By working within the APS, I hope to get a better personal understanding of the specific challenges facing the APS with regards to technology, and hopefully assist in developing strategies to be a more agile, responsive and citizen-centric public service. I will also continue helping to move the Open Government agenda ahead both in my own time and, where appropriate, within my new role. My commitment to Open Government (and Gov 2.0) lies in my understanding that it provides a path to a public service and democracy that is most relevant to, engaged with, responsive to, representative of and accountable to its citizens.

I’ll finish by saying that after three years in her office, my respect for Kate has only grown. She is a person who has engaged fully in her role with integrity, responsibility, grace and a firm grip on her own principles. She is a politician that makes me believe politics isn’t just a dirty word and I wish we had more like her. Even in spite of the fact the last time I socialised with her, I ended up with a fractured scaphoid! I have learnt a keen respect for the torque of a 2 stroke, especially on a motocross track.

My shiny black carbon fibre cast. Shiny!My shiny black carbon fibre cast. Shiny!

So, I’m diving into the deep end and I look forward to seeing how well I swim. Wish me luck :)


January 25, 2012 07:00 AM

-----

January 24, 2012

Erik de Castro Lopo

Benchmarking and QuickChecking readInt.

I'm currently working on converting my http-proxy library from using the Data.Enumerator package to Data.Conduit (explanation of why in my last blog post).

During this conversion, I have been studying the sources of the Warp web server because my http-proxy was originally derived from the Enumerator version of Warp. While digging through the Warp code I found the following code (and comment) which is used to parse the number provided in the Content-Length field of a HTTP header: 


  -- Note: This function produces garbage on invalid input. But serving an
  -- invalid content-length is a bad idea, mkay?
  readInt :: S.ByteString -> Integer
  readInt = S.foldl' (\x w -> x * 10 + fromIntegral w - 48) 0

The comment clearly states that that this function can produce garbage, specifically if the string contains anything other than ASCII digits. The comment is also correct that an invalid Content-Length is a bad idea. However, on seeing the above code, and remembering something I had seen recently in the standard library, I naively sent the Yesod project a patch replacing the above code with a version that uses the readDec function from the Numeric module: 


  import Data.ByteString (ByteString)
  import qualified Data.ByteString.Char8 as B
  import qualified Numeric as N

  readInt :: ByteString -> Integer
  readInt s =
      case N.readDec (B.unpack s) of
          [] -> 0
          (x, _):_ -> x

About 3-4 hours after I submitted the patch I got an email from Michael Snoyman saying that parsing the Content-Length field is a hot spot for the performance of Warp and that I should benchmark it against the code I'm replacing to make sure there is no unacceptable performance penalty.

That's when I decided it was time to check out Bryan O'Sullivan's Criterion bench-marking library. A quick read of the docs and bit of messing around and I was able to prove to myself that using readDec was indeed much slower than the code I wanted to replace.

The initial disappointment of finding that a more correct implementation was significantly slower than the less correct version quickly turned to joy as I experimented with a couple of other implementations and eventually settled on this: 


  import Data.ByteString (ByteString)
  import qualified Data.ByteString.Char8 as B
  import qualified Data.Char as C

  readIntTC :: Integral a => ByteString -> a
  readIntTC bs = fromIntegral
          $ B.foldl' (\i c -> i * 10 + C.digitToInt c) 0
          $ B.takeWhile C.isDigit bs

By using the Integral type class, this function converts the given ByteString to any integer type (ie any type belonging to the Integral type class). When used, this function will be specialized by the Haskell compiler at the call site to to produce code to read string values into Ints, Int64s or anything else that is a member of the Integral type class.

For a final sanity check I decided to use QuickCheck to make sure that the various versions of the generic function were correct for values of the type they returned. To do that I wrote a very simple QuickCheck property as follows: 


  prop_read_show_idempotent :: Integral a => (ByteString -> a) -> a -> Bool
  prop_read_show_idempotent freader x =
      let posx = abs x
      in posx == freader (B.pack $ show posx)

This QuickCheck property takes the function under test freader and QuickCheck will then provide it values of the correct type. Since the function under test is designed to read Content-Length values which are always positive, we only test using the absolute value of the value randomly generated by QuickCheck.

The complete test program can be found on Github in this Gist and can be compiled and run as: 


  ghc -Wall -O3 --make readInt.hs -o readInt && ./readInt

When run, the output of the program looks like this: 


  Quickcheck tests.
  +++ OK, passed 100 tests.
  +++ OK, passed 100 tests.
  +++ OK, passed 100 tests.
  Criterion tests.
  warming up
  estimating clock resolution...
  mean is 3.109095 us (320001 iterations)
  found 27331 outliers among 319999 samples (8.5%)
    4477 (1.4%) low severe
    22854 (7.1%) high severe
  estimating cost of a clock call...
  mean is 719.4627 ns (22 iterations)

  benchmarking readIntOrig
  mean: 4.653041 us, lb 4.645949 us, ub 4.663823 us, ci 0.950
  std dev: 43.94805 ns, lb 31.52653 ns, ub 73.82125 ns, ci 0.950

  benchmarking readDec
  mean: 13.12692 us, lb 13.10881 us, ub 13.14411 us, ci 0.950
  std dev: 90.63362 ns, lb 77.52619 ns, ub 112.4304 ns, ci 0.950

  benchmarking readRaw
  mean: 591.8697 ns, lb 590.9466 ns, ub 594.1634 ns, ci 0.950
  std dev: 6.995869 ns, lb 3.557109 ns, ub 14.54708 ns, ci 0.950

  benchmarking readInt
  mean: 388.3835 ns, lb 387.9500 ns, ub 388.8342 ns, ci 0.950
  std dev: 2.261711 ns, lb 2.003214 ns, ub 2.585137 ns, ci 0.950

  benchmarking readInt64
  mean: 389.4380 ns, lb 388.9864 ns, ub 389.9312 ns, ci 0.950
  std dev: 2.399116 ns, lb 2.090363 ns, ub 2.865227 ns, ci 0.950

  benchmarking readInteger
  mean: 389.3450 ns, lb 388.8463 ns, ub 389.8626 ns, ci 0.950
  std dev: 2.599062 ns, lb 2.302428 ns, ub 2.963600 ns, ci 0.950

At the top of the output is proof that all three specializations of the generic function readIntTC satisfy the QuickCheck property. From the Criterion output its pretty obvious that the Numeric.readDec version is about 3 times slower that the original function. More importantly, all three version of this generic function are an order of magnitude faster than the original.

That's a win! I will be submitting my new function for inclusion in Warp.

Update : 14:13

At around the same time I submitted my latest version for readInt Vincent Hanquez posted a comment on the Github issue suggesting I look at the GHC MagicHash extension and pointed me to an example.

Sure enough, using the MagicHash technique resulted in something significantly faster again.

Update #2 : 2012-01-29 19:46

In version 0.3.0 and later of the bytestring-lexing package there is a function readDecimal that is even faster than the MagiHash version.


January 24, 2012 12:52 AM

-----

January 21, 2012

Pia Waugh

Twitter Weekly Updates for 2012-01-22


January 21, 2012 11:00 PM

Twitter Weekly Updates for 2012-01-22


January 21, 2012 11:00 PM

-----

January 18, 2012

James Morris

Save the date: 2012 Linux Security Summit, 30-31 August, San Diego

This is a pre-announcement so people can start planning travel for the year.

The Linux Security Summit for 2012 will be held on the 30th and 31st of August in San Diego, CA, USA.  It will be co-located with LinuxCon North America, plumbers and the kernel summit.

More details to follow.


January 18, 2012 12:43 AM

-----

January 16, 2012

James Morris

New git repository for the Linux kernel security subsystem

I’ve set up a new git repository for the Linux kernel security subsystem on the new kernel.org server.

The URLs are:

git://git.kernel.org/pub/scm/linux/kernel/git/jmorris/linux-security.git
http://git.kernel.org/pub/scm/linux/kernel/git/jmorris/linux-security.git
https://git.kernel.org/pub/scm/linux/kernel/git/jmorris/linux-security.git

Developers should work against the “next” branch.

A web-browsable interface via gitweb may be found at:

http://git.kernel.org/?p=linux/kernel/git/jmorris/linux-security.git;a=summary

The temporary repo on selinuxproject.org will go away soon, so please update your repositories.


January 16, 2012 05:02 AM

-----

January 14, 2012

Pia Waugh

Twitter Weekly Updates for 2012-01-15


January 14, 2012 11:00 PM

Twitter Weekly Updates for 2012-01-15


January 14, 2012 11:00 PM

Erik de Castro Lopo

A Simple Telnet Client Using Data.Conduit.

Below is a simple telnet client written using Haskell's new Conduit library. This library was written by Michael Snoyman the man behind the Yesod Web Framework for Haskell.

The Conduit library is a second generation approach to the problem of guaranteeing bounded memory usage in the presence of lazy evaluation. The first generation of these ideas were libraries like Iteratee, Enumerator, and IterIO. All of these first generation libraries use the the term enumerator for data producers and iteratee for data consumers. The new Conduit library calls data producers "sources" and data consumers "sinks" to make them a little more approachable.

The other big difference between Conduit and the early libraries in this space is to do with guaranteeing early clean up of potentially scarce resources like sockets. Although I have not looked in any detail at the IterIO library, both Iteratee and Enumerator simply rely on Haskell's garbage collector to clean up resources when they are no longer required. The Conduit library on the other hand uses Resource transformers to guarantee release of these resources as soon as possible.

The client looks like this (latest available here): 


  import Control.Concurrent (forkIO, killThread)
  import Control.Monad.IO.Class (MonadIO, liftIO)
  import Control.Monad.Trans.Resource
  import Data.Conduit
  import Data.Conduit.Binary
  import Network (connectTo, PortID (..))
  import System.Environment (getArgs, getProgName)
  import System.IO


  main :: IO ()
  main = do
      args <- getArgs
      case args of
          [host, port] -> telnet host (read port :: Int)
          _ -> usageExit
    where
      usageExit = do
          name <- getProgName
          putStrLn $ "Usage : " ++ name ++ " host port"


  telnet :: String -> Int -> IO ()
  telnet host port = runResourceT $ do
      (releaseSock, hsock) <- with (connectTo host $ PortNumber $ fromIntegral port) hClose
      liftIO $ mapM_ (`hSetBuffering` LineBuffering) [ stdin, stdout, hsock ]
      (releaseThread, _) <- with (
                            forkIO $ runResourceT $ sourceHandle stdin $$ sinkHandle hsock
                            ) killThread
      sourceHandle hsock $$ sinkHandle stdout
      release releaseThread
      release releaseSock

There are basically three blocks, a bunch of imports at the top, the program's entry point main and the telnet function.

The telnet function is pretty simple. Most of the function runs inside a runResourceT resource transformer. The purpose of these resources transformers is to keep track of resources such as sockets, file handles, thread ids etc and make sure they get released in a timely manner. For example, in the telnet function, the connectTo function call opens a connection to the specified host and port number and returns a socket. By wrapping the connectTo in the call to with then the socket is registered with the resource transformer. The with function has the following prototype: 


  with :: Resource m
       => Base m a             -- Base monad for the current monad stack
       -> (a -> Base m ())     -- Resource de-allocation function
       -> ResourceT m (ReleaseKey, a)

When the resource is registered, the user must also supply a function that will destroy and release the resource. The with function returns a ReleaseKey for the resource and the resource itself. Formulating the with function this way makes it hard to misuse.

The other thing of interest is that because a telnet client needs to send data in both directions, the server-to-client communication path and the client-to-server communication run in separate GHC runtime threads. The thread is spawned using forkIO and even though the thread identifier is thrown away, the resource transformer still records it and will later call killThread to clean up the thread.

The main core of the program are the two lines containing calls to sourceHandle and sinkHandle. The first of these lines pulls data from stdin and pushes it to the socket hsock while the second pulls from the socket and pushes it to stdout.

It should be noted that the final two calls to release are not strictly necessary since the resource transformer will clean up these resources automatically.

The experience of writing this telnet client suggests that the Conduit library is certainly easier to use than the Enumerator or Iteratee libraries.


January 14, 2012 02:22 AM

-----

January 12, 2012

Dave Ruys

More board mods from the ghetto: Vault Chopmount (below a...



More board mods from the ghetto: Vault Chopmount

(below a standard Comet Voodoo Doll 2)

Playing around with topmount board shapes I decided to chop down an Athena and remount with a shorter wheelbase. Result: crazy responsive livewire freeriding board. So rad.


January 12, 2012 02:21 AM

-----

January 09, 2012

Sam Johnston

Cloud computing's concealed complexity

Cloud gears cropped

James Urquhart claims Cloud is complex—deal with it, adding that "If you are looking to cloud computing to simplify your IT environment, I’m afraid I have bad news for you" and citing his earlier CNET post drawing analogies to a recent flash crash.

Cloud computing systems are complex, in the same way that nuclear power stations are complex — they also have catastrophic failure modes, but given cloud providers rely heavily on their reputations they go to great lengths to ensure continuity of service (I was previously the technical program manager for Google's global tape backup program so I appreciate this first hand). The best analogies to flash crashes are autoscaling systems making too many (or too few) resources available and spot price spikes, but these are isolated and there are simple ways to mitigate the risk (DDoS protection, market limits, etc.)

Fortunately this complexity is concealed behind well defined interfaces — indeed the term "cloud" itself comes from network diagrams in which complex interconnecting networks became the responsibility of service providers and were concealed by a cloud outline. Cloud computing is, simply, the delivery of information technology as a service rather than a product, and like other utility services there is a clear demarcation point (the first socket for telephones, the meter for electricity and the user or machine interface for computing).

Everything on the far side of the demarcation point is the responsibility of the provider, and users often don't even know (nor do they need to know) how the services actually work — it could be an army of monkeys at typewriters for all they care. Granted it's often beneficial to have some visibility into how the services are provided (in the same way that we want to know our phone lines are secure and power is clean), but we've developed specifications like CloudAudit to improve transparency.

Making simple topics complex is easy — what's hard is making complex topics simple. We should be working to make cloud computing as approachable as possible, and drawing attention to its complexity does not further that aim. Sure there are communities of practitioners who need to know how it all works (and James is addressing that community via GigaOm), but consumers of cloud services should finally be enabled to apply information technology to business problems, without unnecessary complexity.

If you find yourself using complex terminology or unnecessary acronyms (e.g. anything ending with *aaS) then ask yourself if you're not part of the problem rather than part of the solution.


January 09, 2012 08:53 AM

Flash/Silverlight: How much business can you afford to turn away?

Tim Anderson was asking about the future of Silverlight on Twitter today so here are my thoughts on the subject, in the context of earlier posts on the future of Flash:

2009: Why Adobe Flash penetration is more like 50% than 99%
2010: Face it Flash, your days are numbered.
2011: RIP Adobe Flash (1996-2011) - now let's bury the dead

In the early days of the Internet, a lack of native browser support for "advanced" functionality (particularly video) created a vacuum that propelled Flash to near ubiquity. It was the only plugin to achieve such deep penetration, though I would argue never as high as 99% (which Adobe laughably advertise to this day). As a result, developers were able to convince clients to adopt the platform for all manner of interactive sites (including, infamously, many/most restaurants).

The impossible challenge for proprietary browser plugins is staying up-to-date and secure across a myriad hardware and software platforms — it was hard enough trying to support multiple browsers on multiple versions of Windows on one hardware platform (x86), but with operating systems like Linux and Mac OS X now commanding non-negligible shares of the market it's virtually impossible. Enter mobile devices, which by Adobe's own reckoning outnumber PCs by 3 to 1. Plugin vendors now have an extremely diverse ecosystem of hardware (AMD, Intel, etc.) and software (Android, iOS, Symbian, Windows Phone 7, etc.) and an impossibly large number of permutations to support. Meanwhile browser engines (e.g. WebKit, which is the basis for Safari and Chrome on the desktop and iOS, Android and webOS on mobile devices) have added native support for the advanced features whose absence created a demand for Flash.

Unsurprisingly, not only is Flash in rapid decline — as evidenced by Adobe recently pulling out of the mobile market (and thus 3 in 4 devices) — but it would be virtually impossible for any competitor to reach its level of penetration. As such, Silverlight had (from the outset) a snowflake's chance in hell of achieving an "acceptable" level of penetration.

What's an "acceptable level of penetration" you ask? That's quite simple — it's the ratio of customers that businesses are prepared to turn away in order to access "advanced" functionality that is now natively supported in most browsers. At Adobe's claimed 99% penetration you're turning away 1 in 100 customers. At 90% you're turning away 1 in 10. According to http://riastats.com, if you're deploying a Flash site down under then you're going to be turning away 13%, or a bit more than 1 in 8. For Silverlight it's even worse — almost half of your customers won't even get to see your site without having to install a plugin (which they are increasingly less likely to do).

How much revenue can your business tolerate losing? 1%? 10%? 50%? And for what benefit?


January 09, 2012 08:00 AM

-----

January 08, 2012

Dave Ruys

VGF: Vault Ghetto Freeride. Some bunch of wack-jobs cut the...



VGF: Vault Ghetto Freeride.

Some bunch of wack-jobs cut the oversized lips off a set of Vault proto wheels to make some seriously rad ghetto-lathed freeride slide wheels. That wannabe-thug at the start looks dodgy as. I’d be keeping well clear of him….and his bastard file….


January 08, 2012 11:48 AM

Russell Coker

My First Cruise

A few weeks ago I went on my first cruise, from Sydney to Melbourne on the Dawn Princess. VacationsToGo.com (a discount cruise/resort web site) has a review of the Dawn Princess [1], they give it 4 stars out of a possible 6. The 6 star ships seem to have discount rates in excess of $500 per day per person, much more than I would pay.

The per-person rate is based on two people sharing a cabin, it seems that most cabins can be configured as a double bed or twin singles. If there is only one person in a cabin then they pay almost double the normal rate. It seems that most cruise ships have some support for cabins with more than two people (at a discount rate), but the cabins which support that apparently sell out early and don’t seem to be available when booking a cheap last-minute deal over the Internet. So if you want a cheap cruise then you need to have an even number of people in your party.

The cruise I took was two nights and cost $238 per person, it was advertised at something like $220 but then there are extra fees when you book (which seems to be the standard practice).

The Value of Cruises

To book a hotel room that is reasonably comfortable (4 star) in Melbourne or Sydney you need to spend more than $100 per night for a two person room if using Wotif.com. The list price of a 4 star hotel room for two people in a central city area can be well over $300 per night. So the cost for a cruise is in the range of city hotel prices.

The Main Dining Room (MDR) has a quality of food and service that compares well with city restaurants. The food and service in the Dawn Princess MDR wasn’t quite as good as Walter’s Wine Bar (one of my favorite restaurants). But Walter’s costs about $90 for a four course meal. The Dawn Princess MDR has a standard 5 course meal (with a small number of options for each course) and for no extra charge you can order extra serves. When you make it a 7 course meal the value increases. I really doubt that I could find any restaurant in Melbourne or Sydney that would serve a comparable meal for $119.

You could consider a cruise to be either paying for accommodation and getting everything else for free or to be paying for fine dining in the evening and getting everything else for free. Getting both for the price of one (along with entertainment etc) is a great deal!

I can recommend a cruise as a good holiday which is rather cheap if you do it right. That is if you want to spend lots of time swimming and eating quality food.

How Cruise Companies Make Money

There are economies of scale in running a restaurant, so having the MDR packed every night makes it a much more economic operation than a typical restaurant which has quiet nights. But the expenses in providing the services (which involves a crew that is usually almost half the number of passengers) are considerable. Paying $119 per night might cover half the wages of an average crew member but not much more.

The casino is one way that the cruise companies make money. I can understand that someone taking a luxury vacation might feel inclined to play blackjack or something else that seems sophisticated. But playing poker machines on a cruise ship is rather sad – not that I’m complaining, I’m happy for other people to subsidise my holidays!

Alcohol is rather expensive on board. Some cruise companies allow each passenger to take one bottle of wine and some passengers try to smuggle liquor on board. On the forums some passengers report that they budget to spend $1000 per week on alcohol! If I wanted a holiday that involved drinking that much I’d book a hotel at the beach, mix up a thermos full of a good cocktail in my hotel room, and then take my own deck-chair to the beach.

It seems that the cruise companies specialise in extracting extra money from passengers (I don’t think that my experience with the Dawn Princess is unusual in any way). Possibly the people who pay $1000 per night or more for a cruise don’t get the nickel-and-dime treatment, but for affordable cruises I think it’s standard. You have to be in the habit of asking the price whenever something is offered and be aware of social pressure to spend money.

When I boarded the Dawn Princess there was a queue, which I joined as everyone did. It turned out that the queue was to get a lanyard for holding the key-card (which opens the cabin door and is used for payment). After giving me the lanyard they then told me that it cost $7.95 – so I gave it back. Next time I’ll take a lanyard from some computer conference and use it to hold the key-card, it’s handy to have a lanyard but I don’t want to pay $7.95.

Finally some things are free at some times but not at others, fruit juice is free at the breakfast buffet but expensive at the lunch buffet. Coffee at the MDR is expensive but it was being served for free at a cafe on deck.

How to have a Cheap Cruise

VacationsToGo.com is the best discount cruise site I’ve found so far [2]. Unfortunately they don’t support searching on price, average daily price, or on a customised number of days (I can search for 7 days but not 7 or less). For one of the cheaper vessels it seems that anything less than $120 per night is a good deal and there are occasional deals as low as $70 per night.

Princess cruises allows each passenger to bring one bottle of wine on board. If you drink that in your cabin (to avoid corkage fees) then that can save some money on drinks. RumRunnerFlasks.com sells plastic vessels for smuggling liquor on board cruise ships [3]. I wouldn’t use one myself but many travelers recommend them highly.

Chocolate and other snack foods are quite expensive on board and there are no restrictions on bringing your own, so the cheap options are to bring your own snack food or to snack from the buffet (which is usually open 24*7). Non-alcoholic drinks can be expensive but you can bring your own and use the fridge in your cabin to store it, but you have to bring cans or pressurised bottles so it doesn’t look like you are smuggling liquor on board.

Generally try not to pay for anything on board, there’s enough free stuff if you make good choices.

Princess offers free on-board credit (money for buying various stuff on-board) for any cruise that you book while on a cruise. The OBC starts at $25 per person and goes as high as $150 per person depending on how expensive the cruise is. Generally booking cruises while on-board is a bad idea as you can’t do Internet searches. But as Princess apparently doesn’t allow people outside the US to book through a travel agent and as they only require a refundable deposit that is not specific to any particular cruise there seems no down-side. In retrospect I should have given them a $200 on the off chance that I’ll book another cruise with them some time in the next four years.

Princess provide a book of discount vouchers in every cabin, mostly this is a guide to what is most profitable for them – and thus what you should avoid if you want a cheap holiday. But there are some things that could be useful such as a free thermos cup with any cup of coffee – if you buy coffee then you might as well get the free cup. Also they have some free contests that might be worth entering.

Entertainment

It’s standard practice to have theatrical shows on board, some sort of musical is standard and common options include a magic show and comedy (it really depends on which cruise you take). On the Dawn Princess the second seating for dinner started at 8PM (the time apparently varies depending on the cruise schedule) which was the same time as the first show of the evening. I get the impression that this sort of schedule is common so if you want to see two shows in one night then you need to have the early seating for dinner. The cruise that I took lasted two nights and had two shows (a singing/dancing show and a magic show), so it was possible to have the late seating for dinner and still see all the main entertainment – unless you wanted to see one show twice.

From reading the CruiseCritic.com forum [4] I get the impression that the first seating for dinner is the most popular. On some cruises it’s easy to switch from first to second seating but not always possible to switch from second to first. Therefore the best strategy seems to be to book the first seating.

Things to do Before Booking a Cruise

Read the CruiseCritic.com forum for information about almost everything.

Compare prices for a wide variety of cruises to get a feel for what the best deals are. While $100 per night is a great deal for the type of cruise that interests me and is in my region it may not be a good match for the cruises that interest you.

Read overview summaries of cruise lines that operate in your area. Some cruise lines cater for particular age groups and interests and are thus unappealing to some people – EG anyone who doesn’t have children probably won’t be interested in Disney cruises.

Read reviews of the ships, there is usually a great variation between different ships run by one line. One factor is when the ships have been upgraded with recently developed luxury features.

Determine what things need to be booked in advance. Some entertainment options on board support a limited number of people and get booked out early. For example if you want to use the VR golf simulator on the Dawn Princess you should probably check in early and make a reservation as soon as you are on board. The forums are good for determining what needs to be booked early.

Also see my post about booking a cruise and some general discussion of cruise related things [5].

Related posts:

  1. Cruises It seems that in theory cruises can make for quite...
  2. Combat Wasps One of the many interesting ideas in Peter F. Hamilton’s...
  3. Victoria Hotel Melbourne I have just stayed at the Victoria Hotel Melbourne. I...


January 08, 2012 11:03 AM

-----

January 07, 2012

Pia Waugh

Twitter Weekly Updates for 2012-01-08


January 07, 2012 11:00 PM

Twitter Weekly Updates for 2012-01-08


January 07, 2012 11:00 PM

-----

January 06, 2012

Robert Collins

Public service announcement: signals implies reentrant code even in Python

This is a tiny PSA prompted by my digging into a deadlock condition in the Launchpad application servers. We were observing a small number of servers stopping cold when we did log rotation, with no particularly rhyme or reason. tl;dr: do not call any non-reentrant code from a Python signal handler. This includes the signal [...]


January 06, 2012 04:38 AM

-----

January 05, 2012

Russell Coker

DRBD Benchmarking

I’ve got some performance problems with a mail server that’s using DRBD so I’ve done some benchmark tests to try and improve things. I used Postal for testing delivery to an LMTP server [1]. The version of Postal I released a few days ago had a bug that made LMTP not work, I’ll release a new version to fix that next time I work on Postal – or when someone sends me a request for LMTP support (so far no-one has asked for LMTP support so I presume that most users don’t mind that it’s not yet working).

The local spool on my test server is managed by Dovecot, the Dovecot delivery agent stores the mail and the Dovecot POP and IMAP servers provide user access. For delivery I’m using the LMTP server I wrote which has been almost ready for GPL release for a couple of years. All I need to write is a command-line parser to support delivery options for different local delivery agents. Currently my LMTP server is hard-coded to run /usr/lib/dovecot/deliver and has it’s parameters hard-coded too. As an aside if someone would like to contribute some GPL C/C++ code to convert a string like “/usr/lib/dovecot/deliver -e -f %from% -d %to% -n” into something that will populate an argv array for execvp() then that would be really appreciated.

Authentication is to a MySQL server running on a fast P4 system. The MySQL server was never at any fraction of it’s CPU or disk IO capacity so using a different authentication system probably wouldn’t have given different results. I used MySQL because it’s what I’m using in production. Apart from my LMTP server and the new version of Postal all software involved in the testing is from Debian/Squeeze.

The Tests

All tests were done on a 20G IDE disk. I started testing with a Pentium-4 1.5GHz system with 768M of RAM but then moved to a Pentium-4 2.8GHz system with 1G of RAM when I found CPU time to be a bottleneck with barrier=0. All test results are for the average number of messages delivered per minute for a 19 minute test run where the first minute’s results are discarded. The delivery process used 12 threads to deliver mail.

P4-1.5 p4-2.8
Default Ext4 1468 1663
Ext4 max_batch_time=30000 1385 1656
Ext4 barrier=0 1997 2875
Ext4 on DRBD no secondary 1810 2409

When doing the above tests the 1.5GHz system was using 100% CPU time when the filesystem was mounted with barrier=0, about half of that was for system (although I didn’t make notes at the time). So the testing on the 1.5GHz system showed that increasing the Ext4 max_batch_time number doesn’t give a benefit for a single disk, that mounting with barrier=0 gives a significant performance benefit, and that using DRBD in disconnected mode gives a good performance benefit through forcing barrier=0. As an aside I wonder why they didn’t support barriers on DRBD given all the other features that they have for preserving data integrity.

The tests with the 2.8GHz system demonstrate the performance benefits of having adequate CPU power, as an aside I hope that Ext4 is optimised for multi-core CPUs because if a 20G IDE disk needs a 2.8GHz P4 then modern RAID arrays probably require more CPU power than a single core can provide.

It’s also interesting to note that a degraded DRBD device (where the secondary has never been enabled) only gives 84% of the performance of /dev/sda4 when mounted with barrier=0.

p4-2.8
Default Ext4 1663
Ext4 max_batch_time=30000 1656
Ext4 min_batch_time=15000,max_batch_time=30000 1626
Ext4 max_batch_time=0 1625
Ext4 barrier=0 2875
Ext4 on DRBD no secondary 2409
Ext4 on DRBD connected C 1575
Ext4 on DRBD connected B 1428
Ext4 on DRBD connected A 1284

Of all the options for batch times that I tried it seemed that every change decreased the performance slightly but as the greatest decrease in performance was only slightly more than 2% it doesn’t matter much.

One thing that really surprised me was the test results from different replication protocols. The DRBD replication protocols are documented here [2]. Protocol C is fully synchronous – a write request doesn’t complete until the remote node has it on disk. Protocol B is memory synchronous, the write is complete when it’s on a local disk and in RAM on the other node. Protocol A is fully asynchronous, a write is complete when it’s on a local disk. I had expected protocol A to give the best performance as it has lower latency for critical write operations and for protocol C to be the worst. My theory is that DRBD has a performance bug for the protocols that the developers don’t recommend.

One other thing I can’t explain is that according to iostat the data partition on the secondary DRBD node had almost 1% more sectors written than the primary and the number of writes was more than 1% greater on the secondary. I had hoped that with protocol A the writes would be combined on the secondary node to give a lower disk IO load.

I filed Debian bug report #654206 about the kernel not exposing the correct value for max_batch_time. The fact that no-one else has reported that bug (which is in kernels from at least 2.6.32 to 3.1.0) is an indication that not many people have found it useful.

Conclusions

When using DRBD use protocol C as it gives better integrity and better performance.

Significant CPU power is apparently required for modern filesystems. The fact that a Maxtor 20G 7200rpm IDE disk [3] can’t be driven at full speed by a 1.5GHz P4 was a surprise to me.

DRBD significantly reduces performance when compared to a plain disk mounted with barrier=0 (for a fair comparison). The best that DRBD could do in my tests was 55% of native performance when connected and 84% of native performance when disconnected.

When comparing a cluster of cheap machines running DRBD on RAID-1 arrays to a single system running RAID-6 with redundant PSUs etc the performance loss from DRBD is a serious problem that can push the economic benefit back towards the single system.

Next I will benchmark DRBD on RAID-1 and test the performance hit of using bitmaps with Linux software RAID-1.

If anyone knows how to make a HTML table look good then please let me know. It seems that the new blog theme that I’m using prevents borders.

Update:

I mentioned my Debian bug report about the mount option and the fact that it’s all on Debian/Squeeze.

Related posts:

  1. I need an LMTP server I am working on a system where a front-end mail...
  2. Some Notes on DRBD DRBD is a system for replicating a block device across...
  3. paper about ZCAV This paper by Rodney Van Meter about ZCAV (Zoned Constant...


January 05, 2012 08:31 AM

-----

January 04, 2012

Dave Ruys

Some local teenager started filming me with his phone whil I was...

[Flash 10 is required to watch video.]

Some local teenager started filming me with his phone whil I was skating so I asked for the footage. That, folks, is how you stop a longboard from around 40-50 km/h (or faster if you’ve got the cojones).


January 04, 2012 10:23 AM

Russell Coker

Autism and a Child Beauty Contest

Fenella Wagener wrote an article for the Herald Sun about an Autistic girl who won the “best personality” award from the controversial new Australian children’s beauty pageant [1]. The girl’s mother is complaining that an Autistic girl shouldn’t win a prize for personality and is critizing the pageant organisers.

A beauty contest involves queuing, being quiet, appearing on stage, wearing cosmetics and unusual/uncomfortable clothes. It probably also involves having someone else assist with dressing and applying cosmetics (being touched by another person). These are all things which tend to be difficult or impossible for Autistic kids. So any girl who can get on stage wearing make-up can probably do whatever is required to avoid being obviously excluded from a personality prize. As any such prize has to be largely subjective I don’t think it would ever be possible to prove that someone was the correct choice for the winner, it would merely be possible to prove that some candidates excluded themselves.

But whether the girl deserved to win isn’t the real issue here. I think that beauty pageants should be restricted to adults, merely entering a child in such a contest is bad enough, but making nasty public statements about a child is horrible. If other children made a Facebook page claiming that the girl in question didn’t deserve to win a “best personality” prize it would probably be reported as cyber-bullying. I don’t think that publishing the name or photo of the girl in question is in the “public interest” either. Many news sites that have picked up the story have shown the same lack of journalistic ethics so now the girl has some high traffic sites with her name linked to this story, it seems unlikely that anything good she might do in the near future will get a higher ranking for her name in search engines. So any time she searches for her name on Google (which most people do regularly) she will be reminded that her mother thinks she has some sort of defective personality because she is Autistic.

High school is generally bad for almost everyone on the Autism Spectrum. Presumably any parent who would abuse their child by allowing such an article to be published would also send them to a regular school (as opposed to Home Schooling which is probably the only good option for Autistic kids in Australia). I’m sure that the standard practice at every high school nowadays is that the kids all use Google to discover things to tease each other about. So in a few years the Herald Sun article will probably be the basis of a high school bullying campaign.

The girl in question is only 9, so she’s got another 6 or 7 years before she can legally leave her mother. In Australia 16 is the minimum legal age to live without parents and the police won’t forcibly return “runaway” children who are almost 16.

The Journalistic Code of Ethics

Here is a link to the Australian Media Alliance code of Journalistic Ethics [2]. Section 8 includes “Never exploit a person’s vulnerability or ignorance of media practice“. I think that publishing the name and photograph of a 9yo girl in a way that is likely to lead to bullying in a few years is a clear example of exploiting a vulnerable person.

The code of ethics has a guidance clause which says “Only substantial advancement of the public interest or risk of substantial harm to people allows any standard to be overridden“. Even if it was a proven fact that a beauty pageant was issuing awards to unqualified children there would not be any substantial advancement of the public interest in publishing that.

Beauty Contests are Evil

The Australian has an article about the same beauty contest by Caroline Overington which quotes adolescent and child psychotherapist Collett Smart calling for government intervention [3].

Catherine Manning has written a good article explaining some of the reasons for opposing child beauty pageants [4].

The American Psychological Association has published a report on the Sexualization of Girls [5], they have lots of references to psychological research which gives a variety of reasons for opposing child beauty contests. IMHO each of the reasons alone should be sufficient to convince people that child beauty pageants are bad.

Finally the pictures of contestants who are less than 10yo but made up to look like they are 20+ are rather disturbing.

Related posts:

  1. Child Abuse and Censorship In Australia we are currently in the middle of a...
  2. Autism Awareness and the Free Software Community It’s Autism Awareness Month April is Autism Awareness month, there...
  3. Autism, Food, etc James Purser wrote “Stop Using Autism to Push Your Own...


January 04, 2012 06:17 AM

-----

January 03, 2012

Russell Coker

Standardising Android

Don Marti wrote an amusing post about the lack of standards for Android phones and the fact that the iPhone has a better accessory market as a result [1].

I’d like to see some Android phones get standardised in a similar manner to the PC. The big thing about the IBM PC compatible market was that they all booted the same way, ran the same OS and applications, had the same expansion options, connectors, etc. The early PCs sucked in many ways (there were many other desktop computers in the 80′s that were better in various ways) but the larger market made the PC win.

The PC even killed the Mac! This is something we should remember now when discussing the iPhone.

I’d like to see different Android phones that can run the same OS with the same boot loader. Having HTC, LG, Samsung, and others all sell phones that can run the same version of CyanogenMod and have the same recovery options if a mistake is made when loading CyanogenMod shouldn’t be any more difficult than having IBM, Compaq, HP, DEC, Dell, and others selling PCs that run the same versions of all the OSs of the day and had the same recovery options.

Then there should be options for common case sizes. From casual browsing in phone stores it seems that most phones on sale in Australia are of a tablet form without a hardware keyboard, they have a USB/charger socket, an audio socket, and hardware buttons for power, volume up/down, and “home” – with the “settings” and “back” buttons being through the touch-screen on the Galaxy S but hardware in most others. A hardware button to take a picture is available in some phones.

The variation in phone case design doesn’t seem to be that great and there seems to be a good possibility for a few standards for common formats, EG large tablet, small tablet, and large tablet with hardware keyboard. The phone manufacturers are currently competing on stupid things like how thin a phone can be while ignoring real concerns of users such as having a phone that can last for 24 hours without being charged! But they could just as easily compete on ways of filling a standard case size, with options for screen resolution, camera capabilities, CPU, GPU, RAM, storage, etc. There could also be ways of making a standard case with several options, EG having an option for a camera that extends from the back of the case for a longer focal length – such an option wouldn’t require much design work for a second version of anything that might connect to the phone.

Also standards would need to apply for a reasonable period of time. One advantage that Apple has is that it has only released a few versions of the iPhone and each has been on sale for a reasonable amount of time (3 different sizes of case in 4 years). Some of the Android phones seem to only be on sale in mass quantities for a few months before being outdated, at which time many of the stores will stop getting stock of matching accessories.

Finally I’d be a lot happier if there was good support for running multiple Android phones with the same configuration. Then I could buy a cheap waterproof phone for use at the beach and synchronise all the configuration before leaving home. This is a feature that would be good for manufacturers as it would drive the average rate of phone ownership to something greater than 1 phone per person.

Related posts:

  1. Choosing an Android Phone My phone contract ends in a few months, so I’m...
  2. My Ideal Mobile Phone Based on my experience testing the IBM Seer software on...
  3. My Prediction for the iPhone I have previously written about how I refused an offer...


January 03, 2012 02:14 PM

-----

December 31, 2011

Pia Waugh

Twitter Weekly Updates for 2012-01-01


December 31, 2011 11:00 PM

Russell Coker

Links December 2011

Barry Ritholtz wrote an insightful post quoting Federal Reserve Bank of Kansas City President Thomas Hoenig, who warns that the nation’s biggest banks are putting the U.S. capitalist society at risk [1]. Big banks oppose capitalism.

Glenn Greenwald has written an insightful article for Salon about the modern definition of American excellence being the killing of supposedly bad people without any due process [2].

Mazuma Mobile buys used mobile phones [3]. They can send a post-pack to ship your old mobile to them. This is good for the environment and also saves some money.

Sam Varghese has written an informative article about the Trans Pacific Partnership Agreement that will probably end up benefiting US corporations at the expense of Australian citizens [4].

Cory Doctorow has written an informative article for The Guardian about the BBC DRM plans[5]. He received information that was denied in a FOI request which shows how poor the BBC case is and how bad the Ofcom oversight is.

Sam Harris has written an insightful blog post about self-defense [6]. He also has many other posts that are worth reading.

Aparna Roa gave an interesting TED presentation about her robotic art [7].

Related posts:

  1. Links October 2011 Ron has written an interesting blog post about the US...
  2. Links May 2011 John W. Dean wrote in insightful series of three articles...
  3. Links July 2011 The Reid Report has an article about the marriage pledge...


December 31, 2011 12:56 PM

Sociological Images

I’ve recently been reading the Sociological Images blog [1]. That site has lots of pictures and videos that are relevant to the study of Sociology (most of which have a major WTF factor) and it’s run by people who have Ph.Ds in Sociology so the commentary is insightful. Since reading that I’ve started photographing relevant things.

woman in straight-jacket advertising energy prices

I can’t work out the logic behind the above advert for Energy Watch which was on a billboard near Ringwood Station in Melbourne, Australia. The only thing that it clear is that it spreads bad ideas about mental illness and psychiatric treatment. It doesn’t make me want to do business with them.

Antons full display

The above picture is a shop-front for the Antons clothing store (I’m not sure if they are a tailor or if they sell ready to wear). It was taken on Lonsdale St, Melbourne where the store apparently used to be, now they are in Melbourne Central.

Antons left display, African and Southern EuropeanAntons right display, Northern European and Japanese

The above pictures show more detail. Unfortunately the combination of lighting and my camera (Xperia X10 phone camera) wasn’t adequate to show the apparent ethnic differences between the two men. It seems that the most likely Australian interpretation of the ethnic groups that are represented are African (maybe Afro-American), Southern-European or maybe American Hispanic, North-Western European, and Japanese. It’s good to have mannequins representing the fact that not everyone in Australia is white, but different facial expressions for different races seems a strange choice (admittedly it might be a choice made by mannequin manufacturers). Also the Japanese woman with fan idea is rather outdated.

I’ve just started reading You May Ask Yourself: An Introduction to Thinking Like a Sociologist (Second Edition) by Dalton Conley. I’ve only read the first chapter, but that was good enough that the entire book has to be good enough to recommend.

Related posts:

  1. images for a web site When I first started putting pictures on my web site...
  2. A First Digital Camera I’ve just been asked for advice on buying a digital...
  3. The Australian Open and Android Phones (Seer) On Monday the 25th of January 2010 I visited the...


December 31, 2011 02:15 AM

My Blog Server was Cracked

On the 1st of August I noticed that the server which runs my blog among other things was having an occasional SEGV from a sshd process. Unfortunately I was busy and didn’t pay much attention to this, which turned out to be a big mistake.

On the 12th of September I started investigating this properly and noticed that when someone tried to connect to ssh with password authentication sshd would SEGV after it was denied access to a shared memory region or a semaphore which had a SE Linux type of unconfined_t. I added some SE Linux auditallow rules and discovered that the memory region in question was created by the ssh client. Shortly after that I came to the conclusion that this wasn’t some strange feature of ssh (or one of the many shared objects it uses) but hostile activity. The ssh client appeared to be storing passwords that it used in a shared memory region and sshd was also collecting passwords in the same region and presumably offering them to a ssh client which uses some extension to the ssh protocol.

The sshd process was crashing because it couldn’t handle EPERM on access to shared memory or semaphores. Presumably if the system in question wasn’t running SE Linux then the exploit would have remained undetected for a lot longer.

At this stage we don’t know how the attacker got in. Presumably one of the people with root access ran a ssh client on a compromised system and had their password sniffed. One such client system was mysteriously reinstalled at about that time, the sysadmin of the system in question claimed to have no backups which made it impossible to determine if that system had been compromised. I believe that the sysadmin of the client system knew that their system was compromised, kept that information secret, and allowed other systems to become and remain compromised.

The attacker made no good effort to conceal their presence, they replaced ssh, sshd, and ssh-add and didn’t bother changing the Debian checksums so the debsums program flagged the files as modified. Note that I have kept copies of the files in question and am willing to share them with anyone who wants to analyse them.

Steinar H. Gunderson has named this trojan Ebury [1].

Recovery

By the evening of the 13th of September I had the system mostly working again. Jabber still isn’t working because ejabberd is difficult to get working at the best of times, I am now investigating whether there is a better Jabber server to use, but as I don’t use Jabber often this hasn’t been a priority for me.

Some of the WordPress plugins I use and all of the WordPress themes that are installed were outside the Debian packaging system, as I couldn’t be sure that they hadn’t been altered (because the people who wrote WordPress plugins don’t keep old versions online) I had to upgrade to the newer versions. Of course the newer versions weren’t entirely compatible so I had to use a different theme and I couldn’t get all plugins working. Link Within no longer works, not that it ever worked properly [2], I wanted to try Outbrain again but their web site won’t let me login (and they haven’t responded to my support request). Does anyone know of a good WordPress plugin to provide links to related content? Either related content on my blog or on the Internet in general will be OK.

Some people have asked me about the change in appearance of my blog. It was simply impossible (for someone with my PHP skills) to get my blog looking the same way as it did before the server was cracked. I think that the new look is OK and don’t mind if people think it looks likw a VW advert – VW make great cars, I was very satisfied with the VW Passat I used to drive.

Future Plans

I had bought some Yubikeys (USB devices that generate one-time passwords) [3] to control access to that server, if I had configured the software to use them then this might not have happened. The use of one-time password devices can prevent passive password-sniffing attacks. It would still allow active attacks (such as using ControlPath/ControlMaster options on the ssh client to allow a hostile party to connect later (EG the -M, -S, and “-o ControlPersist” options for the ssh client). It’s a pity that there doesn’t seem to be a way to configure the ssh server to disable ControlMaster.

Conclusion

It would be good to have some changes to sshd to allow more restrictions on what a client can request, as ControlMaster functionality isn’t needed by most users it should be possible to disable it.

SE Linux doesn’t protect against a compromised client system or any other way of stealing passwords. It did do a good job of stopping Ebury from doing all the things it wanted to do and thus making me aware of the problem. So I count this as a win for SE Linux.

Yubikeys are the cheapest and easiest way of managine one-time passwords. I had already bought some for use on the system in question but hadn’t got around to configuring them. I have to make that a priority.


December 31, 2011 12:01 AM

-----

December 28, 2011

Russell Coker

Secure Boot and Protecting Against Root

There has been a lot of discussion recently about the recent Microsoft ideas regarding secure boot, in case you have missed it Michael Casadevall has written a good summary of the issue [1].

Recently I’ve seen a couple of people advocate the concept of secure boot with the stated idea that “root” should be unable to damage the system, as Microsoft Software is something that doesn’t matter to me I’ll restrict my comments to how this might work on Linux.

Restricting the “root” account is something that is technically possible, for much of the past 9 years I have been running SE Linux “Play Machines” which have UID 0 (root) restricted by SE Linux such that they can’t damage the system [2] – there are other ways of achieving similar goals. But having an account with UID 0 that can’t change anything on the system doesn’t really match what most people think of as “root”, I just do it as a way of demonstrating that SE Linux controls all access such that cracking a daemon which runs as root won’t result in immediately controlling the entire system.

As an aside my Play Machine is not online at the moment, I hope to have it running again soon.

Root Can’t Damage the System

One specific claim was that “root” should be unable to damage the system. While a secure boot system can theoretically result in a boot to single user mode without any compromise that doesn’t apply to fully operational systems. For a file owned by root to be replaced the system security has to be compromised in some way. The same compromise will usually work every time until the bug is fixed and the software is upgraded. So the process of cracking root that might be used to install hostile files can also be used at runtime to exploit running processes via ptrace and do other bad stuff.

Even if the attacker is forced to compromise the system at every boot this isn’t a great win for the case of servers with months of uptime or for the case of workstations that have confidential data that can be rapidly copied over the Internet. There are also many workstations that are live on the Internet for months nowadays.

Also the general claim doesn’t really make sense on it’s own. “root” usually means the account that is used for configuring the system. If a system can be configured then the account which is used to configure it will be able to do unwanted things. It is theoretically possible to run workstations without external root access (EG have them automatically update to the latest security fixes). Such a workstation configuration MIGHT be able to survive a compromise by having a reboot trigger an automatic update. But a workstation that is used in such a manner could be just re-imaged as it would probably be used in an environment where data-less operation makes sense.

An Android phone could be considered as an example of a Linux system for which the “root” user can’t damage the system if you consider “root” to mean “person accessing the GUI configuration system”. But then it wouldn’t be difficult to create a configuration program for a regular Linux system that allows the user to change some parts of the system configuration while making others unavailable. Besides there are many ways in which the Android configuration GUI permits the user to make the system mostly unusable (EG by disabling data access) or extremely expensive to operate (EG by forcing data roaming). So I don’t think that Android is a good example of “root” being prevented from doing damage.

Signing All Files

Another idea that I saw advocated was to have the “secure boot” concept extended to all files. So you have a boot loader that loads a signed kernel which then loads only signed executables and then every interpreter (Perl, Python, etc) will also check for signatures on files that they run. This would be tricky with interpreters that are designed to run from standard input (most notably /bin/sh but also many other interpreters).

Doing this would require changing many programs, I guess you would even have to change mount to check the signature on /etc/fstab etc. This would be an unreasonably large amount of work.

Another possibility would be to change the kernel such that it checks file signatures and has restrictions on system calls such as open() and the exec() family of calls. In concept it would be possible to extend SE Linux or any other access control system to include access checks on which files need to be signed (some types such as etc_t and bin_t would need to be signed but others such as var_t wouldn’t).

Of course this would mean that no sysadmin work could be performed locally as all file changes would have to come from the signing system. I can imagine all sorts of theoretically interesting but practically useless ways of implementing this such as having the signing system disconnected from the Internet with USB flash devices used for one-way file transfer – because you can’t have the signing system available to the same attacks as the host system.

The requirement to sign all files would reduce the use of such a system to a tiny fraction of the user-base. Which would then raise the question of why anyone would spend the effort on that task when there are so many other ways of improving security that involve less work and can be used by more people.

Encrypted Root Filesystem

One real benefit of a secure boot system is for systems using encrypted filesystems. It would be good to know that a hostile party hasn’t replaced the kernel and initrd when you are asked for the password to unlock the root filesystem. This would be good for the case where a laptop is left in a hotel room or other place where a hostile party could access it.

Another way of addressing the same problem is to boot from a USB device so that you can keep a small USB boot device with you when it’s inconvenient to carry a large laptop (which works for me). Of course it’s theoretically possible for the system BIOS to be replaced with something that trojans the boot process (EG runs the kernel in a virtual machine). But I expect that if someone who is capable of doing that gets access to my laptop then I’m going to lose anyway.

Conclusion

The secure boot concept does seem to have some useful potential when the aim is to reboot the system and have it automatically apply security fixes in the early stages of the boot process. This could be used for Netbooks and phones. Of course such a process would have to reset some configuration settings to safe defaults, this means replacing files in /etc and some configuration files in the user’s home directory. So such a reboot and upgrade procedure would either leave the possibility that files in /etc were still compromised or it would remove some configuration work and thus give the user an incentive to avoid applying the patch.

Any system that tries to extend signature checks all the way would either be vulnerable to valid but hostile changes to system configuration (such as authenticating to a server run by a hostile party) or have extreme ease of use issues due to signing everything.

Also a secure boot will only protect a vulnerable system between the time it is rebooted and the time it returns to full operation after the reboot. If the security flaw hasn’t been fixed (which could be due to a 0-day exploit or an exploit for which the patch hasn’t been applied) then the system could be cracked again.

I don’t think that a secure boot process offers real benefits to many users.


December 28, 2011 04:16 AM

-----

December 25, 2011

Matt Palmer

The Other Way...

Chris Siebenmann sez:

The profusion of network cables strung through doorways here demonstrates that two drops per sysadmin isn’t anywhere near enough.

What I actually suspect it demonstrates is that Chris’ company hasn’t learnt about the magic that is VLANs. All of the reasons he cites in the longer, explanatory blog post could be solved with VLANs. The only time you can’t get away with one gigabit drop per office and an 8 port VLAN-capable switch is when you need high capacity, and given how many companies struggle by with wifi, I’m going to guess that sustained gigabit-per-machine is not a common requirement.

So, for Christmas, buy your colleages a bunch of gigabit VLAN capable switches, and you can avoid both the nightmare of not having enough network ports, and the more hideous tragedy of having to crawl around the roofspace and recable an entire office.


December 25, 2011 05:00 AM

-----

December 24, 2011

Pia Waugh

Twitter Weekly Updates for 2011-12-25


December 24, 2011 11:00 PM

-----

December 23, 2011

Dave Ruys

"Mummy: Daddy needs more skateboards and more wheels!"

“Mummy: Daddy needs more skateboards and more wheels!”

- I love my daughter.


December 23, 2011 07:19 AM

Time to get busy on the ghetto lathe…..



Time to get busy on the ghetto lathe…..


December 23, 2011 06:27 AM

-----

December 21, 2011

Dave Ruys

Ghetto lathe. This blog being Ad Hoc, the picture is pretty...



Ghetto lathe.

This blog being Ad Hoc, the picture is pretty fitting. Test riding some sample wheels for Vault, the wheel profile had about 10mm of (in my opinion) completely redundant outer lip making the wheel highly unpredictable. One jerry-rigged lathe session later and I end up with a centreset* freeride wheel that is smooth sliding, predictable and just plain rad. Time to get some.

Next batch of wheels will be pretty much like this from factory. Sorted.

* When the wheel is symmetrical about the bearing hub, meaning you can mount it either way to get even wear


December 21, 2011 11:15 PM

-----

December 20, 2011

Andrew Cowie

Poisoning DNS perhaps a bad idea

This is insane. I’m sitting at a café in Sydney using their hotspot. Went to search for something, and I kept getting strange looking “site not found” pages. Huh? Thy were working a few hours ago. So I started digging.

The café’s upstream ISP is “Optus”, one of the major Australian carriers. To my astonishment I found that Optus’s DNS servers are interfering with Google searches, stealing their DNS lookups and serving results pages on their own (shitty quality) branded search instead! Try https:? No connection; and Google+ wouldn’t load either.

Obviously as soon as realized what’s going on I immediately changed DNS servers to something reliable. Before I did I found a tiny “about this page” link at the bottom of the heinous Optus search results page, where I was told how great this was for me, but how I could opt out of their “default” search engine if I wanted to but was warned this was an “advanced setting”.

Seriously, what do Optus think they’re doing? From a commercial standpoint, do they really think that their captive audience matters to anyone advertising on the web? Of course not, but in the mean time they’re certainly going to alienate customers who just maybe actually do want to use (in this case) Google sites.

There’s a bigger issue, though. Unaltered answers to DNS queries is a backbone of net neutrality. That’s our problem, but once carriers start poisoning nameservers in their own favour it will be but a blink before everyone is doing it to each other and lookups will become worthless. While I’m sure the morons in Marketing who thought that sabotaging DNS queries would be a good idea won’t be worried about the wreckage that will cause for everyone else, such a war wouldn’t be good for any of the companies involved, either. And meanwhile, if they really want everyone to learn how to install an app to “fix” the internet…

Of course, this is only a taste of what we’ll be in for when the communications minister finally gets his compulsory Great Firewall of Australia censorship in place, but one thing at a time. If you’re looking for internet access down here, clearly Optus or anything that uses their network should be blacklisted.

AfC


December 20, 2011 04:00 AM

-----

December 17, 2011

Russell Coker

Some Notes on DRBD

DRBD is a system for replicating a block device across multiple systems. It’s most commonly used for having one system write to the DRBD block device such that all writes are written to a local disk and a remote disk. In the default configuration a write is not complete until it’s committed to disk locally and remotely. There is support for having multiple systems write to disk at the same time, but naturally that only works if the filesystem drivers are aware of this.

I’m installing DRBD on some Debian/Squeeze servers for the purpose of mirroring a mail store across multiple systems. For the virtual machines which run mail queues I’m not using DRBD because the failure conditions that I’m planning for don’t include two disks entirely failing. I’m planning for a system having an outage for a while so it’s OK to have some inbound and outbound mail delayed but it’s not OK for the mail store to be unavailable.

Global changes I’ve made in /etc/drbd.d/global_common.conf

In the common section I changed the protocol from “C” to “B“, this means that a write() system call returns after data is committed locally and sent to the other node. This means that if the primary node goes permanently offline AND if the secondary node has a transient power failure or kernel crash causing the buffer contents to be lost then writes can be lost. I don’t think that this scenario is likely enough to make it worth choosing protocol C and requiring that all writes go to disk on both nodes before they are considered to be complete.

In the net section I added the following:

sndbuf-size 512k;
data-integrity-alg sha1;

This uses a larger network sending buffer (apparently good for fast local networks – although I’d have expected that the low delay on a local Gig-E would give a low bandwidth delay product) and to use sha1 hashes on all packets (why does it default to no data integrity).

Reserved Numbers

The default port number that is used is 7789. I think it’s best to use ports below 1024 for system services so I’ve setup some systems starting with port 100 and going up from there. I use a different port for every DRBD instance, so if I have two clustered resources on a LAN then I’ll use different ports even if they aren’t configured to ever run on the same system. You never know when the cluster assignment will change and DRBD port numbers seems like something that could potentially cause real problems if there was a port conflict.

Most of the documentation assumes that the DRBD device nodes on a system will start at /dev/drbd0 and increment, but this is not a requirement. I am configuring things such that there will only ever be one /dev/drbd0 on a network. This means that there is no possibility of a cut/paste error in a /etc/fstab file or a Xen configuration file causing data loss. As an aside I recently discovered that a Xen Dom0 can do a read-write mount of a block device that is being used read-write by a Xen DomU, there is some degree of protection against a DomU using a block device that is already being used in the Dom0 but no protection against the Dom0 messing with the DomU’s resources.

It would be nice if there was an option of using some device name other than /dev/drbdX where X is a number. Using meaningful names would reduce the incidence of doing things to the wrong device.

As an aside it would be nice if there was some sort of mount helper for determining which devices shouldn’t be mounted locally and which mount options are permitted – it MIGHT be OK to do a read-only mount of a DomU’s filesystem in the Dom0 but probably all mounting should be prevented. Also a mount helper for such things would ideally be able to change the default mount options, for example it could make the defaults be nosuid,nodev (or even noexec,nodev) when mounting filesystems from removable devices.

Initial Synchronisation

After a few trials it seems to me that things generally work if you create DRBD on two nodes at the same time and then immediately make one of them primary. If you don’t then it will probably refuse to accept one copy of the data as primary as it can’t seem to realise that both are inconsistent. I can’t understand why it does this in the case where there are two nodes with inconsistent data, you know for sure that there is no good data so there should be an operation to zero both devices and make them equal. Instead there

The solution sometimes seems to be to run “drbdsetup /dev/drbd0 primary -” (where drbd0 is replaced with the appropriate device). This seems to work well and allowed me to create a DRBD installation before I had installed the second server. If the servers have been connected in Inconsistent/Inconsistent state then the solution seems to involve running “drbdadm -- --overwrite-data-of-peer primary db0-mysql” (for the case of a resource named db0-mysql defined in /etc/drbd.d/db0-mysql.res).

Also it seems that some commands can only be run from one node. So if you have a primary node that’s in service and another node in Secondary/Unknown state (IE disconnected) with data state Inconsistent/DUnknown then while you would expect to be able to connect from the secondary node is appears that nothing other than a “drbdadm connect” command run from the primary node will get things going.


December 17, 2011 08:59 AM

Matt Palmer

Rethtool: How I Learned to Stop Worrying and Love the ioctl

Damn those unshaven yaks…

I’m trying to write a Nagios plugin for work that will comprehensively monitor network interfaces and make sure they’re up, passing traffic, all those sorts of things. Of course, I’m doing it all in Ruby, because that’s how I roll.

So, I need to Know Things about the interface. Everyone does that with ethtool. Right? Sure, if your eyeballs are parsing it. But have you ever tried to machine parse it? To put it as eloquently as possible:

# ethtool eth0
Settings for eth0:
 Supported ports: [ TP MII ]
 Supported link modes:   10baseT/Half 10baseT/Full 
                         100baseT/Half 100baseT/Full 
                         1000baseT/Half 1000baseT/Full 
 Supports auto-negotiation: Yes
 Advertised link modes:  10baseT/Half 10baseT/Full 
                         100baseT/Half 100baseT/Full 
                         1000baseT/Half 1000baseT/Full 
 Advertised pause frame use: No
 Advertised auto-negotiation: Yes
 Link partner advertised link modes:  10baseT/Half 10baseT/Full 
                                      100baseT/Half 100baseT/Full 
                                      1000baseT/Half 1000baseT/Full 
 Link partner advertised pause frame use: No
 Link partner advertised auto-negotiation: Yes
 Speed: 1000Mb/s
 Duplex: Full
 Port: MII
 PHYAD: 0
 Transceiver: internal
 Auto-negotiation: on
 Supports Wake-on: pumbg
 Wake-on: g
 Current message level: 0x00000033 (51)
 Link detected: yes

Parse that, bitch!

Or… perhaps not.

At any rate, I decided that it would be most advantageous if I went straight to the source and twiddle the ioctl until it did my bidding.

And thus, about 5 hours later, was Rethtool born.

Once I worked out a less-than-entirely-crackful way of dealing with C structs in Ruby (after a bit of digging around, I went with the appallingly-undocumented-but-sufficiently-featureful CStruct), and after I finally worked out I was passing the wrong damned struct to ioctl(SIOCETHTOOL) (speaking of appallingly-undocumented: fuck you, ioctl, and all your twisty-passages children), it was smooth sailing.

So, if you’re one of the eight or so people on earth who will ever need to get at the grubby internals of your network interfaces using Ruby (and can’t do it via some sysfs magic), Rethtool is for you.


December 17, 2011 05:00 AM

-----

December 14, 2011

Russell Coker

Hetzner Failover Konfiguration

The Wiki documenting how to configure IP failover for Hetzner servers [1] is closely tied to the Linux HA project [2]. This is OK if you want a Heartbeat cluster, but if you want manual failover or an automatic failover from some other form of script then it’s not useful. So I’ll provide the simplest possible documentation.

Below is a sample of shell code to get the current failover settings and change them to point the IP address to a different server. In my tests this takes between 19 and 20 seconds to complete, when the command completes the new server will be active and no IP packets will be lost – but TCP connections will be broken if the servers don’t support shared TCP state.

# username and password for the Hetzner robot
USERPASS=USER:PASS
# public IP
IP=10.1.2.3
# new active server
ACTIVE=10.2.3.4
# get current values
curl -s -u $USERPASS https://robot-ws.your-server.de/failover.yaml/$IP
# change active server
curl -s -u $USERPASS https://robot-ws.your-server.de/failover.yaml/$IP -d active_server_ip=$ACTIVE

Below is the output of the above commands showing the old state and the new state.

failover:
ip: 10.1.2.3
netmask: 255.255.255.255
server_ip: 10.2.3.3
active_server_ip: 10.2.3.4
failover:
ip: 10.1.2.3
netmask: 255.255.255.255
server_ip: 10.2.3.4
active_server_ip: 10.2.3.4


December 14, 2011 10:44 PM

Mary Gardiner

An appeal for the Ada Initiative

When I was 15 I went on the web for the first time. A boy in my computing class went to Yahoo!, typed in “girls” and spent some time showing me porn.

Photograph of Mary Gardiner

I’ve programmed since I was a kid. I’ve loved the idea of open technology since I read a curious article in the 1990s about people all over the world, fixing complex bugs in an operating system that a university student had named after himself.

But every so often, I’m reminded how my Internet experience began. Women friends haven’t been safe on mailing lists, they haven’t been safe on Wikipedia’s talk pages, and they haven’t been safe at conferences. And even when they are safe, sometimes they’re lonely: estimates of women’s participation in open source run to about 2%, and as Wikipedia editors at 9%.

Thus, I’ve been a volunteer creating communities by and for women in open source since 2000. It’s been the equivalent of an unpaid part-time job for several of those years. But a year ago, Valerie Aurora became more ambitious, and proposed that since we were doing real work, we should do it as our real job. Together we created the Ada Initiative, a non-profit supporting women in open technology and culture. We rely on your support for our work:

Donate now!

Within a year we’ve organised our first AdaCamp, surveyed thousands of people about their perspective on women in open technology and culture, wrote and encouraged adoption of an anti-harassment policy by over 30 conferences and organizations in open tech/culture, and much more.

To continue our work in 2012, we need your help! Please donate to the Ada Initiative, and contribute to our planned work, including future AdaCamps, methodologically rigorous research into women in open source, and training for women contributors to open tech/culture projects and their allies.

Donate now: we can’t do it without you!


December 14, 2011 09:00 PM

-----

December 10, 2011

Sam Johnston

A word on the future of Europe (without the United Kingdom)

It's rare that I rant about politics but given the train wreck that we've woken up to here in Europe I thought I'd make the exception as this is important for all of us — both here in the 27 member European Union (technically while part of Europe, Switzerland's not part of the European Union nor the 17 member Eurozone as it has its own currency, but we're landlocked by it and affected by its instability) as well as abroad, including the United States.

I'm no expert on European politics, but having been a resident of the region for almost a decade now and lived and/or worked in three member states (in addition to Switzerland) I have the unusual advantage of having seen it from many angles:
The United Kingdom is a large and important economy in the zone, but even down to the grassroots level they see themselves as independent and assess every single decision solely on the basis of what it will do for them — there are regularly mini scandals in the papers about their relationship with their fellow Europeans (who are typically seen to be somehow benefiting at their expense). This shortsighted tweet captures the sentiment nicely:
As a prime example, the Common Agricultural Policy which is designed "to provide farmers with a reasonable standard of living, consumers with quality food at fair prices and to preserve rural heritage", tends to redistribute funds from more urbanised countries like the Netherlands and the United Kingdom to those where agriculture actually takes place. It's an important (albeit changing) function and it commands almost half of the EU's budget.

Another example of unnecessary friction is their [self-]exclusion from the Schengen Agreement, which creates a borderless area within Europe, thus facilitating transport and commerce. You still have to pass border control when you enter or leave the Schengen area, including when traveling to/from the Common Travel Area (consisting only of the United Kingdom and Ireland, which are connected on the island of Ireland by the border between the Republic of Ireland and Northern Ireland), but you can travel freely within it once you're there and there are visas which cover the entire region.

Cutting to the chase, it is of no surprise then that the brits would be stubborn when it came to changing the treaty by unanimous vote — indeed I've been predicting that for a while and was certain it would happen a few days ago. What is a surprise though is just how belligerent and childish they've been about it — as a frenchman said in reference to the following video from The Telegraph's excellent article EU suffers worst split in history as David Cameron blocks treaty change:

Another user tweeted:
Also retweeting:
A fair question if you ask me (and if the language upsets you more than the situation itself then you might want to reconsider your priorities). Others agreed:
And:
I think Simon Wardley sums it up nicely though:
From my point of view the brits are [allowing their representatives to get away with] acting like petulant children, benefiting from the European Union when it suits them, and taking their toys home when it doesn't. Their argument that the very establishment that got us into this mess must absolutely be protected above all else is weak — and that it is in the interests of the city, let alone the entire country, deceptive.

They "very doggedly" (their words) sought "a 'protocol' giving the City of London protection from a wave of EU financial service regulations related to the eurozone crisis". That's right, they didn't want to play by the same rules as everyone else, and exercised their veto when it became apparent that was the only option.

To add insult to injury, they "warned the new bloc that it would not be able to use the resources of the EU, raising real doubts as to whether the eurozone would be able to enforce fiscal rules in order to calm the markets". So not only are they going to not participate in cleaning up the mess they played a key role in creating, but they're going to do their best to make sure nobody else can either.

Fortunately there's light at the end of the tunnel: "Cameron was clumsy in his manoeuvring," a senior EU diplomat said. “It may be possible that Britain will shift its position in the days ahead if it discovers that isolation really is not a viable course of action.” Please take a moment today to express your discontent with this decision as sometimes in order to serve your own interests you also need to consider those of others — in much the same way as the tragedy of the commons (where in this case the commons is the European and global markets).

Update: Another great [opinion] piece from The Telegraph: Cameron: the bulldog has no teeth:
Cameron (and Britain) are now in a no-win situation. If the eurozone countries start to rally, then we shall be isolated from the new bloc and stuck in the slow lane of Europe. Should the euro problems deepen, then we shall bear the consequences in full. As George Osborne has indicated, a disorderly collapse of the euro would drag a voiceless Britain into depression.
...
In France and Germany, Cameron will be blamed for exacerbating a crisis by leaders who will brand him the pariah of Europe. Overnight, Britain has changed from a major player to an isolated outpost which, if this goes on, will become about as significant on the global stage as the Isle of Mull. Churchill would be turning in his grave.
Related:



December 10, 2011 09:59 AM

-----

December 09, 2011

Adrian Chadd

Let's blow this popsicle stand..

It's ~ 7:30am on (hopefully!) my last day in Australia for a while. I'm not really sure how to feel. Excited? Scared? A bit of both?

I feel like I've had my life on hold for so long (and made some ridiculous decisions along the way!) that I .. I dunno, maybe now I can let go a bit?

It's all very odd, truth be told.

Well, let's blow this popsicle stand and see what happens.


December 09, 2011 08:31 PM

Russell Coker

Cocolo Chocolate

Cocolo Overview

I recently wrote about buying a fridge for storing chocolate [1].

Jason Lewis (the co-founder of Organic Trader [2]) read that post and sent me some free samples of Cocolo chocolate [3] (Cocolo is an Organic Trader product that is made in Switzerland).

It’s interesting to note that Cocolo seem very focussed on a net presence [3], their URL is printed on the back of the packet in an equal size font to the main label on the front (although the front label is in upper case). The main web page has a prominent link to their Twitter page which appears to be updated a couple of times a month.

PIcture of Cocolo chocolate packaging

Cocolo makes only organic fair-trade chocolate. Every pack lists the percentage of ingredients that are Fairtrade (presumably milk and some other ingredients are sourced locally in Switzerland and Fairtrade doesn’t apply to them). Their chocolate packages have the URL www.fairtrade.com.au printed on them and their web site links to an international Fairtrade organisation. The packages also list the organic and Fairtrade certification details and state that they are GMO free. The final geek data on the package is advise to store the chocolate at a temperature between 16C and 18C (I have now set my fridge thermostat to 17C). The above picture shows the front of a pack of Dark Orange chocolate and the back of a pack of Milk chocolate.

Reviews

One thing that is different about Cocolo is that they use only unrefined evaporated organic cane sugar juice to sweeten their chocolate. This gives it a hint of molasses in the flavor. Children who like white sugar with brown coloring might not appreciate this, but I think that the use of natural cane sugar juice will be appreciated by most people who appreciate products with complex and subtle flavors.

The Milk chocolate contains a minimum of 32% cocoa solids, this compares to the EU standard of a minimum of 25% for milk chocolate and the UK standard of a minimum of 20% for “Family Milk Chocolate”. The EU standard for dark chocolate specifies a minimum of 35% cocoa solids, so it seems that Cocolo milk chocolate is almost as strong as dark chocolate. If you are used to eating dark and bittersweet chocolate then the Cocolo milk chocolate is obviously not that strong, but it is also significantly more concentrated than most milk chocolate that is on the market. The high chocolate content combined with the evaporated cane sugar extract gives a much stronger flavor than any of the milk chocolates that I have eaten in recent times.

The Dark Mint Crisp chocolate has a minimum of 61% cocoa mass. The mint crisp is in very small pieces that give a good texture to the chocolate with a faint crunch when you bite it. It has a good balance of mint and chocolate flavors.

The Dark Orange chocolate contains 58% cocoa solids and has a subtle orange flavor.

The white chocolate tastes quite different from most white chocolate. While most white chocolate is marketed to children the Cocolo white chocolate will probably appeal more to adults than children. This is one of the few white chocolates that I’ve wanted to eat since the age of about 14.

They also have many other flavors, most common types of chocolate (such as with almonds or hazelnuts) are available.

I highly recommend Cocolo products!


December 09, 2011 05:20 AM

Some OWS Stuff

There is an interesting interview with Richard Wilkinson on ted.com about the damage to society that is caused by excessive inequality [1]. The conclusion is interesting, it seems that one part of the solution is to send the message that excessive greed is anti-social. The demonstrations against the greedy 1% seem likely to do some good in this regard. He also has a TED talk about this in which he shows that even the rich don’t benefit from great inequality [2]. If nothing else the links between great inequality and high rates of mental illness and murder should convince people that they don’t want to live in such a society.

Buying Cameras

Some of the radio news reports have claimed that some of the people in OWS camps in Australia are somehow unworthy because they have smart-phones and digital cameras. They seem to believe that people have to demonstrate great poverty to have standing to complain. The problem here is that owning a mobile phone is essential to getting employment in many industries, so the vast majority of people who are old enough to work but not near retirement age just have to own a mobile phone. Once you have decided to use a phone if you are going to use it a lot (which you probably will if living in a tent in a public place) then being on a contract usually makes economic sense. As the cheapest contracts include a “free” Android phone that means almost everyone who is old enough to be allowed out alone at night and young enough to be willing to sleep rough for a cause will have such a phone.

A related complaint is about people owning good cameras. One reason for this is the fact that the huge number of cameras deters the police from using excessive force – that’s a good justification for spending money on a good camera. But then a good camera doesn’t cost much nowadays, the specs on $100 cameras are quite impressive and a camera that costs $400 new will have features that are useful for professional photographers – and cameras are even cheaper on ebay.

In Australia social security payments for a single person who is out of work are as much as $486.80 per fortnight [3]. Therefore someone who is living on social security could buy a great camera if they saved up two weeks of social security payments or buy a similar camera on ebay for one week’s worth of payments.

When at a protest the widespread use of cameras is essential for protection against police brutality. It’s worth saving up for a camera.

But really they are missing the point, OWS is about representing ~99% of the population. Even people on the average income are suffering because of the greed of the rich.

Is Australia such a Great Place?

There have been claims that Australia is such a great place that OWS protests are not needed, supposedly we have a very egalitarian society. The ABS data shows that the level of inequality as expressed by numbers isn’t that great when restricted to just income if you restrict the analysis to the top 20% [4]. The mean gross household income per week is $340 for the lowest quintile and $2,380 for the highest quintile giving a ratio of 1:7.0. But when you look at total household net worth it’s $27,400 vs $1,720,700 – a ratio of 1:62.8! The ratio for the richest 1% would surely be a lot greater but a quick search of abs.gov.au didn’t turn up any statistics on them. Also Richard Wilkinson’s analysis puts Australia quite close to the US.

If the people who are in the top 1% earned their money, paid taxes, and didn’t pervert the political process then there wouldn’t be any OWS protests. The corruption in the political process and rent seeking by corporations isn’t as bad here as it is in the US, but it’s bad enough that we need some serious changes. Emily Manuel wrote an interesting article for Tiger Beatdown about the need for an Occupy movement in Australia [5]. Her description of the political problems in Australia is quite clear. But I wasn’t convinced by her analysis which suggested that Australia is different from the US in any significant way.

Really Comparing Income

To properly compare income it doesn’t make sense to compare a ratio of money earned, or disposable income which is often misleading due to disagreements about what is considered disposable. It’s really about which of the things that you want to do can be achieved on your income. One possible way of comparing it is based on Maslow’s hierarchy of human needs. The poorest 20% of households in Australia have a gross income of $340 per week. The ABS shows that the average of private rental prices in the 8 capital cities (where most of the population lives) is $218 per week [6]. Obviously the poorest households can’t afford an average rent unless you have lots of people packed in one house. No matter how you look at the numbers it’s going to be difficult for low income people to pay for rent and food. That puts them on the bottom level of Maslow’s higherarchy.

It seems to me that the benefits of increased income diminish and it becomes more of a contest to be richer than other people. Owning an old car is a significant benefit over having no car. Owning a new car has some real benefits over an old car. Luxury cars usually have safety features before they are introduced to cheaper cars, but apart from that the benefit of owning a luxury car is small. Owning multiple Rolls-Royces doesn’t seem to provide any real benefit, someone who would want to do that would probably be better served by hiring a good psychologist.

Update:

In a comment Brendan Scott cited The Theory of the Leisure Class by Thorstein Veblen. That’s a great point, and I’ll have to read that book! I still maintain that owning two Rolls Royces is a symptom of psychological problems, people who can sort out such problems won’t feel the need to try and prove that they have more money than others.

Charles Stross has an interesting article about wealth disparity etc [7]. Being a sci-fi author he also explores what life might be like after Drexler boxes satisfy most of our current materiel needs.

The Age has an informative article by Suzy Freeman-Greene contrasting the OWS protests in Melbourne which were broken up by police to the Catholic anti-abortion protests that have been going on for more than 20 years [8]. It seems that corporations aren’t the only large organisations that are pwning democracy!


December 09, 2011 04:58 AM

-----

December 07, 2011

Mary Gardiner

Interested in women in open tech and culture? AdaCamp Melbourne wants you!

My non-profit organisation, the Ada Initiative, wants to go full steam ahead into 2012, and we’re holding an AdaCamp event in Melbourne to kick off the year!

The Ada Initiative supports women in open technology and culture, ranging from open source to free culture to grassroots community organising to makerspaces to remix and fandom culture to open government initiatives and more. This stuff is powerful: it’s already shaping society and is going to continue to do so more and more. The Ada Initiative is focussed on supporting women in becoming an integral part of these communities.

AdaCamp will be a one day “unconference” (that is, it will have free-form sessions scheduled by participants) focussed on furthering women’s work in open technology and culture. It will be held on Saturday January 14 in Melbourne, some travel funding is available.

AdaCamp places are by invitation, if you’re interested in coming along please apply today. Applications close December 14. Hoping to meet some readers and ‘net friends there!


December 07, 2011 12:00 AM

-----

December 06, 2011

Dave Ruys

"People are taking the piss out of you everyday. They butt into your life, take a cheap shot at you..."

“

People are taking the piss out of you everyday. They butt into your life, take a cheap shot at you and then disappear. They leer at you from tall buildings and make you feel small. They make flippant comments from buses that imply you’re not sexy enough and that all the fun is happening somewhere else. They are on TV making your girlfriend feel inadequate. They have access to the most sophisticated technology the world has ever seen and they bully you with it. They are The Advertisers and they are laughing at you.

You, however, are forbidden to touch them. Trademarks, intellectual property rights and copyright law mean advertisers can say what they like wherever they like with total impunity.

Fuck that. Any advert in a public space that gives you no choice whether you see it or not is yours. It’s yours to take, re-arrange and re-use. You can do whatever you like with it. Asking for permission is like asking to keep a rock someone just threw at your head.

You owe the companies nothing. Less than nothing, you especially don’t owe them any courtesy. They owe you. They have re-arranged the world to put themselves in front of you. They never asked for your permission, don’t even start asking for theirs.

”

-

Banksy (via thenewavengers)

I’m feeling the rage and I’m loving it.


December 06, 2011 09:51 PM

-----

December 05, 2011

Russell Coker

CyanogenMod and the Galaxy S

Thanks to some advice from Philipp Kern I have now got my Galaxy S running CyanogenMod 7.1.0 which is based on Android 2.3.7 [1]. CyanogenMod has lots of configuration options that seem to be lacking in the stock releases and also supports some advanced features such as OpenVPN and a command-line. I can’t properly compare CyanogenMod to the stock Android as I’ve only used versions 2.1 and 2.2 of the stock Android. Presumably some of the things that I like about CyanogenMod are in the stock Android 2.3.7 release.

The process of updating a phone is difficult and has some risk. Fortunately Samsung provided “Download mode” in the BIOS to allow recovery. If you mess up the process of updating a Galaxy S and you can get Download mode by holding down volume-down, home, and then power buttons then you can almost certainly recover (so don’t panic).

The CyanogenMod people don’t provide any documentation on upgrading from Android 2.2 (which is what Optus is still shipping AFAIK). So you will probably have some difficulty when upgrading a Galaxy S that you get in Australia (it seems that Optus is the only company shipping them in volume).

As an aside if you want to buy a Gel Case for a Galaxy S in Australia then visit an Optus store. It seems that Optus is the only phone store that hasn’t run out their Galaxy S accessories in favor of the Galaxy S2.

I have previously written about the Galaxy S and Three Networking [2]. Now that I have the Galaxy S as my primary phone on the Virgin network all my data corruption problems are solved, the problem is entirely related to Three. With CyanogenMod there is an option to be able to toggle the LED “Flash” as a torch from the drop-down menu, this makes the lack of such a LED on the Galaxy S even more of an annoyance.

I have also discovered that the Galaxy S apparently doesn’t have a status LED! This makes it the only phone that I’ve ever owned that has no clear way of informing me when the battery is charged! It’s also really useful to have a flashing LED to indicate low battery when running a full screen app, and to have a flashing LED to indicate that email has been received.

Someone should design a phone with multiple LEDs to indicate different things. I’d like to have one LED to indicate charging status and another to indicate whether there is unread email or SMS. Whatever the cost of including a LED during manufacture it would have to be almost nothing compared to the ~$500 sale price of a phone. Wikipedia says “The Samsung Galaxy S features a PowerVR graphics processor, yielding 20 million triangles per second, making it the fastest graphics processing unit in any smartphone at the time of release. Also, upon release, the Galaxy S was both the first Android phone to be certified for DivX HD, and at 9.9 mm was the thinnest smartphone available”. I don’t care about any of that, I want a phone with decent battery life, a LED “Flash”, and a status LED.

The main benefit I get from the Galaxy S over the Xperia X10 is the greater storage. The Xperia X10 has a total of 1G of storage and only 465M of that is available for application install. My Galaxy S has 16G of internal storage of which 1.8G is available for phone apps and 13G is available for pictures and other mass storage. Having 1.8G for phone apps and internal phone storage used by such apps (which includes the offline IMAP cache) is a massive benefit, enough to outweigh the lack of a staus LED and a Flash LED.

What I Really Want

I’d rather have a Samsung Galaxy Note. The Note has a LED flash, a 5.3″ screen with 1280*800 resolution which is much better for running as a SSH client and also good for web browsing. I’m not inclined to spend money on a phone now, so I’ll probably use the Galaxy S until Virgin offers me a new phone or someone just gives me a new phone (I can always hope). One of the many nice features in the Galaxy Note is a built in stylus. When using my current phones for web browsing I sometimes find it difficult to have a touch registered to the desired part of the screen, this is a real problem with the Opera web browser which requires a long press to open a URL in a new tab.


December 05, 2011 02:31 AM

-----

December 02, 2011

Craige McWhirter

Nicola in the Spring

Wandering through one of our paddocks today, I noticed all the alpaca's staring intently in one direction, like they do when there's a threat (usually a dog). I headed in the direction they were looking and begin to hear faint, plaintive goat bleats, although I could see no goats in the paddock.

It didn't take long to discover one of this year's kids had found a hole exposing one of our many underground spring flows.

This hole is big enough for me to get into and spring flow underneath is large enough for me to crawl in. It's worth noting that this paddock is almost 100% covered in blackberries. If you look closely, despite abundant blackberries, Nicola had leaned across this opening, managed to take two bites from the blackberry leaves before slipping into the hole.

After I stopped laughing and put the camera away, I pulled out one cold, wet and muddy kid who immediately bolted through the sunshine in search of her mum (who licked off all the mud).

Blog topics: 
The Farm


December 02, 2011 08:00 AM

-----

November 30, 2011

Russell Coker

Links November 2011

Forbes has an interesting article about crowd-sourcing by criminals and law enforcement [1].

Ulissescastr0 made a Youtube video showing how to install SE Linux on Debian/Etch [2]. Probably no-one is using Etch nowadays so this video is outdated, but it’s a good way of teaching people. It would be good if someone made a similar video showing how to do SE Linux things on Squeeze.

I discovered the above SE Linux video through Explow which provides a neat interface to multiple searches and information sources [3]. I don’t think I will be using Explow much in future as I could get the same result through Google video search. They also have a news portal but there are other sites for that. But it does seem that Explow would be useful for newbies.

Eric Michael Johnson wrote an interesting article about the inherent bias in Psychological research based in the US [4]. People who live in urban environments think differently in some ways to people who live in different environments or who have different lifestyles. Therefore generalising from university students in the US to the entire human race is likely to get incorrect results. This is something to consider the next time you are tempted to generalise to the wider population from your own friends, colleagues, etc.

The Daily Kos has a scary article about the TSA having a woman detained for reciting part of the US constitution [5]. The US will remain on my list of countries to avoid for the forseeable future.

Vorlon has written an informative article about the use of hardening options when building Debian packages [6]. It’s now even easier to do this, so every package that simultaneously deals with data of differing levels of integrity or sensitivity should be built this way.

Bunker Roy gave an interesting TED talk about his Barefoot College that teaches useful skills to people in rural parts of India who don’t have a traditional school education [7]. His talk really shows up some of the arrogance in the people who run traditional education.

Justin Hall-Tipping gave an interesting TED talk about ways of solving the world energy problems [8]. He started with explaining the problems and why they need to be urgently solved and then described in detail some of the research that his group has done to solve the problems. This includes flexible photo-voltaic cells, infra-red vision to save on lighting, and a way of using carbon nano-tubes to control the thermal properties of windows.


November 30, 2011 01:54 PM

-----

November 28, 2011

Mary Gardiner

Speaking of being tall

Of course, if you blog about it it will happen again: “I thought they only made them that tall in Texas!” said the woman in the elevator with us this afternoon.

At least she gets points for originality. Texas? Why Texas?


November 28, 2011 07:14 AM

Andrew Cowie

Learning Haskell

In the land of computer programming, newer has almost always meant better. Java was newer than C, and better, right? Python was better than Perl. Duh, Ruby is better than everything, so they’d tell you. But wait, Twitter is written in Scala. Guess that must be the new hotness, eh?

Haskell has been around for quite a while; somehow I had it in my head that it was outdated and only for computer science work. After all, there are always crazy weirdos out there in academia working on obscure research languages — at least, that’s the perspective from industry. After all, we’re the ones getting real work done. All you’re doing is sequencing the human genome. We invented Java 2 Enterprise Edition. Take that, ivory tower.

The newness bias is strong, which is why I was poleaxed to find people I respect like Erik de Castro Lopo and Conrad Parker working hard in, of all things, Haskell. And now they’re encouraging me to program in it, too (surely, cats and dogs are sleeping together this night). On their recommendation I’ve been learning a bit, and much to my surprise, it turns out Haskell is vibrant, improving, and really cutting edge.

The next thing

I get the impression that people are tired of being told that the some cool new thing makes everything else they’ve been doing irrelevant. Yet many professional programmers (and worse, fanboy script kiddies) are always looking to the next big thing, the next cool language. Often the very people you respect about a topic have already moved on to something else (there’s a book deal in it for them if they can write it fast enough).

But still; technology is constantly changing and there’s always pressure to be doing the latest and greatest. I try my best to resist this sort of thing, just in the interest of actually getting anything done. Not always easy, and the opposite trap is to adopt a bunker mentality whereby you defend what you’re doing against all comers. Not much learning going on there either.

There is, however, a difference between the latest new thing and learning something new.

One of the best things about being active in open source is the opportunity to meet people who you can look up to and learn from. I may know a thing or two about operations and crisis and such, but my techie friends and colleagues are my mentors when it comes to software development and computer engineering. One thing they have taught me over the years is the value of setting out deliberately to “stretch” your mind. Specifically, experimenting with a new programming language that is not your day-to-day working environment, but something that will force your to learn new ways of looking at problems. These guys are professionals; they recognize that whatever your working language(s) are, you’re going to keep using them because you get things done there. It’s not about being seduced by the latest cool project that some popular blogger would have you believe is the be-all-and-end-all. Rather, in stretching, you might be able to bring ideas back to your main work and just might improve thereby. I think there is wisdom there.

Should I attempt to learn Haskell?

I’ve had an eye on functional programming for a while now; who hasn’t? Not being from a formal computer science or mathematics background — (“damnit Jim, I’m an engineer, not an english major” when called upon to defend my atrocious spelling) — the whole “omigod, like, everything is function and that’s like, totally cool” mantra isn’t quite as compelling by itself as it might be. But lots of people I respect have been going on about functional programming for a while now, and it seemed a good direction to stretch. So I asked which language should I learn?

My colleagues suggested Haskell for a number of reasons. That cutting edge research was happening there and that increasingly powerful things were being implemented in the compiler and runtime as a result sounded interesting. That Haskell being a pure functional language (I didn’t know yet what that meant but that’s beside the point) would really force me to learn a functional way of doing at things (as opposed to some others where you can do functional things but can easily escape those constraints; pragmatic, perhaps, but since the idea was to learn something new, that made Haskell sound good rather than perceiving this as a limitation). Finally, they claimed that you could express problems concisely (brevity good, though not if it’s so dense that it’s write-only).

Considering a new language (or, within a language, considering various competing frameworks for web applications, graphical user interface, production deployment, etc) my sense is that when we look at such things we are all fairly quick to judge, based on our own private aesthetic. Does it look clean? Can I do things I need to do with this easily? How do the authors conceive of the problem space? (in web programming especially, a given framework will make some things easy and other things nigh on impossible; you need to know what world-view you’re buying into).

I don’t know about you, but elegance by itself and in the abstract is not sufficient. Elegance is probably the most highly valued characteristic of good engineering design, but it must be coupled with practicality. In other words, does the design get the job done? So before I was willing to invest time learning Haskell, I wanted to at least have some idea that I’d be able to use it for something more than just academic curiosity.

Incidentally, I’m not sure the Haskell community does itself many favours by glorifying in how clever you can be in the language; the implied corollary is that you can’t do anything without being exceedingly clever about it. If true, that would be tedious. I get the humour of the commentary that as we gain experience we tend to overcomplicate things, as seen in the many different ways there are to express a factorial function. But I saw that article linked from almost every thread about how clever you can be with Haskell; is that the sort of thing that you want to use as an introduction for newcomers? Given the syntax is so different from what people are used to in mainstream C-derived programming languages, the code there just looks like mush. The fact that there are people who studied mathematics are doing theorem proving in the language is fascinating, but the tone is very elevated as a result. A high bar for a newcomer — even a professional with 25 years programming experience — to face.

It became clear pretty fast that I wouldn’t have the faintest idea what I was looking at, but I still tried to see if I could get a sense of what using Haskell would be like. Search on phrases like “haskell performance”, “haskell in production”, “commercial use of haskell”, “haskell vs scala”, and so on. You get more than enough highly partisan discussion. It’s quick to see people love the language. It’s a little harder to evidence see it being used in anger, but eventually I came across pages like Haskell in Industry and Haskell Weekly News which have lots of interesting links. That pretty much convinced me it’d be worth giving it a go.

A brief introduction

So here I am, struggling away learning Haskell. I guess I’d have to say I’m still a bit dubious, but the wonderful beginner tutorial called Learn You A Haskell For Great Good (No Starch Press) has cute illustrations. :) The other major starting point is Real World Haskell (O’Reilly). You can flip through it online as well, but really, once you get the idea, I think you’ll agree it’s worth having both in hard copy.

Somewhere along the way my investigations landed me on discussion of something called “software transactional memory” as an approach to concurrency. Having been a Java person for quite some years, I’m quite comfortable with multi-threading [and exceptionally tired of the rants from people who insist that you should only write single threaded programs], but I’m also aware that concurrency can be hard to get right and that solving bugs can be nasty. The idea of applying the database notion of transactions to memory access is fascinating. Reading about STM led me to this (short, language agnostic) keynote given at OSCON 2007 by one Simon Peyton-Jones, an engaging speaker and one of the original authors of GHC. Watching the video, I heard him mention that he had done an “introduction to Haskell” earlier in the conference. Huh. Sure enough, linked from here, are his slides and the video they took.

Watching the tutorial implies a non-trivial investment in time, and a bit of care to manually track the slides with him as he is presenting, but viewing it all the way through was a very rewarding experience. By the time I watched this I’d already read Learn You A Haskell and a goodly chunk of Real World Haskell, but if anything that made it even more fascinating; I suppose I was able to concentrate more on what he was saying for the emphasis on why things in Haskell are the way they were.

I was quite looking forward to how he would introduce I/O to an audience of beginners; like every other neophyte I’m grinding through learning what “monads” are and how they enable pure functional programming to coexist with side effects. Peyton-Jones’s discussion of IO turns up towards the end (part 2 at :54:36), when this definition went up on a slide:

IO (a) :: World -> (a, World)

accompanied by this description:

“You can think of it as a function that takes a World to a pair of a and a new World … a rather egocentric functional programmer’s view of things in which your function is center of the universe, and the entire world sort of goes in one side of your function, gets modified a bit by your function, and emerges, in a purely functional way, in a freshly minted world which comes out the other…”

“Oh, so that’s a metaphor?” asked one of his audience.

“Yes. The world does not actually disappear into your laptop. But you can think of it that way if you like.”

Ha. :)

Isolation and reusability

A moment ago I mentioned practicality. The most practical thing going these days is the web problem, i.e. using a language and its toolchain to do web programming. Ok, so what web frameworks are there for Haskell? Turns out there are a few. Two newer ones in particular, Yesod and the Snap Framework. Their raw performance as web servers looks very impressive, but the real question is how does writing web pages & application logic go down? Yesod’s approach, called “Hamlet“, doesn’t do much for me. I can see why type safety across the various pieces making up a web page would be something you’d aspire to, but it ain’t happening (expecting designers to embed their work in a pseudo-but-not-actually HTML language has been tried before. Frequently. And it’s been a bust every time). Snap, on the other hand, has something called “Heist“. Templates are pure HTML and when you need to drop in programmatically generated snippets you do so with a custom tag that gets substituted in at runtime. That’s alright. As for writing said markup from within code there’s a different project called “Blaze” which looks easy enough to use.

Reading a thread about Haskell web programming, I saw explicit acknowledgement on the part of framework authors from all sides that it would be possible to mix and match, at least in theory. If you like Yesod’s web server but would rather to use Snap’s Heist template engine, you could probably do so. You’d be in for all the glue code and knowing what you’re about, but this still raises an interesting point.

A big deal with Haskell — and one of the core premises of programming in a functional language that emphasizes purity and modularity — is that you can rely on code from other libraries not to interfere with your code. It’s more than just “no global variables”; pure functions are self contained, and when there are side effects (as captured in IO and other monads) they are explicitly marked and segregated from pure code. In IT we’ve talked about reusable code for a long time, and we’ve all struggled with it: the sad reality is that in most languages, when you call something you have few guarantees that nothing else is going to happen over and above what you’ve asked for. The notion of a language and its runtime explicitly going out of its way to inhibit this sort of thing is appealing.

Hello web, er world

Grandiose notions aside, I wanted to see if I could write something that felt “clean”, even if I’m not yet proficient in the language. I mentioned above that I liked the look of Snap. So, I roughed out some simple exercises of what using the basic API would be like. The fact that I am brand new at Haskell of course meant it took a lot longer than it should have! That’s ok, I learnt a few things along the way. I’ll probably blog separately about it, but after an essay about elegance and pragmatism, I thought I should close with some code. The program is just a little ditty that echos your HTTP request headers back to you, running there. You can decide for yourself if the source is aesthetically pleasing; ’tis a personal matter. I think it’s ok, though I’m not for a moment saying that it’s “good” style or anything. I will say that with Haskell I’ve already noticed that what looks deceptively simple often takes a lot of futzing to get the types right — but I’ve also noticed that when something does finally compile, it tends to be very close to being done. Huh.

So here I am freely admitting that I was quite wrong about Haskell. It’s been a bit of a struggle getting started, and I’m still a bit sceptical about the syntax, but I think the idea of leveraging Haskell shows promise, especially for server-side work.

AfC


November 28, 2011 05:35 AM

Mary Gardiner

Your friendly guide to talking to me about being tall

Scene setting: I’m 193cm/6’4″ tall. The average height of an Australian woman is about 163cm, so conveniently you can think of me as being a whole ruler taller, or that the average Australian woman’s head is about my shoulder height. This is a weird enough height that I’ve had all kinds of weird conversations about it. Let me get you past the weird.

Rule 1: consider not talking to a tall person about their height. It’s hard to do well. Think of it like this:
Person 1: “your body has a very very unusual feature! very unusual! very unusual!”
Person 2: “whereas your body does not! very normal! very normal!”

It’s a pretty one way conversation, basically. It’s unlikely (statistically) that they can reciprocate in kind by asking you/informing you about your visible weirdnesses, and if they can, it’s likely you don’t want to hear about your weirdnesses. The conversation in reality goes something like this:

Person 1: you are very very tall!
Person 2: um, indeed.
Person 1: [waits patiently for tall person to work harder to pull their turn out of the magical conversation hat]

Or alternatively, the general rule is start conversations where the person you are talking to has some chance of reciprocation.

Rule 2: especially consider not talking to a tall child or teenager about their height! This is because people generally make free with subjecting children and teenagers to every thought that crosses their mind, usually prescriptively at that. I am probably down to a conversation every few months about my height now. When I was a teenager, I had a conversation with a stranger about my height about once a week. That person who by virtue of youth (*cough* and gender) is extra socially obliged to stand there and look polite while they hear your every thought about human height variations? You’re not the only person taking advantage.

Rule 3: I’ve heard the jokes. Useful rule in general for anyone who has what you consider an unusual body, name, accent, hair colour, job, dress, religious belief, ethnic identity, mobility aid, manner of speaking, hobby, and/or other thing.

I have to say, I’m yet to hear what I’d call a good tall joke, but then, I would be biased, wouldn’t I?

Rule 4: I don’t need to know about how unattractive you find it. I won’t belabour this: if you’re the kind of person who tells tall people they are ugly or freaky (in my case, this was almost exclusively done by men to my teenage self, men in late middle age still occasionally do it now), you’re the kind of person who isn’t reading.

Incidentally, the favoured insult for a tall slender woman you’ve just seen on the street and instantly been repelled by is “lanky bitch” or “fucking lanky bitch“. In case it ever comes up in a trivia quiz or something. Who the hell uses the word ‘lanky’?

Rule 5: I don’t want to hear about how jealous you are. This is more complicated and interesting. When I was in my late teens, most of those people stopping me to talk to me about it were middle-aged women* wanting to tell me I was beautiful and special and should stand up straight and be proud and they wished they were me.

It took me ages to work out what was going on, which is that each of these women thought she was the only one and was lighting a torch in the misery of my teen years. Since it happened several times a month, I had no notion that they thought that, and they must have been rather unsettled by my awkward and slightly hostile reaction to their attempt to reach through the fog of human cruelty with a kind thought. Sorry, kind women.

* Um, possibly adult women? I wasn’t good at picking adult’s ages at the time.

Rule 6: unless you are my doctor, I don’t want to discuss my genetic history with you. I’m not sure why everyone wants to know whether my parents are tall (oh what the hell: yes, they are, and if the human race consisted entirely of my father’s relatives, I would be at the tall end of normal, rather than at the “having conversations with strangers and writing blog entries” level). It seems kind of weird to be led through a laundry list of my relatives and asked if they are tall. Are people trying to find out if their own children will/won’t/might be tall?

A special note to doctors on this one: you don’t get out of gaol free! It might help to explain why you’re asking. “There are some diseases and syndromes which have extreme height as a symptom, but if your whole family is tall that’s less likely” is an example of a helpful thing to say. (At my height-for-sex, I suspect you can just about get away with saying “so, Marfan syndrome**, you either have it or have been investigated for it, yeah?”) But since quite a few doctors have done this out of either a desire for chitchat equivalent to the general public or a desire to satisfy some medical curiosity irrelevant to their treatment of me, I don’t like it much from doctors without explanation either. I am all good with doctor chitchat, but not about something where I can’t tell if you think I have a disease or you have a few minutes to shoot the breeze with me.

** Not the only medically interesting cause of tallness, I know.

Rule 7: I will be the judge of whether I can wear heels, thank you. I don’t wear high ones because OUCH and also because there’s absolutely no social advantage to me from being taller, quite the reverse. But I sometimes wear low ones because I like the shoes they are attached to, and every so often a sales assistant refuses to sell them to me. What the hell?

Rule 8: It’s not good news for me that there’s someone taller than you. Actual remark addressed to me on several occasions: “wow, oh my god, you’re taller than me! I feel so good knowing that there’s a woman taller than me out there!” Only about half the time do they go on to realise what that implies from my point of view.

I do see the temptation to start conversations with other tall people about how they are taller than me, but when I do I remember this.

Rule 9: You don’t need to worry about what your kids say. Well, unless it’s “fucking lanky bitch” I guess. But kids specialise in drive-bys: “that lady is very tall!” I don’t mind stating-the-obvious drive-bys, it’s cute.

The champion kid remark to date was while I was pregnant: “Mummy, that lady is very tall and she has a baby in her tummy!” Indeed!

Rule 10: I am all good with reaching stuff on high shelves for you. Maybe this bugs some tall people, certainly people apologise a lot for asking me to do this, but it seems fair enough, really. Why do shelves intended for the general public go so high anyway?

Rule 11: I like to show off. I can touch the ceiling (on tiptoes) in normal height modern rooms. (I use this to change lightbulbs.) I can stand flat-feet on the bottom of a 1.8m depth pool (the usual depth of recreational pools) and it comes up to about my mouth. I almost never get the chance to mention these things to people! Humour me. (OK, you don’t have to, now that you’ve read this.)

Rule 12: If you’ve known me for ages and have secretly always wanted to talk to me about being tall, I usually don’t mind much of this from people I know. I guess the ugly thing would be an exception, but really, it’s strangers bowling up to me and asking about the height of my great-great-grandfather’s sister that comprises 99% of the problem.


November 28, 2011 04:36 AM

-----

November 24, 2011

Mary Gardiner

Parenting economics

From Matt Yglesias:

Family life is subject to a vicious economic conundrum known as Baumol’s cost disease. Economy-wide wages are linked to economy-wide productivity. That means that over time sectors of the economy that don’t feature productivity gains will see rapidly rising costs…

Child-rearing is basically stick stuck in a kind of dark ages of artisanal production, but as market wages have risen the opportunity cost of this extremely labor intensive line of work has steadily increased. The implication is that societies that want to continue existing in the future are increasingly going to have to find ways to subsidize parental investment in the next generation.


November 24, 2011 12:07 AM

-----

November 23, 2011

Sam Johnston

Infographic: Diffusion of Social Networks — Facebook, Twitter, LinkedIn and Google+

Social networking market

They say a picture's worth a thousand words and much digital ink has been spilled recently on impressive sounding (yet relatively unimpressive) user counts, so here's an infographic showing the diffusion of social networks as at last month to put things in perspective.

There are 7 billion people on the planet, of which 2 billion are on the Internet. Given Facebook are now starting to make inroads into the laggards (e.g. parents/grandparents) with 800 million active users already under their belt, I've assumed that the total addressable market (TAM) for social media (that is, those likely to use it in the short-medium term) is around a billion Internet users (i.e. half) and growing — both with the growth of the Internet and as growing fraction of Internet users. That gives social media market shares of 80% for Facebook, 20% for Twitter and <5% for Google+. In other words, Twitter is 5x the size of Google+ and Facebook is 4x the size of Twitter (e.g. 20x the size of Google+).

It's important to note that while some report active users, Google report total (e.g. best case) users — only a percentage of the total users are active at any one time. I'm also hesitant to make direct comparisons with LinkedIn as while everyone is potentially interested in Facebook, Twitter and Google+, the total addressable market for a professional network is limited, by definition, to professionals — I would say around 200 million and growing fast given the penetration I see in my own professional network. This puts them in a similar position to Facebook in this space — up in the top right chasing after the laggards rather than the bottom left facing the chasm.

Diffusion of innovations

The graph shows Rogers' theory on the diffusion of innovations, documented in The Innovator's Dilemma, where diffusion is the process by which an innovation is communicated through certain channels over time among the members of a social system. There are 5 stages:
  1. Knowledge is when people are aware of the innovation but don't know (and don't care) about it.
  2. Persuasion is when people are interested in learning more.
  3. Decision is when people decide to accept or reject it.
  4. Implementation is when people employ it to some degree for testing (e.g. create an account).
  5. Confirmation is when people finally decide to use it, possibly to its full potential.
I would suggest that the majority of the total addressable market are at stage 1 or 2 for Google+ and Twitter, and stage 4 or 5 for Facebook and LinkedIn (with its smaller TAM). Of note, users' decisions to reject an innovation at the decision or implementation phase may be semi-permanent — to quote Slate magazine's Google+ is Dead article, "by failing to offer people a reason to keep coming back to the site every day, Google+ made a bad first impression. And in the social-networking business, a bad first impression spells death." The same could be said for many users of Twitter, who sign up but fail to engage sufficiently to realise its true value. Facebook, on the other hand, often exhibits users who leave only to subsequently return due to network effects.

Social networking is also arguably a natural monopoly given, among other things, dramatically higher acquisition costs once users' changing needs have been satisfied by the first mover (e.g. Facebook). Humans have been using social networking forever, only until recently it's been manual and physiologically limited to around 150 connections (Dunbar's number, named after British anthropologist Robin Dunbar). With the advent of technology that could displace traditional systems like business cards and rolodexes came a new demand for pushing the limits for personal and professional reasons — I use Facebook and LinkedIn extensively to push Dunbar's number out an order of magnitude to ~1,500 contacts for example, and Twitter to make new contacts and communicate with thousands of people. I don't want to maintain 4 different social networks any more than I want to have to search 4 different directories to find a phone number — I already have 3 which is 2 too many!

Rogers' 5 factors

How far an innovation ultimately progresses depends on 5 factors:
  1. Relative Advantage — Does it improve substantially on the status quo (e.g. Facebook)?
  2. Compatibility — Can it be easily assimilated into an individual's life?
  3. Simplicity or Complexity — Is it too complex for your average user?
  4. Trialability — How easy is it to experiment?
  5. Observability — To what extent is it visible to others (e.g. for viral adoption)
Facebook, which started as a closed community at Harvard and other colleges and grew from there, obviously offered significant relative advantage over MySpace. I was in California at the time and it seemed like everyone had a MySpace page while only students (and a few of us in local/company networks) had Facebook. It took off like wildfire when they solved the trialability problem by opening the floodgates and a critical mass of users was quickly drawn in due to the observability of viral email notifications, the simplicity of getting up and running and the compatibility with users' lives (features incompatible with the unwashed masses — such as the egregiously abused "how we met" form — are long gone and complex lists/groups are there for those who need them but invisible to those who don't). Twitter is also trivial to get started but can be difficult to extract value from initially.

Network models

Conversely, the complexity of getting started on Google+ presents a huge barrier to entry and as a result we may see the circles interface buried in favour of a flat "follower" default like that of Twitter (the "suggested user list" has already appeared), or automated. Just because our real-life social networks are complex and dynamic does not imply that your average user is willing to invest time and energy in maintaining a complex and dynamic digital model. The process of sifting through and categorising friends into circles has been likened to the arduous process of arranging tables for a wedding and for the overwhelming majority of users it simply does not offer a return on investment:
In reality we're most comfortable with concentric rings, which Facebook's hybrid model recently introduced by way of "Close Friends", "Acquaintances" and "Restricted" lists (as well as automatically maintained lists for locations and workplaces — a feature I hope gets extended to other attributes). By default Facebook is simple/flat — mutual/confirmed/2-way connections are "Friends" (though they now also support 1-way follower/subscriber relationships ala Twitter). Concentric rings then offer a greater degree of flexibility for more advanced users and the most demanding users can still model arbitrarily complex networks using lists:
In any case, if you give users the ability to restrict sharing you run the risk of their actually using it, which is a sure-fire way to kill off your social network — after all, much of the value derived from networks like Facebook is from "harmless voyeurism". That's why Google+ is worse than a ghost town for many users (including myself, though as a Google Apps users I was excluded from the landrush phase) while being too noisy for others. Furthermore, while Facebook and Twitter have a subscribe/follow ("pull") model which allows users to be selective of what they hear, when a publisher shares content with circles on Google+ other users are explicitly notified ("push") — this is important for "observability" but can be annoying for users.

Nymwars

The requirement to provide and/or share your real name, sex, date of birth and a photo also presents a compatibility problem with many users' expectations of privacy and security, as evidenced by the resulting protests over valid use cases for anonymity and pseudonymity. For something that was accepted largely without question with Facebook, the nymwars appear to have caused irreparable harm to Google+ in the critically important innovator and early adopter segments, for reasons that are not entirely clear to me. I presume that there is a greater expectation of privacy for Google (to whom people entrust private emails, documents, etc.) than for Facebook (which people use specifically and solely for controlled sharing).

Adopter categories

Finally, there are 5 classes of adopters (along the X axis) varying over time as the innovation attains deeper penetration:
  1. Innovators (the first 2.5%) are generally young, social, wealthy, risk tolerant individuals who adopt first.
  2. Early Adopters (the next 13.5%) are opinion leaders who adopt early enough (but not too early) to maintain a central communication position.
  3. Early Majority (the next 34%, to 50% of the population) take significantly longer to adopt innovations.
  4. Late Majority (the next 34%) adopt innovations after the average member of society and tend to be highly sceptical.
  5. Laggards (the last 16%) show little to no opinion leadership and tend to be older, more reclusive and have an aversion to change-agents.
I've ruled out wealth because while buying an iPhone is expensive (and thus a barrier to entry), signing up for a social network is free.

The peak of the bell curve is the point at with the average user (e.g. 50% of the market) has adopted the technology, and it is very difficult both to climb the curve as a new technology and to displace an existing technology that is over the hump.

The Chasm

The chasm (which exists between Early Adopters and Early Majority i.e. at 16% penetration), refers to Moore's argument in Crossing the Chasm that there is a gap between early adopters and the mass market which must be crossed by any innovation which is to be successful. Furthermore, thanks to accelerating technological change they must do so within an increasingly limited time for fear of being equaled by an incumbent or disrupted by another innovation. The needs of the mass market differ — often wildly — from the needs of early adopters and innovations typically need to adapt quickly to make the transition. I would argue that MySpace, having achieved ~75 million users at peak, failed to cross the chasm by finding appeal in the mass market (ironically due in no small part to their unfettered flexibility in customising profiles) and was disrupted by Facebook. Twitter on the other hand (with some 200 million active users) has crossed the chasm, as evidenced by the presence of mainstream icons like Bieber, Spears and Obama as well as their fans. LinkedIn (for reasons explained above) belongs at the top right rather than the bottom left.

Disruptive innovations

The big question today is whether Google+ can cross the chasm too and give Facebook a run for its money. Facebook, having achieved "new-market disruption" with almost a decade head start in refining the service with a largely captive audience, now exhibits extremely strong network effects. It would almost certainly take another disruptive innovation to displace them (that is, according to Clayton Christensen, one that develops in an emerging market and creates a new market and value network before going on to disrupt existing markets and value networks), in the same way that Google previously disrupted the existing search market a decade ago.

In observing that creating a link to a site is essentially a vote for that site ("PageRank"), Google implemented a higher quality search engine that was more efficient, more scalable and less susceptible to spam. In the beginning Backrub Google was nothing special and the incumbents (remember Altavista?) were continuously evolving — they had little to fear from Google and Google had little to fear from them as it simply wasn't worth their while chasing after potentially disruptive innovations like Backrub. They were so disinterested in fact that Yahoo! missed an opportunity to acquire Google for $3bn in the early days. Like most disruptive technologies, PageRank was technologically straightforward and far simpler than trying to determine relevance from the content itself. It was also built on a revolutionary hardware and software platform that scaled out rather than up, distributing work between many commodity PCs, thus reducing costs and causing "low-end disruption". Its initial applications were trivial, but it quickly outpaced the sustaining innovation of the incumbents and took the lead, which it has held ever since:

Today Facebook is looking increasingly disruptive too, only in their world it's no longer about links between pages, but links between people (which are arguably far more valuable). Last year while working at Google I actively advocated the development of a "PageRank for people" (which I referred to as "PeopleRank" or "SocialRank"), whereby a connection to a person was effectively a vote for that person and the weight of that vote would depend on the person's influence in the community, in the same way that a link from microsoft.com is worth more than one from viagra.tld (which could actually have negative value in the same way that hanging out with the wrong crowd negatively affects reputation). I'd previously built what I'd call a "social metanetwork" named "meshed" (which never saw the light of day due to cloud-related commitments) and the idea stemmed from that, but I was busy running tape backups for Google, not building social networks on the Emerald Sea team.

With the wealth of information Google has at its fingertips — including what amounts to a pen trace of users' e-mail and (courtesy Android and Google Voice) phone calls and text messages — it should have been possible for them to completely automate the process of circle creation, in the same way that LinkedIn Maps can identify clusters of contacts. But they didn't (perhaps because they got it badly wrong with Buzz), and they're now on the sustaining innovation treadmill with otherwise revolutionary differentiating features being quickly co-opted by Facebook (circles vs lists, hangouts vs Skype, etc).

Another factor to consider is that Google have a massive base of existing users in a number of markets that they can push Google+ to, and they're not afraid to do so (as evidenced by its appearance in other products and services including Android, AdWords, Blogger, Chrome, Picasa, Maps, News, Reader, Talk, YouTube and of course the ubiquitous sandbar and gratuitous blue arrow which appeared on Google Search). This strategy is not without risk though as if successful it will almost certainly attract further antitrust scrutiny, in the same way that Microsoft found itself in hot water for what was essentially putting an IE icon on the desktop. Indeed I had advocated the deployment of Google+ as a "social layer" rather than isolated product (ala the defunct Google Buzz), but stopped short of promoting an integrated product to rival Facebook — if only to maintain a separation of duties between content production/hosting and discovery.

The solution

While I'm happy to see some healthy competition in the space, I'd rather not see any of the social networks "win" as if any one of them were able to cement a monopoly then us users would ultimately suffer. At the end of the day we need to remember that for any commercial social network we're not the customer, we're the product being sold:
As such, I strongly advocate the adoption of open standards for social networking, whereby users select a service or host a product that is most suitable for their specific needs (e.g. personal, professional, branding, etc) which is interoperable with other, similar products.

What we're seeing today is similar to the early days of Internet email, where the Simple Mail Transfer Protocol (SMTP) broke down the barriers between different silos — what we need is an SMTP for social networking.

References:
Sources:


November 23, 2011 08:04 PM

Ozone

Two new mixes

I've been pretty dormant in my music for the past few years, but I have been working on two two mixes in my sparse spare time: "Tes Lyric":/music/tes_lyric, a weird blend of electronica, classical and rock, and "Stage Superior":/music/ss, a progressive house mix. They're up on my "music":/music page now; enjoy!


November 23, 2011 04:10 PM

Russell Coker

Receiving Death Threats

On occasion I receive death threats in comments on my blog. This doesn’t bother me, I had more than a few pathetic losers say similar things to me in high school but the difference is that a pathetic loser on the other side of the world can’t do much about it.

The latest one is on my post “Is Asperger Syndrome a Good Thing?” [1]. At 11:04AM my time (00:04 UTC) on the 23rd of November someone using IP address 82.14.50.190 (registered to “NTL Infrastructure for Tesco – Guildford”) and with the claimed email address of sharkbait @ tesco.net told me that I’m on his “to kill list“.

He was angry because my post apparently didn’t satisfy his need to “prove to people that they don’t want AS“. As it’s not possible to choose your DNA there is really no point in trying to prove that one set of genes is better than another, unless of course you want to harass people who have different genes to you.

Naturally he didn’t read my post, in fact the differences between his comment and what I wrote made me suspect that it’s something he was pasting in to lots of blogs, but it seems unlikely that he would use an iPhone for such a copy/paste job so he must just not read. For example he rhetorically asks “did you even mention it was a form of autism in your ‘negative points’” when I gave a link to the proposed DSM changes where Asperger Syndrome is being merged with “Autistic Disorder” and the word “Autism” appears 5 times in the post.

He has one specific complaint about people he knows who are on the Autism Spectrum, he says “Every time someone speaks to them they get angry and tell said person to back the fuck off. I’ve been on the receiving end.” Generally there are two possible reasons for being told to “fuck off”, one is that the person telling you is an asshole and the other is that you are being an asshole. If you are the type of person who sends death threats to bloggers then you really should consider the latter explanation.

He also suggests that I watch Boston Legal – presumably because that show has one character who is an Aspie. Now I have actually watched that show and it’s the worst legal show I’ve ever seen, when a legal show has such unrealistic portrayal of court scenes I don’t think it should be relied on for information on any other topic. I watched a couple of episodes of Boston Legal and only saw one scene with the Aspie guy, so I can’t even determine whether it’s a realistic portrayal – he’s a minor character. In any case there are plenty of Aspie guys in the Free Software community who I interact with regularly, I read forums such as AspiesForFreedom.com, and I’ve read some text books on the topic. So even if a TV show had a very accurate portrayal of an Aspie character I still doubt that I would learn anything by watching it.

As a general tip for flamers, if you want to accuse someone of being a member of a NAZI or KKK type organisation then the word you are after is Aryan not “ayrien“. If you don’t know a word then don’t use it.

Please note the conditions of sending me email [2], I reserve the right to publish all threats in any way that I wish.

A Final Note

Thanks sharkbait for adding me to your “to kill list“, I’m sure that there are lots of nice people on it and I’ve got good company.

If I didn’t get the occasional death threat I would worry that I’m not writing well enough or failing to address important issues. I will write more about Autism in the future.


November 23, 2011 02:14 PM

-----

November 22, 2011

Dave Ruys

ewokracing: directionlessdrive: Oh gawd yes. *drops to knees*...



ewokracing:

directionlessdrive:

Oh gawd yes. *drops to knees* please santa.

My Mazda 1300 will look like this one day.

I think we need to be clear here: NO Mazda 1300 will ever look anywhere near as good as a Mk 1 Escort. I say that as a dyed-in-the-wool 1300 nut. I love the little beasts to death but the styling of the Mk 1 is effectively 1970s design perfection. Sad that the Mk 2 looked so incredibly dull.

I do, however, look forward to seeing the ewok 1300 in this state of uber-workedness one day.


November 22, 2011 01:56 AM

-----

November 21, 2011

Mary Gardiner

Computational linguists

xkcd suddenly exploded in my circles in 2006, thanks to the comic Randall Munroe calls Computational Linguists and most people refer to as “Fuck Computational Linguistics” getting around at the annual conference of the Association for Computational Linguistics.

There’s been requests for the xkcd store to sell it before, but it’s never been done.

I just ordered a batch through Sticker Mule, both of the full comic and of a smaller badge version I did. (They will do proofs of them, I’ll be interested to see if the “Fuck” bugs them.) In order to do so I did a vector version of the comic (via Inkscape’s “trace bitmap”), and because the original comic, and these variants, are under Creative Commons Attribution NonCommercial, I can share them with you here. If you want them, order copies from the sticker vendor of your choice!

Full comic:
Indicative PNG | Compressed Inkscape SVG | PDF (fonts as paths)

Smaller badge-like variant:

Fuck Computational Linguistics
Compressed Inkscape SVG | PDF (fonts as paths)

The vector versions aren’t very clean, but neither is the original comic, so I’m hoping these look like the spirit of the original, rather than a nasty hack.

Reminder: these are licensed for free noncommercial use (the precise condition is noncommercial use with attribution to the original author, modifications OK). So don’t sell them!


November 21, 2011 09:33 AM

Andrew Cowie

A good GNOME 3 Experience

I’ve been using GNOME 3 full time for over 9 months, and I find it quite usable. I’ve had to learn some new usage patterns, but I don’t see that as a negative. It’s a new piece of software, so I’m doing my best to use it the way it’s designed to be used.

Sure, it’s different than GNOME 2. It’s vastly different. But it is a new UI paradigm. The GNOME 2 experience was over 9 years old, and largely based on the experience inherited from the old Windows 95 muxed with a bit of CDE. There were so many things that the GNOME hackers wanted to do — and lots of things all the UI studies said needed changing — that the old pattern simply couldn’t support.

Still, a lot of people are upset. Surprise. Most recently it’s been people running Debian Testing who just recently found that their distro has migrated its packages from GNOME 2.32 to GNOME 3.x. Distros like Ubuntu have been shipping GNOME 2.32 for ages; but it has been well over 2 years since anyone actually worked on that code. It’s wonderful that nothing has changed for you in all that time [a true Debian Stable experience!] but I think it’s a bit odd not to expect that something that was widely advertised as being such a different user experience is … different.

What I find annoying about these conversations is that if they had gone and bought an Apple laptop with Mac OS X on it they would be perfectly reasonably working through learning how to use a new Desktop and not complaining about it at all. But here we are admonishing the GNOME hackers had the temerity to do something new and different.

Installing

I went to some trouble to run GNOME 3 on Ubuntu Linux during the Natty cycle; that was a bit of work but I needed to be current; now with Oneiric things are mostly up to date. GNOME 3.0 was indeed a bit of a mess, but then so was GNOME 2.0. The recently released 3.2 is a big improvement. And it looks like the list of things that seem targeted to 3.4 will further improve things.

I’m now running GNOME 3 on a freshly built Ubuntu Oneiric system; I just did a “command line” install of Ubuntu and then installed gdm, gnome-shell, xserver-xorg and friends. Working great, and not having installed gnome-desktop saved me a huge amount of baggage. Of course a normal Oneiric desktop install and then similarly installing and switching to gnome-shell would work fine too; either way you probably want to enable the ppa:gnome3-team/gnome3 PPA.

Launchers

One thing I do recommend is mapping (say) CapsLock as an additional “Hyper” and then Caps + F1 .. Caps + F12 as launchers. I have epiphany browser on F1, evolution on F2, my IRC client on F3 and so on. Setting up Caps + A as to do gnome-terminal --window means you can pop a term easily from anywhere. You do the mapping in:

    System Settings → Keyboard Layout → Layout tab → Options...

and can set up launchers via:

    System Settings → Keyboard → Shortcuts tab → "Custom Shortcuts" → `[+]` button

(you’d think that’d all just be in one capplet, but anyway)

Not that my choices matter, per se, but to gives you an idea:

AcceleratorLaunchesDescription
Caps + F1 epiphany Web browser (primary)
Caps + F2 evolution Email mail
Caps + F3 pidgin IRC client
Caps + F4 empathy Jabber client
Caps + F5 firefox Web browser (alternate)
Caps + F6 shotwell Photo manager
Caps + F7 slashtime Timezone utility
Caps + F8 rhythmbox Music player
Caps + F9 eclipse Java IDE
Caps + F10 devhelp GTK documentation
Caps + F11 gucharmap Unicode character picker
Caps + F12 gedit Text editor
Caps + Z gnome-terminal --window New terminal window

That means I only use the Overview’s lookup mechanism (ie typing Win, T, R, A… in this case looking for the Project Hamster time tracker) for outlying applications. The rest of the time it’s Caps + F12 and bang, I’ve got GEdit in front of me.

Of course you can also set up the things you use the most on the “Dash” (I think that’s what they call it) as favourites. I’ve actually stopped doing that (I gather the original design didn’t have favourites at all); I prefer to have it as an alternative view of things that are actually running.

Extensions

People love plugin architectures, but they’re quite the anti-pattern; over and above the software maintenance headache (evolving upstream constantly breaks APIs used by plugins, for one example; the nightmare of packaging plugins safely being another) before long you get people installing things with contradictory behaviour and which completely trash the whole experience that your program was designed to have in the first place.

Case in point is that it didn’t take long after people discovered how to use the extension mechanism built into gnome-shell for people to start using it to implement … GNOME 2. Gawd.

Seeking that certainly is not my recommendation; as I wrote above the point of GNOME 3 and it’s new shell is to enable a new mode of interaction. Still, everyone has got their itches and annoyances, and so for my friends who can’t live without their GNOME 2 features, I thought I’d point out a few things.

There are a collections of GNOME Shell Extensions some of which appear to be packaged, i.e. gnome-shell-extensions-drive-menu for an plugin which gives you some kind of menu when removable devices are inserted. I’m not quite sure what the point of that is; the shell already puts something in the tray when you’ve got removable media. Whatever floats your boat, I guess. Out in the wild are a bunch more. The charmingly named GNOME Shell Frippery extensions by Ron Yorston has a bunch of plugins to recreate GNOME 2 features. Most are things I wouldn’t touch with a ten-foot pole (a bottom panel? Who needs it? Yo, hit the Win key to activate the Overview and you see everything!).

My personal itch was wanting to have 4 fixed workspaces. The “Auto Move Workspaces” plugin from gnome-shell-extensions was close (and would be interesting if its experience and UI were properly integrated into the primary shell experience), but the “Static Workspaces” plugin from gnome-shell-frippery did exactly the trick. Now I have four fixed workspaces and I can get to them with Caps + 1 .. Caps + 4. Hurrah.

You install the plugin by dropping the Static_Workspaces@rmy.pobox.com/ directory into ~/.local/share/gnome-shell/extensions/, then restarting the Shell via Alt + F2, R, and then firing up gnome-tweak-tool and activating the extension:

    Advanced Settings → Shell Extension tab → switch "Static Workspaces Extension" to "On"

Hopefully someone will Debian package gnome-shell-frippery soon.

Not quite properly integrated

Having to create custom launchers and fiddle around with plugins just to get things working? “Properly integrated” this ain’t, and that’s my fault. I respect the team hacking on GNOME 3, and I know they’re working hard to create a solid experience. I feel dirty having to poke and tear their work apart. Hopefully over the next few release cycles things like this will be pulled into the core and given the polish and refined experience that have always been what we’ve tried to achieve in GNOME. What would be really brilliant, though, would be a way to capture and export these customizations. Especially launchers; setting that up on new machines is a pain and it’d be lovely to be able to make it happen via a package. Hm.

AfC


November 21, 2011 06:28 AM

Rusty

The Power of Undefined Values

Tools shape the way we work, because they change where we perceive risk when we write code.  If common compilers warn about something, I’ll code in a way that will trigger it in case of mistakes.  eg: instead of:

    int err = -EINVAL;
    if (something())
         goto out;
    err = -ENOSPC;
    if (something_else())
         goto cleanup_something;
...
cleanup_something:
    undo_something();
out:
    return err;

I would now set err in every branch:

    int err;
    if (something()) {
        err = -EINVAL;
        goto out;
    }
    if (something_else()) {
        err = -ENOSPC;
        goto out;
    }

Because when I add another clause to the initialization and forget to set err, gcc will warn me about it being uninitialized.  This bit me once, and it can be hard to spot the problem when you’re only reviewing a patch, not the code as a whole.

These days, we have valgrind, and despite its fame as a use-after-free debugger, it really shines at telling you when you rely on the results of an uninitialized field.  So, I’ve adapted to lean on it.  I explicitly don’t initialize structure members I don’t use in a certain path.  I avoid calloc(): while 0 is often less harmful than any other value, I’d much rather know that I’ve thought about and set up every field I actually use.  When changing code this is particularly important, and I spend a lot of my time changing code.  I have even changed to doing malloc() in some cases where I previously used on-stack or file-scope variables.  Valgrind doesn’t track on-stack usage very well, and static variables are defined to be zeroed, so valgrind can’t tell when I wander into the weeds.  I think these days, that’s a misfeature.

So, if I were designing a C-like language today, I’d bake in the concept of undefined values, knowing that the tools to leverage it are widely available.  10 years ago, I’d have said 0-by-default is safest, but times change.  I think Go chose wrong here, but it may not be as bad as C for other reasons.  I’d have to code in it for a few years to really tell.


November 21, 2011 06:04 AM

-----

November 20, 2011

Russell Coker

Galaxy S vs Xperia X10 and Android Network Access

Galaxy S Review I’ve just been given an indefinite loan of a Samsung Galaxy S which is more useful than the Sony Ericsson Xperia X10 that I own. I think that the main benefit is that it runs Android 2.2 instead of Android 2.1 on the Xperia. 2.2 is what gives it USB tethering support [...]


November 20, 2011 03:47 PM

-----

November 17, 2011

Russell Coker

Cruises

It seems that in theory cruises can make for quite economical vacations. The cheapest prices tend to be around $100 per person per night for an “inside” cabin (IE no window) with two people (there is a significant extra fee for having a cabin with only one person). If you book a room in a [...]


November 17, 2011 06:47 AM

-----

November 14, 2011

Dave Ruys

Female colleague 1: Your tie looks nice today.

Female colleague 1: Your tie looks nice today.
Me: That wasn't the intention.
FC2: It looks like a candy store!
Me: Is that a good thing?
FC2: It depends on whether you're trying to lure the children
Me: That's....not a good thing....


November 14, 2011 11:15 PM

-----

November 12, 2011

Russell Coker

Good Riddance to Flash

The Age reports that Adobe has ceased development of Flash for mobile systems [1]. This is described as leading to an improvement in the web experience for iPhone and iPad users, but the more important thing is that it will improve the experience for everyone. The Flash plugin has always been a resource hog and [...]


November 12, 2011 12:47 PM

Matt Palmer

Misleading error messages from blktrace

If you ever get an error message from the blktrace tool that looks like this:

BLKTRACESETUP(2) /dev/dm-0 failed: 2/No such file or directory
Thread 3 failed open /sys/kernel/debug/block/(null)/trace3: 2/No such file or directory
Thread 2 failed open /sys/kernel/debug/block/(null)/trace2: 2/No such file or directory
Thread 0 failed open /sys/kernel/debug/block/(null)/trace0: 2/No such file or directory
Thread 1 failed open /sys/kernel/debug/block/(null)/trace1: 2/No such file or directory
FAILED to start thread on CPU 0: 1/Operation not permitted
FAILED to start thread on CPU 1: 1/Operation not permitted
FAILED to start thread on CPU 2: 1/Operation not permitted
FAILED to start thread on CPU 3: 1/Operation not permitted

Don’t be alarmed – your disk hasn’t suddenly disappeared out from underneath you. In fact, it means quite the opposite of what “No such file or directory” might imply. In fact, it means that there is already a blktrace of that particular block device in progress, and you’ll need to kill that one off before you can start another one.

Thank $DEITY for the kernel source code – it was the only hope I had of diagnosing this particular nit before I went completely bananas and smashed my keyboard into small pieces.


November 12, 2011 05:00 AM

-----

November 11, 2011

Sam Johnston

RIP Adobe Flash (1996-2011) - now let's bury the dead

Adobe kills mobile Flash, giving Steve Jobs the last laugh, reports The Guardian's Charles Arthur following the late Steve Jobs' epic Thoughts on Flash rant 18 months ago. It's been about 2.5 years since I too got sick of Flash bringing my powerful Mac to its knees, so I went after the underlying lie that perpetuates the problem, explaining why Adobe Flash penetration is more like 50% than 99%. I even made progress Towards a Flash free YouTube killer, only it ended up being YouTube themselves who eventually started testing a YouTube HTML5 Video Player (while you're there please do your bit for the open web by clicking "Join the HTML5 Trial" at the bottom of that page).
I heard a sound as though a million restaurant websites cried out at once — Charles Arthur
You see, armed with this heavily manipulated statistic, armies of developers are to this day fraudulently duping their paying clients into deploying a platform that will invariably turn away a percentage of their business at the door, in favour of annoying flaming logos and other atrocities that blight the web:



How much business can you tolerate losing? If you've got 95% penetration then you're turning away 1 in 20 customers. At 90% you're turning away 1 in 10. At 50% half of your customers won't even get to see your product. I don't know too many businesses who can afford to turn away any customers in this economic climate.

In my opinion the only place Flash technology has in today's cloud computing environment is as a component of the AIR runtime for building (sub-par) cross-platform applications, and even then I'd argue that they should be using HTML5. As an Adobe Creative Suite Master Collection customer I'm very happy to see them dropping support for this legacy technology to focus on generating interactive HTML5 applications, and look forward to a similar announcement for desktop versions of the Flash player in the not too distant future.

In any case, with the overwhelming majority of devices being mobile today and with more and more of them including browser functionality, the days of Flash were numbered even before Adobe put the mobile version out of its misery. Let's not drag this out any longer than we have to, and bury the dead by uninstalling Flash Player. Here's instructions for Mac OS X and Windows, and if you're not ready to take the plunge into an open standards based HTML5 future then at least install FlashBlock for Chrome or Firefox (surely you're not still using IE?).

Update: Flash for TV is dead too, as if killing off mobile wasn't enough: Adobe Scrapping Flash for TV, Too‎

Update: Rich Internet Application (RIA) architectures in general are in a lot of trouble — Microsoft are killing off Silverlight as well: Mm, Silverlight, what's that smell? Yes, it's death

Update: In a surprising move that will no doubt be reversed, RIM announced it would continue developing Flash on the PlayBook (despite almost certainly lacking the ability to do so): RIM vows to keep developing Flash for BlackBerry PlayBook – no joke


November 11, 2011 01:06 PM

Mary Gardiner

linux.conf.au: program choices

I’m all but all booked in for linux.conf.au in Ballarat! (Need some accommodation in Melbourne for AdaCamp and to book the train to Ballarat.) So, time to share my early picks of the program:

Saturday (in Melbourne):

Monday:

Tuesday:

Wednesday:

Thursday:

Friday:

It’s skewed a little by my interests for the Ada Initiative now, that’s where all the mentoring stuff comes from. And I doubt I will get to all of this although presumably Valerie and I won’t be whisking people off to private meetings about the Ada Initiative as much. (At LCA 2011, when we were yet to launch it, we did almost nothing else.) It looks like Tuesday is a day to catch my breath before Wednesday. My family have decided to travel home Friday, so sadly Friday won’t be.


November 11, 2011 02:57 AM

-----

November 10, 2011

Dave Ruys

Thursday Gnar - Collingrove Cup by Gabe. Such a rad mix backed...



Thursday Gnar - Collingrove Cup by Gabe.

Such a rad mix backed with awesome jazz soundtrack. via skatehousemedia.


November 10, 2011 04:48 AM

-----

November 09, 2011

Sam Johnston

How NOT to respond to vulnerability reports


Reuven Cohen and the guys at Enomaly could write the book on how NOT to respond to vulnerability reports:
  1. Don't disavow vulnerabilities in products you've previously taken credit for
  2. Don't claim issues are not valid while denying researchers a right of reply
  3. Don't claim obvious issues are "unactionably vague" and then ignore them, even after a working exploit is publicly available
  4. Don't claim trivial remote root exploits are "theoretically valid but extremely difficult to exploit"
  5. Don't claim it's ok to rely on security by obscurity or race conditions
  6. Don't turn on moderation because a researcher posts a vulnerability report to your lists
  7. Don't subsequently ban a researcher from your lists because they tried to notify your users when you failed to
  8. Don't claim that security vulnerabilities are ok because there have been "no reports of any security compromise"
  9. Don't claim "other mitigating factors that have been present in the environment from the beginning" when the vulnerability has already been demonstrated
  10. Don't ask for private notification of vulnerabilities only to then ignore/dispute them
  11. Don't publicly call researchers unethical for opting for full disclosure, especially when they do so because you have been reticent and unresponsive in the past
  12. Don't release ineffective fixes, especially when the researcher has told you exactly how to fix it
  13. Don't dispute the vulnerability when a clearinghouse like Secunia contacts you to verify it
  14. Don't criticise researchers for reviewing your product
  15. Don't shoot the messenger
  16. Don't downplay critical vulnerabilities as "relatively minor", "random" paths as "pretty hard to guess", etc.
  17. Don't send in board members to fight your battles
  18. Don't claim new products having "significant new and enhanced functionality" is a valid excuse
  19. Don't make security claims like "High Assurance" if you're not going to take security seriously
  20. Don't claim that "Enomaly shall be entitled to (i) suspend or de-activate your account without notice, and (ii) retain any remaining funds in your account", and definitely don't actually do it.
After my recent SploitCloud: exploiting cloud brokers for fun and profit article and the follow-up Retro vulnerability of the day: cleartext passwords over the wire you'd have thought the publicly demonstrated vulnerabilities would have been quietly fixed and we'd have moved on. But no — they've decided instead to suspend my Spotcloud account so as I can't find any more holes, keeping funds they were holding in trust for payment to third-party providers as "compensation" — something I'm more inclined to refer to as "theft":

Enomaly have also not only failed to notify Spotcloud buyers and sellers that they are vulnerable themselves, but moderated (e.g. deleted) my notification to them and banned me from the lists in the process:

If I were one of the (apparently few) users of the Spotcloud service then I'd be extremely dissatisfied, to say the least, that this information was being actively concealed from me. At the end of the day you owe it to yourselves and your users to only ever work with providers who take security seriously.


November 09, 2011 12:57 PM

Retro vulnerability of the day: cleartext passwords over the wire

While spending my Sunday looking at what people are doing with various cloud platform services I came across these 4 case studies on the Google App Engine (GAE) pricing page:
Ignoring WebFilings (who have an Amazon EC2 backend) and gigya (who have their own platform and only use GAE for their live chat applet), Best Buy caught my eye as I already caught them sending employee credentials in the clear with the Twelpforce GAE app written by Enomaly a few years ago and Giftag was also done in "partnership" with Best Buy (whatever that means): "Enomaly Launches Giftag.com for Best Buy".

I also stumbled on a cross-site request forgery vulnerability in Enomaly's own flagship SpotCloud product earlier this year, which I wrote up last week — some 6 months after the initial report: SploitCloud: exploiting cloud brokers for fun and profit.

Sure enough when you crack out Wireshark and sniff the wire you can clearly see they're sending credentials in the clear over the public Internet, both at signup:

...and for good measure, on every login:
This wouldn't be such a problem were it not for rampant password reuse — I would not be at all surprised if most of the email/password combinations captured also worked on the email account itself. That is, by sniffing Giftag signups/logins you also have a good chance of a type of privilege escalation to the email account and from there to other services like Facebook:

To their credit(!?!), the other GAE case study application (Apmasphere, a property management application by Ray White, Australia's largest real estate group) exhibits exactly the same vulnerability, both at signup:


...and at login:

The moral of the story is that it doesn't matter how trivial your app is, given enough rope users will hang themselves by re-using passwords. As developers you owe it to your employers, clients and users to protect them from themselves, in this case by requiring SSL using Google App Engine's "secure: always" configuration directive which was introduced over 3 years ago. Very soon you'll also be able to use your own domains with SSL (rather than *.appspot.com) which, due to limitations in the protocol, is technically challenging to implement for a multi-tenant service at scale.

Update: While Best Buy's Giftag IP address is owned by Google (according to whois) and runs on the Google platform (according to the Server: Google Frontend HTTP header), the IP address for Ray White's Apmasphere is owned by Primus Telecommunications (according to whois) and runs an Apache web server (according to the Server: Apache HTTP header). Does anyone know whether one of the four main GAE case studies has indeed migrated to an in-house platform and if so, when and why? More to the point, is anyone aware of anyone doing anything of any consequence on GAE? I'm still looking for decent case studies of GAE native applications.

Update: Enomaly founder, Reuven Cohen disavows the vulnerability, claiming "Interestingly, the GAE version the giftag site wasn't developed by enomaly." SFAICT the "GAE version" is the only version so in my opinion they're responsible or they're plagiarists — taking someone else's work or ideas and passing them off as one's own. I'll let you decide for yourselves:

And 6 months later:

Update: The Giftag extension for Firefox is also vulnerable:

Update: Even the bookmarklet is vulnerable... if you add this to your toolbar and click on it then it will insecurely retrieve Javascript (gift-bookmarklet-loader.js) and execute it, even within an SSL session. That is, an attacker can trivially execute trusted code that has full access to secure pages:

javascript: (function () {
    var d = document;
    var s = d.createElement('script');
    s.id = "gt_boot";
    s.setAttribute('src', 'http://www.giftag.com:80/media/js/gift-bookmarklet-loader.js');
    d.getElementsByTagName('head')[0].appendChild(s);
})();


November 09, 2011 02:19 AM

SploitCloud: exploiting cloud brokers for fun and profit

My friends at Enomaly have been beating up on Amazon Web Services (AWS) over the XML signature element wrapping&nbsp;vulnerability currently being overhyped by the press, which&nbsp;is ironic given their security track record&nbsp;and unfortunate given I rather like what Amazon have achieved. Back in March I reported multiple vulnerabilities&nbsp;in&nbsp;SpotCloud&nbsp;(including their having copied Amazon's vulnerable signatures years after they were&nbsp;reported and fixed) and I was told I was unethical and my report that they "may not validate incoming web and/or API requests and if so, may be vulnerable to cross-site request forgery in which an attacker could make unauthorised management requests on behalf of a user" was "unactionably vague". To demonstrate the severity of the outstanding vulnerability go grab yourself a SpotCloud account, charge it up&nbsp;(ignoring PCI-DSS for a second given they're collecting credit card numbers via App Engine) and click the image below. I'll silently create an instance for you using a hidden IFRAME, but you're welcome to experiment with more destructive experiments like deleting existing instances and uploading malicious workloads. Update: If you look at the code you'll see the hourly rate is passed to the client as "cost" and presumably trusted on return (if not, why is it there?). I haven't seen a price manipulation vulnerability in over a decade, but I'm not tinkering with it because I don't fancy being accused of stealing from them or their providers. Update:&nbsp;While the consumer API now uses OAuth, the provider API still uses Amazon's vulnerable signatures for authentication: #sorts by key.lowercase(). ie A b c Dee e ffFf sorted_keys = sorted(parameters.keys(), key=lambda k: k.lower()) #concatenates key,value pairs. a=1,b=2,C=32 becomes "a1b2C32" data = ’’.join(key + parameters[key] for key in sorted_keys) #Data is now: ecp_usernamespotcloudusernameparamAvalueTimestamp2006-12-08T07:48:03Z digest = hmac.new(’spotcloudpassword’, data, sha).digest() This may have been safe over SSL were it not for the fact that client libraries (including python) typically don't validate the certificate chain by default. Update: Wells Fargo reports "CHECK CRD PURCHASE SPOT CLOUD ETOBICOKE CD"&nbsp;as "Unusual Activity" in emailed alert… canceling card, requesting re-issue. Should have used a virtual card. Wonder if Google know their&nbsp;App Engine poster child is using it to collect credit card details? Update: It is believed that Private SpotCloud and Enomaly Elastic Computing Platform (ECP) are also vulnerable to cross-site request forgery, but without access to the software I have no way to verify. Update: This is how Enomaly deals with security researchers:


November 09, 2011 01:42 AM

-----

November 08, 2011

Mary Gardiner

freelish.us: mental outage

It’s not absolutely clear to me that anyone at Geek Feminism has missed the linkspams, of which there hasn’t been one since 18th September. No one’s said anything, anyway.

What happened? freelish.us happened. Or it didn’t.

freelish.us, a bookmarking site using the open source status.net code, launched in April (April 1 actually, was that a good idea?) By that stage I was looking for an alternative to Delicious for bookmarking due to the new terms of service. I’ve been using flagship status.net site Identi.ca for microblogging for a long time (it cross-posts to Twitter) and while I’m inconsistent, I do like contributing to the commons to some degree, so a Creative Commons attribution bookmarking stream also appealed to me.

But the entire experience produced what I’d call “micro-burnout”. As in, I didn’t stop feeling pleasure or joy in stuff in general as would happen with burnout, but sharing links became a giant pain in the neck. Micro-burnout. Sharing links sucked.

First, there was the month or more on freelish.us where I just couldn’t seem to add bookmarks or import my Delicious backup file for love or money. I’d click “OK” and nothing would appear in my stream. It turned out that that was because I’d never validated my email address, but there was no error message to that effect, in fact no error message at all. I happened to see an understated warning elsewhere on the site that it was unvalidated, validated it, and suddenly the site actually worked.

Then there was the bookmarklet. The theory is visit a site, go to the bookmarklet, it’s bookmarked! On freelish.us it worked like this:

  1. go to the bookmarklet. This is pretty annoying in the first place, because I have a small laptop screen and bookmarklets require me to leave the bookmark toolbar visible. (I much prefer the Instaright approach, which places a small button in the URL bar, which is otherwise dead space anyway.)
  2. almost inevitably, find that I had been logged out of freelish.us, which must have had the most aggressively timed out cookies since linux.conf.au’s Zookeepr software (memo to Zookeepr: keep me logged in please)
  3. log in on the bookmarklet’s pop up
  4. be greeted with a small page saying I’ve logged in successful, but no sign of the entry form to bookmark what I needed reappearing
  5. back back back reload back back retry bookmarklet finally bookmark thing

And then, finally, on September 16, it and other status.net sites were taken down for upgrade. And now, nearly two months later, freelish.us home page still reads: “StatusNet cloud sites, including Identi.ca, are under maintenance. See status blog post for details and updates.”

Some facts about that:

  1. it’s not actually true any more: Identi.ca came back up after 24 hours or so
  2. it appears from comments there that any number of status.net sites are still down, and there’s been very little public comment on any of them that I can find. Several people asked specifically about freelish.us.

Also, freelish.us missed a probably once-off opportunity to captialise on the flight of horrified users of the new Delicious. But that’s not my concern.

All up, for two months the thought of bookmarking sites at all has made me distinctly “meh”, so, no linkspam for GF. This is what the software meh takes from the world.

I eventually decided that it was important to talk about what an annoying experience freelish.us has been, important enough to actually ask them for comment (via their press email contact). Here’s the information that as far as I can tell status.net has not communicated otherwise:

Q. What is the status of freelish.us? Is it going to return at some point or is it gone?

Evan Prodromou of status.net replied on the 30th October:

Freelish.us didn’t upgrade very well during the 1.0 process.

We’re moving to a new data centre this week, and I’m going to try to revive it then.

I fully intend to see it operational in early November.

There was a second question to which he didn’t directly reply, which was Q. In either event, is it possible for users of freelish.us to recover their bookmarks either for their own use or for import into another site? I take it from the lack of separate response that the re-appearance of the site will be the way in which users can recover their bookmarks and there is not an earlier alternative.

For the sake of the linkspams, I’m giving Pinboard a go. I’ll let you know how I do.


November 08, 2011 08:10 AM

-----

November 05, 2011

Craige McWhirter

Penelope and Creampuff take a Dip

On a hot November Saturday I've let the alpacas into fresh pasture. Instead of diving into the fresh pasture, they dived straight into the small dam for a swim.

Blog topics: 
The Farm


November 05, 2011 10:50 PM

-----

November 03, 2011

Dave Ruys

Oops….yesterday was Wednesday. A bit premature on the...



Oops….yesterday was Wednesday. A bit premature on the Thursday Gnar….

Anyways, my sponsor (-ish) just sent me a new deck. It’s basically a platform for testing their wheels when they send me different prototypes. Bathtub-like concave for the foot-locking goodness. Much radness is occurring as a result of it…


November 03, 2011 11:25 AM

-----

November 02, 2011

Dave Ruys

Thursday Gnar - Gabe’s Slide Jam. Three hills and a lot of...



Thursday Gnar - Gabe’s Slide Jam.

Three hills and a lot of tortured urethane. The video does not do justice to how OFF THE HOOK the last hill was.

I seem to have managed to completely avoid getting on film…

via skatehousemedia


November 02, 2011 01:28 AM

-----

October 31, 2011

Russell Coker

SE Linux Status in Debian 2011-10

Debian/Unstable Development deb http://www.coker.com.au wheezy selinux The above APT sources.list line has my repository for SE Linux packages that have been uploaded to Unstable and which will eventually go to testing and then the Wheezy release (if they aren’t obsoleted first). I have created that repository for people who want to track SE Linux development [...]


October 31, 2011 02:47 PM

Links October 2011

Ron has written an interesting blog post about the US as a “lottery economy” [1]. Most people won’t win the lottery (literally or metaphorically) so they remain destined for poverty. Tim Connors wrote an informative summary of the issues relating to traffic light timing and pedestrians/cyclists [2]. I have walked between Southgate and the Crown [...]


October 31, 2011 02:47 AM

-----

October 28, 2011

Sam Johnston

Enomaly's SpotCloud and 'The Case Against Commodity Cloud Exchanges' (Redux)

I wouldn't have bothered commenting on Enomaly's launch of the SpotCloud "private beta" today as I've been busy and neglecting my blog of late, but their Founder and CTO, Reuven Cohen, recently wrote an insightful critique of their competitor in this space, Zimory, which just needed a quick find and replace to be largely applicable here. See for yourself:

ElasticVapor: The Case Against Commodity Cloud Exchanges

The concept of a commodity cloud exchange is something that I've been talking about for several years. Notably Sun Microsystems also proposed it back in 2005. Recently a new start-up spun out out of Deutsche Telekom called ZimoryEnomaly is attempting to use this as the nexus of their enterprise focused hybrid cloud platform. The company describes itself as the first 'global marketplace for cloud resourcescloud computing clearinghouse and marketplace' to enable organizations to buy or sell extra computing capacity.

For those of you unfamiliar with the concept of a cloud exchange, the concept is to provide a central financially focused exchange where companies are able to trade standardized cloud capacity in the form of a futures contract; that is, a contract to buy specific quantities of a compute / storage / bandwidth capacity in the form of a commodity at a specified price with delivery set at a specified time in the future. The contract details what cloud asset is to be bought or sold, and how, when, where and in what quantity it is to be delivered, similar to a bandwidth exchange or clearing house. The exchange may be public or private akin to private exchanges / ECNs on the stock market, where membership is by invitation only

As I dug a little deeper into the Zimory'sEnomaly's SpotCloud web interface I noticed that ZimoryEnomaly SpotCloud is actually not really a marketplace so much as a multi-cloud management platform. The platform does little to address security, audibility, accountability, or trading/futures contracts. My first question is why should I trust their cloud providers and how do I know they're secure?

Another problem with the platform is their approach to capacity access. It appears that you are forced to use their platform, a platform that has no API or web services that I could see. Also their approach to a SLA is not very obvious, they broadly describe three levels, Bronze, Silver and Gold with no insight into what these levels actually represent. We are too just take them at their word.

Upon closer examination of the ZimoryEnomaly SpotCloud platform it appears to be nothing more than an open source hybrid cloud computing platform with an "ebay" marketing spin. So lets for a moment assume that a Commodity Cloud Exchange is a service that businesses actually want. (I'm not convinced they do) If this is the case, is a random start-up such as ZimoryEnomaly really in a position to offer such a service? And if so, should this exchange look like ebay or should it look more like a traditional commodities exchange? My opinion is the latter. What worries me about such a cloud exchange is the first thought that comes to mind is Enron, who attempted a similar bandwidth focused offering in the late 90's.

If we truly want to enable a cloud computing exchange / marketplace, maybe a better choice would be to build upon an existing exchange platform with a proven history. A platform with an existing level of trust, governance as well as compliance such as the Chicago Mercantile Exchange's Globex electronic trading platform or even the NasDaq.

Creating a cloud exchange has less to do with the technology and more to do with the concept of trust and accountability. If I'm going to buy XX amount of regional capacity for my Christmas rush I want to rest assured that the capacity will be actually available. And more importantly at a quality of service and location that I've agreed upon. I also need to be assured that the exchange is financially stable and will remain in business for the foreseeable future. All of which ZimoryEnomaly's SpotCloud doesn't offer.

Trust and security aside, To make ZimoryEnomaly SpotCloud attractive, they need to enable a marketplace that allows its users to buy additional capacity based on economic / costing factors that matter. For example being able to define a daily budget for your app, similar to the way AdWords spending works. There needs to be a fine-grained control over this budget so you can apply it across CPU, network bandwidth, disk storage, location with a focus on future requirements (futures). I should be able to trade / swap any unused capacity as easily as I originally bought it. There needs to be provider quota system that allows for the assurance that a certain amount of cloud capacity is always available to the "exchange" with a priority level. There should be multiple types of trading contracts as well as an indepth audit trail with a clear level of transparency within the entire trading process.

At the end of the day, I'm not convinced we're ready for "standardized" cloud exchanges. The cloud computing industry is still emerging, there are no agreed upon standards for how we as an industry can collaborate as partners yet alone trade capacity. In a lot of ways I feel ZimoryEnomaly is putting the cart before the horse and is probably 5-10 years too early.


October 28, 2011 06:14 PM

Russell Coker

Desktop Equivalent Augmented Reality

Augmented reality is available on all relatively modern smart phones. I’ve played with it on my Android phone but it hasn’t delivered the benefits that I hoped, there is a game where you can walk through a virtual maze which didn’t work for me, and a bunch of programs which show me the position of [...]


October 28, 2011 11:47 AM

Matt Palmer

rsync for LVM-managed block devices

If you’ve ever had to migrate a service to a new machine, you’ve probably found rsync to be a godsend. It’s ability to pre-sync most data while the service is still running, then perform the much quicker &l