I have released version 1.19-rc5 of Perdition. The key changes are:
The code and related libraries are available as tarballs
here.
More information is available in the announcement email.
More information about perdition here.
Tue, 20 Mar 2012 21:25:20 +0900| Permalink
I am looking for an intern to work on a project with me as a Linux Kernel Developer focused on networking. For details please see horms.net/employment/
Mon, 15 Aug 2011 16:41:34 +0900| Permalink
Yesterday afternoon I made a presentation
introducing Open vSwitch at Linux Con Japan.
This is an update on the presentation of the same title that
I made at the Netfilter Workshop 2011.
Fri, 03 Jun 2011 11:17:23 +0900| Permalink
This afternoon I made a short presentation
at Japan Technical Jamboree 37
the work that I have been doing to allow
booting Linux directly on the SH-Mobile ARM platform.
This is an update on the presentation on the same topic
in January at MobileFOSS Miniconf, part of linux.conf.au Brisbane 2011.
Fri, 20 May 2011 14:58:27 +0900| Permalink
The following command is in my shell history
A青ポ0〜〜〜〜?
I think we can assume that is the work of Hikari.
I think it translates as:
A blue "po" Oooooooooh?
Thu, 10 Mar 2011 11:01:08 +0900| Permalink
I recently managed to make a bit of a mess of things while doing some multi-process programming using fork(). While the box was still semi-usable I was unable to kill off the processes faster than they were being created and I ended up resorting to a reboot.
At the time I wasn't entirely sure what the problem was and not cherishing the prospect of more reboots I used taskset to constrain my shell, its child processes including test runs of my program and of course its forked children to a single CPU.
# taskset -p 01 $$
The result? A subsequent fork explosion was indeed constrained to one CPU and I was able to kill off all the processes quite easily.
My system is a single socket with four cores. I have disabled hyper-threading
so there is only one thread per core. I am unsure how well this technique
would work in other situations, especially in the case of multiple threads
but only one socket and one core.
Thu, 10 Mar 2011 11:01:08 +0900| Permalink
On the 18th of March I will be making a short presentation at Japan Technical
Jamboree #36, on the work that I have been doing to allow booting Linux
directly on the SH-Mobile ARM platform.
This will be a follow-up to a presentation I made on the same topic
at the MobileFOSS Mini-Conf at Linux.conf.au 2011 in January.
Material from that presentation
Mon, 07 Mar 2011 08:07:43 +0900| Permalink
This afternoon I will be making a presentation at Linux.Conf.Au on Network Bandwidth Control in Virtualised Environments. This is primarily on work that I have done on network QoS with Xen. I the slides and related information are now available. slides, etc...
Wed, 26 Jan 2011 11:06:44 +1000| Permalink
This morning I made a short presentation on the work that I have been doing to allow booting Linux directly on the SH-Mobile ARM platform. slides, etc...
Tue, 25 Jan 2011 11:53:58 +1000| Permalink
Over the past few weeks eastern Australia, including Brisbane, has been subjected to devastating floods. My sympathy and condolences are with anyone adversely effected by these events.
Last night a decision was made by the organisers and Linux Australia that despite the floods Linux.Conf.Au 2011 will go ahead next week in Brisbane as scheduled. I am delighted to be able to be part of the event.
In Tuesday I will be making a presentation at the MobileFOSS miniconf on using zboot to boot Linux directly from flash or, SD or MMC cards on SH-Mobile ARM.
On Wednesday at 15:45 I will be making a presentation at the main conference on Network Bandwidth Control in Virtualized Environments. This covers the use of Linux's QoS facilities to control guest bandwidth usage.
Its going to be a great conference, I'm looking forward to it.
Tue, 18 Jan 2011 08:25:54 +0900| Permalink
After a bit of a hiatus I have updated my photo gallery.
Sun, 21 Nov 2010 17:15:30 +0900| Permalink
I have released version 1.19-rc4 of Perdition. The key changes are:
The code and related libraries are available as tarballs
here.
More information is available in the announcement email.
More information about perdition here.
Mon, 13 Sep 2010 17:48:10 +0900| Permalink
I'm going solo, working for myself at
Horms Solutions,
a boutique free and open source software consultancy.
Tue, 31 Aug 2010 17:09:06 +0900| Permalink
I will be in Boston next week to give a presentation on
Network Bandwidth Control in Virtualized Environments at
LinuxCon
and attend the related mini-summits. Its been a while
since I've attended an event in North America, I'm looking forward to it.
Fri, 06 Aug 2010 11:54:25 +0900| Permalink
I have released version 1.19-rc3 of Perdition. This release fixes several problems. Perhaps most significantly, Thunderbird 3.1 can now re-connect to perdition SSL/TLS.
The code and related libraries are available as tarballs
here.
More information is available in the announcement email.
More information about perdition here.
Thu, 29 Jul 2010 17:16:17 +0900| Permalink
I have released version 2.0.2 of kexec-tools. the user-space portion of kexec a soft-reboot and crash-dump facility of Linux and Xen.
There are changes since 2.0.2-rc1.
The code is available as a tarball here and in git here. More information is available in the announcement email.
kexec-tools recently moved, so the URLs above are new.
Thu, 29 Jul 2010 14:01:27 +0900| Permalink
My home page and blog have moved. The new locations are:
Thu, 29 Jul 2010 12:23:52 +0900| Permalink
Due to tyre troubles with my other two bikes I was "forced" to ride my Cannondale road bike to work this morning. The first time I've ridden it since well before Hikari was born last November. Wow, that bike is light and voom it flies. (And I put the brakes back together correctly after shipping it from Sydney!)
In semi-related news, its really warm in Tokyo at the moment.
Tue, 20 Jul 2010 09:55:18 +0900| Permalink
I have released version 2.0.2-rc2 of kexec-tools. the user-space portion of kexec a soft-reboot and crash-dump facility of Linux and Xen. The only change since 2.0.2-rc1 is to include a number of files that were missing from the tarball which caused build failures on x86_64, ppc, ppc64 and arm.
The code is available as a tarball here and in git here.
More information is available in the announcement email.
Tue, 13 Jul 2010 12:01:14 +0900| Permalink
I have released version 1.19-rc2 of Perdition. There is a single change since 1.19-rc1 to fix a crash-bug on invocation of perdition in managesieve mode. Merge madness on my part. Thanks to Filipe Carvalho for reporting it. The fix is here.
The code and related libraries are available as tarballs
here.
More information is available in the announcement email.
More information about perdition here.
Sat, 10 Jul 2010 10:13:03 +0900| Permalink
I have released version 1.19-rc1 of Perdition, a mail retrieval proxy. This release includes numerous bug fixes and support for the manage sieve protocol.
The code and related libraries are available as tarballs
here.
More information is available in the announcement email.
More information about perdition here.
Tue, 06 Jul 2010 11:47:19 +0900| Permalink
I have released version 2.0.2-rc1 of kexec-tools,
the user-space portion of kexec
a soft-reboot and crash-dump facility of Linux and Xen.
The code is available as a tarball here and in git here.
More information is available in the announcement email.
Mon, 05 Jul 2010 16:15:14 +0900| Permalink
I wish that I was writing about a new code for style but instead I'm writing about poor coding style. I'll keep it brief.
Fri, 25 Jun 2010 11:29:35 +0900| Permalink
Today Chiz, Hikari and I were lucky enough to join Robert and Lynne, and their friends and family at Eden Gardens in North Ryde to celebrate their wedding. It was a very happy and relaxed wedding, very fitting for Robert and Lynne. Congratulations to you both.
Photos here
Sat, 27 Mar 2010 20:15:03 +1100| Permalink
Is it ok to use chat to communicate with my wife?
If she is sitting at the next desk?
If there is a sleeping baby in between?
Is any of this relevant if she laughs?
Tue, 02 Feb 2010 15:06:07 +1100| Permalink
This evening
Chiz, Hikari and I went down to Darling Harbour to watch the Australia Day
Fireworks. Although there were rather a lot of people down there we managed
to find a good vantage point. Close enough to be impressed by the
show. Much to my surprise I managed to take some reasonable snaps
of the show.
Photos
here.
Wed, 27 Jan 2010 00:47:52 +1300| Permalink
Hikari and Chiz came home from hospital today. It was nice and
uneventful and everyone seems relaxed now.
To celebrate I've expanded Hikari's photo page, its
here.
Sun, 15 Nov 2009 18:04:56 +1100| Permalink
Born at 1:47 pm on the 11th November 2009 at the Royal Hospital for Women in Randwick. 2.72kg healthy boy.
Photos here
Thu, 12 Nov 2009 17:45:03 +1100| Permalink
I've been playing with Google Chrome Dev Builds of late. That is the web browser by Google, not the OS or any other tenuously related project that has a similar name. It seems nice and fast. And even though its billed as work in progress, it seems quite stable to me.
But alas there are no add-ons. And while I'm prepared to accept that some
of my favourites, vimpretor and rikaichan, are a bit esoteric. The
world without adblock is a pretty
scary place that I'm not sure I want to deal with. All those banner-adds
and the like really lower the tone of the place. This seems amusing as
I believe that chrome basically means to sex things up.
Thu, 30 Jul 2009 18:32:06 +1000| Permalink
Photos of our recent trip to Dubai, Milan and Florence.
The highlight of the trip was Michael and Ding's wedding
just south of Florence. photos
Sat, 25 Jul 2009 00:09:34 +1000| Permalink
Its been a while between drinks, so I have released
kexec-tools 2.0.1-rc1.
Now is the time to point out what is missing and needs to go into 2.0.1 :-)
Wed, 22 Jul 2009 12:28:00 +1000| Permalink
Xenclient is one of those projects where you do a fairly small check out, perform some configuration, press the go button, and it builds a small universe worth of software, downloading things as needed. In my experience build-breakage usually ensues in such situations.
Here is the winning combination I came up with to compile the system on Lenny. Installing and testing I will leave for another day.
The build needs to be done on an x86 32bit system. It should be possible to do this non-natively using a cross-compiler or a chroot. I didn't feel like enhancing the difficulty of the task at hand, so I just found a system on which I could run a native compile.
A mailing list post breifly discussing this topic can be found here
Ensure that the following packages are installed: build-essential, bison, flex, guilt, texinfo, git, bcc, gettext, rsync, iasl, locales, python, gawk and subversion.
Ensure that the en_US.UTF-8 locale is present on the system. This can be done this before the locales package is installed by running the following. Note that debconf-set-selections needs to run as root, which can be achieved by prefixing its invocation with sudo, assuming sudo is set up.
# echo "locales locales/locales_to_be_generated string en_US.UTF-8 UTF-8" | \ debconf-set-selections
Or this can be done by after the locales package is installed by running the following as root:
# dpkg-reconfigure locales
Check-out the initial source code.
# git clone http://xenbits.xen.org/git-http/xenclient/build.git # cd build
Unset some environment variables.
# unset MAKE # unset CFLAGSThe build system bailed-out early on and asked me to unset CFLAGS. I bailed-out of my MAKE="make -j3" build when the load went over 3000!
Copy the sample configuration, disable partd and select the 2.6.18 kernel. My preference was for the 2.6.27.5 kernel, but this failed to build.
# cp configs/xenclient_config ./.config # echo "# BR2_PACKAGE_PARTED is not set" >> .config # echo "BR2_XENCLIENT_KERNEL_2_6_18=y" >> .config # echo "# BR2_XENCLIENT_KERNEL_2_6_27 is not set" >> .config
Activate the configuration.
# make oldconfig
Build.
# makeThis may take quite a while. It will download things, though it is nice enough to cache most of the downloads in
Ponder the installation instructions in the
Fri, 22 May 2009 13:46:03 +1000| Permalink
Xen 3.4.0 has been released and includes two improvements to PCI pass-through that I was involved in:
Allow any unused PCI slot to be used for PCI pass-through, including hot-plug.
Previously only slots 6 & 7 could be used, limiting users to two pass-through devices per domU. The new limit is around 28, as there are 32 possible slots on a PCI bus, and at typically 4 are used by a domU for emulated devices. Fewer devices will be available for pass-through if extra ioemu devices are attached to the domU.
Allow the user to specify slot used for pass-through of devices attached to domU at boot time. This mirrors the ability to select the slot when a device is hot-plugged.
Unfortunately there have also been a number of bugs in pass-through. Many of these have been fixed. But and as of the release a known problem is that domains can't be started if they have previously had PCI devices hot-plugged. At this time I recommend using xm_pci_list_v3.patch and xm_pci_attach_v2.patch which have been provided by Masaki Kanno in his post [PATCH] Fix xm pci-attach/detach for inactivemanaged as a fix for this problem.
Update: Kanno-san's changes have been combined into a single patch
and applied to xen-unstable as
changset: 19618:780041c4a96d "xend: Fix xm pci commands for inactive
managed domains".
Tue, 19 May 2009 12:03:50 +1000| Permalink
Of late I've been working on and off towards adding support for the Intel 82566DM-2 and 82576 to gPXE. My current technique involves a combination of studying data-sheets and programming manuals available on Intel's web site, scrounging around the internet for existing patches and examining the working Linux drivers, which have a common ancestor to the driver in gPXE. Over the past week this work has been more on than off.
So far I have a driver for the 82566DM-2 that can receive packets and send one packet. As an amusing work-around I tried reseting after each transmit and that does allow it to send more packets. For the 82576 so far the driver can only send packets. Perhaps for another amusing hack I should try sending with one card and receiving with the other.
Update: I now have the 82576 working and have posted a
patch which the mailing list archive mangled.
I haven't had any more luck with the 82566DM-2 and have shelved
that project for now.
Fri, 08 May 2009 11:10:54 +1000| Permalink
I recently wanted a way to set up a machine as a serial console server. That is to connect two of its serial ports to serial ports on two other machines using a null-modem. The idea being that these other machines would put their console on their serial port which would be accessable remotely. Good for debugging, good for handling crashes.
Reading Matt Palmer's recent post SheevaPlug-based serial console concentrators, I decided to write up how I achieved this goal.
Rather than users logging into the console server and then executing a command to access the relevant serial port, I wanted access to be I wanted access to the consoles by telneting to a designated port, one port per serial console that is available.
To achieve this I wrote a small script, screenshell and put it in /usr/local/bin. It is pretty simple, it creates a new screen session for the requested TTY if one doesn't already exist, else it connects to the existing session. I couldn't find an argument to screen that would do this in one hit, so the script looks at what screen sessions are running and acts accordingly - some room for improvement here (see update).
In order to accept connections via telnet, I just added the following lines to /etc/inetd.conf:
8000 stream tcp nowait scs.scs /usr/sbin/tcpd /usr/sbin/in.telnetd -h -L/usr/local/bin/screenshell-ttyS0 8001 stream tcp nowait scs.scs /usr/sbin/tcpd /usr/sbin/in.telnetd -h -L/usr/local/bin/screenshell-ttyS1
This basically says, listen for connections on port 8000, if you get a connection, invoke /usr/sbin/in.telnetd and use /usr/local/bin/screenshell-ttyS0 to handle their session. Similarly for port 80001. scs.scs denotes that /usr/sbin/in.telnetd should be invoked as user scs, group scs. The scs (Serial Console Server) user and group were created for this sole purpose, though that is really just a matter of taste.
I couldn't work out how to determine which port the user was connecting to from within screenshell when it is invoked in this way, nor could I work out how to pass extra arguments to screenshell. So instead I just symlinked screenshell to screenshell-ttyS0 and screenshell-ttyS1, which allowed the command name to be used to determine which TTY was being used. Again, room for improvement.
For the problem at hand telnet was sufficient so I didn't investigate how well ssh would work with the solution that I cam up with. For the task at hand, the solution above seems to work quite well.
Download: screenshell
Home: ~horms/junk/
Update: 4th May 2009
Kfish pointed
out that screen -xRR -S NAME will attach
to an existing session or start a new one as necessary. I have
slimmed down the script accordingly.
Fri, 01 May 2009 11:10:34 +1000| Permalink
I'm not sure how/why/when /var/lib/xend/state/sr.xml became zero length, but I am sure that Xen's reaction was fairly difficult to read: xend wouldn't start.
# /etc/init.d/xend restart .........
The log, at 61 lines, was much more verbose, though equally useless unless you are prepared to delve into the code. It can be found here. It turns out the interesting portion for debugging the problem is:
File "usr/lib/python2.5/site-packages/xen/xend/XendNode.py", line 240, in _init_SRs
saved_srs = self.state_store.load_state('sr')
Diving into the code, I put a few debugging statements into XendStateStore.py, which seemed to indicate that the breakage was occurring around the time that /var/lib/xend/state/sr.xml was read. On inspection I found this file to be empty, while similar files in the same directory were not.
A solution: # rm /var/lib/xend/state/sr.xml
A Lesson Learnt: If you follow xen-unstable, from time to time things will break in bizarre ways.
A Mystery: How did the file end up empty in the first place?
Thu, 30 Apr 2009 11:56:25 +1000| Permalink
![[Rainbow over Sydney]](http://www.vergenet.net/~horms/pleb_blossom/pics/2009-04-29-155448.0.jpg)
Wed, 29 Apr 2009 17:02:47 +1000| Permalink
As the ATM was processing my withdrawal and preparing to hand over some cash to me, the screen displayed something along the lines of "Who is Watching?". Presumably this was to encourage me to be aware of people looking over my shoulder trying to grab my PIN. Or perhaps inviting me to be wary of someone watching me from a strategic location, ready to pounce on me and relieve me of my grocery money. There is a small mirror fitted to the ATM, ostensibly to aid customers in spotting who is watching.
I'm not sure about the machine I was using, but many ATMs are fitted with
cameras to help banks do their watching. And many street corners in the
Sydney city now feature loud-speakers attached to the traffic lights in the
name of allowing the authorities to broadcast instructions in the case of
an terrorist attack - presumably there are cameras involved too. I wonder
if perhaps the ATM was inviting me to think about who is watching me at a
slightly deeper level.
Tue, 28 Apr 2009 17:30:18 +1000| Permalink
What started as a quick edit to mention some features that I got merged into the forthcoming 3.4.0 release of Xen ended up as an all day reworking. But its done now and hopefully the VT-d HowTo is in better shape for it.
This was my first experience editing using a wiki for something that took more than a few moments. At first it was a bit painful - a meta language for a mark-up language does seem like at least one layer of abstraction to many to me - but by the end of it I was less bothered. The version of MoinMoin's GUI editor that is on wiki.xensource.com does seem to be somewhat broken/feature lacking. Perhaps there are newer/better versions around. But it was a nice change of sceen from the MoinMoin Text editor. And between those two views of reality I managed to get the job done.
Originally I was going to write this post to see if anyone has any
advice to offer on using wikis for large edits. But I've come to realise
that now I'm just writing it as an excuse to do something in HTML -
something I though I would never hear myself say.
Tue, 21 Apr 2009 17:39:46 +1000| Permalink
Today I was klined on Freenode. I don't know why.
Perhaps I've been devious while I wasn't looking.
In any case, no IRC on Freenode for me until my
ticket lodged with kline@freenode.net is resolved.
* Connected. Now logging in... * *** Looking up your hostname... * *** Checking ident * *** Found your hostname * *** No identd (auth) response * *** Banned: Please do not DoS or exploit users on freenode. If in error, * please contact kline@freenode.net. Thanks! (2009/04/15 02.25) * Closing Link: 127.0.0.1 (Banned) * Disconnected (Remote host closed socket).
Update: It was a mix-up and all has been fixed now :-)
Wed, 15 Apr 2009 16:55:29 +1000| Permalink
A few months ago I acquired an HP dc7800 which amongst other
things includes vPro, an on-board baseboard management system of sorts.
vPro can do quite a number of things, but the features that interested me
were:
As someone who is quite keen on having machines in a different room to me, this was all quite appealing. And for the most part it works. However, there are a few cases where it falls down, and this has led me to gradually replace it with other solutions.
Problems encountered with vPro:
Tue, 14 Apr 2009 10:58:39 +1000| Permalink
If you're the kind of person who thinks that your keyboard configuration in your xorg.conf should be used by Xorg, then read on. If on the other hand you like your settings to be overriden by whatever dbus tells Xorg, then this isn't for you.
This afternoon I updated my Debian unstable system, which amongst quite a few other things upgraded Xorg, I think that package of particular relevance is xserver-xorg 1:7.4+1. This all seemed to go well enough, except that my Japanese keyboard was clearly being used by Xorg as a US keyboard - they letters worked byt most of the symbols didn't.
AFter much messing about and, I noticed that /var/log/Xorg.0.log was reporting that AutoAddDevices was enabled, and that as a result it would ignore the configuration in the xorg.conf for all keyboards, mouses and various other classes of input devices. And a bit further down information about the devices that hal had autodetected, which were almost correct, but not quite.
The following magic tells X to actually use its own configuration for devices. Which has resolved my problem.
Section "ServerFlags" Option "AutoAddDevices" "off" EndSection
Mon, 13 Apr 2009 19:28:59 +1000| Permalink
I'm working on what appears to be a regression in
xen-unstable.
But I'm having a little bit of trouble narrowing down the change-set
that is causing the problem. Here are my notes so far:
revision 19248: good
revision
19249: is bad!
good bad
good
bad
good
revision
19250: is bad
The problem? HVM domains can't connect to virtual block devices (sometimes).
Update:
16 March 2009:
I never did get to the bottom of this,
but
revision
19354 seems to work fine
Wed, 11 Mar 2009 15:34:25 +1100| Permalink
After the exhaustion of a week at LCA in Hobart, and a few days at home in Sydney, I'm now in the office Tokyo.
When I mention Tokyo a common response is "isn't that a really crowded place?". Especially from Japanese people that I have met, though their version is usually a statement rather than a question.
This morning I caught the subway from Ginza-Ichome Station at about 9:30am. There was one other person on the entire platform, form the time I arrived until I got on the train, about 2 minutes later. I wish I had my camera, but you'll just have to try and imagine an empty Tokyo subway station on an otherwise busy morning.
Thu, 29 Jan 2009 12:24:34 +0900| Permalink
Computerworld are running an interview with me on some opinions about Xen and KVM. more...
Fri, 23 Jan 2009 16:42:39 +1100| Permalink
My last post was titled "Building qemu-dm for HVM VT-d PCI Pass-Through", a piece about building a component of Xen. But what does all this terminology mean?
Firstly, a very brief overview of the Xen architecture. The three main components of a system running Xen are the hypervisor, privileged domains and unprivileged domains. The hypervisor is run when the system first boots and controls access to the hardware. Privileged domains essentially provide both drivers and a management interface to xen. Usually there is one privileged domain, called dom0. They hypervisor by itself is not a functional system and generally the hypervisor + dom0 is required to boot the system. Unprivileged domains are the guest OSes on the system, often referred to as domU. As the name suggests they generally have to access hardware via the hypervisor - which may delegate the task to a privileged domain.
Xen supports two methods of domain (guest) virtualisation. In the beginning there were paravirtualised (PV) domains. In paravirtualised domains portions of the guest OS that execute privileged instructions are replaced by hypercalls. The hypercall asks the hypervisor to perform the relevant action. At the time that Xen was originally developed this was the only type of virtualisation that was possible on x86 and ia64.
With the advent of Intel's VT-x for x86 and VT-i for ia64, and AMD's AMD-V it has became possible to run unmodified operating systems as guests. This is called full-virtualisation or hardare virtual machine (HVM). It has obvious advantages when the guest operating sytem can't be modified.
There are several techniques used to give unprivileged domains access to PCI devices. For PV domains the common approach is to split the driver into front-end and back-end portions. The front-end driver sits in the unprivileged domain and the back-end driver sits in a privileged domain. The front-end driver communicates with the back-end driver. And the back-end driver communicates with the hardware. The back-end driver may multiplex IO from multiple front-end drivers. And thus a PCI device may be shared between domains.
Another approach used with PV domains is PCI pass-through. This basically involves giving a PCI device exclusively to an unprivileged domain. The advantage of this is a potential performance increase, as I/O no longer has to go through a back-end driver sitting in a privileged domain. The disadvantages of this approach are:
Systems that have Intel VT-d have IOMMU capabilities. The IOMMU constrains the area to with a device passed-through to a guest may DMA. Which removes the security problem relating to passing-through PCI devices to unprivileged guests. Xen allows pass-through to paravirtualised to optionally use the IOMMU if present.
For fully virtualised domains devices are commonly emulated.
This is done by running the device emulation code from qemu as
qemu-dm in a privileged domain.
In order to improve I/O performance in fully virtualised domains
it is possible to use PV drivers. These essentially provide
front-end drivers to fully virtualised domains.
And with the advent of Intel's VT-d it is possible to pass-through PCI
devices to fully virtualised domains.
Fri, 12 Dec 2008 17:35:11 +1100| Permalink
If you are building the qemu-dm, device models for xen, from http://xenbits.xensource.com/git-http/qemu-xen-unstable.git or the tools/ioemu-dir/ directory that is created when building http://xenbits.xensource.com/xen-unstable.hg (due to the wonders of the xen build system, they are the same thing). And you are doing this on Debian. And you would like PCI pass-through to work for HVM domains. Then make sure you have the libpci-dev package installed before building. Otherwise qemu-dm will be built without pass-through support. If you install libpci-dev after building then run make clean before rebuilding, to ensure that all files are rebuilt correctly - they won't be otherwise.
If you do this then the instructions in the Xen VT-d How-To should work.
Otherwise you will end up the following nasty message, which just cost me more time than I care to admit, in /var/log/xen/xend.log
ERROR (XendDomainInfo:581) Device model didn't tell the vslots for PCI device
On a semi-related note, if you wish to use VT-d PCI pass-through in
paravirtualised domains then pass iommu=pv as an argument to xen when
booting. Otherwise non-VT-d pass-through will be used for paravirtualised
domains. This is independent whether qemu-dm was compiled with
libpci-dev present or not.
Fri, 12 Dec 2008 16:58:52 +1100| Permalink
Today was a beautiful pre-summer day in Sydney and Chiz and I went to see Sculpture by the Sea, which is held on the coastal walk between Bondi and Tamarama Beaches.
Sun, 19 Oct 2008 21:32:53 +1100| Permalink
Some photos from our ski trip to Perisher in August.
more...
Mon, 13 Oct 2008 21:56:15 +1100| Permalink
Yesterday Chiz and I had the pleasure to gather together with Struan and Genny's friends and family for their wedding in the Royal Botanical Gardens in Sydney. It was a wet morning but the weather cleared just in time for the event. I took a few snaps of the occasion. more...
I also have some snaps of the Skydiving that took place in Picton two weeks ago as part of Struan's bucks party. more...
Sun, 05 Oct 2008 13:20:59 +1100| Permalink
Flow-control is implemented from netfront all the way to the end of life for a packet, typically transmit in the destination interface.
This post explains briefly how flow-control is implemented in netfront/netback. The paths examined are for the case where dom0 bridges traffic from netback to the destination interface. The same logic should also hold for packets from netback that are routed by dom0.
Netback/Netfont is used to provide networking to paravirtualised domain Us. Thus, much of this discussion is not applicable to fully-virtualised domains, also known as HVM domains.
The code reference is from the linux-2.6.18-xen tree on xenbits.xensource.com, around the time of the Xen 3.3 release.
This analysis is based upon the notion of domU sending a stream of packets as fast as it can. There is no congestion control, no packet filtering, and the CPU(s) of domU are fast enough to processes packets as fast as the physical interface can send them without dropping any packets. Furthermore there are no return packets. In short, the notion is of UDP packets being blindly sent by domU.
Packets are received by netback from netfront using a ring-buffer that is implemented as a grant table. Typically there are 256 slots in this buffer. And once it becomes full netfront can no longer send packets.
The relevant netfront code in the reference netfront implementation is in drivers/xen/netfront/netfront.c:network_start_xmit().
if (!netfront_tx_slot_available(np))
netif_stop_queue(dev);
For each packet transmitted by netfront to netback, one ring-buffer slot is used for header information and one additional slot is used per fragment. So in the common case of a packet with one fragment (that is, an unfragmented packet), two slots will be used.
Assuming that the number of packets to be sent is significantly greater than the number of slots in ring-buffer, the rate at which netfront can send packets will be the same as the rate at which netback frees slots in the ring-buffer. So the ring-buffer provides a mechanism for netback to enforce flow-control on netfront.
Strictly speaking the above logic applies to fragments not packets. But as it is common to have unfragmented packets it seems more comfortable to talk in terms of packets.
Once all fragments for a packet have been received by netback from netfront via the ring-buffer it packs them into an sk_buff, the structure that the Linux networking code uses to represent packets. This work is done by drivers/xen/netback/netback.c:net_tx_action().
At this point the slot used in the ring-buffer for header information is queued to be released for reused by netfront. This is done by calling drivers/xen/netback/netback.c:netif_idx_release(). The release queue is processed by calling net_tx_action_dealloc() the next time that net_tx_action() is called.
The ring-buffer slot associated with each fragment contains a page which in turn contains the data for the fragment. This page is associated with a fragment within the sk_bufs created for the packet. This work is done by drivers/xen/netback/netback.c:netbk_fill_frags() which is called by net_tx_action(). The critical line is:
frag->page = virt_to_page(idx_to_kaddr(pending_idx));
Once all fragments have been added to the sk_buff it is passed to the Linux networking core using net/core/dev.c:netif_rx(). At this point the packet has been received by the netback device driver.
Usually netif_rx() will simply move the sk_buff onto a received queue and schedule processing of the queue if it is not scheduled already. In any case, sk_buff will end up being destroyed, either after bing transmitted or being dropped - for example because of packet filtering rules.
At this point a packet is in the Linux networking core in in the form of an sk_buff. The page that was associated with each fragment of each sk_buff in netback by netbk_fill_frags() have a particular property that is of relevance to flow-control. These pages are members of drivers/xen/netback/netback.c:mmap_pages[] which is set up in drivers/xen/netback/netback.c:netback_init(). Each member of mmap_pages[] is marked as foreign and has drivers/xen/netback/netback.c:netif_page_release() set as its destructor using include/linux/page-flags.h:SetPageForeign().
SetPageForeign(page, netif_page_release);
When an sk_buff is freed, the destructor is called for each page that has both been associated with a fragment within the sk_buff and marked as foreign using SetPageForeign(). This is done in mm/page_alloc.c:free_hot_cold_page().
#ifdef CONFIG_XEN
if (PageForeign(page)) {
PageForeignDestructor(page);
return;
}
#endif
sk_buffs are freed using kfree_skb(), the call path from there to free_hot_cold_page() is:
|
 net/core/skbuf.c:kfree_skb() ⟶ net/core/skbuf.c:__kfree_skb() ⟶ net/core/skbuf.c:kfree_skbmem() ⟶ net/core/skbuf.c:skb_release_data() ⟶ mm/swap.c:put_page() ⟶ mm/swap.c:__page_cache_release() ⟶ mm/page_alloc.c:free_hot_page() ⟶ mm/page_alloc.c:free_hot_cold_page() |
Thus when an sk_buff is freed, for any reason, netif_page_release() will be called. It simply calls netif_idx_release(), which as discussed above queues a ring-buffer slot up to be released for reuse by netfront. This freeing will typically take place in the physical NIC's driver, when the packet is transmitted on the physical interface.
As sk_buffs are generally freed as packets are transmitted on the physical interface, the rate at which slots are freed relates to the rate at which packets are sent. Thus the rate at which netfront can send packets is effectively the rate at which the destination interface, usually a physical ethernet interface, can sent packets, assuming the dom0 CPU can keep up.
The following diagram shows the usage cycle of ring-buffer slots. They are consumed by domU. Slots used for meta-data are added to the free list immediately by Netback. Slots used for fragments are added to the free list once the skb to which they are associated is freed. Slot buffers that are placed on the free list become available for reuse in the ring-buffer by DomU.
It is worth noting that if packets are dropped, for instance by packet filtering, before they make it to the destination device, then the ring-buffer slot will be freed. This should allow for faster transmission by netfilter. That is, packets that get dropped don't take as long to transmit and thus netback can transmit at a faster rate.
Thu, 25 Sep 2008 21:45:19 +1000| Permalink
The Army has been practicing running two helicopters up Pitt St. this afternoon. So far they have done about half a dozen laps, down from Circular Quay to about Town Hall where they do some maneuver that I can't quite see out of my window before looping back North over the Domain. They seem to take about a 15-30 minute rest between each lap. Not sure where they go during this time. Perhaps up to Palm Beach. Or perhaps they drop in for scones with the PM at Kirribilli House. In any case, I'm glad that they are keeping themselves busy and giving the office workers something to peer at our of their windows.
Mon, 18 Aug 2008 16:19:55 +1000| Permalink
Am I the only person who invokes firefox -P profile and expects it to work even if firefox is already running?
Mon, 18 Aug 2008 12:12:27 +1000| Permalink
Sometimes when you live in a foreign country interesting things happen. For Chiz, yesterday was one of those days.
Before, during and after dinner she was complaining of not feeling very well due to a sore stomache. She was wondering if perhaps she was coming down with a cold - there is at least one going around. On the short walk home from dinner she asked me what the sasuage like things below ones stomache are called, as they hurt too. I let her know that intestines was the word that she was after.
When we got home she showed me the elite new tea that she had been drinking on recommendation from a friend. Four or five cups since she started the day before...
![[Clearing Intestines to Reduce Weight Tea]](http://www.vergenet.net/~horms/pleb_blossom/pics/5-3919-crop1.jpg)
Wed, 06 Aug 2008 09:56:52 +1000| Permalink
Some time back in 2006 I sifted through many mailing list posts and pulled together all the patches to kexec-tools that were floating about. I subsequently put these in a git tree and released them as kexec-tools-testing.
As time went on, kexec-tools-testing ended up having releases about every 3 months or so, and became the focus for development of kexec-tools.
It has become a bit silly to call this code kexec-tools-testing, as it is the main development branch. The name only seems to serve to confuse people and perhaps scare some people off.
Along with the latest release of the code I have renamed the tree kexec-tools. So don't be confused any longer!
More about how to get kexec-tools
here
Sat, 19 Jul 2008 11:04:38 +1000| Permalink
Last Friday morning I had the joy of Jackhammers [1] not to long after waking up. This Friday I had the joy of not being able to log into X on my desktop machine.
I fairly quickly narrowed this down to some fairly bogus permissions on /dev/null, /dev/zero and likely many others. And I fairly quickly started to suspect udev. To cut a long story short, after much fussing about I narrowed the problem down to the presence of /etc/udev/rules.d/65_dmsetup.rules by comparing things with a working system.
After that, I found that this is Debian Bug #491107in dmsetup 2:1.02.27-2. The bug report describes the problem as well as I could, I experienced both the permissions and the no booting variants in the course of my journey :-)
From: Modestas Vainius
To: Debian Bug Tracking System
Subject: something in dmsetup.rules renders udev useless
Date: Wed, 16 Jul 2008 21:06:02 +0300
I'm reporting this as critical because it rendered my systems unbootable. When /etc/udev/rules.d/65_dmsetup.rules is present, udev stops functioning properly (various anomalies can be seen like wrong permissions on /dev/null etc.). If, unfortunately, initramfs image is regenerated and that rules gets included, system becomes unbootable due to "Waiting for root file system". Actually, even if udev daemon is started in this case, it simply sits there and won't coldplug a single device. Moving this file away resolves the problem. I reproduced this problem on both my computers with 2.6.26-trunk-amd64, 2.6.25-2-amd64 and a few other kernels (which had initramfs regenerated).
Lessons learnt:
Fri, 18 Jul 2008 10:45:45 +1000| Permalink
Chiz and I took a nice break in the Kangaroo Valley over the weekend. It was very relaxing and a great chance to unwind.
Not much to report other than that we took some nice snaps along the way.
Sun, 13 Jul 2008 23:29:01 +1000| Permalink
It seems that ripping stuff up and replacing it with other stuff seems to be much in vogue in our area. The City of Sydney Council is replacing the footpaths near our apartment with, if I understand the hoarding correctly, cycleways. The existing footpaths were asphalt, reasonably smooth and in quite good condition.
The new "cycleways" consist of what look to be marginally more slippery paving stones. How this less even surface constitutes a cycleway is beyond me. Other than the increased chance of slipping, there is also the matter of the small gaps and potential undulations where paving stones meet, about every 15cm. But frankly unless they have an idea of how to stop pedestrians from walking around the city, its not going to be much use for bikes anyway. Perhaps the hoardings have jbeen recycled from another project, fair enough, we are all green these days, just like the colour of the hoardings.
Posturing about what constitutes a cycleway aside, all I can
really conclude is that the council has money to burn.
Perhaps someone could recommend a good charity to the mayor?
Not wanting to be left out, the owner of one of the other three apartments on our floor is renovating. Today seems to be jackhammer day, starting at 8:30am. This has greatly enhanced my work day, which starts at a similar time, in a similar location.
To escape for a bit I went to the shops - carefully negotiating the still under construction footpaths/cycleways. Kfish, if you still need graph paper, I have found a source.
Update
Simon Rumble emailed me some links to about what is going on [1], [2]. These links make a lot more sense than my wild conjecture :-)
It seems that the cycle way will actually be between the footpath and the road, or in other words, part of the road will be sectioned off for cyclists, allong side the recently beautified footpath. I guess the construction will last a while longer as they seem to have only got as far as the footpath to date.
In other news, the jackammer stopped at around 11:30.
1: http://www.cityofsydney.nsw.gov.au/Development/CityImprovements/RoadsAndStreetscapes/KingStCycleRoute.asp
2: http://www.cityofsydney.nsw.gov.au/Development/CityImprovements/RoadsAndStreetscapes/KingStCycleRoute/KingStCycleway.html
Thu, 10 Jul 2008 15:44:04 +1000| Permalink
On Sunday we went to yum-cha at Maroubra Junction, very crowded and totally awesome food, and then brefily to La Perouse. The weather was really nice. I didn't have my usual camera with me, but I did take some nice snaps with Chiz's compact camera, which luckily she had in her bag.
Fri, 04 Jul 2008 12:52:50 +1000| Permalink
On Saturday Chiz and I did the manly scenic walk from The Spit to Manly (actually from Seaforth to The Spit to Manly as we missed the bus stop). I think that it ended up being about 10km for us and taking about 3 hours, which was pretty much ideal for us.
It was a bit cold and the day was starting to die by the time we took the ferry back to Circular Quay, next time we will try and start our hiking a bit earlier.
Fri, 04 Jul 2008 11:01:53 +1000| Permalink
Wednesday was a wonderful sunny day in day in Boston and I had plenty of time to explore and take some more snaps before my flight left in the evening. They were generally of much better quality than my efforts on Sunday, which was overcast and rainy.
I also managed to find some excellent lunch on Charles St. in Beacon Hill. I don't remember the name of the little Italian place on the 0.5th floor, but their chicken salad was awesome.
Boston is a pretty nice place and the people seemed pretty friendly - though perhaps because the Red Sox were playing at home and mostly winning while I was there. It also seemed pretty safe, perhaps due to the police presence. I observed four separate police services in operation during the course of my stay: Boston City Police, Middlesex County Police, Massachusetts State Police and Transit Police.
Fri, 27 Jun 2008 15:58:22 +1000| Permalink
I'm currently in Boston for Xen Summit which kicks off today.
Yesterday I had a little bit of time to explore Boston and
catch up with Geoff Harrison, who I hadn't seen since our
days working together at VA Linux about 7 years ago.
Mon, 23 Jun 2008 09:56:48 -0400| Permalink
In April Chiz and I stopped by Hong Kong for a couple of days. Lots of tall buildings. Lots of great food. And somewhat suprprisingly, lots of greenery, birds and turtles at Hong Kong Park right in the middle of town.
Tue, 17 Jun 2008 14:02:40 +1000| Permalink
In February Chiz and I went to Zao Onsen in the north of Japan for a few days Skiing. The weather great on the first day, and pretty average from then on in. But the snow was awesome and 40 lifts was more than enough to keep us busy.
As the name implies Zao Onsen has Onsen (hot spring baths). And the small stream that ran past our hotel was steaming (photo). Zao Onsen also features a forrest Juhyo, pine trees covered with snow to the point of looking like giant snow monsters (photo). Totally awewsome.
If you like to ski and are thinking about a trip to Japan then I think Zao Onsen is certainly on a par with Nagano and Hokkaido. Chiz thinks that is is better :-)
Sun, 15 Jun 2008 18:20:11 +1000| Permalink
Chiz and I just got back from a week in Tokyo. I spent some time in the office catching up with people, eating good food and doing pretty much the same outside the office, though with fewer computers and more beers. Chiz's week was pretty similar, though amusingly we tended to catch up with different friends, so we didn't see much of each other during the week.
On Saturday our good friends Chiaki and Alex were married at a shinto ceremony at Meiji Jingu. It was great to be able to be a part of such a wonderful day. And so nice to see so many faces that we don't see very often - as well as quite a few that we do see quite often.
After the ceremony there was a reception at Meiji Kinen. Then a bit later an after party in Daikanyama, an improptu after-after party in Ebisu, and more drinks in a different part of Ebisu. At that point Chiz and I went home - It was middnight and we had started the days festivities at 8:30am. Alex, Chiaki and others kicked on until 4am doing karaoke.
Congratulations Alex & Chiaki!
I have some photos of the early portions of the day. Sorry none of the ceremoney, as photography wasn't permitted at that point. And it seems that around about the start of the reception I became rather more interested in the party than taking snaps of it.
Fri, 13 Jun 2008 14:24:28 +1000| Permalink
Sunday was yet another beautiful day in Sydney and Chiz and I headed down to Circular Quay and then across to the Botanical Gardens. We found a Scotish couple inside feeding the Cockatoos and they kindly let Chiz have a go too. Not to long after that it started to get dark and we went off to find the resident fruit bats.
Mon, 26 May 2008 11:25:53 +1000| Permalink
Once again I've been working away at my patches to move the mapping of EFI on ia64 Xen to match that of ia64 Linux [*].
Why? Because it turns out that you can only teach the firmware about this mapping a maximum of one times between (hard) reboots. This of course is a problem when kexecing between Xen and Linux (aka soft reboot).
Since my last proclamation that things seemed to be working, circa LCA, two fairly serious bugs have been found (by Alex Willamson) and fixed (by me).
The first bug involved registering with the firmware where exactly the handler for PAL and SAL calls exist. I managed to find this problem quite quickly - a stroke of luck to be honest - and it was just a simple matter of teaching the relevant code the new virtual address.
The second bug was that launching an HVM (fully virtualised) domain would cause the hypervisor to lock up - clearly a problem. This took a bit longer to find the cause of and a lot longer to fix. My old enemy TLB repinning was the culprit. But after much pain I seem to have worked out what to do with the VPD (privregs) entry and appear to have a winning combination.
Now that is all finally under control I'm catching up on kexec-tools and LVS patches, some outstanding Debian bugs against packages that I maintain, and finally getting a chance to look at doing something interesting with the xen networking code. Ok, that is a long list, I wonder how far I will get before EFI consumes me again.
Thu, 24 Apr 2008 10:24:47 +1000| Permalink
These go out to all those who are partial towards either photos of
buildings on Swanson St. near the University of Melbourne or dead trees by
the Yarra River a bit further downstream. Of particular note
was the notice put up by the Bunny
Sherrif nearby some of the dead trees. more...
In unrelated news, my Kexec for ia64 Xen code still appears to be working.
I've upported to the latest ia64 xen-unstable.hg, consolidated some
TLB unpining code, and made a fresh release, v20080206 to celebrate.
Now if only someone other than me would test the code and point
out where the next big bug lies.
more....
Wed, 06 Feb 2008 18:08:19 +0900| Permalink
alt="[Building in Melbourne]" width="79" height="120" align="left"> I've just got back from a week in Melbourne, attending Linux.Conf.Au.
Linux.Conf.Au
I'll try and publish some sort of report at some stage, but suffice to say its been an absolute ripper. For those of you who aren't here, or are here but missed something, I encourage you to look at the video's of the presentations that are online, linked from the programme.
Perhaps my favourite was a lightning talk on Greasemonkey by Paul Fenwick. It doesn't seem that the lightening talks were recorded by the AV guys, but an informal summary by Paul is here.
Update: The lightening talks are now online. [OGG] [SPX]. The greasmonkey talk is the very last one for those who feel like seeking. It is also available by iteself on Youtube here.
Getting Home
I flew home via Sydney and the second leg was punctuated by and unexpected stop in Cairns around 2am to fix the de-icing equipment on one of the engines - they said they expected ice further north and couldn't take any risks, fair enough given the layer of snow that was covering Tokyo when I arrived. As the plane was still heavy with fuel, we had to circle around Cairns for an hour or so before landing. In all, we arrived about 2 1/2 hours late. No complains, the staff did a good job keeping everyone informed of what was going on.
The lovely child behind me decided that about an hour or so out of Cairns was a good time to start talking to himself. This got louder and louder and eventually turned into screaming which lasted until breakfast. The poor thing was probably distressed to the max. Though his antics did nothing to soothe myself or nearby passengers.
Just to spice things up an extra bit, the train from the airport into town was delayed - due to snow of course.
Lastly, I've heard stories of people operating checkout-counters paying more attention to the conversation with their colleague than the customer they are serving. But I was surprised to see the same tactic used by the lady who handled paperwork as I prepared to leave Sydney.
Code
On the coding front, I posted an updated version of my kexec for Xen on ia64 patches just before I left Tokyo. I haven't had time to break it yet, but it does solve a bunch of tedious bugs - mainly relating to TLB flushing or pinning - and I'm feeling pretty good about it right now. more...
I have a couple of patches and emails awaiting my attention, mainly relating to LVS, ldirectord or kexec-tools. I will try to attend to them along with my email backlog this week.
Web
Raster and I took a few moments at Linux.Conf.Au to copy the photos that he took at my wedding onto my laptop. I have put an edited version up. more...
Chizuko has also been working on a fun new site, Psychommunity.Org.
Life
Chizuko's visa has finally come through and we are planning to move
ourselves to Sydney in April. So to all our friends down there, see you
soon :-)
Mon, 04 Feb 2008 18:25:23 +0900| Permalink
Its not often somone interviews me, but this month my number was up. Howard Dahdah of Computerworld spoke to me recently about Linux-HA in the lead up to the Linux-HA BOF at Linux.Conf.au 2008 to be held at Melbourne University later this month / early next month.
Article: The low-down on the Linux High-Availability Project - Cluster improvements deepen Linux HA’s appeal
Update: The article seems to have been syndicated on
Network World,
here.
My number must really be up this time.
Wed, 16 Jan 2008 11:19:44 +0900| Permalink
As I mentioned in my last
entry for last year I've been working on some tedious problems relating
to re-pining TLB entries when switching region registers on IA64.
The good news is that the fix I found and wrote about really does work. The bad news is that only solves 2 out of 3 cases. VHPT disabled works. VHPT enabled works. per-vcu-VHPT fails.
Using the HW debugger I can look and see what the TLB entries are. Its
rather slow to use, so it isn't practical to check every time an re-pin is
to be done. But spot checks indicate that I'm twiddling the right values.
So I guess its time for some rigorous head-scratching.
Thu, 10 Jan 2008 23:35:33 +0900| Permalink
I always find myself travelling at this time of year and this year is no exception. I opened the account by travelling to my parents-in-law's place in Tsuruga, Fukui where my wife Chizuko and I spent a couple of days with her family - as is the done thing for New Year's in Japan.
Map: Tokyo to Tsuruga
The evening that we arrived it snowed quite heavily and when we awoke the world had turned white. Which was quite fortunate as Chizuko and I had it in mind to go skiing.
Imajo 365 had a reasonable amount of snow but a combination of very close weather and a very small patronage - I guess to be expected for New Years Day - meant that only one lift was open and thus only one run was available. None the less it was good to stretch our legs.
Map: Tsuruga to Imajo
A few days later we went to Hakodateyama for some more skiing. This time Kfish came up from Kyoto to join us for the day. All but one lift was open and although the runs were quite short, where as a reasonable amount of powder and I managed to end up face down once or twice.
Maps: Tsuruga to Hakodateyama, Kyoto University to Hakodateyama
Though somewhat limited, the skiing wet our appetite and upon returning to Tokyo we promptly booked a trip to go and ski at Zao-Onsen in mid-February. Neither Chizuko or I have been to the Tohoku region before and we are quite excited about the trip. We plan to spend a night in Yamagata before heading off to the slopes.
Maps: Tokyo to Yamagata, Yamagata to Zao-Onsen
We will also be joining my work-buddies in their annual ski trip, which will take us to Rusutsu in early March. The trip was in Rusutsu last year as well and was completely awesome, fingers crossed it will be this year too.
Map: Tokyo to Rusutsu
And just to save this entry from being entirely about skiing, the last item on my travel list (though not chronologically), is Linux.Conf.Au at Melbourne University in late January / early February. I'm not doing any presentations this year, and about my only commitment appart from listening to awesome talks and catching up with people is to help organise the Linux-HA BOF. Linux.Conf.Au is always a blast and I'm looking forward to this year's edition.
Map: Tokyo to Melbourne
* These aren't the actual routes between destinations taken / planned to be taken, and in some cases they aren't even close, but they are fun anyway.
Update: Without realising it my wife wrote the more-or-less the same information a few hours later. more...
Tue, 08 Jan 2008 21:41:51 +0900| Permalink
I've been looking at the cause of a particularly tedious bug for the past couple of weeks. Pinning and unpinning page-table entries when switching the region register for region 7 on ia64 Xen to be precise. Something that is needed for my seemingly endless quest to get EFI mapping to work correctly when Kexecing from Xen to Linux of vice versa.
I found the final problem at about 15:13 this afternoon, which in the end was as simple as passing a physical address to a function which was expecting a virtual one, though there were several more critical issues along the way.
Debugging this problem has been a bit like unlocking a safe. There are a variety of things that need to be done. And if they aren't all done exactly correctly, the bug manifests. However like entering the combination to the safe, there is no feedback on which of the steps might be wrong. Thus a very long and tedious trial and error procedure on my part had to be undertaken. To make things just that little bit more interesting, I wasn't even entirely sure what items needed to be on the list - a bit like not knowing how long the combination to the safe is. And I didn't know if the code was actually going to solve the problem I was intersted in fixing - fortunately it does.
I'm pretty satisfied to have found the problem, though it took a long time, though the solution needs cleaning up, and though there are several other issues that need to be addressed that may end up being just as tedious.
Bug found at 15:13.
Office closes for the year (with beer) at 16:00.
(watch this space for "bother it wasn't solved after all" early next year)
Fri, 28 Dec 2007 15:38:32 +0900| Permalink
Actually, I'm not sure if its a bashism or not, but I am pretty sure that it amuses me.
history -s owned
Then hit the up-cursor-key
Thu, 20 Dec 2007 19:06:08 +0900| Permalink
If you have a in interest in Japan beyond bizzare fashion trends and manga, then "Shebaeredomo, Shaberedomo" - dubiously "Talk, Talk, Talk" in English - a film about rakugo - a form of Japanese commedy - directed by Hideyuki Hirayama may interest you. offsite review...
I'm not entirely sure how interesting it would be to people who are not
familiar with Japan, but to explain the young boy character -
Osaka != Tokyo. Think Sydney vs Melbourne or San Francisco
vs Los Angeles, or any other rivally between two not too
geographically-distant cities, but with different dialects thrown in to
spice things up a bit.
Fri, 07 Dec 2007 15:50:09 +0900| Permalink
This year seems to be they year for wedding for my friends. I've been to no fewer than 4 in the past few months, not including my own, and missed about as many due to travel constraints.
I had already way put up way too many photos of my
wedding and honeymoon up a month or so back. But since then I have
accumulated even more. To this end I'm happy to announce that I have put
up photos by Chihiro
Kinoshta, a university friend of my wife; Seiichi
Takinami, my wife's uncle; and Isako
Takinami, my mother-in-law.
Fri, 07 Dec 2007 15:04:05 +0900| Permalink
As I wrote recently [*]
I have been having a lot of trouble getting my EFI mapping patches working.
This is a fairly essential part of my port of kexec to IA64 Xen. It turns
out that the problem was caused by not setting the protection keys up
correctly - I did this in the original C implementation, but it got left
out when I ported everything over to assembly. Easy to fix. A real pain to
find.
Thanks to Alex Williamson from HP for making a machine available that exhibited the problem - so far it only shows up on an HP rx3600 with an dual-ported Intel e1000 plugged into a hot-plug PCI slot with one of the ports hidden from domain0 using the pcihide command line option to the domain0 kernel.
Alex also reported that some user-space applications were segfaulting. This turned out to be easily reproduced by either trying to build xen - gcc would segfault at some point; or running vim xen/arch/ia64/xen/ivt.S - the file that I was hacking on. It turned out that this problem was fixed by removing a patch that I wrote to clean up the assembly code by re-using some portions at the expense of extra branches. A dubious change at best. Good riddance!
EFI Mapping Patches Take IV posted [*]. Happy times.
For a brief moment of time I had the great feeling of making two steps forward. This didn't last long. Even before I had posted the patches three new problems reared their ugly heads: The Intel Tiger2 that I have hangs on a SAL call when booting the second hypervisor when doing Xen-to-Xen kexec; The rx3600 seems to hang in purgatory when doing kexec - I hate it when that happens, its really painful to debug; And the rx3600 seems to have trouble registering the crashkernel region, which is reserved memory for kdump to run its second kernel in.
Two problems solved. Three problems found. Progress? Barely.
In unrelated news, my wife Chizuko has been learning HTML and CSS and
pushed up the latest version of her home page www.chizart.net a little earlier today.
At least something seems to be moving forwards.
Thu, 29 Nov 2007 17:45:02 +0900| Permalink
On my bike ride to
and from work I wind my way around the imperial palace for a kilometre
or so. So far on three occasions over the past year and a half, including
twice in the last week a bunch of police and the like were at one of the
larger entrances to the palace. And police were at all the minor entrances
to the footpath that encircles the palace that I pass. I assume that this
indicates that the emperor, his son the crown-price or some other dignatory
is coming or going. Today there were several helecopters circling
over-head. I wonder if they are on to me...
Wed, 28 Nov 2007 14:16:15 +0900| Permalink
I'm off to Santa Clara for a couple of days to attend Xen Summit.
Its my first trip to California for a couple of years, I'm looking
forward to catching up with some old faces and meeting some new ones.
On the hacking front I'm hoping to shift over IPVS to use CTL_UNNUMBERED for its sysctl interfaces. Several of the sys_sysctl interfaces are broken and seem to have been in that state ever since they were created several years ago. I did some work on fixing them up, but the prevailing logic seems to be that sys_sysctl isn't used by anyone so its better to rip out support for it. And who am I to argue?
And I am still puzzling over why my EFI mapping patches blow
up on Alex Willamson's machines but not mine. It looks like the problem
is that the hypervisor is trying to handle a fault for EFI memory
that should be left to domain~0 to handle. But I can neither reproduce the
problem nor work out why its occuring. Its a bit of a bummer because
the rest of the ia64 xen kexec work has been merged but without this
change its not functional. Mailing list mublings about this.
Tue, 13 Nov 2007 12:35:18 +0900| Permalink
I've been riding my bike to and from work a bit after a longer break than I'd care to admit. This morning the odometer clocked over 4000km. Which means I've ridden about that far since I bought the bike about 18 months ago.
Some semi-interesting facts about this:
Ok, I'll stop talking about bikes now.
Wed, 07 Nov 2007 17:28:48 +0900| Permalink
Anyone who has lived on Japan with a visa will know that its mandatory to have an alien registration card (gaikokujintorukushomeisho). Its also mandatory to carry it around, though interestingly other than when you go through immigration more or less the only people who can demand to see it are the police and only if they suspect you of committing a crime.
There are two other documents that you need, a passport and a visa. And if you'd like to leave Japan and come back without invalidating your visa you also need a re-entry permit. So there are quite a few documents, and they are all related to each other. After a fairly enlightening experience at the Shinjuku ward office this morning I thought I would write a few paragraphs about my understanding of the relationship between these documents and how updating one effects updating the others. In all of this the key thing to remember is that there are three separate governments involved and they only update their own documentation.
Passport
This is pretty straight forward. If you are going to get a Japanese visa then you need a passport issued by your home country. And if it is going to expire while you are living in Japan then you need to get it renewed.
As an asside, the Australian consulate in Tokyo managed to get me a new passport in 4 days last December, which included a round trip to Canberra. I was well impressed as I didn't even ask for speedy service.
Visa
If you are not a Japanese national and wish to stay in Japan for an extended stay then you are supposed to get a visa (nihonkokusasho). Unsurprisingly these are handled by an arm of the Japanese federal government. I'm won't to go into how to get one. But basically it ends up being a document which is affixed to your passport. If you get a a re-entry permit (sennyukokukyoka) then it will also be a document that is affixed to your passport.
The interesting thing here is that if you renew your passport while holding a visa, then you are likely to run into a situation where the visa in your old passport has not yet expired. However, this isn't a problem. The procedure is to just carry both the current passport and the old passport which holds the visa visa, along with your alien registration card, when you need to deal with Japanese officials - which basically means when immigration officials or the people who handle alien registrations at your local ward or city office.
So there is no need to update your visa just because your passport was updated. Ditto for re-entry permits. Nice and simple.
Alien Registration Card
Alien registration cards are administered by local governments in Japan. Officially its the mayor giving you his blessing to stay in his neighbourhood. Not a problem, just more red-tape.
The slight trick in this is that any of the details on your alien registration card change then you need to go to your local ward or city office and get them updated. I was aware that you need to do this when your address or address of your employer changed. But what I didn't realise is that you are also supposed to do this if your passport number changes, which mine did when I renewed my passport. You also need to get the period of stay on the registration card updated if you get a visa extension (zairyukikankoshinkyoka). I did both this morning. 11 and 19 months late respectively.
The curious thing about all of this that I have travelled through Japanese
immigration several times before anyone noticed. Which seems doubly
curious given the long caution that the nice lady at the ward office gave
me this morning.
Wed, 07 Nov 2007 16:40:50 +0900| Permalink
Chizuko and I were legally married at the Shinjuku-ku ward office in Tokyo
on the 7th of July 2007 (7/7/7). We had a civic ceromony and reception
at Ristorante Donnaloia in Kobe on the 23rd September. We took our
honeymoon in Fiji and Sydney from the 30th September until the
15th October. During the time in Sydney we held a party at the Coogee
Bay Hotel.
A rather raw and uncut dump of the photos from these events
is up. We hope to find some time to cut things
down to a more reasonable size in the not to distant future.
more...
Mon, 29 Oct 2007 17:23:24 +0900| Permalink
I'll forgive those who think that "SMP Xen IA64 Kexec" is a meaningless arrangement of characters, but its actually the SMP (multi-processor) for IA64 (a exotic architecture from Intel) potion a project that I have been working on for quite some time. The project itself is to port Kexec (a soft-reboot mechanism) to Xen (a virtualisation technology). If this isn't making any sense and you'd like it to, please see the presentation that I made at Linux.Conf.Au this year on the topic Kexec: Soft-Reboot and Crash-Dump Analysis for Linux and Xen.
Today I was very happy to be able to make the 20070713 alpha release of the code, which includes SMP support. This may not sound very exciting, but I seem to remember promising myself that uni-processor version would be ready for Christmas. A goal which I reached with a few days to spare. And distractions aside, I've been working on the SMP stuff ever since. Ok, I admit it, there were a lot of distractions :-)
Most of the work involved porting portions of CPU_HOTPLUG to xen, the portions that allow CPUS to be taken off-line, which is done just before kexecing - the OS boot code handles bringing them back online. The catch was making sure that the entries that had been pinned into the TLB were removed correctly. Empirically both removing an entry that wasn't present and failing to remove one that was present turned out to be fatal. A tricky thing to debug when the OS has basically been shut down. Well, not so tricky once you work out thats what the problem is, and narrowing down that as the cause took most of the time.
As its now the Friday afternoon before a 3 day weekend,
I'm happy with the process thats been made, as long as I don't
reflect on how long it took... oops.
Fri, 13 Jul 2007 17:56:01 +0900| Permalink
Recently I realised that I had absolutely no idea where the passbook for my Japanese bank account was. Some of my Japanese friends informed me that this was quite a problem, as if someone had come across it they could quite easily use it to access my funds. After ransacking both where I live and where I work to no avail I concluded that I really needed to head off to the bank to try and sort things out.
Upon getting to the branch of Tokyo Mitsubishi UFJ Bank a few minutes walk away from where I work I intuitively looked for the ticket machine so I could get a ticket for the appropriate section and join the queue. But for some reason, despite my instincts, I decided to go and chat to the elderly attendant who seemed quite bored. This was fortunate because upon describing my predicament he informed me that the branch we were in was of the formerly UFJ variety but to order a new passbook I would need one of the formerly Tokyo Mitsubishi variety due to my account being of that variety. He promptly gave me a map and sent me to the right place, about 10 minutes away.
On the one hand I might have been quite annoyed about this, but as it happened I was quite happy to have the chance to take a walk. So off I went, pondering the stories that friends of mine had told me about how many years into the future it would be until Tokyo Mitsubishi and UFJ actually merged their computer systems so they could function as a single bank. I guess that things really do go pretty slowly when trying to merge two gigantic organisations.
The Tsukiji branch was able to cancel my old passbook and order a new one for me without any bother, and in particular no tedium revolving around what form of identification that I was going to use - sometimes being a foreigner in Japan is a real pain, fortunately this wasn't one of those times.
While in the branch someone tried to suggestive-sell me an upgrade of sorts on my ATM card. Fortunately before saying yes I remembered the warning a friend had given me about the perils of the new card, so I was able to politely turn down the offer.
On the way back I wandered through the fish markets in Tsukiji, and pondered what a relaxing day I was having. Then I remembered I'd just been to the bank. I guess that everything is what you make of it.
Wed, 11 Jul 2007 15:38:42 +0900| Permalink
Things that I found out on the way to work this morning:
Mon, 21 May 2007 10:59:51 +0900| Permalink
Having lived near a fire station for the past few years, I conclude that every time it rains someone sets fire to their house.
Wed, 02 May 2007 17:02:35 +0900| Permalink
I went over to Seattle to visit my good friends Dave Miller and Soyong
Park. While I was there we had some fine times exploring Seattle, and even
found time to take a day-trip to Vancouver. Here are some photos that I
took on my journey.
more...
Mon, 09 Apr 2007 20:49:16 -0700| Permalink
This year the cherry blossoms are blooming early, and the early-blooming variety is already in full blossom. Magnus, Chiro-chan, Chiz-chan and I took advantage of this with some mid-March hanami action yesterday.
Mon, 19 Mar 2007 10:56:16 +0900| Permalink
Today I went along to a symposium held by The Linux Foundation Japan. At the Q&A session at the end of his presentation Andrew Morton was asked when he thought some new work might me merged into the mainline tree. He began his response with:
Its a major change to the CPU scheduler by Con Kolivas.
He's from Australia, and thats where all the best code comes from.
I wonder what the traslators made of that.
Tue, 13 Mar 2007 16:38:36 +0900| Permalink
I've been using crosstool for quite a while now to produce cross compilers which I use primarily to produce code to run on ia64 while working on kexec.
One of the nice things about crosstool is has a vast selection of possible architecture, gcc and glibc combinations available. Around 3,700 by my counting. One of the difficult things about using crosstool is choosing which configuration to use for a given architecture. There is a build matrix available to help. However I found that I was not getting the same results as it indicates and that there seem to be a lot of combinations missing. Is this because they are invalid, to save space, or for some other reason? I decided to investigate further.
I developed a small script to test every combination available. I set this running on my desktop at work for a day or so, and given its rate of progress it seemed like it would take about 3 months to complete. So I fired it up on my machine at home to try and half this time. This all worked pretty well, except that the machine at home which is usually off has to run its fan quite a lot to keep itself cool now. An annoying whirring noise that permeates my apartment for about 5 seconds every 30 seconds. No big deal, that whirring noise is for the cause.
This morning I noticed that the machine at home actually hadn't made any progress for about 4 days. The whirring was still going on, so I knew it was doing something. A quick investigation showed that with glibc-2.2.3 if you pass in an target that it doesn't know about (e.g. sh-unknown-linux-gnu) then it's configure gets stuck in an endless loop. Examining a little closer it seems that the loop examines a string of ever growing length, so presumably it will eventually run out of memory and segfault, but it was still going strong after 4 days, and thats just too long.
I came up with a trivial hack to fix this. If config.sub fails, configure should too. The version below prints out config.sub's output (error message which is curiously printed to sdtout instead of stderr) before bailing to make the log a little less cryptic.
It all seems to be working fine now. And as I am now someway into a lazy Sunday afternoon, I thought it would be a great time to share my pain.
--- glibc-2.2.3/configure 2007-03-11 12:06:32.000000000 +0900
+++ glibc-2.2.3/configure 2007-03-11 12:06:35.000000000 +0900
@@ -914,7 +914,10 @@
esac ;;
esac
-host=`${CONFIG_SHELL-/bin/sh} $ac_config_sub $host_alias`
+if [ ! host=`${CONFIG_SHELL-/bin/sh} $ac_config_sub $host_alias` ]; then
+ echo "$host"
+ exit 1
+fi
host_cpu=`echo $host | sed 's/^\([^-]*\)-\([^-]*\)-\(.*\)$/\1/'`
host_vendor=`echo $host | sed 's/^\([^-]*\)-\([^-]*\)-\(.*\)$/\2/'`
host_os=`echo $host | sed 's/^\([^-]*\)-\([^-]*\)-\(.*\)$/\3/'`
@@ -1388,7 +1388,10 @@
esac ;;
esac
-build=`${CONFIG_SHELL-/bin/sh} $ac_config_sub $build_alias`
+if [ ! build=`${CONFIG_SHELL-/bin/sh} $ac_config_sub $build_alias` ]; then
+ echo "$build"
+ exit 1
+fi
build_cpu=`echo $build | sed 's/^\([^-]*\)-\([^-]*\)-\(.*\)$/\1/'`
build_vendor=`echo $build | sed 's/^\([^-]*\)-\([^-]*\)-\(.*\)$/\2/'`
build_os=`echo $build | sed 's/^\([^-]*\)-\([^-]*\)-\(.*\)$/\3/'`
@@ -1777,7 +1777,10 @@
esac ;;
esac
-build=`${CONFIG_SHELL-/bin/sh} $ac_config_sub $build_alias`
+if [ ! build=`${CONFIG_SHELL-/bin/sh} $ac_config_sub $build_alias` ]; then
+ echo "$build"
+ exit 1
+fi
build_cpu=`echo $build | sed 's/^\([^-]*\)-\([^-]*\)-\(.*\)$/\1/'`
build_vendor=`echo $build | sed 's/^\([^-]*\)-\([^-]*\)-\(.*\)$/\2/'`
build_os=`echo $build | sed 's/^\([^-]*\)-\([^-]*\)-\(.*\)$/\3/'`
Sun, 11 Mar 2007 12:48:15 +0900| Permalink
It occurred to me that while I have seen quite a few dictionaries of Aussie slang that explain things in (alternate?) English, I hadn't run across a Japanese translation of such important concepts as bonzer and ripper*. To this end I have started adding Australian slang to saidict, an add-on Japanese/English dictionary that is intended to supplement Jim Breen's edict.
So far there are only about a dozen entries. They are all tagged as "Australian slang", so you can easily grep for them. Corrections, additions, and other feedback more than welcome.
Download: saidict.utf8
Home: ~horms/junk/
This is slang, and as such some terms may be vulgar or offensive.
Those who are easily offended should consider
entertaining themselves elsewhere.
* According to Kfish ripper is actually a term originates from Chinese and was introduced to Australia by gold diggers of Chinese origin the 1850s. The same word, rippa also exists in Japanese, and has the same meaning :-)
Mon, 05 Feb 2007 13:44:24 +0900| Permalink
I got sick of the old brokenness of by RSS feed,
and have written by own feed generator, with the help
of Alex.
Out with the old brokenness, in with the new!
The RSS feed is, as before, available from http://www.vergenet.net/~horms/pleb_blossom/rss.xml.
The Atom feed is no longer available.
The code, though work in progress and probably not useful to anyone except me, is available.
Download: genBlog
Home: ~horms/junk/
Feedback, please.
Fri, 02 Feb 2007 17:53:18 +0900| Permalink
This is a perl script based on Mail::Box::Manager which given an mbox on stdin, will execute the command given as arguments once on each file. Kind of like xargs --max-args 1, but for mailboxes.
e.g.: mail.xargs sendmail < ~/INBOX
Download: mail.xargs
Home: ~horms/junk/
Wed, 09 Aug 2006 15:55:00 +0900| Permalink
I'm off to Canada for two weeks tomorrow. First stop is Ottawa for a week to attend OLS and surrounding events. Then a week of not doing much at all, probably in Motreal.
I probably won't be doing much email during this time, so if you need me for something, please be patient.
Thu, 13 Jul 2006 22:46:14 +0900| Permalink
I spent far to long working out that the Kuroutou Shikou KRTV-7131 can use linux's saa7131 (~2.6.17-rc5) driver without modification by specifying card=65, which uses the settings for a V-Stream Studio TV Terminator. This is basically the same board, except that the one I have is missing the FM Tuner option.
modprobe saa7134 card=65
If your kernel doesn't automatically insert the tuner module at this point, you should add it manually. It should automatically detect a type 54 tuner. If this doesn't happen, then you might be able to specify type=54 on the modprobe command line. Failing that, recompile the kernel with CONFIG_KMOD so that the saa7134 module insertion will automatically pull in tuner.
modprobe tuner #type=54
Sun, 04 Jun 2006 21:58:49 +0900| Permalink
I poped over to Hong Kong with Chiz to visit my brother. The weather was crap, but we had a great time anwyay. It was my first trip to Hong Kong, I hope I can go back soon. photos...
Tue, 30 May 2006 18:48:44 +0900| Permalink
It seems that I'll be heading up to Ottawa for OLS in July with my colleague Magnus Damm. Its my first trip to that part of the world for a while, I'm looking forward to the conference, and looking forward with catching up with lots of people.
Excitement!
Thu, 18 May 2006 16:36:40 +0900| Permalink
I started the day with a shower,
then I put the wrong pants on...
Fri, 12 May 2006 11:31:15 +0900| Permalink
I've been using xdm as my login manager since I've been using X. Its simple, if you configure it up its not too ugly, and it works. Unfortunately the Debian Xorg maintainers have a habit of breaking it regularly (in unstable anyway). Sometimes backing out to the previous version works, sometimes its just easier to use something else.
On one such occasion I tried using gdm. Gdm has some nice featues, like being able to configure it from within itself, but watching it draw the background on a P4 2.5Ghz was just insulting (or perhaps infuriating), so I managed to get xdm up and running again. The next time I had issues with xdm I ended up reverting to "exec xinit" from a console. On reflection the console is less insulting than gdm.
The other day I bought myself a sexy new monitor, installed xdm, and on that nice big screen, Debian's default xdm config was well, just so damn ugly it insulted me (though not as much as gdm). As my xdm config had bitrotted over the years I got too and finished packaging entrance, the display manager from Enligthenment. That definately was not insulting, its fast, looks good, and with a bit of fixing on my part, now works. I'm planning to hack on it some more to get better error reporting, and more configuratbility in there.
| Insult-O-Meter |
gdm > xinit > xdm > entrance (slow) (console ain't X) (Debian's config too ugly) (nice) |
At the end of all of those fun and games, Debian packages for entrance, entice (Enlightenment's somewhat bitrotted image viewer), and the required libraries are now available on packages.vergenet.net/vergenet. And all the changes I made to make this happen are in CVS. The packages are apt-gettable, though you'll end up with all the other packages that are lying around there. Drop me a line if you want me to break it up a bit.
Mon, 08 May 2006 17:29:50 +0900| Permalink
Today was the first decently warm day of the year here in the metropolis of crying babies. I'm looking forward to the warm party months.
In completely unrelated news, I went to watch my friends' band, The Back Horn, play at Liquid Room in Ebisu last week. So much energy I was bouncing off the walls for three or four days afterwards. Rock on!
Mon, 01 May 2006 19:21:29 +0900| Permalink
I recently started working on an Imlib2 loader for Nikon's NEF format as I want to play with the NEF files that my my D70 can produce. The loader implementation will probably used libtiff, and the code I have so far does that. However, I was having some trouble working out exactly what values are where in the TIFF, so I wrote a tiff directory decoder from scratch. Its some what limited, but it should be able to give you a directory listining given a TIFF. Well, an NEF from a D70 anyway.
I put it online on the off chance that its useful to someone else too. more...
Thu, 27 Apr 2006 23:23:30 +0900| Permalink
Another quickie. Sort the human readable output of du -h, ls -h, etc...
On request for my good mate Fusion94.
Download: sort-hr
Home: ~horms/junk/
Wed, 26 Apr 2006 18:01:46 +0900| Permalink
A quick script that I brewedup to create a Debian root directory, suitable for use with NFS-root, using debootstrap. Actually, its probably useful for other things too. debootstrap is almost there, but not quite.
Download: ndebootstrap
Home: ~horms/junk/
Mon, 17 Apr 2006 18:41:48 +0900| Permalink
Apparently someone has packaged Pia for Debian.
# dpkg -p pia Package: pia Priority: extra Section: x11 Installed-Size: 112 Maintainer: Gerd KnorrArchitecture: i386 Source: xawtv Version: 3.94-1.1 Depends: libc6 (>= 2.3.5-1), libfontconfig1 (>= 2.3.0), libfreetype6 (>= 2.1.5-1 ), libice6, libjpeg62, libsm6, libx11-6, libxaw7 (>> 4.1.0), libxext6, libxft2 ( >> 2.1.1), libxinerama1, libxmu6, libxpm4, libxrandr2, libxrender1 (>= 1:0.9.0.2 ), libxt6, libxv1, libxxf86dga1, libxxf86vm1, xlibmesa-gl | libgl1, zlib1g (>= 1 :1.2.1), xawtv-plugins (= 3.94-1.1) Size: 34328 This is a simple movie player. It is used by xawtv and motv to playback movie files which are recorded from TV.
(I lied about the pumpkins)
Tue, 11 Apr 2006 10:53:19 +0900| Permalink
Japanese news papers are notorious for being difficult to read because of the use of obscure vocabulary and compound nouns. Today I came across this beauty.
Gaimushobeisyukyofukuyocho means "ministry of foreign affairs' deputy director United States affairs", which is by all accounts a position within the North Korean government. Or in other words, the North Korean official that is leading up their delegation to the six-way talks with China, Japan, South Korea, Russia and the Uinted States on North Korea's nulear policy.
I added it to saidict for safe keeping.
References: asyura.com, asahi.com.
Tue, 04 Apr 2006 21:12:07 +0900| Permalink
In the same artticle that I was talking about in my previous post, I found another nice long job title.
Gaikoutsushoshogaikouseisakushitsucho means "ministry of foreign trade's director of foreign policy, a position in the South Korean government. He'll be chatting away with North Korea's Gaimushobeishukyofukukyocho in the six-way talks relating to the North's neuclear aspirations.
I also addded this to added it saidict for safe keeping.
References: asahi.com.
Tue, 04 Apr 2006 21:12:07 +0900| Permalink
For the past few months I have been working on code relating to Xen. This week a colleage joined the team. To get him up to speed I've been helping him to get xen built, which can be a less than trivial task. On the third day he came out with this gem.
| An interesting mix of kernel and user space crap. |
| Magnus Damm |
Wed, 29 Mar 2006 11:43:36 +0900| Permalink
While doing some work on okiniri I stumbled upon a feature of date that I wasn't aware of. I'm not sure when these got introduced, but they are certainly there in 5.94.
You can ask date to do arbitary timezone conversions for you. So lets say you're sitting at your desk in a lifeless office complex in Tokyo, and you're chatting to Kfish on IRC, who is in Sydney wondering about scheduling some downtime for a server in San Francisco.
"Mate, what time will midnight in Sydney be in San Francisco?", he asks.
Rather than be lame and run off to some web-based for, I dig out date.
# TZ="America/Los_Angeles" date --date='TZ="Australia/Sydney 00:00"' --rfc-2822
Tue, 21 Mar 2006 16:00:00 -0800
This works despite my prevailing timezone being Japan/Tokyo.
# date --rfc-2822
Thu, 23 Mar 2006 09:22:21 +0900
The --rfc-2822 isn't neccessary, but I like that format. Of course you can ask date for any format, and it will default to the prefered format for your prevailing locale. You can also supply the input in various formats. Of course you can use the current time instead of the current time, and omiting one or both of the TZ specifications will use the prevailing timezone.
The available timezones are in /usr/share/zoneinfo/, and the prevailing one being /etc/localtime which is typically a symlink into /usr/share/zoneinfo/
For more information about date and timezones see "27.9 Specifying time zone rules" in the corutils manual, online or locally by running info date
Thu, 23 Mar 2006 09:26:29 +0900| Permalink
I've got in on Hanami early this year, snapping these shots on the way to work this morning. Most of the trees havn't started blossoming yet. But Shinjuku Chuo Park has a few trees of a variety that bloom about a week early, I first noticed them showing colour on Saturday. So I guess that next weekend the real fun will start. more...
Mon, 20 Mar 2006 12:12:53 +0900| Permalink
When working with Git I sometimes make wrappers to help me out. The only one I am currently using is okiniri-export. Its pretty simple, given a changeset, report its commit message, diffstat and diff. I think the date-foo that it ues is pretty new, it works with "date (GNU coreutils) 5.94", but might break with earlier versions. more...
Mon, 20 Mar 2006 10:05:19 +0900| Permalink
Attending Linux.Conf.Au is always a time of inspiration for me. This year's excellent edition in Dunedin was no exception. I'm hoping to find time to explore all sorts of ideas that in the coming year. Here is the first cab off the rank.
As part of his key-note, Mark Shuttleworth discussed how the Rosetta component of Launchpad is working to break down barriers for those wanting to contribute translations to open source projects at his key-note. I started thinking about an unreleased piece that I wrote last year about Open Source and multilingual desktops. And after having a chance to chat about my ideas with Mark over lunch while he was in Tokyo last week I was encouraged to give my ideas a wider audience.
Using Free and Open Source Software Principles to Create an
Integrated Multilingual Desktop Environment
Free and Open Source sofware makes use of the ability for users to create
their own solutions, and collaboration to create high quality software at a
rapid rate. This discussion looks at how similar techinques could be used
to improve multilingual desktop environments.
Checking the dates I see that was inspired to make the original after various discissions with Kfish not long after Linux.Conf.Au last year. Figures.
Sun, 12 Feb 2006 23:58:46 +0900| Permalink
A first cut of some photos from in and around Linux.Conf.Au 2006. Includes Dave Miller, Greh Lehey, Rusty and Jeff Waugh shaving for the John Lions Chair. more...
Mon, 30 Jan 2006 13:02:16 +1300| Permalink
I've finished a draft of my presentation to be made at the Debian Miniconf at Linux.Conf.Au in Dunedin next week. Feedback welcome. more...
Thu, 19 Jan 2006 12:02:27 +0900| Permalink
I can tell that LCA 2006 (23rd - 28th January) is getting close, becasue I've spent the weekend so far preparing presentation slides and a laptop to take. So, for those of you who have been waiting to track me down for good or for bad, you won't have to wait too much longer. I'm abandoning the cilly Tokyo winter and heading down to New Zealand for the entire week.
I start off with a quick trip to the Bay of Islands, north of Aukland to visit my Dad (no I'm not from New Zealand and neither is he, but he's been living there for about a year now). I should get down to Dunedin on Monday afternoon, in time to freshen up for the performance night at Arc Cafe as part of the Dicital Art Mini-Conf, organised by Kfish and the gang.
On Tuesday, if all goes to plan I'll be giving a presentation at the Debian Mini-Conf about what the Debian Kernel Team has been up to in the past year. And on Thursday my good mate Dave Miller is giving a keynote. I haven't even looked at the programme yet and already my week is filling up.
LCA 2006, Dunedin.
I'll be there, and I will be wearing
Speights underwear!
Sat, 14 Jan 2006 17:47:36 +0900| Permalink
You can complain about the DVD firmware in Macs only letting you change the region 5 times. But the thing is that Apple doesn't want to be a hardware company. Or a sofware company for that matter. They want to be a music company, and perhaps a video company. Like a Sony that doesn't suck.
Fri, 13 Jan 2006 15:11:18 +0900| Permalink
I've been thinking about what I'd do if I bought an aparemtment and renovated it. Natrually, embeding technology in the walls is near the top of the list. But I'm not entirely sure of my options. For instance, does anyone know if there is sound equipment available that will refuse to play Alanis Morissette?
Thu, 12 Jan 2006 17:37:26 +0900| Permalink
If you have a PSP (Play Station Portable) in Japanese, and you want to change it to English, and you don't read Japanese and thus can't read the menus, this page is for you. more...
Sat, 07 Jan 2006 13:26:11 +0900| Permalink
I hope its not to late, I just made my new year's resoution
Only do things that are awsome.
Wed, 04 Jan 2006 14:02:28 +0900| Permalink
Photos from new year's in and around Shinjuku in Tokyo,
including hatsumode, a trip to a temple in the cold of the
night on new years day.
more...
Sun, 01 Jan 2006 18:00:21 +0900| Permalink
Given a tar-ball, if there is a debian/ directory inside unpack the tar-ball and try to build a debian package from it.
Given a .dsc file (and associated tarball and possibly diff), try to unpack and build a debian package from it.
Download: dpkg-unpack-build
Home: ~horms/junk/
Tue, 27 Dec 2005 14:35:41 +0900| Permalink
Query and Change the power states of outlets controlled by an APC PDU. It always toggles the first available outlet, Assuming that each user only having access to one outlet. I'm not sure if the other configurations are even possible.
# power-toggle 172.17.100.1 7 pass status STATE: ON # power-toggle 172.17.100.1 7 pass off Turning off... # power-toggle 172.17.100.1 7 pass off Already off... # power-toggle 172.17.100.1 7 pass reset Turning on... # power-toggle 172.17.100.1 7 pass reset Turning off... Sleeping for 10s... Turning on...
Download: power-toggle
Home: ~horms/junk/
Mon, 19 Dec 2005 13:21:07 +0900| Permalink
ccache is a compiler cache. I find it greatly reduces compile time when I have to recompile the same code over and over, which is often the case when I prepare Debian packages.
It can be invovked in a variety of ways. For me the easist is to make a symlink the name of the compiler in my path. For example, ~/local/bin is in my path before /usr/bin, and I want all invocations to gcc to use the compiler, so I do this
# which gcc /usr/bin/gcc # (cd ~/local/bin && ln -s /usr/bin/ccache gcc; ) # which gcc /home/horms/local/bin/gcc |
So now when I invoke gcc it actually runs ccache, it does its magic and calls /usr/bin/gcc as needed. As ccache invocations look exactly the same as gcc invocations, this means its easy to get ccache into a build.
# which gcc /usr/bin/gcc # ccache-update-links ~/local/bin # which gcc /home/horms/local/bin/gcc |
But with different versions of gcc, and recently doing some work with cross-compilers, I have a lot of symlinks I want to create, and this is a bit of a chore. So I wrote a ccache-update-links to do this for me. Note that it actually just creates and overwrites links, it doesn't know how to delete dead links. Perhaps I should add that...
Download: ccache-update-links
Home: ~horms/junk/
Thu, 08 Dec 2005 13:20:56 +0900| Permalink
Last night a went and picked up my camera which packed it in during my trip to Korea. Its was the second time I went to pick it up as the first time round it was still malfunctioning when the shopkeeper gave it to me. It seems that the fix was to flash the firmware up to version 2.00. Kind of like having a new camera on the inside.
Must away, time to play.
Fri, 18 Nov 2005 09:48:45 +0900| Permalink
Saidict has expanded 100% from one word to two! More...
Fri, 18 Nov 2005 09:45:29 +0900| Permalink
I have had an unverified report form myself that the following will allow file to correctly detect Ulead Photo Explorer5 (.pe5) files.
cat >> $HOME/.magic << EOF # Ulead Photo Explorer5 # Source: http://www.jisyo.com/cgibin/view.cgi?EXT=pe5 (Japanese) 0 string IIO2H Ulead Photo Explorer5 EOF
Its Debian Bug #339397 for now.
Wed, 16 Nov 2005 10:29:47 +0900| Permalink
I made it back from Korea in once piece (several weeks ago), however, sadly, my camera did not. Well, its in once piece, but one not working piece. I took it back to the shop for repair under warranty, they called to tell me it was fixed. But alas when I went to pick it up, it was still not working - though in a slightly different way. I think its an electronics failure. I hope it gets fixed soon so I can take some new snaps.
Fortunately Dave's attempts at taking photos were more successful than mine. As an added bonus, his trip with Soyong was around 2 weeks, whereas Chiz and I only dropped in for 2 days. You can read all about it on his blog [Oct] [Nov] which also has links to lots of awesome photos.
In semi-related news, by brother sent me a copy of Lee Frost's "The Complete Guide to Night and Low-Light Photography". Mmm, wish I had my camera back...
Wed, 16 Nov 2005 10:25:00 +0900| Permalink
I love my inbox so much that I made a graph of it.
Ok, thats a lie, except the graph bit.
See what I've been reading here
Thu, 27 Oct 2005 11:41:44 +0900| Permalink
New images for enlightenment's E-Biff.
|
No New Messages |
New Messages |
9 Messages, 7 New | |
| New: |
[png] [xcf] |
[png] [xcf] |
[png] [xcf] |
| Original: |
[png] |
[png] |
[png] |
Fri, 07 Oct 2005 16:19:11 +0900| Permalink
I've been thinking of writing a supplementry Japanese-English diectionary, basically for words I can't find in Edict.
So far I have an entry for "retranslation function", courtesy of Yuichi Yoshida, the uim-anthy author.
Works wonderfully with Gjiten, and should work equally well in anything that supports edict format dictionaries. Though it would be arguably more useful if it had more entries.
Download: saidict.utf8
If you need another format, pkf and friends can do that for you.
This and other exciting can be found in my junk-yard.
Fri, 07 Oct 2005 16:16:31 +0900| Permalink
My good mate Derek sent me this link explaining the ways of 25-way Janken - Extreme Scissors Paper Rock.
Two words: Insane!
Sat, 01 Oct 2005 15:22:26 +0900| Permalink
I decided to take a break from email.
So I left the office early, went home,
and answered some email.
Fri, 16 Sep 2005 20:35:45 +0900| Permalink
As a follow up to my breif report on uim-anthy, I should mention that for many (prehaps all) applications the locale also needs to be set for Japanese input to work. This means you a) need a Japanese locale (dpkg-reconfigure locales on Debian) and b) you need to set the locale.
If you want all messages and interaction to be in Japanese, when available, then set LC_ALL:
export LC_ALL=ja_JP.UTF-8
If you want messages and the like in the default language, but also want to be able to do input in Japanese, then set LC_CTYPE:
export LC_CTYPE=ja_JP.UTF-8
Of course, you can set LC_ALL to other values, to use different languages, and LC_CTYPE can be set independantly.
If you use EUC-JP instead of UTF-8, or more to the point, have an application, such as Eterm, that supports EUC-JP but not UTF-8, then use ja_JP.eucJP as the locale instead. Noting that you can set this environment on a per programme invocation basis.
Miscelaneous Notes:
export PAGER=jless
Mon, 12 Sep 2005 12:26:03 +0900| Permalink
If you are runing Debian's 6.8.2.dfsg.1-2 version of xlibs and you have a Japanese keyboard, you might need this patch to get your backslash/underscore and hirugana/katakana keys.
--- /etc/X11/xkb/keycodes/xfree86 2005-07-12 21:58:36.000000000 +0300
+++ /etc/X11/xkb/keycodes/xfree86 2005-07-16 13:21:25.000000000 +0300
@@ -151,8 +151,10 @@
// Keys that are generated on Japanese keyboards
alias <HZTG> = <TLDE>; // Hankaku_Zenkaku toggle
- <HKTG> = 208; // Hiragana_Katakana toggle
- <AB11> = 211; // backslash/underscore
+ <HKTG> = 120; // Hiragana_Katakana toggle
+ //<HKTG> = 208; // Hiragana_Katakana toggle
+ alias <AB11> = <KPDC>; // backslash/underscore
+ //<AB11> = 211; // backslash/underscore
<XFER> = 129; // Henkan
<NFER> = 131; // Muhenkan
<AE13> = 133; // Yen
Sun, 17 Jul 2005 15:44:36 +0300| Permalink
After years of struggling with kinput2 with cannaserver, and occasionally dabling with with im-ja with wnn, I have switched over to uim-anthy, and its completely awsome. The canna conversion actually works.
On my Debian system I just installed uim-anthy, uim-gtk2.0 and uim-xim to get the input method itself, gtk module, and xim bridge. Combined, these packages allow uim-anthy to be used with gnome applications, firefox and X aplications such as Eterm.
I also made sure that uim-xim is running and that following the XMODIFIERS and GTK_IM_MODULE environment variables are always set. If you have a .xession, just add this sinppit in there before your window manager runs.
uim-xim& XMODIFIERS=@im=uim GTK_IM_MODULE=uim
To configure uim-anthy run uim-pref-gtk. This also provides a list of what key combinations are used to do conversions.
I also tried using scim, which can in turn use uim-anthy, but I found that for my needs using uim-anthy was a better fit.
Sun, 17 Jul 2005 15:33:53 +0300| Permalink
I am happy to announce the first release of perdition in over a year. It includes lots of updates, of particular note; A potential buffer overflow in the Berkely DB module has been fixed. The LDAP module can now use arbitary URLs if compiled against OpenSSL 2.0.0 or newer. All asyncrhonous IMAP messages are now passed back to the end-user. The level to which add_domain pears down a domain is configurable. And SSL now works in inetd mode. Thanks to all who helped out with patches and bug reports.
A complete list is avalable in the ChangeLog.
Perdition 1.17 is available for download in the Perdition download directory.
For more information on Perdition, please see the Perdition home page.
Wed, 22 Jun 2005 15:53:25 +0900| Permalink
Theo de Raadt, famed for his work on OpenBSD has given some analysis of why he dislikes Linux in a Forbes.Com article. Thanks to Jaq for putting me on to this.
I found it particularly interesting the way he makes reference to the having lots of developers working on rapidly developing new features to reach the needs of users as a bad things. And furthermore commented on how big companies fuding Linux development fueled its popularity while in practice they were using the opensource community as an unpaid workforce.
The thing is, that is the whole idea. People working together to produce software that meets the needs of people, when they want it, not when some ivory-tower development model says they can have it. It turns out that this also works pretty well for companies, and they realise that by funding open source development they get a whole lot more out than they put in. And the really great thing is, everyone gets a whole lot more out.
While I don't have anything against OpenBSD, I really think Theo is missing the point. I'd like to close off with a recent contribution I made (vanity?) allow someone to do local portmaping for LVS, because they needed to be able to do that.
Tue, 21 Jun 2005 12:58:14 +0900| Permalink
If anyone out there in the ether has a Dynabook with a Synaptics touch pad, and is having trouble with keyboard input on 2.6 kernels, try this patch. Applies against 2.6.12. The sattelite check was added between 2.6.11 and 2.6.12, so if you have an earlier kernel, just graft in toshiba_dmi_table, or in a pinch, just always set the rate to 40.
--- linux-2.6.12.bak/drivers/input/mouse/synaptics.c 2005-06-18 14:24:58.000000000 +0900
+++ linux-2.6.12/drivers/input/mouse/synaptics.c 2005-06-18 18:10:22.000000000 +0900
@@ -608,6 +608,13 @@ static struct dmi_system_id toshiba_dmi_
DMI_MATCH(DMI_PRODUCT_NAME , "Satellite"),
},
},
+ {
+ .ident = "Toshiba Dynabook",
+ .matches = {
+ DMI_MATCH(DMI_SYS_VENDOR, "TOSHIBA"),
+ DMI_MATCH(DMI_PRODUCT_NAME , "dynabook"),
+ },
+ },
{ }
};
#endif
@@ -656,7 +663,7 @@ int synaptics_init(struct psmouse *psmou
* thye same as rate of standard PS/2 mouse.
*/
if (psmouse->rate >= 80 && dmi_check_system(toshiba_dmi_table)) {
- printk(KERN_INFO "synaptics: Toshiba Satellite detected, limiting rate to 40pps.\n");
+ printk(KERN_INFO "synaptics: Toshiba Satellite or Dynabook detected, limiting rate to 40pps.\n");
psmouse->rate = 40;
}
#endif
Sat, 18 Jun 2005 18:34:04 +0900| Permalink
It seems that somewhere between 0.10 and 0.11.2 there format that git uses to report diffs between trees changed. This broke okiniri (my fun scripts written about git/cognito), so I fixed it. I also added the poorly named okiniri-linearise-head to make it easier to break up the stable 2.6.11.X patches into something like this.
Thu, 09 Jun 2005 19:38:59 +0900| Permalink
I have recently discovered that using a 0.3mm mechanical pencil with B leads as a writing instrument is roughly eqivalent to writing with a slightly dulled knife. With the emphasis on slightly.
And in other etch news, I wonder how much fun Japanese people will have with that release of Debian (pronounced echi) when combined with the concept of ecchi (pronounced ecchi, comes from the h in hentai, means about the same thing).
Mon, 30 May 2005 22:58:55 +0900| Permalink
I've decided to establish Planet Horms using the fine Planet feed aggregator. I hope that the mish-mash of content accurately reflects my life.
[not suitable for children... or anyone else for that matter]
Tue, 17 May 2005 16:45:05 +0900| Permalink