Archive for June, 2009

Validating Scripts after a fresh installation

Saturday, June 20th, 2009

On #gobolinux last night an issue came up that has appeared occasionally before – the signature format for packages has changed between the 014.01 release and now, so the Scripts package won’t validate when you upgrade. The change was necessary to make for security, but it creates a problem for new installations trying to bring their tools up-to-date.

For those that don’t care for the fuss, and are willing to take the (very small) risk, you can just run SymlinkProgram Scripts 2.10.1 to activate the new package after installation fails with the validation error. Alternatively, you can upgrade piecewise through intermediate releases to get the fullest possible validation. The first stop should be 2.9.1, which will allow validating the latest package but does not include the version-validating code introduced for another problem. That will also be good enough for most people – the chance of a compromised mirror is pretty slight.

However for complete security you would need to validate the entire package as installed using a trusted signature from the CD release. Here’s a quick script to do that – it builds a verifiable checksum and validates it against the live system. The core code is embedded in a signed block using my key, with a small piece of code outside to verify the signature and execute the code automatically (not necessary, but the GPG command line to use the GoboLinux keyring is pretty long, so this makes it easier – you can copy the internal block out and validate manually if you prefer).

HopValidate will validate the Scripts 2.10.1 package using a known good checksum, preserving the chain of trust all the way. For full security, you should review the unsigned portion of the code, which uses the system keyring to validate the rest.

The manual command to decrypt is gpg –decrypt –no-default-keyring –keyring=/Programs/Scripts/Current/Data/gpg/goboring.gpg HopValidate, or –verify to verify the signature only. If you’re using this on Rootless, you’ll need to adjust the wrapper script and the command to use your local path to the Scripts directory – there’s no autodetection to keep the code easily reviewable.

[ed. 2010-01-31: Updated for 2.10.1]

Facebook usernames arrive

Saturday, June 13th, 2009

And while I’m not sure about that, I figured I should lock it down while I can, so here it is.

s3w 0.6.0

Friday, June 12th, 2009

It’s time for another release. s3w is a client to access and synchronise with Amazon Web Services’ Simple Storage Service (Amazon S3). It supports both direct access to S3 operations (GET, PUT, LIST, COPY, …) and higher-level functionality like bucket-to-bucket copy and pushing local directory structures into buckets.

s3w 0.6.0 is now out. As well as the standard array of bug fixes, the major new feature in this release is “stacked push”. In a stacked push, another bucket (or prefix within a bucket) is used as a base, and any files that match the corresponding object in the stacked-upon location won’t be pushed again. Incremental backups are one use for that; I also use it as a quick way to publish slightly different versions of files. Multiple stacked locations can be used, but each new location in the stack adds another network round-trip to slow the process down, so you probably want to keep the number low. They’re searched left-to-right as on the command line, push –stacked base1 –stacked base2 … src/ dest/.

Pull has a corresponding stacked mode for reconstructing the directory tree, which reads in the same order. There’s also new copybucket –move support, which deletes the source keys after they’re copied, and which can be paired with the new copybucket –save-date, which preserves the original Last-Modified date in a special piece of metadata which push will use (incremental backups, again). Etags (file checksums) can be calculated to save redundant pushes as well, which is useful if you have severe clock skew or unreliable modification times.

There are also scattered bug fixes to problems that emerged during the development cycle, and some extra niceties like offering suggestions to mistyped commands or bucket names. The configuration file can customise more behaviour – per-bucket exclusions and custom short names for buckets. See s3w config –help for details of how to set these values – push.bucket.<dom>.excludes (list) and shortname.<shortname> (string bucket name).

s3w depends on Python 2.6 and Boto. There are build and quickstart instructions in the readme. Running just s3w with no arguments will give a list of commands and a brief summary, while comprehensive documentation for each is available with s3w <commandname> –help.

Development takes place in a Bazaar branch, currently hosted on Launchpad (not necessarily going to stay there). You can access it using bzr branch lp:s3w. Patches welcome! There are some usage examples on the homepage, as well as in the internal documentation.

Reviews of books that should never have been written I: Ice Station

Wednesday, June 3rd, 2009

Matthew Reilly’s Ice Station is somewhat in the Crichton or Clancy vein, of which, well, you know. It has the cringing down pat. Nevertheless, can’t blame it for what it is, I knew that going in and I knew I was going to regret it, but the real problem: Reilly appears to have reached adulthood unaware that whales and dolphins have blowholes through which to breathe. I am unable to fathom this.