Linux Users of Victoria, Inc.

Made this vid this morning on my mobile phone. You may need to turn your sound up quite high to hear it.

So much like last year, the Linux Australia face to face meeting has somewhat spoiled my WoBloMo posting frequency. Though, technically it’s still the 13th in UTC, New York and Hawaii, so there’s that.

Anyway, I’ve bitten the bullet and signed up for the Upstarta Meetup. I’ve been in two minds about Upstarta for a while — on the plus side, cool local people chatting about startups; but on the minus side, I disagree with some of the Upstarta Principles. But I’ve decided I like them in practice enough not to care.

But like that’s going to stop me disagreeing about them on my blog! The main one I don’t accept philosophically (as opposed to in practice) is the first one:

Neither a borrower nor a lender be: no credit or external funding.

In practice, or on a day to day basis, I think it’s a good idea — Paul Graham’s essays on ramen profitability or the challenges of fundraising argue for a similar approach for startups specifically, and in the broader sense of things you don’t have to look far for negative consequences of either borrowing or lending more than (it later turns out) was affordable.

But on the other hand there are a bunch of times when I think borrowing and lending is useful.

I wouldn’t have had the opportunity to go to university straight out of high school without borrowing money in some form or another — as it happened it was paid via HECS which meant paying a fairly small portion of my course’s cost to the government explicitly either upfront or through my taxes at a low compounding interest rate, and having the rest of it paid by taxpayers at the government’s discretion. The new HELP system does the same. If I’d had to earn enough cash to pay for that education in full in advance on my own — no loans from family even — I can’t see how I would have had the opportunity to learn the same stuff, which would have resulted in not knowing how to make Debian’s “testing” actually happen.

In theory at least, I’m also something of a fan of self-funded pensions — that is, investing some of your income over the course of your life, so that you can eventually live off the proceeds without having to work. Personally, I’d love to be able to put a million dollars or so in the bank at 5% or more and get fifty grand or so every year before I even have to lift a finger — but the only way that works is if that million dollars is being lent to some borrower, who’s making enough use of that million dollars that they can afford to give me fifty grand just for the privilege.

And it seems like in at least some cases investment of cash — borrowing — is a valuable contribution: per this criticism of StackOverflow’s VC hunt, giving a bunch of Starbucks franchisees money to open stores to follow a proven business model seems to have worked out pretty well for pretty much everyone involved.

But there’s a lot of space between all of those and giving people millions of dollars to spend on foosball tables and beanbags because they’ve had an idea for a webapp; and ultimately it’s not entirely worth making sure there’s a clear distinction between things that are “good advice in the here and now” and “fundamental principles to be adhered to forever” when the aim of the moment is to make an interesting business from nothing.

So much like last year, the Linux Australia face to face meeting has somewhat spoiled my WoBloMo posting frequency. Though, technically it’s still the 13th in UTC, New York and Hawaii, so there’s that.

Anyway, I’ve bitten the bullet and signed up for the Upstarta Meetup. I’ve been in two minds about Upstarta for a while — on the plus side, cool local people chatting about startups; but on the minus side, I disagree with some of the Upstarta Principles. But I’ve decided I like them in practice enough not to care.

But like that’s going to stop me disagreeing about them on my blog! The main one I don’t accept philosophically (as opposed to in practice) is the first one:

Neither a borrower nor a lender be: no credit or external funding.

In practice, or on a day to day basis, I think it’s a good idea — Paul Graham’s essays on ramen profitability or the challenges of fundraising argue for a similar approach for startups specifically, and in the broader sense of things you don’t have to look far for negative consequences of either borrowing or lending more than (it later turns out) was affordable.

But on the other hand there are a bunch of times when I think borrowing and lending is useful.

I wouldn’t have had the opportunity to go to university straight out of high school without borrowing money in some form or another — as it happened it was paid via HECS which meant paying a fairly small portion of my course’s cost to the government explicitly either upfront or through my taxes at a low compounding interest rate, and having the rest of it paid by taxpayers at the government’s discretion. The new HELP system does the same. If I’d had to earn enough cash to pay for that education in full in advance on my own — no loans from family even — I can’t see how I would have had the opportunity to learn the same stuff, which would have resulted in not knowing how to make Debian’s “testing” actually happen.

In theory at least, I’m also something of a fan of self-funded pensions — that is, investing some of your income over the course of your life, so that you can eventually live off the proceeds without having to work. Personally, I’d love to be able to put a million dollars or so in the bank at 5% or more and get fifty grand or so every year before I even have to lift a finger — but the only way that works is if that million dollars is being lent to some borrower, who’s making enough use of that million dollars that they can afford to give me fifty grand just for the privilege.

And it seems like in at least some cases investment of cash — borrowing — is a valuable contribution: per this criticism of StackOverflow’s VC hunt, giving a bunch of Starbucks franchisees money to open stores to follow a proven business model seems to have worked out pretty well for pretty much everyone involved.

But there’s a lot of space between all of those and giving people millions of dollars to spend on foosball tables and beanbags because they’ve had an idea for a webapp; and ultimately it’s not entirely worth making sure there’s a clear distinction between things that are “good advice in the here and now” and “fundamental principles to be adhered to forever” when the aim of the moment is to make an interesting business from nothing.

I use Apache Solr for search on several projects, including a few using Drupal. Solr has built in support for replication and load balancing, unfortunately the load balancing is done on the client side and works best when using a persistent connection, which doesn't make a lot of sense for php based webapps. In the case of Drupal, there has been a long discussion on a patch in the issue queue to enable Solr's native load balancing, but things seem to have stalled.

In one instance I have Solr replicating from the master to a slave, with the plan to add additional slaves if the load justifies it. In order to get Drupal to write to the master and read from either node I needed a proxy or load balancer. In my case the best lightweight http load balancer that would easily run on the web heads was haproxy. I could have run varnish in front of solr and had it do the load balancing but that seemed like overkill at this stage.

Now when an update request hits haproxy it directs it to the master, but for reads it balances the requests between the 2 nodes. To get this setup running on ubuntu 9.10 with haproxy 1.3.18, I used the following /etc/haproxy/haproxy.cfg on each of the web heads:

global log 127.0.0.1 local0 log 127.0.0.1 local1 notice maxconn 4096 nbproc 4 user haproxy group haproxy daemon defaults log global mode http option httplog option dontlognull retries 3 maxconn 2000 balance roundrobin stats enable stats uri /haproxy?stats frontend solr_lb bind localhost:8080 acl master_methods method POST DELETE PUT use_backend master_backend if master_methods default_backend read_backends backend master_backend server solr-a 192.168.201.161:8080 weight 1 maxconn 512 check backend slave_backend server solr-b 192.168.201.162:8080 weight 1 maxconn 512 check backend read_backends server solr-a 192.168.201.161:8080 weight 1 maxconn 512 check server solr-b 192.168.201.162:8080 weight 1 maxconn 512 check

To ensure the configuration is working properly run "wget http://localhost:8080/solr -O -" on each of the web heads. If you get a connection refused message haproxy may not be running. If you get a 503 error make sure solr/jetty/tomcat is running on the solr nodes. If you get some html output which mentions Solr, then it should be working properly.

For Drupal's apachesolr module to use this configuration, simply set the hostname to localhost and the port to 8080 in the module configuration page. Rebuild your search index and you should be right to go.

If you had a lot of index updates then you could consider making the master write only and having 2 read only slaves, just change the IP addresses to point to the right hosts.

For more information on Solr replication refer to the Solr wiki, for more information on configuring haproxy refer to the manual. Thanks to Joe William and his blog post on load balancing couchdb using haproxy which helped me get the configuration I needed after I decided what I wanted.

On Thursday February 25th we held the first meeting for 2010 of the Sydney Functional Programming group. The meeting was held at Google's Sydney offices and we had 17 people show up to hear our two presenters.

First up we had your correspondent (Erik de Castro Lopo), giving a presentation titled "Hacking DDC" on my bug fixing work on Ben Lippmeier's DDC compiler. I explained a little about what DDC and Disciple were; a Haskell like language with some interesting extensions to the type system. I then suggested that anyone curious as to why these extensions were interesting should read the first chapter of Ben's PhD thesis "Type Inference and Optimisation for an Impure World". I then went on how using Darcs for the revision control made it easy to use one branch per bug or feature I'm working on, specifically, it allowed me to work on one until I got stuck and then move on to another without the debugging of the first interfering with the second.

Our second presenter for the evening was Tim Docker who gave us an explanation of a Domain Specific Language (DSL) for handling dates in financial systems. Code written in his DSL looked a lot like Ocaml, but the implementation was in C++.

A big thanks to Tim for presenting and Google for providing the meeting venue and the snacks.

Brendan Scott linked to a couple of articles about CAL (the Copyright Agency Limited) [1]. I have previously written about CAL and the way that they charge organisations for the work of others without their consent [2]. My personal dispute with CAL is that they may be charging people to use my work, I have not given them permission to act on my behalf and will never do so. If they ever bill anyone for my work then it will be an act of piracy. The fact that the government through some bad legislation permitted them to do such things doesn’t prevent it from being piracy – you can’t disagree with this claim without supporting the past actions of China and other countries that have refrained from preventing factories from mass-producing unauthorised copies of software.

The first article concerns the fact that last year CAL paid more than $9,400,000 in salary to it’s employees (including $350,000 to it’s CEO) while it only paid $9,100,000 directly to the authors [3]. It also spent another $300,000 to send it’s executives to a junket in Barbados. It did give $76,000,000 to publishers “on the assumption that a proportion of this money will be returned to authors” – of course said publishers could have used the money to have holidays in Barbados. CAL doesn’t bother to check who ends up with shares of the $76,000,000 so it’s anyone’s guess where it ends up.

The second article is by James Bradley who is an author and director of CAL [4]. He claims that “much” of the $76,000,000 was distributed to authors, although I’m not sure how he would have any idea of how much it was – which is presumably why he used the word “much” instead of some other word with a clearer meaning such as “most“. He also notes that CAL invested $1,000,000 in “projects specifically designed to promote the development and dissemination of Australian writing“, which sounds nice until you consider the fact that none of the authors (apart from presumably the few who sit on the CAL board) had any say in the matter. Can I take a chunk of the $9,400,000 that is paid to CAL employees and invest it in something? If not then why not? If they can “invest” money that was owed to other people then why can’t I invest their salaries?

James also says “The issue of how well CAL serves rights-holders – and authors and artists in particular – is a vital one” which is remarkably silly. He is entirely ignoring the fact that some rights holders don’t want to be “served” by CAL at all. The fact that CAL can arbitrarily take money for other people’s work is an infringement on their rights. He further demonstrates his ignorance by saying “Without CAL and the licences we administer, users – educational institutions, government agencies and corporate organisations, to name just a few – would be required to seek permission every time they reproduced copyright material or run the risk of legal action for copyright infringement” – of course any educational institution can use Creative Commons licensed work [5].

I’ve previously written about the CK12 project to develop CC licensed text books for free use [6]. There’s no reason why the same thing can’t be done for university text books. In the discussion following Claudine Chionh’s LUV talk titled “Humanities computing, Drupal and What I did on my holidays” [7] it was suggested that it should be possible to gain credit towards a post-graduate degree based on work done to share information – this could mean setting up a Drupal site and populating the database or it could mean contributing to CC licensed text books. Let’s face it, a good CC text book will be read by many more people than the typical conference proceedings!

James says that CAL is used “Instead of having to track down individual rights-holders every time they want to reproduce copyright material“. The correct solution to this problem would be to change the copyright law such that if a reasonable attempt to discover the rights-holder fails then work is deemed to be in the public domain. The solution to the problem of tracking down rights-holders is not to deny them their rights entirely and grant CAL the right to sub-license their work!

He also makes the ridiculous claim “Whereas in the age of the physical book schools and universities could have bought fewer books and made up the difference by using photocopies, it is now possible for an organisation to buy a single set of digital materials and reproduce them ad infinitum” which implies that CAL is the only thing saving the profits of authors from unrestricted digital copying. Of course as CAL seems to have no active enforcement mechanisms and they apparently charge a per-student fee they really have no impact on the issue of a single licensed copy being potentially used a million times – extra use apparently won’t provide benefits to the author and use in excess of the licensing scheme won’t be penalised.

He asks the rhetorical question “After all, why go to the expense of creating a textbook (or some form of digital course materials) if you are going to sell only a half-dozen copies to state education departments“. The answer is obvious to anyone who has real-world experience with multiple licensing schemes – you can sell one single copy and make a profit if the price is high enough. The smart thing for the education departments to do would be to pool their resources and pay text book companies for writing CC licensed texts (or releasing previously published texts under the CC). The average author of a text book would probably be very happy to earn $100,000 for their work, the editorial process probably involves a similar amount of work. So if the government was to offer $300,000 for the entire rights to a text book then I’m sure that there would be more than a few publishers tendering for the contract.

According to the CIA World Fact Book there are 2,871,482 people in Australia aged 0-14 [8], that means about 205,000 per year level. CAL charges $16 for each primary and secondary student so the government is paying about $3,280,000 every year per year level. Even in year 12 the number of text books used is probably not more than 10, so it seems to me that if all the money paid to CAL by schools in a single year was instead used to fund Creative Commons licensed text books then the majority of the school system would be covered! The universities have a much wider range of text books but they also have higher CAL fees of $40 per student. After cutting off the waste of taxpayer money on CAL fees for schools that money could be invested in the production of CC licensed university text books.

• [1] http://brendanscott.wordpress.com/2010/03/10/james-bradley-responds-on-slattery-cal-article/
• [2] http://etbe.coker.com.au/2009/09/07/bad-but-appealing-copyright-arguments/
• [3] http://www.theaustralian.com.au/news/arts/copyright-staff-get-more-than-they-give-to-authors-and-artists/story-e6frg8n6-1225831556653
• [4] http://www.theaustralian.com.au/higher-education/opinion-analysis/licensing-body-copyright-agency-limited-has-lasting-value/story-e6frgcko-1225838858623
• [5] http://creativecommons.org/
• [6] http://etbe.coker.com.au/2009/08/29/free-k-12-text-books/
• [7] http://luv.asn.au/2010/03/02
• [8] https://www.cia.gov/library/publications/the-world-factbook/geos/as.html

The Patent War of All Against All

“Hereby it is manifest that during the time men live without a common power to keep them all in awe, they are in that condition which is called war; and such a war as is of every man against every man. For war consisteth not in battle only, or the act of fighting, but in a tract of time, wherein the will to contend by battle is sufficiently known: and therefore the notion of time is to be considered in the nature of war, as it is in the nature of weather. For as the nature of foul weather lieth not in a shower or two of rain, but in an inclination thereto of many days together: so the nature of war consisteth not in actual fighting, but in the known disposition thereto during all the time there is no assurance to the contrary. All other time is PEACE.”

Hobbes, Leviathan, Chapter XIII, paragraph 8

Glyn Moody has written a post about a system called U-Prove.  Glyn notes that the software is being licensed under a BSD licence and notes that is a good thing, but then observes that there is a patent encumbrance on the code, and indicates this is a bad thing.    In a comment, Sam Ramji refers off to this article of mine, kindly remarking it is a ‘good post’.

The essence of Glyn’s argument is that the OSP does not preserve the freedom of free software, so it is not sufficient.  Sam appears to be adopting my comments to the effect that it’s hard to single out one company for the patent blame game.

In the context of free software patents are problematic.  In the ideal world patents on software wouldn’t exist and there wouldn’t be a problem.  However, they do exist.  Moreover, part of the reason they exist is because of a variation of mutually assured destruction – many businesses believe they need to acquire patents in order to defend against other patents.*  Jonathan Schwartz sets out some of the sad, tawdry circumstances in which this logic plays out here.

I think it is a non trivial problem to find wording which preserves just the defensive potential of patents (which, is actually their offensive potential limited to specific circumstances of exercise)  while preserving freedom when licensing software.  Some of the more detailed free software licences attempt this.   It is, I think, a more difficult problem to craft such wording to apply to standards – because standards purport to be agreed by some collection of people, while freedom requires that everyone be permitted to pursue their own goals.   Thus, any ‘promise’ or ‘covenant’** which is limited to an agreed specification must necessarily be inconsistent with freedom in a way qualitatively different to a patent clause in an open source licence.   Moreover, any wording which applies to a particular version of a specification will be inconsistent with the evolution of that specification.  In short, promises made in relation to specifications are likely to always be problematic (the best to hope for is a disclaimer  – per W3C).

Therefore, if the words themselves are likely inadequate, the issue of who is saying the words, and what one can reasonably read into them becomes much more important.  In the OSP post that Sam refers to I explicitly reference  another, earlier, article on the OSP which calls this issue out.  With this in mind, I note that Microsoft has recently chosen to specifically draw out the Linux/open source angle in its cross licensing deals with Amazon[3] and IO Data[4].  It didn’t need to, but presumably chose to.  It seems reasonable to conclude this is signaling.  These are grounds that would justify a reasonable person finding the OSP inadequate.

If Microsoft wants people like Glyn to trust them, they should perhaps incline against, rather than toward making the will to contend by battle [...] sufficiently known.

Notes:

* This, by the way, is much the same argument used by Hobbes in favour of the need for a common power to counter the war of all against all (hence the quote above).  In this case however the common power (parliament) rather than saving us from a state of war of all against all has plunged some of us into a version of it.

** (I distrust these words – if it is a licence why not call it that, but perhaps they are US terms of art?)

[3] ‘…said Horacio Gutierrez, corporate vice president and deputy general counsel, in the news release. “Microsoft’s patent portfolio is the largest and strongest in the software industry, and this agreement demonstrates … our ability to reach pragmatic solutions to IP issues regardless of whether proprietary or open source software is involved.”  The press release is here, it doesn’t fall over itself as much as the IO Data one below.

[4] ‘...said David Kaefer, Microsoft general manager of intellectual property, in a statement announcing the latest deal. “Microsoft has a strong track record of collaboration with companies running Linux-based offerings, and this agreement is a reflection of our commitment to partner with industry leaders around the world.” ‘ In fact, on Microsoft’s press release page (at 12 March 2010) the call out comprises of the words:  “I-O Data customers will receive patent covenants for their use of devices running Linux and related open source software.“  The press release is at pains to refer to it.

Assemblyman seeking to ban all salt in restaurant cooking

My relationship with Puppet is one of love and hate. I am forced to use it simply because there is no better tool around, but I hate it in so many ways that I don’t even want to start to enumerate (hint: most have to do with Ruby, actually).

Today I decided to put an end to one thing that has been driving me insane: the fact that puppetd (the client) and puppetmasterd (the server) use the same working directory, /var/lib/puppet. Since I consider and would like to treatthe machine on which puppetmasterd is running just another puppet client, I was running into funky issues related to SSL certificate confusion, obscure errors, and SSL revocation horrors.

The following hence assumes that you have installed or are planning to install puppetd on the machine running your puppetmaster, and that you have two fully-qualified domain names for the machine. For instance, I run puppetmaster on vera.madduck.net, and puppetmaster.madduck.net is an alias for the same machine. I’ll use these names in the following as examples.

The following may be Debian-specific, as I am solely using the puppet and puppetmaster packages for my experimentation and verification. Your mileage may vary, but the concept shall be the same.

1. Stop everything:

   /etc/init.d/puppetmaster stop /etc/init.d/puppet stop

   (also verify that you have not instructed cron to restart these services)

2. Rename the working directory:

   mv /var/lib/puppet /var/lib/puppetmaster

   and amend /etc/puppet/puppet.conf accordingly:

   [main] # … vardir=/var/lib/puppetmaster ssldir=$vardir/ssl # … [puppetmasterd] certname=puppetmaster.madduck.net # …

   I am doing this in [main], planning to override it for puppetd later, because puppetd is the only program which makes sense to be separated from the rest. Since only the puppetmaster needs a special certificate name, that is set specifically in the [puppetmasterd] section.

   If you use apache2 or nginx in front of your puppetmasters, make sure to amend the SSL file locations in the virtual host definition and restart (!) the service.

   You can verify that the configuration has been amended by making sure that there is no output from the following command:

   # puppetmasterd --genconfig | grep -q '/var/lib/puppet/' && echo SOMETHING IS WRONG

3. Now restart puppetmaster:

   /etc/init.d/puppetmaster start

   and verify that it starts.

   If your puppetmaster previously ran under a different name, it will create itself a new certificate and sign it.

   Since the client will get its own working directory (and thus a new SSL certificate), you want to remove all records of the old certificate:

   # puppetca --list --all + puppetmaster.madduck.net + vera.madduck.net # puppetca --clean vera.madduck.net

4. Change the configuration file to tell puppetd about its working directory:

   [puppetd] server=puppetmaster.madduck.net vardir=/var/lib/puppetmaster ssldir=$vardir/ssl # …

   This you can verify with the following command, which should not print anything:

   # puppetd --genconfig | grep -q '/var/lib/puppet[^/]' && echo SOMETHING IS WRONG

5. Now install puppet, or (re)start it if it’s already installed:

   # /etc/init.d/puppet stop # puppetd --no-daemonize --onetime --verbose --waitforcert 30 & info: Creating a new SSL key for vera.madduck.net warning: peer certificate won't be verified in this SSL session info: Caching certificate for ca info: Creating a new SSL certificate request for vera.madduck.net # puppetca --list vera.madduck.net # puppetca --sign vera.madduck.net notice: Signed certificate request for vera.madduck.net notice: Removing file Puppet::SSL::CertificateRequest vera.madduck.net at '/var/lib/puppetmaster/ssl/ca/requests/vera.madduck.net.pem' # fg info: Caching certificate for vera.madduck.net info: Caching certificate_revocation_list for ca […] # puppetca --list --all + puppetmaster.madduck.net + vera.madduck.net # /etc/init.d/puppet start

   Do yourself the favour and check that it’s all working.

6. Optionally, you can now clean up the client stuff in the server’s working directory, for instance like this (it worked for me, but this is the sledgehammer approach):

   # /etc/init.d/puppetmaster stop # cd /var/lib/puppetmaster # tar -cf /tmp/puppetmaster.workingdir-backup.tar . # find ../puppet -type f -printf '%P\n' | xargs rm # /etc/init.d/puppetmaster start

7. If you stopped cron before (and your puppet recipes have not since restarted it):

   /etc/init.d/cron start

All done. I wish puppet, or at least Debian’s puppet packages would do this by default. Please let me know if the above conversion works for you. Then I might start working on an automated migration.

NP: Genesis: Selling England by the Pound

The DDL code paths for Drizzle are increasingly different from MySQL. For example, the embedded_innodb StorageEngine CREATE TABLE code path is completely different than what it would have to be for MySQL. This is because of a number of reasons, the primary one being that Drizzle uses a protobuf message to describe the table format instead of several data structures and a FRM file.

We are pretty close to having the table protobuf message format being final (there’s a few bits left to clean up, but expect them done Real Soon Now (TM)). You can see the definition (which is pretty simple to follow) in drizzled/message/table.proto. Also check out my series of blog posts on the table message (more posts coming, I promise!).

Drizzle allows either your StorageEngine or the Drizzle kernel to take care of storage of table metadata. You tell the Drizzle kernel that your engine will take care of metadata itself by specifying HTON_HAS_DATA_DICTIONARY to the StorageEngine constructor. If you don’t specify HTON_HAS_DATA_DICTIONARY, the Drizzle kernel stores the serialized Table protobuf message in a “table_name.dfe” file in a directory named after the database. If you have specified that you have a data dictionary, you’ll also have to implement some other methods in your StorageEngine. We’ll cover these in a later post.

If you ever dealt with creating a table in MySQL, you may recognize this method:

virtual int create(const char *name, TABLE *form, HA_CREATE_INFO *info)=0;

This is not how we do things in Drizzle. We now have this function in StorageEngine that you have to implement:

int doCreateTable(Session* session, const char *path,                   Table& table_obj,                  drizzled::message::Table& table_message)

The existence of the Table parameter is largely historic and at some point will go away. In the Embedded InnoDB engine, we don’t use the Table parameter at all. Shortly we’ll also get rid of the path parameter, instead having the table schema in the Table message and helper functions to construct path names.

Methods name “doFoo” (such as doCreateTable) mean that there is a method named foo() (such as createTable()) in the base class. It does some base work (such as making sure the table_message is filled out and handling any errors) while the “real” work is done by your StorageEngine in the doCreateTable() method.

The Embedded InnoDB engine goes through the table message and constructs a data structure for the Embedded InnoDB library to create a table. The ARCHIVE storage engine is much simpler, and it pretty much just creates the header of the ARZ file, mostly ignoring the format of the table. The best bet is to look at the code from one of these engines, depending on what type of engine you’re working on. This code, along with the table message definition should be more than enough

Typical beach activity, Santa Cruz

Originally uploaded by Nooks

I would have liked to catch the lighthouse to the left of the frame in this shot but it didn't happen and I live the wave activity too much to use another image instead.

Ooops. Emergency woblomo post coz I forgot. Here’s the link to the other day’s screencast that apparently didn’t make it through aggregators. Hohum.

Ooops. Emergency woblomo post coz I forgot. Here’s the link to the other day’s screencast that apparently didn’t make it through aggregators. Hohum.

I’ve been an advocate for free software ever since I started using it in 1999 (or was it 1998?). Some who know me would probably say that I’m an anti-Microsoft, pro Free software zealot. Maybe that’s true.

The way I see it, there are three camps. Those who think that open source software is good and so choose to use it; those who think that all software should be free (libre) and that proprietary software is evil; and, those who hate Microsoft and will use anything but.

I want to be in the first camp.

However, it’s true that I also want to convert people to free software. I do. Sorry about that. I want to get people off Windows and Mac and whatever else they are running and onto a free system like Linux. I want others to join me, because I think it’s the better way. Who am I though, to suggest what’s best for someone else?

I don’t like the fact that our Government is a Microsoft shop and spends half a billion dollars on proprietary software licenses a year. Likewise the fact that they create all their data in closed proprietary data formats which I have to try and make available years down the road (as a part of my day job). I wish that people could use an operating system of their choice at work, instead of being forced to use Windows.

Most importantly, I wish that people had choice when it comes to buying a new machine and generally I wish that Microsoft didn’t have their monopoly (can anyone argue that it’s actually been good for the industry as a whole? I think it’s a stretch to say that we wouldn’t have cheap hardware without Microsoft).

Recently, Internode (my much beloved ISP) dumped free software and instead rolled out Microsoft Exchange for mail, calendaring, etc. CEO Simon Hackett outlined his reasons for the migration and it’s a compelling argument.

We’ve been trying to get a working, open source based, cross platform calendaring system happening at Internode for years and years.

And we’ve never managed it.

There’s always something that doesn’t work right, or a need to run a plethora of ‘conduit’ tools between multiple calendar systems. Somehow, no matter how hard you try, that seems to turn into a dogs breakfast, and involves some horrid compromises on at least one of your operating systems in terms of user access to the resulting information.

OK, so that does read a lot like a Microsoft PR statement (I wonder what discount they received for printing that), but if Internode can’t get it right what hope does the rest of the world have?

I remember a prominent free software developer (someone whom I respect greatly) saying to me something along the lines of (and I’m paraphrasing here):

Just because something is open source, doesn’t make it better. Use the best tool for the job, even if it’s proprietary and closed source.

Is that true? If so, then why do I persist with various free software projects which are inferior to their proprietary counterparts?

I guess that it really comes down to your definition of “best” tool for the job. If your set of criteria for best tool does not exclude costly licensing fees, then a closed source solution be the best for the job. On the other hand, if access to the code is important to you, then it doesn’t matter how “great” a closed tool is, it’s never going to be the best tool for the job. So, this is why for some people, Apple iPod might be the best tool for the job, but for others it doesn’t even rank (like me, who has all their music in FLAC).

So you see for Internode, the best tool for the job was one that offered seamless syncing with their iPhones.

C’est la vie.

In writing this I am reminded of a friend who would laugh at me while I spent half an hour on the terminal trying to connect to a WPA encrypted wireless network. His Mac would just work straight away (like what we have now with NetworkManager). For me, a closed operating system wasn’t the best tool for the job, so I persisted with countless hours of command line work to associate with an access point.

I know that this boils down to a world view and I don’t consider myself a free software Zionist, but maybe I have been – or maybe I am. If so, maybe I shouldn’t be.

It’s too tiring to keep on fighting for the adoption of free software and maybe we just shouldn’t be in the first place. I’m reminded of that quote from Linus:

The thing is, at least to me personally, Microsoft just isn’t relevant to what I do. That might sound strange, since they are clearly the dominant player in the market that Linux is in, but the thing is: I’m not in the ”market.” I’m interested in Linux because of the technology, and Linux wasn’t started as any kind of rebellion against the ”evil Microsoft empire.” Quite the reverse, in fact: from a technology angle, Microsoft really has been one of the least interesting companies. So I’ve never seen it as a ”Linus versus Bill” thing. I just can’t see myself in the position of the nemesis, since I just don’t care enough. To be a nemesis, you have to actively try to destroy something, don’t you? Really, I’m not out to destroy Microsoft. That will just be a completely unintentional side effect.

I like that. He’s just getting on with the job of making great open source software because he believes it’s a better way. He doesn’t care what the rest of the world does (at least, that’s what I read from it). That’s what I want. I just don’t want to care any more.

So what’s the point of this post? Well I feel that more and more my own attitude is changing from one of championing free software regardless of circumstance, to a more apathetic position something along the lines of, “I use Linux, you use whatever you want. If you want Microsoft Windows and Office, then go use that.”

If you think that .NET is the future of the Linux desktop, then go right ahead and use it.

Wanna store all your data in proprietary formats? Feel free.

My brother loves Linux and free software now (it’s taken about 10 years of constant barraging to get him to use it) but his wife simply “must” use Outlook. So he still runs Outlook for all their mail. Weird, but OK.

From now on, I won’t push Linux and free software (I’ll probably re-do MakeTheMove to reflect this, someday). This has all been the result of a long time frustration bubbling under the surface, but more recently the culmination of hundreds of events, conversations and epiphanies I’ve had and experienced over the last 6 months or so. I’m done. I’ll simply use what I want to use and leave everyone else to their own devices. If they want to also use free software, as always I’ll be happy to help.

Free Software is Principled

I recall, several years back now, being in some sort of forum somewhere arguing over the implementation of anti-circumvention legislation in Australia.  I recall Rusty Russell talking about ghostscript’s [?] handling of pdf documents at the time and how it respected restrictions settings in the pdf documents.  That is, despite being able to ignore them, ghostscript’s authors decided to respect them.   In practice that would mean that most ghostscript users would also respect those settings.

By way of contrast, today, looking for information about pdf to text conversion tools I came across closed source software whose primary purpose is apparently to remove restrictions from pdf files.  In my experience free software is typically more principled than its closed source counterparts – perhaps stupidly so.

In my day job I do Linux embedded work and as people in the embedded world know, Linux is a pretty commonly used embedded OS. Today I was evaluating a new board and found it had an Intel graphics chip that was not properly detected by Ubuntu 9.10. The ever trusty lspci said this:

00:02.0 VGA compatible controller: Intel Corporation System Controller Hub (SCH Poulsbo) Graphics Controller (rev 07)

We all know that Intel employs a bunch of well known Xorg developers, so this shouldn't be a problem, right?

Unfortunately, it is a problem. Intel's offering for this chipset is the Intel® Embedded Graphics Drivers web page where they offer a 124 megabyte download (registration required). After registration you get to choose which driver pack you want and which OS you are downloading it for. Ubuntu was not on the list and neither was Debian. I chose Fedora 10 (released in 2008) as that was the most recent one.

Now, you can image my surprise when the driver download for Fedora Linux contained just four files:

Archive: /tmp/IEGD_10_3_GOLD.zip testing: UsersGuide_10_3_1525.pdf OK testing: IEGD_10_3_GOLD_1525.exe OK testing: IEGD_SU_10_3_GOLD.pdf OK testing: RELNOTES_10_3_1525.txt OK No errors detected in compressed data of /tmp/IEGD_10_3_GOLD.zip.

Yep, thats right, the driver download for Fedora Linux contains two PDF files, a text file and an executable installer for Windows.

Being the curious (and paranoid) type I decided to explore this further, by running the installer under WINE in a chroot. After the installer you get left with several metric craploads of Java Jar files, and another windows executable iegd-ced.exe that supposedly configures this nightmare. I ran it (again, under WINE in a chroot) but it didn't seem to do anything sensible or worthwhile so I looked around amongst the other installed files and found IEGD_10_3_Linux.tgz.

Inside that tarball there are a bunch of Xorg library binaries (for several different versions of Xorg), a large chunk of source code that gets compiled into the Linux kernel and even better yet, a couple of Microsoft Visual Studio project files. WTF?

Unbe-fscking-lievable. Needless to say, I will avoid any hardware which uses this chipset and any other hardware that requires binary only kernel blobs packaged this badly. Doing so makes my life easier.

The people at Intel who thought this was a good idea must have their own personal mother-lode of stupid.

In my day job I do Linux embedded work and as people in the embedded world know, Linux is a pretty commonly used embedded OS. Today I was evaluating a new board and found it had an Intel graphics chip that was not properly detected by Ubuntu 9.10. The ever trusty lspci said this:

00:02.0 VGA compatible controller: Intel Corporation System Controller Hub (SCH Poulsbo) Graphics Controller (rev 07)

We all know that Intel employs a bunch of well known Xorg developers, so this shouldn't be a problem, right?

Unfortunately, it is a problem. Intel's offering for this chipset is the Intel® Embedded Graphics Drivers web page where they offer a 124 megabyte download (registration required). After registration you get to choose which driver pack you want and which OS you are downloading it for. Ubuntu was not on the list and neither was Debian. I chose Fedora 10 (released in 2008) as that was the most recent one.

Now, you can image my surprise when the driver download for Fedora Linux contained just four files:

Archive: /tmp/IEGD_10_3_GOLD.zip testing: UsersGuide_10_3_1525.pdf OK testing: IEGD_10_3_GOLD_1525.exe OK testing: IEGD_SU_10_3_GOLD.pdf OK testing: RELNOTES_10_3_1525.txt OK No errors detected in compressed data of /tmp/IEGD_10_3_GOLD.zip.

Yep, thats right, the driver download for Fedora Linux contains two PDF files, a text file and an executable installer for Windows.

Being the curious (and paranoid) type I decided to explore this further, by running the installer under WINE in a chroot. After the installer you get left with several metric craploads of Java Jar files, and another windows executable iegd-ced.exe that supposedly configures this nightmare. I ran it (again, under WINE in a chroot) but it didn't seem to do anything sensible or worthwhile so I looked around amongst the other installed files and found IEGD_10_3_Linux.tgz.

Inside that tarball there are a bunch of Xorg library binaries (for several different versions of Xorg), a large chunk of source code that gets compiled into the Linux kernel and even better yet, a couple of Microsoft Visual Studio project files. WTF?

Unbe-fscking-lievable. Needless to say, I will avoid any hardware which uses this chipset and any other hardware that requires binary only kernel blobs packaged this badly. Doing so makes my life easier.

The people at Intel who thought this was a good idea must have their own personal mother-lode of stupid.

A couple of months ago (December 1st for those playing along at home) it marked five years to the day that I started at MySQL AB (now Sun, now Oracle). A good part of me is really surprised it was for that long and other parts surprised it wasn’t longer. Through MySQL and Sun, I met some pretty amazing people, worked with some really smart ones and formed really solid and awesome friendships. Of course, not everything was perfect (sometimes not even close), but we did have some fun.

Up until November 2008 (that’s 3 years and 11 months for those playing at home) I worked on MySQL Cluster. Still love the product and love how much better we’re making Drizzle so it’ll be the best SQL interface to NDB :)

The ideas behind Drizzle had been talked about for a while… and with my experience with internals of the MySQL server, I thought that some change and dramatic improvement was sorely needed.

Then, in 2008, Brian created a tree. I was soon sending in patches at nights, we announced to the whole world at OSCON and it captured a lot of attention.

Since November 2008 I’ve been working on Drizzle full time. It was absolutely awesome that I had the opportunity to spend all my days hacking on Drizzle – both directly with fantastic people and for fantastic people.

But… the Sun set… which was exciting and sad at the same time.

Never to fear! There were plenty of places wanting Drizzle hackers (and MySQL hackers). For me, it came down to this: “real artists ship”. While there were other places where I would no doubt be happy and work on something really cool, the only way I could end up working out where I should really be was: what is the best way to have Drizzle make a stable release that we’d see be suitable for deployment? So, Where Am I Now?

Rackspace.

Where I’ll again be spending all my time hacking Drizzle.

Time to learn Javascript properly.

The Onion: Nation Shudders At Large Block Of Uninterrupted Text

‘…”Why won’t it just tell me what it’s about?” said Boston resident Charlyne Thomson, who was bombarded with the overwhelming mass of black text late Monday afternoon. “There are no bullet points, no highlighted parts. I’ve looked everywhere—there’s nothing here but words.”‘

Story here