Over the weekend, the hard drive in the server most of my life is on died.  Well, about 1% of the data on the system is readable.  Amazingly, it's pretty much the most important 1%.

I have somewhat up-to-date backups (I was once a middling-good sysadmin, after all).  Backups that I haven't restored from, mostly because I've come to realise that I just don't care about that content anymore.  It's rather freeing, actually.

The hosting company I lease the server from replaced the drive and reinstalled on that drive (Goodbye Debian stale, Hello Ubuntu Dapper Drake with LTS).

The things that I thought I'd miss that I've found I haven't is my email.  Oh, I'll grab a copy of my aliases file for the people I want to keep in contact with, but the exact emails aren't all that important.  And, anyway, I'd already moved my mail life over to Google Mail (I'll write something up about Google Apps For Your Domain someday).

I recovered my 4.0.24 MySQL tables for gibe (the blogging engine for TurboGears I'm working on and this runs on), and for engal (the photo gallery for TurboGears I wrote).  I installed 4.0.24 (not a simple task via package management with Dapper, so I resorted to a custom install), discovered only the "visit" tables were broken, exported, and installed on package-managed MySQL 5 (on my way to PostgreSQL, of course).

I recovered my SVN repository from the disk itself, which contains my web site and all the code for gibe and engal.  The static resource content (mostly images of Dante linked to from my posts) was also all fine.

/etc was toast (Input/Output error if you tried to change working directory to it), which meant my exim and name server configuration and zones weren't recoverable from the drive.  But since I'd moved to Google Mail, everything was simplified, and I added back the backup MXs I run for friends.  The name server configuration was basically duplicated on another server, and that was about five minutes of work.

The backups are still useful - they're the "archive" of stuff I once thought was interesting.  I'll eventually get it whittled down to what I care about, and get it down locally (which may not make sense if you don't realise how bad Internet connectivity in South Africa is), and archive it here.

And, of course, if I never had the backups, I would almost certainly have not been lucky enough to be able to recover without using them.

(And, of course, this post will almost certainly cause people to notice that everything isn't perfectly put back together - but then, I'm counting on that so that I can find out about it... )

Getting out of the never-ending mad dash that ruled my life for much of last year is starting to give me time to think about and hopefully learn from the events from that time.

Lately this thinking has revolved around how to do technology right, and how technology should be treated by business.  There's always lip service about what technology should be to a business, and how the engagements should work, and so forth.  But I think we so often find ourselves working from a flawed understanding of what success is to the business that is initiating the project.

A common position I end up in is taking over what pretty much anyone would call a failed project - a project where a lot of money has been spent and the resultant product is not even of sufficient quality to expose to others, let alone started to make money.  Aspects of the project that have already been paid for have been found not to be of sufficient quality (or even delivered at all), and need to have more money spent on them.

But I hack on the existing project for a few months to get it to the point where I wouldn't put a bag over my head to avoid being associated with it.  The project launches, and after a few rough patches and late nights, I get the last major kinks out, and the project starts making money.  Some might even call it a "success".

But is it?  If I'd been on the project from the beginning, my arrogance forces me to insist that the project would have been delivered properly the first time, with less back and forth to QA, with fewer publically-visible problems post-launch, and thus would be cheaper in terms of direct costs and indirect costs to regain the confidence lost during this whole process.  So, while the second part of the project (which some may call another project entirely) in itself is successful, the entire process of developing the technology was more expensive than it needed to be.

And that's not even the most damning difference.  If I'd been on the project from the beginning, my arrogance forces me to believe that the end result would be better in the way that most matters about technology in the long run - how it facilitates or hinders changes in future.  Or, put simply, a solution I'm involved in would give the company agility.

Many companies have a great idea, and get to market by the skin of the teeth of the people they could find and afford to do their initial technology.  And, because of the way the technology was rushed and was cobbled together, changing it is often scarily hard.  Which isn't a problem at first (when they have the existing developers who remember all the ins and outs of the system), and so the company's great idea and timing and efforts make them profitable and renowned.  Fast forward a few years, and they're cursing their main source of income and often the thing they're most famous for (if you're talking a web site, for example).

Why?

The developers are usually gone by now.  Even if they aren't, they no longer have the ability to keep in their heads all the hacks they've been forced to put in to the original system to keep it ticking over "until we rebuild it all".  There's entire pages of code dedicated to special cases for particular user names, groups, and so forth.  When changes do happen, all effort is put into reducing the intrusion on the original system, because the company has been bitten by the hard-to-detect consequences of previous changes.

Success or failure?

At the beginning, a great idea was all that was needed to get into the market, and delivery could be made timeously.  Now, there are more people available, and there's more money available.  Sounds good...  But, despite the additional people and money, even small ideas and small changes take a long time to make, and the next great idea is near impossible to implement.  These are the small and big ideas and changes necessary to continue being relevant - to not become an also-ran of the very area they once owned.

Unless your business usage of the technology is once-off - a gimmick web site that has a specified shelf life - then you have got to think about the cost of change built into that technology.  (By the way, this is also why open source and/or open standards are such a no-brainer in the long term - you never end up having your data and processes tied up in some proprietary system that makes the cost of change too high when you need to change something about your business.)

I don't claim to have the answer to how one can define success, but this is the simplest way to describe what I'm feeling now:

Success can't be measured only by what you have achieved - the resources, the accolades, and the good will.  More important than those, it should be measured also by what you can achieve from this point on.

(This also applies to any development project independently of the business that initiates it.  Taking the laziness as a virtue approach, you can measure your success by how much effort you save yourself by building something that's easy to maintain and extend in future.)

The prison outreach programme at Pollsmoor initiated by The Shuttleworth Foundation and now continued by the awesome Inkululeko Technologies is starting to produce tangible fruit, with 43 inmates to write their examinations after an OpenICDL-based course.

The programme, which has been running for six months, aims at teaching advanced end-user computing skills, including the use of OpenOffice.org and the Linux desktop included in the tuXlab distro which is built on EduBuntu. Inmates are also given access to Wikipedia, which they can use for research and to further their studies 

Not only is it a way to reduce recidivism rates (which I whole-heartedly support), it doesn't come with a massive initial capital outlay in terms of courseware, software, and hardware from governments before the programme's efficacy is determined, and it scales out easily and without two different types of relicensing hurdles if it turns out well. 

(Okay, so you could just subscribe to the Daily Python-URL...)

Two potentially life-saving-for-me-in-a-few-months SOAP bindings have become available - Kevin Dangoor's TGWebServices and Optio's soaplib.  On a few minute's investigation, I prefer TGWebServices slightly - the code seems easier to get into (although it may just be that I'm used to Kevin's style from TurboGears?), and it makes nice use of generic functions, which should make doing unexpected-to-the-developers things possible.  Neither does WSSE (but who'd want to fight with that early on?), and TGWebServices doesn't try the client side of things.  (Personally, I hope that I can always convince people to use alternatives to SOAP.)

The much-anticipated new declarative layer over the excellent SQLAlchemy ORM was announced, called Elixir.  I'm not certain I like the syntax immediately, but I'm confident that the process of having a number of competing implementations and the associated experiences and observations from them has led the developers to something better than any of them individually, and which can handle the changes inherent with creating a new system more smoothly.

I've mentioned Brevé before, but not directly.  It's a templating system for XML (and HTML, if you don't do XHTML) that's also pure Python code, like this:

    html [
        head [
            title [ 'A Brevé Template' ]
        ],
        body [
            h1 [ 'Briefly, Brevé' ], br,
            div ( style = 'text-align: center;' ) [
                span [ '''
                    As you can see, Brevé maps very
                    directly to the final HTML output.
                ''' ]
            ]
        ]
    ]

It's not for every occasion, but it can make generating certain types of XML easier than alternatives that require actual XML templates - I'd use it for XML chunk generation, and use an XML-based template for broader structural work.  And, like an XML-based template, it escapes content correctly based on whether it's in an attribute or in general content, so you can feel reasonably safe from XSS and just plain old broken output.  Generating malformed XML is just hard.

Also, Gustav Niemeyer released (possibly not recently) dateutil.  It's a set of additional powerful functions for date manipulation, carrying on from where datetime stops.  Of particular interest is the rrule package for a superset of iCalendar recurrence rules from RFC2445.

Ian Bicking wrote a high-level comparison between Pylons and TurboGears.  Basically, they're very close philosophically, but each have areas where they currently lead the other.  There should be a great interplay between the communities going forward, and I can see myself having a more idealistic attachment to Pylons while probably pragmatically using TurboGears often because it'll be the more popular platform with better documentation (well, one day) for new developers to get up to speed on.

PyCon's looks like it's going to be great.  Over 500 registrations as of a few days ago.  Unfortunately I couldn't afford to go, and was too embarrassed to ask for funding - which, realistically, would and should have been declined in favour of letting a whole lot more, and more deserving, people attend.  Next year for sure!  (Unless, say, I, yet again, spend more money than I can really afford on open source events in Africa)

It seems KT's doing well - they were invited to attend the Olliance Group's Open Source Think Tank event later this year.  (Nice venue!  The Californian winelands have long been the #1 location I'd love to visit in the US, even before becoming even more enamoured after watching Sideways)

And (belatedly), they seem to be looking for developers again - if you can stand programming in PHP and want to be paid to work on an open source project in beautiful Cape Town, why not apply?

The great guys at frogfoot are hiring - they've got junior and senior sysadmin posts going, as well as some non-technical positions.

Or if you've always wanted to work with me (how could you not?) - or maybe just with Python in Cape Town - you can also check out this Python web developer job at CareerJunction.  As far as I understand, there are two positions, and you'll be working with Bryn and I.  (If you apply, give me a shout, and if I don't know you well, a geekier version of your experience vs. what's on your CV, so I can put in a good word.)

The Western Cape Linux User Group (CLUG with an invisible W) is having a meeting tonight (Tuesday, February 13, 2007) on MythTV, the personal video recorder and more ("home media convergence box", if you can stand the hype-words).  Join us at our usual venue (actually, might be downstairs tonight), and then for dinner afterwards for geeky chat too if you want.

One of the innovative frogfoot frogs, Joe, mentions that two of my favourite companies, Amobia and Inkululeku, are working together to wirelessly "wire" up schools in the Western Cape.

Until such time that there are decent fixed-line options, wireless is an easy winner in terms of installation time for the last-mile connectivity, and Amobia's investment into creating software to manage their network should pay off in terms of implementation time and future maintenance.

Later this month is the inaugural Cape Town 27 dinner, although it follows on the heels of other geek dinners here and around the country.  The first 27 dinner was in Johannesburg last week - the dinners are planned to alternate between the two cities.

It's like a younger, hipper, and bloggier version of First Tuesday (which seems to be defunct in South Africa now, since the first three pages on their web site have content last updated in 2004, 2005, and 2006 respectively).  I guess it's also a hipper, bloggier, and hypier version of the CLUG meeting+dinner held twice a month.

I've decided to give the CLUG meeting+dinner combo a miss to attend this month's meeting.  The up-side is that, if I like it enough to go again, it won't conflict with CLUG every time.  The down-side is that many of the 27s will be on weekend-nights (and thus means having to compete against the masses) or will conflict with rock climbing or other commitments.

I guess that might actually be an upside for the event itself - getting the same group of people every time isn't great for the creative juices, and nor is missing out on the same people every time.

One of the strangest things I've seen in my career (and I use that term lightly) is how companies react to bad morale and attrition - they meddle even more than they did to cause the bad morale and attrition in the first place.

Kathy Sierra has an, as usual, excellent post on her insightful web log, Creating Passionate Users, on the topic - Don't ask employees to be passionate about the company!  The title isn't quite as useful advice as this quote:

The company should behave just like a good user interface -- support people in doing what they're trying to do, and stay the hell out of their way.

Whenever we see government increasing regulation to improve competition in an industry, we wonder how they can be so blind.  Similarly, often it's hard for companies to know exactly what all their employees do.  And, as they try formalise the process of their employees doing their work, the overhead and the reduced employee satisfaction causes things that used to get done not to get done.  And they see this as a reason to place more formality in things - which the employees see as getting in their way, and ultimately leads to them being unhappy.

Growth stages are the most opportune and dangerous for companies - a whole whack of people being added, changing the culture as new elements and ideas are added.  New ways of thinking of things.  New values.  The danger comes in the need for growth lowering the criteria for entry - less referrals from existing staff, more unknowns, more pressure to hire fast enough to meet requirements.  A few bad apples get by, leading to measures to halt the damage.  These measures change the culture too, potentially alienating the original employees, and also potentially scaring away the people most like the original employees from working there from the vibe they pick up in the interviews.  Attrition starts.  And then beatings will continue until morale improves...

I've had two pretty good interventions in my work life.  I've had many more bad interventions.  The good interventions sought to find the root of the problem - the misunderstandings and unvoiced thoughts.  The bad interventions sought to find someone to blame, and to try treat the symptoms.

Some of the worst interventions were enforced team building exercises.  Or "bunny hugging", as Barry would call it.  Or exercises in exactly how fake we can be when being fake will make the pain end faster.  Not that these exercises didn't help solidify existing alliances between employees, or facilitate the making of new friends - it's just that they didn't have the ability to seek the root problems.  If anything, the solidification of existing alliances and the forming of others usually come at the expense of the company, not to its ultimate benefit, as the topic of conversation is usually the stupidity of the company to try use bunny-hugging to "fix" things.

When you enter an environment where one is forced to sing happy birthday to colleagues on their birthday, or forced to attend a meeting to celebrate an employee you haven't even met having a baby, you have to wonder what underlying problems the company has had, and the likelihood that they still exist.

If you want to make someone happy, make it possible for them to do their job the way they want to do them - for a lot of people, that means doing it properly.  Take them seriously when they talk.  If they complain about the light levels where they work, don't simply lament the situation - do something about it.  If they express disinterest in something, try find a way for them to avoid it.  If they think a particular book would be useful, buy it.  Be the facilitator of their success, filter of outside forces, and the deliverer of praise.  Sure, you can't accommodate your employees every time, but make sure they think that you're doing what you can to avoid the company making their life difficult.

(By the way, I'm not saying that team-building is a bad thing.  It's just that if it's being done in response to identifiable problems, then it's likely not going to help.)