A New Car

A couple of weekends ago I finally bit the bullet and put the deposit down on a new car - an Audi TT. I went with the 3.2 litre quattro in Ibis White with a Mineral Grey interior. The salesman (the third that I had dealt with - the other two had left the industry soon after I saw them) was quite enthusiastic as he had been trying to convince his boss that the TT in white would make a nice show car and he would finally have the chance to show him.

I went moderately wild on the extras. iPod attachment, decent stereo, Audi magnetic ride etc. As this is my first ever brand new car I have to admit to lessened self-control. I did avoid things like the electric seats and the self-dipping xenon headlights though. I'm still torn as to which was the most useless option that I chose though: The luggage nets or the extended leather trim.

I'm still debating whether to get the car debadged as I'm not particularly interested in advertising the fact that I've gone for the most expensive model to all and sundry.

What I'm most irritated about is the fact that I may have a fight on my hands when it comes to getting the licence plate that I want BO07 HBY. I checked the DVLA website and it is up for auction 'soon'. I'm not absolutely desperate to get it, but I would be upset if it ended up not with a Boothby but with some obsessed Star Trek fan.

Balancing Ease of Use in Designs

I've long held that in software design and development that the only wrong way of solving a problem is one that does not work. I do however believe that some ways of solving problems are better than others.

In various entries I've talked about the ways I approach trying to arrive at better solutions, some of them were quite abstract. Most of them are ways of evaluating whether one solution is better than another.

In conversation yesterday with Neil, one evaluation came up that I have found very useful: which solution makes the everyday usage easiest?

This evaluation is probably quite familar to consumers of Application Programming Interfaces (APIs) but often does not seem to be familiar to producers of APIs.

When designing something always consider adding a bit of complexity to doing the unusual things in order to make it easier to do the usual things.

Albert the Mongoose

Time for another family history blog entry. Today I want to tell some stories about Albert the mongoose.

Albert was ridiculously tame, he was a particular pet of my father's who had a poachers pocket sewn into many of his coats so that Albert could travel with him. Often, when my father was having a chat with an acquaintance or talking to a shopkeeper, the conversation would grind to a sudden, staring halt as Albert stuck his head out of the coat to see what was going on.

My grandfather was aware that Albert needed plenty of exercise. The favoured form was to tie a brick to a piece of rope and swing it around. Albert would bounce over the brick on each pass with evident enjoyment.

Albert loved people as they were always doing something interesting and would often wander off to see what they were up to. Once shrieking was heard from one of the neighbouring houses and on investigation it was found that Albert had introduced himself to a new cook. The cook did not appreciate the introduction and proceeded to defend herself with what was to hand: A basket of eggs. Albert thought this was wonderful as he not only got to dodge each egg as it was hurled but also had a race to see how much he could eat before the next egg arrived.

Possibly Albert's finest hour was one afternoon as he rested in a sunny spot on the large back lawn. A local tom cat spotted him. This cat was well known, he had caused no end of havoc among the local feline population, had despoiled birds nest for miles around and had caused much angst to the local humans too. The tom cat went into the full stalking mode; body low to the ground, ears pointed and slow, careful movements. Apparently it was a masterpiece of stalking, no noise, barely a shiver of the grasses and always downwind.

After nearly half an hour of stalking, the cat was perfectly positioned to pounce. What the cat had failed to notice was that thoughout all his zig-zag stalking from cover to cover, Albert had managed to always keep his back square to the cat.

With a tail twitch and a wriggle the attack was launched, but when he landed on the patch of ground that was Albert's resting place, the mongoose was no longer there - he was three feet up and dropping perfectly onto the cat's back. In went all of Albert's claws and with evident satisfaction a firm bite was made to the fold of loose skin on the back of the cat's neck.

The cat took off, squalling round and around the garden. As time went by the cat got more and more tired, more and more quiet until finally, panting heavily, he slowed to a staggering walk and stopped by where Albert had been resting. The mongoose let go and daintily jumped down.

The cat staggered from the garden and was apparently never seen again.

Latin is a language....

Much of my programming is informed by my mother's career as a linguist.

She grew up part of an Anglo-Scots family in Argentina and is fluent in both Spanish and English. From an early age she worked hard to expose me to a range of languages through friends. Not just European languages but also any others (including Nepalese, Urdu and others from an old Gurkha officer).

At school I was made to study French, Latin and Ancient Greek.

I did not turn out as she expected - she taught me the analysis and understanding of languages in general but I instead focussed it on English and computer languages.

When dealing with customers it is essential to understand what they are trying to communicate. I became better at this when I fully realised that English is not just one language but a family of dialects and jargons. The way that different groups of people use English can lead to all kinds of misunderstandings. Just think of the different uses of the word 'Monitor':- A computer display, a thread monitor, a school prefect/official, the verb etc.

In many cases I have found that disagreements arise not because of differences about how a system should work but instead because of differences in how the words describing the system are used.

When it comes to computer languages the awareness of language can be very revealing. One of my favourite examples of this is a brilliant speech by Guy Steele: Growing a Language.

I'd recommend that any software developer, designer or architect pay a little more attention to linguistics.

The Quantum Boolean Anti-Pattern in XML

One of my pet peeves is the Quantum Boolean anti-pattern.

'What the heck is that?' I hear you cry!

Well it is where some bright spark defines an element or attribute as boolean (true or false) but it may or may not be present (minOccurs="0").

What then happens is that you never get to see false. Either the element or attribute is present and set to true or it is absent and you have to guess that it is implicitly false.

Please decide to set it to boolean and make it mandatory or make it an optional marker element with no sub-elements or attributes.

George the Rock Python.

After some positive responses to the family anecdotes about Dulce de Leche, I though that I might share another one.

My father's family certainly accumulated a fair share of these anecdotes. My grandfather kept some quite interesting pets: George an Indian Rock Python and his old friend Albert the Mongoose.

Albert and George were good friends despite their natural instincts. Many was the time that George would be seen asleep in a sunlit place with Albert asleep on top of him.

Next door to my grandfather lived a retired British India Army colonel (my father always referred to him as a 'Boom-Puna' British India Army Colonel). He and his wife had returned to England and had settled into a contented if liverish life filled mainly by complaining how much worse off they were in England. He was a bit of a drinker; my father at the age of 12 delivered a message to him and was offered a gin and tonic.

Soon after moving in, my grandfather (who had an evil sense of humour) bribed my father to take George and deposit him on the colonel's front lawn. He was told to hide in the hedge to keep an eye on George and to report the reaction back faithfully.

With a certain amount of effort (as my father was only a boy at the time and George was a big snake - 12 feet or so), George was deposited on the lawn. My father retired to the nearby privet hedge and, having made himself comfortable, awaited the colonel's arrival with interest.

Soon after the colonel strode out of his front door to take his morning constitutional and found himself confronted by George who was just stirring sluggishly in the morning sun.

He froze for nearly a minute and paled dramatically before rushing back into the house crying 'Maud! Maud draw a cold bath! I've got the DTs again!'

One Way to Reduce the Risk of Social Engineering Attacks

In my day to day job I work with a number of secure systems. A number of them are entirely internal, but the others are customer facing. There are also certain operational procedures which are instigated at a customer's behest (the customer phones up and asks - I've been taken to task for my sesquipedalian posts...).

The problem comes when the customer is asked to provide identification. The classical approach is via some form of shared secret - you know the kind of thing - a maiden name or some other personal piece of information.

I've never been comfortable with this kind of security scheme as it relies on security through obscurity in the first instance and on the honesty of the company staff in the second. Anyone can now trawl through public records to find out the kind of personal information used for this security scheme and increasing numbers of cases of personal information being stolen at call centers are coming to light.

Many of the systems I work with have dual channels for customer interaction:

• A secure online channel - for day to day customer use of the system.
• A phone number - for out of the ordinary requests.

To reduce the risk of Social Engineering attacks I advocate using a scheme that provides authentication for the phone calls through the secure online channel.

The steps of the scheme work like this:

1. The customer decides that he or she needs to make a call.
2. The customer accesses a function on the secure online channel to enter a password for that call.
3. The customer enters the password.
4. The system digests the password and stores it.
5. The customer makes a phone call.
6. The customer representative requests the password and enters it on his system.
7. The system digests this password and compares it with the stored one.
8. If they do not match then the customer has not been authenticated and the representative politely handles this.
9. If they do match then the customer is authenticated and the call proceeds.
10. The system clears the digest preparatory for a new support call.

This scheme does rely on a couple of things:

• The presence of the online channel
• The fact that the customer does not mind the added complexity versus the increased security (this can be handled by clear public relations explaining how this scheme reduces the risk of identity theft etc.)

We are building a chain of trust here. We base the authentication of the customer over the phone on the authentication of the customer over the online channel.

The security of this system can be fine tuned in a number of ways:

• A more complex password (length, character types etc.) can increase security but at the cost of increasing the complexity of the customer interaction with the system and the representative - try explaining !B00thbY over the phone. The fact that the password is intended to be only used once should allow for a simpler password.
• Store the digests of previous passwords and introduce a clear reuse policy. This introduces a marginal increase in customer complexity for a good increase in security.
• Allow (limited) reuse of passwords. This reduces the security but can be very useful in a scenario where support is needed when the online channel is down. The user can generate a password in advance for, say, 5 uses after which time one would hope that the online channel is back and a new password can be generated. Further modifications can come in the event of an extended outage where the password use could be extended.

This security scheme produces a limited shared secret in a relatively simple and secure way and neatly eliminates the use of most personal information from customer interactions. I hope that I have illustrated not only the scheme but the compromises involved.

Mock Gravy

I think that it's time for another culinary discussion.

I love gravy. To me gravy is one of the great culinary delights and adds immeasurably to the right dishes.

The trouble is that I hate all instant gravies and have never felt that gravy recipes that did not involve the juices from roasting were really gravy.

In the end I resolved to come up with a way of making a good 'mock' gravy. It turns out that with the right tweaks it can make a really superior vegetarian gravy.

The fundamental idea is simple. To emulate the roasting process and produce the same flavours.

To do this I put oil (as a substitute for fat) in the bottom of a pan. Depending on whether I am trying to make a chickem, pork or beef 'mock' gravy, I use more or less oil.

To the oil I add a tablespoon of vegetable bouillon and two tablespoons of the appropriate other bouillon. I add a tablespoon of soy sauce, a drop of mushroom ketchup and a couple of tablespoons of lemon or lime juice.

You can optionally add some of the herbs that you would use for roasting, a little garlic and/or onion powder. If I'm making a beef gravy I will often also add a teaspoon of beef Bovril.

The next job is to heat the ingredients in the pan over a medium heat, stirring frequently. The intent is to recreate the process that in the bottom of a roasting tin.

The oil will stay somewhat separate from the other ingredients. You are looking to end up with the oil separated from a thick, glutinous liquor after cooking the ingredients. This should take a few minutes only. If you end up with a gritty mixture that will generally mean that the ingredients have been overcooked.

Now add enough flour to absorb the oil, mixing thoroughly.

Cook the flour and oil mixture for a minute or two. You want the flour cooked but not burnt so stir it continuously. The pan should now look to contain very dark brown bread crumbs.

Add 100-150 ml of cold water to the pan and stir hard. The flour will absorb the water and start to come together.

Now start adding the stock to the mixture a 50 ml or so at a time. Each time stir until the mixture (roux) has become smooth again and there are no more large lumps. The pan remains on the heat throughout this process.

When enough stock has been added the roux will have become runny and the rest of the stock can be addded.

Bring the pan back to the boil and taste. Season if desired. One can either add gravy granules/cubes or boil the gravy down (stirring frequently) until the desired intensity of flavour is achieved. Other flavours such as redcurrant jelly can be added at this time.

The gravy is now ready for serving. Pour it over a large helping of bangers & mash, a bowl of couscous or anything else that takes your fancy.

To make the vegetarian version simply use all vegetable bouillon and stock. This recipe scales up very well.

Ingredients

• Oil (vegetable, olive or something similar)
• Bouillon (I tend to use the Knorr 'Touch of Taste' bouillons)
• Soy Sauce
• Mushroom Ketchup
• Lemon or Lime Juice
• Roasting Herbs (Optional - Rosemary, sage, thyme etc.)
• Garlic Powder (Optional)
• Onion Powder (Optional)
• Beef Bovril (Optional)
• Flour
• 100 - 150 ml Cold Water
• 1 Litre of Good Stock
• Salt and Pepper

Good Threaded Code 2

I've been writing concurrent code in Java for some years now and like to believe that I've learnt a few things in the process.

In Java all objects have a mutex and monitor associated with them and so any object can be used to provide concurrency control and signalling. Java allows code to synchronize on any objects to which it has access, this includes the object declaring the code. In fact Java positively encourages you to synchronize on the declaring object by providing synchronized methods and allowing synchronized(this).
For example:

public class ConcurrentClass
{
    public void doSomething() 
    {
        //do something thread safe...
        ...
        //do something not thread safe...
        synchronized(this)
        {
            ...
        }
    }

    public synchronized void doSomethingElse() 
    {
        //do something not thread safe...
        ...
    }
}

When you call doSomething() or doSomethingElse() the mutex of the ConcurrentClass instance is only held for the duration of the synchronized block or the synchronized method.

I believe that this approach violates encapsulation. The mutex used to control the internal concurrency issues of the ConcurrentClass instance is available to other classes and so other classes can participate in the internal concurrency concerns of the ConcurrentClass instance. At first sight this might not seem a bad thing but just stop and think for a second. This means that concurrency control for a specific resource can be spread across several classes. Rather than debugging one class you may have to debug many.

I would far rather write something like:

public class ConcurrentClass 
{
    private Object mutex = new Object();
    ...
        
    public void doSomething() 
    {
        //do something thread safe...
        ...
        //do something not thread safe...
        synchronized(mutex)
        {
            ...
        }
    }

    public void doSomethingElse() 
    {
        //do something not thread safe...
        synchronized(mutex)
        {
            ...
        }
    }

}

This means that the mutex and thus the concurrency control is fully encapsulated within the class.

An example where the lack of encapsulation can cause a problem is in java.lang.Thread. I have seen application code that sub-classes java.lang.Thread and uses synchronized methods or synchronize on the thread object. If the application code deadlocks it becomes impossible to use any of the normal methods to control the thread as they are all synchronized on the thread themselves.

Idempotence? What's that?

When I'm architecting enterprise systems I often get that response when I talk about my preferred alternative to using 2 phase commits and XA.

Idempotence is not a particularly difficult concept. I define it as a behaviour that has the same outcome on the second, third, fourth etc. time of trying as it had on the first.

A lot of things in our day to day life are idempotent. Say you are going to the car with some bags. On reaching the car you put the bags down and go to unlock the car; you are then interrupted by a friend calling to you and you have a chat. When you turn back to the car you can't remember if you unlocked it or not so you go to unlock it again. If you had actually already unlocked the car, it will remain unlocked. Unlocking a car is normally idempotent - I'm not answerable for odd car designs or for people that actually lock their cars again when they do this.

This can be extremely useful when you are trying to integrate two disparate transactional systems. Lets call one system A and the other B. If we are doing something in A that needs a change to be applied in B we can make the change to B idempotent.

The process would often work like this.

1. Open a transaction in A.
2. Do the work in A.
3. Open a transaction in B.
4. Do the work in B. Idempotent
   
5. Close the transaction in B.
6. Close the transaction in A.

If a problem occurred in steps 1 to 5, systems A and B would be in a consistent state as both transactions would be rolled back. If a problem occurs at stage 6 then B would have the change while A doesn't.

When the process is re-run successfully A and B will end up in a consistent state as long as the work done at B is idempotent.

This is a much lighter weight mechanism that the 2 phase commit / XA one but it does mean that there is a window in which A and B can be inconsistent. For many systems this window may not be a problem in which case I'd recommend the use of idempotence.

Another month between posts.

A lot has happened to me in the past month. My cousin James came to visit, I faced eviction due to my landlord's non-payment of mortgage, I moved to a lovely new place in a converted chapel, I went speed dating and have spent an awful lot of time trying to get broadband into my new place so that I can work from home.

This morning I had a good breakthrough with JOGL - working out how to initiate an offscreen pixel buffer without needing a full blown AWT or Swing GL canvas.

With my life calming down a little (off to see the Two Gallants tomorrow in Brighton), I'll hopefully blog a little more.

Test Driving a New Model Audi TT

I currently drive an X-reg Ford Puma and am now looking for a replacement. I've really enjoyed the Puma, it handles well, has nice pick up and as a 2+2 can carry 4 people over moderate distances. It has swallowed enough camping gear to take me and a friend to music festivals and has regularly slogged around the M25.

However the time has come to think about something else as the Puma is getting on.

What I've been looking has been some kind of successor to the Puma; another sporty coupe that fits well with my life. The trouble is that Ford no longer seems to have cars of that kind available to buy. I'd be forced to choose between one of the Ka or Focus range or end up with a barge.

I ended up casting my eyes further afield and there seemed to be three major choices:
BMW z4 Coupe
Nissan 350Z
Audi TT (New model)

By process of elimination I ended up at the Audi TT, allegedly the handling was massively improved in the new model, it was a 2+2 and can definitely shift. The reviewers had suggested that the 2.0T was nicer handling than the 3.2 quattro, so that was the one that I asked to test drive.

An old friend of mine had asked to come along on the test drive and so, having picked him up a little late, I roared on to the garage.

On arrival we spotted the new and old TT models sat side by side. It was very interesting to compare them. At first blush they looked very similar. The face of the car had obviously changed. The headlights and grille were definitely 'new Audi' and had given it a more aggressive look. The two other obvious changes had occurred at the back of the car with the introduction of a pop-up rear spoiler (my friend and I were both a little dubious - something else to go wrong) and the roofline had shifted back where it met the boot lid.

On further inspection it became clear that the new model was a little wider and a little longer. While most of the distinctive creases and curves remained, they had been subtly altered.

On entering the dealership, I was greeted by our friendly salesman Matt who had cheerfully ignored my request to test the new 2.0T and had lined me up with the 3.2 quattro. At least it had the S-Tronic gearbox so I could happily play with flappy-paddles.

The salesman took us out of the showroom and then out of town with me in the passenger seat and my friend in the back. It was obvious that the +2 seats were not much use at all except for headless people and contortionists. The roofline meant that any adult sitting in the seats had to bend their neck very uncomfortably. It was possible to provide survivable leg room.

Sat in the front passenger seat, I was able to look around the cabin. In basic trim there was quite a lot of basic plastic and, while the build quality was good, I did not much like it. The controls were very well laid out and the seats were extremely comfortable.

The noise in the interior was very pleasant - a nice V6 rumble, pleasantly audible but not overpowering.

Once we had left town I exchanged places with the salesman and prepared myself to drive the car. Fortunately the test drive took in several roads that I had often driven before in a number of cars so I was able to concentrate on the feel of the car and not on the route.

I started off with the S-Tronic box in automatic mode and attempted to pull away. Not having driven an automatic for a while I flubbed the basics - I forgot to put it in drive. Having sorted that out (and remembering to put my foot on the brake) I gently lifted my foot off the brake to see if the car would creep in automatic mode - it did. The indicator stalk on the left was used and I went to pull away. Carefully keeping my left foot out of play I manage a relatively clean pull away and quickly accelerated up to 60. The gearbox was almost unnoticeable, the only sign that gears were being changed was the change in exhaust note.

Rapidly reaching the first corner - a long right hander suitable for 60 miles an hour I simply steered around it. The steering was light but surprisingly communicative. A few hundred yards later I was coming up behind another road user and the road was about to drop away a long way down to a sharp left turn. I deliberately slowed to open up a gap between me and the car in front and let engine braking open up a good gap so that I could come down the hill at speed and take the left hander quite fast. The left hander was dispatched with no fuss. Once again I was coming up behind a small queue of cars so I eased back and followed them. I noticed that it was possible to occasionally wrong-foot the automatic and end up in a less than perfect gear on exiting the corner.

After a little pleasant gentle driving we reached a round about and fortunately the drive took us away from the other cars and onto a lovely half mile of road where I was able to plant my foot and see what would happen. The car took off and I felt that my kidneys were making intimate conversation with the seat. Half way down the road is a right hand bend which rises to a crest half way. The car handled this with ease and dropped down to join the dual carriageway back into town. Joining was no hassle, the only thing that I had to do was make sure that I did not accelerate too much.

On the way back into town I got the salesman to talk me through the gearbox and paddles and I had a pleasant time trying them out. Very neat, very easy. Move the gear stick left to be able to use it to push up and down through the gears, back right for automatic. The paddles worked at any time and could override the choice of the automatic. If I were to get the car I would probably use the paddles almost exclusively.

The final part of the test drive involved me coming up a long hill with a patch of road at the bottom that always manages to unsettle two wheel drive cars. I used the accelerator aggressively and the quattro system worked perfectly. There was no wriggle or squirm and the car accelerated up the hill again. At the top of the hill I got out and sat in the +2 seats, of that the less said the better.

Climbing thankfully out at the showroom I sat down with the salesman and went through the options and colours. The deposit was only to be £1000 and I was sorely tempted to buy then and there. The car was very handsome, the handling impeccable, the engine intoxicating and the gearbox nearly perfect.

I think that I probably shall get a TT despite the cramped rear passenger accomodation. The boot is large and the car is a very good compromise.

I may just need to invest in a guillotine.

Dulce de Leche

Despite my grandparents living in Argentina, my mother was born in Uruguay as they did not want her to be conscripted if she were a boy. As I understand it the Argentines are not always well liked in South America so my mother is often quite firm on the fact that she was born Uruguayan.

Despite largely growing up in England I was exposed to a fair chunk of another culture for which I will always be grateful. A significant part of that is South American cuisine. Dulce de Leche (Milk Jam) is one of the elements of South American cooking that plays no small part in a number of my family's stories.

The recipe for Dulce de Leche is deceptively simple:

• Two cups of milk.
• One cup of sugar.
• A pinch of cream of Tartar.
• Vanilla.

One can use vanilla sugar, good vanilla essence or infuse the vanilla into the milk before cooking the rest.

Simply place all the ingredients into a thick pan over a low heat and cook until the mixture turns a medium caramel brown and thickens, stirring occasionally. This can take anything up to an hour and a half.

The resulting concoction can be eaten with a spoon, used in cake and pastry fillings or just spread on bread.

A rather more simple way of preparing it (not without its own risks) is to boil a can of sweetened condensed milk for 2 and a half hours. Remember to keep an eye on it and top up the water if necessary. Do not open the can in any circumstance until it has had a chance to cool. If the can starts bulging while cooking take it off the heat immediately and stand well back.

The Dulce de Leche produced in this way tends to be a little coarser and less flavoursome.

An english cousin was shown the simple recipe once but unfortunately forgot that it was cooking. 6 hours later the pan had boiled dry and the can had exploded spraying hot metal and hot overcooked Dulce de Leche everywhere. As it was my mother's kitchen and it still smelled of Dulce de Leche 3 months later she was less than pleased.

Another time a very old friend of my mother's was living in the States and started to crave real Dulce de Leche - just like grandmother used to make. She had been using the condensed milk recipe but this time it just wasn't good enough.

She called her Argentine friend in the next town, who after some deliberation said 'No I don't have a recipe for it, but if you take a tin of sweetened condensed milk...'.

My mother's friend then called an Argentine friend in the next county, who said, after some thought, 'Well if you take a tin of sweetened condensed milk...'.

Calls were placed to assorted Argentine friends throughout the Eastern Seaboard. Who without exception replied: 'I don't know the proper recipe but if you take a tin of...'.

My mother's friend not being one to give up without a fight said to herself 'Right, I'll call up the Argentine Embassy in Washington D.C.'.

The operator receiving her call was more than a little surprised by her request and had to reply that she didn't know but if my mother's friend would hold then the operator would ask around in the embassy.

Twenty minutes passed and the operator returned to the call.

'Well I asked everybody, even the Ambassador's chef and the Ambassador's wife and none of them had the recipe, but if you take a tin of...'.

A few weeks after, my mother received a letter from her friend telling this story and my mother was able to send back her own grandmother's recipe as I have recorded above.

Extension Points

This entry is dedicated to Hardev who came up and ever so politely reminded me that it has been some weeks since my last one.

I've been accumulating topics and do have a few things that I want to talk about so here is one.

A classic refrain in the IT business is 'future proofing', a dangerous and terrible thing. All too often I have sat through requirements and design meetings where people sit and try to anticipate all future ways in which the software could be used. If you ever find yourself in such a meeting, either take it over and talk about 'extension points' or run (don't walk) for the nearest exit and wait for the bloated mess that results to collapse in on itself.

An extension point is an abstraction that allows the easy extension of application behaviour by using patterns such as decorator, factory, strategy etc. Unfortunately as I've already discussed many moons ago, abstractions do make it more difficult to comprehend the code; this means that just adding extension points do have a cost. I find that the extension points that are likely to be used in the future are the ones that are 'sympathetic' to the thrust of the functionality.

In the end extension points should fall naturally out of the design process, however they can be used as a good argument to use against the 'future proofers'.

A meeting with an old friend...

I was pleasantly surprised the other day to hear from Ian Hannaford. He wanted to benefit from what little wisdom I could muster. We had a good chat about his new role (thrown in the deep end as usual) and generally had a good old chat. Ian boosted my ego by telling me that he had directed other people to my blog (Hi other people!) and we gossipped about old friends and talked techie.

In the course of our chat, Ian told me about his pattern use but that he found the Gang of Four book a touch dry and academic. He suggested that I could at some stage write something to make it a little more approachable and relevant.

I'm going to think on this a bit and may start writing entries on situations where I have used GOF patterns and why I used them.

I'm going to toddle now and have me a bite of dinner (Teryaki beef, sticky rice and pak choi.).

P.S. I got LocoRoco on the PSP this weekend - at last I am using the PSP as a games machine instead of a mobile video player. LocoRoco is great!

GPGPU with JOGL part 1.

Well I've started tinkering with the GPGPU concept that I mentioned in an earlier post. I've landed on JOGL as it seems to provide me with all the basic functionality that I am going to need and is reasonably cross-platform.

I need to be able to bind a texture (representing an input array of data) into a buffer against which I will run a shader program (the function that I want to perform on the data) and place the outcome of the shader program into a buffer from which I can extract the output array.

I'm struggling with a few handicaps -

• I'm using my PowerBook for this and support for OSX by JOGL is still a bit flakey.
• I've never programmed using GL APIs before so I'm having to learn them from scratch.
• Once I've grasped enought of the APIs, I still have to master at least one shader language.

It's certainly a fun learning curve.

I'm going to struggle on and hopefully I'll have a bitonic sort algorithm implemented before too long.

Apple and Bacon Omelette

I was visiting with some friends the other day, enjoying a very pleasant evening catching up. As I was just heading out the door to go home and cook myself supper, Henry jokingly asked me whether there was anything that he could do with some apples that he had in front of him for his own supper. In short order we had determined that there was some bacon in the fridge and in front of the bemused eyes of his mother, we had rolled up our sleeves and proceeded to cook a new omelette.

We used two to three rashers of salted rindless back bacon, two to three eggs, half a cup of milk and a quarter of a Pink Lady apple per person. The bacon was cut into lardons and the apple was finely diced. The eggs and milk were beaten together with a generous pinch of black pepper. A nob of butter was placed in the bottom of a very large pan and gently melted, we then added the bacon and cooked it through slowly, avoiding frying it to crunchiness, the apples were cooked with the bacon until soft and the egg mixture was added. The low heat was kept on until the eggs were nearly cooked through, the omelette was folded over with a palette knife and then cut into portions and served.

It was extremely delicious.

Facets

I first started thinking about facets after reading an article on java.net nearly two years ago.

I kind of liked the idea that object can gain attributes and behaviours and indeed gain and lose types through it's life cycle.

In normal OO practices where an object is defined having all the types that it is ever going to have (A String is an Object and a CharArray and that is all it will ever be). Facets would allow an object to gain and lose types (A panel in Swing could become a type of Window or a scrollable viewport in another Window). In the real world when a person joins a footbal team they gain the facet of being a footbal player and when they leave the team they lose it. The facet of being a football player brings a team association and a shirt number with it plus a lot of interesting behaviours involving a ball and a pitch.

A number of similar patterns exist such as Mixins, but mixins only really bring behaviour not type information. Unfortunately such a useful word has already been used here.

In fact the idea has already been had by this bunch, and unfortunately they have patented it as US Patent 6,513,157. I did nothing more about facets for quite some time after finding this.

A couple of weeks ago I came across some of the code that I had knocked together to play with facets and decided to bite the bullet and see if I could do something with it. I went through the patent again and realised that the patent covered an implementation of facets that did not need changes to source code. As long as I made sure that what I implemented required changes to existing objects to add the functionality then I would be fine.

I'm finally getting around to completing my implementation and when I've had a chance to explore it I'm going to make it available and blog its uses.

One thing has come out of my work on Facets - Abstract super classes are a snare and a delusion. Take an interface 'Faceted', implement it with a concrete class called 'FacetedSupport'. 'FacetedSupport' can be used as an ancestor of 'Faceted' objects OR it can be used as a support class for objects that are implementing the 'Faceted' interface. If you made it an abstract super class then you would be forced to create a sub-class to make use of its functionality.

Water, Water Everywhere and not a Drop to Drink

I never fail to be irritated by short-sighted stupidity.

I live in the South of England and have been watching all the green spaces where I live being relentlessly developed. I often feel that in order to eliminate the North-South divide, the government is trying to get the entire population to live in the South. Every few months there seems to be a new diktat that several thousand new homes be built in one of the Southern Counties.

One of the problems is that we are in the middle of one of the worst droughts to ever hit the Southern Counties. Why are we moving thousands of people to an area that does not have the infrastructure to support them? It seems that someone is failing their basic planning course - ensure that you have sufficient resource available before you start.

Something that adds heavily to the irony is that, as the South is already very densely populated, most of the decent land has already been used. This means that many of the new developments are occurring on less-suitable land such as flood-plains. Already people are being flooded out of their homes while they can't use their sprinklers.

Will the time come when people are forced to use stand-pipes while their downstairs is under water?

More on Threads and Parallelisation

Two posts in as many days - a brand new record for me.

I talked a little while ago about the future of programming being parallel / multi-core: Good Threaded Code.

Trawling through the one of my favourite web sites I came upon this piece: The Register: Deconstructing databases with Jim Gray. The title is a little misleading, as Mr. Gray actually spends more time talking about the use of GPUs for massively parallel processing and the future of programming in such an environment. He mentions that a common language called 'Accelerator' is already being defined for programming GPUs for parallel processing within a Microsoft environment. Now what I want to know is whether there is similar functionality available for Java? If there isn't I suppose that we'll need to start something in this area.

I want to be able to implement this algorithm and see how well we can get it accelerated: Bitonic Sort.

Process Singleton or Cluster Mutex.

My friend Neil and I have been discussing this pattern for some time. I finally came up with a couple of sober yet pithy names for it as to date we've been calling it the 'Talking Stick Pattern'. This pattern is intended to be used primarily in clustered applications though there is nothing to prevent it being used elsewhere.

The singleton pattern is well known and various attempts have been made to implement it within a cluster. For most instances a node-singleton is sufficient as the singleton pattern is used to prevent excessive memory usage - effectively a pool of one. However on occasions the singleton pattern is used to provide access control to a shared resource that cannot support multiple concurrent accesses or to control the running of a business process that could have issues if run concurrently. Failover is the weakness with implementing a cluster-singleton: You are either dependent on a vendor-specific approach or you have to come up with a solution of your own.

The solution that Neil and I have arrived at is derived from the 'Talking Stick' idea used in group discussions. Rather than have a pell-mell of competing voices, the talking stick is handed to an individual who can then talk, when he is done the talking stick is handed on to the next individual who is allowed to talk. When you do not have the talking stick you are not allowed to talk.

We apply this to a set of objects. Before the object can act, it must get hold of the 'talking stick'. This means that only one object can act at a time. The implementation of the talking stick must be robust in the event of a failure and we have two tried and tested implementations that work.

The first talking stick implementation is a little database dependent. The use of a row-lock in a database that supports either a read-past semantic or like Oracle supports SELECT...FOR UPDATE NOWAIT. When the transaction commits, the lock is released for the next access. If something goes wrong the database transaction rolls back and the row lock becomes available for another transaction.

The second implementation relies on a transactional JMS implementation and a message on a queue is used as the talking stick. An object waits on the queue for a message and when it has it, it is allowed to act. When it has finished it places the message back on the queue for the next object to access. Again, when a failure occurs, the transaction tolls back and the talking stick becomes available again.

Depending on whether it is being used to manage processes or manage access control I would call it a Process Singleton or a Cluster Mutex.

Edit - Gil, a colleague working with me, pointed out that the talking stick is a form of Token.

Are They Running the Country or Running for Re-Election?

This is an huge question, but one which has a simple answer.

I believe that all political parties in Britain are about running for re-election and running the country is a side issue.

This has not always been the case, but the temptation to run for election is one of the great weaknesses of democracy. It is much harder to get re-elected by doing a good job of running the country than by standing around for photo opportunities and telling us what we want to hear. Especially when you consider that your opponents only have to do the photo opportunities. Of course the machiavelllian solution is to get your opponents deeply enmeshed in running the country 'in the interest of non-partisan politics'. The opposition would get credit for not being out of practice in running the country and for any good work they do. You'd increase the availability of competent people for key positions, tie your opponents up and level the field when it came time for elections. With more competent people, the country would benefit overall and you'd get the credit for that (and of course for the good work that your opposition do...).

The trouble is exacerbated by the press. We are told that a strong fifth estate is one of the great pillars of a strong democracy. Unfortunately at the moment our press is lazy and lets the politicians set the agenda. Many stories are handed to them by the current political leadership and by their opposition in their struggle to be elected. It is also in the press' interest to keep politics partisan as the latest bickering is an easy story that sells papers and airtime.

I wonder whether the press could ever be convinced that the real story is in the fact that politicians are running for re-election and not running the country. What would happen if the press were banned from naming individual politicians except when they were doing something wrong? Can democracy ever produce a leadership that truly focusses on running the country and is able to work together with their opponents for the common good? Any other ideas?

Learning: By Rote Vs, Asking Why.

I had a very pleasant lunch yesterday with two Voca colleagues, Roger and Peter. As my lunches tend to always go, we ended up discussing a number of interesting topics, this lunch steered entirely clear of technology.

We discussed counselling, consciousness, upbringing and education. I told Roger and Peter in my typical bombastic manner about a very interesting piece of research I read recently in New Scientist. The article touched on the manners in which humans and our cousins great apes learn from our parents. If you were asked which species is more likely to learn by simple, exact copying of parental activities, which species would you pick?

It turns out that humans are far more likely than any other great ape to shortcut learning by simple imitation. The example that is often quoted is the Mother making a 'pot roast', she cuts off one end of the joint and puts it in the pot beside the main part. Her daughter asks her why she does that and she is forced to reply that she doesn't know, her mother did it that way. So the Mother asks here mother who replies that her own mother always did it that way. The Mother then calls her grandmother who replies that she started doing that because she didn't have a big enough pot.

It appears that humans are extremely likely to learn by rote. It seems to be a shortcut that we have evolved. I believe that this shortcut emerged because of the volume of information that we are forced to learn in order to survive in our culture. If we questioned every single tiny fact, we would never learn enough to sustain the complexity of our culture before we were 50 years old.

That's not to say that questioning is not important. Without 'Why?' we would never have progressed our culture.

I would suggest that the best education a child can get is one that is primarily by rote that does not suppress the desire to question. In effect there is a balance: sufficient rote learning to provide a basic body of knowledge to survive in the world coupled with time spent teaching how to question this body of knowledge.

I believe that you actually have to have a basic level of knowledge before you can decide which questions are worth asking.

When I say 'Culture' I mean it in the more scientific sense of the set of learnt behaviours and knowledge that we as a species learn rather than are born knowing.

Driving Concentration

When I first started driving I had an old MGB Roadster (known as Summer). It was a spartan driving experience the height of luxury being a heater that had only two settings off and BURN YOU TO DEATH!

My driving was probably at its best when I drove Summer, I kept my temper better, drove more defensively and was generally more considerate.

When I started racking up the motorway miles I got a new car. It was a lovely little Ford Puma (known as Sue). It was immensely more reliable and more luxurious.

However it was with Sue that my driving started to disimprove.

My concentration became worse, I failed to anticipate and, when something happened unexpectedly, I was more inclined to lose my temper. I came to realise that something needed to be done, so I thought about it and finally came to a conclusion.

It was the CD/Radio.

In dear old rackety Summer, there was no point to putting a radio in as I wouldn't have been able to hear it above the engine and the noises of the world outside. I was forced to keep all my concentration on my driving and the environment around me. With Sue the radio almost automatically went on and my concentration drifted. I've now taken to turning the radio off and my driving is beginning to return to the quality that it enjoyed before my fall from grace.

The Real Questions.

I've been watching the various manoeuverings going on in parliament to change various aspects of the balance between the legal system and the political system in this country.

When misgivings are raised, the politicians always seem to justify their actions on two bases:

• It is necessary because the current system is unwieldy/expensive.
• Don't you trust us/me?

The real questions that should be asked of the politicians are:

• Whether I trust you or not is irrelevant. Can I trust your successors? Can you guarantee that the 2nd, 3rd or 4th set of politicians to be voted in won't abuse the powers that you are giving them?
• Don't you think it is worth the inconvenience/expense in order to guarantee a reasonable level of freedom from oppression?

I wonder whether we will ever see these questions posed clearly, well and in a situation where the politician is forced to answer.

My Preferred try...finally Semantic

Carrying on a JDBC motif from my previous entry, I want to talk about the use of try...finally blocks in managing resources.

All too often I see null-checking used in finally blocks because this kind of construct is used:

Connection conn = null;
try
{
    conn = ds.getConnection();
    //do some work with the connection.
    ...
}
catch(SQLException sqle)
{
    LOG.error(sqle);
}
finally
{
    if(conn!=null)
    {
        try
        {
            conn.close();
        }
        catch(SQLException sqle)
        {
            LOG.error(sqle);
        }
    }
}

I personally prefer this kind of construct which eliminates the null check at the expense of adding another try..catch block:

try
{
    Connection conn = ds.getConnection();
    try
    {
        //do some work with the connection.
        ...
    }
    finally
    {
        try
        {
            conn.close();
        }
        catch(SQLException sqle)
        {
            LOG.error(sqle);
        }
    }
}
catch(SQLException sqle)
{
    LOG.error(sqle);
}

This is purely stylistic, but I prefer not to have the null check and I do like having the exceptions all towards the end of the unit of code. Do not be tempted to remove the catch from the connection closure as you will mask the original cause of the error. This semantic gets even better if you need to propagate the core SQL exception up to a common handler as it then looks even tidier:

Connection conn = ds.getConnection();
try
{
    //do some work with the connection.
    ...
}
finally
{
    try
    {
        conn.close();
    }
    catch(SQLException sqle)
    {
        LOG.error(sqle);
    }
}
//SQLException gets propagated out to another block of code handling it.

A Connection Pool That Satisfies a Previous Rant

A little while ago I was talking about how many connection pools do not properly satisfy the JDBC specifications (Here.). Many home rolled connection pools do not close the resources associated with a connection before returning it to the pool.

I finally pulled my finger out and decided to have a look for a connection pool that behaved properly. Having pulled the source code for a number of connection pools (and shuddered once or twice when reading), I found a connection pool that actually works properly: Apache Commons-DBCP. The connection pool keeps track of all Statements that are created and closes them and their associated resources when the connection is returned to the pool.

As long as this pool is used, we can avoid all the JDBC clutter in our finally blocks and just close the connection when we're done.

A Persuasive Theory on the Origin of Consciousness

I've been reading a very interesting book these past few days, Julian Jaynes' "The Origin of Consciousness in the Breakdown of the Bicameral Mind".

I've often thought about how we came to be as we are and read upon the subject, this book presents the best theory that I have yet seen.

According to the theory, consciousness is a late comer to the party.

Humanity's evolution of speech and reason did not need consciousness. Speech allowed far larger social groups to be coherent and reason allowed us to cope with the increased complexity of our environments.

As groups increased in size and settlements were formed, humanity needed some means of "sustaining" activities that have no immediate apparent reward.

The mechanism postulated is that of part of a brain that evolved to store the admonitions and advice given by parents and those in authority and to play it back. To a modern mind this playback would appear to be an auditory hallucination. Thus a non-conscious human could persist in preparing a field for planting despite the lack of immediate reward. A voice would be heard, possibly of a parent or of a leader continually reminding them to 'prepare the field for planting'.

Initially this mechanism would have been a simple playback mechanism, but driven by societal advances, it would have increased in complexity and would have been able to synthesise original commands.

The potential ramifications are interesting, the auditory hallucinations based on the voice of a person would have persisted long after the person had died, leading to a belief in life after death and even to worship of ancestors. As the complexity of the mechanism increased, there is no reason not to assume that the voices were limited to known people, and instead could have been interpreted as coming from gods.

It is interesting to note how much of the early writings available to us, talk about how people acted on the promptings of gods or goddesses and how there is very little about personal motivation. Indeed a case can be made that the introspection is a later addition to the text.

In effect early humanity were hallucinating schizophrenics (Bicameral) and much of the structure of ancient societies can be explained by this.

Consciousness only emerged when language gained enough complexity to support a concept of "I" and humanity was forced to evolve mentally by the breakdown of their bicameral societies. Consciousness emerged as an outcome of the integration of the hallucinatory aspects of our minds with the logical/active portions and the ability to conceptualise a self.

I've summarised very briefly here my understanding of a much more complete argument and have only touched on its consequences. I came across this theory first in fiction through the works of Neal Stepenson and other authors, now I can see where they took there inspiration.

I'm certain that this theory is not totally correct, but I would argue that it needs to be considered seriously as we continue to try to arrive at a full understanding of consciousness. It addresses for me how consciousness could arise without a significant physiological change.

The 'self' may be just a construct and this speaks interestingly for future human social and mental evolution.

Change and Stability

I've been giving some thought of late to how groups of people are run, not just at a team level but at an organisational or even a national level.

I've been trying to draw lessons from out politicians past and present. Not many of the lessons have been how to do it well...

One of the major things that have come out of this thought is the dynamic tension between change and stability. Notice the words that I have used, another way of expressing it is the tension between chaos and stasis.

An example of this is the NHS. It is obvious to all politicians that something must be done. The trouble is they never seem to get the something quite right. The old NHS worked as well as it did before the politicians started fiddling because the patients, nurses, doctors and administrators had worked out a modus vivendi. They had discovered ways to work around the grossesr of flaws and it more or less worked. Unfortunately for the NHS it has become a political football, a month doesn't go by without some new announcement. This change is done with the best of intentions but it never gives the participants a chance to settle into the new practices and so the grossest of flaws never get worked around as new ones are introduced the whole time.

Change is essential, especially in this day and age. New technologies, new ideas and new social groups all mean that many of the old ways of doing things do not remain correct. You can't stand still.

But neither do you have to keep running.

I think that the lesson that we and our politicians need to learn is to moderate the rate of change. The real skill comes in introducing changes that only do what is needed and no more. The necessity is to fine-tune the structures we have and only do significant restructuring when absolutely necessary.

This even extends into my domain. It is very tempting to completely re-write systems from the ground up with no regard for the havoc that those changes will cause. All in pursuit of some perception of perfection.

Lasting perfection is unattainable in a dynamic world. All we can ever do is approach it by making sensible, minor changes to proven, stable systems.

The real skill may be in knowing when not to do something...

Good Threaded Code.

I've recently upgraded my main x86 development machine. After a lot of to-ing and fro-ing I decided on an AMD Athlon 64 X2 processor. It has become increasingly clear that the wave of the future is multi-processor / multi-core. Intel and AMD are both talking about 8 core processors before the end of the decade, Sun have just release their Niagara core, Microsoft/IBM are using a 3 core Power PC CPU in the new XBox and Sony/IBM have produced the Cell processor initially going multi-core in the PlayStation 3.

With the advent of consumer level multi-core processors in both home computers and consoles it is becoming clear that we are all going to write more in the way of threaded and concurrent code.

I for one am looking forward to having all that processor power, but as usual the question is how are we, as developers, going to make best use of it?

A part of the solution will be the increasing rise of APIs and frameworks such as that by Doug Lea http://gee.cs.oswego.edu/dl/classes/EDU/oswego/cs/dl/util/concurrent/intro.html for Java. By providing well designed and well tested concurrency code, programming multi-threaded applications will be simplified.

Application Servers will take up part of the burden, but at the usual price of lowered performance and increased cost.

I suspect that certain diagrams will become worth their weight in gold such as UML Activity diagrams.

Lesser known concurreny paradigms such as spin-locks will become more widely understood.

The major expense in using these multi-core systems will be in the synchronization points when different threads / processes exchange data. The skill will be in minimising the impact of the synchronization points.

I'd recommend all developers / designers / architects intending on making any money over the next decade to get their hands on this new generation of SMP machines as soon as possible and start understanding the complexities and opportunities.

Neil negotiating with Meeraj...

Poor Neil, another day in the Voca offices and another day trying to get Meeraj's approval for a code change.

The First Post-Industrial Technology?

I was sat with Neil Ellis one lunchtime last week, we were discussing how and where mass-production techniques could be applied to software.

I explained to Neil why I belived that this couldn't be done.

I feel that the point of mass-production is to reduce the cost of producing copies of a prototype.

In car manufacturing, for example, even the simplest of prototypes for the cheapest of cars cost upwards of £100,000. Millions of pounds will be spent fitting out a production line. After applying mass-production techniques, the copies will be sold for a twentieth of the cost of the original.

With software we are in a very different space, the cost of mass-production is essentially zero. All the cost is in the prototype.

Building a car prototype involves designing components, testing them, putting them together and testing the whole. Engineer will often design a car prototype re-using components from other vehicles and will design components to be re-used. Does this sound familiar?

The advent of devices that will print components and eventually nanofactories mean that the cost of mass-production will shrink down to purely that of the raw materials and the energy.

This means that we may find that trends in software development may be pointers to the future based on these new devices. I wonder what an open-source washing machine will look like?

OptimalJ - My Review.

I've been holding off on this entry for some time. I wanted to wait until I left the place where I was using it so that I could feel able to be completely honest.

I'll be giving an user's eye view of it both from the perspectives of an architect and a developer, I'll talk about designing using it and developing with the artifacts. I've been using OptimalJ 3.2 which is a relatively old version, newer versions will have addressed some of the problems that I'll be raising. I'll mention any fixes that I am aware of.

Model Driven Architecture is yet another attempt to increase the application complexity while reducing development difficulty by drawing picures (usually UML) to define the components and code generation to produce them.

OJ contains a set of UML modelling tools, which are separate and distinct from the MDA tools. It did not seem to be possible to move the analysis and design done using OJ's own UML tools directly into the MDA tools. On the project that I was working on we used Rational Rose to do the analysis and design, before using OJ's MDA tools. The inbuilt UML tools were inferior to those in Rose.

Using OJ to produce the MDA models is little different from using a class modeller in an UML tool, defining classes, attributes and methods. Unfortunately aside from the class modelling most of the rest of the process is about walking through wizards or setting properties. A lot more thought could have gone into using more UML diagram types. For example when one wants to define dependencies between services, one has to add to the 'UsedComponent' property values. These kind of dependencies could be easily defined using collaboration or sequence diagrams.

The three major 'models' that OJ works with are the Domain Model (used to define the domain objects and services), Application Model (fleshing out the domain model and getting quite platform specific) and lastly the code model which is the generated code.

The following layers are defined by OJ by default in the Application Model:

• DBMS - The physical data model.
• Common - cross layer objects such as DTOs (in OJ speak UpdateObjects and DataObjects), OJ enumerations and structs.
• BusinessLogic - if I need to explain this one to you, perhaps you shouldn't be reading this... Well maybe I should explain one thing; BusinessLogic lumps Entity and Session Beans together and does not attempt to guide one down more structured approaches such as using the Session Facade pattern.
• BusinessFacade - a curious set of auto-generated facades that will try to use UserTransactions if you don't watch them very carefully. Mainly useful.
• Presentation - auto-generated struts forms and actions that are only really useful for data entry and prodding the services. In more recent versions of OJ a workflow designer has been added which will hopefully make this a lot more useful.

Out of the box one can only really architect J2EE/EJB applications, persistence only uses entity beans.

In practical use, OJ gets increasingly sluggish as the application increases in size. For what I would call a medium-small application we needed to wait anything up to a minute while OJ digested simple property changes; essential tools that check and update models that needs to be run frequently needed 10 minutes or more. It is also incredibly memory hungry, architects' machines were running with 2 gigabytes of RAM.

These problems can be managed by splitting your application into independent subsystems.

The code generation produces code that is split up into free and guarded blocks. Developers are expected to place business and application logic into the free blocks. I personally believe that weaving hand-cranked and generated code together in this way is a bad idea:

• It makes re-designing and re-factoring unnecessarily difficult as OJ is not properly joined up. If you change a class name or package, the developer code gets put into a 'recycle bin' and needs to be retrieved.
• The quality of code generated by the default patterns is more than a little suspect and makes it very difficult to get useful data out of reporting tools such as findbugs and checkstyle.
• It means that the model is not the thing. Not only do you have to check the model into your repository, you have to check in large swathes of generated code.

The code produced by the default patterns does not inspire confidence in the Java abilities of the people producing the application. For example a form of dirty marker pattern in provided to support the 'UpdateObject's (DTOs) produced in the common layer. Unfortunately it is one of the worst implementations that I have ever come across. The authors seem to have so little understanding of encapsulation that they require a developer to actually call a method to set a changed flag for each field manipulated. When one looks at an UpdateObject's interface, one sees nearly the entire workings of the OJ dirty marker pattern marked public for all to see and it is not a pretty sight. I was so outraged that I was very tempted to go and find the authors and slap them till they promised never to produce something like that ever again.

The version that I was using had one other significant failing, the model-merge functionality that allows multiple modellers to work simultaneously was broken and this introduced a major bottle nect.

Out of the box OJ falls into negative ROI. Any modern project using open source code generation can out-produce it.

Reading over what I have written it would seem that I hate it with a passion. That would be untrue. The potential is huge especially when one realises that one can rewrite the meta-models and patterns.

I would only recommend OJ to an organisation that had significant up-front time to invest:

• Take the time to really understand OJ's capabilities
• Improving the meta-model to provide a better breakdown of the business logic tier,
• Throw out many of the patterns and introduce new ones to use callback or dependency injection to move developer code out of the generated code.
• Produce patterns that make use of a wider range of technology.
• Write a decent dirty marker pattern.
• Wait for Compuware to make the generation steps full scriptable in ant.
• In monster computers for its architects.

Once all this had been done, then a positive ROI should materialise.

Another time to use dependency injection?

I was talking to some friends this lunchtime, explaining IOC containers and what dependency injection could mean for them.

A though came to me. What if IOC containers could also inject dependencies when objects were deserialized? Objects could behave completely differently across tiers. Say an object has a dependency on a persistence interface. On the middle tier the object would get stored to the database buton the client the object would get serialised to the middle tier for storage.

Chilidish Men? Or Childish Women?

I'm sure that the women in my life will give me a lot of grief for this entry, but here goes...

I'm a contentedly childish man, I like my toys, I like cartoons; generally I like having the time to play from time to time. When I have to be I can be as adult as necessary, dealing with difficult things in personal or professional life.

What I have noticed over the years is a certain undercurrent in our society where certain of the feminine elements belittle men because of the very form of childishness that I acknowledge in myself. Expressions such as 'He's such a big kid.' or 'Boys and their toys...' are a mild form of this.

I have started to wonder however whether women are any more grown up than men. I think that they are just better at public relations.

When a boy is playing with toy cars and guns he is being childish, when a girl is playing with dolls, makeup and dressing up they are preparing for adulthood.

This transfers forward to later life, when women spends hours trying on different outfits or being pampered at a beauticians - that is adult. I wonder whether are really just reverting to childhood.

Oracle OCI JDBC Driver problems

Yesterday, I spent a couple of hours trying to work out why we were getting an UnsatisfiedLinkError on the OCI8 libraries when we were running the 9.2 JDBC OCI drivers against an Oracle 9.2 database on Windows Server 2003.

After a lot of head scratching it turned out that the application's ojdbc14.jar and the OCI libraries were mutually incompatible despite appearing to be for the same point release of Oracle.

The application's driver jar was the one that was checked into CVS, whereas the libraries came as part of a new install of Oracle. When we used the driver jar from the new install, our problems went away.

It is definitely worth remembering to always use the driver jars that came with the libraries that you are using.

Fast Web Applications - Design thoughts

Whenever I design web applications there are a number of steps that I go through in order to achieve decent performance. Most of these you will all know about, however there is one thing that I do that does not seem to be well recorded in any book or online article.

Most of you will already have thought about static and dynamic content, in a java web application that will probably mean that there will be a set of HTML pages and images on the apache server and JSPs or Servlets on a Servlet engine.

I have a third classification of content that is very useful to think about: Semi-static.

Semi-static content is content that is data-driven (like dynamic content), but the underlying data changes relatively infrequently compared to the number of times that the content is viewed. Good examples would be a daily graph of stocks or a airport flight information page that is displayed across hundreds of displays.

There are various ways of dealing with this type of content, but they all come down to one thing - caching. Many people when they come across this type of content end up creating a jack-of-all-trades servlet that manages the caching and invalidation of this content. I tend to favour a different approach. I like to generate this content using a template engine (or equivalent for images) to the file system behind my web-server and let the web-server and the downstream caches do their work. The template engine should be event driven, I would normally use a lightweight JMS system to do this.

With a little careful configuration you can leverage the strengths of HTTP to allow an HTTP HEAD request to be issued, checking whether the browser cache, proxy cache or web-server in-memory cache has the up to date version and serve that directly from the cache or retrieve the new version from the web server file system accordingly.

A True Hybrid

On the project that I am on at the moment, we are following an increasingly familiar pattern. A primarily relational database with one or two XML fields. The XML fields store either data that requires an extremely flexible structure or data that is very object-oriented.

I was wondering whether this signals the need for a new kind of database. Relational databases answer many challenges bu not always perfectly, Object databases are often less performant but meet data requirements well. Maybe it is time to bring the two together and create a true ORDBMS, not the type that Oracle tried but one where a relational database has object fields that are in effect mini but true ODBMS instances.

Shiny New Toy on the Horizon...

Finally it looks like we might be getting something that I have always dreamed of.

http://java.sun.com/developer/technicalArticles/Programming/mvm/

I (among many others) have long wanted a way to reliably use a JVM as a true Virtual Machine. To be able to run multiple applications with only the overhead of starting one runtime. It should speed up Swing applications and make ridiculous java startup times a thing of the past.

It has always been feasible to go some way to achieve this kind of thing using a custom java application. This application would take several parameters at the command line:

• The classpath for the application class to run.
• The name of the application class to run.
• Any parameters to be passed to the application class.

When the custom application class was run it would try to connect to a local socket trying to find a running instance of itself. If it found a running instance it would communicate the parameters for the application to be run over the socket and terminate. If it didn't find a running instance it would instantiate a classloader with the relevant classpath, reflectively instantiate the class from the new classloader and run it's main method, passing in the specific parameters. It would then continue running, listening on a socket for any more applications to run.

While this approach does a very effective job of reducing the footprint of running applications, it does not address the startup time (a new JVM instance is started for every application) nor does it properly isolate the applications (locales, system properties, System.exit() all combine to throw a spanner into the works).

I'm looking forward to playing with the research release of the MVM when it becomes available.

Evolution and Religion

Sorry about the length of time between posts, but I've spent an awful lot of time thinking about this post.

Recently I've been aware that several of my friends and acquaintances fall into one of two camps. One camp is the believers who feel that the theory of evolution is an argument used by others to attack the notion that God exists. The other camp is the sceptic who believes that the theory of evolution can be used to deflate certain of the 'absurd' beliefs endemic to most religions.

I happen to fall into another and much quieter camp, those who believe both in a religion and in evolution (whether Darwinian or a more modern form).

So, you ask, how do I reconcile the two?

My first thought is that God tends to use the most effective and elegant means of addressing his will. To my mind that means that there is no reason why evolution isn't the technique that God chose to use in creating the many and varied creatures on this planet.

Secondly I tend to take very seriously the assertion that God stands somehow outside of our conception of time. This means that to God it makes no difference whether it took 4 billion years of our time for us to evolve or we appeared in an instant. What matters is the result.

Something like evolution is almost required, because one thing that most religions are clear about is the importance of faith. Without evolution or its like, we would be able to point and say we know that a supreme being created us. Where would faith be then?

I also believe that there are reasons why we were created with intelligence; one of them is that we can start to approach an understanding of God (however slight an understanding), by investigating the marvellous universe around us..

J2EE Miconceptions - Part 2

Another popular misconception is around SQL and JDBC. I find that an awful lot of database specific SQL get written and a lot of it occurs for no good reason.

I would suggest that you go and read the 'Scalar Functions' section in the JDBC specification and then review all your SQL. I think that you will find a lot of the database specific SQL could be rolled up into generic SQL if you used these functions.

JDBC 'Scalar Functions' are by no means a panacea, but with judicious use you can seriously reduce the amount of DB specific SQL you are using which in turn will reduce your maintenance / migration burdens.

New found liking for dessert wines.

I have just had a bit of an education...

I've just spent a weekend on the Isle of Wight with my sister; Saturday afternoon was spent sharing a significant part of my whisky collection with some old friends. Chris (who happens to be a very talented musician) returned my collection of CDs that I accumulated on my '03 roadtrip and told me how much he had enjoyed 'Los Hombres Calientes'. I'd forgotten that those CDs were in that particular part of my collection, but was very please to know that they had given him so much pleasure.

After a couple of hours drinking whisky, we drifted across into discussing wines. I stated that I had never come across a dessert wine that I liked, with the honourable exception of the occasional Italian Vinsanto. I declared that they were all too sweet and sickly.

Rather well lubricated by this stage, Chris returned with a wax-sealed bottle and introduced me to 'Royal Tokay'. I had heard of 'Tokay' before, but today was the day when I discovered that it was a tiny wine producer in Hungary, costing £30 for an half bottle of the 'Royal Tokay'. I'm not a wine writer so I won't even try to describe what it tasted like; but after cleansing my palate with a little water, I enjoyed one of the finest flavours that it has been my pleasure to experience.

I am now on the look out for Tokay, and I want to try the 'Imperial'.

The Speed of Light?

THe night before last I must have been having some corking dreams, because I woke up this morning with a question that I would love an answer to.

'How does the quantum foam effect the speed of light?'.

For my readers who are not physicists, let me break this question down.

The 'quantum foam' is a descriptive term for the fact that there really is no such thing as a vacuum. Due to the probabilistic nature of quantum mechanics even in a space where all matter as we recognise it has been removed, particles continually pop in and out of existence. Energy is conserved because these particles disappear as rapidly as they appear.

This being the case, I wondered whether the speed of light is in some way determined by it's interaction with this quantum foam.

A side query is whether the increase in mass associated with objects moving relatively faster (most obvious as the speed of light is approached) is in some way connected with interactions with this foam. as masses move faster relatively speaking, they sweep out more and more space, vastly increasing their chance of interacting with these virtual particles.

Singleton - Pattern or Anti-Pattern?

It's interesting to see how various developers respond when you ask them about the singleton pattern. Most react positively, others say that it is an anti-pattern of the worst form.

Why it is considered an anti-pattern by some?

1. In Java it doesn't always meet the contract of the 'Singleton Pattern' as defined in the GoF book. Classloaders mean that more than one instance of a Singleton can exist in the same JVM.
2. Developers often use it as a dumping ground for de facto global variables, harking back to the most unstructured early days of programming.
3. Implementations often feature 'Singleton' in the names of implementing classes and other code becomes dependent on the fact that it is an instance of the singleton pattern.

I believe however that there is nothing wrong with the singleton pattern, just in implementations of it.

One should think of the singleton pattern as an implementation pattern for the factory and pool patterns. Developers should be handed a factory or pool and told only that it will give them an instance of a particular class or interface on request. They shouldn't know that it is the same object instance each time.

This allows the developer of the pool or factory to make efficient use of available resource, if possible, but also allows the implementation to be changed if necessity dictates. By hiding the usage of singleton behind the pool or factory patterns, code fragility is avoided.

J2EE misconceptions - Part 1 (of quite a few actually).

On every project I never fail to be surprised at how little supposedly experienced J2EE developers actually know. This week I found myself asking team members if they actually had read the EJB specification and the answer came back a resounding 'No!'.

It seems that all too often when people are learning J2EE and EJBs in particular, they go and look at the specifications with a beginner's eye and form the impression that the specification documents are dense and impenetrable. They never go back with a more experienced eye and have another go. When I went back and re-read the specifications half way through my first J2EE project I suddenly realised that they were really useful and were packed full of vital information that allowed me to treat J2EE more as a science than as the black art practiced by so many.

The misconception that I want to address with this post is around Container Managed Transactions (CMTs). Let me ask you a question and please be sure what you think your answer is before you move on.

In what circumstances do CMTs roll back?

Take a second or two.

The answer that I usually get is: 'When an exception is thrown.'

Trouble is, the answer is sort of wrong.

The correct answer is: 'When an unchecked exception is thrown out of the transaction context or when EjbContext.setRollbackOnly() is called.'

What many of the developers I talk to fail to realise is that checked exceptions do not roll the transaction back.

So did you get the right answer?

UML Abuse

Had a mixed day at work today, good and bad.

The bad today was around a discussion of how we were going to use UML to analyse use cases. All was going well until we got on to discussing how to model the user interface. We need to record the navigation between screens, the decision made was to do this with a mixture of class and statechart diagrams. I argued against this and suggested Visio, but was overridden on the basis that we wanted to do it all in UML via Rational Rose.

Thinking back I realise that I have actually seen this a lot, people determinedly using UML when another style of diagram would have been more appropriate. I seem to recall that Jacobson, Grady and Rumbaugh, when they first wrote up UML, were quite clear that while the diagrams covered the majority of design cases, they by no means covered them all. Where has this attitude gone?

Does anyone else see UML abuse as a problem?

Name Dropping

I had an interesting day yesterday, some guys from JBoss came over to sell a support package to the project that I am working on. So I found myself, just before lunch, shut in a meeting room hearing the pitch made by a commercial venture on the back of open source software.

I can certainly see that there is money to be made by their business model. In commercial and public sector there is the concept of 'due diligence' hereafter known as 'arse covering'. The support packages on offer provide exactly the right feeling of security for management and customers, in this case backed up by genuine knowledge of the product.

Certain of the more rabid open source fans I have met believe that this is a dilution of their long term goal. At present I have to disagree, because of this commercialisation the rate of development in JBoss is extremely high and that benefits anyone who needs an application server and can't afford the cost models imposed by companies like BEA. I don't know whether a successful business can be established long-term by paid for arse covering, but we will see.

After the meeting I adjourned to the cafeteria to share a coffee with the gentlemen from JBoss and had an interesting time talking to one of the more vocal EJB3 supporters that I have come across. He was very clear that the current 'issues' with JDO are all the fault of the JDO specification team and how they 'snuck out' the candidate specification over Christmas. I certainly agree that it was not a time best calculated to make the right impression. I have a suspicion that the actual cause of the release date was the feeling 'Right let's get it finished and have a good Christmas.'

I have to admire the detachment displayed in discussing how the current situation was in part due to the frictions between a specification driven by large business and a specification driven by a thriving user community. I found myself agreeing that there was room for both specifications. Having more than one persistence specification allows me to exercise one of my dictums: 'The right tool for the right job.'.

My radical proposal to resolve the current problem is to go ahead and put the mechanisms in place for migration from JDO and EJB3, but also state that mechanisms for the reverse migration should be part of the EJB3 specification.

Motorway Driving.

With my current contract, I'm having to do a lot of motorway miles, to and from work. I tend to odd hours and so tend to avoid the worst of rush hour. However there are a few things that I need to get off my chest...

I hate:

• Under-takers - Not the sombre men that come to take away granny, but the fast furious drivers that come up inside lanes past a queue of cars. When they get to the slower moving obstacle in their lane that has caused the stack of cars in the outer lanes, they force their way back out causing everyone else to brake.
• Underpowered White Van Men - White Van man is an acknowledged hazard of modern driving, but in his way he is predictable and possible to get along with. The only one I have any problems with are the ones with heavy loads and small engines. They zoom downhill in the outside lane at 80 mph. I have no problems with them zooming, it's when they go uphill and they use that brief spell of 80mph to justify remaining in the outside lane as they crawl uphill at 60 mph or below.
• Sticky Drivers - I define a sticky driver as someone who stays in a lane regardless of whether the inner lane is empty. They are the cause of a lot of dangerous driving, either because a marginally faster driver is forced to overtake them, or because a significantly faster driver is forced to undertake them, or (worst of all) an impatient driver trying to persuade them to move drives right up their backside.

There are many minor irritations on the roads; roadworks, speed cameras and potholes; but it seems to be a constant that other road users cause the most irritation.

First impression of OptimalJ

Well, I've been using OptimalJ 3.2 for 4 days now and I can say that my first impression is 'stability' or more to the point the lack of it.

Part of the problem is that it is based on 3.5 of Netbeans, which was never noted for it's fabulous robustness, but a large amount of the instabilities that I have encountered are in the OptimalJ functionality. This is a tool that promises to give great productivity enhancements, but to be honest they have all been soaked up by the time spent trying to work around the issues.

This instability reflects on an idea that I've been kicking around for some little time now. The limits of complexity that can be achieved in a piece of software. The improvements in development; assembler, functional programming, procedural programming, object orientation, UML etc.; have all increased the complexity of software that can be produced.

I get the feeling with OptimalJ that this particular software is pushing the complexity limits of the development processes used to produce it.

PVRs, what I really want in one now.

Been chatting with my friend Dan about my upcoming purchase of a Hauppauge DEC 3000-S USB Satellite decoder. We were uncertain about the lack of an hardware encoder for recording until we realised that Digital Video Broadcasts (DVBs) are encoded in MPEG II in the first place.

I got to thinking about what it is that I want in a PVR beyond what is currently available and the major one is being able to view one channel while another is being recorded. This is something that is available with the current analogue technology VCR and television as they each have their own hardware to decode the signal and either display or record it.

Ideally I would like a modular PVR being able to plug in additional receiving/decoding units, so that I can view/record across as many channels as I have decoding units.

Of course there are practical design problems, it would be possible to overwhelm both the CPU and the filing system with the amount of data being written. Within those practical limitations I would love to be able to record 3 or 4 channels at a time.

I suppose the next thing I need to do is look at what can be cobbled together now with available technologies. The easiest thing to prove the concept will be to use two USB compatible decoders. Next will be to sort out drivers and software capable of dealing with two separate streams. I think that I'll probably have to start with MythTV and carry on from there. I know that the Hauppauge software won't cope.

Anyone reading this with ideas, I would appreciate your feedback.

Scrambled Eggs - My Way.

Well, this is going to be my first cooking post; I've been cooking since I was 7 when my mother first came down with M.E., it was only baked beans but I have come a little way since then.

It has taken me quite some time to perfect my scrambled eggs recipe. The quest began when I realised that the white of an egg and the yolk of an egg need quite different cooking regimens to achieve perfection. Undercooked whites are slippery and unpleasant, overcooked yolks are dry as dust and taste vaguely sulphurous.

So here is how I cook my scrambled eggs now:

Ingredients:

• Eggs, 1 or 2 per person depending on famishment and size of egg.
• Milk, about 50 ml per egg. I prefer organic full fat for this but it does work with semi-skimmed.
• Vegetable oil (I prefer sunflower oil).
• Seasoning

Put enough vegetable oil into a small saucepan to cover the base and swirl around the sides. Separate the egg whites from the yolks, putting the whites in the saucepan and the yolks in a bowl with the milk.

Place the saucepan containing the whites and vegetable oil on a low heat. The intent is to cook the whites without frying them to crunchiness. While the whites are cooking whisk the yolks and the milk together.

When the whites have solidified, you can take the opportunity to pour away any excess oil from the saucepan. A little oil gives sheen and prevents sticking, too much oil is just plain greasy.

I now tend to use the edge of a wooden spoon and break the cooked whites down into pieces. The size tends to reflect how 'refined' I'm feeling. Now add the yolk / milk mixture to the pan and return to the heat.

How high I now have the temperature tends to reflect how much attention I am prepared to pay to it. If I'm in a hurry and am prepared to stir continuously I will up the temperature; if I want to pay attention to cooking other things, I will leave it on the lowest possible temperature and stir occasionally.

Before long the scrambled eggs will start to go 'blop' and steam slightly, this is the signal that you have to pay attention and stir to stop it sticking to the pan. Hereafter it is up to you as to how thick and well set you want your scrambled eggs, the longer you cook it the harder they will get. Don't forget that they will continue to thicken slightly after you take them off the heat.

Serve the scrambled eggs and only now apply seasoning. If you add salt any time before this you risk the eggs curdling and separating giving you a texture of grit floating in tasteless water.

I personally like my eggs served either on wholemeal pitta bread or a nice malted wholegrain toast. Cooking up a few bacon lardons and sprinkling them on top can look really smart too.

The final thing to do is enjoy eating them!

A chance to experience MDA.

'Model Driven Architecture'.

There has certainly been a lot of discussion over the past couple of years about MDA. I have sat on the sidelines, an interested observer, wondering quite how well the whole thing will work.

Unlike most development approaches, the model is the primary artifact and not the source code. This leads to the following concerns:

• The versioning of the model will be interesting. Depending on how it is stored and encoded. Remembering the nightmares with Rational Rose Models I'm not sanguine.
• XMI and UML are not of themselves programming languages and so are not able to encode business logic. This means that a significant secondary artifact is the business logic code. The interaction between the model and the business logic will be interesting.
• Tuning and customisation. The templates for conversion from the Platform Independent Model (PIM) to the Platform Specific Model (PSM) may be adjustable, but how far can you go? Can you apply specifc tunings/customisations to specific components?
• The tools to accomplish this seem likely to be complex, what is the scope for vendor lock in?

I've tinkered with AndroMDA and had mixed results. However I will be working with OptimalJ now, so as I gain experience, I will try and record what I learn here.

Connection Pools - are they always a good thing?

Sorry about the hiatus, it was mainly because I had nothing that I wanted to say... I suspect that that is going to change.

I had a talk with a friend (hi Meeraj!) yesterday and we were discussing a connection pool that he was using. After certain queries anything up to 100 megabytes were consumed and not garbage collected. The theory that we arrived at was that the PreparedStatement caching was the problem. I don't know if that was the case, I hope that Meeraj will let me know, but that set me to thinking.

Why did we introduce the complexity of a connection pool to our code in the first place?

I don't just mean the complexity of writing them (that's mainly dealt with by DB vendor's own implementations nowadays); tuning them and ensuring the robustness of our own code around them can be an headache.

As I recall the major reason for pooling connections was the relative expense of creating the connection in the first place. This expense was only large compared to the time that a small query would take to run and be used. It made a lot of sense to reuse a single connection for several of these little queries. The longer running the queries the smaller the amount of time (proportionately) was spent in creating the connection.

So, for a system in which queries are long running and the connections tend to consume a lot of resources, does it make sense to have a connection pool? Would it be better to have some more simple form of connection management?

The 'meta' lesson is that even for the most frequently used patterns / components, we should always remember why they were used in the first place and recognise the situations where they add no value or perhaps even cause problems.