December 20, 2006

Latin is a language....

Much of my programming is informed by my mother's career as a linguist.

She grew up part of an Anglo-Scots family in Argentina and is fluent in both Spanish and English. From an early age she worked hard to expose me to a range of languages through friends. Not just European languages but also any others (including Nepalese, Urdu and others from an old Gurkha officer).

At school I was made to study French, Latin and Ancient Greek.

I did not turn out as she expected - she taught me the analysis and understanding of languages in general but I instead focussed it on English and computer languages.

When dealing with customers it is essential to understand what they are trying to communicate. I became better at this when I fully realised that English is not just one language but a family of dialects and jargons. The way that different groups of people use English can lead to all kinds of misunderstandings. Just think of the different uses of the word 'Monitor':- A computer display, a thread monitor, a school prefect/official, the verb etc.

In many cases I have found that disagreements arise not because of differences about how a system should work but instead because of differences in how the words describing the system are used.

When it comes to computer languages the awareness of language can be very revealing. One of my favourite examples of this is a brilliant speech by Guy Steele: Growing a Language.

I'd recommend that any software developer, designer or architect pay a little more attention to linguistics.

December 14, 2006

The Quantum Boolean Anti-Pattern in XML

One of my pet peeves is the Quantum Boolean anti-pattern.

'What the heck is that?' I hear you cry!

Well it is where some bright spark defines an element or attribute as boolean (true or false) but it may or may not be present (minOccurs="0").

What then happens is that you never get to see false. Either the element or attribute is present and set to true or it is absent and you have to guess that it is implicitly false.

Please decide to set it to boolean and make it mandatory or make it an optional marker element with no sub-elements or attributes.

December 08, 2006

George the Rock Python.

After some positive responses to the family anecdotes about Dulce de Leche, I though that I might share another one.

My father's family certainly accumulated a fair share of these anecdotes. My grandfather kept some quite interesting pets: George an Indian Rock Python and his old friend Albert the Mongoose.

Albert and George were good friends despite their natural instincts. Many was the time that George would be seen asleep in a sunlit place with Albert asleep on top of him.

Next door to my grandfather lived a retired British India Army colonel (my father always referred to him as a 'Boom-Puna' British India Army Colonel). He and his wife had returned to England and had settled into a contented if liverish life filled mainly by complaining how much worse off they were in England. He was a bit of a drinker; my father at the age of 12 delivered a message to him and was offered a gin and tonic.

Soon after moving in, my grandfather (who had an evil sense of humour) bribed my father to take George and deposit him on the colonel's front lawn. He was told to hide in the hedge to keep an eye on George and to report the reaction back faithfully.

With a certain amount of effort (as my father was only a boy at the time and George was a big snake - 12 feet or so), George was deposited on the lawn. My father retired to the nearby privet hedge and, having made himself comfortable, awaited the colonel's arrival with interest.

Soon after the colonel strode out of his front door to take his morning constitutional and found himself confronted by George who was just stirring sluggishly in the morning sun.

He froze for nearly a minute and paled dramatically before rushing back into the house crying 'Maud! Maud draw a cold bath! I've got the DTs again!'

December 06, 2006

One Way to Reduce the Risk of Social Engineering Attacks

In my day to day job I work with a number of secure systems. A number of them are entirely internal, but the others are customer facing. There are also certain operational procedures which are instigated at a customer's behest (the customer phones up and asks - I've been taken to task for my sesquipedalian posts...).

The problem comes when the customer is asked to provide identification. The classical approach is via some form of shared secret - you know the kind of thing - a maiden name or some other personal piece of information.

I've never been comfortable with this kind of security scheme as it relies on security through obscurity in the first instance and on the honesty of the company staff in the second. Anyone can now trawl through public records to find out the kind of personal information used for this security scheme and increasing numbers of cases of personal information being stolen at call centers are coming to light.

Many of the systems I work with have dual channels for customer interaction:
  • A secure online channel - for day to day customer use of the system.
  • A phone number - for out of the ordinary requests.
To reduce the risk of Social Engineering attacks I advocate using a scheme that provides authentication for the phone calls through the secure online channel.

The steps of the scheme work like this:
  1. The customer decides that he or she needs to make a call.
  2. The customer accesses a function on the secure online channel to enter a password for that call.
  3. The customer enters the password.
  4. The system digests the password and stores it.
  5. The customer makes a phone call.
  6. The customer representative requests the password and enters it on his system.
  7. The system digests this password and compares it with the stored one.
  8. If they do not match then the customer has not been authenticated and the representative politely handles this.
  9. If they do match then the customer is authenticated and the call proceeds.
  10. The system clears the digest preparatory for a new support call.
This scheme does rely on a couple of things:
  • The presence of the online channel
  • The fact that the customer does not mind the added complexity versus the increased security (this can be handled by clear public relations explaining how this scheme reduces the risk of identity theft etc.)
We are building a chain of trust here. We base the authentication of the customer over the phone on the authentication of the customer over the online channel.

The security of this system can be fine tuned in a number of ways:
  • A more complex password (length, character types etc.) can increase security but at the cost of increasing the complexity of the customer interaction with the system and the representative - try explaining !B00thbY over the phone. The fact that the password is intended to be only used once should allow for a simpler password.
  • Store the digests of previous passwords and introduce a clear reuse policy. This introduces a marginal increase in customer complexity for a good increase in security.
  • Allow (limited) reuse of passwords. This reduces the security but can be very useful in a scenario where support is needed when the online channel is down. The user can generate a password in advance for, say, 5 uses after which time one would hope that the online channel is back and a new password can be generated. Further modifications can come in the event of an extended outage where the password use could be extended.
This security scheme produces a limited shared secret in a relatively simple and secure way and neatly eliminates the use of most personal information from customer interactions. I hope that I have illustrated not only the scheme but the compromises involved.

November 24, 2006

Mock Gravy

I think that it's time for another culinary discussion.

I love gravy. To me gravy is one of the great culinary delights and adds immeasurably to the right dishes.

The trouble is that I hate all instant gravies and have never felt that gravy recipes that did not involve the juices from roasting were really gravy.

In the end I resolved to come up with a way of making a good 'mock' gravy. It turns out that with the right tweaks it can make a really superior vegetarian gravy.

The fundamental idea is simple. To emulate the roasting process and produce the same flavours.

To do this I put oil (as a substitute for fat) in the bottom of a pan. Depending on whether I am trying to make a chickem, pork or beef 'mock' gravy, I use more or less oil.

To the oil I add a tablespoon of vegetable bouillon and two tablespoons of the appropriate other bouillon. I add a tablespoon of soy sauce, a drop of mushroom ketchup and a couple of tablespoons of lemon or lime juice.

You can optionally add some of the herbs that you would use for roasting, a little garlic and/or onion powder. If I'm making a beef gravy I will often also add a teaspoon of beef Bovril.

The next job is to heat the ingredients in the pan over a medium heat, stirring frequently. The intent is to recreate the process that in the bottom of a roasting tin.

The oil will stay somewhat separate from the other ingredients. You are looking to end up with the oil separated from a thick, glutinous liquor after cooking the ingredients. This should take a few minutes only. If you end up with a gritty mixture that will generally mean that the ingredients have been overcooked.

Now add enough flour to absorb the oil, mixing thoroughly.

Cook the flour and oil mixture for a minute or two. You want the flour cooked but not burnt so stir it continuously. The pan should now look to contain very dark brown bread crumbs.

Add 100-150 ml of cold water to the pan and stir hard. The flour will absorb the water and start to come together.

Now start adding the stock to the mixture a 50 ml or so at a time. Each time stir until the mixture (roux) has become smooth again and there are no more large lumps. The pan remains on the heat throughout this process.

When enough stock has been added the roux will have become runny and the rest of the stock can be addded.

Bring the pan back to the boil and taste. Season if desired. One can either add gravy granules/cubes or boil the gravy down (stirring frequently) until the desired intensity of flavour is achieved. Other flavours such as redcurrant jelly can be added at this time.

The gravy is now ready for serving. Pour it over a large helping of bangers & mash, a bowl of couscous or anything else that takes your fancy.

To make the vegetarian version simply use all vegetable bouillon and stock. This recipe scales up very well.

Ingredients
  • Oil (vegetable, olive or something similar)
  • Bouillon (I tend to use the Knorr 'Touch of Taste' bouillons)
  • Soy Sauce
  • Mushroom Ketchup
  • Lemon or Lime Juice
  • Roasting Herbs (Optional - Rosemary, sage, thyme etc.)
  • Garlic Powder (Optional)
  • Onion Powder (Optional)
  • Beef Bovril (Optional)
  • Flour
  • 100 - 150 ml Cold Water
  • 1 Litre of Good Stock
  • Salt and Pepper

November 17, 2006

Good Threaded Code 2

I've been writing concurrent code in Java for some years now and like to believe that I've learnt a few things in the process.

In Java all objects have a mutex and monitor associated with them and so any object can be used to provide concurrency control and signalling. Java allows code to synchronize on any objects to which it has access, this includes the object declaring the code. In fact Java positively encourages you to synchronize on the declaring object by providing synchronized methods and allowing synchronized(this).
For example:
public class ConcurrentClass
{
public void doSomething()
{
//do something thread safe...
...
//do something not thread safe...
synchronized(this)
{
...
}
}

public synchronized void doSomethingElse()
{
//do something not thread safe...
...
}
}
When you call doSomething() or doSomethingElse() the mutex of the ConcurrentClass instance is only held for the duration of the synchronized block or the synchronized method.

I believe that this approach violates encapsulation. The mutex used to control the internal concurrency issues of the ConcurrentClass instance is available to other classes and so other classes can participate in the internal concurrency concerns of the ConcurrentClass instance. At first sight this might not seem a bad thing but just stop and think for a second. This means that concurrency control for a specific resource can be spread across several classes. Rather than debugging one class you may have to debug many.

I would far rather write something like:
public class ConcurrentClass 
{
private Object mutex = new Object();
...

public void doSomething()
{
//do something thread safe...
...
//do something not thread safe...
synchronized(mutex)
{
...
}
}

public void doSomethingElse()
{
//do something not thread safe...
synchronized(mutex)
{
...
}
}

}
This means that the mutex and thus the concurrency control is fully encapsulated within the class.

An example where the lack of encapsulation can cause a problem is in java.lang.Thread. I have seen application code that sub-classes java.lang.Thread and uses synchronized methods or synchronize on the thread object. If the application code deadlocks it becomes impossible to use any of the normal methods to control the thread as they are all synchronized on the thread themselves.

November 03, 2006

Idempotence? What's that?

When I'm architecting enterprise systems I often get that response when I talk about my preferred alternative to using 2 phase commits and XA.

Idempotence is not a particularly difficult concept. I define it as a behaviour that has the same outcome on the second, third, fourth etc. time of trying as it had on the first.

A lot of things in our day to day life are idempotent. Say you are going to the car with some bags. On reaching the car you put the bags down and go to unlock the car; you are then interrupted by a friend calling to you and you have a chat. When you turn back to the car you can't remember if you unlocked it or not so you go to unlock it again. If you had actually already unlocked the car, it will remain unlocked. Unlocking a car is normally idempotent - I'm not answerable for odd car designs or for people that actually lock their cars again when they do this.

This can be extremely useful when you are trying to integrate two disparate transactional systems. Lets call one system A and the other B. If we are doing something in A that needs a change to be applied in B we can make the change to B idempotent.

The process would often work like this.
  1. Open a transaction in A.
  2. Do the work in A.
  3. Open a transaction in B.
  4. Do the work in B. Idempotent
  5. Close the transaction in B.
  6. Close the transaction in A.
If a problem occurred in steps 1 to 5, systems A and B would be in a consistent state as both transactions would be rolled back. If a problem occurs at stage 6 then B would have the change while A doesn't.

When the process is re-run successfully A and B will end up in a consistent state as long as the work done at B is idempotent.

This is a much lighter weight mechanism that the 2 phase commit / XA one but it does mean that there is a window in which A and B can be inconsistent. For many systems this window may not be a problem in which case I'd recommend the use of idempotence.

November 02, 2006

Another month between posts.

A lot has happened to me in the past month. My cousin James came to visit, I faced eviction due to my landlord's non-payment of mortgage, I moved to a lovely new place in a converted chapel, I went speed dating and have spent an awful lot of time trying to get broadband into my new place so that I can work from home.

This morning I had a good breakthrough with JOGL - working out how to initiate an offscreen pixel buffer without needing a full blown AWT or Swing GL canvas.

With my life calming down a little (off to see the Two Gallants tomorrow in Brighton), I'll hopefully blog a little more.

September 26, 2006

Test Driving a New Model Audi TT

I currently drive an X-reg Ford Puma and am now looking for a replacement. I've really enjoyed the Puma, it handles well, has nice pick up and as a 2+2 can carry 4 people over moderate distances. It has swallowed enough camping gear to take me and a friend to music festivals and has regularly slogged around the M25.

However the time has come to think about something else as the Puma is getting on.

What I've been looking has been some kind of successor to the Puma; another sporty coupe that fits well with my life. The trouble is that Ford no longer seems to have cars of that kind available to buy. I'd be forced to choose between one of the Ka or Focus range or end up with a barge.

I ended up casting my eyes further afield and there seemed to be three major choices:
BMW z4 Coupe
Nissan 350Z
Audi TT (New model)

By process of elimination I ended up at the Audi TT, allegedly the handling was massively improved in the new model, it was a 2+2 and can definitely shift. The reviewers had suggested that the 2.0T was nicer handling than the 3.2 quattro, so that was the one that I asked to test drive.

An old friend of mine had asked to come along on the test drive and so, having picked him up a little late, I roared on to the garage.

On arrival we spotted the new and old TT models sat side by side. It was very interesting to compare them. At first blush they looked very similar. The face of the car had obviously changed. The headlights and grille were definitely 'new Audi' and had given it a more aggressive look. The two other obvious changes had occurred at the back of the car with the introduction of a pop-up rear spoiler (my friend and I were both a little dubious - something else to go wrong) and the roofline had shifted back where it met the boot lid.

On further inspection it became clear that the new model was a little wider and a little longer. While most of the distinctive creases and curves remained, they had been subtly altered.

On entering the dealership, I was greeted by our friendly salesman Matt who had cheerfully ignored my request to test the new 2.0T and had lined me up with the 3.2 quattro. At least it had the S-Tronic gearbox so I could happily play with flappy-paddles.

The salesman took us out of the showroom and then out of town with me in the passenger seat and my friend in the back. It was obvious that the +2 seats were not much use at all except for headless people and contortionists. The roofline meant that any adult sitting in the seats had to bend their neck very uncomfortably. It was possible to provide survivable leg room.

Sat in the front passenger seat, I was able to look around the cabin. In basic trim there was quite a lot of basic plastic and, while the build quality was good, I did not much like it. The controls were very well laid out and the seats were extremely comfortable.

The noise in the interior was very pleasant - a nice V6 rumble, pleasantly audible but not overpowering.

Once we had left town I exchanged places with the salesman and prepared myself to drive the car. Fortunately the test drive took in several roads that I had often driven before in a number of cars so I was able to concentrate on the feel of the car and not on the route.

I started off with the S-Tronic box in automatic mode and attempted to pull away. Not having driven an automatic for a while I flubbed the basics - I forgot to put it in drive. Having sorted that out (and remembering to put my foot on the brake) I gently lifted my foot off the brake to see if the car would creep in automatic mode - it did. The indicator stalk on the left was used and I went to pull away. Carefully keeping my left foot out of play I manage a relatively clean pull away and quickly accelerated up to 60. The gearbox was almost unnoticeable, the only sign that gears were being changed was the change in exhaust note.

Rapidly reaching the first corner - a long right hander suitable for 60 miles an hour I simply steered around it. The steering was light but surprisingly communicative. A few hundred yards later I was coming up behind another road user and the road was about to drop away a long way down to a sharp left turn. I deliberately slowed to open up a gap between me and the car in front and let engine braking open up a good gap so that I could come down the hill at speed and take the left hander quite fast. The left hander was dispatched with no fuss. Once again I was coming up behind a small queue of cars so I eased back and followed them. I noticed that it was possible to occasionally wrong-foot the automatic and end up in a less than perfect gear on exiting the corner.

After a little pleasant gentle driving we reached a round about and fortunately the drive took us away from the other cars and onto a lovely half mile of road where I was able to plant my foot and see what would happen. The car took off and I felt that my kidneys were making intimate conversation with the seat. Half way down the road is a right hand bend which rises to a crest half way. The car handled this with ease and dropped down to join the dual carriageway back into town. Joining was no hassle, the only thing that I had to do was make sure that I did not accelerate too much.

On the way back into town I got the salesman to talk me through the gearbox and paddles and I had a pleasant time trying them out. Very neat, very easy. Move the gear stick left to be able to use it to push up and down through the gears, back right for automatic. The paddles worked at any time and could override the choice of the automatic. If I were to get the car I would probably use the paddles almost exclusively.

The final part of the test drive involved me coming up a long hill with a patch of road at the bottom that always manages to unsettle two wheel drive cars. I used the accelerator aggressively and the quattro system worked perfectly. There was no wriggle or squirm and the car accelerated up the hill again. At the top of the hill I got out and sat in the +2 seats, of that the less said the better.

Climbing thankfully out at the showroom I sat down with the salesman and went through the options and colours. The deposit was only to be £1000 and I was sorely tempted to buy then and there. The car was very handsome, the handling impeccable, the engine intoxicating and the gearbox nearly perfect.

I think that I probably shall get a TT despite the cramped rear passenger accomodation. The boot is large and the car is a very good compromise.

I may just need to invest in a guillotine.

September 15, 2006

Dulce de Leche

Despite my grandparents living in Argentina, my mother was born in Uruguay as they did not want her to be conscripted if she were a boy. As I understand it the Argentines are not always well liked in South America so my mother is often quite firm on the fact that she was born Uruguayan.

Despite largely growing up in England I was exposed to a fair chunk of another culture for which I will always be grateful. A significant part of that is South American cuisine. Dulce de Leche (Milk Jam) is one of the elements of South American cooking that plays no small part in a number of my family's stories.

The recipe for Dulce de Leche is deceptively simple:
  • Two cups of milk.
  • One cup of sugar.
  • A pinch of cream of Tartar.
  • Vanilla.
One can use vanilla sugar, good vanilla essence or infuse the vanilla into the milk before cooking the rest.

Simply place all the ingredients into a thick pan over a low heat and cook until the mixture turns a medium caramel brown and thickens, stirring occasionally. This can take anything up to an hour and a half.

The resulting concoction can be eaten with a spoon, used in cake and pastry fillings or just spread on bread.

A rather more simple way of preparing it (not without its own risks) is to boil a can of sweetened condensed milk for 2 and a half hours. Remember to keep an eye on it and top up the water if necessary. Do not open the can in any circumstance until it has had a chance to cool. If the can starts bulging while cooking take it off the heat immediately and stand well back.

The Dulce de Leche produced in this way tends to be a little coarser and less flavoursome.

An english cousin was shown the simple recipe once but unfortunately forgot that it was cooking. 6 hours later the pan had boiled dry and the can had exploded spraying hot metal and hot overcooked Dulce de Leche everywhere. As it was my mother's kitchen and it still smelled of Dulce de Leche 3 months later she was less than pleased.

Another time a very old friend of my mother's was living in the States and started to crave real Dulce de Leche - just like grandmother used to make. She had been using the condensed milk recipe but this time it just wasn't good enough.

She called her Argentine friend in the next town, who after some deliberation said 'No I don't have a recipe for it, but if you take a tin of sweetened condensed milk...'.

My mother's friend then called an Argentine friend in the next county, who said, after some thought, 'Well if you take a tin of sweetened condensed milk...'.

Calls were placed to assorted Argentine friends throughout the Eastern Seaboard. Who without exception replied: 'I don't know the proper recipe but if you take a tin of...'.

My mother's friend not being one to give up without a fight said to herself 'Right, I'll call up the Argentine Embassy in Washington D.C.'.

The operator receiving her call was more than a little surprised by her request and had to reply that she didn't know but if my mother's friend would hold then the operator would ask around in the embassy.

Twenty minutes passed and the operator returned to the call.

'Well I asked everybody, even the Ambassador's chef and the Ambassador's wife and none of them had the recipe, but if you take a tin of...'.

A few weeks after, my mother received a letter from her friend telling this story and my mother was able to send back her own grandmother's recipe as I have recorded above.

September 13, 2006

Extension Points

This entry is dedicated to Hardev who came up and ever so politely reminded me that it has been some weeks since my last one.

I've been accumulating topics and do have a few things that I want to talk about so here is one.

A classic refrain in the IT business is 'future proofing', a dangerous and terrible thing. All too often I have sat through requirements and design meetings where people sit and try to anticipate all future ways in which the software could be used. If you ever find yourself in such a meeting, either take it over and talk about 'extension points' or run (don't walk) for the nearest exit and wait for the bloated mess that results to collapse in on itself.

An extension point is an abstraction that allows the easy extension of application behaviour by using patterns such as decorator, factory, strategy etc. Unfortunately as I've already discussed many moons ago, abstractions do make it more difficult to comprehend the code; this means that just adding extension points do have a cost. I find that the extension points that are likely to be used in the future are the ones that are 'sympathetic' to the thrust of the functionality.

In the end extension points should fall naturally out of the design process, however they can be used as a good argument to use against the 'future proofers'.

July 30, 2006

A meeting with an old friend...

I was pleasantly surprised the other day to hear from Ian Hannaford. He wanted to benefit from what little wisdom I could muster. We had a good chat about his new role (thrown in the deep end as usual) and generally had a good old chat. Ian boosted my ego by telling me that he had directed other people to my blog (Hi other people!) and we gossipped about old friends and talked techie.

In the course of our chat, Ian told me about his pattern use but that he found the Gang of Four book a touch dry and academic. He suggested that I could at some stage write something to make it a little more approachable and relevant.

I'm going to think on this a bit and may start writing entries on situations where I have used GOF patterns and why I used them.

I'm going to toddle now and have me a bite of dinner (Teryaki beef, sticky rice and pak choi.).

P.S. I got LocoRoco on the PSP this weekend - at last I am using the PSP as a games machine instead of a mobile video player. LocoRoco is great!

July 19, 2006

GPGPU with JOGL part 1.

Well I've started tinkering with the GPGPU concept that I mentioned in an earlier post. I've landed on JOGL as it seems to provide me with all the basic functionality that I am going to need and is reasonably cross-platform.

I need to be able to bind a texture (representing an input array of data) into a buffer against which I will run a shader program (the function that I want to perform on the data) and place the outcome of the shader program into a buffer from which I can extract the output array.

I'm struggling with a few handicaps -
  • I'm using my PowerBook for this and support for OSX by JOGL is still a bit flakey.
  • I've never programmed using GL APIs before so I'm having to learn them from scratch.
  • Once I've grasped enought of the APIs, I still have to master at least one shader language.
It's certainly a fun learning curve.

I'm going to struggle on and hopefully I'll have a bitonic sort algorithm implemented before too long.

July 04, 2006

Apple and Bacon Omelette

I was visiting with some friends the other day, enjoying a very pleasant evening catching up. As I was just heading out the door to go home and cook myself supper, Henry jokingly asked me whether there was anything that he could do with some apples that he had in front of him for his own supper. In short order we had determined that there was some bacon in the fridge and in front of the bemused eyes of his mother, we had rolled up our sleeves and proceeded to cook a new omelette.

We used two to three rashers of salted rindless back bacon, two to three eggs, half a cup of milk and a quarter of a Pink Lady apple per person. The bacon was cut into lardons and the apple was finely diced. The eggs and milk were beaten together with a generous pinch of black pepper. A nob of butter was placed in the bottom of a very large pan and gently melted, we then added the bacon and cooked it through slowly, avoiding frying it to crunchiness, the apples were cooked with the bacon until soft and the egg mixture was added. The low heat was kept on until the eggs were nearly cooked through, the omelette was folded over with a palette knife and then cut into portions and served.

It was extremely delicious.

June 19, 2006

Facets

I first started thinking about facets after reading an article on java.net nearly two years ago.

I kind of liked the idea that object can gain attributes and behaviours and indeed gain and lose types through it's life cycle.

In normal OO practices where an object is defined having all the types that it is ever going to have (A String is an Object and a CharArray and that is all it will ever be). Facets would allow an object to gain and lose types (A panel in Swing could become a type of Window or a scrollable viewport in another Window). In the real world when a person joins a footbal team they gain the facet of being a footbal player and when they leave the team they lose it. The facet of being a football player brings a team association and a shirt number with it plus a lot of interesting behaviours involving a ball and a pitch.

A number of similar patterns exist such as Mixins, but mixins only really bring behaviour not type information. Unfortunately such a useful word has already been used here.

In fact the idea has already been had by this bunch, and unfortunately they have patented it as US Patent 6,513,157. I did nothing more about facets for quite some time after finding this.

A couple of weeks ago I came across some of the code that I had knocked together to play with facets and decided to bite the bullet and see if I could do something with it. I went through the patent again and realised that the patent covered an implementation of facets that did not need changes to source code. As long as I made sure that what I implemented required changes to existing objects to add the functionality then I would be fine.

I'm finally getting around to completing my implementation and when I've had a chance to explore it I'm going to make it available and blog its uses.

One thing has come out of my work on Facets - Abstract super classes are a snare and a delusion. Take an interface 'Faceted', implement it with a concrete class called 'FacetedSupport'. 'FacetedSupport' can be used as an ancestor of 'Faceted' objects OR it can be used as a support class for objects that are implementing the 'Faceted' interface. If you made it an abstract super class then you would be forced to create a sub-class to make use of its functionality.

June 11, 2006

Water, Water Everywhere and not a Drop to Drink

I never fail to be irritated by short-sighted stupidity.

I live in the South of England and have been watching all the green spaces where I live being relentlessly developed. I often feel that in order to eliminate the North-South divide, the government is trying to get the entire population to live in the South. Every few months there seems to be a new diktat that several thousand new homes be built in one of the Southern Counties.

One of the problems is that we are in the middle of one of the worst droughts to ever hit the Southern Counties. Why are we moving thousands of people to an area that does not have the infrastructure to support them? It seems that someone is failing their basic planning course - ensure that you have sufficient resource available before you start.

Something that adds heavily to the irony is that, as the South is already very densely populated, most of the decent land has already been used. This means that many of the new developments are occurring on less-suitable land such as flood-plains. Already people are being flooded out of their homes while they can't use their sprinklers.

Will the time come when people are forced to use stand-pipes while their downstairs is under water?

May 31, 2006

More on Threads and Parallelisation

Two posts in as many days - a brand new record for me.

I talked a little while ago about the future of programming being parallel / multi-core: Good Threaded Code.

Trawling through the one of my favourite web sites I came upon this piece: The Register: Deconstructing databases with Jim Gray. The title is a little misleading, as Mr. Gray actually spends more time talking about the use of GPUs for massively parallel processing and the future of programming in such an environment. He mentions that a common language called 'Accelerator' is already being defined for programming GPUs for parallel processing within a Microsoft environment. Now what I want to know is whether there is similar functionality available for Java? If there isn't I suppose that we'll need to start something in this area.

I want to be able to implement this algorithm and see how well we can get it accelerated: Bitonic Sort.

May 30, 2006

Process Singleton or Cluster Mutex.

My friend Neil and I have been discussing this pattern for some time. I finally came up with a couple of sober yet pithy names for it as to date we've been calling it the 'Talking Stick Pattern'. This pattern is intended to be used primarily in clustered applications though there is nothing to prevent it being used elsewhere.

The singleton pattern is well known and various attempts have been made to implement it within a cluster. For most instances a node-singleton is sufficient as the singleton pattern is used to prevent excessive memory usage - effectively a pool of one. However on occasions the singleton pattern is used to provide access control to a shared resource that cannot support multiple concurrent accesses or to control the running of a business process that could have issues if run concurrently. Failover is the weakness with implementing a cluster-singleton: You are either dependent on a vendor-specific approach or you have to come up with a solution of your own.

The solution that Neil and I have arrived at is derived from the 'Talking Stick' idea used in group discussions. Rather than have a pell-mell of competing voices, the talking stick is handed to an individual who can then talk, when he is done the talking stick is handed on to the next individual who is allowed to talk. When you do not have the talking stick you are not allowed to talk.

We apply this to a set of objects. Before the object can act, it must get hold of the 'talking stick'. This means that only one object can act at a time. The implementation of the talking stick must be robust in the event of a failure and we have two tried and tested implementations that work.

The first talking stick implementation is a little database dependent. The use of a row-lock in a database that supports either a read-past semantic or like Oracle supports SELECT...FOR UPDATE NOWAIT. When the transaction commits, the lock is released for the next access. If something goes wrong the database transaction rolls back and the row lock becomes available for another transaction.

The second implementation relies on a transactional JMS implementation and a message on a queue is used as the talking stick. An object waits on the queue for a message and when it has it, it is allowed to act. When it has finished it places the message back on the queue for the next object to access. Again, when a failure occurs, the transaction tolls back and the talking stick becomes available again.

Depending on whether it is being used to manage processes or manage access control I would call it a Process Singleton or a Cluster Mutex.

Edit - Gil, a colleague working with me, pointed out that the talking stick is a form of Token.

May 21, 2006

Are They Running the Country or Running for Re-Election?

This is an huge question, but one which has a simple answer.

I believe that all political parties in Britain are about running for re-election and running the country is a side issue.

This has not always been the case, but the temptation to run for election is one of the great weaknesses of democracy. It is much harder to get re-elected by doing a good job of running the country than by standing around for photo opportunities and telling us what we want to hear. Especially when you consider that your opponents only have to do the photo opportunities. Of course the machiavelllian solution is to get your opponents deeply enmeshed in running the country 'in the interest of non-partisan politics'. The opposition would get credit for not being out of practice in running the country and for any good work they do. You'd increase the availability of competent people for key positions, tie your opponents up and level the field when it came time for elections. With more competent people, the country would benefit overall and you'd get the credit for that (and of course for the good work that your opposition do...).

The trouble is exacerbated by the press. We are told that a strong fifth estate is one of the great pillars of a strong democracy. Unfortunately at the moment our press is lazy and lets the politicians set the agenda. Many stories are handed to them by the current political leadership and by their opposition in their struggle to be elected. It is also in the press' interest to keep politics partisan as the latest bickering is an easy story that sells papers and airtime.

I wonder whether the press could ever be convinced that the real story is in the fact that politicians are running for re-election and not running the country. What would happen if the press were banned from naming individual politicians except when they were doing something wrong? Can democracy ever produce a leadership that truly focusses on running the country and is able to work together with their opponents for the common good? Any other ideas?

May 19, 2006

Learning: By Rote Vs, Asking Why.

I had a very pleasant lunch yesterday with two Voca colleagues, Roger and Peter. As my lunches tend to always go, we ended up discussing a number of interesting topics, this lunch steered entirely clear of technology.

We discussed counselling, consciousness, upbringing and education. I told Roger and Peter in my typical bombastic manner about a very interesting piece of research I read recently in New Scientist. The article touched on the manners in which humans and our cousins great apes learn from our parents. If you were asked which species is more likely to learn by simple, exact copying of parental activities, which species would you pick?

It turns out that humans are far more likely than any other great ape to shortcut learning by simple imitation. The example that is often quoted is the Mother making a 'pot roast', she cuts off one end of the joint and puts it in the pot beside the main part. Her daughter asks her why she does that and she is forced to reply that she doesn't know, her mother did it that way. So the Mother asks here mother who replies that her own mother always did it that way. The Mother then calls her grandmother who replies that she started doing that because she didn't have a big enough pot.

It appears that humans are extremely likely to learn by rote. It seems to be a shortcut that we have evolved. I believe that this shortcut emerged because of the volume of information that we are forced to learn in order to survive in our culture. If we questioned every single tiny fact, we would never learn enough to sustain the complexity of our culture before we were 50 years old.

That's not to say that questioning is not important. Without 'Why?' we would never have progressed our culture.

I would suggest that the best education a child can get is one that is primarily by rote that does not suppress the desire to question. In effect there is a balance: sufficient rote learning to provide a basic body of knowledge to survive in the world coupled with time spent teaching how to question this body of knowledge.

I believe that you actually have to have a basic level of knowledge before you can decide which questions are worth asking.

When I say 'Culture' I mean it in the more scientific sense of the set of learnt behaviours and knowledge that we as a species learn rather than are born knowing.

May 11, 2006

Driving Concentration

When I first started driving I had an old MGB Roadster (known as Summer). It was a spartan driving experience the height of luxury being a heater that had only two settings off and BURN YOU TO DEATH!

My driving was probably at its best when I drove Summer, I kept my temper better, drove more defensively and was generally more considerate.

When I started racking up the motorway miles I got a new car. It was a lovely little Ford Puma (known as Sue). It was immensely more reliable and more luxurious.

However it was with Sue that my driving started to disimprove.

My concentration became worse, I failed to anticipate and, when something happened unexpectedly, I was more inclined to lose my temper. I came to realise that something needed to be done, so I thought about it and finally came to a conclusion.

It was the CD/Radio.

In dear old rackety Summer, there was no point to putting a radio in as I wouldn't have been able to hear it above the engine and the noises of the world outside. I was forced to keep all my concentration on my driving and the environment around me. With Sue the radio almost automatically went on and my concentration drifted. I've now taken to turning the radio off and my driving is beginning to return to the quality that it enjoyed before my fall from grace.

March 29, 2006

The Real Questions.

I've been watching the various manoeuverings going on in parliament to change various aspects of the balance between the legal system and the political system in this country.

When misgivings are raised, the politicians always seem to justify their actions on two bases:

  • It is necessary because the current system is unwieldy/expensive.
  • Don't you trust us/me?

The real questions that should be asked of the politicians are:

  • Whether I trust you or not is irrelevant. Can I trust your successors? Can you guarantee that the 2nd, 3rd or 4th set of politicians to be voted in won't abuse the powers that you are giving them?
  • Don't you think it is worth the inconvenience/expense in order to guarantee a reasonable level of freedom from oppression?

I wonder whether we will ever see these questions posed clearly, well and in a situation where the politician is forced to answer.

March 16, 2006

My Preferred try...finally Semantic

Carrying on a JDBC motif from my previous entry, I want to talk about the use of try...finally blocks in managing resources.

All too often I see null-checking used in finally blocks because this kind of construct is used:

Connection conn = null;
try
{
conn = ds.getConnection();
//do some work with the connection.
...
}
catch(SQLException sqle)
{
LOG.error(sqle);
}
finally
{
if(conn!=null)
{
try
{
conn.close();
}
catch(SQLException sqle)
{
LOG.error(sqle);
}
}
}

I personally prefer this kind of construct which eliminates the null check at the expense of adding another try..catch block:

try
{
Connection conn = ds.getConnection();
try
{
//do some work with the connection.
...
}
finally
{
try
{
conn.close();
}
catch(SQLException sqle)
{
LOG.error(sqle);
}
}
}
catch(SQLException sqle)
{
LOG.error(sqle);
}

This is purely stylistic, but I prefer not to have the null check and I do like having the exceptions all towards the end of the unit of code. Do not be tempted to remove the catch from the connection closure as you will mask the original cause of the error. This semantic gets even better if you need to propagate the core SQL exception up to a common handler as it then looks even tidier:

Connection conn = ds.getConnection();
try
{
//do some work with the connection.
...
}
finally
{
try
{
conn.close();
}
catch(SQLException sqle)
{
LOG.error(sqle);
}
}
//SQLException gets propagated out to another block of code handling it.

March 10, 2006

A Connection Pool That Satisfies a Previous Rant

A little while ago I was talking about how many connection pools do not properly satisfy the JDBC specifications (Here.). Many home rolled connection pools do not close the resources associated with a connection before returning it to the pool.

I finally pulled my finger out and decided to have a look for a connection pool that behaved properly. Having pulled the source code for a number of connection pools (and shuddered once or twice when reading), I found a connection pool that actually works properly: Apache Commons-DBCP. The connection pool keeps track of all Statements that are created and closes them and their associated resources when the connection is returned to the pool.

As long as this pool is used, we can avoid all the JDBC clutter in our finally blocks and just close the connection when we're done.

February 23, 2006

A Persuasive Theory on the Origin of Consciousness

I've been reading a very interesting book these past few days, Julian Jaynes' "The Origin of Consciousness in the Breakdown of the Bicameral Mind".

I've often thought about how we came to be as we are and read upon the subject, this book presents the best theory that I have yet seen.

According to the theory, consciousness is a late comer to the party.

Humanity's evolution of speech and reason did not need consciousness. Speech allowed far larger social groups to be coherent and reason allowed us to cope with the increased complexity of our environments.

As groups increased in size and settlements were formed, humanity needed some means of "sustaining" activities that have no immediate apparent reward.

The mechanism postulated is that of part of a brain that evolved to store the admonitions and advice given by parents and those in authority and to play it back. To a modern mind this playback would appear to be an auditory hallucination. Thus a non-conscious human could persist in preparing a field for planting despite the lack of immediate reward. A voice would be heard, possibly of a parent or of a leader continually reminding them to 'prepare the field for planting'.

Initially this mechanism would have been a simple playback mechanism, but driven by societal advances, it would have increased in complexity and would have been able to synthesise original commands.

The potential ramifications are interesting, the auditory hallucinations based on the voice of a person would have persisted long after the person had died, leading to a belief in life after death and even to worship of ancestors. As the complexity of the mechanism increased, there is no reason not to assume that the voices were limited to known people, and instead could have been interpreted as coming from gods.

It is interesting to note how much of the early writings available to us, talk about how people acted on the promptings of gods or goddesses and how there is very little about personal motivation. Indeed a case can be made that the introspection is a later addition to the text.

In effect early humanity were hallucinating schizophrenics (Bicameral) and much of the structure of ancient societies can be explained by this.

Consciousness only emerged when language gained enough complexity to support a concept of "I" and humanity was forced to evolve mentally by the breakdown of their bicameral societies. Consciousness emerged as an outcome of the integration of the hallucinatory aspects of our minds with the logical/active portions and the ability to conceptualise a self.

I've summarised very briefly here my understanding of a much more complete argument and have only touched on its consequences. I came across this theory first in fiction through the works of Neal Stepenson and other authors, now I can see where they took there inspiration.

I'm certain that this theory is not totally correct, but I would argue that it needs to be considered seriously as we continue to try to arrive at a full understanding of consciousness. It addresses for me how consciousness could arise without a significant physiological change.

The 'self' may be just a construct and this speaks interestingly for future human social and mental evolution.

January 31, 2006

Change and Stability

I've been giving some thought of late to how groups of people are run, not just at a team level but at an organisational or even a national level.

I've been trying to draw lessons from out politicians past and present. Not many of the lessons have been how to do it well...

One of the major things that have come out of this thought is the dynamic tension between change and stability. Notice the words that I have used, another way of expressing it is the tension between chaos and stasis.

An example of this is the NHS. It is obvious to all politicians that something must be done. The trouble is they never seem to get the something quite right. The old NHS worked as well as it did before the politicians started fiddling because the patients, nurses, doctors and administrators had worked out a modus vivendi. They had discovered ways to work around the grossesr of flaws and it more or less worked. Unfortunately for the NHS it has become a political football, a month doesn't go by without some new announcement. This change is done with the best of intentions but it never gives the participants a chance to settle into the new practices and so the grossest of flaws never get worked around as new ones are introduced the whole time.

Change is essential, especially in this day and age. New technologies, new ideas and new social groups all mean that many of the old ways of doing things do not remain correct. You can't stand still.

But neither do you have to keep running.

I think that the lesson that we and our politicians need to learn is to moderate the rate of change. The real skill comes in introducing changes that only do what is needed and no more. The necessity is to fine-tune the structures we have and only do significant restructuring when absolutely necessary.

This even extends into my domain. It is very tempting to completely re-write systems from the ground up with no regard for the havoc that those changes will cause. All in pursuit of some perception of perfection.

Lasting perfection is unattainable in a dynamic world. All we can ever do is approach it by making sensible, minor changes to proven, stable systems.

The real skill may be in knowing when not to do something...