A blog about my life, thoughts and work. This blog will consist of programming, philosophy, politics, poetry and anything else that I want to talk about.
July 30, 2006
A meeting with an old friend...
In the course of our chat, Ian told me about his pattern use but that he found the Gang of Four book a touch dry and academic. He suggested that I could at some stage write something to make it a little more approachable and relevant.
I'm going to think on this a bit and may start writing entries on situations where I have used GOF patterns and why I used them.
I'm going to toddle now and have me a bite of dinner (Teryaki beef, sticky rice and pak choi.).
P.S. I got LocoRoco on the PSP this weekend - at last I am using the PSP as a games machine instead of a mobile video player. LocoRoco is great!
July 19, 2006
GPGPU with JOGL part 1.
I need to be able to bind a texture (representing an input array of data) into a buffer against which I will run a shader program (the function that I want to perform on the data) and place the outcome of the shader program into a buffer from which I can extract the output array.
I'm struggling with a few handicaps -
- I'm using my PowerBook for this and support for OSX by JOGL is still a bit flakey.
- I've never programmed using GL APIs before so I'm having to learn them from scratch.
- Once I've grasped enought of the APIs, I still have to master at least one shader language.
I'm going to struggle on and hopefully I'll have a bitonic sort algorithm implemented before too long.
July 04, 2006
Apple and Bacon Omelette
We used two to three rashers of salted rindless back bacon, two to three eggs, half a cup of milk and a quarter of a Pink Lady apple per person. The bacon was cut into lardons and the apple was finely diced. The eggs and milk were beaten together with a generous pinch of black pepper. A nob of butter was placed in the bottom of a very large pan and gently melted, we then added the bacon and cooked it through slowly, avoiding frying it to crunchiness, the apples were cooked with the bacon until soft and the egg mixture was added. The low heat was kept on until the eggs were nearly cooked through, the omelette was folded over with a palette knife and then cut into portions and served.
It was extremely delicious.
June 19, 2006
Facets
I kind of liked the idea that object can gain attributes and behaviours and indeed gain and lose types through it's life cycle.
In normal OO practices where an object is defined having all the types that it is ever going to have (A String is an Object and a CharArray and that is all it will ever be). Facets would allow an object to gain and lose types (A panel in Swing could become a type of Window or a scrollable viewport in another Window). In the real world when a person joins a footbal team they gain the facet of being a footbal player and when they leave the team they lose it. The facet of being a football player brings a team association and a shirt number with it plus a lot of interesting behaviours involving a ball and a pitch.
A number of similar patterns exist such as Mixins, but mixins only really bring behaviour not type information. Unfortunately such a useful word has already been used here.
In fact the idea has already been had by this bunch, and unfortunately they have patented it as US Patent 6,513,157. I did nothing more about facets for quite some time after finding this.
A couple of weeks ago I came across some of the code that I had knocked together to play with facets and decided to bite the bullet and see if I could do something with it. I went through the patent again and realised that the patent covered an implementation of facets that did not need changes to source code. As long as I made sure that what I implemented required changes to existing objects to add the functionality then I would be fine.
I'm finally getting around to completing my implementation and when I've had a chance to explore it I'm going to make it available and blog its uses.
One thing has come out of my work on Facets - Abstract super classes are a snare and a delusion. Take an interface 'Faceted', implement it with a concrete class called 'FacetedSupport'. 'FacetedSupport' can be used as an ancestor of 'Faceted' objects OR it can be used as a support class for objects that are implementing the 'Faceted' interface. If you made it an abstract super class then you would be forced to create a sub-class to make use of its functionality.
June 11, 2006
Water, Water Everywhere and not a Drop to Drink
I live in the South of England and have been watching all the green spaces where I live being relentlessly developed. I often feel that in order to eliminate the North-South divide, the government is trying to get the entire population to live in the South. Every few months there seems to be a new diktat that several thousand new homes be built in one of the Southern Counties.
One of the problems is that we are in the middle of one of the worst droughts to ever hit the Southern Counties. Why are we moving thousands of people to an area that does not have the infrastructure to support them? It seems that someone is failing their basic planning course - ensure that you have sufficient resource available before you start.
Something that adds heavily to the irony is that, as the South is already very densely populated, most of the decent land has already been used. This means that many of the new developments are occurring on less-suitable land such as flood-plains. Already people are being flooded out of their homes while they can't use their sprinklers.
Will the time come when people are forced to use stand-pipes while their downstairs is under water?
May 31, 2006
More on Threads and Parallelisation
I talked a little while ago about the future of programming being parallel / multi-core: Good Threaded Code.
Trawling through the one of my favourite web sites I came upon this piece: The Register: Deconstructing databases with Jim Gray. The title is a little misleading, as Mr. Gray actually spends more time talking about the use of GPUs for massively parallel processing and the future of programming in such an environment. He mentions that a common language called 'Accelerator' is already being defined for programming GPUs for parallel processing within a Microsoft environment. Now what I want to know is whether there is similar functionality available for Java? If there isn't I suppose that we'll need to start something in this area.
I want to be able to implement this algorithm and see how well we can get it accelerated: Bitonic Sort.
May 30, 2006
Process Singleton or Cluster Mutex.
The singleton pattern is well known and various attempts have been made to implement it within a cluster. For most instances a node-singleton is sufficient as the singleton pattern is used to prevent excessive memory usage - effectively a pool of one. However on occasions the singleton pattern is used to provide access control to a shared resource that cannot support multiple concurrent accesses or to control the running of a business process that could have issues if run concurrently. Failover is the weakness with implementing a cluster-singleton: You are either dependent on a vendor-specific approach or you have to come up with a solution of your own.
The solution that Neil and I have arrived at is derived from the 'Talking Stick' idea used in group discussions. Rather than have a pell-mell of competing voices, the talking stick is handed to an individual who can then talk, when he is done the talking stick is handed on to the next individual who is allowed to talk. When you do not have the talking stick you are not allowed to talk.
We apply this to a set of objects. Before the object can act, it must get hold of the 'talking stick'. This means that only one object can act at a time. The implementation of the talking stick must be robust in the event of a failure and we have two tried and tested implementations that work.
The first talking stick implementation is a little database dependent. The use of a row-lock in a database that supports either a read-past semantic or like Oracle supports SELECT...FOR UPDATE NOWAIT. When the transaction commits, the lock is released for the next access. If something goes wrong the database transaction rolls back and the row lock becomes available for another transaction.
The second implementation relies on a transactional JMS implementation and a message on a queue is used as the talking stick. An object waits on the queue for a message and when it has it, it is allowed to act. When it has finished it places the message back on the queue for the next object to access. Again, when a failure occurs, the transaction tolls back and the talking stick becomes available again.
Depending on whether it is being used to manage processes or manage access control I would call it a Process Singleton or a Cluster Mutex.
Edit - Gil, a colleague working with me, pointed out that the talking stick is a form of Token.
May 21, 2006
Are They Running the Country or Running for Re-Election?
I believe that all political parties in Britain are about running for re-election and running the country is a side issue.
This has not always been the case, but the temptation to run for election is one of the great weaknesses of democracy. It is much harder to get re-elected by doing a good job of running the country than by standing around for photo opportunities and telling us what we want to hear. Especially when you consider that your opponents only have to do the photo opportunities. Of course the machiavelllian solution is to get your opponents deeply enmeshed in running the country 'in the interest of non-partisan politics'. The opposition would get credit for not being out of practice in running the country and for any good work they do. You'd increase the availability of competent people for key positions, tie your opponents up and level the field when it came time for elections. With more competent people, the country would benefit overall and you'd get the credit for that (and of course for the good work that your opposition do...).
The trouble is exacerbated by the press. We are told that a strong fifth estate is one of the great pillars of a strong democracy. Unfortunately at the moment our press is lazy and lets the politicians set the agenda. Many stories are handed to them by the current political leadership and by their opposition in their struggle to be elected. It is also in the press' interest to keep politics partisan as the latest bickering is an easy story that sells papers and airtime.
I wonder whether the press could ever be convinced that the real story is in the fact that politicians are running for re-election and not running the country. What would happen if the press were banned from naming individual politicians except when they were doing something wrong? Can democracy ever produce a leadership that truly focusses on running the country and is able to work together with their opponents for the common good? Any other ideas?
May 19, 2006
Learning: By Rote Vs, Asking Why.
We discussed counselling, consciousness, upbringing and education. I told Roger and Peter in my typical bombastic manner about a very interesting piece of research I read recently in New Scientist. The article touched on the manners in which humans and our cousins great apes learn from our parents. If you were asked which species is more likely to learn by simple, exact copying of parental activities, which species would you pick?
It turns out that humans are far more likely than any other great ape to shortcut learning by simple imitation. The example that is often quoted is the Mother making a 'pot roast', she cuts off one end of the joint and puts it in the pot beside the main part. Her daughter asks her why she does that and she is forced to reply that she doesn't know, her mother did it that way. So the Mother asks here mother who replies that her own mother always did it that way. The Mother then calls her grandmother who replies that she started doing that because she didn't have a big enough pot.
It appears that humans are extremely likely to learn by rote. It seems to be a shortcut that we have evolved. I believe that this shortcut emerged because of the volume of information that we are forced to learn in order to survive in our culture. If we questioned every single tiny fact, we would never learn enough to sustain the complexity of our culture before we were 50 years old.
That's not to say that questioning is not important. Without 'Why?' we would never have progressed our culture.
I would suggest that the best education a child can get is one that is primarily by rote that does not suppress the desire to question. In effect there is a balance: sufficient rote learning to provide a basic body of knowledge to survive in the world coupled with time spent teaching how to question this body of knowledge.
I believe that you actually have to have a basic level of knowledge before you can decide which questions are worth asking.
When I say 'Culture' I mean it in the more scientific sense of the set of learnt behaviours and knowledge that we as a species learn rather than are born knowing.
May 11, 2006
Driving Concentration
My driving was probably at its best when I drove Summer, I kept my temper better, drove more defensively and was generally more considerate.
When I started racking up the motorway miles I got a new car. It was a lovely little Ford Puma (known as Sue). It was immensely more reliable and more luxurious.
However it was with Sue that my driving started to disimprove.
My concentration became worse, I failed to anticipate and, when something happened unexpectedly, I was more inclined to lose my temper. I came to realise that something needed to be done, so I thought about it and finally came to a conclusion.
It was the CD/Radio.
In dear old rackety Summer, there was no point to putting a radio in as I wouldn't have been able to hear it above the engine and the noises of the world outside. I was forced to keep all my concentration on my driving and the environment around me. With Sue the radio almost automatically went on and my concentration drifted. I've now taken to turning the radio off and my driving is beginning to return to the quality that it enjoyed before my fall from grace.
March 29, 2006
The Real Questions.
I've been watching the various manoeuverings going on in parliament to change various aspects of the balance between the legal system and the political system in this country.
When misgivings are raised, the politicians always seem to justify their actions on two bases:
- It is necessary because the current system is unwieldy/expensive.
- Don't you trust us/me?
The real questions that should be asked of the politicians are:
- Whether I trust you or not is irrelevant. Can I trust your successors? Can you guarantee that the 2nd, 3rd or 4th set of politicians to be voted in won't abuse the powers that you are giving them?
- Don't you think it is worth the inconvenience/expense in order to guarantee a reasonable level of freedom from oppression?
I wonder whether we will ever see these questions posed clearly, well and in a situation where the politician is forced to answer.
March 16, 2006
My Preferred try...finally Semantic
All too often I see null-checking used in finally blocks because this kind of construct is used:
Connection conn = null;
try
{
conn = ds.getConnection();
//do some work with the connection.
...
}
catch(SQLException sqle)
{
LOG.error(sqle);
}
finally
{
if(conn!=null)
{
try
{
conn.close();
}
catch(SQLException sqle)
{
LOG.error(sqle);
}
}
}
I personally prefer this kind of construct which eliminates the null check at the expense of adding another try..catch block:
try
{
Connection conn = ds.getConnection();
try
{
//do some work with the connection.
...
}
finally
{
try
{
conn.close();
}
catch(SQLException sqle)
{
LOG.error(sqle);
}
}
}
catch(SQLException sqle)
{
LOG.error(sqle);
}
This is purely stylistic, but I prefer not to have the null check and I do like having the exceptions all towards the end of the unit of code. Do not be tempted to remove the catch from the connection closure as you will mask the original cause of the error. This semantic gets even better if you need to propagate the core SQL exception up to a common handler as it then looks even tidier:
Connection conn = ds.getConnection();
try
{
//do some work with the connection.
...
}
finally
{
try
{
conn.close();
}
catch(SQLException sqle)
{
LOG.error(sqle);
}
}
//SQLException gets propagated out to another block of code handling it.
March 10, 2006
A Connection Pool That Satisfies a Previous Rant
I finally pulled my finger out and decided to have a look for a connection pool that behaved properly. Having pulled the source code for a number of connection pools (and shuddered once or twice when reading), I found a connection pool that actually works properly: Apache Commons-DBCP. The connection pool keeps track of all Statements that are created and closes them and their associated resources when the connection is returned to the pool.
As long as this pool is used, we can avoid all the JDBC clutter in our finally blocks and just close the connection when we're done.
February 23, 2006
A Persuasive Theory on the Origin of Consciousness
I've often thought about how we came to be as we are and read upon the subject, this book presents the best theory that I have yet seen.
According to the theory, consciousness is a late comer to the party.
Humanity's evolution of speech and reason did not need consciousness. Speech allowed far larger social groups to be coherent and reason allowed us to cope with the increased complexity of our environments.
As groups increased in size and settlements were formed, humanity needed some means of "sustaining" activities that have no immediate apparent reward.
The mechanism postulated is that of part of a brain that evolved to store the admonitions and advice given by parents and those in authority and to play it back. To a modern mind this playback would appear to be an auditory hallucination. Thus a non-conscious human could persist in preparing a field for planting despite the lack of immediate reward. A voice would be heard, possibly of a parent or of a leader continually reminding them to 'prepare the field for planting'.
Initially this mechanism would have been a simple playback mechanism, but driven by societal advances, it would have increased in complexity and would have been able to synthesise original commands.
The potential ramifications are interesting, the auditory hallucinations based on the voice of a person would have persisted long after the person had died, leading to a belief in life after death and even to worship of ancestors. As the complexity of the mechanism increased, there is no reason not to assume that the voices were limited to known people, and instead could have been interpreted as coming from gods.
It is interesting to note how much of the early writings available to us, talk about how people acted on the promptings of gods or goddesses and how there is very little about personal motivation. Indeed a case can be made that the introspection is a later addition to the text.
In effect early humanity were hallucinating schizophrenics (Bicameral) and much of the structure of ancient societies can be explained by this.
Consciousness only emerged when language gained enough complexity to support a concept of "I" and humanity was forced to evolve mentally by the breakdown of their bicameral societies. Consciousness emerged as an outcome of the integration of the hallucinatory aspects of our minds with the logical/active portions and the ability to conceptualise a self.
I've summarised very briefly here my understanding of a much more complete argument and have only touched on its consequences. I came across this theory first in fiction through the works of Neal Stepenson and other authors, now I can see where they took there inspiration.
I'm certain that this theory is not totally correct, but I would argue that it needs to be considered seriously as we continue to try to arrive at a full understanding of consciousness. It addresses for me how consciousness could arise without a significant physiological change.
The 'self' may be just a construct and this speaks interestingly for future human social and mental evolution.
January 31, 2006
Change and Stability
I've been trying to draw lessons from out politicians past and present. Not many of the lessons have been how to do it well...
One of the major things that have come out of this thought is the dynamic tension between change and stability. Notice the words that I have used, another way of expressing it is the tension between chaos and stasis.
An example of this is the NHS. It is obvious to all politicians that something must be done. The trouble is they never seem to get the something quite right. The old NHS worked as well as it did before the politicians started fiddling because the patients, nurses, doctors and administrators had worked out a modus vivendi. They had discovered ways to work around the grossesr of flaws and it more or less worked. Unfortunately for the NHS it has become a political football, a month doesn't go by without some new announcement. This change is done with the best of intentions but it never gives the participants a chance to settle into the new practices and so the grossest of flaws never get worked around as new ones are introduced the whole time.
Change is essential, especially in this day and age. New technologies, new ideas and new social groups all mean that many of the old ways of doing things do not remain correct. You can't stand still.
But neither do you have to keep running.
I think that the lesson that we and our politicians need to learn is to moderate the rate of change. The real skill comes in introducing changes that only do what is needed and no more. The necessity is to fine-tune the structures we have and only do significant restructuring when absolutely necessary.
This even extends into my domain. It is very tempting to completely re-write systems from the ground up with no regard for the havoc that those changes will cause. All in pursuit of some perception of perfection.
Lasting perfection is unattainable in a dynamic world. All we can ever do is approach it by making sensible, minor changes to proven, stable systems.
The real skill may be in knowing when not to do something...
December 19, 2005
Good Threaded Code.
With the advent of consumer level multi-core processors in both home computers and consoles it is becoming clear that we are all going to write more in the way of threaded and concurrent code.
I for one am looking forward to having all that processor power, but as usual the question is how are we, as developers, going to make best use of it?
A part of the solution will be the increasing rise of APIs and frameworks such as that by Doug Lea http://gee.cs.oswego.edu/dl/classes/EDU/oswego/cs/dl/util/concurrent/intro.html for Java. By providing well designed and well tested concurrency code, programming multi-threaded applications will be simplified.
Application Servers will take up part of the burden, but at the usual price of lowered performance and increased cost.
I suspect that certain diagrams will become worth their weight in gold such as UML Activity diagrams.
Lesser known concurreny paradigms such as spin-locks will become more widely understood.
The major expense in using these multi-core systems will be in the synchronization points when different threads / processes exchange data. The skill will be in minimising the impact of the synchronization points.
I'd recommend all developers / designers / architects intending on making any money over the next decade to get their hands on this new generation of SMP machines as soon as possible and start understanding the complexities and opportunities.
October 20, 2005
Neil negotiating with Meeraj...
August 20, 2005
The First Post-Industrial Technology?
I explained to Neil why I belived that this couldn't be done.
I feel that the point of mass-production is to reduce the cost of producing copies of a prototype.
In car manufacturing, for example, even the simplest of prototypes for the cheapest of cars cost upwards of £100,000. Millions of pounds will be spent fitting out a production line. After applying mass-production techniques, the copies will be sold for a twentieth of the cost of the original.
With software we are in a very different space, the cost of mass-production is essentially zero. All the cost is in the prototype.
Building a car prototype involves designing components, testing them, putting them together and testing the whole. Engineer will often design a car prototype re-using components from other vehicles and will design components to be re-used. Does this sound familiar?
The advent of devices that will print components and eventually nanofactories mean that the cost of mass-production will shrink down to purely that of the raw materials and the energy.
This means that we may find that trends in software development may be pointers to the future based on these new devices. I wonder what an open-source washing machine will look like?
July 31, 2005
OptimalJ - My Review.
I've been holding off on this entry for some time. I wanted to wait until I left the place where I was using it so that I could feel able to be completely honest.
I'll be giving an user's eye view of it both from the perspectives of an architect and a developer, I'll talk about designing using it and developing with the artifacts. I've been using OptimalJ 3.2 which is a relatively old version, newer versions will have addressed some of the problems that I'll be raising. I'll mention any fixes that I am aware of.
Model Driven Architecture is yet another attempt to increase the application complexity while reducing development difficulty by drawing picures (usually UML) to define the components and code generation to produce them.
OJ contains a set of UML modelling tools, which are separate and distinct from the MDA tools. It did not seem to be possible to move the analysis and design done using OJ's own UML tools directly into the MDA tools. On the project that I was working on we used Rational Rose to do the analysis and design, before using OJ's MDA tools. The inbuilt UML tools were inferior to those in Rose.
Using OJ to produce the MDA models is little different from using a class modeller in an UML tool, defining classes, attributes and methods. Unfortunately aside from the class modelling most of the rest of the process is about walking through wizards or setting properties. A lot more thought could have gone into using more UML diagram types. For example when one wants to define dependencies between services, one has to add to the 'UsedComponent' property values. These kind of dependencies could be easily defined using collaboration or sequence diagrams.
The three major 'models' that OJ works with are the Domain Model (used to define the domain objects and services), Application Model (fleshing out the domain model and getting quite platform specific) and lastly the code model which is the generated code.
The following layers are defined by OJ by default in the Application Model:
- DBMS - The physical data model.
- Common - cross layer objects such as DTOs (in OJ speak UpdateObjects and DataObjects), OJ enumerations and structs.
- BusinessLogic - if I need to explain this one to you, perhaps you shouldn't be reading this... Well maybe I should explain one thing; BusinessLogic lumps Entity and Session Beans together and does not attempt to guide one down more structured approaches such as using the Session Facade pattern.
- BusinessFacade - a curious set of auto-generated facades that will try to use UserTransactions if you don't watch them very carefully. Mainly useful.
- Presentation - auto-generated struts forms and actions that are only really useful for data entry and prodding the services. In more recent versions of OJ a workflow designer has been added which will hopefully make this a lot more useful.
Out of the box one can only really architect J2EE/EJB applications, persistence only uses entity beans.
In practical use, OJ gets increasingly sluggish as the application increases in size. For what I would call a medium-small application we needed to wait anything up to a minute while OJ digested simple property changes; essential tools that check and update models that needs to be run frequently needed 10 minutes or more. It is also incredibly memory hungry, architects' machines were running with 2 gigabytes of RAM.
These problems can be managed by splitting your application into independent subsystems.
The code generation produces code that is split up into free and guarded blocks. Developers are expected to place business and application logic into the free blocks. I personally believe that weaving hand-cranked and generated code together in this way is a bad idea:
- It makes re-designing and re-factoring unnecessarily difficult as OJ is not properly joined up. If you change a class name or package, the developer code gets put into a 'recycle bin' and needs to be retrieved.
- The quality of code generated by the default patterns is more than a little suspect and makes it very difficult to get useful data out of reporting tools such as findbugs and checkstyle.
- It means that the model is not the thing. Not only do you have to check the model into your repository, you have to check in large swathes of generated code.
The code produced by the default patterns does not inspire confidence in the Java abilities of the people producing the application. For example a form of dirty marker pattern in provided to support the 'UpdateObject's (DTOs) produced in the common layer. Unfortunately it is one of the worst implementations that I have ever come across. The authors seem to have so little understanding of encapsulation that they require a developer to actually call a method to set a changed flag for each field manipulated. When one looks at an UpdateObject's interface, one sees nearly the entire workings of the OJ dirty marker pattern marked public for all to see and it is not a pretty sight. I was so outraged that I was very tempted to go and find the authors and slap them till they promised never to produce something like that ever again.
The version that I was using had one other significant failing, the model-merge functionality that allows multiple modellers to work simultaneously was broken and this introduced a major bottle nect.
Out of the box OJ falls into negative ROI. Any modern project using open source code generation can out-produce it.
Reading over what I have written it would seem that I hate it with a passion. That would be untrue. The potential is huge especially when one realises that one can rewrite the meta-models and patterns.
I would only recommend OJ to an organisation that had significant up-front time to invest:
- Take the time to really understand OJ's capabilities
- Improving the meta-model to provide a better breakdown of the business logic tier,
- Throw out many of the patterns and introduce new ones to use callback or dependency injection to move developer code out of the generated code.
- Produce patterns that make use of a wider range of technology.
- Write a decent dirty marker pattern.
- Wait for Compuware to make the generation steps full scriptable in ant.
- In monster computers for its architects.
Once all this had been done, then a positive ROI should materialise.
July 19, 2005
Another time to use dependency injection?
A though came to me. What if IOC containers could also inject dependencies when objects were deserialized? Objects could behave completely differently across tiers. Say an object has a dependency on a persistence interface. On the middle tier the object would get stored to the database buton the client the object would get serialised to the middle tier for storage.
