I gave my "need for speed" presentation a JAOO in Denmark on Monday (4 Oct 2010). It's an overview of using "exotic hardware" like FPGAs, NFPs, SIMD processors and GPUs for accelerating applications. Must admit didn't feel I gave my best performance (sorry to attendees for that) (Update: got my audience feedback and I got a "great presentation", top marks! Shows what a judge I am) - it was a bit of a formal set up with banked seating and bright lights pointing at the presenter so it was difficult to see the audience and engage them.
Having said that, the questions and discussions afterwards did re-emphasize a belief (prejudice?) I have had for some time : algorithms are a weakness in current software developers. In fact, I told a story of using Linear Programming to solve an optimisation problem rather than using the brute force approach used by the original programmers. Of the 9 people listening, all software developers, only 2 were aware of the technique and neither of them knew how to even approach solving a problem with it. I found this shocking. Although I shouldn't be: in my experience I find software developers are unaware of many techniques to solve certain types of problem effectively and efficiently.
I wonder whether this is a co-evolutionary outcome of computers becoming more computationally powerful and programming tools becoming more accessible and productive. It has become possible over the last 10 years to solve bigger problems brute-force within the cycle-time programmers are comfortable operating within (you know: the code/compile/debug/coffee cycle). As a consequence, it seems to me the emphasis on good algorithms has faded. Additionally, the removal of technical requirements "solved" by things like garbage collection and the misinterpretation of techniques like "do the simplest thing" seems to have lead to a place where algorithmics are "Terra incognita".
I think their are visible signs of this when you start investigating systems with performance issues. Most commonly these days I find the problems are due to the algorithm used not taking the computational constraints into account. I see everything from creation of unnecessary amounts of garbage to disregard for the effect of network traffic. There seems to be a belief that the system should "solve" these problems for you - and more often than not when I give this "exotic hardware" talk the hope is that these solutions will magically solve performance without the need for the developer to make any changes to their simplistic approach to algorithms. Unfortunately, this is not true, and furthermore I see no chance of current development tools being able to hide the impacts of these newer hardware architectures from the developer.
Some of the people I talked with were students who told me that they have never had any kind of education about the practical implementation of algorithms. They have done classic computer science algorithms (bubble sort, etc) and have been taught programming languages, but they had never had to join these two things up! In fact, when asked, I could think of no reference apart from the greet 'numerical recipes' books - and they don't cover the use of higher level algorithms like Linear Programming or Map-Reduce or Data Parallelism (I.e. How to implement create a model to utilise these approaches for efficiency purposes).
So it seems to me there is a big hole in software development currently: algorithms, using them and implementing them. I feel that we are all the poorer for it and fear it may actually be limiting us. In fact, I think that we are going to hit a technological speed-bump when shortly we need to start using hardware that incorporates on chip SIMDs and FPGAs for performance reasons. The compilers will not be able to save us. We need to go back to a more basic algorithmic approach to get the advantage, and it seems to me current software developers have lost or have never gained this skill.
- Posted using BlogPress from my iPad
0 comments:
Post a Comment