It is no joke that computer hardware has advanced by leaps and bounds over the past decade.

10 years ago, multicore systems were expensive and high-end; today, your grandmother may have one and probably have no clue what she has! Alas, application software has not kept pace. The Linux OS has done a fair job at being able to leverage some of the power multicore systems offer us, but applications running on them have not. The same can be said more or less for Windows, but its been a long while since I did anything systems-level with Windows. But the same issues do apply, however. We are today with the multicore situation where we were in the 80s and the 90s with the multithreaded issues. Back then, CPUs grew support for multithreaded programming, but software including some OSes were slow to adopt. The Macintosh, when it was first released in 1984, would only support cooperative multitasking when the underlying 68000 was perfect for preemption, which was leveraged by the Amiga that came out a year later in 1985. Even I wrote a preemptive multitasking OS back in 1980, when I was 18 years old! I had a hard time understanding why Apple and Microsoft were having such a hard time with this 5 years later. The first preemptive OS released by Microsoft was Windows NT, that came out in the late 80s, shortly after the time Commodore closed its doors. I to this day am still mystified why big corporations like Apple and Microsoft couldnt pull off until much later what an 18 year-old such as myself could do in a couple of months! Today, we are in a similar situation with applications especially those applications that should be able to leverage multicore power such as Databases and Games and Web Servers. Actually, Web Servers such as Apache doesnt fair too bad in this area from what I can tell. Cant speak to IIS, since Ive never used it. The MySQL Database struggles to be able to leverage the multicore, but the developers there seem to only take incremental steps. For a while MySQLs performance used to actually degrade on multicore platforms! Now its been optimized for at least 8 cores, but you still may not see the expected gain on 16 or 32-core systems. Why would this be the case? Surely, MySQL and other applications are written to be *multithreaded*. Ah, there lies the rub. Without getting deep into the details of system resource allocation, spin-locking, and the like, I wish to discuss this issue from a high-level perspective. Im sure some of you not-so-tech-savvy tech managers would appreciate that! And, by the way, its nothing wrong with not understanding all the nitty-gritty details of semaphores and synchronization issues if you are a tech manager. At least, as long as your people do. But then, thats part of the problem. Some do; many dont. And those that dont may not be as forthcoming as youd like for fear of being fired or ridiculed or being given a lower status, etc. Thats the way it is. From the birds point of view, its not too hard to understand at all. The basic difference in multithreaded programming and multicore programming is this:

If you are doing multithreaded programming for single-core systems, you gain nothing for trying to do a lot of processing simultaneously in multiple threads. The goal of multithreaded programming is to, instead, keep the idle time of the CPU as low as possible whilst doing the most work possible where the work will be done in serial fashion anyway. If you actually do attempt to do real processing in multiple threads, in most cases your performance will actually degrade faster than if you did it using serial programming. Why? Because the processor takes time to context-switch between tasks, and the more it has to do that, the more overhead youll incur. On a multicore system, on the other hand, your goals are quite different you actually do want to spread the work out among the cores so that it DOES executes simultaneously, because your performance gain should be directly related to how many cores you have. So, a quad core system should be able to get the work done 4 times faster than a single core system. A 16-core platform should be 4-times faster than a quad-core system, and 16-times faster than a single core. Ah, but as always, theres a catch. Hence my use of the words, should be, rather than will be. Making efficient use of multiple cores in highly non-trivial. For starters, you may not be able to break the tasks down to a parallelizable form. Or if you can, there may still be dependencies between the tasks where one would have to wait on another for information, or wait for a common resource such as the hard drives or network cards to become available. In a dynamic situation, such as a database server, the issues can become even more convoluted as you deal with the order many resources such as rows in a table are locked by many tasks running on many cores. If you are having to deal with legacy code not designed for multicore systems, as was the case with the MySQL codebase, the issues becomes even hairier. Also, most languages in popular use, such as C++, Java, Python and Ruby, have little to no facillities for multicore or distributed programming. Interpreted scripting languages like Python dont even handle multithreading very well, at least Python pre 3.0. Ruby has issues in this regard as well. The common wisdom with some is to run your program in multiple processes, which does work for those situations that doesnt require a lot of state dependencies or resource sharing. That approach, when it works, is nice, because it scales well with the number of cores you have and it also scales well in a cluster/cloud computing scenario. But if those simultaneously running systems DO require a lot of sharing of data, resources, and other dependencies, the scale factor is severely restricted. It may call for a complete rework of the algorithms involved or some clever system hacks, or both. I dont think there is any common wisdom that can be applied in all cases. And when you are dealing with time-to-market constraints, budgetary realities, and the like, you may be forced to take a sub-optimal path.

So it may be safe to say that it may be a little while before we see software truly leverage the true power of multicore systems. We will see it here and there where the effort can be applied and the understanding is present, but the rest will entail a slow evolutionary process. MIT offers a course on multicore programming. Cant say how good or bad it is, but its MIT. How can you go wrong?