IEEE enthuses about new cache memory management

The IEEE is getting excited about a new process for managing the fast-access memory inside a CPU which it claims can lead to as much as a twofold speedup and to energy-use reductions of up to 72 percent.

According to its inhouse mag, the idea fixes several important problems.

At the moment, control is hard-wired into the CPU’s circuitry, but the substantial speedup came when the designers let the operating system do the job.

According to an IEEE paper, the idea, dubbed Jigsaw, is the brainchild of two MIT researchers Daniel Sanchez and his graduate student Nathan Beckmann.

They realised that as computers and portable devices accumulate  more memory and CPU cores, it makes less  sense to leave cache management entirely up to the CPU. Instead, they say, it might be better to let the operating system share the burden.

The idea is not new, it was first seen in IBM’s Cell processors, as well as Sony’s PlayStation 3, but what is new about the MIT technology, called Jigsaw, is that it allows software to configure some on-chip memory caches but without requiring so much control that programming becomes a memory-management nightmare.

This solves many of the problems that people had with the PS3.  It was a bugger to programme because of the memory management requirement.

On-chip caches must be designed to handle every job, from pure floating-point number crunching to intensive searches and queries of a computer’s memory banks.

CPUs have no higher-level knowledge of the kinds of jobs they’re doing. This means a self-contained numerical simulation with complex equations but little need for memory access would run with exactly the same cache resources as would a graph search, a memory-hogging hunt for relationships between stored data.

But using the Sanchez and Beckmann plan you give a percent of the CPU’s footprint to a simple piece of hardware that could monitor in real time the cache activity in each core. Hardware cache monitors would give Jigsaw the independent oversight it would need to play air traffic controller with the CPU’s caches.

It means that the OS’s kernel needs at most a few thousand more lines of code which is not that much considering that the lighter weight Linux’s kernel in 2012 weighed in with 15 million lines.

One of Jigsaw’s features is a software module, to be folded in with the OS, that the researchers call Peekahead which computes the best configuration of CPU caches based on the upcoming jobs it expects the cores to do in the coming clock cycles.

However IEEE has warned that the technology is not baked yet.  It is proven to work well on one chip, but might not be effective on other processors with a different hardware design. It also means that every time the processor changes, you have to redo your software.

But Sanchez thinsk that software applications and utilities would remain unaffected by Jigsaw. The only thing that needs to be done is the operating system code needs to be aware of that intimate knowledge of the hardware, like the topology of the different portions of the cache.