The future calculus of memory management

Posted Jan 19, 2012 7:44 UTC (Thu) by alankila (guest, #47141)
Parent article: The future calculus of memory management

This article makes me think that it's time to start to look for a new line of work. I so dislike the notion of replacing the predictable reliability of fixed resources with this vague quasi-economical logic of boom and crunch periods whose timing will depend on the dynamic demands placed on unrelated systems which nevertheless share the same resources.

The problem here is that computer systems have a minimum expectation of the availability of the various resources, and that in turn dictates the minimum which must always be available to each service, or that service will risk collapsing under load. Of course, this minimum guaranteed level looks a lot like spare capacity to somebody who doesn't realize that the system is already at the optimal minimum. (The other big complication is that throughput of complex systems tends to be limited by the most constrained resource, so screwing this up in any of the major resource categories is going to be bad news for throughput.)

However, I suppose that sometimes it is guaranteed that resource demand peaks occur at non-overlapping times. So you might put nightly backup service on the host that also services some other function during daytime, or something. It is also possible to guarantee minimal resource availability to some virtual machines and leave the rest to struggle under "best effort" guarantees. This kind of competition between resources doesn't sound anything like a "RAM market" to me, though.

I will refrain from criticizing this article much further, but I'll be personally be rather happy if there will be no more like it. RAM servers to make a tidy profit? I find the notion ridiculous and have no idea how it could really work or even make any sense.

to post comments

The future calculus of memory management

Posted Jan 19, 2012 8:17 UTC (Thu) by alankila (guest, #47141) [Link] (2 responses)

Hmm. Reading this again I think the economic logic was not meant to be taken literally, but it was only there to sort of illustrate the motivation and the thinking involved. I think I made a mistake around that paragraph and flipped my bozo bit too early. However, I still think that systems can't be overcommitted safely, so I remain skeptical what possible tricks there are that an administrator could pull while not violating service's expectations.

One thing occurred to me, though: it might be possible to trade one resource for another. Suppose, for instance, that one workload is very memory and cpu hungry, but doesn't do almost any i/o, and another workload is able to operate at same efficiency with less memory if it gains more i/o to replace it. It would make sense to trade the resource bounds between the workloads if relevant data to build a valid model of the workload's throughput within resource constraints exists. However, I doubt the models will ever be anything better than purely empirical, and that means they sometimes don't work, and then all things crash. Ugly.

One example...

Posted Jan 19, 2012 9:35 UTC (Thu) by khim (subscriber, #9252) [Link] (1 responses)

However, I still think that systems can't be overcommitted safely, so I remain skeptical what possible tricks there are that an administrator could pull while not violating service's expectations.

This depends on what services you have available. Google's infrastructure is good example. It runs different kinds of taks on the same machines. Some serve search results and have "no overquota" policy. Some are batch processes, crawlers, exacycle, etc. These can be killed at any time because you can always just start them on another system.

Now, not only batch processes can be run in overcommit mode - Google can even take memory reserved for critical search process! Because if it actually will ask for the memory later you can kill non-critical process with extreme prejudice and give memory to critical process. Not sure if Google actually does it or not, but this is obviously doable.

If you'll think about what real clusters are doing you may be surprised just how much work is done by processes which can actually be killed and restarted. Sadly today such tasks are usually run synchroniously in the context of critical user-facing process thus to use memory efficiently you'll need to do serious refactoring.

This is a variant on what once was called "Goal Mode" in resource management

Posted Jan 19, 2012 17:29 UTC (Thu) by davecb (subscriber, #1574) [Link]

Once more, it's "back to the future" time in computer science (;-))

What you describe here is a superset of a problem that we suffered in the days of the mainframe, that of optimizing resource usage against a "success" criteria. One wanted, in those days, to adjust dispatch priority and disk storage to benefit a program that was overloaded, to get it out of trouble.

Resource management initially allowed one to set guaranteed minimums, and to share when one wasn't using all your allocation, but rather statically. IBM then introduced a scheme that allowed it to be done in a way that often diagnosed a slowdown and added more resources, called "goal mode". It still exists on mainframes.

Modern resource management schemes don't go quite that far. We guarantee minimums, provide maximums so as to avoid letting us shoot ourselves in the foot, make sharing of unused resources easy, and selective penalize memory hogs by making them page against themselves.

We need a modern goal mode: as Linux is a hot-bed of resource management research, I wouldn't be unduly surprised to see it happen here.

--dave

The future calculus of memory management

Posted Feb 3, 2012 3:52 UTC (Fri) by kevinm (guest, #69913) [Link] (1 responses)

Your first paragraph is quite interesting - I like the idea that such a system would be a kind of analogue of a real world financial economy. I think there is a lot of truth to this - you might indeed expect to see big booms and busts in the "datacentre RAM economy" if it's composed of a free market of locally-optimising agents.

I wonder if in the future research into this will lead to results that can be applied in macroeconomics, or vice-versa? If you solve the cross-datacentre RAM allocation problem, have you also invented an algorithm that could be used to optimally drive a real-world command economy?

The future calculus of memory management

Posted Feb 3, 2012 20:11 UTC (Fri) by nix (subscriber, #2304) [Link]

If you solve the cross-datacentre RAM allocation problem, have you also invented an algorithm that could be used to optimally drive a real-world command economy?

Well, that has already been invented, by Leonid Kantorovich way back before WWII, originally with the declared intention of treating the entire Soviet economy as a single vast optimization problem. It didn't work, and not just because the models were insanely complex and the necessary computer power didn't then exist. No algorithm can deal with the fact that people lie, and no algorithm exists, nor likely can ever exist, that can produce correct outputs given largely-lying inputs. (If only a few lie, you can deal with it: but in many command economies, espeically those that can resort to force when needed, there's an active incentive to lie to your superiors. So, in the end, most people will stretch the truth, and your beautiful optimizer fails ignominiously.)

I wish someone could find a way to globally optimize economies without the wastage inherent in competition, but then I also wish for immortality and FTL travel.