The Utility of Exploiting Idle Memory for Data-IntensiveComputations

In this paper, we examine the utility of exploiting idle memory in workstationpools. We attempt to answer the following questions. First, given aworkstation pool, what fraction of the memory can be expected to be idle? Thisprovides an estimate of the opportunity for hosting guest data. Second, whatfraction of a individual host\'s memory can be expected to be idle? This helpsdetermine the recruitment policy -- what is the maximum amount of memory thatshould be recruited on a single host? Third, what is the distribution ofmemory idle-times? That is, what is the probability that a chunk of memorythat is currently idle will be idle for longer than time t? This informationindicates how long guest data can be expected to survive; applications thataccess their data-sets frequently within the expected life-time of guest dataare more likely to benefit from exploiting idle memory. Fourth, how muchbenefit can a user expect? We use two metrics for the benefit of exploitingidle memory: (1) if I have a pool with w workstations, how much memory shouldI expect to get for free by harvesting idle memory; (2) how much improvementcan be achieved in end-to-end execution time? Finally, how long and howfrequently might a user have to wait to reclaim her machine if she volunteersto host guest pages on her machine? This helps answer the question of socialacceptability. To answer the questions relating to the availability of idlememory, we have analyzed two-week long traces from five workstation pools withdifferent sizes, locations, and patterns of use. To evaluate the expectedbenefits and costs, we have simulated three data-intensive applications(0.5GB-5GB) on these workstation pools.