FreeLoader: Scavenging desktop storage resources for scientific data

Sudharshan S. Vazhkudai; Xiaosong Ma; Vincent W. Frech; Jonathan W. Strickland; Nandan Tammineedi; Stephen L. Scott

doi:10.1109/SC.2005.27

FreeLoader: Scavenging desktop storage resources for scientific data

Sudharshan S. Vazhkudai^*, Xiaosong Ma, Vincent W. Frech, Jonathan W. Strickland, Nandan Tammineedi, Stephen L. Scott

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

48 Citations (Scopus)

Abstract

High-end computing is suffering a data deluge from experiments, simulations, and apparatus that creates overwhelming application dataset sizes. End-user workstations - despite more processing power than ever before - are ill-equipped to cope with such data demands due to insufficient secondary storage space and I/O rates. Meanwhile, a large portion of desktop storage is unused. We present the FreeLoader framework, which aggregates unused desktop storage space and I/O bandwidth into a shared cache/scratch space, for hosting large, immutable datasets and exploiting data access locality. Our experiments show that FreeLoader is an appealing low-cost solution to storing massive datasets, by delivering higher data access rates than traditional storage facilities. In particular, we present novel data striping techniques that allow FreeLoader to efficiently aggregate a workstation's network communication bandwidth and local I/O bandwidth. In addition, the performance impact on the native workload of donor machines is small and can be effectively controlled.

Original language	English
Title of host publication	Proceedings - Thirteenth International Symposium on Temporal Representation and Reasoning, TIME 2006
DOIs	https://doi.org/10.1109/SC.2005.27
Publication status	Published - 2005
Externally published	Yes
Event	ACM/IEEE 2005 Supercomputing Conference, SC'05 - Seatle, WA, United States Duration: 12 Nov 2005 → 18 Nov 2005

Publication series

Name	Proceedings of the ACM/IEEE 2005 Supercomputing Conference, SC'05
Volume	2005

Conference

Conference	ACM/IEEE 2005 Supercomputing Conference, SC'05
Country/Territory	United States
City	Seatle, WA
Period	12/11/05 → 18/11/05

Keywords

Distributed storage
Parallel I/O
Scientific data management
Serverless storage system
Storage cache
Storage scavenging
Striped storage

Access to Document

10.1109/SC.2005.27

Cite this

Vazhkudai, S. S., Ma, X., Frech, V. W., Strickland, J. W., Tammineedi, N., & Scott, S. L. (2005). FreeLoader: Scavenging desktop storage resources for scientific data. In Proceedings - Thirteenth International Symposium on Temporal Representation and Reasoning, TIME 2006 Article 1560008 (Proceedings of the ACM/IEEE 2005 Supercomputing Conference, SC'05; Vol. 2005). https://doi.org/10.1109/SC.2005.27

@inproceedings{01c6809626c44fd4a3b080fb1f51cc44,

title = "FreeLoader: Scavenging desktop storage resources for scientific data",

abstract = "High-end computing is suffering a data deluge from experiments, simulations, and apparatus that creates overwhelming application dataset sizes. End-user workstations - despite more processing power than ever before - are ill-equipped to cope with such data demands due to insufficient secondary storage space and I/O rates. Meanwhile, a large portion of desktop storage is unused. We present the FreeLoader framework, which aggregates unused desktop storage space and I/O bandwidth into a shared cache/scratch space, for hosting large, immutable datasets and exploiting data access locality. Our experiments show that FreeLoader is an appealing low-cost solution to storing massive datasets, by delivering higher data access rates than traditional storage facilities. In particular, we present novel data striping techniques that allow FreeLoader to efficiently aggregate a workstation's network communication bandwidth and local I/O bandwidth. In addition, the performance impact on the native workload of donor machines is small and can be effectively controlled.",

keywords = "Distributed storage, Parallel I/O, Scientific data management, Serverless storage system, Storage cache, Storage scavenging, Striped storage",

author = "Vazhkudai, \{Sudharshan S.\} and Xiaosong Ma and Frech, \{Vincent W.\} and Strickland, \{Jonathan W.\} and Nandan Tammineedi and Scott, \{Stephen L.\}",

note = "Publisher Copyright: {\textcopyright} 2005 IEEE.; ACM/IEEE 2005 Supercomputing Conference, SC'05 ; Conference date: 12-11-2005 Through 18-11-2005",

year = "2005",

doi = "10.1109/SC.2005.27",

language = "English",

isbn = "1595930612",

series = "Proceedings of the ACM/IEEE 2005 Supercomputing Conference, SC'05",

booktitle = "Proceedings - Thirteenth International Symposium on Temporal Representation and Reasoning, TIME 2006",

}

Vazhkudai, SS, Ma, X, Frech, VW, Strickland, JW, Tammineedi, N & Scott, SL 2005, FreeLoader: Scavenging desktop storage resources for scientific data. in Proceedings - Thirteenth International Symposium on Temporal Representation and Reasoning, TIME 2006., 1560008, Proceedings of the ACM/IEEE 2005 Supercomputing Conference, SC'05, vol. 2005, ACM/IEEE 2005 Supercomputing Conference, SC'05, Seatle, WA, United States, 12/11/05. https://doi.org/10.1109/SC.2005.27

FreeLoader: Scavenging desktop storage resources for scientific data. / Vazhkudai, Sudharshan S.; Ma, Xiaosong; Frech, Vincent W. et al.
Proceedings - Thirteenth International Symposium on Temporal Representation and Reasoning, TIME 2006. 2005. 1560008 (Proceedings of the ACM/IEEE 2005 Supercomputing Conference, SC'05; Vol. 2005).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - FreeLoader

T2 - ACM/IEEE 2005 Supercomputing Conference, SC'05

AU - Vazhkudai, Sudharshan S.

AU - Ma, Xiaosong

AU - Frech, Vincent W.

AU - Strickland, Jonathan W.

AU - Tammineedi, Nandan

AU - Scott, Stephen L.

PY - 2005

Y1 - 2005

N2 - High-end computing is suffering a data deluge from experiments, simulations, and apparatus that creates overwhelming application dataset sizes. End-user workstations - despite more processing power than ever before - are ill-equipped to cope with such data demands due to insufficient secondary storage space and I/O rates. Meanwhile, a large portion of desktop storage is unused. We present the FreeLoader framework, which aggregates unused desktop storage space and I/O bandwidth into a shared cache/scratch space, for hosting large, immutable datasets and exploiting data access locality. Our experiments show that FreeLoader is an appealing low-cost solution to storing massive datasets, by delivering higher data access rates than traditional storage facilities. In particular, we present novel data striping techniques that allow FreeLoader to efficiently aggregate a workstation's network communication bandwidth and local I/O bandwidth. In addition, the performance impact on the native workload of donor machines is small and can be effectively controlled.

AB - High-end computing is suffering a data deluge from experiments, simulations, and apparatus that creates overwhelming application dataset sizes. End-user workstations - despite more processing power than ever before - are ill-equipped to cope with such data demands due to insufficient secondary storage space and I/O rates. Meanwhile, a large portion of desktop storage is unused. We present the FreeLoader framework, which aggregates unused desktop storage space and I/O bandwidth into a shared cache/scratch space, for hosting large, immutable datasets and exploiting data access locality. Our experiments show that FreeLoader is an appealing low-cost solution to storing massive datasets, by delivering higher data access rates than traditional storage facilities. In particular, we present novel data striping techniques that allow FreeLoader to efficiently aggregate a workstation's network communication bandwidth and local I/O bandwidth. In addition, the performance impact on the native workload of donor machines is small and can be effectively controlled.

KW - Distributed storage

KW - Parallel I/O

KW - Scientific data management

KW - Serverless storage system

KW - Storage cache

KW - Storage scavenging

KW - Striped storage

UR - https://www.scopus.com/pages/publications/33845399711

U2 - 10.1109/SC.2005.27

DO - 10.1109/SC.2005.27

M3 - Conference contribution

AN - SCOPUS:33845399711

SN - 1595930612

SN - 9781595930613

T3 - Proceedings of the ACM/IEEE 2005 Supercomputing Conference, SC'05

BT - Proceedings - Thirteenth International Symposium on Temporal Representation and Reasoning, TIME 2006

Y2 - 12 November 2005 through 18 November 2005

ER -

FreeLoader: Scavenging desktop storage resources for scientific data

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this