Computation and storage in the cloud: understanding the trade-offs

Front Cover......Page 1
Computation and Storage in the Cloud......Page 4
Copyright Page......Page 5
Contents......Page 6
Acknowledgements......Page 10
About the Authors......Page 12
Preface......Page 14
1.1 Scientific Applications in the Cloud......Page 16
1.3 Overview of This Book......Page 18
2.1 Data Management of Scientific Applications in Traditional Distributed Systems......Page 20
2.1.1 Data Management in Grid......Page 21
2.1.2 Data Management in Grid Workflows......Page 23
2.1.3 Data Management in Other Distributed Systems......Page 24
2.2.1 Cost-Effectiveness of Deploying Scientific Applications in the Cloud......Page 25
2.2.2 Trade-Off Between Computation and Storage in the Cloud......Page 26
2.4 Summary......Page 27
3.1 Motivating Example......Page 30
3.2.1 Requirements and Challenges of Deploying Scientific Applications in the Cloud......Page 32
3.2.2 Bandwidth Cost of Deploying Scientific Applications in the Cloud......Page 33
3.3.1 Cost Model for Data Set Storage in the Cloud......Page 34
3.3.3 Cost-Effective Storage Strategies......Page 35
3.4 Summary......Page 36
4.2 Data Provenance and DDG......Page 38
4.3 Data Set Storage Cost Model in the Cloud......Page 40
4.4 Summary......Page 42
5 Minimum Cost Benchmarking Approaches......Page 44
5.1.1 CTT-SP Algorithm for Linear DDG......Page 45
5.1.2 Minimum Cost Benchmarking Algorithm for DDG with One Block......Page 47
5.1.2.1 Constructing CTT for DDG with One Block......Page 48
5.1.2.2 Setting Weights to Different Types of Edges......Page 49
5.1.2.3 Steps of Finding MCSS for DDG with One Sub-Branch in One Block......Page 51
5.1.3.1 General CTT-SP Algorithm for Different Situations......Page 53
5.1.3.2 Pseudo-Code of General CTT-SP Algorithm......Page 54
5.2 Dynamic On-the-Fly Minimum Cost Benchmarking Approach......Page 58
5.2.1.1 Different MCSSs of a DDG_LS in a Solution Space......Page 59
5.2.1.2 Range of MCSSs’ Cost Rates for a DDG_LS......Page 60
5.2.1.3 Distribution of MCSSs in the PSS of a DDG_LS......Page 62
5.2.2 Algorithms for Calculating PSS of a DDG_LS......Page 65
5.2.3 PSS for a General DDG (or DDG Segment)......Page 68
5.2.3.1 Three-Dimensional PSS of DDG Segment with Two Branches......Page 69
5.2.3.2 High-Dimensional PSS of a General DDG......Page 71
5.2.4.1 Minimum Cost Benchmarking by Merging and Saving PSSs in a Hierarchy......Page 73
5.2.4.2 Updating of the Minimum Cost Benchmark on the Fly......Page 76
5.3 Summary......Page 79
6.1 Data-Accessing Delay and Users’ Preferences in Storage Strategies......Page 80
6.2 Cost-Rate-Based Storage Strategy......Page 81
6.2.1.1 Algorithm for Deciding Newly Generated Data Sets’ Storage Status......Page 82
6.2.1.3 Algorithm for Deciding Regenerated Data Sets’ Storage Status......Page 83
6.3 Local-Optimisation-Based Storage Strategy......Page 84
6.3.1.1 Enhanced CTT-SP Algorithm for Linear DDG......Page 85
6.3.1.2 Rules in the Strategy......Page 87
6.3.2 Cost-Effectiveness Analysis......Page 88
6.4 Summary......Page 89
7.2 Evaluation of Minimum Cost Benchmarking Approaches......Page 90
7.2.1 Cost-Effectiveness Evaluation of the Minimum Cost Benchmark......Page 91
7.2.2 Efficiency Evaluation of Two Benchmarking Approaches......Page 92
7.3.1 Cost-Effectiveness of Two Storage Strategies......Page 97
7.3.2 Efficiency Evaluation of Two Storage Strategies......Page 99
7.4.1 Utilisation of Minimum Cost Benchmarking Approaches......Page 101
7.4.2 Utilisation of Cost-Effective Storage Strategies......Page 102
7.5 Summary......Page 105
8.1 Summary of This Book......Page 106
8.2 Key Contributions of This Book......Page 107
Appendix A: Notation Index......Page 110
Appendix B: Proofs of Theorems, Lemmas and Corollaries......Page 112
Appendix C: Method of Calculating λ Based on Users’ Extra Budget......Page 122
Bibliography......Page 124