Main Image
Volume 299 - The 7th International Conference on Computer Engineering and Networks (CENet2017) - Session II - Wireless communication
A Load Balancing Strategy Based on Multiple Metrics in Spatial Image File Access Process in Hadoop
L. Yuan,* Y. Feng, B. Li, F. Li
*corresponding author
Full text: pdf
Pre-published on: 2017 July 17
Published on: 2017 September 06
To solve the problem of load imbalance when accessing spatial image files in HDFS, we present a loading capacity model based on multiple attributes. In our model, five attributes, including
disk space load capacity, CPU processing capacity, memory processing capacity, disk read-write processing capacity and bandwidth, are introduced to measure the load balancing of each node. Then we used the objective weighting method of variation coefficient method to process experiment data, calculated the weight coefficient of the five attributes and obtained a load
capacity model. By this way, the nodes can calculate the actual load ability of their own, which will help improve the efficiency of the load balance when accessing spatial image files in HDFS.
To validate the proposed model, a large number of spatial image files are adopted to be accessed in our experiments. Experiment results show that, when accessing the spatial image files in HDFS, better load balance ability, system performance and efficiency can be obtained by using the strategy of the proposed multi-attribute load balance model.
Open Access
Creative Commons LicenseCopyright owned by the author(s) under the term of the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.