OATAO - Open Archive Toulouse Archive Ouverte Open Access Week

Modulo Based Data Placement Algorithm for Energy Consumption Optimization of MapReduce System

Song, Jie and He, HongYan and Wang, Zhi and Yu, Ge and Pierson, Jean-Marc Modulo Based Data Placement Algorithm for Energy Consumption Optimization of MapReduce System. (2016) Journal of Grid Computing, 1. 1-16. ISSN 1570-7873

(Document in English)

PDF (Author's version) - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader

Official URL: http://dx.doi.org/10.1007/s10723-016-9370-2


With the explosion of data production, the efficiency of data management and analysis has been concerned by both industry and academia. Meanwhile, more and more energy is consumed by the IT infrastructure especially the larger scale distributed systems. In this paper, a novel idea for optimizing the Energy Consumption (EC for short) of MapReduce system is proposed. We argue that a fair data placement is helpful to save energy, and then we propose three goals of data placement, and a modulo based Data Placement Algorithm (DPA for short) which achieves these goals. Afterwards, the correctness of the proposed DPA is proved from both theoretical and experimental perspectives. Three different systems which implement MapReduce model with different DPAs are compared in our experiments. Our algorithm is proved to optimize EC effectively, without introducing the additional costs and delaying data loading. With the help of our DPA, the EC for the WordCount , Sort and MRBench can be reduced by 10.9 %, 8.3 % and 17 % respectively, and time consumption is reduced by 7 %, 6.3 % and 7 % respectively.

Item Type:Article
Additional Information:Thanks to Springer editor. This papers appears in Journal of Grid Computing ISSN 1570-7873 The original PDF is available at: http://link.springer.com/article/10.1007/s10723-016-9370-2
HAL Id:hal-01514611
Audience (journal):International peer-reviewed journal
Uncontrolled Keywords:
Institution:French research institutions > Centre National de la Recherche Scientifique - CNRS (FRANCE)
Université de Toulouse > Institut National Polytechnique de Toulouse - Toulouse INP (FRANCE)
Université de Toulouse > Université Toulouse III - Paul Sabatier - UT3 (FRANCE)
Université de Toulouse > Université Toulouse - Jean Jaurès - UT2J (FRANCE)
Université de Toulouse > Université Toulouse 1 Capitole - UT1 (FRANCE)
Other partners > Northeastern University (CHINA)
Laboratory name:
Deposited On:03 Apr 2017 15:13

Repository Staff Only: item control page