OATAO - Open Archive Toulouse Archive Ouverte Open Access Week

On Processing Extreme Data

Petcu, Dana and Iuhasz, Gabriel and Pop, Daniel and Talia, Domenico and Carretero, Jesus and Prodan, Radu and Fahringer, Thomas and Grasso, Ivan and Doallo, Ramon and Martin, Maria J. and Fraguela, Basilio B. and Trobec, Roman and Depolli, Matjaz and Almeida Rodriguez, Francisco and De Sande, Francisco and Da Costa, Georges and Pierson, Jean-Marc and Anastasiadis, Stergios and Bartzokas, Aristides and Lolis, Christos and Gonçalves, Pedro and Brito, Fabrice and Brown, Nick On Processing Extreme Data. (2016) Scalable computing : Practice and Experience, 16 (4). 467-489. ISSN 2194-6876

(Document in English)

PDF (Author's version) - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader

Official URL: http://dx.doi.org/10.12694/scpe.v16i4.1134


Extreme Data is an incarnation of Big Data concept distinguished by the massive amounts of data that must be queried, communicated and analyzed in near real-time by using a very large number of memory or storage elements and exascale computing systems. Immediate examples are the scientific data produced at a rate of hundreds of gigabits-per-second that must be stored, filtered and analyzed, the millions of images per day that must be analyzed in parallel, the one billion of social data posts queried in real-time on an in-memory components database. Traditional disks or commercial storage nowadays cannot handle the extreme scale of such application data. Following the need of improvement of current concepts and technologies, we focus in this paper on the needs of data intensive applications running on systems composed of up to millions of computing elements (exascale systems). We propose in this paper a methodology to advance the state-of-the-art. The starting point is the definition of new programming paradigms, APIs, runtime tools and methodologies for expressing data-intensive tasks on exascale systems. This will pave the way for the exploitation of massive parallelism over a simplified model of the system architecture, thus promoting high performance and efficiency, offering powerful operations and mechanisms for processing extreme data sources at high speed and/or real time.

Item Type:Article
Additional Information:Thanks to FCPE, Scalable Computing: Practice and Experience, Scientific International Journal for Parallel and Distributed Computing. ISSN 2194-6876 The definitive version is available at http://www.scpe.org/index.php/scpe/article/view/1134
HAL Id:hal-01524718
Audience (journal):International peer-reviewed journal
Uncontrolled Keywords:
Institution:French research institutions > Centre National de la Recherche Scientifique - CNRS (FRANCE)
Université de Toulouse > Institut National Polytechnique de Toulouse - Toulouse INP (FRANCE)
Université de Toulouse > Université Toulouse III - Paul Sabatier - UT3 (FRANCE)
Université de Toulouse > Université Toulouse - Jean Jaurès - UT2J (FRANCE)
Université de Toulouse > Université Toulouse 1 Capitole - UT1 (FRANCE)
Other partners > Institut "Jožef Stefan" - IJS (SLOVENIA)
Other partners > Terradue (ITALY)
Other partners > Università della Calabria (ITALY)
Other partners > University of Edinburgh (UNITED KINGDOM)
Other partners > University of Innsbruck (AUSTRIA)
Other partners > Universidad Carlos III de Madrid - UC3M (SPAIN)
Other partners > Universidad de La Laguna - ULL (SPAIN)
Other partners > Universidade da Coruña - UDC (SPAIN)
Other partners > University of Ioannina - UoI (GREECE)
Other partners > West University of Timisoara (ROMANIA)
Laboratory name:
Deposited On:27 Apr 2017 15:45

Repository Staff Only: item control page