Big data analysis with skeletons on SOFA

Publikation: Bidrag til bog/antologi/rapport › Konferencebidrag i proceedings › Forskning › fagfællebedømt

Standard

Big data analysis with skeletons on SOFA. / Skovhede, Kenneth; Vinter, Brian.

Communicating Process Architectures 2017 and 2018, WoTUG-39 and WoTUG-40 - Proceedings of CPA 2017 (WoTUG-39) and Proceedings of CPA 2018 (WoTUG-40). red. / Jan Baekgaard Pedersen; Kevin Chalmers; Jan F. Broenink; Brian Vinter; Kevin Vella; Peter H. Welch; Marc L. Smith; Kenneth Skovhede. IMIA and IOS Press, 2019. s. 5-17 (Concurrent Systems Engineering Series, Bind 70).

Publikation: Bidrag til bog/antologi/rapport › Konferencebidrag i proceedings › Forskning › fagfællebedømt

Harvard

Skovhede, K & Vinter, B 2019, Big data analysis with skeletons on SOFA. i JB Pedersen, K Chalmers, JF Broenink, B Vinter, K Vella, PH Welch, ML Smith & K Skovhede (red), Communicating Process Architectures 2017 and 2018, WoTUG-39 and WoTUG-40 - Proceedings of CPA 2017 (WoTUG-39) and Proceedings of CPA 2018 (WoTUG-40). IMIA and IOS Press, Concurrent Systems Engineering Series, bind 70, s. 5-17, 39th WoTUG Conference on Communicating Process Architectures, CPA 2017 and 40th WoTUG Conference on Communicating Process Architectures, CPA 2018, Dresden, Tyskland, 19/08/2018. https://doi.org/10.3233/978-1-61499-949-2-5

APA

Skovhede, K., & Vinter, B. (2019). Big data analysis with skeletons on SOFA. I J. B. Pedersen, K. Chalmers, J. F. Broenink, B. Vinter, K. Vella, P. H. Welch, M. L. Smith, & K. Skovhede (red.), Communicating Process Architectures 2017 and 2018, WoTUG-39 and WoTUG-40 - Proceedings of CPA 2017 (WoTUG-39) and Proceedings of CPA 2018 (WoTUG-40) (s. 5-17). IMIA and IOS Press. Concurrent Systems Engineering Series Bind 70 https://doi.org/10.3233/978-1-61499-949-2-5

Vancouver

Skovhede K, Vinter B. Big data analysis with skeletons on SOFA. I Pedersen JB, Chalmers K, Broenink JF, Vinter B, Vella K, Welch PH, Smith ML, Skovhede K, red., Communicating Process Architectures 2017 and 2018, WoTUG-39 and WoTUG-40 - Proceedings of CPA 2017 (WoTUG-39) and Proceedings of CPA 2018 (WoTUG-40). IMIA and IOS Press. 2019. s. 5-17. (Concurrent Systems Engineering Series, Bind 70). https://doi.org/10.3233/978-1-61499-949-2-5

Author

Skovhede, Kenneth ; Vinter, Brian. / Big data analysis with skeletons on SOFA. Communicating Process Architectures 2017 and 2018, WoTUG-39 and WoTUG-40 - Proceedings of CPA 2017 (WoTUG-39) and Proceedings of CPA 2018 (WoTUG-40). red. / Jan Baekgaard Pedersen ; Kevin Chalmers ; Jan F. Broenink ; Brian Vinter ; Kevin Vella ; Peter H. Welch ; Marc L. Smith ; Kenneth Skovhede. IMIA and IOS Press, 2019. s. 5-17 (Concurrent Systems Engineering Series, Bind 70).

Bibtex

@inproceedings{13200a5a1db346298cdfbb54b2c3ef22,
title = "Big data analysis with skeletons on SOFA",
abstract = "This paper explores how a skeleton based approach can be used to perform big data analysis. We introduce a restricted storage system based on blocks with a fixed maximum size. The storage design removes the residual data problem commonly found in storage systems, and enables processing on individual blocks. We then introduce a stream-oriented query system that can be used on top of the distributed storage system. The query system is built on a limited number of core operations. Each of the perform a specified function, such as filtering elements, but are skeleton operations where the programmer needs to fill in how to perform the operation. The operations are designed to allow splitting across the blocks in the storage system, giving concurrent execution while maintaining a completely sequential program description. To assist in understanding the data flow, we also introduce a graphical representation for each of the methods, enabling a visual expression of an algorithm. To evaluate the query system we implement a number of classic Big-Data queries and show how to implement them with code, and how the queries can be visualized with the graphical representation.",
keywords = "Big data, CSP, Process oriented programming, SODA, SOFA",
author = "Kenneth Skovhede and Brian Vinter",
year = "2019",
doi = "10.3233/978-1-61499-949-2-5",
language = "English",
series = "Concurrent Systems Engineering Series",
publisher = "IMIA and IOS Press",
pages = "5--17",
editor = "Pedersen, {Jan Baekgaard} and Kevin Chalmers and Broenink, {Jan F.} and Brian Vinter and Kevin Vella and Welch, {Peter H.} and Smith, {Marc L.} and Kenneth Skovhede",
booktitle = "Communicating Process Architectures 2017 and 2018, WoTUG-39 and WoTUG-40 - Proceedings of CPA 2017 (WoTUG-39) and Proceedings of CPA 2018 (WoTUG-40)",
note = "39th WoTUG Conference on Communicating Process Architectures, CPA 2017 and 40th WoTUG Conference on Communicating Process Architectures, CPA 2018 ; Conference date: 19-08-2018 Through 22-08-2018",

}

RIS

TY - GEN

T1 - Big data analysis with skeletons on SOFA

AU - Skovhede, Kenneth

AU - Vinter, Brian

PY - 2019

Y1 - 2019

N2 - This paper explores how a skeleton based approach can be used to perform big data analysis. We introduce a restricted storage system based on blocks with a fixed maximum size. The storage design removes the residual data problem commonly found in storage systems, and enables processing on individual blocks. We then introduce a stream-oriented query system that can be used on top of the distributed storage system. The query system is built on a limited number of core operations. Each of the perform a specified function, such as filtering elements, but are skeleton operations where the programmer needs to fill in how to perform the operation. The operations are designed to allow splitting across the blocks in the storage system, giving concurrent execution while maintaining a completely sequential program description. To assist in understanding the data flow, we also introduce a graphical representation for each of the methods, enabling a visual expression of an algorithm. To evaluate the query system we implement a number of classic Big-Data queries and show how to implement them with code, and how the queries can be visualized with the graphical representation.

AB - This paper explores how a skeleton based approach can be used to perform big data analysis. We introduce a restricted storage system based on blocks with a fixed maximum size. The storage design removes the residual data problem commonly found in storage systems, and enables processing on individual blocks. We then introduce a stream-oriented query system that can be used on top of the distributed storage system. The query system is built on a limited number of core operations. Each of the perform a specified function, such as filtering elements, but are skeleton operations where the programmer needs to fill in how to perform the operation. The operations are designed to allow splitting across the blocks in the storage system, giving concurrent execution while maintaining a completely sequential program description. To assist in understanding the data flow, we also introduce a graphical representation for each of the methods, enabling a visual expression of an algorithm. To evaluate the query system we implement a number of classic Big-Data queries and show how to implement them with code, and how the queries can be visualized with the graphical representation.

KW - Big data

KW - CSP

KW - Process oriented programming

KW - SODA

KW - SOFA

U2 - 10.3233/978-1-61499-949-2-5

DO - 10.3233/978-1-61499-949-2-5

M3 - Article in proceedings

AN - SCOPUS:85082385381

T3 - Concurrent Systems Engineering Series

SP - 5

EP - 17

BT - Communicating Process Architectures 2017 and 2018, WoTUG-39 and WoTUG-40 - Proceedings of CPA 2017 (WoTUG-39) and Proceedings of CPA 2018 (WoTUG-40)

A2 - Pedersen, Jan Baekgaard

A2 - Chalmers, Kevin

A2 - Broenink, Jan F.

A2 - Vinter, Brian

A2 - Vella, Kevin

A2 - Welch, Peter H.

A2 - Smith, Marc L.

A2 - Skovhede, Kenneth

PB - IMIA and IOS Press

T2 - 39th WoTUG Conference on Communicating Process Architectures, CPA 2017 and 40th WoTUG Conference on Communicating Process Architectures, CPA 2018

Y2 - 19 August 2018 through 22 August 2018

ER -

ID: 241091096