Event-driven, collaborative and adaptive scientific workflows on the grid
Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review
Standard
Event-driven, collaborative and adaptive scientific workflows on the grid. / Bardino, Jonas; Rehr, Martin; Vinter, Brian.
Communicating Process Architectures 2017 and 2018, WoTUG-39 and WoTUG-40 - Proceedings of CPA 2017 (WoTUG-39) and Proceedings of CPA 2018 (WoTUG-40). ed. / Jan Baekgaard Pedersen; Kevin Chalmers; Jan F. Broenink; Brian Vinter; Kevin Vella; Peter H. Welch; Marc L. Smith; Kenneth Skovhede. IMIA and IOS Press, 2019. p. 61-80 (Concurrent Systems Engineering Series, Vol. 70).Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review
Harvard
APA
Vancouver
Author
Bibtex
}
RIS
TY - GEN
T1 - Event-driven, collaborative and adaptive scientific workflows on the grid
AU - Bardino, Jonas
AU - Rehr, Martin
AU - Vinter, Brian
PY - 2019
Y1 - 2019
N2 - Exponential growth in scientific data set sizes and corresponding computation needs, forces scientists and engineers to structure and automate experiments in workflows running on distributed architectures. In eScience the flows typically evolve gradually from intensive experimentation and often involve multiple participants from separate organisations. Thus, there's a need for infrastructures supporting such highly dynamic and collaborative workflows. Despite much attention to scientific workflows in recent years, most existing systems tend to be single-user top-down approaches, which are inherently best suited for static and fixed flows with all steps known up front. In this work we introduce a simple general rule-based model for event-driven workflows based on data change triggers. A bottom-up workflow approach, that enables a high level of automation and allows dynamically changing flows - with or without manual user interaction. It is realised with an implementation on top of the Minimum intrusion Grid (MiG), which helps de-couple workflow design from underlying execution concerns, and provides built-in collaboration and sharing across organisation boundaries. However, the model itself applies to much wider range of scenarios, and other such possible implementation methods are briefly outlined.
AB - Exponential growth in scientific data set sizes and corresponding computation needs, forces scientists and engineers to structure and automate experiments in workflows running on distributed architectures. In eScience the flows typically evolve gradually from intensive experimentation and often involve multiple participants from separate organisations. Thus, there's a need for infrastructures supporting such highly dynamic and collaborative workflows. Despite much attention to scientific workflows in recent years, most existing systems tend to be single-user top-down approaches, which are inherently best suited for static and fixed flows with all steps known up front. In this work we introduce a simple general rule-based model for event-driven workflows based on data change triggers. A bottom-up workflow approach, that enables a high level of automation and allows dynamically changing flows - with or without manual user interaction. It is realised with an implementation on top of the Minimum intrusion Grid (MiG), which helps de-couple workflow design from underlying execution concerns, and provides built-in collaboration and sharing across organisation boundaries. However, the model itself applies to much wider range of scenarios, and other such possible implementation methods are briefly outlined.
KW - Adaptive
KW - Collaborative
KW - EScience
KW - Event-driven
KW - Grid
KW - Workflows
U2 - 10.3233/978-1-61499-949-2-61
DO - 10.3233/978-1-61499-949-2-61
M3 - Article in proceedings
AN - SCOPUS:85082401950
T3 - Concurrent Systems Engineering Series
SP - 61
EP - 80
BT - Communicating Process Architectures 2017 and 2018, WoTUG-39 and WoTUG-40 - Proceedings of CPA 2017 (WoTUG-39) and Proceedings of CPA 2018 (WoTUG-40)
A2 - Pedersen, Jan Baekgaard
A2 - Chalmers, Kevin
A2 - Broenink, Jan F.
A2 - Vinter, Brian
A2 - Vella, Kevin
A2 - Welch, Peter H.
A2 - Smith, Marc L.
A2 - Skovhede, Kenneth
PB - IMIA and IOS Press
T2 - 39th WoTUG Conference on Communicating Process Architectures, CPA 2017 and 40th WoTUG Conference on Communicating Process Architectures, CPA 2018
Y2 - 19 August 2018 through 22 August 2018
ER -
ID: 241091303