Distributed Hash-tables for Scientific Computing

Distributed Hash-Tables are a common data-store in distributed applications and have proven to be useful in the context of webservices and Big Data applications.


In this project, we investigate how the field of High Performance Computing benefits from this technique. Therefore, we explore two research questions. The first one deals with the efficient implementation of a DHT. Compute-Clusters are equipped with High-performance networks like InfiniBand or Omni-Path and offer special communication APIs.


The second question is which scientific applications may benefit from DHTs? Therefore, we are adapting existing applications. This is done in cooperation with partners at the GFZ, Potsdam.


Publications

POET (v0.1): Speedup of Many-Cores Parallel Reactive Transport Simulations with Fast DHT-Lookups
Marco De Lucia, Michael Kühn, Alexander Lindemann, Max Lübke and Bettina Schnor
Geoscientific Model Development
May 2021


Talks

Combining machine learning and equation based models in reactive transport: POET
Marco De Lucia, Michael Kühn, Max Lübke, Bettina Schnor
Goldschmidt Conference
July 2023


Student Thesis

Einsatz und Bewertung von verschiedenen Key-Value-Stores zur Beschleunigung der gekoppelten hydrodynamisch-geochemischen Simulation POET
Nico Sauerbrei
Bachelor Thesis, University of Potsdam, 2023


Design, implementation and validation of an interpolation-based surrogate model for the POET simulator
Max Lübke
Master Thesis, University of Potsdam, 2023


Einsatz und Bewertung einer verteilten Hash-Table zur Beschleunigung von gekoppelten hydrodynamisch-geochemischen Simulationen
Max Lübke
Bachelor Thesis, University of Potsdam, 2020


Parallelisierung und Modularisierung einer Reaktiven-Transportsimulation
Alexander H. W. Lindemann
Master Thesis, University of Potsdam, 2019


Implementierung einer DistributedHashtable mittels MPI/OSC
Max Lübke
Student Thesis, University of Potsdam, 2018