Higher Degree by Research Application Portal
Title | Space filling curve based optimization for multi-D data distribution |
---|---|
Supervisor | Prof Andreas Wicenec |
Course | Doctor of Philosophy |
Research area | Physical Sciences |
Project description | Radio astronomy is producing huge multi-dimensional data cubes, which require the application of complex computational workflows on High Performance Compute (HPC) clusters. These workflows require optimization of I/O. Even when assuming that we would be able to load all the data into the cluster memory, there will be enormous amounts of inter and cross-node communication required to get the data from one process to the next. HPC clusters are inherently non homogeneous in terms of I/O performance within a single computer (due to NUMA architectures) and worse across multiple computers due to the network topology of the cluster and even worse when it comes to accessing data on the (shared), multi-level file system. Access patterns in typical radio astronomy algorithms on the other side are highly dependent on the algorithms applied to the data and thus ultimately on the science goals. These access patterns can differ from the one extreme of being perfectly aligned with the native order of the data cube to the other extreme of being orthogonal to it. Having such an issue on a single machine has severe impacts on overall performance already, but on a cluster it might lead to a situation where every worker of a N times distributed algorithm initiates access to pieces of data residing on many, if not all, nodes of the cluster. In this project we will investigate the effect of such conflicting access patterns and try to address the issue by distributing the multi-dimensional data using so called space filling curves such like Hilbert curves. Thus it is necessary to identify at least the minimal set of parameters required to adjust the data distribution to differing target HPC systems. |
Opportunity status | Closed |
Open date | 01 Aug 2024 |
Close date | 28 Sep 2024 |
School | Graduate Research School |
Contact | Professor Andreas Wicenec | andreas.wicenec@uwa.edu.au |
Additional information | We are interested to hear from potential candidates from a computer science or software engineering background with a firm interest in applying this expertise to scientific exploration and knowledge extraction. People with a background in other sciences, but with a solid knowledge of software development practices and tools would be equally suited. The candidate would join an active multi-disciplinary research and development group with many scientific and commercial cross fertilisation possibilities as well as excellent international collaborations. The work will be in collaboration with the American Oak Ridge Laboratory, one of the leading institutions in high performance computing. |
Course type | Doctorates |
Description | The Doctor of Philosophy (PhD) is a program of independent, supervised research that is assessed solely on the basis of a thesis, sometimes including a creative work component, that is examined externally. The work presented for a PhD must be a substantial and original contribution to scholarship, demonstrating mastery of the subject of interest as well as an advance in that field of knowledge. Visit the course webpage for full details of this course including admission requirements, course rules and the relevant CRICOS code/s. |
Duration | 4 years |
Guidance
Computer Science
Data Science
Astronomy
SKA