The Inter-university Institute for Data Intensive Astronomy (IDIA) is a partnership of three South African universities, the Universities of Cape Town, of the Western Cape and of Pretoria. IDIA also has an industry partner in SAP. The overarching goal of IDIA is to build within the South African university research community the capacity and expertise in data intensive research to enable global leadership on MeerKAT large survey science projects and large projects on other SKA pathfinder telescopes, leading to leadership on SKA phase 1 Key Science programs.
One of the first elements to reach this goal was for IDIA to set up a data-intensive research cloud facility to service its scientific community. Currently, IDIA is the primary platform to service five out of eight MeerKAT large survey projects.
Big scientific data changes the way we do science. Scientists themselves have to change their habits and approaches when data have to live in the cloud. Platforms like IDIA’s research cloud also need to enable scientists to keep applying the scientific method; visualising their data, re-processing it, testing hypotheses on it, etc. without having to wait for weeks for results because the data sets are so big.
Cloud technology also opens the door to real-time collaboration across the globe as analysis tools live in the cloud and can be run by collaborators on the same platform. Scientists also need assistance from technologies such as machine learning and artificial intelligence as the data are way too vast to look at manually.
The Inter-University Institute for Data Intensive Astronomy is here to answer some of those questions and facilitate research in the age of astronomical quantities of scientific data.