Osaka University and NEC will demonstrate and exhibit a jointly developed data transfer prototype at SC24 in Atlanta

Tokyo & Osaka, Japan – Osaka University and NEC Corporation (NEC; TSE: 6701) are moving forward with efforts to realize a data infrastructure supporting Open Science. In 2021, The Joint Research Laboratory for Integrated Infrastructure of High Performance Computing and Data Analysis was established within the D3 Center, Osaka University (Director: Professor Susumu Date*1) by Osaka University and NEC. The result of the Joint Research Laboratory will be showcased at the International Conference for High Performance Computing, Networking, Storage, and Analysis (SC24) scheduled to be held from November 17 to 22, 2024 in Atlanta, Georgia. The experiment to be conducted will transfer huge research data between servers using a 100 Gbps line between Japan and the United States. One file of huge research data stored in the storage of one server is transferred to the other server at high speed. In the preliminary evaluation, we have confirmed that the transfer of 1TB of data can be completed in 87 seconds (effective 92.0Gbps) over a short distance. This experiment will confirm the performance of the long-distance transfer between Japan and the U.S.

Background
In recent years, universities and other research institutions have been strongly urged to promote Open Science enabling research data to be shared with society. At the G7 Science and Technology Ministers’ Meeting held in Sendai in May 2023, the G7 Science and Technology Ministers’ Communique was issued as a joint statement citing the need to promote Open Science as a shared common understanding among G7 countries and the EU. To this end, it is imperative to create a data infrastructure that enables the sharing of research data produced through day-to-day academic research among collaborating institutions, and makes the data public so that it can be utilized by various institutions.

The D3 Center, Osaka University, has conducted test trials since May 2021 for ONION (Osaka university Next-generation Infrastructure for Open research and innovatioN), a data aggregation platform that enables users to store research data and share it among collaborating researchers inside and outside the university. The Joint Research Laboratory has been working on the research and development of RED-ONION (Research EnhanceD ONION), which will upgrade ONION with a high-speed data transfer service.

Today, sharing large-scale research data among collaborative research groups is done over very slow networks and is quite time consuming. In some cases, the data is copied once onto a hard disk and physically transported. As a result, data sharing is very time-consuming and labor intensive, and has been an obstacle to the promotion of data-driven research. RED-ONION aims to support highly efficient academic research that handles large-scale data. RED-ONION provides a high-speed data transfer service that enables the rapid transfer and sharing of the research data generated within the university between major locations within the university. This will enhance the research data sharing within the Osaka University campus and significantly contribute to the productivity of today's academic research, which handles large-scale data and requires high-performance computing and high-performance data analysis.

Research Result
The Joint Research Laboratory is developing a prototype of a high-speed data transfer system that will enable the high-speed sharing of large-scale research data between research centers. This system will be centered on ONION, and will include high-speed networks, servers, and storage devices dedicated to transferring research data at major sites within the university. This system will not only enable the fast and efficient transfer between major sites but will also allow the data to be housed in the data aggregation platform ONION.

The research result is a system technology that streamlines the transfer of large-scale research data. It enables easy and high-speed transfer stored in the storage of one research center to the storage of another research center or to the shared storage of the D3 Center at a speed of approximately 100 Gbps. The feature of this technology is that a single large file stored in the storage can be transferred to the other storage with a single copy command at high speed. Preliminary evaluation has confirmed that a file consisting of 1TB can be transferred in 87 seconds (effective 92.0Gbps), bringing the 100Gbps network performance close to its limit.

Exhibit at SC24
At SC24, we will experiment a long-distance transfer of the high-speed data transfer system, and exhibit at the research exhibition booth of the National Institute of Information and Communications Technology (NICT *2). The purpose of this experiment is to confirm the performance of the system not only for sharing research data within the university, but also for sharing data with other universities and research organizations in Japan, as well as for long-distance data transfer with overseas research groups. Using the prototype developed, data transfer experiments between Japan and the U.S. and within the U.S., assuming international long-distance data transfer, will be conducted using the network and experimental environment established by NICT with the cooperation of NREN (*3) in Japan and abroad.

In developing this experiment and prototype, we received technical cooperation from CLEALINK TECHNOLOGY Co., Ltd., a developer of high-speed file transfer software Archaea. Super Micro Computer, Inc. provides the server equipment, DataDirect Networks Japan, Inc. provides the storage devices, and Macnica, Inc. provides some of the network equipment.

Notes:
*1 Joint Research Laboratory for Integrated Infrastructure of High Performance Computing and Data Analysis (https://www.nri.cmc.osaka-u.ac.jp/)
*2 National Institute of Information and Communications Technology (NICT): Japan's only public research institute specializing in the field of information and communications (https://www.nict.go.jp/)
*3 Organizations that cooperated in the construction of the Japan-U.S. experimental network: NII, SingAREN, Internet2, PacificWave, TransPAC, NICT
Organizations cooperating in the construction of the U.S.-U.S. experimental network: SCinet, Internet2, CENIC