QRIScloud Compute Cloud
Our expanded compute cloud is up and running providing large amounts of additional capacity and a number of specialised facilities. The service went live for Queensland researchers during June 2014 and was added to the NeCTAR Production Federation on 20 October 2014.
As well as a eight-fold expansion in the number of cores available, the quality of service from the expanded cloud has improved considerably. The 1.6 times over-commitment of cores from the original stage 1 facility has been removed.
Researchers now have access to:
QRIScloud has installed some GPU cores and these will be made available to researchers in the new year.
- Almost 4,000 regular cores
- 4 nodes (4x64 cores) with large (1 TB) memories.
The regular cores can be accessed here, or see nectar.org.au for more information. If you need your cores to be allocated at the QRIScloud node choose QRIScloud as your Availability Zone. For special requirements and assistance with the specialised cores of the compute cloud, please talk to your local eResearch Analyst.
QRIScloud Data Storage
During November and December the QRIScloud data storage facilities in Brisbane are being enhanced to reach their full planned 2014 capacity. Researchers can expect to be able to use the greatly expanded facilities to store data from January 2015. Talk to your local eResearch Analyst about the use of the new capabilities.
The Townsville data storage facility was expanded by 1.6 usable petabytes of high-performance disk in October bringing the total QRIScloud storage in Townsville up to about two petabytes. This storage is available now and applications are being received by QRIScloud’s eResearch Analysts.
In Brisbane the total QRIScloud storage will consist of six usable petabytes of high-performance disk and nine petabytes of tape. The tape systems are organised using HSM (hierarchical storage management). The HSM system will accelerate tape performance by staging active data collections onto disk for rapid user access. This is a huge upgrade in capacity providing the storage to house more than double the allocations that have been granted to date.
QRIScloud is designed to replicate all primary data stored there onto tapes, providing data security for QRIScloud users. The design provides for up to two replicas of primary data, with one of the replicas stored at an offsite data centre. The total tape capacity for primary and replica data in use in January will exceed 40 petabytes.
Data Access Facilities
Following the commissioning of the data storage QRIScloud will speed up ingest. This is particularly important for all the collections on the backlog we have had to manage because of the limited interim storage available.
Alongside the ingest program QRIScloud will add layers of data access services designed to open up access to QRIScloud data to researchers. These will include:
They are being installed now in both Brisbane and Townsville, and QRIScloud will be conducting tests over early 2015 before releasing them to researchers. More details of these services will be revealed in future NewsFeeds.
- User authentication based on the AAF (Australian Access Federation) service that is already used widely across the Australian research sector
- A data asset management system known as Mediaflux or LiveArc that makes it easy to store and manage data, to discover data stored and to access the data
- A rapid data ingest and movement system known as Aspera. Aspera can be tuned to use as much of the available bandwidth as permitted so that large amounts of data can be transmitted to and from QRIScloud as quickly as possible.
Consolidation at Polaris
Brisbane-based QRIScloud facilities will be consolidated at the Dell Polaris data centre in Springfield during the early part of 2015. Virtual machines and collections currently operating on the equipment housed in DC2 (data centre 2) at St Lucia will be transitioned to the Polaris facilities, and, once this has been completed, the DC2-based equipment will be moved to Polaris and re-commissioned. This will provide an excellent modern professional production environment for the QRIScloud facilities with high-speed connections to campuses and offsite replica storage.
Thanks to the Development Team
The creation of QRIScloud would not have been possible without a tremendous amount of dedicated work by the teams at QCIF, and at our data centre suppliers and operators, UQ and JCU, and their expert teams. There have been many people involved but David Green and Jay van Schyndel deserve special mention.
QCIF would also like to thank the NeCTAR and RDSI Projects for providing the Commonwealth Department of Education NCRIS and EIF funding to make QRIScloud possible, and the Queensland Government Department of Science IT Innovation and the Arts (DSITIA) for the Co-investment Fund grant that supported the development and operation of QRIScloud.
Thanks also to our major suppliers, Dell, SGI and AARNet for their efforts over and above the call of duty to deliver, install and commission the data centre, networks, capital equipment and software to meet often demanding milestones.
And thanks to Queensland’s researchers who have already demonstrated how much can be achieved when they have available easy-to-acquire and easy-to-use high-performance cloud computing capacity available to them. Your enthusiasm in adopting QRIScloud facilities has been a great incentive to provide more and better facilities.
A happy Christmas and New Year to everyone. When you come back in January the complete QRIScloud capacity will be waiting for you.