Management of an Academic HPC Cluster: The UL Experience

in Proc. of the 2014 Intl. Conf. on High Performance Computing Simulation (HPCS 2014) (2014, July)

The intensive growth of processing power, data storage and transmission capabilities has revolutionized many aspects of science. These resources are essential to achieve high-quality results in many ... [more ▼]

The intensive growth of processing power, data storage and transmission capabilities has revolutionized many aspects of science. These resources are essential to achieve high-quality results in many application areas. In this context, the University of Luxembourg (UL) operates since 2007 an High Performance Computing (HPC) facility and the related storage. The aspect of bridging computing and storage is a requirement of UL service – the reasons are both legal (certain data may not move) and performance related. Nowa- days, people from the three faculties and/or the two Interdisciplinary centers within the UL, are users of this facility. More specifically, key research priorities such as Systems Bio-medicine (by LCSB) and Security, Reliability & Trust (by SnT) require access to such HPC facilities in order to function in an adequate environment. The management of HPC solutions is a complex enterprise and a constant area for discussion and improvement. The UL HPC facility and the derived deployed services is a complex computing system to manage by its scale: at the moment of writing, it consists of 150 servers, 368 nodes (3880 computing cores) and 1996 TB of shared raw storage which are all configured, monitored and operated by three per- sons using advanced IT automation solutions based on Puppet [1], FAI [2] and Capistrano [3]. This paper covers all the aspects in relation to the management of such a complex infrastructure, whether technical or administrative. Most design choices or implemented approaches have been motivated by several years of experience in addressing research needs, mainly in the HPC area but also in complementary services (typically Web-based). In this context, we tried to answer in a flexible and convenient way many technological issues. This experience report may be of interest for other research centers belonging either to the public or the private sector looking for good if not best practices in cluster architecture and management. [less ▲]

Computational infrastructures for data and knowledge management in systems biology

in Prokop, Ales; Csukás, Bela (Eds.) Systems Biology: Integrative Biology and Simulation Tools (2013)

The volume, complexity and heterogeneity of data originating from high throughput functional genomics technologies have created challenges and opportunities for Information Technology (IT) departments ... [more ▼]

The volume, complexity and heterogeneity of data originating from high throughput functional genomics technologies have created challenges and opportunities for Information Technology (IT) departments. These increased demands have also led to increasing costs for IT infrastructure such as necessary computing power and storage devices, as well as further costs for manpower effort, required for maintenance. This chapter describes some of the challenges for computational analysis infrastructure, including bottlenecks and most pressing needs that have to be addressed to effectively support the development of systems biology and its application in medicine. [less ▲]