Programmable Cyberinfrastructure: Introduction to Building Clusters in the Cloud

Abstract

Cloud computing is growing area for educating students and performing meaningful scientific research. The challenge for many educators and researchers is knowing how to use some of the unique aspects of computing in the cloud. One key feature is true elastic computing - resources on demand. This can be as simple as being able to provision a single purpose-driven virtual machine by hand quickly and easily. It can be taken a step further into creating scripted launch of additional resources as needed. Beyond that, elastic computing techniques can go to the next level and create modest virtual clusters on demand. While cloud resources won't replace traditional HPC environments for large research projects, there are many smaller research and education projects that would benefit from the highly customizable, highly configurable, programmable cyberinfrastructure afforded by cloud computing environments. This tutorial will discuss the basic methods required for interacting with elastic computing environments. It will then show a hands-on approach to creating virtual clusters in an Openstack environment. Please come prepared with a laptop with working wireless internet, and the following packages installed: Python 2.7 or later, setuptools, pip (the following pip packages may be installed in a virtualenv if you are comfortable with that!), "pip install python-glanceclient python-cinderclient python-openstackclient python-novaclient python-neutronclient python-keystoneclient python-heatclient shade". The goal is to have a working openstack client interface on your machine; for more details, see: https://docs.openstack.org/user-guide/common/cli-install-openstack-command-line-clients.html Required reading: It would be useful to explore the Jetstream wiki: https://wiki.jetstream-cloud.org - Particularly the sections on using the Jetstream API! We will provide training accounts and working openrc.sh files during the session!