Jetstream User Guide
Last update: August 15, 2017

System Overview

Jetstream is a user-friendly cloud environment designed to give researchers access to interactive computing and data analysis resources on demand, whenever and wherever they want to analyze their data. It provides a library of virtual machines designed to do discipline specific scientific analysis. Software creators and researchers will also be able to create their own customized virtual machines (VM) or their own private computing system within Jetstream.

Jetstream features a web-based user interface based on the popular Atmosphere cloud computing environment developed by the CyVerse (formerly known as the iPlant Collaborative) and extended to support science and engineering research generally. The operational software environment is based on OpenStack.

Accessing Jetstream

Jetstream is accessible through a web interface using XSEDE credentials via Globus Auth. Jetstream is NOT accessible via the XSEDE Single Sign-On Login Hub. Newly created XSEDE accounts must be added to a specific allocation by the PI or Resource manager in order to access Jetstream.

Jetstream is meant primarily for interactive research, small scale processing on demand, or as the backend to science gateways to send research jobs to other HPC or HTC resources. Jetstream is different in that you can work with GUIs that you couldn't otherwise use on most HPC systems.

Jetstream may be used for prototyping, for creating tailored workflows to either use at smaller scale with a handful of CPUs or to port to larger environments after doing your proof of concept work at a smaller level. Jetstream is not a typical High Performance Computing (HPC) or High Throughput Computing (HTC) environment and won't be used for large scale parallel processing or high-throughput computing.

Figure 1: System overview of Jetstream environment

Key Features

Jetstream utilizes Atmosphere, an easy to use web application, on-demand environment that is designed to accommodate computationally and data-intense research tasks, including Infrastructure as a Service (IaaS) with advanced APIs; Platform as a Service (PaaS) for developing and deploying software to the science community; and Software as a Service (SaaS).

Some of the key features include:

  • Access virtual machine images preconfigured with an operating system and software to help you do scientific computations in domain-specific tasks
  • Find and use tools with the intuitive self-service portal
  • Easily manage virtual machines
  • Publish your own software suites, create your own work environments, and run the software for community use
  • Integrate with existing infrastructure components using API services
  • Easily generate and manage statistical reporting of user resources for total CPU hours and memory usages, total instances and applications launched by user, cloud monitoring, and on-demand intelligence resource allocation

Within Atmosphere, you launch an instance (a launched image of a virtual machine), selecting from the list of available images (a template of a virtual machine containing an installed operating system, software, and configuration). It is recommended that you familiarize yourself with the Linux command-line as some actions require some degree of knowledge of the command-line interface.

System Configuration

The computing environment consists of two homogenous clusters at Indiana University and Texas Advanced Computing Center (TACC) with a test environment at the University of Arizona. The system provides over 1/2 a PetaFLOPS of computational capacity and 2 petabytes of block and object storage. The individual nodes contain two Intel "Haswell" processors, 128 GB of RAM, 2 terabytes of local storage, and 10 gigabit Ethernet networking. The system leverages 40 gigabit Ethernet for network aggregation and each of the production clusters connect to Internet2 at 100 Gbps. The physically distributed system allows Jetstream to be highly available and resilient. GlobusAuth is used for large-scale file transfer and authentication.

Figure 2: System configuration of Jetstream environment

System Configuration Aggregate information Per Node (Compute Node)
Machine type Dell (Various) Dell M630 Nodes
Operating system RHEL/CentOS/Ubuntu CentOS
Memory model N/A N/A
Processor cores 7,680 per site 24
CPUs 640 per site 2
Nodes 320 per site -
RAM 40TB 128GB
Network 100gbps to Internet2
40gbps x 4 to local infrastructure
10gbps to XSEDE
10gbps
Local Storage 640TB 2TB
Storage information Aggregate information Per Node
File systems Block storage for VMs, other storage options forthcoming Varies
Total disk space 640TB Local
960TB Attached Storage
2TB local
Total scratch space N/A N/A

XSEDE Service Units

Jetstream allocations are measured in XSEDE Service Units (SUs). On Jetstream, SUs are consumed at a rate of 1 SU per vCPU hour. For more information on XSEDE Service Units, see the XSEDE KB document "On XSEDE, how are compute jobs charged?"

SU cost per hour for each Jetstream VM size is outlined in Virtual Machine (VM) Sizes and Configurations.

Only instances that have an Active (green status light) decrement SUs.

Image name Description

BioLinux 8

CentOS 6 MATLAB

  • Tags: Featured, CentOS, desktop, development, gui, m1_medium, VNC
  • Created: 8/2/2016
  • Requires a Medium VM size or larger

CentOS 6 Rstudio

  • Tags: Featured, ...
  • Tags: Featured, CentOS, desktop, development, gui, R, Rstudio, VNC
  • Created: 8/30/2016
  • Based on CentOS 6.8 - Microsoft R Open, MKL (Rblas), and Rstudio
  • Requires a Small VM size or larger

CentOS 6 (6.8) Development

  • Tags: Featured, CentOS, development
  • Created: 7/22/2016
  • Based on Centos 6 Cloud 1601 Image
  • Development tools added
  • Patched up to date as of 6/29/16 -- upgraded to CentOS 6.8

CentOS 6 (6.8) Development GUI

  • Tags: Featured, CentOS, desktop, development, gui, VNC
  • Created: 7/15/2016
  • Based on CentOS 6 (6.7) Development
  • Patched up to date as of 5/31/16 -- updated from 6.7 to 6.8
  • Turned off the following from default startup in GNOME:MAKER 2.31.8 with CCTools 5.4
  • Bluetooth Power Manager Volume Control PulseAudio Package Manager

CentOS 7.2 Development

  • Tags: Featured, CentOS, development
  • Created 7/22/2016
  • Based on Centos 7 Cloud 1601 Image, development tools added
  • Patched up to date as of 6/29/16
  • Development tools added

CentOS 7 R with Intel compilers

  • Tags: Featured, CentOS, desktop, development, gui, Intel, vnc
  • Created 9/06/2016
  • R-3.3.1 with Intel compilers built on CentOS 7 (7.2)

CentOS 7 (7.2) Development GUI

  • Tags: Featured, CentOS, desktop, development, gui, x2go
  • x2go installed -- Instructions for x2go
  • Based on Centos 7 Cloud 1601 Image, Patched up to date as of 8/11/16.
  • Development tools added
  • X Window System and Xfce Groups added
  • Added yum-plugin-changelog, irods-icommands-4.1.9. firefox . thunderbird, xfce4-terminal, nm-connection-editor, network-manager-applet, net-tools, xterm, Lmod
  • NOTE: Ongoing issues with VNC installation on this image - Do not use

Galaxy 16.01 Standalone

  • Tags: Featured, community-contributed, Ubuntu
  • Created: 3/30/2016Galaxy 16.01 Standalone
  • Based on Ubuntu 14.04.4 LTS
  • This is a standalone Galaxy server that comes preconfigured with hundreds of tools and commonly used reference datasets: just launch and use.
  • Requires a Large VM size or larger

Intel Development (CentOS 7)

  • Tags: Featured, CentOS, desktop, development, gui, Intel, vnc
  • Created: 09/01/2016
  • Intel compilers and development environment
  • Based on Centos 7 Cloud 1601 Image, Development tools + GUI + iRODs access added
  • Patched up to date as of 8/23/2016
  • REQUIRES a m1.small or larger VM to launch

Ubuntu 14.04.3 Development

  • Tags: Featured, development, Ubuntu
  • Created: 6/24/2016
  • Ubuntu 14.04.3
  • Trusty Tahr v1.1
  • Patched up to date as of 5/12/16

Ubuntu 14.04.3 Development GUI

  • Tags: Featured, desktop, development, gui, Ubuntu, VNC
  • Created: 7/22/2016
  • Based on Ubuntu 14.04.3 Development
  • Patched up to date as of 6/24/16
  • Base Ubuntu 14.04.3 + Xfce + Xfce-goodies, firefox, icon sets and themes, added Emacs and cmake

Wrangler iRODS - CentOS 6.7

Software licenses

Software licenses are generally the responsibility of the user to obtain, coordinate, implement, and maintain.

Some licenses may require users to coordinate with Jetstream staff to allow for proper configuration. Such coordination is handled on a case-by-base basis.

Licensed software maintained by staff but available to users are listed below.

Compilers and Parallel Libraries

  • The Intel compiler is covered at IU and TACC by a site license and has been configured to point to the proper license server upon boot.
  • All components of the Intel Parallel Studio are covered by the IU and TACC site licenses, including the MKL libraries.

Specialty software

  • MATLAB licenses are also available at both providers. A pref-configured MATLAB-specific image is available within Jetstream.

If you need licenses for any other software then you will have to provide your own license.

Please be aware that if you take an image or software from Jetstream and run it somewhere else then the license will not work. Jetstream license servers restrict access to only valid Jetstream IP addresses.

Known Issues

Number Issue Date Reported Date Resolved Mitigation

2

Jetstream may not recognize XSEDE-associated usernames that differ from TACC usernames due to unresolved issues with identity mapping. This primarily affects long-time users of XSEDE or Teragrid.

8/30/2016

 

This can be mitigated by filing a ticket to help@xsede.org requesting that your TACC and XSEDE usernames be brought into alignment.

3

Some Jetstream virtual machines remain stuck in the Networking stage when they launch. This prevents users from logging into or using the virtual machine.

8/30/2016

 

Try redeploying the virtual machine as described in Instance Management Actions.

4

X2GO on the Biolinux8 image is currently experiencing keyboard mapping issues on first login.

9/28/2016

 

Using the x2goclient software on your remote machine * suspend the x2go session * restore the x2go session by re-authenticating This restored session will have the correct keyboard map.

1

Jetstream is not currently recognizing all XSEDE staff allocations, such as TG-STA060015, as being valid for some users. This is in part because these allocations are roaming allocations. XSEDE users whose only Jetstream allocation is one of these staff allocations may not be able to log into the system or manage VMs.

8/30/2016

9/12/2016

XSEDE users can resolve this by requesting a non-roaming or startup allocation on Jetstream. See Jetstream Allocations for details or email help@xsede.org.

Allocations

Jetstream allocations are awarded exclusively through the eXtreme Science and Engineering Discovery Environment (XSEDE). XSEDE provides XSEDE User Portal (XUP) accounts free of charge. XSEDE allocations require that the PI be a US-based researcher.

You can read the Jetstream Allocations Guide for more detail on Jetstream allocations. Samples of successful requests are available there as well.

  1. Browse to the XSEDE User Portal
  2. Click "Create Account" on the left side of your screen.
  3. Fill out the form and click Submit.
  4. Upon receipt of the email notification click the link in the email to verify your account and set your username and password. If the link doesn't work, go to the XSEDE User Portal, click "Sign In" and then select "Verify Account" under the "Other Sign In Options".
  5. Following account verification, if not already logged in, go to the XSEDE User Portal, click "Sign In" and sign in with the username and password set in the verification step.
  6. You will be asked to read and accept the User Responsibilities form. This outlines acceptable use to protect shared resources and intellectual property.
  1. Read the XSEDE Allocations Overview. There are sample allocation requests in the overview that you may find helpful.
  2. Go to XRAS, the XSEDE Resource Allocation System. On the Available Opportunities page, click "Start a New Submission" under "Startup". If you are not familiar with the process, select "Begin Guided Submission" for step-by-step instructions.
  3. Before submitting an allocation request have the following information available:
    1. PI, Co-PIs, and Allocation Managers (username)
    2. Add additional users that will be able to use your allocation time and resources (optional)
    3. Title
    4. Abstract (typically a paragraph or two for a Startup request will suffice)
    5. Keywords
    6. Field of science (secondary areas of science may be also be added)
    7. Resources
      1. Select "Jetstream IU/TACC" from the list.
      2. A startup allocation is typically 50,000 SUs for Jetstream.
      3. Fill in number of Virtual Machines needed
      4. Fill in number of public IP addresses needed
      5. Select "Jetstream Storage". For Startup allocations, the block storage provided will be limited. Read Virtual Machine Sizes and Configurations for a list of the various VM sizes and the RAM and associated storage. If additional storage for your VM is not needed, enter 1 in the Amount Requested box. Note: The maximum allowed is 500GB. A justification will be needed in the comments for any significant storage request in the Comments box.
    8. Supporting documents - PDF format required
      1. PI CV (2 page limit)
      2. CoPI CV required for every CoPI added to request (2 page limit)
    9. Supporting Grants (Optional)
    10. Publications of previous/supporting work (optional)
  4. Submit allocation request. At this point, all entered information is validated, errors or omissions are flagged.

Allow 1-2 business days for your application to go through the approval process. You can view detailed information, with screenshots, about the allocation request process.

The Getting Started Guide describes the process of getting onto XSEDE, applying for allocations and using XSEDE resources.

To review the types of allocations XSEDE and the process to get an allocation, here are some links you might find useful:

How often can I get a startup allocation?

Applications for startup allocations will only be accepted once. After the startup runs out it is best to apply for a research allocation.

How do I request additional SUs?

If you already have an XSEDE allocation and need to request additional service units (SUs) the PI or co-PI may submit a request via the XSEDE User Portal. For instructions on how to submit the request, see How do I request supplemental service units for an XSEDE allocation?

Request additional SUs

If you already have an XSEDE allocation and need to request additional service units (SUs) the PI or co-PI may submit a request via the XSEDE User Portal. For instructions on how to submit the request, see How do I request supplemental service units for an XSEDE allocation?

Additional resources, such as CPUs and memory, may be requested via Atmosphere.

After logging in, click on Change Your Settings.

Screenshot: Change Settings

Figure 1: The Jetstream Dashboard, with access to the Change Your Settings button

Then click on request more resources in the Allocation section.

Screenshot: Request More

Figure 2: The Jetstream Change Your Settings UI, showing the link to use when requesting more resources

On the popup form:

  1. Select the cloud for which you would like additional resources - IU or TACC.
  2. Identify the Allocation Source
  3. List the resources needed, e.g., 4 CPUs and 8GB memory, running 4 cores for 1 week
  4. Enter a justification for the resource request
  5. Describe how the additional resources will be used.

Click on Request Resources to submit the form.

Trial Access allocation

Jetstream is an NSF/XSEDE resource designed to promote and provide configurable cyberinfrastructure in the form of cloud computing to both novice and experienced users.

XSEDE offers Trial Access allocations to Jetstream. Trial Access allocations are designed to give potential users faster but limited access to Jetstream.

Since there is no formal allocation proposal request, within one business day users will be able to access and evaluate Jetstream prior to requesting a more involved larger startup or research allocation.

Trial Access allocations are limited to:

  • 1000 Service Units
  • 1 small (2-core) Virtual Machine (VM) instance per cloud at a time
  • 1 VM backup snapshot per instance,
  • 1 small 10 GB disk external storage volume.

This is enough power to give new users an experience with virtual computing and try some "cloud-native" work, but is not a substantial usage of the system.

The Trial Access (aka Easy Button) is for exploratory purposes only and currently the limits are hard-wired into the allocation. The additional limits placed on an Easy Button account are not present for Startup and Full allocations.

XSEDE currently has quite a rapid turn around on Startup allocations, often less than a week (excluding holidays).

You can read all about allocations.

Request a Trial Access allocation

To enroll in the Trial Access allocation:

  1. Browse to the XSEDE User Portal.
  2. Click "Create Account" on the left side of your screen.
  3. Fill out the form and click Submit.
  4. Upon receipt of the email notification click the link in the email to verify your account and set your username and password. If the link doesn't work, go to the XSEDE User Portal, click "Sign In" and then select "Verify Account" under the "Other Sign In Options".
  5. Following account verification, if not already logged in, go to the XSEDE User Portal, click "Sign In" and sign in with the username and password set in the verification step.
  6. You will be asked to read and accept the User Responsibilities form. This outlines acceptable use to protect shared resources and intellectual property.
  7. Now logged in to the portal, click on "MyXSEDE" tab.
  8. In the lower left bar, Click "Enroll".
  9. This will take you to a page describing Trial Access and allowing you to Enroll or Unenroll.
  10. Click the "Enroll Trial Access" button.
  11. A Successful Enrollment will show a message like "Your request is being processed. You will receive an immediate ticket message from our helpdesk system confirming your request. Please expect a follow-up email in 24 hours confirming your request was completed.
  12. Wait approximately 4 hours for your allocation to propagate through the authentication system.
  13. Continue with the Quick Start Guide instructions.
    1. Please note that the Trial Access allocation has ID ASC160018.
  14. You may unenroll at any time on the same page by clicking the "Unenroll Trial Access" button.

Account Conflicts

If you have a deactivated or otherwise disabled account with the Texas Advanced Computing Center (TACC), you will need to reactivate that account by [changing your password](https://portal.tacc.utexas.edu/password-reset) before your Trial allocation can be finalized.

VM Sizes and Configurations

Jetstream can be used in several different virtual machine (VM) sizes which are charged in service units (SUs) based on how much of the total system resource is used. The table below outlines the VM sizes created for Jetstream.

VM Size vCPUs RAM (GB) Local Storage (GB) SU cost per hour Can be saved as an image?
m1.tiny 1 2 8 1 yes
m1.small 2 4 20 2 yes
m1.medium 6 16 60 6 yes
m1.large 10 30 60 10 yes
m1.xlarge 24 60 60 24 yes
m1.xxlarge 44 120 60 44 yes
s1.large 10 30 120 10 no
s1.xlarge 24 60 240 24 no
s1.xxlarge 44 120 480 44 no

This allocation information may be subject to changes in the future.

Please note that s1.* based customized instances will NOT be able to be used to create images in Atmosphere.

If your work requires 24 GB of RAM and 60 GB of local storage, then you would request 10 SUs per hour to cover a single Large VM instance.

If your work requires 10 GB of local storage in 1 thread using 3 GB of RAM, then you would request 2 SUs per hour for a Small VM instance. You would then multiply by the number of hours you will use that size VM in the next year and multiply by the number of VMs you will need.

To calculate the number of SUs you will need in the next year, first estimate the number of hours you expect to work on a particular project. For example, if you typically work 40 hours per week and expect to spend 25% of your time on this project that would be 10 hours per week. Next, calculate the total number of hours per year for this project:

Total hours = 10 hours per week * 52 weeks per year 
Total hours = 520

Finally, calculate the total SUs for the year for a single instance medium VM:

Total SUs = 520 hours per year * vCPUs
Total SUs = 520 hours per year * 6vCPUs Total SUs = 3120

If project requires more than 1 medium size VM multiply total SUs by the number of VMs that you will need:

Total SUs needed for 3 medium size VMs = 3 * 3120
Total SUs = 9360

Note: SU cost per hour is described in the XSEDE KB document On XSEDE, how are compute jobs charged?.

Shutdown your VM properly
The calculations above assume that your VM is shutdown properly. For instructions see Shutting down, suspending, stopping instances.

For information on submitting a Research Allocation Request, please see https://portal.xsede.org/successful-requests. Note that all allocations above the startup level require a strong justification for the time being requested.

Quick Start Guide

Accessing Jetstream

Jetstream allocations are awarded exclusively through the eXtreme Science and Engineering Discovery Environment (XSEDE). XSEDE provides XSEDE User Portal (XUP) accounts free of charge. To get a startup allocation, please see Jetstream Allocations for step by step instructions.

For complete details and screenshots, see the System Access section of this guide.

Note
Jetstream is accessible through a web interface using XSEDE credentials via Globus Auth. Jetstream is not accessible via the XSEDE Single Sign-On Login Hub. Newly created XSEDE accounts must be added to a specific allocation by the PI or Resource manager in order to access Jetstream.

Getting started

To start the VM provisioning process, navigate to https://use.jetstream-cloud.org.

  1. Click Login in the top right to authenticate using your XSEDE credentials.
  2. On the Globus Auth screen, click Continue.
  3. Enter your XSEDE credentials; confirm whether you will allow your credentials to be used to access Jetstream.
  4. To proceed, click Allow and the web interface to Jetstream will load.
  5. Once you are authenticated via Globus Auth, you will end up on the Jetstream landing page, also called the Dashboard. On this page you will be able to:
    1. launch a new instance
    2. browse help resources
    3. change your settings
    4. see your resources and usage history
    5. view a Jetstream Community Activity feed
  6. As SSH is a primary method to access Jetstream resources, we recommend you add SSH keys for each host machine from which you will connect to Jestream.

Launch an instance

To launch an instance, you will need to:

  1. open up the Jetstream Atmosphere page, https://use.jetstream-cloud.org
  2. follow the instructions below

  3. Click Launch New Instance from the Dashboard screen

  4. On the Featured Images screen, search for the image you would like to use or to scroll through the images that you have permission to use
  5. Click on the image you would like to use then click Launch to begin the process of creating an instance
  6. On the 'Launch an Instance / Basic Options' page:
    1. Give your instance a name
    2. Select the version if there are multiple versions available
    3. Select a project for your instance or create a new project
    4. Select the allocation source
    5. Choose which provider you want to run on, Indiana or TACC
    6. Select the instance size
    7. Click Continue
  7. Review the information entered for provisioning your instance
    1. If everything is OK click Launch instance to start the build process
    2. To make changes, click Back to return to previous screens
    3. To add or create a deployment script, click Advanced Configuration
  8. When the status shows as Active, click on the name of the instance to display the instance details
  9. To log in to the instance, click on Open Web Shell, located on the lower right hand side of the screen under Links. If this is unavailable, try refreshing your window. If the link is still not enabled, log in to your instance via SSH for your operating system
  10. If you are using Open Web Shell and you did not install your SSH key before provisioning and launching the instance, you will be prompted to enter your XSEDE username and password

System Access

Step-by-step guide

  1. To start the VM provisioning process, navigate to https://use.jetstream-cloud.org.
  2. Click Login in the top right to authenticate using your XSEDE credentials.

    Figure 1: The Jetstream Login access for XSEDE credentials

  3. On the Globus Auth screen click Continue.

    Figure 2: The Globus Auth screen for Jetstream Web App with XSEDE credentials

  4. Enter your XSEDE credentials.

    Figure 3: The XSEDE credentials screen

  5. After you type in your XSEDE username and password, it will ask you to confirm whether you will allow your credentials to be used to access Jetstream. If you wish to use Jetstream, click Allow. You may wish to review the terms of service and privacy policies linked on that page. Generally, you will only see this screen the first time you log into Jetstream. However, changes to Globus Auth might mean you see this screen on a later login to Jetstream.

    Figure 4: Allow Jetstream access with XSEDE credentials

  6. To proceed, click Allow and the web interface to Jetstream will load.

    Figure 5: The Jetstream interface loading

  7. Once you are authenticated via Globus Auth, you will end up on the Jetstream landing page, also called the Dashboard. On this page you will be able to:

    • launch a new instance
    • browse help resources
    • change your settings
    • see your resources and usage history
    • view a Jetstream Community Activity feed

    Figure 6: The Jetstream UI Dashboard

Add users to an instance

These steps will let the user that you create ssh to a running instance using a password you set. The user can reset the password once they login and/or add their ssh keys.

Step-by-step guide

All steps to be run as root or using sudo.

  1. Run 'adduser username'
  2. Run 'passwd username' and assign a temporary password
  3. Run 'usermod -a -G users username'

Notes

If you have the user's public SSH key, do the following:

  1. Run 'mkdir ~username/.ssh/'
  2. Run 'chmod 700 ~username/.ssh/'
  3. Run 'chown username:username ~username/.ssh/'
  4. Copy the user's public ssh into ~username/.ssh/ or use an editor to create the file authorized_keys in that directory and paste the contents of their public SSH into that file
  5. Run 'chown username:username ~username/.ssh/authorized_keys'

If you do not have the user's public SSH key, you will also need to do these steps. It is best, from a security standpoint, to ONLY allow public key access.

  1. Edit /etc/ssh/sshd_config and add the line PasswordAuthentication yes and then save the file
  2. Restart sshd ('service sshd restart' for CentOS 6, 'service ssh restart' for Ubuntu 14.04 systems, -OR- 'systemctl sshd restart' for CentOS 7)

SSH keys

Adding SSH keys to the Jetstream Atmosphere environment

While Jetstream provides a web-based terminal for accessing your VM once it has been deployed, you might find that you wish to access your VM via SSH if you've provisioned it with a routable IP number. Please note that during early operations, all IP numbers offered will be routable - this will change in production.

If you need assistance creating SSH keys, please refer to the XSEDE KB article "How do I set up SSH public-key authentication to connect to a remote system?" Please do note that to get your keys on Jetstream, you will not follow the instructions In that article on placing your keys on a remote system.

Step-by-step guide

  1. To add your ssh key(s) to Jetstream, click on your username in the upper right hand corner and then click Settings.

    Figure 1: The Jetstream dashboard, showing how to access user settings

  2. On the Settings screen, under Advanced, click Show More, to expand the section for adding your SSH key. Check the box that says "Enable ssh access into launched instances" and then click the green plus sign to actually add your key.

    Figure 2: Accessing SSH Key advanced settings in the Jetstream dashboard

  3. On the next screen give the key a descriptive name and then paste the contents of your PUBLIC ssh key into the dialog box.

    Figure 3: The public SSH key dialog in the Jetstream UI dashboard

  4. After you have pasted in your SSH key, click Confirm. You will then be back at the Settings screen with your key shown in the SSH Configuration section.

Launching your VM

  1. To get started using a Jetstream virtual machine, click Launch New Instance from the Dashboard screen. This will take you to a search screen where you can search the image name, description, or tags of the image you would like to use or to scroll through all of the images that you have permission to use.

    For instance, if you want images named or tagged with "CentOS", enter that text in the search bar. The search is not case sensitive. Once you find an image that you wish to use, click on the name or icon and it will take you to the image information screen.

    Figure 1: The Jetstream UI viewing Images available for launch

  2. On image information screen, you will see more details on the image, such as version history and what systems it is available on (Indiana, TACC, or both). On this screen you may add an image (photo) to your project, click the star to save it as favorite image, or actually launch the image.

    Click Launch to begin the process of creating an instance.

    Figure 2: The Launch button, used to create an instance

  3. Give your instance a name, select the version if there are multiple versions available, and choose which provider you want to run on, Indiana or TACC. Click Continue.

    Figure 3: Setting the Instance details

  4. Choose the instance size. This indicates the vCPUs, memory, and disk size for the VM. See the Virtual Machine Sizes table to show the available options and the SUs consumed per hour. Check projected resource usage then click Continue to move to the next screen.

    Figure 4: Setting the size of an Instance

  5. Select or create a project to hold this instance. If you have any existing projects, they will be shown here and you can select one. If you don't have any existing projects, click Create New Project and fill in Project Name and Description. A detailed description is optional, but it is recommended to include any grant names or other easily identifying details so others working with you may easily find it. Click Create to create the new project.

    Figure 5: Adding an instance to an existing project, or creating a new project

  6. On the project selection screen, click Launch to start the initialization of your instance.

    Figure 6: Starting the initialization of an instance

  7. On the last screen, review the choices for provisioning your instance. If you need to make changes, click Back to return to previous screens.

    Figure 7: Review choices prior to creation of instance

  8. If all of your choices are correct, click Launch instance to start the build process.

    Figure 8: Process screen during creation of an instance

  9. Below are several screens you might see during the provisioning process.

    Figure 9a: Instance creation process: Build - networking

    Figure 9b: Instance creation process: Active - initializing

    Figure 9c: Instance creation process: Active - deploying

  10. The instance will be ready for use when you see a green dot and "Active" in the Status column.

    Figure 10: Instance ready for use: Green Dot means Active

Please note that it may take some time for instances to become active, 5-10 minutes on average. The start up time also depends on how busy the system is and on the size of the VM you requested.

Once the instance is Active additional management actions, may be performed on the image.

From the VM command line, find the public IP assigned to your instance

If your VM has a public IP and you need to find that IP (and don't have ready access to the Jetstream interface, you can issue this command from the command line to get your public IP:
wget http://ipinfo.io/ip -qO -

Instance Management Actions

After launching an instance, several options are available under the Actions menu located on the right hand side of your screen.

Figure 11: Management options available on a launched instance

Report an instance if these situations occur:

  • you can't connect via SSH or VNC
  • the status never changes from pending to running
  • you receive errors when running or installing software
  • the metrics for the instance do not display
  • the instance exhibits any other unexpected behavior

Image request - see Customizing and Saving a VM

Suspend an instance to:

  • free up resources for other users
  • safely preserve the state of your instance without imaging
  • preserve your resources and time allocation (your resource usage charts will only reflect the freed resources when the instance is suspended)
  • NOTE: The IP address is likely to change when the instance is resumed.

Shelve an instance to:

  • free up compute resources for other users
  • preserve your allocation a
  • allow for imaging of the instance
  • NOTE: The IP address is likely to change when the instance is resumed.
  • It may take 5-15 minutes to resume (Unshelve) your instance
  • status will change from "Active" to "Shelved_offloaded"

Stop an instance to:

  • free up resources for other users
  • NOTE: Your instance will continue to burn XSEDE Service Units (SUs). To preserve your resources and time allocation you must suspend your instance.)

Redeploy an instance:

  • to fix intances that show up as 'active - deploy_error'
  • Contact support if your VM returns to the deploy_error state after redeployment.

Reboot an instance to send:

  • an 'ACPI Restart' request to the VM that will start the reboot process for your VM
  • a 'Hard Reboot', which will forcibly restart your VM, if it doesn't respond to a 'Reboot'

Delete an instance:

  • Unmount volumes within your instance before deleting the instance or risk corrupting your data and the volume.
  • Your instance will be shut down and all data will be permanently lost!
  • Your resource usage charts will not reflect changes until the instance is completely deleted and has disappeared from your list of instances.

Logging in to your VM

Accessing Jetstream

Jetstream is accessible through a web interface using XSEDE credentials via Globus Auth. Jetstream is not accessible via the XSEDE Single Sign-On Login Hub. Newly created XSEDE accounts must be added to a specific allocation by the PI or Resource manager in order to access Jetstream.

Jetstream allows multiple methods for accessing and using your VM.

Open Web Shell is the preferred method.

You may also log in to your instance directly via an SSH command line or graphic VNC desktop from your host computer.

Some images support additional mechanisms for accessing your VM. Some of the additional access methods are noted in image-specific documentation e.g. x2go for BioLinux.

Logging in with Web Shell

  1. Log in to Jetstream via the web interface and launch the instance.
  2. When the status shows as Active, click the Open Web Shell link found on the lower right hand side of the screen. If this link is unavailable, try refreshing your window. If the link is still not enabled, log in to your instance via SSH for your operating system.
  3. Enter your XSEDE username and password.
  4. Click Connect and then enter your password again.

To become root
Enter sudo su - at the command prompt or type sudo command and replace command with the command for which you want to use sudo.

Figure 1: Active instance with Web Shell link in lower right corner

A successful Web Shell login will look similar to the following.

Figure 2: Screenshot of a successful Web login shell

Logging in with SSH

Regardless of whether you're logging from a Linux, Mac, or a Windows machine,

  1. add SSH keys to your account and
  2. copy the instance IP address, either from the confirmation email or from the IP address displayed in the My Instances list.

MacOS X

  1. Open a terminal window for Mac OS X (from Finder, go to Applications, click Utilities, and then double-click Terminal).
  2. In the terminal window, enter the following command, using your XSEDE username and password, and the instance IP address:
$ ssh xsede_username@instance_ip_address
  1. Press Enter.

A successful login will look similar to the following:

Figure 3: Screenshot of a successful MacOS X login shell

Windows using PuTTY

PuTTY is an SSH client for Windows. It operates a bit differently than Terminal to make the initial SSH connection. For a useful guide to using PuTTY, see PuTTY - Remote Terminal and SSH Connectivity.

  1. Download the PuTTY application.
  2. Launch PuTTY.
  3. The first time PuTTY is used for login, add your private key.
    1. Click the 'Default Settings' session to save your private key for all future sessions.
    2. Click on the + symbol next to the 'SSH' category on the left side.
    3. Click on the 'Auth' category to bring up the PuTTY Configuration screen (see screenshot below).
    4. The key is set down at the bottom under 'Private key file for authentication'. Click on the Browse button next to the 'Private key file for authentication' field and locate your private key file on the file system. Select the file and press 'Ok'. (It is probably in your My Documents folder.)
    5. Click the 'Session' category from the left hand side.
    6. Make sure "Default Settings" is still selected.
    7. Click Save.
  4. Enter the IP address, either copied from your My Instances list or from the confirmation email, and click Connect.
  5. Enter your XSEDE username when prompted and click Enter.
From the VM command line, find the public IP assigned to your instance
If your VM has a public IP address and you need to find that IP (and don't have ready access to the Jetstream interface), you can use wget or curl from the command line to get your public IP:
wget http://169.254.169.254/latest/meta-data/public-ipv4 -qO -
wget http://ipinfo.io/ip -qO -
curl http://169.254.169.254/latest/meta-data/public-ipv4
curl http://ipinfo.io/ip

*Note: http://169.254.169.254/latest/meta-data/public-ipv4 works even in conditions in which external DNS servers are not accessible.

To become root
Enter sudo su - at the command prompt or type sudo command and replace command with the command for which you want to use sudo.

Logging in with VNC desktop

VNC is only available on certain images. Please look for the GUI or Desktop tags on the Featured images.

Images with VNC/GUI/Desktop enabled will display Web Desktop under the LINKS items in the instance detailed view (found by clicking on the instance title). You may have to refresh your browser to make it appear.

Simply click on Web Desktop to start a browser-based VNC implementation.

Alternatively, if you would like to use a different VNC client (which must support encrypted connections), for example realvnc viewer (free), please follow these instructions.

For VNC launches

  1. Once your instance comes up, login via ssh.
  2. Type 'sudo passwd your_username'.
  3. Use realvnc viewer (free) to connect to x.x.x.x:1 (your_ip:1) with your username and the password you just set.
  4. When finished with the VNC, please disconnect your VNC session via the Real VNC viewer or by closing the VNC window on your host computer.

**DO NOT LOG OUT INSIDE THE DESKTOP OF THE VM: this will make your VNC unusable until you manually restart the VNC.

Note on resizing: Use xrandr -s 1920x1080 to get a larger screen; adjust the numbers to get the screen size that you need. xrandr works reliability in both operation systems.

Note: the default VNC session will only work for the user that launched the instance. Other user accounts on that instance that wish to have an individual desktop will have to start their own VNC manually.

Disconnecting your VNC session via the Real VNC Viewer:

Figure 4: Disconnecting a VNC instance using the Real VNC Viewer

To become root
Enter sudo su - at the command prompt or type sudo command and replace command with the command for which you want to use sudo.

Managing a VNC server

Starting a VNC server manually:

By default for images with VNC capability, a VNC is only launched for the User that launched the Instance.

If you have added additional users accounts to your Instance, and those users have set their password, they must manually start their own VNC desktop using the 'vncserver' command:

[USER@js-19-210 ~]$ vncserver :1

where :1 is a display number (1-9).

Note: If a VNC server is already running on that particular display number, you will receive an error:

Error: A VNC or X Server is already running as :1 [DisplayInUse]

If this arises, choose a different display number.

Note: If a User has logged out within the VNC desktop, this manuallly stops the VNC server.

Stopping a VNC server

A VNC server may be stopped using the command:

[USER@js-19-210 ~]$ vncserver -kill :1

where :1 is the display number ofa VNC server owned by that User.

Logging out of the desktop in the VNC also stops the VNC server.

Note: Killing an active VNC server before data/files have been save or icon settings stored may result in a loss of data/settings.

Restarting a VNC server

If a User has stopped their VNC server either by logging out of the desktop within the VNC or with the -kill option to the vncserver command, they may simply restart the server after SSH'ing in, as above.

Customizing and saving a VM

An image is a type of template for a virtual machine (VM).

You can launch an instance, install the software and files you want to use, then request an image of the instance. This will save all of the changes and updates within Atmosphere. Saving instances as images saves resources. The saved image can be relaunched at any time so that it won't keep running, and using resources, when it is not being used.

CAUTION!
  • THE IMAGE MUST BE ACTIVE OR STOPPED in order to request an image. Currently, Jetstream cannot image Suspended instances.
  • Imaging Guidelines: There are several other restrictions and notes of which you must be aware in order to achieve a successful image. For example, several directories and files are removed during image creation.
  • At this time, only the creator of an image may update an image.
  • Please note that s1.* based customized instances will NOT be able to be used to create images in Atmosphere.

After submitting the form, the Jetstream Atmosphere support staff will review and process the request. Future versions of Atmosphere will allow users to initiate the VM imaging process automatically.

Be sure to test launch any image created to validate that it behaves as expected BEFORE suspending or removing the current active instance, since only active instances can be imaged.

VERY IMPORTANT NOTE
Creating an image on the SMALLEST possible size VM on which it will run will allow the image to be launched on VMs of the same size and larger.
For example, an image created on a Tiny size VM can be launched on a VM of any size; an image created on a Medium VM can only be launched on a Medium or larger size VM.

Imaging Guidelines

Before you request an image

Here are some tips to help ensure a viable importable image:

  • Operating system: Base the image on
    • CentOS6, CentOS 7 and later
    • Ubuntu 14.04 and later, Long Term Support (LTS) versions of Ubuntu recommended
  • File system: Ext3, Ext4, or XFS.
  • Image format: RAW or QCOW2.
  • Software: Image must contain no licensed software that would prohibit use within a cloud or virtualized environment. It is recommended that software be installed in /opt or /usr/local/.
  • New Image Name:
    DO NOT use the name of an existing featured image. Please be mindful of re-using the name of one of your own existing images.
    DO NOT use a period(".") as any of the last 5 characters of the name.
  • Description of the Image: DO add an informative description of your image and make note of how it varies from any base images.
  • Instance status: The instance MUST be ACTIVE or STOPPED in order to image. Suspended images cannot be imaged at this time.
  • Volumes: All Volumes must be DETACHED from the instance. Failure to detach Volumes will result in a image whose child instances fail to boot.
  • Owner: At this time, only the creator of an image may update an image. Developer teams wishing to update a shared image should contact Jetstream support.

Before submitting a request for an image of your instance, remove the following software from the instance:

  • Licensed software: including software not purchased by Jetstream or otherwise not in the public domain.
  • Not cloud suitable: software in which the licensing, features, or activity of the software otherwise precludes its use or inhibits the activity of other software within a cloud or virtualized environment.

Volumes and iRODS FUSE mounts are not copied as part of the image.

Caution!

The following directories are deleted as part of the imaging process:

  • /home/
  • /mnt/
  • /tmp/
  • /root/

The following system files are typically overwritten by the Jetstream imaging process for security and operational reasons:

  • /etc/fstab
  • /etc/group
  • /etc/host.allow
  • /etc/host.conf
  • /etc/host.deny
  • /etc/hosts
  • /etc/ldap.conf
  • /etc/passwd
  • /etc/resolve.conf
  • /etc/shadow
  • /etc/sshd/
  • /etc/sysconfig/iptables
  • /root/
  • /var/log

Request an image

You can request an image (a type of template for a virtual machine) of a running instance. This saves a complete copy of all changes and updates made to the instance since it was launched it so it can be reused at any time. It also saves resources by launching the instance only when you need it.

You also can see the list of all image requests you have made.

You can add a script before requesting the image that executes after an instance using the image is launched and active.

To get started:

  1. Log in to Jetstream web interface https://use.jetstream-cloud.org/.
  2. Detach all attached volumes from the instance. Detailed instructions for this will be coming later.
  3. Click Projects on the menu bar and open the project with the instance to use for the new image.
  4. Click the instance name. The instance must be in Active status.
  5. In the Actions list on the right, click Image.

Image Info

The information you provide on here will help others to discover this image.

  1. New Image Name (required): Enter the name, up to 30 characters, to assign to the new image.
  2. Description of the Image (required): The description should include key words that concisely describe the tools installed, the purpose of the tools (e.g., This image performs X analysis), and the initial intent of the machine image (e.g. designed for XYZ workshop). Include key words that will help users search for this image.
  3. Image Tags (optional): Click in the field and select tags that will enhance search results for this image. You may include the operating system, installed software, or configuration information (e.g. Ubuntu, NGS Viewers, MAKER, QIIME, etc.). Tags can be added and removed later, if needed.
  4. Click Next.

Figure 1: Image Request - Image Info screen

Version Info

Versioning is an important part of the imaging process. Use this information to track how your image changes over time. This information will also be helpful to others that wish to use your image.

  1. New Version Name (required): Enter the new (unique) name or number of the image. Versioning helps users understand how your changes relate to the overall progress of the Application. Versions are alphanumeric (e.g. 2.0-stable, 2.1-beta, 2.2-testing). Limit the name to 30 characters and keep versioning consistent.
  2. Change Log (required): Concisely describe what you've changed in this specific version. This description will help users understand how your application as changed over time.
  3. Click Next.

Figure 2: Image Request - Version Info screen

Provider

  1. Select the cloud provider to use for the image. If you would like the image to be available on multiple clouds, email help@jetstream-cloud.org.
  2. Indicate minimum CPU and memory requirements (optional).
  3. Click Next.

Figure 3: Image Request - Provider screen

Privacy

  1. Select the visibility for the image:

    • Public: The image will be visible to all users and anyone will be able to launch it.
    • Private: The image will be visible only to you and only you will be able to launch it.
    • Specific Users: The image will be visible to only you and the users you specify. Only you and those specific users will be able to launch it. If you chose Specific Users, select the users who will be able to launch the image.
  2. Click Advanced Options or Submit.

    Advanced Options will allow you to:

    • Exclude files from the image
    • Add a deployment script
    • Require the user to verify understanding of any license restrictions

Figure 4: Image Request - Privacy screen

Advanced Option

Exclude Files

Note the list of directories that will automatically be excluded form the image:

  • /home/
  • /mnt/
  • /tmp/
  • /root/

In the box provided, list any additional files or directories to be excluded from the image. Write one path per line.

Figure 5: Image Request - Exclude Files screen

Advanced Option

Boot Scripts & Licenses

Deployment scripts are executed when a user launches the image and each time an instance is 'Started', 'Resumed', or 'Restarted'. These scripts should be able to handled being run multiple times without adverse effects.

Click Next to continue to the next screen without adding a new script or a software license.

  1. To add a deployment script, click in the search field and search for the title of the script.
  2. To create a new deployment script:
    • Click Create New Script, enter a title for the script, then either click URL and enter the script URL or click Full Text and enter the deployment script.
    • When done, click Create and Add, then click Next.
  1. To list any licensed software used in the image and require users to agree to the license agreement before launching, click in the search field and search for the license title.
  2. To create a new license:
    • Click Create New License, enter a title for a license, then either click URL and enter the license URL or click Full Text and enter the full license text.
    • When done, click Create and Add, then click Next.

    Figure 6: Image Request - Boot Scripts & Licenses screen

Review Image Request

On the Review screen, verify the information entered on the previous screens. Click Back to return to the previous screens and make corrections. When all is OK, click the checkbox certifying that the license does not contain any license-restricted software that is prohibited from being distributed within a virtual or cloud environment..

  1. Click Request Image.

You will receive an email from Support when the image is completed. Please email questions to help@jetstream-cloud.org.

Figure 7: Image Request - Review screen

Viewing your images

You can view your lists of images and image requests.

  1. Click Images on the top menu bar.
  2. Click:
    • MY IMAGES to view the list of your images.
    • MY IMAGE REQUESTS to view the list of your image requests.

Figure 8: Viewing a list of images

Request image deletion

Currently, the only way to delete (archive) an image you requested is to email help@jetstream-cloud.org. You will receive an email confirmation when your image has been archived.

Image publicaton

Submit your Jetstream image for storage and digital object identifier (DOI)

Images that you have customized and saved may be made available for publications or to the public, within certain limits (for example, they may not contain secure or protected information or data).

To request an image be stored, given a digital object identifier (DOI), please follow these instructions.

Please be aware that at the current time, such a process requires the manual intervention of staff and is therefore subject to scheduling limits.

Disabling a VM

Shutting down, suspending, stopping instances

Why suspend an instance?

  • free up resources for other users
  • safely preserve the state of your instance without imaging
  • preserve your resources and quota allocation (SUs are not charged when an instance is suspended)
    • NOTE: Though not currently charged, Memory and Disk are stored and continue to place demand on system resources.
  • resource usage charts will only reflect the inactive resources when the instance is fully suspended

The instance status will change from "Active" to "Suspended" in the Atmosphere Instances status panel as shown in Figures 1 and 2.

Values and settings stored only in memory will be maintained, as in putting a physical computer in sleep mode.

Note

  • The IP addresses will almost certainly change when the instance is resumed, whether it was suspended or shutdown. Configurations and settings may need to be adjusted accordingly.
  • It may take 5-15 minutes (depending on system load) to resume your instance.

Stop an active instance:

  • to free up computing and some storage resources
  • preserve your resources and quota allocation
    • NOTE: Though not currently charged, Memory and Disk are stored and continue to place demand on system resources.
  • your resource usage charts will only reflect the freed resources when the instance is fully suspended

The instance status will change from "Active" to "Shutoff" in the Atmosphere Instances status panel (Figures 1 and 2).

Values and settings stored only in memory will be lost, as in shutting down a physical computer.

Shutting down a VM from Linux

If you are going to stop your instance, shut down the VM gracefully and securely. In a GUI environment on Linux, the methods may vary. Launching a terminal and running the shutdown command as root should work consistently across Linux versions:

/sbin/shutdown -h now

or

sudo /sbin/shutdown -h now

This will stop all operations, log out other active users on your VM as the system powers down and show your VM as "Shutoff" in Atmosphere once the VM has shut down.

Note that sometimes the Atmosphere interface doesn't update in a timely manner to reflect the true status of the VM. If you have done a shutdown from the console, then do a hard refresh of your browser about a minute after shutdown. This is usually Command-R on a Mac and Ctrl-F5 on a PC. Check your browser's documentation to verify.

In general, refresh the browser to verify the VM state if in doubt.

For information on the other options available from the Actions menu see Instance management actions.

Figure 1: Instance with Active status. Instance is consuming SUs and resources.

Figure 2: Instance with Shutoff status. Instance is no longer consuming SUs and resources.

Getting Scientific Software

If you are using a CentOS/rpm based VM, you can utilize software packaged by the XSEDE Campus Bridging team for the XSEDE National Integration Toolkit (XNIT).

The XSEDE National Integration Toolkit (XNIT), formerly known as the XSEDE Yum Repository, is a collection of Red Hat Package Manager (RPM) packages assembled to simplify the process of converting a "bare-bones" Linux cluster into a high-performance, parallel computing system that can be used to support scientific discovery. The packages included in the repository are specific versions and builds of scientific, mathematical, and visualization applications recommended by the Extreme Science and Engineering Discovery Environment (XSEDE) for optimal compatibility with XSEDE digital services.

Please see the Knowledge Base entry about XNIT for details on what software is available and how to set up the XNIT repository on your VM.

If you are using a Debian based system such as Ubuntu, you can use the Alien package to convert RPMs to DEB packages for installation. This is not supported and may not perform exactly as expected. For more information on Alien, see Converting .rpm Packages To Debian/Ubuntu .deb Format With Alien.

Launching Jupyter Notebook within an instance

The following steps will easily install and start a jupyter notebook on any active instance.

Step-by-step guide

  1. Login to the instance using web shell or using an ssh client.
  2. Type "ezj". Jupyter notebook will then be installed and configured within a couple of minutes.
  3. After the installation is complete, a URL will be displayed that is accessible from any browser.

    Closing your Jupyter notebook
    If you close your web shell or your ssh session, the Jupyter notebook will shutdown.

  4. To restart to your Jupyter notebook, repeat steps 1-3.

Image Specific Documentation

Some of the more commonly used custom images include:

Images and the Intel compiler

The environment variables for using the Intel compiler are set for users in /etc/skel/.bash_profile. If you want to set them manually and/or use it for the root user, you can copy the variables below.

Please note that while we don't anticipate there being issues for the root user to have these in the default root user environment, there may be unintended consequences.

Intel environment variables

environment variables for Intel parallel_studio_xe_2016_updater3

export INTEL_LICENSE_FILE=/opt/intel/licenses/

export PATH=/opt/intel/compilers_and_libraries/linux/bin/intel64:/opt/intel/compilers_and_libraries/linux/mpi/intel64/bin:/opt/intel/debugger_2016/gdb/intel64_mic/bin:$PATH

export MANPATH=/opt/intel/man/common:/opt/intel/compilers_and_libraries/linux/mpi/man:/opt/intel/compilers_and_libraries/linux/man/en_US:/opt/intel/documentation_2016/en/debugger/gdb-ia/man/:/opt/intel/documentation_2016/en/debugger/gdb-mic/man/:/opt/intel/documentation_2016/en/debugger/gdb-igfx/man/:$MANPATH

export LIBRARY_PATH=/opt/intel/compilers_and_libraries/linux/ipp/lib/intel64:/opt/intel/compilers_and_libraries/linux/compiler/lib/intel64:/opt/intel/compilers_and_libraries/linux/mkl/lib/intel64:/opt/intel/compilers_and_libraries/linux/tbb/lib/intel64/gcc4.4:/opt/intel/compilers_and_libraries/linux/daal/lib/intel64_lin:/opt/intel/compilers_and_libraries/linux/daal/../tbb/lib/intel64_lin/gcc4.4:/opt/intel/compilers_and_libraries/linux/daal/../compiler/lib/intel64_lin:$LIBRARY_PATH

export LD_LIBRARY_PATH=/opt/intel/compilers_and_libraries/linux/compiler/lib/intel64:/opt/intel/compilers_and_libraries/linux/mpi/intel64/lib:/opt/intel/compilers_and_libraries/linux/mpi/mic/lib:/opt/intel/compilers_and_libraries/linux/ipp/lib/intel64:/opt/intel/compilers_and_libraries/linux/compiler/lib/intel64:/opt/intel/compilers_and_libraries/linux/mkl/lib/intel64:/opt/intel/compilers_and_libraries/linux/tbb/lib/intel64/gcc4.4:/opt/intel/debugger_2016/libipt/intel64/lib:/opt/intel/compilers_and_libraries/linux/daal/lib/intel64_lin:/opt/intel/compilers_and_libraries/linux/daal/../tbb/lib/intel64_lin/gcc4.4:/opt/intel/compilers_and_libraries/linux/daal/../compiler/lib/intel64_lin:$LD_LIBRARY_PATH

export CPATH=/opt/intel/compilers_and_libraries/linux/ipp/include:/opt/intel/compilers_and_libraries/linux/mkl/include:/opt/intel/compilers_and_libraries/linux/tbb/include:/opt/intel/compilers_and_libraries/linux/daal/include:$CPATH

export NLSPATH=/opt/intel/compilers_and_libraries/linux/compiler/lib/intel64/locale/en_US:/opt/intel/compilers_and_libraries/linux/mkl/lib/intel64/locale/en_US:/opt/intel/debugger_2016/gdb/intel64_mic/share/locale/en_US:/opt/intel/debugger_2016/gdb/intel64/share/locale/en_US:$NLSPATH

export INFOPATH=/opt/intel/documentation_2016/en/debugger/gdb-ia/info/:/opt/intel/documentation_2016/en/debugger/gdb-mic/info/:/opt/intel/documentation_2016/en/debugger/gdb-igfx/info/:INFOPATH

export CLASSPATH=/opt/intel/compilers_and_libraries/linux/daal/lib/daal.jar:$CLASSPATH

export IPPROOT=/opt/intel/compilers_and_libraries/linux/ipp

export MKLROOT=/opt/intel/compilers_and_libraries/linux/mkl/

export TBBROOT=/opt/intel/compilers_and_libraries/linux/tbb

export DAALROOT=/opt/intel/compilers_and_libraries/linux/daal

export I_MPI_ROOT=/opt/intel/compilers_and_libraries/linux/mpi

export INTEL_PYTHONHOME=/opt/intel/debugger_2016/python/intel64/

export GDBSERVER_MIC=/opt/intel/debugger_2016/gdb/targets/mic/bin/gdbserver

export GDB_CROSS=/opt/intel/debugger_2016/gdb/intel64_mic/bin/gdb-mic

export MPM_LAUNCHER=/opt/intel/debugger_2016/mpm/mic/bin/start_mpm.sh

BioLinux 8

Using x2go with the BioLinux 8 image

The BioLinux 8 image comes bundled by default with x2go. While x2go is not the preferred desktop sharing method for Jetstream, it is very functional and does work well. At this time, VNC is not functioning properly on the BioLinux 8 image. To get started, download the x2go client. The client for Windows, OSX, and multiple versions of Linux is available from the x2go site, includes installation instructions (all platforms).

After installation, launch the client and set up a new connection:

  • Session name: enter a descriptive name for the connection
  • Host: Your_VM_IP_address (or DNS name)
  • Login: Your_XSEDE_username
  • To login via SSH enter your Jetstream SSH key.
    • Optional: check Try auto login (via SSH agent or default SSH key)
  • Session Type: Choose "Custom Desktop" and Command: "MATE"
    • or Session Type "MATE" and Command : "" (leave blank)
  • Click OK to save the connection.

Once saved, click on the connection to bring up the virtual machine GUI desktop. Note: it may take 1-2 minutes for the connection to come up.

To use password authentication rather than SSH, set the password on your VM.

  1. Once the instance comes up, login via ssh.
  2. Type 'sudo passwd your_username'.

TROUBLE SHOOTING: x2go keyboard map wrong

If your x2go session starts with a keyboard map that doesn't match your device, use your _x2goclient_ program on your **remote host** to _suspend_, then _resume_ your x2go session.

Using the Jetstream API

To request OpenStack API access, email help@xsede.org with the subject "Requesting Jetstream API access for the XYZ PROJECT". In your email, answer as many of the questions listed below as possible.

  • Briefly describe what you want to do and how you plan to do it.
  • What will runtime operations look like?
  • Does this application exist already or is this primarily a development effort? If yes, what parts exist and where can they be referenced?
  • What API are you planning on using, Openstack, RADOS, Atmosphere, EC2, S3, something else?
  • Are any of these gateways? Might it spawn a gateway in the future?
  • What resources could you conceive this project consuming once in production? Please include sustained as well as peak resource demands.
  • Do you anticipate bursting into/out of Jetstream?
  • Jetstream contains more than one OpenStack cloud. Were you aware of this and do you think your implementation will be able to leverage multiple clouds initially? Eventually?
  • Are there long term storage needs, for example for reference data, that need to be accessible from the running instances? If so, how much storage is needed?
  • Are there run-time scratch/transient/workbench storage needs?
  • What is/are the XSEDE project name(s)?
  • Who is working on this project and what are their roles?
  • Is there a group account that development will occur under?
  • Is there one primary developer that will be using this account; or, will a group of individuals be developing code within this account?
Note

Early access to Jetstream's OpenStack API is being granted to select developers only. At this time, developers should understand the following:

  1. The current API environment is unstable.
  2. The current API environment is considered to be in Early User/Friendly User mode. Not everything is installed and/or fully functional.
  3. Jetstream staff and systems administrators will prioritize and address issues as they arise.
  4. Access may be revoked or restricted to insure the effective and efficient delivery of production services.
  5. This is an OpenStack API not an Atmosphere API. When you open up Atmosphere your OpenStack VMs will not be displayed.
  6. PIs and developers must be aware of, and conform to, any and all policies required for users of the systems owned and/or operated by the service providers, Indiana University (IU) and Texas Advanced Computing Center (TACC).

    IU

    TACC

After access granted

  • The project PI should ensure that all accounts have been associated with their XSEDE project(s).
  • Users and group account owners should verify that they can authenticate at the XSEDE User Portal.
  • Users and group account owners should verify their TACC username and password. See the TACC password reset page.
  • Once your XSEDE and TACC credentials are in order, confirm that you can authenticate to the OpenStack Horizon portal, in the TACC domain.
  • Start up a VM instance via Atmosphere and install the OpenStack clients. For more information see Install the OpenStack command-line clients.
For example

Using Atmosphere, launch a CentOS-7 and then, at a minimum, install these packages:

  • Set the environment variables in openrc.sh. For instructions, see Setting up openrc.sh.
  • After the clients have been installed and the environment variables set, OpenStack client commands such as glance image-list should now return useful information.
  • If you have never worked with OpenStack and need help in creating your first running instance email help@xsede.org.

Setting up openrc.sh

Jetstream has two clouds: the IU cloud and the TACC cloud; each cloud has one domain, the 'TACC' domain. API users will only be interacting with one cloud at a time. Transferring entities such as images and volumes from one domain to the other requires intervention by systems personnel. To request these services email help@xsede.org.

When using the 'TACC' domain, the username and password in the openrc.sh file are the same login credentials that would be used to access any TACC resource. For assistance with TACC login credentials, visit https://portal.tacc.utexas.edu/password-reset/-/password/request-reset.

IU cloud, TACC domain

TACC domain

export OS_PROJECT_DOMAIN_NAME=tacc
export OS_USER_DOMAIN_NAME=tacc
export OS_PROJECT_NAME=TG_your_xsede_project_name
export OS_USERNAME=your_tacc_username
export OS_PASSWORD=your_tacc_password
export OS_AUTH_URL= Contact help@xsede.org for the available endpoint URL
export OS_IDENTITY_API_VERSION=3

TACC cloud, TACC domain

TACC domain

export OS_PROJECT_DOMAIN_NAME=tacc
export OS_USER_DOMAIN_NAME=tacc
export OS_PROJECT_NAME=TG_your_xsede_project_name
export OS_USERNAME=your_tacc_username
export OS_PASSWORD=your_tacc_password
export OS_AUTH_URL= Contact help@xsede.org for the available endpoint URL
export OS_IDENTITY_API_VERSION=3

Use the Horizon dashboard to generate openrc.sh

  1. Log in to Horizon:
    IU: https://jblb.jetstream-cloud.org/dashboard
    TACC: https://tacc.jetstream-cloud.org/dashboard
    Domain: TACC
    User Name: your TACC username
    Password: your TACC password
  2. Click on Access & Security (located under Compute on the left hand side)
  3. Click on the API Access tab (4th tab)
  4. Click on Download OpenStack RC File v3 (2nd tab) The password is not stored in the script. This openrc.sh file will prompt for a password when sourced.

Note: To select an active project prior to generating openrc.sh, click on Identity (left hand side) to see a list of your projects.

OpenStack command line

There are many options and tools for using the OpenStack API from the command line. Follow the instructions in the table below to set up a security policy and network, launch and manage a VM and then remove the entire structure. All of the commands EXCEPT creating and removing security groups may be done from the Horizon OpenStack dashboard.

For more information, see the OpenStack command-line interface cheat sheet. Help is also available directly from the command line tools as shown in this example.

Get help from a command line tool (an example): nova help secgroup-create

usage: nova secgroup-create <name> <description>
Create a security group.
Positional arguments:
<name> Name of security group.
<description> Description of security group.

Openstack component basics for this process are:

Create a security group - do this once at IU and/or TACC before launching instances. Comand line

Create a security group that will enable inbound ping and SSH. For more info see,

https://wiki.openstack.org/wiki/Neutron/SecurityGroups

See also Add/Remove security groups.

Important note: On OpenStack, the default is that NO ports are open versus the traditional all ports are open unless specifically closed. For this reason, a security group must be established and the SSH port opened in order to allow login.

nova secgroup-create global-ssh "ssh & icmp enabled"

nova secgroup-add-rule global-ssh tcp 22 22 0.0.0.0/0

nova secgroup-add-rule global-ssh icmp -1 -1 0.0.0.0/0

Upload SSH key - do this once

 

If you have an SSH key upload id_rsa & id_rsa.pub to nova (note: Key filenames may vary)

cd ~/.ssh

nova keypair-add --pub-key id_rsa.pub id_rsa

If you don't have an SSH key then create a new key and upload to nova.

ssh-keygen -b 2048 -t rsa -f ${OS_PROJECT_NAME}-api-key -P ""

nova keypair-add --pub-key id_rsa.pub ${OS_PROJECT_NAME}-api-key

Setup the network - do this once OpenStack neutron command

Create a private network

neutron net-create ${OS_PROJECT_NAME}-api-net

Verify that the private network was created

neutron net-list

Create a subnet within the private network space

neutron subnet-create ${OS_PROJECT_NAME}-api-net 10.0.0.0/24 --name ${OS_PROJECT_NAME}-api-subnet1

Verify that subnet was created neutron net-list

 

Create a router

neutron router-create ${OS_PROJECT_NAME}-api-router

Connect the newly created subnet to the router (use names instead of UUIDs)

neutron router-interface-add

neutron router-interface-add ${OS_PROJECT_NAME}-api-router ${OS_PROJECT_NAME}-api-subnet1

Connect the router to the gateway named "public"

neutron router-gateway-set ${OS_PROJECT_NAME}-api-router public

Verify that the router has been connected to the gateway

neutron router-show ${OS_PROJECT_NAME}-api-router

Create and launch a VM

OpenStack nova commands

See what sizes (flavors) are available

nova flavor-list

Create and boot an instance

Make sure your SSH keyname matches.

nova boot ${OS_PROJECT_NAME}-api-U-1 --flavor m1.tiny --image 3c3db94e-377b-4583-83d7-082d1024d74a --key-name ${OS_PROJECT_NAME}-api-key --security-groups global-ssh --nic net-name=${OS_PROJECT_NAME}-api-net

Create a public IP address for an instance

nova floating-ip-create public

Associate that IP address with that instance

nova floating-ip-associate ${OS_PROJECT_NAME}-api-U-1 your.ip.number.here

SSH in!

Note that your key was inserted in root's .ssh dir.

SSH root@your.ip.number.here

Reboot, suspend, stop an instance

nova reboot ${OS_PROJECT_NAME}-api-U-1

nova suspend ${OS_PROJECT_NAME}-api-U-1

nova stop ${OS_PROJECT_NAME}-api-U-1

Remove an instance

OpenStack nova and neutron commands

Delete an instance

nova delete ${OS_PROJECT_NAME}-api-U-1

Disassociate the IP address from the instance

nova floating-ip-disassociate ${OS_PROJECT_NAME}-api-U-1 149.165.170.87

Disconnect the router from the gateway

neutron router-gateway-clear ${OS_PROJECT_NAME}-api-router

Delete the subnet from the router

neutron router-interface-delete ${OS_PROJECT_NAME}-api-router ${OS_PROJECT_NAME}-api-subnet1

Delete the router

neutron router-delete ${OS_PROJECT_NAME}-api-router

Add/Remove security groups  

These commands do not have an equivalent GUI operation and can only be performed via the command line clients.

When using the GUI, security groups must be associated with an instance when it is created and/or booted.

nova add-secgroup ${OS_PROJECT_NAME}-api-U-1 global-ssh

nova remove-secgroup ${OS_PROJECT_NAME}-api-U-1 global-ssh

Installing the Openstack clients on OS X

This will help you get the Openstack clients working on Mac OS X 10.11.x. It may work on recent older versions of Mac OS X but it has not been tested.

Follow the instructions below at your own risk.

source .bash_profile

Task Comand

If necessary, install Xcode from the App Store.

 

Add these lines to .bash_profile

# Set architecture flags

export ARCHFLAGS="-arch x86_64"

# Ensure user-installed binaries take precedence

export PATH=/usr/local/bin:$PATH

Run this command

Verify that the Xcode required tools are installed and functional (This command may require root access to run.)

xcode-select --install

Set the permissions that Brew expects

sudo chflags norestricted /usr/local && sudo chown $(whoami):admin /usr/local && sudo chown -R $(whoami):admin /usr/local

Install Brew if not already installed

/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

If Brew is installed, update as needed.

brew update

If prompted to agree to the Xcode license run:

xcodebuild -license

Check the Brew install with this command. If errors found doctor will provide instructions to fix them.

brew doctor

Verify that Brew is up to date

brew update

Install python

brew install python

Fix the links

brew linkapps python

Install the OpenStack clients

pip install python-keystoneclient python-novaclient python-heatclient python-swiftclient python-neutronclient python-cinderclient python-glanceclient python-openstackclient

Set up your OpenStack credentials See Setting up openrc.sh for details.

source .openrc

Test an Open Stack command

glance image-list

If you get python errors try these commands:

pip uninstall six

pip install six

Following future OpenStack updates, all installed pip modules can be updated with this command:

pip freeze --local | grep -v '^-e' | cut -d = -f 1 | xargs -n1 pip install -U

Creating and using volumes from the command line

Assuming you have an allocation for Jetstream storage, you can create and use volumes from the command line.

To view any volumes you might have:

cinder list

To create a 10 GB volume, you can do:

cinder create 10 --display-name ${OS_PROJECT_NAME}-10GVolume

Then you can attach it to an instance for use:

nova volume-attach vm-uid-number volume-uid-number auto

Nova will report back with information like the following:

Property Value
device /dev/sdb
id af59d4fa-ced2-4049-a062-7b2a15807b7f
serverId 6987520f-eae4-4505-8100-4857b5d7e3db
volumeId af59d4fa-ced2-4049-a062-7b2a15807b7f

From your instance, you can now create a mount point, view the device, and create the filesystem (using ext4 as the filesystem type for this example).

# mkdir /vol1
# fdisk -l /dev/sdb
# mkfs.ext4 /dev/sdb

If you get the following warning, it's safe to hit 'y' to proceed:

/dev/sdb is entire device, not just one partition!
Proceed anyway? (y,n)
# mount /dev/sdb /vol1

Assuming you didn't get any errors, /dev/sdb should now be mounted on /vol1

# touch /vol1/foo
# ls -la /vol1/
    total 24
    drwxr-xr-x   3 root root  4096 Jul 13 13:37 .
    dr-xr-xr-x. 18 root root  4096 Jul 13 11:50 ..
    -rw-r--r--   1 root root     0 Jul 13 13:37 foo
    drwx------   2 root root 16384 Jul 13 13:36 lost+found
    
# df -h
    Filesystem      Size  Used Avail Use% Mounted on
    /dev/sda1       8.0G  2.4G  5.7G  30% /
    devtmpfs        902M     0  902M   0% /dev
    tmpfs           920M     0  920M   0% /dev/shm
    tmpfs           920M   17M  904M   2% /run
    tmpfs           920M     0  920M   0% /sys/fs/cgroup
    tmpfs           184M     0  184M   0% /run/user/0
    /dev/sdb        9.8G   37M  9.2G   1% /vol1
    

When you are done with your volume or want to use it with another VM, if you are not shutting down the VM, you'll need to unmount it.

umount /vol1

To detach it from the VM, you'll do a nova detach-volume:

nova volume-detach vm-uid-number volume-uid-number

Doing a cinder list now should show the volume as available:

cinder list
ID Status Name Size Volume Type Bootable Attached to
af59d4fa-ced2-4049-a062-7b2a15807b7f available jlf599-10GVolume 10 false  

If you want to completely destroy a volume, you can do:

cinder delete <volume-uid-number>

Setup for Horizon API user

Part 1: Create Private Network

1. Login to Horizon:

IU: https://jblb.jetstream-cloud.org/dashboard

TACC: https://tacc.jetstream-cloud.org/dashboard

Domain: TACC

User Name: your TACC username

Password: your TACC password


2. Click on Network > Network Topology > +Create Network

create network screenshot

3. Enter a network name, for example, username_net.

create network screenshot

4. Enter a subnet name, username_subnet, and a network address, 10.10.10.0/24.

create network screenshot 2

5. Click Create to create the new network.

create network screenshot 3

6. Click on +Create Router.

network topo 2 screenshot

7. Enter a router name, username_router, connect to the External Network: public, click Create Router.

create router screenshot

8. Connect your private network to the router. Click on the router you just created then +Add Interface.

connect private screenshot

9. Select the subnet you just created from the dropdown list. Click Add Interface.

add interface screenshot

10. The network should now be connected to the new router. The end result will look similar to this diagram.

network diagram screenshot

Part 2: Create Firewall Rules

Each project has its own firewall rules. You will need to add at least an ssh rule to allow yourself to ssh into your instances.

  • On the left side bar, click on Compute > Access & Security > Security Groups
  • Find the default group, then click Manage Rules
  • Click Add Rule
  • Rule: Choose SSH from the dropdown
  • Click Add

Part 3: Upload SSH Key

You will need to upload at least 1 ssh public key in order to access the instances you create.

  • On the left side bar, click on Compute > Access & Security > Key Pairs
  • Click Import Key Pair
    • Key Pair Name: username_key
    • Public Key: your_public_key
    • Click Import Key Pair

Snapshots

Creating snapshots and new Glance images from the command line

When you have created a custom workflow or configuration on the API side, you can create a snapshot for your own use. In OpenStack, an instance snapshot is an image. The only difference between an image that has been uploaded directly to glance and an image you create by snapshot is that an image created by snapshot has additional properties in the glance database and defaults to being private. You can create an snapshot from a running server instance, but if you want to preserve data, you must shut down the source VM and verify the instance status is SHUTOFF before creating the snapshot.

To create the snapshot from the command line:

nova image-create --poll instance-name snapshot-image-name

(e.g. nova image-create --poll my-CentOS7-instance MyCustomCentos7Image-Feb-7-2017)

Snapshots won't show in Horizon (this has been submitted as a bug and may be fixed real soon now) but for now, to make it visible you'll need to export it and bring it back as a Glance image. If you just plan to use this within your project and from the command line only, the rest of the steps aren't necessary.

glance image-download UID --file whatever_file_name_you_like.raw

(e.g. glance image-download 569677d8-c7b0-4606-86d8-7673a5ecd5cf --file c7custom-image.raw)

Then bring it into Glance - e.g.

glance image-create --name "My-Custom-Image-Name"
--visibility public --disk-format raw --container-format bare
--property skip_atmosphere=yes --property hw_disk_bus=scsi
--property hw_scsi_model=virtio-scsi --property hw_qemu_guest_agent=yes
--property os_require_quiesce=yes --file c7custom-image.raw
Note

There are a lot of metadata tags there but those are important to insure that your instances will create properly from the stored image. You definitely want to make sure you get them all.

Boot the new image. Test it. Make sure it works. Do this before deleting. Please. Once it's gone, it's really gone. Be sure.

Delete your snapshot if you no longer need it. For example:

glance image-delete 569677d8-c7b0-4606-86d8-7673a5ecd5cf

Wrangler

Using Jetstream to access Wrangler data collections with iRODS

Jetstream allows you to quickly spin up new science resources in a managed cloud. You can even leverage other XSEDE resources, such as Wrangler, to retrieve or store additional data sets.

Wrangler iRODS accounts Wrangler iRODS accounts must be requested. They are not automatically generated upon account creation. Please see the Wrangler user guide for more information.

Step-by-step guide

  1. To get started, choose one of the following featured images as source for a new Jetstream node:
    1. Ubuntu 14.04.3 Development
    2. Ubuntu 14.04.3 Development GUI
    3. CentOS 6 (6.8) Development
    4. CentOS 6 (6.8) Development GUI
    5. Centos 7 (7.2) Development
    6. Centos 7 (7.2) Development GUI
    7. Intel Development (CentOS 7)
  2. Click on the Launch button to create a new instance based on this image.
  3. After the node finishes deployment, a compatible client package for iRODS will already be installed.
  4. Run isetup.
  5. At the prompt, enter the username used by Wrangler for the collection. There is no need to match usernames within the Jetstream node and iRODS usernames. Multiple Jetstream usernames can access the same iRODS collection.
  6. Next, you will be prompted to choose either the IU or TACC iRODS instance. Although both iRODS zones are linked, they contain separate data sets. It is best to communicate directly with the instance where the data collection is stored.

The isetup script will create the initial iRODS environment file, "irods_environment.json" in the .irods folder in your home directory and ask to overwrite it, if it already exists.

atmosphere login screenshot

Figure 1: Running isetup on Jetstream Atmosphere

Once the proper environment is set, initialize a session using iinit using the Wrangler account credentials, which will use the Wrangler password for the account you specified. This creates a temporary credential which will be valid for two weeks. Once connected, the file system can be navigated and the standard iRODS commands are available.

The ihelp command lists the common iRODS commands, such as ils, ipwd, iget, iput, and imeta, which are used to transfer files to iRODS or perform tagging, queries, or other operations. Detailed instructions on using these commands can be found at the iRODS site.

Any files submitted into iRODS are given a checksum, and access is logged for reads and writes. These files will then also be available via the Wrangler data analysis system on XSEDE. It is possible to prototype program flow using Jetstream, and then move to Wrangler to perform larger calculations on the same data collection.

Volumes

Volumes: Small virtual filesystems that may be attached to the user's running/active Instances.

Files/data saved to a Volume are maintained across successive attachment/detachment actions to the user's Instances.

Volume actions with the Atmosphere interface:

Volumes can also be controlled directly from the OpenStack API.

Cost and Size:

While Volumes are available to facilitate research at no additional Service Unit charge, and may be requested during initial or supplemental Jetstream allocation requests, large capacity storage is beyond the scope of Jetstream.

Users who have been approved for Volumes are limited to 10 volumes with an aggregate capacity of 500 GB across all.

Project and Providers:

As with Instances, Volumes are associated/organized with User Projects and with particular Providers (e.g. IU or TACC).

Sharing:

Volumes may only be attached to one active Instance at a time.

Volumes may be shared using standard methods (e.g. NFS) to other active Instances within Jetstream.

Backup & Exporting:

Users should regularly [backup[(#volumes:backup) (via ssh, rsync, tar, or the like) any critical data contained on Volumes as no automated backup functions are currently provided by Jetstream.

Create a Volume

A Volume must be created in the Jetstream Atmosphere interface before it can be attached to an active Instance.

From the Jetstream Atmosphere Dashboard:

  • Click on "Projects"
  • Select your Project
  • Click "New"
  • Select "Volume"

Figure 1: Volume creation dialog

  • A window will pop up (Figure 1) in which you can:
    • Name the Volume
    • Enter the desired volume size (subject to your allocation limit)
    • Select the Provider on which the Volume will reside
  • Click "Create volume"
  • After creation, the volume will appear in your list of available volumes (Figure 2)

Figure 2: Available volumes in a project

Attach a Volume

In order to access a Volume, it must be Attached to an active Instance.

From the Jetstream Atmosphere Dashboard:

  • Click on "Projects"
  • Select your Project
  • Select the desired Volume and click "Attach"
    • NOTE: You will only be able to select Active Instances running on the same Provider as the Volume
  • The Volume will now be automatically mounted in the selected instance, e.g. on device /dev/sdb as /vol1
[USER@js-169-6 ~]$ df -kh
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda1        59G  3.0G   54G   6% /
tmpfs           7.8G  148K  7.8G   1% /dev/shm
/dev/sdb        9.9G  151M  9.2G   2% /vol1
* NOTE: A _Volume_ may only be attached to one active _Instance_ at a time. However, _Volumes_ may be shared using standard methods (e.g. NFS) to other active _Instances_ within Jetstream. After attaching, the _Volume_ status in the Atmosphere interface will change to reflect the _Instance_ to which it is attached. **Figure 3:** Volume status after attaching to an instance

Detach a Volume

In order to remove a Volume from an active Instance, it must be detached.

Files and data saved to a Volume are maintained across successive attach and detach actions.

From the Jetstream Atmosphere Dashboard:

  • Click on "Projects"
  • Select your Project
  • Select the desired Volume and click "Detach"
    • WARNING: If data are being written to the volume when it is detached, the data may become corrupted. Therefore, we recommend you make sure there are no data being written to the volume before detaching it.
  • Click "Yes, detach the volume"
    • NOTE: A Volume will fail to detach if it is in active use on the Instance to which it is attached.
  • After the Volume has successfully detached, it may be subsequently re-attached to any of the user's active Instances.

Backup and Export a Volume

Users should regularly backup any critical data, especially data contained on Volumes, as no automated backup functions are currently provided by Jetstream.

Listed below are examples of methods to transfer files from your Volume:

SCP, SSH, and TAR

scp: One simple way to transfer the data on your Volume is using scp:

$ scp /vol1/file remote-user@destination_server_ip:/path/

tar with ssh: Use tar with ssh to transfer an entire directory or a file:

$ tar czf - /vol1/directory-or-file | ssh remote-user@destination_server_ip 'tar xzf -'

tar with ssh: As above, but leave the files in a remote archive:

$ tar czf - /vol1/directory | ssh remote-user@destination_server_ip 'cat - > archive.tar.gz'

DD with SSH

dd with ssh: Use dd with ssh to copy your entire Volume to a remote image file:

$ df -kh
    
    $ dd if=/dev/sdb | ssh remote-user@destination_server_ip 'dd of=vol1.img'
    

RSYNC

rsync: Use rsync to transfer an entire directory without encryption:

$ rsync -avtr /vol1 remote-user@destination_server_ip:/path/

rsync with ssh: Use rsync with ssh to transfer an entire directory with encryption and between different local and remote users:

$ rsync -avtr -e 'ssh -l local-user' /vol1 remote-user@destination_server_ip:/path/

Globus Connect file transfer tools

Globus Online is a fast, reliable, and secure file transfer service for easily moving data to, from, and between digital resources on the Extreme Science and Engineering Discovery Environment (XSEDE).

Please refer to the Globus Connect Personal GUI documentation for detailed instructions on installing and using Globus Connect Personal on your Jetstream VM.

You will need a Globus account (sign up at http://www.globus.org/app/signup) and a valid SSH key (see instructions here if necessary) prior to the installation of Globus Personal Connect software.

Please note: You will need Tcl/TK for the GUI Globus tools.

To install Tcl/Tk you need to execute with root privileges:

For Ubuntu:

# apt-get install tk tcllib

For Fedora, CentOS, RedHat:

# yum install tk tcllib
Problem Possible Solution

VM gets stuck trying to provision the network

  1. Refresh: Refresh your browser and confirm that the Atmosphere interface has synchronized with the latest Jetstream information.
  2. Wait: Please be aware that while provisioning is generally rapid, it can sometimes take several minutes during times of heavy activity. (wait 5-10 minutes)
  3. Redeploy/Reboot: Click Projects (on the top menu) and select the Project with the instance that is stuck. Then click on the name of the stuck instance. From the Actions menu on the right, try Redeploy first (waiting about 2 minutes before proceeding), Reboot if that fails (waiting about 5-10 minutes before going to the next step), and Hard Reboot if the first two actions fail (waiting about 5-10 minutes).

If none of these Actions work, DO NOT DELETE THE INSTANCE. Make note of the Alias (UID) and IP Address of the Instance, and email that information to help@jetstream-cloud.org or click the Feedback & Support button.

Hostname mismatch

If you are having hostname mismatch issues with some software, try the following:

  1. Type su
  2. Type crontab -e
  3. Insert this line: @reboot /bin/hostname | awk -F'.' '{print $1}' > /etc/hostname
  4. Save and Exit

Volume fails to detach

  1. make sure the instance to which the volume is attached is actively running
  2. make sure no jobs or process are running on the instance are using the volume

The command fuser -km /vol_b will terminate all processes usign the volume named vol_b

Volume fails to attach

  1. make sure the instance to which the volume should be attached is actively running

My instance has been locked by an administrator

  1. Are you running an insecure web-service like Apache or MongoDB with no security features enabled ?
    1. MongoDB: https://docs.mongodb.com/manual/administration/security-checklist/
    2. Apache: https://httpd.apache.org/docs/2.4/misc/security_tips.html

Web-Shell (Gate-One) does not connect (via ssh)

If your Web-Shell / Gate-One stalls at a prompt like: Host/IP or ssh:// URL [localhost]:

  1. Close the session by clicking on the "X" in the top-right menu bar:
  2. Close the browser window/tab
  3. Go back to your Jetstream Atmosphere window and re-click on the link

Web Desktop does not launch

Try using the instructions for an external VNC viewer.

API command line access to TACC fails with "requires authentication" error

Add the following two lines to the openrc.sh script if they are not there:

missing TACC openrc.sh env variables

export OS_USER_DOMAIN_NAME=tacc
export OS_IDENTITY_API_VERSION=3

btrfs in conjunction with OpenStack's ceph has known kernel bug

please use a different file system. e.g XFS, EXT4

Mouse does not work in web desktop

Laptops with touchscreens are unable to use the mouse in the web desktop using Chrome and Firefox on Windows 10.

There is a bug in the web browser code in both Chrome and Firefox for using a touchscreen PC with the web desktop of Jetstream. This is not a bug we can easily work around since it's in the browser code itself.

While we do not recommend using Internet Explorer, it does not have the issue. Using a VNC Viewer also is a viable workaround.

web-shell works for my Ubuntu 16.04 version or newer instance, but I cannot ssh to it directly from an external ssh client

Are you using DSA keys? DSA keys are deprecated for Ubuntu 16.04 and newer instances. Please use RSA keys.

Users with a brace " { " or parenthesis "(" in their passwords may experience errors with OpenStack command line commands.

  • There is a known issue with braces in openstack passwords when using the OpenStack command line client. At this time, do not use a brace in your password.
  • A potential work around is to enclose your password in single quotes in your .openrc

fail2ban has locked you out from vnc or terminal-based shell sessions

Use the web shell and do the following:

  • sudo iptables -L -n
    • Look for your IP number in that output, if it's there, proceed
  • sudo fail2ban-client status

Should show you fail2ban is working and your jail name(s):

[js-157-95] root ~-->fail2ban-client status

Status

|- Number of jail: 1

`- Jail list: ssh-iptables

  • sudo fail2ban-client set ssh-iptables unbanip YOUR_IP_NUMBER
An Introduction to Linux Cornell Virtual Workshop (full access to the training materials requires logging in with XSEDE credentials)
Learn Linux section from Linux.com website
HowtoForge user-friendly Linux tutorials
Linux Knowledge Base tutorials, forums, and how-tos for Linux
Learning the Shell learn the Linux command line
Command-line bootcamp learn all the basic skills needed to start being productive in the UNIX terminal

Policies

Good citizenship

  • Each VM burns SUs for the time it is in operation.
    • It is beneficial to you and other Jetstream users to shut it down when it is not in active use.
    • This frees up resources for other users and also preserves your SUs for future use.

Security

  • Remember to log out from the menu at the top right (where it shows your username).
    • This ensures that you do not inadvertently allow others to access your Jetstream account.
  • Update and configure software that utilizes the network to preclude unauthorized access.
    • Commonly neglected examples include MongoDB.
Title Creator Modified Using Jetstream to access Wrangler data collections with iRODS Peg Lindenlaub Nov 04, 2016 System Access Peg Lindenlaub Oct 20, 2016 Adding SSH keys to the Jetstream Atmosphere environment Peg Lindenlaub Oct 20, 2016 Logging in with SSH Peg Lindenlaub Oct 03, 2016 Quick Start Guide Peg Lindenlaub Sep 13, 2016 Launching your VM Peg Lindenlaub Aug 30, 2016 Logging in with Web Shell Peg Lindenlaub Jul 11, 2016 Logging in with VNC desktop Peg Lindenlaub Jul 11, 2016 How to add users to a running instance Peg Lindenlaub May 25, 2016
Image template of a virtual machine containing an installed operating system, software, and configuration
Instance launched image of a virtual machine
Information as a Service (IaaS) form of cloud computing that provides virtualized computing resources over the Internet. Read more...
Platform as a Service (PaaS) cloud computing model that delivers applications over the Internet. Read more...
Software as a Service (SaaS) software distribution model in which applications are hosted by a vendor or service provider and made available to customers over a network, typically the Internet. Read more...
Volumes small virtual filesystems that may be attached to running instances. Read more...