XSEDE Allocation Policies

1.0. Introduction

This document states the policies and instructions for preparing requests to use the XSEDE-allocated resources of the National Science Foundation. All interested persons are encouraged to review this guide prior to making a submission; any questions regarding these policies should be directed to the XSEDE Help Desk or email help@xsede.org. Answers to Frequently Asked Questions (FAQs) are available in the XSEDE Knowledge Base.

Proposals for XSEDE allocations have a single Principal Investigator (PI) who may request allocations on one or more resources. The PI uses the same XRAS (XSEDE Resource Allocation Service) award number to request or renew allocations in subsequent years, even though the focus and computational projects may change from year to year. There are three types of projects: Startup, Educational, and Research.

Principal Investigators (PIs) of research programs should use Research Project Requests to obtain allocations for their computational research. These requests are reviewed quarterly by the XSEDE Resource Allocation Committee (XRAC). The XRAC committee consists of researchers and computational scientists who review these submissions and make allocation recommendations for all XSEDE resources. Submissions are due 2.5 months before the quarterly review in which they will be considered.

Startup Project Requests provide a mechanism to new users for starting their research or initial experience on XSEDE resources without delay. Education Projects support classroom instruction and training classes. Startup and Education projects are accepted and reviewed continuously throughout the quarter. The startup request form was designed to make application easy, requiring only basic biography information, an abstract and a CV. A course syllabus and resource justification are needed for all educational requests.

Support for the XSEDE project is provided by NSF under award number ACI 10-53575. Support for the acquisition and operation of the allocated High Performance Computing (HPC) systems (referred to as Service Providers) is provided under separate NSF awards. Many of the policies have been derived under the direction of the XAB (XSEDE Advisory Board) and the management of XSEDE in consultation with XSEDE User Advisory Committee (UAC), SPs (Service Providers), members of the XRAC and the NSF. The allocations procedures and processes are managed and implemented through the XSEDE Allocations group.

1.1. Summary of Major Changes (as of 02/19/2015)

Most new changes focus on streamlining the allocation process, providing new features or clarifying the documentation. Any new policy updates that reflect significant changes from prior policies and request preparation instructions will be posted in this section for one year. PIs should review the relevant sections containing updates and the Resource Catalog before their next submission.

  1. Storage allocation requests for Archival Storage in conjunction with compute and visualization resources and/or Stand Alone Storage need to be requested explicitly both in your proposal (research proposals) and also in the resource section of XRAS.

  2. Furthermore, the PI must describe the peer-reviewed science goal that the resource award will facilitate. These goals must match or be sub-goals of those described in the listed funding award for that year.

  3. After the Panel Discussion of the XRAC meeting, the total Recommended Allocation is determined and compared to the total Available Allocation across all resources. Transfers of allocations may be made for projects that are more suitable for execution on other resources; transfers may also be made for projects that can take advantage of other resources, hence balancing the load. When the total Recommended considerably exceeds Available Allocations a reconciliation process adjusts all Recommended Allocations to remove oversubscription. This adjustment process reduces large allocations more than small ones and gives preference to NSF-funded projects or project portions. Under the direction of NSF, additional adjustments may be made to achieve a balanced portfolio of awards to diverse communities, geographic areas, and scientific domains.

  4. Conflict of Interest (COI) policy will be strictly enforced for large proposals. For small requests, the PI may participate in the respective meeting, but leave the room during the discussion of their proposal.

  5. XRAC proposals for allocations request resources that represent a significant investment of the National Science Foundation. The XRAC review process therefore strives to be as rigorous as for equivalent NSF proposals.

  6. The actual availability of resources is not considered in the review. Only the merit of the proposal is. Necessary reductions due to insufficient resources will be made after the merit review, under NSF guidelines, as described in Section 6.4.1.

  7. 10% max advance on all research requests, as described in Section 3.5.4

In an effort to ensure that current and potential users benefit as soon as possible, these policy changes are being rolled out while related changes to the XSEDE's Resource Allocation Service (XRAS) are being implemented. Users should watch for changes to XRAS and pay close attention to instructions within the XRAS data entry forms.

2.0. Appropriate Uses for XSEDE Allocations

XSEDE provides access to resources in high-performance computing, scientific visualization, data storage, and extended collaborative support, to enhance scientific discovery. These policies and procedures are followed to ensure a fair and efficient allocation of these resources.

XSEDE encourages and supports projects in all disciplines for a wide range of purposes. In general, XSEDE allocations policies provide for projects that

  1. Allow individuals or groups to pursue their research objectives
  2. Permit researchers or research groups to port codes, get started in the XSEDE environment, or evaluate XSEDE capabilities
  3. Support a range of educational objectives, from classroom instruction to training classes
2.1. Categories of Research Activities

Within this broad scope, XSEDE supports various categories of research groups and services. Most often, the involved projects can be classified in one of the following categories:

  1. Single Principal Investigator (PI): Most projects support a single PI and possibly a small research group working closely and co-authoring papers.

  2. Multi-PI, Large Research Collaborations: Projects of this type are characterized by a single PI representing a large group of collaborating co-PIs who are working on sub-projects within the overall collaboration. A single request is submitted, and a single project is allocated. The management of the allocated resources is left to the discretion of the principals on the request.

  3. Large-scale Consortiums: Projects of this type are intended to support large-scale, funded projects that work together as a consortium. Often in these cases, a mechanism already exists for allocating community or project resources (e.g., an instrument such as a telescope or detector), and that mechanism will also be used to make allocations from the time granted to the consortium project to individual investigators. Requests for this type of project typically describe the internal processes for managing access of individual investigators within the consortium.

  4. Gateways or Community Services: Projects of this type provide services to a large community of users who are typically not directly collaborating with the project PI. An example of such a project would be an application portal service providing access to software and computer time to a community of biology researchers via a web-based interface. Requests to provide such a service must describe the details of the services provided, the methods used, the expected consumption of resources, and mechanisms for monitoring the users and usage of the service. Statistics of community usage should be reported quarterly and in renewal requests for resources, progress reports, and end-of-project reports. XSEDE provides How to Write a Winning Gateway Proposal to assist in writing these requests.

2.2. PI Eligibility

The Principal Investigator (PI) of an allocated project is the person responsible for the accuracy of the resource request and the management of the ensuing allocation. Most commonly the PI is a researcher or educator at a U.S. academic or non-profit research institution. A PI may not be a high school or undergraduate student; a qualified advisore.g., a high school teacher or faculty member, must serve in this capacity. In most cases, a graduate student may not be a PI. A post-doctoral researcher is eligible to be a PI. See Section 8 for exceptions.

A U.S.-based researcher, engineer, or scholar who has a joint appointment with a university or non-profit research institution may submit a request using that affiliation. The appointment may be adjunct, instructional, or any other official position.

Research staff from federal and state agencies or federally funded research and development centers (FFRDCs) are also welcome to apply for an XSEDE allocation if their agency or center does not typically provide research staff with access to non-XSEDE High Performance Computing (HPC) resources of adequate scope for the planned research. The lack of access to such resources should be documented in the request, as described in Section 7.3.

Section 8 lists further categories of individuals who are eligible to request XSEDE allocations. The most common class of ineligible PIs are researchers based outside of the United States. While XSEDE supports collaborations between U.S.-based PIs and their foreign colleagues, PIs on XSEDE requests and allocations must be based at a U.S. institution.

Allocated projects need not be supported by NSF grants; PIs may have support from any funding agency or funding source. Furthermore, while most requests do have peer-reviewed funding awards, a PI need not have such a grant in order to submit resource requests; however, the scientific merit of requests for use in research not supported by peer-reviewed funding will be evaluated by XSEDE reviewers. See Section 7.1.

3.0. General Allocation Policies

To support the wide range of research objectives and project uses described above, the XSEDE allocations process defines three types of projects.

  1. Startup: Startup projects target new users planning to submit more substantial requests in the future as well as users who have modest computational requirements that, nevertheless, cannot be satisfied by their campus resources.

  2. Education: Education projects target faculty or teachers intending to use XSEDE-allocated resources for classroom instruction or training classes related to the use of advanced cyberinfrastructure technologies.

  3. Research: Research projects are designated for projects that have progressed beyond the startup phase and are conducting production usage of XSEDE-allocated resources in pursuit of their research objectives.

Details specific to Startup and Education projects, including allocation size restrictions, are described in Section 5. Details specific to Research requests and projects are covered in Section 6. However, projects of all three types are subject to a common, basic set of policies and procedures, which are summarized in this section.

3.1. XSEDE-Allocated Resources

XSEDE manages the allocation requests and awards for a set of NSF cyberinfrastructure (CI) resources. These generally can be categorized into computational, storage, and specialized resources. For a full listing of resources and architecture details, please refer to the XSEDE Resource Overview.

Computational resources are typically High Performance Computing (HPC) systems oriented toward batch job submission. They include systems of various architectures and from many vendors. Note that computational resources typically have associated disk systems for short-term data storage by users.
Storage resources include both disk-based systems for medium- and long-term storage of data and tape-based systems for long-term storage. Some storage resources available to users are not part of the allocations process (e.g. scratch storage)

Specialized resources include dedicated visualization systems, systems for virtual hosting, and high-throughput computing.

Extended Collaborative Support Service (ECSS) resources include staff experts from the XSEDE partner sites whose time is devoted to working with allocated projects to accelerate progress toward its research objectives. ECSS support may include working on code performance, adapting codes to XSEDE capabilities, improving gateway capabilities, and so on.

3.2. Submission of Allocation Requests

Allocation requests are only accepted electronically and must be made through the XRAS system via the XSEDE User Portal (XUP). Requests include annual submissions to initiate or continue allocations as well as manage allocations during an allocation period.

The XRAS system recognizes two "Opportunities" submission types and five "Request Actions", which can apply to all three types of "Projects" (Startup, Educational and Research):

Opportunities:

  1. New: A submission is considered "new" if any of the following conditions are met:

    • a PI's first submission to XSEDE, or
    • a request in a completely different area of science than a PI's previously submitted requests, or
    • a PI's first request for a particular project type (Startup, Education, or Research).

    For example, a PI with a prior or current Startup allocation but now submitting a Research request is considered a new Research request.

  2. Renewal: A submission is classified as a "renewal" if a PI meets the following conditions:

    • has had an allocation of the same request type, which was active within the past two years
    • is continuing the same or similar line of research
    • is in the same field of science.

    A renewal request must address the progress of the research supported by the prior allocation.

Request Actions

  1. Supplement: A supplement is a request for additional resources during an existing allocation's one year time frame. Its purpose is to support changes in the original computational research plan that are required to achieve the scientific goals of the project. This may include altered or new projects or support for projects proceeding more rapidly than anticipated or that require more resources than anticipated (see Section 3.5.3). Supplement awards are highly dependent upon availability of resources and limited when allocation awards at the previous XRAC meeting have been reduced to eliminate oversubscriptions. Supplements are not a mechanism to acquire additional resources for awards that were recommended for less than the amount originally requested (see Justification).

  2. Transfer: Transfers (Section 3.5.5) of allocated units between XSEDE resources are requested through XRAS. No formal documentation is required.

  3. Extension: Extensions (Section 3.5.6) of allocation periods beyond the normal 12-month duration are requested through XRAS. A brief reason is required but no formal documentation is needed.

  4. Justification (Appeal): If the XRAC has requested that the PI give more details or justification before the project can be allocated any time or any additional time, the PI should submit a justification (appeal). Also, a PI may submit a rebuttal on a review criticism as a Justification (Appeal) request. This type of submission should specifically address information that the reviewers have requested or have criticized. (See Section 6.4.4) This request type does not apply to Startup or Education requests; PIs for those types should submit another, revised Startup or Education request. Justifications are not considered for rejected proposals.

  5. Advance: You can request an advance of up to 10% of your submitted Research request. Usage against this advance will be deducted from your eventual allocation award. Approval of the advance is subject to review by the Service Provider, and the availability of sufficient Service Units on the selected resource(s).

3.3. One Project per PI

An individual may be a PI on only one active XSEDE allocation request/project at a given time. Several distinct research activities can be combined in a single resource request, however, the resource request for each activity must be justified, and any allocation-size limits apply to the aggregate request.

The single-project rule is designed to minimize the effort required by PIs for submitting resource requests and the overhead to the XSEDE process for reviewing those requests. While PIs may have several different funded grants that require computational support, these should be included as sub-projects within a single request.

There are several exceptions to this rule:

  • A PI may submit a Research request at any time during an active Startup allocation. A successful Research allocation will supersede any Startup allocations and start a new 12-month allocation period.

  • If a PI has research activities that are in significantly different fields of science and, therefore, cannot be reviewed by the same set of reviewers, multiple requests/projects will be considered (e.g. physics and biology).

  • If a PI is leading an individual research activity as well as a community consortium activity or community service activity, separate requests for these distinct activities will be considered.

  • A PI with active Startup or Research projects may submit requests for any number of Education projects to keep classroom and training users separate from research activities.

Similarly, to minimize the effort required to gain access to XSEDE-allocated resources, closely collaborating researchers should submit a single collaborative request rather than several individual requests. For example, a PI and associated post-doctoral researchers; investigators supported by the same funding grant; and researchers in the same lab group should consider submitting a request describing and justifying the various subactivities. One of the collaborators is designated as the PI, and others can be designated as co-PIs. For further guidance on whether multiple requests or a single, collaborative request is appropriate in a specific case, please contact the XSEDE Help Desk.

3.4. Allocation Duration

All allocations for XSEDE resources are made for a 12-month period. PIs can continue their activities in subsequent years through annual renewal requests (Section 3.5.1).

3.4.1. Forfeiture of Unused Allocation Units

At the end of each 12-month allocation period, projects forfeit any unused compute SUs (see Section 4.2.1). For storage allocations that have used less than their allocated space, a PI must justify continued need for the extra space or risk a reduced allocation amount in subsequent years. However, see Section 3.5.6 below.

3.5 Allocation Management

Once a project has received its allocation, XSEDE gives the PI and any co-PIs a number of options to ensure the flexibility needed to complete their research objectives.

3.5.1. Project Continuation

The most important consideration for PIs is whether they wish to continue computing after the expiration of their current allocation period.

In general, Startup allocations are not eligible for continuation or renewal; PIs are expected to submit Research requests to a XRAC meeting during their Startup allocation period to continue pursuing their research objectives. Certain exceptions apply; see Section 5.2.

For single-year Research awards, the PI should submit a Renewal request to the XRAC for consideration. In most cases, they should submit this request approximately one year after their initial request submission, so that it can be reviewed and awarded to avoid any interruption. In general, the appropriate submission period is the one approximately three months prior to the expiration of their allocation.

3.5.2. Additional Users

A PI may share his/her allocation by establishing accounts under the allocation with any number of collaborators, including graduate or undergraduate students. Research collaborators may include colleagues based outside of the U.S. Additional users can be added through the XSEDE User Portal (under the My XSEDE tab). Each user must first create their own portal account.

PIs on Education allocations can contact the XSEDE Help Desk for assistance in establishing accounts for large groups of students under their allocated project.

3.5.3. Supplement Requests

To request additional resources during an allocation award period, a PI may submit a Supplement Request for resources through XRAS via the XSEDE User Portal. Supplement Requests are accepted at any time and are generally considered within two to three weeks of submission. However, large requests must be reviewed by the panel along with new and renewal Research requests at the next XRAC meeting. Generally, supplemental allocation requests for startup awards are capped at the startup award limits for the particular resource. All other supplemental allocation requests have no limit.

The Main Document of the Supplement Request should contain the following: minimal information about the science and methodologies concerning the supplement; a description of new projects or why more resources are needed to complete the project in progress; a work plan that includes justifications that detail the new resource requirements; and any new efficiencies in codes or methods.

Do not upload the Main Document of the original request. Since a Supplement Main Document addresses the concerns of only an additional allocation it should be appropriately short compared to the Main Document of the original request and no greater than five pages.

A Progress Report is also required and should contain the following:

  1. New scientific discoveries

  2. Computational accomplishments of the previous computational work plan (list any SUs used)

  3. Summary of publication information, including conference presentations, technical reports, etc. (Put the Publication list in a separate document, but report the number of publications and other proceedings and reports in the Progress report.)

  4. Contributions to other research efforts and fields of science (experimental/computational/instrumental, etc.).

In practice, to ensure a group can continue computing without an interruption, a PI should not wait until an allocation is completely spent before submitting a Supplement request. Please allow at least two to three weeks for review and processing.

Supplemental allocations to Startup and Education projects are constrained by the allocation-size restrictions on these projects (see Section 5.0). PIs should submit Supplement requests via the XRAS system.

3.5.4. Advances

Upon submitting a new or renewal Research request, PIs are allowed to ask for an immediate advance of up to 10% of the resources requested in anticipation of a favorable review. The size of the advance will depend on current availability on the target resource and staff review. Any usage of this advance will be debited against the eventual allocation. Advances are accepted up to three weeks before the upcoming allocation period.

3.5.5. Transfers

PIs can request that allocated time be transferred from one platform to another, subject to resource availability on the target resource. On Research allocations, such transfers are subject to conversion of SUs according to established weighting factors (see Section 4.2.2). On Startup allocations, transfers are subject to the size restrictions for the target resource (see Section 5). PIs must submit transfer requests via the XRAS system.

3.5.6. Extensions

Projects that encounter problems in consuming their allocations, such as unexpected staffing changes, can request an extension to their allocation end date. In such instances, PIs may request a single extension of an allocation, extending the expiration date by a maximum of six months The PI will be asked to specify the project number and the length of the extension (1-6 months), along with a brief reason for the extension.

Note that Extensions apply to all current resource allocations on the same project; PIs cannot extend some allocations and submit overlapping Renewal requests for other resources.

If a project still has unused allocation units at the end of the extended allocation period, the PI must submit a Renewal request to continue their XSEDE project; the request should explain the reasons for the unused allocation.

Extension requests should be submitted via the XRAS system.

3.6. Acknowledging Support

An acknowledgement of support from the appropriate service provider and XSEDE should appear in any publication of material, whether copyrighted or not, that describes work which benefited from access to XSEDE cyberinfrastructure resources. For suggested language, see How to Acknowledge XSEDE.

PIs should include a bibliography of articles or other manuscripts - published, accepted, submitted or in preparation - that benefited from support by XSEDE-allocated resources as part of their annual Progress Reports and Final Reports (see Section 6.3.2).

4.0. Selecting Resources and Calculating Allocation Sizes

PIs who need the special capabilities of particular resources can specify those resources in their requests. This section contains information related to selecting resources and calculating allocation amounts for all requests.

For each resource available for allocation, a statement of recommended use is available in the XSEDE Resources Overview. These guidelines describe Service Provider policies or usage models that may be emphasized or, conversely, may not be permitted on certain resources. PIs are advised to review these guidelines before submitting requests for specific resources.

4.2. Calculating Resource Requests

The requested allocation amounts need to be detailed and justified in the supporting documents. Allocation amounts for compute, storage, and/or Extended Collaborative Support Services should be clearly linked to the scientific goals and the proposed research plan.

  • For compute resources, calculations should be based on small-scale test runs on the resources of interest and scaling tests, as well as the number and type of runs to be made.

  • For storage resources, calculations should describe the data to be stored and the amount of storage required for the data (record/file size, number of files/records, and/or other relevant characteristics). Also the rationale for persistent storage on XSEDE allocated storage resources should be provided. Storage resource allocations are made directly in gigabytes.

  • A request for Extended Collaborative Support Services (ECSS) to enhance the effectiveness and productivity of work can accompany an application for resources at any level, or it can be submitted stand-alone as a Supplement to an active grant. It requires the Principal Investigator (PI) to answer five (5) questions in the Resource request section of XRAS. This request will be reviewed, usually in conjunction with your allocation proposal. ECSS management will then assess whether there is also a good potential match with the experts available to collaborate with your team. For more information please visit the ECSS page.

4.2.1. Definitions for Allocation Units

Different resource types define different units for making resource requests. The XSEDE Resource Catalog shows the allocation request unit for each resource. Throughout the remainder of this document, the following resource types and allocation units are used:

Compute resources (including most visualization resources) are requested in terms of "service units" (SUs). In general, 1 SU equals 1 processor core-hour, or one wallclock-hour on one processor core, on a given resource. Local resource policies may define more complex formulas for SUs that include such factors as queue priority and so on. A submission that requests allocations on several resources should specify each of those requests in terms of local SUs.

Storage resources are requested in terms of gigabytes (GBs).

ECSS is requested in terms of 1 (yes) or 0 (no). Also, the disposition of ECSS requests should be explained clearly: the reviewers provide a recommendation regarding the suitability and importance of the collaborative project between the PI's group and ECSS experts. ECSS management follows up by assigning a staff expert to work with the PI to formulate a project plan which must be agreed upon between the PI and ECSS management within 45 days.

Specialized resources may define specialized request units. Refer to the XSEDE Resources Catalog for the units used in requesting allocations on these resources.

4.2.2. Converting Units Between Compute Resources

When allocated units are transferred from one compute resource to another, a conversion is applied to normalize the relative computational power of the two resources. The most common situation is a transfer of SUs from one compute resource to another. In this case, the standard conversion is generally based on a formula derived from the resource's performance on the HPL benchmark. Users can review the standard conversions using the SU Conversion Calculator.

Alternate conversion rates. If a PI has performance results of his/her group's own code on the source and target resources in a transfer request, the PI can request a conversion factor based on that code's performance be applied. The alternate conversion rate should be documented and noted in the transfer request.

At this time, it is not possible to convert allocable units on one resource type to units on another resource type. For example, storage allocations cannot be converted to compute allocations.

5.0. Startup and Education Requests

To enable researchers to quickly gain access to XSEDE-allocated resources, XSEDE provides two small-scale project types, both of which require a project abstract and information about the PI (see next section). Education requests require a course syllabus and resource justification.

  • Startup projects target researchers getting started on XSEDE, developing code, testing applications, and other similar purposes. PIs making Startup requests can select from a full list of available resources.

  • Education projects are specifically designated for class instruction or training activities. Given the difference in sizes of the new HPC systems, Startup and Education allocations can be as much as 200,000 SUs on the largest systems. The limit on the smaller systems ranges from 1,000 to 50,000 SUs. The grand total number of SUs in a Startup or Education request cannot exceed 200,000 SUs. The Startup allocation limit will be posted in the XSEDE Resource Catalog for each resource. In addition, transfers to a single resource on a Startup allocation cannot exceed that resource's Startup limit. The XSEDE reviewers and allocations staff will enforce these limits. Startup and Education allocations are for a period of one year.

5.1. Submission

Startup and Education requests can be submitted at any time via the XRAS system. The requests are reviewed and processed continually. An individual request is typically reviewed and processed within two weeks of submission.

Within XRAS, Startup and Education requests primarily require basic information about the PI, any co-PIs, and funding grants supporting the work being conducted. The XRAS forms also request an abstract describing the work to be conducted.

An abstract with a general description of the work may be sufficient, but additional information that will enhance your request may include explanations for why the project needs access to XSEDE-allocated resources, how the estimate of resources request was reached, and how the resources requested were selected.

A curriculum vitae (CV) for the PI and each co-PI is required. CVs should be in standard NSF or NIH format (two page maximum). The Documents upload section of XRAS will expect a CV upload for each co-PI (and the PI) entered in the Co-PI information area. PIs having NSF Fellowship or Honorable Mention awards must attach a copy of their NSF award letter to their request.

5.2. Renewal

In general, Startup allocations are presumed to be for one year only. However, renewals will be permitted with appropriate justification and subject to XSEDE reviewer approval; researchers may be directed to submit Research project requests to the XRAC. Valid rationales for renewing Startup projects may include progress on Startup work that has proceeded more slowly than anticipated, or lack of access to campus resources to support the research activities being proposed even at the scale of XSEDE Startup allocations. This may include the need to access specialized resources or larger-scale resources not typically available on campuses.

Education projects can be renewed indefinitely as long as the education activity - for example, a class repeated each academic year - is continued.

In submitting a renewal for a Startup or Education project, a PI is required to include a Progress Report describing how the prior year's allocation was used and a listing of any publications (published, accepted, submitted or in preparation) that resulted from the prior year's allocation.

5.3. Review and Processing

Each Startup and Education request will be reviewed by at least one XSEDE staff reviewer, and if the first review rejects the request, a second review will automatically be conducted. For Startup and Education requests, each SP will define a primary reviewer, and the Allocations Coordinator or designee will assign the reviewer from each SP whose resource has been selected by the PI. The Education & Outreach Director will appoint reviewers for Education requests; the XSEDE ECSS Area Directors will appoint reviewers to evaluate XSEDE Extended Support requests.

PIs for Startup and Education requests are notified by email when reviews are completed.

6.0. Submitting Research Requests

For projects that have progressed beyond the Startup phase, a Research request is appropriate. Research requests include a well-documented resource-use plan that describes how the requested allocations are necessary and sufficient to accomplish the project's research objectives. A single Research request and resulting project may combine compute, storage and Extended Collaborative Support Services Program requests.

Research requests of all sizes are submitted for consideration to the XSEDE Resource Allocation Committee (XRAC). There is no limit to the size of allocations that can be provided to Research requests; however, the Allocation Coordinator may determine that some smaller-scale Research requests can be processed via staff-only review.

6.1. Deadlines for Submissions

The XRAC submission deadlines are January 15, April 15, July 15, and October 15. These deadlines are approximately six weeks prior to the meeting at which they will be reviewed, and 10 weeks prior to the allocation start date. The corresponding allocations begin April 1, July 1, October 1, and January 1, respectively, and the corresponding allocation end dates are therefore March 31, June 30, September 30, and December 31.

In general, once a PI enters the allocations process, the PI should plan to submit a Renewal request at the same deadline each year to avoid interruption in the project's allocations, assuming they consume their allocation within the 12-month period. (See, however, Section 3.5.6 on extending projects that are proceeding more slowly than anticipated.) All potential and continuing Research projects are encouraged to keep these deadlines in mind for their planning purposes.

6.2. XRAS Entry Forms

The following elements are entered directly into XRAS web forms as part of the submission.

  • PI and Co-PI Information: The entries consist of the standard cover sheet information regarding the PI and any co-PIs.

  • Project Information: The project title, relevant field(s) of science, the project abstract and keywords. Accurate primary and secondary fields of science ensure that a request can be assigned to experts within the appropriate domain.

  • Supporting Grants: Information regarding any grants, awards, funding or other support provided to accomplish the goals of the submitted request. These should be the annual dollar amounts that have been provided for support to the entire project, not just the budget for computing-related activities. If the supporting grant funds activities beyond those proposed for XSEDE-allocated resources (such as experimental work, or work by other collaborators), the PI must specify the percentage of the annual funding for the grant that will be associated with the activities to be supported by XSEDE resources. Furthermore, the PI must describe the peer-reviewed science goals that the computational award will facilitate. These goals must match or be sub-goals of those described in the listed funding award for that year.

  • Resource Request: The resources being requested, along with a summary of the resource needs (i.e., responses to the questions associated with each resource).

  • Request Documents: For Research submissions, specific documents are required as described in Section 6.3.

6.3. Research Request Documents

The supplied documents must discuss the work planned, justify the resource request, and provide any relevant additional information (e.g., Special Requirements or Code Performance and Scaling). The Main Document must adhere to page length rules. See Section 7 for guidelines in crafting a successful submission.

6.3.1. Document Formatting

While readability is of greatest importance, documents must satisfy the following minimum requirements. Documents that conform to NSF proposal format guidelines will satisfy these guidelines.

  • Margins: Documents must have 2.5-cm (1-inch) margins at the top, bottom, and sides.

  • Fonts and Spacing: The type size used throughout the documents must conform to the following three requirements:

    Use one of the following typefaces identified below:

    • Arial 11, Courier New, or Palatino Linotype at a font size of 10 points or larger;
    • Times New Roman at a font size of 11 points or larger; or
    • Computer Modern family of fonts at a font size of 11 points or larger.

      A font size of less than 10 points may be used for mathematical formulas or equations, figures, table or diagram captions and when using a Symbol font to insert Greek letters or special characters. PIs are cautioned, however, that the text must still be readable.

    • Type density must be no more than 15 characters per 2.5 cm (1 inch).
    • No more than 6 lines must be within a vertical space of 2.5 cm (1 inch).
  • Page Numbering: Page numbers should be included in each file by the submitter. Page numbering is not provided by XRAS.

  • File Format: XRAS accepts only PDF file formats.

6.3.2. Required and Optional Documents

Submissions should include the following required documents and may include several optional documents. No other documents will be reviewed. This set of information corresponds to what is currently requested by XRAC reviewers and largely to what is provided in current submissions, but it does expand the total permissible page count for many types of submissions. Many projects are not expected to use the full page-limit in all sections. Concise submissions are appreciated.

  • Main Document: Required, 10 or 15 pages (5 pp. for Supplement Request). This document should have the scientific background, research objectives, and the justification of the request for all resources and resource types. The reviewers will focus on this document, and the document should include sufficient detail to permit reviewers to make a recommendation. As with a renewal research request, the Main Document is required, however, the previous year's Main Document can be re-used with updates to the computational plan and justification for the upcoming year's research. If there are changes to the scientific direction this should be addressed in the Main Document as well. The following page limits apply:

    • Single-Year Request for more than 10 million SUs: 15 pages.
    • All Other Requests: 10 pages.

    IMPORTANT: If page limits for the Main Document are exceeded, the Allocation Coordinator will return the submission without review.

  • Progress Report: Required for renewals and supplements, three pages. For renewal requests the reviewers also consider a PI's record of using their allocations and achieving meaningful results. This should be submitted as a separate document. The reviewers will consider this document for rating the successful use of prior allocations.

  • Publications resulting from XSEDE Support: Required for Renewals and Progress Reports. Publications should be entered in the XSEDE portal profile of one of the authors of the publication. The XRAS submission site has a section to attach publications resulting from research that benefited from access to XSEDE-allocated resources. These publications, authored by the PI or users of the allocation, may be in preparation, submitted, accepted, or published. This information is separated for XSEDE metrics purposes and should not include citations for literature cited elsewhere in the submission (see "References and Figures" below). Please see the XSEDE page about proper acknowledgments for any publication resulting from research performed on resources allocated by XSEDE. Acknowledgments

  • CVs: Required, no limit. This is includes CVs for the PI and all co-PIs. The two-page NSF or NIH format is highly recommended. For supplement or justification (rebuttals) submissions for which CVs for the same PI and co-PIs have already been submitted in the original request, this document can be omitted.

  • Special Requirements: Optional, 1 page. This document should be included if the completion of the research plan requires capabilities that fall outside a Service Provider's (SPs) standard user and usage environment. For example, jobs longer than the standard queue lengths, specific scheduling demands, or dedicated file system space. The XRAC is not required to read this document; the intended audience is the staff at the relevant SP(s).

  • Code Performance and Scaling: Required, 5 pages. This document should contain information to support claims of code performance and scaling that does not fit within the Main Document. Reviewers should not be required to closely scrutinize this document to evaluate the overall request.

  • References: Optional, no limit. A PI may separate a lengthy bibliography (for literature cited in the Main Document) in order to take full advantage of the maximum page limits. Please note that this bibliography is different from the "Publications Resulting from XSEDE Support" described above.

SUMMARY OF DOCUMENT REQUIREMENTS FOR XSEDE RESEARCH ALLOCATION RESOURCE REQUESTS

na = not applicable

REQUEST TYPE IN XRAS MAIN DOC PROGRESS REPORT PUBS RESULTING FROM XSEDE SUPPORT CVS SPECIAL RQMTS CODE PERFORMANCE & SCALING REFERENCE
PAGE LIMITS 10 pp.; 15 pp. for >10 million SU requests (see below for exceptions) 3 pp. no limit 2 pp.-format per CV; single combined document 1 p. 5 pp. no page limit
NEW Required na na Required Optional Required Optional
RENEWAL Required Required Required to be entered on the XRAS submission form Required Optional Required Optional
SUPPLEMENT Required5pp. max. Required na Not required if no change in PI/CoPIs Optional Required Optional
JUSTIFICATION Required na na Not required if no change in PI/CoPIs Optional Required Optional
EXTENSION na na na na Na na na
TRANSFER na na na na Na na na
ADVANCE na na na na Na na na
6.4. Review and Processing

Research requests are reviewed quarterly by the XSEDE Resource Allocations Committee (XRAC). The XRAC consists of volunteer experts from the faculty and staff of U.S. universities, laboratories, and other research institutions. All committee members have expertise in at least one area of computational science or engineering and serve a term of approximately three years. PI and co-PIs of allocation awards may be asked to serve on the XRAC.

The XRAC will ensure that:

  • All requests for resources that exceed a threshold level determined by XSEDE shall be peer-reviewed.
  • Written reviews of the resource requests shall be completed in a timely way and made available to the requestors and the allocation coordinators.
  • Recommendations to XSEDE for the allocation of resources based on the requests, reviews, and the available resources shall be developed and documented.
  • The environment for this process shall be consistent with the conflict-of-interest policy and shall maintain confidentiality of requestors and their reviews.
    For details on the selection of XRAC members and the XRAC review and meeting process, please refer to the XRAC Policies and Procedures web page or email the help desk at help@xsede.org
6.4.1. Review and Allocation

Each XRAC Research Request (New/Renewal/Supplement/Appeal) is reviewed by members of the XSEDE Resource Allocations Committee (XRAC). The reviewers evaluate the merits of the proposal, based on the criteria of Section 7.2.. When peer-reviewed funding information that supports the Computational Research Plan is provided, the science is not re-reviewed. Only when funding information is not supplied is the science reviewed. Each quarter, requests are assigned to reviewers. After a period of one month to complete the reviews, the reviewers convene at a XRAC meeting to determine a Recommended Allocation for each XRAC Research Request (and any large Supplement or Justification). A preliminary discussion among the review team for each request is held to derive a Consensus Allocation for each request. Panel Discussions, including all reviewers, are held to evaluate each team's review summary in an open forum. Special considerations are noted. After a discussion, a Recommended Allocation for each request is arrived at by consensus.

After the Panel Discussion of the XRAC meeting, the total Recommended Allocation is determined and compared to the total Available Allocation across all resources. Transfers of allocations may be made for projects that are more suitable for execution on other resources; transfers may also be made for projects that can take advantage of other resources, hence balancing the load. When the Recommended considerably exceeds the Available Allocations, a reconciliation process adjusts all Recommended Allocations to remove oversubscription. This adjustment process reduces large allocations more than small ones and gives preference to NSF-funded projects or project portions. Under the direction of NSF, additional adjustments may be made to achieve a balanced portfolio of awards to diverse communities, geographic areas, and scientific domains.

All funding used to support the Research Plan of an XRAC Research Request must be reported in the Supporting Grants form in the XRAS submission. Reviewers use this information to assess whether the PI has enough support to accomplish the Research Plan, analyze data, prepare publications, etc. This assessment is blind to the type of funding support. For Reconciliation, Funding information is used to provide priority to the NSF funding portion of a request.

6.4.2. Conflicts of Interest

Every effort is made to avoid conflicts of interest. XRAC members are not allowed to review or be present for the discussion of requests from their home institution, former students, postdocs, advisors, or current and recent collaborators. In addition, XRAC members who are also PIs or co-PIs on XSEDE resource requests do not review requests or participate in XRAC deliberations for the meeting to which they submit their requests. This policy will be strictly enforced for large proposals. For small requests, the PI may participate in the respective meeting, but leave the room during the discussion of their proposal.

If in the opinion of a PI, a certain individual has a conflict of interest, the PI may request that the individual not act as reviewer on their request or potential subsequent appeal. The Allocations Coordinator will consider such requests for the particular review. Such a request should be included as part of a PI's resource request submission in the Special Requirements document. (XRAS will be modified to accept these names directly in the near future.)

The full XRAC conflict of interest policy is at [URL forthcoming] and is incorporated by reference into this document.

6.4.3. Notification of Results

PIs will be notified of the results of the review of Research requests via an email notification within three weeks following the XRAC meeting.

6.4.4. Confidentiality of Reviews and Requests

All XRAC reviews are considered confidential and are only made available to:

  • the PI and co-PIs of the submitted request;
  • members of the XRAC without a conflict of interest;
  • XRAC reviewers of appeals (justifications), supplement requests, progress reports, or subsequent renewals for continuing allocations; and
  • XSEDE allocations officers; and
  • Representatives from XSEDE Service Providers; and
  • NSF staff

XRAC requests and documentation are generally considered confidential, with access limited to the same sets of individuals. However, the requests may be made available to XSEDE User Services or the ECSS staff to assist a research group or evaluate their needs for Extended Collaborative Support Services.

XSEDE allocation amounts and descriptive project information (PI name and institution, title, and abstract) are not considered to be confidential.

6.4.5. Justification (Appeal) Process

PIs who wish to appeal the recommendations of the XRAC may do so by submitting a Justification (Appeal). A Justification should be submitted via XRAS within four weeks of the date when the PI is notified of the review results.

A Justification is also used to supply additional information or clarification requested by the reviewers in order to make a final recommendation. A PI may also use the Justification to specify, in the PI's opinion, why the reviews of the original request and the resulting allocation were incorrect or unfair. A Justification can be used only to supply requested information or to provide the rebuttal. It may not be used to re-submit a rejected request. In the latter case, the PI should submit a revised request in the next submission period (comments in the Request Summary will indicate options available for a rejected request).

If possible, the XRAC members who reviewed the original proposal will consider the Justification (Appeal). Reviewers are expected to respond within two weeks; the Allocations Coordinator may make a determination as to any increase to be made to any initial allocation, if resources are available. If the resources to be allocated are not available on the requested resources, the allocation may be made on another resource. For large requests, at least one favorable review is required before the Allocations Coordinator may make an assessment for an allocation. Alternatively, the allocation may be made at the beginning of the next allocation cycle on the requested resource.

7.0. Preparing a Successful Request Document

On average, more than 70 percent of Research requests receive allocations. However, some allocations may be for less than the level requested. To help ensure that you receive the allocation needed for you to successfully complete your research project, it is important that you provide an appropriate level of justification. The single most important review criterion is the justification of the resource request in the Main Document of the request. Inadequate justification for requested resources is often the reason for reduced or denied allocations. This section provides policies and guidelines to help you craft a successful submission.
In short, the bulk of the Main Document should focus on the resource request, whether for compute, storage, or [Extended Collaborative Support Services]](https://www.xsede.org/ecss).

  • For compute resources, the request should address the appropriateness of the computational methodology, the algorithmic capabilities that make this problem a good fit for the requested systems, and the rationale for the number of runs being proposed.
  • For storage resources, the request should address the appropriateness of the storage resource for the data sharing or management needs, the planned use and usage patterns for the data to be stored, and the rationale for the amount of storage space requested.

  • A request for Extended Collaborative Support to enhance the effectiveness and productivity of work can accompany an application for resources at any level, or it can be submitted stand-alone as a Supplement to an active grant. It requires the Principal Investigator (PI) to answer five (5) questions in the Resource request section of XRAS. This request will be reviewed, usually in conjunction with your XSEDE grant proposal. ECSS management will then assess whether there is also a good potential match with the experts available to collaborate with your team.
7.1. Scientific Merit

XRAC proposals for large allocations request resources that represent a significant investment of the National Science Foundation. The XRAC review process therefore strives to be as rigorous as for equivalent NSF proposals. The Main Document of a request for resources will succinctly state the scientific impact of the research to be conducted, and the existing merit-reviewed supporting grants for the research should be listed in the XRAS forms together with a clear description of the part that will be facilitated by the compute allocation.

If the research has had independent review and has been deemed worthy of scientific support as demonstrated by current financial support from a national agency or foundation, the scientific merit and approach will not be subject to further review by the XRAC.

In those circumstances where independent merit-review has not resulted in current financial support from a national agency or foundation, the Allocations Coordinator will highlight this to the reviewers and the XRAC will review the scientific merit and approach of the proposed work. For ongoing computational activities, the XRAC will also consider the progress made using prior allocations, including the publication of peer-reviewed manuscripts and other communications within the community.

7.2. Review Criteria

The Main Document of the resource request will be reviewed against three criteria, which apply across all types of resources, with the level of detail of the review rising with the size of the requested resources:

  1. Appropriateness of Methodology: For compute requests, the choice of applications, methods, algorithms and techniques to be employed to accomplish the stated scientific objectives should be reasonably described and motivated. For storage requests, the data usage, access methods, algorithms and techniques to be employed to accomplish the stated research objectives should be reasonably described and motivated. For shared collections, requestors should describe the public or community access methods to be provided.

  2. Appropriateness of Research Plan: The steps in the research plan should explain how the research objectives would be achieved. For compute resources, the proposed computations should encompass simulation parameters (step size, time scale, ensemble parameters, etc.) that are needed to obtain accurate and meaningful results, as well as the human resources that can be devoted to the task. The amount of resources requested should be derived from the methodology and the research plan. If there are serious concerns about the research plan, reviewers will document those concerns in their reviews, and the PI may choose to address those concerns in an Appeal.

  3. Efficient Use of Resources: The resources selected should be used as efficiently as is reasonably possible and in accordance with the recommended use guidelines of those resources. For computational resources, performance and parallel scaling data should be provided along with a discussion of optimization and/or parallelization work to be done to improve the applications. If the reviewers conclude that the request is more appropriate on XSEDE resources other than those requested, they may recommend an allocation on those other resources instead.

In considering these criteria, the XRAC recognizes that scientific productivity is the end goal. If adapting to less familiar but, in the view of the panel, better architectures or algorithms requires a significant learning curve for the proposer, with a concomitant interruption of scientific productivity, the XRAC may suggest the alternatives, but nevertheless grant the requested resources with the proposed architecture and methods. In exceptional cases, where the reviewers conclude "beyond a reasonable doubt" that the proposed methods are so inefficient that they amount to a waste of public resources, they should not approve the request until their concerns are addressed by the proposer. Reviewers are encouraged to recommend extended collaborative support, even for those proposers who have not requested it, as part of their recommendations for addressing shortcomings. For storage resources, information on required performance and expected access patterns should be provided for all data and collections to be stored and used along with a discussion of work done or planned to improve the efficiency of the data use.

Note that the availability of resources is not considered in the review. Only the merit of the proposal is. Necessary reductions due to insufficient resources will be made after the merit review, under NSF guidelines, as described in Section 6.4.1.

7.3. Access to Other CI Resources

In addition, all PIs should describe their local cyberinfrastructure resources and other non-XSEDE resources available to them.

Local Computing Environment: Requests should describe briefly the local (e.g., campus or lab) computing environment available to a research team and, most significantly, how XSEDE resources will provide capabilities beyond those of local resources or why the requested XSEDE resources are required in addition to those resources.

Other Supercomputer Resources: If a PI has access to other supercomputer resources (e.g. Blue Waters, NCAR resource, INCITE awards, or other agency resources), or is logically expected to have such access, as with researchers who are affiliated with or are collaborating with other agencies with resource facilities, the document should describe why those resources are not available to the group or why the requested XSEDE resources are required in addition to those resources.

7.4. Samples of Successful Request Documents

To assist PIs in writing their documents, examples of successful resource requests are available as models for others to follow. With these samples, reviewers have expressed the view that these were excellent examples of how an allocation request should be presented. In particular, the reviewers noted that in each case:

  • The request summarizes the research in context of the current state-of-the-art, outlines the computational algorithms to be used, and relates these algorithms to the research subsections.
  • The reviewers were provided sufficient information but not overwhelmed by details.
  • Justification for the allocation request was clear and closely coupled to computational experiments or projects. As such, if the committee needed to reduce the original request, the reduction could have been done rationally with minimum disruption to the investigator.
  • Results from previous allocations were summarized and related to this request.

8.0. Further Details on Eligibility

A PI who applies for an XSEDE resource allocation is usually a researcher or educator at a U.S. academic or research institution. A principal investigator (PI) may not be a high school, undergraduate, or graduate student (but see Section 8.1); a qualified advisor must serve in this capacity. A postdoc is eligible to serve as a PI. XSEDE welcomes allocation requests from throughout the research community. This section clarifies and makes explicit the eligibility rules that apply to specific classes of researchers and educators.

8.1. NSF Grad Student Fellows and Honorable Mentions

While in most cases, a graduate student is ineligible to be PI of an allocation request, an exception is made for NSF Graduate Student Fellows and Honorable Mention recipients. Recipients of these NSF awards can submit requests for Startup allocations for compute time, per arrangement with NSF. In the XRAS Startup submission form a checkbox is provided for a person to identify him- or herself as a fellow/honorable mention recipient. Include a CV and supporting documentation (grant number or an award letter) as part of the request submission.

8.2. Other Federal Agencies

Research staff employed by federal agencies or non-NSF FFRDCs are eligible to apply for an XSEDE allocation if their agency or center does not typically provide research staff with access to non-XSEDE resources of adequate scope for the planned research.

8.2.1. Military Service Academies

Allocation requests submitted on behalf of faculty members at the military service academies, including the U.S. Military Academy, the U.S. Naval Academy, the U.S. Air Force Academy, the U.S. Coast Guard Academy, the U.S. Naval Postgraduate School, and the Uniformed University of the Health Sciences, will be accepted on the same basis as requests from other academic institutions.

8.3. State and Local Governments

State educational offices or organizations and local school districts may submit allocation requests intended to broaden the impact, accelerate the pace, and increase the effectiveness of improvements in science, mathematics, and engineering education in both K-12 and post-secondary levels. A teacher or educator at an accredited public or private K-12 school is eligible to apply for an allocation as PI.

8.4. Non-Profit, Non-Academic Organizations

Independent museums, observatories, libraries, research laboratories, professional societies and similar organizations in the United States that are directly associated with educational or research activities are eligible.

8.5. Unaffiliated Individuals

Scientists, engineers or educators located within the U.S. may be eligible for support, even if the individual is not employed by or affiliated with an organization provided that:

  • the project is sufficiently meritorious and otherwise complies with the conditions of any allocation request;
  • the requestor has demonstrated the capability to carry out the project.

Unaffiliated individuals should contact the XSEDE Help Desk before preparing a request for submission.

8.6. Foreign Collaborators

A PI on a request must be a researcher or educator employed at a U.S. institution. Allocation requests with "foreign components" may be made through a domestic PI (with a foreign Co-PI). A foreign component is defined as the performance of any part of the project outside the U.S. either by the PI or a researcher, or researchers employed by a foreign institution. XSEDE will not provide an allocation to a PI that does not have a substantive role in the project; that is the PI may not simply serve as a proxy for a foreign researcher.

Collaborative projects involving non-U.S. researchers are encouraged as long as they include substantive intellectual participation by the U.S. researchers. In joint research projects, foreign collaborators are eligible to make use of that allocation in a manner consistent with the request.

Allocations requests involving foreign collaborators will be evaluated using the standard review criteria.

8.7. For-Profit Organizations

U.S. commercial organizations, especially small businesses with strong capabilities in scientific or engineering research or education may apply for an allocation. The NSF XD Program is interested in supporting projects that couple industrial research resources and perspectives with those of universities; therefore, it especially welcomes requests from cooperative projects involving both universities and the private commercial sector. It is necessary for these projects to submit their work in an open forum, and make the work readily available to the public.

A Principal Investigator may declare a commercial organization as the "institution" supporting his open-science research. In this case open-science support means that the organization has funding or an agreement (MOU) with an open-science agency, or a component of an open-science agency (usually a University, national open-science agency such as NSF and NIH, national laboratory such as Sandia, Argonne National Lab, etc., or a national funding agency such as DOE, DoD, etc.), to do research that is guaranteed to be accessible and submitted to an open-scientific organization or journal for peer reviewed dissemination of the scientific results. Proposals that are not supported by collaborative open-science funding are generally not considered for review. A submission through a commercial "institution" is also subject to screening by the Allocations Working Group, consisting of site-representatives and the XSEDE Allocations Coordinator.

In addition to supporting scientific research by commercial organizations under the terms described above using the normal XSEDE resource allocation process, many of the XSEDE Service Providers have active industrial partnership programs, including funded access to resources without the restrictions associated with free allocations; for additional information see the XSEDE Industry Partnerships page.

Last update: March 2, 2015