NAIRR Structure and Specifications for Resource ElementsThis chapter provides details of [the] key components, along with desired capabilities when the NAIRR begins initial operations. Given the fast pace of technological development, the Operating Entity should maintain the flexibility to adjust approaches to the elements detailed below, in consultation with the Steering Committee and Program Management Office.The NAIRR Operating Entity should develop an integrated portal to provide the user base described in Chapter 2 with access to a federated mix of on-premise and commercial computational and data resources and services. Computational resources would include conventional servers, computing clusters, HPC, and cloud computing, and should also support access to edge computing resources and testbeds for AI R&D. The NAIRR Operating Entity should make open and protected data available via resource providers and partnerships. Data should be co-located with computational resources where possible. Data providers should facilitate user access to restricted statistical data through the Standard Application Process (SAP) established under the 2018 Foundations for Evidence-Based Policymaking Act, where appropriate and possible. ^^ The NAIRR Operating Entity and resource providers should make software, training, and educational resources available to support a diverse set of users with varying levels of AI research experience and proficiency. National Artificial Intelligence Research Resource Task ForceNAIRRTF_6ca63702-ceaf-11ed-b4fb-9e69fe82ea00A widely-accessible, national cyberinfrastructure that will advance and accelerate the U.S. AI R&D environment and fuel AI discovery and innovation in the United States_6ca638a6-ceaf-11ed-b4fb-9e69fe82ea00To provide details of the key components of the NAIRR along with desired capabilities_48f034ca-cf63-11ed-b3e1-ed4b1883ea00Artificial IntelligenceResearchDevelopmentCapabilityPortal & InterfaceDevelop an NAIRR user portal_48f03696-cf63-11ed-b3e1-ed4b1883ea001Access Portal and User Interface ~ The Operating Entity is responsible for development of an NAIRR user portal that supports key user functionalities such as single sign-on, team allocations, data search and discovery, collaboration tools, resource discovery, job submission, consolidated accounting, spend alerts, information about data use, and cost-optimization of workflows. The portal will be one way to access NAIRR resources. Alternate access methods such as secure shell or scripting interfaces should also be made available for advanced users.Catalog & JobsAllow users to select AI applications, computational resources, and data sources from a curated catalog and to launch and monitor jobs_48f037c2-cf63-11ed-b3e1-ed4b1883ea001.1The portal will allow users to select their AI applications, computational resources, and data sources from a curated catalog, and to launch and monitor jobs from a portal that provides a uniform, integrated view.HelpProvide built-in help functions and an integrated help desk ticketing system_48f038d0-cf63-11ed-b3e1-ed4b1883ea001.2The portal should have built-in help functions and an integrated help desk ticketing system.Documentation & TrainingMaintain a catalog of resource provider user documentation and training materials_48f039de-cf63-11ed-b3e1-ed4b1883ea001.3The portal should maintain an up-to-date catalog of resource provider user documentation and training materials.Collaboration & CommunitiesSupport collaboration and community building_48f03baa-cf63-11ed-b3e1-ed4b1883ea001.4StudentsResearchersResource ProvidersChat functions, meeting rooms, forums, and other functionality may be included to support collaboration and community building among students, researchers, resource providers, and other users.Search & DiscoveryEnable data search and discovery and leverage automated technologies_48f03cc2-cf63-11ed-b3e1-ed4b1883ea001.5The portal should also enable data search and discovery and leverage automated technologies so that (1) metrics on data use can drive data acquisition and (2) diverse, community-driven data curation, linkage, and validation activities can be fostered.MetricsUse metrics on data use to drive data acquisition_d978c784-d29a-11ed-adce-039f0c83ea001.5.1Curation, Linkage & ValidationFoster community-driven data curation, linkage, and validation activities_d978c928-d29a-11ed-adce-039f0c83ea001.5.2User AccountsMaintain user accounts_48f03dd0-cf63-11ed-b3e1-ed4b1883ea001.6A user account would be required to manage computational allocations, monitor usage, submit jobs, and post to the community forum.WebsiteProvide a website through which key elements are publicly available_48f03ed4-cf63-11ed-b3e1-ed4b1883ea001.7The Operating Entity should provide a public website through which some key elements are available without the need for a user account and sign-on. For example, linked catalogs of AI education tools and testbeds, as well as an index of AI datasets with metadata, annotations of known problems and deprecation status, and community-contributed code, should be readily available.CostAssess the cost of building the user portal and public website inhouse versus acquiring it commercially_48f03ff6-cf63-11ed-b3e1-ed4b1883ea001.8The Operating Entity should assess the cost of building the user portal and public website inhouse versus acquiring it commercially. To speed development, the Operating Entity could outsource the design, construction, and maintenance of the user portal to a commercial entity that has previously created successful user portals. All major aspects of the portal should be included in NAIRR initial operational capabilities.AccessMake access to computational and data resources available to a variety of new users who otherwise would face financial, logistical, or capacity challenges engaging in the AI research ecosystem_48f04104-cf63-11ed-b3e1-ed4b1883ea002Computational Resources ~ To lower barriers to entry into AI research, the Operating Entity and resource providers must make access to computational and data resources available to a variety of new users who otherwise would face financial, logistical, or capacity challenges engaging in the AI research ecosystem.LeverageLeverage existing resources in all sectors_48f04212-cf63-11ed-b3e1-ed4b1883ea002.1Expanded access should be provided by leveraging existing resources in all sectors, augmenting the capacity of federally provided resources as appropriate, creating new research computing and data infrastructure to serve the AI R&D community, and providing financial support where needed.Federal ResourcesAugment the capacity of federally provided resources as appropriate_48f0437a-cf63-11ed-b3e1-ed4b1883ea002.1.1InfrastructureCreate new research computing and data infrastructure to serve the AI R&D community_48f04492-cf63-11ed-b3e1-ed4b1883ea002.1.2Financial SupportProvide financial support where needed_48f045aa-cf63-11ed-b3e1-ed4b1883ea002.1.3FederationSupport the federation of user-supplied computing resources, testbeds, and sensors at the edge_48f046f4-cf63-11ed-b3e1-ed4b1883ea002.2The NAIRR should also support the federation of user-supplied computing resources, testbeds, and sensors at the edge.Capacity & CapabilityAddress both the capacity and capability needs of the AI research community_48f04816-cf63-11ed-b3e1-ed4b1883ea003AI Research CommunityCapacity and Capability ~ When fully implemented, the NAIRR should address both the capacity (i.e., ability to support many users) and capability (i.e., ability to train the most resource-intensive AI models) needs of the AI research community.Computational ResourcesProvide a mix of computational resources with a range of central processing unit (CPU) and graphics processing unit (GPU) options_48f0492e-cf63-11ed-b3e1-ed4b1883ea003.1To meet existing capacity needs, the NAIRR should provide a mix of computational resources (i.e., on-premise and commercial cloud, dedicated, and shared resources) with a range of central processing unit (CPU) and graphics processing unit (GPU) options with multiple accelerators per node, high-speed networking, and sufficient memory capacity (i.e., at least one terabyte per node). The exact balance of computational resources will depend on the results of resource provider funding opportunities. Users should have the option of selecting which resources they would like to use through a range of mechanisms, including the user portal, direct command-line access, or optionally interactive “notebook”-like environments.SupercomputerSupport at least one large-scale machine-learning supercomputer_48f04a50-cf63-11ed-b3e1-ed4b1883ea003.2To meet users’ capability needs, the NAIRR system should include at least one large-scale machine-learning supercomputer capable of training 1 trillion-parameter models. This could be made available by leveraging an existing supercomputer or newly procured through a competitive bid process managed by the Operating Entity in consultation with the Steering Committee and relevant advisory boards.Software ResourcesAssess OSS packages most used by AI researchers_48f04b68-cf63-11ed-b3e1-ed4b1883ea004AI ResearchersTensorFlowPyTorchNAIRR Software Resources ~ AI research has grown explosively through the development and dissemination of open source software (OSS) frameworks including TensorFlow, PyTorch, and their derivatives. Both these packages were developed by commercial entities and could have been kept proprietary. Instead, they were released as OSS projects, to the benefit of, and for further development by, the AI research community. The success of these projects has inspired many other OSS projects and tools. ^^ The Operating Entity, with advice from the Technology Advisory Board, should assess OSS packages most used by AI researchers and specify a standard software environment for the NAIRR federation.StandardSpecify a standard software environment for the NAIRR federation_48f04c8a-cf63-11ed-b3e1-ed4b1883ea004.1This software environment should be containerized as a lightweight virtual machine, and be supported across resource providers. Academic teams with their own on-premise servers would be encouraged to adopt the NAIRR federation standard.WorkflowsExplore new AI workflow orchestration tools and templates for standard AI analysis tasks_48f04db6-cf63-11ed-b3e1-ed4b1883ea004.2In addition, the Operating Entity should explore new AI workflow orchestration tools and templates for standard AI analysis tasks, such as cnvrg.io, which can meet the needs of industry researchers and might be suitable for adoption by the NAIRR federation.Data & DatasetsProvide a search and discovery service with metadata about the usage of all datasets_48f04ed8-cf63-11ed-b3e1-ed4b1883ea005Data and Datasets ~ The Operating Entity should provide a search and discovery service with metadata about the usage of all datasets. Such a service should be consistent with Section 202(c) of the Evidence Act. It should be designed to dovetail with the capabilities anticipated through development of a Federal data catalog, but extend beyond Federal data.FundingFund the creation of or provide continuing support to existing AI data repositories_48f05018-cf63-11ed-b3e1-ed4b1883ea005.1The Operating Entity should support data resource providers by either funding the creation of or providing continuing support to existing AI data repositories.Interoperability & CompetitionPublish interoperability guidelines for data repositories and encourage competion to become NAIRR data resource providers_48f05158-cf63-11ed-b3e1-ed4b1883ea005.2NAIRR Data Resource ProvidersSubcommittee on Open ScienceNational Science and Technology CouncilIn coordination with the Technology Advisory Board, the Operating Entity should publish interoperability guidelines for such data repositories, and encourage data repositories to compete to become NAIRR data resource providers. These guidelines should be informed by the Desired Characteristics of Data Repositories for Federally Funded Research developed by the National Science and Technology Council’s Subcommittee on Open Science. Having such repositories and datasets visible, searchable, and discoverable inside the NAIRR, as well as implementing mechanisms to track dataset use, are important to the success of the NAIRR.Security & FederationFederate computational, network, and data resources operating in accordance with security and access-control policies_48f052b6-cf63-11ed-b3e1-ed4b1883ea005.3NAIRR-Open and NAIRR-Secure zones should federate computational, network, and data resources operating in accordance with security and access-control policies that are uniform within the zone, but different between zones, reflecting the restrictions associated with the data in each zone. NAIRR-Secure should coordinate and collaborate with the program office designated by the Office of Management and Budget to oversee the SAP, and others as appropriate, in making available and specifying security and user access controls required for restricted (confidential) government and third-party data. SAP is required by the Evidence Act to be the “front door” for accessing restricted data within the possession of Federal statistical agencies.Datasets & MetadataEvaluate and characterize datasets into tiers, each with a different level of acceptance criteria_48f053f6-cf63-11ed-b3e1-ed4b1883ea006Dataset Acceptance Criteria and Metadata Standards ~ The Operating Entity should evaluate and characterize datasets into tiers, each with a different level of acceptance criteria. Examples include high, medium, and low levels of metadata; provenance; information about dataset usage, and the availability of persistent identifiers.Evaluation & CategorizationEnsure that each dataset is evaluated according to industry standards or best practices_48f0554a-cf63-11ed-b3e1-ed4b1883ea006.1The Operating Entity should ensure that each dataset is evaluated according to industry standards or best practices and that a determination is made on how each should be categorized.Data CatalogDevelop a Federal data catalog_48f0568a-cf63-11ed-b3e1-ed4b1883ea006.2Where possible, such cataloging efforts should be aligned with efforts to develop a Federal data catalog.FormatsProvide a list of acceptable formats and leverage community-driven principles and standards_48f057c0-cf63-11ed-b3e1-ed4b1883ea006.3Research Data AllianceNISTThe Operating Entity should not define dataset standards, as this area continues to evolve rapidly and would be best addressed by the community of users. However, the Operating Entity should provide a public-facing list of acceptable formats to ensure compatibility with resources and tools, encourage broader use, and leverage existing community-driven principles and standards such as those developed by the Research Data Alliance and NIST, among others.DocumentationProvide documentation with each directory or file_48f05914-cf63-11ed-b3e1-ed4b1883ea006.4Regardless of category, substantive documentation should be provided with each directory or file containing data.Analysis ReadinessSpecify the meaning of “analysis-ready” and categorize datasets accordingly_48f05a54-cf63-11ed-b3e1-ed4b1883ea006.5The Operating Entity should also specify what it means for a dataset to be “analysis-ready” and categorize datasets accordingly. For example, an analysis-ready dataset should be in a structured format (e.g., a relational table or JSON34 or Neo4j35 formats) and should include details such as the semantics and provenance, information about the data-generation process, a data dictionary, related code, summary statistics for quality-assurance purposes, and information about how it has been used in previous analyses. Further, such a dataset should conform to standards in cases where datatypes are normally represented in a standard ontology (e.g., geographic information system [GIS] vector objects, gene ontology codes for molecules).Rarity & ImportanceTransform important and rare datasets to become analysis-ready_48f05b9e-cf63-11ed-b3e1-ed4b1883ea006.6Not all datasets need be in analysis-ready form. Some types of data or partial datasets are important or rare, and can be contributed with the goal that others can help transform them into analysis-ready data.Incentives & CurationIncentivize and curate contributed datasets and other resources_48f05cf2-cf63-11ed-b3e1-ed4b1883ea007Role of the Operating Entity in Incentivizing and Curating Contributed Datasets and Other ResourcesData ServiceEstablish a data service that facilitates access to and additional use of existing curated datasets_48f05e3c-cf63-11ed-b3e1-ed4b1883ea007.1Since the quality of many AI models depends on high-quality training and test data, the Operating Entity should establish a data service that facilitates access to and additional use of existing curated datasets of value and interest to the NAIRR user community. Curation of AI data, models, tools, and workflows should be done by the user community in an AI data commons, facilitated by the NAIRR search and discovery platform. Such a community system, governed by terms of use as well as a review system, would facilitate data sharing and curation by members of the community.Recognition & AccessIncentivize researchers who contribute to the common good through high-profile NAIRR recognition and/or preferential access to NAIRR resources_48f05f86-cf63-11ed-b3e1-ed4b1883ea007.2In the context of a commons model, researchers who contribute to the common good through data curation and code sharing, and whose contributions are recognized and valued by relevant communities, could be incentivized through high-profile NAIRR recognition and/or preferential access to NAIRR resources.Query, Discovery & CurationTest a service for searching for, discovering, and curating valuable external data as well as data generated with NAIRR resources_48f060ee-cf63-11ed-b3e1-ed4b1883ea007.3The NAIRR Operating Entity should test, on a trial basis, a service for searching for, discovering, and curating valuable external data as well as data generated with NAIRR resources.AI MarketplacesContract with one or more commercial AI marketplaces to meet user data curation needs_48f062f6-cf63-11ed-b3e1-ed4b1883ea007.3.1One option would be to contract with one or more commercial AI marketplaces to meet its users' data curation needs. The “AI marketplace” is a powerful concept that has emerged in the commercial sector; it refers to the social and technical infrastructure through which the user community contributes, documents, and shares data, codes, and models. Contributions are validated and valued by the community, and community standards are enforced by the company managing the marketplace.Data CommonsDevelop an “AI data commons”_48f06454-cf63-11ed-b3e1-ed4b1883ea007.3.2Another option is for the NAIRR to develop its own “AI data commons” with attributes similar to a commercial marketplace. Such an option is likely to be preferable for the federally funded NAIRR. However, since both commons and marketplace options have merit, the Operating Entity should have flexibility regarding development of data curation services, and the services should be implemented on a trial basis and evaluated for efficacy by the Operating Entity in the first five years of NAIRR operation.Technical SupportDedicate resources to technical support staff who can support community-driven curation efforts_48f06620-cf63-11ed-b3e1-ed4b1883ea007.4Data UsersData ContributorsData CuratorsSubstantial Operating Entity resources should be dedicated to technical support staff who can support community-driven curation efforts. Data users, contributors, and curators will require support to understand and meet the technical standards of NAIRR data repositories.Training & SupportProvide training and additional support to ensure the integrity and quality of NAIRR datasets as well as to protect privacy, civil rights, and civil liberties_48f0677e-cf63-11ed-b3e1-ed4b1883ea007.5Further, training and additional support will be critical to the integrity and quality of NAIRR datasets, and to protect privacy, civil rights, and civil liberties.Federal DataMake federal datasets accessible_48f068dc-cf63-11ed-b3e1-ed4b1883ea008The NAIRR and Existing Federal Government Data ~ Federal agencies hold data that could fuel foundational, use-inspired, and translational AI research in domains such as transportation, healthcare, and natural hazards research. Sources of Federal agency data include statistical data, administrative data, and data from federally funded intramural and extramural research. While some of these datasets are already accessible to the public, many others are not. ^^ Since Federal datasets could be highly valuable to AI research and advance national goals, there are three other Federal Government data efforts with which the NAIRR could engage.data.govWork with data.gov to encourage additional contributions_48f06a62-cf63-11ed-b3e1-ed4b1883ea008.1data.govOne is data.gov, which is a website that points to other resources containing information and data generated by agency or agency-funded projects. Most of the retrievable data on data.gov are in web or text form, which might be of interest to some NAIRR researchers. However, scientific numerical datasets are deeply buried in data.gov and not easily accessible. The Operating Entity and Program Management Office could work with data.gov to encourage additional contributions conforming to NAIRR data acceptance criteria, which should include measures of data use.SAPEnable discovery and access to restricted data acquired by Federal statistical agencies through a single application process and portal_48f06bca-cf63-11ed-b3e1-ed4b1883ea008.2SAPAnother is the SAP, through which researchers will be able to discover and apply for access to restricted data acquired by Federal statistical agencies through a single application process and portal.Acquisition, Linkage & ProtectionProvide additional capability for data acquisition, linkage, and protection_48f06d3c-cf63-11ed-b3e1-ed4b1883ea008.3Finally, the National Secure Data Service (NSDS) demonstration project, established by the CHIPS and Science Act of 2022, has the potential to complement the SAP and existing statistical agency efforts with additional capability for data acquisition, linkage, and protection (see Box 6 for more details). Statistical PolicyEstablish a NAIRR-Federal Interagency Council on Statistical Policy (ICSP) working group_48f06ec2-cf63-11ed-b3e1-ed4b1883ea008.4Interagency Council on Statistical Policy (ICSP)The Steering Committee should facilitate the establishment of a NAIRR-Federal Interagency Council on Statistical Policy (ICSP) working group.Secure NodeAssess options for establishing a secure node for the purpose of enabling large-scale AI analysis of government data for statistical purposes_48f07034-cf63-11ed-b3e1-ed4b1883ea008.4.1This working group should collaborate to assess options for establishing a secure node for the purpose of enabling large-scale AI analysis of government data for statistical purposes.Protocols & ControlsDefine the Confidential Information Protection and Statistical Efficiency Act-compliant data access protocols and controls_48f071a6-cf63-11ed-b3e1-ed4b1883ea008.4.1Where such resources are not intended to be made accessible via the SAP or the NSDS demonstration project, the working group should define the Confidential Information Protection and Statistical Efficiency Act-compliant data access protocols and controls. This NAIRR-ICSP collaboration should facilitate the provisioning of timely access for appropriate (i.e., approved) projects to restricted (i.e., confidential) government and third-party data.State & Local DatasetsEncourage and support additional contributions of State and local datasets_48f07336-cf63-11ed-b3e1-ed4b1883ea008.5State Government AgenciesLocal Government AgenciesThe NAIRR should also encourage and support additional contributions of State and local datasets conforming to NAIRR data acceptance criteria, and subject to the legal requirements of the State and local government agencies, either by working with data.gov or the eventual NSDS. In terms of existing high-quality data repositories managed by agencies such as the National Institutes of Health (NIH) and NASA, the Operating Entity will need to determine whether to reproduce large datasets that are already available from these other sources or find other means of coordinating access for NAIRR researchers. This coordination could benefit from regular convening of leadership from various Federal data efforts to identify ways to improve coordination and avoid inefficiency or redundancy.2023-01-312023-04-04https://www.ai.gov/wp-content/uploads/2023/01/NAIRR-TF-Final-Report-2023.pdfOwenAmburOwen.Ambur@verizon.net