Finalized:Monday, October 5, 2015
Author(s):TAC Standards Working group
Report of the EarthCube Standards Working Group
1.1 Workgroup charge
1.2 Workgroup membership
1.3 Workgroup process
1.4 What is a standard
Proposal for new specification
Adopt a registered specification as an EC recommendation
Scope: the intention is to develop a framework for adoption of standards, focused on technology
1.1 Workgroup charge
Compile information on the use of existing standards, and make recommendations on standards (based on usage, community acceptance/adaption) to the TAC to meet EarthCube requirements
Identify opportunities for standardization and recommend course of action
Evaluate proposals for standards activities and make recommendations to TAC.
Guide the EarthCube community to systematically integrate existing standards, and contribute to external standards-making processes to assure that they address EarthCube requirements.
1.2 Workgroup membership
Stephen Richard: email@example.com (chair) OK
Ibrahim Demir: firstname.lastname@example.org (co-chair) OK
David Arctur: email@example.com OK
Bob Arko: firstname.lastname@example.org OK
Ethan Davis: email@example.com OK
Doug Fils: firstname.lastname@example.org
John Graybeal: email@example.com
Christopher Tucker: firstname.lastname@example.org
1.3 Workgroup process
Phone calls, face to face discussion at Tech Hands Meeting, collaborative work on Google Docs, user surveys.
The subject of standardization consistently foments considerable discussion and a wide variety of opinions. The essence of what is needed for the purposes of EarthCube is documentation of practice to achieve some objective so that it is reproducible and understandable by others. There is a continuum starting with any documented practice in some community. If lots of people use a particular documented practice it could be adopted as a best practice. If almost everyone uses some documented practice, then it is a de facto standard.
A formal standard is a specification of some practice that is adopted by a recognized standards body. The set of formal standards and set of de facto standards intersect, but are not the same; some formal standards are not very widely used. Nonetheless, because of the community participation and rigor required to formalize the standard we recognize that they merit careful evaluation.
We propose that for EarthCube, the term ‘standard’ be defined as “a public specification documenting some practice or technology that is adopted and used by a community.” (See EC Glossary for definitions of terms).
A specification is a document that describes the technical characteristics of an artifact or practice, possibly including a description of what it should do, or an explicit set of requirements that it must satisfy.
We propose to consider all standards that affect the flow of information between EarthCube components, and between EarthCube and external systems or actors. Standards are particularly important to consider in internal EarthCube information flow because they have the strongest potential to improve (or worsen) the interoperability of those components. At the same time, any standards that define EarthCube’s interfaces with external elements will strongly impact the adoption of EarthCube by its target communities.
2. Purpose of standards activities for EarthCube
EarthCube needs standards (see definition above!) to enable interoperability of data sources and data consumers in the system, and to define the interfaces necessary to link components and sub-systems in a loosely-coupled federated system of systems. The goal is documentation of practice used to achieve some objective, so that it is reproducible and understandable by others. In order to make this documentation accessible, EarthCube infrastructure will need to include a registry of standards that are in use by the EarthCube community, and a process to determine which standards are recommended for use. The registry might include all documented practices in use, with varying levels of recommendation.
The EarthCube governance authority does not appear likely to control funding, but will play an advisory role. Thus, EarthCube will not be in a position to dictate community practice; it will play an enabling and facilitating role. Decisions about documented practices (standards) that will be used by the community will be individual project decisions based on technology considerations and policies of the funding authority (NSF). In any event, the operation of the system will depend on the ability of component implementers to verify that software does indeed conform to selected standards, and will interoperate with with other system components that implement an interface or interchange standard. The purpose of standards recommendations in EarthCube is to provide guidance and facilitate component implementation, not to add an onerous layer of requirements and testing.
EarthCube will not be a formal standards development organization, but will have a “standards authority” for purposes of finding, recognizing, and promoting use of relevant and useful standards and practices. The EarthCube standards authority will work with its funded projects to facilitate the adoption of standards and systematic practices, which may include the development of specifications of value to the EarthCube community. The work of the EarthCube standards authority will also inform and leverage the architectural development of EarthCube, as appropriate.
3. Workflow for coordinating standards activities
For the purposes outlined above, the WG recommends defining a continuum of standards value to EarthCube along the following lines:
any documented practice in any geoscience community.
best practice used by lots of people
de facto standard used by almost everyone
formal standard adopted by a recognized standards body.
EarthCube requirement. Funding agencies (e.g. NSF) require conformance for EarthCube project deliverables.
The EarthCube standards authority will need to implement processes to evaluate documented practice in the progression through this continuum. The barrier to entry for new specifications is kept low by choice; the WG places higher value on good documentation of what is actually implemented and working than on technically appealing but untested mandates. The maintenance and use of recommendations will be greatly facilitated if require automated processes to validate conformance with recommended practice are available. However, the EarthCube standards authority is expressly not acting to support or perform component standards conformance testing.
User feedback from communities that are utilizing a standard will be the best source of data to determine the quality of documented practice and elevation of practices to higher levels of recommendation. Quality assessments will be based on ease of use, success in using described practice to link components, performance, and ability to meet application requirements. This aspect of the EarthCube standards system requires the ability to collect and systematically analyze usage information and user feedback. User feedback may be obtained from internal EarthCube projects, and from relevant external projects (particularly those creating comparable cyberinfrastructures).
Several workflow scenarios have been envisioned by the workgroup. This section outlines some of the standards coordination activities that will need to be sustained by the EarthCube organization and infrastructure.
Proposal for new specification
In an active community of technology developers supporting geoscience research, new practices and approaches to cyberinfrastructure requirements will (and should!) emerge. In the absence of known best practices or standards, scientists and developers need to have a means to determine good ways to solve technical problems. Developing a culture of interoperability and component reuse to avoid duplication of effort requires the means to locate and contact people who have knowledge of relevant technical solutions to problems that arise. These contacts might lead to the formation of an informal interest group that would determine the areas of commonality in their practice, and agree on and document a technical approach that they would test and use. In many cases this might simply be adoption of a set of existing specifications, which could be from formal standards organizations or specifications from the EarthCube registry. If some new approach is found to be necessary, the specification could be registered as a new documented practice.
The WG proposes that the barrier to entry be kept low for new specifications to be introduced into the registry of practice, with requirements focused on completeness and clarity of documentation of the proposed practice or technology. The entry level might simply require that the specification documentation is freely available in a stable repository or as a published document, and metadata for the specification is accessible via the EarthCube resource catalog.
Individual projects and developers will need to assess the new practice relative to existing practice to decide which to use, and this will constitute a ‘natural selection’ process that will lead to emergence of community practice within EarthCube. The role of the standards authority in this system will be to review documentation, elicit input on pros and cons of competing technology, monitor the selections that are being made, and present the facts to the community to assist in design decisions.
One of the functions of a standards authority within EarthCube will be to constantly evaluate the benefit of new approaches that are not compatible with existing technology, but bring new capabilities, relative to the interoperability and ease of use fostered by use of existing practice. An obvious consideration in evaluating new technology proposals relative to existing practice is the landscape of current practice. The EarthCube standards authority should encourage and enable participation in relevant, active standardization activities by established organizations. This can increase adoption of existing, compatible standards, avoid duplication of effort through reinvention, and improve alignment of new standards development with EarthCube needs.
Adopt a registered specification as an EC recommendation
The basis of a successful evolutionary system is a diversity of approaches, but diversity of options complicates implementation decisions. As a service to the community, the EarthCube standards authority should establish criteria to evaluate specifications in the EarthCube registry, and assign categorizations to support design decisions. A mechanism to nominate a registered specification for elevation to a higher level of recommendation should be available to the community. Decisions on such recommendation should be based on pragmatic considerations, such as wide usage (number of implementers, frequency of use) or demonstrable technical advantages for the EarthCube system. The Standards WG suggest that the EarthCube specification registry include a mechanism to identify operational applications that are utilizing a particular specification, and provide links from specifications to evaluation metrics such as user feedback, usage volume, locale, and product impacts.
Elevation from the entry level recommendation might include steps such as:
Attachment of formal reviews and responses to the specification
Adoption and deployment of the specified technology by multiple parties, with associated feedback
Endorsement of the specification by community members from different domains
Demonstration of a maintenance process to collect feedback and update the specification as issues are identified.
A mature system of systems will likely include some widely adopted standards for interfaces and information interchange. As EarthCube matures, policies and procedures might be needed for nomination, review, and community decision on adoption of particular specifications as required for EarthCube integration. We recognize this as a future requirement, but consider the current state of EarthCube governance too ill-defined to make recommendations on how this process might work.
The standards authority mechanisms discussed in this section, and throughout the document, must also be in service to incorporate new versions of existing standards.
4. Recommendations on existing standards for EarthCube adoption
The workgroup has been compiling a list of specifications that are in use by the EarthCube Community. At this point the focus has been on specifications identified by the funded projects as in use or planned for use. The compilation is accessible in a Google spreadsheet.
This discussion of standards, and the activity to collect standards information, has already resulted in several agreements for improving specified EarthCube interfaces. We believe that the elevation of standards definition and evaluation will continue to have a concrete and positive impact on the technology selections, and corresponding interoperability, of EarthCube projects
An initial review indicates that most teams that are developing software components are in relatively early stages of choosing and implementing standards. Relatively few standards are adopted or planned across multiple projects, and some commonly used standards, like “RESTful API”, are incomplete without profiles or customizations, which typically are not interoperable.
While it is necessary to describe the precise specification being used, it is also useful to understand the number of projects using the same broader specifications or practices, so that communities of practice can be identified and leveraged. The lesson we derive is that the hierarchical and versioned nature of standards will require development of a more structured taxonomy, allowing specific customizations and profiles to be carry the context of the parent standard(s).
While relatively few EarthCube-used specifications have been identified so far, the value and pace of EarthCube specifications identification and entry will increase with the following factors:
number of EarthCube components;
maturity of EarthCube components;
interconnections between EarthCube components;
recommendations from comparable cyberinfrastructures; and
identification of specific external needs (use cases), including customer expectations of support for certain standards.
5. Recommendations on new standards
One topic that has emerged in the various EarthCube community meetings is the need for conventions on how EarthCube web services are documented in various metadata formats to facilitate interoperable service discovery and automation of data access from client applications based on metadata. This is an example of an interoperability need that is partially or wholly met by multiple existing specifications (Swagger, WADL, ISO19115/119/139...). Different projects within EarthCube may have competing interests in defining a practice and supporting its adoption across the EarthCube community. Furthermore, the engineering tradeoffs involved in the technology choice may be complex.
The standards authority is in a position to identify and document such potential conflicts, and in some cases facilitate a resolution. In these cases, one outcome may be the development of new specification recommended for EarthCube adoption, ideally in the context of an existing community activity or standards organization. It is too early to tell if a mechanism will exist to require projects to adopt a particular standard, or how such a mechanism might work. We note that the lack of an effective mechanism for resolving this type of design conflict has been a cause of failure for many cyberinfrastructure projects, and we encourage its thoughtful consideration at the highest levels of EarthCube governance.
6. Summary Recommendations for standards process
EarthCube needs a registry for specifications of EarthCube practice; the barrier for entry into the registry should be kept low—focused on the quality and completeness of documentation and testing.
Utilize a tiered scheme to categorize specifications in different levels of adoption/recommendation
Collect feedback and evaluation on the utility, usage, and performance of registered specifications and use this to categorize adoption/recommendation levels; this must be a transparent process.
Implement validation processes for recommendations with a high level of adoption.
EarthCube governance should constitute a permanent “standards authority” to carry out the evaluations and recommendations of the Standards WG.
The hierarchical and versioned nature of standards will require development of a structured taxonomy allowing specific customizations and profiles to carry the context of the parent standard(s).