VOSI.tex

\documentclass[11pt,letter]{ivoa}
\input tthdefs

\input gitmeta

\usepackage{listings}
\lstloadlanguages{XML}
\lstset{flexiblecolumns=true,basicstyle=\footnotesize,tagstyle=\ttfamily,showspaces=false}
\usepackage{todonotes}

\title{IVOA Support Interfaces}

\ivoagroup{Grid and Web Services Working Group}

\author{Patrick Dowler}
\author{Matthew Graham}
\author{Brian Major}
\author{Guy Rixon}

\editor{Patrick Dowler}

\previousversion[https://ivoa.net/documents/VOSI/20170524/]{REC-1.1}
\previousversion[http://www.ivoa.net/documents/VOSI/20110531]{REC-1.0}

\begin{document}
\begin{abstract}
This document describes the minimum interface that a web service
requires to participate in the IVOA. Note that this is not required of
standard VO services developed prior to this specification, although
uptake is strongly encouraged on any subsequent revision. All new
standard VO services, however, must feature a VOSI-compliant interface. 
\end{abstract}


\section*{Acknowledgments}

This document has been developed with support from the National Science
Foundation's Information Technology Research Program under Cooperative
Agreement AST0122449 with The Johns Hopkins University, from the UK
Particle Physics and Astronomy Research Council (PPARC), from the
European Commission's (EC) Sixth Framework Programme via the Optical
Infrared Coordination Network (OPTICON), and from EC's Seventh Framework
Programme via its eInfrastructure Science Repositories initiative.

This work is based on discussions and actions from the 2003 IVOA meeting
in Strasbourg and further discussions on registry functionality at JHU
late in 2003. Later inputs came from a local meeting at JHU in Sept.
2004. William O'Mullane and Ani Thakar were the editors and primary
authors for these early versions.

The decision to split the interfaces into a mandatory set and optional
logging interfaces was taken by GWS-WG at the IVOA meeting of May 2006. 

\section*{Conformance-related definitions}

The words ``MUST'', ``SHALL'', ``SHOULD'', ``MAY'', ``RECOMMENDED'', and
``OPTIONAL'' (in upper or lower case) used in this document are to be
interpreted as described in IETF standard RFC2119 \citep{std:RFC2119}.

The \emph{Virtual Observatory (VO)} is a
general term for a collection of federated resources that can be used
to conduct astronomical research, education, and outreach.
The \href{http://www.ivoa.net}{International
Virtual Observatory Alliance (IVOA)} is a global
collaboration of separately funded projects to develop standards and
infrastructure that enable VO applications.


\section{Introduction}
\label{sec:introduction}

This document describes a set of common basic functions that VO web
services provide in the form of a standard support interface in order to
allow for the effective management of the VO.  There are two basic
support functions that this document describes:  The reporting of
capability metadata and
the reporting of table metadata (if applicable).  A previous version of
this document also required an interface for reporting service
availability.  As discussed in sect.~\ref{sect:availability}, this
part of the specification was dropped in version 1.2.

VO service standards previous to VOSI may not be forced to
retrospectively implement VOSI (although that should be encouraged).
Nonetheless, all new VO service standards (or updated existing ones)
must enforce the VOSI implementation.


\subsection{Role within the VO Architecture}

The IVOA Architecture \citep{2021ivoa.spec.1101D} provides a high-level
view of how IVOA standards work together to connect users and
applications with providers of data and services, as depicted in the
diagram in Figure ~\ref{fig:archdiag}. 

\begin{figure}
\centering
\includegraphics[width=0.9\textwidth]{role_diagram.pdf}
\caption{VOSI in the IVOA Architecture. VOSI is the standard that
defines the basic functions that all VO services should provide in order
to support management of the VO.}
\label{fig:archdiag}
\end{figure}

In this architecture, users employ a variety of tools (from the User
Layer) to discover and access archives and services -- that is,
resources -- of interest (represented in the Resource Layer). A registry
plays a role in discovery by harvesting metadata that describe archives
and services and making them searchable from a central service. The VOSI
interface provides a means for a service to provide some of this
metadata itself; this allows a registry to pull the metadata from the
service rather than relying on a human to provide it (e.g. by typing the
data into a registration form manually). This mechanism can make it
easier to collect highly detailed metadata (e.g. descriptions of columns
in a catalog) that might not be practically provided otherwise. As some
of this metadata describes the service interface and how it behaves,
other applications can use this information for controlling how they use
the service. Even when the service is ``discovered'' through some means
other than a registry, an application can still understand how to use
the service by querying for this information directly. (See Appendix
\ref{appendix:harvesting} for a more detailed description of this use
case.)

\section{Interface bindings}
\label{sec:bindings}

This section explains how services are to report VO support metadata
through a standard interface.

The standard interface returns metadata without changing the state of
the service with which it is associated. This standard requires a REST
binding of VOSI even when applied to services that do not have a RESTful
interface.

In the REST binding, the support interfaces shall have distinct URLs in
the HTTP scheme and shall be accessible by the GET operation in the HTTP
protocol. The response to an HTTP POST, PUT or DELETE to these resources
is not defined by this specification. However, if an implementation has
no special action to perform for these requests, the normal response
would be a HTTP 405 "Method not allowed" status code.  The
capabilities binding must be available to anonymous
requests.\todo{This is probably no longer true (see SSO) -- material for
another PR}

The endpoints and interface types for the support interface shall be
defined in the service's registration using one \xmlel{capability}
element for each interface. The values of the \xmlel{standardID}
attribute for these Capabilities are given in
section~\ref{sec:endpoints}.

When using the REST binding, any HTTP URLs may be used. The client must
find the appropriate URLs from the service's entry in the VO registry
and, in general, should not try and infer the URLs from any other URLs
for that service.\todo{Which is in a bit of a conflict with the
introducton's ``by some other means''. Resolve this in another PR?} However, standards for specific services may put extra
constraints on the form of the URLs.

\section{Metadata specification}
\label{sec:metadata}

 There are various classes of metadata that might be returned by a
 service through its standard interface:

\begin{itemize}
\item those describing its functional capabilities
\item those describing its operational behaviour - availability, reliability, etc.
\item those describing tabular data handled by the service
\item those describing other aspects of the service 
\end{itemize}

This section defines how each of these classes is represented.\todo{I'd
say let's switch this to our canonical prefixes as in other Registry
documents (vr:, vs:, \dots).  It's more readable and less of a 
typographic chore.} The
following typographic convention is used to represent a XML element
defined within a particular namespace:

$$\hbox{\xmlel{http://some.name.space\#elementName}}$$

For example, \xmlel{http://www.ivoa.net/xml/VOResource/v1.0\#resource}
indicates a XML element named \xmlel{resource} that is described in the
XML schema associated with the 'http://www.ivoa.net/xml/VOResource/v1.0'
namespace -- in this case, this would be VOResource.xsd
\citep{2018ivoa.spec.0625P}. 

\subsection{Capability metadata}

Note:
    'Capability' is unfortunately an overloaded term in the VO referring
    to both a functional aspect of a service and also particular pieces
    of metadata defined by various XML schema. When referring to an XML
    element called \xmlel{capability}, it shall be typeset in italic
    typewriter. Its parent namespace may also be included (using the
    syntax described above) if it is ambiguous which XML schema is being
    referred to. 
    
This interface provides the service metadata in the form of a list of
Capability descriptions. Each of these descriptions is an XML element
that:

\begin{itemize}
\item states that the service provides a particular, IVOA-standard function;
\item lists the interfaces for invoking that function;
\item records any details of the implementation of the function that are
not defined as default or constant in the standard for that function. 
\end{itemize}

For example, one Capability might describe a cone search function and
another the TAP implementation but these two might well apply to the
same service.

An entry for a service in the resource registry -- i.e., its VOResource --
contains the Dublin Core resource metadata (identifier, curation
information, content description, etc.) followed by the service's
capability descriptions (expressed as a series of
\xmlel{http://www.ivoa.net/xml/VOResource/v1.0\#Ca\-pa\-bi\-lity}
elements). Effectively, the resource metadata describes the service to
human users and the capability list describes it to software. Therefore,
the latter list has two uses:

\begin{itemize}
\item it may be read by a client application to find out how to invoke
the service. This presumes that the service has been already been
selected and the VOSI endpoint located.
\item it may be read by the registry itself to compile the registry
entry for the service. In this case, the resource metadata are entered
into the registry directly and the service metadata are then read from
the service. Since the service implementation usually knows its
capabilities, this removes the need for a human to type them into the
registry. 
\end{itemize}

The service metadata shall be represented as an XML document with the root element:\\
$$\hbox{\xmlel{http://www.ivoa.net/xml/VOSICapabilities/v1.0\#capabilities}}$$\\
(See Appendix \ref{appendix:capabilities} for the definition of the
VOSICapabilities XML schema.) This element must contain one or more
child capability elements that describe the capabilities of the service.
Given that the capability element is defined to be of type\\
$$\hbox{\xmlel{http://www.ivoa.net/xml/VOResource/v1.0\#Capability}}$$\\
a capability element may be represented by a legal sub-type of\\
$$\hbox{\xmlel{http://www.ivoa.net/xml/VOResource/v1.0\#Capability}}$$\\
in which case, the capability element must use an \xmlel{xsi:type}
attribute to identify the sub-type (see section 2.2.1 of VOResource
\citep{2018ivoa.spec.0625P}). 

\begin{admonition}{Note}
The value of the capability element's \xmlel{standardID} attribute is
used to indicate the service's support for particular standard protocols
(such as Simple Image Access, Simple Cone Search, etc.). In the case of
some protocols, the support for the standard is further characterized by
additional metadata provided by a standard XML schema extension of
\xmlel{Capability} for that protocol. The extension metadata is enabled
by adding a \xmlel{xsi:type} attribute to the capability element set to
the Capability sub-type value defined in the extension schema for that
protocol (see example below). 
    
The VOResource list of capabilities should include capabilities
describing VOSI endpoints as specified in section \ref{sec:endpoints}.
\end{admonition}

In the REST binding, the service metadata shall be a single web resource
with a registered URL. The date and time at which the metadata last
changed shall be obtained from the Last-Modified HTTP Header keyword
sent in the response to a GET or HEAD request to the registered URI.

All VO services should provide this interface. 

\subsection{Availability metadata}
\label{sect:availability}

Versions 1.0 and 1.1 of this specification required services to declare
information on their current and future availability.  It was found that
this specification was insufficient to work with multi-capability and/or
mirrored services \citep{note:caproles}.  Also, the interface has not
found the use that was originally anticipated.  Therefore, availability
reporting is no longer part of VOSI, although the standards record still
defines a corresponding standards key, indicating support for the legacy
endpoint.


\subsection{Table metadata}

Some services deal with tabular data. These data may be the target of
ADQL queries, as in TAP \citep{2019ivoa.spec.0927D}, or they may be the
output of other operations, as in SIAP queries. In each case, it is
useful if the service describes the details of the tables concerned. It
is more useful if this description can be captured in the resource
registry.

The VODataService standard \citep{2021ivoa.spec.1102D} defines XML
elements for describing a set of tables. These elements can be included
in a resource document for a service.

A service which uses tables in its interface should define a VOSI
endpoint from which table metadata can be read. The table metadata for
all content of a service shall be represented as an XML document of
which the root element is of type\\
$$\hbox{\xmlel{http://www.ivoa.net/xml/VODataService/v1.1\#TableSet}}$$\\
The table metadata for a single table (see below) shall be represented as an
XML document of which the root element is of type\\
$$\hbox{\xmlel{http://www.ivoa.net/xml/VODataService/v1.1\#Table}}$$\\
This element may contain any mix of elements allowed by the VODataService XML schema.
The table metadata for a single table can also be representated using an empty VOTable.

The REST interface for the tables endpoint in described in detail in the OpenAPI
specification (see ???). Here we give a brief summary of the main use and some
examples but defer to the OpenAPI documents as the definitive specification.

\subsubsection{TableSet}
The TableSet metadata is a hierarchical web resource with two levels of detail.
Maximum detail (max) is the complete metadata including all details of the table
structure (columns and foreign keys).  Minimum detail (min) includes the full
set of tables (organised by schema) with names and optional descriptions, but does not
include column of foreign key information.

For example, the request:

\begin{verbatim}
GET http://example.net/srv/tables
\end{verbatim}
\noindent would return the default (service-chosen) amount of detail (either just table
listing or the complete TableSet metadata). 

The caller can use the \verb|detail| parameter to request a specific amount of detail.
For example:
\begin{verbatim}
GET http://example.net/srv/tables?detail=min
\end{verbatim}
\noindent should be taken as a suggestion to return the minimum level of TableSet
metadata detail, while:
\begin{verbatim}
GET http://example.net/srv/tables?detail=max
\end{verbatim}
\noindent should be taken as a suggestion to return the maximum level of TableSet
metadata detail.

Services with a large number of tables and/or columns cannot normally
respond with a usable TableSet document that has the maximum level of
detail so may wish to always respond with the minimum level of detail.

\subsubsection{Table}
The REST endpoint must also support a child resource for each table
described in the TableSet document.  The child resource must be named as
it appears in the name of the corresponding child Table element.  For
example:

\begin{verbatim}
GET http://example.net/srv/tables/ivoa.ObsCore
\end{verbatim}
\noindent would return a Table document describing the ivoa.ObsCore table in full
detail, starting:

\begin{verbatim}
<vosi:table xmlns:vosi="http://www.ivoa.net/xml/VOSITables/v1.0">
    <name>ivoa.ObsCore</name>
    ...
    <column>
    ...
</vosi:table>
\end{verbatim}

The above \xmlel{vosi:table} document is the default response format; clients 
may request the alternate VOTable format using the HTTP \verb|accept| header with a
value of the VOTable mime-type (\verb|"application/x-votable+xml"|).

\subsubsection{User-managed Tables}
Some services may allow users to create tables using the tables endpoint. The lifetime
of such tables and the authentication and authorization requirements for using this feature
are an implementation detail that may vary from one service to another. At one extreme,
users may be able to create tables anonymously and the table will persist for a short 
period of time (days or weeks) before being removed. At the other extreme, users would
have to authenticate, have permission to create tables in a specific schema, and the tables
would persist forever.

The visibility of user-managed tables is also an implementation detail not specified here.
Expectation: short-lived anonymously created tables would not be visible (e.g. not listed 
by the tables endpoint of the service or be added to the \verb|tap_schema| of a TAP service).
Expectation: permanent tables created (owned) by an authenticated user could be made
visible to and usable by other users (e.g. included in tables endpoint output and added to
the \verb|tap_schema| of a TAP service).

The additional actions to allow users to manage tables are optional. The OpenAPI specification
says that a tables endpoint that does not support these actions must respond with an HTTP
405 status code; we expect that current services already respond this way. \todo{Do we need a
registry extension for VOSI-tables? just listing http verbs would be enough to avoid probing
for a 405 aka try and see if it works}

To create a new table:
\begin{verbatim}
PUT <xml> http://example.net/srv/tables/schema1.table1
\end{verbatim}
\noindent where the \verb|<xml>| payload can be either a VOSI-table or VOTable document. For
a VOTable payload, the table may contain rows and those rows would be inserted into the created
table.

To update table metadata:
\begin{verbatim}
POST <xml> http://example.net/srv/tables/schema1.table1
\end{verbatim}
\noindent where only some of the table metadata can me modified (see below). In addition, if
the VOTable format is used for updating metadata, the VOTable would not contain any rows (but
see below).

To delete a table:
\begin{verbatim}
DELETE http://example.net/srv/tables/schema1.table1
\end{verbatim}

Some services may allow the caller to append additional rows to a table. This could be
implemented by accepting a table update that carries additional rows:
\begin{verbatim}
POST <votable> http://example.net/srv/tables/schema1.table1
\end{verbatim}
\noindent where some metadata may be updated and rows in the input VOTable are appended
to the existing table. Other tabular data formats that are not sufficient for create may be
supported for appending rows: csv, tsv, fits, etc. \todo{Service should advertise the
list of acceptable formats}

\subsubsection{Mutable Table Metadata}
Metadata that can be updated:
\begin{itemize}
\item table metadata that can be updated: utype, description
\item column metadata that can be updated: utype, ucd, unit, description
\end{itemize}

Column metadata where update would imply a structural change to the table (e.g. an SQL
alter table statement):
\begin{itemize}
\item column datatype, arraysize, xtype: alter column datatype?
\item input table description has fewer columns: drop column?
\item input table description has more columns: add column?
\item rename table
\item rename column(s)
\end{itemize}
\noindent These are probably too fraught with pain in the details. \todo{Some CADC users have
asked for table and column renaming, which should be easy enough to implement}

\subsection{Non-service metadata (non-normative)}

There may be other metadata associated with a service than the
capability metadata described above.

\begin{itemize}
\item Every service has the Dublin Core resource metadata \citep{std:DUBLINCORE}.
\item Some services are associated with registered applications.
\item Some services are associated with registered data collections. 
\end{itemize}

None of these are explicitly provided for in this version of VOSI. Some
might be covered in later versions of VOSI. 

\section{Registration of VOSI endpoints}
\label{sec:endpoints}

The endpoints for the service metadata shall be
included in the registration of each service that provides them.

\begin{tabular}{l l l l l}
\label{tab:registration}
Endpoint type & standardID value \\
capabilities & \nolinkurl{ivo://ivoa.net/std/VOSI#capabilities} \\
tables (1.0) & \nolinkurl{ivo://ivoa.net/std/VOSI#tables} \\
tables (1.1 and 1.2) & \nolinkurl{ivo://ivoa.net/std/VOSI#tables-1.1} \\
\end{tabular}

A capabilities endpoint should be represented by an element named
\xmlel{capability}, of type
\xmlel{http://www.ivoa.net/xml/VOResource/v1.0\#Capability}. If such a
capability\todo{Since we require this endpoint, isn't this qualification
a bit silly?} is provided then the value of the \xmlel{standardID}
attribute must be \nolinkurl{ivo://ivoa.net/std/VOSI#capabilities}.

A tables endpoint should be represented by an element named
\xmlel{capability}, of type
\xmlel{http://www.ivoa.net/xml/VOResource/v1.0\#Capability}. If such a
capability is provided then the value of the \xmlel{standardID}
attribute must be \nolinkurl{ivo://ivoa.net/std/VOSI\#tables}, or, for
version 1.1, \nolinkurl{ivo://ivoa.net/std/VOSI\#tables-1.1}.

With both VOSI functions, the \xmlel{capability} element that
describes the function must contain an \xmlel{interface} element of a
type semantically appropriate for the binding of the function to the
service; the \xmlel{accessURL} element within the \xmlel{interface}
element indicates the endpoint for the VOSI function. For the REST
binding, this \xmlel{accessURL} element must set the \xmlel{use}
attribute to \texttt{"full"}. Furthermore, for the REST binding, this
document recommends using the
\xmlel{http://www.ivoa.net/xml/VODataService/v1.1\#ParamHTTP} interface
type to encode VOSI endpoints (see the examples given in
section~\ref{sec:examples}). 

The capabilities endpoint must not require any
credentials to view\todo{See above}.  Thus, the \xmlel{interface} registry entries for
capabilities must not contain a securityMethod element.

\section{Example VOSI responses}
\label{sec:examples}

\subsection{Example 1: SIA 1.0 capabilities}

A sample response from a capabilities resource describing an SIA service. 

\begin{lstlisting}[language=XML,basicstyle=\footnotesize]
<?xml version="1.0" encoding="UTF-8"?>
<vosi:capabilities xmlns:vosi="http://www.ivoa.net/xml/VOSICapabilities/v1.0"
    xmlns:vr="http://www.ivoa.net/xml/VOResource/v1.0"
    xmlns:vs="http://www.ivoa.net/xml/VODataService/v1.1"
    xmlns:sia="http://www.ivoa.net/xml/SIA/v1.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">

  <!-- a generic capability (for custom, non-standard interfaces) -->
  <capability>
    <interface xsi:type="vr:WebBrowser">
      <accessURL use="full"> http://adil.ncsa.uiuc.edu/siaform.html 
      </accessURL>
    </interface>
  </capability>
  
  <!-- the SIA capability -->
 
  <capability xsi:type="sia:SimpleImageAccess" 
      standardID="ivo://ivoa.net/std/SIA">
    <interface xsi:type="vs:ParamHTTP" role="std">
      <accessURL> http://adil.ncsa.uiuc.edu/cgi-bin/voimquery?survey=f&amp; 
      </accessURL>
    </interface>

    <imageServiceType>Pointed</imageServiceType>

    <maxQueryRegionSize>

      <long>360.0</long>

      <lat>180.0</lat>

    </maxQueryRegionSize>

    <maxImageExtent>

      <long>360.0</long>

      <lat>180.0</lat>

    </maxImageExtent>

    <maxImageSize>

      <long>5000</long>

      <lat>5000</lat>

    </maxImageSize>

    <maxFileSize>100000000</maxFileSize>

    <maxRecords>5000</maxRecords>

  </capability>

  <!-- the interface that returns this capability -->
  <capability standardID="ivo://ivoa.net/std/VOSI#capabilities">
    <interface xsi:type="vs:ParamHTTP" role="std">
      <accessURL use="full"> 
        http://adil.ncsa.uiuc.edu/cgi-bin/voimquery/capabilities 
      </accessURL>
    </interface>
  </capability>

</vosi:capabilities>
\end{lstlisting}


\subsection{Example 2: TAP tables, maximum detail}

A sample response from a tables resource describing a TAP service in
full (maximum) detail.

\begin{lstlisting}[language=XML]
<?xml version="1.0" encoding="UTF-8"?>
<vosi:tableset
  xmlns:vosi="http://www.ivoa.net/xml/VOSITables/v1.0"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xmlns:vs="http://www.ivoa.net/xml/VODataService/v1.1"
  version="1.1">

  <schema>
    <name>cfht</name>
    <table type="output">
      <name>cfht.deepU</name>
      <column>
        <name>cfhtlsID </name>
        <dataType xsi:type="vs:TAPType" size="30">adql:VARCHAR</dataType>
      </column>
      <column>
        <name>survey </name>
        <dataType xsi:type="vs:TAPType" size="6">adql:VARCHAR</dataType>
      </column>
      <column>
        <name>field </name>
        <dataType xsi:type="vs:TAPType" size="2">adql:VARCHAR</dataType>
      </column>
      <column>
        <name>pointing </name>
        <dataType xsi:type="vs:TAPType" size="6">adql:VARCHAR</dataType>
      </column>
      <column>
        <name>selectionFilter </name>
        <dataType xsi:type="vs:TAPType" size="2">adql:VARCHAR</dataType>
      </column>
    </table>
  </schema>
  
  <!-- additonal schema element in minimum detail example omitted for brevity -->

</vosi:tableset>
\end{lstlisting}

\subsection{Example 3: Tables, minimum detail}

A sample response from a tables resource with no Column or ForeignKey elements.

\begin{lstlisting}[language=XML]
<?xml version="1.0" encoding="UTF-8"?>
<vosi:tableset
  xmlns:vosi="http://www.ivoa.net/xml/VOSITables/v1.0"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xmlns:vs="http://www.ivoa.net/xml/VODataService/v1.1"
  version="1.1">

  <schema>
    <name>cfht</name>
    <table type="output">
      <name>cfht.deepU</name>
    </table>
  </schema>
  
  <schema>
    <name>TAP_SCHEMA</schema>
    <table type="output">
      <name>TAP_SCHEMA.tables</name>
    </table>
    <table type="output">
      <name>TAP_SCHEMA.columns</name>
    </table>
    <table type="output">
      <name>TAP_SCHEMA.keys</name>
    </table>
    <table type="output">
      <name>TAP_SCHEMA.key_columns</name>
    </table>
  </schema>

</vosi:tableset>
\end{lstlisting}

\subsection{Example 4: Child table resource}

A sample response from a child table resource (e.g.
http://example.net/srv/tables/TAP\_SCHEMA.columns).

\begin{lstlisting}[language=XML]
<?xml version="1.0" encoding="UTF-8"?>
<vosi:table xmlns:vosi="http://www.ivoa.net/xml/VOSITables/v1.0" 
    xmlns:vs="http://www.ivoa.net/xml/VODataService/v1.1" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    type="output" version="1.1">
  <name>TAP_SCHEMA.columns</name>
  <column>
    <name>table_name</name>
    <description>the table this column belongs to</description>
    <dataType xsi:type="vs:TAPType" size="64">VARCHAR</dataType>
  </column>
  <column>
    <name>column_name</name>
    <description>the column name</description>
    <dataType xsi:type="vs:TAPType" size="64">VARCHAR</dataType>
  </column>
  <column>
    <name>utype</name>
    <description>lists the utypes of columns in the tableset</description>
    <dataType xsi:type="vs:TAPType" size="512">VARCHAR</dataType>
  </column>
  <column>
    <name>ucd</name>
    <description>lists the UCDs of columns in the tableset</description>
    <dataType xsi:type="vs:TAPType" size="64">VARCHAR</dataType>
  </column>
  <column>
    <name>unit</name>
    <description>lists the unit used for column values in the tableset</description>
    <dataType xsi:type="vs:TAPType" size="64">VARCHAR</dataType>
  </column>
  <column>
    <name>description</name>
    <description>describes the columns in the tableset</description>
    <dataType xsi:type="vs:TAPType" size="512">VARCHAR</dataType>
  </column>
  <column>
    <name>datatype</name>
    <description>lists the ADQL datatype of columns in the tableset</description>
    <dataType xsi:type="vs:TAPType" size="64">VARCHAR</dataType>
  </column>
  <column>
    <name>size</name>
    <description>lists the size of variable-length columns in the tableset</description>
    <dataType xsi:type="vs:TAPType">INTEGER</dataType>
  </column>
  <column>
    <name>principal</name>
    <description>a principal column; 1 means 1, 0 means 0</description>
    <dataType xsi:type="vs:TAPType">INTEGER</dataType>
  </column>
  <column>
    <name>indexed</name>
    <description>an indexed column; 1 means 1, 0 means 0</description>
    <dataType xsi:type="vs:TAPType">INTEGER</dataType>
  </column>
  <column>
    <name>std</name>
    <description>a standard column; 1 means 1, 0 means 0</description>
    <dataType xsi:type="vs:TAPType">INTEGER</dataType>
  </column>
  <foreignKey>
    <targetTable>TAP_SCHEMA.tables</targetTable>
    <fkColumn>
      <fromColumn>table_name</fromColumn>
      <targetColumn>table_name</targetColumn>
    </fkColumn>
  </foreignKey>
</vosi:table>
\end{lstlisting}


\appendix
\section{The Complete VOSICapabilities Schema}
\label{appendix:capabilities}

\lstinputlisting[language=XML]{VOSICapabilities-v1.0.xsd}

\section{The Complete VOSITables Schema}
\label{appendix:tables}

\lstinputlisting[language=XML]{VOSITables-v1.1.xsd}

\section{Use Case for Capability Harvesting (non-normative)}
\label{appendix:harvesting}

 In the section \ref{sec:introduction}, we summarized the role that the
 metadata retrieval functions play in the discovery of services. In
 particular, it mentions that a registry can harvest this information
 from a service's VOSI interface to save the provider from entering the
 information explicitly into a web form. In this appendix, we describe
 this use case in more detail, including both the publishing of the
 metadata and its typical use by service clients.

Some publishing registries \citep{2018ivoa.spec.0723D} provide a
publicly accessible publishing tool: it allows any data service provider
to ``register'' his service by providing the necessary metadata adequate
to describe it. Such a tool typically provides a form that the provider
fills out to enter all the metadata; the form processor uses those
inputs to format a VOResource record that describes the service based on
that metadata. Prior to the registry's support for the VOSI metadata
functions, entering all of the metadata (particularly, fully describing
all of the table columns) would often be laborious; consequently,
providers are effectively discouraged from providing the mostly optional
information.

With support for the VOSI metadata functions, the registry's publishing
tool can now offer an alternative mechanism for providing much of the
information. After generally describing the service via core metadata
(e.g., its title, identifier, general description, contact information,
etc.), the registry can offer the option of entering the VOSI URLs that
provide the capability and table metadata. In the case of TAP, where
these VOSI functions are mandated as part of the TAP interface, it would
only be necessary for the user to enter the TAP service's base URL (in
VOResource parlance, the ``access URL''). In either case, the tool would
access the VOSI URLs, pull over the metadata, integrate it into the core
metadata, and show the provider the combined results before publishing
it to the registry. The tool might also ask the provider if this data is
expected to change over time and thus whether the VOSI URLs should be
polled regularly to update the service description held by the registry.
Alternatively, the tool may may allow the provider to quickly update the
service description via a single button click that causes the tool to
re-access the VOSI endpoints and refresh service description held by the
registry.

We note that pulling certain metadata from the service itself is
expected to save the provider time and effort because the information is
in large part inherently available to the service implementation. This
most obviously applies to the table metadata: the provider's underlying
database will normally have access to table schemas which can be used to
provide, for instance, detailed descriptions of all the tables and their
columns. It is less natural, perhaps, for the service to have access to
the capability information; however, with the growing use of service
toolkits that allow a provider to quickly deploy compliant IVOA
services, it is possible that the capability information could naturally
be assembled from the configuration information that was used to set up
the service. Of course, if the provider is forced to create static XML
documents manually to implement the VOSI functions, it's unlikely that
this has saved her any time over entering the metadata into a publishing
tool.

A key goal of the VOSI metadata functions is to encourage the capture of
metadata that is useful for discovering and selecting services that a
user will want to work with. For example, a user may wish to find
services that access a table containing redshift values. Or, a user may
wish to find Simple Spectral Services (a particular capability) that are
fully compliant. A second goal is to make that metadata available so
that the user can plan its use of the service: for example, the user,
through some tool, might browse through the table column descriptions to
figure out how to form her query. In this latter use, the client tool
can either use the registry as the source of this information or the
service itself via its VOSI functions. The question that arises for the
client tool developer then is, which source -- the registry or the
service -- should be preferred?

This VOSI specification does not recommend the use of one source over
the other. The choice, in general, will depend on the context of a
particular client tool and what it is trying to do, and the preferences
of developers may indeed evolve over time as, say, VOSI support becomes
more ubiquitous. At least initially, client tools -- particularly
general ones that can engage different kinds of service protocols --
will likely prefer to use the registry for the source of capability and
table metadata. The main reason would be that not all services will be
implemented to support VOSI. By going to the registry in this case, the
client gets this metadata for both services where it was retrieved via
VOSI and where it was entered explicitly into a publishing tool by the
provider. Under certain circumstances however, say, where the client
works with just one kind of service protocol like TAP in which VOSI
support is mandated for compliance, the VOSI interface might be the
preferred source. In particular, if the service URL was obtained by the
client through some means other than the discovery in a registry, then
it would not be necessary for the client to go to the registry to
understand what can be done with the service; the tool can get this
information from the service itself. 

We have implied in the above discussion that the capability and table
metadata are the same whether they are retrieved from the registry or
from the service. It is possible, however, that the service could change
-- new capabilities or table columns could be added -- and the registry
could (at least temporarily, depending on the registry) get out of sync
with the service. This circumstance may occur rarely; nevertheless, if
being up-to-date is important, then the client may need to be more
sophisticated in its retrieval. That is, it could retrieve the resource
description from the registry; then if the description indicates support
for VOSI, the VOSI URLs would be accessed to get the latest, up-to-date
information. 

\section{Changes from Previous Versions}
\label{appendix:changes}

\subsection{Changes since REC-1.1}

\begin{itemize}
\item Removed specification for the availability endpoint; added a brief
discussion of the rationale.
\end{itemize}

\subsection{Changes since REC-VOSI-20110531}

Added alternate root element (table) to VOSITables schema.

Extended VOSITables REST binding to include detail level and parameter
and child table resource for scalability.

Defined \#tables-1.1 standardID.

Removed references to SOAP bindings.

\subsection{Changes since PR-20101206}

Added Appendix B, use case discussion
    
Formatting comments from RFC addressed: typos, etc.

\subsection{Changes since PR-20100311}

Inclusion of IVOA Architecture text

Restructuring and clarification in response to RFC comments

Inclusion of VOSITables schema in appendix

Second example added for a TAP service response

\subsection{Changes since WD-20090825}

Mandate the use of VOSICapabilities to return capabilities

S2.1: added non-normative note about capability sub-types; added example
capabilities metadata

Recommend the inclusion of VOSI interfaces in capability metadata

S2.5: When returning capabilities metadata, require VOSI (REST)
accessURLs to have use="full"; recommend this use of ParamHTTP.

Rename Availability schema to VOSIAvailability; added VOSICapabilities schema.

\subsection{Changes since WD-20081030}

The REST binding is made mandatory for all kinds of service. Details of
the SOAP binding, including its WSDL contract, are removed.

The definition of the root element for the table-metadata document is
corrected. Instead of requiring the tableset element from VODataService
1.1 (which element does not exist in that schema), the text now requires
an element of type TableSet. 


\bibliography{ivoatex/ivoabib,ivoatex/docrepo}


\end{document}