Rust library interface

This document describes the design of the functions forming the Rust library and tracks the implementation status. All of these functions are external-facing functions that are used internally in the CLI commands.

We use symbols to indicate the status of implementation (see table below). For planned or in-progress work, we might include signatures, docstrings, and pseudocode to clarify the design. Once the interface is implemented, these are replaced by links to the reference documentation.

A table showing the symbols used to indicate the status of interface components, along with their descriptions.

Status	Description
	Interface that has been implemented.
	Interface that is currently being worked on.
	Interface that is planned, but isn’t being worked on currently.

read_package()

This function reads in the data package’s metadata file (currently only the datapackage.json file) into a Package struct. While we called it source in the args of many CLI commands, we call it package here to be clearer. “Source” on it’s own is a vague term, “source of what?”, so we wanted to be explicit that it is the data package’s metadata we are reading in. Plus, the struct is called Package. However, the read_package() has the argument called source, which is the location given in the SOURCE arg of many of the CLI commands.

At some point we aim to support other metadata formats, but for now only datapackage.json is supported.

The signature is (with an internal PackageSource enum for the different source locations):

read_package(source: PackageSource) -> Result<Package, Error>

read_request()

This reads in the request.yaml file into the Request struct. This functionality is very similar to read_package(). The signature is:

read_request(path: &str) -> Result<Request, Error>

check()

The check() function first reads and parses a request.yaml and a metadata file, datapackage.json, checking that they contain the required keys. It then runs the checks described below in order, returning a CheckError if any check fails.

flowchart TD
    request["request<br>path to request.yaml"]
    metadata["metadata<br>path to datapackage.json"]
    output[/"Output<br>Result&lt;(), CheckError&gt;"/]

    subgraph check ["check()"]
      read_request["read_request()<br>Parse request into a Request struct"]
      read_metadata["read_metadata()<br>Parse metadata file into a Package struct"]
      check_package_name["check_package_name()<br>Package name matches metadata"]
      check_package_version["check_package_version()<br>Package version matches metadata"]
      check_resource_names["check_resource_names()<br>Resource names match metadata"]
      check_column_names["check_column_names()<br>Column names match metadata"]
      check_row_filters["check_row_filters()<br>Row filters match metadata columns"]
    end

    request --> read_request --> check_package_name --> check_package_version --> check_resource_names --> check_column_names --> check_row_filters --> output
    metadata --> read_metadata --> check_package_name

    check:::outer_function

  classDef outer_function fill:none

Figure 1: The flow of input and output through the check function.

/// Checks a request yaml against the metadata file.
///
/// The checks include:
///
/// - Required sections and keys are present in the request and metadata.
/// - Data package name and version in the request matches the metadata.
/// - The resources and columns in the request exist in the metadata.
/// - The row filters match columns in the metadata.
///
/// Errors if the request or metadata file can't be read, or if any
/// check fails.
pub fn check(request: Path, metadata: Path) -> Result<(), CheckError> {
    let request = read_request(request);
    let metadata = read_metadata(metadata);
    // Package name and version matches metadata.
    check_package_name(&request, &metadata);
    check_package_version(&request, &metadata);
    // Resource names match metadata.
    check_resource_names(&request, &metadata);
    // Column names match metadata.
    check_column_names(&request, &metadata);
    // Row filters match metadata columns (including whether the operator
    // types and values are allowed with the column type).
    check_row_filters(&request, &metadata);

    Ok(())
}