COVID Policy Modelling Model Connector How-To: Creating a connector

In this section, you will create a connector for a model of your choice. After completion, you should have a complete connector which is capable of running simulations on your local machine, and which could be integrated with COVID-UI.

Contents

Assumptions

The main body of these instructions assume the following:

The instructions also contain additional notes to support any of the following:

Creating a connector

Develop your connector by editing the Dockerfile, connector code and test input, then building, running and validating your connector. The below process is best followed iteratively, starting with a basic connector with dummy behaviour, and then making alterations, testing and validating throughout. This process includes only the high-level steps required - further details may be found in the README.md file in your connector repository.

  1. Edit the file Dockerfile to install the model, along with your connector code:

    • Set an appropriate base image for your language e.g. python, r-base etc.
      • (Recommended) Pin to a specific version, e.g. python:3.9 - the level of specificity may vary according to the language’s conventions, requirement for reproducibility, frequency of development etc.
    • Install the model code using the usual conventions for your language, e.g. using RUN pip for Python (as shown in the tutorial)
    • Install a JSON Schema validator as well.
    • Add the connector code using COPY.
    • Add the schema files using COPY as well
    • Set the CMD to run your connector code, passing in as parameters:
      • input location (/data/input/inputFile.json)
      • output location (/data/output/output.json)
      • input schema location (from your COPY command)
      • output schema location (from your COPY command)
    • If your model is not available from a public package repository, refer to the notes below:

  2. Create your connector code, which needs to:

    • Parse the input JSON file.
    • Validate the input against the schema file.
    • Run a simulation using the model, interpreting the input parameters appropriately as suitable for the model.
      • You must check the region and subregion parameters, and either produce results pertaining to that subregion or exit with an error if the subregion is not supported.
      • You should support as many of the interventionPeriod parameters as is appropriate for the model, but can ignore any (or all) of them if necessary.
      • For more information on the format of the file, you can refer to an annotated example or schema documentation
    • Convert the output of the simulation into an object matching the output schema.
      • All metric keys are required. For any metrics that the model does not support, you should return an array of appropriate length, consisting of 0 values.
      • For more information on the format of the file, you can refer to annotated example or schema documentation.
    • Include the input parameters as the metadata key of the output.
    • Include metadata about the model and connector as the model key of the output.
      • The sample Dockerfile provides a CONNECTOR_VERSION environment variable, which can be used as the value for connectorVersion, e.g. in Python, using os.getenv("CONNECTOR_VERSION").
      • For modelVersion, use a value meaningful to the model in use. For example, Covasim provides a covasim.__version__ attribute.
    • Validate the output against the schema file.
    • Exit with a zero status if the simulation succeeded, or a non-zero status otherwise.
      • For many languages, this is default behaviour.
    • If your model is not available as a library or your connector does not use the common schema, refer to the notes below:

  3. Edit the test data in test-job.json. This may be necessary for example if your model does not support the parameters specified in the file.

  4. Build your image (this might take some time on first run, but subsequent runs will usually be quicker):

    $ docker-compose build run-model
...
Successfully tagged <MODEL>-connector_run-model:latest

  5. Test your connector code:

    $ docker-compose run --rm run-model
Creating <MODEL>-connector_run-model_run ... done
...

  6. Validate the output of your connector code:

    $ docker-compose run --rm validate
Creating <MODEL>-connector_validate_run ... done
...

  7. Commit your changes:

    $ git add ...
$ git commit -m "..."

Next steps

Follow the steps for documenting your connector.

Additional steps for alternative approaches

The model is available as a non-interactive script or application

In your connector, use whatever is common in your language for calling external applications, e.g. subprocess in Python, child_process in Node.js etc. Depending on the exact nature of the application, you may also have to read or write the input and output parameters to files on disk. If necessary, make sure to check the output and result code for errors.

The model can be installed as a container

Use the existing model container as the base image. You should not usually have to take further steps to install the model or any dependencies.

It’s recommended to pin to a specific version - the level of specificity may vary according to the model’s development process, requirement for reproducibility, frequency of development etc.

The model can be installed from an external location

In your Dockerfile, obtain a copy of the model with something like RUN curl ... or RUN wget ... etc. Depending on your base image, you may need to install curl or wget first, e.g. with RUN apk add curl, RUN apt-get install curl etc. You may also need to add other RUN ... commands to install any prerequisites of the model.

The model can be installed from a source repository

In your Dockerfile, obtain a copy of the model with something like RUN git clone .... Depending on your base image, you may need to install git first, e.g. with RUN apk add git, RUN apt-get install git etc. If necessary, add other commands of the form RUN ... to carry out build steps. You may also need to add other RUN ... commands to install any prerequisites of the model.

The connector will use a custom schema

In your connector code, you will need to interpret the input and output parameters according to whatever your custom schema expects. You may wish to update the schema documentation if necessary. You will need to edit the test-job.json file to match a valid test input for the schema.

The connector will use a different shared schema

In your connector code, you will need to interpret the input and output parameters according to whatever the shared schema expects. There may be more information available in the schema documentation if the schema author provided it. You will need to edit the test-job.json file to match a valid test input for the schema.