How to update from API spec

These are the steps to follow when bringing this repo up to date from an updated API spec.

How this was built

  • Initially using then-current (2020-Aug-07) Docker container openapitools/openapi-generator-cli:latest
  • Now via the offical v5.0.0 release openapitools/openapi-generator-cli:v5.0.0
  • Standard invocation using TileDB Cloud API Spec v1, setting package name and version
  • Some manual edits (detailed below)
  • Invoking roxygen2::roxygenise() to build interactive R documentation followed by a one-word fix to one Rd file
  • Invoking pkgdown:build_site() to build HTML documentation
    • Update 2022-03-25: now done as a GitHub Action.

Prep

Prep: the Docker command in runMe.sh will read its current directory and write the following into ./tiledbcloud-generated: DESCRIPTION, NAMESPACE, README.md, R, docs, man, tests, and git_push.sh. Unfortunately it will read the current files from the working directory as well.

In either case we need a current openapi-v1.yaml copied from https://github.com/TileDB-Inc/TileDB-Cloud-API-Spec into ..

Autogen option 1

mkdir /tmp/gen
cp openapi-v1.yaml   /tmp/gen
cp runMe.sh          /tmp/gen
cp regenerate-docs.r /tmp/gen/
cp -a .openapi-generator* /tmp/gen

cd /tmp/gen
./runMe.sh

Then remove DESCRIPTION, NAMESPACE, R, docs, man, test from this repo directory (maybe mkdir old and mv them to old) with outputs from /tmp/gen. Then move /tmp/gen/README.md to ./origREADME.md.

Autogen option 2

mkdir old
mv DESCRIPTION old
mv NAMESPACE   old
mv README.md   old
mv R           old
mv docs        old
mv man         old
mv tests       old
mv git_push.sh old

./runMe.sh

Then move ./README.md to ./origREADME.md and old/README.md to ..

Post-processing

In either case, be sure any manual-layer files (such as init.R – listed in runMe.sh) are copied back to ./R. Then remove NAMESPACE and (in R) library(devtools) and devtools::document(). (The NAMESPACE file must be removed first since it was generated by openapi-generator so devtools will skip it.) This will ensure that exported functions in the manual-layer files are properly exported.

Important The generated API wants to also parse a field User which we apparently do not send. So the default deserializer breaks. See PR #2 and its small diff to R/api_client.R file for a fix that needs to be re-applied.

Also diff -r old/R R and see what other manual edits need to be re-applied. In particular, all manual edits needing re-application should be marked with MANUAL EDIT AFTER OPENAPI AUTOGEN.

Also manually edit DESCRIPTION, carrying over manual edits from before the autogen run.

Also:

R CMD build .
R CMD INSTALL .

See HowToRelease.md for how to update documentation pages and commit them to version control.

Testing

Just as the library has a higher line-count of code autogenerated by OpenAPI, and a lower line-count of manual-layer files, so too with testing. In tests/testthat are many autogenerated files – with function bodies written commented out by OpenAPI. We do not focus our testing in this direction.

In inst/tinytest are manually written tests. These are opt-in: they make TileDB Cloud calls, which we don’t want to push on all users who build the package locally.

To opt in:

  • Have either environment variable TILEDB_REST_TOKEN, or the pair TILEDB_REST_USERNAME and TILEDB_REST_PASSWORD – or all three if you like.
  • Optionally, TILEDB_REST_HOST if your cloud installation is local, and/or for TileDB employees pointing at our staging environment.
  • In R: tinytest::test_all(".")