Upgrading from v12 to v13.0¶

Prerequisites¶

The steps listed in this article require an existing local installation of InvenioRDM v12.

Upgrade Steps¶

Make sure you have the latest invenio-cli installed. For InvenioRDM v13, it should be v1.9.0+.

$ invenio-cli --version
invenio-cli, version 1.9.0

Upgrade InvenioRDM¶

Requirements¶

Python 3.9 or 3.11 or 3.12 is required to run InvenioRDM v13.

The minimum required OpenSearch version is now v2.12. See below on how to upgrade older versions.

Upgrade option 1: In-place¶

This approach upgrades the dependencies in place. At the end of the process, your virtual environment for the v12 version will be completely replaced with the v13 environment and dependencies.

cd <my-site>

# Upgrade to InvenioRDM v13
invenio-cli packages update 13.0.0
# The old `invenio-admin` dependency has been removed and must be uninstalled
pipenv run pip uninstall -y invenio-admin
# Re-build assets
invenio-cli assets build

Upgrade option 2: New virtual environment¶

This approach will create a new virtual environment and leaves the v12 one as-is. If you are using a docker image on your production instance this will be the option you choose.

Step 1¶

• create a new virtual environment
• activate your new virtual environment
• install invenio-cli by running pip install invenio-cli

Step 2¶

Update the <my-site>/Pipfile by changing the version of invenio-app-rdm to ~=13.0.0 and removing the unnecessary postgresql extra (it is already installed by default and will trigger a warning if left in the file):

[packages]
---invenio-app-rdm = {extras = [..., "postgresql"], version = "~=12.0.0"}
+++invenio-app-rdm = {extras = [...], version = "~=13.0.0"}

If you're using Sentry (tool for monitoring or error tracking), update the dependency in <my-site>/Pipfile to:

---invenio-logging = {extras = ["sentry_sdk"], version = "~=2.0"}
+++sentry-sdk = {extras = ["flask"], version = ">=1.0.0,<2.0.0"}

Step 3¶

Update the Pipfile.lock file:

invenio-cli packages lock

Step 4¶

Install InvenioRDM v13:

invenio-cli install

Activate the virtual environment¶

Before running any invenio commands, activate your virtual environment shell:

$ invenio-cli shell
Launching subshell in virtual environment...
source <path to virtualenvs>/bin/activate

This step ensures that all subsequent commands use the correct Python environment and installed dependencies.

Database migration¶

Execute the database migration:

invenio alembic upgrade

Data migration¶

Execute the data migration:

invenio shell $(find $(pipenv --venv)/lib/*/site-packages/invenio_app_rdm -name migrate_12_0_to_13_0.py)

Rebuild search indices¶

invenio index destroy --yes-i-know
search_prefix=$(invenio shell -c "print(app.config['SEARCH_INDEX_PREFIX'])")
invenio index delete --force --yes-i-know "${search_prefix}rdmrecords-records-record-*-percolators"
invenio index init
# if you have records custom fields
invenio rdm-records custom-fields init
# if you have communities custom fields
invenio communities custom-fields init
invenio rdm rebuild-all-indices

From v12 onwards, search indices for statistics (record's views and downloads) are not affected by invenio index destroy --yes-i-know and are totally functional after the rebuild step.

opensearchpy.exceptions.AuthorizationException: AuthorizationException(403, 'security_exception', 'no permissions for [cluster:admin/component_template/put] and User [name=<my-name>, backend_roles=[], requestedTenant=null]')
opensearchpy.exceptions.AuthorizationException: AuthorizationException(403, 'security_exception', 'no permissions for [indices:admin/index_template/put] and User [name=<my-name>, backend_roles=[], requestedTenant=null]')

Updated affiliations and funders¶

InvenioRDM now integrates the updated schema version v2 of ROR (see announcement here). This new version introduces additional fields and improvements, so you will need to re-import both the affiliations and funders vocabularies to ensure your data is up to date.

To re-import, you can either set up a job or perform the import manually via the CLI. Please follow the instructions in the affiliations and funders documentation pages for detailed steps. Since these imports use external data sources, local overrides via {funders,affiliations}.yml in your instance's app_data/ aren't needed anymore and can be discarded.

Infrastructure/configuration changes¶

Required changes¶

OpenSearch version¶

The minimum required OpenSearch version is now v2.12. This change is necessary due to a bug in earlier OpenSearch versions that affects the handling of geo-shape fields introduced in InvenioRDM v13. For more details, see the related InvenioRDM issue and the OpenSearch issue and discussion.

To check the current version, connect to the OpenSearch Dashboard with your browser or run the following CLI command:

$ curl -X GET "http://localhost:9200"
{
  ...
  "version" : {
    "distribution" : "opensearch",
    "number" : "2.17.1",
    ...
  },
  ...
}

Add -u <username>:<password> if you require authentication, or -k to ignore SSL certificate warnings.

Please refer to the official OpenSearch upgrade documentation. If you choose to delete and re-create your search cluster as part of the upgrade, remember to back up and restore your statistics indices (see how to do this here). Be sure to repeat the Rebuild search indices step to ensure your system is fully functional.

Jobs¶

The new Jobs feature uses a custom celery task scheduler which requires a separate celery beat. See the related documentation or the scheduler service in the InvenioRDM boilerplate for how to add it.

Updated PDF previewer¶

The updated PDF file previewer (PDF.js v4) now uses ECMAScript modules (.mjs) instead of CommonJS files (.js). By default, the nginx web server does not recognize .mjs files in its default MIME types configuration. As a result, the MIME type may be reported incorrectly, causing browsers to block the file and resulting in broken PDF previews.

To resolve this, simply add a custom types entry in your nginx.conf (for example, in the http block). See this change in the InvenioRDM boilerplate for reference.

  ...
  include       /etc/nginx/mime.types;
  default_type  application/octet-stream;
+ types {
+     # Ensure nginx treats ECMAScript modules as JavaScript
+     application/javascript js mjs;
+ }
  ...

Optional changes¶

Deprecations¶

APP_ALLOWED_HOSTS

With the upgrade to Flask version 3, the configuration variable APP_ALLOWED_HOSTS has been renamed to TRUSTED_HOSTS. The value remains unchanged. You should review your invenio.cfg, environment variables, and deployment configuration for any occurrences of the old variable name. It is recommended to update all references to TRUSTED_HOSTS to avoid deprecation warnings in the console and prepare for its eventual complete removal.

invenio_records_resources.services.Link

Usage of invenio_records_resources.services.Link is deprecated in favor of invenio_records_resources.services.EndpointLink for InvenioRDM links and invenio_records_resources.services.ExternalLink for external third-party links. Replace instances of Link in your custom code, if any, appropriately. Link will be removed in a future major InvenioRDM release.

APP_RDM_DEPOSIT_FORM_PUBLISH_MODAL_EXTRA

Usage of the configuration APP_RDM_DEPOSIT_FORM_PUBLISH_MODAL_EXTRA is deprecated and will be removed in a future major InvenioRDM release. Override the modal component with your own customized compliance information instead.

Display versions in administration panel¶

As described in the release notes, you can now display the versions of your installed modules directly in the Administration panel. To enable this feature, add the following to your invenio.cfg:

from invenio_app_rdm import __version__
ADMINISTRATION_DISPLAY_VERSIONS = [
    ("invenio-app-rdm", f"v{__version__}"),
    ("<my module name>",
]

FAIR signposting level 1¶

If you have enabled FAIR Signposting, ensure that you have updated your web server configuration as required. See the documentation here for detailed instructions.

Failing to apply these changes may result in errors when accessing certain records.

Enhanced File Uploader (Uppy)¶

If you plan to use the new Uppy file uploader, ensure that your Content Security Policy (CSP) settings are updated in your invenio.cfg as described in the enhanced file uploader documentation. Failing to update your CSP rules may prevent the uploader from functioning correctly.

New configuration variables¶

These are the new configuration variables introduced in this release. Make sure that you read the related documentation before enabling them. Add them to your invenio.cfg as needed:

# Enable FAIR Signposting
APP_RDM_RECORD_LANDING_PAGE_FAIR_SIGNPOSTING_LEVEL_1_ENABLED = True
# Enable Audit Logs
AUDIT_LOGS_ENABLED = True
# Enable Curation Checks (manual setup required)
CHECKS_ENABLED = True
# Enable the new Uppy file uploader (experimental feature)
APP_RDM_DEPOSIT_NG_FILES_UI_ENABLED = True