Exposing a R function

You can expose any R function as a endpoint on the API node. Calling the endpoint will call your function with the parameters you specify and return the results of the function.

This might look similar to the R prediction endpoint, but there are a few key differences:

  • A Python prediction endpoint has a strict concept of input records, and output prediction. It must output a prediction, and thus can only be used for prediction-like use cases. In contrast, a R function can do any kind of action and return any form of result (or even no result). For example, you can use a R function endpoint to store data in a database, a file, …
  • Since there is no concept of input records, you cannot use the dataset-based enrichment features in a R function endpoint

Creating the R function endpoint

To create a custom prediction endpoint, start by creating a service. (See Your first API service for more information). Then, create an endpoint of type “Custom function (R)”.

DSS prefills the Code part with a sample

Structure of the code

The code of a R function endpoint must include at least one function, whose name must be entered in the “Settings” tab.

The parameters of the function will be automatically filled from the parameters passed in the endpoint call.

The result of the function must be JSON-serializable.

Using managed folders

A R function endpoint can optionally reference one or several DSS managed folders. When you package your service, the contents of the folders are bundled with the package, and your custom code receives the paths to the managed folder contents.

The paths to the managed folders (in the same order as defined in the UI) is available by calling the dkuAPINodeGetResourceFolders() function. This function returns a vector of character vectors, each one being the absolute path to the folder.

Testing your code

Developing a custom model implies testing often. To ease this process, a “Development server” is integrated in the DSS UI.

To test your code, click on the “Deploy to Dev Server” button. The dev server starts and load your model. You are redirected to the Test tab where you can see whether your model loads.

You can also define Test queries, i.e. JSON objects akin to the ones that you would pass to the API node user API. When you click on the “Play test queries” button, the test queries are sent to the dev server, and the result is printed.

Python packages and versions

We strongly recommend that you use code environments for deploying Python function endpoints if these packages use any external (not bundled with DSS) library

Available APIs in a custom model code

Note that, while the dataiku.* libraries are accessible, most of the APIs that you use in Python recipes will not work: the code is not running with the DSS Design node, so datasets cannot be read by this API. If you need to access DSS managed folders, see Using managed folders above.

Using your own libraries

You will sometimes need to write custom library functions (for example, shared between your custom training recipe and your custom model).

You can place these custom Python files in the project’s “libraries” folder, or globally in the lib/python folder of the DSS installation. Both recipes and custom models can import modules defined there.

When you package a service, the whole content of the lib/python folders (both project and instance) are bundled in the package. Note that this means that it is possible to have several generations of the service running at the same time, using different versions of the custom code from lib/python.

Server-side tuning

It is possible to tune the behavior of R prediction endpoints on the API node side. You can tune how many concurrent requests your API node can handle. This depends mainly on your model (its speed and in-memory size) and the available resources on the server running the API node.

You can configure the parallelism parameters for an endpoint by creating a JSON file in the config/services folder in the API node’s data directory.

mkdir -p config/services/<SERVICE_ID>

Then create or edit the config/services/<SERVICE_ID>/<ENDPOINT_ID>.json file

This file must have the following structure and be valid JSON:

{
    "pool" : {
        "floor" : 1,
        "ceil" : 8,
        "cruise": 2,
        "queue" : 16,
        "timeout" : 10000
    }

}

This configuration allows you to control the number of allocated pipelines.

One allocated pipeline means one R process running your code, preloaded with your initialization code, and ready to serve a prediction request. If you have 2 allocated pipelines (meaning 2 R processes), 2 requests can be handled simultaneously, other requests will be queued until one of the pipelines is freed (or the request times out). When the queue is full, additional requests are rejected.

Those parameters are all positive integers:

  • floor (default: 1): Minimum number of pipelines. Those are allocated as soon as the endpoint is loaded.
  • ceil (default: 8): Maximum number of allocated pipelines at any given time. Additional requests will be queued. ceil ≥ floor
  • cruise (default: 2): The “nominal” number of allocated pipelines. When more requests come in, more pipelines may be allocated up to ceil. But when all pending requests have been completed, the number of pipeline may go down to cruise. floor ≤ cruise ≤ ceil
  • queue (default: 16): The number of requests that will be queued when ceil pipelines are already allocated and busy. The queue is fair: first received request will be handled first.
  • timeout (default: 10000): Time, in milliseconds, that a request may spend in queue wating for a free pipeline before being rejected.

Each R process will only serve a single request at a time.

It is important to set “cruise”:

  • At a high-enough value to serve your expected reasonable peak traffic. If you set cruise too low, DSS will kill excedental R processes, and will need to recreate a new one just afterwards.
  • But also at a not-too-high value, because each pipeline implies a running R process consuming the memory required by the model.

You can also deploy your service on multiple servers, see High availability and scalability.