Portability
Intent
The portability specification allows for the representations of entire workflow projects "as code", lets user move between various deployment pathways (cloud, local, DIY, etc.) and proposes a globally-applicable way of specifying workflow automation and systems integration that might be applied across workflow-engines/integration platforms across the sector. Nothing about the spec must be specific to OpenFn or any one of our individual products. We envision a future in which software built with Lightning, the OpenFn Integration Toolkit, and entirely new and different integration/workflow tools can adopt this specification.
If you're interested in contributing to the specification, reach out to OpenFn via the community forum, write to us, or suggest changes by submitting a pull request here.
"Projects as code"
The portability specification v4 defines how entire projects (groups of workflows with their associated triggers, edges, credentials and jobs) can be represented as code. It improves the OpenFn developer experience, allowing workflows to be built and tested locally; (b) enables project version control and an audit trail of project changes; and (c) allows users to port existing workflows from OpenFn v1 to v2, as well as between instances or deployments of Lightning.
The project "spec"
The project specification (or "spec") is often saved as a project.yaml file.
name: openhie-project
description: Some sample
# credentials:
# globals:
workflows:
OpenHIE-Workflow:
name: OpenHIE Workflow
jobs:
FHIR-standard-Data-with-change:
name: FHIR-standard-Data-with-change
adaptor: '@openfn/language-http@latest'
enabled: true
# credential:
# globals:
body: |
fn(state => {
console.log("hello github integration")
return state
});
Send-to-OpenHIM-to-route-to-SHR:
name: Send-to-OpenHIM-to-route-to-SHR
adaptor: '@openfn/language-http@latest'
enabled: true
# credential:
# globals:
body: |
fn(state => state);
Notify-CHW-upload-successful:
name: Notify-CHW-upload-successful
adaptor: '@openfn/language-http@latest'
enabled: true
# credential:
# globals:
body: |
fn(state => state);
Notify-CHW-upload-failed:
name: Notify-CHW-upload-failed
adaptor: '@openfn/language-http@latest'
enabled: true
# credential:
# globals:
body: |
fn(state => state);
triggers:
webhook:
type: webhook
edges:
webhook->FHIR-standard-Data-with-change:
source_trigger: webhook
target_job: FHIR-standard-Data-with-change
condition: always
FHIR-standard-Data-with-change->Send-to-OpenHIM-to-route-to-SHR:
source_job: FHIR-standard-Data-with-change
target_job: Send-to-OpenHIM-to-route-to-SHR
condition: on_job_success
Send-to-OpenHIM-to-route-to-SHR->Notify-CHW-upload-successful:
source_job: Send-to-OpenHIM-to-route-to-SHR
target_job: Notify-CHW-upload-successful
condition: on_job_success
Send-to-OpenHIM-to-route-to-SHR->Notify-CHW-upload-failed:
source_job: Send-to-OpenHIM-to-route-to-SHR
target_job: Notify-CHW-upload-failed
condition: on_job_failure
The project "state"
The project state is a representation of a particular project as on a specific
Lightning instance. It is often saved as projectState.json and contains UUIDs
for resources on a particular Lightning deployment.
{
"workflows": {
"OpenHIE-Workflow": {
"id": "27ae2937-0959-48b8-a597-b1646aae8c14",
"name": "OpenHIE Workflow",
"jobs": {
"Transform-data-to-FHIR-standard": {
"id": "e44f65bb-5038-4e17-8d93-b63cbe95254a",
"delete": true
},
"Send-to-OpenHIM-to-route-to-SHR": {
"id": "977b87ff-f347-42b5-832f-6ae2ca726f32",
"name": "Send-to-OpenHIM-to-route-to-SHR",
"adaptor": "@openfn/language-http@latest",
"body": "fn(state => state);\n",
"enabled": true
},
"Notify-CHW-upload-successful": {
"id": "86b743a3-fd00-4629-b9fb-d5f38fb56d0b",
"name": "Notify-CHW-upload-successful",
"adaptor": "@openfn/language-http@latest",
"body": "fn(state => state);\n",
"enabled": true
},
"Notify-CHW-upload-failed": {
"id": "be85df30-0abd-4f8e-be17-501f67e18b8d",
"name": "Notify-CHW-upload-failed",
"adaptor": "@openfn/language-http@latest",
"body": "fn(state => state);\n",
"enabled": true
},
"FHIR-standard-Data": {
"id": "55016dda-42e3-4ee1-8a9c-24e3f23d42f1",
"delete": true
},
"FHIR-standard-Data-with-change": {
"id": "28dd0846-a6ae-40c0-8ab4-3e0a6b487afe",
"name": "FHIR-standard-Data-with-change",
"adaptor": "@openfn/language-http@latest",
"body": "fn(state => state);\n",
"enabled": true
}
},
"triggers": {
"webhook": {
"id": "530cde0b-0de4-4f68-8834-0a4356a2fe53",
"type": "webhook"
}
},
"edges": {
"webhook->Transform-data-to-FHIR-standard": {
"id": "b2c7407b-0ae9-4ca5-9d6b-ee624976fa54",
"delete": true
},
"Transform-data-to-FHIR-standard->Send-to-OpenHIM-to-route-to-SHR": {
"id": "d22ed6f4-26a2-4c85-b261-cc110a6851e6",
"delete": true
},
"Send-to-OpenHIM-to-route-to-SHR->Notify-CHW-upload-successful": {
"id": "26c12f7f-7806-4008-87cd-6747998f95f4",
"condition": "on_job_success",
"source_job_id": "977b87ff-f347-42b5-832f-6ae2ca726f32",
"source_trigger_id": null,
"target_job_id": "86b743a3-fd00-4629-b9fb-d5f38fb56d0b"
},
"Send-to-OpenHIM-to-route-to-SHR->Notify-CHW-upload-failed": {
"id": "0630ac96-4f67-4de7-8c3d-0bf3f89f80d9",
"condition": "on_job_failure",
"source_job_id": "977b87ff-f347-42b5-832f-6ae2ca726f32",
"source_trigger_id": null,
"target_job_id": "be85df30-0abd-4f8e-be17-501f67e18b8d"
},
"webhook->FHIR-standard-Data": {
"id": "5ce3a8ed-b9eb-464a-a2cd-ba55adc393c2",
"delete": true
},
"FHIR-standard-Data->Send-to-OpenHIM-to-route-to-SHR": {
"id": "5f459cd9-2882-4a61-a2cc-ec45e58d4837",
"delete": true
},
"webhook->FHIR-standard-Data-with-change": {
"id": "75e7f7d8-274b-410d-9600-730bbd535229",
"condition": "always",
"source_job_id": null,
"source_trigger_id": "530cde0b-0de4-4f68-8834-0a4356a2fe53",
"target_job_id": "28dd0846-a6ae-40c0-8ab4-3e0a6b487afe"
},
"FHIR-standard-Data-with-change->Send-to-OpenHIM-to-route-to-SHR": {
"id": "1e5ba385-2c49-4241-8cd2-042c99a810ec",
"condition": "on_job_success",
"source_job_id": "28dd0846-a6ae-40c0-8ab4-3e0a6b487afe",
"source_trigger_id": null,
"target_job_id": "977b87ff-f347-42b5-832f-6ae2ca726f32"
}
}
}
},
"id": "8deff39d-8189-4bd7-9dc7-f9f08e7f2c60",
"name": "openhie-project"
}
Using the CLI to deploy or describe projects projects as code
The project spec and project state can be used for a variety of reasons, e.g. one could generate the state and spec as backups of the project or one could generate these files and use them for auditing and record keeping, etc. The OpenFn CLI comes with commands that can be used to pull project configurations down from a running Lightning server, and to deploy or push updates to existing projects on a Lightning server. To learn more about automated version control via pull and deploy, head over to our Version Control docs.
Install it by running npm install -g @openfn/cli
Before using the CLI, configure it either by passing in environment variables:
OPENFN_ENDPOINT=https://app.openfn.org
OPENFN_API_KEY=yourSecretApiToken
Or through a config.json file:
{
// Required, can be overridden or set with `OPENFN_API_KEY` env var
"apiKey": "***",
// Optional: can be set using the -p, defaults to project.yaml
"specPath": "project.yaml",
// Optional: can be set using -s, defaults to .state.json
"statePath": ".state.json",
// Optional: defaults to OpenFn.org's API, can be overridden or set with
// `OPENFN_ENDPOINT` env var
"endpoint": "https://app.openfn.org"
}
More details on the CLI can be found here.
openfn pull to generate a project spec and state
To generate the spec and state files for an existing project, use:
openfn pull {YOUR-PROJECT-UUID} -c ./config.json
This command will save (or overwrite) a project spec and state file based on the path you've set in your configuration.
openfn deploy to create a project on a Lightning instance
To deploy a new project to a Lightning instance from a project spec (without a project state) file use:
openfn deploy -c config.json
openfn deploy to update an existing project
With a valid project state defined in your config.json, the same
openfn deploy command will beam up your changes as described by a difference
between your project spec and what's found on the server.
openfn deploy -c config.json
Checking https://demo.openfn.org/api/provision/4adf2644-ed4e-4f97-a24c-ab35b3cb1efa for existing project.
Project found.
[CLI] ♦ Changes:
{
workflows: [
{
jobs: [
{
- body: "fn(state => {\n console.log(\"ok\")\n return state\n});"
+ body: "fn(state => {\n console.log(\"some changes here!\")\n return state\n});\n"
}
...
...
...
]
}
]
}
? Deploy? yes
[CLI] ♦ Deployed.
Getting Help with the cli
The cli package comes with an inbuilt help. Adding --help to a command such
as openfn deploy --help will result in a help message describing the command
and the options available when using this command. See an example below
openfn deploy --help
openfn deploy
Deploy a project's config to a remote Lightning instance
Options:
--version Show version number [boolean]
--help Show help [boolean]
-c, --config, --config-path The location of your config file [default: "./.config.json"]
--no-confirm Skip confirmation prompts (e.g. 'Are you sure?') [boolean]
--describe Downloads the project yaml from the specified instance [boolean]
-l, --log Set the log level [string]
--log-json Output all logs as JSON objects [boolean]
-p, --project-path The location of your project.yaml file [string]
-s, --state-path Path to the state file
Other Versions
- Portability Proposal v4
- Portability Proposal v3
- Portability Proposal v2 (
@latestfor platform-app/microservice compatibility.) - Portability Proposal v1