Passer au contenu principal

Microservice

Microservice and devtools are being replaced by Lightning

Please note that OpenFn/microservice and OpenFn/devtools are being deprecated and replaced by OpenFn/Lightning, When lighting is released.

But​

OpenFn is used by numerous health and humanitarian organizations around the world to scale their programs through real-time interoperability, systems integration, and workflow automation. OpenFn/microservice makes use of OpenFn's open-core technology—namely OpenFn/core, OpenFn/engine, and the various OpenFn adaptors—to create standalone microservices which can be deployed on any hardware.

This microservice approach helps to ensure that governments and NGOs are never locked-in to OpenFn's SaaS offering, and can port their existing jobs, triggers, and credentials from OpenFn.org to their own infrastructure easily.

Introduction​

Similar to platform, OpenFn/microservice runs on project.yaml files. This means that when organizations or governments have an open-source license requirement, all their jobs, credentials, and project configurations can be exported from OpenFn's iPaaS and used to create a microservice deployment.

While this approach doesn't provide the OpenFn platform front-end with its various project management and configuration features, it's perfect for groups with DevOps experience and 100% compatible with the platform. You can even build and test entire projects on platform and then export the project.yaml file to run on your own servers using microservice.

This microservice approach provides flexibility to governments and NGOs, so they are never locked-in to OpenFn's SaaS platform offering. At any time, an organization can port their existing jobs, triggers, and credentials from OpenFn.org to run with our FOSS integration toolkit, using their own infrastructure.

Prérequis​

Familiarity with other elements of OpenFn's open source integration toolkit is helpful when considering the microservice approach.

Docker opérationnel​

Assuming you've got an .env and a sample project at ./sample-project directory with a project.yaml spec:

docker-compose up

You can configure either the compose file or the .env, or run the container using docker run:

docker run -v <path-to-your-project-folder>:/home/microservice/<path-to-your-project-folder> \
  --env-file <path-to-your-env-file> \
  --network host \
  openfn/microservice:v0.3.2

Guide de développement et de fonctionnement​

  • Cloner ce dépôt avec git clone git@github.com:OpenFn/microservice.git
  • Entrez le répertoire avec cd microservice
  • Installez les dépendances avec mix setup
  • Exécutez les tests avec mix test
  • Créez un répertoire de projet pour conserver les artefacts de votre projet mkdir sample-project
  • Créez une nouvelle spécification de projet avec cp project.yaml.example ./sample-project/project.yaml
  • Créez un fichier .env avec cp .env.example .env
  • Installez les adaptateurs nécessaires via npm install @openfn/language-http --prefix priv/openfn/runtime/node_modules --no-save --no-package-lock --global-style
  • Démarrez votre serveur de microservice avec env $(cat .env | grep -v "#" | xargs ) iex -S mix phx.server

Fonctionnement dans Docker​

  • Construire une image de docker avec docker build -t openfn/microservice:v0.3.0 .
  • Exécutez avec docker run command

Configuration de projet​

You can configure the jobs, triggers, credentials and language packs used in your microservice in the project.yaml config file.

Première configuration en utilisant l'exemple de configuration​

The sample project configuration file describes an example project setup to help you get acquainted with this structure.

By default microservice is configured with 4 sample jobs:

  1. job-1 est déclenché lorsqu'un message correspondant arrive à l'inbox (voir trigger-1).
  2. recurring-job est un job programmé pour s'exécuter toutes les minutes et est liée au trigger cron every-minute.
  3. flow-job et catch-job s'exécutent respectivement après success et failure de job-1.

All of the jobs are configured with the language pack openfn/language-common.

In the default sample configuration a new message posted to localhost:4000/inbox that matches trigger-1 (i.e. the message contains "number":2) is greeted with an asynchronous acknowledgement receipt (HTTP 202 Data accepted and processing has begun) and will trigger job-1 to run.

You can try this out with the following snippet:

curl -X POST -H "Content-Type: application/json" \
 -d '{
  "number":2,
  "surveyId": 37479
}' \
 http://localhost:4000/inbox

Posting a message not matching any of the triggers (e.g. “number”:3) equally prompts an acknowledgement but doesn’t trigger any jobs.

Example message post for this non-match scenario:

curl -X POST -H "Content-Type: application/json" \
 -d '{
  "number":3,
  "surveyId": 37479
}' \
 http://localhost:4000/inbox

HTTP post requests made to localhost:4000/inbox will be processed by the Receiver, according to the credential, expression, and adaptor defined in the project configuration YAML file.

Time-based jobs will be run by Engine according to the credential, expression, and adaptor defined in your project.yaml file.

Configuration depuis votre projet de plateforme OpenFn existant​

If you have a project configured on OpenFn, you have two ways for exporting your config on the Project Settings page and running your project in microservice.

  1. Si vous exportez en tant que project.yaml, vous pouvez télécharger vos paramètres au format yaml à partir de la page de téléchargement de votre projet de plateforme ou à partir d'un lien dans l'e-mail généré automatiquement envoyé à votre adresse. Vous pouvez insérer ce fichier dans votre environnement tel que configuré en utilisant le Development Up and Running Guide.

  2. Si vous exportez en tant que microservice.zip, vous obtiendrez votre dossier microservice prêt à fonctionner avec docker, contenant

  • un fichier de configuration docker-compose.yaml
  • un dossier du projet contenant project.yaml
  • Fichier .env avec les variables d'environnement par défaut pour docker
  • un fichier Readme

cd into the folder and run the project with docker-compose up. If you don't have the docker image, it will be auto-pulled from hub.docker.com.

Export Microservice Zip