common@2.1.1

arrayToString(arr, separator)
asData(data, state)
chunk(array, chunkSize)
combine(operations)
composeNextState(state, response)
cursor(value, options)
dataPath(path)
dataValue(path)
each(dataSource, operation)
expandReferences(value, [skipFilter])
field(key, value)
fields(fields)
fn(func)
fnIf(condition, operation)
group(arrayOfObjects, keyPath, callback)
humanProper(str)
index()
join(targetPath, sourcePath, targetKey)
jsonValue(obj, path)
lastReferenceValue(path)
map(path, operation, state)
merge(dataSource, fields)
parseCsv(csvData, [parsingOptions], [callback])
referencePath(path)
scrubEmojis(text, replacementChars)
source(path)
sourceValue(path)
splitKeys(obj, keys)
toArray(arg)

This adaptor exports the following namespaced functions:

helpers.decode(base64Data)
helpers.encode(data)
helpers.uuid()
beta.each(dataSource, operation)
dateFns.format()
dateFns.parse()
http.get(url, options)
http.options(options)
http.post(url, options)
http.request(method, url, options)

Functions

arrayToString

arrayToString(arr, separator) ⇒ string

Turns an array into a string, separated by X.

Param	Type	Description
arr	`array`	Array of toString'able primatives.
separator	`string`	Separator string.

Example

field("destination_string__c", function(state) {
  return arrayToString(dataValue("path_of_array")(state), ', ')
})

asData

asData(data, state) ⇒ array

Simple switcher allowing other expressions to use either a JSONPath or object literals as a data source.

JSONPath referencing a point in state
Object Literal of the data itself.
Function to be called with state.

Param	Type	Description
data	`String` \| `object` \| `function`
state	`object`	The current state.

Example

asData('$.key'| key | callback)

chunk

chunk(array, chunkSize) ⇒ Object

Chunks an array into an array of arrays, each with no more than a certain size.

Param	Type	Description
array	`Object`	Array to be chunked
chunkSize	`Integer`	The maxiumum size of each chunks

Example

chunk([1,2,3,4,5], 2)

combine

combine(operations) ⇒ Operation

Combines two operations into one

Param	Type	Description
operations	`Operations`	Operations to be performed.

Example

combine(
  create('foo'),
  delete('bar')
)

composeNextState

composeNextState(state, response) ⇒ State

Prepares next state

Param	Type	Description
state	`State`	state
response	`Object`	Response to be added

Example

composeNextState(state, response)

cursor

cursor(value, options) ⇒ Operation

Sets a cursor property on state. Supports natural language dates like now, today, yesterday, n hours ago, n days ago, and start, which will be converted relative to the environment (ie, the Lightning or CLI locale). Custom timezones are not yet supported. You can provide a formatter to customise the final cursor value, which is useful for normalising different inputs. The custom formatter runs after natural language date conversion. See the usage guide at https://docs.openfn.org/documentation/jobs/job-writing-guide#using-cursors

Param	Type	Description
value	`any`	the cursor value. Usually an ISO date, natural language date, or page number
options	`object`	options to control the cursor.
options.key	`string`	set the cursor key. Will persist through the whole run.
options.defaultValue	`any`	the value to use if value is falsy
options.format	`function`	custom formatter for the final cursor value

Example: Use a cursor from state if present, or else use the default value

cursor($.cursor, { defaultValue: 'today' })

Example: Use a pagination cursor

cursor(22)

dataPath

dataPath(path) ⇒ string

Ensures a path points at the data.

Param	Type	Description
path	`string`	JSONPath referencing a point in `data`.

Example

dataPath('key')

dataValue

dataValue(path) ⇒ Operation

Picks out a single value from the source data object—usually state.data. If a JSONPath returns more than one value for the reference, the first item will be returned.

Param	Type	Description
path	`String`	JSONPath referencing a point in `data`.

Example

dataValue('key')

each

each(dataSource, operation) ⇒ Operation

Iterates over an array of items and invokes an operation upon each one, where the state object is scoped so that state.data is the item under iteration. The rest of the state object is untouched and can be referenced as usual. You can pass an array directly, or use lazy state or a JSONPath string to reference a slice of state.

Param	Type	Description
dataSource	`DataSource`	JSONPath referencing a point in `state`.
operation	`Operation`	The operation needed to be repeated.

Example: Using lazy state ($) to iterate over items in state.data and pass each into an "insert" operation

each(
  $.data,
  // Inside the callback operation, `$.data` is scoped to the item under iteration
  insert("patient", {
    patient_name: $.data.properties.case_name,
    patient_id: $.data.case_id,
  })
);

Example: Iterate over items in state.data and pass each one into an "insert" operation

each(
  $.data,
  insert("patient", (state) => ({
    patient_id: state.data.case_id,
    ...state.data
  }))
);

Example: Using JSON path to iterate over items in state.data and pass each one into an "insert" operation

each(
  "$.data[*]",
  insert("patient", (state) => ({
    patient_name: state.data.properties.case_name,
    patient_id: state.data.case_id,
  }))
);

expandReferences

expandReferences(value, [skipFilter]) ⇒ Operation

Recursively resolves objects that have resolvable values (functions).

Param	Type	Description
value	`object`	data
[skipFilter]	`function`	a function which returns true if a value should be skipped

field

field(key, value) ⇒ Field

Returns a key, value pair in an array.

Param	Type	Description
key	`string`	Name of the field
value	`Value`	The value itself or a sourceable operation.

Example

field('destination_field_name__c', 'value')

fields

fields(fields) ⇒ Object

Zips key value pairs into an object.

Param	Type	Description
fields	`Fields`	a list of fields

Example

fields(list_of_fields)

fn

fn(func) ⇒ Operation

Creates a custom step (or operation) for more flexible job writing.

Param	Type	Description
func	`function`	is the function

Example

fn(state => {
  // do some things to state
  return state;
});

fnIf

fnIf(condition, operation) ⇒ Operation

A custom operation that will only execute the function if the condition returns true

Param	Type	Description
condition	`Boolean`	The condition that returns true
operation	`Operation`	The operation needed to be executed.

Example

fnIf((state) => state?.data?.name, get("https://example.com"));

group

group(arrayOfObjects, keyPath, callback) ⇒ Operation

Groups an array of objects by a specified key path.

Param	Type	Description
arrayOfObjects	`Array.<Object>`	The array of objects to be grouped.
keyPath	`string`	The key path to group by.
callback	`function`	(Optional) Callback function

Example

const users = [
  { name: 'Alice', age: 25, city: 'New York' },
  { name: 'Bob', age: 30, city: 'San Francisco' },
  { name: 'Charlie', age: 25, city: 'New York' },
  { name: 'David', age: 30, city: 'San Francisco' }
];
group(users, 'city');
// state is { data: { 'New York': [/Alice, Charlie/], 'San Francisco': [ /Bob, David / ] }

humanProper

humanProper(str) ⇒ string

Substitutes underscores for spaces and proper-cases a string

Param	Type	Description
str	`string`	String that needs converting

Example

field("destination_string__c", humanProper(state.data.path_to_string))

index

index() ⇒ DataSource

Returns the index of the current array being iterated. To be used with each as a data source.

Example

index()

join

join(targetPath, sourcePath, targetKey) ⇒ Operation

Adds data from a target object

Param	Type	Description
targetPath	`String`	Target path
sourcePath	`String`	Source path
targetKey	`String`	Target Key

Example

join('$.key','$.data','newKey')

jsonValue

jsonValue(obj, path) ⇒ Operation

Picks out a single value from a JSON object. If a JSONPath returns more than one value for the reference, the first item will be returned.

Param	Type	Description
obj	`object`	A valid JSON object.
path	`String`	JSONPath referencing a point in given JSON object.

Example

jsonValue({ a:1 }, 'a')

lastReferenceValue

lastReferenceValue(path) ⇒ Operation

Picks out the last reference value from source data.

Param	Type	Description
path	`String`	JSONPath referencing a point in `references`.

Example

lastReferenceValue('key')

map

map(path, operation, state) ⇒ State

Scopes an array of data based on a JSONPath. Useful when the source data has n items you would like to map to an operation. The operation will receive a slice of the data based of each item of the JSONPath provided.

Param	Type	Description
path	`string`	JSONPath referencing a point in `state.data`.
operation	`function`	The operation needed to be repeated.
state	`State`	Runtime state.

Example

map("$.[*]",
  create("SObject",
    field("FirstName", sourceValue("$.firstName"))
  )
)

merge

merge(dataSource, fields) ⇒ DataSource

Merges fields into each item in an array.

Param	Type	Description
dataSource	`DataSource`
fields	`Object`	Group of fields to merge in.

Example

merge(
  "$.books[*]",
  fields(
    field( "publisher", sourceValue("$.publisher") )
  )
)

parseCsv

parseCsv(csvData, [parsingOptions], [callback]) ⇒ Operation

Takes a CSV file string or stream and parsing options as input, and returns a promise that resolves to the parsed CSV data as an array of objects. Options for parsingOptions include:

delimiter {string/Buffer/[string/Buffer]} - Defines the character(s) used to delineate the fields inside a record. Default: ','
quote {string/Buffer/[string/Buffer]} - Defines the characters used to surround a field. Default: '"'
escape {Buffer/string/null/boolean} - Set the escape character as one character/byte only. Default: "
columns {boolean / array / function} - Generates record in the form of object literals. Default: true
bom {boolean} - Strips the byte order mark (BOM) from the input string or buffer. Default: true
trim {boolean} - Ignore whitespace characters immediately around the delimiter. Default: true
ltrim {boolean} - Ignore whitespace characters from the left side of a CSV field. Default: true
rtrim {boolean} - Ignore whitespace characters from the right side of a CSV field. Default: true
chunkSize {number} - The size of each chunk of CSV data. Default: Infinity
skip_empty_lines {boolean} - Ignore empty lines in the CSV file. Default: true

Returns: Operation - The function returns a Promise that resolves to the result of parsing a CSV stringOrStream.

Param	Type	Description
csvData	`String` \| `Stream`	A CSV string or a readable stream
[parsingOptions]	`Object`	Optional. Parsing options for converting CSV to JSON.
[callback]	`function`	(Optional) callback function. If used it will be called state and an array of rows.

referencePath

referencePath(path) ⇒ string

Ensures a path points at references.

Param	Type	Description
path	`string`	JSONPath referencing a point in `references`.

Example

referencePath('key')

scrubEmojis

scrubEmojis(text, replacementChars) ⇒ string

Replaces emojis in a string.

Param	Type	Description
text	`string`	String that needs to be cleaned
replacementChars	`string`	Characters that replace the emojis

Example

scrubEmojis('Dove🕊️⭐ 29')

source

source(path) ⇒ Array.<(String|Object)>

Picks out a value from source data. Will return whatever JSONPath returns, which will always be an array. If you need a single value use sourceValue instead.

Param	Type	Description
path	`String`	JSONPath referencing a point in `state`.

Example

source('$.key')

sourceValue

sourceValue(path) ⇒ Operation

Picks out a single value from source data. If a JSONPath returns more than one value for the reference, the first item will be returned.

Param	Type	Description
path	`String`	JSONPath referencing a point in `state`.

Example

sourceValue('$.key')

splitKeys

splitKeys(obj, keys) ⇒ Array.<Object>

Splits an object into two objects based on a list of keys. The first object contains the keys that are not in the list, and the second contains the keys that are.

Returns: Array.<Object> - - Tuple of objects, first object contains keys not in list, second contains keys that are.

Param	Type	Description
obj	`Object`	The object to split.
keys	`Array.<string>`	List of keys to split on.

toArray

toArray(arg) ⇒ array

Ensures primitive data types are wrapped in an array. Does not affect array objects.

Param	Type	Description
arg	`any`	Data required to be in an array

Example

each(function(state) {
  return toArray( dataValue("path_of_array")(state) )
}, ...)

helpers

These functions belong to the helpers namespace.

helpers.decode

decode(base64Data) ⇒ string

Decodes a Base64 encoded string back to its original format.

Returns: string - - The decoded string.

Param	Type	Description
base64Data	`string`	The Base64 encoded string.

Example: Decode a Base64 string

const decoded = decode('SGVsbG8gV29ybGQ=');
console.log(decoded); // Output: Hello World

helpers.encode

encode(data) ⇒ string

Encodes a given string into Base64 format.

Returns: string - - The Base64 encoded string.

Param	Type	Description
data	`string`	The string to be encoded.

Example: Encode a string

const encoded = encode('Hello World');
console.log(encoded); // Output: SGVsbG8gV29ybGQ=

helpers.uuid

uuid() ⇒ string

Generates a UUID (Universally Unique Identifier).

Returns: string - - A newly generated UUID.
Example: Generate a UUID

const id = uuid();
console.log(id); // Output:'3f4e254e-8f6f-4f8b-9651-1c1c262cc83f'

beta

These functions belong to the beta namespace.

beta.each

each(dataSource, operation) ⇒ Operation

It also ensures the results of an operation make their way back into the state's references.

Param	Type	Description
dataSource	`DataSource`	JSONPath referencing a point in `state`.
operation	`Operation`	The operation needed to be repeated.

Example

each("$.[*]",
   create("SObject",
   field("FirstName", sourceValue("$.firstName")))
 )

dateFns

These functions belong to the dateFns namespace.

dateFns.format

format()

The format function from the date-fns library. See https://date-fns.org/v3.6.0/docs/parse

dateFns.parse

parse()

The parse function from the date-fns library. See https://date-fns.org/v3.6.0/docs/parse

http

These functions belong to the http namespace.

http.get

get(url, options) ⇒ Operation

Make a GET request.

Param	Type	Description
url	`string`	URL to access
options	`CommonRequestOptions`	Request options

This operation writes the following keys to state:

State Key	Description
data	the parsed response body
response	the response from the HTTP server, including headers, statusCode, body, etc
references	an array of all previous data objects used in the Job

Example: Request a resource

http.get('https://jsonplaceholder.typicode.com/todos')

Example: Request a resource with basic auth

http.get(
 'https://jsonplaceholder.typicode.com/todos',
 http.options().basic('user', 'pass')
)

Example: Request a resource with oauth

http.get(
 'https://jsonplaceholder.typicode.com/todos',
 http.options().oauth($.configuration.access_token)
)

http.options

options(options) ⇒ OptionsHelpers

Builder function to create request options. Returns an object with helpers to easily add commonly used options. The return object is chainable so you can set as many options as you want. Pass an object to set your own options.

Param	Type	Description
options	`CommonRequestOptions`	options to pass to the request

Example: Get with a query an oath token

get($.data.url, http.options({ query: $.query }).oath($.configuration.access_token)

http.post

post(url, options) ⇒ Operation

Make a POST request.

Param	Type	Description
url	`string`	URL to access
options	`CommonRequestOptions`	Request options

This operation writes the following keys to state:

State Key	Description
data	the parsed response body
response	the response from the HTTP server, including headers, statusCode, body, etc
references	an array of all previous data objects used in the Job

Example: Post a JSON object (setting the content-type header)

 http.post(
   'https://jsonplaceholder.typicode.com/todos',
   $.data,
   options().json(),
 })

http.request

request(method, url, options) ⇒ Operation

Make a HTTP request.

Param	Type	Description
method	`string`	The HTTP method to use.
url	`string`	URL to resource.
options	`CommonRequestOptions`	Request options

This operation writes the following keys to state:

State Key	Description
data	the parsed response body
response	the response from the HTTP server, including headers, statusCode, body, etc
references	an array of all previous data objects used in the Job

Example

http.request(
  'GET',
  'https://jsonplaceholder.typicode.com/todos'
)

Interfaces

OptionsHelpers

Helper functions provided by http.options.

Properties

Name	Type	Description
json	`function`	Sets the `content-type' header to 'application/json'
basic	`function`	Sets basic auth on the Authorization header. Pass username and password
bearer	`function`	Sets a Bearer token on the Authorization header. Pass the token.
oauth	`function`	Sets a Bearer token on the Authorization header. Pass the oauth token.

Functions​

arrayToString​

asData​

chunk​

combine​

composeNextState​

cursor​

dataPath​

dataValue​

each​

expandReferences​

field​

fields​

fn​

fnIf​

group​

humanProper​

index​

join​

jsonValue​

lastReferenceValue​

map​

merge​

parseCsv​

referencePath​

scrubEmojis​

source​

sourceValue​

splitKeys​

toArray​

helpers​

helpers.decode​

helpers.encode​

helpers.uuid​

beta​

beta.each​

dateFns​

dateFns.format​

dateFns.parse​

http​

http.get​

http.options​

http.post​

http.request​

Interfaces​

OptionsHelpers​

Functions

arrayToString

asData

chunk

combine

composeNextState

cursor

dataPath

dataValue

each

expandReferences

field

fields

fn

fnIf

group

humanProper

index

join

jsonValue

lastReferenceValue

map

merge

parseCsv

referencePath

scrubEmojis

source

sourceValue

splitKeys

toArray

helpers

helpers.decode

helpers.encode

helpers.uuid

beta

beta.each

dateFns

dateFns.format

dateFns.parse

http

http.get

http.options

http.post

http.request

Interfaces

OptionsHelpers