common@1.15.1
- arrayToString(arr, separator)
- asData(data, state)
- chunk(array, chunkSize)
- combine(operations)
- composeNextState(state, response)
- cursor(value, options)
- dataPath(path)
- dataValue(path)
- each(dataSource, operation)
- expandReferences(value, [skipFilter])
- field(key, value)
- fields(fields)
- fn(func)
- fnIf(condition, operation)
- group(arrayOfObjects, keyPath, callback)
- humanProper(str)
- index()
- join(targetPath, sourcePath, targetKey)
- jsonValue(obj, path)
- lastReferenceValue(path)
- map(path, operation, state)
- merge(dataSource, fields)
- parseCsv(csvData, [parsingOptions], [callback])
- referencePath(path)
- scrubEmojis(text, replacementChars)
- source(path)
- sourceValue(path)
- splitKeys(obj, keys)
- toArray(arg)
This adaptor exports the following namespaced functions:
- http.del(requestParams)
- http.expandRequestReferences(value)
- http.get(requestParams)
- http.head(requestParams)
- http.options(requestParams)
- http.patch(requestParams)
- http.post(requestParams)
- http.put(requestParams)
- beta.each(dataSource, operation)
- dateFns.format()
- dateFns.parse()
Functions
arrayToString
arrayToString(arr, separator) ⇒ string
Turns an array into a string, separated by X.
Param | Type | Description |
---|---|---|
arr | array | Array of toString'able primatives. |
separator | string | Separator string. |
Example
field("destination_string__c", function(state) {
return arrayToString(dataValue("path_of_array")(state), ', ')
})
asData
asData(data, state) ⇒ array
Simple switcher allowing other expressions to use either a JSONPath or object literals as a data source.
- JSONPath referencing a point in
state
- Object Literal of the data itself.
- Function to be called with state.
Param | Type | Description |
---|---|---|
data | String | object | function | |
state | object | The current state. |
Example
asData('$.key'| key | callback)
chunk
chunk(array, chunkSize) ⇒ Object
Chunks an array into an array of arrays, each with no more than a certain size.
Param | Type | Description |
---|---|---|
array | Object | Array to be chunked |
chunkSize | Integer | The maxiumum size of each chunks |
Example
chunk([1,2,3,4,5], 2)
combine
combine(operations) ⇒ Operation
Combines two operations into one
Param | Type | Description |
---|---|---|
operations | Operations | Operations to be performed. |
Example
combine(
create('foo'),
delete('bar')
)
composeNextState
composeNextState(state, response) ⇒ State
Prepares next state
Param | Type | Description |
---|---|---|
state | State | state |
response | Object | Response to be added |
Example
composeNextState(state, response)
cursor
cursor(value, options) ⇒ Operation
Sets a cursor property on state.
Supports natural language dates like now
, today
, yesterday
, n hours ago
, n days ago
, and start
,
which will be converted relative to the environment (ie, the Lightning or CLI locale). Custom timezones
are not yet supported.
You can provide a formatter to customise the final cursor value, which is useful for normalising
different inputs. The custom formatter runs after natural language date conversion.
See the usage guide at https://docs.openfn.org/documentation/jobs/job-writing-guide#using-cursors
Param | Type | Description |
---|---|---|
value | any | the cursor value. Usually an ISO date, natural language date, or page number |
options | object | options to control the cursor. |
options.key | string | set the cursor key. Will persist through the whole run. |
options.defaultValue | any | the value to use if value is falsy |
options.format | function | custom formatter for the final cursor value |
Example: Use a cursor from state if present, or else use the default value
cursor($.cursor, { defaultValue: 'today' })
Example: Use a pagination cursor
cursor(22)
dataPath
dataPath(path) ⇒ string
Ensures a path points at the data.
Param | Type | Description |
---|---|---|
path | string | JSONPath referencing a point in data . |
Example
dataPath('key')
dataValue
dataValue(path) ⇒ Operation
Picks out a single value from the source data object—usually state.data
.
If a JSONPath returns more than one value for the reference, the first
item will be returned.
Param | Type | Description |
---|---|---|
path | String | JSONPath referencing a point in data . |
Example
dataValue('key')
each
each(dataSource, operation) ⇒ Operation
Scopes an array of data based on a JSONPath.
Useful when the source data has n
items you would like to map to
an operation.
The operation will receive a slice of the data based of each item
of the JSONPath provided.
It also ensures the results of an operation make their way back into the state's references.
Param | Type | Description |
---|---|---|
dataSource | DataSource | JSONPath referencing a point in state . |
operation | Operation | The operation needed to be repeated. |
Example
each("$.[*]",
create("SObject",
field("FirstName", sourceValue("$.firstName"))
)
)
expandReferences
expandReferences(value, [skipFilter]) ⇒ Operation
Recursively resolves objects that have resolvable values (functions).
Param | Type | Description |
---|---|---|
value | object | data |
[skipFilter] | function | a function which returns true if a value should be skipped |
field
field(key, value) ⇒ Field
Returns a key, value pair in an array.
Param | Type | Description |
---|---|---|
key | string | Name of the field |
value | Value | The value itself or a sourceable operation. |
Example
field('destination_field_name__c', 'value')
fields
fields(fields) ⇒ Object
Zips key value pairs into an object.
Param | Type | Description |
---|---|---|
fields | Fields | a list of fields |
Example
fields(list_of_fields)
fn
fn(func) ⇒ Operation
Creates a custom step (or operation) for more flexible job writing.
Param | Type | Description |
---|---|---|
func | function | is the function |
Example
fn(state => {
// do some things to state
return state;
});
fnIf
fnIf(condition, operation) ⇒ Operation
A custom operation that will only execute the function if the condition returns true
Param | Type | Description |
---|---|---|
condition | Boolean | The condition that returns true |
operation | Operation | The operation needed to be executed. |
Example
fnIf((state) => state?.data?.name, get("https://example.com"));
group
group(arrayOfObjects, keyPath, callback) ⇒ Operation
Groups an array of objects by a specified key path.
Param | Type | Description |
---|---|---|
arrayOfObjects | Array.<Object> | The array of objects to be grouped. |
keyPath | string | The key path to group by. |
callback | function | (Optional) Callback function |
Example
const users = [
{ name: 'Alice', age: 25, city: 'New York' },
{ name: 'Bob', age: 30, city: 'San Francisco' },
{ name: 'Charlie', age: 25, city: 'New York' },
{ name: 'David', age: 30, city: 'San Francisco' }
];
group(users, 'city');
// state is { data: { 'New York': [/Alice, Charlie/], 'San Francisco': [ /Bob, David / ] }
humanProper
humanProper(str) ⇒ string
Substitutes underscores for spaces and proper-cases a string
Param | Type | Description |
---|---|---|
str | string | String that needs converting |
Example
field("destination_string__c", humanProper(state.data.path_to_string))
index
index() ⇒ DataSource
Returns the index of the current array being iterated.
To be used with each
as a data source.
Example
index()
join
join(targetPath, sourcePath, targetKey) ⇒ Operation
Adds data from a target object
Param | Type | Description |
---|---|---|
targetPath | String | Target path |
sourcePath | String | Source path |
targetKey | String | Target Key |
Example
join('$.key','$.data','newKey')
jsonValue
jsonValue(obj, path) ⇒ Operation
Picks out a single value from a JSON object. If a JSONPath returns more than one value for the reference, the first item will be returned.
Param | Type | Description |
---|---|---|
obj | object | A valid JSON object. |
path | String | JSONPath referencing a point in given JSON object. |
Example
jsonValue({ a:1 }, 'a')
lastReferenceValue
lastReferenceValue(path) ⇒ Operation
Picks out the last reference value from source data.
Param | Type | Description |
---|---|---|
path | String | JSONPath referencing a point in references . |
Example
lastReferenceValue('key')
map
map(path, operation, state) ⇒ State
Scopes an array of data based on a JSONPath.
Useful when the source data has n
items you would like to map to
an operation.
The operation will receive a slice of the data based of each item
of the JSONPath provided.
Param | Type | Description |
---|---|---|
path | string | JSONPath referencing a point in state.data . |
operation | function | The operation needed to be repeated. |
state | State | Runtime state. |
Example
map("$.[*]",
create("SObject",
field("FirstName", sourceValue("$.firstName"))
)
)
merge
merge(dataSource, fields) ⇒ DataSource
Merges fields into each item in an array.
Param | Type | Description |
---|---|---|
dataSource | DataSource | |
fields | Object | Group of fields to merge in. |
Example
merge(
"$.books[*]",
fields(
field( "publisher", sourceValue("$.publisher") )
)
)
parseCsv
parseCsv(csvData, [parsingOptions], [callback]) ⇒ Operation
Takes a CSV file string or stream and parsing options as input, and returns a promise that
resolves to the parsed CSV data as an array of objects.
Options for parsingOptions
include:
delimiter
{string/Buffer/[string/Buffer]} - Defines the character(s) used to delineate the fields inside a record. Default:','
quote
{string/Buffer/[string/Buffer]} - Defines the characters used to surround a field. Default:'"'
escape
{Buffer/string/null/boolean} - Set the escape character as one character/byte only. Default:"
columns
{boolean / array / function} - Generates record in the form of object literals. Default:true
bom
{boolean} - Strips the byte order mark (BOM) from the input string or buffer. Default:true
trim
{boolean} - Ignore whitespace characters immediately around thedelimiter
. Default:true
ltrim
{boolean} - Ignore whitespace characters from the left side of a CSV field. Default:true
rtrim
{boolean} - Ignore whitespace characters from the right side of a CSV field. Default:true
chunkSize
{number} - The size of each chunk of CSV data. Default:Infinity
skip_empty_lines
{boolean} - Ignore empty lines in the CSV file. Default:true
Returns: Operation
- The function returns a Promise that resolves to the result of parsing a CSV stringOrStream
.
Param | Type | Description |
---|---|---|
csvData | String | Stream | A CSV string or a readable stream |
[parsingOptions] | Object | Optional. Parsing options for converting CSV to JSON. |
[callback] | function | (Optional) callback function. If used it will be called state and an array of rows. |
referencePath
referencePath(path) ⇒ string
Ensures a path points at references.
Param | Type | Description |
---|---|---|
path | string | JSONPath referencing a point in references . |
Example
referencePath('key')
scrubEmojis
scrubEmojis(text, replacementChars) ⇒ string
Replaces emojis in a string.
Param | Type | Description |
---|---|---|
text | string | String that needs to be cleaned |
replacementChars | string | Characters that replace the emojis |
Example
scrubEmojis('Dove🕊️⭐ 29')
source
source(path) ⇒ Array.<(String|Object)>
Picks out a value from source data.
Will return whatever JSONPath returns, which will always be an array.
If you need a single value use sourceValue
instead.
Param | Type | Description |
---|---|---|
path | String | JSONPath referencing a point in state . |
Example
source('$.key')
sourceValue
sourceValue(path) ⇒ Operation
Picks out a single value from source data. If a JSONPath returns more than one value for the reference, the first item will be returned.
Param | Type | Description |
---|---|---|
path | String | JSONPath referencing a point in state . |
Example
sourceValue('$.key')
splitKeys
splitKeys(obj, keys) ⇒ Array.<Object>
Splits an object into two objects based on a list of keys. The first object contains the keys that are not in the list, and the second contains the keys that are.
Returns: Array.<Object>
- - Tuple of objects, first object contains keys not in list, second contains keys that are.
Param | Type | Description |
---|---|---|
obj | Object | The object to split. |
keys | Array.<string> | List of keys to split on. |
toArray
toArray(arg) ⇒ array
Ensures primitive data types are wrapped in an array. Does not affect array objects.
Param | Type | Description |
---|---|---|
arg | any | Data required to be in an array |
Example
each(function(state) {
return toArray( dataValue("path_of_array")(state) )
}, ...)
http
These functions belong to the http namespace.
http.del
del(requestParams) ⇒ Operation
Make a DELETE request
Returns: Operation
- - Function which takes state and returns a Promise
Param | Type | Description |
---|---|---|
requestParams | object | Supports the exact parameters as Axios. See here |
Example: Deleting a record with data that comes from state
http.delete({
url: state => `https://www.example.com/api/items/${state.id}`,
})(state);
http.expandRequestReferences
expandRequestReferences(value) ⇒ Operation
Recursively resolves objects that have resolvable values (functions), but
omits HTTP request specific modules like FormData
.
Param | Type | Description |
---|---|---|
value | object | data |
http.get
get(requestParams) ⇒ Operation
Make a GET request
Returns: Operation
- - Function which takes state and returns a Promise
Param | Type | Description |
---|---|---|
requestParams | object | Supports the exact parameters as Axios. See here |
Example: Get an item with a specified id from state
http.get({
url: state => `https://www.example.com/api/items/${state.id},
headers: {"content-type": "application/json"}
});
http.head
head(requestParams) ⇒ Operation
Make a HEAD request
Returns: Operation
- - Function which takes state and returns a Promise
Param | Type | Description |
---|---|---|
requestParams | object | Supports the exact parameters as Axios. See here |
Example: Gets the headers that would be returned if the HEAD request's URL was instead requested with the HTTP GET method
http.head({
url: 'https://www.example.com/api/items',
});
http.options
options(requestParams) ⇒ Operation
Make a OPTIONS request
Returns: Operation
- - Function which takes state and returns a Promise
Param | Type | Description |
---|---|---|
requestParams | object | Supports the exact parameters as Axios. See here |
Example: Requests permitted communication options for a given URL or server, with data from state.
http.options({
url: 'https://www.example.com/api/items',
});
http.patch
patch(requestParams) ⇒ Operation
Make a PATCH request
Returns: Operation
- - Function which takes state and returns a Promise
Param | Type | Description |
---|---|---|
requestParams | object | Supports the exact parameters as Axios. See here |
Example: Applies partial modifications to a resource, with data from state.
http.patch({
url: state => `https://www.example.com/api/items/${state.id}`,
data: state => state.data
});
http.post
post(requestParams) ⇒ Operation
Make a POST request
Returns: Operation
- - Function which takes state and returns a Promise
Param | Type | Description |
---|---|---|
requestParams | object | Supports the exact parameters as Axios. See here |
Example: Sending a payload with data that comes from state
http.post({
url: "https://example.com",
data: (state) => state.data
});
Example: Capturing the response for later use in state
fn((state) => {
return http.post({
url: "https://example.com",
data: (state) => state.data
})(state).then(({response}) => {
state.responseData = response.data
})
});
http.put
put(requestParams) ⇒ Operation
Make a PUT request
Returns: Operation
- - Function which takes state and returns a Promise
Param | Type | Description |
---|---|---|
requestParams | object | Supports the exact parameters as Axios. See here |
Example: Creates a new resource or replaces a representation of the target resource with the request payload, with data from state.
http.put({
url: state => `https://www.example.com/api/items/${state.id}`,
data: state => state.data
});
beta
These functions belong to the beta namespace.
beta.each
each(dataSource, operation) ⇒ Operation
Scopes an array of data based on a JSONPath.
Useful when the source data has n
items you would like to map to
an operation.
The operation will receive a slice of the data based of each item
of the JSONPath provided.
It also ensures the results of an operation make their way back into the state's references.
Param | Type | Description |
---|---|---|
dataSource | DataSource | JSONPath referencing a point in state . |
operation | Operation | The operation needed to be repeated. |
Example
each("$.[*]",
create("SObject",
field("FirstName", sourceValue("$.firstName")))
)
dateFns
These functions belong to the dateFns namespace.
dateFns.format
format()
The format function from the date-fns library. See https://date-fns.org/v3.6.0/docs/parse
dateFns.parse
parse()
The parse function from the date-fns library. See https://date-fns.org/v3.6.0/docs/parse