common@1.13.3

Functions

alterState(func)
arrayToString(arr, separator)
asData(data, state)
chunk(array, chunkSize)
combine(operations)
composeNextState(state, response)
cursor(value, options)
dataPath(path)
dataValue(path)
del(requestParams)
each(dataSource, operation)
each(dataSource, operation)
expandReferences(value, [skipFilter])
expandRequestReferences(value)
field(key, value)
fields(fields)
fn(func)
get(requestParams)
head(requestParams)
humanProper(str)
index()
join(targetPath, sourcePath, targetKey)
jsonValue(obj, path)
lastReferenceValue(path)
map(path, operation, state)
merge(dataSource, fields)
options(requestParams)
parseCsv(csvData, [parsingOptions], [callback])
patch(requestParams)
post(requestParams)
put(requestParams)
referencePath(path)
request(method, fullUrlOrPath, [options])
scrubEmojis(text, replacementChars)
source(path)
sourceValue(path)
splitKeys(obj, keys)
toArray(arg)
validate(schema, data)
withAgent(params)

alterState

alterState(func) ⇒ Operation

alias for "fn()"

Param	Type	Description
func	`function`	is the function

arrayToString

arrayToString(arr, separator) ⇒ string

Turns an array into a string, separated by X.

Param	Type	Description
arr	`array`	Array of toString'able primatives.
separator	`string`	Separator string.

Example

field("destination_string__c", function(state) {
  return arrayToString(dataValue("path_of_array")(state), ', ')
})

asData

asData(data, state) ⇒ array

Simple switcher allowing other expressions to use either a JSONPath or object literals as a data source.

JSONPath referencing a point in state
Object Literal of the data itself.
Function to be called with state.

Param	Type	Description
data	`String` \| `object` \| `function`
state	`object`	The current state.

Example

asData('$.key'| key | callback)

chunk

chunk(array, chunkSize) ⇒ Object

Chunks an array into an array of arrays, each with no more than a certain size.

Param	Type	Description
array	`Object`	Array to be chunked
chunkSize	`Integer`	The maxiumum size of each chunks

Example

chunk([1,2,3,4,5], 2)

combine

combine(operations) ⇒ Operation

Combines two operations into one

Param	Type	Description
operations	`Operations`	Operations to be performed.

Example

combine(
  create('foo'),
  delete('bar')
)

composeNextState

composeNextState(state, response) ⇒ State

Prepares next state

Param	Type	Description
state	`State`	state
response	`Object`	Response to be added

Example

composeNextState(state, response)

cursor

cursor(value, options) ⇒ Operation

Sets a cursor property on state. Supports natural language dates like now, today, yesterday, n hours ago, n days ago, and start, which will be converted relative to the environment (ie, the Lightning or CLI locale). Custom timezones are not yet supported. You can provide a formatter to customise the final cursor value, which is useful for normalising different inputs. The custom formatter runs after natural language date conversion. See the usage guide at https://docs.openfn.org/documentation/jobs/job-writing-guide#using-cursors

Param	Type	Description
value	`any`	the cursor value. Usually an ISO date, natural language date, or page number
options	`object`	options to control the cursor.
options.key	`string`	set the cursor key. Will persist through the whole run.
options.defaultValue	`any`	the value to use if value is falsy
options.format	`function`	custom formatter for the final cursor value

Example (Use a cursor from state if present, or else use the default value)

cursor($.cursor, { defaultValue: 'today' })

Example (Use a pagination cursor)

cursor(22)

dataPath

dataPath(path) ⇒ string

Ensures a path points at the data.

Param	Type	Description
path	`string`	JSONPath referencing a point in `data`.

Example

dataPath('key')

dataValue

dataValue(path) ⇒ Operation

Picks out a single value from the source data object—usually state.data. If a JSONPath returns more than one value for the reference, the first item will be returned.

Param	Type	Description
path	`String`	JSONPath referencing a point in `data`.

Example

dataValue('key')

del

del(requestParams) ⇒ Operation

Make a DELETE request

Returns: Operation - - Function which takes state and returns a Promise

Param	Type	Description
requestParams	`object`	Supports the exact parameters as Axios. See here

Example (Deleting a record with data that comes from state)

delete({
   url: state => `https://www.example.com/api/items/${state.id}`,
 })(state);

each

each(dataSource, operation) ⇒ Operation

Scopes an array of data based on a JSONPath. Useful when the source data has n items you would like to map to an operation. The operation will receive a slice of the data based of each item of the JSONPath provided.

It also ensures the results of an operation make their way back into the state's references.

Param	Type	Description
dataSource	`DataSource`	JSONPath referencing a point in `state`.
operation	`Operation`	The operation needed to be repeated.

Example

each("$.[*]",
  create("SObject",
    field("FirstName", sourceValue("$.firstName"))
  )
)

each

each(dataSource, operation) ⇒ Operation

It also ensures the results of an operation make their way back into the state's references.

Param	Type	Description
dataSource	`DataSource`	JSONPath referencing a point in `state`.
operation	`Operation`	The operation needed to be repeated.

Example

each("$.[*]",
   create("SObject",
   field("FirstName", sourceValue("$.firstName")))
 )

expandReferences

expandReferences(value, [skipFilter]) ⇒ Operation

Recursively resolves objects that have resolvable values (functions).

Param	Type	Description
value	`object`	data
[skipFilter]	`function`	a function which returns true if a value should be skipped

expandRequestReferences

expandRequestReferences(value) ⇒ Operation

Recursively resolves objects that have resolvable values (functions), but omits HTTP request specific modules like FormData.

Param	Type	Description
value	`object`	data

field

field(key, value) ⇒ Field

Returns a key, value pair in an array.

Param	Type	Description
key	`string`	Name of the field
value	`Value`	The value itself or a sourceable operation.

Example

field('destination_field_name__c', 'value')

fields

fields(fields) ⇒ Object

Zips key value pairs into an object.

Param	Type	Description
fields	`Fields`	a list of fields

Example

fields(list_of_fields)

fn

fn(func) ⇒ Operation

Creates a custom step (or operation) for more flexible job writing.

Param	Type	Description
func	`function`	is the function

Example

fn(state => {
  // do some things to state
  return state;
});

get

get(requestParams) ⇒ Operation

Make a GET request

Returns: Operation - - Function which takes state and returns a Promise

Param	Type	Description
requestParams	`object`	Supports the exact parameters as Axios. See here

Example (Get an item with a specified id from state)

 get({
     url: state => `https://www.example.com/api/items/${state.id},
     headers: {"content-type": "application/json"}
});

head

head(requestParams) ⇒ Operation

Make a HEAD request

Returns: Operation - - Function which takes state and returns a Promise

Param	Type	Description
requestParams	`object`	Supports the exact parameters as Axios. See here

Example (Gets the headers that would be returned if the HEAD request's URL was instead requested with the HTTP GET method)

head({
  url: 'https://www.example.com/api/items',
});

humanProper

humanProper(str) ⇒ string

Substitutes underscores for spaces and proper-cases a string

Param	Type	Description
str	`string`	String that needs converting

Example

field("destination_string__c", humanProper(state.data.path_to_string))

index

index() ⇒ DataSource

Returns the index of the current array being iterated. To be used with each as a data source.

Example

index()

join

join(targetPath, sourcePath, targetKey) ⇒ Operation

Adds data from a target object

Param	Type	Description
targetPath	`String`	Target path
sourcePath	`String`	Source path
targetKey	`String`	Target Key

Example

join('$.key','$.data','newKey')

jsonValue

jsonValue(obj, path) ⇒ Operation

Picks out a single value from a JSON object. If a JSONPath returns more than one value for the reference, the first item will be returned.

Param	Type	Description
obj	`object`	A valid JSON object.
path	`String`	JSONPath referencing a point in given JSON object.

Example

jsonValue({ a:1 }, 'a')

lastReferenceValue

lastReferenceValue(path) ⇒ Operation

Picks out the last reference value from source data.

Param	Type	Description
path	`String`	JSONPath referencing a point in `references`.

Example

lastReferenceValue('key')

map

map(path, operation, state) ⇒ State

Param	Type	Description
path	`string`	JSONPath referencing a point in `state.data`.
operation	`function`	The operation needed to be repeated.
state	`State`	Runtime state.

Example

map("$.[*]",
  create("SObject",
    field("FirstName", sourceValue("$.firstName"))
  )
)

merge

merge(dataSource, fields) ⇒ DataSource

Merges fields into each item in an array.

Param	Type	Description
dataSource	`DataSource`
fields	`Object`	Group of fields to merge in.

Example

merge(
  "$.books[*]",
  fields(
    field( "publisher", sourceValue("$.publisher") )
  )
)

options

options(requestParams) ⇒ Operation

Make a OPTIONS request

Returns: Operation - - Function which takes state and returns a Promise

Param	Type	Description
requestParams	`object`	Supports the exact parameters as Axios. See here

Example (Requests permitted communication options for a given URL or server, with data from state.)

options({
  url: 'https://www.example.com/api/items',
});

parseCsv

parseCsv(csvData, [parsingOptions], [callback]) ⇒ Operation

Takes a CSV file string or stream and parsing options as input, and returns a promise that resolves to the parsed CSV data as an array of objects. Options for parsingOptions include:

delimiter {string/Buffer/[string/Buffer]} - Defines the character(s) used to delineate the fields inside a record. Default: ','
quote {string/Buffer/[string/Buffer]} - Defines the characters used to surround a field. Default: '"'
escape {Buffer/string/null/boolean} - Set the escape character as one character/byte only. Default: "
columns {boolean / array / function} - Generates record in the form of object literals. Default: true
bom {boolean} - Strips the byte order mark (BOM) from the input string or buffer. Default: true
trim {boolean} - Ignore whitespace characters immediately around the delimiter. Default: true
ltrim {boolean} - Ignore whitespace characters from the left side of a CSV field. Default: true
rtrim {boolean} - Ignore whitespace characters from the right side of a CSV field. Default: true
chunkSize {number} - The size of each chunk of CSV data. Default: Infinity
skip_empty_lines {boolean} - Ignore empty lines in the CSV file. Default: true

Returns: Operation - The function returns a Promise that resolves to the result of parsing a CSV stringOrStream.

Param	Type	Description
csvData	`String` \| `Stream`	A CSV string or a readable stream
[parsingOptions]	`Object`	Optional. Parsing options for converting CSV to JSON.
[callback]	`function`	(Optional) callback function. If used it will be called state and an array of rows.

patch

patch(requestParams) ⇒ Operation

Make a PATCH request

Returns: Operation - - Function which takes state and returns a Promise

Param	Type	Description
requestParams	`object`	Supports the exact parameters as Axios. See here

Example (Applies partial modifications to a resource, with data from state.)

patch({
  url: state => `https://www.example.com/api/items/${state.id}`,
  data: state => state.data
});

post

post(requestParams) ⇒ Operation

Make a POST request

Returns: Operation - - Function which takes state and returns a Promise

Param	Type	Description
requestParams	`object`	Supports the exact parameters as Axios. See here

Example (Sending a payload with data that comes from state)

post({
  url: "https://example.com",
  data: (state) => state.data
});

Example ( Capturing the response for later use in state )

alterState((state) => {
  return post({
    url: "https://example.com",
    data: (state) => state.data
  })(state).then(({response}) => {
   state.responseData = response.data
  })
});

put

put(requestParams) ⇒ Operation

Make a PUT request

Returns: Operation - - Function which takes state and returns a Promise

Param	Type	Description
requestParams	`object`	Supports the exact parameters as Axios. See here

Example (Creates a new resource or replaces a representation of the target resource with the request payload, with data from state.)

put({
  url: state => `https://www.example.com/api/items/${state.id}`,
  data: state => state.data
});

referencePath

referencePath(path) ⇒ string

Ensures a path points at references.

Param	Type	Description
path	`string`	JSONPath referencing a point in `references`.

Example

referencePath('key')

request

request(method, fullUrlOrPath, [options]) ⇒

request is a helper function that sends HTTP requests and returns the response body, headers, and status code. Use the error map to provide custom error messages or get hold of the response in case of errors.

Returns: an object with the following properties:

method: the request method
url: the request url
code: the status code of the response
headers: the headers of the response
body: the body of the response
message: the status text of the response
duration: the response time

Param	Description
method	The HTTP method to use for the request (e.g., "GET", "POST", "PUT", "DELETE", etc.).
fullUrlOrPath	The full or partial URL for the request.
[options]	The `options` parameter is an object that contains additional configuration options for the request.

scrubEmojis

scrubEmojis(text, replacementChars) ⇒ string

Replaces emojis in a string.

Param	Type	Description
text	`string`	String that needs to be cleaned
replacementChars	`string`	Characters that replace the emojis

Example

scrubEmojis('Dove🕊️⭐ 29')

source

source(path) ⇒ Array.<(String|Object)>

Picks out a value from source data. Will return whatever JSONPath returns, which will always be an array. If you need a single value use sourceValue instead.

Param	Type	Description
path	`String`	JSONPath referencing a point in `state`.

Example

source('$.key')

sourceValue

sourceValue(path) ⇒ Operation

Picks out a single value from source data. If a JSONPath returns more than one value for the reference, the first item will be returned.

Param	Type	Description
path	`String`	JSONPath referencing a point in `state`.

Example

sourceValue('$.key')

splitKeys

splitKeys(obj, keys) ⇒ Array.<Object>

Splits an object into two objects based on a list of keys. The first object contains the keys that are not in the list, and the second contains the keys that are.

Returns: Array.<Object> - - Tuple of objects, first object contains keys not in list, second contains keys that are.

Param	Type	Description
obj	`Object`	The object to split.
keys	`Array.<string>`	List of keys to split on.

toArray

toArray(arg) ⇒ array

Ensures primitive data types are wrapped in an array. Does not affect array objects.

Param	Type	Description
arg	`any`	Data required to be in an array

Example

each(function(state) {
  return toArray( dataValue("path_of_array")(state) )
}, ...)

validate

validate(schema, data) ⇒ Operation

Validate against a JSON schema. Any erors are written to an array at state.validationErrors. Schema can be passed directly, loaded as a JSON path from state, or loaded from a URL Data can be passed directly or loaded as a JSON path from state. By default, schema is loaded from state.schema and data from state.data.

Param	Type	Description
schema	`string` \| `object`	The schema, path or URL to validate against
data	`string` \| `object`	The data or path to validate

Example (Validate `state.data` with `state.schema`)

validate()

Example (Validate form data at `state.form` with a schema from a URL)

validate("https://www.example.com/schema/record", "form")

Example (Validate the each item in `state.records` with a schema from a URL)

each("records[*]", validate("https://www.example.com/schema/record"))

withAgent

withAgent(params) ⇒ Operation

Creates an https agent for axios from the agentOptions key passed in params.

Param	Type	Description
params	`object`	data

Functions​

alterState​

arrayToString​

asData​

chunk​

combine​

composeNextState​

cursor​

dataPath​

dataValue​

del​

each​

each​

expandReferences​

expandRequestReferences​

field​

fields​

fn​

get​

head​

humanProper​

index​

join​

jsonValue​

lastReferenceValue​

map​

merge​

options​

parseCsv​

patch​

post​

put​

referencePath​

request​

scrubEmojis​

source​

sourceValue​

splitKeys​

toArray​

validate​

withAgent​

Functions

alterState

arrayToString

asData

chunk

combine

composeNextState

cursor

dataPath

dataValue

del

each

each

expandReferences

expandRequestReferences

field

fields

fn

get

head

humanProper

index

join

jsonValue

lastReferenceValue

map

merge

options

parseCsv

patch

post

put

referencePath

request

scrubEmojis

source

sourceValue

splitKeys

toArray

validate

withAgent