Qrtz

qrtz is the processing language for hybrixd. It is used to handle routing requests. Each routing request spawns a process that follows qrtz steps to a result, or a failure.

A qrtz command always consists of 4 characters. The parameters are separated by spaces:

head parameter1 parameter2 ...

Recipe

Recipes are JSON files that contain parameters and qrtz methods for asset handling, data source queries or actionable engines. A recipe, depending on what it does, fits in one of three categories: asset, source, or engine.

A source-recipe in principle contains only methods to query read-only data sources.

An engine-recipe contains methods to perform actions based on incoming data, business logic and often interfaces with other programming languages.

An asset-recipe contains methods to query value-related ledger or cryptocurrency API's. It returns path options to the hybrixd routing engine for users or machines to perform actions like, for example, querying the balance of an address.

The categories above make it easier to recognize the functionality of a recipe instantly, and analyze it from the proper context. Source and engine recipes are interpreted identically by the scheduler. However, they are available through separate paths: /source and /engine. Asset recipes are specifically tailored to be used in the context of decentralized banking and are available through the routing path: /asset.

Data flow

Every step in a qrtz recipe has the potential to alter the data of the process in which the script is running. Process data is available by using the dollar sign $. It is also available as input to every new process step. It behaves like a variable in qrtz in that it can be read anywhere. However, the difference with a variable is that the data of the process can be changed by every step of the process. The design consideration is that this coaxes you to design your qrtz script with a concise data flow in mind.

Here is an example of some qrtz steps and the resulting data flow.

data 'Hello dog!'       // data 'Hello dog!'    - set the data of the process
drop -4                 // data 'Hello '        - drops the last 4 characters
repl ' ' ' world!'       // data 'Hello world!'  - replaces the ' '  (space) by ' world!'

Variables and scopes

$ the main stream variable it is the output of the previous step. Use $$ to escape the dollar sign (for example in regular expressions).

$variable Retrieve a property from the recipe scope (read only)or process scope if no recipe variable is found.

$local::variable Use a local scope (maintained between all processes spawned from the same recipe).

$otherRecipe::variable the recipe scope of another recipe (read-only)

${.property} To retrieve a sub properties of objects and arrays. For example: if data is {a:{b:1}}} then ${.a.b} will return 1

Debugging

qrtz processes flow can be analyzed using the debug tool. These are only available to node operators with root access.

CLI debugging

Adding the -d flag to a cli api call will provide the debug output. This will illustrate the programe flow of all subprocesses and steps.

$ ./hybrixd /a/dummy/balance/wrongaddress -d
pid: 1575447893066808
.      ╻ /asset/dummy/balance/wrongaddress                 "Invalid address wrongaddress"
.0     ┗┳ call validate/$1                                 "invalid"
.0.0    ┣┳ /asset/dummy/validate/wrongaddress              "invalid"
.0.0.0  ┃┗┳ data '$addressRegex'                           "^_dummyaddress_$"
.0.0.1  ┃ ┣ flow 'undefined' 1 3                           "^_dummyaddress_$"
.0.0.2  ┃ ┣ rout '/source/wavalidator/$symbol/$1'          null
.0.0.3  ┃ ┣ done                                           null
.0.0.4  ┃ ┣ data '$1'                                      "wrongaddress"
.0.0.5  ┃ ┣ regx '$addressRegex' 1 2                       "wrongaddress"
.0.0.6  ┃ ┣ done valid                                     null
.0.0.7  ┃ ┗ done invalid                                   "invalid"
.1      ┣ flow valid 2 1                                   "invalid"
.2      ┣ fail "Invalid address $1"                        "Invalid address wrongaddress"
.3      ┣ data '$1'                                        null
.4      ┣ flow _dummyaddress_ 2 1                          null
.5      ┣ stop 1,'Error: invalid address!'                 null
.6      ┣ pass '10000.00000000'                            null
.7      ┣ form                                             null
.8      ┣ done                                             null

  • $variables
  • @labels
  • Process steps that contain errors.
  • Process steps that have not been executed.

Using the Browser debug interface

An interactive debug interface is available through:

http://localhost:1111/proc/debug

This interface is similar to the cli interface but includes live updated progress views.

Shell

qrtz can be ran as a shell and interactive mode.

./qrtz Start qrtz in interactive mode

cat data ./qrtz -i or ./qrtz < cat data Pipe data to interactive mode

qrtz script.qrtz $1 $2 or ./qrtz -s script.qrtz $1 $2 Execute script with command parameters 1,2

qrtz $1 $2 < cat script.qrtz or cat script.qrtz | qrtz $1 $2 Execute script provided trough pipe with command parameters 1,2

Script Example

Create a file script.qrtz with the following content:

data "hello $1"
done

Then use the following to execute:

./qrtz script.qrtz "world"

This results in:

hello world

By adding a "shebang" the file can be made into a qrtz executable:

#!/$QRTZ_HOME/qrtz
data "hello $1"
done

Then use the following to execute:

./script.qrtz "world"

Interactive Shell Example

$ ./qrtz
> data "hello world"
hello world
> quit
$

Pass command line parameters:

$ ./qrtz universe
> data "hello $1"
hello universe
> quit
$

Reference

No results. Change your search filter or reset the filter.

Array/String

case mode   Change case of a string.
drop [offset=1] [ending=0]   Drop elements from the start or end of an array or string. (Reversed function of pick and push.)
excl value [multi=false] [onFound=1] [onNotFound=1]   Exclude elements from an array or input string based on their value. Jump to a target based on success or failure. (Reversed function of filt.)
filt [property=''] value [multi=false] [onFound=1] [onNotFound=1]   Filter all specified elements from an array, removing all other entries. (Reversed function of excl.)
find needle [onFound=1] [onNotFound=1]   Find a string in an array of objects, and return the object it was found in.
flip   Flip reverse the contents of an array or string.
fuse data [offset=length] [delete=0]   Append or splice new data onto array or string.
head [size=1]   Get the front of a string or array.
incl key   Check if array or string includes a certain value. Boolean version of indx.
indx needle [backwardSearch=false] [arraySearch=false]   Find a string in a string or an array of strings, and return the index(es) of it. For a boolean version see incl.
isct data   Return the intersect of an array or string.
join [separator=''] [separator2='']   Join an array into a string.
many [amount=2]   Keep all duplicate elements in an array of which there are many, removing all single, or less available entries.
move sourceKey [sourceCount=0] [sourceOffset=0] [targetOffset=0] [targetCount=0]   Move elements from one array to another.
pick [start] [end]   Pick elements from an array or input string. Indexes, a range or selection array of indexes can be entered.
pull [start] [end]   Pull elements from an array or input string. Indexes, a range or selection array of indexes can be entered.
push input [offset=length]   Push new data into array or string.
regx [variable=data] pattern [flags=[]] [onMatch=1]] [onNoMatch=1]]   Test the data against the regex pattern
repl [string=] [replace=''] [regexModifier=false]   Replace part or parts of a string. All matched occurrences are replaced. Regular expressions can be used by specifying regexModifier.
shuf [amount]   Shuffle elements in an array or input string randomly or shift them left or right.
size [input]   Count the amount of entries a list, string or object has.
sort method   Sort data by using a certain method.
splt [separator=] [elements=undefined]   Explode a string into an array, split by separator.
tail [size=1]   Get the last part or tail of a string or array.
take start [length=all]   Take elements cutting them from an array or input string.
trim [leftOrBothTrim=] [rightTrim=leftOrBothTrim] [reverseTrim=false]   Trim elements from left and or right side of a string or array.
uniq [groupSize=1]   Remove all duplicate elements from an array, leaving only unique entries.

Cryptography

decr [keys] keys.publicKey keys.secretKey [salt] [onSuccess=1] [onFail]   Decrypt serialized data using a public and private keypair.
encr [keys] keys.publicKey keys.secretKey [salt] [onSuccess=1] [onFail=1]   Encrypt and serialize data into an URL-safe format using a static key.
hash [method='sha256']   Create a sha256 hash from an input string.
keys [secret]   Create or reproduce cryptographic signing keys.
node [request]   Return information about the hybrix node
sign key [signature] [onSuccess] [onFail]   Sign a message with a secret key or verify the signature for a public key.

Flow

flow [onTrue=1] [onFalse=1]   Change the flow of execution based on a string or value comparison.
hook [onError] [errorCode=1]   Indicates a fallback target when an error is encountered.
jump step   Jump forward or backward several instructions. Using labels is also supported.
loop position variable condition [initValue=1] [stepOperation=+1]   Loop back to position or label based on criteria.
root [onRoot]] [onNotRoot]]   Check whether process has sufficient permissions.
ship [variable] [onPositive=1] [onZero=1] [onNegative=1] [onNaN=1]   Test a number to be positive, zero or negative (named after the spaceship operator)
test [variableName] onTrue [onFalse=1]   Evaluate condition and jump accordingly. For mathematical conditionals use the 'true' function.
true condition [onTrue=1] [onFalse=1]   Test truth of a math condition and choose to jump. The condition is evaluatated mathematically and jumped accordingly.
type [type]   Return or set the type of the data stream.
void [variableName] or [onTrue=1] [onFalse=1]   Change the flow of execution based on void analysis.

Interactive

exit   Exits the interactive shell, only available in interactive cli mode.
help [func]   Provides help on provided qrtz command, only available in interactive cli mode.
mode switch1 [switch2...]   Toggles modes in interactive shell, only available in interactive cli mode.
quit   Quits the interactive shell, only available in interactive cli mode.

Logging

info messages...   Logs data to error output (information).
logs [messages...]   Logs data to log output (progress).
warn messages...   Logs data to error output (warnings).

Numbers

atom [reduce=false] [factor=$factor]   Convert a number from atomic units (large integer), or convert to atomic units.
form [factor=$factor]   Format a (floating point) number and return a neatly padded string, rounding down for sufficiency.
math equation [followUpEquation...]   Calculates numbers and variables. Mathematical calculations can be done with very large numbers.
rand [start] [until]   Return a random number between 0 and 1, an integer, or return a random index number from an array.

Process

call command [data=data]   Execute a quartz routine in the same engine. Call creates a new process and waits for the data of that process to be returned (combining Qrtz commands fork and read).
data object   Set data stream to a value.
done [data=data]   Stop processing and return message.
each command -   Iterate through elements of a container. eachute a given function for all elements in parallel.
exec name   Execute a command line script or binary. Allows passing of environment variables. This method is needs root.
fail [data=data]   Fail processing and return message.
fold command -   Iterate through elements of a container. Execute a given function for every element sequentially, failing completely if a single iteration fails. The parallel version of this method is each.
fork command [data=data] [childReferenceID]   Creates a new process by calling the command asynchronously.
func name   Execute a javascript module function. Only if the script is run from a recipe with a non-Quartz module defined.
mime data   Set the mime type field of a process.
pass [data=data]   Pass data to parent process.
prog [start] [end] [second]   Set progress of parent process directly, or incrementally in a set amount of time.
read processIDs [interval=500]   Wait for one or more processes to finish and return the results.
serv path   Serve a file Note: can only be defined as singular command in a method
stop [err=0] [data=data]   Stop processing and return data.
sync name   Execute a javascript module function and return the result directly (used for serving web content through api). Only if the script is run from a recipe with a non-Quartz module defined. Note: can only be defined as singular command in a method
time [millisecs] [data=data] [err=1]   Set timeout of current process.
wait millisecs   Wait a specified amount of time before continuing the process.
with variable qrtz a,   Performs a data command using a specific variable. The data buffer is left untouched.

Storage

burn key   Delete a storage item
file [success=1] [failure=1]   File reads the data of a filekey into process memory.
hops key [value=0]   Read or set the value of the amount of hops that a file is from its originator.
list [key] [onSuccess=1] [onFail=1] [onEmpty=onSuccess]   Return a list of storage keys based on a string. Wildcards are supported.
load [key]   Retrieve a string value stored under key. NOTE: When loading saved objects, pass them through jpar for object conversion.
meta key   Retrieve the storage metadata based on a key.
save key [value=data]   Store a key value pair. When value is an object, it will be stored as a string.
seek [key=data] [onFound=1] [onNotFound=1]   Seek a storage key and jump on existence or not. Wildcards are supported.

Transformers

code target [source='base256']   Change encoding of a string from one format to the other.
date [format='epoch'] [timestamp='']   Convert a date String and return it's Unix timestamp.
flat [depth=infinite]   Flatten an array or object.
jpar [onSuccess=1] [onFailure=1]   Attempt to turn a string into a JSON object. In case of success or failure jump.
jstr   Turns JSON object into a string.
merg [mergeOperator]   Merge objects.
pack   Compress and serialize data into an URL-safe format.
scan test [cumulative] [cinit=0]   Scan through the values of an array or object, and return values that pass the test. This method will filter out any non-numeric characters.
tran pattern [success=1] [failure=1]   Transforms data using a pattern definition. Also for arrays and dictionaries of property paths. Also for explicit primitive values.
unpk [onSuccess=1] [onFailure=1] [parse=true]   Deserialize and decompress data, jump on success or failure.

Variables/Objects

copy source target   Copy a variable to another variable. The data stream is left untouched.
have pattern objectDefinition [onExist] [onNotExist]   Checks for data object to contain other objects. Also for arrays and dictionaries of property paths, and for explicit primitive values.
peek [key] [default]   Gets data from a variable that was defined by poke. Used inside of other instructions.
poke key [value=data] [default]   Put data in a variable for later use. Use peek to retrieve.

Misc

bank method [login] [login.username] [login.password] [login.publicKey] [login.secretKey] [login.accountFile] [login.importKey] [login.importKeyFile] [login.auth] [onSuccess=1] [onFailure]   Banking functions to control hybrix accounts from Qrtz. bank uses the hybrix-jslib methods to check balances, manage keys and perform transactions. See https://api.hybrix.io/help/hybrix-jslib#reference for all methods. Note: if you want to use deterministic transactions client-side, please use the hybrix-jslib for this purpose. For security, this command can only be used as sessionID root.
byte direction variable method parameter [code]   Encoding and decoding data from/to a bitstream.
curl [target=$host] [querystring=""] [method="GET"] [headers={}] [overwriteProperties={}] [onSuccess=1] [onFail=1]   Use API queue to perform a curl call to an external host.
qrtz [method]   List all available methods (use call to call them) or the qrtz code for a given method
rout path [host] [onSuccess=1] [onFailure]   Route a path through hybrixd and pass the result to the next step. /api/help for API information.