Qrtz

qrtz is the processing language for hybrixd. It is used to handle routing requests. Each routing request spawns a process that follows qrtz steps to a result, or a failure.

A qrtz command always consists of 4 characters. The parameters are separated by spaces:

head parameter1 parameter2 ...

Recipe

Recipes are JSON files that contain parameters and qrtz methods for asset handling, data source queries or actionable engines. A recipe, depending on what it does, fits in one of three categories: asset, source, or engine.

A source-recipe in principle contains only methods to query read-only data sources.

An engine-recipe contains methods to perform actions based on incoming data, business logic and often interfaces with other programming languages.

An asset-recipe contains methods to query value-related ledger or cryptocurrency API's. It returns path options to the hybrixd routing engine for users or machines to perform actions like, for example, querying the balance of an address.

The categories above make it easier to recognize the functionality of a recipe instantly, and analyze it from the proper context. Source and engine recipes are interpreted identically by the scheduler. However, they are available through separate paths: /source and /engine. Asset recipes are specifically tailored to be used in the context of decentralized banking and are available through the routing path: /asset.

Data flow

Every step in a qrtz recipe has the potential to alter the data of the process in which the script is running. Process data is available by using the dollar sign $. It is also available as input to every new process step. It behaves like a variable in qrtz in that it can be read anywhere. However, the difference with a variable is that the data of the process can be changed by every step of the process. The design consideration is that this coaxes you to design your qrtz script with a concise data flow in mind.

Here is an example of some qrtz steps and the resulting data flow.

data 'Hello dog!'       // data 'Hello dog!'    - set the data of the process
drop -4                 // data 'Hello '        - drops the last 4 characters
repl ' ' ' world!'       // data 'Hello world!'  - replaces the ' '  (space) by ' world!'

Variables and scopes

$ the main stream variable it is the output of the previous step. Use $$ to escape the dollar sign (for example in regular expressions).

$variable Retrieve a property from the recipe scope (read only)or process scope if no recipe variable is found.

$local::variable Use a local scope (maintained between all processes spawned from the same recipe).

$otherRecipe::variable the recipe scope of another recipe (read-only)

${.property} To retrieve a sub properties of objects and arrays. For example: if data is {a:{b:1}}} then ${.a.b} will return 1

Debugging

qrtz processes flow can be analyzed using the debug tool. These are only available to node operators with root access.

CLI debugging

Adding the -d flag to a cli api call will provide the debug output. This will illustrate the programe flow of all subprocesses and steps.

$ ./hybrixd /a/dummy/balance/wrongaddress -d
pid: 1575447893066808
.      ╻ /asset/dummy/balance/wrongaddress                 "Invalid address wrongaddress"
.0     ┗┳ call validate/$1                                 "invalid"
.0.0    ┣┳ /asset/dummy/validate/wrongaddress              "invalid"
.0.0.0  ┃┗┳ data '$addressRegex'                           "^_dummyaddress_$"
.0.0.1  ┃ ┣ flow 'undefined' 1 3                           "^_dummyaddress_$"
.0.0.2  ┃ ┣ rout '/source/wavalidator/$symbol/$1'          null
.0.0.3  ┃ ┣ done                                           null
.0.0.4  ┃ ┣ data '$1'                                      "wrongaddress"
.0.0.5  ┃ ┣ regx '$addressRegex' 1 2                       "wrongaddress"
.0.0.6  ┃ ┣ done valid                                     null
.0.0.7  ┃ ┗ done invalid                                   "invalid"
.1      ┣ flow valid 2 1                                   "invalid"
.2      ┣ fail "Invalid address $1"                        "Invalid address wrongaddress"
.3      ┣ data '$1'                                        null
.4      ┣ flow _dummyaddress_ 2 1                          null
.5      ┣ stop 1,'Error: invalid address!'                 null
.6      ┣ pass '10000.00000000'                            null
.7      ┣ form                                             null
.8      ┣ done                                             null

  • $variables
  • @labels
  • Process steps that contain errors.
  • Process steps that have not been executed.

Using the Browser debug interface

An interactive debug interface is available through:

http://localhost:1111/process/debug

This interface is similar to the cli interface but includes live updated progress views.

Shell

qrtz can be ran as a shell and interactive mode.

./qrtz Start qrtz in interactive mode

cat data ./qrtz -i or ./qrtz < cat data Pipe data to interactive mode

qrtz script.qrtz $1 $2 or ./qrtz -s script.qrtz $1 $2 Execute script with command parameters 1,2

qrtz $1 $2 < cat script.qrtz or cat script.qrtz | qrtz $1 $2 Execute script provided trough pipe with command parameters 1,2

Script Example

Create a file script.qrtz with the following content:

data "hello $1"
done

Then use the following to execute:

./qrtz script.qrtz "world"

This results in:

hello world

By adding a "shebang" the file can be made into a qrtz executable:

#!/$QRTZ_HOME/qrtz
data "hello $1"
done

Then use the following to execute:

./script.qrtz "world"

Interactive Shell Example

$ ./qrtz
> data "hello world"
hello world
> quit
$

Pass command line parameters:

$ ./qrtz universe
> data "hello $1"
hello universe
> quit
$

Reference

No results. Change your search filter or reset the filter.

Array

isct dataReturn the intersect of an array or string.

Array/String

case modeChange case of a string.
code target [source='base256']Change encoding of a string from one format to the other.
date [format='epoch'] [timestamp='']Convert a date String and return it's Unix timestamp.
drop [offset=1] [ending=0]Drop elements from the start or end of an array or string. (Reversed function of pick and push.)
excl value [multi=false] [onFound=1] [onNotFound=1]Exclude elements from an array or input string based on their value. Jump to a target based on success or failure. (Reversed function of filt.)
filt [property=''] value [multi=false] [onFound=1] [onNotFound=1]Filter all specified elements from an array, removing all other entries. (Reversed function of excl.)
find needle [onFound=1] [onNotFound=1]Find a string in an array of objects, and return the object it was found in.
flat [depth=infinite]Flatten an array or object.
flipFlip reverse the contents of an array or string
fuse data [offset=length] [delete=0]Append or splice new data onto array or string.
head [size=1]Get the front of a string or array.
indx needle [backwardSearch=false] [arraySearch=false]Find a string in a string or an array of strings, and return the index(es) of it.
join [separator=''] [separator2='']Join an array into a string.
jpar [onSuccess=1] [onFailure=1]Attempt to turn a string into a JSON object. In case of success or failure jump.
jstrTurns JSON object into a string.
many [amount=2]Keep all duplicate elements in an array of which there are many, removing all single, or less available entries.
move sourceKey [sourceCount=0] [sourceOffset=0] [targetOffset=0] [targetCount=0]Move elements from one array to another
pick [start] [end]Pick elements from an array or input string. Indexes, a range or selection array of indexes can be entered.
pull [start] [end]Pull elements from an array or input string. Indexes, a range or selection array of indexes can be entered.
push input [offset=length]Push new data into array or string.
repl [string=] [replace=''] [regexModifier=false]Replace part or parts of a string. All matched occurrences are replaced. Regular expressions can be used by specifying regexModifier.
shuf [amount]Shuffle elements in an array or input string randomly or shift them left or right.
size [input]Count the amount of entries a list, string or object has.
sort methodSort data by using a certain method.
splt [separator=] [elements=undefined]Explode a string into an array, split by separator.
tail [size=1]Get the last part or tail of a string or array.
take start [lenght=all]Take elements cutting them from an array or input string.
trim [leftOrBothTrim=] [rightTrim=leftOrBothTrim]Trim elements from left and or right side of a string or array.
uniq [groupSize=1]Remove all duplicate elements from an array, leaving only unique entries.

Cryptography

keys secretCreate or reproduce cryptographic signing keys.
node requestReturn information about the hybrix node
sign key [signature] [onSuccess] [onFail]Sign a message with a secret key or verify the signature for a public key.

Finance

bank username passwordOrKeyOrFile Banking functions to control hybrix accounts from Qrtz. It is useful for quick implementations of automated and responsive transactions on the node side. Note that this command is not intended for situations where a great amount of parallelized transactions are required. Please use the interface library for this. Also, if you want to use deterministic transactions client-side, please use the hybrix interface library for this purpose. For security, this command can only be used as sessionID root.

Flow

flow [onTrue=1] [onFalse=1]Change the flow of execution based on a string or value comparison.
hook onError [err=1]Indicates a fallback target when an error is encountered.
jump stepJump forward or backward several instructions. Using labels is also supported.
loop position variable condition [initValue=1] [stepOperation=+1]Loop back to position or label based on criteria.
ship [variable] [onPositive=1] [onZero=1] [onNegative=1] [onNaN=1]Test a number to be positive, zero or negative (named after the spaceship operator)
test [variableName] onTrue [onFalse=1]Evaluate condition and jump accordingly. For mathematical conditionals use the 'true' function.
true condition [onTrue=1] [onFalse=1]Test truth of a math condition and choose to jump. The condition is evaluatated mathematically and jumped accordingly.
type [type]Return or set the type of the data stream.
void or [onTrue=1] [onFalse=1]Change the flow of execution based on void analysis.

Interactive

exitExits the interactive shell, only available in interactive cli mode.
help [func]Provides help on provided qrtz command, only available in interactive cli mode.
mode switch1 [switch2...]Toggles modes in interactive shell, only available in interactive cli mode.
quitQuits the interactive shell, only available in interactive cli mode.

Interface

initInitiates a client stack program. This stack can be filled by using step and executed using exec.

Introspection

qrtz [method]List all available methods (use call to call them) or the qrtz code for a given method

Logging

logs [messages...]Logs data to log output.
warn messages...Logs data to error output.

Object

merg [mergeOperator]Merge objects.

Process

call command [data=data]Execute a quartz routine in the same engine. Call creates a new process and waits for the data of that process to be returned (combining Qrtz commands fork and read).
data objectSet data stream to a value.
done [data=data]Stop processing and return message.
each command -Iterate through elements of a container. Execute a given function for each element in parallel.
fail [data=data]Fail processing and return message.
fork command [data=data] [childReferenceID]Creates a new process by calling the command asynchronously.
func nameExecute a javascript module function. Only if the script is run from a recipe with a non-Quartz module defined.
mime dataSet the mime type field of a process.
pass [data=data]Pass data to parent process.
read processIDs [interval=500]Wait for one or more processes to finish and return the results.
stop [err=0] [data=data]Stop processing and return data.
sync nameExecute a javascript module function and return the result directly (used for serving web content through api). Only if the script is run from a recipe with a non-Quartz module defined. Note: can only be defined as singular command in a method
time [millisecs] [data=data] [err=1]Set timeout of current process.
wait millisecsWait a specified amount of time before continuing the process.
with variable qrtz a,Performs a data command using a specific variable. The data buffer is left untouched.

Storage

burn keyDelete a storage item
list [key] [onSuccess=1] [onFail=1] [onEmpty=onSuccess]Return a list of storage keys based on a string. Wildcards are supported.
load [key]Retrieve a string value stored under key. NOTE: When loading saved objects, pass them through jpar for object conversion.
meta keyRetrieve the storage metadata based on a key.
save key [value=data]Store a key value pair. When value is an object, it will be stored as a string.
seek [key=data] [onFound=1] [onNotFound=1]Seek a storage key and jump on existence or not. Wildcards are supported.

String

regx [variable=data] pattern [onMatch=1]] [onNoMatch=1]]Test the data against the regex pattern

TODO

byte direction variable method parameter [code]Encoding and decoding data from/to a bitstream.

Variables

copy source targetCopy a variable to another variable. The data stream is left untouched.
peek [key] [default]Gets data from a variable that was defined by poke. Used inside of other instructions.
poke key [value=data] [default]Put data in a variable for later use. Use peek to retrieve.

Misc

atom [reduce=false] [factor=$factor]Convert a number from atomic units (large integer), or convert to atomic units.
curl [target=$host] [querystring=""] [method="GET"] [headers={}] [overwriteProperties={}] [onSuccess=1] [onFail=1]Use API queue to perform a curl call to an external host.
file [success=1] [failure=1]File reads the data of a filekey into process memory.
form [factor=$factor]Format a (floating point) number and return a neatly padded string.
hash [method='sha256']Create a sha256 hash from an input string.
have pattern objectDefinition [success=1] [failure=1]Checks for data object to contain other objects. Also for arrays and dictionaries of property paths, and for explicit primitive values.
math equationCalculates numbers and variables. Mathematical calculations can be done with very large numbers.
packCompress and serialize data into an URL-safe format.
prog [start] [end] [second]Set progress of parent process directly, or incrementally in a set amount of time.
rand [start] [until]Return a random number between 0 and 1, or an integer.
rout pathRoute a path through hybrixd and pass the result to the next step. /api/help for API information.
scan test [cumulative] [cinit=0]Scan through the values of an array or object, and return values that pass the test.
tran pattern [success=1] [failure=1]Transforms data using a pattern definition. Also for arrays and dictionaries of property paths. Also for explicit primitive values.
unpk [onSuccess=1] [onFailure=1] [parse=true]Deserialize and decompress data, jump on success or failure.