(cache)T-ORCH SPEC

Authorship and Legal

T-ORCH is intended to be a scalable service orchestration layer for Service Oriented Architecture. It is wholly designed and implemented by Chris Punches in 2018. Its design is owned by this project’s parent group, SILO GROUP, LTD. and Chris Punches and is released under the Attribution-NonCommercial-NoDerivs license.

Abstract

T-ORCH allows you to centrally control and administrate, and orchestrate, services and microservices across environments or within the same environment. It is intended for complex application layer orchestration as a command and control system triggering actions from within the services that are aware of it.

The diagram below outlines component interoperation at an abstract level:

Components

The controller, which will be called bonebox.
The API, which will be called orchard.
The MQ, which will always be just MQ.
The consumer, which will be called harvester.
The database, which will always be just DB.
The service, which is what this whole thing revolves around and isn’t part of the solution really, but needs to be aware of how it should interact, presumably through shared library calls. For simplicity we’ll just call this service but it’s technically a client to orchard.

Bonebox Controller

The bonebox controller registers a request or cancels a request. It also can check on the state of requests. It is the point at which bonebox GUIDs are registered. It is able to obtain all available service GUIDs and request GUIDs.

Orchard API

The Orchard RESTful API reports on state for requests, creates, updates, reports, and deletes requests from the controller, which it relays to the MQ for buffering. It polls request state directly from the DB, provides a way for services to check for new requests, as well as sending request state updates from the service to the MQ. It is the point at which request GUIDs and bonebox controller GUIDs are created. It is the point at which GUIDs are paired to each other. It also authenticates users.

Service

The service, besides performing it’s normal function in the environment, gets new requests once they’ve been registered, it also acknowledges requests, and also marks requests as complete or failed. The service is registered by Bonebox via Orchard to create a service GUID consumable by Bonebox for request associations.

MQ

The MQ receives all request creations and updates (state changes), including request cancellations and deletions from the Orchard API.

Harvester Consumer

The harvester consumer relays requests from the MQ and inserts them into a database.

Database

The database receives creation, update, or deletion requests from the consumer only. It also provides the table used to report on request state to the Orchard API.

Component Distribution

There can be multiple instances of bonebox.
There can be multiple instances of orchard.
There can be multiple services being controlled.
There can not be multiple instances of MQ for each orchard and harvester.
There can not be multiple instances of harvester for each MQ, orchard and DB.
There can not be multiple instances of DB for each orchard, harvester, MQ.

Access Control

Orchard will serve as the point of reference for the data model and component access model. It’s going to be very simple. API authentication will be present:

A user for each bonebox instance.
A user for each service.
A super user whose credentials are provided by the bonebox end user during registration of a bonebox instance/user and registration of a service instance/user. Registration is the point at which Orchard creates, stores, and provides a GUID.

Requests

States

There are five states for a request:

(REG) Registered
(ACK) Acknowledged
(CPT) Completed
(ERR) Failed
(CCL) Cancelled

Lifecycle

These states also represent the request lifecycle:

REG -> [ ACK ] -> ( CPT | ERR | CCL )

Data Fields

A request is ultimately an object in transport and a table row at rest. A rigid object structure is used uniformly across all components.

A JSON representation of the fields a Request object must have is below. Items marked in bold are state-specific, meaning that they are only present when updating to their associated state or when reporting to bonebox. All items are displayed when reporting to bonebox.

{
  "bonebox guid": "9001-9001-9001-9001",
  "service guid": "9002-9002-9002-9002",
  "request guid": "9003-9003-9003-9003",
  "state": "REG|ACK|CPT|ERR|CCL",
  "payloads": { 
    'REG': { 'timestamp': '@timestamp', 'payload': 'dump_to_disk_mode 1' },
    'ACK': { 'timestamp': '@timestamp', 'payload': 'preparing to dump to disk' },
    'CPT': { 'timestamp': '@timestamp', 'payload': 'now logging to disk' },
    'ERR': { 'timestamp': '@timestamp', 'payload': 'failed to transition mode: I/O error' },
    'CCL': { 'timestamp': '@timestamp', 'payload': 'no longer needed, cancel if possible' }
  }
}

For instance, request guid is not present in a “REG” request body from bonebox because it does not exist until this is created by Orchard. Neither would the ACK, CPT, ERR, or CCL payload entries. These entries would, however, all be present when bonebox fetches states for requests. The whole object is returned.

Actor/Request Interoperation

The Orchard API is the arbiter of how components are allowed to behave with each other and with the state of requests. The logic around access control will be based on the state the request is in, the source of the action being taken, and the target of the action.

The format is as follows: The top level hiearchy represents a CRUD structure (Create, Read, Update, Delete). Each top level hierarchy is divided into “All” or “Single” to categorize movements against all request targets in a CRUD category.

Entries are of the format:

<actor> by [(<field> [and])] [via <actor mechanism>] for <request criteria> [ –<scope clarification> ]

Registered (REG)

Can Create:
- Single:
  - bonebox by bonebox guid and service guid via orchard for a single request associated with a single service
- All:
  - None. Requests are registered one at a time, and are one per service.

Can Read:
- Single:
  - service by request guid via orchard
  - bonebox by bonebox guid and request guid via orchard for a single request associated with a service associated with that bonebox
- All:
  - bonebox by bonebox guid [and service guid] via orchard for all requests associated with that [service and] bonebox

Can Update:
- Single:
  - bonebox by request guid and bonebox guid via orchard for a request associated with that bonebox — can only update to CCL
  - service by request guid and service guid via orchard for a request associated with that service — can only update to ACK
- All:
  - bonebox by bonebox guid [and service guid] via orchard for all requests [to a single service] associated with that bonebox — can only update to CCL
Can Delete:
- Single:
  - None. This is not a terminal state.
- All:
  - None. This is not a terminal state.

Acknowledged (ACK)

Can Create:
- Single:
  - None. This is not an entry state.
- All:
  - None. This is not an entry state.
Can Read:
- Single:
  - bonebox by request guid via orchard for a request associated with that bonebox
- All:
  - bonebox by [service guid and] bonebox guid via orchard for all requests [to a service] associated with that bonebox
Can Update:
- Single:
  - service by request guid and service guid via orchard for a request associated with that service — can only update to CPT or ERR.
  - bonebox by request guid and bonebox guid via orchard for a request associated with that bonebox — can only update to CCL.
- All:
  - None. Requests are processed one at a time.
Can Delete:
- Single:
  - None. Only requests in a terminal state can be deleted.
- All:
  - None. Requests are processed one at a time and can only be deleted in a terminal state.

Completed (CPT)

Can Create:
- Single:
  - None. This is not an entry state.
- All:
  - None. This is not an entry state.
Can Read:
- Single:
  - bonebox by request guid and bonebox guid via orchard for a request associated with that bonebox
- All:
  - bonebox by [service guid and] bonebox guid via orchard for requests associated with that [service and] bonebox.
Can Update:
- Single:
  - None. A completed request can only be deleted.
- All:
  - None. A completed request can only be deleted.
Can Delete:
- Single:
  - bonebox by request guid and bonebox guid via orchard for requests associated with that bonebox.
- All:
  - bonebox by [service guid and] bonebox guid via orchard for requests associated with that [service and] bonebox

Cancelled (CCL)

Can Create:
- Single:
  - None. This is not an entry state.
- All:
  - None. This is not an entry state.
Can Read:
- Single:
  - bonebox by request guid and bonebox guid via orchard for requests associated with that bonebox
- All:
  - bonebox by [service guid and] bonebox guid via orchard for requests associated with that [service and] bonebox
Can Update:
- Single:
  - None. This is a terminal state.
- All:
  - None. This is a terminal state.
Can Delete:
- Single:
  - bonebox by request guid and bonebox guid via orchard for requests associated with that bonebox
- All:
  - bonebox by [service guid and] bonebox guid for requests associated with that [service and] bonebox

Failed (ERR)

Can Create:
- Single:
  - None. This is not an entry state.
- All:
  - None. This is not an entry state.
Can Read:
- Single:
  - bonebox by request guid and bonebox guid via orchard for requests associated with that bonebox
- All:
  - bonebox by request guid [and service guid] via orchard for requests associated with that [service and] bonebox
Can Update:
- Single:
  - None. This is a terminal state.
- All:
  - None. This is a terminal state.
Can Delete:
- Single:
  - bonebox by request guid and bonebox guid via orchard for requests associated with that bonebox
- All:
  - bonebox by request guid [and service guid] via orchard for requests associated with that [service and] bonebox