MartiLQ

martiLQ stands for metadata reconcilation for transfer information, load quality.

The objective is to provide transfer information for high volume data such as in files. The files can be transferred via HTTPS, SFTP, message queue, network share or other. The transfer information being described here does not need to arrive via the same channel and could be received via email or even synchronous / asynchronous API. The transfer information does not dictate or determine how the data is formatted.

The transfer information can provide details on the file format, but in itself it does not understand the data format.

Note: The terms file and document are intended to be interchangeable through out this documentation.

martiLQ is intended to provide minimum basic information on the transfer with
ability to include optional information. The metadata reconcilation transfer document being decscribed here wil be referred to as the martiLQ document throughout this documentation.

The transfer information is supplied as a separate document which could be another file or supplied via API by the publisher notifying the consumer(s).

Tools and Scenarios

Tools and code snippets are provided to generate the transfer information and then assist in reconciling the document contents once received. Refer to the source programming folders for more details or Tools for more general information

• Java
• golang
• python
• powershell
• docker

Transfer information

The information in the martiLQ document is summarised below. For more detailed information see martiLQ definition

Mandatory information

The mandatory information is:

• Title
• Unique identifier
• Resource list - See Resource section summary below or detailed document Resource

Optional information

The optional information is:

• Description
• Modified
• Tags or keywords
• Publisher
• Contact point
• Acces level
• Rights
• Batch
• License
• Spatial (*)
• Temporal (*)
• Described By - A link to the metadata describing the document.
  More details information could be supplied in the distribution
• Landing page
• Theme

Information extension

The information supplied can be extended by party agreement and there are place holders in the defintion.

Resource

The resource section is intended to allow multiple data files to be grouped together. The resource section can be repeated, but at least one must be included. If the resource is repeated it will commonly be for definiting multiple formats of the same data or batching of different data together from the same extract process.

• Title
• Unique identifier
• Document name
• Issued date - When the document was made available. The date can include time
• Modified - When the document was created or modified. This is the data and time
• Size of document - The document size in bytes
• Hash of document - The hash of the document, which can be blank especially for large documents
• Hash algorithm

Resource optional

The following are some of the optional items in the resource section. See Resource for more details

• Description
• Download URL
• Version
• Format
• Compression
• Encryption