48 lines
1.8 KiB
Markdown
48 lines
1.8 KiB
Markdown
|
# Quality definition
|
||
|
|
|
||
|
|
The **martiLQ** definition allows for the inclusion of load quality
|
||
|
|
definitions. The load quality definition is intended to be
|
||
|
|
able to be applied universally with common tools. Not
|
||
|
|
all needs are covered with the base definition but can be extended.
|
||
|
|
|
||
|
|
## Defined load quality metrics
|
||
|
|
|
||
|
|
* Sequential batch number - This is a decimal number defined at the **martiLQ** definition
|
||
|
|
and applies to all resources. The integer portion is for new batches and the fraction
|
||
|
|
part can be used for issues with the same data extract. such as requiring resend because
|
||
|
|
a resource was missing.
|
||
|
|
|
||
|
|
* Number of records in the document - This is the number of data primary records not the
|
||
|
|
count of end of lines and is agreed between parties. XML record counts could be based
|
||
|
|
on the number of primary segments under root. JSON records can be counted in a similar way.
|
||
|
|
The headers or trailling records are not counted
|
||
|
|
|
||
|
|
## Addresses deficiencies
|
||
|
|
|
||
|
|
The **martiLQ** objective is to address deficiencies with alternative
|
||
|
|
data load quality approaches such as:
|
||
|
|
|
||
|
|
* magic formats in file names
|
||
|
|
* identifying the number of files
|
||
|
|
* knowing when all files are ready
|
||
|
|
* separate documentation that is unlinked
|
||
|
|
* securing the data
|
||
|
|
* adding footers to the data, requiring custom file handlers
|
||
|
|
|
||
|
|
## Extending load quality metrics
|
||
|
|
|
||
|
|
**martLQ** framewokr and importantly is open to extension so that extra
|
||
|
|
load metrics appropriate to the situation can be included.
|
||
|
|
|
||
|
|
### Extension ideas
|
||
|
|
|
||
|
|
The following extensions for load quality can easily be included:
|
||
|
|
|
||
|
|
* Mandatory data in column
|
||
|
|
* Uniqueness of data values
|
||
|
|
* Data values are within defined tolerances
|
||
|
|
* Check for data exclusions
|
||
|
|
|
||
|
|
And all this information is included in the **martiLQ** definition
|
||
|
|
allowing for self describing load quality.
|