diff --git a/docs/source/filetransfer_ack.png b/docs/source/filetransfer_ack.png new file mode 100644 index 0000000..600936f Binary files /dev/null and b/docs/source/filetransfer_ack.png differ diff --git a/docs/source/filetransfer_appcode.png b/docs/source/filetransfer_appcode.png new file mode 100644 index 0000000..db5446d Binary files /dev/null and b/docs/source/filetransfer_appcode.png differ diff --git a/docs/source/filetransfer_base.png b/docs/source/filetransfer_base.png new file mode 100644 index 0000000..a69f707 Binary files /dev/null and b/docs/source/filetransfer_base.png differ diff --git a/docs/source/filetransfer_injection.png b/docs/source/filetransfer_injection.png new file mode 100644 index 0000000..c54c727 Binary files /dev/null and b/docs/source/filetransfer_injection.png differ diff --git a/docs/source/why.md b/docs/source/why.md index e29eb38..6316bce 100644 --- a/docs/source/why.md +++ b/docs/source/why.md @@ -1,8 +1,7 @@ -Why use marti -============= +# Why use marti **martiLQ** is a framework for providing a degree of auditability and reconciliation of -documents transferred between systems in an organisation and externally. It does not intend +documents transferred between systems in an organization and externally. It does not intend define the format or content of the document. It defines controls that can be used to: 1. verify source @@ -14,12 +13,11 @@ define the format or content of the document. It defines controls that can be u You would use **martiLQ** if any of the controls are a requirement for you. -Documents ---------- +## Documents Documents in this context are digital storage objects such as operating system files, cloud storage objects or blobs. The document content can have structure and contains multiple -records or it can be unstructered such as PDFs. If you are including PDFs then they are +records or it can be unstructured such as PDFs. If you are including PDFs then they are likely to be be supporting documents for your actual data files. The **martiLQ** framework is not intended to be used for single record transfers such as @@ -27,8 +25,7 @@ in single web transactions. It is for providing controls when moving large amou data as one event. This data are commonly referred to as batch extracts and performed at scheduled times such as end of day. -Security --------- +## Security The framework does not replace your security, inflight encryption or encryption at rest. diff --git a/pattern.md b/pattern.md index a8ef8f9..75641d6 100644 --- a/pattern.md +++ b/pattern.md @@ -23,14 +23,14 @@ to demonstrate generating the **martiLQ** document. ## Problem statement -Even though event streaming is a stragetic goal for many organizations, there exists legacy processes and there +Even though event streaming is a strategic goal for many organizations, there exists legacy processes and there will continue to be a need to transfer data flies and other documents from one system to another. When a handover of a data file or document occurs, the best practice is to include metrics with the transfer to assure the recipient of provenance and quality of the data file or document. This is the metadata associated with the data file or document. -A document includes unstructered data, letters, pictures, binary objects while data files could be though of +A document includes unstructured data, letters, pictures, binary objects while data files could be though of as structured data that is describes multiple records. ### Assurance Problem @@ -105,6 +105,12 @@ loaded back onto the same compute resource (think mainframe) or physically couri reference book [Enterprise Integration Patterns](https://www.enterpriseintegrationpatterns.com/patterns/messaging/FileTransferIntegration.html) by Hohpe and Woolf recognizes this by inclusion of the pattern written by Martin Fowler. +--- +**The basic _file transfer_ pattern** +![](docs/source/filetransfer_base.png) + +--- + In the explanation written by Martin Fowler, he makes observations about the "File Transfer" including: "Part of what makes _File Transfer_ simple is that no extra tools or integration packages are needed, but that also means that developers @@ -114,6 +120,29 @@ have to do a lot of the work themselves. The applications must agree om file-nam The pattern being described here addresses the issues and concerns that relate to file transfer. Many of these are related to the common non functional requirements that architects cover in solution designs. +## Solution + +The **martiLQ** document pattern can be implemented in various combinations. Some combinations are illustrated below. + +--- +**Using the pattern without large scale change to existing processing** +![](docs/source/filetransfer_injection.png) + +--- + +--- +**Inclusion of the pattern into existing application code to generate and read the martiLQ document** +![](docs/source/filetransfer_appcode.png) + +--- + + +--- +**If there is a requirement for acknowledgement to the publisher, then maybe this approach is applicable** +![](docs/source/filetransfer_ack.png) + +--- + ### Security, robustness, reliability, fault-tolerance The pattern defines how security and assurance is applied to the data files and documents. The pattern does diff --git a/source/golang/server/src/docs/why.md b/source/golang/server/src/docs/why.md index e29eb38..6316bce 100644 --- a/source/golang/server/src/docs/why.md +++ b/source/golang/server/src/docs/why.md @@ -1,8 +1,7 @@ -Why use marti -============= +# Why use marti **martiLQ** is a framework for providing a degree of auditability and reconciliation of -documents transferred between systems in an organisation and externally. It does not intend +documents transferred between systems in an organization and externally. It does not intend define the format or content of the document. It defines controls that can be used to: 1. verify source @@ -14,12 +13,11 @@ define the format or content of the document. It defines controls that can be u You would use **martiLQ** if any of the controls are a requirement for you. -Documents ---------- +## Documents Documents in this context are digital storage objects such as operating system files, cloud storage objects or blobs. The document content can have structure and contains multiple -records or it can be unstructered such as PDFs. If you are including PDFs then they are +records or it can be unstructured such as PDFs. If you are including PDFs then they are likely to be be supporting documents for your actual data files. The **martiLQ** framework is not intended to be used for single record transfers such as @@ -27,8 +25,7 @@ in single web transactions. It is for providing controls when moving large amou data as one event. This data are commonly referred to as batch extracts and performed at scheduled times such as end of day. -Security --------- +## Security The framework does not replace your security, inflight encryption or encryption at rest.