Compare commits
6 Commits
| Author | SHA1 | Date |
|---|---|---|
|
|
d8a8ff25a9 | |
|
|
55beb4ecab | |
|
|
46a09520b6 | |
|
|
3906144a1c | |
|
|
cb2dc3cbf8 | |
|
|
f05d8b72be |
|
|
@ -5,7 +5,7 @@ There are various options for using PeterDocs. The following sections will cove
|
|||
## File Filter
|
||||
|
||||
The ``-FileFilter`` parameter allows selection of files that are to be included into the archive file.
|
||||
The parameter only applies to the compress function or buidling the reconciliation file.
|
||||
The parameter only applies to the compress function or building the reconciliation file.
|
||||
|
||||
For example:
|
||||
|
||||
|
|
@ -23,7 +23,7 @@ will only include files with the extension ".jpg" and starting with the characte
|
|||
|
||||
## ReconcileFile
|
||||
|
||||
The ``-ReconcileFile`` parameter allows specification of the reocnciliation file if you
|
||||
The ``-ReconcileFile`` parameter allows specification of the reconciliation file if you
|
||||
wish to select your own name.
|
||||
|
||||
For example:
|
||||
|
|
@ -67,7 +67,7 @@ includes the date.
|
|||
## Compression Level
|
||||
|
||||
By setting the Compression level to a value recognized by the 7Zip4Powershell module you can gain more control
|
||||
of the compresison. The main use case here is for documents that are already compressed and would
|
||||
of the compression. The main use case here is for documents that are already compressed and would
|
||||
not benefit from future compression. To use this feature you need to set the environment variable.
|
||||
|
||||
If all documents are JPEG pictures then setting this value can speed up the compress process
|
||||
|
|
|
|||
|
|
@ -11,7 +11,7 @@ You can use ``PeterDocs`` to reconcile files transferred using the Windows
|
|||
system.
|
||||
|
||||
Robocopy does require your source and target folders to be accessible from
|
||||
the coputer that is executing the command.
|
||||
the computer that is executing the command.
|
||||
|
||||
To use ``PeterDocs`` and ``Robocopy`` install PeterDocs from the PowerShell Gallery
|
||||
and execute the below commands in a PowerShell terminal, changing the values to suit.
|
||||
|
|
@ -52,7 +52,7 @@ below command in a PowerShell terminal, changing the values to suit.
|
|||
New-PeterReconcile -ReconcileFile .\mypictures_metadata.csv -SourceFolder <Source> -ExcludeHash -IncludeExif
|
||||
```
|
||||
|
||||
At the conclusion of the exceution, you will have a file named ``##peter_exif##.csv`` that
|
||||
At the conclusion of the execution, you will have a file named ``##peter_exif##.csv`` that
|
||||
contains your pictures metadata. You will also have a CSV file with picture file
|
||||
general metadata named ``mypictures_metadata.csv`` such as creation time and size.
|
||||
|
||||
|
|
|
|||
|
|
@ -7,7 +7,7 @@ or cloned to a remote location securely. The documents can be binary or text do
|
|||
including personal photographs or sensitive Microsoft Word documents.
|
||||
|
||||
At the remote location a reconciliation can be performed to verify that the documents
|
||||
have been recieved and no alteration occurred.
|
||||
have been received and no alteration occurred.
|
||||
|
||||
## Why
|
||||
|
||||
|
|
@ -49,7 +49,7 @@ You can ignore the remaining parameters if you are happy with the defaults.
|
|||
|
||||
## What
|
||||
|
||||
The ```Compress-Peter``` compressess the contet of the ```SourceFolder``` and saves the result
|
||||
The ```Compress-Peter``` compresses the content of the ```SourceFolder``` and saves the result
|
||||
as the encrypted ```ArchiveFile```. The archive file also contains the reconciliation file
|
||||
so that the recipient of the archive is able to reconcile the restore at the remote location.
|
||||
|
||||
|
|
@ -67,7 +67,7 @@ generate delta archive files.
|
|||
|
||||
## Send Usage
|
||||
|
||||
Once the archive file is created you will commonly send or transfer it to anohter
|
||||
Once the archive file is created you will commonly send or transfer it to another
|
||||
location where it wll be unpacked.
|
||||
|
||||
Please read next the documentation on [sending the archive](SendArchive.md)
|
||||
|
|
|
|||
|
|
@ -3,7 +3,7 @@
|
|||
## Objective
|
||||
|
||||
The objective is to provide a simple capability to create a secure
|
||||
archive file containiung documents to be restored and reconciled at the destination.
|
||||
archive file containing documents to be restored and reconciled at the destination.
|
||||
|
||||
The design artefacts must provide all the tools required to pack,
|
||||
secure, unpack and reconcile the restored/cloned documents.
|
||||
|
|
|
|||
|
|
@ -22,7 +22,7 @@ encryption. The secret (or password) needs to be complex and at least
|
|||
The complex secret needs to contain lower case letter, upper case letter,
|
||||
numeric digit and special symbol.
|
||||
|
||||
The secret is used directly on the 7ZIP compresison and you can use the
|
||||
The secret is used directly on the 7ZIP compression and you can use the
|
||||
secret to decrypt the archive file and check its contents.
|
||||
|
||||
Make a note of the secret as you will need it to decrypt the contents.
|
||||
|
|
@ -34,10 +34,10 @@ file itself.
|
|||
## RecipientKey
|
||||
|
||||
The PeterDocs parameter labelled ```RecipientKey``` is used for asymmetric
|
||||
keys provided by the Microsft Certificate Manager for encrypting content.
|
||||
keys provided by the Microsoft Certificate Manager for encrypting content.
|
||||
|
||||
This is the most secure method for transfer as it is secured with the
|
||||
certficates. Using certificates requires:
|
||||
certificates. Using certificates requires:
|
||||
|
||||
1. your recipient to send you their public certificate
|
||||
2. you to load the public certificate into your Windows Certificate Manager
|
||||
|
|
@ -49,7 +49,7 @@ contents.
|
|||
|
||||
The recipient will need to receive the archive file plus the ```.key```
|
||||
file generated by PeterDocs. Both files must be accessible to the
|
||||
recipient for decryptingg the contents.
|
||||
recipient for decrypting the contents.
|
||||
|
||||
### Internal process
|
||||
|
||||
|
|
@ -58,7 +58,7 @@ is saved into the ```.key``` file which is encrypted using the recipients
|
|||
public key.
|
||||
|
||||
The reason for doing this process is because there are technical limitations
|
||||
in encrypting large files using the certficate public keys. The maximum
|
||||
in encrypting large files using the certificate public keys. The maximum
|
||||
size is around 60MB when using public keys.
|
||||
|
||||
### Generating your Certificate
|
||||
|
|
@ -70,13 +70,31 @@ Manager or executing the below PowerShell command.
|
|||
New-SelfSignedCertificate -Subject "CN=PeterDocs" -FriendlyName "PeterDocs" -KeyDescription "Encryption key for PeterDocs data encipherment" -CertStoreLocation "Cert:\CurrentUser\My" -KeyUsage KeyEncipherment,DataEncipherment, KeyAgreement -Type DocumentEncryptionCert
|
||||
```
|
||||
|
||||
To list your current certficates use the below PowerShell command.
|
||||
To list your current certificates use the below PowerShell command.
|
||||
|
||||
```powershell
|
||||
Get-Childitem -Path "Cert:\CurrentUser\My" -DocumentEncryptionCert
|
||||
```
|
||||
|
||||
**Note**: The certficates from above is assigned to your current logged in user and not the local machine
|
||||
|
||||
### Exporting your Certificate
|
||||
|
||||
You will need to export your public key and send it to the person who will generate the archive
|
||||
file for you. That person with your public key will need to import your public key.
|
||||
|
||||
On Windows, execute the command ``certmgr`` and export the certificate under "Personal\Certificates" for example.
|
||||
|
||||
### Certificate Backup
|
||||
|
||||
You should create a backup of your certificate (public and private) in case your local
|
||||
device suffers a failure. Please secure the backup copy.
|
||||
|
||||
You can use the below as a sample code for exporting the default ``PeterDocs``
|
||||
certificate is give below
|
||||
|
||||
```powershell
|
||||
Get-ChildItem -Path "Cert:\CurrentUser\My" | where{$_.Subject -eq "CN=PeterDocs"} | Export-Certificate -Type CERT -FilePath C:\Temp\PeterDocs_cert.cer -Force
|
||||
```
|
||||
|
||||
Change the values and file name to suit your situation. This file can be imported using the ``certmgr``
|
||||
|
|
|
|||
|
|
@ -36,7 +36,7 @@ To expand the archive you will need write access to the ```RestoreFolder``` loca
|
|||
## What
|
||||
|
||||
The ```Expand-Peter``` decrypts the archive file and expands the contents into
|
||||
the specified restore folder. It does not peform a reconciliation which is the
|
||||
the specified restore folder. It does not perform a reconciliation which is the
|
||||
next step.
|
||||
|
||||
## Reconcile Usage
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
# Install
|
||||
|
||||
PeterDocs is a module that can be donwloaded or installed from
|
||||
PeterDocs is a module that can be downloaded or installed from
|
||||
[PowerShell Gallery](https://xx.com/)
|
||||
|
||||
## Pre-requisites
|
||||
|
|
@ -43,7 +43,7 @@ On a **computer with Internet (PowerShell Gallery) access** do the following ste
|
|||
1. First check you have PowerShell version 5 or later
|
||||
|
||||
```powershell
|
||||
$PSVersionTabe.PSVersion
|
||||
$PSVersionTable.PSVersion
|
||||
```
|
||||
|
||||
2. Download the PeterDocs module
|
||||
|
|
@ -63,7 +63,7 @@ On the **computer lacking Internet (PowerShell Gallery) access** do the followin
|
|||
1. First check you have PowerShell version 5 or later
|
||||
|
||||
```powershell
|
||||
$PSVersionTabe.PSVersion
|
||||
$PSVersionTable.PSVersion
|
||||
```
|
||||
|
||||
2. Run the following command to determine where the ZIP needs to be unpacked
|
||||
|
|
@ -86,3 +86,11 @@ Import-Module PeterDocs
|
|||
```
|
||||
|
||||
The summary details on PeterDocs should be displayed.
|
||||
|
||||
## Uninstall
|
||||
|
||||
To uninstall ``PeterDocs``, execute the following PowerShell command
|
||||
|
||||
```powershell
|
||||
UnInstall-Module -Name PeterDocs
|
||||
```
|
||||
|
|
|
|||
|
|
@ -3,13 +3,13 @@
|
|||
## Why
|
||||
|
||||
Once the archive file is sent, you need to download it from its intermediate
|
||||
location if the source and destination locations are not directky connected.
|
||||
location if the source and destination locations are not directly connected.
|
||||
|
||||
You can user other tools you have available to download the archive file.
|
||||
|
||||
## When
|
||||
|
||||
The archive is received after it sent. The assunmption is that cloud storage
|
||||
The archive is received after it sent. The assumption is that cloud storage
|
||||
is being used as an intermediary.
|
||||
|
||||
## How
|
||||
|
|
@ -36,8 +36,8 @@ The ```SourcePath``` is specified as follows:
|
|||
The "s3" prefix is to download from AWS S3. The "b2" prefix
|
||||
is to download from Backblaze.
|
||||
|
||||
If you are dowloading from AWS you can specify the AWS profile name
|
||||
in parameter ```SourceProfile```. In this situtation the profile
|
||||
If you are downloading from AWS you can specify the AWS profile name
|
||||
in parameter ```SourceProfile```. In this situation the profile
|
||||
needs to exist in the AWS credentials on your local device and user profile.
|
||||
|
||||
If you are downloading from Backblaze you specify the ```AccountId``` and the
|
||||
|
|
@ -57,7 +57,7 @@ use other tools to download the archive file and the key file.
|
|||
|
||||
The function will not expand or reconcile the restore at the destination.
|
||||
|
||||
Please ensure you have sufficent storage to accomodate the local copy of the
|
||||
Please ensure you have sufficient storage to accommodate the local copy of the
|
||||
archive and space to unpack it.
|
||||
|
||||
## Expand Usage
|
||||
|
|
|
|||
|
|
@ -4,7 +4,7 @@ A reconcile file is generated as part of the Compress process and packed with th
|
|||
|
||||
## Why
|
||||
|
||||
When transferring or cloning documenmts to another location, you will want to
|
||||
When transferring or cloning documents to another location, you will want to
|
||||
verify that the same documents have been restored unaltered at the destination.
|
||||
|
||||
## When
|
||||
|
|
@ -44,7 +44,7 @@ The document last update and time is not checked because the value
|
|||
will reflect the date and time of restore.
|
||||
|
||||
The reconciliation summary is displayed in the terminal and the log
|
||||
wil lhave more information.
|
||||
will have more information.
|
||||
|
||||
If any errors are listed, please investigate the discrepancy.
|
||||
|
||||
|
|
|
|||
|
|
@ -43,7 +43,7 @@ The "s3" prefix is to upload to AWS S3. The "b2" prefix
|
|||
is to upload to Backblaze.
|
||||
|
||||
If you are uploading to AWS you can specify the AWS profile name
|
||||
in parameter ```TargetProfile```. In this situtation the profile
|
||||
in parameter ```TargetProfile```. In this situation the profile
|
||||
needs to exist in the AWS credentials on your local device and user profile.
|
||||
|
||||
If you are uploading to Backblaze you specify the ```AccountId``` and the
|
||||
|
|
|
|||
|
|
@ -75,4 +75,4 @@ PeterTask
|
|||
```
|
||||
|
||||
**NOTE** Please do not use the VolumeSize parameter as it will fail due to a current
|
||||
issue with the underlying 7zip4 Powershell module.
|
||||
issue with the underlying 7zip4 Powershell module.
|
||||
|
|
|
|||
Binary file not shown.
|
|
@ -33,7 +33,7 @@ $global:default_metaFile = "##peter##.json"
|
|||
$global:default_errorListFile = Join-Path -Path ".\" -ChildPath "##peter_error_list##.txt"
|
||||
$global:LogPathName = ""
|
||||
$global:MetadataPathName = Join-Path -Path ".\" -ChildPath ".peter-metadata"
|
||||
$global:Version = "0.31"
|
||||
$global:Version = "0.32"
|
||||
|
||||
|
||||
function Open-Log {
|
||||
|
|
@ -415,7 +415,21 @@ Param(
|
|||
if ($ExcludeHash) {
|
||||
$sourceHash = ""
|
||||
} else {
|
||||
$sourceHash = (Get-FileHash -Path $_.FullName).Hash
|
||||
$fullN = $_.FullName
|
||||
try
|
||||
{
|
||||
$sourceHash = (Get-FileHash -Path $fullN -ErrorAction Stop).Hash
|
||||
}
|
||||
catch
|
||||
{
|
||||
$sourceHash = ""
|
||||
$errorDetail = $_.Exception.Message
|
||||
Write-Log "Error in hash with message: $errorDetail"
|
||||
Write-Log "Hash creation failed for '$fullN'. Maybe the file is open in another process"
|
||||
Close-Log
|
||||
Throw "Hash creation failed for '$fullN'. Maybe the file is open in another process"
|
||||
}
|
||||
|
||||
}
|
||||
$record = '"'+$_.FullName.Replace($RootFolder, "")+'","'+$_.LastWriteTime.ToString("yyyy-MM-ddTHH:mm:ss")+'"'
|
||||
$record = $record + ',"'+$_.CreationTime.ToString("yyyy-MM-ddTHH:mm:ss")+'","'+$_.LastAccessTime.ToString("yyyy-MM-ddTHH:mm:ss")+'"'
|
||||
|
|
@ -454,7 +468,21 @@ Param(
|
|||
if ($ExcludeHash) {
|
||||
$sourceHash = ""
|
||||
} else {
|
||||
$sourceHash = (Get-FileHash -Path $_.FullName).Hash
|
||||
$fullN = $_.FullName
|
||||
try
|
||||
{
|
||||
$sourceHash = (Get-FileHash -Path $fullN -ErrorAction Stop).Hash
|
||||
}
|
||||
catch
|
||||
{
|
||||
$sourceHash = ""
|
||||
$errorDetail = $_.Exception.Message
|
||||
Write-Log "Error in hash with message: $errorDetail"
|
||||
Write-Log "Hash creation failed for '$fullN'. Maybe the file is open in another process"
|
||||
Close-Log
|
||||
Throw "Hash creation failed for '$fullN'. Maybe the file is open in another process"
|
||||
}
|
||||
|
||||
}
|
||||
$record = '"'+$_.FullName.Replace($RootFolder, "")+'","'+$_.LastWriteTime.ToString("yyyy-MM-ddTHH:mm:ss")+'"'
|
||||
$record = $record + ',"'+$_.CreationTime.ToString("yyyy-MM-ddTHH:mm:ss")+'","'+$_.LastAccessTime.ToString("yyyy-MM-ddTHH:mm:ss")+'"'
|
||||
|
|
@ -696,6 +724,7 @@ Param(
|
|||
} else {
|
||||
Write-Log "Parameter: SecretKey Value: ************** "
|
||||
}
|
||||
|
||||
Write-Log "Parameter: ArchiveFile Value: $ArchiveFile "
|
||||
Write-Log "Parameter: ReconcileFile Value: $ReconcileFile "
|
||||
Write-Log "Parameter: FileFilter Value: $FileFilter "
|
||||
|
|
@ -770,18 +799,17 @@ Param(
|
|||
$ArchiveFile = $(Get-SoftwareName) + $(Get-Date -Format "yyyyMMdd_HHmm") + ".7z"
|
||||
}
|
||||
|
||||
if ($SecretKey -eq "") {
|
||||
if ($SecretFile -eq "")
|
||||
{
|
||||
if ($SecrettKey -eq "") {
|
||||
if ($SecretFile -eq "") {
|
||||
$SecretFile = $ArchiveFile + ".key"
|
||||
}
|
||||
$secret = New-RandomPassword -Length 80
|
||||
Protect-CmsMessage -To $recipientKey -OutFile $SecretFile -Content $secret
|
||||
if (!(Test-Path -Path $SecretFile)) {
|
||||
Write-Log "Secret file '$SecretFile' not found"
|
||||
Close-Log
|
||||
Throw "Secret file '$SecretFile' not found"
|
||||
}
|
||||
$secret = New-RandomPassword -Length 80
|
||||
Protect-CmsMessage -To $recipientKey -OutFile $SecretFile -Content $secret
|
||||
} else {
|
||||
if (!(Test-PasswordQuality -TestPassword $SecretKey)) {
|
||||
Write-Log "Secret Key does not meet complexity rules"
|
||||
|
|
@ -795,21 +823,18 @@ Param(
|
|||
Write-Log "Saving folders/files to archive file '$ArchiveFile'"
|
||||
Write-Host "Saving folders/files to archive file '$ArchiveFile'"
|
||||
|
||||
if ($ReconcileFile -eq "")
|
||||
{
|
||||
if ($ReconcileFile -eq "") {
|
||||
if (!(Test-Path -Path $global:MetadataPathName)) {
|
||||
$null = New-Item -Path $global:MetadataPathName -ItemType Directory
|
||||
}
|
||||
$ReconcileFile = Join-Path -Path $global:MetadataPathName -ChildPath $default_reconcileFile
|
||||
}
|
||||
|
||||
if ($FileFilter -eq "")
|
||||
{
|
||||
if ($FileFilter -eq "") {
|
||||
$FileFilter = "*"
|
||||
}
|
||||
|
||||
if ($SourceFolder.EndsWith("*"))
|
||||
{
|
||||
if ($SourceFolder.EndsWith("*")) {
|
||||
Write-Log "Archive primary folder is '$SourceFolder'"
|
||||
$firstCompress = $true
|
||||
Get-ChildItem $SourceFolder -Force | ForEach-Object {
|
||||
|
|
|
|||
28
README.md
28
README.md
|
|
@ -1,4 +1,4 @@
|
|||
# PeterDocs - Protect, Transfer, Reconcile Dcouments
|
||||
# PeterDocs - Protect, Transfer, Reconcile Documents
|
||||
|
||||
## Summary
|
||||
|
||||
|
|
@ -14,7 +14,7 @@ that execute the code are required to have Windows PowerShell installed.
|
|||
Use the script to create an encrypted archive of the source folder and its contents, then
|
||||
transfer the archive file to your target, where the content are unpacked using the decryption
|
||||
key. After archive contents are restored you can execute the reconcile function
|
||||
to veriy that the contents are transferred, unaltered.
|
||||
to verify that the contents are transferred, unaltered.
|
||||
|
||||
See [Quick Start](QuickStart.md) if you are ready to start and don't need the details.
|
||||
|
||||
|
|
@ -90,11 +90,11 @@ will need write access on the target storage. A log file is written at execution
|
|||
to record activity.
|
||||
|
||||
Your bulk file transfer is encrypted in transit. Note that if you use the
|
||||
SecretKey method the ecnrypted contents will only be as secure as the strength
|
||||
SecretKey method the encrypted contents will only be as secure as the strength
|
||||
of your secret.
|
||||
|
||||
You can use storage providers such as Dropbox, AWS S3, Google Drive, OneDrive or BackBlaze
|
||||
and your documents have additonal protection.
|
||||
and your documents have additional protection.
|
||||
|
||||
A log file is produced on execution. Repeated executions on the same day
|
||||
will add text content to the same log file. The default log name takes the form:
|
||||
|
|
@ -105,6 +105,26 @@ via local file NuGet package file if Internet access is limited.
|
|||
|
||||
See the [Advanced Usage](Docs/Advanced.md) for more advanced options.
|
||||
|
||||
## Limitations
|
||||
|
||||
### Secure string
|
||||
|
||||
The current version does not use secure strings for password protection
|
||||
within the code. You data is stil protected with encryption.
|
||||
|
||||
## Path length
|
||||
|
||||
There is a limitation with the PowerShell functions used within PeterDocs
|
||||
of file paths having to be 260 characters in length or less.
|
||||
|
||||
If you have long file paths, the processing will fail. A possible
|
||||
work around is to use mapped net work drive even on your local sourced
|
||||
file. The command in PowerShell would be something like:
|
||||
|
||||
```powershell
|
||||
New-PSDrive "X" -PSProvider FileSysytem -Root "$Source"
|
||||
```
|
||||
|
||||
## Further Reading
|
||||
|
||||
[Design](Docs/Design.md)
|
||||
|
|
|
|||
|
|
@ -0,0 +1,34 @@
|
|||
param (
|
||||
[Parameter(Mandatory)]
|
||||
[String] $Source,
|
||||
[Parameter(Mandatory)]
|
||||
[String] $Destination
|
||||
)
|
||||
|
||||
# Note that there is a path limitation for files of 260 characters
|
||||
# beyond which PeterDocs will fail
|
||||
# You could also use drive mapping to overcome this
|
||||
# New-PSDrive "X" -PSProvider FileSysytem -Root "$Source"
|
||||
|
||||
$step ="Starting"
|
||||
Try {
|
||||
$step ="Creating initial reconcile"
|
||||
New-PeterReconcile -ReconcileFile .\myrobocopy.csv -SourceFolder $Source
|
||||
$step ="Running robocopy"
|
||||
Write-Host "Running robocopy for source '$Source' and destination '$Destination'"
|
||||
# Change the command line switches to suit
|
||||
robocopy `"$Source`" "$Destination" /e /copy:DAT /dcopy:DAT /log+:./robocopy.log /r:1000 /w:10
|
||||
if ($LastExitCode -lt 8) {
|
||||
Write-Host "Robocopy succeeded"
|
||||
} else {
|
||||
Write-Host "Robocopy failed with exit code:" $LastExitCode
|
||||
throw "Robocopy error"
|
||||
}
|
||||
$step ="Running copy reconcile"
|
||||
Compare-Peter -ReconcileFile .\myrobocopy.csv -RestoreFolder $Destination
|
||||
# You can modify the code here to add a success email notification
|
||||
} Catch {
|
||||
Write-Host "Error: $_"
|
||||
Write-Error "Processing encountered error at step '$step'"
|
||||
# You can modify the catch to add a simlpe email notification on errors
|
||||
}
|
||||
|
|
@ -0,0 +1,6 @@
|
|||
# Samples
|
||||
|
||||
This folder contains sample scripts using ``PeterDocs``
|
||||
|
||||
The samples are intended to be self documenting. Please adjust
|
||||
the code to suit your circumstances.
|
||||
Loading…
Reference in New Issue