PeterDocs/ptfrFiles/ptrFiles.ps1

<#
 .Synopsis
   Allows the secure transfer and reconciliation of a large number of files

   PTRfile: Protect, Transfer, Reconcile files

 .Description
   Packages source folder contents into a 7ZIP file, adding a reconciliation 
   file to the 7ZIP file and then encrypting the contents.  Send 
   * this script
   * the 7ZIP package file 
   * plus optional SecretFilename ( if using RecipientKeyName )
   to the target or recipient.
   
   The source folder is not altered and only read rights are required. A log
   file is written at exceution to record activity.

   The SecretFileName can be sent via email, while the 7ZIP can go different routes 
   due to possible size such as:
   * Cloud storage provider
   * HTTPS web file upload
   * SFTP transfer
   * USB stick

   At the target, unpack the contents to a folder and reconcile the results.  You
   will need write access on the target storage. A log file is written at exceution
   to record activity.

   Your bulk file transfer is encrypted in transit.  Note that if you use the
   SecretKey method the ecnrypted contents will only be as secure as the strength
   of your secret.

   You can use storage providers such as Dropbox, AWS S3, Google Drive, OneDrive or BackBlaze
   and your documents have additonal protection.

   A log file is produced on execution.  Repeated executions on the same day
   will add text content to the same log file.  The default log name takes the form:
   "ptr_files_yyyy-MM-dd.log"

   You will need to have installed the 7Zip4Powershell PowerShell cmdlet 
   before using the pack or unpack actions.  You can install the cmdlet
   by executing 
   .\ptrFiles.ps1 -Action install -Path ".\" 

   Author:  Tom Peltonen

 .Parameter Action
  Action to perform, which can be:
  - Install             : Install 7Zip4PowerShell
  - Pack                : Archive the contents of a folder(s)
  - Unpack              : Unpack the archive, but no reconfile is performed
  - Reconcile           : Reconcile the contents in the restored folder
  - ReconcileFile       : Generate reconfile file.  The pack process does this.
  - ArchiveInformation  : Fetch archive information

 .Parameter Path
  The path to the files and folders to pack or the path to the unpack location. 
  The path can include a trailing * as a wildcard to only include a subset of 
  directories.

  When using the trailing * for names, the filtering is only applied to immediate
  folder names under the parent folder.  The filter does not cascade to lower folders.

  The path can be a local drive, mapped network drive or a network shared folder
  location such as \\stora\MyLibrary.

  The Path can also be a file containing a list of paths, one per line.  To use a
  list file, prefix the Path value with a "@" and name the file. Do not use a folder
  for @ defined path.

  A file (@ prefix) containing a list of paths cannot contain generic path names, that 
  is paths with trailing wildcard of "*"

 .Parameter RecipientKeyName
  The recipient of the package which is used to find the appropriate
  certificate for encrypting with the public key.  Either the RecipientKeyName 
  or the SecretKey is required for packing or unpacking the 7ZIP file.
  Using the RecipientKeyName is the most secure transfer option as a
  asymmetric cryptographic key is used that can only be decrypted by the 
  holder of the private key.

  If you are using the RecipientKeyName, then the 7ZIP file contents can only
  be unzipped by the holder of the private key and the SecretFileName file.
  If you don't have the private, which you should not unless you are sending
  to yourself, then you cannot unpack the 7ZIP file.

 .Parameter SecretKey
  A tradiitional secret to encrypt or decrypt the 7ZIP package. Either the RecipientKeyName 
  or the SecretKey is required for packing or unpacking the 7ZIP file.  This method
  uses a symmetric cryptographic key exchange which is less secure then the 
  RecipientKeyName approach.

  Note: Currently the script doe snot user Secure Strings

 .Parameter ArchiveFileName
  The location and name of the 7ZIP file.  If not supplied a default 7ZIP file name
  will be generated in the current directory for the pack action.

  The default name will take the form ".\transfer_protect_yyyyMMdd_hhmm.7z"

  For unpack actions, the archive file name parameter is mandatory.

 .Parameter RootFolderName
  The root folder, which should be used if using wildcard (*) for the
  path.  A guess will be made as to value if not supplied, which will
  work in many circumstances.

 .Parameter FileFilter
  A filter on file names.  This does not filter directories.
  An example to only include JPEG file is "*.jpg".  You can also
  filter on picture file names starting with "IMG*.jpg"

 .Parameter ReconcileFileName
  The name of the reconfile file name to generate during pack or use 
  during unpack.  This is a file name without path.  If no value is 
  supplied, then a default name is generated.
  The reconcile file is included into the root of the 7ZIP file.
  Once a reconcile is executed, you can delete this file from the 
  restored location.

  The default name is "##protect_transfer_reconcile_files##.csv"

 .Parameter SecretFileName
  The secret file name is used with RecipientKeyName to secure the
  internally generated password for the 7ZIP file.  When unpacking the
  7ZIP file you will need access to this file if RecipientKeyName
  was used. If not supplied a default name is used.  This file is 
  encrypted with RecipientKeyName.

  The default name is the archive file name with postfix  ".key"

 .Parameter CloudProfile
  The profile name to use for Install and Transfer actions.  The
  default for Install is "UserScope".  The default for "Transfer"
  is "default"
  Profile name can also be specifed with Environment variable 
  "PTRFILES_PROFILE"

 .Parameter ExcludeHash
  Exclude the file hash from the reconcile file.  As producing a file
  hash takes compute cycles during pack, you can select to bypass this 
  generation to speed up the packaging.  Excluding the hash does reduce 
  the functionality of the reconciliation at unpack.

 .Parameter LogPath
  The log folder where log files are written.  If the folder does not
  exist then it is created.  You need write access rights to this location.

 .Notes
  This script has been written to use the 7ZIP function as it is open source
  and provides a secure encryption mechanism, plus portability on Windows,
  Linux and MacOS.

  It is also beneficial that 7ZIP has efficient compression algorithms.

  Compressing and packing a large data set can take significant time and also
  require storage space.  The script does not check if you have sufficient
  free storage to package the source contents into a single 7ZIP file.  It is your
  responsibility to ensure sufficient storage space exists.

  If you need to copy files from one directory to another accessible directory from
  your Windows desktop, you might consider using ROBOCOPY.  If the target directory
  is not accessible and you want to reconcile, then this tool is appropriate. 

  The following environment variables are supported:
  - PTRFILES_RECIPIENTKEYNAME
  - PTRFILES_PROFILE

 
 .Example
   # Pack and encrypt all files in folder ".\transferpack\" using a private-public key
   # A file with the postifx ".key" is also generated alongside the 7ZIP file
   .\ptrFiles.ps1 -Action pack -Path ".\transferpack\" -RecipientKeyName data@mycompany
 
 .Example
   # Unpack all files in 7ZIP file "transfer_protect_yyyMMdd_hhmm.7z" to folder ".\targetdir" using a private-public key
   # You will need the file "transfer_protect_yyyMMdd_hhmm.7z.key" to unpack the encrypted 7ZIP file
   .\ptrFiles.ps1 -Action unpack -ArchiveFileName "transfer_protect_yyyMMdd_hhmm.7z" -Path ".\targetdir" -RecipientKeyName data@mycompany
 
 .Example
   # Reconcile files in folder ".\targetdir"
   .\ptrFiles.ps1 -Action reconcile -Path ".\targetdir" 

 .Example
   # Pack and encrypt all files in folder ".\transferpack\" using a password
   .\ptrFiles.ps1 -Action pack -Path ".\transferpack\" -SecretKey "fjks932c-x=23ds"
 
 .Example
   # Unpack all files in 7ZIP file "transfer_protect_yyyMMdd_hhmm.7z" to folder ".\targetdir" using a password
   .\ptrFiles.ps1 -Action unpack -ArchiveFileName "transfer_protect_yyyMMdd_hhmm.7z" -Path ".\targetdir" -SecretKey "fjks932c-x=23ds"

 .Example
   # Pack and encrypt all files in folder ".\transferpack\02*" where the folder name starts with "02" using a password
   .\ptrFiles.ps1 -Action pack -Path ".\transferpack\02*" -SecretKey "fjks932c-x=23ds"

#>

param (
    [Parameter(Mandatory)][String] $Action, 
    [Parameter(Mandatory)][String] $Path, 
    [String] $RecipientKeyName,
    [String] $SecretKey, 
    [String] $ArchiveFileName, 
    [String] $RootFolderName,
    [String] $FileFilter,
    [String] $ReconcileFileName, 
    [String] $SecretFileName, 
    [String] $CloudProfile,
    [switch] $ExcludeHash,
    [String] $LogPath

)

$default_dateLocal = Get-Date -Format "yyyyMMdd_HHmm"
$default_archiveFile = ".\ptr_file_##date##.7z"
$default_reconcileFile = "##protect_transfer_reconcile_files##.csv"
$default_profile = "default"

function Write-Log {
    param(
        [String] $LogEntry
    )

    $date = Get-Date -f "yyyy-MM-dd"

    if ($LogPath -eq "")
    {
        $logPath = Join-Path -Path ".\" -ChildPath "Logs"
    }
    $logName = "ptr_files_$date.log"
    $sFullPath = Join-Path -Path $logPath -ChildPath $logName 

    if (!(Test-Path -Path $logPath)) {
        $null = New-Item -Path $logPath -ItemType Directory
    }

    if (!(Test-Path -Path $sFullPath)) {
        Write-Host "Log path: $sFullPath"
        $null = New-Item -Path $sFullPath -ItemType File
    }
    $dateTime = Get-Date -f "yyyy-MM-dd HH:mm:ss"
    Add-Content -Path $sFullPath -Value "[$dateTime]. $LogEntry"

}

function Close-Log {
    $dateTime = Get-Date -f "yyyy-MM-dd HH:mm:ss"
    Write-Log "***********************************************************************************"
    Write-Log "*   End of processing: [$dateTime]"
    Write-Log "***********************************************************************************"
}

function Get-SoftwareName {
    return [String] "PTRFILES"
}

function New-RandomPassword {
param(
    [int] $length = 20,
    [String] $characters = "abcdefghiklmnoprstuvwxyzABCDEFGHKLMNOPRSTUVWXYZ1234567890!()?}][{@#*+-",
    [switch] $ConvertToSecureString
)
    $password = 1..$length | ForEach-Object { Get-Random -Maximum $characters.length }
    $private:ofs=""
    
    if ($ConvertToSecureString.IsPresent) {
        return ConvertTo-SecureString -String [String]$characters[$password] -AsPlainText -Force
    } else {
        return [String]$characters[$password]
    }
}

function Test-FilesExist
{
    Param( 
        [Parameter(Mandatory)][String] $FolderName,
        [String] $FileFilter
    ) 

    Get-ChildItem $folderName -Recurse | Where-Object {!$_.PSIsContainer} | ForEach-Object {
        return $true
    }
    
    return $false
}

function Get-ConvenientFileSize
{
    Param( 
        [Parameter(Mandatory)][long] $Size
    ) 
 
    
    if ($totalFileSize -ge 1000000000000) {
        $totalRightLabel = "TB"
        $totalFileXbytes = [math]::Round(($size / 1000000000000), 2)        
    } else {
        if ($totalFileSize -ge 1000000000) {
            $totalRightLabel = "GB"
            $totalFileXbytes = [math]::Round(($size / 1000000000), 2)        
        } else { 
            if ($totalFileSize -ge 1000000) {
                $totalRightLabel = "MB"
                $totalFileXbytes = [math]::Round(($size / 1000000), 2)        
            } else {
                $totalRightLabel = "KB"
                $totalFileXbytes = [math]::Round(($size / 1000), 2)
            }
        }
    }

    return $totalFileXbytes.ToString() + " " + $totalRightLabel
}



function Get-B2ApiToken {
    Param
    (
        [Parameter(Mandatory)] [ValidateNotNull()] [ValidateNotNullOrEmpty()] [String] $AccountId,
        [Parameter(Mandatory)] [ValidateNotNull()] [ValidateNotNullOrEmpty()] [String] $AccountKey
    )


    Begin
    {
        if(-not $AccountID -or -not $AccountKey)
        {
            [PSCredential]$b2Creds = Get-Credential -Message 'Enter your B2 account ID and application key below.'
            try
            {
                [String]$AccountId = $b2Creds.GetNetworkCredential().UserName
                [String]$AccountKey = $b2Creds.GetNetworkCredential().Password
            }
            catch
            {
                throw 'You must specify the account ID and application key.'
            }
        }
        
        [String]$plainCreds = "${AccountId}:${AccountKey}"
        [String]$encodedCreds = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes($plainCreds))
        [Hashtable]$sessionHeaders = @{'Authorization'="Basic $encodedCreds"}
        [Uri]$b2ApiUri = 'https://api.backblaze.com/b2api/v1/b2_authorize_account'
    }
    Process
    {
        try
        {
            $b2Info = Invoke-RestMethod -Method Get -Uri $b2ApiUri -Headers $sessionHeaders
            [String]$script:SavedB2AccountID = $b2Info.accountId
            [Uri]$script:SavedB2ApiUri = $b2Info.apiUrl
            [String]$script:SavedB2ApiToken = $b2Info.authorizationToken
            [Uri]$script:SavedB2DownloadUri = $b2Info.downloadUrl

            $b2ReturnInfo = [PSCustomObject]@{
                'AccountID' = $b2Info.accountId
                'ApiUri' = $b2Info.apiUrl
                'DownloadUri' = $b2Info.downloadUrl
                'Token' = $b2Info.authorizationToken
            }

            return $b2ReturnInfo
        }
        catch
        {
            $errorDetail = $_.Exception.Message
            Write-Error -Exception "Unable to authenticate with given APIKey.`n`r$errorDetail" `
                -Message "Unable to authenticate with given APIKey.`n`r$errorDetail" -Category AuthenticationError
        }
    }    

}


function Get-B2Bucket {
    Param
    (
        [Parameter(Mandatory)] [ValidateNotNull()] [ValidateNotNullOrEmpty()] [String] $ApiToken,
        [Parameter(Mandatory)] [ValidateNotNull()] [ValidateNotNullOrEmpty()] [String] $AccountId,
        [Parameter(Mandatory)] [ValidateNotNull()] [ValidateNotNullOrEmpty()] [String] $ApiUri,
        [Parameter(Mandatory)] [ValidateNotNull()] [ValidateNotNullOrEmpty()] [String] $BucketHost
    )

    Begin
    {
        [Hashtable]$sessionHeaders = @{'Authorization'=$ApiToken}
        [String]$sessionBody = @{'accountId'=$AccountID} | ConvertTo-Json
        [Uri]$b2ApiUri = "$ApiUri/b2api/v1/b2_list_buckets"
    }
    Process
    {
        $b2Info = Invoke-RestMethod -Method Post -Uri $b2ApiUri -Headers $sessionHeaders -Body $sessionBody
        foreach($info in $b2Info.buckets)
        {
            if ($bucketHost -eq $info.bucketName) {
                $b2ReturnInfo = [PSCustomObject]@{
                    'BucketName' = $info.bucketName
                    'BucketID' = $info.bucketId
                    'BucketType' = $info.bucketType
                    'AccountID' = $info.accountId
                }
                return $b2ReturnInfo
            }
        }

        return $null
    }

}

function Get-B2UploadUri {
    Param
    (
        [Parameter(Mandatory)] [ValidateNotNull()] [ValidateNotNullOrEmpty()] [String] $BucketHost,
        [Parameter(Mandatory)] [ValidateNotNull()] [ValidateNotNullOrEmpty()] [String] $FileName,
        [Parameter(Mandatory)] [ValidateNotNull()] [ValidateNotNullOrEmpty()] [Uri] $ApiUri,
        [Parameter(Mandatory)] [ValidateNotNull()] [ValidateNotNullOrEmpty()] [String] $ApiToken
    )

    Begin
    {
        [Hashtable]$sessionHeaders = @{'Authorization'=$ApiToken}
        [Uri]$b2ApiUri = "$ApiUri/b2api/v1/b2_get_upload_url"
    }
    Process
    {
        try
        {
            [String]$sessionBody = @{'bucketId'=$bucketHost} | ConvertTo-Json
            $b2Info = Invoke-RestMethod -Method Post -Uri $b2ApiUri -Headers $sessionHeaders -Body $sessionBody
            $b2ReturnInfo = [PSCustomObject]@{
                'BucketId' = $b2Info.BucketId
                'UploadUri' = $b2Info.uploadUrl
                'Token' = $b2Info.authorizationToken
            }

            return $b2ReturnInfo
        }
        catch
        {
            $errorDetail = $_.Exception.Message
            Write-Error -Exception "Unable to retrieve the upload uri.`n`r$errorDetail" `
                -Message "Unable to retrieve the upload uri.`n`r$errorDetail" -Category ReadError
        }
    }

}


function Invoke-B2SUpload {
    Param
    (
        [Parameter(Mandatory)] [ValidateNotNull()] [ValidateNotNullOrEmpty()] [String] $BucketHost,
        [Parameter(Mandatory)] [ValidateNotNull()] [ValidateNotNullOrEmpty()] [String] $TargetPath,
        [Parameter(Mandatory)] [ValidateNotNull()] [ValidateNotNullOrEmpty()] [String] $FileName,
        [Parameter(Mandatory)] [ValidateNotNull()] [ValidateNotNullOrEmpty()] [Uri] $ApiUri,
        [Parameter(Mandatory)] [ValidateNotNull()] [ValidateNotNullOrEmpty()] [String] $ApiToken
    )

    try
    {
        
        [String] $b2FileName = [System.Uri]::EscapeDataString($TargetPath)       
        [String] $b2FileMime = [System.Web.MimeMapping]::GetMimeMapping($fileName)        
        [String]$b2FileSHA1 = (Get-FileHash -Path $fileName -Algorithm SHA1).Hash
        [String]$b2FileAuthor = (Get-Acl -Path $fileName).Owner
        
        $b2FileAuthor = $b2FileAuthor.Substring($b2FileAuthor.IndexOf('\')+1)
        
        [Hashtable]$sessionHeaders = @{
            'Authorization' = $ApiToken
            'X-Bz-File-Name' = $b2FileName
            'Content-Type' = $b2FileMime
            'X-Bz-Content-Sha1' = $b2FileSHA1
            'X-Bz-Info-Author' = $b2FileAuthor
        }
        
        $b2Info = Invoke-RestMethod -Method Post -Uri $ApiUri -Headers $sessionHeaders -InFile $fileName
        
        $b2ReturnInfo = [PSCustomObject]@{
            'Name' = $b2Info.fileName
            'FileInfo' = $b2Info.fileInfo
            'Type' = $b2Info.contentType
            'Length' = $b2Info.contentLength
            'BucketID' = $b2Info.bucketId
            'AccountID' = $b2Info.accountId
            'SHA1' = $b2Info.contentSha1
            'ID' = $b2Info.fileId
        }
        
        return $b2ReturnInfo 
    }
    catch
    {
        $errorDetail = $_.Exception.Message
        Write-Error -Exception "Unable to upload the file.`n`r$errorDetail" `
            -Message "Unable to upload the file.`n`r$errorDetail" -Category InvalidOperation
    }



}



function Invoke-B2SDownload {
    Param
    (
        [Parameter(Mandatory)] [ValidateNotNull()] [ValidateNotNullOrEmpty()] [String] $BucketHost,
        [Parameter(Mandatory)] [ValidateNotNull()] [ValidateNotNullOrEmpty()] [String] $SourcePath,
        [Parameter(Mandatory)] [ValidateNotNull()] [ValidateNotNullOrEmpty()] [String] $FileName,
        [Parameter(Mandatory)] [ValidateNotNull()] [ValidateNotNullOrEmpty()] [Uri] $ApiDownloadUri,
        [Parameter(Mandatory)] [ValidateNotNull()] [ValidateNotNullOrEmpty()] [String] $ApiToken
    )

    Begin
    {
        if(-not (Test-Path -Path $FileName -IsValid))
        {
            throw 'The file path given ($FileName) is not valid.`n`rThe file cannot be saved.'
        }
        [Hashtable]$sessionHeaders = @{'Authorization'=$ApiToken}
    }
    Process
    {
        [Uri]$b2ApiUri = "${ApiDownloadUri}file/$BucketHost/$SourcePath"
            try
            {
                Invoke-RestMethod -Method Get -Uri $b2ApiUri -Headers $sessionHeaders -OutFile $FileName
            }
            catch
            {
                $errorDetail = $_.Exception.Message
                Write-Error -Exception "Unable to upload the file.`n`r$errorDetail" `
                    -Message "Unable to upload the file.`n`r$errorDetail" -Category InvalidOperation
            }
    }

}

# ==============================================================================


# Reconcile
function Set-Reconcile
{
Param( 
    [Parameter(Mandatory)][String] $ReconcileFile,
    [Parameter(Mandatory)][String] $FolderName,
    [String] $RootFolderName,
    [String] $FileFilter,
    [switch] $Feedback = $false
) 

    if ($reconcileFile -eq "")
    {
        $reconcileFile = $default_reconcileFile
    }

    if ($folderName.StartsWith("@")) {
        If (!(Test-Path -Path $folderName.Substring(1) )) {    
            Write-Log "File '$($folderName.Substring(1))' does not exist"
            Write-Host "File '$($folderName.Substring(1))' does not exist" -ForegroundColor Red
            Close-Log
            Exit
        }
    } else {
        If (!(Test-Path -Path $folderName )) {    
            Write-Log "Folder '$folderName' does not exist"
            Write-Host "Folder '$folderName' does not exist" -ForegroundColor Red
            Close-Log
            Exit
        }
    }

    Write-Log "Generating reconciliation file '$reconcileFile'"
    Write-Host "Generating reconciliation file '$reconcileFile'"
    
    $totalFileCount = 0
    $totalFileSize = 0

    if ($rootFolderName -eq "") {
        $rootFolderName = $folderName
    }
    if ($ExcludeHash) {
        $messageFrequency = 1000
    } else {
        $messageFrequency = 500
    }


    Set-Content -Path $reconcileFile  -Value '"FullName","LastWriteTime","CreationTime","LastAccessTime","Length","Hash","ParentFolder","Object","Attributes","Extension"'

    if ($folderName.StartsWith("@")) {
        Write-Log "Using @ file '$($folderName.Substring(1))'"
        Write-Host "Using @ file '$($folderName.Substring(1))'"

        Get-Content -Path $($folderName.Substring(1)) | ForEach-Object {
            if ($_ -ne "") {
                If (!(Test-Path -Path $_ )) {    
                    Write-Log "Folder/file '$($_)' does not exist"
                    Write-Host "Folder/file '$($_)' does not exist" -ForegroundColor Red
                }
                else {
                    Get-ChildItem $_ -Filter $fileFilter -Recurse | Where-Object {!$_.PSIsContainer} | ForEach-Object {

                        $totalFilecount = $totalFileCount + 1
                        $totalFileSize = $totalFileSize + $_.Length 
            
                        if (($totalFilecount % $messageFrequency) -eq 0) {            
                            Write-Log "Read $totalFileCount files and size $(Get-ConvenientFileSize -Size $totalFileSize ).  Currently at folder '$($_.Directory)' " 
                            Write-Host "Read $totalFileCount files and size $(Get-ConvenientFileSize -Size $totalFileSize ).  Currently at folder '$($_.Directory)' " 
                        }
            
                        if ($ExcludeHash) {
                            $sourceHash = ""
                        } else {
                            $sourceHash = (Get-FileHash -Path $_.FullName).Hash
                        }
                        $record = '"'+$_.FullName.Replace($rootFolderName, "")+'","'+$_.LastWriteTime.ToString("yyyy-MM-ddTHH:mm:ss")+'"'
                        $record = $record + ',"'+$_.CreationTime.ToString("yyyy-MM-ddTHH:mm:ss")+'","'+$_.LastAccessTime.ToString("yyyy-MM-ddTHH:mm:ss")+'"'
                        $record = $record + ','+$_.Length+',"'+$sourceHash+'","'+ $_.Directory + '","' + $_.Name + '","' + $_.Attributes+'","'+$_.Extension+'"'
                        Add-Content -Path  $reconcileFile  -Value $record
                    
                    }
                }
            }
        }

    } else {
        Get-ChildItem $folderName -Filter $fileFilter -Recurse | Where-Object {!$_.PSIsContainer} | ForEach-Object {

            $totalFilecount = $totalFileCount + 1
            $totalFileSize = $totalFileSize + $_.Length 

            if (($totalFilecount % $messageFrequency) -eq 0) {            
                Write-Log "Read $totalFileCount files and size $(Get-ConvenientFileSize -Size $totalFileSize ).  Currently at folder '$($_.Directory)' " 
                Write-Host "Read $totalFileCount files and size $(Get-ConvenientFileSize -Size $totalFileSize ).  Currently at folder '$($_.Directory)' " 
            }

            if ($ExcludeHash) {
                $sourceHash = ""
            } else {
                $sourceHash = (Get-FileHash -Path $_.FullName).Hash
            }
            $record = '"'+$_.FullName.Replace($rootFolderName, "")+'","'+$_.LastWriteTime.ToString("yyyy-MM-ddTHH:mm:ss")+'"'
            $record = $record + ',"'+$_.CreationTime.ToString("yyyy-MM-ddTHH:mm:ss")+'","'+$_.LastAccessTime.ToString("yyyy-MM-ddTHH:mm:ss")+'"'
            $record = $record + ','+$_.Length+',"'+$sourceHash+'","'+ $_.Directory + '","' + $_.Name + '","' + $_.Attributes+'","'+$_.Extension+'"'
            Add-Content -Path  $reconcileFile  -Value $record
        
        }

    }

    Write-Log "Total reconcile file count is $totalFileCount and size $(Get-ConvenientFileSize -Size $totalFileSize ) ($totalFileSize)"
    if ($feedback) {
        Write-Host "Total reconcile file count is $totalFileCount and size $(Get-ConvenientFileSize -Size $totalFileSize )" -ForegroundColor Green
    }
}

function Invoke-SinglePack
{
    Param( 
        [Parameter(Mandatory)][String] $ArchiveFolder,
        [Parameter(Mandatory)][String] $ArchiveFile,
        [String] $FileFilter,
        [Boolean] $FirstCompress
    ) 

    if (!(Test-Path -Path $ArchiveFolder -PathType Leaf)) {
        Write-Log "Archive folder '$ArchiveFolder'"
        Write-Host "Archivefolder '$ArchiveFolder'"
    }
    if (Test-FilesExist -FolderName $ArchiveFolder -FileFilter $FileFilter) {
        try {
            if ($FirstCompress) {
                Compress-7Zip -Path $ArchiveFolder -ArchiveFileName $ArchiveFile -Format SevenZip -PreserveDirectoryRoot -Filter $FileFilter   
            } else {
                Compress-7Zip -Path $ArchiveFolder -ArchiveFileName $ArchiveFile -Format SevenZip -PreserveDirectoryRoot -Filter $FileFilter -Append    
            }
            $FirstCompress = $false
        } catch {
            Write-Log "Compress error with folder/file '$ArchiveFolder'.  See any previous errors.  $Error"
            Write-Host "Compress error with folder/file '$ArchiveFolder'.  See any previous errors.  $Error" -ForegroundColor Red
        }
    } else {
        Write-Log "Empty folder/file '$ArchiveFolder'"
        Write-Host "Empty folder/file '$ArchiveFolder'"
    }

    return $FirstCompress
}


# Compress / Package
function Invoke-Pack
{
Param( 
    [Parameter(Mandatory)][String] $TransferFolder,
    [String] $RootFolder,
    [String] $FileFilter,
    [Parameter(Mandatory)][String] $Secret,
    [Parameter(Mandatory)][String] $CompressFile,
    [String] $ReconcileFile
) 

    if ($transferFolder.StartsWith("@")) {
        If (!(Test-Path -Path $transferFolder.Substring(1) )) {    
            Write-Log "File '$($transferFolder.Substring(1))' does not exist"
            Write-Host "File '$($transferFolder.Substring(1))' does not exist" -ForegroundColor Red
            Close-Log
            Exit
        }
    } else {
        If (!(Test-Path -Path $transferFolder )) {    
            Write-Log "Folder '$transferFolder' does not exist"
            Write-Host "Folder '$transferFolder' does not exist" -ForegroundColor Red
            Close-Log
            Exit
        }
    }

    Write-Log "Saving folders/files to archive file '$compressFile'"
    Write-Host "Saving folders/files to archive file '$compressFile'"

    if ($reconcileFile -eq "")
    {
        $reconcileFile = $default_reconcileFile
    }

    if ($fileFilter -eq "")
    {
        $fileFilter = "*"
    }

    if ($transferFolder.EndsWith("*"))
    {
        Write-Log "Archive primary folder is '$transferFolder'"
        $firstCompress = $true
        Get-ChildItem $transferFolder| ForEach-Object {
            $firstCompress = Invoke-SinglePack -ArchiveFolder $_.FullName -ArchiveFile $compressFile -FileFilter $fileFilter -FirstCompress $firstCompress
        }
    } else {
        if ($transferFolder.StartsWith("@")) {
            Write-Log "Using @ file '$($transferFolder.Substring(1))'"
            Write-Host "Using @ file '$($transferFolder.Substring(1))'"
            $firstCompress = $true

            Get-Content -Path $($transferFolder.Substring(1)) | ForEach-Object {
                if ($_ -ne "") {

                    if ($_.EndsWith("*")) {
                        Get-ChildItem $_ | ForEach-Object {
                            $firstCompress = Invoke-SinglePack -ArchiveFolder $_.FullName -ArchiveFile $compressFile -FileFilter $fileFilter -FirstCompress $firstCompress
                        }
                    } else {
                
                        If (!(Test-Path -Path $_ )) {    
                            Write-Log "Folder/file '$($_)' does not exist"
                            Write-Host "Folder/file '$($_)' does not exist" -ForegroundColor Red
                        }
                        else {
                            $firstCompress = Invoke-SinglePack -ArchiveFolder $_ -ArchiveFile $compressFile -FileFilter $fileFilter -FirstCompress $firstCompress
                        }
                    }
                }
            }
        } else {
            Write-Log "Archive folder '$transferFolder'"
            Write-Host "Archive folder '$transferFolder'"
            Compress-7Zip -Path $transferFolder -ArchiveFileName $compressFile -Format SevenZip -Filter $fileFilter    
        }
    }

    If (!(Test-Path -Path $compressFile )) {    
        Write-Log "Archive file '$compressFile' was not created.  See any previous errors"
        Write-Host "Archive file '$compressFile' was not created.  See any previous errors" -ForegroundColor Red
        Close-Log
        Exit
    }

    Set-Reconcile -ReconcileFile $reconcileFile -FolderName $transferFolder -FileFilter $fileFilter -RootFolderName $rootFolder
    If (!(Test-Path -Path $reconcileFile )) {    
        Write-Log "Reconcile file '$reconcileFile' was not created.  See any previous errors"
        Write-Host "Reconcile file '$reconcileFile' was not created.  See any previous errors" -ForegroundColor Red
        Close-Log
        return
    }

    Write-Log "Add reconcile file '$reconcileFile' to file '$compressFile'"
    $fullReconcileName = (Get-Item $reconcileFile).FullName
    $fullZipName = (Get-Item $compressFile).FullName
    Compress-7Zip -Path $fullReconcileName -ArchiveFileName $fullZipName -Format SevenZip -Append -Password $secret -EncryptFilenames
    Remove-Item $fullReconcileName

    Write-Log "Archive file '$compressFile' created from folder '$transferFolder'"
    Write-Host "Archive file '$compressFile' created from folder '$transferFolder'"  -ForegroundColor Green
}


# Send package
function Invoke-PutArchive
{
Param( 
    [Parameter(Mandatory)][String] $CompressFile,
    [Parameter(Mandatory)][String] $TargetPath,
    [String] $SecretFile,
    [String] $TargetProfile,
    [String] $AccountId,
    [String] $AccountKey
) 

    if ($compressFile -eq "") {
        Write-Log "Archive file name required" 
        Write-Host "Archive file name required"  -ForegroundColor Red
        Close-Log
        return
    }

    if ($targetProfile -eq "") {
        $getEnvName = $(Get-SoftwareName) + "_PROFILE"
        if ([System.Environment]::GetEnvironmentVariable($getEnvName) -ne "" -and $null -ne [System.Environment]::GetEnvironmentVariable($getEnvName)) {
            $targetProfile = [System.Environment]::GetEnvironmentVariable($getEnvName)
        } 
        if ($null -eq $targetProfile -or $targetProfile -eq "") {
            $targetProfile = $default_profile
        }
    }

    if (!(Test-Path -Path $compressFile )) {
        Write-Log "Archive file '$compressFile' not found"
        Write-Host "Archive file '$compressFile' not found"  -ForegroundColor Red
        Close-Log
        return
    }
    if ($secretFile -eq "") {
        $secretFile = $compressFile + ".key"
    }

    $remoteType = $false

    if ($targetPath.StartsWith("s3://")) {
        $remoteType = $true

        [int] $offset = "s3://".Length
        $parts = $targetPath.Substring($offset).Split("/")
        $bucketHost = $parts[0]
        $offset = $offset + $bucketHost.Length + 1

        if ($bucketHost -eq "") {
            Write-Log "Bucket name required" 
            Write-Host "Bucket name required"  -ForegroundColor Red
            Close-Log
            return
        }

        Set-AWSCredential -ProfileName $targetProfile

        $targetObject = $targetPath.Substring($offset)
        Write-Log "Transferring '$compressFile' file to host '$bucketHost' folder '$targetObject'"
        Write-Host "Transferring '$compressFile' file to host '$bucketHost' folder '$targetObject'"
        Write-S3Object -BucketName $bucketHost -File $compressFile -Key $targetObject
        if (Test-Path -Path $secretFile) {
            $targetObject = $targetPath.Substring($offset) + ".key"
            Write-Log "Transferring '$secretFile' file to host '$bucketHost' folder '$targetObject'"
            Write-Host "Transferring '$secretFile' file to host '$bucketHost' folder '$targetObject'"
            Write-S3Object -BucketName $bucketName -File $secretFile -Key $targetObject 
        }
        $targetObject = $targetPath.Substring($offset)
        Write-Log "Archive file '$compressFile' stored on S3 bucket '$bucketHost' at '$targetObject'"
        Write-Host "Archive file '$compressFile' stored on S3 bucket '$bucketHost' at '$targetObject'" -ForegroundColor Green

    }



    if ($targetPath.StartsWith("b2://")) {
        $remoteType = $true

        [int] $offset = "b2://".Length
        $parts = $targetPath.Substring($offset).Split("/")
        $bucketHost = $parts[0]
        $offset = $offset + $bucketHost.Length + 1

        if ($null -eq $accountId -or $accountId -eq "") {
            $accountId = $targetProfile
        }

        if ($accountKey -eq "") {
            $getEnvName = $(Get-SoftwareName) + "_ACCOUNTKEY"
            if ([System.Environment]::GetEnvironmentVariable($getEnvName) -ne "" -and $null -ne [System.Environment]::GetEnvironmentVariable($getEnvName)) {
                $accountKey = [System.Environment]::GetEnvironmentVariable($getEnvName)
            } 
            if ($null -eq $accountKey -or $accountKey -eq "") {
                Write-Log "Account key required" 
                Write-Host "Account key required"  -ForegroundColor Red
                Close-Log
                return
            }
        }

        $b2ApiToken = Get-B2ApiToken -AccountId $accountId -AccountKey $accountKey

        $b2Bucket = Get-B2Bucket -ApiToken $b2ApiToken.Token -AccountId $b2ApiToken.accountId -ApiUri $b2ApiToken.ApiUri -BucketHost $bucketHost
        if ($null -eq $b2Bucket -or $b2Bucket.BucketID -eq "") {
            Write-Log "Bucket '$bucketHost' not found" 
            Write-Host "Bucket '$bucketHost' not found" -ForegroundColor Red
            Close-Log
            return
        }

        $b2UploadUri = Get-B2UploadUri -BucketHost $b2Bucket.bucketId -FileName $compressFile -ApiUri $b2ApiToken.ApiUri -ApiToken $b2ApiToken.Token 
        $targetObject = $targetPath.Substring($offset)
        Write-Log "Transferring '$compressFile' file to host '$bucketHost' folder '$targetObject'"
        Write-Host "Transferring '$compressFile' file to host '$bucketHost' folder '$targetObject'"
        $b2Upload = Invoke-B2SUpload -BucketHost $b2UploadUri.bucketId -TargetPath $targetObject  -FileName $compressFile -ApiUri $b2UploadUri.uploadUri -ApiToken $b2UploadUri.Token
        Write-Log "Upload: $b2Upload"
        if (Test-Path -Path $secretFile) {
            $targetObject = $targetPath.Substring($offset) + ".key"
            Write-Log "Transferring '$secretFile' file to host '$bucketHost' folder '$targetObject'"
            Write-Host "Transferring '$secretFile' file to host '$bucketHost' folder '$targetObject'"
            $b2Upload = Invoke-B2SUpload -BucketHost $b2UploadUri.bucketId -TargetPath $targetObject  -FileName $secretFile -ApiUri $b2UploadUri.uploadUri -ApiToken $b2UploadUri.Token
            Write-Log "Upload: $b2Upload"
        }
        $targetObject = $targetPath.Substring($offset)
        Write-Log "Archive file '$compressFile' stored on Backblaze bucket '$bucketHost' at '$targetObject'"
        Write-Host "Archive file '$compressFile' stored on Backblaze bucket '$bucketHost' at '$targetObject'" -ForegroundColor Green

    }

    if (!($remoteType)) {
        Write-Log "Unknown remote path '$targetFolder.'.  No transfer performed" 
        Write-Host "Unknown remote path '$targetFolder.'.  No transfer performed"  -ForegroundColor Red
        Write-Host "Recognised transfer prefixes: "
        Write-Host "    s3://         : Send to S3 compatible location"
        Write-Host " "    
        Write-Host "If you are saving to local drives or network shared folders,"    
        Write-Host "please use your OS tools to move the file"    
    Write-Host " "    
    }


}


# Receive package
function Invoke-GetArchive
{
Param( 
    [Parameter(Mandatory)][String] $CompressFile,
    [Parameter(Mandatory)][String] $SourcePath,
    [String] $SecretFile,
    [String] $SourceProfile,
    [String] $AccountId,
    [String] $AccountKey
) 
    
        if ($compressFile -eq "") {
            Write-Log "Archive file name required" 
            Write-Host "Archive file name required"  -ForegroundColor Red
            Close-Log
            return
        }
    
        if ($sourceProfile -eq "") {
            $getEnvName = $(Get-SoftwareName) + "_PROFILE"
            if ([System.Environment]::GetEnvironmentVariable($getEnvName) -ne "" -and $null -ne [System.Environment]::GetEnvironmentVariable($getEnvName)) {
                $sourceProfile = [System.Environment]::GetEnvironmentVariable($getEnvName)
            } 
            if ($null -eq $sourceProfile -or $sourceProfile -eq "") {
                $sourceProfile = $default_profile
            }
        }
    
    
        $remoteType = $false
    
        if ($sourcePath.StartsWith("s3://")) {
            $remoteType = $true

            [int] $offset = "s3://".Length
            $parts = $sourcePath.Substring($offset).Split("/")
            $bucketHost = $parts[0]
            $offset = $offset + $bucketHost.Length + 1

            Set-AWSCredential -ProfileName $sourceProfile
    
            $sourceObject = $sourcePath.Substring($offset)
            Write-Log "Fetching '$compressFile' file from host $bucketHost folder $sourceObject"
            Write-Host "Fetching '$compressFile' file from host $bucketHost folder $sourceObject"
            $null = Read-S3Object -BucketName $bucketHost -File $compressFile -Key $sourceObject
            if (!(Test-Path -Path $compressFile)) {
                Write-Log "Archive file '$sourceObject' not found." 
                Write-Host "Archive file '$sourceObject' not found."  -ForegroundColor Red
            } else {
                $sourceObject = $sourcePath.Substring($offset) + ".key"
                $secretFile = $compressFile + ".key"
                Write-Log "Fetching '$secretFile' file from host '$bucketHost' folder '$sourceObject'"
                Write-Host "Fetching '$secretFile' file from host '$bucketHost' folder '$sourceObject'"
                $null = Read-S3Object -BucketName $bucketHost -File $secretFile -Key $sourceObject 
                if (!(Test-Path -Path $secretFile)) {
                    Write-Log "Secret file '$sourceObject' not found. Required if you are using recipient keys" 
                    Write-Host "Secret file '$sourceObject' not found. Required if you are using recipient keys" 
                }
                $sourceObject = $sourcePath.Substring($offset)
                Write-Log "Archive file '$compressFile' fetched from S3 '$sourcePath'"
                Write-Host "Archive file '$compressFile' fetched from S3 '$sourcePath'" -ForegroundColor Green
            }
    
        }
    

    
        if ($sourcePath.StartsWith("b2://")) {
            $remoteType = $true

            [int] $offset = "b2://".Length
            $parts = $sourcePath.Substring($offset).Split("/")
            $bucketHost = $parts[0]
            $offset = $offset + $bucketHost.Length + 1

            if ($null -eq $accountId -or $accountId -eq "") {
                $accountId = $sourceProfile
            }
    
            if ($accountKey -eq "") {
                $getEnvName = $(Get-SoftwareName) + "_ACCOUNTKEY"
                if ([System.Environment]::GetEnvironmentVariable($getEnvName) -ne "" -and $null -ne [System.Environment]::GetEnvironmentVariable($getEnvName)) {
                    $accountKey = [System.Environment]::GetEnvironmentVariable($getEnvName)
                } 
                if ($null -eq $accountKey -or $accountKey -eq "") {
                    Write-Log "Account key required" 
                    Write-Host "Account key required"  -ForegroundColor Red
                    Close-Log
                    return
                }
            }
    
            $b2ApiToken = Get-B2ApiToken -AccountId $accountId -AccountKey $accountKey

            $b2Bucket = Get-B2Bucket -ApiToken $b2ApiToken.Token -AccountId $b2ApiToken.accountId -ApiUri $b2ApiToken.ApiUri -BucketHost $bucketHost
            if ($null -eq $b2Bucket -or $b2Bucket.BucketID -eq "") {
                Write-Log "Bucket '$bucketHost' not found" 
                Write-Host "Bucket '$bucketHost' not found" -ForegroundColor Red
                Close-Log
                return
            }
    
            $sourceObject = $sourcePath.Substring($offset)
            Write-Log "Fetching '$compressFile' file from host '$bucketHost' folder '$sourceObject'"
            Write-Host "Fetching '$compressFile' file from host '$bucketHost' folder '$sourceObject'"
            Invoke-B2SDownload -BucketHost $bucketHost -SourcePath $sourceObject -FileName $compressFile -ApiDownloadUri $b2ApiToken.DownloadUri -ApiToken $b2ApiToken.Token
            if (!(Test-Path -Path $compressFile)) {
                Write-Log "Archive file '$sourceObject' not found." 
                Write-Host "Archive file '$sourceObject' not found."  -ForegroundColor Red
            } else {
                $sourceObject = $sourcePath.Substring($offset) + ".key"
                $secretFile = $compressFile + ".key"
                Write-Log "Fetching '$secretFile' file from host '$bucketHost' folder '$sourceObject'"
                Write-Host "Fetching '$secretFile' file from host '$bucketHost' folder '$sourceObject'"
                Invoke-B2SDownload -BucketHost $bucketHost -SourcePath $sourceObject -FileName $secretFile -ApiDownloadUri $b2ApiToken.DownloadUri -ApiToken $b2ApiToken.Token
                if (!(Test-Path -Path $secretFile)) {
                    Write-Log "Secret file '$sourceObject' not found. Required if you are using recipient keys" 
                    Write-Host "Secret file '$sourceObject' not found. Required if you are using recipient keys" 
                }
                $sourceObject = $sourcePath.Substring($offset)
                Write-Log "Archive file '$compressFile' fetched from Backblaze '$sourcePath'"
                Write-Host "Archive file '$compressFile' fetched from Backblaze '$sourcePath'" -ForegroundColor Green
            }
        
        }


        if (!($remoteType)) {
            Write-Log "Unknown remote path '$sourcePath'.  No get performed" 
            Write-Host "Unknown remote path '$sourcePath'.  No get performed"  -ForegroundColor Red
            Write-Host "Recognised transfer prefixes: "
            Write-Host "    s3://bucket/path/path     : Fetch from S3 compatible location"
            Write-Host "    b2://bucket/path/path     : Fetch from Backblaze location"
            Write-Host " "    
            Write-Host "If you are fetching from local drives or network shared folders,"    
            Write-Host "please use your OS tools to move the file"    
            Write-Host " "    
        }
    

}


# Unpack package
function Invoke-Unpack
{
Param( 
    [Parameter(Mandatory)][String] $RestoreFolder,
    [Parameter(Mandatory)][String] $Secret,
    [Parameter(Mandatory)][String] $CompressFile
) 

    If (!(Test-Path -Path $CompressFile )) {    
        Write-Log "Archive file '$CompressFile' does not exist"
        Write-Host "Archive file '$CompressFile' does not exist" -ForegroundColor Red
        Close-Log
        Exit
    }

    Write-Log "Restoring files transferred to '$restoreFolder'"
    Write-Log "Archive file is '$compressFile'"

    # Uncompress the data files
    Expand-7Zip -ArchiveFileName $compressFile -TargetPath $restoreFolder -Password $secret
    Write-Log "Contents unpacked from archive file '$compressFile' to folder '$restoreFolder'"
    Write-Host "Contents unpacked from archive file '$compressFile' to folder '$restoreFolder'" -ForegroundColor Green
}


# Reconcile
function Invoke-Reconcile
{
Param( 
    [Parameter(Mandatory)][String] $ReconcileFile,
    [Parameter(Mandatory)][String] $Folder,
    [String] $TargetReconcileFile,
    [String] $RootFolder,
    [Switch] $ExtendedCheck
) 

    if ($reconcileFile -eq "")
    {
        $reconcileFile = $default_reconcileFile
    }

    Write-Log "Reconciling documents transferred"
    Write-Host "Reconciling documents transferred"
    If (!(Test-Path -Path $reconcileFile )) {    
        Write-Log "Reconciliation file '$reconcileFile' does not exist"
        Write-Host "Reconciliation file '$reconcileFile' does not exist" -ForegroundColor Red
        Close-Log
        Exit
    }
    If (!(Test-Path -Path $folder )) {    
        Write-Log "Folder '$folder' does not exist"
        Write-Host "Folder '$folder' does not exist" -ForegroundColor Red
        Close-Log
        Exit
    }
    Write-Log "Using reconciliation file '$reconcileFile'"
    
    $totalFileCount = 0
    $totalFileSize = 0
    $errorCount = 0
    $errorCreateCount = 0
    $missingFileCount = 0
    $missingHash = $false

    # For each entry in the reconcile file
    #     find the file and compare hash
    Import-Csv $reconcileFile | ForEach-Object {
        $totalFileCount = $totalFileCount +1 
        if ($rootFolder -ne "") {
            $adjustedName = $_.FullName.Replace($rootFolder, "\")
            $restoreFileName = $(Join-Path -Path $folder -ChildPath $adjustedName)    
        } else {
            $restoreFileName = $(Join-Path -Path $folder -ChildPath $_.FullName)    
        }
        If (Test-Path -Path $restoreFileName ) {    
            if ($_.Hash -ne "") {
                $targetHash= (Get-FileHash -Path $restoreFileName).Hash
                if ($_.Hash -ne $targetHash) {
                    $errorCount = $errorCount + 1
                    Write-Log "Hash mismatch for file '$restoreFileName' with target value $targetHash"
                }
            } else {
                $missingHash = $true
            }
            if ((Get-Item -Path $restoreFileName).CreationTime.ToString("yyyy-MM-ddTHH:mm:ss") -ne $_.CreationTime) {
                Write-Log "Creation mismatch for file '$restoreFileName' with target value $((Get-Item -Path $restoreFileName).CreationTime.ToString("yyyy-MM-ddTHH:mm:ss"))"
                $errorCreateCount = $errorCreateCount + 1

                $dateTimeValue = [Datetime]::ParseExact($_.CreationTime, 'yyyy-MM-ddTHH:mm:ss', $null)
                $fileValue = (Get-Item -Path $restoreFileName).CreationTime
                $diff = ($dateTimeValue - $fileValue).Seconds
                # Allow +/- 2 second discrepancy
                if (($diff.Seconds -lt -2) -or ($diff.Seconds -gt 2)) {
                    $errorCount = $errorCount + 1
                }
            }
            if ((Get-Item -Path $restoreFileName).Length -ne $_.Length) {
                $errorCount = $errorCount + 1
                Write-Log "Length mismatch for file '$restoreFileName' with target value $(Get-Item -Path $restoreFileName).Length)"
            }

            # Note that last / write access time is not checked by default as it will comonly be changed after restore
            if ($extendedCheck) {
                if ((Get-Item -Path $restoreFileName).LastAccessTime.ToString("yyyy-MM-ddTHH:mm:ss") -ne $_.LastAccessTime) {
                    $errorCount = $errorCount + 1
                    Write-Log "Last access mismatch for file '$restoreFileName' with target value $((Get-Item -Path $restoreFileName).LastAccessTime.ToString("yyyy-MM-ddTHH:mm:ss"))"
                }
                if ((Get-Item -Path $restoreFileName).LastWriteTime.ToString("yyyy-MM-ddTHH:mm:ss") -ne $_.LastWriteTime) {
                    $errorCount = $errorCount + 1
                    Write-Log "Last write mismatch for file '$restoreFileName' with target value $((Get-Item -Path $restoreFileName).LastWriteTime.ToString("yyyy-MM-ddTHH:mm:ss"))"
                }
            }

            $totalFileSize = $totalFileSize + (Get-Item -Path $restoreFileName).Length             
        } else {
            $missingFileCount = $missingFileCount + 1
            $errorCount = $errorCount + 1
            Write-Log "Non existant target file '$restoreFileName'"
        }
    }

    Write-Log "Total file storage size is $(Get-ConvenientFileSize -Size $totalFileSize ) ($totalFileSize)"
    Write-Host "Total file storage size is $(Get-ConvenientFileSize -Size $totalFileSize )"

    if ($missingHash)
    {
        Write-Log "Reconcile file had one or many or all blank hash entries"
        Write-Host "Reconcile file had one or many or all blank hash entries"  -ForegroundColor Yellow
    }

    Write-Log "Total file count is $totalFileCount with $errorCount errors"
    Write-Log "There are $missingFileCount missing files"

    if ($errorCreateCount -gt 0) {
        Write-Log "File create mismatch count is $errorCreateCount" 
        Write-Host "File create mismatch count is $errorCreateCount" -ForegroundColor Red
    }

    if ($errorCount -gt 0) {
        Write-Host "Total file count is $totalFileCount with $errorCount errors" -ForegroundColor Red
    } else {
        Write-Host "Total file count is $totalFileCount with $errorCount errors"  -ForegroundColor Green
    }
    if ($missingFileCount -gt 0) {
        Write-Host "There are $missingFileCount missing files" -ForegroundColor Red
    }
}


# Main code logic starts here
function Invoke-Main {
    
    $actioned = $false

    if ($action -eq "Install") {
        $actioned = $true
        if ($cloudProfile -eq "") {
            Install-Module -Name 7Zip4Powershell -Scope CurrentUser
            Install-Module -Name AWS.Tools.Installer -Scope CurrentUser
            Install-Module -Name AWS.Tools.S3  -Scope CurrentUser    
        } else {
            Install-Module -Name 7Zip4Powershell -Scope $cloudProfile
            Install-Module -Name AWS.Tools.Installer -Scope $cloudProfile
            Install-Module -Name AWS.Tools.S3  -Scope $cloudProfile
        }
    }

    if ($action -eq "Pack") {
        $actioned = $true

        if ($RecipientKeyName -eq "") {
            $getEnvName = $(Get-SoftwareName) + "_RECIPIENTKEYNAME"
            if ([System.Environment]::GetEnvironmentVariable($getEnvName) -ne "" -and $null -ne [System.Environment]::GetEnvironmentVariable($getEnvName)) {
                $RecipientKeyName = [System.Environment]::GetEnvironmentVariable($getEnvName)
            }
        }

        if (($RecipientKeyName -eq "") -and ($SecretKey -eq "")) {
            Write-Log "Recipient Key Name or Secret Key required for packing" 
            Write-Host "Recipient Key Name or Secret Key required for packing"  -ForegroundColor Red
            Close-Log
            return
        } 
        
        if ($rootFolderName -eq "") {
            if ($path.EndsWith("*")) {
                Write-Log "Root folder required for packing when using wild card for Path" 
                Write-Host "Root folder required for packing when using wild card for Path"  -ForegroundColor Red
                Close-Log
                return
            } else {
                $rootFolderName = $path
            }
        }

        if ($ArchiveFileName -eq "") {
            $ArchiveFileName = $default_archiveFile.Replace("##date##", $default_dateLocal)
        }

        if ($SecretKey -eq "") {
            if ($secretFileName -eq "")
            {
                $secretFileName = $ArchiveFileName + ".key"
            }
            $secret = New-RandomPassword -Length 80
            Protect-CmsMessage -To $recipientKeyName -OutFile $secretFileName -Content $secret 
        } else {
            $secret = $SecretKey
        }

        Invoke-Pack -TransferFolder $path -Secret $secret -CompressFile $ArchiveFileName -ReconcileFile $reconcileFileName -RootFolder $rootFolderName -FileFilter $fileFilter
    }


    if ($action -eq "Put") {
        $actioned = $true
        
        if ($ArchiveFileName -eq "") {
            Write-Log "Archive file name required" 
            Write-Host "Archive file name required"  -ForegroundColor Red
            Close-Log
            return
        }

        if (!(Test-Path -Path $ArchiveFileName )) {
            Write-Log "Archive file '$ArchiveFileName' not found"
            Write-Host "Archive file '$ArchiveFileName' not found"  -ForegroundColor Red
            Close-Log
            return
        }

        Invoke-PutArchive -CompressFile $archiveFileName -TargetPath $path -SecretFile $secretFileName -TargetProfile $cloudProfile
    }


    if ($action -eq "Get") {
        $actioned = $true
        
        if ($ArchiveFileName -eq "") {
            Write-Log "Archive file name required" 
            Write-Host "Archive file name required"  -ForegroundColor Red
            Close-Log
            return
        }
        
        Invoke-GetArchive -CompressFile $archiveFileName -SourcePath $path -SecretFile $secretFileName -SourceProfile $cloudProfile
    }


    if ($action -eq "Unpack") {
        $actioned = $true

        if ($RecipientKeyName -eq "") {
            $getEnvName = $(Get-SoftwareName) + "_RECIPIENTKEYNAME"
            if ([System.Environment]::GetEnvironmentVariable($getEnvName) -ne "" -and $null -ne [System.Environment]::GetEnvironmentVariable($getEnvName)) {
                $RecipientKeyName = [System.Environment]::GetEnvironmentVariable($getEnvName)
            }
        }

        if (($RecipientKeyName -eq "") -and ($SecretKey -eq "")) {
            Write-Log "Recipient Key Name or Secret Key required for unpacking" 
            Write-Host "Recipient Key Name or Secret Key required for unpacking" -ForegroundColor Red
            Close-Log
            return
        } 
        if ($ArchiveFileName -eq "") {
                Write-Log "Archive file Name required for unpacking" 
                Write-Host "Archive file Name required for unpacking" -ForegroundColor Red
                Close-Log
                return
        } 
        
        if ($SecretKey -eq "") {
            if ($secretFileName -eq "")
            {
                $secretFileName = $ArchiveFileName + ".key"
            }
            $secret = Unprotect-CmsMessage -To $recipientKeyName -Path $secretFileName
        } else {
            $secret = $SecretKey
        }
        Invoke-Unpack -RestoreFolder $path -Secret $secret -CompressFile $ArchiveFileName
        
    }


    if ($action -eq "ReconcileFile") {
        $actioned = $true
        if ($reconcileFileName -eq "")
        {
            $reconcileFileName = $default_reconcileFile
        }
        Set-Reconcile -ReconcileFile $reconcileFileName -FolderName $path -Feedback -RootFolderName $rootFolderName -FileFilter $fileFilter
    }


    if ($action -eq "Reconcile") {
        $actioned = $true
        if ($reconcileFileName -eq "")
        {
            $reconcileFileName = $default_reconcileFile
        }
        $localReconcileFile = Join-Path -Path $path -ChildPath $reconcileFileName
        Invoke-Reconcile -ReconcileFile $localReconcileFile -Folder $path -RootFolder $rootFolderName
    }

    if ($action -eq "ArchiveInformation") {
        $actioned = $true
        if (($RecipientKeyName -eq "") -and ($SecretKey -eq "")) {
            Write-Log "Recipient Key Name or Secret Key required for 7Zip information" 
            Write-Host "Recipient Key Name or Secret Key required for 7Zip information"  -ForegroundColor Red
            Close-Log
            return
        } 
        
        if ($SecretKey -eq "") {
            if ($secretFileName -eq "")
            {
                $secretFileName = $ArchiveFileName + ".key"
            }
            $secret = Unprotect-CmsMessage -To $recipientKeyName -Path $secretFileName
        } else {
            $secret = $SecretKey
        }
        Write-Log "Retrieving archive information"
        Write-Host "Retrieving archive information"
        
        Get-7ZipInformation -ArchiveFileName $ArchiveFileName -Password $secret
    }


    if ($action -eq "MakeCert") {
        $actioned = $true
        if (($RecipientKeyName -eq "") -and ($SecretKey -eq "")) {
            Write-Log "Recipient Key Name required to create a standard certificate" 
            Write-Host "Recipient Key Name required to create a standard certificate"  -ForegroundColor Red
            Close-Log
            return
        } 
        if ($Path -ne "Cert:\CurrentUser\My") {
            Write-Log "The -Path value needs to be 'Cert:\CurrentUser\My'" 
            Write-Host "The -Path value needs to be 'Cert:\CurrentUser\My'"  -ForegroundColor Red
            Close-Log
            return
        } 

        Write-Log "Making a file encryption certificate"
        Write-Host "Making a file encryption certificate"
        
        New-SelfSignedCertificate -Subject $RecipientKeyName -KeyFriendlyName $RecipientKeyName -DnsName $RecipientKeyName -CertStoreLocation $Path -KeyUsage KeyEncipherment,DataEncipherment, KeyAgreement -Type DocumentEncryptionCert
    }


    if ($action -eq "ListCert") {
        $actioned = $true
        if ($Path -ne "Cert:\CurrentUser\My") {
            Write-Log "The -Path value needs to be 'Cert:\CurrentUser\My'" 
            Write-Host "The -Path value needs to be 'Cert:\CurrentUser\My'"  -ForegroundColor Red
            Close-Log
            return
        } 

        Write-Log "Listing encryption certificates"
        Write-Host "Listing encryption certificates"
        
        if ($RecipientKeyName -eq "")
        {
            Get-Childitem -Path $Path -DocumentEncryptionCert
        } else {
            Write-Host ""
            Write-Host "   PSParentPath: Microsoft.PowerShell.Security\Certificate::$Path"
            Write-Host ""
            Write-Host "Thumbprint                                Subject"
            Write-Host "----------                                -------"
            Get-Childitem -Path $Path -DocumentEncryptionCert | ForEach-Object {
                if ($_.Subject -eq ("CN=$RecipientKeyName"))
                {
                    Write-Host "$($_.Thumbprint)  $($_.Subject)"
                }
            }
        }
    }


    if (!($actioned))
    {
        Write-Log "Unknown action '$action'.  No processing performed" 
        Write-Host "Unknown action '$action'.  No processing performed"  -ForegroundColor Red
        Write-Host "Recognised actions: "
        Write-Host "    Pack                 : Pack folder contents into secure 7Zip file"
        Write-Host "    Put                  : Put or send the archive file to remote destination"
        Write-Host "    Get                  : Get or fetch the archive from remote location"
        Write-Host "    Unpack               : Unpack folder contents from secure 7Zip file"
        Write-Host "    Reconcile            : Reconcile files in unpack folder with list of packed files"
        Write-Host "    ReconcileFile        : Generate a reconcile file without packing"
        Write-Host "    Install              : Install required packages"
        Write-Host "    ArchiveInformation   : Fetch archive information from archive file"
        
        Write-Host ""
        Write-Host "For help use command "
        Write-Host "    Get-Help .\ptrFiles.ps1"
    }

    Close-Log
}


$dateTimeStart = Get-Date -f "yyyy-MM-dd HH:mm:ss"
Write-Log "***********************************************************************************"
Write-Log "*   Start of processing: [$dateTimeStart]"
Write-Log "***********************************************************************************"


Write-Log "Script parameters follow"
ForEach ($boundParam in $PSBoundParameters.GetEnumerator())
{
    if ($boundParam.Key -eq "SecretKey") {
        Write-Log "Parameter: $($boundParam.Key)   Value: ************** "
    } else {
        Write-Log "Parameter: $($boundParam.Key)   Value: $($boundParam.Value) "
    }
}
Write-Log ""


Invoke-Main