Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Suggestion: Multi document output #187

Open
pacorreia opened this issue Feb 27, 2025 · 4 comments · May be fixed by #188
Open

Suggestion: Multi document output #187

pacorreia opened this issue Feb 27, 2025 · 4 comments · May be fixed by #188

Comments

@pacorreia
Copy link

First of all awesome work with this module!

It really helped me a lot, but I had to find a workaround for saving multi documents.

My use case is very simple, a K8s manifest with several resources defined.

I Import an existing yaml file, do my changes and by the time I have to save it to a new yaml file this is what I use:

function Save-ArrayAsYamlFile {
<#
.SYNOPSIS
    Saves an array of objects as a YAML file.

.PARAMETER InputObject
    The array of objects to save as a YAML file.

.PARAMETER OutputPath
    The path to save the YAML file.

.EXAMPLE
    Save-ArrayAsYamlFile -InputObject $array -OutputPath "path/to/output.yaml"
#>
  param(
    [Parameter(Mandatory = $true, Position = 0, ValueFromPipeline = $true, ValueFromPipelineByPropertyName = $true, HelpMessage = "The array of objects to save as a YAML file.")]
    [array]$InputObject,

    [Parameter(Mandatory = $true, Position = 1, ValueFromPipeline = $true, ValueFromPipelineByPropertyName = $true, HelpMessage = "The path to save the YAML file.")]
    [string] $OutputPath
  )

  $ErrorActionPreference = "Stop"

  try {
    Import-YamlModule -Force -NoClobber

    Write-Debug "Document count: $($InputObject.Count)"
    Write-Debug "Document array: $InputObject"

    for ($i = 0; $i -lt $InputObject.Count; $i++) {
      if ($i -eq 0) {
        Write-Debug ("Writing the first document to '$OutputPath' as:`n{0}`n" -f ($InputObject[$i] | ConvertTo-Yaml) )

        $InputObject[$i] | ConvertTo-Yaml | Out-File -Path $OutputPath -Force

        Write-Debug "Content saved."
      }
      else {
        Write-Debug ("Appending the next document to '$OutputPath' as:`n{0}`n" -f ($InputObject[$i] | ConvertTo-Yaml) )

        "---" | Out-File -Path $OutputPath -Append

        $InputObject[$i] | ConvertTo-Yaml | Out-File -Path $OutputPath -Append

        Write-debug "Content appended."
      }
    }
    Write-Debug "Updated deployment manifest has been written to '$OutputPath'."
  }
  catch {
    Write-Error -Message "Error in script $($MyInvocation.MyCommand.Name) at line $($MyInvocation.ScriptLineNumber): $($_.Exception.Message)"
    throw $_
  }
}

Found that when converting from YAML, I get an object of type System.Array, indexed by [int]

So was easy for me to be lazy and generate the YAML for each item at a time, add the "---" at the end, save to the file and keep going

Hope it can serve as base for a much more elegant solution, for now, this one just works like charm for me
@gabriel-samfira
Copy link
Member

Hi @pacorreia

The issue of outputting multiple documents has been brought up before and the decision back then was to wait and see.

The current implementation of ConvertTo-Yaml allows you to pipe objects to the commandlet. When you do this, the input that comes in through the pipe is iterated over. So if you pipe an array, an action is taken for each object in the array.

We work around this by using the BEGIN, PROCESS and END blocks of an "advanced" powershell commandlet. But a simple array is indistinguishable from an array of documents. So the result you get will probably not look like anything you would expect.

We can add a new commandlet as you suggested, but I believe we could potentially have a cleaner solution to this. We can define a new custom type like PowershellYamlMultiDocument (I am terrible at naming things), which would represent an array of objects that would be serialized as a single document.

We could then have ConvertTo-Yaml recognize that object as a collection of objects that need to be outputted as a single multi-document file. Some checks would need to be done in the END block of the commandlet to make sure we don't pipe in a mix of PowershellYamlMultiDocument and non PowershellYamlMultiDocument objects, and then output the result.

This would allow us to still use ConvertTo-Yaml as the sole commandlet for serializing to yaml.

What do you think?

@pacorreia
Copy link
Author

Hi @pacorreia

The issue of outputting multiple documents has been brought up before and the decision back then was to wait and see.

The current implementation of ConvertTo-Yaml allows you to pipe objects to the commandlet. When you do this, the input that comes in through the pipe is iterated over. So if you pipe an array, an action is taken for each object in the array.

We work around this by using the BEGIN, PROCESS and END blocks of an "advanced" powershell commandlet. But a simple array is indistinguishable from an array of documents. So the result you get will probably not look like anything you would expect.

We can add a new commandlet as you suggested, but I believe we could potentially have a cleaner solution to this. We can define a new custom type like PowershellYamlMultiDocument (I am terrible at naming things), which would represent an array of objects that would be serialized as a single document.

We could then have ConvertTo-Yaml recognize that object as a collection of objects that need to be outputted as a single multi-document file. Some checks would need to be done in the END block of the commandlet to make sure we don't pipe in a mix of PowershellYamlMultiDocument and non PowershellYamlMultiDocument objects, and then output the result.

This would allow us to still use ConvertTo-Yaml as the sole commandlet for serializing to yaml.

What do you think?

Hey,

Got you perfectly, and knew it was about the type, indeed is hard to distinguish where to draw a 'line' between each documento.

Your suggestion looks sensible to me and would work.

@gabriel-samfira
Copy link
Member

I'll try to create a proof of concept soon. And we can discuss on the PR.

@pacorreia
Copy link
Author

Reach out if you feel like discussing it before the PR

@gabriel-samfira gabriel-samfira linked a pull request Feb 28, 2025 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[WiP] Add some multi-document output support gabriel-samfira/powershell-yaml
2 participants
@gabriel-samfira @pacorreia