Issues related to the yaml.v3 library

[braydonk]

Hello yamlfmt users!

I wanted to make a discussion post to explain the relationship between yamlfmt and... formatting yaml.

TL;DR

Fixing bugs in yaml.v3 is hard and time consuming. If you know your stuff about yaml and/or Go and are willing to contribute some fixes, please give it a go and reach out if you have issues!

yaml.v3

Despite what the name would make you think, yamlfmt itself does not implement any actual yaml parsing or formatting itself. Instead, I leveraged the library gopkg.in/yaml.v3 (or yaml.v3 for short), the most accurate yaml library for Go that I am personally aware of. That library is the engine for any actual yaml interaction, except for the minor hacks implemented to work around it (most of which has been fixed, only one remains; see the internal/hotfix package).

yaml.v3 is, in my eyes, an idle project. Maintainers don't accept contributions and (at least when I checked) don't respond to issues. To be clear, I think this is fine and I mean no ill will to the maintainers of the project. They don't owe anything on it. However, it did mean that what I opted to do was fork the library to resolve some features that were causing problems for yamlfmt users that I would be incapable of pushing upstream. When I did this, I was optimistic for the future and felt like I would be able to push even greater changes using yaml.v3 as a basis.

yaml.v3 at it's core is a Go port of libyaml, a C library. Modifying this library is very difficult. It is enough of a time-sink to come up with solutions for some of the challenging parsing edge-cases, so the idea of adding new features is essentially not even on the table in my current scenario.

Open yaml issues

This is why there are a number of open issues on this repo that are related to problems that you might think a yaml formatting tool would be able to solve. In reality, this tool is more of an orchestration around an existing yaml parser, and when that parser has some kind of issues it is very challenging for me to intervene.

The following are the issues that are related to bugs or shortcomings in the yaml.v3 library:
#74
#86
#110
#112
#131
#132
#137
#142
#147

I'm opening the floor to contributors here. If you are good with Go, or with parsers, or with libyaml for that matter, and you are able to resolve these issues, I would welcome any assistance on my fork, https://github.com/braydonk/yaml. I added a new package there called formattest where it should be reasonably easy to add test cases related to whatever you are fixing.

For those still affected by these issues that I can't easily fix, I do apologize. It is a bit of a strange scenario where the guy who made a tool called yaml fumt can't easily fix the issues it has fumting the yaml.

[TheSpiritXIII]

Thanks for the fabulous tool! Just wanted to point out that the Kubernetes team (I'm not affiliated with them) decided to fork yaml.v3 as well: kubernetes-sigs/kustomize#4033 (https://github.com/kubernetes-sigs/yaml)

[braydonk]

Interesting, thanks for letting me know! I'll look through their fork and see if there's anything there that may help address the existing issues, that would be awesome.

[stevehipwell]

@braydonk I'm not sure if you looked into this further but it looks like you could use the forked go-yaml.v3 package in kubernetes-sigs/yaml.

[braydonk]

I didn't look into it too deeply, it is possible that some of the patches they had done might end up being useful in the fork yamlfmt uses. However, the fork yamlfmt uses has added a bunch of features that would not get accepted based on this opening section of the README, so even if it were based on this version I would still need a fork managed specifically for this tool.

There are also a small number of people using braydonk/yaml for the fixes/features I've added, and while I haven't specifically promised anything to those users I would like to keep it around for that reason.