Use a development branch for API breaking changes towards v0.8.0 and beyond

[CameronTStark]

Is your feature request related to a problem? Please describe.
In discussing the next release in #955 two difficulties have arisen:

1. An uncertainty in how to describe releases via semantic versioning (bugfix release E.g. 0.7.3 vs minor release E.g. v0.8.0).
2. Releasing incremental, API breaking changes along with bugfixes/enhancements.

Describe the solution you'd like
Utilize a development branch to centralize API breaking changes towards the goals of the v1.0.0.

Bugfixes and enhancements would still be merged straight into master as they currently are. Such changes are more straightforward and could be released to PyPI faster without concern of self-afflicted API changes and could enable a more bug-free and feature-filled PyPI version available for the masses.

Describe alternatives you've considered
Mark API breaking PRs as v0.8.0 and wait to merge them until they're all ready.

This would not allow the community a chance to tryout these changes ahead of time and would be difficult for developers to ensure API consistency.

Additional context
It looks like this was used in the past #178. Was there a reason for moving away from this workflow?

[wholmgren]

If we have to have multiple branches then I'd rather keep master on the bleeding edge and create bug fix branches for the older release series. But I think when looked at from that point of view it's clear that we don't have the person-power or development speed to make multiple branches worthwhile.

I think the major impediment is our own lack of will to push API changes faster.

This would not allow the community a chance to tryout these changes ahead of time and would be difficult for developers to ensure API consistency.

The community can more easily try out changes by simply using the master version on github. This is common practice for CI configurations for libraries with more infrastructure (e.g. pandas tests against numpy master).

[adriesse]

I think the major impediment is our own lack of will to push API changes faster.

We certainly do have the "Will" to push API changes! :)

[CameronTStark]

If we have to have multiple branches then I'd rather keep master on the bleeding edge and create bug fix branches for the older release series. But I think when looked at from that point of view it's clear that we don't have the person-power or development speed to make multiple branches worthwhile.

What I'm suggesting would simply make the development branch a single feature branch that could be managed by the team we have available.

I think the major impediment is our own lack of will to push API changes faster.

I agree with you and don't see a way to getting it done faster which is why I'm suggesting this path.

The community can more easily try out changes by simply using the master version on github. This is common practice for CI configurations for libraries with more infrastructure (e.g. pandas tests against numpy master).

From what I've witnessed, the current mode of operation is to push partial API improvements toward v0.8.0 within bugfix releases and decide that the API has changed enough to warrant a change to a minor release. That causes bugfixes and enhancements to take a long time to get to PyPI because there's fear that if we release API breaking changes every month then we'll lose users. By tucking them away and releasing them when they're finished in their entirety we could reasonably release bugfix releases monthly (v0.7.x) and minor releases with API improvements quarterly or even semi-annually (v0.x.0). This way we know what to release when and the community knows when significant changes have happened that they need to look out for. I otherwise have a hard time figuring it all out.

Maybe I'm making it too hard? 🤷‍♂️

[mikofski]

IMO a dev branch is not necessary. We have not reached v1.0 so breaking changes should be expected. Anyone who doesn't want to deal with breaking changes is free to use an older version. If they want new features they can manage the upgrades.

We already help users with deprecation cycles. We can help them more with better documentation and migration guides.

IMHO breaking changes only becomes an issue after v1.0, when we may choose to support multiple release branches such as a long term support (LTS) release to which we backport new features without making API changes. But I think that's a bridge we can cross when we get there.

My feelings on this are not set in stone, I could convinced otherwise, but I think those who would bare the greatest burden should have the most say. Will & Cliff

If our goal is to grow user base, there may be other concerns to users higher priority than breaking API changes. If it were up to me, I would invest in better engagement. Surveys, user group meeting, discourse or other modern forum, etc. I'm not convinced that a dev branch will deliver the expected impact for what it would cost in effort.

[CameronTStark]

I'd like to first clarify that the intention for using a development branch is to manage API breaking changes together as a group of merged PRs not to move all development to this branch. It would be more correct to call it a feature branch. Such a branch would be named to indicate its purpose and be temporary. It's main purpose would be to group API breaking changes and contain them through a "third-digit" bug release(s) toward a "second-digit" minor release. My apologies for being less than clear.

Last Friday in a phone conversation between @cwhanse, @wholmgren and I, we discussed possible scenarios for this method. There we decided that we would push toward v0.8.0 now without such a branch but potentially implement one for future API changes if we decide it's useful and worth the effort which will be determined at a later date.

That resolves my reason for raising this issue and so I'll close this issue out. If there's further discussion I'll be happy to reopen.

Thanks to all for their insight!