Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Arrange How We'll Work Here and Collaborate With Upstream #2

Open
RandomDSdevel opened this issue Jan 18, 2019 · 20 comments
Open

Arrange How We'll Work Here and Collaborate With Upstream #2

RandomDSdevel opened this issue Jan 18, 2019 · 20 comments
Labels
enhancement New feature or request help wanted Extra attention is needed

Comments

@RandomDSdevel
Copy link
Member

RandomDSdevel commented Jan 18, 2019

     I'm initially somewhat inclined to want to work off of upstream's latest stable released tag (by tracking a 'stable' branch like what I have set up locally) and rebase as needed, but we could also work off of 'master.' Regardless, we need to find some way to make it easy to stay abreast of upstream commits, or at least most of them and/or the ones we'd be interested in. I also want us to consider how we'd upstream commits if we decided to do so for any before fully upstreaming our work here.

@rw
Copy link
Collaborator

rw commented Jan 18, 2019

Short term, overriding pour_bottle? to always be false should be enough IMO.

Long term, their changes to building packages (no more options?!) for v2 might force us to diverge.

@RandomDSdevel
Copy link
Member Author

@rw:

Long term, their changes to building packages (no more options?!) for v2 might force us to diverge.

     There would, at least initially, likely remain a non-trivial number of commits we could keep in common, however, and I'd want to make maintaining that brew code similarity tenable. We'd handle differences in formulae between upstream's and our copies in an organization fork of Homebrw/homebrew-core, anyway, and, per portage-brew/portage-brew-staging-and-evolution#8, that doesn't exist yet.

@RandomDSdevel
Copy link
Member Author

     With respect to branch organization here in portage-brew/brew, here, tentatively, is what I have set up for now:

  • 'upstream-master' tracks upstream's 'master.'
    • Our 'master' branch tracks 'upstream-master.'
  • 'upstream-stable' tracks upstream's latest stable tagged release.
    • Our 'stable' branch tracks 'upstream-stable.' (Odds could be that this should track our 'master' branch…?)

I haven't quite hammered out whether our PRs should be against our 'master' or our 'stable' branch and would appreciate guidance on that.

@RandomDSdevel
Copy link
Member Author

RandomDSdevel commented Jan 25, 2019

     An update: I've reorganized which branches have which others as upstreams. Here's the new tentative layout:

  • 'upstream-master' tracks upstream's 'master.'
    • Our 'master' branch tracks 'upstream-master.'
      • Our 'stable' branch tracks our 'master' branch.
      • PRs are tentatively against our 'master' branch.
  • 'upstream-stable' tracks upstream's latest stable tagged release.

Having our 'stable' branch track our 'master' branch instead of our 'upstream-stable' branch comes with a trade-off:

  • It makes it easier to pull our downstream commits in from 'master,' but more difficult to otherwise keep based off of a stable upstream release.

    (Instead of only manually pulling the latest tag into 'upstream-stable' and then rebasing 'stable' on top of that, one instead has to manually pull the latest tag into 'upstream-stable' and separately rebase 'stable' on top of that same upstream tag.)

I'm uncertain how this interacts, if at all well, with pulling commits from our 'master' branch into our 'stable' branch. Maybe somebody can suggest a cleaner solution here… (We could also switch back to the previous strategy if that's better.)

@RandomDSdevel
Copy link
Member Author

     Also, in case it wasn't clear, I want to have tooling and/or automation set up to ease maintaining whatever we settle on here.

@SMillerDev
Copy link

Long term, their changes to building packages (no more options?!) for v2 might force us to diverge.

Options aren't going away in brew, just in the core repo.

@sjackman
Copy link

I'm the lead of maintainer of Linuxbrew/brew, which has been a fork of Homebrew/brew for about five years. Each stable release of Homebrew, I merge the stable tag (say 1.9.3) into branch a branch named merge-1.9.3 on Linuxbrew/brew. I open a pull request from that branch. When it passes CI, I merge that pull request (using git checkout master && git merge --ff-only merge-1.9.3 && git push origin master and delete the branch merge-1.9.3). Developers and our CI will now be using that master branch. I leave it for about a day, to wait for bug reports. After about a day, I push the stable tag (1.9.3) and release it.

@RandomDSdevel
Copy link
Member Author

@SMillerDev:

Options aren't going away in brew, just in the core repo.

     We know, but see portage-brew/portage-brew-staging-and-evolution#8 and #1.

@RandomDSdevel
Copy link
Member Author

@sjackman:

     So you don't have any analog of our 'upstream-master' branch? You only keep track of stable upstream releases?

@sjackman
Copy link

Correct. We merge only stable upstream releases into our master branch.

@RandomDSdevel
Copy link
Member Author

     OK, that makes things simpler. I'll implement that soon.

@RandomDSdevel
Copy link
Member Author

RandomDSdevel commented Jan 26, 2019

     All right, here's what I have now:

  • 'upstream-master' tracks upstream's 'master.'
    • 'upstream-stable' tracks upstream's latest stable tagged release. (This is to let us avoid constantly creating temporary branches for merging into our 'master' branch each time upstream tags a new release.)
      • Our 'master' branch tracks 'upstream-stable.'
        • PRs are against our 'master' branch.
        • Our 'stable' branch tracks our 'master' branch, but is updated only when we actually want to tag a downstream release.

I'm leaving this issue open until we tackle tooling/automation, however.

@sjackman
Copy link

upstream-master: I have a remote named homebrew and call this branch homebrew/master, which is updated with git fetch homebrew. No harm in giving it a pseudonym though.

@RandomDSdevel
Copy link
Member Author

@sjackman:

     Thought I'd seen that as a convention somewhere, codifying/mirroring the terminology. Eh, it can always get renamed later to be more clear if need be.

@sjackman
Copy link

sjackman commented Jan 28, 2019

No harm in giving it a pseudonym though.

So long as you configure git fetch to automatically update upstream-master from its remote. Otherwise it's one more branch name to get stale and confuse you.

Edit: Change brew fetch to git fetch

@RandomDSdevel
Copy link
Member Author

@sjackman:

     You mean 'brew update.' 'brew fetch' is for formula artifacts, as I recall.

@sjackman
Copy link

Typo. I meant git fetch. I swap git and brew multiple times per day. I may never get them straight.

@RandomDSdevel
Copy link
Member Author

     No worries; I do it sometimes, too!

@S-C-Johnston
Copy link

I thought it'd be more appropriate to comment here than start a new issue on this: I think it'd be wise to document some of the architecture of homebrew, either as a part of this project or as a part of another project in this organization. The chief impetus is that homebrew org doesn't seem to make public much of their design except in raw ruby. It seems an odd half-measure between cathedral and bazaar styles, there. Anyway, I say this because I think it'd help lower the barrier to contributing in a meaningful way.

At the very least, some simple graphviz diagrams (a good medium precisely because it is compiled from markup, and therefore put in git and collaborated on) that illustrate the interaction between the user and homebrew and source code.

I have some informed guesses I could make about its structure and interaction. For example, just sampling a few commits, it looks like the maintenance of most formula revolves around incrementing the version number and updating the hashes. That doesn't do a whole lot to tell me in which files I should read the relevant source, though.


I am a student with a full-time job commitment on top of that, so I'm not sure how much I could help, frankly. And perhaps it's wildly unhelpful to be spitballing as an idea-guy. I just know that it seems super intimidating from my perspective to even begin to understand where things should be worked on.

@sjackman
Copy link

I'm a maintainer of Homebrew. I'm happy to answer specific questions that you may have about the architecture of Homebrew. In the mean time, there's a fair amount of information in the docs: https://docs.brew.sh particularly https://docs.brew.sh/Formula-Cookbook

RandomDSdevel pushed a commit that referenced this issue Jan 28, 2021
> Bootsnap is a library that plugs into Ruby, with optional support
> for ActiveSupport and YAML, to optimize and cache expensive
> computations.
https://github.com/Shopify/bootsnap

For our case that translates to "repeated calls to `brew` have
reductions in the time spend `require`ing speeding up the process
boot time".

For example:

```
$ hyperfine --warmup=2 "unset HOMEBREW_BOOTSNAP; brew info wget" "export HOMEBREW_BOOTSNAP=1; brew info wget"
Benchmark #1: unset HOMEBREW_BOOTSNAP; brew info wget
  Time (mean ± σ):      2.417 s ±  0.032 s    [User: 659.0 ms, System: 855.5 ms]
  Range (min … max):    2.382 s …  2.464 s    10 runs

Benchmark #2: export HOMEBREW_BOOTSNAP=1; brew info wget
  Time (mean ± σ):      1.862 s ±  0.064 s    [User: 425.3 ms, System: 566.8 ms]
  Range (min … max):    1.736 s …  1.952 s    10 runs

Summary
  'export HOMEBREW_BOOTSNAP=1; brew info wget' ran
    1.30 ± 0.05 times faster than 'unset HOMEBREW_BOOTSNAP; brew info wget'
```
RandomDSdevel pushed a commit that referenced this issue Mar 13, 2021
Port `brew --version` to Bash to speed it up.

While we're here:
- remove (now) unused `Tap` Git methods
- use `--quiet` instead of `-q` to be more verbose

Benchmarks:
```
$ hyperfine --min-runs=3 --warmup=2 "git checkout origin/master; brew --version" "git checkout speedup_brew_version; brew --version"
Benchmark #1: git checkout origin/master; brew --version
  Time (mean ± σ):      2.083 s ±  0.004 s    [User: 396.8 ms, System: 597.2 ms]
  Range (min … max):    2.080 s …  2.088 s    3 runs

Benchmark #2: git checkout speedup_brew_version; brew --version
  Time (mean ± σ):     847.9 ms ±  35.2 ms    [User: 100.0 ms, System: 247.9 ms]
  Range (min … max):   824.1 ms … 888.3 ms    3 runs

Summary
  'git checkout speedup_brew_version; brew --version' ran
    2.46 ± 0.10 times faster than 'git checkout origin/master; brew --version'
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

5 participants