git-subrepo.swim

git-subrepo
===========

Git Submodule Alternative

<badge travis ingydotnet/git-subrepo>

= Synopsis

  git subrepo -h    # Help Overview

  git subrepo clone <remote-url> [<subdir>]
  git subrepo init <subdir>
  git subrepo pull <subdir>
  git subrepo push <subdir>

  git subrepo fetch <subdir>
  git subrepo branch <subdir>
  git subrepo commit <subdir>
  git subrepo config <subdir>

  git subrepo status [<subdir>]
  git subrepo clean <subdir>

  git subrepo help [<command> | --all]
  git subrepo version
  git subrepo upgrade

= Description

This git command "clones" an external git repo into a subdirectory of your
repo. Later on, upstream changes can be pulled in, and local changes can be
pushed back. Simple.

= Benefits

This command is an improvement from `git-submodule` and `git-subtree`; two
other git commands with similar goals, but various problems.

It assumes there are 3 main roles of people interacting with a repo, and
attempts to serve them all well:

* *owner* - The person who authors\/owns\/maintains a repo.
* *users* - People who are just using/installing the repo.
* *collaborators* - People who commit code to the repo and subrepos.

The `git-subrepo` command benefits these roles in the following ways:

* Simple and intuitive commandline usage (with tab completion).
* Users get your repo and all your subrepos just by cloning your repo.
* Users do not need to install `git-subrepo`, ever.
* Collaborators do not need to install unless they want to push/pull.
* Collaborators know when a subdir is a subrepo (it has a `.gitrepo/config` file).
* The `.gitrepo/config` file never gets pushed back to the subrepo upstream.
* Well named branches and remotes are generated for manual operations.
* Owners do not deal with the complications of keeping submodules in sync.
* Subrepo repositories can contain subrepos themselves.
* Branching with subrepos JustWorks™.
* Different branches can have different subrepos in different states, etc.
* Moving\/renaming\/deleting a subrepo subdir JustWorks™.
* You can `init` an existing subdirectory into a subrepo.
* Your git history is kept squeaky clean.
* Upstream history (clone/pull) is condensed into a single commit.
* Pulls can use a `merge`, `rebase` or `force` strategies.
* You can see the subrepo history with `git log subrepo/<subdir>/fetch`.
* Commits pushed back upstream are *not* condensed (by default).
* Trivial to try any subrepo operations and then reset back.
* No configuration required.
* Does not introduce history that messes up other git commands.
* Fixes known rebase failures with `git-subtree`.

= Installation

The best short answer is:

  git clone https://github.com/ingydotnet/git-subrepo /path/to/git-subrepo
  echo 'source /path/to/git-subrepo/.rc' >> ~/.bashrc

The complete "Installation Instructions" can be found below.

Note: git-subrepo needs a git version (> 2.5) that supports worktree:s.

= Commands

All the *subrepo* commands use names of actual Git commands and try to do
operations that are similar to their Git counterparts. They also attempt to
give similar output in an attempt to make the subrepo usage intuitive to
experienced Git users.

Please note that the commands are /not/ exact equivalents, and do not take all
the same arguments. Keep reading…

- `git subrepo clone <repository> [<subdir>] [-b <branch>] [-f] [-m <msg>] [-e] [--method <merge|rebase>]`

  Add a repository as a subrepo in a subdir of your repository.

  This is similar in feel to `git clone`. You just specify the remote repo
  url, and optionally a sub-directory and/or branch name. The repo will be
  fetched and merged into the subdir.

  The subrepo history is /squashed/ into a single commit that contains the
  reference information. This information is also stored in a special file
  called `<subdir>/.gitrepo/config`. The presence of this file indicates that the
  directory is a subrepo.

  All subsequent commands refer to the subrepo by the name of the /subdir/.
  From the subdir, all the current information about the subrepo can be
  obtained.

  The `--force` option will "reclone" (completely replace) an existing subdir.

  The `--method` option will decide how the join process between branches are
   performed. The default option is merge.

  The `clone` command accepts the `--branch=` `--edit`, `--force` and
  `--message=` options.

- `git subrepo init <subdir> [-r <remote>] [-b <branch>] [--method <merge|rebase>]`

  Turn an existing subdirectory into a subrepo.

  If you want to expose a subdirectory of your project as a published subrepo,
  this command will do that. It will split out the content of a normal
  subdirectory into a branch and start tracking it as a subrepo. Afterwards
  your original repo will look exactly the same except that there will be a
  `<subdir>/.gitrepo/config` file.

  If you specify the `--remote` (and optionally the `--branch`) option, the
  values will be added to the `<subdir>/.gitrepo/config` file. The `--remote` option
  is the upstream URL, and the `--branch` option is the upstream branch to push
  to. These values will be needed to do a `git subrepo push` command, but they
  can be provided later on the `push` command (and saved to `<subdir>/.gitrepo/config`
  if you also specify the `--update` option).

  Note: You will need to create the empty upstream repo and push to it on your
  own, using `git subrepo push <subdir>`.

  The `--method` option will decide how the join process between branches
  are performed. The default option is merge.

  The `init` command accepts the `--branch=` and `--remote=` options.

- `git subrepo pull <subdir>|--all [-M|-R|-f] [-m <msg>] [-e] [-b <branch>] [-r <remote>] [-u]`

  Update the subrepo subdir with the latest upstream changes.

  The `pull` command fetches the latest content from the remote branch pointed
  to by the subrepo's `.gitrepo/config` file, and then tries to merge the changes into
  the corresponding subdir. It does this by making a branch of the local
  commits to the subdir and then merging or rebasing (see below) it with the
  fetched upstream content. After the merge, the content of the new branch
  replaces your subdir, the `.gitrepo/config` file is updated and a single 'pull'
  commit is added to your mainline history.

  The `pull` command will attempt to do the following commands in one go:

    git subrepo fetch <subdir>
    git subrepo branch <subdir>
    git merge/rebase subrepo/<subdir>/fetch subrepo/<subdir>
    git subrepo commit <subdir>
    # Only needed for a consequential push:
    git update-ref refs/subrepo/<subdir>/pull subrepo/<subdir>

  In other words, you could do all the above commands yourself, for the same
  effect. If any of the commands fail, subrepo will stop and tell you to finish
  this by hand. Generally a failure would be in the merge or rebase part, where
  conflicts can happen. Since Git has lots of ways to resolve conflicts to your
  personal tastes, the subrepo command defers to letting you do this by hand.

  When pulling new data, the method selected in clone/init is used. This has
  no effect on the final result of the pull, since it becomes a single commit.
  But it does affect the resulting `subrepo/<subdir>` branch, which is often
  used for a subrepo `push` command. See 'push' below for more information.
  If you want to change the method you can use the `config` command for this.

  When you pull you can assume a fast-forward strategy (default) or you can
  specify a `--rebase`, `--merge` or `--force` strategy. The latter is the same
  as a `clone --force` operation, using the current remote and branch.

  When you pull you can assume a fast-forward strategy (default) or you can
  specify a `--rebase`, `--merge` or `--force` strategy. The latter is the same
  as a `clone --force` operation, using the current remote and branch.

  Like the `clone` command, `pull` will squash all the changes (since the last
  pull or clone) into one commit. This keeps your mainline history nice and
  clean. You can easily see the subrepo's history with the `git log` command:

    git log refs/subrepo/<subdir>/fetch

  The set of commands used above are described in detail below.

  The `pull` command accepts the `--all`, `--branch=`, `--edit`, `--force`,
  `--message=`, `--remote=` and `--update` options.

- `git subrepo push <subdir>|--all [<branch>] [-r <remote>] [-b <branch>] [-M|-R] [-u] [-f] [-s] [-N]`

  Push a properly merged subrepo branch back upstream.

  This command takes the subrepo branch from a successful pull command and
  pushes the history back to its designated remote and branch. You can also use
  the `branch` command and merge things yourself before pushing if you want to
  (although that is probably a rare use case).

  The `push` command requires a branch that has been properly merged/rebased
  with the upstream HEAD (unless the upstream HEAD is empty, which is common
  when doing a first `push` after an `init`). That means the upstream HEAD is
  one of the commits in the branch.

  By default the branch ref `refs/subrepo/<subdir>/pull` will be pushed, but
  you can specify a (properly merged) branch to push.

  After that, the `push` command just checks that the branch contains the
  upstream HEAD and then pushes it upstream.

  The `--force` option will do a force push. Force pushes are typically
  discouraged. Only use this option if you fully understand it. (The `--force`
  option will NOT check for a proper merge. ANY branch will be force pushed!)

  The `push` command accepts the `--all`, `--branch=`, `--dry-run`, `--force`,
  `--merge`, `--rebase`, `--remote=`, `--squash` and `--update` options.

- `git subrepo fetch <subdir>|--all [-r <remote>] [-b <branch>]`

  Fetch the remote/upstream content for a subrepo.

  It will create a Git reference called `subrepo/<subdir>/fetch` that points at
  the same commit as `FETCH_HEAD`. It will also create a remote called
  `subrepo/<subdir>`. These are temporary and you can easily remove them with
  the subrepo `clean` command.

  The `fetch` command accepts the `--all`, `--branch=` and `--remote=` options.

- `git subrepo branch <subdir>|--all [-f] [-F]`

  Create a branch with local subrepo commits.

  Scan the history of the mainline for all the commits that affect the `subdir`
  and create a new branch from them called `subrepo/<subdir>`.

  This is useful for doing `pull` and `push` commands by hand.

  Use the `--force` option to write over an existing `subrepo/<subdir>` branch.

  The `branch` command accepts the `--all`, `--fetch` and `--force` options.

- `git subrepo commit <subdir> [<subrepo-ref>] [-m <msg>] [-e] [-f] [-F]`

  Add subrepo branch to current history as a single commit.

  This command is generally used after a hand-merge. You have done a `subrepo
  branch` and merged (rebased) it with the upstream. This command takes the
  HEAD of that branch, puts its content into the subrepo subdir and adds a new
  commit for it to the top of your mainline history.

  This command requires that the upstream HEAD be in the `subrepo/<subdir>`
  branch history. That way the same branch can push upstream. Use the
  `--force` option to commit anyway.

  The `commit` command accepts the `--edit`, `--fetch`, `--force` and
  `--message=` options.

- `git subrepo status [<subdir>|--all|--ALL] [-F] [-q|-v]`

  Get the status of a subrepo. Uses the `--all` option by default. If the
  `--quiet` flag is used, just print the subrepo names, one per line.

  The `--verbose` option will show all the recent local and upstream commits.

  Use `--ALL` to show the subrepos of the subrepos (ie the "subsubrepos"), if
  any.

  The `status` command accepts the `--all`, `--ALL`, `--fetch`, `--quiet` and
  `--verbose` options.

- `git subrepo clean <subdir>|--all|--ALL [-f]`

  Remove artifacts created by `fetch` and `branch` commands.

  The `fetch` and `branch` operations (and other commands that call them)
  create temporary things like refs, branches and remotes. This command
  removes all those things.

  Use `--force` to remove refs. Refs are not removed by default because they
  are sometimes needed between commands.

  Use `--all` to clean up after all the current subrepos. Sometimes you might
  change to a branch where a subrepo doesn't exist, and then `--all` won't find
  it. Use `--ALL` to remove any artifacts that were ever created by subrepo.

  To remove ALL subrepo artifacts:

    git subrepo clean --ALL --force

  The `clean` command accepts the `--all`, `--ALL`, and `--force` options.

- `git subrepo config <subdir> <option> [<value>] [-f]`

  Read or update configuration values in the subdir/.gitrepo/config file.

  Because most of the values stored in the .gitrepo/config file are generated you
  will need to use `--force` if you want to change anything else then the
  `method` option.

  Example to update the `method` option for a subrepo:

    git subrepo config foo method rebase

- `git subrepo help [<command>|--all]`

  Same as `git help subrepo`. Will launch the manpage. For the shorter usage,
  use `git subrepo -h`.

  Use `git subrepo help <command> to get help for a specific command. Use
  `--all` to get a summary of all commands.

  The `help` command accepts the `--all` option.

- `git subrepo version [-q|-v]`

  This command will display version information about git-subrepo and its
  environment. For just the version number, use `git subrepo --version`. Use
  `--verbose` for more version info, and `--quiet` for less.

  The `version` command accepts the `--quiet` and `--verbose` options.

- `git subrepo upgrade`

  Upgrade the `git-subrepo` software itself. This simply does a `git pull` on
  the git repository that the code is running from. It only works if you are on
  the `master` branch. It won't work if you installed `git-subrepo` using `make
  install`; in that case you'll need to `make install` from the latest code.

= Command Options

- `-h`

  Show a brief view of the commands and options.

- `--help`

  Gives an overview of the help options available for the subrepo command.

- `--version`

  Print the git-subrepo version. Just the version number. Try the `version`
  command for more version info.

- `--all` (`-a`)

  If you have multiple subrepos, issue the command to all of them (if
  applicable).

- `--ALL` (`-A`)

  If you have subrepos that also have subrepos themselves, issue the command to
  ALL of them. Note that the `--ALL` option only works for a subset of the
  commands that `--all` works for.

- `--branch=<branch-name>` (`-b <branch-name>`)

  Use a different upstream branch-name than the remote HEAD or the one saved in
  `.gitrepo/config` locally.

- `--dry-run` (`-N`)

  For the push command, do everything up until the push and then print out the
  actual `git push` command needed to finish the operation.

- `--edit` (`-e`)

  Edit the commit message before committing.

- `--fetch` (`-F`)

  Use this option to fetch the upstream commits, before running the command.

- `--force` (`-f`)

  Use this option to force certain commands that fail in the general case.

  NOTE: The `--force` option means different things for different commands.
  Read the command specific doc for the exact meaning.

- `--merge` (`-M`)

  Use a `merge` strategy to include upstream subrepo commits on a pull (or
  setup for push).

- `--message=<message>` (`-m <message>`)

  Specify your own commit message on the command line.

- `--rebase` (`-R`)

  Use a `rebase` strategy to include upstream subrepo commits on a pull (or
  setup for push).

- `--remote=<remote-url>` (`-r <remote-url>`)

  Use a different remote-url than the one saved in `.gitrepo/config` locally.

- `--squash` (`-s`)

  Squash all commits on a push into one new commit.

- `--update` (`-u`)

  If `--branch` or `--remote` are used, and the command updates the `.gitrepo/config`
  file, include these values to the update.

= Output Options

- `--quiet` (`-q`)

  Print as little info as possible. Applicable to most commands.

- `--verbose` (`-v`)

  Print more information about the command execution and results. Applicable
  to most commands.

- `--debug` (`-d`)

  Show the actual git (and other) commands being executed under the hood.
  Applicable to most commands.

- `--DEBUG` (`-x`)

  Use the Bash `set -x` option which prints every command before it is run.
  VERY noisy, but extremely useful in deep debugging. Applicable to all
  commands.

= Environment Variables

The `git-subrepo` command exports and honors some environment variables:

- `GIT_SUBREPO_ROOT`

  This is set by the `.rc` file, if you use that method to install / enable
  `git-subrepo`. It contains the path of the `git-subrepo` repository.

- `GIT_SUBREPO_RUNNING`

  This variable is exported when `git-subrepo` is running. It is set to the pid
  of the `git-subrepo` process that is running. Other processes, like git hooks
  for instance, can use this information to adjust accordingly.

- `GIT_SUBREPO_COMMAND`

  This variable is exported when `git-subrepo` is running. It is set to the
  name of the `git-subrepo` subcommand that is running.

- `GIT_SUBREPO_PAGER`

  Use this to specify the pager to use for long output commands. Defaults to
  `$PAGER` or `less`.

- `GIT_SUBREPO_QUIET`

  Set this for quiet (`-q`) output.

- `GIT_SUBREPO_VERBOSE`

  Set this for verbose (`-v`) output.

- `GIT_SUBREPO_DEBUG`

  Set this for debugging (`-d`) output.

= Installation Instructions

There are currently 3 ways to install `git-subrepo`. For all of them you need
to get the source code from GitHub:

  git clone https://github.com/ingydotnet/git-subrepo /path/to/git-subrepo

The first installation method is preferred: `source` the `.rc` file. Just add a
line like this one to your shell startup script:

  source /path/to/git-subrepo/.rc

That will modify your `PATH` and `MANPATH`, and also enable command completion.

The second method is to do these things by hand. This might afford you more
control of your shell environment. Simply add the `lib` and `man` directories
to your `PATH` and `MANPATH`:

  export GIT_SUBREPO_ROOT="/path/to/git-subrepo"
  export PATH="/path/to/git-subrepo/lib:$PATH"
  export MANPATH="/path/to/git-subrepo/man:$MANPATH"

See below for info on how to turn on Command Completion.

The third method is a standard system install, which puts `git-subrepo` next to
your other git commands:

  make install        # Possibly with 'sudo'

This method does not account for upgrading and command completion yet.

== Windows

This command is known to work in these Windows environments:

* Git for Windows -- https://git-for-windows.github.io/
* Babun -- http://babun.github.io/
* Cygwin -- https://www.cygwin.com/

Let us know if there are others that it works (or doesn't work) in.

= Testing

The `git-subrepo` repository comes with a extensive test suite. You can run it
with:

  make test

or if you don't have `make` on your system:

  prove -v test

= Upgrading

If you used the `.rc` or `PATH` method of installation, just run this to
upgrade `git-subrepo`:

  git subrepo upgrade

Or (same thing):

  cd /path/to/git-subrepo
  git pull

If you used `make install` method, then run this again (after `git pull`):

  make install        # Possibly with 'sudo'

= Command Completion

The `git subrepo` command supports `<TAB>`-based command completion. If you
don't use the `.rc` script (see Installation, above), you'll need to enable
this manually to use it.

== In Bash

If your Bash setup does not already provide command completion for Git, you'll
need to enable that first:

  source <Git completion script>

On your system, the Git completion script might be found at any of the
following locations (or somewhere else that we don't know about):

* `/etc/bash_completion.d/git`
* `/usr/share/bash-completion/git`
* `/usr/share/bash-completion/completions/git`
* `/opt/local/share/bash-completion/completions/git`
* `/usr/local/etc/bash_completion.d/git`
* `~/.homebrew/etc/bash_completion.d/git`

In case you can't find any of these, this repository contains a copy of the
Git completion script:

  source /path/to/git-subrepo/share/git-completion.bash

Once Git completion is enabled (whether you needed to do that manually or
not), you can turn on `git-subrepo` completion with a command like this:

  source /path/to/git-subrepo/share/completion.bash

== In zsh

In the Z shell (zsh), you can manually enable `git-subrepo` completion by
adding the following line to your `~/.zshrc`, *before* the `compinit` function
is called:

  fpath=('/path/to/git-subrepo/share/zsh-completion' $fpath)

= Status

The git-subrepo command has been in use for well over a year and seems to get
the job done. Development is still ongoing but mostly just for fixing bugs.

Trying subrepo out is simple and painless (this is not `git submodule`).
Nothing is permanent (if you do not push to shared remotes). ie You can always
play around and reset back to the beginning without pain.

This command has a test suite (run `make test`), but surely has many bugs. If
you have expertise with Git and subcommands, please review the code, and file
issues on anything that seems wrong.

If you want to chat about the `git-subrepo` command, join `#gitcommands` on
`irc.freenode.net`.

= Notes

* Works on POSIX systems: Linux, BSD, OSX, etc.
* Works on various Windows environments. See "Windows" section above.
* The `git-subrepo` repo itself has 2 subrepos under the `ext/` subdirectory.
* Written in (very modern) Bash, with full test suite. Take a look.
* A `.gitrepo/config` file never is in the top level dir (next to a `.git/` dir).

= Authors

* Ingy döt Net <ingy@ingy.net>
* Magnus Carlsson <grimmymail@gmail.com>

= License and Copyright

The MIT License (MIT)

Copyright (c) 2013-2018 Ingy döt Net