When running gitconfig --local, run it in subdirectories as well.

Summary:
The advantage of local is it doesn't pollute global state, but the
disadvantage is that the settings are not inherited by submodules.
(This is a misfeature, in my opinion.)  So whenever we set anything
--local, we need to manually do it in all submodules as well.

This isn't a perfect solution, since if new submodules are added later
they won't get the setting -- we'd have to hook into `git p` or
something for that -- but it's better than nothing!

In particular, it fixes (modulo the caveat above)
https://app.asana.com/0/31965416896056/172514774666235
and also
https://app.asana.com/0/31965416896056/51708574715815

Update submodules recursively, and do a better job parsing `status` output.

I wasn't doing recursive submodules before because I figured they
typically weren't *our* code so there was no point in using our
configs on them.  But there's no harm either, and they *can* be code
we care about, so let's just do it.

Just use no-arg `split` so we ignore leading spcae and runs of whitespace.

No-arg `split` is the best.

Test Plan:
I ran
   cd /tmp
   ~/khan/devtools/ka-clone/bin/ka-clone -p [email protected]:Khan/webapp
   cd webapp/intl/translations
   git config -l --local --includes | grep transport
and saw it say 'git-bigfile.transport=s3'.

On an old version of ka-clone, it did not say that.

(same)

(same)

Reviewers: mroth, benkraft

Reviewed By: benkraft

Differential Revision: https://phabricator.khanacademy.org/D34584

Author: csilvers

bin/ka-clone

@@ -80,6 +80,31 @@ def die_if_not_valid_git_repo():
         sys.exit(revparse_retcode)
 
 
+def _run_in_main_repo_and_subrepos(cmd):
+    """Run the given command in the main repo and all subrepos.
+
+    This is typically needed with `git config --local`, since local
+    settings are not inherited by submodules.
+
+    Note this is not a perfect fix!, since if new submodules are created
+    after the initial clone, they will not pick up the config setting.
+    We'd have to hook into `git p` for that.  But it's better than nothing!
+    """
+    # We cannot use `git submodule foreach` since the submodules may
+    # not have been checked out yet.
+    all_submodules = subprocess.check_output(
+        ['git', 'submodule', 'status', '--recursive']
+    )
+    all_dirs = [l.split()[1] for l in all_submodules.splitlines()]
+    all_dirs.append('.')      # do the main repo as well!
+
+    for subdir in all_dirs:
+        # For recursive submodules, the directory doesn't exist until we
+        # do a `git submodule checkout` of the containing submodule.
+        if os.path.exists(subdir):
+            subprocess.check_call(cmd, cwd=subdir)
+
+
 def _default_email():
     try:
         kac_email = subprocess.check_output(["git", "config", "kaclone.email"])
@@ -89,7 +114,8 @@ def _default_email():
 
 
 def set_email(email=_default_email()):
-    subprocess.check_call(
+    # We have to run this in subrepos explicitly due to `--local`.
+    _run_in_main_repo_and_subrepos(
         ['git', 'config', '--local', 'user.email', email]
     )
     _cli_log_step_success("Set user.email to {}".format(email))
@@ -156,11 +182,15 @@ def link_gitconfig_extras():
 
 
 def _gitconfig_local_reference(config_key, location, name="reference"):
-    """Configure reference to userdir template, but only if exists."""
+    """Configure reference to userdir template, but only if exists.
+
+    This also updates the gitconfig reference in submodules, since
+    `--local` does not cross submodule boundaries.
+    """
     home = os.path.expanduser("~")  # safe for cross platform
     tmpl = os.path.join(home, location)
     if os.path.isfile(tmpl):
-        subprocess.check_call(
+        _run_in_main_repo_and_subrepos(
             ['git', 'config', '--local', config_key, tmpl]
         )
         _cli_log_step_success("Linked {}".format(name))
@@ -189,6 +219,12 @@ def _clone_repo(src, dst, quiet=False):
     retcode = subprocess.call(cmds)
     if retcode is not 0:
         sys.exit(retcode)
+
+    # Initialize submodules, so we can set configs on them too.
+    retcode = subprocess.call(['git', 'submodule', 'init'], cwd=dst)
+    if retcode is not 0:
+        sys.exit(retcode)
+
     return dst
 
 
@@ -232,6 +268,7 @@ def _cli_process_current_dir(cli_args):
     if cli_args.protect_master:
         protect_master()
 
+
 if __name__ == '__main__':
     parser = _cli_parser()
     args = parser.parse_args()