Added support for remote repositories (with and without authentication) #101

GeertvanHorrik · 2014-03-17T11:36:37Z

When no local .git folder is available, it is now possible to execute the logic on remote branches:

gitversion -url [repositoryurl]

to use authentication:

gitversion -url [repositoryurl] -u [username] -p [password]

andreasohlund · 2014-03-17T12:08:57Z

Looks good, @nulltoken can you do a review as well and pull in if ok?

nulltoken · 2014-03-17T13:24:03Z

GitVersion/GitPreparer.cs

+
+        public string Url { get; private set; }
+
+        public string BranchName { get; private set; }


How is this property leveraged by the code?

Hmmm, you are right. Let me check...

Fixed, good catch!

… remote repository

nulltoken · 2014-03-17T14:17:21Z

GitVersion/GitFlow/BranchClassifier.cs

@@ -43,12 +43,12 @@ public static string GetSuffix(this Branch branch, BranchType branchType)

        public static bool IsDevelop(this Branch branch)
        {
-            return branch.Name == "develop";
+            return branch.Name.EndsWith("develop");


Why do we need this change?

I noticed it was in my previous pull request as well. It will always work on remote/develop as well. GitVersion is downloading the right branch anyway now, but this will always work (where == won't).

The problem is that this code won't protect us from branches that are named "mydevelop" or "origin/wrong/develop"

Ok, since it is working with local branches now anyway, I will revert this file.

nulltoken · 2014-03-17T14:20:06Z

GitVersion/GitPreparer.cs

+                // Normalize (download branches) before using the branch
+                GitHelper.NormalizeGitDirectory(gitDirectory);
+
+                using (var repository = new Repository(gitDirectory))


Can't we just check the branch out?

If you have an easier way, yes. But I couldn't find it (clone doesn't support default branch to check out). I don't want to download all the files though (since this might be a lot of data).

As the repository is already fully downloaded, I think that something like this should work (Warning: untested code)

var branchName = "..." var b = repo.FindBranch(branchName); repo.Checkout(b, CheckoutModifiers.Force);

That code works, but is much slower. Just let me know what you prefer, I don't mind what is being used in the end.

That code works, but is much slower

Hmmm. You're right. We don't actually need to check the files out.

How about

var branchName = "..." var b = repo.FindBranch(branchName); Debug.Assert(b != null); repo.Refs.UpdateTarget("HEAD", b.CanonicalName); Logger.WriteInfo(string.Format("Moving HEAD to branch '{0}'", b.CanonicalName));

That should work 😉

Just pushed (with other options commented out). This works great and is slightly simpler than what we had before.

…ight branch

…epositories

nulltoken · 2014-03-17T15:12:33Z

GitVersion/GitPreparer.cs

+                        Logger.WriteInfo(string.Format("Switching to branch '{0}'", BranchName));
+
+                        var branch = repository.FindBranch(BranchName);
+                        if ((branch != null) && !branch.IsCurrentRepositoryHead)


I think this line (and the one above) can be safely dropped

GeertvanHorrik · 2014-03-18T11:18:24Z

So the pull request is now being accepted?

nulltoken · 2014-03-18T11:23:16Z

GitVersion/BuildServers/GitHelper.cs

@@ -34,6 +34,7 @@ public static void NormalizeGitDirectory(string gitDirectory)

        static FetchOptions BuildFetchOptions()
        {
+            // TODO: Respect username/password of arguments?


Good catch!

@andreasohlund What's your take about this? How should we behave when both arguments and environment variables are valued? Which one should we consider?

I'd say commandline first, then fallback to env?

I agree. Commandline should override general settings.

I'd say commandline first, then fallback to env?

I agree. Commandline should override general settings.

@GeertvanHorrik Makes sense. Could you please update the PR to cope with this?

Working on it. I will pass the arguments around everywhere then. Normally I would use an IoC container to inject the arguments automatically, but since this app is so simple, I will just pass it into the ctors manually.

nulltoken · 2014-03-18T11:31:01Z

@andreasohlund Should we create an issue to make this feature also available to the Task?

andreasohlund · 2014-03-18T13:49:45Z

Should we create an issue to make this feature also available to the Task?

Please do

…password are first read from the environment variables, but can be overridden by command line arguments

…er is found so the build server can also pick up the version without using it as msbuild task

JakeGinnivan · 2014-03-19T21:59:09Z

GitVersion/Program.cs

@@ -36,12 +40,18 @@ static void Main()

                var versionAndBranch = VersionCache.GetVersion(gitDirectory);

+                foreach (var buildServer in applicableBuildServers)


This should be opt in.

This change will break shouldly, because I parse the json output directly to use in my build scripts. I added a comment to #106 to discuss this particular issue

You are saying you use TeamCity but don't want to use the TeamCity integration?

Yeah,

The build scripts use the outputted version, patch assembly info and write the build number to teamcity themselves. This way I can run a complete build locally without teamcity

…type (json or buildserver) (-) Removed logging to console

GeertvanHorrik · 2014-03-20T21:55:08Z

I just merged with the latest upstream master. I think this pull request is ready and can be accepted.

Added support for remote repositories (with and without authentication)

dazinator · 2015-01-11T19:28:41Z

Just wondering if this feature makes it possible to run team city builds without having to set "checkout on agent" ? Or am I getting the wrong end of the stick?

By default, Team city is set to Automatically check out on the Server, and historically unless you change it to "agent" side checkouts, then there is no git directory. Does this solve this issue?

GeertvanHorrik · 2015-01-11T19:29:32Z

Yes, this feature solves that issue.

dazinator · 2015-01-11T20:33:26Z

Wowser that's brilliant. Any chance of someone who knows how this feature works, creating a simple wiki entry showing how to use server side checkout with team city and gitversion? I have historically used jake's blog entry: http://jake.ginnivan.net/blog/2014/07/09/my-typical-teamcity-build-setup/ but I think that only applies for agent checkouts.
The reason being is i'd love to switch back to server checkout asap.

If I can get this working, I will happily create the above documentation. At the moment, there appears to be a bug as per the issue below.

dazinator · 2015-01-11T20:50:57Z

I have tried to configure this within Team City, and here are my issues:

I can't seem to use an SSH Key to authenticate, I'd like to use that as opposed to my github username and password. Is this supported?
If I use my username and password, authentication works, but I then get an error. You can see the error details on the following gist: https://gist.github.com/dazinator/87b4bfe7ade61e8cdea9

It looks like, (from the stack trace in the above gist) that when the constructor for NextVersionTxtFileFinder is called, NULL is passed in as the repositoryDirectory. This subsequently causes an error when GitVersion.NextVersionTxtFileFinder.TryGetNextVersion() is called.

Any ideas why repositoryDirectory would be NULL? Is it misuse on my part, or is this a legitimate defect?

I'd love to get this working.

nulltoken · 2015-01-11T21:31:43Z

@dazinator GitVersion relies on LibGit2Sharp which doesn't support SSH yet. Feel free to subscribe to libgit2/libgit2sharp#852 in order to gets notified about progress.

`
"G:\Program Files\TeamCity\buildAgent\work\2b1e52cf22ed9beb\lib\GitVersion.CommandLine.2.0.0\Tools\GitVersion.exe" /url https://github.com/dazinator/CrmAdo.git /u ******* /output buildserver /updateassemblyinfo true -assemblyVersionFormat MajorMinorPatch

Where the argument being passed in for /s is a SSH key on my github account listed here: https://github.com/settings/ssh

The error I get is:


[Step 1/5] Starting: "G:\Program Files\TeamCity\buildAgent\work\2b1e52cf22ed9beb\lib\GitVersion.CommandLine.2.0.0\Tools\GitVersion.exe" /url https://github.com/dazinator/CrmAdo.git /u ******* /output buildserver /updateassemblyinfo true -assemblyVersionFormat MajorMinorPatch
[20:43:46][Step 1/5] in directory: G:\Program Files\TeamCity\buildAgent\work\2b1e52cf22ed9beb
[20:43:47][Step 1/5] Retrieving git info from url 'https://github.com/dazinator/CrmAdo.git'
[20:43:47][Step 1/5] An unexpected error occurred:
[20:43:47][Step 1/5] System.NullReferenceException: Object reference not set to an instance of an object.
[20:43:47][Step 1/5] at LibGit2Sharp.RemoteCallbacks.GitCredentialHandler(IntPtr& ptr, IntPtr cUrl, IntPtr usernameFromUrl, GitCredentialType credTypes, IntPtr payload)
[20:43:47][Step 1/5] at LibGit2Sharp.Core.NativeMethods.git_clone(RepositorySafeHandle& repo, String origin_url, FilePath workdir_path, GitCloneOptions& opts)
[20:43:47][Step 1/5] at LibGit2Sharp.Core.Proxy.git_clone(String url, String workdir, GitCloneOptions& opts)
[20:43:47][Step 1/5] at LibGit2Sharp.Repository.Clone(String sourceUrl, String workdirPath, CloneOptions options)
[20:43:47][Step 1/5] at GitVersion.GitPreparer.GetGitInfoFromUrl()
[20:43:47][Step 1/5] at GitVersion.GitPreparer.Prepare()
[20:43:47][Step 1/5] at GitVersion.Program.Run()

Regarding the log above, it looks like only passing in the username causes an NullReferenceException which is... Hmmm... unfortunate.

The error may originate from here. Either we should change this way of blindly building the callback or we should ensure that we do not accept partial credentials as parameters (eg. only the username is this case).

@SimonCropp @JakeGinnivan Thoughts?

dazinator · 2015-01-11T21:39:05Z

Oh gees, was I only passing username. I thought I was passing username and password.. Dear lord. I will try it again. this time I'll try and pass in the actual arguments that I need to! :)

dazinator · 2015-01-11T21:42:40Z

@nulltoken - Sorry I am actually passing both username and password and I get the same exception.

Here is a new log:

Starting: "G:\Program Files\TeamCity\buildAgent\work\2b1e52cf22ed9beb\lib\GitVersion.CommandLine.2.0.0\Tools\GitVersion.exe" /url https://github.com/dazinator/CrmAdo.git /u dazinator /p ******* /output buildserver /updateassemblyinfo true -assemblyVersionFormat MajorMinorPatch
[21:40:11][Step 1/5] in directory: G:\Program Files\TeamCity\buildAgent\work\2b1e52cf22ed9beb
[21:40:11][Step 1/5] Deleting existing .git folder from 'G:\Program Files\TeamCity\buildAgent\work\2b1e52cf22ed9beb\_dynamicrepository\.git' to force new checkout from url
[21:40:11][Step 1/5] Setting up credentials using name 'dazinator'
[21:40:11][Step 1/5] Retrieving git info from url 'https://github.com/dazinator/CrmAdo.git'
[21:40:21][Step 1/5] Working directory: G:\Program Files\TeamCity\buildAgent\work\2b1e52cf22ed9beb\_dynamicrepository
[21:40:21][Step 1/5] Applicable build agent found: 'TeamCity'.
[21:40:21][Step 1/5] One remote found (origin -> 'https://github.com/dazinator/CrmAdo.git').
[21:40:21][Step 1/5] Fetching from remote 'origin' using the following refspecs: +refs/heads/*:refs/remotes/origin/*.
[21:40:22][Step 1/5] Creating local branch from remote tracking 'refs/remotes/origin/dev'.
[21:40:22][Step 1/5] Creating local branch from remote tracking 'refs/remotes/origin/gh-pages'.
[21:40:22][Step 1/5] Skipping local branch creation since it already exists 'refs/remotes/origin/master'.
[21:40:22][Step 1/5] HEAD points at branch 'refs/heads/master'.
[21:40:22][Step 1/5] Running against branch: master
[21:40:22][Step 1/5] GitHubFlow version strategy will be used
[21:40:22][Step 1/5] An unexpected error occurred:
[21:40:22][Step 1/5] System.ArgumentNullException: Value cannot be null.
[21:40:22][Step 1/5] Parameter name: path1
[21:40:22][Step 1/5]    at System.IO.Path.Combine(String path1, String path2)
[21:40:22][Step 1/5]    at GitVersion.NextVersionTxtFileFinder.TryGetNextVersion(SemanticVersion& semanticVersion) in c:\BuildAgent\work\68ea6bfb44e5afb\GitVersionCore\GitHubFlow\NextVersionTxtFileFinder.cs:line 19
[21:40:22][Step 1/5]    at GitVersion.NextSemverCalculator.<GetPossibleVersions>d__2.MoveNext() in c:\BuildAgent\work\68ea6bfb44e5afb\GitVersionCore\GitHubFlow\NextSemverCalculator.cs:line 53
[21:40:22][Step 1/5]    at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
[21:40:22][Step 1/5]    at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
[21:40:22][Step 1/5]    at GitVersion.NextSemverCalculator.NextVersion() in c:\BuildAgent\work\68ea6bfb44e5afb\GitVersionCore\GitHubFlow\NextSemverCalculator.cs:line 29
[21:40:22][Step 1/5]    at GitVersion.BuildNumberCalculator.GetBuildNumber(GitVersionContext context) in c:\BuildAgent\work\68ea6bfb44e5afb\GitVersionCore\GitHubFlow\BuildNumberCalculator.cs:line 22
[21:40:22][Step 1/5]    at GitVersion.Program.Run()

dazinator · 2015-01-11T21:47:07Z

I will try and debug this locally for a bit see if I can work out what's going on.

nulltoken · 2015-01-11T21:48:44Z

@dazinator Sorry, I was unclear.

The exception above definitely looks like another issue.

The log I was referring to (and which is pasted in my comment) is extracted from the initial mail I received when your first issued your comment (which you may have edited meanwhile).

dazinator · 2015-01-11T21:49:36Z

@nulltoken ah yes - I did edit things - probably my bad - no worries.

dazinator · 2015-01-11T21:56:57Z

Ok the bug is here:

As to why context.Repository.Info.WorkingDirectory returns null I have no idea..

This causes NextVersionTxtFileFinder to be constructed with a null repository directory, which ultimately causes the exception.

Any ideas?

dazinator · 2015-01-11T22:04:58Z

Also, I went to create a regression test to replicate this problem, but it hit me that I have absolutely no idea how to do such a thing. I guess I am just not familiar enough with GitVersion at this point.

So at the moment, to replicate this problem I simply put some command line arguments in the visual studio project configuration, and then ran the GitVersion.exe project.

Added support for remote repositories (with and without authentication)

173518b

nulltoken reviewed Mar 17, 2014
View reviewed changes

GitVersion now actually uses the specified target branch when using a…

81fb3ec

… remote repository

nulltoken reviewed Mar 17, 2014
View reviewed changes

Changed tabs to spaces in HelpWriter

314cf17

nulltoken reviewed Mar 17, 2014
View reviewed changes

GeertvanHorrik added 3 commits March 17, 2014 15:22

Revert changes in BranchClassifier to make sure it always picks the r…

d56fde9

…ight branch

Using repo.Refs.UpdateTarget instead of adding a new ref for remote r…

20c935d

…epositories

Removed not required code from GitPreparer

981c4f6

nulltoken reviewed Mar 17, 2014
View reviewed changes

nulltoken reviewed Mar 18, 2014
View reviewed changes

nulltoken mentioned this pull request Mar 18, 2014

Make GitVersionTask able to dynamically fetch remote repositories when the CI server doesn't #104

Closed

GeertvanHorrik added 2 commits March 18, 2014 16:02

Parameters are now being passed around to ensure that the username / …

f3bde9c

…password are first read from the environment variables, but can be overridden by command line arguments

GitVersion now writes results to console when a compatible build serv…

2ff757f

…er is found so the build server can also pick up the version without using it as msbuild task

GeertvanHorrik mentioned this pull request Mar 18, 2014

GitVersion.exe should also integrate with build servers #106

Closed

GeertvanHorrik added 2 commits March 18, 2014 17:24

Improved integration with Continua CI

2ed9693

GitVersion now checks out bare repositories

623f617

JakeGinnivan reviewed Mar 19, 2014
View reviewed changes

JakeGinnivan mentioned this pull request Mar 19, 2014

TeamCity without MSBuild #68

Closed

(+) Added /output parameter to allow the customization of the output …

292270b

…type (json or buildserver) (-) Removed logging to console

JakeGinnivan mentioned this pull request Mar 19, 2014

Added /exec and /proj arguments #112

Merged

Merge branch 'master' of https://github.com/Particular/GitVersion

2052f25

JakeGinnivan added a commit that referenced this pull request Mar 20, 2014

Merge pull request #101 from GeertvanHorrik/master

2028024

Added support for remote repositories (with and without authentication)

JakeGinnivan merged commit 2028024 into GitTools:master Mar 20, 2014

This was referenced Jan 14, 2015

Dynamic Repository with GitHubFlow throws an exception #336

Closed

Improved support for dynamic repositories #269

Closed

fixes #101 fixes #336 #349

Closed


		public string Url { get; private set; }

		public string BranchName { get; private set; }

		@@ -36,12 +40,18 @@ static void Main()

		var versionAndBranch = VersionCache.GetVersion(gitDirectory);

		foreach (var buildServer in applicableBuildServers)

Added support for remote repositories (with and without authentication) #101

Added support for remote repositories (with and without authentication) #101

Conversation

GeertvanHorrik commented Mar 17, 2014

andreasohlund commented Mar 17, 2014

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

GeertvanHorrik commented Mar 18, 2014

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nulltoken commented Mar 18, 2014

andreasohlund commented Mar 18, 2014

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

GeertvanHorrik commented Mar 20, 2014

dazinator commented Jan 11, 2015

GeertvanHorrik commented Jan 11, 2015

dazinator commented Jan 11, 2015

dazinator commented Jan 11, 2015

nulltoken commented Jan 11, 2015

dazinator commented Jan 11, 2015

dazinator commented Jan 11, 2015

dazinator commented Jan 11, 2015

nulltoken commented Jan 11, 2015

dazinator commented Jan 11, 2015

dazinator commented Jan 11, 2015

dazinator commented Jan 11, 2015