Skip to content

Move native binaries into separate NuGet package #984

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
May 28, 2015
Merged

Move native binaries into separate NuGet package #984

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,8 @@ _ReSharper*/
*.DotSettings
#Ignore custom generated files
LibGit2Sharp/Core/UniqueIdentifier.cs
LibGit2Sharp/Core/NativeDllName.cs
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we start ignoring it, shouldn't we also drop it from the index?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, thought 76eaeea included that, or is there something more that you're wanting me to do here?


!Lib/NativeBinaries/*/*.pdb
!nuget.package/build/
_NCrunch_LibGit2Sharp/
packages/
3 changes: 0 additions & 3 deletions .gitmodules
View file
Open in desktop

This file was deleted.

2 changes: 1 addition & 1 deletion CI/build.msbuild
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@
</Target>

<Target Name="Test" DependsOnTargets="Build">
<xunit Assembly="$(TestBuildDir)/LibGit2Sharp.Tests.dll" Xml="$(DeployFolder)/Test-result.xml" />
<xunit Assembly="$(TestBuildDir)/LibGit2Sharp.Tests.dll" ShadowCopy="false" Xml="$(DeployFolder)/Test-result.xml" />
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please share what led to this change?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mentioned it a bit in my comments above somewhere, but didn't go into a ton of detail.

The build.libgit2sharp.sh script used to set the LD_LIBRARY_PATH / DYLD_LIBRARY_PATH variables to the location of the newly built libgit2 binaries. One of the things I learned during all of my research for this is that setting these shouldn't be necessary. Mono will first look in the directory of the referencing assembly when searching for libraries referenced by DllImport attributes. By doing this, they are mimicking the loading behavior of windows.

I was curious about when this might have been added to mono, and I ended up grabbing a copy of the mono source and tracked down the library loading code. After looking through the history, I found that the particular side-by-side behavior has been around since mono 2.10, and before that it would first look in the current directory instead, which was changed because it was deemed to be dangerous.

I removed the path variables from the scripts, and everything worked fine on my mac. However, the build was failing on my ubuntu vm, with an exception indicating it couldn't find libgit2. I spent some time in the mono debug logs, and I saw that the libgit2sharp libraries were being shadow copied while being run by the xunit msbuild test runner. When run on linux, this was causing a problem, but osx had no problem, was was loading the library fine.

After turning off shadow copy in the build.msbuild file, the linux build started being able to load libgit2 as it should, but would fail on the CanProbeForNativeBinariesFromAShadowCopiedAssembly() test, indicating that linux was indeed having problems when shadow copied assemblies are involved.

If you look through my commits, you'll see at one point I had worked around the problem on linux by adding a LD_LIBRARY_PATH value back, but since that needs an absolute path, I added a Travis value to get the path passed in. The linux Travis build passed after that.

I wasn't thinking about that not existing when not running Travis, and I ran the script on my local ubuntu vm... and it worked!

After experimenting, I realized that the path didn't even need to be valid to work, it just need to have a semicolon in the value. If you look at the latest version of build.libgit2sharp.sh, you'll see that I've set it to just a ':' and that is enough to make everything work properly on linux again!

So, with that workaround in place, strictly speaking, the change to build.msbuild isn't really needed... but I don't see any harm in keeping it. You don't really need to worry about locked files on CI build, so shadow copying isn't really gaining you anything.. and with it on it was actually masking the shadow copy specific test failure!

So, that's my super-longwinded explanation for why that got changed! 😄

I'm hoping I'll be able to track down the bug in the mono code and see if I can get it fixed.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW, you're aware that build.msbuild is only used by Travis, and not AppVeyor, right? I'm thinking that must not have always been the case, because there are some things in build.msbuild that seem to think it might be run on Windows. It's also still doing things like trying to copy a NativeBinaries folder, which won't on non-windows builds.

</Target>

<Target Name="Deploy" DependsOnTargets="Test">
Expand Down
1 change: 1 addition & 0 deletions Lib/CustomBuildTasks/CustomBuildTasks.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@
</ItemGroup>
<ItemGroup>
<Compile Include="GenerateUniqueIdentifierTask.cs" />
<Compile Include="GenerateNativeDllNameTask.cs" />
</ItemGroup>
<Import Project="$(MSBuildToolsPath)\Microsoft.CSharp.targets" />
</Project>
Binary file modified Lib/CustomBuildTasks/CustomBuildTasks.dll
View file
Open in desktop
Binary file not shown.
43 changes: 43 additions & 0 deletions Lib/CustomBuildTasks/GenerateNativeDllNameTask.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
using System;
using System.IO;
using Microsoft.Build.Framework;
using Microsoft.Build.Utilities;

namespace CustomBuildTasks
{
public class GenerateNativeDllNameTask : Task
{
public ITaskItem InputHashFile { get; set; }

public string OutputFile { get; set; }

public override bool Execute()
{
var fileName = InputHashFile.GetMetadata("FullPath");
string hash;

using (var sr = new StreamReader(fileName))
{
hash = sr.ReadLine();
}

var shortHash = hash.Substring(0, 7);

var nativeDllName = @"namespace LibGit2Sharp.Core
{{
internal static class NativeDllName
{{
public const string Name = ""git2-{0}"";
}}
}}
";

using (var sw = new StreamWriter(OutputFile))
{
sw.Write(nativeDllName, shortHash);
}

return true;
}
}
}
Binary file removed Lib/NativeBinaries/amd64/git2-9bbc8f3.dll
View file
Open in desktop
Binary file not shown.
Binary file removed Lib/NativeBinaries/amd64/git2-9bbc8f3.pdb
View file
Open in desktop
Binary file not shown.
930 changes: 0 additions & 930 deletions Lib/NativeBinaries/libgit2.license.txt
View file
Open in desktop

This file was deleted.

Binary file removed Lib/NativeBinaries/x86/git2-9bbc8f3.dll
View file
Open in desktop
Binary file not shown.
Binary file removed Lib/NativeBinaries/x86/git2-9bbc8f3.pdb
View file
Open in desktop
Binary file not shown.
4 changes: 0 additions & 4 deletions LibGit2Sharp.Tests/LibGit2Sharp.Tests.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -136,10 +136,6 @@
<None Include="packages.config" />
</ItemGroup>
<Import Project="$(MSBuildToolsPath)\Microsoft.CSharp.targets" />
<PropertyGroup>
<NativeBinariesDirectory>$(MSBuildProjectDirectory)\..\Lib\NativeBinaries</NativeBinariesDirectory>
</PropertyGroup>
<Import Project="..\LibGit2Sharp\CopyWindowsNativeDependencies.targets" />
<Import Project="..\LibGit2Sharp\ExtraDefine.targets" />
<Import Project="VisualStudio.Tests.targets" />
<PropertyGroup>
Expand Down
13 changes: 0 additions & 13 deletions LibGit2Sharp/CopyWindowsNativeDependencies.targets
View file
Open in desktop

This file was deleted.

7 changes: 0 additions & 7 deletions LibGit2Sharp/Core/NativeDllName.cs
View file
Open in desktop

This file was deleted.

16 changes: 10 additions & 6 deletions LibGit2Sharp/LibGit2Sharp.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<Import Project="..\packages\LibGit2Sharp.NativeBinaries.1.0.25\build\LibGit2Sharp.NativeBinaries.props" Condition="Exists('..\packages\LibGit2Sharp.NativeBinaries.1.0.25\build\LibGit2Sharp.NativeBinaries.props')" />
<PropertyGroup>
<Configuration Condition=" '$(Configuration)' == '' ">Debug</Configuration>
<Platform Condition=" '$(Platform)' == '' ">AnyCPU</Platform>
Expand Down Expand Up @@ -355,22 +356,25 @@
<CodeAnalysisDictionary Include="CustomDictionary.xml" />
</ItemGroup>
<ItemGroup>
<EmbeddedResource Include="libgit2_hash.txt" />
<EmbeddedResource Include="libgit2sharp_hash.txt" />
</ItemGroup>
<ItemGroup>
<EmbeddedResource Include="libgit2sharp_hash.txt" />
<None Include="packages.config" />
</ItemGroup>
<Import Project="$(MSBuildToolsPath)\Microsoft.CSharp.targets" />
<Import Project="UniqueIdentifier.targets" />
<PropertyGroup>
<NativeBinariesDirectory>$(MSBuildProjectDirectory)\..\Lib\NativeBinaries</NativeBinariesDirectory>
</PropertyGroup>
<Import Project="CopyWindowsNativeDependencies.targets" />
<Import Project="NativeDllName.targets" />
<Import Project="ExtraDefine.targets" />
<PropertyGroup>
<PreBuildEvent>
</PreBuildEvent>
</PropertyGroup>
<Target Name="EnsureNuGetPackageBuildImports" BeforeTargets="PrepareForBuild">
<PropertyGroup>
<ErrorText>This project references NuGet package(s) that are missing on this computer. Enable NuGet Package Restore to download them. For more information, see http://go.microsoft.com/fwlink/?LinkID=322105. The missing file is {0}.</ErrorText>
</PropertyGroup>
<Error Condition="!Exists('..\packages\LibGit2Sharp.NativeBinaries.1.0.25\build\LibGit2Sharp.NativeBinaries.props')" Text="$([System.String]::Format('$(ErrorText)', '..\packages\LibGit2Sharp.NativeBinaries.1.0.25\build\LibGit2Sharp.NativeBinaries.props'))" />
</Target>
<!-- To modify your build process, add your task inside one of the targets below and uncomment it.
Other similar extension points exist, see Microsoft.Common.targets.
<Target Name="BeforeBuild">
Expand Down
1 change: 0 additions & 1 deletion LibGit2Sharp/LibGit2Sharp.v2.ncrunchproject
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,5 +22,4 @@
<UseCPUArchitecture>AutoDetect</UseCPUArchitecture>
<MSTestThreadApartmentState>STA</MSTestThreadApartmentState>
<BuildProcessArchitecture>x86</BuildProcessArchitecture>
<AdditionalFilesToInclude>..\Lib\NativeBinaries\**.*</AdditionalFilesToInclude>
</ProjectConfiguration>
23 changes: 23 additions & 0 deletions LibGit2Sharp/NativeDllName.targets
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
<?xml version="1.0" encoding="utf-8"?>
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">

<UsingTask TaskName="GenerateNativeDllNameTask" AssemblyFile="..\Lib\CustomBuildTasks\CustomBuildTasks.dll" />

<PropertyGroup>
<LibGit2SharpPath Condition="'$(MSBuildThisFileDirectory)' == ''">.</LibGit2SharpPath>
<LibGit2SharpPath Condition="'$(MSBuildThisFileDirectory)' != ''">$(MSBuildThisFileDirectory)</LibGit2SharpPath>
<NativeDllNamePath>$(LibGit2SharpPath)\Core\NativeDllName.cs</NativeDllNamePath>
<CoreCompileDependsOn>$(CoreCompileDependsOn);GenerateNativeDllNameCs</CoreCompileDependsOn>
<CoreCleanDependsOn>$(CoreCleanDependsOn);CleanNativeDllNameCs</CoreCleanDependsOn>
</PropertyGroup>

<Target Name="GenerateNativeDllNameCs"
Inputs="@(EmbeddedResource)"
Outputs="$(NativeDllNamePath)">
<GenerateNativeDllNameTask InputHashFile="@(EmbeddedResource)" Condition=" '%(Filename)%(Extension)' == 'libgit2_hash.txt' " OutputFile="$(NativeDllNamePath)" />
</Target>

<Target Name="CleanNativeDllNameCs">
<Delete Files="$(NativeDllNamePath)" />
</Target>
</Project>
1 change: 0 additions & 1 deletion LibGit2Sharp/libgit2_hash.txt
View file
Open in desktop

This file was deleted.

4 changes: 4 additions & 0 deletions LibGit2Sharp/packages.config
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
<?xml version="1.0" encoding="utf-8"?>
<packages>
<package id="LibGit2Sharp.NativeBinaries" version="1.0.25" targetFramework="net40" allowedVersions="[1.0.25]" />
</packages>
212 changes: 0 additions & 212 deletions UpdateLibgit2ToSha.ps1
View file
Open in desktop

This file was deleted.

Loading