macOS can't notarize 2.9.0

[tresf]

Any attempts to notarize the .dylib that we ship with 2.9.0 fail, resulting in the following:

      "severity": "error",
      "code": null,
      "path": "my-app-1.0.0.pkg/my-app-1.0.0.pkg Contents/Payload/my-app.jar/natives/osx_64/libjssc.dylib",
      "message": "The binary uses an SDK older than the 10.9 SDK.",
      "docUrl": null,
      "architecture": "x86_64"

2.8.0 did NOT suffer this problem. This is a regression. Pinging @lukas-w for possible help/recommendations with this.

[tresf]

Hmm... otool -l shows the following in 2.9, but not in 2.8:

Load command 8
      cmd LC_VERSION_MIN_MACOSX

Both target libstdc++.6.dylib. Here's the full output of the command...

2.8.0

Mach header
      magic cputype cpusubtype  caps    filetype ncmds sizeofcmds      flags
 0xfeedfacf 16777223          3  0x00           6    12       1320 0x00110085
Load command 0
      cmd LC_SEGMENT_64
  cmdsize 552
  segname __TEXT
   vmaddr 0x0000000000000000
   vmsize 0x0000000000002000
  fileoff 0
 filesize 8192
  maxprot 0x00000007
 initprot 0x00000005
   nsects 6
    flags 0x0
Section
  sectname __text
   segname __TEXT
      addr 0x00000000000007d0
      size 0x000000000000118a
    offset 2000
     align 2^4 (16)
    reloff 0
    nreloc 0
     flags 0x80000400
 reserved1 0
 reserved2 0
Section
  sectname __symbol_stub1
   segname __TEXT
      addr 0x000000000000195a
      size 0x000000000000006c
    offset 6490
     align 2^0 (1)
    reloff 0
    nreloc 0
     flags 0x80000408
 reserved1 0 (index into indirect symbol table)
 reserved2 6 (size of stubs)
Section
  sectname __stub_helper
   segname __TEXT
      addr 0x00000000000019c8
      size 0x00000000000000d8
    offset 6600
     align 2^2 (4)
    reloff 0
    nreloc 0
     flags 0x00000000
 reserved1 0
 reserved2 0
Section
  sectname __const
   segname __TEXT
      addr 0x0000000000001aa0
      size 0x000000000000002c
    offset 6816
     align 2^5 (32)
    reloff 0
    nreloc 0
     flags 0x00000000
 reserved1 0
 reserved2 0
Section
  sectname __cstring
   segname __TEXT
      addr 0x0000000000001acc
      size 0x0000000000000007
    offset 6860
     align 2^0 (1)
    reloff 0
    nreloc 0
     flags 0x00000002
 reserved1 0
 reserved2 0
Section
  sectname __eh_frame
   segname __TEXT
      addr 0x0000000000001ad8
      size 0x0000000000000510
    offset 6872
     align 2^3 (8)
    reloff 0
    nreloc 0
     flags 0x6000000b
 reserved1 0
 reserved2 0
Load command 1
      cmd LC_SEGMENT_64
  cmdsize 312
  segname __DATA
   vmaddr 0x0000000000002000
   vmsize 0x0000000000001000
  fileoff 8192
 filesize 4096
  maxprot 0x00000007
 initprot 0x00000003
   nsects 3
    flags 0x0
Section
  sectname __dyld
   segname __DATA
      addr 0x0000000000002000
      size 0x0000000000000010
    offset 8192
     align 2^3 (8)
    reloff 0
    nreloc 0
     flags 0x00000000
 reserved1 0
 reserved2 0
Section
  sectname __nl_symbol_ptr
   segname __DATA
      addr 0x0000000000002010
      size 0x0000000000000008
    offset 8208
     align 2^2 (4)
    reloff 0
    nreloc 0
     flags 0x00000006
 reserved1 18 (index into indirect symbol table)
 reserved2 0
Section
  sectname __la_symbol_ptr
   segname __DATA
      addr 0x0000000000002018
      size 0x0000000000000090
    offset 8216
     align 2^2 (4)
    reloff 0
    nreloc 0
     flags 0x00000007
 reserved1 19 (index into indirect symbol table)
 reserved2 0
Load command 2
      cmd LC_SEGMENT_64
  cmdsize 72
  segname __LINKEDIT
   vmaddr 0x0000000000003000
   vmsize 0x0000000000006000
  fileoff 12288
 filesize 21056
  maxprot 0x00000007
 initprot 0x00000001
   nsects 0
    flags 0x0
Load command 3
          cmd LC_ID_DYLIB
      cmdsize 56
         name libjSSC-2.8_x86_64.jnilib (offset 24)
   time stamp 1 Wed Dec 31 19:00:01 1969
      current version 0.0.0
compatibility version 0.0.0
Load command 4
     cmd LC_UUID
 cmdsize 24
    uuid 42816ADC-83A4-F3FF-6E2F-422A583B0174
Load command 5
     cmd LC_SYMTAB
 cmdsize 24
  symoff 12480
   nsyms 45
  stroff 13352
 strsize 1184
Load command 6
            cmd LC_DYSYMTAB
        cmdsize 80
      ilocalsym 0
      nlocalsym 4
     iextdefsym 4
     nextdefsym 22
      iundefsym 26
      nundefsym 19
         tocoff 0
           ntoc 0
      modtaboff 0
        nmodtab 0
   extrefsymoff 0
    nextrefsyms 0
 indirectsymoff 13200
  nindirectsyms 37
      extreloff 0
        nextrel 0
      locreloff 12288
        nlocrel 18
Load command 7
      cmd LC_SEGMENT_SPLIT_INFO
  cmdsize 16
  dataoff 12432
 datasize 48
Load command 8
          cmd LC_LOAD_DYLIB
      cmdsize 56
         name /usr/lib/libstdc++.6.dylib (offset 24)
   time stamp 2 Wed Dec 31 19:00:02 1969
      current version 7.4.0
compatibility version 7.0.0
Load command 9
          cmd LC_LOAD_DYLIB
      cmdsize 56
         name /usr/lib/libgcc_s.1.dylib (offset 24)
   time stamp 2 Wed Dec 31 19:00:02 1969
      current version 1.0.0
compatibility version 1.0.0
Load command 10
          cmd LC_LOAD_DYLIB
      cmdsize 56
         name /usr/lib/libSystem.B.dylib (offset 24)
   time stamp 2 Wed Dec 31 19:00:02 1969
      current version 111.0.0
compatibility version 1.0.0
Load command 11
      cmd LC_CODE_SIGNATURE
  cmdsize 16
  dataoff 14544
 datasize 18800

2.9.0

Mach header
      magic cputype cpusubtype  caps    filetype ncmds sizeofcmds      flags
 0xfeedfacf 16777223          3  0x00           6    15       1432 0x00110085
Load command 0
      cmd LC_SEGMENT_64
  cmdsize 632
  segname __TEXT
   vmaddr 0x0000000000000000
   vmsize 0x0000000000003000
  fileoff 0
 filesize 12288
  maxprot 0x00000007
 initprot 0x00000005
   nsects 7
    flags 0x0
Section
  sectname __text
   segname __TEXT
      addr 0x00000000000017c0
      size 0x0000000000001248
    offset 6080
     align 2^4 (16)
    reloff 0
    nreloc 0
     flags 0x80000400
 reserved1 0
 reserved2 0
Section
  sectname __stubs
   segname __TEXT
      addr 0x0000000000002a08
      size 0x000000000000006c
    offset 10760
     align 2^1 (2)
    reloff 0
    nreloc 0
     flags 0x80000408
 reserved1 0 (index into indirect symbol table)
 reserved2 6 (size of stubs)
Section
  sectname __stub_helper
   segname __TEXT
      addr 0x0000000000002a74
      size 0x00000000000000c4
    offset 10868
     align 2^2 (4)
    reloff 0
    nreloc 0
     flags 0x80000400
 reserved1 0
 reserved2 0
Section
  sectname __const
   segname __TEXT
      addr 0x0000000000002b40
      size 0x000000000000004c
    offset 11072
     align 2^4 (16)
    reloff 0
    nreloc 0
     flags 0x00000000
 reserved1 0
 reserved2 0
Section
  sectname __cstring
   segname __TEXT
      addr 0x0000000000002b8c
      size 0x0000000000000009
    offset 11148
     align 2^0 (1)
    reloff 0
    nreloc 0
     flags 0x00000002
 reserved1 0
 reserved2 0
Section
  sectname __unwind_info
   segname __TEXT
      addr 0x0000000000002b95
      size 0x0000000000000098
    offset 11157
     align 2^0 (1)
    reloff 0
    nreloc 0
     flags 0x00000000
 reserved1 0
 reserved2 0
Section
  sectname __eh_frame
   segname __TEXT
      addr 0x0000000000002c30
      size 0x00000000000003c8
    offset 11312
     align 2^3 (8)
    reloff 0
    nreloc 0
     flags 0x00000000
 reserved1 0
 reserved2 0
Load command 1
      cmd LC_SEGMENT_64
  cmdsize 312
  segname __DATA
   vmaddr 0x0000000000003000
   vmsize 0x0000000000001000
  fileoff 12288
 filesize 4096
  maxprot 0x00000007
 initprot 0x00000003
   nsects 3
    flags 0x0
Section
  sectname __nl_symbol_ptr
   segname __DATA
      addr 0x0000000000003000
      size 0x0000000000000010
    offset 12288
     align 2^3 (8)
    reloff 0
    nreloc 0
     flags 0x00000006
 reserved1 18 (index into indirect symbol table)
 reserved2 0
Section
  sectname __got
   segname __DATA
      addr 0x0000000000003010
      size 0x0000000000000008
    offset 12304
     align 2^3 (8)
    reloff 0
    nreloc 0
     flags 0x00000006
 reserved1 20 (index into indirect symbol table)
 reserved2 0
Section
  sectname __la_symbol_ptr
   segname __DATA
      addr 0x0000000000003018
      size 0x0000000000000090
    offset 12312
     align 2^3 (8)
    reloff 0
    nreloc 0
     flags 0x00000007
 reserved1 21 (index into indirect symbol table)
 reserved2 0
Load command 2
      cmd LC_SEGMENT_64
  cmdsize 72
  segname __LINKEDIT
   vmaddr 0x0000000000004000
   vmsize 0x0000000000000bf4
  fileoff 16384
 filesize 3060
  maxprot 0x00000007
 initprot 0x00000001
   nsects 0
    flags 0x0
Load command 3
          cmd LC_ID_DYLIB
      cmdsize 48
         name @rpath/libjssc.dylib (offset 24)
   time stamp 1 Wed Dec 31 19:00:01 1969
      current version 0.0.0
compatibility version 0.0.0
Load command 4
            cmd LC_DYLD_INFO_ONLY
        cmdsize 48
     rebase_off 16384
    rebase_size 8
       bind_off 16392
      bind_size 96
  weak_bind_off 16488
 weak_bind_size 32
  lazy_bind_off 16520
 lazy_bind_size 248
     export_off 16768
    export_size 520
Load command 5
     cmd LC_SYMTAB
 cmdsize 24
  symoff 17440
   nsyms 43
  stroff 18284
 strsize 1160
Load command 6
            cmd LC_DYSYMTAB
        cmdsize 80
      ilocalsym 0
      nlocalsym 1
     iextdefsym 1
     nextdefsym 22
      iundefsym 23
      nundefsym 20
         tocoff 0
           ntoc 0
      modtaboff 0
        nmodtab 0
   extrefsymoff 0
    nextrefsyms 0
 indirectsymoff 18128
  nindirectsyms 39
      extreloff 0
        nextrel 0
      locreloff 0
        nlocrel 0
Load command 7
     cmd LC_UUID
 cmdsize 24
    uuid AFD62728-14B1-3F5B-BE4F-6B341D3AE44F
Load command 8
      cmd LC_VERSION_MIN_MACOSX
  cmdsize 16
  version 10.8
      sdk 10.8
Load command 9
      cmd LC_SOURCE_VERSION
  cmdsize 16
  version 0.0
Load command 10
          cmd LC_LOAD_DYLIB
      cmdsize 56
         name /usr/lib/libstdc++.6.dylib (offset 24)
   time stamp 2 Wed Dec 31 19:00:02 1969
      current version 56.0.0
compatibility version 7.0.0
Load command 11
          cmd LC_LOAD_DYLIB
      cmdsize 56
         name /usr/lib/libSystem.B.dylib (offset 24)
   time stamp 2 Wed Dec 31 19:00:02 1969
      current version 169.3.0
compatibility version 1.0.0
Load command 12
      cmd LC_FUNCTION_STARTS
  cmdsize 16
  dataoff 17288
 datasize 40
Load command 13
      cmd LC_DATA_IN_CODE
  cmdsize 16
  dataoff 17328
 datasize 0
Load command 14
      cmd LC_DYLIB_CODE_SIGN_DRS
  cmdsize 16
  dataoff 17328
 datasize 112

[lukas-w]

otool -l shows the following in 2.9, but not in 2.8:

I noticed this as well and I'm a bit baffled about how it's possible to produce a dylib without the LC_VERSION_MIN_MACOSX block. XCode's Clang appears to insert this automatically and I don't think there's a way to disable it. Is there any info about how the existing .jnilib from 2.8.0 was created? Maybe it was cross-compiled?

[tresf]

It's very unlikely it was cross-compiled. I wonder if CMake is injecting this under the covers. Probably worth enabling VERBOSE=1.

Here's where (I believe) CMake sets it....

https://cmake.org/cmake/help/latest/variable/CMAKE_OSX_DEPLOYMENT_TARGET.html

Alexey didn't use CMake, he compiled it by hand directly. Blaming cmake is purely speculative of course. I'll see what I can churn up.

[lukas-w]

I wonder if CMake is injecting this under the covers. Probably worth enabling VERBOSE=1.

I tried that and CMake does not pass any additional options to the linker or compiler unless CMAKE_OSX_DEPLOYMENT_TARGET is specified, which gets passed on as -mmacosx-version-min to Clang. If this option is not specified, LC_VERSION_MIN_MACOSX is still inserted, but with the current system's version. Hence my statement XCode's Clang appears to insert this automatically. Maybe LC_VERSION_MIN_MACOSX was only introduced after SDK 10.5? I'll try compiling with an older SDK and see what happens. Currently I'm using 10.13.

BTW, Apple claims that we can have backwards compatibility while using a new SDK. So using a newer SDK and setting CMAKE_OSX_DEPLOYMENT_TARGET should fix this in theory.

[tresf]

Apple claims that we can have backwards compatibility while using a new SDK. So using a newer SDK and setting CMAKE_OSX_DEPLOYMENT_TARGET should fix this in theory.

I haven't had a lot of luck with this with another project, so I've always built on older machines. Usually it comes down to the build dependencies being too new and requiring recompiling. Since this library only has JNI as a dependency it's a good candidate for what you're describing.

This flag will still only yield binaries for 10.9 though. Setting -DCMAKE_OSX_DEPLOYMENT_TARGET=10.8 will ultimately yield the same SDK level we're offering now, causing the same rejection during notarization.

[tresf]

This flag will still only yield binaries for 10.9 though. Setting -DCMAKE_OSX_DEPLOYMENT_TARGET=10.8 will ultimately yield the same SDK level we're offering now, causing the same rejection during notarization.

Hmm... maybe I'm wrong here... the following command...

cmake . -DCMAKE_OSX_SYSROOT= -DCMAKE_OSX_DEPLOYMENT_TARGET=10.8

Yields the following:

cmd LC_VERSION_MIN_MACOSX
  cmdsize 16
  version 10.8
      sdk 10.13

I'm skeptical this will work, but I can try. Note, -DCMAKE_OSX_SYSROOT= is a hack so I don't need the older SDK installed.

[lukas-w]

Setting -DCMAKE_OSX_DEPLOYMENT_TARGET=10.8 will ultimately yield the same SDK level we're offering now, causing the same rejection during notarization.

This is exactly what won't happen according to Apple's page about notarization problems:

notarization only works for binaries linked against macOS 10.9 or later. If you use an older SDK, notarization fails [...].
Using a newer SDK doesn’t affect your binary’s compatibility with earlier versions of macOS. Instead, version compatibility depends on the app’s deployment target

[tresf]

@lukas-w I'll try installing the 10.8 SDK on my machine and see if I can get this compiled and tested.

[lukas-w]

@tresf For the purpose of testing if it fixes notarization, any newer SDK should do. I tried doing this myself, but I'm having trouble getting Maven to build jssc. Off-topic, but maybe this error is familiar to you:

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.0:compile (default-compile) on project jssc: Fatal error compiling: error: release version 6 not supported -> [Help 1]

[tresf]

I tried to force 10.8 here 3918b68

1. Installed the 10.8 SDK (steps here: https://github.com/java-native/jssc/wiki/Publishing-a-Release#macos)
2. Adding the step to force 10.8

... unfortunately otool -l still shows the following

      cmd LC_VERSION_MIN_MACOSX
  cmdsize 16
+ version 10.8
+     sdk 10.8

... which I believe will still be rejected by Apple.

@lukas-w should I just attempt to go back to this hack? #54 (comment) Tested on 10.14 and it doesn't appear to work (was previously testing on 10.13). Ideas?

[lukas-w]

@tresf I think our only options are either

1. Produce a library without the LC_VERSION_MIN_MACOSX command. LLVM's LLD linker has a -no_version_load_command that does this. I tested this with the following CMake options after running brew install llvm:

   cmake .. -DCMAKE_CXX_COMPILER=/usr/local/opt/llvm/bin/clang++ -DCMAKE_C_COMPILER=/usr/local/opt/llvm/bin/clang -DCMAKE_SHARED_LINKER_FLAGS="-fuse-ld=lld -Wl,-no_version_load_command" -DCMAKE_LINKER=/usr/local/opt/llvm/bin/lld
   

2. Just use SDK 10.9+. I was able to compile a test library using SDK 10.9 with OSX 10.7 as the target:

        cmd LC_VERSION_MIN_MACOSX
    cmdsize 16
    version 10.7
        sdk 10.9
   

This should be accepted by Apple's notarization (except if your tests show that it isn't, see my question below). I have no way to test the binary under an older macOS version though.

Tested on 10.14 and it doesn't appear to work (was previously testing on 10.13).

What exactly doesn't work? Notarization?

[tresf]

What exactly doesn't work? Notarization?

The part that didn't work was the version and sdk being different. Specifically, I can't compile using the hacked cmake approach.

/usr/local/opt/llvm/bin/clang++

So you're using clang from Homebrew instead of Apple? Why?

-no_version_load_command

I'll have to try this. I think this achieves what @scream3r's build had previously achieved. Apple may eventually block these though. Seems like a loophole. 😕