feat: cache and verify downloaded archive

[lidel]

This PR saves us time on CI and gives more confidence that downloaded archive is valid:

• ♻️ retries twice before failing to download (replaced node-fetch with got)
• 🗃️ adds caching of downloaded archives
  • right now go-ipfs download slows down each build of ipfs-desktop
  • electron, electron-builder and even our own aegir already do this type of caching for big third-party downloads
  • faster install and CI will provide additional incentive for people to migrate from go-ipfs-dep (Deprecate in favor of npm-go-ipfs ipfs-inactive/npm-go-ipfs-dep#45)
• 🛑 adds pre-commit hook to protect against commiting real binary to git
• 🕵️ downloads .sha512 manifest and compares it with sha512 of downloaded archive

Demo: caching and verifying in action

First time downloads archive to the cache:

Downloading https://dist.ipfs.io/go-ipfs/v0.7.0/go-ipfs_v0.7.0_linux-amd64.tar.gz to /home/lidel/.cache/npm-go-ipfs
Downloaded https://dist.ipfs.io/go-ipfs/v0.7.0/go-ipfs_v0.7.0_linux-amd64.tar.gz
Downloading go-ipfs_v0.7.0_linux-amd64.tar.gz.sha512
Downloaded go-ipfs_v0.7.0_linux-amd64.tar.gz.sha512
Verifying go-ipfs_v0.7.0_linux-amd64.tar.gz.sha512
OK (1d5910f27e8d7ea333145f15c6edcbacc1e8db3a99365f0847467bdfa7c73f4d7a05562e46be8e932056c8324ed0769ca1b6758dfb0ac4c2e1b6066b57c4a086)
Unpacked /home/lidel/project/ipfs/npm-go-ipfs
Linking /home/lidel/project/ipfs/npm-go-ipfs/go-ipfs/ipfs to /home/lidel/project/ipfs/npm-go-ipfs/bin/ipfs

Second time reused archive from the cache:

https://dist.ipfs.io/go-ipfs/versions
Found /home/lidel/.cache/npm-go-ipfs/go-ipfs_v0.7.0_linux-amd64.tar.gz
Verifying go-ipfs_v0.7.0_linux-amd64.tar.gz.sha512
OK (1d5910f27e8d7ea333145f15c6edcbacc1e8db3a99365f0847467bdfa7c73f4d7a05562e46be8e932056c8324ed0769ca1b6758dfb0ac4c2e1b6066b57c4a086)
Unpacked /home/lidel/project/ipfs/npm-go-ipfs
Linking /home/lidel/project/ipfs/npm-go-ipfs/go-ipfs/ipfs to /home/lidel/project/ipfs/npm-go-ipfs/bin/ipfs

Note that SHA512 is compared on every run.
If a single bit was flipped, it will return an error:

Found /home/lidel/.cache/npm-go-ipfs/go-ipfs_v0.7.0_linux-amd64.tar.gz
Verifying go-ipfs_v0.7.0_linux-amd64.tar.gz.sha512
Expected   SHA512: 2d5910f27e8d7ea333145f15c6edcbacc1e8db3a99365f0847467bdfa7c73f4d7a05562e46be8e932056c8324ed0769ca1b6758dfb0ac4c2e1b6066b57c4a086
Calculated SHA512: 1d5910f27e8d7ea333145f15c6edcbacc1e8db3a99365f0847467bdfa7c73f4d7a05562e46be8e932056c8324ed0769ca1b6758dfb0ac4c2e1b6066b57c4a086
Error: SHA512 of /home/lidel/.cache/npm-go-ipfs/go-ipfs_v0.7.0_linux-amd64.tar.gz' (1d5910f27e8d7ea333145f15c6edcbacc1e8db3a99365f0847467bdfa7c73f4d7a05562e46be8e932056c8324ed0769ca1b6758dfb0ac4c2e1b6066b57c4a086) does not match expected value from /home/lidel/.cache/npm-go-ipfs/go-ipfs_v0.7.0_linux-amd64.tar.gz.sha512 (2d5910f27e8d7ea333145f15c6edcbacc1e8db3a99365f0847467bdfa7c73f4d7a05562e46be8e932056c8324ed0769ca1b6758dfb0ac4c2e1b6066b57c4a086)
    at cachingFetchAndVerify (/home/lidel/project/ipfs/npm-go-ipfs/src/download.js:67:11)

cc @andrew

[andrew]

👍

[lidel]

@achingbrain as agreed, I've updated tests and removed dead code ported from npm-go-ipfs-dep + added git workflow to run npm test on PRs.

I'm merging this, so the cache is included when 0.9.0 ships.