CURL: Download of Runner bin not working

[m8t88]

Hi all,

When executing the image build with packer for Ubuntu 22, I get the following error when it tries to download the runner bin:

ubuntu@ip-10-21-108-145:~$ curl -o test -f "https://github.com/actions/runner/releases/download/v2.321.0/actions-runner-linux-x64-2.321.0.tar.gz"
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0

I then created an instance in AWS and logged into that. I confirmed that it can reach github by nslookup.
buntu@ip-10-21-108-145:~$ nslookup github.com
Server: 127.0.0.53
Address: 127.0.0.53#53

Non-authoritative answer:
Name: github.com
Address: 140.82.121.4

I then tried the following curl flags since I thought it might be certificate related:
-k should ignore the SSL cert
-L handles redirects

That also didnt help. Below the returns

untu@ip-10-21-108-145:~$ curl -o test -fL "https://github.com/actions/runner/releases/download/v2.321.0/actions-runner-linux-x64-2.321.0.tar.gz"
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.
ubuntu@ip-10-21-108-145:$ curl -o test -fk "https://github.com/actions/runner/releases/download/v2.321.0/actions-runner-linux-x64-2.321.0.tar.gz"
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
ubuntu@ip-10-21-108-145:$ curl -o test -fLk "https://github.com/actions/runner/releases/download/v2.321.0/actions-runner-linux-x64-2.321.0.tar.gz"
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
curl: (22) The requested URL returned error: 403
ubuntu@ip-10-21-108-145:~$ curl -o test -Lk "https://github.com/actions/runner/releases/download/v2.321.0/actions-runner-linux-x64-2.321.0.tar.gz"
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 29690 0 29690 0 0 1207k 0 --:--:-- --:--:-- --:--:-- 1207k
curl: (56) OpenSSL SSL_read: error:0A000126:SSL routines::unexpected eof while reading, errno 0

I tried the same command on my ubuntu22 machine in my office behind a corp firewall. There it is no issue to download the bin with the same command
123@123~/.ssh $ curl -o test -Lk "https://github.com/actions/runner/releases/download/v2.321.0/actions-runner-linux-x64-2.321.0.tar.gz"
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 113M 100 113M 0 0 29.6M 0 0:00:03 0:00:03 --:--:-- 38.5M

I checked the OpenSSL and curl versions on both machine and they are the same

123@123~/.ssh $ openssl version
OpenSSL 3.0.2 15 Mar 2022 (Library: OpenSSL 3.0.2 15 Mar 2022)
123@123~/.ssh $ curl --version
curl 7.81.0 (x86_64-pc-linux-gnu) libcurl/7.81.0 OpenSSL/3.0.2 zlib/1.2.11 brotli/1.0.9 zstd/1.4.8 libidn2/2.3.2 libpsl/0.21.0 (+libidn2/2.3.2) libssh/0.9.6/openssl/zlib nghttp2/1.43.0 librtmp/2.3 OpenLDAP/2.5.18
Release-Date: 2022-01-05
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS brotli GSS-API HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM NTLM_WB PSL SPNEGO SSL TLS-SRP UnixSockets zstd

I dont understand why there is a 403 in AWS but not from my office and how to authenticate where with what if that is the issue.

It might be a long shot, but did anyone had the same issue or has an idea what I can do, so that the packer build works in my AWS account?

[m8t88]

I figured it out.. it was a change in our company firewall approach in where I have to specifically whitelist every domain I want to access from our EC2 instances. I thought I got all of them but did not realize that the DL call to github.com redirects to objects.githubusercontent.com.

After whitelisting that domain too, it works just fine. I figured this out after checking the verbose responses with the -v curl flag.

Leaving this here for any other GHES user that might have the same issue.