Merge pull request #279 from tmaher/tls-verify-hostnames

enable TLS hostname validation

Author: jch

Diff for: README.rdoc

@@ -52,12 +52,10 @@ This task will run the test suite and the
 
   rake rubotest
 
-To run the integration tests against an LDAP server:
-
-  cd test/support/vm/openldap
-  vagrant up
-  cd ../../../..
-  INTEGRATION=openldap bundle exec rake rubotest
+CI takes too long? If your local box supports
+{Vagrant}[https://www.vagrantup.com/], you can run most of the tests
+in a VM on your local box. For more details and setup instructions, see
+{test/support/vm/openldap/README.md}[https://github.com/ruby-ldap/ruby-net-ldap/tree/master/test/support/vm/openldap/README.md]
 
 == Release
 

Diff for: lib/net/ldap.rb

@@ -476,61 +476,73 @@ def self.result2string(code) #:nodoc:
   #   specify a treebase. If you give a treebase value in any particular
   #   call to #search, that value will override any treebase value you give
   #   here.
+  # * :force_no_page => Set to true to prevent paged results even if your
+  #   server says it supports them. This is a fix for MS Active Directory
+  # * :instrumentation_service => An object responsible for instrumenting
+  #   operations, compatible with ActiveSupport::Notifications' public API.
   # * :encryption => specifies the encryption to be used in communicating
   #   with the LDAP server. The value must be a Hash containing additional
   #   parameters, which consists of two keys:
   #     method: - :simple_tls or :start_tls
-  #     options: - Hash of options for that method
+  #     tls_options: - Hash of options for that method
   #   The :simple_tls encryption method encrypts <i>all</i> communications
   #   with the LDAP server. It completely establishes SSL/TLS encryption with
   #   the LDAP server before any LDAP-protocol data is exchanged. There is no
   #   plaintext negotiation and no special encryption-request controls are
   #   sent to the server. <i>The :simple_tls option is the simplest, easiest
   #   way to encrypt communications between Net::LDAP and LDAP servers.</i>
-  #   It's intended for cases where you have an implicit level of trust in the
-  #   authenticity of the LDAP server. No validation of the LDAP server's SSL
-  #   certificate is performed. This means that :simple_tls will not produce
-  #   errors if the LDAP server's encryption certificate is not signed by a
-  #   well-known Certification Authority. If you get communications or
-  #   protocol errors when using this option, check with your LDAP server
-  #   administrator. Pay particular attention to the TCP port you are
-  #   connecting to. It's impossible for an LDAP server to support plaintext
-  #   LDAP communications and <i>simple TLS</i> connections on the same port.
-  #   The standard TCP port for unencrypted LDAP connections is 389, but the
-  #   standard port for simple-TLS encrypted connections is 636. Be sure you
-  #   are using the correct port.
-  #
+  #   If you get communications or protocol errors when using this option,
+  #   check with your LDAP server administrator. Pay particular attention
+  #   to the TCP port you are connecting to. It's impossible for an LDAP
+  #   server to support plaintext LDAP communications and <i>simple TLS</i>
+  #   connections on the same port. The standard TCP port for unencrypted
+  #   LDAP connections is 389, but the standard port for simple-TLS
+  #   encrypted connections is 636. Be sure you are using the correct port.
   #   The :start_tls like the :simple_tls encryption method also encrypts all
   #   communcations with the LDAP server. With the exception that it operates
   #   over the standard TCP port.
   #
-  #   In order to verify certificates and enable other TLS options, the
-  #   :tls_options hash can be passed alongside :simple_tls or :start_tls.
-  #   This hash contains any options that can be passed to
-  #   OpenSSL::SSL::SSLContext#set_params(). The most common options passed
-  #   should be OpenSSL::SSL::SSLContext::DEFAULT_PARAMS, or the :ca_file option,
-  #   which contains a path to a Certificate Authority file (PEM-encoded).
-  #
-  #   Example for a default setup without custom settings:
-  #     {
-  #       :method => :simple_tls,
-  #       :tls_options => OpenSSL::SSL::SSLContext::DEFAULT_PARAMS
-  #     }
+  #   To validate the LDAP server's certificate (a security must if you're
+  #   talking over the public internet), you need to set :tls_options
+  #   something like this...
   #
-  #   Example for specifying a CA-File and only allowing TLSv1.1 connections:
-  #
-  #     {
-  #       :method => :start_tls,
-  #       :tls_options => { :ca_file => "/etc/cafile.pem", :ssl_version => "TLSv1_1" }
+  #   Net::LDAP.new(
+  #     # ... set host, bind dn, etc ...
+  #     encryption: {
+  #       method: :simple_tls,
+  #       tls_options: OpenSSL::SSL::SSLContext::DEFAULT_PARAMS,
   #     }
-  # * :force_no_page => Set to true to prevent paged results even if your
-  #   server says it supports them. This is a fix for MS Active Directory
-  # * :instrumentation_service => An object responsible for instrumenting
-  #   operations, compatible with ActiveSupport::Notifications' public API.
+  #   )
+  #
+  #   The above will use the operating system-provided store of CA
+  #   certificates to validate your LDAP server's cert.
+  #   If cert validation fails, it'll happen during the #bind
+  #   whenever you first try to open a connection to the server.
+  #   Those methods will throw Net::LDAP::ConnectionError with
+  #   a message about certificate verify failing. If your
+  #   LDAP server's certificate is signed by DigiCert, Comodo, etc.,
+  #   you're probably good. If you've got a self-signed cert but it's
+  #   been added to the host's OS-maintained CA store (e.g. on Debian
+  #   add foobar.crt to /usr/local/share/ca-certificates/ and run
+  #   `update-ca-certificates`), then the cert should pass validation.
+  #   To ignore the OS's CA store, put your CA in a PEM-encoded file and...
+  #
+  #   encryption: {
+  #     method:      :simple_tls,
+  #     tls_options: { ca_file:     '/path/to/my-little-ca.pem',
+  #                    ssl_version: 'TLSv1_1' },
+  #   }
+  #
+  #   As you might guess, the above example also fails the connection
+  #   if the client can't negotiate TLS v1.1.
+  #   tls_options is ultimately passed to OpenSSL::SSL::SSLContext#set_params
+  #   For more details, see
+  #    http://ruby-doc.org/stdlib-2.0.0/libdoc/openssl/rdoc/OpenSSL/SSL/SSLContext.html
   #
   # Instantiating a Net::LDAP object does <i>not</i> result in network
   # traffic to the LDAP server. It simply stores the connection and binding
-  # parameters in the object.
+  # parameters in the object. That's why Net::LDAP.new doesn't throw
+  # cert validation errors itself; #bind does instead.
   def initialize(args = {})
     @host = args[:host] || DefaultHost
     @port = args[:port] || DefaultPort

Diff for: lib/net/ldap/connection.rb

@@ -52,6 +52,15 @@ def open_connection(server)
     hosts.each do |host, port|
       begin
         prepare_socket(server.merge(socket: @socket_class.new(host, port, socket_opts)), timeout)
+        if encryption
+          if encryption[:tls_options] &&
+             encryption[:tls_options][:verify_mode] &&
+             encryption[:tls_options][:verify_mode] == OpenSSL::SSL::VERIFY_NONE
+            warn "not verifying SSL hostname of LDAPS server '#{host}:#{port}'"
+          else
+            @conn.post_connection_check(host)
+          end
+        end
         return
       rescue Net::LDAP::Error, SocketError, SystemCallError,
              OpenSSL::SSL::SSLError => e
@@ -392,12 +401,11 @@ def search(args = nil)
         # should collect this into a private helper to clarify the structure
         query_limit = 0
         if size > 0
-          if paged
-            query_limit = (((size - n_results) < 126) ? (size -
-                                                              n_results) : 0)
-          else
-            query_limit = size
-          end
+          query_limit = if paged
+                          (((size - n_results) < 126) ? (size - n_results) : 0)
+                        else
+                          size
+                        end
         end
 
         request = [

Diff for: script/generate-fixture-ca

@@ -0,0 +1,48 @@
+#!/bin/bash
+
+BASE_PATH=$( cd "`dirname $0`/../test/fixtures/ca" && pwd )
+cd "${BASE_PATH}" || exit 4
+
+USAGE=$( cat << EOS
+Usage:
+  $0 --regenerate
+
+Generates a new self-signed CA, for integration testing. This should only need
+to be run if you are writing new TLS/SSL tests, and need to generate
+additional fixtuer CAs.
+
+This script uses the GnuTLS certtool CLI. If you are on macOS,
+'brew install gnutls', and it will be installed as 'gnutls-certtool'.
+Apple unfortunately ships with an incompatible /usr/bin/certtool that does
+different things.
+EOS
+)
+
+if [ "x$1" != 'x--regenerate' ]; then
+  echo "${USAGE}"
+  exit 1
+fi
+
+TOOL=`type -p certtool`
+if [ "$(uname)" = "Darwin" ]; then
+  TOOL=`type -p gnutls-certtool`
+  if [ ! -x "${TOOL}" ]; then
+    echo "Sorry, Darwin requires gnutls-certtool; try `brew install gnutls`"
+    exit 2
+  fi
+fi
+
+if [ ! -x "${TOOL}" ]; then
+  echo "Sorry, no certtool found!"
+  exit 3
+fi
+export TOOL
+
+
+${TOOL} --generate-privkey > ./cakey.pem
+${TOOL} --generate-self-signed \
+  --load-privkey ./cakey.pem \
+  --template ./ca.info \
+  --outfile ./cacert.pem
+
+echo "cert and private key generated! Don't forget to check them in"

Diff for: script/install-openldap

@@ -2,8 +2,8 @@
 set -e
 set -x
 
-BASE_PATH="$( cd `dirname $0`/../test/fixtures/openldap && pwd )"
-SEED_PATH="$( cd `dirname $0`/../test/fixtures          && pwd )"
+BASE_PATH=$( cd "`dirname $0`/../test/fixtures/openldap" && pwd )
+SEED_PATH=$( cd "`dirname $0`/../test/fixtures"          && pwd )
 
 dpkg -s slapd time ldap-utils gnutls-bin ssl-cert > /dev/null ||\
   DEBIAN_FRONTEND=noninteractive apt-get update  -y --force-yes && \
@@ -48,47 +48,58 @@ chown -R openldap.openldap /var/lib/ldap
 rm -rf $TMPDIR
 
 # SSL
+export CA_CERT="/usr/local/share/ca-certificates/rubyldap-ca.crt"
+export CA_KEY="/etc/ssl/private/rubyldap-ca.key"
 
-sh -c "certtool --generate-privkey > /etc/ssl/private/cakey.pem"
+# The self-signed fixture CA cert & key are generated by
+# `script/generate-fiuxture-ca` and checked into version control.
+# You shouldn't need to muck with these unless you're writing more
+# TLS/SSL integration tests, and need special magic values in the cert.
 
-sh -c "cat > /etc/ssl/ca.info <<EOF
-cn = rubyldap
-ca
-cert_signing_key
-EOF"
+cp "${SEED_PATH}/ca/cacert.pem" "${CA_CERT}"
+cp "${SEED_PATH}/ca/cakey.pem"  "${CA_KEY}"
 
-# Create the self-signed CA certificate:
-certtool --generate-self-signed \
---load-privkey /etc/ssl/private/cakey.pem \
---template /etc/ssl/ca.info \
---outfile /etc/ssl/certs/cacert.pem
+# actually add the fixture CA to the system store
+update-ca-certificates
 
 # Make a private key for the server:
 certtool --generate-privkey \
---bits 1024 \
---outfile /etc/ssl/private/ldap01_slapd_key.pem
+  --bits 1024 \
+  --outfile /etc/ssl/private/ldap01_slapd_key.pem
 
 sh -c "cat > /etc/ssl/ldap01.info <<EOF
 organization = Example Company
 cn = ldap01.example.com
+dns_name = ldap01.example.com
+dns_name = ldap02.example.com
+dns_name = localhost
 tls_www_server
 encryption_key
 signing_key
 expiration_days = 3650
 EOF"
 
+# The integration server may be accessed by IP address, in which case
+# we want some of the IPs included in the cert. We skip loopback (127.0.0.1)
+# because that's the IP we use in the integration test for cert name mismatches.
+ADDRS=$(ifconfig -a | grep 'inet addr:' | cut -f 2 -d : | cut -f 1 -d ' ')
+for ip in $ADDRS; do
+  if [ "x$ip" = 'x127.0.0.1' ]; then continue; fi
+  echo "ip_address = $ip" >> /etc/ssl/ldap01.info
+done
+
 # Create the server certificate
 certtool --generate-certificate \
   --load-privkey /etc/ssl/private/ldap01_slapd_key.pem \
-  --load-ca-certificate /etc/ssl/certs/cacert.pem \
-  --load-ca-privkey /etc/ssl/private/cakey.pem \
+  --load-ca-certificate "${CA_CERT}" \
+  --load-ca-privkey "${CA_KEY}" \
   --template /etc/ssl/ldap01.info \
   --outfile /etc/ssl/certs/ldap01_slapd_cert.pem
 
 ldapmodify -Y EXTERNAL -H ldapi:/// <<EOF | true
 dn: cn=config
 add: olcTLSCACertificateFile
-olcTLSCACertificateFile: /etc/ssl/certs/cacert.pem
+olcTLSCACertificateFile: ${CA_CERT}
 -
 add: olcTLSCertificateFile
 olcTLSCertificateFile: /etc/ssl/certs/ldap01_slapd_cert.pem
@@ -110,6 +121,14 @@ chmod g+r /etc/ssl/private/ldap01_slapd_key.pem
 chmod o-r /etc/ssl/private/ldap01_slapd_key.pem
 
 # Drop packets on a secondary port used to specific timeout tests
-iptables -A OUTPUT -p tcp -j DROP --dport 8389
+iptables -A INPUT -p tcp -j DROP --dport 8389
+
+# Forward a port for Vagrant
+iptables -t nat -A PREROUTING -p tcp --dport 9389 -j REDIRECT --to-port 389
+
+# fix up /etc/hosts for cert validation
+grep ldap01 /etc/hosts || echo "127.0.0.1 ldap01.example.com" >> /etc/hosts
+grep ldap02 /etc/hosts || echo "127.0.0.1 ldap02.example.com" >> /etc/hosts
+grep bogus /etc/hosts  || echo "127.0.0.1 bogus.example.com" >> /etc/hosts
 
 service slapd restart

Diff for: test/fixtures/ca/ca.info

@@ -0,0 +1,4 @@
+cn = rubyldap
+ca
+cert_signing_key
+expiration_days = 7200

Diff for: test/fixtures/ca/cacert.pem

@@ -0,0 +1,24 @@
+-----BEGIN CERTIFICATE-----
+MIID7zCCAlegAwIBAgIMV7zWei6SNfABx6jMMA0GCSqGSIb3DQEBCwUAMBMxETAP
+BgNVBAMTCHJ1YnlsZGFwMB4XDTE2MDgyMzIzMDQyNloXDTM2MDUxMDIzMDQyNlow
+EzERMA8GA1UEAxMIcnVieWxkYXAwggGiMA0GCSqGSIb3DQEBAQUAA4IBjwAwggGK
+AoIBgQDGe9wziGHZJhIf+IEKSk1tpT9Mu7YgsUwjrlutvkoO1Q6K+amTAVDXizPf
+1DVSDpZP5+CfBOznhgLMsPvrQ02w4qx5/6X9L+zJcMk8jTNYSKj5uIKpK52E7Uok
+aygMXeaqroPONGkoJIZiVGgdbWfTvcffTm8FOhztXUbMrMXJNinFsocGHEoMNN8b
+vqgAyG4+DFHoK4L0c6eQjE4nZBChieZdShUhaBpV7r2qSNbPw67cvAKuEzml58mV
+1ZF1F73Ua8gPWXHEfUe2GEfG0NnRq6sGbsDYe/DIKxC7AZ89udZF3WZXNrPhvXKj
+ZT7njwcMQemns4dNPQ0k2V4vAQ8pD8r8Qvb65FiSopUhVaGQswAnIMS1DnFq88AQ
+KJTKIXbBuMwuaNNSs6R/qTS2RDk1w+CGpRXAg7+1SX5NKdrEsu1IaABA/tQ/zKKk
+OLLJaD0giX1weBVmNeFcKxIoT34VS59eEt5APmPcguJnx+aBrA9TLzSO788apBN0
+4lGAmR0CAwEAAaNDMEEwDwYDVR0TAQH/BAUwAwEB/zAPBgNVHQ8BAf8EBQMDBwQA
+MB0GA1UdDgQWBBRTvXSkge03oqLu7UUjFI+oLYwnujANBgkqhkiG9w0BAQsFAAOC
+AYEATSZQWH+uSN5GvOUvJ8LHWkeVovn0UhboK0K7GzmMeGz+dp/Xrj6eQ4ONK0zI
+RCJyoo/nCR7CfQ5ujVXr03XD2SUgyD565ulXuhw336DasL5//fucmQYDeqhwbKML
+FTzsF9H9dO4J5TjxJs7e5dRJ0wrP/XEY+WFhXXdSHTl8vGCI6QqWc7TvDpmbS4iX
+uTzjJswu9Murt9JUJNMN2DlDi/vBBeruaj4c2cMMnKMvkfj14kd8wMocmzj+gVQl
+r+fRQbKAJNec65lA4/Zeb6sD9SAi0ZIVgxA4a7g8/sdNWHIAxPicpJkIJf30TsyY
+F+8+Hd5mBtCbvFfAVkT6bHBP1OiAgNke+Rh/j/sQbyWbKCKw0+jpFJgO9KUNGfC0
+O/CqX+J4G7HqL8VJqrLnBvOdhfetAvNQtf1gcw5ZwpeEFM+Kvx/lsILaIYdAUSjX
+ePOc5gI2Bi9WXq+T9AuhSf+TWUR874m/rdTWe5fM8mXCNl7C4I5zCqLltEDkSoMP
+jDj/
+-----END CERTIFICATE-----