BUG/MINOR: ssl: suboptimal certificate selection with TLSv1.3 and dual ECDSA/RSA

When using TLSv1.3, the signature algorithms extension is used to chose
the right ECDSA or RSA certificate.

However there was an old test for previous version of TLS (< 1.3) which
was testing if the cipher is compatible with ECDSA when an ECDSA
signature algorithm is used. This test was relying on
SSL_CIPHER_get_auth_nid(cipher) == NID_auth_ecdsa to verify if the
cipher is still good.

Problem is, with TLSv1.3, all ciphersuites are compatible with any
authentication algorithm, but SSL_CIPHER_get_auth_nid(cipher) does not
return NID_auth_ecdsa, but NID_auth_any.

Because of this, with TLSv1.3 when both ECDSA and RSA certificates are
available for a domain, the ECDSA one is not chosen in priority.

This patch also introduces a test on the cipher IDs for the signaling
ciphersuites, because they would always return NID_auth_any, and are not
relevent for this selection.

This patch fixes issue #2300.

Must be backported in all stable versions.

(cherry picked from commit 23093c72f139eddfce68ea5580193ee131901591)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
(cherry picked from commit 3f1e6f01a9a5748a9d4fa8800628b9a561cb3fc8)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
(cherry picked from commit 82c45fde0ebb930d072b264297a2e4c97e337352)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
(cherry picked from commit c29217e3a6e535ae29f2ecf5323172a8cf871d64)
Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
1 file changed