Today I spent a few hours wrapping my head around an issue where some websites weren’t able to connect properly to PayPal.com. I found it pretty tricky to understand because it involved quite a few technologies and programs:
- PHP: a programming language, and a program to interpret lines of code written in that programming language (aka interpreter)
- cURL: a program for composing messages to be sent over the Internet
- openSSL: a program for encrypting messages according to one of many encryption algorithms
- TLS (sometimes the same thing as SSL): a protocol for encrypting HTTP messages so they can be securely sent between two servers
- PayPal: a separate server that handles payment processing, which just recently started requiring all communication with its servers be over the protocol TLS 1.2
And each of those programs has various versions, some which don’t play nicely with each other.
I like making analogies to the real world in order to better understand computer stuff. So here we go:
Imagine a gradeschool penpals initiative. Principal Helen Powers tells 5th grade teacher, Mr. Curls, to have the kids write letters off to another school somewhere else in the world. Mr Curls tells one of his students, Open-mouthed Susie-Lee Lewinski, to write a letter to another child on the other side of the planet. Here’s their pictures, in case you were wondering:
Except we don’t know what language the other child speaks (in fact, they haven’t decided where to send the letter yet). So the first letter Susie should send will find out what language the other child speaks, and if Susie can also write in that language (by the way, Susie is a polyglot: she speaks Arabic, Greek, Latin, Spanish, Esperanto and English).
So when everything goes smoothly:
- Principal Helen Powers asks Mr Curls to send a message to a given address
- Mr Curls tells Susie to compose a message to the unknown child
- Susie’s first letter is a “handshake” letter, briefly telling the other child what languages she speaks, and asking them which one they’d like to use.
- The other child, we’ll call them Palay, responds saying they’d like to speak in Esperanto
- Susie and Palay continue to correspond in Esperanto
What each character represents:
- Principal Helen Powers: PHP interpreter
- Mr Curls: cURL
- Open-Mouthed Susie-Lee Lewinski: OpenSSL
- various versions of TLS and SSL: different languages Susie can speak
- Palay : PayPal
When everything works smoothly,
- PHP code has instructions to use cURL to send a request to PayPal (like Principal Powers instructing Mr Curls to send a message to Palay)
- cURL takes those instructions and, in turn, instructs openSSL to send a request to PayPal
- openSSL sends a handshake request to PayPal, telling its server what versions of TLS and SSL it can handle
- PayPal’s server responds with what versions of TLS and SSL it can handle
- a version of TLS or SSL is chosen (or “negotiated”), and subsequent messages are sent and received using that protocol
There are a few problems that can occur both between clients and servers communicating over the Internet, and between grade school penpals.
Or, the servers don’t have a shared supported version of TLS or SSL. In the case with PayPal, for security, they decided their server would only communicating using TLS version 1.2 or higher (and no version of SSL). This is fine for most websites on servers which have had openSSL updated in the last 5 years or so (specifically, servers should at least use openSSL version 1.0.1c/). However, if openSSL is too old, it won’t know how to communicate in TLS 1.2, and so PayPal will reply with an error like this:
curl: (35) error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure
Or, the PHP code specifies a specific version of TLS or SSL must be used, but the other server refuses to use that version.
Eg, in PHP code you can set
curl_setopt ( $handle, CURLOPT_SSLVERSION, CURL_SSLVERSION_SSLv2) that will tell cURL to tell openSSL to only communicate using SSL version 2, not any other version of SSL or TLS. That’s fine if the other server is willing to communicate in SSLv2, but if they aren’t, like in the case of PayPal, they’ll respond with another error message.
Or, there is a bug in the version of cURL or openSSL in use, which causes openSSL to not auto-negotiate the best version of TLS or SSL to communicate with (the best usually being the most recent version of TLS they both support, or failing that, the most recent version of SSL they both support).
cURL version 7.29 had a problem that it supports TLS 1.2, but it needs to specifically told to use it instead of an different version of SSL or TLS. (This is like unless you tell Mr Curls that it’s ok for Susie to communicate in English, he’ll tell her it’s not ok.) According to this WordPress Trac ticket, the problem could be solved by specifically instructing cURL to use TLS 1.0 or higher (as of 2015, 99% of servers supported it) by using
curl_setopt ( $handle, CURLOPT_SSLVERSION, efefefefefefefef);.
Also, cURL version 7.24 with openSSl 1.0.1e appears to have a different bug: if you added the above-mentioned line telling cURL to use TLS version 1 or higher,
curl_setopt ( $handle, CURLOPT_SSLVERSION, CURL_SSLVERSION_TLSv1 );, openSSL wouldn’t communicate in TLS 1.2. (This is like Principal Powers telling Mr Curls “Susie is allowed to communicate in Spanish, Esperanto, or English” but Mr Curls misheard and missed the last two languages mentioned, and so thought Susie should only be allowed to communicate in Spanish). Ironically, if the PHP code doesn’t specify any SSL or TLS version (or specifies TLS 1.2), openSSL is able to auto-negotiate the best version of TLS to use.
In section “Miscommunications with Mr Curls”, you see that for cURL 7.29 we should add
curl_setopt ( $handle, CURLOPT_SSLVERSION, CURL_SSLVERSION_TLSv1 ); to fix handshake issues, but for cURL 7.24 with openSSL 1.0.1e that actually creates issues. So what’s the best general approach (that works for most version of cURL)?
It was suggested you should just check which version of openSSL is in use, and if it’s a version that know how to use TLS 1.2, the PHP code should tell cURL to use that. But if not, the PHP code should tell cURL to just use the latest version of TLS possible.
But then, it was realized that the version of openSSL that PHP knows about may be different than the version of openSSL cURL is using, so it’s impossible for PHP to determine which version of openSSL cURL is using. Also, we’d still have the second “Miscommunication between Mr Curls and Susie”.
An important point though: for our current purposes, it’s mainly communication with PayPal that was causing us grief.
So, our PHP code now specifically instructs requests going to PayPal to only use TLS1.2. Like so:
curl_setop($handle, CURLOPT_SSLVERSION, 6); (note that 6 is the value of
CURL_SSLVERSION_TLSv1_2, but it’s nice to use its value in case the constant isn’t defined). If the version of openSSL in-use on the server doesn’t know how to use TLS 1.2, the communication will fail. But it’s doomed anyway: it doesn’t know how to communicate with TLS 1.2 and PayPal won’t accept anything else. So the PHP code explicitly tells cURL: use TLS 1.2 or bust.
But, requests going to other servers, which might not yet support TLS 1.2, can fallback to openSSL’s normal TLS/SSL-version auto-negotiation, which get it right 99% of the time.
What’s more, we saw the most popular e-Commerce WordPress plugin, WooCommerce, is doing exactly that too. So we’ll be in pretty good company anyway.
Do you see a problem in my logic? Or any parts that don’t make very much sense? Please let me know in the comments!