June 7, 2019
Solving problems calling out of CF via https, by updating JVM
Comments
(2)
June 7, 2019
Solving problems calling out of CF via https, by updating JVM
ColdFusion troubleshooter
Legend 101 posts
Followers: 75 people
(2)

(Originally written Jun 2019, updated Feb 2021)

Sometimes the solution to a problem is not what many may suggest it is. And the right solution may even be easier than a commonly suggested one.

Over the years I’ve had clients present to me (or have seen people write on the web ) that they were getting errors when trying to do a cfhttp call to an https URL, or it could be other tags like cfmail or cfftp, anything calling out from CF to some other server via https. It may be a payment gateway, or an API, or a web service. And they “didn’t change anything” in CF, but suddenly they failures. The good news is that there is generally a simple solution.

TLDR: And I have found that in most cases the solution is NOT necessarily about importing certs or tweaking JVM args but is much simpler: they probably just needed to update the JVM that CF is using.

Importing certs may well have worked for such problems in the past (and may be needed still in some unique cases), but in many cases, that’s NOT the only or even best solution. And if a cert DOES need to be imported, do be careful to import it into the CORRECT location (the JVM CF is using).

Let me explain both situations.

Common error messages

If you have code trying to call out from CF to another server via https, you may at some point find CF reporting errors like these:

I/O Exception: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

or

I/O Exception: peer not authenticated

Or others. And even if not in a CF error, you might find these in the saved output of a CF scheduled task (calling out to an https URL).

Or the more generic error you may see in the output of a cfhttp could just be just a less-helpful:

Connection Failure

Still other errors you may see include:

  • I/O Exception: Could not generate secret
  • I/O Exception: Couldn’t kickstart handshaking

What might be the real cause, when “nothing changed”?

It’s of course very frustrating when this happens and you feel, “nothing changed” on your end.

First, a very common reason for this sort of problem is simply that the destination server being called has changed what SSL or TLS versions or protocols it supports. And it may now be using a version of those which the JVM that your CF is relying on DOES NOT YET SUPPORT. And that JVM CF is relying on may not have been updated in years, as I will explain below.

Second, the destination server may well have changed something about its server certificates, such that the client certs (or root certs) stored within the CF JVM can no longer work with those newer certs. These may be certificates that you did install in the past, or they merely be the ones built into the java certificate store (cacerts, in the lib/security folder of the JVM that CF is using), but now they “don’t work”, again because the server you’re calling has changed, and the JVM (and perhaps default certs) that it uses no longer work.

What if the problem comes and goes?

This can drive you batty, where the failing URL “sometimes works” and “sometimes fails”. In this case, often the problem is that the URL you’re calling is being served by a load balancer on the “other end”, and one of the destination servers (in that cluster) has NOT yet had whatever change was done. So SOME of your requests end up on a server WITH some change (affecting you, that fail) and some of your requests end up on a server WITHOUT the change (and “work” for you).

Why updating the JVM that CF uses may help

Anyway, often the problem is simply that the JVM which underlies CF is just old (perhaps very old). Over the years, the JVM has been updated to support more and more recent protocols (and indeed to drop support for older ones), and to have more and more updated certs by default.

This could happen with ANY version of CF, since CF began running atop Java with CF6. Even with the VERY latest CF version, the JVM it’s using might not have been updated since you installed it months or years ago.

In the case of this one client of mine, they had left CF running the original JVM that came with their CF version released in 2014, which was Java 1.8.0_25. To put things in perspective, as of this writing (2019), that JVM update 25 was 5 years old!, being from 2014, the time when CF11 also came out! FWIW, the latest Java 8 version at this writing is 1.8.0_212 (and Oracle is still updating Java 8 and 11, as “long term support” releases of Java.)

But again, this could happen even with CF2016, 2018 and later. [Update in 2021: it has even helped people running CF2021, as it comes out of the box running Java 11.0.1–which is strange, as the Java 11 version current before it came out was 11.0.9. Anyway, they had these sort of errors, and updating to the then-current 11.0.10 made the problems go away.]

So all we did was to simply install the current latest available Java update (supported by the version of CF they were running), and we told CF to use that, then we restarted CF, and now their https calls via cfhttp and scheduled tasks were working!

How to go about updating the JVM CF uses: questions you may have

When I say “all we did was to simply update” the Java that underlies CF, you may have lots of questions about that. 🙂 Here’s good news, I have done various posts that help with this, including one announcing some recent Java updates and where I addressed typical questions you may have about updating the JVM, including:

  • where to download the latest Java updates, from Adobe (and why)
  • where to find more about HOW TO update the JVM CF that uses
  • what versions of CF support what versions of Java, including Java 11
  • why you should NOT use Java 12, even though it’s “more recent” than Java 11
  • how to recover if you make a mistake trying to update the JVM CF that uses

(I should add that even updating to a slightly older Java update than the very latest one possibly could solve this problem as well, for those who “don’t want to be on the bleeding edge” of latest Java updates. But it’s generally best to be on the latest available update of the Java version supported by your version of CF, and Adobe does recommend that.)

Whether you’ll be updating to that specific Java update or not, check do out that other post for answers to those and some other common questions about updating the JVM that CF users.

What if you really DO need to import certs? Be careful about where you do!

I’ve shared here mostly how it is that for most people they do NOT need to import any certificates into CF’s java certificate store (cacerts).

But some people WILL need to do that. Maybe the server you’re calling is one that only allows calling into it if the http client (in this case, CF) has implemented a special client cert that the server owner has provided. OK, that sometimes IS necessary. Why then may the error above happen when you feel you DID import the cert?

I can tell you that a very common problem is that someone HAS updated the JVM that CF uses (see above), but now they have found instructions on the web showing how to use the java keytool to import the cert into cacerts file found in the lib/security…only the instructions tell you to point it at the cacerts in the /jre/lib/security directly under the CF folder. But if someone has told CF to use a new JVM in a new location (again, see above), then CF is NO LONGER paying attention to that cacerts in its jre/lib/security folder.

In that case, the simple solution is to import the cert (if indeed it’s really needed) into the lib/security/cacerts of whatever JVM you are using. Look at the CF Admin and its “java and JVM” page, to see what the java home (first field) points to. THAT is the JVM CF is using (also found in the java.home line of CF’s jvm.config file.)

Note that if the JVM is a Java 8 JRE, the folder will be in jre/lib/security, while in a JDK (and for Java 11, that’s all there is), you will find the cacerts in the JVM’s lib/security folder.

What about cacerts in other places within CF?

You may also discover that there are lib/security/cacerts folders in other places within, particularly its jnibridge and jetty folders (under CF’s cfusion folder, or under any sibling “instance” folder, if you are running CF Enterprise and created multiple instances.). Those are NOT certs you need to worry about. Those are for the JVM’s implemented in support of the process of INSTALLING the features that use those two folders, namely .NET integration (for the jnibridge folder) and Solr operations and CFHTMLtoPDF (for the jetty folder). You can safely ignore those.

What you may not need to bother with

So really, the JVM update may be all most need, and not any other approaches. But what are the other approaches that some may suggest, that you may not need?

First, you may NOT need to import any certificates after all. You will often see resources on the web proposing that as the solution. Indeed, one of the resources my client had found (which led them to ask if they needed to go about importing a new cert) was this blog post of years ago from the venerable hass.de site.

He was explaining why importing certs may be needed to solve some SSL problems.  Again, sometimes it may be. Again, for the client I helped which prompted this post, they were using an older CF with an older JVM from years ago. It may well be that such an old JVM had expired certificates and root certs, leading to a need for some (at some time in the past) to do such cert imports, before a newer jvm included the better certs.   (Indeed, sometimes people have been importing certs by rote for years every time they did update JVMs thinking they needed them, when instead all they needed was to update the JVM to get the better certs and root certs it would have by default.)

But again just updating Java worked for my client and I hope it may for some of my readers.

And that hass.de blog did indeed mention in passing that “an update to latest Java sometimes also help, too.” That would be easy to miss amid all else in the post.  One reason I point this out here is that sadly that hass.de site no longer allows comments, so I couldn’t add this clarification there. (I get it that more and more sites are not permitting comments. I see the pros AND cons.)

Second, the commonly suggested Dhttps.protocols arg may not help or be needed: again you may see resources on the web proposing adding those as a solution for some cf ssl issues. But it is not the right solution for this more common problem (of the destination server changing what it supports, and that being something that the JVM CF is running on does NOT support).

This Dhttps.protocols arg would be for limiting what protocols CF/the JVM should use. But the SSL negotiation between CF and the destination server (such as in a CFHTTP call or scheduled task URL) should cause CF to use whatever the best protocol the destination DOES support. But again, if the ones that the destination DOES support are not ones that the JVM underlying CF supports, you will get the error. As such, the args won’t help, and really it’s rare that you need this arg anyway to “limit” what protocols CF would try to use.

Bottom line: Try updating the Java that CF uses

My goal here was just to get the word out that for THAT error above (and perhaps other failures to call out of CF to https pages), you may not need to bother importing certs. You may need to just update the JVM. It’s worth a shot. And you can easily revert back to the previous JVM version, if you follow best practices in updating the JVM that CF users, as discussed in resources I point out in that other post. Or see also a resource page I have on updating Java, and the page also covers updating CF, the CF web server config, and more..

Hope all this may help some readers.


For more blog posts from Charlie Arehart, see his posts here as well as his posts at carehart.org. And follow him on Twitter and other social media as carehart.

2 Comments
2019-06-09 23:45:31
2019-06-09 23:45:31

Hi Charlie,

thanks you for your article. You speak of “some unique cases”. I’d say it depend’s on when you get the exception. If you try to connect to some internal server it likely is the case that the cert is not created by some trusted autority and needs to be imported. If you try to access a public facing website and get the exception, it likely uses some recent cert that was not part of your original Java install.

Like
(2)
(1)
>
Bardnet
's comment
2019-06-10 00:07:25
2019-06-10 00:07:25
>
Bardnet
's comment

Right, thanks Bernhard. I was torn about elaborating on the other cases. Fair point that that’s the most likely one.

I figured that for anyone who might find this jvm update alone didn’t help, they’d easily find other discussions of importing certs.

I will add that those who do that should beware that IF someone has indeed already changed the jvm CF uses, they need to be sure to import the cert into the lib/security /cacerts of THAT jvm, not the one in CF’s jre/lib–as most resources would presume to tell folks.

Like
Add Comment