April 29, 2019
How to get CF to know a user’s real IP address, when behind a proxy, load balancer, caching solution, etc.
Comments
(17)
April 29, 2019
How to get CF to know a user’s real IP address, when behind a proxy, load balancer, caching solution, etc.
ColdFusion troubleshooter
Guide 146 posts
Followers: 119 people
(17)

Someone asked how they could get CF to track the “real” ip address for someone, when their CF server was operating behind a particular proxy/caching solution–in their case Cloudflare, but it could happen with any number of such solutions. Often, such proxies/load balancers/caching servers will cause the IP address of that “other” server to show up to CF, not that of the originating user.

Why is this a problem? For one thing, some may expect to see the “real” user ip address in the cgi.remote_addr variable, which perhaps they store in a database or log, but instead they will find what is tracked is the proxy’s ip address. A different challenge is for folks expecting to see CF Admin debugging output (assuming it’s enabled, which is not recommended for production): in this case, even if their originating IP address is in the “allowed ip addresses” list in the CF Admin, they won’t see the debug output because CF sees their request coming from the proxy’s IP address instead.

The good news is that there is a solution for this problem. It takes a combination of knowing first what HEADER is passed from the proxy/cache/load balance server to identify the originating IP address, and then setting up CF to now REGARD THAT HEADER as the one holding the IP address. You can enable it with the Tomcat RemoteIPValve (CF 10 and above run on Tomcat out of the box), as I show below.

To be clear, enabling this change will require that you have the authority to change CF config files, and you will need to restart CF for it to take effect. But anyone can use the info in the first section below, to find out WHAT the header is, and then they could ask someone with authority to make the needed change.

Getting the header with the real IP address

So first, note that most such caching/proxy/load balancer solutions which “change the ip address” to be their own will also send along an “http header” to identify the originating user’s IP address. In most cases, the header name is “X-Forwarded-For”, or perhaps “X-Real-IP”.

In the case of Cloudflare, it also passes it in as “CF-Connecting-IP”. (Here the “CF” stands for Cloudflare, not ColdFusion.) That’s useful, because in some environments, another server between CloudFlare and the origin server (where ColdFusion is installed) might itself set those “standard” headers (to some other unexpected value), and in that case can rely on this CF-Connecting-IP header instead.  (For more on these headers from a Cloudflare perspective, see this support page of theirs.)

But next, whatever the header may be, you could find out WHAT headers are coming into a CF page by dumping the result of CF’s gethttprequestdata() function, which shows all incoming headers (and more) passed in to the page (which is showing that dump):

<cfdump var="#getHttpRequestData()#">

More specifically, see the headers struct within that:

<cfdump var="#getHttpRequestData().headers#">

And someone with a CF site behind CloudFlare should see that it does have that header for CF-Connecting-IP, among others. But again look for X-Forwaded-For, X-Real-IP, or others.  Basically, use your browser’s “find” feature to locate what you know to be your “real” IP address, and see what header tracks that value.

Once you know, you could also access its value in CFML code with:

getHttpRequestData().headers["X-Forwarded-For"]

or

getHttpRequestData().headers["CF-Connecting-IP"]

and so on.

Changing CF to USE that header for the IP address it tracks

Even so, getting that value (showing it, or storing it into a variable) is not going to solve the problems above, of either someone wanting CF’s cgi.remote_addr to know the value, or to have the CF admin regard the “real” ip address when deciding whether to show debugging output. Instead, you need to get CF to regard THAT header and its value as the IP that CF knows for those purposes.

To do that, you need to implement a change in the underlying Tomcat configuration of CF. You will enable Tomcat’s RemoteIPValve feature, which is designed specifically for this challenge (not unique to CF at all). You will make a one-line change to a file, and restart CF, then test your changes.

Specifically, in the CF folder (such as C:/ColdFusion2018), you will find the cfusion/runtime/conf folder, and in there a server.xml file. Make a copy of it, before changing it, to revert if you have trouble–CF won’t come up if you make certain mistakes in that file.  (I will note also that if you may run “multiple instances” of CF, then you will find a runtime/conf folder with its own separate server.xml, in the folder for the name of that instance just under your CF folder. )

In the file, you will find an <engine element. It should look like this:

<Engine name="Catalina" defaultHost="localhost" jvmRoute="cfusion">

(Note that if you are looking at the server.xml of some instance other than the cfusion one, then the jvmroute value will name that instance name. I’m NOT proposing you CHANGE that line at all. I just point it out with regard to what you should be searching for.)

Under that line, you can add this a new line, when enables the valve (at the “engine” level) and names that whatever header you identified in the first section as the header holding the real IP address. If it’s X-Forwarded-For, use this:

<Valve className="org.apache.catalina.valves.RemoteIpValve" remoteIpHeader="X-Forwarded-For" />

or to tell it to use the Cloudflare header, use this:

<Valve className="org.apache.catalina.valves.RemoteIpValve" remoteIpHeader="CF-Connecting-IP" />

And keep reading, for one more attribute that you may need to add, to get this to work in your particular environment (an update at the bottom here, though do read the rest before jumping to there).

You may want to make sure there’s not already such a RemoteIpValve in that file first (someone else could have put one there, for use with some other proxy). For instance, one can also enable this valve at the host level, or context level, both under the engine level.

I won’t try to explain that any further here, or if indeed you may need or could want to use any of the various available attributes on that RemoteIPValve, as documented at the Tomcat site:

https://tomcat.apache.org/tomcat-8.5-doc/api/org/apache/catalina/valves/RemoteIpValve.html

But the good news is that that single simple line above may be all you need (along with a restart of CF). If anything goes amiss on restart, revert back to the original server.xml and compare it to what you changed, to see where you may have made a mistake.

Update: I have had a couple of folks share that they DID need to add more attributes to the valve. In particular, if there may be multiple proxies in your environment, or if you find that you KNOW that the desired header is being passed in but STILL not being used for the “right” IP address, you may need to add the internalproxies attribute for the remoteipvalve.

This is needed only if the ip of that proxy is NOT already in the list of IP ranges which the valve supports by default. The default value is a regex, listing several ranges:

10\.\d{1,3}\.\d{1,3}\.\d{1,3}|192\.168\.\d{1,3}\.\d{1,3}| 169\.254\.\d{1,3}\.\d{1,3}|127\.\d{1,3}\.\d{1,3}\.\d{1,3}| 172\.1[6-9]{1}\.\d{1,3}\.\d{1,3}|172\.2[0-9]{1}\.\d{1,3}\.\d{1,3}| 172\.3[0-1]{1}\.\d{1,3}\.\d{1,3}| 0:0:0:0:0:0:0:1|::1

For those not savvy with regex’s and cidr specifications, these are first several ipv4 ranges (10.*.*.*, 192.168.*.*, 169.254.*.*, 127.*.*.*,172.1(6-9)*.*.*, 172.2(0-9).*.*, 172.3(0-1)*.*.*) and the ipv6 localhost addresses (the ipv4 localhost addresses were in the ipv4 list just indicated.)

So if you wanted to ADD to that list, for instance, note that you’d need to repeat that current long value (to keep the defaults) and ADD to it. Then you should be adding a regular expression for the ip address or range that you want to tell it to ignore. So for instance, if your other internal proxy’s IP address is in a range starting with 159.233, you could use the following:

<Valve className="org.apache.catalina.valves.RemoteIpValve" remoteIpHeader="X-Forwarded-For" internalProxies="10\.\d{1,3}\.\d{1,3}\.\d{1,3}|192\.168\.\d{1,3}\.\d{1,3}|169\.254\.\d{1,3}\.\d{1,3}|127\.\d{1,3}\.\d{1,3}\.\d{1,3}|172\.1[6-9]{1}\.\d{1,3}\.\d{1,3}|172\.2[0-9]{1}\.\d{1,3}\.\d{1,3}|172\.3[0-1]{1}\.\d{1,3}\.\d{1,3}|0:0:0:0:0:0:0:1|159\.233\.\d{1,3}\.\d{1,3}"/>

Handling this IP header detection in your web server, instead of CF

Before concluding, let me address something that some readers may be itching to ask about: couldn’t this be handled instead whatever external web server you may have fronting CF (such as IIS or Apache or nginx)? Can’t those be modified so that the WEB SERVER receives and handles the conversion of the “real” ip header? If so, then that would pass into CF that converted IP address.

Yes, that is possible, in different ways with different web servers. It would also then affect what IP address is tracked in that web server’s access logs, tracking visits to your site/s. I will not elaborate on how to do that in different web servers. There are various resources on that, with varying approaches and value.

For instance, in the case of Cloudflare, I will note they have a support section called “Restoring Visitor IPs“, which shows links to pages discussing how to cause their header to be regarded by various web servers and app servers. Sadly, most either talk about affecting the web server access log only (not the IP address that would be passed on to a backend application server, such as CF).

And there’s indeed a page shown there for configuring Tomcat:

https://support.cloudflare.com/hc/en-us/articles/203715940-How-do-I-restore-original-visitor-IP-with-Tomcat7-

But sadly its focus is on configuring Tomcat to affect what appears in the *Tomcat* access log (which is not even enabled in CF by default, except in CF10). Affecting that log may be interesting for some, but it would not help in the cases described at the opening here. who instead need the remoteipvalve solution above.

Even if changing the web server may somehow suit some better, still other readers may find that they can’t make such a change, but perhaps they CAN change CF (by modifying CF’s Tomcat config), which is why I write the above.

I’ll look forward to hearing what others may have to say.  Did I forget anything? Get anything wrong? As always, I’m just trying to help.


For more blog content from Charlie Arehart, see his posts here as well as his posts at carehart.org. And follow him on Twitter and other social media as carehart.

17 Comments
2024-08-15 06:09:21
2024-08-15 06:09:21

I have an update on this post.

First, note that a couple years after my post, Tomcat came out with a new option for naming trustedProxies as an alternative to the internalProxies–which has the advantage that we don’t need to ADD the regex for our desired IPs/CIDR values to the default regex value (as discussed in my post). Instead, we can just list the regex of IPs/CIDR values.

A simple example would be to add the IP range I’d mentioned above,  starting with  a range starting with 159.233, which could be specified simply as:

trustedProxies="159.233.d{1,3}.d{1,3}"

Second, as for working with CloudFlare, note that it offers a web site with a list of all the IPv4 and IPv6 CIDR values from which it could send requests, https://www.cloudflare.com/ips/. There are about two dozen total–and I have created the regex’s for them–which are quite complex. Then they need to all be specified on one line. It makes for a VERY long regex, but it is what it is.

Indeed, while I tried to specify here the valve as set to use those, the formatting of this portal just made it very challenging to present. Here it is instead as a github gist.

Thankfully the list of IP ranges that Cloudflare uses doesn’t change often. But to help future readers wondering if this value is still “correct”, here also is the current list of the range values available at that page, as I write:

173.245.48.0/20
103.21.244.0/22
103.22.200.0/22
103.31.4.0/22
141.101.64.0/18
108.162.192.0/18
190.93.240.0/20
188.114.96.0/20
197.234.240.0/22
198.41.128.0/17
162.158.0.0/15
104.16.0.0/13
104.24.0.0/14
172.64.0.0/13
131.0.72.0/22

2400:cb00::/32
2606:4700::/32
2803:f800::/32
2405:b500::/32
2405:8100::/32
2a06:98c0::/29
2c0f:f248::/32

I will eventually work on editing the post above (or creating a new one). But I wanted to get this out while I was working out this info for a client.

Like
2022-03-24 16:48:14
2022-03-24 16:48:14

I’m using a fully patched CF 10 Enterprise, which is sitting on top of Tomcat 7.0.75. Link to that version’s docs:

https://tomcat.apache.org/tomcat-7.0-doc/api/org/apache/catalina/valves/RemoteIpValve.html

We were having problems trying to get the correct (either) CloudFlare header to pass through as cgi.remote_addr. We’re connecting to our servers over VPN, so we could see the headers outside of the VPN, but not inside it. Fixed an issue there, but we’re not using any internal proxies and still could not see the correct value. Then it dawned on me that CloudFlare’s IP addresses ARE essentially internal proxies from the POV of the server. Rather than trying to filter out their entire range of IP addresses, this is what ended up working:

<Valve className=”org.apache.catalina.valves.RemoteIpValve” internalProxies=”d{1,3}.d{1,3}.d{1,3}.d{1,3}” />

The attribute remoteIPHeader defaults to `x-forwarded-for`, so there’s no point specifying the value unless you want one of the other headers.

Like
(1)
>
iKnowKungFoo
's comment
2022-03-24 17:00:50
2022-03-24 17:00:50
>
iKnowKungFoo
's comment

Thanks for sharing, Adrian. So we should make clear to people that you are basically telling it to trust ANY intervening proxy that may manipulate and share the header that you tell it to use.

That’s understandable when you face a situation where it seems you can’t figure out the correct value to use (just like those who use “.*” for the new allowedRequestAttributesPattern added to Tomcat’s AJP connector in Mar 2020). But again in both cases one is basically circumventing a protection that Tomcat added.

As long as one does it with their eyes open (feeling that the protection is less important than the feature they need to enable), I understand how it may seem that pragmatism must override purity/security sometimes.

If you or anyone sees this differently (and feels that change is “totally safe” to do in all cases), I welcome feedback. I’m just another walker on the path, pointing out highlights as I come across them. 🙂

Finally, as for your observation about x-forwarded-for being the default, fair enough. I suggested specifying it as much for the sake of making clear to someone (who may not know that’s the default) what the header is that’s being used.

Like
2020-07-15 23:31:02
2020-07-15 23:31:02

True-Client-IP is another header name that is used by CloudFlare and others.

Like
(1)
>
Benjamin Reid
's comment
2020-07-16 18:43:46
2020-07-16 18:43:46
>
Benjamin Reid
's comment

Sure, Benjamin, and thanks. To be clear, that’s why when I indicated headers I did refer to “in most cases”, as they are many and varied. (And again, they may be injected by various intercepting resources/proxies along the way.)

More important this is why I showed HOW one would see what header IS being passed in. In some years, still some other header may be come more popular. 🙂

Like
2019-07-01 18:01:32
2019-07-01 18:01:32

Is there any way to identify the “remote port”?  I was using ColdFusion to log some abuse and the remote network admin asked me if I could identify the port used (since they operate a VPN).  Is there any way to identify this using IIS & ColdFusion?  (While researching, I discovered that modern PHP returns this as $_SERVER.REMOTE_PORT.)

Like
(1)
>
James Moberg
's comment
2019-07-02 03:40:34
2019-07-02 03:40:34
>
James Moberg
's comment

Might you mean simply what’s reported in cgi.server_port?

Like
2019-06-07 11:08:39
2019-06-07 11:08:39

Hi, this did not work for me on Azure. We added the valve, restarted, and nothing in the headers for the CF-Connecting-IP.

Like
(2)
>
tribule
's comment
2019-06-07 11:25:24
2019-06-07 11:25:24
>
tribule
's comment

Check out the docs for the valve, for additional attributes that may help.

As I said in a reply to someone else below, see especially the internalproxies attribute, if there is yet some other proxy within your environment (like a load balancer), whose ip is not in the default list of them: 10\.\d{1,3}\.\d{1,3}\.\d{1,3}|192\.168\.\d{1,3}\.\d{1,3}|169\.254.\d{1,3}.\d{1,3}|127\.\d{1,3}\.\d{1,3}\.\d{1,3}|0:0:0:0:0:0:0:1.

If that works, please write back to help confirm for other readers.

Like
>
Charlie Arehart
's comment
2019-06-28 17:15:13
2019-06-28 17:15:13
>
Charlie Arehart
's comment

Tribule, did it work for you, with that additional info?

Like
2019-05-23 14:04:27
2019-05-23 14:04:27

Would you recommend using GetHttpRequestData(false).headers?

I believe we had to do this w/Adobe ColdFusion because it defaults to “true” and then headers can only be read once. (CFDocs.org indicates that this flag defaults to “false”, but official Adobe documentation indicates that it defaults to “true”.)

Like
(1)
>
James Moberg
's comment
2019-05-23 20:41:37
2019-05-23 20:41:37
>
James Moberg
's comment

James, I would think it would be inconsequential for this need (to get the header). That arg is about whether to get the body, and the stated limitation that you can only get it once seems to be talking about the body instead, not the headers.

But if you or anyone gets clarification on it, feel free to share. I have only ever used it without any arg, which (from the docs) defaults to false.

Like
2019-05-22 22:39:25
2019-05-22 22:39:25

Hi Charlie-

I did the change on my production server and did not get cgi.REMOTE_ADDR to reflect the IP in the CF-Connecting-IP but a work around I found was to just use this wherever I was using cgi.REMOTE_ADDR:

var curIP = cgi.REMOTE_ADDR;
if ( structKeyExists(cgi,’CF_Connecting_IP’) )
curIP = cgi.CF_Connecting_IP;

 

Like
(1)
>
suaveodean
's comment
2019-05-22 23:55:24
2019-05-22 23:55:24
>
suaveodean
's comment

Interesting idea, Matt. Thanks for sharing it. It may help some.

As for the remoteipvalve not working, there can be various reasons that would be so. If you visit the Tomcat docs, you’ll see the various additional attributes, which solve various problems. See especially the internalproxies attribute, if there is yet some other proxy within your environment (like a load balancer), whose ip is not in the default list of them: 10\.\d{1,3}\.\d{1,3}\.\d{1,3}|192\.168\.\d{1,3}\.\d{1,3}|169\.254.\d{1,3}.\d{1,3}|127\.\d{1,3}\.\d{1,3}\.\d{1,3}|0:0:0:0:0:0:0:1.

Like
2019-05-01 20:28:05
2019-05-01 20:28:05

Great article Charlie, as always! 

We had this problem years ago here at work when a new proxy was installed and we found that it wasn’t forwarding the real IP at all. We had to ask our network team to change the configuration in the proxy appliance (I’d prefer not to say the name of the device, for security reasons). After they did that we started seeing X-Forwarded-For in the header data.

Like
(2)
>
yacoubean
's comment
2019-05-02 03:23:35
2019-05-02 03:23:35
>
yacoubean
's comment

Thanks for the kind regards, Jake. And had you guys used the remoteipvalve to get CF to regard that header? Or was this perhaps before CF10 (and CF’s Tomcat integration)?

Like
>
Charlie Arehart
's comment
2019-05-07 18:17:50
2019-05-07 18:17:50
>
Charlie Arehart
's comment

Yeah, before CF10.

Like
Add Comment