The ColdFusion 10 Update 4 is now available for install within your administrator. It includes several important bug fixes. ColdFusion 10 Update 4 is a cumulative update. It includes all the bug fixes from previous updates of ColdFusion 10. All the issues reported in Update 3 have been resolved in this update.
The details of the update is available here.
Staff
109 posts
This issue with the tech note has been fixed. You can access it here:
http://helpx.adobe.com/coldfusion/kb/coldfusion-10-update-4.html
The update has some connector fixes which is why you need to reconfigure the connector
The link to the technote is broken – both in this blog post and in the CFIDE. It doesn’t seem to exist on Adobe.com any more.
Why are we reinstalling the connector? I read on this blog [ http://www.cfdad.com/tag/coldfusion-10/ ] that it fixes a problem with CGI.
Silly me. I was trying to install the 64bit version on a 32bit OS.
Hello,
Could you help me please, I’m running a Windows Server 2003 SP2 and am trying to install CF10, and it just can not, and WILL not install, I’ve spent many hours so far and got nowhere. The Win 2k3 server is a hosted OS running under Hyper-V. The error I get once InstallAnywhere has got to 100% is: Windows error 216 occured while loading the Java VM
Any help much appreciated! 🙂
Update 5 worked great for me and resolved the vulnerability. I had previously performed the steps in the Tuning coldFusion 10 IIS Connector Configuration article. These were sent to me by Adobe Support about 10 days before they were published in the blog post. That had taken care of the random 503’s we were experiencing during or after a heavy load. Just a note that if you do the tuning, then remove and add the connector, it will blow away your tuning, so take a note of which values you add/change first or use the -upgrade flag with wsconfig.exe.
@Doug S, on that matter of “when will CF support Windows 8 (and also OS X Mountain Lion)?”, note that the CF team blog has addressed that in a separate entry:
https://coldfusion.adobe.com/post.cfm/coldfusion-10-support-on-windows-8-and-os-x-mountain-lion
I’m not saying you’ll like the answer there (support is still some months away), of course. I’m just proposing that it would be better to offer any further discussion there, rather than here, especially for others interested in the matter who might be following that entry and its comments.
Since no one has yet mentioned it, there was an updater 5 released today, and it addresses specific issues with IIS. For more info, see the blog entry on it:
https://coldfusion.adobe.com/post.cfm/coldfusion-10-update-5-security-update-now-available
Also, readers of this blog entry on updater 4 will also want to note the other new blog entry here created a couple of days ago, “Tuning ColdFusion 10 IIS Connector configuration”:
https://coldfusion.adobe.com/post.cfm/tuning-coldfusion-10-iis-connector-configuration
@Krishna – It turned out to be permissions related. When I switched the runtime user for the service from my custom user to ‘local system user’ the update worked. Then I just switched my service back to the correct runtime user after the update completed.
Another customer with big CF10 problems. Subscribing to thread.
Well Windows 8 is well and truly here. So where are the fricking updated installers? 12 months in pre-release and Adobe is still not ready? FFS!
11/16/2012 09:11:26 AM has passed, and I do not see the anticipated error grouping within the Event Viewer.
So, this has stopped the faults. But what’s causing the requests every 10 minutes that were creating the faults in the first place!?
Still a little baffled.
FYI, just added the “/jakarta/* = cfusion” line to uriworkermap.properties and cycled services. Laying in wait to see if the anticipated 11/16/2012 09:11:26 AM fault appears in the event viewer.
@Aaron
So, I was able to do a little debugging and server monitoring this morning. I had thought about doing it last night, but we get off-peak traffic at that point, so I opted to wait until this morning (under our heaviest daily loads) to test.
As you recommended, I stopped IIS and ColdFusion, then modified the isapi_redirect.properties files to put the connector into “debug” mode. After that, I fired ColdFusion up, then IIS, and then waited a bit. As was expected, I could see the isapi_redirect.log file growing rapidly, so I knew the configuration change had taken place.
After a few minutes, I decided to try the “exploit” myself (as you did on your server). I typed in that URL into my web browser and, sure enough, got an error back from the server. I was simultaneously monitoring the Application Log via Event Viewer on the server in question. As my request failed, I watched as 7 sets of error/info messages were logged to the Event Viewer – all of the exact same nature I’d been seeing in the past. One error message followed by two information messages – but repeated 7 times, in this case, for some reason.
I then continued to wait in debug mode until another entry appeared in Event Viewer that wasn’t deliberately triggered by me. 7 minutes later, it appeared. One error and two information entries – just as expected.
At that point, I stopped IIS, stopped ColdFusion, put the web connector back in “info” mode, started ColdFusion, and started IIS in order to prevent further rapid growth of the connector log. I archived the 10MB log and downloaded it to my machine for local inspection.
The catalyst event in the Event Viewer had a timestamp of 11/16/2012 08:21:16, so I scanned forward through the log looking for instances of “attempting to map uri” around that same time period. To my surprise and your prediction, I found a matching entry in the log at 11/16/2012 08:21:15.968 that matches the exploit that had been mentioned in comment #45 (remaining vague in order to avoid deletion of this VERY important comment).
So, as you suspected, something… somehow… somewhere… with some frequency… is attempting to access that URL directly and is causing the faults we’ve been seeing. I’m still left guessing as to why these requests are coming in.
But one odd thing I noticed is that frequency and spacing of some of these requests. After harvesting this log, I returned to the server to refresh the Event Viewer and found 4 additional faults in the Application Log after the first one I observed that prompted me to harvest the log and put the server back in “info” logging mode. What’s odd about them is the time series of all 5 faults that occurred naturally (without my specific request of the URL).
Fault 1: 11/16/2012 08:21:16 AM
Fault 2: 11/16/2012 08:31:18 AM
Fault 3: 11/16/2012 08:41:20 AM
Fault 4: 11/16/2012 08:51:22 AM
Fault 5: 11/16/2012 09:01:24 AM
The faults are almost exactly 10 minutes apart from each other. There’s absolutely NO WAY that’s a mere coincidence. And that led me to believe one of two theories.
My first theory is that perhaps an outside source (rival, competitor, “script kiddie”) may have setup a repeating scheduled job to hit this exploit URL on our server as a means of initiating a DOS incident. Then I thought to myself – if that were the case, said outside source would CERTAINLY need to be calling this URL more frequently. Even if we had IIS’s Rapid-Fail Protection enabled (which we presently do not), the default trigger point for an app pool shutdown is 5 faults within 5 minutes. So, one fault request every 10 minutes wouldn’t get the job done if a DOS attack were the goal.
Which leads me to my second theory.
These requests MUST be coming from something internal to ColdFusion. I ravaged through all the time/interval settings I could located within ColdFusion to see if anything is configured on a 10 minute interval, but I came up empty handed. Nevertheless, this doesn’t eliminate some ColdFusion process as the culprit. It just means it’s not a user-configurable option or interval.
At this point, I’m planning to make the suggested change and add “/jakarta/* = cfusion” to the uriworkermap.properties file. However, considering all I’ve gone through in hunting this down and all the pressure I’ve gotten from my client to produce some sort of explanation, I’d really like to find out what’s initiating these requests.
Any ideas?
@Russ : Yes, it is possible to request for a CF9 license key via backward licensing policy.
I will put you in touch with the team at Adobe who can help you with this.
@Russ, that part of my reply was responding to your having said, “I’ve been hunting for CF9 Standard Upgrade, but can’t find it anywhere. Anyone know where it might be available?”
If I somehow misunderstood, sorry. But maybe the info may help someone else, or another reader here may be able to pass it on to someone else if they see it asked elsewhere. Like others here, I’m just always trying to move the ball down the field. 🙂
@charlie
My issue really isn’t with the availability of an upgrade installer (or lack thereof), but more to do with being able to purchase a valid CF9 license key. As you said, hopefully someone from Adobe will have a solution for that.
Russ, there is no “upgrade” installer for 8 to 9. There is only ever the installer for each release, and the last step detects if you have a previous install and offers to import settings to the new release (which co-exists).
You can get the installer at http://www.adobe.com/support/coldfusion/downloads_updates.html#cf9. That is 9.0.2 (for more on that, and differences from 9.0 and 9.0.1, see http://helpx.adobe.com/coldfusion/release-note/coldfusion-9-0-update-2.html).
Basically 9.0.2 is 9.0.1, minus Verity, plus all the hotfixes, cumulative hotfixes, and security hotfixes that were available for 9.0.1 at the time of 9.0.2’s release in late May. It also throws in a couple of tiny features that were added in CF10. See the technote for details.
Note as well that the 9.0.2 installer is a full installer, not an updater (from 9.0 or 9.0.1.)
Those with a support agreement with Adobe can get 9.0 or 9.0.1, but it’s not available publicly due to the expiration of the agreement with Verity, which required removal of all public links to releases containing it.
As for a CF10 license key working with 9, I do not believe that will work. They are different licenses. Someone from Adobe may chime in with more details.
I need to upgrade a client who is currently running CF8, but I sure don’t want to go to CF10 yet. I’ve been hunting for CF9 Standard Upgrade, but can’t find it anywhere. Anyone know where it might be available? Will a CF10 license key work with a downloaded copy of CF9? I’d certainly be happy to go that route if it would work.
@Christian
Thanks for pointing out the differentiation between the update 2 connector and update 4 connector. Unfortunately, I was able to verify that the mod date and file size of my isapi_redirect.dll indicate that it is, in fact, the update 4 connector and we’re still experiencing problems with it in place. I have plans this evening to take Aaron’s advice on throwing the connector into “debug” mode until an error is encountered, and then dive into the logged information to determine the nature of our errors. Once I’ve done that, I’ll report back here with more information.
@Aaron
Yes, the server in #56 & #75 is the same. Apparently I failed to reconfigure the connector with the wsconfig tool after Update 1 as per the instructions so your Date Last Modified timestamps for isapi_redirect.dll
are more accurate
Forgot that I gave the false impression that #67 would 100% resolve #45’s vulnerability. It *currently* does not, due to a case-sensitive Tomcat sitting on a case-insensitive Windows/IIS. However, it should resolve about 99% of the vulnerability. This case-sensitivity issue (#3199283) is known and is marked Fixed/ToTest. Thus, it is safe to say that *soon* there should be 100% resolution of the vulnerability.
Thanks,
-Aaron
@Adobe,
After uninstalling all updates, then installing the Mandatory Update, and then reconfiguring the connector, CF Admin always said 0 Updates Available. Repro:
1) Uninstall all updates
2) Install Mandatory Update
3) Reconfigure connector, restart CF, and CTRL+F5 the browser
4) CF Admin says 0 Updates Available and logs the following to update.log: “Not able to connect to Update Site: Variable VERSIONSTR is undefined.”
5) Install Update 1 manually, then repeat #3 and see #4
6) Install Update 2 manually, then repeat #3 and see #4
7) Install Update 4 manually, then repeat #3 and see CF Admin says 0 Updates Available (understandable, since U4 is most recent update) and no error is logged in update.log (so perhaps Update4 has the VERSIONSTR fix, but Update1 and 2 do not).
Just an observation. It could cause confusion if user wants to just install Update 1 or Update 2.
Thanks,
-Aaron
First, here are the Date Last Modified timestamps for the connectors:
– If CF10 w/ no updates, then 3/29/2012
– If CF10 Update 1 or 2, then 8/9/2012
– If CF10 Update 4, then 10/22/2012
@Tyson, Apologies. I thought you were subscribed before #45 and had its email. Regarding the 403, is it specifically “403.14 – Directory listing denied”? If so, that does not resolve the issue in #45.
@Christian, Is the server in #56 & #75 the same? If so, based on the timestamps mentioned in #75, IIS was never configured for Update 2 and perhaps this is the cause of the Update 2 issue mentioned in #56. I experienced same issue with Remove not deleting the /1/ directory, and Add creating the /2/ directory. This was b/c I had one of /1/’s files open in notepad which was preventing the deletion of the /1/ directory. Once I closed the file, then Remove was able to delete /1/ and then Add recreated /1/.
@Matt, The steps I mentioned in #67 are not the official fix for #45. Until Adobe confirms it, then it’s still technically ‘use at your own risk’. I am using it b/c not using it allows anyone to stop the site’s application pool if any of CF10’s updates are installed and if IIS’s Rapid-Fail is enabled (which is the default for IIS7, and I _believe_ IIS6 as well).
@All, Since comment #45 was deleted, I can’t go into further detail about it. Also, I cannot take credit for the steps in #67 and also I cannot reveal where I learned of them from. Just suffice it to say that “IMO” those steps are necessary for: 1) closing a security hole, and 2) determining the cause of the 503 errors. While putting the connector into debug mode can help in determining the cause of the 503 errors, it does not close the security hole. Additionally, debug mode will cause the connector’s log file to grow large fast and make it difficult to analyze.
The steps I mentioned in #67 will allow CF to throw a 404 (instead of the 503) which will be logged in the site’s IIS log. Thus, when the Event Viewer shows a isapi_redirect.dll error, then we can look to the site’s IIS log and see what URL was requested. There are possibly various causes of the 503 error. The 503 error that #67 addresses does not exist in CF10 with no updates. It only exists if Update 1 or higher has been installed. Seeing the logged request URLs should help reveal any patterns and specific causes of the 503 each is seeing.
@All, I would recommend corresponding with Adobe via email (as mentioned in #71) for troubleshooting the cause of the 503 error. I will contact Adobe regarding the difference I noticed between CF10 with no updates and CF10 w/ updates 1+ installed.
Thanks,
-Aaron
@Aaron,
This is in regards to the steps you describe in comment #67 for fixing the vulnerability (previously) described in #45. Is this the official fix or is this considered a use at your own risk type of fix? I made the change on my development server and the vulnerability appears to have been closed. I just don’t want to make the change on my production box if it could cause other issues. Thanks!
For those of you still having issues, you may want to double check that the connectors were reconfigured properly. The instructions in the tech note say to use the wsconfig tool, but no details on how to use it. My first go around I did not do it properly. My second go around I hit the “Remove” button, but the “Add” button did not configure everything correctly when I selected “All” IIS websites. In fact, all it did was create a C:ColdFusion10configwsconfig2 folder and then a C:ColdFusion10configwsconfig3 folder when I ran it again.
Within the /2/ and /3/ folders I noticed that the isapi_redirect.dll had a different date modified(10/22/12) and file size even though both are version 1.2.32.0, but the /1/ folder still had the “old” file with date modified 3/29/12.
I then stumbled on the Update_all_connectors.bat file which “upgraded” the connector automatically for me
C:ColdFusion10/cfusionbinconnectors>Upgrade_all_connectors.bat
WARNING! This will upgrade ALL ColdFusion MX web server connectors.
Make sure ColdFusion MX Application Server and Microsoft IIS are not running before running the upgrade.
Press Control+C to abort.
Press any key to continue . . .
command line: -upgrade -v
Created file C:ColdFusion10configwsconfigcfwin32.dll
Created file C:UsersCPOLIN~1AppDataLocalTemp1ExecuteAppCmdExecuteAppCmd.exe
Stopped “World Wide Web Publishing Service” service
Created file C:ColdFusion10configwsconfig1isapi_redirect.dll
Started “World Wide Web Publishing Service” service
The Internet Information Server (IIS) connector was upgraded in All
C:ColdFusion10cfusionbinconnectors>
I don’t know if this process is correct since there aren’t any instructions and I’m no Tomcat expert, but that is one place I’d look.
“old” isapi_redirect.dll – 388,608 bytes
“new” isapi_redirect.dll – 389,120 bytes
@Aaron
I’m unable to view comment #45, so I can’t tell you whether it’s related to the problems we’re having or not.
I’m not personally observing 503 errors on my client’s server. My only indication that anything is going wrong are the entries in the Event Viewer.
But at this point, I’m a bit leery of poking around and starting to make configuration changes on the client’s production server without a clear understanding of the nature and impact of the changes. Specifically, I’m not certain why I would want to add “/jakarta/* = cfusion” to the uriworkermap.properties file. Presently if I attempt to access anything within the /jakarta/ virtual directory, I get a 403 Forbidden error from IIS – which is perfectly fine with me. If you can explain to me the justification for making this change and how it potentially affects my particular issue, I’d be able to consider it further.
In order to put the server into connector “debug” mode instead of “information” mode, I’ll need to wait until an off-peak time since it requires a service restart. Since this is non-invasive, I’ll probably give this a shot and see what, if anything, I’m able to drudge up from the logs when these errors are taking place.
I’ll return with more information as it’s available.
For anyone else that runs into my same error…obviously in order to get around it, I had to edit the “Sharing & Permissions” to “Read & Write” and then run the “Install” again. Cheers!
Attempting to install this hotfix results in the following error found in the log:
Failed to copy hotfix files:/tmp/808645.tmp/dist/cfusion
Status: FATAL ERROR
Additional Notes: FATAL ERROR – Failed to copy the hotfix files to the target location:/Applications/ColdFusion10/cfusion
FATAL ERROR – /Applications/ColdFusion10/cfusion/bin/cfcompile.bat (Permission denied)
Forgot to mention, in that last post, those recycle events will be logged to the Event Viewer.
FWIW, here is a script, that can be ran from command prompt, which will enable logging of all IIS7+ app pool recycle events:
C:WindowsSystem32inetsrvappcmd set config /section:applicationPools /applicationPoolDefaults.recycling.logEventOnRecycle:”Time, Requests, Schedule, Memory, IsapiUnhealthy, OnDemand, ConfigChange, PrivateMemory”
To undo that, just run:
C:WindowsSystem32inetsrvappcmd set config /section:applicationPools /-applicationPoolDefaults.recycling
Thanks,
-Aaron
@Tyson,
Here’s another thing to try:
1) Open ColdFusion10configwsconfig1isapi_redirect.properties
2) Change log_level= info to log_level= debug
3) Restart CF & WWW services
4) Immediately after seeing a 503 in Event Viewer, undo #2 (change debug to info and restart CF & WWW). B/c isapi_redirect.log will be growing rapidly in debug mode.
5) CTRL+F the ColdFusion10configwsconfig1isapi_redirect.log for “is not a servlet url” and then “Attempting to map URI” to see if you can find any URIs that failed to resolve and that you find suspicious.
Just a thought. But that’s the only thing I could think of ATM to get more verbose logging of what’s going on.
If it helps, I can attach a script which you can schedule to run via cfschedule which will start any stopped IIS7+ application pools and then send you an email letting you know which app pools had been stopped and which ones it had started. Then you can re-enable Rapid-Fail and just be notified upon each failure so that you can investigate.
Thanks,
-Aaron
@Tyson, what timing 🙂 At 1:26am EST I’d intentionally triggered a 503 by directly accessing the URL mentioned in the deleted comment #45.
Requests for that URL are logged nowhere that I can find. Not in the IIS logs, CF logs, the ColdFusion10cfusionruntimelogs etc. (I only verified the 1:26 timestamp by looking at my browser’s history log)
If the 503 issue mentioned in #45 is the same issue that you are seeing, then the following should stop the 503s:
1) Open ColdFusion10configwsconfig1uriworkermap.properties
2) Add /jakarta/* = cfusion
3) Save and then restart CF and the WWW service
Could you try that and see if the 503 errors stop?
Thanks,
-Aaron
Well, my client is running one single web site on their server, so there’s only one app pool involved.
As I check in on the newly patched server this morning, I’m still seeing the same issues in the Event Viewer.
—[ SNIP ]—
Faulting application name: w3wp.exe, version: 7.5.7601.17514, time stamp: 0x4ce7afa2
Faulting module name: isapi_redirect.dll, version: 1.2.32.0, time stamp: 0x50850ee6
Exception code: 0xc0000005
Fault offset: 0x00000000000118c1
Faulting process id: 0x1d8
Faulting application start time: 0x01cdc1b69b53318f
Faulting application path: c:windowssystem32inetsrvw3wp.exe
Faulting module path: D:ColdFusion10configwsconfig2isapi_redirect.dll
Report Id: 978633cb-2daa-11e2-9b05-0050568a6da8
—[ SNIP ]—
… and …
—[ SNIP ]—
Fault bucket , type 0
Event Name: APPCRASH
Response: Not available
Cab Id: 0
Problem signature:
P1: w3wp.exe
P2: 7.5.7601.17514
P3: 4ce7afa2
P4: isapi_redirect.dll
P5: 1.2.32.0
P6: 50850ee6
P7: c0000005
P8: 00000000000118c1
P9:
P10:
Attached files:
These files may be available here:
C:ProgramDataMicrosoftWindowsWERReportQueueAppCrash_w3wp.exe_c7987f849945058256ab720f046114e117b593e_21c4f00e
Analysis symbol:
Rechecking for solution: 0
Report Id: 978633cb-2daa-11e2-9b05-0050568a6da8
Report Status: 4
—[ SNIP ]—
Now, I haven’t specifically seen a server crash since the update last night, but I hadn’t really seen a server crash in the past few days prior to the update either. Remember, I already disabled IIS 7.5’s Rapid-Fail Protection that was causing the respective app pool to be killed if 5 of these faults were received within a 5 minute period. However, I have had a few server crashes prior to update 4 even with that Rapid-Fail Protection disabled. So with these faults continuing to amass in our Event Viewer (there are 614 of them already in just the past 10 hours), I highly expect that the server is at risk of similar 503 errors and/or crashes.
Side note: When I attempted to visit your site last night as I was posting, Aaron, I was getting a 503 service unavailable error message from your web host.
Subscribing
I wonder if it is specific CFML which causes the 503 Service Unavailable.
Could those experiencing the 503 Service Unavailable see if it is typically the same app pools that are getting stopped?
If some app pools never get stopped, but some app pools “randomly” stop, then that may help us narrow down the issue.
It may not be the load, but rather some specific CFML which is causing the app pool to crash. If anyone has further clues it would help.
Thanks,
-Aaron
To clarify, I had made the changes to IIS Rapid-Fail Protection many weeks ago prior to installing update 4 since it was the only way I could manage to keep the server from constantly crashing. However, there’s no acceptable reason why I should still be seeing w3wp.exe/isapi_redirect.dll faults in the Event Viewer now that this update is installed. Adobe, what gives!? I’ve been told this update was “tested” with numerous clients before it was released. It took all of 4 minutes hitting our site with LoadImpact for me to product Event Viewer faults. How could this have been overlooked?
I’m back with an update. I just installed update 4 on our production server this evening (after performing a VM snapshot as an emergency fallback). After installing the update, I ran a load test against the server. Sadly, I’m still seeing errors in the Application Log within the Windows Event Viewer that indicate that the w3wp.exe service and isapi_redirect.dll are still throwing faults under load. When I had first discovered this, I modified IIS 7.5’s Rapid-Fail Protection and simply turned it off since it was resulting in our IIS AppPools being killed. So now my AppPools are no longer being killed, but I’m still seeing these w3wp.exe/isapi_redirect.dll faults in Windows Event Viewer with update 4 installed. To me, that means update 4 hasn’t resolved the issues it claims to resolve. We’ll see how the server holds up (or not) under normal customer load tomorrow during business hours.
I see one cause of 503 Service Unavailable on IIS is a stopped application pool. And one resolution is starting or restarting the site’s application pool.
Can those experiencing 503 Service Unavailable please check the site’s application pool in IIS Manager and see if it became stopped?
Thanks,
-Aaron
We experienced the 503 errors on our development server – so held off upgrading production from 9 to 10.
After the release of update 4 I figured it sounded like Adobe had fixed the issues. I was wrong…
Upgrade on production went smoothly – but within half an hour the 503/unavailable errors started…
I attempted to revert to cf9 – but that was a disaster too – issues with web connectors and 403 issues.
Finally got it back up – with 2 hours downtime – outside of any maintenance window.
This really isn’t on. I don’t know how this issue got past beta – let alone 4th update. When update 5 comes out – I’ll be waiting to hear others feedback before believing Adobe again…
Really disappointed!
Please disregard my comment (#31) about difficulty installing Updater 4 Final. Unable to reproduce that. Must’ve been user error on my part (I prob didn’t go thru the motions in the correct order). I’ve reinstalled it on same machine, as well as two others, and it installed flawlessly every time.
Thanks,
-Aaron
Just subscribing
Christian, thank you so much for the feedback. As I author this reply, I’m literally wrapping up on the phone with Adobe obtaining a backwards license for ColdFusion 9. I’ve just spent the past 46 minutes on the phone being bounced through various call centers at Adobe (Customer Support, Customer Support Supervisor, Volume Licensing Support, and ultimately ColdFusion Tech Support) attempting to accomplish this. It has proven a bit difficult since the client I’m representing technically upgraded from 8 to 10, so they never owned 9 – and they purchased their license through a 3rd party distributor. Ugh. Nevertheless, I persevered and am now patiently awaiting email delivery of license keys for ColdFusion 9. Sadly, I’ve got a massive client whose production server is sitting on CF10u2 with persistent service crashes, and I just need to get them stabilized as quickly as possible before I do anything else. My plan is to first get their production server degraded to ColdFusion 9, just to stop the crashing and get them on something stable. Once their production server is stabilized, I plan to proceed with ColdFusion 10 update 4 on their test server. There, I will perform various metrics and stress and load tests on that server to ensure that we are no longer victim to the awful 503 errors under load. Once I’m able to verify that, I then plan to proceed cautiously with upgrading their production server to ColdFusion 10 update 4 as a final resting place.
In order to better serve the community and those who have been affected by similar scenarios, I plan to report back here with my final experience and findings. There’s definitely a path to resolution ahead of me, I just never would have expected it to be so daunting and unnecessarily complex. Frankly, the whole experience has crushed a bit of my nearly fan-boy confidence in and support of Adobe as the flagship representative for ColdFusion. That’s a pretty significant statement when you consider that I’ve been working with ColdFusion since version 0.9 beta back in mid 1995. Believe it or not, I probably still have an old ColdFusion 1.0 binary (anyone remember /cgi-bin/dbml.exe?template=/index.dbm ???) and accompanying application code sitting on a Zip Drive (another nostalgic reference) somewhere in a cabinet in my office.
As mentioned, I’ll report back (likely next week) with my findings. Thanks to all those who have continued to interact with this thread. 🙂
@Tyson
We just finished scanning our site with Security Metrics and CF10 Update 4 and it survived without any issues whereas it crashed a horrible death last week when only Update 2 was installed so you can add me to the happy camper category.
@Adam
Considering the circumstances and the “black eye” from update 3, I can only partially agree. That said, there’s simply no way that I’ll proceed (nor will my client -allow- me to proceed) with update 4 after the embarrassing debacle and ultimate recall of update 3. That whole fiasco cost me and a client 2 full days of installation, testing, troubleshooting, hair pulling, and an ultimate contingency fall back to a pre-update VM snapshot nearly 48 hours later to ColdFusion 10 update 2. The only feedback I’ve heard anywhere in the ColdFusion community thus far about update 4 consists of complaints from numerous sources that the 503 service problems still persist. I’ve not been able to personally with even a -single- source that the update solved these problems for anyone. At this point, I don’t consider Adobe’s self-proclaimed “thumbs up” on this update a reliable indicator of its effectiveness.
Please… ANYBODY… just point me to ONE single client or person that was having 503 service errors resulting from bug 3222748, 3216317, 3318104, or 3300889 that has installed update 4, and maybe I can reconsider my position on this. Anyone? Anyone!?
Tyson, I suspect your analysis is a good one, but bear in mind one seldom hears from *happy* clients; one only hears from the ones that are annoyed.
—
Adam
@Kiran
If that’s the case, then why is it that the only public feedback I’m seeing is that of people who have installed the update and are indicating that they are continuing to experience service outages and 503 errors with update 4 applied? What sort of testing could have been performed that completely missed the fact that 503 errors are still being reported with this update installed?
Not to mention, if update 4 truly resolved the issues, then why is Matt working with Adobe over the span of many days to implement and test additional changes? This means that update 4 is NOT a successful release, and I won’t go anywhere near it with my own clients until I can assure them that the update will resolve the problems they’ve been ravaged by for the past few weeks.
Still completely dissatisfied.
Tyson,
I’ve been working with Adobe the last few days to implement some additional changes after applying Update 4. Our system was lasting anywhere from 1-4 days before we’d be plagued with the 503. The verdict is still out as to whether these latest tuning settings completely fix the problem. As soon as I know whether they work or not I will be in contact with Adobe and will post the results here as well.
So, I still don’t see anyone in here rejoicing and claiming that “update 4” has resolved the service unavailable (503) errors plaguing the IIS web connector. Should I be assuming, at this point, that this update is no more effective than update 3? I still have a massive client sitting on ColdFusion 10 update 2 on 64-bit Windows 2008, IIS 7.5 that is experiencing frequent unpredictable service crashes resulting from the constant faults being thrown internally by the IIS connector. This particular client upgraded directly from ColdFusion 8 to ColdFusion 10 (yikes, I know). At this point, I’m planning to call Adobe and demand that they issue my client two license keys for ColdFusion 9 so that we can AT LEAST fall back and return to a stable build of the application server. The current state we’re stuck in is absolutely absurd and unacceptable. I haven’t seen problems as bad as this in ColdFusion since the days of leaky ODBC drivers back before MX. As a 17-year ColdFusion veteran and evangelist, I’m feel absolutely embarrassed by the outstanding issues and with ColdFusion 10’s stability, as well as with the sloppiness and ineffectiveness of the updates that are being released. Adobe, you ought to be embarrassed, too.
@Jake Hand,
Re-running the Update again should fix the issue for you.
Can you please re-install from Administrator and check and let me know.
@raZorTT,
Can you please check the Comment#14 and if there are errors in the log can you please re-run the installation.
Thanks,
Krishna
Anyone have any thoughts about why the version number might not have been updated? I ran the update through the admin UI and it again didn’t update the number. I’m thinking it’s something to do with file permissions?
Cheers,
Simon
We have the same problem as Matt. We lost motions days to install, disinstalare CF 10. We went back to 9. Great inconvenience caused to customers. Such a thing has never occurred in the history of CF. Hopefully it will be resolved all at the earliest.
Mine was still crashing, even more than before the update. Became unusable. Solved all the problems by going back to CF9. Now have a stable production ready server again.
*** This comment has been deleted as this seems like a security vulnerability. We have taken a note of this and we are investigating this. ***
– Adobe ColdFusion team
ours is still crashing with the “Service Temporary Unavailable!” after applying update 4 and removing/adding the connector. We’re running on 2008 R2 x64, IIS 7.5
(Commenting again to subscribe to comments.)
@Tyson, I’ll be able to answer that on Thursday when Security Metrics scans our site again. The last scan crashed ColdFusion
So, has anyone been able to verify that this updater (when installed properly) is an effective fix to the bugs 3222748, 3216317, and 3318104? So far, I see nothing in the CF community that instills confidence that this updater has resolved those bugs effectively. I’m stuck on a 64-bit Windows Server 2008 with IIS 7.5 that is routinely crashing per the bug IDs I’ve mentioned, but I don’t see anyone yet (besides Adobe) claiming that these bugs have been fixed by this updater. Is anyone out there able to verify the effectiveness of this updater?
any ideas? Please
Tried all that except for the .car file, now when I try to access cf admin or any website it is looking I get
HTTP Error 500.0 – Internal Server Error
Calling LoadLibraryEx on ISAPI filter “D:ColdFusion10configwsconfig1isapi_redirect.dll” failed
Hi folks,
I’ve found something interesting while trying to apply updater 4 on my CF10 servers (RHEL6).
My servers are behind a firewall so I have to copy the jar onto the server and do the installation via the console.
The user I use to ssh onto the box has got sudo access, so when I run the installer all seems to go fine. After it completes the log file showed all the updates were successful, but when I logged into the CF Administrator the version number was still 10,0,0,2xxxxx.
Out of interest I tried doing the install as root (which I can’t do in my acceptance and production environments) and the log file shows the same success messages, but the version is correctly updated to 10,0,4,283281.
Is there perhaps a file or directory that I might need to check permissions?
Cheers,
Simon
Just an update to my original question. Our sysadmin figured it out. Not really sure what the solution was but its fixed.
@Jason L, here are the steps (slightly modified from above):
1. Create .car file in CF10 (“ColdFusion Archive” file containing CF Admin settings)
— note: step #1 can *currently* only be done in Developer or Enterprise Edition (hopefully CF11 will support backing up CF Admin’s settings in Standard Edition too – IMO, backups should ALWAYS be supported). If Standard Edition, then clear serial number from C:ColdFusion10cfusionlib to convert back to Developer Edition.
2. Remove all CF10 Connectors
3. Install CF9 and configure for all IIS websites during install.
4. Deploy .car file into CF9
5. Restart CF9 Service
Steps to create a .car file: http://help.adobe.com/en_US/ColdFusion/10.0/Admin/WSc3ff6d0ea77859461172e0811cbf3638e6-7fc5.html
Thanks,
-Aaron
Can anyone offer help with #30? I’m about to do it and don’t want to brake more than is already broken.
Just noting I had difficulty installing Update 4 Final, *BUT* perhaps it was related to me having installed the “Prerelease” version of Update 4. CF Admin’s Updates page said Update 4 Final was installed, but it kept saying Update 4 Final was still available for download. And when I tried to run the .jar file for uninstalling Update 4, I was getting an ‘access denied’ error.
Again, the issue that I saw may not actually affect everyone. For me, I had to do these steps:
1) Delete the Update 4 Prerelease jar from {cf_install_home}/{instance_name}/lib/updates.
2) Copy all folders from {cf_install_home}/{instance_name}/hf-updates/{hf-10-00004}/backup directory to {cf_install_home}/{instance_name}/
3) Delete the Update 3 jar (and I remember there being another file in there too) from {cf_install_home}/{instance_name}/lib/updates.
4) Copy all folders from {cf_install_home}/{instance_name}/hf-updates/{hf-10-00003}/backup directory to {cf_install_home}/{instance_name}/
Then I was able to install Update 4 Final via CF Admin.
I’ll try a few install/uninstall scenarios to see if this is specific to users that had installed the “Prerelease” version of Update 4.
Thanks,
-Aaron
I’ve had it with CF10. What’s the best way to go back to CF9 with minimum down time? I was simply going to:
1. Remove all CF10 Connectors
2. Install CF9 and configure for all IIS websites during install.
3. Move over neo-cron.xml and neo-database.xml from CF10 to CF9 directory.
4. Restart CF9 Service
My intention was to leave CF10 installed but just turn off the services, in case I want to switch in the future or just uninstall it later on. Will this process work? Any better suggestions?
Tim-
I would try re-installing the mandatory update since it seems like it did not take.
I would also like to get more details on how to update the connector and how to verify. In my case, the Remove button is greyed out and only the Add button is available. I click on the Add button, then hit OK. When it says the web server is currently running and must be restarted to add this configuration I click Yes. Is this the correct procedure?
I tried to install the new Update 4 in the CF10 Administrator but I get the following message:
| Error occurred while installing the update: Failed Signature verification
I also checked the log and found the following:
| 19791 Nov 5, 2012 11:26:20 AM Error [cfthread-1] – File is not signed by a trusted provider.
| 19792 Nov 5, 2012 11:26:20 AM Error [cfthread-1] – Error While Downloading File From http://download.adobe.com/pub/adobe/coldfusion/hotfix_004.jar at D:cfusionhf-updates – Failed Signature verification
| 19793 Nov 5, 2012 11:26:20 AM Information [ajp-bio-8012-exec-4] – Failed Signature verification
Everyone is saying the fix is to install the mandatory update but we already did. I even tried to do it again and it says:
| This update is already installed
So what’s next?
Help is very much appreciated.
Thanks, Tim
@Krishna, The log file is named:
Adobe_ColdFusion_10_Update_4_Install_11_04_2012_19_19_05.log
The error in the log file was:
“Failed to copy hotfix files: C:Userscfruntimeuser870868.tmpdistcfusion
Status: FATAL ERROR
Additional Notes: FATAL ERROR – Failed to copy the hotfix files to the target location: {cf_install_home}cfusion
FATAL ERROR – {cf_install_home}cfusionbincoldfusionsvc.exe
(The process cannot access the file because it is being used by another process)”
Wow this is really unstable. 3-4 times per hour. Not all websites, but looks like when one fails than the next one and so on. Rebooting ColdFusion service doesn’t solve the issue. The connectors have to be updated each time.
I have coldfusion 10 on a windows 2008 R2 64bit server with IIS 7.5. Have 8 websites. The websites are connected to a second server running Windows 2008 R2 with SQL 2008. The error just randomly appears on any of the websites, even ones that are barely used.
The service is unavailable.
Service Temporary Unavailable!
The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later.
Â
Â
Â
 Jakarta/ISAPI/isapi_redirector/1.2.32 ()
Version 10,0,4,283281 so installed.
How can I confirm it is successfully installed?
Roughly 150 people on the website. Had this before with CF10 before the update but only like 2 times a day, now maybe every 30 clicks is exaggeration but it’s crashing a few times per hour. Win 2008 / IIS 7.5 ran the wsconfig tool as Administrator (right click run as admin) twice now.
Yikes. Under what sort of load? Anything in the logs? And you didn’t experience this sort of thing before applying U/4?
—
Adam
And here come the problems. “Service Unavailable” every 30 clicks or so.
@Charlie: The technote already has a note to reconfigure connectors.
Here’s what it says under Install ColdFusion 10 Update 4
After applying the update, reconfigure the connectors using wsconfig tool. It is n{cf_install_home}/cfusion/runtime/bin.
@Jake Hand
Looks like there should have been errors while installing.
Can you please tell me the full name of the log files that are there under /cfusion/hf-updates/hf-10-00004/
so that I can know the cause of the problem.
Also can you please open the log file and check if there is any error?
Thanks,
Krishna
I’m running into an odd issue where CF Administrator keeps telling me that Update 4 is available for installation after I just installed it. I verified the Information page shows my server at update level “cfusion/lib/updates/chf10000004.jar”.
I’ve already ran the wsconfig “-upgrade” that Aaron mentioned – do I have anything to worry about? (ie. How can I confirm for sure the update installed properly?)
[Note: just subscribing to get comments alert]
Hey Hemant:
It’s fair enough I guess that the WSConfig stuff is a separate step currently, but is there any reason why this bit cannot be automated as part of the upgrade process? That said… if you do that you also gotta back-up the files you update properly too 😉
Cheers.
—
Adam
Adobe,
Instead of unconfig/re-config, can we use wsconfig’s ‘-upgrade’ option?
http://help.adobe.com/en_US/ColdFusion/10.0/Admin/WSc3ff6d0ea77859461172e0811cbf364104-7fd9.html
Will that be fine? B/c the -upgrade option upgrades the modules while preserving the settings files (uriworkermap.properties, IIS’s applicationHost.config, etc).
Just wondering why we’re being told to actually unconfig/re-config instead of -upgrade.
Thanks,
-Aaron
Caveat Emptor: the updater screwed my Apache install up slightly:
http://adamcameroncoldfusion.blogspot.co.uk/2012/11/oh-ffs-updater-4-hosed-my-apache-config.html
This is unavoidable, but something to be aware of. I am a bit pissed-off with WSConfig though.
—
Adam
Never mind! I had not installed mandatory update. After that it installed.
Cheers.
Same problem as update 3, it is not downloading. “Error occurred while downloading the update: Failed signature verification.” Also I have big issue of ArrayIndexOutofBound error when page has multiple ajax call. It happens randomly. May be I should report in bug report section.
Subscribing
Hey Rakshith, that technote does not mention that people need to re-configure the web server connectors.
Fortunately, the note in the updater interface (in the admin) for updater 4 does mention it (“IMPORTANT: After applying the update, reconfigure the connectors using wsconfig tool. It is in {cf_install_home}/cfusion/runtime/bin”)
But some people may miss that who might see it if it was listed in this technote.
Hope you can get someone to tweak that technote to add that. (You may want to mention there also that users on Windows 2008 or 7 might need also to “run as administrator” when doing that.)
Well….If you’re running an unsupported Win8/Server 2012 configuration, don’t install the update. You will not be able to perform the uninstall/reinstall connector step because a check has been added which prevents the WSCT from completing the install part of the process.
I can’t fault them for not working on an unsupported configuration. At least there are workarounds.
[no comment, just subscribing to see others’ comments]
Hello. Applying update 4 went smoothly for me (I had lots of issues with update 3).
How can we confirm we are using the updated connector? I removed and added the connector but it would be nice if there was a way to confirm everything is correct.
Thanks.
Jeremy
You must be logged in to post a comment.
23 Nov
10AM Pacific
Online
