Monthly Archives: August 2013

My ColdFusion 11 Wishlist

I’ve been trying to put together a “ColdFusion 11 Wishlist” post for quite a while. The time consuming part has been that it is difficult to think of things that I want to see added to CF. Over the years I’ve become quite accustomed to its shortcomings, pitfalls and long-standing bugs. I don’t pretend CF is something that it is not. There are a lot of things that it’s just not good at (there are also many things that it IS good at). However, tacked-on features like <cfmap> don’t appeal to me because I know how to implement a Google map – and I can do it better, faster and can keep it up to date.

I mention the <cfmap> tag (and the feared <cffacebook> tag that we hope to not see in the upcoming version of CF) because these additions don’t sell the product to developers – they sell it to managers. Developers realize that these incomplete features are ultimately glued on to a somewhat antiquated language that only recently got the ability to understand null. I think videos like this (produced by The Onion?) will demonstrate the point I’m trying to make.

Adobe has done well to publish a road map for its next versions of CF with some general bullet points regarding upcoming features. The good news? They are listening to the community to some extent because the install process and PDF features are to be improved. Huzzah! The bad news? The other upcoming features appear to be “add-ons” for developing mobile sites (apps?) and social media integration. No thanks…

I’m all for, as the CF team puts it, “embracing futuristic technologies” – but the lack of these integrated “futuristic technologies” is not what prevents me from creating great web applications in ColdFusion. What does prevent me from doing that are the long-standing bugs in very important functions like SerializeJSON(). These are significant hold ups that chew-up development time are not addressed quickly (or even acknowledged).

So, what’s ultimately in my ColdFusion 11 wishlist? Just two things:

  1. Address the backlog of bugs.
  2. Listen to CF developers. We care about the the future of the product – it is our livelihood, after all. We won’t steer CF in the wrong direction!

Spidering / Link Checking With wget

I use XENU for link checking sites and finding missing assets but I couldn’t figure out how to make sure that it was following the redirects it encountered. For example, if an inline image source is “/images/sitelogo.jpg” but that 301 redirects to “/images/sitelogo-new.jpg”, XENU will report the redirect (as an error if you prefer), but what I really want to know is whether the destination of that redirect was a 200 OK (or a 404, or something else unintended). It wasn’t clear to me if XENU was ensuring that the file existed after being redirected.

I tried out a few other free tools but none seemed even as good as XENU. It was then that I stumbled upon the “spider” option in wget. You can set it free on a URL like so:

wget --spider -l 2 -r -p -o wgetOutput.log

This will spider the URL up to 2 levels deep and ensure that any inline assets on the pages within those levels are also downloaded. The “-p” option ensures that inline assets like images or css are downloaded from a page even when the maximum number of levels in the “-l” option is reached. The output is logged to wgetOutput.log

At the very end of wgetOutput.log you’ll find a list of broken links that looks something like this. You will also get a ton of other useful information about every request that it made – so you know exactly what it’s doing!

Spider mode enabled. Check if remote file exists.
--2013-08-06 20:10:40--
Reusing existing connection to
HTTP request sent, awaiting response... 200 OK
Length: 4153 (4.1K) [image/png]
Remote file exists but does not contain any link -- not retrieving.
unlink: No such file or directory

Other Useful Options

Specify a user agent:

-U "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)"

Spider a site that forces you to log in:

  1. Get the Cookie Exporter Add-on for Firefox.
  2. Log into the site you want to spider.
  3. From Firefox, run Tools -> Export Cookies -> cookiesFile.txt
  4. Use the “–load-cookies” option:
    --load-cookies cookiesFile.txt

Complete Example:

wget --spider -l 2 -r -p -o wgetOutput.log -U "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)" --load-cookies cookiesFile.txt