Q: How are the times measured?
A: The test URLs are loaded in an iframe. The time measurement is the time from when the iframe's URL was set, to the time the iframe's onload event fired.
Q: What's the difference between "Clear Disk & Mem Cache" and "Clear Mem Cache"?
A: "Clear Disk & Mem Cache" simulates the scenario of a user visiting your page for the first time, when they don't have any of your page's resources in their browser's cache. "Clear Mem Cache" simulates the scenario of a user coming back to your site after a previous session, for example, the previous day. In this scenario the user would have resources in the disk cache, but not in Firefox memory, and so the resources must be read from disk. There's a third scenario that Hammerhead doesn't (currently) test - a user coming back to your site within the same browser session. In this scenario, your page's resources would not only be in disk cache, they would also (likely) be in memory, so the page would load even faster. In my experience, we work to make resources cacheable in order to optimize the "coming back tomorrow" scenario, and that's why Hammerhead clears the memory cache between reloads for "primed cache" measurements.
Q: How do these time measurements compare to my performance in the real world?
A: In my blog post, Hammerhead: moving performance testing upstream, I talk about how gathering data from real page views is the most precise performance metric. But, to foster faster development cycles, it's important to measure performance closer to the developer. This will be less precise than data gathered from real traffic, but Hammerhead helps by handling the cache, iterating over multiple URLs, and gathering and reporting the results.

It's not a question of one or the other – both should be done. Testing performance on your dev box will give you more immediate feedback on the impact of your changes. Real page view data is ground-truth for what your users are actually experiencing. As discussed in the blog post, using a bandwidth throttler on your dev box is the most important thing you can do to make your results closer to what you'll see in the real world.

Q: Does Hammerhead measure Web 2.0 pages?
A: Not currently. I planning on adding a callback capability for logging time measurements. This could be used by Web 2.0 apps to record the time for an Ajax action, and initiate the next action.
Q: Why do the load times differ between Fasterfox, YSlow, Firebug Net Panel, and Hammerhead?
A: YSlow and Hammerhead should be pretty close - within 15 milliseconds. Both use the same technique (I know - I wrote both pieces of code). The reason they can be off by 15 milliseconds is due to the time resolution of JavaScript's Date implementation. The finest granularity is 15 milliseconds, so if the YSlow and Hammerhead timers aren't collected in the exact same execution loop, they're guaranteed to be off by 15 milliseconds.
Q: How come the first page load time when I open the browser or open a new tab isn't measured?
A: Hammerhead measures page loads starting from the previous page's onbeforeunload event. Since there is to "previous page", the first page loads in a tab or browser aren't measured. I hope to fix this.
Q: What version(s) of Firefox does Hammerhead support?
A: Firefox 2 and 3.
Q: Do I need Firebug?
A: Yes. You can get Firebug from getfirebug.com.
Q: Why should I disable Firebug panels?
A: It's possible that Firebug could affect page load times. To mitigate this effect, disable the Console, Script, and Net panels. You can do this by selecting each of those tabs, clicking on the dropdown menu, and choosing "Disabled".
Q: Any known bugs or issues?
A: Current known issues with Hammerhead:
Q: What's on the todo list?
A: