This suggestion would be for an application that would work on the site. This would probably not want to be a focus until after the site is up and running, but I figured I would jot it down while I was thinking about it.
The basic idea is that some resources are very easy to track usage of. (Water, natural gas/oil/whatever heating fuel, electricity, gasoline).
The first three categories tend to give you monthly bills which would give you a monthly data point. By charting these over time, and providing overlays with typical, and target usages for the particular location/setup (have some config choices). This would provide a way of showing the impact of various fixes on the usage. You could put points on the charts where various fixes went into effect etc. You might also be able to take more continuous data from whatever forms the smart meters give out.
Gasoline could be estimated fairly exactly using a basic setup. Since most people pay for gas using a card, they have a record of how much they spent on gas on what day. Knowing what type of gas they use and what the average price for their location is, you could get a pretty good estimate of actual gasoline used with minimal effort. Especially if someone is already keeping a budget or using something like quicken online that automatically can pull and categorize expenses.
Not sure if this would be something practical or not, but it would provide a good way of quantifying impact. I'm also not sure how to effectively quantify some of the other categories easily.
SixLinks Developers' response:
This is a cool idea, but as you suggest, pretty programming-intensive.
Let us know what you think with your votes and comments!