Saturday, December 24, 2011

Merry Christmas - and a new gallery

To all, I wish you happy holidays, and a viable New Year.

My favorite blogging opportunity this year was being able to mess around with Javascript and scientific graphics. It's colorful, but I hope that it does convey a lot of information in a fairly small space. Anyway that's the aim.

So I've overhauled the gallery of examples during the year that I maintain. It's no longer practical to have these all on one page, so I've made separate pages, with a tableau of images you can click on to reach them. The gallery is here, but I'm showing the tableau below.




Temperature shown with HTML5 canvas

Google maps view of temperature stations

Monthly temperatures - spherical hamonics

Millenium proxy plots

Surface Temperature Trends



Statistically significant warming

SST and model trends

Foster and Rahmstorf adjusted trends

KML and KMZ files

Significant trend differences in temperature indices

Jaxa Arctic Ice Extent

Javascript topic index



Finally, I guess I should provide something Christmassy. Here's Enya:


Tuesday, December 20, 2011

November GISS is out - down 0.07°C from October


Monday, December 19, 2011

Significant trends in Foster/Rahmstorf


Sunday, December 18, 2011

Significant trend differences

In earlier posts I have shown plots of all possibly linear trends in about a century of surface temperature data from various sources. In this post I modified the plot by fading regions where the trend was not significantly different from zero.

But as commenter Frank noted, this is not the only significant difference that might be of interest. So in this post, I'll show some plots that help to bound the range of signficance.

The first plot shows, for each start and end month, the lowest trend that the observed trend would be significantly less than. That is useful for testing predictions that might be failing on the cool side. For example, if you think that there is a claim that the trend over a period should have been 2°C/century, you can see where the actual trend was significantly (at 95%) below that.

The second plot, below the jump, shows the converse. What is the highest trend that was significantly (95%) below the observed. This is probably mainly of interest in establishing whether a trend was significantly above zero, but you might be interested in other values - if you think a theory significantly under-predicts.

The final plot is of the t-statistic - the trend normalized by its standard error. This lets you look at other degrees of significance - mainly relative to zero, but in conjunction with the other plots, you can work out other trend comparisons too.

I've retained the apparatus whereby you can check each point (by clicking) against its plot, echoing the numerical trend value (and period). I've changed from earlier plots of this kind by allowing trend periods down to 1 year.

Purpose

I should at this stage say that my purpose here is to show how the much invoked arithmetic of trend fitting works out. I'm not saying that it is always a good thing to do, and there are cautions about what significance means.

Saying that a trend is significantly different from a base trend is saying that the on the null hypothesis that the data is formed from the base trend plus random noise the observed result is improbable. Cautions:
  • Lack of significance does not mean the base trend is right. It just means it is consistent with this data. Many other possibilities would also be consistent.
  • Significance does not mean that any physics, say AGW, is disproved. It just means that there may be something more than random variation plus trend.
  • And it may not even mean that. It says the result, on the null hypothesis, is improbable. But improbable things happen. There are about half a million dots on the longest plots. If they were independent, and each had a 5% chance of being in a certain range, then that means 25000 significant dots. Even with correlation, you might still expect something like 5% of the area to show as significant. 


Plots

So here is the first plot, showing which trends the observed trend  would be significantly less than. To use it, pick a trend (color) you want to test from the legend. That and bluer redder colors indicate regions where the trend observed was significantly less than the test (color).  I have reversed the coloring custom of earlier posts, marking zero with dark brown, and 1.7°C/Century with gray. For the associated plot and mode of operation, see this post. But note that you can click anywhere on the plot to show the real trend there (shown in text and also by the time series plot).


1999-now
1989-now
1960-now
1901-now
Land and Ocean
Hadcrut
GISSLO
NOAA
UAH
MSU.RSS
TempLS
Land Only
BEST
GissTs
CRUTEM
NOAAland
Sea Surface
HADSST2




And here is the second plot, showing trends where the observed would be significantly greater. The brown marks the edge of the area where the trend is significantly greater than zero.


1999-now
1989-now
1960-now
1901-now
Land and Ocean
Hadcrut
GISSLO
NOAA
UAH
MSU.RSS
TempLS
Land Only
BEST
GissTs
CRUTEM
NOAAland
Sea Surface
HADSST2









And the third plot, which shows the t-statistic, or ratio of the trend to its standard error. For the number of degrees of freedom here, this is distributed normally, and 1.96, marked in brown, is the level of 95% significance. 1.64 is 90%, and 2.58 is 99%. One observation here is that there is only a small fringe region where a choice of a different test level would alter the result..


1999-now
1989-now
1960-now
1901-now
Land and Ocean
Hadcrut
GISSLO
NOAA
UAH
MSU.RSS
TempLS
Land Only
BEST
GissTs
CRUTEM
NOAAland
Sea Surface
HADSST2












































































Friday, December 16, 2011

Google Map problem fixed

Well, I'm hoping. I posted a while ago a Google Maps rendition of the GHCN surface temperature network. The Javascript mechanics made it possible to select and display all kinds of subsets, and even show a movie.

I heard some reports that people were having trouble making it display. I'd been struggling a bit to make robust Javascript, but that was getting better. There's a complication that I have to have a valid GM ID, but that seemed OK - users shouldn't need one.

Today I found what I think was the problem. I've recently started using an Amazon bucket, and using a newly acquired Web address www.moyhu.org. But it seems that DNS servers don't always get to the right place using that. So I'm using the full Amazon address.

It seems to work now. It's here.


Thursday, December 8, 2011

Nov temps displayed with HTML 5

It's time for November temperatures with TempLS. The headline number is a cooling - from 0.48°C to 0.36°C. I'll show the usual comparative plot below.

But what I really want to talk about is a new style of presentation. The last few posts have tracked my learning curve with Javascript; now I'm coupling it with the canvas feature of HTML 5. This means I can do client-side graphics - much more flexible and interactive

What this means for November GHCN temperatures is that I can show the anomalies directly - without cell aggregation or smoothing with orthogonal functions. This is done by a triangular mesh and HTML5's neat linear color gradient function. Every station has it's anomaly directly represented with the correct color.

I found this very interesting, as it shows graphically how correlated the anomalies are. Most of the color picture is actually quite smooth. There are exceptions, and because these can be associated with large triangles, it gives an exaggerated effect there. But for the most part, large scale patterns show through well.

So here's the plot of November average temperature anomalies, in °C. It is of course interactive - you can rearrange it, magnify, show the stations, click to see station detail and numbers etc. Use Ctrl+. Ctrl- to get it the right size for your screen. The mechanics are explained below the jump.








your browser does not support the canvas tag
Refresh
Show Stations
Show Mesh
Magnification
Station




















































How it works

The flat map at top right is your navigator. If you click a point in that, the sphere will rotate so that point appears in the centre. The buttons below allow modification. Set what you want, and press refresh. You can show stations, and the mesh, and magnify 2×, 4×, or 8× (by setting both). You can click again to unset (and press refresh).

Then you can click in the sphere. At the bottom on the right, the nearest station name and anomaly will appear. You may want to have stations displayed here. You'll see two faint numbers next to "stations". This indicates how much your clicked missed the station by (in pixels). It's not really a test of your mousing, but of my getting the alignment right (fiddly).

I'm continuing to work on the shading. The HTML5 function is good, but it works by RGB linear interpolation. The rainbow scheme that I use isn't linear (linear schemes are monochromatic and boring), so they get out of alignment. This sometimes means that one of the corners of the triangle does not get quite the right color. That can happen when there is a lot of change within a triangle.

Comparative plot of recent months, and the usual spherical harmonics fit.





and the Spherical Harmonics map



Previous Months

October
September
August

More data and plots


Monday, December 5, 2011

A Javascript Gallery

My last post had a Javascript worldmap of GHCN stations. I think this approach is very promising, and I'm hoping to improve with HTML 5 features soon.

Something new - I have an Amazon S3 bucket which I'm using for backup storage. This caused a hiccup with the last post, when I had not adjusted the permissions properly. I could see it, but probably no-one else could. I hope it works now. Please let me know if there are problems.

Anyway, this post is about a gallery that I've compiled with the various recent Javascript and other interactive graphics collected. It's a page on the blog, which you can also get to through the links on top right. There is a table of contents. I'm hoping to keep adding to it.

Saturday, December 3, 2011

Google Maps display of GHCN stations, with Javascript

I have been using KML files with Google Earth for displaying temperature stations so that they can be colored and selected according to inventory information. Steven Mosher got me started on this, and he has again pointed me towards using Google Maps interactively.

This is the next in my Javascript learning process. V3 of the Google maps API is well integrated with JS, and the language gives much more flexible control than KML. So in the display presented here, you can select subsets of the GHCN V3 stations according to inventory and data properties, changing their color or making them disappear. Properties include start, end and duration of data, and urban and airport status.

There is also an animation capability. You can set up a movie to run over a period, and you will see the stations in the chosen category appaer and fade reflecting the availability of data in those years.

You can click on each marker to bring up a balloon with information. Rolling the mouse over shows the name of the station.



Implementation of Javascript is not wholly reliable - IE requires tweaking for permission, and there can be clashes with the Blogger software etc. You may find that looking at the original html from here will work better, and will also give an uncluttered screen. An option is to download the html. Use Ctrl+ and Ctrl- to fit well on the screen.






















































More detailed usage information is below the jump.



You have available all the Google Maps controls. On the right are the controls for this widget, in three sections. You should make selections from the middle section. For the selection to be effective, the left button should be checked. All regular buttons act as toggles, and the label reflects the current state; clicking moves to an opposite state. You can write numbers in the text boxes, and toggle the "<" sign to affect the interpretation. The clear button clears the choices, and All overrides.

When you click an action button, all stations which satisfy any one of the chosen conditions will be re-rendered in that color. You may want to make a note of your choice, because the logic can get tangles after a few choices. Of course, to can just use the All button to restore.

To work the movies, choose a time interval and a pause (time between frames in sec). Then click the movie button, where the label will switch to "Show". Then click an action button; currently selected stations will be rendered in that color in each year for which they have data. The years tick over just above the Movie button. Normally you'll start from no stations visible, and they'll become invisible again after data finishes. Even if they start colored, they will still disappear after data ends.

Here are some suggestions:
  • At start, the All button will show. Try just changing the colors with an action button.
  • Try coloring by urban status. First All and yellow, then clicked the Mixed button, to show Not M. That means Mixed, the GHCN class between Urban and Rural, will stay yellow. Then, with Urban showing (and All not showing - toggle if it is), click the left radio button beside it, and then click pink. Urban stations will then be pink. Then change Urban to Rural, and click cyan. Now you will have a display showing the three different classes.
  • Then click Clear, and then set Duration to less than 70. Click Invisible, and you'll be left with the Urban/Rural coloring for stations with at least 70 years data.
  • Finally, a movie. Click Clear, All, Invisible. Then choose some movie years - say 1900-2011, and a pause of 1 sec. Click the Movie button; it changes to Show. Then click a color (Yellow, say) to run the movie. It doesn't matter what order you click buttons prior to the Action.


You can run a movie on, say, airports, but be aware that it is using current classifications, so you'll see 19th century airports.

I'll produce more of these, and maybe one for all stations, but data size (loading time) may be a drag.