Archive | science RSS for this section

Plotly on cygwin; the absolute basics

This has to be about the absolute basics. I don’t know anything else.

Plotly ( is an interactive, online graphing tool. It can be called from JavaScript, Python, whatever. This post is about getting it to work through Python on Cygwin.

This all mostly follows instructions on the Plotly website.

(1) Installed via pip. What’s pip? A Python package manager. I ran the Cygwin setup.exe program and made sure that Puython was installed (in my case it was 2.7) and then installed the matching pip2.7 (Cygwin package python2-pip). So installed that and all its dependencies.

(2) Opened a Cygwin terminal (not an X terminal, just mintty) and typed:

$ pip install plotly

and watched some magic occur.

(3) Went to the Plotly website and created my (free) account. Went to my account settings and selected ‘API keys’. Could not see key — just looked like a row of dots! But hitting ‘Regenerate key’ gave me a new, visible one. Copied that text and noted my username.

(4) In Cygwin, (note, $ is the Cygwin prompt, >>> is the python prompt) typed:

$ python
Python 2.7.13 (default, Mar 13 2017, 20:56:15)
[GCC 5.4.0] on Cygwin

>>> import plotly
>>>'DarrenG2', api_key='<<insert your key here>>')


This set up the info needed for the local Plotly/Python installation to talk to the website where the graph will appear.

(5) Checked that this had worked out. Back at Cygwin prompt, in home directory, typed:

$ cat .plotly/.credentials
    "username": "DarrenG2",
    "stream_ids": [],
    "api_key": "<<your key here>>",
    "proxy_username": "",
    "proxy_password": ""

(6) OK, looked good. Now, tested it by grabbing an example from the Plotly website. Created a file ‘’ and pasted in some text copied from the website:

import plotly.plotly as py	
from plotly.graph_objs import *

trace0 = Scatter(
    x=[1, 2, 3, 4],
    y=[10, 15, 13, 17]
trace1 = Scatter(
    x=[1, 2, 3, 4],
    y=[16, 5, 11, 9]
data = Data([trace0, trace1])

py.plot(data, filename = 'basic-line')

(7) Then saved and ran the script

$ python2.7.exe
High five! You successfuly sent some data to your account on plotly. View your plot in your browser at or inside your account where it is named 'basic-line'

Looked good, though they’ve spelled ‘successfully’ unsuccessfully.

(8) But where was the graph? Well, I was working in a basic terminal window. It sent the graph to the web, but then tried to open it using the default links, the text browser. So all I got was a blank screen (typed ‘q’ to quit links). There are a couple of options to see the graph — one is just to paste the given URL into Edge, Chrome, Firefox. Another is to tell Cygwin to look elsewhere for its browser…

(9) Edited my .bash_profile file in my Cygwin home directory and added these two lines:

$ BROWSER=/cygdrive/c/Users/darren/AppData/Local/Mozilla\ Firefox/firefox.exe
$ export BROWSER

This set up the environment variable BROWSER and pointed it at the firefox.exe file (non-admin install, so in an unusual place). I also ran these two lines in the terminal window to save me closing and reopening it.

(10) Repeated step (7) and — lo and behold! — a Firefox window popped up and the graph was in it!

Plotly graph in Firefox, after running the script in Cygwin.

Plotly graph in Firefox, after running the script in Cygwin.

Now, mastery of Plotly and Python is a much bigger project, but at least this offers the beginnings. Note also that the graph can be edited interactively within the webpage where it appears.


Plots away!



Journal self-archiving policies

One of the most-viewed pages here (it’s all relative; still not very many hits really) is the list of dodgy publishers. It’s a drop in the ocean, but it gets a few hits. Something else I’d like to bring all into one place, even though it’s been done elsewhere, is to summarise self-archiving polices. I’ll focus on the journals I’ve published in, preparatory to putting together a web archive of all my papers that I am allowed to self-archive (on my website and maybe on ResearchGate). It’s really just a resource for me, but I might as well make it public.

Some journals are open access; you don’t need to self-archive those, but usually you can.

Some allow you to archive a proper reprint, processed and edited by the journal — like the IUCr, as shown below.

Some suggest you archive the ‘author’s final version’ but don’t want you to put up anything with the journal’s imprimatur.

Some say ‘mine mine mine’ and don’t let you host it at all. I hope to make this clear.

The page lives at, and so far has exactly one (1) entry, the good old IUCr, which has a very enlightened policy. They allow self-archiving as long as you use the official e-reprint, rather than just the downloaded PDF, and they request that you provide a link to the journal. Seems very reasonable. The official e-reprint is easy to recognise; it has a distinctive front page with some official words on it, something like this (colours may vary):

Front page of IUCr reprint, showing it is dfferent from download from journal.

Note the box with the notice near the bottom of the page.


I think their policy is very reasonable because the IUCr has a very professional publication and editorial team who need to be paid and ought to be paid. Subscriptions are part of the mix, yet they allow authors to house their own work and to distribute copies to colleagues freely. It seems a very sensible mix.

More updates as they come to hand.

The last domino falls: A little bit of science.

This paper and its deposited material explore clustering of 2 × 1 dimers (dominoes) subject to simple interactions and temperature. Much of the work in domino tilings has been statistical, combinatorial and thermodynamic in nature. Instead, here, the domino is used as a simple model of a non-spherical molecule to explore aggregation, rather as if the molecules were interacting in solution. As a result, the work does not look at how many ways there are to tile a plane, but at how the cluster evolves with different parameters in the potential that governs the clustering. These parameters include the rules used to select which of the many possible dominoes will be added to the cluster, and temperature. It is shown that qualitative changes in clustering behaviour occur with temperature, including affects on the shape of the cluster, vacancies and the domain structure.

The paper is on the web, open access, at and It comes with a bundle of software anyone can use to play with the model, modify it, whatever. Please do!

It’s basically a toy model, but it shows some nice behaviour. Apologies to the red/green colour-blind.



T=0. At low temperature, the model gives strongly structured domains.



T=0.16. At intermediate temperatures, the ordering begins to break down, though remains apparent. If temperature continues to increase, vacancies appear.



T=10. Vacancies, no domains, really. Addition of new dominoes is almost random.




This week I started work at Biotext, a company that specialises in writing and editing complex scientific documents. It’s incredibly exciting — it’s the kind of opportunity that does not often come along. There’s a huge amount to learn, but that is part of the enjoyment.

As the name suggests, their focus has often been on biological material, though in the broadest sense — agriculture, environment, and medicine feature strongly. I’m hoping to increase the expertise in the physical sciences.

I looks like a chance to bring together science and writing, and it has come along at a time when I was on the lookout for a new job.

Good luck to me!



Old g95 (for the make use of)

I needed to compile a Windows g95 binary for someone. I have Win 7 in a VM on VirtualBox. I had the g95 install tree from a (quite old) back-up of a previous computer, but not a proper install.

The install tree looks like this:

C:\util>dir /s g95
 Volume in drive C has no label.
 Volume Serial Number is 8440-3CD9

 Directory of C:\util\g95

23/08/2012  09:42 AM              .
23/08/2012  09:42 AM              ..
23/08/2012  09:42 AM              bin
23/08/2012  09:42 AM              doc
23/08/2012  09:42 AM              lib
23/08/2012  09:42 AM            55,782 uninstall-g95.exe
               1 File(s)         55,782 bytes

 Directory of C:\util\g95\bin

23/08/2012  09:42 AM              .
23/08/2012  09:42 AM              ..
26/03/2008  11:49 AM           553,984 ar.exe
26/03/2008  11:49 AM           827,904 as.exe
17/06/2009  09:44 PM           123,046 g95.exe
26/03/2008  11:49 AM           782,848 ld.exe
28/12/2007  12:23 AM            15,964 mingwm10.dll
26/03/2008  11:49 AM           554,496 ranlib.exe
26/03/2008  11:49 AM           685,568 strip.exe
               7 File(s)      3,543,810 bytes

 Directory of C:\util\g95\doc

23/08/2012  09:42 AM              .
23/08/2012  09:42 AM              ..
16/10/2004  09:36 PM            17,015 bg.gif
27/11/2004  03:51 AM            18,007 COPYING.txt
08/12/2005  06:19 AM            22,431 docs.html
23/11/2004  12:00 PM           107,122 g95.bmp
02/01/2007  01:59 AM           170,619 G95Manual.pdf
31/05/2008  06:59 AM            11,858 Readme.html
31/05/2008  07:00 AM             6,687 README.txt
               7 File(s)        353,739 bytes

 Directory of C:\util\g95\lib

23/08/2012  09:42 AM              .
23/08/2012  09:42 AM              ..
28/12/2007  12:23 AM             2,192 crt1.o
28/12/2007  12:23 AM             2,288 crt2.o
28/12/2007  12:23 AM             1,239 dllcrt2.o
23/08/2012  09:42 AM              gcc-lib
22/10/2006  01:27 AM           408,608 libadvapi32.a
30/10/2005  11:13 AM           253,890 libgdi32.a
22/10/2006  01:27 AM           594,018 libkernel32.a
28/12/2007  12:23 AM               458 libm.a
28/12/2007  12:23 AM             7,514 libmingw32.a
28/12/2007  12:23 AM           267,880 libmingwex.a
28/12/2007  12:23 AM            82,558 libmoldname.a
28/12/2007  12:23 AM           503,692 libmsvcrt.a
22/10/2006  01:27 AM           128,262 libshell32.a
22/10/2006  01:27 AM           435,754 libuser32.a
30/10/2005  11:13 AM            82,086 libws2_32.a
              14 File(s)      2,770,439 bytes

 Directory of C:\util\g95\lib\gcc-lib

23/08/2012  09:42 AM              .
23/08/2012  09:42 AM              ..
23/08/2012  09:42 AM              i686-pc-mingw32
               0 File(s)              0 bytes

 Directory of C:\util\g95\lib\gcc-lib\i686-pc-mingw32

23/08/2012  09:42 AM              .
23/08/2012  09:42 AM              ..
23/08/2012  09:42 AM              4.0.4
               0 File(s)              0 bytes

 Directory of C:\util\g95\lib\gcc-lib\i686-pc-mingw32\4.0.4

23/08/2012  09:42 AM              .
23/08/2012  09:42 AM              ..
23/08/2012  09:42 AM             1,022 cc1.lnk
17/06/2009  09:44 PM         5,242,021 f951.exe
17/06/2009  09:44 PM           859,168 libf95.a
17/06/2009  09:44 PM            61,284 libgcc.a
               4 File(s)      6,163,495 bytes

     Total Files Listed:
              33 File(s)     12,887,265 bytes
              20 Dir(s)  42,207,707,136 bytes free

Now, all that is required to make this work is to set some environment variables. (Start → Control Panel → System → Advanced System Settings → Environment Variables).

First, I had to put some directories in the path, so I edit the PATH variable. It works if I have the path to the g95.exe binary and also to the f951.exe file. That is:

PATH=C:\util\g95\bin;c:\util\winvi;c:\util\g95\lib\gcc-lib\i686-pc-mingw32\4.0.4;[[INSERT REST OF PATH HERE]]

I’ve put [[INSERT REST OF PATH HERE]] at the back, but I’ve actually got the g95-related paths at the back (you want the most commonly used bits of the path at the front, not that it matters with modern fast coomputers). Then, I want to create a new variable called LIBRARY_PATH:


Now, I don’t know if I need all three directories in the LIBRARY_PATH, but it works so I am not complaining.

So to compile my tiny little program I can now type:

X:\Downloads\Brill_dir>g95 -o Brill.exe brillouin.f90

And it works. Is it statically linked? Well, on Linux, dynamically linked, compiled with GFortran, it’s about 68 kB. This binary is about 360 kB, so I sure hope it is statically linked!

Caveats: I have not tried anything fancier than this absolutely basic compile. It works. Compiling with the -static flag makes no difference to the size of the binary.


My Appendix: Chains of molecules, planes of scattering

In a very recent post, I mentioned an appendix to an article I wrote. I rather like it. The appendix grew out of a little document I put together. That document is longer, vaguer and a little different from the published appendix, and so I am putting it here. Now, the article was written in LaTeX, and this is a website, so I tried running htlatex on the file. It was very complicated:

$ htlatex planes
$ firefox planes.html

And it worked. Next thing is to get it into WordPress… Easy enough to cut and paste the HTML code into the window here, but what about all the graphics that were turned into png files?Ah well…bit of manual fiddling. Equations and symbols seem to sit high, and some of the inline equations have been broken into a mix of graphics and characters… still, not too bad. The PDF version is available here.

Planes perpendicular to vectors

Say you have a vector in real space, expressed say in direct lattice terms, for

example  planes0x= paplanes1x + pbplanes2x + pcplanes3x where planes1x is the a lattice parameter, which is a vector.

You may want the reciprocal plane(s) perpendicular to this vector.


Because correlations in a crystal collapse the scattering into features perpendicular to the direction of the correlation. In a normal, fully ordered three dimensions (3D) crystal, this collapsing happens in all three directions, so the scattered intensity coming off the atoms gets concentrated at points, the reciprocal lattice points, usually denoted hkl.

If you have only two dimensional ordering, the scattering is collapsed down in two directions but not the third, giving rise to rods or lines of scattering in reciprocal space (that is, in diffraction space). If there are only one dimensional correlations, the scattering collapses into sheets, that is, it is delocalised in two dimensions and only localised in one dimension (because there are only correlations in one dimension).

In diffuse scattering the crystal is typically long-range ordered in three dimensions, and the diffraction pattern shows nice Bragg peaks (hkl reflections). However, there can also be disorder, for example in the motions of the molecules or the chemical; substitution of one species of atom or molecule for another.

In a molecular crystal, one can sometimes identify a chain of molecules running through the crystal, and interactions within these chains are likely to be much stronger than those within. That tends to mean that the motions of the molecules along the direction of the chain (call that ‘longitudinal’ motion) is highly correlated, while it is not well correlated laterally.

In such a situation, the single crystal diffuse scattering will show ‘sheets’ of scattering perpendicular to the length of the chain.

Let’s say the chain of molecules extends along an arbitrary real-space direction, which we’ll define by the vectorplanes0x as above.

Now, a plane perpendicular to planes0x can be specified by giving two (nor more) non-collinear vectors that lie in it. Let’s look at two vectors and we’ll call them planes7x and planes8x, for no good reason.

Then we can say that

planes9x (1)

but note that we are not assuming planes7xplanes8x = 0, since right angles within the plane are not that important — especially as for generality I am not assuming orthogonal axes.

Now, planes7x exists in reciprocal space, so it is a linear combination of the reciprocal lattice vectors, planes13x, planes14x and planes15x like this

planes16x (2)

and these reciprocal vectors are defined in terms of the direct space vectors like this

planes17x (3)

and similarly for the other reciprocal vectors. The important thing for us to note is that this means planes13x is perpendicular to planes3x and planes2x . This is important when we go to take dot products later on. The bottom line here is basically the volume of the unit cell, and 2π is just a scalar, so from the point of view of defining the plane that we want, these are not important.

Ignoring the scalar parts, substituting eq. 3 into eq. 2 gives

planes21x (4)

and since we have more variables than we need if we are to satisfy eq. 1, we can arbitrarily set qc = 0.

Now, considering the dot product of planes7x and planes0x , in full it is

planes24x (5)

and this is useful because, to take the last term on the first line as an example, planes1x is perpendicular to (planes3x × planes1x) by the very nature of the cross product. This means that any terms with a repeated vector go to zero. Further, in the remaining terms the vector part is just of the form planes1x planes29x which is the unit cell volume and a constant, which we can also factor out to be left with

planes30x (6)

which is nice and simple. This is not a surprise but still…

The next step is to find another vector in that plane. This is just planes8x, and if we use the same logic but, to make planes8x non-collinear with planes0x , we choose rb to be zero, we get an equation analogous to eq. 6. These can be summed up as

planes34x (7)

where planes35x is a fairly straightforward extension.

Now, in terephthalic acid (TPA), triclinic polymorph of form II, each molecule has a -COOH group at each end. These H-bond strongly with the groups on neighbouring molecules and you get strongly correlated chains of molecules running along the [-111] (direct space) direction. This then suggests that the planes of scattering perpendicular to these chains will extend in the directions

planes36x (8)


planes37x (9)

Now, does this work? Figure 1 is some data from TPA, diffuse scattering data measured on a synchrotron. It also shows the reciprocal axes and the white, two-ended arrows show the directions of the diffuse planes and by
counting Bragg spots it can be seen that these agree with the calculation above.


Figure 1: Observed data, measured at the Advanced Photon Source, showing what looks like multiple crystallites with similar orientations.

This means that we can ascribe these features to correlations in the displacements of the TPA molecules linked by the -COOH groups.

Before, before. Stop living in the past.

A Paper! Good God, a Paper: ‘Synchrotron X-ray diffuse scattering from a stable polymorphic material: terephthalic acid, C8H6O4’

I’ve been doing science for a long time, and while I’m in a bit of a career transition at the moment (see here for example), I’ve still got a few fingers in a few pies, and a few pieces of work slowly wending their ways through the system. Most recently, Eric Chan and I put out ‘Synchrotron X-ray diffuse scattering from a stable polymorphic material: terephthalic acid, C8H6O4‘. It’s a paper about the fuzzy, diffuse scattering from two polymorphs of the title compound.

It’s out in Acta Crystallographica Section B: STRUCTURAL SCIENCE, CRYSTAL ENGINEERING AND MATERIALS, a highly reputable but not open access journal, although they do allow authors to self-archive. At the moment, what that means is if you want a copy send me a message and I’ll punt one back to you.

Terephthalic acid molecule, drawn in Mercury.

Terephthalic acid molecule, drawn in Mercury.

What is terephthalic acid (TPA)? Well, it is a chemical used a lot in industry (plastics and such) and at room temperature it can crystallise out of solution in two forms, called (wait for it) form I and form II. (Well, actually the word ‘form’ is poorly defined in this context, technically, and it’s better to just say ‘polymorph I’ and ‘polymorph II’). In this context, a molecule is polymorphic if it can form more than one crystal structure and these structures can co-exist. Many materials change structure as you heat them up or squash them, but in a polymorphic system separate crystals of the structures can sit there side by side, under the same conditions. In most case, those conditions are room temperature and one atmosphere of pressure.

The two room temperature polymorphs are both triclinic, so of low symmetry. The difference is in how the molecules are arranged relative to each other. In both cases the -COOH groups on the ends of the molecules connect strongly to those on neighbouring molecules, so long chains of molecules form. (In the picture here, the -COOH groups are those at the ends of the molecule consisting of two red (oxygen) atoms, one white (hydrogen) and the grey (carbon) atom attached to the two whites.) These chains are sort of like one dimensional crystals, and then they are stacked up (like logs or a pile of pipes), but you can stack them up with, say, the -COOH in neighbouring chains close together, or you might have the phenyl rings (that is, the hexagon of grey carbon atoms) in one chain adjacent to the -COOH in the next. So in that sort of way you can get different crystal structures depending on how you stack things up.

Anyway, the paper looks at these polymorphs and how they are similar and how they differ. It uses my old ZMC program, which you can download from here (it comes with an example simulation, though not this one I’m talking about now). (That link goes to a paper I wrote and published for an Open Access journal, which I chose specifically so that you could go and download ZMC and everything for free…)

So in doing this I think about the connectivity of the molecule — how do the atoms depend on each other and where does the molecule need to be able to flex and twist? That means I end up drawing diagrams like this one:



That’s exciting, isn’t it? I start at the middle (X) and then each atom is positioned relative to the ones that went before. Here’s another picture (because I happen to have it handy)…. This shows how the atoms were numbered, and how by numbering them correctly and building the molecule up in the right order it is easy to let the -COOH groups spin around. mol_num

The X-ray diffuse scatting in the <i>h</i>0<i>l</i> layer of reciprocal space of TPA.

The X-ray diffuse scattering in the h0l layer of reciprocal space of TPA. Measured at the Advanced Photon Source.

Here I show typical data. You can see the little white spots — these are the sharp diffraction peaks, Bragg peaks, and they indicate where a lot of X-rays were reflected off the crystal. They are what is used to work out what is usually called the ‘crystal structure’ which consists of the unit cell (the repeating unit) that the crystal is made up from. But you can also see blobs and streaks and stuff, and these are wider (‘diffuse’) features, and these tell us about how the molecules interact and shuffle each other around, and stuff like that.

Anyway, the paper is online now. The DOI link is One thing I really like about it is it’s got a mathematical appendix. I always wanted to write an article with a mathematical appendix. I think I might post on that separately.


Is it better to go off-line when teaching?

Students, just like most of us including me, are too distractible, especially younger ones lacking self discipline, and by younger I mean first year university, not genuinely young. These days we put the content and the tutorial questions on the Learning Management System (LMS, really just a website) and we tell them to use the LMS to access the questions and the supporting materials and such. Once upon a time they’d just get a bunch of photocopies (‘photostats’) or before that roneos (mimeographs) or just “copy this down off the board.” I’m not pining for the past, I’m trying to work out how we can combine the best of then and now.

What happened then was we’d come to class having not looked at anything beforehand, we’d copy down a bunch of questions or question numbers off the blackboard (it wasn’t a whiteboard) like ‘Ch 8 Q 12-18’) then we’d have the book open in front of us and we’d whisper to each other while we were supposed to be working out the answers. Hmm.

What happens now is this:

They come to class having not looked at anything beforehand (just like in the old days), because the know they can access it when they get there (we knew we’d be given it when we got there, back in the day, so no difference there). But, and this is different now, they then spend ten minutes getting onto the university network and getting distracted by Facebook or whatever and don’t download the questions until the tutorial is half over. Then they get out their notebook (or tablet and stylus) and read the question and… check their messages. Then they show the guy sitting next to them a cat video. Then they laugh and eat some Skittles (fine, fine, that is not the internet’s fault), then they look at Pinterest or for all I know Tinder, and then I ask them how they’re going and they mumble and we’re over half way through now and they have written down a few bits of data pertaining to the first question and that’s it.

Okay, maybe I’m overstating, but I have seen it happen that way. I’m not just fighting any innate apathy or disinterest (or depression or sense of futility) to get them to do the work, I am fighting the single most interesting thing the human race has ever constructed — a world wide distraction machine that has everything on it and available at the touch of a screen.

At best, even when they are doing some physics or mathematics, their attention is divided — they are always ready to pounce on an alert from whatever bit of social media they use, so their brain is never really thinking about the questions we give them to (we hope) help them learn.

Now, in the past when you copied a question off the board, it went in your eyes, through your brain and out your fingers onto the paper. I’m not sure that’s much better than not engaging with it at all, but it can’t be worse. You could only really talk to the people either side of you, just as students can now, so there were by definition fewer distractions because now there are all the ones I had as a student plus smart phones, so at the very least students now have more distractions. Do they deal with them better than I used to? Valid question. Maybe these days they have extra information, extra connectivity, and the ability to use that without being consumed by it.

I’m not sure.

I started thinking about this post while I stood there watching students flick away from Snapchat (or whatever it was) and back to the LMS whenever they saw me coming. A few were able to use the ‘net to find useful information, or a website with some helpful content, and that’s good because a working scientist or problem solver (engineer, IT, whatever) does just that, calling on the info around them as well as what they know. But those students were a small minority.

I recall thinking how I would really, really like to given them all a paper copy of the questions or, better, ask them to bring their own copies (then at least they would have looked at it to the extent of downloading and printing it off and getting it from the printer with their own actual physical fingers before they got there — does that count as ‘engagement’?), and then use just their notebook, their bog basic calculator and their textbook (they still exist, they do!) to tackle the problems.

I don’t say the web is useless. It is great for communication, for extra activities and resources. They can use the web to access the material easily and flexibly when they are not in my class. I use it to distribute videos to buttress the material, to direct them to external resources, though Britney Spears’ Guide to Semiconductor Physics is getting a little behind the zeitgeist now… The WWW ought to be great for collaboration, for ready access to what the students have not internalised. For simulations, for VR, for virtual laboratories, for Skype visits to major laboratories, for feedback, for interaction, for… the sky is the limit.

But not if you can’t sit still long enough to actually do it.

We’ve tried to engage the students to make them want to be there. I mean, that should solve everything. And there’s always a few who do want to be there and  that’s great, they learn almost regardless of what the teachers do. But some students are in the class because they have been told to be there, because the subject is a prerequisite for what the really want, because they thought they would like it and now it’s too late to drop out without recording a fail, whatever. By giving them the option to more easily be mentally elsewhere when they have not developed the self-discipline to choose to do what needs to be done, I’m not sure we’re helping. I wonder if more distraction-free classroom time would have its benefits as part of a broader suite of learning opportunities? Some of the environments would use all the tech at our disposal, and some would just have the student and their brain and the stuff to be tackled.

I just want the best of both worlds; is that too much to ask?


Old fart, I am.

Irrationality: The Enemy Within by Stuart Sutherland. Too true.

Penguin, 1994, 357 pages.

Well. This book is replete with summaries of studies that on the whole show that we are creatures of habit, instinct and fear more than thought and reason. We suffer from the illusion of control. We make emotional decisions and then convince ourselves they were carefully reasoned. We avoid data that might prove us wrong, even when being proved wrong is the best thing that could happen to us.

The cover of <i>Irrationality</i> by Stuart Sutherland.

The cover of Irrationality by Stuart Sutherland.

I can’t say I was shocked. There’s a time and a place for aiming for the utmost in rationality, of course, and times when that’s not sensible, and it is useful to know the difference. If you’re being chased by a bear a quick but sub-optimal decision may be better than making the right one too late. And it’s useful to know when it doesn’t really matter and you can just please your inner reptile, and when you really do need to sit down and analyse things properly.

And in a sense that is the key point. He basically says that only by understanding statistics and by essentially falling back on some means of scoring the alternatives and then picking the one with the best score can we really make rational decisions. Otherwise we rely on impressions, feelings and hunches, none of which are actually reliable. In the end, only by breaking down the problem and applying some kind of rigorous-as-possible analysis, generally relying on mathematics, can a really rational decision be made. And what fraction of decisions are made like than? In my life, relatively few.

Each chapter tackles various forms of irrationality, and each ends with a ‘moral’ which is really a bullet-point summary, the last one of which is usually humorous/facetious. (‘Eat what you fancy.’)

There is some repetition, but the points being made deserve hammering home. There are some lovely little ‘try this yourself’ puzzles, where even though I knew there was a trick and I desperately did not want to answer like an irrational creature, I still got it wrong. The simple two card trick, for example, which I won’t describe in detail here since it would be too much like giving away the twist in the tail.

In summary, if you think you are good at making decisions, you might find this book useful. If you already believe that we’re basically animals in clothes, this will not disabuse you. It’s funny, opinionated, amusing and entertaining, but a little, I repeat, repetitive. Some of the case studies of how really really really important ‘decisions’ were made are a little worrisome, especially because (of course) human nature has not really changed in the meantime. I sometimes look around at a skyscraper, or read about a decision to go to war or spend billions of dollars on a useless aeroplane, and this book comes to mind. Will the building fall down? Is the war really worthwhile? Will the aeroplane get off the ground, and if it does will it stay up?

In some ways the book makes our achievements all the greater. Okay, the planet is in trouble. Okay, we don’t always elect great leaders or do the right thing by our neighbours, family, friends. Yet so much has been done. We’re not always rational, no, and neither should we be. Would more people be happier if the balance shifted towards more rationality? Probably. Yet on the whole we go forward, stumbling sometimes, by accident sometimes, yet we do live longer, we have sent people (okay, men) to the moon, vastly fewer children and mothers die in childbirth. It’s not all bad, this world.

Anyway, it’s a good book.


Book book book.

Why I Dislike Metrics

Metrics are used to measure a researcher’s output. How many publications? Patents? Students? Where are they publishing? Are they being cited? How many dollars in grants are they pulling in?

It’s tricky, because researchers at universities do need to be held accountable for the money invested in them — and the opportunity given to them that may have been given to another. Yet the outcomes of research can be diffuse, slow to materialise and hard to evaluate. A great conceptual breakthrough may have little impact initially. The investigator may have been fired by the time it is recognised. How does a non-expert administrator (who holds the purse strings) distinguish between a researcher who is ahead of the curve, and so not being cited because there are few others working on similar ideas, and one who is poorly cited because they are simply dull? Both are likely to have a tough time getting grant money, too.

Such an administrator falls back on metrics. Impact factors, grant dollars accrued, and so on. Complex formulas are developed. So much for a publication in one of these journals, less for one in these; citation rates are multiplied by this and divided by that, papers with a lot of authors [are|are not] (choose one) down-rated…and when government agencies that dole out grant money choose a particular metric, there’s really no choice.

Just looking at publications, once sheer numbers was the ‘in’ thing. Then it was citations. Then the H-index, the M-index, the insert-your-own-clever-metric-here-index, who knows. Now there are scores that mean publications in lower-ranked journals will actually count against a researcher, such that when comparing two researchers, one with four papers in ‘top’ journals and one with four in top and three in middle, the latter would actually be penalised relative to the first.

I cannot understand how this can be considered equitable, reasonable or sensible. I recognise that it is better to have high impact than low. I recognise that staff who consistently fail to have high impact need to improve that record. I have no problem with that. But the idea that a tail of papers in lower ranked journals is to be penalised is short-sighted, counter-productive and shows a lack of understanding about how science works. I will not speak for other fields.

(1) If I have a postgraduate student, or even an honours student, who had produced a nice result, a novel result, but not a high-impact result, I must now deny them the right to publish that result and build their publication record. They will finish their studies with fewer papers, less experience in writing up their work, a poor publication record, and less chance of employment. Writing for publication is a valuable part of a student’s training. By publishing a (possibly minor) paper extracted from their thesis before the thesis is submitted, a scholar gets feedback on their work and their writing ability from a wide audience, begins to build a profile, and can be more confident that the thesis will be passed because a component of it has already passed peer review.

(2) It would be easy for any such rules to be biased against staff publishing in certain areas. Who decides what is a ‘top’ journal? How is this harmonised across fields? Some fields are oddly replete with high-ranking journals and some have a dearth. This needs to be recognised.

(3) Science is a dialogue, a discussion. Many important results come from bringing together many small results. By forcing staff to only publish their highest-impact work, many results that might be useful to other workers in the field will never see the light of day, will never contribute to the debate. This holds back the field. To give a simple example, databases like the Inorganic Crystal Structure Database are populated by thousands of individually minor results. Most of these were not published in high-impact journals, yet data mining across that database and others has produced powerful results that are of great value. Great cathedrals can be made from many small bricks. This policy prevents those bricks from accumulating. It works against the fundamental (okay, and idealised) nature of science as a transparent, collaborative enterprise.

(4)  Building of collaborations will be inhibited. If I have a colleague at a university or facility (like a synchrotron, say) who is not subject to the same rules, they will quite reasonably say that a piece of work may not be ‘high-impact’, but is worth publishing nonetheless, and I will have to either accept the impact on my own record or deny publication. That is hardly a great way to build a relationship.

(5) Metrics have a habit of being applied retrospectively. Evaluating my performance in 2015 or 2016 (or even further back) against criteria that were not in force or even available at the time is simply unethical. If organisations are going to use metrics, it is because they want to (a) select staff that are performing at a high level and (b) encourage staff to perform at what is considered to be a high level. Evaluating staff who have been trying to satisfy one regime against totally new criteria is unfair and unreasonable, yet happens all the time. There need to be grandfather clauses.

I fully agree that we need to do high-impact science. I fully agree that staff need to be encouraged to publish in top journals. But actively precluding publishing in lesser, but still sound, journals is short-sighted and dangerous, and an example of how the careless use of metrics is destructive. Perhaps metrics are a necessary evil, but I have yet to see whether they do more good than harm.


Pontification over.