I put this here as a warning. While growing degree-days (GDD) are well-known as an effective model to predict yields, they don’t perform so hot at the country-scale.

I used mean temperature GDDs, between 8 and 24 degrees C, estimated at many locations from station data, and then using the weighted average by production within each country. Here are the results:

Statistical models

	Barley	Maize	Millet	Rice	Sorghum	Wheat
GDD / 1000	-0.03	0.01	-0.07**	0.08	0.04*	-0.08***
	(0.01)	(0.01)	(0.03)	(0.06)	(0.02)	(0.02)
Precip. (m)	0.09	0.11***	0.12*	0.02	0.14***	-0.04
	(0.05)	(0.03)	(0.05)	(0.03)	(0.04)	(0.04)
Country Cubic	Y	Y	Y	Y	Y	Y
R2	0.95	0.97	0.91	0.97	0.92	0.96
Adj. R2	0.94	0.96	0.90	0.97	0.91	0.95
Num. obs.	1639	3595	1516	1721	2300	1791
***p < 0.001, **p < 0.01, *p < 0.05

As you can see, for most crops, these GDDs aren’t even significant, and as frequently negative as positive. This defies a century of agricultural research, but the same data at a fine spatial scale seems to work just fine.

Schlenker and Roberts (2009) use daily minimum and maximum temperatures to calculate growing degrees, rather than daily mean temperatures. This is important when the effect of extreme temperatures is an issue, since these often will not show up in mean temperatures.

Growing degree days form a useful model of crop productivity. DMAS has examples of these for maize, soybeans, and cotton.

To do this, they use a sinusoidal approximation, integrating the area of a curve through the minimum and maximum temperatures:

(adapted from here– but don’t use their calculations!)

The calculations aren’t very difficult, but require some careful math. I had a need to write them in python and translate them to R, so I’m providing them here for anyone’s benefit.

import numpy as np
import warnings

warnings.simplefilter("ignore", RuntimeWarning)

def above_threshold(mins, maxs, threshold):
    """Use a sinusoidal approximation to estimate the number of Growing
Degree-Days above a given threshold, using daily minimum and maximum
temperatures.

mins and maxs are numpy arrays; threshold is in the same units."""

    # Determine crossing points, as a fraction of the day
    plus_over_2 = (mins + maxs)/2
    minus_over_2 = (maxs - mins)/2
    two_pi = 2*np.pi
    # d0s is the times of crossing above; d1s is when cross below
    d0s = np.arcsin((threshold - plus_over_2) / minus_over_2) / two_pi
    d1s = .5 - d0s

    # If always above or below threshold, set crossings accordingly
    aboves = mins >= threshold
    belows = maxs <= threshold

    d0s[aboves] = 0
    d1s[aboves] = 1
    d0s[belows] = 0
    d1s[belows] = 0

    # Calculate integral
    F1s = -minus_over_2 * np.cos(2*np.pi*d1s) / two_pi + plus_over_2 * d1s
    F0s = -minus_over_2 * np.cos(2*np.pi*d0s) / two_pi + plus_over_2 * d0s
    return np.sum(F1s - F0s - threshold * (d1s - d0s))

def get_gddkdd(mins, maxs, gdd_start, kdd_start):
    """Get the Growing Degree-Days, as degree-days between gdd_start and
kdd_start, and Killing Degree-Days, as the degree-days above
kdd_start.

mins and maxs are numpy arrays; threshold is in the same units."""

    dd_lowup = above_threshold(mins, maxs, gdd_start)
    dd_above = above_threshold(mins, maxs, kdd_start)
    dd_lower = dd_lowup - dd_above

    return (dd_lower, dd_above)

Download the code for R or python.

Below is my Prezi presentation, from a 4 minute “speed talk” on my research a week ago:

The main points of the narrative are below:

1. Frame 1: My pre-thesis research:
   • I’m going to give a brief sense of where my research has gone, and my hopes for where it’s going.
   • My research has evolved– or wandered– but I’ve realized that it centers around food.
   • Food represents some of the most important ways humanity both relies upon and impacts the planet.
   • We need better models to understand this relationship– models that are sensitive to changing environmental conditions (including climate), and changing management (including adaptation).
2. Frame 2: My thesis:
   • Perhaps more importantly, we need models powerful enough to capture feedbacks.
   • Harvests feedback and affect the harvesters: Fluctuations in a harvest affect producers in immediate and long-term ways.
   • Producers aren’t mechanical or linear: they respond dynamically to conditions.
   • Producers are part of the system, and I’m interested in understanding these systems as wholes.
3. Frame 3: The greater whole:
   • These feedback loops are part of a larger integrated system of how food works.
   • Example: When farmers use excessive inputs in the course of their management, it impacts the environment, including that of fishers.
   • Example: 1/3 of fish caught is sold as fishmeal, which goes to feed livestock. These are all connected.
4. Frame 4: What we eat becomes a driving force throughout this integrated system.
5. Frame 5: A call for help:
   • Consumption patterns feed back on the production system, which feeds back on large environmental processes like climate change.
   • We need integrated models with all these parts to understand the whole and find leverage points where policies can make a difference.
   • I’d like to invite people from many fields to help out: biologists, computer scientists, economists, and foodies too.
   • Help me better represent the vast opportunities the food system holds for a sustainable future.

There’s a lot we don’t know about future climates. Despite continuous improvements, our models still do notoriously poorly with some fairly fundamental dynamics: ocean energy transport, the Indian monsoon, and just reproducing past climate. Worse, we have the computing resources to run each model a hand-full of times. We haven’t estimated the distribution of possible temperatures that even one model predicts.

Instead, we treat the range of models as a kind of distribution over beliefs. This is useful, but a poor approximation to an actual distribution. For one, most of the models are incestuously related. Climate is chaotic, no matter how you model it, so changing the inputs within measurement error can produce a whole range of future temperatures.

This is all motivation for a big innovation from the Risky Business report, on the part of Bob Kopp and DJ Rasmussen.

First, they collected the full range of 44 CMIP5 models (downscaled and adjusted to predict every US county). They extracted each one’s global 2080-2099 temperature prediction.

Then they ran an *hemisphere* scale model (MAGICC), for which its possible to compute enough runs to produce a whole distribution of future temperatures.

Here’s the result: a matching between models and the temperatures distribution for end-of-century global temperatures.

[From American Climate Prospectus, Technical Appendix I: Physical Climate Projections]

The opaque maps are each the output of a different GCM. The ghosted maps aren’t– at least, not alone. In fact, every ghosted map represents a portion of the probability distribution which is currently missing from our models.

To fix this, they made “surrogate” models, by decomposing existing GCM outputs into “forced” seasonal patterns which scale with temperature, and unforced variability. Then they scaled up the forced pattern, added back in the variability, and bingo! A new surrogate model.

Surrogates can’t tell us the true horrors of a high-temperature world, but without them we’d be navigating without a tail.

This animation is dedicated to my advisor Upmanu Lall, who’s been so supportive of all my many projects, including this one! Prabhat Barnwal and I have been studying the effects of Hurricane Sandy on the New York area through the lens of twitter emotions, ever since we propitiously began collecting tweets a week before the storm emerged (see the working paper).

The animation shows every geolocated tweet in between October 20 and November 5, across much of the Northeastern seaboard– about 2 million tweets in total.

Each tweet is colored based on our term-usage analysis of its emotional content. The hue reflects happiness, varying from green (happy) to red (sad), greyer colors reflect a lack of authority, and darker colors reflect a lack of engagement. The hurricane, as a red circle, jutters through the bottom-left of the screen the night of Oct. 29.

The first thing to notice is how much more tweeting activity there is near the hurricane and the election. Look at the difference between seconds 22 (midnight on Oct. 25) and 40 (midnight on Oct. 30).

The night of the hurricane, the tweets edge toward pastel as people get excited. The green glow near NYC that each night finishes with becomes brighter, but the next morning the whole region turns red as people survey the disaster.

My girlfriend uses a “Moka Express”-style stovetop Bialetti espresso-maker most mornings for her cappuccino. These devices are wonderful, reliable, and simple, but they have a nearly fatal flaw. A basin collects the espresso as it condenses, and for the five minutes before the steam brewing happens, there will be no indication of anything happening. Then espresso will start to quietly gurgle up, and if you don’t stop the process in the 20 seconds after it starts, the drink will be ruined.

I built a simple device to solve this problem. It has two wires that sit in the basin, and a loud buzzer that sounds as soon as coffee touches them.

How it works

Here is the circuit diagram for the coffee buzzer, designed to be powered by a 9V battery:

The core of the coffee buzzer is a simple voltage divider. Normally, when the device is not in coffee, the resistance through the air between the two leads on the left (labeled “LOAD”) is very high. As a result, the entire voltage from 9V battery is applied to that gap, so that the voltage across the 500 KΩ resistor is 0.

The IRF1104 is a simple MOSFET, which acts like a voltage-controlled switch. With no voltage across the resistor, the MOSFET is off, so the buzzer doesn’t sound.

To turn the MOSFET on, the voltage across the 500 KΩ resistor needs to be about 2 V. As a result, anything between the two LOAD leads with a resistance of less than about 2000 KΩ will cause the buzzer to turn on.

Coffee resistance seems to vary quite a bit, and the 137 KΩ shown here is on the low end. For this, you need a resistor of at least 40 KΩ. I suggest using something higher, so the detector will be more sensitive.

What you need

Tools

You will need wire-strippers, a multimeter (to check your circuit), and a soldering iron (to put everything together).

Parts

These wires will be the main detector of the coffee buzzer.

Here I use a 1500 KΩ resistor, so the buzzer will sound for resistances less than 5000 KΩ will be detected. Make sure that you use a resistor of more than 300 KΩ.

The MOSFET is a voltage controlled switch, and it’s what allows the buzzer to get plenty of current even though there’s a lot of resistance for current moving through the coffee.

You can get a 9V battery connector (top) and a buzzer (middle, rated for between 4 and 8 V) at RadioShack or from DigiKey. And, of course, the battery itself (not included).

Optional (but highly recommended!)

A simple switch (which may look very different from this one) is a great way to build an integrity tester into the device itself.

A breadboard will let you put the whole circuit together and test it before you solder it.

Wires like these make plugging everything into a breadboard easier.

I suggest taping up the whole device after you solder it, both for protection and to keep everything in one tight package.

How to make an Espresso Buzzer

Start by preparing the detector wires (the black and white wires above). Take at least 30 cm of wire, so the device can sit on the counter away from the flame. Use the wire stripper to strip one end of the wires for use in the circuit. You may want to strip the “detection” end, or not: if you leave the detector end unstripped, the buzzer won’t go off prematurely when if both wires touch the bottom of the basin, but you’ll have to wipe off the end of the wires to get them to stop buzzing once they have started.

Now connect all of the pieces on the breadboard. Here’s one arrangement:

If you aren’t sure how to use a breadboard or how to read a circuit diagram, you can still make the buzzer by skipping this step and soldering the wires together as specified below.

Once everything is connected on the breadboard, you should be able to test the coffee buzzer. If you use a switch, pressing it should make the buzzer sound. Then make a little experimental coffee, and dip the leads into the coffee to check that it works.

Next, solder it all together. As you can see in the circuit diagram, you want to solder together the following groups of wires:

• Ground: The black wire from the battery connector; one wire from the resistor; and the source lead on the MOSFET (furthest right).
• Power: The red wire from the battery connector; one wire from the buzzer; the (stripped) circuit end of one of the detector wires; and optionally one end of the switch.
• Gate: The (stripped) circuit end of the other detector wire; the other end of the resistor; the gate lead on the MOSFET (furthest left); and optionally the other end of the switch.
• Buzzer: The gate lead on the MOSFET (middle); and the remaining wire to the buzzer.

After you have soldered everything together, test it and then wrap tape around it all.

The results

That’s it! Now, to use it, place the detector leads at the bottom of the coffee basin. I suggest putting a bend into the wires a couple inches from the end, so they can sit easily in the basin. If you’ve stripped the ends, the buzzer may buzz when the leads touch the base, but if you just let them go at this point, one wire will sit slightly above the other and just millimeters away from the bottom for perfect detection.

As soon as the coffee reaches a level where both leads are submerged, the detector will start buzzing!

Enjoy!

R doesn’t have a built-in method for calculating heteroskedastically-robust and clustered standard errors. Since this is a common need for us, I’ve put together a little library to provide these functions.

Download the library.

The file contains three functions:

• get.coef.robust: Calculate heteroskedastically-robust standard errors
• get.coef.clust: Calculate clustered standard errors
• get.conf.clust: Calculate confidence intervals for clustered standard errors

The arguments for the functions are below, and then an example is at the end.

get.coef.robust(mod, var=c(), estimator=”HC”)
Calculate heteroskedastically-robust standard errors

mod: the result of a call to lm()
     e.g., lm(satell ~ width + factor(color))
var: a list of indexes to extract only certain coefficients (default: all)
     e.g., 1:2 [to drop FE from above]
estimator: an estimator type passed to vcovHC (default: White's estimator)
Returns an object of type coeftest, with estimate, std. err, t and p values

get.coef.clust(mod, cluster, var=c())
Calculate clustered standard errors

mod: the result of a call to lm()
     e.g., lm(satell ~ width + factor(color))
cluster: a cluster variable, with a length equal to the observation count
     e.g., color
var: a list of indexes to extract only certain coefficients (default: all)
     e.g., 1:2 [to drop FE from above]
Returns an object of type coeftest, with estimate, std. err, t and p values

get.conf.clust(mod, cluster, xx, alpha, var=c())
Calculate confidence intervals for clustered standard errors

mod: the result of a call to lm()
     e.g., lm(satell ~ width + factor(color))
cluster: a cluster variable, with a length equal to the observation count
     e.g., color
xx: new values for each of the variables (only needs to be as long as 'var')
     e.g., seq(0, 50, length.out=100)
alpha: the level of confidence, as an error rate
     e.g., .05 [for 95% confidence intervals]
var: a list of indexes to extract only certain coefficients (default: all)
     e.g., 1:2 [to drop FE from above]
Returns a data.frame of yhat, lo, and hi

Example in Stata and R

The following example compare the results from Stata and R for an example from section 3.3.2 of “An Introduction to Categorical Data Analysis” by Alan Agresti. The data describes 173 female horseshoe crabs, and the number of male “satellites” that live near them.

Download the data as a CSV file. The columns are the crabs color (2-5), spine condition (1-3), carapace width (cm), number of satellite crabs, and weight (kg).

The code below calculates a simple model with crab color fixed-effects, relating width (which is thought to be an explanatory variable) and number of satellites. The first model is a straight OLS regression; then we add robust errors, and finally crab color clustered errors.

The analysis in Stata.

First, let’s do the analysis in Stata, to make sure that we can reproduce it in R.

import delim data.csv

xi: reg satell width i.color
xi: reg satell width i.color, vce(hc2)
xi: reg satell width i.color, vce(cluster color)

Here are excerpts of the results.

The OLS model:

------------------------------------------------------------------------------
      satell |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
       width |   .4633951   .1117381     4.15   0.000     .2428033    .6839868
       _cons |  -8.412887   3.133074    -2.69   0.008    -14.59816   -2.227618
------------------------------------------------------------------------------

The robust errors:

------------------------------------------------------------------------------
             |             Robust HC2
      satell |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
       width |   .4633951   .1028225     4.51   0.000     .2604045    .6663856
       _cons |  -8.412887   2.939015    -2.86   0.005    -14.21505   -2.610726
------------------------------------------------------------------------------

Note that we’ve used HC2 robust errors, to provide a direct comparison with the R results.

The clustered errors.

------------------------------------------------------------------------------
             |               Robust
      satell |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
       width |   .4633951   .0466564     9.93   0.002     .3149135    .6118767
       _cons |  -8.412887   1.258169    -6.69   0.007    -12.41694   -4.408833
------------------------------------------------------------------------------

The robust and clustered errors are a little tighter for this example.

The analysis in R.

Here’s the equivalent code in R.

source("tools_reg.R")
tbl <- read.csv("data.csv")

mod <- lm(satell ~ width + factor(color), data=tbl)

summary(mod)
get.coef.robust(mod, estimator="HC2")
get.coef.clust(mod, tbl$color)

And excerpts of the results. The OLS model:

               Estimate Std. Error t value Pr(>|t|)    
(Intercept)     -8.4129     3.1331  -2.685  0.00798 ** 
width            0.4634     0.1117   4.147 5.33e-05 ***

The coefficient estimates will be the same for all three models. Stata gives us -8.412887 + .4633951 w and R gives us -8.4129 + 0.4634 w. These are identical, but with different reporting precision, as are the standard errors.

The robust errors:

(Intercept)    -8.41289    2.93902 -2.8625  0.004739 ** 
width           0.46340    0.10282  4.5067 1.229e-05 ***

Again, we use HC2 robust errors. Stata provides 2.939015 and .1028225 for the errors, matching these exactly.

The clustered errors:

                Estimate Std. Error  t value  Pr(>|t|)    
(Intercept)    -8.412887   1.258169  -6.6866 3.235e-10 ***
width           0.463395   0.046656   9.9321 < 2.2e-16 ***

Stata gives the exact same values as R.

I’ve started a long project of identifying my top 500 articles and chapters– the papers that either had a great impact on me or that I keep returning to to illustrate a point. One of those is Jeremey Jackson et al. (2001), Historical overfishing and the recent collapse of coastal ecosystems.

The main argument– that overfishing precedes, predicts, and predisposes the present fragility of ecosystems to modern drivers like pollution– is less interesting than the case studies themselves: kelp forests, coral reefs, seagrass beds, oyster estuaries, and benthic communities. This before-after diagram drives the point home (I colored the changes):

The most depressing line is in the abstract:

Paleoecological, archaeological, and historical data show that time lags of decades to centuries occurred between the onset of overfishing and consequent changes in ecological communities, because unfished species of similar trophic level assumed the ecological roles of overfished species until they too were overfished or died of epidemic diseases related to overcrowding.