Wednesday, 29 January 2014

There are no “Schoolboy Errors” in our report

by Christian Dustmann and Tommaso Frattini

Centre for Research of Analysis and Migration (CReAM) at University College London

On January 2nd, Civitas published two reports on our paper The Fiscal Effect of Immigration on the UK on their website, and added a press-release entitled “Schoolboy errors in UCL report claiming fiscal benefit to immigration”. Reading the two reports carefully, it is puzzling what has led Civitas to this headline, and to some of the statements made in the press release.

The first piece [] is by Prof. Mervyn Stone, an emeritus statistics professor at UCL.

Prof. Stone thought that the report is an ambitious and largely scholarly study, in which crucial assumptions (about how to share expenditures and revenue between immigrants and natives) are set out with commendable clarity – and are therefore open to a degree of critical comment…. He then raises some econometric/statistical criticisms of our analysis. However, the main and most important part of the report does not contain any econometrics, as Prof. Stone admits (“Econometric modelling was not invoked for the estimation of fiscal effects”). His piece has therefore not much to say at all about the main part of our paper, which relates to the fiscal effects of immigration.

The emphasis of his piece is rather on the estimation of probability models to determine whether immigrants from different groups are more or less likely than natives to receive state benefits/tax credits or live in social housing. Overall, all of these comments are quite minor and indeed “text-book” like, and we cannot detect any hint to a fundamental flaw in the way we have conducted our analysis. As with any analysis of data, the analyst has to make some assumptions, which is what we have done here as well, and which - as Prof. Stone admits – we “set out with commendable clarity”. None of the assumptions Prof. Stone mentions in his piece would in our view change the main conclusions that we draw from this part of our analysis, as we illustrate in our brief appendix below.

The second piece [], by Nigel Williams, focuses in turn on the fiscal contribution analysis. His emphasis is on the government data we use and the assumptions needed for conducting analysis. Some of these comments are indeed well taken – and we are very clear in our report about the difficulties in conducting such analysis, based on the data that is available. In fact, we devote an entire section (section 2) to discussion of issues involved. However, overall, it is very unlikely that any other, equally reasonable, assumptions would change the general conclusions of our report. In fact, most of the points raised by Mr Williams will lead to a more positive, rather than more negative, conclusion on the net contribution of immigrants.

For instance, Nigel Williams argues that the cost of interest on public sector debt should accrue only to natives, and not to immigrants, since it is natives who accumulated the debt. This means that – if anything - we are over-estimating the fiscal cost of immigration. Similarly, he argues that ”apportioning the cost of immigration and citizenship police services (as we do in our main scenario)  entirely to immigrants is debatable.” We agree, but nevertheless we chose this option in our analysis because it provides a “worst case scenario” from the immigrants’ point of view. Assigning the cost of interest on public sector debt only to natives, as suggested by Mr Williams, would decrease the estimated fiscal cost of immigrants and correspondingly increase the fiscal cost of natives.

Therefore, again, we appreciate the comments by this author and the interest he took in our study. Although we agree on a number of issues brought up, we also do not believe that any of the points raised would change our main conclusions.

Thus, we were surprised by the mismatch between the content of the reports and the aggressively condescending tone of Civitas’ press release [].

In this context, we would like to raise two points.

First, in the two reports there seems to be the suggestion that we should better not have done any analysis at all, as the data is not “perfect” and there is some remaining uncertainty in our findings. We totally disagree with this view. No analysis based on data will ever lead to results that are absolutely free of “uncertainty”, and no data is ever “perfect”. We have followed good academic practice and set out clearly the assumptions we have made in this piece, as has been acknowledged by Prof. Stone. We have (in earlier replies to comments []) computed some extreme scenarios and we have shown that even that would not have changed our main conclusions. Thus, different assumptions may lead to slightly higher or slightly lower net contributions of immigrants, but they will not change the general conclusions of the study – namely that EEA immigrants who arrived after 1999 have made a substantial net fiscal contribution to the UK. We believe that – in a climate where anecdotal evidence rather than well researched data work dominate the public and policy debate – this is an important piece of information that the public ought to know.

Second, both pieces mention the 2003 report on the likely inflow of immigrants from the A8 countries to the UK [] that Christian Dustmann co-authored. This report has nothing to do with our latest piece, but their criticism is eagerly taken up by Civitas director David Green to insult us and our reputation. Christian Dustmann and Ian Preston have responded to the ill-informed criticisms of that report in a separate piece [].

In conclusion, we welcome constructive comments on our analysis. We are pleased that our report is so thoroughly publicly scrutinised, and we believe that this interchange will help improve the way we inform the public debate on this important and sensitive issue. However, we reject the offensive tones used by Civitas’ press release, and we believe that if accusing someone of "schoolboy errors", as done by Civitas’ director David Green, you ought to be able to point to more actual errors.

Mervyn Stone, “Plain Assumptions and Unexplained Wizardy Called in Aid of “The Fiscal Effects of Immigration to the UK”

The piece has two parts.

In part one (“The Cream fiscal effect calculation”), Prof. Stone discusses our fiscal effect calculations. It remains unclear to us what the point of this section is – Stone’s report simply repeats our calculations and lists numbers in slightly different ways in his first two tables.

Part two (“The econometrics that Cream calls on to estimate putative ‘probability gaps’) refers to Table 3 in our paper where we fit linear probability models  to investigate the probability of welfare receipt of different immigrant populations, as compared to native born individuals. The main observations of Prof. Stone refer to two issues, the fit of our estimated model, and the model specification.
  1. The fit of the model, as measured by the coefficient of determination (R2). R2 is a statistics that measures how much of the variation in the outcome variable that is explained by the independent variables is included in the model. It is a useful statistic if we would want to use our model for predictions. However, our analysis is aimed at estimating the difference in the probability of welfare receipt between two groups of individuals, immigrants and natives. In the simplest case (if we were interested in the unconditional difference, and the data was for one cross section only), this difference in probabilities would simply be the difference in the mean of the share of immigrants and the share of natives who receive welfare. No statistical measure of fit is needed to understand this difference, obviously. Further, this difference in the proportion of immigrants and natives’ welfare receipt would be estimated more precisely the more data points are available, as this adds information, and would thus reduce sampling error and increase statistical significance (see his point (iii) (a) on page 13). When we condition on observables, what matters is not R2 per se, but how different characteristics may affect benefit take-up and whether these characteristics are correlated with immigrant status.
  2. The specification of the model. (i) Choice of estimator: We estimate a simple linear probability model, which is easy to interpret. The method essentially fits cell probabilities (see above example), especially when all regressors are binary and mutually orthogonal (in which case probit models and LPM produce exactly the same cell probabilities). When we condition on observables, there may be some extrapolation because of functional form assumptions. However, re-estimating our specification using a probit estimator results in very similar conclusions (we are happy to provide the estimates). (ii) Specification: Our data covers many periods (quarters), and we are interested in a summary measure of the differences in welfare receipt between immigrants and natives. The specification we have chosen conditions on a set of time dummies (to allow for variation in welfare receipt over time that affects immigrants and natives alike), but does not allow for interactions between quarters and immigrant status in addition. As Prof Stone points out himself, our coefficient estimate is therefore interpretable as the weighted combination of the differences between welfare receipt between immigrants and natives across all quarters, which is precisely the coefficient we wish to estimate, as it has a meaningful and simple interpretation in this context. It is therefore a representation of the “weighted averaged” difference in welfare receipt between immigrants and natives over all quarters observed, conditioning on fluctuations in welfare receipt over time that affect immigrants and natives alike. We do agree however that the formulation “… difference in the probability of receiving benefits or living in social housing between immigrants and natives observed at the same moment in time” is imprecise, as Prof Stone points out – what we should have said is “…the weighted averaged difference across quarters in the probability of receiving benefits or living in social housing between immigrants and natives, conditioning on fluctuations in welfare receipt over time that affect immigrants and natives alike.” Thanks for pointing this out to us. (iii) Conditional models: To capture differences between immigrants and natives in demographic characteristics, we condition on gender and a quadratic in age. Again, this is a standard procedure. Of course, it implies an assumption about functional form – which we believe is not implausible but at the same time simple and transparent. One could relax functional form assumptions by including a full set of dummy variables for age, and interact them with gender dummies, or use matching type estimators. Using such estimators, results show an even larger difference in welfare and transfer receipt between immigrants and natives than reported in our Table 3. For instance the gap resulting from a fully interacted model specification is  -0.125 for immigrants arriving since 2000, compared to the estimates of  our more restricted specification reported in the Table, which gives an estimate of -0.084.
  3. Robust standard errors: This is standard jargon in econometrics for the Huber and White estimator of the variance (see White, 1980 and MacKinnon and White, 1985), an estimator that corrects for the heteroscedasticity  implied by the linear probability model. It is a textbook-like correction to make when calculating standard errors in this context (see e.g. Wooldridge (2001), page 454).

MacKinnon, J. G., and H. White (1985), “Some heteroskedasticity-consistent covariance matrix estimators with improved finite sample properties.” Journal of Econometrics, 29, 305–325.

White, H. (1980), “A Heteroskedasticity-Consistent Covariance Matrix Estimator and a Direct
Test for Heteroskedasticity,” Econometrica, 48, 817–838.

Wooldridge, J.M. (2001), “Econometric Analysis of Cross Section and Panel Data”, 1st edition, MIT Press.

1 comment:

  1. The central problem with your report is that it omits many of the costs arising from mass immigration, viz:

    I cannot post the whole article here because of the limitations on characters but a few of the omitted costs are:

    7. The cost of education given to immigrants, this to include the additional costs arising from those with poor or non-existent English.

    8. The cost of benefits, education and NHS care for the children of immigrants born in the UK.

    9. The costs of benefits paid to immigrants to support children born abroad and living abroad.

    10. The inflation of housing costs caused by immigrants and their children born in the UK increasing the demand for housing.


Note: only a member of this blog may post a comment.