The comment chain that I am RIing is here. The context is a discussion of the impact of immigration on native wages. In discussing the evidence about the impact of immigration, /u/simcar01 makes the following claims about immigration research:

The papers don't show the effects of immigration compared to less immigration. So we don't know it has had no/little effect - because we don't know where we would be with lower levels.

Have you got a paper that looks at the effects of immigration level on the wages where "all other things are equal"?

No - the studies are attempting to show the effects immigration have had, compared to the historical trend. Since nothing is equal from one year to the next, it is anecdotal at best.

They don't show what effect a different rate of immigration would have - and they never attempt to.

This represents a basic misunderstanding of how econometrics works. This RI does not take a position on what the real effect of immigration on wages is; it is purely about the econometric techniques used to estimate it.

Let's go back to fundamentals. Suppose the only factor that affected wages was the share of immigrants in a local area's population - an individual's education, training, the unemployment rate, the year and any other factors all had no effect on wages. And also suppose that the wage in an area has no effect on the level of immigration to an area. If we want to measure the effect of immigration on wages in that scenario, it's easy. All we have to do is look at the average wage in each area in each time period, and see how much lower it is in times and places with a higher immigrant share. To put a precise number on this, we suppose that each individual's wages are determined by this equation:

wage = b0 + b1*immigration + u

where u is some random variation that is equally likely to be positive as negative. This implies that in an area with no immigration (where immigration=0) we would expect the wage to be equal to b0, and in areas with higher immigration, the wage would be higher if b1 were positive and lower if b1 were negative. The larger the magnitude of b1, the larger the true effect of immigration on wages. We have data on wages and immigration. We can generate a predicted wage, w-hat, if we estimate values for b0 and b1, and say that the error is 0. We use the data to select values for b0 and b1 that minimise the sum of the differences between w-hat and the true wage for each individual. Basically, we are trying to find the estimate of the baseline wage (with no immigration) and the effect of immigration on wages that best fit the data. Once we have this estimate, we can easily use it to make a prediction about what wages might be if the immigration rate were different - we simply plug in a different value for immigration into the equation, and see what wage is predicted.

Of course, the real world is not that simple. We ignored two specific problems in the first stage - one problem being that factors other than immigration affect wages, the other that there might be an effect of wages on immigration (since immigrants are likely to choose to migrate to places where they can get good wages). Both of these things cause problems for our estimate of the effect of immigration that we calculated before. Suppose we found that b1 was positive, so areas and times with higher immigration tended to have higher wages on average. If wages encourage higher immigration, that correlation might be because immigrants go to the places where there are higher wages. It could still be possible that when immigrants are there, they pull wages down relative to how high wages would have been in the area if the immigrants hadn't gone there. Similarly, suppose we found b1 was positive - areas and times with higher immigration had higher wages - and we somehow knew this wasn't because of immigrants going to the area when wages are higher. It's possible that that is just because wages increased across the board over time, and immigration also increased over time. So within each year, the places with higher immigration would tend to have lower wages, but over time data points with higher immigration would tend to have higher wages, just because they were later in time than the data points with lower immigration. Both these things might lead us to get our estimate of the real effect of immigration on wages wrong. And if we get that estimate wrong, we can't make an accurate prediction about what would happen if we lowered immigration.

But faced with these problems, econometricians don't just say "Woe is me, whatever shall we do" and just look at comparing the trends in immigration and wages over time, giving up on finding out the true effect of immigration on wages. They have a whole battery of techniques that they can use to try and work out the true effect of immigration, holding everything else constant.

The most basic approach is just to include other factors that affect wages in the regression. If you do that, then b1 represents the effect of immigration on wages holding the other factors that you include constant. For instance, to account for the fact that wages increase over time, you could include dummy variables that reflect the year each data point comes from. If you do that, then b1 measures the effect of immigration within a particular year - it measures whether areas with lower immigration in a given year have higher wages in that year. Any change in wages over time that is not driven by changes in immigration, but by natural growth, will be measured as being part of the effect of the year, not of immigration. This can be done for any factor that might affect wages, and all the papers referenced below include some kinds of additional controls for other variables, so that the effect of immigration that they estimate is holding some other factors constant.

Another approach is to try to find some kind of "natural experiment". Sometimes there is an inflow of immigrants to an area that is not driven by the immigrants choosing places with higher wages, but is for some reason random. If this happens, we can compare the areas with an inflow of immigrants to ones without the same inflow. We would expect wages in both areas to increase over time, but if wages in the area with high immigration increase less than in the area without high immigration, then we can attribute this difference to the effect of the immigration - particularly if we control for other factors that might cause the two areas to have different wage growth. We can then see how much immigration affected wage growth and use this to make predictions about how wages would grow if immigration were higher or lower. If we do not have a natural experiment where the inflow of immigrants is random, we cannot do this, as we can't rule out the possibility areas with higher wages attracted more immigrants, thereby obscuring the true effect of immigration on wages. This approach was taken in a famous paper by David Card, who looked at the impact of Cuban immigration to Miami in the Mariel Boatlift, comparing Miami to similar US cities that did not see a sudden inflow of immigrants. Another paper using this strategy is by Foged and Peri, who used the fact that refugees arriving in Denmark were allocated more or less randomly to different locations within the country to estimate the effect of these migrants on the wages of native workers.

However, natural experiments may not be that common, and if they only happen in certain countries we may not be able to get results from them that generalise across countries. In the absence of natural experiments, an instrumental variables approach can be used. The first step is to look at how historic levels of migration affect current migration levels. Historic migration levels cannot have been affected by current levels of wages, and they affect current migration levels because migrants will tend to want to move to places with a higher migration density, as they have family values. So we can extract from that a measure of the migration that we would predict an area today would have based on historic migration levels. This predicted migration cannot be affected by current wages, which solves the problem of current wages determining where immigrants move to. If we then estimate the effect of the predicted immigration density on wages, this effect should be the true effect of immigration on wages, not distorted by the fact that wages also affect immigration. This approach has been taken by some papers that focus on immigration to the UK, such as one by Dustmann, Fabbri and Preston, and a Bank of England working paper by Nickell and Saleheen that has been discussed a lot on /r/ukpolitics since it was published.

So, in fact, all worthwhile economic research on immigration is trying to isolate the effect that immigration has on wages all other things equal, and all of it is trying to measure that effect in such a way that you could make a quantitative prediction of what wages would have been if immigration were higher or lower than it actually was, but nothing else changed. When the Bank of England paper says that in the semi/unskilled services sector, a 10 percentage point rise in the proportion of immigrants is associated with a 2 percent reduction in pay, that is a statement about the effect of immigration holding other things equal. And it implies that if there had been less immigration of semi/unskilled service workers, wages would have been correspondingly higher, in the ratio derived in the paper; it thus shows what effect a different rate of immigration would have. Other studies are estimating similar things. They are certainly not simply looking at the effect immigration had compared to the historical trend - in fact, some of the papers include dummy variables for time, so they are controlling for any change over time not driven by variation in immigration levels. Econometrics is a lot more credible than many people think, and is able to at least make a good attempt at estimating the answer to the question "what would happen to wages if immigration were lower?" This is not a question about which we know nothing because no-one has tried to answer it. It should not be treated as such.