Document(s) 17 of 20

16.1 Sampling design

There exist in the population of interest a number of statistical units. For simplicity, we can think of these units as households or individuals. From an ethical perspective, it is usually preferable to consider individuals as statistical units of interest since it is in the welfare of individuals that we are ultimately interested, but for some purposes (such as the distribution of aggregate household wellbeing) households may also be appropriate statistical units.

These statistical units are those for which we would like to observe socio-economic information such as their household composition, labor activity, income or consumption. Since it is usually too costly to gather information on all of the statistical units of a large population, one would typically be constrained to obtain information on only a sample of such units. Distributive analysis is therefore usually done using survey data.

Since surveys are not censuses, we must take care to distinguish unobserved "true" population values from observed sample values. Sample differences across surveys are indeed due both to true population differences and to sampling variability. Population values are generally not observed (otherwise, we would not need surveys). Sample values as such are rarely of interest: they would be of interest in themselves only if the statistical units that appeared by chance in a sample were also precisely those which were of ethical interest. This is not usually the case. Hence, sample values matter in as much as they can help infer true population values. The statistical process by which such inference is performed is called statistical inference. The sampling process should thus ideally be such that it can be used to make some statistically-sensible distributive analysis at the level of the population, and not solely for the samples drawn.

Sampling errors thus arise because distributive estimates are typically made on the basis of only some of the statistical units of interest in a population. The fact that we have no information on some of the population statistical units makes us infer with sampling error the population value of the distributive indicators in which we are interested. The error made when relying solely on the information content of one sample depends on the statistical units present in that sample. The drawing of other samples would generate different sampling errors. Because samples are drawn randomly, the sampling errors that arise from the use of these samples are also random.

Since the true population values are unknown, the sampling error associated with the use of a given sample is thus also unknown. Statistical theory does, however, allow one to estimate the distribution of sampling errors from which actual (but unobserved) sampling errors arise. This nevertheless requires samples to be probabilistic, viz, that there be a known probability distribution associated to the distribution of statistical units in a sample. This also strictly means that there is absence of unquantifiable and subjective criteria in the choice of units. If this were not so, it would not be possible to assess reliably the sampling distribution of the estimators.

To draw a sample, a sampling base is used. A sampling base is made of all the sampling units (SU) from which a sample can be drawn. The base of sampling units — e.g., the census of all households within in a country — is usually different from the entire population of statistical units — e.g., the population of individuals, say. There are several reasons for this, an important one being that it is generally cost effective to seek information only within a limited number of clusters of statistical units, grouped geographically or socio-economically. This also facilitates the collection of cluster-level (e.g., village-level) information.

A process of simple random sampling draws sample observations randomly and independently from a base of sampling units, each with an equal probability of selection. Simple random sampling is rarely used in practice to generate household surveys. Instead, a population of interest (a country, say) is often first divided into geographical or administrative zones and areas, called strata. The first stage of random selection takes place from within a list of primary sampling units (denoted as PSU's) built for each stratum. Within each stratum, a number of PSU's are then randomly selected. PSU's are often departments, villages, etc.. This random selection of PSU's provides "clusters" of information.

Since the cost of surveying all statistical units un each of these clusters may be prohibitive, it may be necessary to proceed to further stages of random selection within each selected PSU. For instance, within each department, a number of villages may be randomly selected, and within every selected village, a number of households may also be randomly selected. The final stage of random selection is done at the level of the last sampling units (LSU's). These LSU's are often households. Each selected LSU can then provide information on all individuals found within that LSU. These individuals are usually not selected — information on all of them appears in the sample. They therefore do not represent LSU's in statistical terminology.

16.2 Sampling weights

Sampling weights (also called inverse probability, expansion or inflation factors) are the inverse of the sampling probabilities, viz, of the probabilities of a sampling unit appearing in the sample. These sampling weights are SU-specific. The sum of these weights is an estimator of the size of the population of SU's.

Samples are sometimes "self-weighted". Each sampling unit then has the same chance of being included in the survey. This arises, for instance, when the number of clusters selected in each stratum is proportional to the size of each stratum, when the clusters are randomly selected with probability proportional to their size, and when an identical number of households (or LSU's) across clusters is then selected with equal probability within each cluster.

It is, however, common for the inclusion probability to differ across households. One reason comes simply from the complexity of sample designs, which makes differential sampling weights occur frequently. Another reason is that the costs of surveying SU's vary, which makes it more cost effective to survey some households (e.g., urban ones) than others. Sampling precision can also be enhanced with differential probabilities of household inclusion. The idea here is to survey with greater probability those households who contribute more to the phenomenon of interest. It leads to a sampling process usually called sampling with "probability proportional to size".

Assume for instance that we are interested in estimating the value of a distribution-sensitive poverty index. The most important contributors to that index are obviously the poor households, and more precisely the poorest among them. An a priori suspicion might be that such poorest households are proportionately more likely to be found in some areas than in others. Making inclusion probabilities larger for households in these more deprived areas will then enhance the sampling precision of the estimator of the distribution-sensitive poverty index since it will gather data that are more statistically informative.

A reverse sample-design argument would apply for a survey intended to estimate total income in a population. The most important contributors to total income are the richest households, and it would thus be sensible to sample them with a greater probability. Yet one more consequence of the principle of "probability proportional to size" is the desirability of sampling with greater probability those households of larger sizes. Distributive analysis is normally concerned with the distribution of individual well-being. Ceteris paribus, larger-size households contribute more information towards such assessment, and should therefore be sampled with a greater probability (roughly speaking, with a probability proportional to their size).

Omitting sampling weights in distributive analysis will systematically bias both the estimators of the values of indices and points on curves as well as the estimation of the sampling variance of these estimators¹. Including such weights will usually make the analysis free of asymptotic biases. To see this, we follow Deaton (1998), p.45, and let Y be the population total of the x's, with a population of size N. An estimator of that population total is then given by

E:18.2.1

where ti is the number of times unit i appears in a random sample of size n and where w_i is the sampling weight. Let π_i be the probability that unit i is selected each time an observation is drawn from the population. Households with a low value of π_i will have a low probability of being selected in the survey, relative to others with a higher πi. Then, is the expected number of times unit i will appear in the sample, or, for large n, it is roughly speaking the probability of being in the sample. Hence,

and Y is therefore an unbiased estimator of Y. An analogous argument applies to show that is an unbiased estimator of population size N.

16.3 Stratification

The sampling base is usually stratified into a number of strata. The basic advantage of stratification is to use prior information on the distribution of the population, and to "partition" it in parts that are thought to differ significantly from each other. Sampling then draws information systematically from each of those parts of the population. With stratification, no part of the sampling base therefore goes totally unrepresented in the final sample.

To be more specific, a variable of interest, such as household per capita income, often tends to be less variable within some stratum than across an entire population. This is because households within the same stratum typically share to a greater extent than within the entire population some socio-economic characteristics — such as geographical locations, climatic conditions, and demographic characteristics — that are determinants of the incomes of these households, stratification helps generate systematic sample information from a diversity of "socio-economic areas".

¹DAD: Poverty| FGT Index.

Because information from a "broader" spectrum of the population leads on average to more precise estimates, stratification generally decreases the sampling variance of estimators. For instance, suppose at the extreme that household income is the same for all households in a given stratum, and this, for each and every stratum. In this case, supposing also that the population size of each stratum is known in advance, it would be sufficient to draw only one household from each stratum to know exactly the distribution of income in the population.

16.4 Multi-stage sampling

Multi-stage sampling implies that SU's end up in a sample only subsequently to a process of multi-stage selection. Groups (or "clusters") of SU's are first randomly selected within a population (which may be stratified). This is followed by further sampling within the selected groups, and followed by yet another process of random selection within the subgroups just selected.

The first stage of random selection is done at the level of primary sampling units (PSU). An important condition would seem to be that first-stage sampling be random and with replacement for the selection of a PSU to be done independently from that of another. There are many cases, however, in which this condition is not met.

1. First-stage sampling is typically made without replacement.
   
   This will not matter in practice for the estimation of the sampling variance if there is multi-stage sampling, that is, if there is an additional stage of sampling within each selected PSU. The intuitive reason is that selecting a PSU only reveals random and incomplete information on the population of statistical units within that PSU, since not all of these statistical units appear in the sample when their PSU is selected. Selecting that same PSU once more (in a process of first-stage sampling with replacement) does therefore reveal additional information, information different from that provided by the first-time selection of that PSU. This extra information is roughly of equal value to that which would have been revealed if a process of sampling without replacement had forced the selection of a different PSU.
   
   Hence, in the case of multi-stage sampling, first-stage sampling without replacement does not extract significantly more information than first-stage sampling with replacement. It does not therefore practically lead to less variable estimators than a process of first-stage sampling with replacement.
   
   If, however, there is no further sampling after the initial selection of PSU's, then a finite population correction (FPC) factor should be used in the computation of the sampling variance. This would generate a better estimate of the true sampling variance. If FPC factors are not used, then the sampling variance of estimators will tend to be overestimated. This means that it will be more difficult to establish statistically significant differences across distributive estimates, making the distributive analysis more conservative and less informative than it could be.
   
   
2. Sampling is often systematic.

Systematic sampling can be done in various ways. For instance, a complete list of N sampling units is gathered. Letting n be the number of sampling units that are to be drawn, a "step" s is defined as s = N/n. A first sampling unit is randomly chosen within the first s units of the sampling list. Let the rank of that first unit be . The n - 1 subsequent units with ranks k + s, k + 2s, k + 3s,..., k + ns then complete the sample.

If the order in which the sampling units appear in the sampling list is random, then such systematic sampling is equivalent to pure random sampling. If, however, this is not the case, then the effect of such systematic sampling on the sampling variance of the subsequent distributive estimators depends on how the sampling units were ordered in the sampling list in the first place.

1. For instance, a "cyclical" ordering makes sampling units appear in cycles. "Similar" sampling units then show up in the sampling list at roughly fixed intervals. Suppose for illustrative purposes that the size of these intervals is the same as s. Then, systematic sampling will lead to a gathering of information on similar units (e.g., with similar incomes), thus reducing the statistical information that is extracted from the sample. This will reduce the sampling precision of estimators, and increase their sampling variance.
   
2. A cyclical ordering of sampling units suggests that there is more sampling-unit heterogeneity around a given sampling unit than across the whole sampling base (since information around sampling units is simply cyclically repeated across the sampling base). A more frequent phenomenon arises when adjacent sampling units show less heterogeneity than that shown by the entire sampling base. A typical occurrence of this is when sampling units are ordered geographically in a sampling list. Households living close to each other appear close to each other in the list. Villages far away from each other are also far away in the sampling list. Since geographic proximity is often associated with socio-economic resemblance, the farther from each other in the list are sampling units, the more likely will they also differ in socio-economic characteristics.

Systematic sampling will then force units from across the entire sampling list to appear in the sample. Representation from implicit strata will thus be compelled into the sample. This will lead to a sampling feature usually called implicit stratification. Pure random sampling from the sampling list will not force such a systematic extraction of information, and will therefore lead to more variable estimators.

By how far implicit stratification reduces sampling variability depends on the degree of between-stratum heterogeneity which stratification allows to extract, just as for explicit stratification. The larger the heterogeneity of units far from each other, the larger the fall in the sampling variability induced by the systematic sampling's implicit stratification.

One way to account for and to detect the impact of implicit stratification in the estimation of sampling variances is to group pairs of adjacent sampling units into implicit strata. Assume again that n sampling units are selected systematically from a sampling list. Then, create n/2 implicit strata and compute sampling variances as if these were explicit strata. If these pairs did not really constitute implicit strata (because, say, the ordering in the sampling list had in fact been established randomly), then this procedure will not affect much the resulting estimate of the sampling variance. But if systematic sampling did lead to implicit stratification, then the pairing of adjacent sampling units will reduce the estimate of the sampling variance — since the variability within each implicit stratum will be found to be systematically lower than the variability across all selected sampling units.

Generally, variables of interest (such as incomes) vary less within a cluster than between clusters. Hence, ceteris paribus, multi-stage selection reduces the "diversity" of information generated compared to SRS and leads to a less informative coverage of the population. The impact of clustering sample observations is therefore to tend to decrease the precision of estimators, and thus to increase their sampling variance. Ceteris paribus, the lower the within-cluster variability of a variable of interest, the smaller the gain of information that there is in sampling further within the same clusters.

To see this, suppose the extreme case in which household income happens to be the same for all households in a cluster, and this, for all clusters. In such cases, it is clearly wasteful to adopt multi-stage sampling: it would be sufficient to draw one household from each cluster in order to know the distribution of income within that cluster. More information would be gained from sampling from other clusters.

16.5 Impact of sampling design on sampling variability

There are two modelling approaches to thinking about how data were initially generated. The first one, which is also the more traditional in the sampling design literature, is the finite population approach. The second approach is the super-population one: the actual population is a sample drawn from all possible populations, the infinite super-population. This second approach sometimes presents analytical advantages, and it is therefore also regularly used in econometrics.

To illustrate the impact of stratification and clustering on sampling variability, consider therefore the following "super-population model", based on Deaton (1998), p.56. Then, the income x_hij of a household j from a cluster i of a stratum h can be modelled as:

For simplicity, assume that the x_hij are drawn from the same number n of clusters in each of the L strata, and that the same number of LSU (or "households") m is selected in each of the clusters. The indices hij then stand for:

• h = 1,...,L: stratum h
  
  
• i = 1,...,n: cluster i (in stratum h)
  
  
• j = 1,...,m: household j (in cluster i of stratum h).

For simplicity, also assume that α_h, is distributed with mean 0 and variance , that is distributed with mean 0 and variance , and that εhij is distributed with mean 0 and variance . Assume moreover that these three random terms are distributed independently from each other.

16.5.1 Stratification

Say that we wish to estimate mean income μ. The estimator, , is given by

Let

be the estimator of the mean μh of stratum h. Clearly, and since by (16.4) and (16.5)

and

Because of the independence of sampling across strata, we also have that

The sampling variability of is thus a simple average of the sampling variances of the L strata's .

Stratification can in fact be thought of as an extreme case of clustering, with the number of selected clusters corresponding to the number of population clusters, and with sampling being done without replacement to ensure that all population clusters will appear in the sample. Suppose instead that one were to select L strata randomly and with replacement, to make it possible that not all of the strata will be selected. This is in a sense what happens when stratification is dropped and clustering is introduced. Using (16.4) and (16.5), we then have that

where th is a random variable showing the number of times stratum h was selected. Then, recalling that µ_h = µ + α_h and linearizing (16.9), we have approximately that

and thus that

since and E [t_h] = 1. Assuming independence between and t_h and between the , we have that

Since t_h follows a multinomial distribution, with var(t_h) = (L - 1)/L and cov(t_h, t_i) = - 1/L, we find that

Hence, using (16.12) and (16.15), we obtain

The last term in (16.16) is the effect upon sampling variability of removing stratification. The larger this term, the greater the fall in sampling variability that originates from stratification.

16.5.2 Clustering

Let us now investigate the effect of clustering on the sampling variance, that is, on var (). We find:

The first line of (16.17) follows by the definition of , and the second line follows from (16.3) — note that αh is fixed for all of the x_hij in the same stratum h. The last line of (16.17) is obtained from the sampling independence between β_hi and ε_hij.

Hence, for a per-stratum given number of observations mn, it is better to have a large n to reduce sampling variability, namely, it is better to draw observations from a large number of clusters. The larger the cross-cluster variability , the more important it is to have a large number of clusters in order to keep var () low. Ceteris paribus, for a given sample size and for a given , the sampling variance of distributive estimators is smaller the smaller the between-cluster heterogeneity, , but the larger the within-cluster heterogeneity, .

16.5.3 Finite population corrections

Sampling without replacement imposes that all of the selected sampling units are different. It therefore extracts on average more information from the sampling base than sampling with replacement, and ensures that the samples drawn are on average closer to the population of sampling units. Sampling without replacement therefore increases the precision of sample estimators. To account for this increase in sampling precision, a FPC factor can be used, although it complicates slightly the estimation of the variance of the relevant estimators.

Assume simple random sampling of n sampling units from a population of N sampling units. Thus, we have that w_i = N/n for all of the n sample observations. To illustrate the derivation of an FPC factor in this simplified case, we follow Cochrane (1977) and Deaton (1998), p.42-44. An estimator of the population total Y of the x's is given by

where the random variable t_i indicates whether — and how many times — the population unit i was included in the sample. Taking the variance of (16.18), we find:

Using (16.18) and (16.19), the distinction between simple random sampling with and without replacement is analogous to the distinction between a binomial and a multinomial distribution for the t_i. With sampling without replacement, the probability that any one population unit appears in the final sample is equal to n/N, i.e., E[t_i] = n/N. Since ti then takes either a 0 or a 1 value, it thus follows a binomial distribution with parameter n/N. The variance of t_i is then given by . The covariance cov(t_i, t_j) can be found by noting that E[t_i t_j] = P(t_i = t_j = 1) = n/N. (n - 1)/(N - 1), and thus that

Substituting var(t_i) and cov(t_i, t_j) into (16.19), and defining

we find

where 1 - f = (N - n)/N is an FPC factor.

Take now the case of simple random sampling with replacement. We can then express t_i for any given population unit i as a sum of n independent draws t_ij, with j = 1,..., n, each one t_ij indicating whether observation i was selected in draw j. Thus:

Since for any draw j, E[t_ij] = 1/N, the expected value of t_i, is again n/N, but t_i may now take values greater than 1. The draws t_ij being independent, and each draw having a binomial distribution with parameter 1/N, we have that

which is the variance of a multinomial distribution with parameters n and 1/N. It can be checked that the covariance cov(t_i, t_j) is given by -n/N². Substituting var(t_i) and cov(t_i, t_j) into (16.19) again, we now find

This is larger than (16.22): the difference between the two results equals

and depends on the magnitude of n relative to N. The larger the value of n relative to N, the greater the sampling precision gains that there are in sampling without replacement.

16.5.4 Weighting

We follow once more the approach of Cochrane (1977) and Deaton (1998), pp.45-49. Suppose that we are again interested in estimating the variance of the estimator Y of a total Y, but for simplicity assume that sampling is done with replacement so that we can for now ignore FPC factors. Y is now defined as:

Taking its variance, we find

t_i follows once more a multinomial distribution, but now with var(t_i) = nπ_i(l- π_i) and cov(t_i, t_j) = -nπ_i π_j. Substituting this into (16.28), we find

To estimate (16.29), we can replace population values by sample values and thus use the estimator

Denote as y_i = w_ix_i,i = 1,...,n, the n sample values of w_ix_i, and let . Then, (16.30) leads to

with the difference that a familiar n/(n - 1) small-sample correction factor has been introduced in (16.31) to correct for the small-sample bias in estimating the variance of the y_i. Incorporating weights in the estimation of sampling variances is thus relatively straightforward.

16.5.5 Summary

The above material calls to mind the importance for statistical offices of making available sampling design information. This includes providing

• the sampling weights;
  
  
• stratum and PSU (cluster) identifying variables;
  
  
• information on the presence or not of systematic sampling (and thus of implicit stratification), including the relationship between the numbering of sampling units and the original ordering of these units in the sampling base;
  
  
• the finite population correction factors, namely, the size of the sampling base, when appropriate.

Equipped with this information, distributive analysts can provide reliable estimates of the sampling precision of their estimators².

E; 18.9.1

16.6 Estimating a sampling distribution with complex sample designs

We provide in this section a detailed account of the computation of sampling variances in DAD, taking full account of the sampling design. Let:

• h = 1,..., L: the list of the strata (e.g. the geographical regions);
  
  
• i = 1,... ,N_h: the list of primary sampling units (PSU; e.g., villages) in stratum h;
  
  
• N_h: the population number of PSU in a stratum h;
  
  
• n_h: the number of selected PSU in a stratum h;
  
  
• M_hi. the population number of last sampling units (LSU) (e.g., households) in PSU hi;
  
  
• m_hi the number of selected LSU in the PSU hi (for instance, the number of households from village hi that appear in the sample);
  
  
• q_hij: the number of observations in selected LSU hij (e.g., the number of household members in a household hij whose socio-economic information is recorded in the survey, with each household member providing 1 line of information in the data file);

²DAD: Edit|Set Sample Design.

• w_hij: the sampling weight of LSU hij;
  
  
• . the population number of LSU (e.g., the number of households in the population);
  
  
• the number of selected LSU (e.g., the number of selected households that appear in the sample);
  
  
• X_hijk: the value of the variable of interest (e.g., adult-equivalent income) for statistical unit hijk in the population;
  
  
• S_hijk: the size of statistical unit hijk in the population (e.g., if the statistical unit is a household, then Shijk may be the number of persons in household hijk, or alternatively the number of adult equivalents);
  
  
• the population total of interest;
  
  
• x_hijk: the value of X (the variable of interest) that appears in the sample for sample observation hijk;
  
  
• S_hijk: the size of selected sample observation hijk;
  
  
•   the estimated population total of interest;
  
  
• the estimated population number of LSU;
  

• the relevant sum in LSU hij;
  
  
• the relevant sum in PSU hi;
  
  
• the relevant mean in stratum h.

The sampling covariance of two totals, and ( being defined similarly to ) is then estimated by

where

— note the similarity with (16.21) and (16.22) — and where f_h, is a function of a user-specified FPC factor, fpc_h, for stratum h, such that,

• if a fpc_h is not specified by the user, then f_h = 0;
  
  
• if fpc_h ≥ n_h, then f_h = n_h/fpc_h;
  
  
• if fpc_h ≤ 1, then f_h = fpc_h.

Recall that setting f_h ≠ 0 is useful only when the sampling design is of the form either of simple random sampling or of stratified random sampling with no subsequent sub-sampling within the PSU's selected. In both cases, sampling must have been done without replacement.

The variance of is obtained from (16.32) simply by replacing by .

An often-used indicator of the impact of sampling design on sampling variability is called the design effect, deff. The design effect is the ratio of the design-based estimator of the sampling variance over the estimate of the sampling variance assuming that we have obtained a simple random sample of m LSU without replacement. Denote this latter estimate as . Then,

For simple random sampling, we would have that

and, recalling (16.22), the sampling variance of would then equal

where var(y) is the variance of the population y_hij, and where if an FPC factor is specified for the computation of and f = 0 otherwise. V_SRS can then be estimated as follows:

Some of the above variables often take familiar forms and names:

• x_hijk can be thought of as an "individual-level" variable, such as height, health status, schooling, or own consumption. This variable is called the "variable of interest" in DAD. If x_hijk is indeed individual-specific, then s_hijk will not exceed 1 in most reasonable instances. Individual outcomes are, however, not always observed. Even if they are, we may sometimes believe that there is equal sharing in the household to which individuals belong. In those cases, x_hijk will typically take the form of adult-equivalent income or other household-specific measure of living standard.
  
  
• s_hijk gives the "size" of the sample observation hijk. This size may be purely demographic, such as the number of individuals in the unit whose living standard is captured by x_hijk. It may also be 1 even if hijk represents a household and if we are interested in a household count for distributive analysis. But s_hijk may also be an ethical size, which depends on normative perceptions of how important the unit is in terms of some distributive analysis. Examples of such sizes include the number of adult-equivalents in the unit (if, say, we wish to assign individuals an ethical weight that is proportional to their "needs"), the number of families, the number of adults, the number of workers, the number of children, the number of citizens, the number of voters, etc..
  
  
• q_hij is the number of sample observations or statistical units provided by the last sampling unit. This LSU may contain a grouping of households, of villages, etc... More commonly for the empirical analysis of poverty and equity, a LSU represents a household.

16.7 References

General references on estimation and inference taking into account survey design include Asselin (1984) and Cochrane (1977). Applications to economic analysis are discussed and presented in Deaton (1998), Howes and Lanjouw (1998) (focussing on poverty analysis), and Zheng (2002) (with a specific focus on Lorenz curves). Alternative approaches to taking into account survey design can be found inter alia in Cowell (1989) (modelling sampling weights as jointly distributed with living standards), and in Biewen (2002b) and Schluter and Trede (2002a) (for dependence across members of the same sampling unit — households in their case). Kennickell and Woodburn (1999) illustrates the impact of a consistent estimation of survey weights in the US surveys of Consumer Finances for the analysis of the distribution of wealth.

Document(s) 17 of 20