All entries for August 2005

August 25, 2005

Econometric Society World Congress 2005


About identification, we can probably look at works by Chescher and Imbens. These are about Triangular equations with endogenous regressor (we can think of R as an endogenous regressor? and put R in the structural equation? if so, we are in the flamework of them). We may be can say why Y has to be discrete in RS(2003)

Xiaohong Chen wrote a paper with Chunrong Ai about efficient sequential estimation of semi-nonparametric moment models which include Newey and Chamberlain's bounds as special cases.

Floren Jean-Pierre, Uni of Toulouse I wrote a paper about Endogeneity in nonseparable midels: Application to Treatment Models where the Outcomes are durations. Very techniqual paper. There is some regularise of irregular estimators.

Han Hong wrote a paper about confident interval of identification regions "Inference on Identified parameter sets"

Francesca Molinari, Cornell, Partial identification of probability disribution with misclassified data. Very interesting. Use Manski idea. Maybe we can use something to solve identification problem in our exrension of RS(2003)

Sergio Firpo, Puc-Rio and UBC, Inequality treatment effacts. See the effect of treatment on the distribution of the targeted variable. Quite interestin to use this idea to measure the effect of introducing NMW (national minimum wage) on the wage distribution.

What is the relationship between selection (missing data) and random censoring model? (see a paper by tamer and Shakeeb Kahn (Rocheter) for this randomly censored regression)

Thomas Stoker (UCL and MIT) works on missing in X. See his website for more details.

Cheti Nicoletti from Essex ISER works on bounds and imputation. Her paper has some issues about using impuation when Z in the impution regression model is not the same as X in the structural model (X is a proper subset of Z). Since, in Skinner (2002), their X is included in Z, this may be an issue.

Does Bhattacharya's paper allow NMAR? but, although he said that his paper started off from Hirano, Imbens,....'s paper on Attrition with IPW, that paper does not allow NMAR??

Tarozzi Chen..'s paper on Semiparametric Efficiency on moment models are very interesting. But they need refreshment samples. Notice that, for IPW estimators, in order to obtain efficiency, they have to weight with both nonparametric and parametric estimated probabilities ( with correct specification). This may mean that results due to Wooldridge (using estimated prob leads to more efficient estimator) carry over to this. However, since all of these estimators attain semiparametric efficiency bound, we cannot say that using estimated probs is more efficient but we can see that they have to use estimated probs to attain such bounds.

Grant Hiller's paper is very interesting. It is about the foundation of estimation procedure.

August 18, 2005

From Meeting with Mark the day before


we might wanna use EDAGE for exp and YERQAL2 for education.
Why? think about a student who drops out of his Uni after his first year. He will be 19 assuming he has entered the Uni at 18. For him, EDAGE (age leaving full time education) = 19 but YERQAL2 (age obtained highest qualification, which should be A-levels in this case) = 18.


We will come back to look at the case where Z is not equal to X later. That is, we might add some extra variables into R's model (such as hourpay) apart of those from X.


We decided to have Z = X for the time being.

the coefficient of "londse" seems to be varried a lot with the change in assumption.

Combine sampling weights and IP weights and use these modified weights in the OLS estimation.

Write a report for Mark about the arguments for using IPW estimator in Wooldridge (2002, 2003).

Do some diagnostic tests.

Try to do Heckman, Newey's selection models.

Next step should be Smith or TLR.

Meet some time in September.

August 17, 2005

Running IPW

Run IPW and Unweighted estimation. For IPW, we use probit model to estimate the selection probabilities. But we do not use sampling weights (provided by the ONS) to combine with the estimated selection probabilities. We only use the inverse probabilities as weights in IPW estimation. Also we havent checked the validity of the probit estimation yet. We should run some diagnostics tests on this. Note that Vector Z in R's model includs all X's and HOURPAY (Derived variable). [Should it be in this model? R, conditional on Z, is independent of Y. But Y (Direct variable) should be correlated with this Z (Derived variable)

For Unweighted estimators, we run OLS (complete case analysis), OLS with income-sampling weights and OLS with normal-weights. Again, have not done the diagnostics tests.

EDAGE variable: there are people who are studying (EDAGE = 96) and had no education (EDUAGE = 97). We delete the group that are studying but quote 97 to be zero.

If EDAGE = 97 has few observations. Maybe we should delete this group as well.

August 16, 2005

Case Analysis concluded from Wooldridge (2002) and (2003)

SCENARY 1: If R depends only on X, the covarites in the model. That is, there is no Z variable and Assumption MAR holds as (R=1|Y,X) = (R=1|X).

Other settings are special cases of this scenary, so we will not discuss it.

SCENARY 2: If R depends only on X and X is completely recorded;

Double Robustness property of Weighted estimator makes it more likely to be used in this situation. This property means that we have to correctly specify either E(Y|X) or the model of R to get consistency. Why? because (in the linear regression context??) if the model of R is correct, Weighted consistenly estimate L(Y|X), the linear projection in the population anyway. If L(Y|X) = E(Y|X), then we have consistency even if the model of R is incorrect.

The drawback is when GCIME holds and (Y|X) = E(Y|X) as Unweighted estimator is consistent and efficient. But Unweighted is not robust since if (Y|X) is not equal to E(Y|X), then it is inconsistent immediately.

SCENARY 3: If R depends only on X, X is incompletely recorded and the feature of interest is correctly specified;
Use Weighted but also see the comment below.

SCENARY 4: If R depends only on X and X is incompletely recorded;
Weighted estimator will be inconsistent as we cannot estimate R. Unweighted analysis may be better as it allow R model to depend on missing X (Ignorability Assumption holds using missing X)

SCENARY 5: a feature of the conditional distribution is correctly specified, R depends only on X, X are completely recorded and some regularity condtions hold.
Both are consistent and we dont have to bother about getting the model of R correct. It is likely that we will use Unweighted estimator hoping GCIME to hold.

SCENARY 6: Same as SCENARY 5 + GCIME holds
We will use Unweighted as it is efficient estimator.

August 15, 2005

IPW in Wooldridge (2003)

To explore the following issues: when Z is not completely recorded and when the model of R is incorrectly specified.

It is shown here that estimated probabilities yield more efficient estimator (than that using the known ones) as long as the generalised version of information matrix equality holds in the first-step estimation.

Z can be missing when R=1 if the model of R is the conditional log-likelihood function for the cencoring values in the context of censored survival or duraiton analysis.

When the sampling is exogenous (or R depends only on X) and the expectation of the objective function is conditional on X (no misspecification), if we you use Weighted estimator then the selection model (R's model) is allowed to be misspecified.

This should work well in SCENARY 2. In this scenary, we fully record X and R depends on X. The incentive for using Unweighted is that if the feature of interest is correctly specified and GCIME holds than it will be consistent and efficiency. (Note that, in MLE, this requires correct specification in the mean function (for consistency) and the conditional density (for GCIME)!!!

However, using Weighted estimator allows misspecification in both the model for the feature of interest (pop mean or median functions) and the model for missing-data mechanism. This sounds very promissing indeed.

In term of efficiency, we note below that asym var of estimated and unestimated (known) IPW estimator are the same under exogenous sampling and correct specification. From the result about misspecification in selection model, we can relax this result a bit since we no longer require the first-step estimation to be MLE and the correct specification of its model. Now, we can allow for any regular estimation problem with conditional variable Z and allow the misspecification in the probability of selection model (as long as sampling is exogenous and, say, conditional median is correctly specified).

This result extends the cases where GCIME holds, that is Unweighted is more efficient than Weighted ( even though selection model in Weighted estimation is allowed to be misspecified)

August 14, 2005

Note about MAR, NMAR and the literature

Note that, from the setting of IPW estimators, we can see the problem of using notions like NMAR and MAR to convey our idea. Because the nature of Econometric is in our interest on the conditional model, we always have Y and X. In IPW, Unweighted estimator is consistent in a case where X is missing and R depends on X. Such case should be called as NMAR but R is not dependent on Y. Normally, when we use NMAR, we would like to imply that R depends on Y as well. In this IPW's case, the notation of NMAR is clearly misleading. So we should use something like endogenous missing instead.

Efficiency of Weight and Unweighted estimators

We will think about efficiency only if the subjects of comparison are consistent estimators. Thus, we have to compare both of them under the circumstance where they are consistent. That is, Theorem of the Weighted estimators holds with Z=X and the objective function is E[q(W,theta)|X]. Alternatively, the following scenary must be true.

SCENARY 5: a feature of the conditional distribution is correctly specified, R depends only on X, X are completely recorded and some regularity condtions hold.

Given this scenary, it can be show that, for Weighted estimator, asym variance is the same whether the missing probabilities are estimated or are known.

This result does not depend on whether or not a generalised conditional information matrix equality (GCIME) holds. Thus, if all conditions of this scenary are satisfied, the result holds even when there is heteroskedasticity in VAR of unknown form in the context of LS (see Wooldridge 2002)

At this point, we know that, according to this scenary, both Weighted and Unweighted estimators are consistent and it does not matter whether we estimate the seletion probabilities are use the known probabilities. (then the quesion remains: Should we weight or not weight?)

However, if GCIME also holds, we can show that the asymvariance of Unweighted estimator is smaller than that of Weighted estimator.

GCIME holds for conditional MLE if the conditional DENSITY is correctly specified with the true variance =1. (so both, say, population conditional mean and the density have to be correctly specified)

So, in SCENARY 5, it is likely that we will use Unweighted estimator.

In the SCENARY 2 below, we say that we will use Weighted estimator. However, if GCIME holds, we might wanna gamble with the model specification of mean function and use Unweighted estimator.

Consistency: Case Comparison


To comepare them, we have to set the situations where both of them can be used as an alternative of one another.

SCENARY 1: If R depends only on X, the covarites in the model. That is, there is no Z variable and Assumption MAR holds as (R=1|Y,X) = (R=1|X). This is the most common situation where both types of analysis can be used. Also, this is quite a realistic situation since, in practice, it will be difficult to find Z which is not a subset of X and makes the independence between R and (Y,X) holds.



(1) Do not require the correct specification of E(Y|X).

(2) Allow some other variables which are not in X to alsp affect R.


(1) the model of R has to be correct.

(2) X has to be completely recorded now so that we can estimate the model of R.

(3) only Y is allowed to be missing.



(1) Y and X can be jointly missing as the conditioning variables in the model of R (X or a subset of it) can be missing.

(2) If there are some variables which are important but incompletely recorded, they can be in X. In attrition, some covariates are missing in the later waves of study, such covariates can be included using this appraoch.


(1) correct specification of the feature of interest

AS CAN BE SEEN, to use which one of them, we have to consider case by case. NOTE that MAR and NMAR is not a good criteria to divide the literature in missing data (at least for Econometricians) anymore. From the above elaboration, the circumstance where X is missing but R depends on X fits with the setting of NMAR. As can be seen, there can be the case where Unweighted M-estimator is consistent.

SCENARY 2: If R depends only on X and X is completely recorded;

Then, we should use Weighted analysis as we can allow for mispecification and some other variables to affect the missing probability.

SCENARY 3: If R depends only on X, X is incompletely recorded and the feature of interest is correctly specified;

Then, we should use Unweighted analysis because the incomplete X can be in the model of R. (As we know for sure that these incomplete X are matter for the R's model and we do not have to worry about the miscpecification.) Thus, Unweighted analysis yields consistency under weaker assumption (= more variables in the R's model to ensure the independency between Y and R)

However, there could be some restriction such that we cannot use these incomplete variables anyway (Wooldridge (2002) gives an example where the structural of the conditional expectation model refrains us from using incomplete variables). So, Weighted might be better.

SCENARY 4: If R depends only on X and X is incompletely recorded;

Now, we do not know whether, say, E(Y|X) is correctly specified or not. This case is also unclear. We might want to use Unweighted analysis and gamble about the model specification. Or, if R is not significantly dependent on those incomplete X, we may want to use Weighted analysis.

Weighted and Unweighted: Consistency

Note that, in all of this discussion, R has to be independent of Y conditional on some variables anyway. ( For Weighted analysis this is Assumption MAR but, in Unweighted analysis, it is not MAR because the conditional variables in R model can be missing)



(1) Do not require the correct specification of the feature of Y|X ( conditional mean, conditional median)

(2) Allow other variables (apart from those in X) in Z to affect R.

(3) Y and X can be jointly missing as long as Z is fully recorded and as Assumption MAR (using Z) is satisfied.


(1) Model of R must be correctly specified

(2) Z must be completely recorded.

(3) Response probability has to be positive (meaning that we cannot exclude a subsection of the population in the sampling process) ( This may imply that wage equation example is not valid here because people who dont work are excluded completely)



(1) Missing variables (except Y) can be allowed into the model of R,i.e., missing mechanism. This is because we do not have to estimate the response probability. Thus, Ignorability Assumption ( this is not MAR) of R's model tends to be weaker than that of Weighted analysis in general ( since more variables can be conditioned upon to make the independent between Y and R more plausible.)

(2) Y and X can be jointly missing even when we dont have any variable as Z.

(3) Response probability can be zero for some subset of population


(1) require correct specification of the conditional mean, conditional median or conditional distribution.

August 12, 2005

IPW continues

About unweight-M-estimator;
(1) Should note first that the weights we are talking here are probabilitiy weights, not the weights that are provided by the survey conductors.

(2) the result of Assumptions 5.1 and 5.2 in Wooldridge (2002) coincides with that from the missing-data workshop in London. That is, the unweight M-estimator (complete case analysis) is valid ( unbiased and consistent) when R depends on X or is independent of both Y and X. (X here must be observable (not a latent variable like ability) but is allowed to be missing when R=0)

Missing-data workshop ( the introductory one) maintains that if R depends on X, the adjusted mean of Y for X is unbiased. ( adjusted mean is the coefficient of that X in a regression model) Also, if R depends on another random variable, say, S where S is independent of Y and X, then the result is also unbiased. Note that, in their example, S is uniformly distributed in (0,1). Can S be normally distributed???

Note that, from this course, the complete-case analysis is biased if R depends on Y or on both Y and X.

Wooldridge (2002) adds another condition to the conditions mentioned above. He shows that correct specification of the distribution of Y|X or that of mean function is also required.

August 2005

Mo Tu We Th Fr Sa Su
Jul |  Today  | Sep
1 2 3 4 5 6 7
8 9 10 11 12 13 14
15 16 17 18 19 20 21
22 23 24 25 26 27 28
29 30 31            

Search this blog

Favourite blogs


Most recent comments

  • Hi im yo man my intenet namee imm 43 i woork in the toilet withh laptops this coool webbsete by yo man on this entry
  • this suck i'm out of here by katie on this entry
  • this is a stupid website im 89 years old in two weeks it's my birthday how old would i be this is an… by glf on this entry
  • What r u talking about? I think u must be confused. No No No ur major is not econometrics. U have b… by MMP on this entry
  • This blog is temporary out of service because the owner is still busy watching TV (no matter what la… by MMP on this entry

Blog archive

Not signed in
Sign in

Powered by BlogBuilder