EFA vs CFA: when to use exploratory or confirmatory factor analysis

What is the difference between EFA and CFA?

Both methods start from one belief, that something unseen like job satisfaction pushes people to answer a group of items the same way. The difference is who decides the structure. In EFA any item can load on any factor, and you interpret the pattern the software finds. In CFA you draw the model first. You say these five items measure Factor A and these four measure Factor B. The software then tells you how far your data departs from it.

There is no hypothesis in EFA, so EFA alone can’t validate a scale. Costello and Osborne (2005) say directly that EFA is not meant for testing hypotheses or theories, and once the instrument is developed you should move to CFA. Writing “validate the instrument using EFA” in the synopsis is common, and that line itself invites questions later.

Should you run EFA, CFA or both on your questionnaire?

This mostly depends on the source of your items.

• Established scales used as published, in a similar population. Normally CFA alone is enough.
• You have adapted or shortened the scales, translated them into Hindi or Tamil, or given them to a very different population (for example a Western burnout scale used with ASHA workers). Run EFA first, and then CFA on separate data.
• You wrote the items yourself. Then also it is EFA first and CFA on separate data. No shortcut.

The mistake: EFA and CFA on the same respondents

I see this error in drafts more than any other. A scholar runs EFA on all 400 responses and removes the badly behaving items. Then she runs CFA on the same 400 and reports excellent fit as confirmation.

But that good fit proves little, because the EFA already shaped the model around the quirks of those same 400 people, so naturally they fit it.

Split the data instead. Divide the respondents into two halves at random. In SPSS this is Data, Select Cases, Random sample of cases. Run EFA on one half and finalise the structure. Then run CFA on the other half. If one half will be too small, collect a fresh sample for CFA. The “10 respondents per item” rule which people quote is only rough. Why so, our sample size guide explains.

An illustrative case: a scholar writes 28 items on workload of government-school teachers and gets 560 responses. She divides them into 280 and 280. On first half, principal axis factoring with promax points to four factors. She reads the wording again and drops two cross-loading items and one weak item. Then she tests the 25-item model in AMOS on the second half, and every number she reports comes from that half only.

How do you run EFA in SPSS, step by step?

SPSS will happily run EFA on its defaults, and that is where the trouble starts. You get principal components with eigenvalues over 1, and most theses then add varimax to it. Costello and Osborne found this mix in well over half the studies they surveyed, and argued it is not the right choice.

1. 1

   Open the dialog

   Go to Analyze, Dimension Reduction, Factor. Move in the items of only those scales you are checking.

2. 2

   Descriptives

   Tick KMO and Bartlett’s test of sphericity. Tick Anti-image also if you want to see sampling adequacy for each item.

3. 3

   Extraction

   Change Principal components to Principal axis factoring. For data that is close to normal, Maximum likelihood is also fine. Tick Scree plot.

4. 4

   Rotation

   Select Promax or Direct Oblimin. Kappa and delta can stay at the default values.

5. 5

   Options

   Tick Sort by size and suppress coefficients below 0.30. Then the matrix is easy to read on the screen.

6. 6

   Rerun

   Fix the number of factors at what you predicted. Run again with one less and one more, and compare the three pattern matrices.

KMO and Bartlett’s test

The Kaiser-Meyer-Olkin (KMO) measure checks if the partial correlations among your items are small. When items share factors these partial correlations come out small, which is what you want. The textbook labels are from Henry Kaiser. A value in the 0.90s is marvellous, the 0.80s meritorious and the 0.70s middling. In the 0.60s it is mediocre, in the 0.50s miserable, and below 0.50 it is unacceptable. (Kaiser first gave this scale in a 1974 Psychometrika paper, which was on a different index. In the same year Kaiser and Rice set 0.50 as the floor for sampling adequacy.) Open the anti-image matrix also, and any item under 0.50 on its diagonal, look at that first.

Bartlett’s test is weaker. It only sees if the items correlate at all, and with a few hundred respondents it comes significant nearly every time. Report it anyway, it just doesn’t prove much.

Principal axis factoring or principal components?

Principal components analysis (PCA) is strictly a method for data reduction, and it is not factor analysis. PCA analyses the total variance of each item including the error, while factor analysis uses only the variance that items share. In Costello and Osborne’s comparison, PCA gave inflated loadings and inflated variance explained. Fabrigar and colleagues (1999) recommended maximum likelihood when the data are reasonably normal. When normality is badly violated, which happens many times with Likert items, they advised principal axis factoring. If your guide is firm about PCA, run it, but in the thesis write “principal components” and avoid calling those components latent factors.

Varimax or promax?

Varimax forces the factors to stay uncorrelated, while promax and direct oblimin allow them to correlate. In one questionnaire the factors are almost always correlated, so an oblique rotation is closer to reality. If they aren’t, the oblique answer is nearly the same as varimax anyway. Read loadings from the pattern matrix and report the factor correlation matrix.

How many factors should you keep, and which items should go?

Eigenvalue greater than 1 is what SPSS uses by default. Costello and Osborne report broad agreement that it is among the least accurate methods. In their own simulations it kept too many factors in 36% of samples.

The scree plot (Cattell, 1966) needs your judgement. Keep the factors above the point where the curve becomes flat.

Parallel analysis (Horn, 1965) keeps a factor only if its eigenvalue is higher than what random data of the same size gives. Most methodologists prefer it. The Factor dialog in SPSS hasn’t usually had it, so scholars use Brian O’Connor’s free SPSS syntax, fa.parallel from the psych package in R, or JASP. My suggestion is to report parallel analysis with the scree plot, and if they disagree, go by theory.

Loadings, cross-loadings and deleting items

Costello and Osborne follow Tabachnick and Fidell here. They take 0.32 as the minimum loading, and an item that loads 0.32 or more on two factors is called a cross-loading. They also point out that a factor with less than three items is weak and unstable. In many theses the cut-off is 0.40 instead. State your cut-off and follow the same one throughout.

Delete one item at a time and run the analysis again, because all the loadings shift each time. Before deleting an item, read it once. Mostly the cross-loaders turn out to be double-barrelled or reverse-coded items, and the questionnaire design guide discuss about such problems. Keep a log of every deletion.

How do you run CFA, and which fit indices should you report?

For CFA you need a covariance-based SEM program. AMOS lets you draw the model, or there is the free cfa() function of lavaan in R.

Most theses cite the cut-offs of Hu and Bentler (1999). From their simulations they suggested values close to 0.95 for CFI and TLI, 0.06 for RMSEA and 0.08 for SRMR. They wanted these read together as a set, not one by one.

Treat these numbers as guidelines and not pass marks. When fit is poor, don’t start chasing the modification indices. Adding correlated errors one by one till CFI crosses 0.90 is exactly the thing examiners are now asking about. Add one only when you can explain it, for example two items with almost identical wording.

What about SmartPLS?

In PLS-SEM each construct is a weighted composite of its items, so its measurement model is not CFA in the covariance sense. Hair, Howard and Nitzl (2020) call it confirmatory composite analysis. So if you used the PLS algorithm in SmartPLS, write “measurement model assessment” in place of CFA, and don’t report CFI or RMSEA from it. For choosing between them, see AMOS vs SmartPLS.

Convergent and discriminant validity

After fit comes composite reliability and AVE (floor of 0.50, from Fornell and Larcker, 1981). Then discriminant validity, which these days is usually checked with HTMT (Henseler, Ringle and Sarstedt, 2015). Thresholds and fixes are in the reliability and validity guide.

What should you report for EFA and CFA in your thesis?

Put the measurement results before any hypothesis testing, because if the measurement looks weak the examiner will not believe the path results coming after it.

• The reason you ran EFA, CFA or both, linked to where your scales came from.
• Which sample went into each analysis, and how you divided it.
• For EFA, give KMO, Bartlett’s test (chi-square, df, p), the extraction and rotation method, and how you decided the number of factors.
• For EFA, also give the full pattern matrix, the communalities and the factor correlations.
• Each item you deleted, at which stage, and the reason for deleting it.
• For CFA, give the software, estimator, standardised loadings, chi-square with df, CFI, TLI, RMSEA with its interval and SRMR. Any change you made to the model also needs its reason written there.
• CR, AVE and discriminant validity for every construct, all taken from the CFA sample.

For EFA, put items in rows grouped by factor, with pattern loadings and communalities, and eigenvalues and KMO at the bottom. For CFA make three tables, for loadings (with CR and AVE), fit and discriminant validity. The full raw output, put that in appendix only.

Once the measurement model holds, go to your structural questions. For indirect or conditional effects see mediation and moderation, and for other analyses the statistical test guide will help.