Are online personality tests accurate?

Online personality tests can be accurate, but it depends on the test. Being online isn't the problem: research suggests web questionnaires can perform about as well as paper ones. What matters is whether the questions have been tested, and many online quizzes have not been.

So the useful question is not "is it online?", but "what evidence sits behind this particular test?". Here is how to judge one, including ours.

Is the internet the problem?

Psychologists once worried about this a great deal. Samuel Gosling and colleagues tested six common fears about internet questionnaires, comparing a sample of 361,703 people with 510 published traditional samples. They concluded that internet samples were relatively diverse, that findings generalised across formats, and that results were consistent with traditional methods. Nonserious and repeat responders did not adversely affect the findings.

John Johnson looked at the same kind of worry in a different way, examining nearly 24,000 responses to an online inventory. Web participants were about as consistent as people completing a paper version, with an estimated 1% of responses invalid for reasons such as inattention. He did find more skipped questions and some repeat participants, so data checks are sensible.

Tom Buchanan, Johnson and Lewis Goldberg also put a short five-factor inventory online, built from the International Personality Item Pool, and tested it on 2,448 people. After revising it, they reported acceptable reliability and preliminary evidence of validity, with similar results when they replicated it using a different recruiting strategy. So an online format can work when the groundwork is done.

All of that speaks to well-made tests, and it doesn't vouch for a five-question quiz about which pasta you are.

Three things accuracy can mean

When someone says a test is "accurate", they could mean a few different things.

• Reliability: do you get a similar result if you take it again soon after?
• Validity: does it measure what it says it measures?
• Usefulness: do the scores predict anything in real life?

A test can do well on one and poorly on another. For trait measures, there is good news on the third point. Christopher Soto attempted to replicate 78 published links between Big Five traits and life outcomes. 87% of the attempts succeeded in the expected direction, and effects were typically 77% as strong as the originals. That suggests the Big Five is a reasonably accurate map, with effects that may be a bit smaller than first reported.

Why feeling accurate isn't enough

A result that feels right doesn't prove much. In a well-known classroom demonstration published in 1949, Bertram Forer gave students what they thought were individual personality descriptions. In fact, all of them got the same one. Students still rated it as a good fit.

Later known as the Barnum or Forer effect, this is why vague statements like "you can be sociable, but sometimes you need time alone" feel so personal. They fit nearly everyone.

So don't judge a test by how nice or how true its report feels. Check what is behind it.

How to judge any online test

Here are some questions you can ask of any test, in about two minutes.

1. Does it say where its questions came from? Look for a named source.
2. Does it say whether it has been validated? A good test tells you plainly, including when it hasn't been.
3. Does it make claims you can check? Look for real citations with links.
4. Does it give you scales or only a box? Most traits vary by degree. Our comparison of personality tests explains why this matters.
5. Could the result apply to anyone? If everything is flattering and vague, be wary.
6. Is it honest about its limits? No personality test should claim to predict your future or give you a clinical answer.

Where Selfcarta sits

We think you should hold us to the same list. Here is where we stand.

Core Personality uses items from the International Personality Item Pool, a public-domain resource described by Goldberg and colleagues. Specifically, we use the Big-Five Factor Markers, which build on Goldberg's 1992 work. These items have a published research history. That does not mean Selfcarta's own presentation of them has been independently validated as a product, and we make no accuracy claim for it beyond that.

All our other Expeditions use Selfcarta's own questions and are not yet independently validated. They draw on published theory, such as attachment dimensions and the RIASEC career types, but the questions are new. We don't publish accuracy figures for them because we don't have any. We show that note on the results too. Treat those results as prompts for reflection.

You can read more about our approach on the science page.

Self-report has its own limits

Even a good questionnaire only captures how you see yourself. As we cover in how accurately people rate themselves, that view is valuable and incomplete. That is why getting ratings from other people adds something a solo quiz can't. Second Sight invites people who know you to rate you, with raters kept anonymous and shown as averages with at least two others, or as a named one-to-one comparison in the Partner circle.

Try one and see

The best test of a test is whether it gives you something true and useful to work with. Take the Core Personality Expedition, which is free with no sign-up. Then read the results with a curious mind and a little healthy doubt.