New evidence that bullet-points don’t work
October 7, 2009 by Olivia Mitchell
Welcome to this blog – my aim is to make a difference to the success of your presentations. If you’re new here, you may want to subscribe to my RSS feed. Thanks for visiting! Olivia
At last, we have some scientifically rigorous evidence to show that slides full of bullet-points don’t work.
The research is the work of Chris Atherton, a cognitive psychologist. Chris recently delivered a presentation at the Technical Communication UK Conference and has put up her slides on slideshare. There’s been a tremendous amount of interest in them, but as they were designed to complement Chris’s talk – they only tell half the story.
In this post I’ll explain the findings of Chris’s research. I’ve written the post based on Chris’s slides and asked Chris to comment on various aspects. Chris has also reviewed this post to make sure I’ve got all the science right.
The research
Chris tested the effects of using two different types of PowerPoint slides in a presentation. Students were randomly assigned to two groups. One group attended a presentation with traditional bullet-point slides (with the occasional diagram) and the second group attended a presentation with what Chris calls “sparse slides”, which contained the same diagrams, but minimized the amount of text, and broke up the information over several different slides. Both presentations were accompanied by the same spoken narrative.
Here are samples of the slides used:
A. Traditional bullet point with the occasional diagram
B. Sparse slides
Chris tested the students’ learning in two ways – multiple choice questions and short essay answers. There was no significant difference between the groups on the multiple choice questions. Chris comments:
This is most likely because it’s not very hard to pick out the correct answer from among distractors when you have only recently been exposed to the material and your memory of it is quite fresh.
Before marking the short essay answers, Chris worked with two independent people to identify the themes of information in the presentation. They identified around 30 themes by consensus. The short essay answers were then marked by counting how many of those themes the students wrote about.
As you can see the students who were in the presentation with the sparse slides did much better than those who saw traditional slides.
Theory behind the research
There are a number of theories which can be used to explain these results (if you’re not interested in the theories, scroll down to the next section “What does this mean for your presentations?”):
1. The limitations of working memory
Even the students who did well in recalling themes, remembered only 6-7 themes out of a possible 30. Chris suggests this is due to the limitations of our working memory. Recent work (Cowan 2001) has estimated working memory capacity to be around 4 chunks of information:
2. Two processing pathways
The brain has two major pathways for processing information.
The auditory cortex and the areas around it are involved in processing language – both spoken and written.
When a presenter uses bullet-point slides, they’re not using both pathways as effectively as they could. The audience member has to read the words on the slide and listen to the presenter at the same time, leading to overloading of the language areas whilst leaving the visual cortex with very little to do:
Chris notes:
The visual cortex is involved in reading the words on the screen - it works on the lines and features to assemble the words that are being read, but it’s not really being used to the full, since there’s usually little color or texture information.
3. Cognitive load
The theory of cognitive load was developed by John Sweller. Cognitive load is the amount of work required to understand or learn something. There are two main types:
- Intrinsic cognitive load – how inherently difficult something is.
- Extraneous cognitive load – extra work imposed by the thinking/learning environment.
Chris suggests that the sparse slides may minimize extraneous cognitive load by creating fewer competing demands on attention — that is, because we don’t need to spend very long processing the visual elements, we have more attention for what the speaker is saying. She adds:
Having anything on a screen invites people to look at it, the same way their gaze would keep returning to a TV screen in a pub. Since you can’t control the audience’s visual attention, it’s all about controlling what visual information you make available at any given moment, and minimising what is there so it’s not distracting from the spoken narrative, while also ensuring that it is congruent with what you are actually saying.
4. Better encoding of information
Encoding is the process of putting something into your memory. McDaniel and colleagues have shown that a little more effort at the encoding stage can be beneficial to learning. Chris suggests:
Sparse visual cues could lead to better encoding of information — that is, having to work a little bit harder to integrate the speaker’s narrative with the pictures might actually improve our storage of the information (obviously this is only true up to a point; having to work too hard at integrating the two could actually be counterproductive, effectively producing a situation with high extraneous cognitive load).
What does this mean for your presentations?
1. Don’t say too much
Limit what you cover in a presentation. Your audience has limited capacity to take it in.
2. Split the load
Take advantage of the brain’s two pathways. Design your slides so that they can be processed quickly by the visual cortex, allowing the language areas to focus on what you’re saying. This means using more pictures and as few words as you think you can get away with.
3. Get rid of visual clutter on your slides
Do what you can to minimize the extraneous cognitive load on your audience. For example:
- Only put on your slides things you want the audience to focus on.
- Split information between slides rather than having it all on one slide, so that you can direct the audience’s attention where you want it.
4. Make your audience work
There’s some evidence that making your audience work a little to understand your point will make your point stick better. A big caveat to this is that obviously you mustn’t make it so hard that they don’t get your point at all. Some ways of doing this are to:
- Show a picture that the audience has difficulty relating to what you’re saying. Either ask them to guess the relationship, or explain the relationship to them.
- Show them the axes of a graph, and ask your audience to guess the way the data goes (give enough clues that they’re fairly likely to get it right – without making it too easy).
Thank you Chris
It’s great to have some solid experimental evidence on the use of slides in a live presentation, to back-up what so many presentation authors, trainers and coaches have been saying. I’m also deeply indebted to Chris for her help with this post. I’m looking forward to whatever research Chris does next.
Go well with your next presentation. If you found this post useful, subscribe to my RSS feed.
Related posts:
- Powerpoint custom animation experiment – check out the animation for yourself
- New research questions the benefits of custom animation in PowerPoint
- Here’s a quick way to make over a bullet-point slide
- New scientific evidence for banning bullets from your PowerPoint slides
- Robbery at Bullet-Point
Trackback URL for this post: http://www.speakingaboutpresenting.com/design/new-evidence-bullet-points/trackback/
Olivia – This is terrific. Thank you! I am a speech coach and have been waiting a long time for this kind of research. Thank you for posting. I am adding your blog to my reading list and invite you to add mine!
sarahgershman.blogspot.com
Thanks,
Sarah
Hi Sarah
Thanks for your comments. I’ve subscribed to your blog. Olivia
Wow, great information and research that all presenters need to know. Thanks for bringing this to my attention.
This speaks volumes to adding some visual element to your presentation and then “making the audience work a little” as you say, so the point sticks. I like to call tha process engaging the audience.
I too am looking forward to Chris’ next research as well as how you continue to explain this information so we can apply it.
Thanks Olivia.
Thanks Jeff
Thanks for sharing this research. So much of this applies to e-learning courses too. I think many of us applied some of these approaches to our online course design, but now we have the research to back it up.
Hi Jeff
In the online learning area there’s a lot of research especially by Richard Mayer. In the past, in the presenting world we’ve relied on this research. So yes there is definitely overlap between the two areas of research.
Olivia
I say we make t-shirts that say “Bullet-Points Don’t Work!” with a bullet-point laying on a bed.
Thank you for this data–now that there are numbers, maybe the presenters will finally believe the things I’ve been telling them as a designer for literally decades.
I agree with the above comments, and love this research POV! Anyone who has been in one of my presentation or facilitation skills classes knows my perspective on this topic: Slides are not the main event. Slides are there to enhance and support the understanding of key messages, but we as presenters are the key visual aid to driving that understanding. If the slide upstages you, you’re doing something wrong.
If I’m reading the research correctly, it looks like the primary problem arises when asking the brain to do two things at once by talking as the presenter while showing a bulleted slide. One way I’ve found to help with this problem, in addition to slimming down slides to the bare minimum (They’re not speaker’s notes!) is to preview a slide I’m about to show, and give the audience a focused viewing task for any bulleted slide. For example, if I’m about to show a traditional ‘Learning Objectives’ slide, I’ll tell the group, “You’re about to see a slide of bullets showing the objectives of this session. Once I show it to you, I’d like you to skim through it and let me know what looks most interesting to you on that slide.” Click to reveal the slide. Then, I provide them with time to read the slide (while I read it silently to make sure I’m allowing enough time), process it, and share whatever they find most interesting. Amazingly, using this approach, you can have a great dialogue about something as mundane as learning objectives, providing the presenter with input as to how to adapt and focus the remainder of the session. It’s the same bulleted slide used differently to better results.
Again, I’m no fan of bullets. However, if you have to have them there for some reason, using this ‘previewing and focused viewing’ approach can help to increase engagement and retention.
Cheers
Ken De Loreto
Hi Ken
That sounds like a great way of using a bullet-point slide. I can see that that would really involve the audience in actively processing the information.
Olivia
Hi Ken
I really like this suggestion — even in my own teaching, where I am striving to eradicate slide text and bullet lists wherever I can, sometimes I still need to show some text on the slide. I love your approach of reaching out to the audience and effectively inviting them to do some metacognition — breaking the fourth wall of presenting/teaching, if you like
Kind regards, and thank you for your interest in my research!
Chris
Olivia
Another excellent, informative post. Thanks for publicising this research – I’ll be following up on Chris’s work – interestigly she’s based 20 miles up the road from me.
I think there are some difficulties translating the “less is more” approach to slides into practice for highly technical presentations – more work is needed on this.
A bigger problem is convicing clients. They are so used to the old style slides crammed full of words that it can be difficult to convince the that the “less is more” approach is more effective. I’ve had to face this recently.
Keep up the good work
Hi Mike
I’d be interested in seeing the technical slides that are challenging for implementing the “less is more” approach.
It may be a case of using a slow build of the information on the slide.
Do send me a slide at olivia@effectivespeaking.co.nz if you’d like to discuss it further.
Olivia
Very helpful and interesting post. Thanks Olivia!
In a discussion about your post at work a colleague wrote this in response: “I want to make a point (that may seem like splitting hairs, but the cognitive psychologist in me insists) and that is this: I don’t think that Chris Atherton said anything about the efficacy of bullet points per se, rather that in his test condition, the words and pictures interfered with one another, that is where the pictures and words said the same thing, they caused cognitive competition that got in the way of encoding the information. Also, that a cluttered slide makes it hard for the brain to figure out where to look. Simplicity, but done in a way that people have to think about what’s being shown or said on the slide (increase extraneous cognitive load), as opposed to being mentally confused by the same info entering two channels, was the ultimate message, not that bullets or words don’t work. (I hate lists of bullet points myself, and don’t want us to throw the baby out with the bathwater here.) I love the article on presenting by Steve Jobs [http://bit.ly/1TVRbx], which adds something to our understanding of this whole issue, I think: “You see, neuroscientists are finding that information is more effectively recalled when the ideas are delivered as text and pictures instead of text by itself.””
Hi Simon
Your colleague has a point. I’m using the term “bullet-points” as an umbrella term for excess text and clutter on a slide.
BTW the article you referenced is by Communications author Carmine Gallo about Steve Jobs.
Olivia
Hi Simon,
Your colleague’s point about bulleted text is interesting: while we weren’t explicitly testing text vs. pictures, there was relatively little text on the non-traditional slides, and certainly less text present on any given slide than in the traditional condition. So I think one could infer from this that bulleted text is less useful than short, simple text, but it depends what you consider to be “bulleted text”, I guess.
Cognitive psychology theory suggests that information goes in more easily when we don’t have to process words with our ears and our eyes at the same time … there are times when I think it’s advantageous to show a bullet-list (Ken De Loreto gives a nice example in the comments above) but my problem with slideware is that it routinely suggests bullet lists, far often than they might actually be genuinely useful.
Kind regards, and thanks so much for your interest in my research!
Chris
[effusive praise] Have I told you lately how much I appreciate your blog? This is fantastic information. I’ll have to come back to reread it because I think I overloaded my working memory with RSS feeds. [/effusive praise]
Seriously, this bears more study. Thank you for sharing this.
Hey, thank you Todd [blush]. Olivia
Hi, Olivia –
Thanks for the great post concerning Chris’ research. It of course confirms and gives a good solid grounding for what many of us have been arguing for years. To Mike’s point about “highly technical” slides, I would respond that I’ve worked with many highly technical people and their slides and always found ways to simplify without distorting or losing information.
Hi Nick
Yes, I tend to agree with your point about highly technical slides. That’s why I’d love to see one of Mike’s slides!
Olivia
Great post! The lack of a strong editing process is usually the culprit of overloaded presentations. We struggle to cut “good” content to focus on the “best” elements. Thanks for showing how keeping all those good points can dilute overall learning.
Hi Todd
Nice point about editing. We should focus on the best. Olivia
Ah, the often skipped task of editing! Agreed. I’ll add to that the often skipped task of rehearsing. When working with individuals or teams responsible for creating customer-facing presentations (for example for sales ‘pitches’) two of my favorite tips are:
1. Edit: Create what you consider to be the ‘perfect’ presentation, and then cut it in half.
2. Rehearse: If you’re working on the slides the day before or of the presentation, you’re focusing on the wrong things, and probably avoiding the larger, more daunting task of preparing to engage the audience.
Bottom line for me: Slides are not the presentation. We are. However, it seems to be easier, and almost seductive, for presenters to dump everything they need/want to share onto slides. Their thinking is, “If it’s ‘in the presentation’, then the presentation must be good.” Oh, the siren’s song of PowerPoint. A metaphorical crashing onto the rocks happens as a result of this thinking. The presenters (or sailors to stay with the metaphor) never even notice that they’re creating dense, murky, and lifeless slides that will prove cumbersome when live with an audience. By not editing down the slides, and not rehearsing how to engage the audience with the edited slides prior to going live, many presenters don’t discover the traps of certain slides until they’re live with an audience. Not the time to realize that, particularly in a selling (or sailing) situation.
Cheers,
Ken
Hi Ken
Agree with the importance of editing – and love your sailor analogy!
Olivia
Thanks for this great post, Olivia (and Chris). I’ve referenced it on my blog and am sending more readers your way!
http://coachlisab.blogspot.com/2009/10/cognitive-load-and-overload.html
Thanks Lisa – and love the photo on your post. Olivia
Thankyou Olivia for adding the meat to the excellent bones of Chris’s slides! I’ve also just read through all the comments, so many good points and examples. I too have had (and continue to have) some difficulties with some clients. Specifically these are clients who have asked me to design learning resources for face-to-face workshops. I have no trouble introducing learning activities, preparing background resource materials BUT so many clients want EVERYTHING on Powerpoint – I mean everything!! I’ve been doing some work trying to explain the learning benefits of altering their use of Powerpoint (facilitators who read their text-dense slides to learners), so thanks to you and Chris for supplying me (us all) with some research that may help make the task more successful!
Hi Michael
I can totally relate to the issue of working with clients who are wedded to their bullet points.
It’s fascinating that when people are in the audience they hate bullet-points, but as soon as it’s their turn to present “give me my bullet-points”. Presenting is scary to most people and bullet-points represent a level of safety, a security blanket. For more on this see http://www.speakingaboutpresenting.com/design/persuade-ditch-bullets/
So when I’m working with clients, we look at finding alternative safety nets. On our courses, we cover preparing notes before we even get to the PowerPoint. Knowing that they’ve going to have notes – makes a huge difference to their receptivity to the PowerPoint message. (When we used to do notes afterwards – we had a lot more resistance – nowadays we hardly have any).
There are different challenges when you’re preparing Powerpoint slides which may be used by a number of different presenters – as in your case. But it’s still about making sure that you provide an alternative – it could be using the notes pane of Powerpoint or providing a separate facilitator guide.
Olivia
Hi Olivia,
I think you raise a really interesting point here – that people are accustomed to starting with the slides, rather than ending them. Great to know you are redressing the balance!
Also thanks to Michael for his very charming and enthusiastic comments
Chris