...or why I am not a fan of word clouds
One of the questions that occasionally we get asked is how to present data from unstructured free text. And sometimes then we get asked, "Do you do a word cloud?"
To be honest, we get asked this less and less these days, and I think as people get more used to data visualisation we could see this end completely.
It is well known, to those who know me well, that I am not and never have been a fan of word clouds, but why? and also why do people want to use them?
Word Clouds are "the Mullets of the Internet"
I do love this, I am biased I know, but I think it is a fab description from 2005 of the faddish adoption of word clouds (tag clouds) back in the day.
What are word clouds?
Word clouds are simply a pictorial representation of words that are taken from unstructured (or structured) text grouped together, and typically larger or smaller depending on the number of times the word is used.
Whats my issue with word clouds?
If we think about the purpose of data visualisation.
Data visualisation takes complex data and turns this into visuals that allow the user to see the information they need quickly, to spot trends, to see data represented in different ways and to tell a story about the data.
They don''t really tell a story
Telling a story is an important part of data visualisation and I am afraid word clouds simply do not do this, they strip out the sentences, the narrative and try to replace this rich (unstructured) data with "tags" based on how often a single word is said.
I will demonstrate with a bad example;
Macbeth, a classic:
Methought I heard a voice cry, ‘Sleep no more! Macbeth does murder sleep: the innocent sleep, Sleep that knits up the ravelled sleeve of care, The death of each day''s life, sore labor''s bath, Balm of hurt minds, great nature''s second course, Chief nourisher in life''s feast.
The book is complex, with lots going on and many different viewpoints and narratives, understanding this requires that you read the book or at least a detailed synopsis of the major themes.
Or could you just read the word cloud...
Are we surprised that the main word used in Macbeth, a story about Macbeth is "Macbeth"?
They provide no context to the sentiment in the text
Words are just displayed with no background, no context of how the person felt at the time.
We will often use the same words when we are asked a question that are in the question, if you ask someone about Teamwork people may say "we are great at teamwork" or "teamwork could be improved" or "teamwork is ok"
So in the word cloud, we will see "Teamwork" dominating but all the other words will be based on the individual - great, awesome, fab, good, ok, done well and amazing all suggest good things, you just won''t get that in a word cloud.
Their design means you see something that potentially is not there
Some words are bigger than other words because they contain more letters - sounds obvious right?
If a short word has a large count and a long word a medium-sized count you run the risk that you lose the scale between the words simply because the word is bigger!
Look at how "CadburysDairyMilk" is so much larger than "Milka" or "Trident" below - are you confident that this is because it is statistically larger or just because it has more letters.
They are not data science - you cannot see the facts
Here is a word cloud representing towns in the UK by population size, what does this tell us?
We can see that Liverpool and Sheffield are the largest here, but we instantly lose any real frames of reference. - How much larger is the population of Liverpool than Sheffield? - What about Hull? is it the same as Derby?
Every time I see a word cloud presented as insight, I die a little inside
Quote from Jacob Harris from the NY Times
So, why do people use word clouds?
I think it is because of the following:
- Unstructured text takes a long time to read so people want a fast solution
- Word clouds were everywhere once and therefore well known and associated with a fast solution
- Most people do not think about data visualisation issues
- We all have biases that things like word clouds play on
Let''s look at those biases and how we respond to word clouds:
I do not have enough meaning
We have already said that word clouds are sparse when it comes to actual meaning so what happens - "We tend to find stories and patterns even when looking at sparse data"
Some of the biases at play:
Clustering Illusion - The clustering illusion is the tendency to erroneously consider the inevitable "streaks" or "clusters" arising in small samples from random distributions to be non-random
Illusory correlation - In psychology, illusory correlation is the phenomenon of perceiving a relationship between variables even when no such relationship exists
I have too much information
Free text by it''s nature is a lot of information so what happens? - "Visually striking or bizarre things stick out more than non-bizarre, less striking things"
Word clouds are designed to be visually striking Some of the biases at play: Von Restorff effect - The Von Restorff effect, also known as the "isolation effect", predicts that when multiple homogeneous stimuli are presented, the stimulus that differs from the rest is more likely to be remembered
I need to act fast
We have already said that in business we default to needing to act quickly, reading lots of free text is time-consuming. This is one of the reasons word clouds are chosen because - "We favour simple-looking options over complex ones"
So, what is better than word clouds?
Hopefully, by now you are in agreement that we do not need a word cloud in our life if we are looking for insights into any large unstructured free text, So, if not word clouds then what?
In our experience, while there are software tools that can help you make sense of unstructured qualitative data, we''ve yet to find anything that works just like that.
As part of the attraction of word clouds is their convenience, it''s also true that even the best tools provide more accurate insights when you put some effort into coding the text.
Check out our other blog topics.
The People Experience Hub
A tech-led, human approach to measuring and improving people experience at work, helping you understand your people better.