Our Blogs  


Tools for too much text

By Wendy Elford  ·  September 13, 2010  ·  Data visualisation

One of the hazards of doing research with a broad focus is getting lost in all the information out there. One of the pleasures is just wandering around the internet looking for stuff. Data overload is one of the main reasons I was so grateful to come across the Cynefin framework in late 2006. Apart from this leading later to the opportunity to use SenseMaker™ as research tool, conversations and stories told in doing the accreditation course in Brisbane also led to some exposure to other ways researchers, governments and organisations were using to find important information in big volumes of data.

In 2008 I had my own close encounter with the detailed analysis of text using transcription and NVivo. Learning first-hand how intense and time consuming this approach is drove me to not only make sure I had the opportunity to use SenseMaker™, but also to keep my eye open for other options. It's not that the results of detailed analysis weren't valuable, just that the price seemed far too high. I started out looking at semantic analysis and word density tools and stumbled into Latent Semantic Analysis, anti-plagarism software Turnitin and iThenticate and text analytics and I'm still looking.

Plagarism is a big concern in academia and even a simple search on Google will turn up many things that have clearly come from the same place, attributed or not attributed. Churning out words can turn into more than an embarassing slip-up for a professional writer . Stories can echo around the place. I get the feeling that often people are either just too busy or don't have the skills with the nuances of language to create an original response to something read or heard, it's easier to just repeat it on just like telegraph did in the 19th century.

Data mining is huge and if you don't have the technical background it seems a bit like divining for water. It does help if you know the water is there already. Text analytics have been developed for academic purposes and more general use. Examples include tools such as Leximancer and quintura though I am sure there are more (please feel free to comment on any good ones you know about). This post just marks some thoughts on how the options of finding ideas in large volumes of information are evolving.

I was reading today about how Benjamin Franklin received a 'could improve' note from his teacher as a young child and was encouraged to teach himself how to write eloquently by reverse engineering a much admired text.
Maybe part of the challenge and part of the solution is that the text and stories that link together big ideas use prose that is well beyond the day to day language people have learned to use. The best explanations make clever use of metaphors and myths to get complex ideas across to others. Part of me does not trust that the hocus pocus of semantics can really find these true gems, many of which I have come across purely through speaking with people, serendipity, rumours and persistence. Which is why I like story and narrative and using naturalistic sensemaking and SenseMaker™.