Using Qualitative Data
Qualitative data analysis is characterised by the organisation, identification, exploration, and interpretation of key themes and patterns in data. Often the data is textual, but can also be visual. Examining these emerging themes provides answers to your evaluation questions. One particularly useful aspect of qualitative data is that, due to its open and flexible nature, it is much more likely to capture any unintended consequences (both positive and negative) of your project.
Qualitative data is more subjective than quantitative data analysis because it requires a significant degree of interpretation. For example, a participant may make an ironic comment that once transcribed on paper implies the reverse of what they intended; without interpretation and context that comment can be misconstrued. Due to the fluidity and flexibility of qualitative data, it is far more likely that unintended outcomes will emerge from the information you have collected and these can be very useful. Involving participants and beneficiaries in the analysis process can reduce the risks of misinterpretation.
Qualitative analysis is often highly dependent on the context of the data collection and the skill of the interviewer or facilitator.
- Frequently results in a large volume of natural information
- Information collected is deep and nuanced
- Coding data across themes is common in most data analysis approaches
- Is likely to adapt as the evaluation continues and themes in the data emerge
When analysing qualitative data it is important to keep your primary evaluation questions in mind. This will help you to identify the key themes and reduce the likelihood of tangents that are not pertinent to the main evaluation questions you want to answer.
Some general questions you should be asking yourself throughout are:
- What main themes and sub-themes are emerging for individuals/groups and how do they inform your evaluation questions?
- Are there any themes that diverge from the main themes? If so, consider possible explanations.
- Do the themes emerging suggest you are adequately collecting data? You may need to revise or adapt your approach or collect additional material.
- Am I only seeing what I want to see? When you start to see a particular pattern emerge, take time to actively look for evidence that contradicts that pattern. This can help prevent the tendency to search for or interpret information based on what you already believe, which is referred to asconfirmation bias.
As soon as you have collected qualitative information it is important to process and check it as soon as possible, when your memory is freshest. The sooner this is done the better. The information from multiple discussions can begin to overlap when you try to recall them, and you can quite easily forget who said what if you don’t have a good record of it. Even if you have an audio or video recording of the evidence, having the conversation fresh in your mind can be hugely helpful. If there was no recording, notes taken in short hand should be expanded, and poor handwriting fixed. Also, keep notes for yourself in addition to participant responses (for example ‘participant was joking/being sarcastic’). These notes are even more important if more than one person will be dealing with the data. Incorrect assumptions can be made if a person typing up or analysing the data was not involved in the interview and was therefore unaware of the context of a comment or response.
You can also add notes about other highlights or aspects of the data that seemed odd, unexpected, or interesting. It may be possible or useful in an evaluation to analyse these notes and observations so that they become part of the evaluation.
Qualitative analysis can seem very complex, especially for someone who has never done it before, but don’t let that put you off. It is surprising how much interesting and fruitful information emerges by having a systematic way of collecting such data. It may complement or contradict the commonly held wisdom about a service or situation. If you are unsure you may wish to seek specialist help for complex analysis.
There are also practical things to look out for:
- Participant ideas that you can incorporate into your project now or in the future.
- The context of comments.
- Quotes that exemplify experiences and feelings for use in reports.
- Particular language that participants use which may help inform other elements of the evaluation (for example, the wording of future interview or survey questions.
It may be tempting to write up everything that a participant says and this is where a subjective balance exists. It should be quite clear early on what is or is not pertinent to your evaluation. Other comments by participants may be useful to your organisation but outside the scope of your self-evaluation activity. As long as you keep the information anonymised and protected you can always come back to the data later to assess further or additional details raised by individuals or groups.
For raw qualitative data to become meaningful, it must be grouped into themes or categorised in some way. This is one of the main aspects of analysing qualitative data. Three approaches are detailed below:
Content analysis: Content analysis is an efficient way of identifying and quantifying broad trends and patterns and is useful or analysing all kinds of texts (written, images and sound). It quantifies the relative occurrence of selected features in texts, generating themes as patterns within and across the categories of analysis. It is performed by looking in the data for certain content or words, identifying the frequency of particular words or content, identifying their patterns and interpreting their meanings. This is done by going through the text and categorising relevant words and/or sentences, after which the data can be explored.
Thematic analysis: Thematic analysis is simply grouping the information into themes that help you answer your evaluation questions. These themes will either come from the interview/focus group guide you devised at the beginning, emerge from the data naturally, or more likely, a combination of both. Once your themes have been identified it is useful to group the data into thematic areas so that you can analyse the meaning of the themes and connect them back to the research question(s). An example of a theme could be the reported effects of a new service on participants’ mental health.
Framework analysis: Framework analysis is useful in evaluations because it allows you to record both an individual’s information (that can also be used as foundation for a case study) as well as allowing thematic information to emerge. Consideration must be given to keeping the data confidential and secure. Password protect spreadsheets or use codes or pseudonyms rather than a real name in the data sheet with a separate key that links to the participant’s real data elsewhere.
The table below is an example of what can be constructed in a spreadsheet from qualitative information. The horizontal rows give you all the data recorded for an individual. This can be used as an exemplar of one person’s experience of a service. The vertical columns show you all the data recorded for any given question which will often be a theme in itself. This allows you to easily group all the information about a certain question together to look for patterns.
Alternatively, you could simply put all the participant responses from a certain question (e.g. ‘did your involvement with service X have any impact on your mental health?’) into one document to analyse that way.
Researchers in the past have printed out transcripts and cut them up with scissors and then organised them into themes but this is quite a labour intensive method and there are now much quicker digital ways to do this e.g. simply by using the cut and paste function in word processing software.
The type of analysis you choose will depend on what you want to find out and what kind of data you have.
Visual data could have been collected across a range of formats: photographs; videos; artwork; or some of the participatory methods described elsewhere in this guide. Often the material will be able to stand alone as data in your evaluation and other times you will need to explore it further to identify particular themes. Working from photographs or video diaries may take a bit of time to analyse.
Important: Regardless of what approach you use, always keep the original transcripts or field notes intact; you may need them later. You should copy any pertinent text into new documents or duplicate the original documents. As always, consider your data storage and ensure that all data is protected, as described in the Ethics section.