All 1 entries tagged Generalisability

No other Warwick Blogs use the tag Generalisability on entries | View entries tagged Generalisability at Technorati | There are no images tagged Generalisability on this blog

October 02, 2016

Introducing Quality Characteristics For Research Design

I have been thinking more about the quality characteristics of my research and therefore the way in which my research could be defined as quality research, and this is an ongoing process that shall no doubt continue throughout the duration of the Ph.D. as quality characteristics need to be integrated across all stages of the research from the formation of the research questions right the way through to thesis writing and presentation.

Four questions immediately come to mind:

What makes a quality research design?

What makes a quality set of research findings?

What makes a quality set of inferences that are born from these findings?

What makes a quality thesis?

Guess what? These questions are all related so when you think about the quality of the research findings you have to think about the quality of the research design. There is in some sense some sort of sequence here: quality findings come from a quality design, and quality inferences should in theory come from quality findings. But actually reaching and working towards such high levels of quality involves an iterative approach. As an example, if you think you have a quality design then it can be reasonable to suggest that you think that the findings will also be of high quality; however, this is not always the case. This is not an exercise of slotting everything where you think best fits and have your fingers crossed for the best without any sort of foundational understanding of whether or not the design really will work. I recently found this with my own research design: I initially slotted the different components into place and then really went through the process of linking everything together and then came across significant problems as has been discussed in recent blog posts leading to case study elements no longer being useful or relevant for the research. The extent to which the various incompatibilities existed would have led to low quality data born from a mismatched design.

The quality of research design, findings and so on are defined by a series of quality characteristics. What is a quality characteristic? A quality characteristic evaluates and determines the extent to which a research design, set of findings and discussions (verbose inferences, descriptions, assumptions, possible application of findings and so on) have been designed, organised, constructed and presented using an adequate, sound, reasonable and careful approach to thinking and reasoning that is free from error and threats to the correctness and soundness of the design and the data. All these quality characteristics combined make a quality criteria framework. There are many quality characteristics grouped into three main categories:

Validity

Generalisability

Reliability

All characteristics within each group and therefore the groups themselves are applied differently in both quantitative and qualitative research, and their differences between each type of research have been the source of much debate and discussion.

My research design is mixed methods therefore not only have I been thinking about the different quality characteristics within both quantitative and qualitative research, but also that which are unique to mixed methods design such as the quality of the design itself relative to the research problem and questions, and the quality of the data integration process. Essentially what I have found when thinking about quality characteristics and thinking about the development of a suitable framework there are three levels of possible criteria: at the method (grounded theory and questionnaire) level, at the methodology (mixed methods) level and at the philosophical level (critical realism). What I might find in the future is a lot of overlapping and integration. For example a characteristic of the qualitative is “trustworthiness” which defines the extent that a particular inference I make is trustworthy, and a question here in a mixed methods sense is in what way can quantitative data add to this trustworthiness? In what way can the specific mixed methods design add to the general trustworthiness of the research? So already I have taken a concept specific to a particular methodology and applied it to an overall design. Therefore the different levels might overlap and interact with each other in many different ways. I can’t confirm this though till I am someway through analysing all the data.

It is quite a complex area and it will differ depending on what methodology and methods you are using for your research but the general categories of validity, generalisability and reliability apply to near enough all research, they just might be treated a little differently depending on your overall research design and research problem.

I will more than likely return to this topic several times in the future as I continue to work towards a quality criteria framework and the way in which all the characteristics are identified as relative and appropriate for my research.

‘Till next time, stay quality!