All entries for October 2016

Updated thinking about researcher influence on research design

A couple of days ago it was a year since I laid out a few questions in a blog post that I was asking myself at the time regarding the role that researcher beliefs and perspectives of reality play in the research design. I thought I would provide an update on the current thinking regarding these questions.

First Question: Could a researcher, even within a Social Science discipline, really be objective?

Whilst this is being continuously thought about, at the moment I do not have the belief that any researcher can really remain objective, even quantitative researchers. When we talk about qualitative researchers, the argument is obvious in that objectivity is difficult to achieve although this really depends on the way in which objectivity is actually defined. For example, the nature of qualitative data requires the construction of a grounded interpretation of what the data is trying to suggest. Qualitative researchers construct meaning of the data, but this construction of meaning is an interpretation but grounded in the data.

It can be argued that it is this interpretation that gives the process subjectivity whilst the grounding of the interpretation can give research a sense of objectivity. This sounds similar to retroductive and abductive analyses and to some it does not make sense, but it is making increasing sense to me as it appears in my opinion to be good common sense to continuously construct an interpretation and ground any beliefs that stem from that interpretation in the data. Grounded interpretation means all beliefs, thoughts, ideas, and so on, coming from an interpretation that do not fit within the data itself should be discarded. This however does not necessitate the use of grounded theory but everything that is observed must be grounded in the data in some way.

Even social scientists using quantitative data could be viewed as subjective because from a broader sense the research questions and the fact that the social scientist has decided to use quantitative approaches is subjective because it is based on the way that they understand the research problem and the research question. So, whilst an experimental or quasi-experimental and other positivist, objective leaning approaches do collect and analyse data in a matter of fact way, the way in which those findings can be applied to different situations and indeed the way in which the findings are perceived will differ between social science researchers, in my opinion.

What I am considering further is the way in which we really relate to the data and therefore the way in which we interpret the data, and this is important for qualitative researchers. What is the relationship between the researcher and the data, and what factors are involved with such a relationship?

Second Question: Is a researcher drawn towards research methodologies more so because that methodology and methods match their framework of perceptions, beliefs, perspectives, values and attitudes of and towards reality?

Though I am still thinking about this, I would say yes: a researcher is drawn towards not what is actually best to answer a research question and to solve a research problem but is drawn towards that which best aligns with a researcher’s framework of perceptions, beliefs, perspectives, values and attitudes of and towards reality. But what are we talking about when we are talking about perceptions? Perceptions of what exactly? The research problem? Our own interpretation of the research questions that we ask? Where do these perceptions come from and what is it that we are meant to perceive? In what way do our perceptions influence our beliefs, values and attitudes?

My Philosophical perspectives have changed during the past year from constructivist to critical realist because I have come to realise the complexity of reality relative to the phenomena of investigation; that neither exploring the process of the phenomena nor exploring the experiences that people have of the phenomena are enough to gain a full understanding of the phenomena. Risk taking has and still is involved, but so far I think I am on the right track with my philosophical and methodological development and development of argumentation for them. It’s been especially easier since dropping case study. Well, that’s the other issue: sometimes we can become quite set on a particular methodology that we come across difficulties and struggles when we attempt to integrate particular methodologies and methods with other methodologies and methods, but this really depends on the way in which methodologies and methods are used. E.g., I was proposing to use a case study approach as a strategy for question formation, data collection and data analysis but came to the realisation that it was not compatible with grounded theory therefore dropped all case study elements. Since then in my opinion the methodology has been more workable.

I suppose we could say that researchers are initially drawn to methods and methodologies that meet their frameworks of preferences, but then later when they really begin to think about their design, the phenomena and the context of exploration they begin to understand what really might or might not work.

Third Question: Are we as individuals within our society really able to reach or understand objective truth about reality, or will people forever be led by their own preconceptions, perspectives, values and attitudes of and towards reality?

Answer to this for me is a lot more stable than it was a year ago: from a critical realist perspective the answer is, whilst objective truth about phenomena might be out there independent of our thinking, experiencing and perceiving such phenomena, our understanding and knowledge of this phenomena is subjective and always prone to fallibility and defeasibility. This is exactly because our personal frameworks of observation and understanding reality are based on our own experiences of and interactions with reality.

Fourth Question: What should be the extent or role of a researcher’s subjective framework of beliefs of reality play on their role of being a researcher and the development of their research design?

I have the current belief that this is really down to the individual researcher to decide. For me personally, my philosophical and methodological approaches have changed as my own understanding of the phenomena and the context and situation within which the phenomena are to be explored has increased and developed. Has the research design altered based on my own framework of beliefs of reality? You could say yes, but then can also abstract a step higher and say that the research design has changed because of being open minded, cautious about being absolutely certain about research designs, and the willingness to change beliefs about reality: to let my understanding of the phenomena and its explorative context influence my beliefs about reality and therefore about the way in which reality and the phenomena, is to be explored and should be explored.

Summary

Thinking about the role of the researcher, the relationship that the researcher has between him/herself and the participants and between him/herself and the data is a current topic of thinking and consideration for my research. This is because if we do not think about ourselves as researchers, our positioning within reality, the way in which we view data, and the way in which we view research designs we are in danger of becoming stagnant and willing to accept any design that we come across just because it basically works. But, this pragmatic approach to “what works” does not necessarily mean that it is the right or best suitable answer.

‘till next time: keep designing!

Alex Darracott : 18 Oct 2016 11:58 |  Tags: Philosophy Qualitative Quantitative Reflections |  Comments (2) |  Close comments |  Report a problem

Introducing Quality Characteristics For Research Design

I have been thinking more about the quality characteristics of my research and therefore the way in which my research could be defined as quality research, and this is an ongoing process that shall no doubt continue throughout the duration of the Ph.D. as quality characteristics need to be integrated across all stages of the research from the formation of the research questions right the way through to thesis writing and presentation.

Four questions immediately come to mind:

What makes a quality research design?

What makes a quality set of research findings?

What makes a quality set of inferences that are born from these findings?

What makes a quality thesis?

Guess what? These questions are all related so when you think about the quality of the research findings you have to think about the quality of the research design. There is in some sense some sort of sequence here: quality findings come from a quality design, and quality inferences should in theory come from quality findings. But actually reaching and working towards such high levels of quality involves an iterative approach. As an example, if you think you have a quality design then it can be reasonable to suggest that you think that the findings will also be of high quality; however, this is not always the case. This is not an exercise of slotting everything where you think best fits and have your fingers crossed for the best without any sort of foundational understanding of whether or not the design really will work. I recently found this with my own research design: I initially slotted the different components into place and then really went through the process of linking everything together and then came across significant problems as has been discussed in recent blog posts leading to case study elements no longer being useful or relevant for the research. The extent to which the various incompatibilities existed would have led to low quality data born from a mismatched design.

The quality of research design, findings and so on are defined by a series of quality characteristics. What is a quality characteristic? A quality characteristic evaluates and determines the extent to which a research design, set of findings and discussions (verbose inferences, descriptions, assumptions, possible application of findings and so on) have been designed, organised, constructed and presented using an adequate, sound, reasonable and careful approach to thinking and reasoning that is free from error and threats to the correctness and soundness of the design and the data. All these quality characteristics combined make a quality criteria framework. There are many quality characteristics grouped into three main categories:

Validity

Generalisability

Reliability

All characteristics within each group and therefore the groups themselves are applied differently in both quantitative and qualitative research, and their differences between each type of research have been the source of much debate and discussion.

My research design is mixed methods therefore not only have I been thinking about the different quality characteristics within both quantitative and qualitative research, but also that which are unique to mixed methods design such as the quality of the design itself relative to the research problem and questions, and the quality of the data integration process. Essentially what I have found when thinking about quality characteristics and thinking about the development of a suitable framework there are three levels of possible criteria: at the method (grounded theory and questionnaire) level, at the methodology (mixed methods) level and at the philosophical level (critical realism). What I might find in the future is a lot of overlapping and integration. For example a characteristic of the qualitative is “trustworthiness” which defines the extent that a particular inference I make is trustworthy, and a question here in a mixed methods sense is in what way can quantitative data add to this trustworthiness? In what way can the specific mixed methods design add to the general trustworthiness of the research? So already I have taken a concept specific to a particular methodology and applied it to an overall design. Therefore the different levels might overlap and interact with each other in many different ways. I can’t confirm this though till I am someway through analysing all the data.

It is quite a complex area and it will differ depending on what methodology and methods you are using for your research but the general categories of validity, generalisability and reliability apply to near enough all research, they just might be treated a little differently depending on your overall research design and research problem.

I will more than likely return to this topic several times in the future as I continue to work towards a quality criteria framework and the way in which all the characteristics are identified as relative and appropriate for my research.

‘Till next time, stay quality!

Alex Darracott : 02 Oct 2016 17:20 |  Tags: Generalisability Quality Reliability Validity |  Comments (0) |  Close comments |  Report a problem