All 4 entries tagged Trial

Thoughts On The Trial Study

Research designs and research activities need to be trialled before they are deemed appropriate for full research implementation to ensure their validity, fitness for task, fitness for purpose, and their reliability, and to make any amendments. Research activity within educational research is the task that research participants interact with, and the interaction between participants and task is the object of investigation. The trail study is effectively ensuring a fit between the activity and the design as best as possible so that the interactions between participants and tasks are captured as effectively as possible relative to the research problem and research questions.

Activity trial

In what way does the activity match the research problem? In what way will the activity facilitate the needed interactions between it and the participants in order to produce suitable data for analysis relevant to the research questions? There are no easy answers due to the dynamism of the human subjects. Even in experimental designs (to some extent depending on the specific design type), famed for allowing the researcher to have freedom of manipulation of the environment and variables, it is difficult to predict exactly what kind of interaction shall occur between participant and activity. An important objective of a trial study therefore is to ensure the activity is suitable and interesting enough to sustain interaction and generate enough data for analysis.

The activity of my research is collaborative therefore it was important to carry out a trial study to ensure interest among the participants and therefore their interactions with the activity is sustainable to the point where enough data can be generated for suitable analysis. Success criteria of any research activity varies widely depending on its type and the way in which it is used to assist with resolving a particular research problem, and can be determined therefore through understanding your research problem, your research questions and your research design.

I considered the trial of the activity a success not in terms of the quantity of participants but the quantity of the data: the activity was able to hold the attention of the participants to the point where enough data had been collected. The nature of the research design enabled this success due to its focus on the importance of the content of the activity and not the quantity of participants. Other research methods using the same activity would likely emphasise quantity of the participants.

Research design trial

Following the activity, the design was then trialled. It has to be emphasised that depending on the design, either a part or the whole design can be testable. Again, because of the nature of my research design only a part of the design is trialled at this time for the purpose of fulfilling the requirements of the upgrade paper, and also obviously because I want to make sure that it works! The other half of the design shall be trialled at a later time with that trial discussed in the thesis.

Essentially, in a sequential exploratory mixed method approach the qualitative phase occurs first followed by the quantitative phase. The findings from the qualitative phase, which in this case is based on grounded theory, shall be used to develop the methods of the quantitative phase. Reasons why are extensive and are beyond the point of this blog post, but essentially because of the nature of this design the grounded theory approach is being trialled first.

Are you still with me? Yes? Brilliant, if any person could explain to me what the heck I am talking about, answers on a post card to………

The first part of the research design trial, the grounded theory trial, is currently ongoing but initial thoughts and findings suggest that the grounded theory aspect is working and has really helped me to understand mixed methods research more to the point that I have altered the mixed methods variety for this research.

Summary thoughts

There has been a lot to understand and process regarding this trial study especially considering that now the trial study can actually turn into a case study as part of the multiple case study approach of the research. It has been, and is, challenging because I am not just doing the trail study as a slap on attachment to the upgrade paper. This has been and continues to be a serious, detailed, reflective exercise of my research design and research activity and has helped a lot with my understanding of the activity data, ways to approach the analysis and understand the data, and understand the effectiveness of the research approach.

The trail study has evidenced the fact that whilst it is extremely important to read the relevant literature around your research design for academic argumentation for and against (and with just grounded theory alone there is a wide variety of different arguments), it is just as important to simply try it out. It is because of this trying out of the design that I figured out a potential fault with the previous favoured mixed method variety, concurrent triangulation, fixable only through switching to another mixed method variety, the sequential explorative variety.

Alex Darracott : 21 Aug 2016 20:19 |  Tags: Trial Upgrade |  Comments (0) |  Close comments |  Report a problem

Research update

Been quite a while since I have written a blog post on here as there have been plenty of things going on to distract me, but now is time for a series of blog updates! The following is a brief overview of the key areas of activity, with further blog posts following that explore these in more detail.

Update on the Trial study

The trial study is based on trialling a collaborative activity and trialling a grounded theory approach adapted from the ideas of Strauss and Corbin. I say adapted because there are a variety of authors suggesting Strauss and Corbin’s version is too regimented and difficult to follow, but Strauss and Corbin did respond by suggesting their procedures for coding do not have to be followed through strictly, therefore researchers should logically and constructively apply grounded theory procedures relevant to the research context. I might alter aspects of the approach as the trail and the main thesis study continues.

The trial appears to be coming along well. As reported in earlier posts I managed to successfully carry out the trail activity so it is now the case of continuing to use grounded theory to learn about it more, and to learn about the data that I have collected. I am fascinated with grounded theory! Lots of information on grounded theory online but for now it suffices to say that grounded theory is a key methodology (or method: it can be viewed both ways depending on its purpose and position within the research design) in developing new conceptualisations, a new theory, or new theorising of the phenomenon of interest. This is challenging and I have the belief that in order to really be successful at this you have to be quite creative and able to think about data, events, observations and aspects of reality at an abstract level. You begin with exploring the data but you have to think about the data at a higher, abstract level. It’s about interpreting the data based on your observations, though the validity of these interpretations can be increased depending on the research design and the way in which existing literature is used. But despite it being challenging it is an exciting approach to exploring data.

Trial Study as the Thesis Study

The most significant feedback and conversations during the past month or so has been the idea of using the trail study as a thesis study, or acting as a case within the thesis study. This was and still is huge, because it effectively means that when I am exploring and coding the data not only am I exploring data to benefit the production of the upgrade paper but also to benefit the thesis and the research itself. This has been deemed feasible therefore I shall be using the activity as part of the thesis and also explore other activities, known in the thesis as cases, in order to locate differences and similarities, which shall increase the validity of the theorising and theory development.

Change of Research Design

This change has actually come about through working through the trial study. Previously I had mentioned about adopting a critical realist, concurrent triangulation variety of mixed methods where the quantitative and qualitative aspects of the study would be carried out at the same time. In other words, I would have been carrying out grounded theory analysis of the activity whilst collecting thoughts and beliefs of the learners at the same time and then integrate the findings and discussions.

The only aspect that has changed is the mixed methods variety: I’ve ditched the concurrent triangulation and adopted the sequential, exploratory variety. Briefly, during the trail study I was struggling with the way in which data from the grounded theory findings and questionnaire findings could merge due to the abstractness of grounded theory. There would have been a chance of new concepts and new conceptualisations emerging from the data but exploring them via quantitative measures would not have been possible because the quantitative phase would have started at the same time, or in other words concurrently.

Using a sequential exploratory approach enables the constructions and conceptualisations, the theory, to emerge first and then test these out using the questionnaire and generalise across a wider population. This makes a lot more sense than to carry out both threads concurrently given the potential for extra insights that would not have been possible to explore had the mixed methods approach remained unchanged.

I’ve also encapsulated the research into a multiple case study, therefore the research design is now a critical realist multiple case based sequential exploratory mixed methods design.

Progress on the Upgrade Paper

This has been rewritten more times than I care to remember and will be reread and edited in various places before actually handing it in. When I rewrite each section I ask myself why did I not think about things in a particular way and communicate in a particular way before. But that is the beauty of re editing: it makes you think and contemplate more as you go through the document and think if you can explain anything in a more concise, or more sophisticated way without dropping the meaning behind the text. I know this, but still, sometimes it takes me by surprise at the differences that are found between each modified version of a particular document.

Most of the sections are now complete. Only sections that need to be completed are the trial study section, and talk a bit more about potential problems and possible outcomes.

That’s the brief overview of the main things that have been going on during the past month or so!

Alex Darracott : 20 Aug 2016 18:59 |  Tags: Design Mixedmethods Trial Upgrade |  Comments (0) |  Close comments |  Report a problem

Rambling load of updates!

Research Design

The components of the research design were decided a few weeks ago and the components are now unlikely to change. The research design is based on a critical realist approach to a convergent paralleled variety of mixed methods, which includes the use of a questionnaire and grounded theory methods based on the Strauss and Corbin version. I did go through a phase of feeling overwhelmed a few weeks ago as I had decided to change the grounded theory method component, which spurred a huge quantity of questions and thoughts about compatibility and research design validity all at once. I think reflecting back I had become so convinced that Charmaz’s Constructivist Grounded Theory would work, till the philosophical stance changed from relativism to critical realism, that the need to change the grounded theory method was impactful on my confidence of the research design and my overall ability to understand it all. But with patience, time, careful thinking and further reading my confidence in the validity and feasibility of the research design is growing.

This is not to say however that I know everything about the research design components and I do not actually know that the research design will take off. However, at the moment I am meticulously and comprehensively studying each of the research design components and threading them together to make a complete, compatible research design that is relevant to the research context and phenomena of interest. This is being achieved through constructing arguments for the need of the research design within the specific context and relevance for the phenomena of interest, and also through building critiques of existing research designs and relevant design components within Educational research, particularly relevant to the phenomena of interest. This is a current, ongoing task with argumentation and critiques continuously being constructed and amended in various ways through reading relevant literature and thinking about this literature.

Upgrade Paper

Coming along nicely though I shall be dealing with a few questions at the next meeting with the supervisor, including questions regarding the methodological section: is it best to focus on describing the process of the research design? E.g., describe critical realism in a way that is relevant to explaining the process of mixed methods then explain the process of mixed methods and so on. Or is it best to focus on argumentation of the research design? Or a mixture of both?

First draft should be completed soon. The sections have been figured out, so now it’s a case of completing the sections and then begin the process of editing. This is an ongoing process.

Trial study

Decided to let an aspect of the trial period run on for a while longer than previously planned, just to find out what would happen further with the trial and I am pleased that I made this choice because I now have more than enough data to build a practical understanding of the qualitative analytical research methods relating to grounded theory. Additionally, I wonder if there is merit in using the data from trial period in the actual research study, given that grounded theory has an “all is data” perspective. It would be interesting if I could use this data, because that would give me a start on theorising from the codes and categories generated from the data, as described by grounded theory.

Questionnaire is yet to be trialled, but this will happen in the future. This shouldn’t take too long, as all that is required is to trial the design, think about possible statistical analysis methods, then trial the analysis methods, and then think about the way in which quantitative and qualitative data can be analysed to complement or converge with each other. Oh, not too much to do then!

Summary

So that’s about it, but enough to get on with! In summary, the immediate tasks are: continue to thread the research design together through argumentation, critique of existing research, and experimenting with the design theoretically and practically; continue with the upgrade paper, and continue with the trial study through trialling the grounded theory methods, and the questionnaire.

Alex Darracott : 10 Jul 2016 20:25 |  Tags: Design Reflection Trial Upgrade |  Comments (0) |  Close comments |  Report a problem

Rambling Update On Progress!

The Trial Study Up And Running!

After gaining permissions from my supervisor and the admin of an online discussion forum, the discussion forum aspect of the trial study has begun! The aim of this aspect of the trial study is to determine the most suitable discussion activities for the main study and this is being determined through quantity of replies and extent to which the participants are taking part in the activities. The trial study has already indicated the topics that are worthy of pursuing further with the main study and also the topics that are not quite as popular and therefore not worth pursuing further, but even then this aspect is making an important contribution to the trial study.

What is most important is that I already have enough qualitative data for analysis and therefore to experiment with the developing Grounded Theory method. The qualitative data shall be used to experiment with this Grounded Theory method in order to assist with further developing the method ideas and to propose a Grounded Theory approach that is appropriate for the type of qualitative data being analysed. It shall no doubt be proven that you can read as many textbooks and papers on Grounded Theory as you desire, but you cannot actually beat being hands on with a method and really putting it through its paces with real, workable data. That is not to say that textbook reading is not important: this needs to be done in order to understand Grounded Theory at the Philosophical level, its place in the wider context of research methods, to understand its history and reasons of use, understand the processes and activities, and to understand its many critiques and debate points, but actually using it is just as important to your learning of Grounded Theory and any other research method.

The discussion trial is going well, and is due to carry on for a little while though since I already have the required data for the trial study I am likely to shut the trial down soon enough after finding out the results of interventions that I have made to spice up the activities that are lacking in use. Needless to say that I am pleased that it appears that the discussion trial’s aims and objectives so far is on track to being achieved.

In the meantime, I shall be working towards redeveloping the questionnaire and organise for a trial run for the questionnaire to take place.

The Upgrade Paper

Because the trial period has begun and all permissions granted I have been able to work on the Upgrade Paper through discussing the trial and its initial findings. I’ve also been rereading and reediting the paper particularly better describing the problem area and introducing the key authors and key literature referring to the phenomena of interest. I have also been introducing my own ideas and reasons why the phenomenon of interest needs to be investigated in a different way and in ways not presented in existing literature. Obviously I will be going into this further in the thesis, but for the upgrade paper I am trying to introduce my ideas and offer argumentation and reasons behind those ideas, based on existing literature.

Quite pleased with the paper so far. It is not allowed to go too far past four thousand words and that is going to be a challenge! Might sound like a lot of words to play with, but when you have a lot of ideas that you want to get across it is quite a challenge to explain everything in four thousand words. The core idea is to present key ideas, key authors, key basis of building the ideas, and the main aspects of the methodology, ethical issues, trial discussions, and anticipated findings and problems. Bit of a challenge trying to get all that into four thousand words (heck this post has just past six hundred words of waffle!) but it shall be done!

Critical Realism? Or not?

As some of you know, I did originally select critical realism as the philosophical basis of my mixed methods research, but I’m not a hundred percent sure now. The other philosophical perspectives are not suitable, therefore this does mean that I do like critical realism’s ontological and epistemological perspective of reality. Where there are problems now with the use of critical realism is its definition and focus on the relationship between structure and agent. From what I can currently understand (obviously need to do more reading, and learning is continuous) critical realism at the application level focusses on the relationship between a power structure (institution, policy, etc) and the agent (humans). From what I can understand at the moment there does not appear to be any literature that moves critical realism away from this idea of exploring the relationship between structure and agent, and focus on the specific aspects of my research because it is not being developed to explore such a relationship or to measure the impact that a structure has on an agent.

There appears to be however a main contributor of critical realism that does insist on the importance of moving away from power relations between structure and agent and focus more on aspects more appropriate for my research. But this I shall have to read further into.

What does all this mean? Well, given that the other philosophical perspectives of mixed methods are not really that suitable, if I cannot apply critical realism as defined by the main contributors I shall have to create a derivative of critical realism that better explores reality and learning context in the way that the research intends. I shall talk more about this in another blog post.

Making progress!

Which is the main thing, though I have to say that I didn’t expect to be challenged by critical realism in quite the way that I anticipated, but that really is the beauty of research and the Ph.D. in general: you are challenged to question your views of reality and find out if the way you think you view reality is really the way that you view reality. It’s all interesting and I shall be reading more into critical realism soon, as well as continuing with developing the questionnaire and continue to write the upgrade paper!

Till next time: keep it real(ism),or if not just keep it (critical) real(ism)!

Alex Darracott : 25 Apr 2016 20:20 |  Tags: Criticalrealism Groundedtheory Trial Upgrade |  Comments (0) |  Close comments |  Report a problem