Pedagogy of the Artificial talk, Café Scientifique, April 28th

Follow-up to Learning as design[ed] at Café Scientifique, April 28th from Inspires Learning - Robert O'Toole

The Café Scientifique proved to be a great event. Congratulations to Jayne and her team! It was really well organised and provided a great "interdisciplinary" forum for new research. The two (by chance related) arts or social science papers contrasted interestingly with the science paper - although there were connections. This immediately raised questions about how we work and the purpose of communicating research - questions addressed directly in my paper.

During the writing of my presentation, the title slowly morphed into "Pedagogy of the Artificial", a reference to Herbert Simon's "Sciences of the Artificial" (Simon, 1969) and Richard Coyne's excellent paper "Wicked Problems Revisited" (Coyne, 2004)  which brings Deleuze and Guattari to the party. Sadly, I had to edit things down to get within any hope of the ten minute limit. An interesting digression on the origins of design thinking in the baroque, via Deleuze's The Fold, was removed.

Thanks to everyone who asked questions and gave feedback (and apologies to Jeremy Ireland for slightly dodging his difficult question). I'll respond to the questions posted by email, and then write up some further ideas.

Something that I realised during the talk was that I am talking about a "radical democracy" made possible by various technical, social and economic development - the notion being that the separation between users-consumers-suppliers-manufacturers-designers is being eroded, and that design agency is being dispersed throughout these networks of actors. We just need to help people to develop the right kinds of theory - one in which "design thinking" is understood to be a shared capability and activity, rather than the preserve of a specialised elite.

Firstly, here's the slides and script. Then at the end of the entry, i add a video of my presentation recorded using my new Apple Podcast Producer 2 server and a MacBook.

Pedagogy of the Artificial, a talk given at the Café Scientifique in April 2010

I’ll start with a quote from the design thinker, Bruce Mau:

My research is about deliberate positive durable change: innovation. In higher education, in every aspect of higher education, ranging from the first year undergraduate performing a seminar presentation for the very first time, up to the foundation of a new university, and across many points in between: a group of researchers writing a funding bid, a library creating a new open learning space, a teacher adopting a different method, a doctoral student synthesizing half a year’s work into a ten minute presentation.

Common experiences and patterns of behaviour recur at all of these diverse points, and are transferred between them.

For the people embroiled in these changes, what might seem to others to be trivial is in reality a matter of what Mau calls “massive change”. That doesn’t necessarily mean revolutionary global change, but rather change that passes across a threshold and becomes irreversible.

Or, as is often the case, the result is paralysis, indecision, confusion, lost opportunities, lost insights, repetition, and dead-ends.

Some such changes are deeply problematic. for example the zones of transition between stages in an individual’s academic career.

Often we face what the design theorist Hors Rittel famously termed “wicked problems”. Richard Buchanan defined such cases as:

“a class of social system problems which are ill-formed, where the information is confusing, where there are many clients and decision makers with conflicting values, and where the ramifications in the whole system are thoroughly confusing”. (Buchanan, 1992: 15)

As designers have started to address issues of quality enhancement and process innovation in areas such as health care, education, welfare and government, our understanding of just how wicked these problems can be has grown - complex feedback loops between the definition of success, stakeholder perceptions and capabilities, resulting in chaos or more often paralysis - a failure to get beyond grand schemes or an obsession with optimization (Six Sigma).

Universities are, perhaps more than any other social institution, intensive sites of metamorphosis. Universities problematise and disrupt to discover and create. They are full of “wicked problems”. They seek “wicked problems”. Through exercising these capabilities, millions of people become more effective agents of change, both personal and collective.

That’s fascinating. And there’s a lot to learn from it.

We can also learn from new techniques developed by the design industry to help tackle “wicked” problems in other fields. Bringing such techniques to higher education is my chief concern.

However, for this to be possible, we must be prepared to accept two things:

1.    design is more than decoration;

2.    and the radically democratic idea that everyone can be a designer.

Let’s now move on to a second statement about the nature of design, this time from the sociologist Bruno Latour (famous for his studies of research scientists):

This is taken from a fascinating paper entitled “A Cautious Prometheus? A Few Steps Towards a Philosophy of Design” presented at a meeting of the Design History Society.

Latour argues that “design” is a philosophically important concept, marking out the activities and practices involved in creating the artificial. ‘Artificial’ is not a negative term, opposed to the natural. Rather, ‘artificial’ refers to inventions through which humans and human-invented entities can evolve with nature, both in nature and beyond. Design also has a duel relationship with time. Design thinking, for Latour, says of the world:

“it will neither be modernized, nor will it be revolutionized” (Latour, 2008: 3)

It will be adapted, designed.

The concept of “design” and the “designer” emerged in the early seventeenth century, as it became possible for a single person to conceive of designs implemented across the full range of crafts and objects. The power of this idea is being reinvigorated to similarly exciting ends, but with a new ingredient: the power of the designer is dispersed, aided by technology and social mobility, into networks of people - often people who would not normally consider themselves to be designers.

There are three ways of interpreting this reinvigorated design capability.

Modernist eyes see a world of obsolete people and things to be re-engineered to meet the needs of the new normative now. Revolutionary thinking looks to change everything so as to remake itself and the world. Design thinking is different. It aims to achieve innovation through a modesty, a mastery of detail and what Latour calls a “semiotic skill”. Most importantly, Latour says, “it is never a process that begins from scratch: to design is always to redesign.” Design has one eye on finding the valuable in the present and past and another on inventing new value.

That’s the theory.

Now for the practical.

In his book Reassembling the Social, Latour re-analyses the ingredients of social change. This is in essence a lesson in creating the artificial.

He gives the formula: actor dash network dash theory. Three essential elements. Each of equal importance. Note that the second dash, indicating that theory is part of the same assemblage as actors and networks, is important and often overlooked. Theory is part of, immanent to, the world of actors and networks.

Three interconnected elements, each necessary for change to be possible. The formula may be expanded as:

Becoming an agent (of change) requires being a network, being a network requires having theories (about other agents and the network).

But each of these steps is a great challenge. Networks are unstable. Theories contested. Being an ANT, as Latour says, requires:

Giving up control, to others and objects (which themselves have agency).

Adopting hybridized, contested and transitory identities.

Being opportunistic, open to disruption, taking risks.

As a teacher, I recognize these challenges as amongst the most difficult and most essential facing every student.

Independent of all the theory, the IDEO design company has come up with a set of strategies for getting collaborations of diverse design stakeholders to take just these risks.

Tim Brown explains how design thinking adopts...

”An inherently shared approach, design thinking brings together people from different disciplines to effectively explore new ideas—ideas that are more human-centered, that are better able to be executed, and that generate valuable new outcomes.” (Brown, 2008)

Key to their success is a clearly signified separation between “inspiration” activities that expose participants to disruption and complexity, “ideation” activities through which discoveries and theories are synthesized into testable lo-fi prototypes (the slogan is “build to think”), and “implementation” activities that see the creation of shippable end products. Time and space is given for becoming collectively concerned, and for creating and testing theories or models.

Brown describes how this translates into a set of design spaces:

“The design process is best described as a system of spaces rather than a pre-defined series of orderly steps. The spaces demarcate different sorts of related activities that together form the continuum of innovation.”

A design team is encouraged to move freely between the spaces, aiming to move more towards the “implementation” space as certainty and consensus emerges.

How might this approach help students and teachers, for example, in becoming creative, forming their own projects and theories? How might it help individuals and collectives in higher education to redesign what they do?

And here is a video recording of my talk:

Robert O'Toole : 01 May 2010 07:49 |  Tags: Ideation |  Comments (0) |  Close comments |  Report a problem

Evaluating the “impact” of non–trivial educational innovations – the reality

Here's a text i've just written for various purposes. It's just a first draft knowledge dump, to become something more substantial when i get a chance. And be warned, there is some shouting...

In the past year I’ve had many conversations with innovation-minded teachers, educational developers, administrators and policy makers ending in something along the lines of:

“we have to objectively demonstrate quantifiable impact on the student experience”.

This has become the mantra amongst decision makers, budget guardians, in HE. Good. About time. If only they had any idea about how we can actually achieve this worthy ambition! If only they could accept that it’s not a simple task. If it were that easy, why hasn’t it been done before? Perhaps it’s because innovators don’t like to be tested with reality? No, I don’t think so. Painful experience has taught us that doing innovation in HE is hard, evaluating the true impact of innovations is even harder.

Let’s make a serious consideration of the options.

Firstly, the easiest route to objective quantifiable evidence is, as they say in the design world, to “get it in the workflow” – that is to say, make the innovation unavoidable. Furthermore, we need to make the actions that provide data for our evaluations unavoidable. In the world of HE, that usually means to make the activity mandatory, and design formal assessment mechanisms that evaluate its impact on performance. That might not give us information about softer and more subtle aspects of impact (or the genuinely longitudinal), but it’s the best chance we’ve got.

Great idea. Let’s do it.

Not. It turns out that opportunities to introduce mandatory non-trivial innovations (i.e. one’s that aren’t just small enhancements or fillers of obvious gaps) are scarce. Opportunities for modifying formal assessment and feedback processes are even more rare. There are many reasons for this: overcrowded curriculum, pressure on time and resources, the long cycle of the academic year (sometimes there will only be one opportunity every year or less), aversion to risk (especially amongst students). And don’t forget the ethical issue. Students usually only get one chance at anything in a degree programme. You can’t make changes that risk screwing up a student’s chances. I recently heard a report from an Australian university at which a course had been re-engineered. Large changes were made simultaneously (often it seems to make more sense to change many things at once). Beta testing was not undertaken (it’s hard to realistically beta test such things). The results were a disaster for a whole year’s intake of students, along with the careers of several teachers.

Management edicts concerning the need to demonstrate impact will not make this any easier. We are most often all impotent in these matters.

That leads to the commonly adopted second best solution: getting voluntary involvement and voluntary feedback. In some cases this might take the form of testing the performance of a sample of students who have voluntarily undertaken the activity. Payments might be necessary. More usually, a prize is on offer. In a few rare occasions, extra academic incentives are available (I once worked on a project that offered students an opportunity to access an additional 5% on their mark). Great in theory. In an ideal world, all students would engage in development activities that offered to improve the quality of the student experiences. In reality, even when additional marks are on offer, participation rates are low. This often means that evaluations are based upon a self-selected group with a tendency to start off with favouring the innovation. Worse still, if we respond to this by offering the innovation more widely (say to target 1% of all students from across a whole university) variations of context introduce extraneous variables that are difficult to identify and account for (ramping up cost and duration).

Next we move on to more sophisticated hybrid methodologies. These approaches are more promising, but force us to abandon simplistic notions of quantitative representation of action and impact. In my experience, university managers find it difficult to let go of such simplicities. Worse still, the involvement of any kind of interpretative element in research leads to suspicion of manipulation (however, I argue, to evaluate impact within reasonable limits of time and cost THEY JUST HAVE TO START TRUSTING).

The fundamental concepts here are: calibration and correlation. Any research process, action-research included, is really built out of a set of different view points. We might be able to get richly descriptive and personal information from a smaller number of participants. We might be able to act as participant observers (with inescapably limited and prejudiced horizons). We might be able to introduce interventions at certain times and locations. And alongside this focussed information, we can gather wider data for quantitative analysis. Now connect these together into an iterative process of investigation. For example, talk to key individuals to suggest models and conjectures. Survey the wider population. Look for correlations and connections. Modify models and return to closer focus to test them with individuals. All the time understanding how each element in the investigation, each form of evidence, is calibrated along the scale of micro and macro, individual and population – being aware of the limitations that implies for our claims, but also what it tells us about the changing dynamics of generic characteristics at the level of the population and divergent deviations at the level of the individual: what the sociologist Bruno Latour calls “actor-network-theory”.

That gets us a long way, despite requiring rather more skill and intelligence than most managers will admit. But we still have the problem of engagement. How can we get a sufficiently large number and range of people involved in this action-research?

Let’s turn to the experts: people who’s job it is to dream-up, test and implement new solutions to some very difficult problems. For example, innovation in health care processes. The IDEO design company are amongst the best, and are certainly the most documented. To start with, they address the problem of engagement, of getting people to take risks with new ideas, by drawing end users into the design process from the outset. By the time a product reaches the stage of being a live prototype, its users have already got ownership and responsibility.

They call it design thinking. That's often misinterpreted as a technique used by professional designers. The real trick is to get everyone design thinking (managers, users) – sharing responsibility from the outset. For the same reason, Latour encourages everyone to do ANT. Solutions are therefore to a great extend solutions created by the people who will benefit from them (indeed a compromise between all stake holders). They will have been part of dreaming it up (along with other options) and making it fit. IDEO call this generative activity “inspiration”. It is complemented by a second and clearly distinguished activity “ideation” – collectively creating and testing (breaking) prototypes under realistic conditions. Only after much inspiration and ideation will there be a move to implementation – the point at which we are sure about impact.

Combine “design thinking” with a more traditional “action research” and we are on to a method for innovating with meaningful and achievable evaluation of impact.

Robert O'Toole : 23 Apr 2010 14:29 |  Tags: Ideation |  Comments (11) |  Close comments |  Report a problem

Decisions that aren't designs aren't decisions

....that could be O'Toole's first law of education management!

If a so called 'management decision' makes no explicit attempt to explain how it will be achieved, what the consquences are and how they will be apprehended, it's an empty gesture or worse, an edict. Decisions design action. Well designed decisions design effective action.

Robert O'Toole : 23 Apr 2010 12:47 |  Tags: Ideation |  Comments (1) |  Close comments |  Report a problem