Generative & evaluative

Design research is often split into generative research and evaluative research.

Generative research is conducted before the design is begun and helps designers segment their users into groups with similar goals, needs, behaviors, attitudes, language, and mental models. Knowing this information helps designers understand what to design and how to design it.

Evaluative research is conducted once there are design artifacts to test with users. The designed artifact can be the completed design, or the underlying concept for the design, or some isolated aspect of the design, such as its taxonomy, or its use of words or imagery.

To this point, the perspective I have offered on research is strictly objective. The researcher is working entirely in the realm of gathering factual information about users (or about the user’s response to artifacts).

But these same research methods can be used to produce subjective insights in the researcher and the design team which can be even more valuable than the informational content.

*

Before I describe the benefits, I will will first need to explain the difference between objective information and subjective insight, because the two are frequently confused and used interchangeably.

Information about characteristics of a person’s subjectivity (for instance, descriptions of attitudes or feelings,) is still objective information, even if it is about something that lacks physicality or cannot be directly proven scientifically. The one acquiring the information still regards its object from a distance, and is not directly subjectively affected by acquiring the knowledge.

Subjective insight is such that it directly changes how one perceives. Insight is not about a subject, it is about the world the subject experiences. For the researcher (and design team) he insight takes the form of seeing one’s own world differently. This does not mean that the researcher begins to see in exactly the same way the research participant sees, but it does mean that the way the researcher sees changes in response to what he learns in an attempt to understand the participant.

(For this reason an insight is always to some degree disruptive to the one having it. Insight is not for the faint of mind or for the dogmatist. It requires tact, imagination, flexibility, openness and resilience. These qualities are not terribly common in the business world, which is why researchers and empathic designers can sometimes feel misunderstood.)

*

Another way to see the difference: Objective information refers to the 3rd person. Subjective insight appeals to the 1st person.

First person, always means “I”, that is, the researcher, or the designer. The insight must come out of the “I” having the insight, in the form of an immediate experience — a shift with an “ah ha!” Anything less is not an insight.

The difference between objective knowledge and subjective knowledge is the difference between hearing the details of a plot versus becoming absorbed in a literary novel or poem, or learning about history in the traditional way versus seeing a historical film. In the former the emphasis is on the facts. The latter provides facts, but more emphasis is placed on how the facts are related and perceived as meaningful for a particular person.

*

When generative research is understood to contain not only factual information — which is very important — but also contains subjective insight, generative research is revealed to be even more valuable than it first appears.

Not only does the research provide the design team with the material it needs to make smart design decisions (what to include, how to prioritize it, what nomenclature will make the most sense, what kinds of imagery the user will respond to, etc.) — it also provides an opportunity to shift the perspective from which the problem is seen and approached. This has two benefits:

1) Accessing this perspective is the same as “knowing where the user is coming from.” Knowing where someone is coming involves much more than knowing how the other feels, or what he is interested in, or knowing what words he uses. It is something that must be demonstrated, and the judgment on whether the demonstration is authentic or not is highly intuitive. To have objective information about the other, and attempting to use it without really understanding the other’s perspective is a sure-fire way to sound inauthentic and manipulative.

2) Acquiring a perspective for seeing a problem is creatively productive. When we get creatively blocked, most often it is because we try to solve a problem with our unexamined and unconscious habits of thought. Think about how breakthroughs happen: We suddenly conceive of a new way to think about the problem, and once we view it in this new way all sorts of possibilities open up to us, and we have a breakthrough.

A similar change occurs in the evaluative research when it is understood to test not only the artifact (as a sort of experiential quality assurance), but also how well the design team has learned to see where the research participant is coming from, and to demonstrate that understanding through designing an artifact that addresses that perspective. Full understanding of a research participant enables a talented design team to design in a way that demonstrates deep understanding.

This kind of deep understanding — insight — combined with thorough understanding — having all relevant information — is the foundation for cultivating the strongest kind of brand relationship.

3 thoughts on “Generative & evaluative

  1. What about ‘Foundational’ or ‘Formative’ research? This is the 3rd type of research that comes at the ‘Front End’ BEFORE generative research, and is usually confused with it. Generative research aims at creating concepts and testing them through participatory methods with end users. Foundational research holistically understands the ‘world’ of the solution or user, so that you know enough to even plan generative activities…

    1. …However, I am noticing more and more that these forms of research blend into one another and function more as points of emphasis rather than absolute divisions. The more research is driven by informing research problems and less by methodological prescription the more we use tools to shed light on various dimensions of questions than establishing solid foundations of fact then building upon them story by story.

Leave a Reply