An anthropologist and a philosopher dig into the meaning of data artifacts in scientific research.
While conducting fieldwork at a lab at Princeton University, Talia Dan-Cohen, associate professor of sociocultural anthropology at Washington University in St. Louis, observed a common but perplexing problem. For her book A Simpler Life: Synthetic Biological Experiments, Dan-Cohen was tracking the work of practitioners in the developing field of synthetic biology, and she noticed two researchers discussing the same peer review report again and again. For months, the scientists had a hard time convincing reviewers that what they saw as the big result of their work was actually a result at all rather than an artifact, or a flaw in the data caused by their equipment or technique.
For Dan-Cohen, the dispute pointed to larger questions: what is an artifact, anyways? When does an artifact become a result, or vice versa? Who gets to decide?
Dan-Cohen posed these questions to the philosophy of science reading group at WashU, thinking that surely an account already existed in a journal somewhere. She soon learned that despite being a common problem for every researcher who works with data, no standard definition could be found in the philosophy of science literature. With Carl F. Craver, professor of philosophy and philosophy-neuroscience-psychology, she set out to write a quick paper outlining a definition.
They would find the task much more challenging than they anticipated.
Making use of artifacts
Scientists deal with artifacts regularly. Researchers are trained to look for the kinds of standard artifacts that come up day-to-day, but other kinds can lead to scientific controversies. Researchers place their careers on the line when they have to parse the difference between an artifact and a result that could lead to a major discovery.
At the Princeton lab where Dan-Cohen conducted fieldwork, the conflict arose from the nature of the experiments. The researchers Dan-Cohen observed were carrying out basic science, but they were also building new things out of biological parts.
“They didn’t even know how exactly they had created something new, but part of their claim was that you didn’t need to know how,” Dan-Cohen said. “The reviewer described their result as an artifact, and the researchers wrote back that the reviewer was misunderstanding what an artifact is.”
Scientists talk about what goes wrong in an experiment in many different ways, and Dan-Cohen and Carver’s paper attempts to make distinctions between the ways that things can go wrong.
“Experiments are where the rubber of science hits the road of the world,” Craver said. “Is there a principled way to say, this is good data and that is bad data? If there is no basis for that judgment, then at the place where the rubber is supposed to be hitting the road, there’s a tremendous amount of slippage. This is a crucial question in understanding the epistemology of science: how can science purport to tell us something about the world?”
A two-part definition
An artifact occurs when an experimental setup or apparatus gives a researcher information that’s seemingly about the thing that they are interested in but, for one reason or another, is misleading or about something else entirely. Dan-Cohen and Craver created a model defining artifacts that approaches the problem from two very different directions.
The first component of their definition is causal. Researchers use instruments and methodologies that are designed to provide a causal chain between the observable world and their results, but sometimes the instruments themselves can introduce artifacts in their data.
“The first part of what an artifact is involves some error in that causal chain, a deviant causal process that enters in and disrupts the regular relationship between what you observe and the thing that you’re trying to learn about by observing,” Craver said.
In the second component, Craver and Dan-Cohen identify a more pragmatic element. Whether something counts as an artifact really depends on what a researcher is trying to learn about the target they have chosen as the subject of their investigation. Sometimes, one researcher’s artifact is another’s result. In the pragmatic component of their definition, what a researcher is interested in determines when the causal chain is the right kind of causal chain.
Dan-Cohen and Craver give the example of sonograms to explain pragmatic artifacts. The artifacts produced in sonograms are useful in identifying kidney stones, which produce sparkly images on the screen. For an anatomy researcher, the sparkles are artifacts, as they are not present in the kidneys themselves. For a medical researcher, however, the sparkles indicate the existence of something scattering the sound waves in a distinctive way.
The nuance of Dan-Cohen and Craver’s two-pronged definition emerges from the interplay between their areas of expertise. Craver’s philosophical work in the past has focused on the crucial role of causality in scientific explanation. As a sociocultural anthropologist, Dan-Cohen studies how subcultures within scientific fields establish rules for how science works.
“I think there was a pull in the direction of ‘artifacts are artifacts are artifacts’ and the idea that artifacts are negotiated in practice,” Dan-Cohen said. “We approached the problem from two very different vantage points as a philosopher and an anthropologist, and those differing perspectives influenced our findings in ways that were rewarding and ultimately reflected in the final product.”
