After "the letter" arguing Integrated Information Theory (IIT) is a pseudoscience, many people responded by condemning the letter writers for not keeping up with their philosophy of science. Popper's falsification theory is out, went the argument, since the writings of Kuhn and Lakatos. I think this argument is too glib. But, also, I think many people miss that much of the best work in the philosophy of science recently has been historical, focusing on specific episodes rather than making cases for whole new theories. The important point of these texts is on how scientific disputes often include an important philosophical element--one that often makes it difficult for outsiders to fully understand. One text I kept thinking of, in this regard, was Shapin and Schaffer's Leviathan and the Air Pump.
Shapin and Schaffer's text is fundamentally about the value of empirical studies, something that is not always taken as probative. There is Boyle, a standard hero of science performing experiments using the air pump and, in the process, building up a community of scientists committed to using experiments to build theories. But there is also the strange figure of Hobbes, who rejects these experiments and their results based on metaphysical assumptions. Specifically, Hobbes argued vacuums couldn't exist by definition--and, thus, the experimental results of the air pump which suggested a vacuum in nature should be written off. Effectively, certain metaphysics beats out the probabilistic, often inaccurate findings of mere mortals playing with faulty machines. This led Hobbes on a quest to prove--using metaphysical arguments, definitions, and deductions--the success of his views and the glaring inadequacies of the alternatives.
I think this text sheds light on a central issue in the critique of IIT. The theory begins from a particular metaphysical understanding of our phenomenal experience--that ineffable qualia exist, that our experience is rich and manifold, that consciousness gives us an integrated whole greater than the parts, that consciousness is not a function we do but a structure we have--and regards these as axiomatically certain. Their argument about the physical instantiation of this experience, called "Phi," is secondary; it is something we infer about the physical world from our primary, indubitable experience of consciousness. Because the "axioms" of conscious experience are certain and indubitable, the physical facts of consciousness are always secondary and can always be amended as needed. This is why there has been so much fluidity in how Phi is defined over the years, what implications to take from it, how to interpret data about it, and whether it includes inactive logic gates or plants or not.
The physical data has been disproven, shown to be incoherent, or resulting in absurdities numerous times, only for the theory to sprout anew in a different form ripe for further testing. As Scott Aaronson complains, after sharing one such counterexample with Tononi:
Tononi’s response, as best as I can reconstruct it, was that it’s wrong to approach IIT like a mathematician; instead one needs to start “from the inside,” with the phenomenology of consciousness, and only then try to build general theories that can be tested against counterexamples. This response perplexed me: of course you can start from phenomenology, or from anything else you like, when constructing your theory of consciousness. However, once your theory has been constructed, surely it’s then fair game for others to try to refute it with counterexamples? And surely the theory should be judged, like anything else in science or philosophy, by how well it withstands such attacks?
Aaronson's frustration is that of much of the field--not just at this moment but for over a decade. Evidence just doesn't matter in the dispute, at least not in any conventional sense. There is something un-scientific about it, but it is often difficult to specify what is wrong because the theoreticians seem indifferent to criticism.
Hobbes's approach gives us a way of thinking about what is happening. In the Shapin and Schaffer book, he is shown as unshakably certain about a demonstrably false metaphysics. The empirical successes simply didn't rank in his eyes because, at best, they are contingent developments which depend on fallible machines and observers. As such, Hobbes has no problem dogmatically denouncing this whole approach. This has struck many scientists as the problem facing IIT: the neural evidence for disputing the theory has been marshalled, time and again, and yet the IIT theorists continue to be nonplussed. There is simply no assailing the metaphysical position with empirical evidence.
This highlights a second dimension to the airpump story. Hobbes's metaphysical commitments underlie his political commitments. As a result, he took the experimental scientists work on proving the existence of a vacuum as politically dangerous, threatening not just the certainty of science but also the kind of political system Hobbes found necessary. A multitude of empirical studies that often proved vexing and confusing struck him as scary in itself, suggesting uncertainty would become a norm. The separation of a scientific authority from political authority, moreover, made possible another way the state's control might fragment.
This, too, has a corollary. As Tononi tells Aaronson, empirical enquiry is the wrong approach altogether for making sense of consciousness; we need to start from the inside. IIT researchers focus on consciousness is often explained as part of a "new science," one that begins with consciousness but eventually is seen as grounding a new idealism--a belief that mind is more fundamental than matter, that consciousness grounds our world itself. The IIT researchers argue their certainty about the axioms of consciousness provide more certainty--a phenomenological, metaphysical certainty--than any empirical theory could. They also contend the other theories, even in principle, can't really explain the axioms; they are all concerned with the "function" of consciousness, rather than the experience-itself. As such, IIT cannot be disproven by physical evidence even in principle. The "fact" of consciousness is indubitable; physical evidence is, by its nature, derivative and uncertain compared to the certainty of consciousness.
It is an ambitious goal--and one that licenses discounting inconvenient physical facts as not really important in the grand scheme of things. And, in its anti-materialism, it is effectively anti-scientific. Science has, for some time, been dominated by naturalists, if not explicitly than at least instrumentally. Even if a scientist takes no metaphysical stance on the nature of the universe, their research largely assumes that further empirical investigation will explain what is at stake (at least to some degree) and will invalidate current assumptions (whatever they are). There may be little method shared by all sciences, but there is a broad assumption that further evidence is how we'll make sense of the world.
And the success of this approach in physics, chemistry, biology, psychology, and the perceptual sciences have all reinforced this assumption. It is only a small step, from this point of view, to assume that further empirical data will help to unravel consciousness. After all, we know much about color, emotion, and pain from studying the nervous system; why would awareness of color, emotion, and pain require non-material appeals? Why would we need an appeal to mathematics, physics, or even mental reality to explain it? The guiding assumption among scientists of consciousness is that their theories are all evidence-dependent, and it is open for further evidence to invalidate the theory down the line.
Most scientists of consciousness are modest about how much they are getting right at the moment; they all largely assume and accept modifications to their theory are inevitable--and some of them will be wrong. That this doesn't apply to IIT--that the theory cannot be wrong on their lights--makes it something outside of science. Does this make it pseudoscience? Questionable philosophical speculation? I think probably both, though labels and semantics are contestable. But treating it as a science is incorrect. It doesn't want to be a science, at least in the sense of any other working, accepted science. Consciousness might strike some folks as the most certain, undeniable fact of all--and thus capable of overturning everything else we know about the world. That's a perfectly acceptable way to feel. But allowing any feeling to take the status of a fact is unscientific.