On consensus
Recently, in my fellowship interview I was asked how I pick what I want to work on.
The first half of my answer was that I pick projects that aren’t more than a couple steps removed from something practical. Aka, am I developing a technology that either directly helps people, or will allow someone to develop a drug that previously was impossible.
The second half of my answer was off the cuff, but I said that I look for the word “consensus” because that means something is missing. I saw the eyebrows of one of the interviewers go up. I can think of two types of expert consensuses.
Inherent in the concept of a consensus is a gap. No scientific theory that is well evidenced and entirely internally consistent will be presented as the consensus of experts. The structure of DNA being a double helix, for example, was discovered by x-ray crystallography. The polymerase chain reaction (PCR) relies on this structure being correct, and it works spectacularly. That various enzymes that act differently on double stranded DNA than single stranded, which allow for construction of DNA confirmable by DNA sequencing methods, provide good evidence that we understand that fundamental truth well. DNA is a double stranded helix. No two ways about it, no consensus required.
The first type of consensus is filling a gap that is inaccessible by current technology but has never been contradicted. This type of consensus is likely right more often than wrong. A perfect recent example comes from the discovery of gravitational waves. Gravitational waves were predicted by experts since the late 1800’s, and formalized by Einstein in his theory of general relativity. They are ripples in spacetime itself. The detection of gravitational waves took 100 years and the construction of the most precise measurement tool ever conceived by humanity. LIGO, a perfectly straight 4.4 kilometer tube, run in a vacuum one trillionth the pressure outside of it, with active and passive damping, houses the most accurate laser ever made, can measure gravitational waves 1/10,000 the width of a proton. It generates terabytes of data every day.
In 2016 it measured a gravitational wave caused by the collision of two black holes. The collision of the black holes released more than 50 times the power of the stars in all of the observable universe combined, and warped spacetime itself. The expert consensus was proven correct.
Now, the other type of consensus is where it gets fun. The second type of consensus seeks to neatly tie together pieces of data that have been contradicted. This type of consensus mostly exists to make a narrative and hand-wave away data that would prove the consensus wrong. If you see the words “it is the consensus of experts”, “experts agree”, or “it is accepted” about a topic where no superhuman measurement techniques are required, it’s wrong more often than it’s right.
About to fire some shots at Nobel Prize winner Dr. Lefkowitz. He was a crucial part of characterizing G-Protein Coupled Receptors (GPCRs), a class of membrane proteins that is responsible for detecting all sorts of hormones, psychedelic and other drugs, environmental conditions, compounds, etc. They’re a diverse, complex, precise and beautiful class of proteins.
When they detect whatever, they primarily signal through two pathways, G-Proteins, and B-Arrestins (but mostly G-Proteins). At some point, Dr. Lefkowitz found some compounds that seemed to activate one of the pathways and not the other. This was called a “biased compound.”
Companies were founded based on the concept of a biased compound, and massive pharmaceutical companies took the bait as well, starting on programs to make biased compounds. (Trevana, below, is publicly traded, and I can’t give financial advice, but maybe short them.)
The idea is tantalizing, that we can reap the benefits of drugging this kind of protein, without some of the side effects and mechanisms of tolerance formation. It became a matter of consensus that biased agonists exist for many if not all GPCRs and finding these compounds all but guaranteed you a big publication and maybe a lot of money.
But a number of problems arise. The assays that measure signaling through these pathways are subject to a lot of artifacts and cross talk from other GPCRs. They rely heavily on amplification of signals, so small discrepancies can have an outsized effect in interpreting the data. The compounds that were pitched as canonical examples of biased signaling turned out to just be weak drugs with some off target effects. Any crystallization data (by people trying to prove biased signaling real) failed to find a difference above the resolution of the crystal structure. The linked paper is really amazing for it’s hand waving to find a way to fit contradictory data into their consensus view of biased signaling, 0.77 Angstroms is the absolute maximum resolution for a protein crystal, and the protein structure is identical.
Oopsie.
Many scientists have not abandoned the idea, thinking that we just don’t have good enough ways to measure signaling, but it strikes me as unlikely. This review of “biased signaling” in the opioid receptor is as close to excoriating as you can get in science.
It is my prediction that biased signaling will be completely discredited in the near future. It’s already falling apart, and when a bunch of clinical trials fail it’s hard to plug your ears and hum any longer.
When you hear the words “it is the expert consensus” that something is true, your first question should be “why?”