When trial and error beat first principles thinking
I loved Physics in school. It was astonishing for me that a few fundamental principles could explain so much about the world. However, this love of Physics made me believe that understanding a problem from first principles was the only legitimate way of thinking. For me, understanding a subject meant learning the theory behind it thoroughly. It was only after working for a few years that I realized how much learning is by doing. I realized that in many fields a theory based approach has severe limitations. You have to do something, find out what works and then do more of it even though you may not know why it works. This is not just true in our personal and professional lives. Some of the most consequential scientific discoveries and inventions have come out of such approaches. I recently read the excellent book, The demon under the microscope* by Thomas Hager, which describes the invention of the first widely used antibiotic. It is astonishing how this medicine was used for many years without an understanding of how it worked.
Many of us have heard the story of Alexander Fleming and his discovery of penicillin. However, Penicillin was not the first antibiotic to be sold and used widely. That distinction belongs to a class of drugs called sulfa drugs and they originated in the labs of the company Bayer. Before we take a look at how this happened, let us take a look at the context.
Up until the second half of the 1930s, the cure for bacterial diseases was very limited. Vaccination had worked against some pathogens — mainly viruses. Some bacterial infections responded to what was called ‘serum therapy’. Drug manufacture would infect animals with particular strains of bacteria and then process the blood serum of the infected animals and use it as medicine for people. The antibodies in the serum of the animal would work against the bacteria in people too. Or at least that was the theory. In practice, there were many problems. Such as, the serum needed to be prepared for the exact strain of bacteria that had infected the patient or it would not work. Or that the serum preparation process would take time. The upshot of these and other problems was that for many diseases, doctors were reduced to providing nursing care and hope for the best.
Strep diseases in Europe and North America alone were estimated to kill 1.5 million victims a year. In a bad year, ‘childbed fever’ could kill up to one in four infected women. Meningitis would kill 98 of the 100 children infected! In many wars, infection in wounds and diseases in camps killed more people than the enemy did directly.
For many decades, scientists across the world had tried to find medicines that worked across infectious diseases and that could be manufactured in bulk. These attempts had worked in a few parasitic diseases, notably Malaria. However, attempts to discover chemicals that worked against bacterial infections hadn’t gone well and most laboratories had given up. Bayer was an exception. After all, it was their tropical diseases division that had discovered the synthetic quinine that worked against malaria.
Bayer set up a laboratory devoted to bacterial diseases. The laboratory would be headed by Gerhard Domagk, a physician and a pathologist. The chemical side would be in the hands of two chemists. Joseph Klarer worked full time on bacterial disease. Fritz Mietzch worked for the antibacterial efforts as well as for the existing tropical diseases effort. They would cook up new chemicals for Domagk to test. Domagk tested these chemicals as well as chemicals made by chemists in the tropical diseases division. The testing was carried out for a panel of diseases in test tubes (in vitro) as well as in animals (in vivo).
Bayer’s anti-bacteria unit was set up in 1927. By 1929, Domagk and his staff were testing 30 different chemicals per week. It took many years and testing of thousands of chemicals before getting a result. Klarer himself had made hundreds of chemicals. The first chemical to give a positive result was KL — 487 in 1931! After few other encouraging but not breakthrough results, in October of 1932, Klarer attached a sulfanilamide molecule to Azo dye. This chemical, KL — 695, worked! Klarer then tinkered around with this molecule and Domargk’s tests revealed a good candidate for patenting and eventual sales. Bayer sold the medicine as Prontosil.
Prontosil worked but no one knew why. Not only that, the drug worked in animals but not in the test tube. Again, no one knew why. This latter mystery was solved in another lab across the border in a French lab.
Soon after the news of Bayer’s success got out, the Pasteur Institute started testing molecules of Azo dyes and sulfanilamide. One day the lab got extra mice. The scientist in charge decided to test those mice with pure sulfanilamide almost on a whim. The next day, they found that the mice treated with sulfanilamide were doing great even though they had been infected with a virulent strain of Strep. The Azo dye had never been needed at all! In fact, the dye was the reason that the drug did not work in test tubes. Enzymes in the digestive tract of animals had broken down the bond between sulfanilamide and the Azo dye and allowed the former to do its work. Pure sulfanilamide worked just fine in test tubes.
There was a frenzy of trial and error experiments in many countries. A big discovery happened in a company across the channel in England. May and Baker were following the German model of making educated guesses. They tested variant after variant of sulfa in animals. In 1937 they got lucky. An assistant found a bottle containing a chemical called aminopyridine. The bottle had been bought by an employee who no longer worked for the company and nobody remembered why it had been bought. The assistant forged a bond between aminopyridine and sulfanilamide and tested it against a bacteria that caused Pneumonia. At the time, the sulfa drugs available did not work against this disease. Not only did the new chemical work for Pneumonia, but also it was later found to be more effective against many other diseases.
Sulfa and its derivatives were eventually replaced by Penicillin. But in the meantime, they saved millions of lives. Remember, the ‘childbed fever’ that could kill up to one in four infected women in a bad year? Hager says,
“By 1938, it was estimated that sulfa was saving the lives of ten thousand new mothers.”
In a trial of sulfa drugs against Meningitis, doctors saved nine out of ten children as against the earlier rate of two out of hundred.
Sulfa drugs transformed battlefield care.
“In World War I, infected wounds had killed hundreds and thousands of men — more soldiers, by one estimate, than died by enemy bullets. In World War II, however, the numbers would be so low that wound infections no longer presented a major medical problem.”
The mechanism of how sulfa drugs worked was discovered eight years after Klarer made the first molecule of Prontosil. Two researchers from London, Donald Woods and Paul Fildes, discovered what the drug did in bodies. It mimicked para-aminobenzoic (PBA) acid, a chemical essential in bacterial metabolism. When sulfa was around, bacteria would try to metabolize it rather than PBA and this was not very good for the bacteria. Finally, eight years after its discovery, there was an explanation of why sulfa worked. Of course, the discovery of the mechanism was not useless. It opened new areas of research.
“The “antimetabolite” approach that began with sulfa would yield a host of new therapies, including a number of drugs like methotrexate and 5-fluorouracil, used to treat cancer.”
To recap, a drug that saved millions of lives was discovered by the process of trial and error. Further trial and error made it simpler and also allowed it to be used for many other diseases. It was only years later that we discovered how it actually worked.
Just to be clear, the term ‘trial and error’ does not mean that the Bayer team was a group of ignorant workers randomly trying things. It was most certainly not the bunch of proverbial monkeys hammering on typewriters that typed out a Shakesperian tragedy by sheer luck. They were highly experienced professionals.
Domagk graduated from medical school with the best possible marks. Subsequently, he specialized in pathology. It was his work in immunology that prompted Bayer to recruit him. Joseph Klarer has been described by Hager as someone “with an inborn genius for lab work, a Mozart at the bench.” He had studied under a Nobel laureate. Fritz Mietzch “was a talented chemist, better grounded than Klarer in traditional techniques.” It is clear that There was a deep understanding of medicine, immunology and chemistry in their trial and error process.
However, there is no denying that the drug was not discovered through a first principles understanding. Discovery came first and theory came later. This is much more common in science and engineering then we may think. This is even more common in our professional and personal lives. As an entrepreneur, you may try out many things before a product clicks with a customer. As a sales person you may try many channels before discovering one that works for your product. You may need to try many diet and exercise routines before you find one that (sort of) works for you. There may be theoretical explanations for why the choices that worked for you actually worked for you. But for all practical purposes, you have very little access to that theory.
In too many areas, ‘doing something’ and ‘learning by doing’ is the only practical way of making progress. My Physics loving younger self took a lot of time to understand this.
Addendum: A reader, Satya, suggested an articulation that makes a lot of sense to me: “Theories help us limit choices.” It was not that the Bayer team and no theoretical basis. They understood that bacteria were causing the diseases they were trying to cure. They could isolate those bacteria and infect the test animals with them. They also believed that if a drug worked against bacteria in animals, it was likely to work in humans. They hoped (?) that a chemical could be found that would work as synthetic quinine had worked for malaria.
This article is part of the series — Tips for early / mid career analytical types.
“How to spot a bad expert” in this series has received a lot of attention.
Follow me on medium. Choose the option to get the stories delivered to your Inbox.
You can follow AskHow India (@AskHowIndia) or me (@YogeshUpadh) on twitter or on LinkedIn
*The Demon under the microscope from Hagel is highly recommended. As is The alchemy of air from the same author.
