Patrick Huie

Senior Software Engineer.

AI and bio-terrorism

·

I recently ran into https://80000hours.org/podcast/episodes/kevin-esvelt-stealth-wildfire-pandemics/#whether-we-can-sufficiently-improve-our-defences-010901.

Kevin Esvelt has a unique view on defense against bio-terrorism: the answer to human designed pandemics lies more with physics and mathematics than biology.

"Assume there is an adversary. Assume the adversary is smarter than you, better resourced than you, and is operating in the future with the benefit of technologies and advances that you don’t know and can’t imagine. And of course, they’ve had the opportunity to look at your defences after you construct them. So design accordingly."

That line of thinking is why Esvelt is pitching that the best solutions are those that "air-gap" humans from the bad biological actors.

An example he brings up is a PPE device (personal protective equipment) that uses positive air pressure to keep microbes out of your airspace. Or, widely deploying light bulbs with UV in them that denatures bacteria.

The issue is that first someone needs to pay for it, and second someone needs to manufacture it.

The supply chain for PPE will not just wake up one morning and be able to supply 100x the capacity. There are lead times. Factories need to setup purchasing agreements with their raw ingredients and source the additional demand if their current suppliers can't rise to the occasion. Factories need to prepare the automation and process to scale up. Then they need to actually start manufacturing. There's a fixed amount of product a factory can generate based on the capacity of its suppliers, its distance from its suppliers, the amount of energy it has access to, and its ability to deploy more production lines - i.e. can we fit another machine in this corner of the plant? another line of 15 people for hand assembly?

Let's look at the surgical mask market during COVID19. Publicly available datasets on market sizing lack any kind of sensible consensus (market sizing varies from 200M to 20Bn), so instead let's look at 3M's public filings.

"In total, 3M produced and delivered 2 billion respirators globally in 2020, with approximately half in the United States.Today, we are producing respirators at a rate of 2.5 billion per year, a 4-fold increase versus 2019." - 3M 2020 Annual Filing

"3M has increased its annual production rate to 2.5 billion N95 respirators, a fourfold increase over 2019, executives said on their Q4 earnings call Tuesday. In 2020, the company produced 2 billion respirators globally.

Roman (CEO) said production lines are operating at full capacity as 3M anticipates continued strong demand for respirators." - 3M 2021 Filing covered by Supply Chain Dive

After 2021, 3M stopped disclosing numbers on mask productions. They did however comment in filings that the winding down of COVID-19 was a headwind and contributed to their losses throughought 2022 and 2023.

Let's assume that 3M tried to service the full extent of the increased demand for masks. From 2019 to 2020 they were able to increase manufacturing of an already well-manufactured and short cycle good only 4x. In the years following, they only eked out an additional multiple, hitting less than 5x the 2019 capacity.

If 3M could have cornered the market on masks by increasing their supply to 10x they would have. They certainly had the cash reserves for it (5.1Bn in 2020). Its better to tolerate slightly worse margins to capture dominant market share. i.e., the bottleneck was manufacturing capacity + materials.

N95 masks are slightly more complicated to manufacture than a textile, and today cost about $2 USD a pop.

Now imagine an astronaut helmet that provides an environment for you to breathe in with positive pressure to keep out diseases. They're $2.3k a pop currently and include plastics, a hose, filters, battery system.

Positive Pressure PPE

Ok, so maybe each household only gets one. And you only use it when you absolutely need to leave the house, like to flip your neighbor off from behind your fence when he mows the lawn at 6 in the morning. So while a mask can last a few weeks, your apocalyse lungs last about a year. We'll still need several Bn a year of these produced. How many years will that take?

The issue with counting on physical goods for harm reduction is that manufacturing takes time. And for widely contagious respiratory diseases with short incubation periods, that manufacturing lead time is all the disease needs to do a lot of damage.

What about the nuclear anaology? The principles of mutually assured destruction don't apply because the cost of sequencing genomes and making edits to genomes has fallen rapidly. The tools to do it can be acquired by an advanced high school chemsistry lab. It only costs several thousand dollars to assemble horsepox. Meanwhile, it costs a little south of 100M to build a nuclear missile in the west. Plus, through sanctions and other tight controls, the west have made it very difficult for state actors they don't like to build nuclear missiles. Mutually assured destruction doesn't work when your enemy isn't a world leader who would (generally) like himself and his friends to live another day at the top of the food chain. The potential enemy here is someone with very bad mental health who wants to end it all for everyone.

So then, what are we left with? Essentially, vaccines. But, if it possible, the most powerful vaccines we can think of - ones that are able to prep the body for permutations of diseases we haven't yet discovered, while still leaving the average person healthy and happy.

One alternative route is locking down the proliferation of AI, but that's a non starter. DeepSeek has already published open sourced models that are powerful enough for an individual evil actor to do quite a bit of damage.