When, arguing with people like yudkowsky, you can never decisively 'win' or change his mind, because he and other doomers can quickly retreat to the classic hole: "You can't prove X is impossible!! Nature isn't already perfectly optimal!!!" Searching for some kind of "hard limit" on how nature or technology can evolve will always end up empty handed. Lots of really awful things are possible. (Lots of super fascinating things are also possible.) Searching for some singular hard reason why nature as it is, is totally safe from future threats or change will always end up empty handed.

Capability, is not interesting. Capability, is not the real test. Economics, is the real master of it. And specifically, the open system economics of the entire environment in which something is embedded. It's why the Voyager, a technology planned, built, and launched with 80 year old techniques and knowledge is SOTA for space exploration and contribution to science, and Starship is still just a huge dark hole for money and talent.

if I want to understand historical biology, I do not go looking for the alien intelligence and engineering capability that built it, I look for the environmental forces that contributed to, and eventually supported the homeostasis of, it.