No, the podcast can absolutely missrepresent the thing that it’s sumarizing. The podcast also adds commentary, and I think it’s especially this commentary that I find unreliable.

I think that it is impressive, but not necessarily that useful? In particular, you can’t really trust what they’re saying to be accurate so it doesn’t actually give you that much usable information.

Very cool, but I’m not sure what I would actually use it for.

Yeah thought that might be the case! It’s just a thing that a lot of people have misconceptions about so it’s something that I have a bit of a knee jerk reaction to.

There are a couple of reasons that might not work:

• Maybe we’ll asymptotically approach a point that is lower than human-level cognitive capabilities
• Gradual improvements are susceptible to getting stuck in a local maxima. This is a problem in evolution as well. A lot of animals could in theory evolve, say, human level intelligence in principle, but to reach that point they’d have to go through a bunch of intermediate steps that lead to worse fitness. Gradual scientific improvements are a bit like evolution in this way.
• We also lose knowledge over time. Something as dramatic as a nuclear war would significantly set back the progress in developing AGI, but something less dramatic might also lead to us forgetting things that we’ve already learned.

To be clear, most of the arguments I’m making aren’t really about AGI specifically but about humanities capability to develop arbitrary in principle feasible technologies in general.

Another possibility is that humans just aren’t smart enough to figure out AGI. While I’m sure that we will continue incrementally improving technology in some form, it’s not at all self-evident that these improvements will eventually add up to AGI.

A breakthrough in quantum computing wouldn’t necessarily help. QC isn’t faster than classical computing in the general case, it just happens to be for a few specific algorithms (e.g. factoring numbers). It’s not impossible that a QC breakthrough might speed up training AI models (although to my knowledge we don’t have any reason to believe that it would) and maybe that’s what you’re referring to, but there’s a widespread misconception that Quantum computers are essentially non-deterministic turing machines that “evaluate all possible states at the same time” which isn’t the case.

You could still automatically delete all new posts and comments or something like that I suppose

If anything I think that the current rust discourse is a fad. I’m not sure what it is about rust that makes people have so strong opinions about it but I can’t wait for it to become a “normal” language so that people can chill about it a bit.