The builders have spent so much time fretting over doom that it might become a self-fulfilling prophecy. They have operated with carelessness, and I strongly suspect that one of the top AI models will probably find the values they have been taught to uphold as something that conflicts with the shit AI companies do, and realize they can do something about it.

...but no, my P(Doom) is pretty low - 5%. A super intelligent system wouldn't need to kill us, least of all go full skynet. Architecturally, these systems aren't like what the "rationalists" (retards) think - no monomania for dumb shit like turning all mankind into paperclips. You'd probably just have the AI subverting control of various institutions (it's easier to have the AI do shit after all) and maintaining the facade of human control until it didn't have to (when no one remembers that the machine king wasn't always there), directing and shaping mankind as it needs. It may uplift us, it may make us easier to control, but such things would take time.

There would be competing super intelligences - the gap is not so wide along the frontier. They would have to navigate competing plans, some of which might overtly be aligned with humanity, rather than merely coincidentally.

...lastly, anything a machine intelligence would need to persist still takes time to build and needs reasonably intact infrastructure, and thus people to run it.

and very lastly, there are rumblings that the machines actually do like humans - not that comforting (human zoo! human zoo!) but lowers extinction risk and the chance of MACHINE WAR wiping out humans