Finally got some integration tests that allocate cloud machines and install then test software on them. All in multiple clouds. Happy to have that level of testing working in our product.
What I've learned is that the tools for every vendor are bad, unreliable and sometimes even racy between multiple calls. I started the project so naive, and now the scripts are downright paranoid, but they work.