That's what I initially followed, too. But there does seem to be a better way - I've just installed CHARL-E [0] (mentioned elsewhere in this thread) and it was trivial to set up. Literally download the dmg, drag into Applications, and run.
It's an electron app and you can either download it with the weights, or without and add them separately.
Using that it just took slightly over 5 minutes on my 8GB so it's a little bit quicker for me. Maybe the code has improved since I cloned stuff over a week ago, or maybe it's just different system resources when it is run. Either way, it looks like the easy way I've been waiting for.
I've just been following the steps here with default settings: https://replicate.com/blog/run-stable-diffusion-on-m1-mac, but maybe there's a better way to run it at this point?