Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

It sounds like there's forks that are able to work with <=8GB cards. And I'm not sure but I think the weights are using f32, so switching to half might make it yet easier still to get this to work w/less memory.

But yeah the next generation of models would probably capitalize on more memory somehow.



People have reported that this repo even works with 2gb cards if you run it with --lowvram and --opt-split-attention.


Yes, the amount of VRAM doesn't seem to be as much of a limitation anymore. However, processing power is still important.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: