Add fp16/int8 weights

#1
by mkshing - opened

This PR enables to use this model with Colab Free plan by int8 quantization.
Here's the link to the demo in colab.

https://colab.research.google.com/github/mkshing/notebooks/blob/main/stabilityai_japanese_stablelm_alpha_7b.ipynb

mkshing changed pull request status to open

Great, thanks for working on this!

leemeng changed pull request status to merged

Sign up or log in to comment