Update README.md

README.md

@@ -30,16 +30,19 @@ This model requires the following prompt template:
 
 ## CHOICE OF MODELS
 
-Two sets of models are provided:
+Three sets of models are provided:
 
-* Groupsize = 1024
+* Groupsize = None
   * Should work reliably in 24GB VRAM
+  * Uses --act-order for the best possible inference quality given its lack of group_size.
+* Groupsize = 1024
+  * Theoretically higher inference accuracy
+  * May OOM on long context lengths in 24GB VRAM  
 * Groupsize = 128
   * Optimal setting for highest inference quality
-  * But may require more than 24GB VRAM, depending on response length
-  * In my testing it ran out of VRAM on a 24GB card around 1500 tokens returned.
+  * Will definitely need more than 24GB VRAM on longer context lengths (1000-1500+ tokens returned)
 
-For each model, two versions are available:
+For the 128g and 1024g models, two versions are available:
 * `compat.no-act-order.safetensor`
   * Works with all versions of GPTQ-for-LLaMa, including the version in text-generation-webui one-click-installers
 * `latest.act-order.safetensors`
@@ -50,10 +53,13 @@ For each model, two versions are available:
 
 I have used branches to separate the models. This means you can clone the branch you want and not got model files you don't need.
 
-* Branch: **main** = groupsize 1024, `compat.no-act-order.safetensor` file
-* Branch: **1024-latest** = groupsize 1024, `latest.no-act-order.safetensor` file
-* Branch: **128-compat** = groupsize 128, `compat.no-act-order.safetensor` file
-* Branch: **128-latest** = groupsize 128, `latest.no-act-order.safetensor` file
+If you have 24GB VRAM you are strongly recommended to use the file in `main`, with group_size = None.  This is fully compatible, and won't OOM.
+
+* Branch: **main** = groupsize None, `OpenAssistant-SFT-7-Llama-30B-GPTQ-4bit.safetensors` file
+* Branch: **1024-compat** = groupsize 1024, `compat.no-act-order.safetensors` file
+* Branch: **1024-latest** = groupsize 1024, `latest.act-order.safetensors` file
+* Branch: **128-compat** = groupsize 128, `compat.no-act-order.safetensors` file
+* Branch: **128-latest** = groupsize 128, `latest.act-order.safetensors` file
 
 ![branches](https://i.imgur.com/PdiHnLxm.png)
   
@@ -68,7 +74,7 @@ Open the text-generation-webui UI as normal.
 5. Click the **Refresh** icon next to **Model** in the top left.
 6. In the **Model drop-down**: choose the model you just downloaded, `OpenAssistant-SFT-7-Llama-30B-GPTQ`.
 7. If you see an error in the bottom right, ignore it - it's temporary.
-8. Fill out the `GPTQ parameters` on the right: `Bits = 4`, `Groupsize = 1024`, `model_type = Llama`
+8. Fill out the `GPTQ parameters` on the right: `Bits = 4`, `Groupsize = None`, `model_type = Llama`
 9. Click **Save settings for this model** in the top right.
 10. Click **Reload the Model** in the top right.
 11. Once it says it's loaded, click the **Text Generation tab** and enter a prompt!