Update README.md
Browse files
README.md
CHANGED
@@ -30,10 +30,12 @@ Merging seems like the way to go when it comes to training language models on a
|
|
30 |
|
31 |
## Model Description
|
32 |
|
33 |
-
This model was created by training LoRAs and
|
34 |
|
35 |
Incedentally, it seems uncensored. It was trained using the ChatML template and can be used with or without a system prompt.
|
36 |
|
|
|
|
|
37 |
## Apache-2.0 + WhiteRabbitNeo Extended Version
|
38 |
|
39 |
### Licence: Usage Restrictions
|
|
|
30 |
|
31 |
## Model Description
|
32 |
|
33 |
+
This model was created by training LoRAs ([rsLoRA](https://arxiv.org/abs/2312.03732) with [NEFTune](https://arxiv.org/abs/2310.05914) noise alpha at 5) and [DELLA merge](https://arxiv.org/abs/2406.11617) them. It saves space and time this way and the result is good. The WhiteRabbitNeo datasets are the focus in this one along with coding.
|
34 |
|
35 |
Incedentally, it seems uncensored. It was trained using the ChatML template and can be used with or without a system prompt.
|
36 |
|
37 |
+
I am very grateful for all the different components that made this model possible. From H2O and their danube models, through Huggingface and Llama-Factory for making the fine-tuning easy, and to all the great dataset creators. **Thank you!**
|
38 |
+
|
39 |
## Apache-2.0 + WhiteRabbitNeo Extended Version
|
40 |
|
41 |
### Licence: Usage Restrictions
|