LucasF dangvantuan commited on
Commit
40a0ebc
0 Parent(s):

Duplicate from dangvantuan/sentence-camembert-large

Browse files

Co-authored-by: Van Tuan DANG <[email protected]>

.gitattributes ADDED
@@ -0,0 +1,28 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ *.7z filter=lfs diff=lfs merge=lfs -text
2
+ *.arrow filter=lfs diff=lfs merge=lfs -text
3
+ *.bin filter=lfs diff=lfs merge=lfs -text
4
+ *.bin.* filter=lfs diff=lfs merge=lfs -text
5
+ *.bz2 filter=lfs diff=lfs merge=lfs -text
6
+ *.ftz filter=lfs diff=lfs merge=lfs -text
7
+ *.gz filter=lfs diff=lfs merge=lfs -text
8
+ *.h5 filter=lfs diff=lfs merge=lfs -text
9
+ *.joblib filter=lfs diff=lfs merge=lfs -text
10
+ *.lfs.* filter=lfs diff=lfs merge=lfs -text
11
+ *.model filter=lfs diff=lfs merge=lfs -text
12
+ *.msgpack filter=lfs diff=lfs merge=lfs -text
13
+ *.onnx filter=lfs diff=lfs merge=lfs -text
14
+ *.ot filter=lfs diff=lfs merge=lfs -text
15
+ *.parquet filter=lfs diff=lfs merge=lfs -text
16
+ *.pb filter=lfs diff=lfs merge=lfs -text
17
+ *.pt filter=lfs diff=lfs merge=lfs -text
18
+ *.pth filter=lfs diff=lfs merge=lfs -text
19
+ *.rar filter=lfs diff=lfs merge=lfs -text
20
+ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
21
+ *.tar.* filter=lfs diff=lfs merge=lfs -text
22
+ *.tflite filter=lfs diff=lfs merge=lfs -text
23
+ *.tgz filter=lfs diff=lfs merge=lfs -text
24
+ *.xz filter=lfs diff=lfs merge=lfs -text
25
+ *.zip filter=lfs diff=lfs merge=lfs -text
26
+ *.zstandard filter=lfs diff=lfs merge=lfs -text
27
+ *tfevents* filter=lfs diff=lfs merge=lfs -text
28
+ model.safetensors filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,125 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ pipeline_tag: sentence-similarity
3
+ language: fr
4
+ datasets:
5
+ - stsb_multi_mt
6
+ tags:
7
+ - Text
8
+ - Sentence Similarity
9
+ - Sentence-Embedding
10
+ - camembert-large
11
+ license: apache-2.0
12
+ model-index:
13
+ - name: sentence-camembert-large by Van Tuan DANG
14
+ results:
15
+ - task:
16
+ name: Sentence-Embedding
17
+ type: Text Similarity
18
+ dataset:
19
+ name: Text Similarity fr
20
+ type: stsb_multi_mt
21
+ args: fr
22
+ metrics:
23
+ - name: Test Pearson correlation coefficient
24
+ type: Pearson_correlation_coefficient
25
+ value: xx.xx
26
+ library_name: sentence-transformers
27
+ ---
28
+ ## Description:
29
+ [**Sentence-CamemBERT-Large**](https://huggingface.co/dangvantuan/sentence-camembert-large) is the Embedding Model for French developed by [La Javaness](https://www.lajavaness.com/). The purpose of this embedding model is to represent the content and semantics of a French sentence in a mathematical vector which allows it to understand the meaning of the text-beyond individual words in queries and documents, offering a powerful semantic search.
30
+ ## Pre-trained sentence embedding models are state-of-the-art of Sentence Embeddings for French.
31
+ The model is Fine-tuned using pre-trained [facebook/camembert-large](https://huggingface.co/camembert/camembert-large) and
32
+ [Siamese BERT-Networks with 'sentences-transformers'](https://www.sbert.net/) on dataset [stsb](https://huggingface.co/datasets/stsb_multi_mt/viewer/fr/train)
33
+
34
+
35
+ ## Usage
36
+ The model can be used directly (without a language model) as follows:
37
+
38
+ ```python
39
+ from sentence_transformers import SentenceTransformer
40
+ model = SentenceTransformer("dangvantuan/sentence-camembert-large")
41
+
42
+ sentences = ["Un avion est en train de décoller.",
43
+ "Un homme joue d'une grande flûte.",
44
+ "Un homme étale du fromage râpé sur une pizza.",
45
+ "Une personne jette un chat au plafond.",
46
+ "Une personne est en train de plier un morceau de papier.",
47
+ ]
48
+
49
+ embeddings = model.encode(sentences)
50
+ ```
51
+
52
+ ## Evaluation
53
+ The model can be evaluated as follows on the French test data of stsb.
54
+
55
+ ```python
56
+ from sentence_transformers import SentenceTransformer
57
+ from sentence_transformers.readers import InputExample
58
+ from datasets import load_dataset
59
+ def convert_dataset(dataset):
60
+ dataset_samples=[]
61
+ for df in dataset:
62
+ score = float(df['similarity_score'])/5.0 # Normalize score to range 0 ... 1
63
+ inp_example = InputExample(texts=[df['sentence1'],
64
+ df['sentence2']], label=score)
65
+ dataset_samples.append(inp_example)
66
+ return dataset_samples
67
+
68
+ # Loading the dataset for evaluation
69
+ df_dev = load_dataset("stsb_multi_mt", name="fr", split="dev")
70
+ df_test = load_dataset("stsb_multi_mt", name="fr", split="test")
71
+
72
+ # Convert the dataset for evaluation
73
+
74
+ # For Dev set:
75
+ dev_samples = convert_dataset(df_dev)
76
+ val_evaluator = EmbeddingSimilarityEvaluator.from_input_examples(dev_samples, name='sts-dev')
77
+ val_evaluator(model, output_path="./")
78
+
79
+ # For Test set:
80
+ test_samples = convert_dataset(df_test)
81
+ test_evaluator = EmbeddingSimilarityEvaluator.from_input_examples(test_samples, name='sts-test')
82
+ test_evaluator(model, output_path="./")
83
+ ```
84
+
85
+ **Test Result**:
86
+ The performance is measured using Pearson and Spearman correlation:
87
+ - On dev
88
+
89
+
90
+ | Model | Pearson correlation | Spearman correlation | #params |
91
+ | ------------- | ------------- | ------------- |------------- |
92
+ | [dangvantuan/sentence-camembert-large](https://huggingface.co/dangvantuan/sentence-camembert-large)| 88.2 |88.02 | 336M|
93
+ | [dangvantuan/sentence-camembert-base](https://huggingface.co/dangvantuan/sentence-camembert-base) | 86.73|86.54 | 110M |
94
+ | [distiluse-base-multilingual-cased](https://huggingface.co/sentence-transformers/distiluse-base-multilingual-cased) | 79.22 | 79.16|135M |
95
+ | [GPT-3 (text-davinci-003)](https://platform.openai.com/docs/models) | 85 | NaN|175B |
96
+ | [GPT-(text-embedding-ada-002)](https://platform.openai.com/docs/models) | 79.75 | 80.44|NaN |
97
+ - On test
98
+
99
+
100
+ | Model | Pearson correlation | Spearman correlation |
101
+ | ------------- | ------------- | ------------- |
102
+ | [dangvantuan/sentence-camembert-large](https://huggingface.co/dangvantuan/sentence-camembert-large)| 85.9 | 85.8|
103
+ | [dangvantuan/sentence-camembert-base](https://huggingface.co/dangvantuan/sentence-camembert-base)| 82.36 | 81.64|
104
+ | [distiluse-base-multilingual-cased](https://huggingface.co/sentence-transformers/distiluse-base-multilingual-cased) | 78.62 | 77.48|
105
+ | [GPT-3 (text-davinci-003)](https://platform.openai.com/docs/models) | 82 | NaN|175B |
106
+ | [GPT-(text-embedding-ada-002)](https://platform.openai.com/docs/models) | 79.05 | 77.56|NaN |
107
+
108
+
109
+ ## Citation
110
+
111
+
112
+ @article{reimers2019sentence,
113
+ title={Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks},
114
+ author={Nils Reimers, Iryna Gurevych},
115
+ journal={https://arxiv.org/abs/1908.10084},
116
+ year={2019}
117
+ }
118
+
119
+
120
+ @article{martin2020camembert,
121
+ title={CamemBERT: a Tasty French Language Mode},
122
+ author={Martin, Louis and Muller, Benjamin and Su{\'a}rez, Pedro Javier Ortiz and Dupont, Yoann and Romary, Laurent and de la Clergerie, {\'E}ric Villemonte and Seddah, Djam{\'e} and Sagot, Beno{\^\i}t},
123
+ journal={Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics},
124
+ year={2020}
125
+ }
config.json ADDED
@@ -0,0 +1,27 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "camembert/camembert-large",
3
+ "architectures": [
4
+ "CamembertModel"
5
+ ],
6
+ "attention_probs_dropout_prob": 0.1,
7
+ "bos_token_id": 0,
8
+ "eos_token_id": 2,
9
+ "gradient_checkpointing": false,
10
+ "hidden_act": "gelu",
11
+ "hidden_dropout_prob": 0.1,
12
+ "hidden_size": 1024,
13
+ "initializer_range": 0.02,
14
+ "intermediate_size": 4096,
15
+ "layer_norm_eps": 1e-05,
16
+ "max_position_embeddings": 514,
17
+ "model_type": "camembert",
18
+ "num_attention_heads": 16,
19
+ "num_hidden_layers": 24,
20
+ "output_past": true,
21
+ "pad_token_id": 1,
22
+ "position_embedding_type": "absolute",
23
+ "transformers_version": "4.3.3",
24
+ "type_vocab_size": 1,
25
+ "use_cache": true,
26
+ "vocab_size": 32005
27
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:36244c6e193462fc06261192a3a7f3c4ca4523b31d4e55a9f8ebe500d11fb2c5
3
+ size 1346695096
pytorch_model.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e0d665ddffedb6ad4a11604948fc9a04b4b5574389596cc43a934c422e9f9efd
3
+ size 1346812279
sentence_bert_config.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ {
2
+ "max_seq_length": 128
3
+ }
sentencepiece.bpe.model ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f98f266fdc548c94216aaadc13ffaaafacf0c8793303e2195322d954549ea261
3
+ size 808767
special_tokens_map.json ADDED
@@ -0,0 +1 @@
 
 
1
+ {"bos_token": "<s>", "eos_token": "</s>", "unk_token": "<unk>", "sep_token": "</s>", "pad_token": "<pad>", "cls_token": "<s>", "mask_token": {"content": "<mask>", "single_word": false, "lstrip": true, "rstrip": false, "normalized": true}, "additional_special_tokens": ["<s>NOTUSED", "</s>NOTUSED"]}
tf_model.h5 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fe497cd8d5b5f48c64919667b78dc77666b8e42a691f68a7d98477dec34c639f
3
+ size 1347193304
tokenizer_config.json ADDED
@@ -0,0 +1 @@
 
 
1
+ {"bos_token": "<s>", "eos_token": "</s>", "sep_token": "</s>", "cls_token": "<s>", "unk_token": "<unk>", "pad_token": "<pad>", "mask_token": {"content": "<mask>", "single_word": false, "lstrip": true, "rstrip": false, "normalized": true, "__type": "AddedToken"}, "additional_special_tokens": ["<s>NOTUSED", "</s>NOTUSED"], "special_tokens_map_file": null, "name_or_path": "camembert/camembert-large"}