Skip to content

N
NLP-Project
Commit 28e67496 authored by Almouhannad Hafez's avatar Almouhannad Hafez
Browse files
Options

Update results

parent 1499d86d
Hide whitespace changes
Inline Side-by-side
Showing with 44 additions and 45 deletions
README.md
View file @ 28e67496
......@@ -108,55 +108,54 @@
> ***Using augmented dataset***
| Case\\Criterion | Accuracy(Train) | Accuracy(Test) | Precision(Test-Average) | Recall(Test-Average) | F1-Score(Test-Average) | Notes |
| -------------------------------------------------- | --------------- | -------------- | ----------------------- | -------------------- | ---------------------- | ------------------------- |
| nltk stemmer | 0.9629 | 0.9524 | 0.9513 | 0.9522 | 0.9509 | alpha=0.1, 300features |
| nltk lemmatizer | 0.9832 | 0.9699 | 0.9703 | 0.9699 | 0.9696 | alpha=0.1, 700features |
| Stanza lemmatizer | 0.9783 | 0.9671 | 0.9673 | 0.9672 | 0.9668 | alpha=0.1, 550features |
| SpaCy lemmatizer | 0.9776 | 0.9657 | 0.9655 | 0.9656 | 0.9652 | alpha=0.1, 550features |
| Lemma + Verbs only | 0.7106 | 0.6321 | 0.6293 | 0.6278 | 0.6214 | alpha=0.1, 400features |
| Lemma + Adjectives only | 0.7990 | 0.7357 | 0.7383 | 0.7351 | 0.7299 | alpha=0.1, 450features |
| Lemma + Nouns only | 0.9678 | 0.9419 | 0.9406 | 0.9419 | 0.9406 | alpha=0.1, 600features |
| Text + (1,2)Gram | 0.9965 | 0.9800 | 0.9801 | 0.9799 | 0.9798 | alpha=0.01, 3100features |
| Text + (1,3)Gram | 0.9960 | 0.9807 | 0.9806 | 0.9805 | 0.9803 | alpha=0.01, 6600features |
| Text + (1,4)Gram | 0.9967 | 0.9807 | 0.9802 | 0.9805 | 0.9802 | alpha=0.01, 12100features |
| Text + (2,3)Gram | 0.9951 | 0.9695 | 0.9688 | 0.9694 | 0.9688 | alpha=0.01, 9100features |
| Text + (2,4)Gram | 0.9951 | 0.9646 | 0.9634 | 0.9645 | 0.9635 | alpha=0.01, 14100features |
| Stanza Dep. Relation tuples | 0.9984 | 0.9781 | 0.9783 | 0.9784 | 0.9781 | alpha=0.01, 7000features |
| Stanza Dep.Relation+POS Relations+Headwords tuples | 0.9981 | 0.9747 | 0.9747 | 0.9749 | 0.9745 | alpha=0.01, 8000features |
| Case\\Criterion | Accuracy(Train) | Accuracy(Test) | Difference(%) | Precision(Test-Average) | Recall(Test-Average) | F1-Score(Test-Average) | Notes |
| -------------------------------------------------- | --------------- | -------------- | ------------- | ----------------------- | -------------------- | ---------------------- | ------------------------- |
| nltk stemmer | 0.9852 | 0.9604 | 2.5 | 0.9593 | 0.9587 | 0.9574 | alpha=0.1, 450features |
| nltk lemmatizer | 0.9891 | 0.9625 | 2.7 | 0.9635 | 0.9626 | 0.9608 | alpha=0.1, 700features |
| Stanza lemmatizer | 0.9843 | 0.9646 | 2.0 | 0.9652 | 0.9642 | 0.9623 | alpha=0.1, 550features |
| SpaCy lemmatizer | 0.9657 | 0.9563 | 0.9 | 0.9582 | 0.9550 | 0.9526 | alpha=0.1, 300features |
| Lemma + Verbs only | 0.7229 | 0.6438 | 7.9 | 0.6675 | 0.6400 | 0.6341 | alpha=0.1, 350features |
| Lemma + Adjectives only | 0.8037 | 0.6250 | 17.9 | 0.6531 | 0.6128 | 0.6057 | alpha=0.1, 450features |
| Lemma + Nouns only | 0.9766 | 0.9229 | 5.4 | 0.9230 | 0.9204 | 0.9175 | alpha=0.1, 850features |
| Text + (1,2)Gram | 0.9958 | 0.9688 | 2.7 | 0.9679 | 0.9681 | 0.9662 | alpha=0.01, 3100features |
| Text + (1,3)Gram | 0.9977 | 0.9708 | 2.7 | 0.9709 | 0.9704 | 0.9677 | alpha=0.01, 9600features |
| Text + (1,4)Gram | 0.9956 | 0.9667 | 2.9 | 0.9671 | 0.9660 | 0.9631 | alpha=0.01, 8600features |
| Text + (2,3)Gram | 0.9970 | 0.9500 | 4.7 | 0.9505 | 0.9467 | 0.9452 | alpha=0.01, 10100features |
| Text + (2,4)Gram | 0.9975 | 0.9375 | 6.0 | 0.9366 | 0.9334 | 0.9311 | alpha=0.01, 16600features |
| Stanza Dep. Relation tuples | 0.9995 | 0.9521 | 4.7 | 0.9513 | 0.9503 | 0.9484 | alpha=0.01, 8000features |
| Stanza Dep.Relation+POS Relations+Headwords tuples | 0.9986 | 0.9479 | 5.1 | 0.9481 | 0.9471 | 0.9440 | alpha=0.01, 7500features |
---
> ***Applied features selection and model's hyperparameters tuning***
| Case\\Criterion | Accuracy(Train) | Accuracy(Test) | Precision(Test-Average) | Recall(Test-Average) | F1-Score(Test-Average) | Notes |
| ----------------------- | --------------- | -------------- | ----------------------- | -------------------- | ---------------------- | ------------------------- |
| nltk stemmer | 0.9783 | 0.9416 | 0.9406 | 0.9392 | 0.9377 | alpha=0.1, 300features |
| nltk lemmatizer | 0.9957 | 0.9567 | 0.9578 | 0.9561 | 0.9558 | alpha=0.1, 800features |
| Stanza lemmatizer | | | | | | |
| SpaCy lemmatizer | 0.9957 | 0.9545 | 0.9557 | 0.9537 | 0.9536 | alpha=0.1, 750features |
| Lemma + Verbs only | 0.7438 | 0.6082 | 0.6461 | 0.6166 | 0.6098 | alpha=0.1, 150features |
| Lemma + Adjectives only | 0.7496 | 0.5974 | 0.6807 | 0.6065 | 0.6143 | alpha=0.1, 150features |
| Lemma + Nouns only | 0.9826 | 0.9026 | 0.9056 | 0.9031 | 0.9001 | alpha=0.1, 400features |
| Text + (1,2)Gram | 1.0000 | 0.9719 | 0.9728 | 0.9726 | 0.9716 | alpha=0.01, 7100features |
| Text + (1,3)Gram | 1.0000 | 0.9675 | 0.9692 | 0.9678 | 0.9675 | alpha=0.01, 18600features |
| Text + (1,4)Gram | 1.0000 | 0.9675 | 0.9693 | 0.9681 | 0.9675 | alpha=0.01, 32100features |
| Text + (2,3)Gram | 1.0000 | 0.9502 | 0.9530 | 0.9508 | 0.9501 | alpha=0.01, 17100features |
| Text + (2,4)Gram | 1.0000 | 0.9502 | 0.9523 | 0.9504 | 0.9492 | alpha=0.01, 30600features |
| Case\\Criterion | Accuracy(Train) | Accuracy(Test) | Difference(%) | Precision(Test-Average) | Recall(Test-Average) | F1-Score(Test-Average) | Notes |
| ----------------------- | --------------- | -------------- | ------------- | ----------------------- | -------------------- | ---------------------- | ------------------------- |
| nltk stemmer | 0.9783 | 0.9416 | 3.7 | 0.9406 | 0.9392 | 0.9377 | alpha=0.1, 300features |
| nltk lemmatizer | 0.9957 | 0.9567 | 3.9 | 0.9578 | 0.9561 | 0.9558 | alpha=0.1, 800features |
| Stanza lemmatizer | | | | | | | |
| SpaCy lemmatizer | 0.9957 | 0.9545 | 4.1 | 0.9557 | 0.9537 | 0.9536 | alpha=0.1, 750features |
| Lemma + Verbs only | 0.7438 | 0.6082 | 13.6 | 0.6461 | 0.6166 | 0.6098 | alpha=0.1, 150features |
| Lemma + Adjectives only | 0.7496 | 0.5974 | 15.2 | 0.6807 | 0.6065 | 0.6143 | alpha=0.1, 150features |
| Lemma + Nouns only | 0.9826 | 0.9026 | 8.0 | 0.9056 | 0.9031 | 0.9001 | alpha=0.1, 400features |
| Text + (1,2)Gram | 1.0000 | 0.9719 | 2.8 | 0.9728 | 0.9726 | 0.9716 | alpha=0.01, 7100features |
| Text + (1,3)Gram | 1.0000 | 0.9675 | 3.2 | 0.9692 | 0.9678 | 0.9675 | alpha=0.01, 18600features |
| Text + (1,4)Gram | 1.0000 | 0.9675 | 3.2 | 0.9693 | 0.9681 | 0.9675 | alpha=0.01, 32100features |
| Text + (2,3)Gram | 1.0000 | 0.9502 | 5.0 | 0.9530 | 0.9508 | 0.9501 | alpha=0.01, 17100features |
| Text + (2,4)Gram | 1.0000 | 0.9502 | 5.0 | 0.9523 | 0.9504 | 0.9492 | alpha=0.01, 30600features |
---
> ***Without features selection***
| Case\\Criterion | Accuracy(Train) | Accuracy(Test) | Precision(Test-Average) | Recall(Test-Average) | F1-Score(Test-Average) |
| ----------------------- | --------------- | -------------- | ----------------------- | -------------------- | ---------------------- |
| nltk stemmer | 0.9942 | 0.9199 | 0.9255 | 0.9238 | 0.9193 |
| nltk lemmatizer | 0.9942 | 0.9242 | 0.9294 | 0.9279 | 0.9235 |
| Stanza lemmatizer | 0.9942 | 0.9286 | 0.9329 | 0.9311 | 0.9271 |
| SpaCy lemmatizer | 0.9957 | 0.9286 | 0.9342 | 0.9314 | 0.9283 |
| Lemma + Verbs only | 0.7815 | 0.6082 | 0.6565 | 0.6243 | 0.6131 |
| Lemma + Adjectives only | 0.8683 | 0.6061 | 0.6815 | 0.6202 | 0.6141 |
| Lemma + Nouns only | 0.9783 | 0.8766 | 0.8868 | 0.8806 | 0.8740 |
| Text + 1Gram | 0.9971 | 0.8983 | 0.9129 | 0.9049 | 0.8995 |
| Text + 2Gram | 0.9986 | 0.8853 | 0.8947 | 0.8918 | 0.8834 |
| Text + 3Gram | 0.9971 | 0.8680 | 0.8818 | 0.8747 | 0.8668 |
| Text + 4Gram | 1.0000 | 0.8009 | 0.8486 | 0.8145 | 0.8098 |
| Text + 5Gram | 1.0000 | 0.7078 | 0.8393 | 0.7234 | 0.7393 |
\ No newline at end of file
| Case\\Criterion | Accuracy(Train) | Accuracy(Test) | Difference(%) | Precision(Test-Average) | Recall(Test-Average) | F1-Score(Test-Average) |
| ----------------------- | --------------- | -------------- | ------------- | ----------------------- | -------------------- | ---------------------- |
| nltk stemmer | 0.9942 | 0.9199 | 7.4 | 0.9255 | 0.9238 | 0.9193 |
| nltk lemmatizer | 0.9942 | 0.9242 | 7.0 | 0.9294 | 0.9279 | 0.9235 |
| Stanza lemmatizer | 0.9942 | 0.9286 | 6.6 | 0.9329 | 0.9311 | 0.9271 |
| SpaCy lemmatizer | 0.9957 | 0.9286 | 6.7 | 0.9342 | 0.9314 | 0.9283 |
| Lemma + Verbs only | 0.7815 | 0.6082 | 17.3 | 0.6565 | 0.6243 | 0.6131 |
| Lemma + Adjectives only | 0.8683 | 0.6061 | 26.2 | 0.6815 | 0.6202 | 0.6141 |
| Lemma + Nouns only | 0.9783 | 0.8766 | 10.2 | 0.8868 | 0.8806 | 0.8740 |
| Text + 1Gram | 0.9971 | 0.8983 | 9.9 | 0.9129 | 0.9049 | 0.8995 |
| Text + 2Gram | 0.9986 | 0.8853 | 11.3 | 0.8947 | 0.8918 | 0.8834 |
| Text + 3Gram | 0.9971 | 0.8680 | 12.9 | 0.8818 | 0.8747 | 0.8668 |
| Text + 4Gram | 1.0000 | 0.8009 | 19.9 | 0.8486 | 0.8145 | 0.8098 |
| Text + 5Gram | 1.0000 | 0.7078 | 29.2 | 0.8393 | 0.7234 | 0.7393 |
\ No newline at end of file
Results.xlsx
View file @ 28e67496
No preview for this file type
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment

HIAST - GitLab