Questions tagged [precision-recall]

1

votes
1

answer
525

Views

Trading in precision for better recall in Keras classification neural net

There's always a tradeoff between precision and recall. I'm dealing with a multi-class problem, where for some classes I have perfect precision but really low recall. Since for my problem false positives are less of an issue than missing true positives, I want reduce precision in favor of increasin...
megashigger
1

votes
2

answer
336

Views

Precision and recall in Fasttext

I am new in Fasttext. And I already have a few questions about this library, they may seem obvious to someone, but I really want to get the right intuition. Your help will be much appreciated. First of all, I'm talking about Text classification part of Fasttext. According to the tutorial which is pr...
Dilshat
1

votes
1

answer
208

Views

Compute Recall and Precision to evaluate CBIR system

I implemented a CBIR with SIFT combined with other feature-based algorithms (with OpenCV and Python3), now I have to evaluate how the combination of them (i.e. SIFT/SURF, ORB/BRISK...) perform. I found that I can use Precision |TP| / (|TP| + |FP|) and Recall |TP| / (|TP| + |FN|). I know that the TP...
RoRy
1

votes
0

answer
53

Views

Decision tree model with R, recall and precision

I made a decision tree model. I obtained recall and precision for my model. However, I need to obtain recall and precision for each node. How can I do that with R? My code: tree1
Ely91
1

votes
0

answer
58

Views

Identifying the evaluation measures for unranked lists (set)

I have a dataframe as follows: Column 0 (0,1,2,3...) refers to document_ids 40041,37962,37985... are ids representing objects related to the documents.For example, document_id 2 has related (truth) object 37985 and predicted objects 37985,37983. Truth – actual objects (gold standard) Predicted –...
kitchenprinzessin
1

votes
1

answer
373

Views

How to use F-score as error function to train neural networks?

I am pretty new to neural networks. I am training a network in tensorflow, but the number of positive examples is much much less than negative examples in my dataset (it is a medical dataset). So, I know that F-score calculated from precision and recall is a good measure of how well the model is tr...
Arindam
1

votes
0

answer
58

Views

Use TensorFlow loss Global Objectives (recall_at_precision_loss) with Keras (not metrics)

Background I have a multi-label classification problem with 5 labels (e.g. [1 0 1 1 0]). Therefore, I want my model to improve at metrics such as fixed recall, precision-recall AUC or ROC AUC. It doesn't make sense to use a loss function (e.g. binary_crossentropy) that is not directly related to the...
NumesSanguis
1

votes
1

answer
27

Views

How to improve Precision without downing the Recall in a unbalanced dataset?

I have to use a Decision Tree for binary classification on a unbalanced dataset(50000:0, 1000:1). To have a good Recall (0.92) I used RandomOversampling function found in module Imblearn and pruning with max_depth parameter. The problem is that the Precision is very low (0.44), I have too many false...
AlanT
1

votes
2

answer
510

Views

Scikit learn SGDClassifier: precision and recall change the values each time

I have a question about the precision and recall values in scikit learn. I am using the function SGDClassifier to classify my data. To evaluate the performances, I am using the precision and recall function precision_recall_fscore_support but each time that I run the program I have different values...
PSan
1

votes
2

answer
721

Views

Average values of precision, recall and fscore for each label

I'm cross validating a sklearn classifier model and want to quickly obtain average values of precision, recall and f-score. How can I obtain those values? I don't want to code the cross validation by myself, instead I'm using the function cross_validation.cross_val_score. Is it possible to use this...
s0kol
1

votes
1

answer
276

Views

Evaluating mahout based Boolean recommendation engine - interpreting precision & recall

I would like to evaluate a mahout based recommendation engine of a fashion E-Commerce Site. They use shopping card information about item bought thogether - so boolean. I want to evaluate the engine using precision and recall. 1) How can I use these metrics to evaluate the recommendation engine? Is...
Reiner Aldar
1

votes
1

answer
29

Views

what is precision if their is no output?

some of metrics of systems like IR are precision and recall. However their definition is clear but I doubt when one system returns no output should we consider its precision 1 or zero. or should we discriminate if their is no gold answer or not for computing precision in this situation? if this ques...
user3070752
1

votes
1

answer
74

Views

How to compute recall and precision if there is not a positive/negative meaning?

How do you compute these metrics when there is not a positive.negative meaning in the classes, but they just represent something neutral? Let's say for example that we have a classification problem, where you have two classes that represent a person (John, Alex) and you want to classify your new ins...
user3309479
1

votes
2

answer
188

Views

Potential BUG in ROSE package: Difference in accuracy, recall and precision in R

When I calculate the measures with the Rose library I get measures for recall, precision and F1. The recall and precision measures differ however when I calculate them manually. How come? install.packages('ROSE') library(ROSE) library(rpart) s = sample(957,200) training = data[-s,] test = data[s,]...
sockevalley
1

votes
1

answer
209

Views

h2o ValueError: No metric tpr

When trying to acquire the recall score using e.g. rf_model.recall() I get the error: h2o ValueError: No metric tpr I can get other metrics, such as the accuracy, AUC, precision and F1 but no recall... This is presumably a bug. If I run: from h2o.model.metrics_base import H2OBinomialModelMetrics as...
EB88
1

votes
1

answer
396

Views

Issues using GridSearchCV with RandomForestClassifier using large data, always showing recall score = 1, so best params becomes redundant

This is my first StackOverflow question and I am in need of help! I have exhaustively searched for answers myself and through experimentation but I am hoping someone from the community can help. This is work for my dissertation at Uni, so any help would be extremely appreciated. I will try to summar...
Harry Graham
1

votes
1

answer
726

Views

TensorFlow PR Curve Plugin: pr_curve_streaming_op

Has anyone figured out a way to use pr_curve_streaming_op for multiclass classification ? I am trying to follow this demo and this demo. But so far I am getting bumped at the data collection stage itself. Its seems like I need to be able to collect data at runtime and I am facing several issues like...
MLNINJA
1

votes
1

answer
292

Views

Keras - finding recall values of binary classification

I am little bit in trouble finding the value of recall using sklearn. I am using keras 2.0 for solving a binary classification problem. For finding recall I need to depend of sklearn metrics, but I am getting a value error for. Here is my sample code and the error stack: >> print(y_true[:5]) >> [[...
Wazed Ali
1

votes
1

answer
678

Views

Area under the precision-recall curve for DecisionTreeClassifier is a square

I'm using the DecisionTreeClassifier from scikit-learn to classify some data. I'm also using other algorithms and to compare them I use the area under the precision-recall metric. The problem is the shape of the AUPRC for the DecisionTreeClassifier is a square and not the usual shape you would expec...
cod3min3
1

votes
1

answer
1k

Views

Calculate precision and recall on WANG database

I have made an CBIR system in MATLAB and have used similarity measurement as euclidean distance. Using this for each query image I retrieve top 20 images. I have used WANG Dataset for testing my system. It contains 10 classes(like African people, Buses, Roses etc.) each containing 100 images.(1000...
CoderBoy
1

votes
1

answer
304

Views

What does the last row mean in classification_report in scikit learn

I wonder what the last line which says avg / total mean in scikit-learn classification report? Is it macro average or micro average? For example in the following table taken from the documentation, what is the precision recall f1-score support class 0 0.50 1.00 0.67 1...
user2161903
6

votes
5

answer
120

Views

Calculating the number of true positives from a precision-recall curve

Using the below precision recall graph where recall is on x-axis and precision is on y-axis can I use this formula to calculate the number of predictions for a given precision, recall threshold ? These calculations are based on orange trend line. Assuming this model has been trained on 100 instances...
blue-sky
1

votes
1

answer
54

Views

Precision and Recall computation for different group sizes

I didn't find an answer for this question anywhere, so I hope someone here could help me and also other people with the same problem. Suppose that I have 1000 Positive samples and 1500 Negative samples. Now, suppose that there are 950 True Positives (positive samples correctly classified as positive...
SomethingSomething
1

votes
1

answer
3.3k

Views

How To Calculate F1-Score For Multilabel Classification?

I try to calculate the f1_score but I get some warnings for some cases when I use the sklearn f1_score method. I have a multilabel 5 classes problem for a prediction. import numpy as np from sklearn.metrics import f1_score y_true = np.zeros((1,5)) y_true[0,0] = 1 # => label = [[1, 0, 0, 0, 0]] y_pre...
KyleReemoN-
1

votes
1

answer
381

Views

Custom macro for recall in keras

I am trying to create a custom macro for recall = (recall of class1 + recall of class2)/2. I came up with the following code but I am not sure how to calculate the true positive of class 0. def unweightedRecall(): def recall(y_true, y_pred): # recall of class 1 true_positives1 = K.sum(K.round(K.clip...
Aditya
1

votes
2

answer
144

Views

Support = 'None'

Using the code below I get values for precision, recall, and F scores but I get None for support import numpy as np from sklearn.metrics import precision_recall_fscore_support ytrue = np.array(['1', '1', '1', '1', '1','1','1','1','0']) ypred = np.array(['0', '0', '0', '1', '1','1','1','1','0']) prec...
EER
1

votes
1

answer
0

Views

[email protected] computation

Mean average precision computed at k (for top-k elements in the answer), according to wiki, ml metrics at kaggle, and this answer: Confusion about (Mean) Average Precision should be computed as mean of average precisions at k, where average precision at k is computed as: Where: P(i) is the precision...
Podgorskiy
0

votes
0

answer
3

Views

Measuring AUPRC in CatBoost

I want to measure area under the curve of precision-recall curve (AUPRC) in catboost, but the CatBoostClassifier, doesn not have AUPRC as an evaluation metric.Any suggestion that helps me to measure this performance metric will be appreciated. This is the code I use: model = CatBoostClassifier( cust...
woody
1

votes
1

answer
311

Views

precision and recall error while using sklearn

I am using sklearn precision and recall to get those scores. I got an error saying value error. Could anyone tell where am I doing wrong? My y_test is as follows 443 positive 3615 positive 2030 negative 2993 positive 2870 positive 2907 negative 2215 positive My Prediction is as...
merklexy
1

votes
2

answer
1.1k

Views

Calculating Precision and Recall in Click Data

I am trying to build a graph of precision and recall using click data. I have two data sources. First data source has all the user clicked item_ids based on a given query_id. Second data source has all the relevant item_ids for given query_id. I used python and put these in two data sources into di...
Null-Hypothesis
1

votes
2

answer
117

Views

Can the Precision, Recall and F1 be the same value?

I am currently working on an ML classification problem and I'm computing the Precision, Recall and F1 using the sklearn library's following import and respective code as shown below. from sklearn.metrics import precision_recall_fscore_support print(precision_recall_fscore_support(y_test, prob_pos, a...
Nayantara Jeyaraj
9

votes
0

answer
161

Views

Using precision recall metric on a hierarchy of recovered clusters

Context: We are two students intending to write a thesis on reverse engineering namespaces using hierarchical agglomerative clustering algorithms. We have a variation of linking methods and other tweaks to the algorithm we want to try out. We will run the algorithm on popular GitHub repositories and...
David
1

votes
1

answer
1.1k

Views

How to interpret this triangular shape ROC AUC curve?

I have 10+ features and a dozen thousand of cases to train a logistic regression for classifying people's race. First example is French vs non-French, and second example is English vs non-English. The results are as follows: ////////////////////////////////////////////////////// 1= fr 0= non-fr Clas...
KubiK888
6

votes
1

answer
1.9k

Views

Good ROC curve but poor precision-recall curve

I have some machine learning results that I don't quite understand. I am using python sciki-learn, with 2+ million data of about 14 features. The classification of 'ab' looks pretty bad on the precision-recall curve, but the ROC for Ab looks just as good as most other groups' classification. What ca...
KubiK888
5

votes
2

answer
4.5k

Views

How to draw a precision-recall curve with interpolation in python?

I have drawn a precision-recall curve using sklearn precision_recall_curvefunction and matplotlib package. For those of you who are familiar with precision-recall curve you know that some scientific communities only accept it when its interpolated, similar to this example here. Now my question is if...
user823743
3

votes
2

answer
303

Views

What is the correct version of Average precision?

I'm trying to compute the Average Precision (and Mean Average Precision) on the Oxford Building image dataset. Below there is the code that they provide for computing Average Precision. Notice that pos_set is the union of the 'optimal' and 'good' images from the ground trouth set, while junk_set is...
justHelloWorld
5

votes
2

answer
16.4k

Views

Calculate Precision and Recall

I am really confused about how to calculate Precision and Recall in Supervised machine learning algorithm using NB classifier Say for example 1) I have two classes A,B 2) I have 10000 Documents out of which 2000 goes to training Sample set (class A=1000,class B=1000) 3) Now on basis of above trainin...
user1051536
22

votes
5

answer
18.3k

Views

Precision/recall for multiclass-multilabel classification

I'm wondering how to calculate precision and recall measures for multiclass multilabel classification, i.e. classification where there are more than two labels, and where each instance can have multiple labels?
MaVe
4

votes
4

answer
8.2k

Views

Computing F-measure for clustering

Can anyone help me to calculate F-measure collectively ? I know how to calculate recall and precision, but don't know for a given algorithm how to calculate one F-measure value. As an exemple, suppose my algorithm creates m clusters, but I know there are n clusters for the same data (as created by a...
mahesh cs
1

votes
1

answer
2.4k

Views

scikit weighted f1 score calculation and usage

I have a question regarding weighted average in sklearn.metrics.f1_score sklearn.metrics.f1_score(y_true, y_pred, labels=None, pos_label=1, average='weighted', sample_weight=None) Calculate metrics for each label, and find their average, weighted by support (the number of true instances for each lab...
com

View additional questions