Can the Operating Characteristic for the LRT be derived from minimizing Bayes Risk $ \varphi(f) = \alpha P_F - \beta P_D + \gamma$?
Can the Operating Characteristic for the LRT be derived from minimizing Bayes Risk $ \varphi(f) = \alpha P_F - \beta P_D + \gamma$?
Loading saved threads...
Charlie Parker · External communityPost link
External question — Cross Validated Stack Exchange
Author: Charlie Parker
Original post: https://stats.stackexchange.com/questions/188677
License: CC BY-SA 3.0 — https://creativecommons.org/licenses/by-sa/3.0/
Adaptation: HTML converted to plain text; contact email addresses removed.
For fun I was reading some notes on Operating Characteristics and it said (paraphrased with notation definition):
As Fig. 1 suggests (omitted from question), good detection probability $P_D = P(\hat{H}(y) = H_1 \mid H = H_1)$ is generally obtained at the expense of high false alarm probability $P_F = P(\hat{H}(y) = H_1 \mid H = H_0)$ and so choosing a (Likely Ratio Test LRT) threshold $\eta$ for a particular problem involves making an acceptable tradeoff... From this perspective, the Bayesian Hypothesis test (i.e. choosing a decision rule that minimizing Bayes risk $\varphi(f) = \mathbb{E}_{H,y}[C(f(y), H)] $) represents a particular tradeoff, and corresponds to a single point on this curve. To obtain this tradeoff, we effectively selected as our objective function a linear combination of $P_D$ and $P_F$. More specifically, we minimize:
$$ \varphi(f) = \alpha P_F - \beta P_D + \gamma$$
over all possible decision rules, where the choice of $\alpha$ and $\beta$ is in turn, determined by the cost assignment $C_{ij}$'s (where i corresponds to the chosen hypothesis by the decision rule and j corresponds to the correct hypothesis) and the a priori probabilities $P_m$'s.
and claims for the Binary case that the following is true:
$$\alpha = (C_{10} - C_{00})P_0,$$
$$\beta=(C_{01} - C_{11})P_1,$$
$$ \gamma = (C_{00} + C_{01})P_1$$
My first question is, why do we minimize:
$$ \varphi(f) = \alpha P_F - \beta P_D + \gamma$$
that doesn't quite makes sense to me. Why do we want to minimize that? Is it because we just want to minimize the expected risk? (Also, why does that equation express the intrinsic trade off between $P_D$ and $P_F$?)From the notation, I infer that one can re-write Bayes Risk and write it in that form, however, I've been unable to do that. Here is what I have tried:
$$ \varphi(f) = \mathbb{E}_{H,y}[C(f(y), H)] = \sum_{H_j,y} C(f(y), H_j) P(y,H_j)$$
$$ \varphi(f) = \sum_{H_j} \sum_{H_i} \sum_{y : f(y) = H_i} C(f(y) = H_i, H_j) P(y \mid H_j) P(H_j)$$
$$ =\sum_{H_j,H_i} C(H_i, H_j) P(f(y) = H_i \mid H_j) P(H_j)$$
for the binary case we have (were $P_i = P(H_i)$, $C_{ij} = C(f(y) = H_i, H_j)$, $P(H_i \mid H_j) = P(f(y) = H_i \mid H = H_j)$):
$$ \varphi(f) = C_{00} P(H_0 \mid H_0)P_0 + C_{11} P(H_1 \mid H_1)P_1 + C_{10} P(H_1 \mid H_0)P_0 + C_{01} P(H_0 \mid H_1)P_1$$
$$ \varphi(f) = C_{00} P(H_0 \mid H_0)P_0 + C_{11} P_DP_1 + C_{10} P_F P_0 + C_{01} P(H_0 \mid H_1)P_1$$
and after this step, is where I get stuck. The reason I am having trouble advance is because, the form the text suggests as an answer does not even have the terms $P(f(y) = H_0 \mid H = H_0) = P(H_0 \mid H_0)$ and $ P(f(y) = H_0 \mid H = H_1) = P(H_0 \mid H_1)$
anywhere. Which makes me suspect that there might be a typo on the text and $\gamma$ should actually be something else involving $P(H_0 \mid H_1)$ or $P(f(y) = H_0 \mid H = H_0)$. Even if that were true, I can't explain myself how Bayes Risk has a minus sign (as in $\alpha P_F - \beta P_D + \gamma$) even if there were true, unless it was cancelled out somewhere again in the $\gamma$ by some hidden terms. Someone know where the typo is or if there is a way of proceeding from the location I got stuck?
Quote
Report
Charlie Parker · External communityPost link
External answer — Cross Validated Stack Exchange
Author: Charlie Parker
Original post: https://stats.stackexchange.com/a/188917
License: CC BY-SA 3.0 — https://creativecommons.org/licenses/by-sa/3.0/
Adaptation: HTML converted to plain text; contact email addresses removed.
The goal of the question is to show that in the (Binary) Bayesian Hypothesis Testing framework, one can write the expected risk $\varphi(f) = \mathbb{E}_{H,y}[C(f(y), H)]$ as follow:
$$ \varphi(f) = \alpha P_F - \beta P_D + \gamma$$
were $\alpha, \beta, \gamma $ are constants that depend on the cost function ($C_{i,j} = C(f(y) = H_i, H = H_j)$) chosen and the prior probabilities on the hypothesis ($P_m$'s).
So lets write down what $ \varphi(f)$ means and do some algebra:
$$ \varphi(f) = \mathbb{E}_{H,y}[C(f(y), H)] = \sum_{H_j,y} C(f(y), H_j) P(y,H_j)$$
$$ \varphi(f) = \sum_{H_j} \sum_{H_i} \sum_{y : f(y) = H_i} C(f(y) = H_i, H_j) P(y \mid H_j) P(H_j)$$
$$ \varphi(f) =\sum_{H_j,H_i} C(H_i, H_j) P(f(y) = H_i \mid H_j) P(H_j)$$
for the binary case we have (were $P_i = P(H_i)$, $C_{ij} = C(f(y) = H_i, H_j)$, $P(H_i \mid H_j) = P(f(y) = H_i \mid H = H_j)$):
$$ \varphi(f) = C_{00} P(H_0 \mid H_0)P_0 + C_{11} P(H_1 \mid H_1)P_1 + C_{10} P(H_1 \mid H_0)P_0 + C_{01} P(H_0 \mid H_1)P_1$$
$$ \varphi(f) = C_{00} P(H_0 \mid H_0)P_0 + C_{11} P_DP_1 + C_{10} P_F P_0 + C_{01} P(H_0 \mid H_1)P_1$$
Now notice that we want to express the expected risk in terms of the detection probability and false alarm probability
only
. For this we need to find expression for $P(H_0 \mid H_0)$ and $P(H_0 \mid H_1)$ in terms of $P_F$ and $P_D$. Its easy to see that the equations that we need are:
$$P(H_0 \mid H_0) = 1 - P_F$$
and
$$ P(H_0 \mid H_1)$ = 1 - P_D$$
So the expected risk $ \varphi(f)$ becomes:
$$ \varphi(f) = C_{00} (1 - P_F) P_0 + C_{11} P_D P_1 + C_{10} P_F P_0 + C_{01} (1 - P_D) P_1$$
$$ \varphi(f) = (C_{10} - C_{00}) P_0 P_F - (C_{01} - C_{11})P_1 P_D + C_{00}P_0 + C_{01} P_1$$
Thus:
$$\alpha = (C_{10} - C_{00}) P_0 $$
$$\beta = (C_{01} - C_{11})P_1 $$
$$\gamma = C_{00}P_0 + C_{01} P_1 $$
as required $ \varphi(f) = \alpha P_F - \beta P_D + \gamma$.
To finish with some interesting remarks, why would we want to express the expected risk like that? Well, the interesting thing is that if one plots an operating characteristic (OC) for some hypothesis test, one can see how a choice of priors determines which point $(P_D, P_F)$ on the OC we will have. So a choice of priors and costs intrinsically chooses a point $(P_D, P_F)$.
Also, notice that there is a constant $\gamma$ that is irreducible (i.e. independent of which decision rule $f$ we choose). Thus, no matter what we do (unless we choose costs and priors that are exactly zero as to make $\gamma$ is zero), there might be some irreducible expected error that we will have.
Lastly, if we are able to show there is an intrinsic trade-off between large $P_D$ and small $P_F$, then this equations express this trade off for the bayesian hypothesis testing case. To intuitively see this notice that if one can show that $P_F$ and $P_D$ are positively related (which one can show $\frac{d P_D}{d P_F} \geq 0$) then if $P_D$ is increases (so to decrease expected/bayes risk) then $P_F$ increases too. Thus, its unclear how increasing $P_D$ would necessarily lead to a decrease in the average risk.
Quote
Report
Post Reply
Quoted from Forex.com.bd-Editorial External answer — Cross Validated Stack Exchange Author: Charlie Parker Source score (net votes, not local likes): 1 Original post: https://stats.stackexchange.com/a/188917 License: CC BY-SA 3.0 — https://creativecommons.org/licenses/by-sa/3.0/ Adaptation: HTML converted to plain text; contact email addresses removed. The goal of the question is to show that in the (Binary) Bayesian Hypothesis Testing framework, one can write the expected risk $\varphi(f) = \mathbb{E}_{H,y}[C(f(y), H)]$ as follow: $$ \varphi(f) = \alpha P_F - \beta P_D + \gamma$$ were $\alpha, \beta, \gamma $ are constants that depend on the cost function ($C_{i,j} = C(f(y) = H_i, H = H_j)$) chosen and the prior probabilities on the hypothesis ($P_m$'s). So lets write down what $ \varphi(f)$ means and do some algebra: $$ \varphi(f) = \mathbb{E}_{H,y}[C(f(y), H)] = \sum_{H_j,y} C(f(y), H_j) P(y,H_j)$$ $$ \varphi(f) = \sum_{H_j} \sum_{H_i} \sum_{y : f(y) = H_i} C(f(y) = H_i, H_j) P(y \mid H_j) P(H_j)$$ $$ \varphi(f) =\sum_{H_j,H_i} C(H_i, H_j) P(f(y) = H_i \mid H_j) P(H_j)$$ for the binary case we have (were $P_i = P(H_i)$, $C_{ij} = C(f(y) = H_i, H_j)$, $P(H_i \mid H_j) = P(f(y) = H_i \mid H = H_j)$): $$ \varphi(f) = C_{00} P(H_0 \mid H_0)P_0 + C_{11} P(H_1 \mid H_1)P_1 + C_{10} P(H_1 \mid H_0)P_0 + C_{01} P(H_0 \mid H_1)P_1$$ $$ \varphi(f) = C_{00} P(H_0 \mid H_0)P_0 + C_{11} P_DP_1 + C_{10} P_F P_0 + C_{01} P(H_0 \mid H_1)P_1$$ Now notice that we want to express the expected risk in terms of the detection probability and false alarm probability only . For this we need to find expression for $P(H_0 \mid H_0)$ and $P(H_0 \mid H_1)$ in terms of $P_F$ and $P_D$. Its easy to see that the equations that we need are: $$P(H_0 \mid H_0) = 1 - P_F$$ and $$ P(H_0 \mid H_1)$ = 1 - P_D$$ So the expected risk $ \varphi(f)$ becomes: $$ \varphi(f) = C_{00} (1 - P_F) P_0 + C_{11} P_D P_1 + C_{10} P_F P_0 + C_{01} (1 - P_D) P_1$$ $$ \varphi(f) = (C_{10} - C_{00}) P_0 P_F - (C_{01} - C_{11})P_1 P_D + C_{00}P_0 + C_{01} P_1$$ Thus: $$\alpha = (C_{10} - C_{00}) P_0 $$ $$\beta = (C_{01} - C_{11})P_1 $$ $$\gamma = C_{00}P_0 + C_{01} P_1 $$ as required $ \varphi(f) = \alpha P_F - \beta P_D + \gamma$. To finish with some interesting remarks, why would we want to express the expected risk like that? Well, the interesting thing is that if one plots an operating characteristic (OC) for some hypothesis test, one can see how a choice of priors determines which point $(P_D, P_F)$ on the OC we will have. So a choice of priors and costs intrinsically chooses a point $(P_D, P_F)$. Also, notice that there is a constant $\gamma$ that is irreducible (i.e. independent of which decision rule $f$ we choose). Thus, no matter what we do (unless we choose costs and priors that are exactly zero as to make $\gamma$ is zero), there might be some irreducible expected error that we will have. Lastly, if we are able to show there is an intrinsic trade-off between large $P_D$ and small $P_F$, then this equations express this trade off for the bayesian hypothesis testing case. To intuitively see this notice that if one can show that $P_F$ and $P_D$ are positively related (which one can show $\frac{d P_D}{d P_F} \geq 0$) then if $P_D$ is increases (so to decrease expected/bayes risk) then $P_F$ increases too. Thus, its unclear how increasing $P_D$ would necessarily lead to a decrease in the average risk.
Checking account access…