文章目录[隐藏]
目标检测 YOLOv5 - loss for objectness and classification
flyfish
下面两句位于utils/loss.py的ComputeLoss类中
BCEcls = nn.BCEWithLogitsLoss(pos_weight=torch.tensor([h['cls_pw']], device=device))
BCEobj = nn.BCEWithLogitsLoss(pos_weight=torch.tensor([h['obj_pw']], device=device))
BCEWithLogitsLoss的公式是
ℓ
(
x
,
y
)
=
L
=
{
l
1
,
…
,
l
N
}
⊤
,
l
n
=
−
w
n
[
y
n
⋅
log
σ
(
x
n
)
+
(
1
−
y
n
)
⋅
log
(
1
−
σ
(
x
n
)
)
]
,
\ell(x, y) = L = \{l_1,\dots,l_N\}^\top, \quad l_n = - w_n \left[ y_n \cdot \log \sigma(x_n) + (1 - y_n) \cdot \log (1 - \sigma(x_n)) \right],
ℓ(x,y)=L={l1,…,lN}⊤,ln=−wn[yn⋅logσ(xn)+(1−yn)⋅log(1−σ(xn))],
使用BCEWithLogitsLoss代码示例
import torch
target = torch.ones([10, 64], dtype=torch.float32) # 64 classes, batch size = 10
output = torch.full([10, 64], 1.5) # A prediction (logit)
pos_weight = torch.ones([64]) # All weights are equal to 1
criterion = torch.nn.BCEWithLogitsLoss(pos_weight=pos_weight)
loss=criterion(output, target) # -log(sigmoid(1.5))
print(loss) #tensor(0.2014)
BCEWithLogitsLoss = Sigmoid + BCELoss
代码示例
import torch
target = torch.ones([10, 64], dtype=torch.float32) # 64 classes, batch size = 10
output = torch.full([10, 64], 1.5) # A prediction (logit)
a=torch.nn.Sigmoid()
criterion=torch.nn.BCELoss()
a=torch.nn.Sigmoid()
loss = criterion(a(output), target)
print(loss)#tensor(0.2014)
调用binary_cross_entropy的实现
import torch
import torch.nn.functional as F
target = torch.ones([10, 64], dtype=torch.float32,requires_grad=False) # 64 classes, batch size = 10
output = torch.full([10, 64], 1.5,requires_grad=True) # A prediction (logit)
a=torch.nn.Sigmoid()
loss = F.binary_cross_entropy(a(output), target)
loss.backward()
print(loss) #tensor(0.2014, grad_fn=<BinaryCrossEntropyBackward>)
推荐使用BCEWithLogitsLoss，而不是其他方式的实现，因为BCEWithLogitsLoss内部使用了log-sum-exp技巧
这里有是如何实现的log-sum-exp，包含整个推理过程 。
以一个实际的例子来理解BCEWithLogitsLoss内部的计算过程
sigmoid的公式是
Sigmoid
(
x
)
=
σ
(
x
)
=
1
1
+
exp
(
−
x
)
\text{Sigmoid}(x) = \sigma(x) = \frac{1}{1 + \exp(-x)}
Sigmoid(x)=σ(x)=1+exp(−x)1
BCELoss的公式是
ℓ
(
x
,
y
)
=
L
=
{
l
1
,
…
,
l
N
}
⊤
,
l
n
=
−
w
n
[
y
n
⋅
log
x
n
+
(
1
−
y
n
)
⋅
log
(
1
−
x
n
)
]
\ell(x, y) = L = \{l_1,\dots,l_N\}^\top, \quad l_n = - w_n \left[ y_n \cdot \log x_n + (1 - y_n) \cdot \log (1 - x_n) \right]
ℓ(x,y)=L={l1,…,lN}⊤,ln=−wn[yn⋅logxn+(1−yn)⋅log(1−xn)]
预测值分别是 0.7,0.2,0.1
类别标签分别是 1 , 0, 0
预测值经过sigmoid结果是0.6682, 0.5498, 0.5250
代入sigmoid公式
print(1 /(1+math.exp(-1 * 0.7)))
print(1 /(1+math.exp(-1 * 0.2)))
print(1 /(1+math.exp(-1 * 0.1)))
math.exp就是e的多少次方
e约等于2.718281828459045，上面第一个式子与print(1/(1+2.718281828459045 ** (-0.7)))相同
在经过BCELoss结果是0.648557191505494
代入BCELoss公式
from math import log
a=0.6682
b=0.5498
c=0.5250
print(((1 * log (a)+(1-1) * log (1-a))+(0 * log (b)+(1-0) * log (1-b))+(0 * log (c)+(1-0) * log (1-c))) /(-3))
最终得到的结果是0.648557191505494
上例如果用BCEWithLogitsLoss代码写是
import torch
pred=torch.tensor([0.7,0.2,0.1],dtype=torch.float)
target=torch.tensor([1,0,0],dtype=torch.float)
criterion = torch.nn.BCEWithLogitsLoss()
loss=criterion(pred, target)
print(loss)#tensor(0.6486)
版权声明：本文为CSDN博主「TheOldManAndTheSea」的原创文章，遵循CC 4.0 BY-SA版权协议，转载请附上原文出处链接及本声明。
原文链接：https://blog.csdn.net/flyfish1986/article/details/118909723
暂无评论