文章目录[隐藏]

deepSORT依据的是两个匹配机制，马氏距离下的特征层匹配以及IOU匹配，而当IOU匹配失去作用时，也就是多目标追踪任务中目标帧与帧之间差距较大而使得IOU匹配无法实现时，就需要考虑将其替换成其他的匹配机制，比如欧氏距离匹配

IOU匹配替换代码

        iou_track_candidates = unconfirmed_tracks + [
            k for k in unmatched_tracks_a if
            self.tracks[k].time_since_update == 1]
        unmatched_tracks_a = [
            k for k in unmatched_tracks_a if
            self.tracks[k].time_since_update != 1]
        # matches_b, unmatched_tracks_b, unmatched_detections = \
        #     linear_assignment.min_cost_matching(
        #         iou_matching.iou_cost, self.max_iou_distance, self.tracks,
        #         detections, iou_track_candidates, unmatched_detections)
        # matches = matches_a + matches_b
        # unmatched_tracks = list(set(unmatched_tracks_a + unmatched_tracks_b))


        matches=matches_a
        unmatched_tracks=list(set(unmatched_tracks_a+iou_track_candidates))
        #欧氏距离匹配
        Euclidean_distance_candidates=unmatched_tracks
        matches_c, unmatched_tracks_c, unmatched_detections = \
            linear_assignment.min_cost_matching(
                iou_matching.Euclidean_distance_cost, self.max_iou_distance, self.tracks,
                detections, Euclidean_distance_candidates, unmatched_detections)
        matches+=matches_c
        unmatched_tracks=list(set(unmatched_tracks_c))

Euclidean_cost_matching是欧氏距离的方法

修改的位置在deep_sort_pytorch/deep_sort/sort/tracker.py

track属性相关修改

在原本的 deepSORT 中是没有 detection 这个属性的，我们需要在 track 中增加这一属性，来对后续的欧氏距离匹配铺路，在这里，唯一的改动只是增加了一个 detection 的属性

    def __init__(self, mean, covariance, track_id, n_init, max_age,
                 feature=None):
        self.mean = mean
        self.covariance = covariance
        self.track_id = track_id
        self.hits = 1
        self.age = 1
        self.time_since_update = 0

        self.state = TrackState.Tentative
        self.features = []
        if feature is not None:
            self.features.append(feature)
        self.detections = []

        self._n_init = n_init
        self._max_age = max_age

之后需要在 track 的更新函数中将每一帧检测匹配上的 detection 加上去。

    def update(self, kf, detection):
        """Perform Kalman filter measurement update step and update the feature
        cache.

        Parameters
        ----------
        kf : kalman_filter.KalmanFilter
            The Kalman filter.
        detection : Detection
            The associated detection.

        """

        self.mean, self.covariance = kf.update(
            self.mean, self.covariance, detection.to_xyah())
        self.features.append(detection.feature)

        self.detections.append(detection)

        self.hits += 1
        self.time_since_update = 0
        if self.state == TrackState.Tentative and self.hits >= self._n_init:
            self.state = TrackState.Confirmed

欧氏距离匹配代码增加

由于是将IOU匹配替换成了欧氏距离匹配，所以在 deep_sort_pytorch/deep_sort/sort/iou_matching.py中增加方法 Euclidean_distance_cost

def Euclidean_distance(bbox,candidates):
    bbox_cx=bbox[0]+bbox[2]/2
    bbox_cy=bbox[1]+bbox[3]/2
    candidates_cx=candidates[:,0]+candidates[:,2]/2
    candidates_cy=candidates[:,1]+candidates[:,3]/2
    distance=np.sqrt((candidates_cx[:]-bbox_cx)**2+(candidates_cy-bbox_cy)**2)
    return distance
def Euclidean_distance_cost(tracks, detections, track_indices=None,
             detection_indices=None):
    if track_indices is None:
        track_indices = np.arange(len(tracks))
    if detection_indices is None:
        detection_indices = np.arange(len(detections))

    cost_matrix=np.zeros((len(track_indices),len(detection_indices)))
    for row,track_idx in enumerate(track_indices):
        bbox=tracks[track_idx].to_tlwh()
        candidates=np.asarray(
            [detections[i].tlwh for i in detection_indices])
        cost_matrix[row,:]=Euclidean_distance(bbox,candidates)
    return cost_matrix

此函数返回的是距离矩阵

最后在距离 linear_assignment.py 下将iou匹配的阈值替换成合适的欧氏距离阈值

    if (len(cost_matrix_1)==2 and cost_matrix_1[1]!='cas') or len(cost_matrix_1)!=2 :
        #默认的是max_distance = 0.7
        # cost_matrix = cost_matrix_1
        # cost_matrix[cost_matrix > max_distance] = max_distance + 1e-5
        #使用欧氏距离替换iou
        cost_matrix = cost_matrix_1
        max_distance=50
        cost_matrix[cost_matrix > max_distance] = max_distance + 1e+5

如果是简单的修改能够使算法满足项目要求的话下半部分就不用去看了

欧氏距离下的路径规划

在实际的项目中，IOU发挥不了作用的情况下，欧氏距离匹配能够勉强发挥作用，但仍不能达到高帧数下的追踪效果

这种情况下就应该加入运动规划了，规范化目标的运行路径，使得不合理的匹配结果尽可能的少。

在这里就需要利用之前增加的 detection 属性了，修改如下

def Euclidean_distance(bbox,candidates):
    bbox_cx=bbox[0]+bbox[2]/2
    bbox_cy=bbox[1]+bbox[3]/2
    candidates_cx=candidates[:,0]+candidates[:,2]/2
    candidates_cy=candidates[:,1]+candidates[:,3]/2
    distance=np.sqrt((candidates_cx[:]-bbox_cx)**2+(candidates_cy-bbox_cy)**2)
    return distance

#距离长宽比矩阵
def distance_xy_a(bbox,candidates):
    #上一帧的位置
    bbox_cx = bbox[0]
    bbox_cy = bbox[1]
    candidates_cx=candidates[:,0]+candidates[:,2]/2
    candidates_cy=candidates[:,1]+candidates[:,3]/2
    juzhen=[]
    for i in range(len(candidates_cy)):
        # if abs(candidates_cx[i]-bbox_cx)>=10 and abs(candidates_cy[i]-bbox_cy)>=10 :
        #     juzhen.append(True)
        #     continue
        if abs(candidates_cy[i]-bbox_cy)>=50:
            juzhen.append(True)
            continue
        if candidates_cy[i]-bbox_cy<-20:
            juzhen.append(True)
            continue
        if candidates_cx[i]-bbox_cx<-10:
            juzhen.append(True)
            continue
        juzhen.append(False)
    return juzhen

def Euclidean_distance_cost(tracks, detections, track_indices=None,
             detection_indices=None):
    if track_indices is None:
        track_indices = np.arange(len(tracks))
    if detection_indices is None:
        detection_indices = np.arange(len(detections))

    cost_matrix=np.zeros((len(track_indices),len(detection_indices)))
    for row,track_idx in enumerate(track_indices):
        track=tracks[track_idx]
        bbox=tracks[track_idx].to_tlwh()
        candidates=np.asarray(
            [detections[i].tlwh for i in detection_indices])
        cost_matrix[row,:]=Euclidean_distance(bbox,candidates)
        if len(track.detections)!=0:
            ret=track.detections[-1].to_xyah()

            cost_matrix[row,distance_xy_a(ret,candidates)]+=1000

distance_xy_a是设定的路径规划，使得 ID 切换和 ID 丢失的情况尽可能的下降，可根据实际的项目进行修改