Online multi-object tracking with efficient track drift and fragmentation handling

Jaeyong Ju; Daehun Kim; Bonhwa Ku; David K. Han; Hanseok Ko

doi:10.1364/JOSAA.34.000280

Journal of the Optical Society of America A
Vol. 34,
Issue 2,
pp. 280-293
(2017)
•https://doi.org/10.1364/JOSAA.34.000280

Online multi-object tracking with efficient track drift and fragmentation handling

Jaeyong Ju, Daehun Kim, Bonhwa Ku, David K. Han, and Hanseok Ko

Get PDF
Email
Share
Get Citation
Copy Citation Text
Jaeyong Ju, Daehun Kim, Bonhwa Ku, David K. Han, and Hanseok Ko, "Online multi-object tracking with efficient track drift and fragmentation handling," J. Opt. Soc. Am. A 34, 280-293 (2017)

Export Citation
- BibTex
- Endnote (RIS)
- HTML
- Plain Text
Citation alert
Save article

Related Topics
Table of Contents Category
- Image Processing
Optics & Photonics Topics
?

The topics in this list come from the Optics and Photonics Topics applied to this article.

About this Article
History
- Original Manuscript: August 1, 2016
- Revised Manuscript: November 30, 2016
- Manuscript Accepted: December 23, 2016
- Published: January 27, 2017

Accepted Manuscript
Download Accepted Manuscript
Access to this article is made available via and is subject to OSA Publishing Terms of Use.

Abstract

This paper addresses the problem of multi-object tracking in complex scenes by a single, static, uncalibrated camera. Tracking-by-detection is a widely used approach for multi-object tracking. Challenges still remain in complex scenes, however, when this approach has to deal with occlusions, unreliable detections (e.g., inaccurate position/size, false positives, or false negatives), and sudden object motion/appearance changes, among other issues. To handle these problems, this paper presents a novel online multi-object tracking method, which can be fully applied to real-time applications. First, an object tracking process based on frame-by-frame association with a novel affinity model and an appearance update that does not rely on online learning is proposed to effectively and rapidly assign detections to tracks. Second, a two-stage drift handling method with novel track confidence is proposed to correct drifting tracks caused by the abrupt motion change of objects under occlusion and prolonged inaccurate detections. In addition, a fragmentation handling method based on a track-to-track association is proposed to solve the problem in which an object trajectory is broken into several tracks due to long-term occlusions. Based on experimental results derived from challenging public data sets, the proposed method delivers an impressive performance compared with other state-of-the-art methods. Furthermore, additional performance analysis demonstrates the effect and usefulness of each component of the proposed method.

Full Article | PDF Article

More Like This

Robust object tracking based on local discriminative sparse representation

Xin Wang, Siqiu Shen, Chen Ning, Yuzhen Zhang, and Guofang Lv
J. Opt. Soc. Am. A 34(4) 533-544 (2017)

Online tracking of deformable objects under occlusion using dominant points

Dilip K. Prasad and Michael S. Brown
J. Opt. Soc. Am. A 30(8) 1484-1491 (2013)

Efficient object detection and tracking in video sequences

Fadi Dornaika and Fadi Chakik
J. Opt. Soc. Am. A 29(6) 928-935 (2012)

Previous Article Next Article

Cited By

You do not have subscription access to this journal. Cited by links are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access Optica Member Subscription

Figures (10)

You do not have subscription access to this journal. Figure files are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access Optica Member Subscription

Tables (3)

You do not have subscription access to this journal. Article tables are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access Optica Member Subscription

Equations (31)

You do not have subscription access to this journal. Equations are available to subscribers only. You may subscribe either as an Optica member, or as an authorized user of your institution.

Contact your librarian or system administrator
or
Login to access Optica Member Subscription

Symbol	Description
$r_{i}^{t}$	$i$ th track at frame $t$
$d_{j}^{t}$	$j$ th detection at frame $t$
$n_{R}$	number of tracks
$n_{D}$	number of detections
$x_{r_{i}}$	state of track $r_{i}$
$F$	affinity of the object tracking step
$Ψ_{a}$	appearance affinity
$Ψ_{s}$	shape affinity
$Ψ_{p}$	position affinity
$ζ_{d_{j}}$	template of detection $d_{j}$
$ζ_{r_{j}}^{l}$	latest template of track $r_{i}$
$ζ_{r_{i}}^{h}$	historical template of track $r_{i}$
$H_{r_{i}}$	historical template set of track $r_{i}$
$η_{r_{i}}^{t}$	number of consecutively missed observations for track $r_{i}$ continued until frame $t$
$A_{r_{i}}^{t}$	indicator of $r_{i}$ ’s association in the object tracking step at frame $t$
$A_{d_{j}}^{t}$	indicator of $d_{j}$ ’s association in the object tracking step at frame $t$
$Ω_{r_{i}}^{t}$	confidence of track $r_{i}$ at frame $t$
$Ω_{A, r_{i}}^{t}$	appearance confidence in $Ω_{r_{i}}^{t}$
$Ω_{C, r_{i}}^{t}$	observation continuity confidence in $Ω_{r_{i}}^{t}$
$r_{i (u)}^{t}$	$i$ th unreliable track at frame $t$
$d_{j (u)}^{t}$	$j$ th unassociated detection at frame $t$
$G$	affinity of the first drift handling step
$Γ_{r_{i (u)}}^{t}$	valid association range of track $r_{i (u)}$ at frame $t$ in the first drift handling step
$Θ_{r_{i}}^{t}$	temporal motion similarity of track $r_{i}$ at frame $t$
$r_{i (c)}^{t}$	$i$ th corrected track at frame $t$
$r_{j (n)}^{t}$	$j$ th newly initialized track at frame $t$
$H$	affinity of the second drift handling step
$Ψ_{m}$	motion affinity between tracks
$r_{i (e)}^{t}$	$i$ th temporarily terminated track at frame $t$
$I$	affinity of the fragmentation handling step

Methods	MOTP (%)	MOTA (%)	MT (%)	ML (%)	IDS
Exp. (1)	74.21	60.66	46.05	7.56	200.0
Exp. (2)	74.30	61.28	46.39	7.56	195.7
Exp. (3)	74.32	65.31	50.52	7.22	188.3
Exp. (4)	74.42	68.67	52.58	6.87	130.3

Data Set	Methods		MOTP (%)	MOTA (%)	MT (%)	ML (%)	IDS
PETS’09 S2.L1	Online	Proposed	75.9	90.4	94.7	0.0	3
	Online	Breitenstein et al. [14]	56.3	79.7	–	–	–
	Online	Yang et al. [12]	53.8	75.9	–	–	–
	Offline	Berclaz et al. [4]	72.0	80.3	73.9	8.7	13
	Offline	Milan et al. [5]	80.2	90.6	91.3	4.3	11
PETS’09 S2.L2	Online	Proposed	71.6	58.6	37.2	0.0	225
	Online	Breitenstein et al. [14]	51.3	50.0	–	–	–
	Offline	Milan et al. [5]	59.4	56.9	37.8	16.2	99
Town-Centre	Online	Proposed	74.9	68.8	57.0	7.7	162
	Online	Pellegrini et al. [13]	70.7	63.4	59.1	7.0	288
	Online	Yamaguchi et al. [21]	71.1	63.3	58.1	6.5	302
	Offline	Leal-Taixe et al. [6]	71.5	67.3	58.6	7.0	165
MOT Challenge	Online	OMT_DFH (Proposed)	70.5	34.3	13.5	22.6	65.8
	Online	RMOT [22]	70.2	30.4	10.4	27.5	74.0
	Online	TC_ODAL [23]	70.9	24.2	5.4	29.0	75.2
	Offline	CEM [5]	71.2	28.1	14.3	24.2	87.2
	Offline	SMOT [10]	71.7	23.9	2.8	32.2	120.2
	Offline	TBD [11]	71.3	28.0	12.7	24.3	192.2
	Offline	DP_NMS [8]	71.4	22.7	6.0	20.8	529.2
ISSIA	Online	Proposed	72.8	68.3	56.6	17.1	63.3
	Online	Yoon et al. [22]	72.3	67.7	52.5	19.0	73.3
	Online	Bae and Yoon [23]	72.7	65.5	41.7	19.0	80.2
	Offline	Pirsiavash et al. [8]	73.1	49.7	28.4	20.6	2035.5

Symbol	Description
$r_{i}^{t}$	$i$ th track at frame $t$
$d_{j}^{t}$	$j$ th detection at frame $t$
$n_{R}$	number of tracks
$n_{D}$	number of detections
$x_{r_{i}}$	state of track $r_{i}$
$F$	affinity of the object tracking step
$Ψ_{a}$	appearance affinity
$Ψ_{s}$	shape affinity
$Ψ_{p}$	position affinity
$ζ_{d_{j}}$	template of detection $d_{j}$
$ζ_{r_{j}}^{l}$	latest template of track $r_{i}$
$ζ_{r_{i}}^{h}$	historical template of track $r_{i}$
$H_{r_{i}}$	historical template set of track $r_{i}$
$η_{r_{i}}^{t}$	number of consecutively missed observations for track $r_{i}$ continued until frame $t$
$A_{r_{i}}^{t}$	indicator of $r_{i}$ ’s association in the object tracking step at frame $t$
$A_{d_{j}}^{t}$	indicator of $d_{j}$ ’s association in the object tracking step at frame $t$
$Ω_{r_{i}}^{t}$	confidence of track $r_{i}$ at frame $t$
$Ω_{A, r_{i}}^{t}$	appearance confidence in $Ω_{r_{i}}^{t}$
$Ω_{C, r_{i}}^{t}$	observation continuity confidence in $Ω_{r_{i}}^{t}$
$r_{i (u)}^{t}$	$i$ th unreliable track at frame $t$
$d_{j (u)}^{t}$	$j$ th unassociated detection at frame $t$
$G$	affinity of the first drift handling step
$Γ_{r_{i (u)}}^{t}$	valid association range of track $r_{i (u)}$ at frame $t$ in the first drift handling step
$Θ_{r_{i}}^{t}$	temporal motion similarity of track $r_{i}$ at frame $t$
$r_{i (c)}^{t}$	$i$ th corrected track at frame $t$
$r_{j (n)}^{t}$	$j$ th newly initialized track at frame $t$
$H$	affinity of the second drift handling step
$Ψ_{m}$	motion affinity between tracks
$r_{i (e)}^{t}$	$i$ th temporarily terminated track at frame $t$
$I$	affinity of the fragmentation handling step

Methods	MOTP (%)	MOTA (%)	MT (%)	ML (%)	IDS
Exp. (1)	74.21	60.66	46.05	7.56	200.0
Exp. (2)	74.30	61.28	46.39	7.56	195.7
Exp. (3)	74.32	65.31	50.52	7.22	188.3
Exp. (4)	74.42	68.67	52.58	6.87	130.3

Abstract

Cited By

Figures (10)

Tables (3)

Equations (31)

Journal of the Optical Society of America A