Added two tables to back up claims made in text

Signed-off-by: Jim Martens <github@2martens.de>
This commit is contained in:
2019-09-25 15:06:44 +02:00
parent 990059784a
commit 8bc3ab5d7a

View File

@ -982,6 +982,25 @@ threshold indicates a worse performance.
\subsection*{Non-Maximum Suppression and Top \(k\)}
\begin{table}[htbp]
\centering
\begin{tabular}{rccc}
\hline
variant & before & after & after \\
& entropy/NMS & entropy/NMS & top \(k\) \\
\hline
Bay. SSD, no dropout, no NMS & 155,251 & 122,868 & 72,207 \\
no dropout, NMS & 155,250 & 36,061 & 33,827 \\
\hline
\end{tabular}
\caption{Comparison of Bayesian SSD variants without dropout with
respect to the number of detections before the entropy threshold,
after it and/or non-maximum suppression, and after top \(k\). The
entropy threshold 1.5 was used for both.}
\label{tab:effect-nms}
\end{table}
Miller et al.~\cite{Miller2018} supposedly did not use non-maximum suppression
in their implementation of dropout sampling. Therefore, a variant with disabled
non-maximum suppression (NMS) was tested. The results are somewhat expected:
@ -994,7 +1013,8 @@ duplicate detections could stay and maxima boxes could be removed.
The number of observations was measured before and after the combination of entropy threshold and NMS filter: both Bayesian SSD without
NMS and dropout, and Bayesian SSD with NMS and disabled dropout
have the same number of observations everywhere before the entropy threshold. After the entropy threshold (the value 1.5 was used for both) and NMS, the variant with NMS has roughly 23\% of its observations left.
have the same number of observations everywhere before the entropy threshold. After the entropy threshold (the value 1.5 was used for both) and NMS, the variant with NMS has roughly 23\% of its observations left
(see table \ref{tab:effect-nms} for absolute numbers).
Without NMS 79\% of observations are left. Irrespective of the absolute
number, this discrepancy clearly shows the impact of non-maximum suppression and also explains a higher count of false positives:
more than 50\% of the original observations were removed with NMS and
@ -1029,6 +1049,25 @@ recall.
\subsection*{Dropout Sampling and Observations}
\begin{table}[htbp]
\centering
\begin{tabular}{rccc}
\hline
variant & after & after \\
& prediction & observation grouping \\
\hline
Bay. SSD, no dropout, NMS & 1,677,050 & 155,250 \\
keep rate 0.9, NMS & 1,617,675 & 549,166 \\
\hline
\end{tabular}
\caption{Comparison of Bayesian SSD variants without dropout and with
0.9 keep ratio of dropout with
respect to the number of detections directly after the network
predictions and after the observation grouping.}
\label{tab:effect-dropout}
\end{table}
The dropout variants have largely worse performance than the Bayesian variants
without dropout. This is expected as the network was not trained with
dropout and the weights are not prepared for it.
@ -1052,7 +1091,7 @@ The number of detections per class was measured before and after the
detections were grouped into observations. To this end, the stored predictions
were unbatched and summed together. After the aforementioned filter
and before the grouping, roughly 0.4\% (in fact less than that) of the
more than 430 million detections are remaining. The variant with dropout
more than 430 million detections are remaining (see table \ref{tab:effect-dropout} for absolute numbers). The variant with dropout
has slightly fewer predictions left compared to the one without dropout.
After the grouping, the variant without dropout has on average between