Added two tables to back up claims made in text
Signed-off-by: Jim Martens <github@2martens.de>
This commit is contained in:
43
body.tex
43
body.tex
@ -982,6 +982,25 @@ threshold indicates a worse performance.
|
|||||||
|
|
||||||
\subsection*{Non-Maximum Suppression and Top \(k\)}
|
\subsection*{Non-Maximum Suppression and Top \(k\)}
|
||||||
|
|
||||||
|
\begin{table}[htbp]
|
||||||
|
\centering
|
||||||
|
\begin{tabular}{rccc}
|
||||||
|
\hline
|
||||||
|
variant & before & after & after \\
|
||||||
|
& entropy/NMS & entropy/NMS & top \(k\) \\
|
||||||
|
\hline
|
||||||
|
Bay. SSD, no dropout, no NMS & 155,251 & 122,868 & 72,207 \\
|
||||||
|
no dropout, NMS & 155,250 & 36,061 & 33,827 \\
|
||||||
|
\hline
|
||||||
|
\end{tabular}
|
||||||
|
|
||||||
|
\caption{Comparison of Bayesian SSD variants without dropout with
|
||||||
|
respect to the number of detections before the entropy threshold,
|
||||||
|
after it and/or non-maximum suppression, and after top \(k\). The
|
||||||
|
entropy threshold 1.5 was used for both.}
|
||||||
|
\label{tab:effect-nms}
|
||||||
|
\end{table}
|
||||||
|
|
||||||
Miller et al.~\cite{Miller2018} supposedly did not use non-maximum suppression
|
Miller et al.~\cite{Miller2018} supposedly did not use non-maximum suppression
|
||||||
in their implementation of dropout sampling. Therefore, a variant with disabled
|
in their implementation of dropout sampling. Therefore, a variant with disabled
|
||||||
non-maximum suppression (NMS) was tested. The results are somewhat expected:
|
non-maximum suppression (NMS) was tested. The results are somewhat expected:
|
||||||
@ -994,7 +1013,8 @@ duplicate detections could stay and maxima boxes could be removed.
|
|||||||
|
|
||||||
The number of observations was measured before and after the combination of entropy threshold and NMS filter: both Bayesian SSD without
|
The number of observations was measured before and after the combination of entropy threshold and NMS filter: both Bayesian SSD without
|
||||||
NMS and dropout, and Bayesian SSD with NMS and disabled dropout
|
NMS and dropout, and Bayesian SSD with NMS and disabled dropout
|
||||||
have the same number of observations everywhere before the entropy threshold. After the entropy threshold (the value 1.5 was used for both) and NMS, the variant with NMS has roughly 23\% of its observations left.
|
have the same number of observations everywhere before the entropy threshold. After the entropy threshold (the value 1.5 was used for both) and NMS, the variant with NMS has roughly 23\% of its observations left
|
||||||
|
(see table \ref{tab:effect-nms} for absolute numbers).
|
||||||
Without NMS 79\% of observations are left. Irrespective of the absolute
|
Without NMS 79\% of observations are left. Irrespective of the absolute
|
||||||
number, this discrepancy clearly shows the impact of non-maximum suppression and also explains a higher count of false positives:
|
number, this discrepancy clearly shows the impact of non-maximum suppression and also explains a higher count of false positives:
|
||||||
more than 50\% of the original observations were removed with NMS and
|
more than 50\% of the original observations were removed with NMS and
|
||||||
@ -1029,6 +1049,25 @@ recall.
|
|||||||
|
|
||||||
\subsection*{Dropout Sampling and Observations}
|
\subsection*{Dropout Sampling and Observations}
|
||||||
|
|
||||||
|
\begin{table}[htbp]
|
||||||
|
\centering
|
||||||
|
\begin{tabular}{rccc}
|
||||||
|
\hline
|
||||||
|
variant & after & after \\
|
||||||
|
& prediction & observation grouping \\
|
||||||
|
\hline
|
||||||
|
Bay. SSD, no dropout, NMS & 1,677,050 & 155,250 \\
|
||||||
|
keep rate 0.9, NMS & 1,617,675 & 549,166 \\
|
||||||
|
\hline
|
||||||
|
\end{tabular}
|
||||||
|
|
||||||
|
\caption{Comparison of Bayesian SSD variants without dropout and with
|
||||||
|
0.9 keep ratio of dropout with
|
||||||
|
respect to the number of detections directly after the network
|
||||||
|
predictions and after the observation grouping.}
|
||||||
|
\label{tab:effect-dropout}
|
||||||
|
\end{table}
|
||||||
|
|
||||||
The dropout variants have largely worse performance than the Bayesian variants
|
The dropout variants have largely worse performance than the Bayesian variants
|
||||||
without dropout. This is expected as the network was not trained with
|
without dropout. This is expected as the network was not trained with
|
||||||
dropout and the weights are not prepared for it.
|
dropout and the weights are not prepared for it.
|
||||||
@ -1052,7 +1091,7 @@ The number of detections per class was measured before and after the
|
|||||||
detections were grouped into observations. To this end, the stored predictions
|
detections were grouped into observations. To this end, the stored predictions
|
||||||
were unbatched and summed together. After the aforementioned filter
|
were unbatched and summed together. After the aforementioned filter
|
||||||
and before the grouping, roughly 0.4\% (in fact less than that) of the
|
and before the grouping, roughly 0.4\% (in fact less than that) of the
|
||||||
more than 430 million detections are remaining. The variant with dropout
|
more than 430 million detections are remaining (see table \ref{tab:effect-dropout} for absolute numbers). The variant with dropout
|
||||||
has slightly fewer predictions left compared to the one without dropout.
|
has slightly fewer predictions left compared to the one without dropout.
|
||||||
|
|
||||||
After the grouping, the variant without dropout has on average between
|
After the grouping, the variant without dropout has on average between
|
||||||
|
|||||||
Reference in New Issue
Block a user