SIO-1483 has nothing common with this report.
Remove submission in Evaluation queue prevents running the eval_mgr job, as I suppose, right?
But it doesn't remove submission, which is created before all that staff. This submission is created with default status "?" (which is converted to "Pending.." ofc).
Submission status is set in controllers:update_submission_score and based on submission report status (precisely - 'INITIAL' kind).
BUT, this method is called by handler during eval_mgr, so submission status wouldn't be updated.
Moreover, there is no submission report (no 'INITIAL' kind report) connected with this submission, because of the same reason - eval_mgr was stopped (notice that submission report is created in handlers during eval_mgr, at the end in fact - look at programs:handlers:_make_base_report which is called by make_report handler).
SIO-1483would probably fix this.