concept continuous task in category machine learning
appears as: continuous tasks, continuous task
This is an excerpt from Manning's book Human-in-the-Loop Machine Learning MEAP V09.
Ground-Truth for continuous tasks is most often implemented as an acceptable range of responses. For example, if you have a sentiment analysis task on a 0-100 scale and you have a very positive item, then you might accept any annotation in an 80-100 range as being “correct” and anything below 80 as “incorrect”. This allows you to treat quality control as if it was a labeling and so all the methods in the last chapter can be applied.
Figure CONT_AGREE Two ways of calculating the expected agreement in a continuous task: the chance that a random number falls in that range, and the percent of annotations across the entire dataset that fall into the range.
