How ScaleOps Delivers Hi-Quality, Pixel Perfect Annotations
Here at ScaleOps, we obsessed with Quality and Acuracy.
Labeling Test as part of Interview
- All Interviewees must pass a labeling test prior to offer. That is our first level of weeding.
Real world Test
- In the first few days, following indoctrination, new annotators are given “gold” tasks – Tasks for which we already have accurate annotations.
- Their output is compared to canned results. All annotators must meet quality consensus prior to moving onto real tasks if they are unable to meet these guidelines within the first week of joining then they are asked to leave.
Training and Indoctrination
- To new joiners, we deliver extensive training during the indoctrination process, with each new joiner performing a few annotations on the presentation screen while trainers, and others, watch.
- Small errors are pointed out and the entire batch is educated on how to avoid these.
EARLY CONSENSUS & REDUNDANCY
- Real Tasks awarded to new annotators in the first week are consensus-based, where the same task is also awarded to a more experienced annotator, and a consensus must be achieved prior to marking the task complete.