Questionnaire about automatic video processing tools to assist in annotation tasks

Reaching our goals requires annotated sign language videos. Sign language corpora are, in general, manually annotated by linguists and computer scientists using annotation software tools. However, annotation is a time-consuming and error-prone process. This is why we are developing automatic video processing tools to assist in annotation tasks. For example, an annotator is working in an annotation software window (Fig. a), annotations are saved in the two tiers: AG1 and AG2. When the annotator needs an automatic treatment, e.g. posture detection, which takes two input parameters, then two additional tiers appear in the window (Fig. b). Filling in the two tiers (input parameters) can be done manually or using AG1 and AG2. Once the treatment has finished processing, the result is displayed as a new tier that the annotator can easily save or modify (Fig. c).

example of access to automatic treatment from an annotation tool

Some proposed processes that we intend to develop are listed below. We would be glad to have your opinions on the usefulness of the automatic processes proposed. In the context of annotation, could you please classify the automatic processes below as Very Useful, Useful or Not Useful for your work ?
Automatic Annotation ToolDescriptionVery
Useful		UsefulNot
Useful
1. Face TrackerSegment out the face and enlarge to aid with expression/lip shape discrepancies.___ Very Useful Useful Not Useful
2. Signing Space calibrationIdentify the extent of the normal signing space used in a performance.___ Very Useful Useful Not Useful
3. SegmentationPotential start and end of a sign.___ Very Useful Useful Not Useful
4. Detention (D) detectorA D is a part of a sign, which is motivated, during which no feature is changing.___ Very Useful Useful Not Useful
5. Posture (P) detectorA P is a part of a sign, which is motivated but transient.___ Very Useful Useful Not Useful
6. Transition (T) trajectory shape detectortransition occurs between 2 Ds/Ps. This is the part(s) if a sign during which at least one feature is changing.___ Very Useful Useful Not Useful
7. Transition ballistic trajectory detectorIf acceleration is null, this is not a ballistic movement, if acceleration is positive, this is a ballistic movement.___ Very Useful Useful Not Useful
8. Symmetry detectorDetection of symmetry between hand trajectories___ Very Useful Useful Not Useful
9. Handedness detectorMark signs as one-handed or two-handed___ Very Useful Useful Not Useful
10. Limb orientation detectorDetection of hand, forearm, arm orientation, regarding two axis: main axis along the limb, and normal axis, for example palm orientation.___ Very Useful Useful Not Useful
11. Detection of elbow locationsThis detector should be applied during a key posture. Site location detector.___ Very Useful Useful Not Useful
12. Detection of hand locationsThis detector should be applied during a key posture. Site location detector.___ Very Useful Useful Not Useful
13. Contact detectorDetection of contact between sites, such as SH (strong hand) index tip with nose, SH elbow with WH (weak hand) palm...___ Very Useful Useful Not Useful
14. Eye Gaze orientation detectorDetection of eye gaze orientation___ Very Useful Useful Not Useful
15. Head Pose orientation detectorDetection of Head Pose orientation___ Very Useful Useful Not Useful
16. Eye lid detectorOpen, shut, partial...___ Very Useful Useful Not Useful
17. Eyebrow detectorRaised, frown___ Very Useful Useful Not Useful
18. Cheek detectorSwollen, sunken...___ Very Useful Useful Not Useful
19. Tongue detectorPosition of the tongue: In, out___ Very Useful Useful Not Useful
20. Shoulder detectorPosition of the tip of the shoulder: shrugged...___ Very Useful Useful Not Useful
21. Movement featuresTo identify certain aspects of the movements, like straight vs. circular, repeated etc. Same for hand shapes and locations___ Very Useful Useful Not Useful
22. NMF alignmentSegment onset and conclusion of NMFs with respect to manual signing.___ Very Useful Useful Not Useful
23. Signer recognitionTry to identify the signer(s) shown in a video. In a database with hundreds (or in the long run even thousands) of informants.___ Very Useful Useful Not Useful
24. Modified token detectorDetection of tokens that are substantially different from the type's prototype.___ Very Useful Useful Not Useful
Page last modified on October 01, 2010, at 11:33 AM