PAGER – Tagging a Parametric Avatar Gesture Expression Response

The Avatar natural motion subsystems were constructed so that synthesized speech SSML tags could be read from the speech response of the TTS system and modify the performance or range of motions during any given response. By passing tags inserted into SILVIA responses the Avatar gesturing performance is synchronized with the speech much like a viseme synchronizes with a phoneme. Unlike a phoneme any range of parameters or blending of pose and gesture assets can generate motion as appropriate to the context in which the response occurs.

The form of the tag is similar to standard XML and is formatted with the pose or gesture subsystem or pose sequence in the opening brackets, then the name of the asset followed by as many parameters as allows you to tune the performance, timing or looping. In its basic form it may pass a name of a pose or gesture asset indicating the subsystem should load the asset named between start tag and end tag and can be placed anywhere in the SILVIA response strings.


<ArmsGesture-MyArmsGesture03/>

Perhaps you want multiple gestures to be used in a semi-random manner. You would want to pass the name of those gestures and the timing min-max range. This could then be used to pass parameters into a method that control the timing or the lerp between random picks of the gestures passed in.


<ArmsGesture-MyArmsGesture03/>

In this tag everything up to the params=? are the names of assets. The parameters indicate how to use the named assets. In this case a List of the gestures is created, which will be selected from randomly, as the first parameter indicates. The next two floats indicate a minimum and maximum range for the blend interval between the randomly selected gestures. The final bool indicates that cycling is true. The tag is stripped during the response and is sent to a parser which then matches ArmsGestureAsset names to compose a List to select gestures from the ArmsGestureData assets library as well as the timing range.

Since the first “params” was random we know to channel the params into a method constructed to handle this form of a tag.


<ArmsGesture-Gesture03,Gesture06,Gesture07,params=?random,0.5,2.25,true/>

To begin with the current and previous gestures need to be stored as a public field in the component for access across multiple methods to allow lerping. We also have fields to store the current duration of the lerp or interpolation as it is more commonly known as well as a timer. The lerpTimer is incremented in the Update loop and when it equals or exceeds the lerpDuration it resets to zero and selects the next gesture. Each frame a lerping method get called which interpolates between previous and currently selected ArmsGestureAssets.


void Update () {
    lerpTimer += Time.deltaTime;
    if (lerpTimer >= lerpDuration) {
        lerpTimer = 0f;
        currGesture =  nextGesture;
        lerpDuration = Random.Range(durationMinMax.x, durationMinMax.y);
}
    LerpArmsGesture ();
}
public void LerpArmsGesture () {
    //get the normalized time for the lerp interpolation
    float t = lerpTimer / lerpDuration;
    gestureCtrl.leftFrontToBack = Mathf.Lerp(currGesture.leftFrontToBack,
nextGesture.leftFrontToBack, t);
    gestureCtrl.leftUpDown = Mathf.Lerp(currGesture.leftUpdown,
nextGesture.leftUpdown, t);
    gestureCtrl.leftInOut = Mathf.Lerp(currGesture.leftInOut, nextGesture.leftInOut,
t);
gestureCtrl.rightFrontToBack = Mathf.Lerp(currGesture.rightFrontToBack,
       nextGesture.rightFrontToBack, t);
    gestureCtrl.rightUpDown = Mathf.Lerp(currGesture.rightUpdown,
nextGesture.rightUpdown, t);
    gestureCtrl.rightInOut = Mathf.Lerp(currGesture.rightInOut,
nextGesture.rightInOut, t);
}

Using a similarly styled tag the gestures could be randomly selected from pre-populated Lists that each comprise a group of poses or gestures in character with the speaking, idling, listening, emphasizing of the Avatar interactive performance. In this example we want a Procedural Idle that uses 7 different ArmsGestureAssets residing in a premade list, chosen randomly and interpolated in a random range of time. To designate a List during parsing that is already declared, surround the name with curly brackets. The name of the predefined List of ArmsGestureAssets after the all uppercase LIST attribute signifying it is a public List by the name of the string following the attribute as in the example below where the list is named “idleGesturesList”.


<ArmsGesture-LIST idleArmsGestures,params=?random,0.5,2.25,true/>

When parsing the tag it gets a reference to the idleArmsGestures List and passes that into the RandomlyCycleGestures method whereas in the previous example the List was created from the comma delimited names of the gestures preceding the params=? keyword string.

n the above example we used the attribute LIST to let the parser know the List is pre-defined and named. You could pass multiple Lists in the PAGER tag by comma delimiting them. A method of name could be called by using the attribute METHOD. In the following tag we tell the parser we are looking for a reference to a List named idleArmsGestures and send that to a method named SetGestureTimings which will create a List of matched indexed timings use them in a sequence which will interpolate from idleArmsGesture[n] to idleArmsGesture[n+1] at timings[n].


<ArmsGesture-LIST idleArmsGestures,METHOD SetGestureTimings,params=?sequence/>

The SILVIA PAGES