Update 1.1.2.0 eyelids getting stuck???

[AirMouse]

I just accepted the update to 1.1.2.0 and now suddenly the eyelids for all of my characters get stuck partially open:

What has changed to make the eyes get stuck now?

How can I change it back?

Thank you.

Edited November 4, 2019 by AirMouse

[AirMouse]

BUG CONFIRMED!!!!

I just down-graded to 1.1.0.1 and the eyelids work WAY BETTER (like they used to):

So this proves that the new update to 1.1.2.0 is totally broken.

Can you please fix it.?

Edited November 4, 2019 by AirMouse

[AirMouse]

Also I have tried several times to uninstall/reinstall v1.1.2.0 on my Win10 i9-9900K (dual 2080TI) PC.

I also reboot and tried it again and again. But no matter what I do the new v1.1.2.0 looks absolutely horrible!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

But v1.1.0.1 looks awesome and works perfectly on all of my PCs.

 

Could you please re-evaluate your latest release to find the bug that causes the eyelids to get stuck partially open?

I will continue to use v1.1.0.1 myself but what the heck do I tell my users??? That they should download https://hub.vive.com/en-US/download/VIVE_SRanipalInstaller_1.1.0.1.msi and uninstall v1.1.2.0, then never update?? That's kind of a lame workaround IMHO.

 

This new bug makes my software look awful and I am very embarrassed to show it. Can you please fix this soon???????????????????????

 

@Daniel_Y @Corvus

[Daniel_Y]

Do you use EyeData.timestamp in your code? 

 v1.1.2.0 addressed an issue existed in v.1.1.0.1 that returns a wrong unit of timestamp. If you did your own workaround for it in your code with v.1.1.0.1, it may have unexpected behavior with  v1.1.2.0.

[AirMouse]

 

7 hours ago, Daniel_Y said:

Do you use EyeData.timestamp in your code? 

 v1.1.2.0 addressed an issue existed in v.1.1.0.1 that returns a wrong unit of timestamp. If you did your own workaround for it in your code with v.1.1.0.1, it may have unexpected behavior with  v1.1.2.0.

I just searched through my script and could find no mention of `timestamp` anywhere, so I don't think I am using it.

I have attached my re-write of the SRanipal_AvatarEyeSample.cs script.

I have no clue where to even look to find the bug, things were working perfectly until the update hit and now it's unusable right at a critical time. This has been very frustrating and I can't solve this by my self as I have no idea what to even look for.

Could you please help me understand what I have to change in my scripts so the eyelids will close normally again when using v1.1.2.0?

 

Thank you.

 

//========= Copyright 2018, HTC Corporation. All rights reserved. ===========
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.Assertions;
using System;
using Unity.Mathematics;
using UnityEditor;

namespace ViveSR
{
   namespace anipal
   {
      namespace Eye
      {
         public class SRanipal_AvatarEyeSample2 : MonoBehaviour
         {
            [SerializeField] private Transform[] EyesModels = new Transform[0];
            public List<EyeShapeTable> EyeShapeTables;
            /// <summary>
            /// Customize this curve to fit the blend shapes of your avatar.
            /// </summary>
            [SerializeField] public AnimationCurve EyebrowAnimationCurveUpper;
            /// <summary>
            /// Customize this curve to fit the blend shapes of your avatar.
            /// </summary>
            [SerializeField] public AnimationCurve EyebrowAnimationCurveLower;
            /// <summary>
            /// Customize this curve to fit the blend shapes of your avatar.
            /// </summary>
            [SerializeField] public AnimationCurve EyebrowAnimationCurveHorizontal;

            public bool NeededToGetData = true;
            public Dictionary<EyeShape, float> EyeWeightings = new Dictionary<EyeShape, float>();
            private AnimationCurve[] EyebrowAnimationCurves = new AnimationCurve[(int)EyeShape.Max];
            private GameObject[] EyeAnchors;
            private const int NUM_OF_EYES = 2;

            private Armature m_armature;

            //private void Start()
            public void Initialize(Armature armature, Transform leftEye, Transform rightEye)
            {        
               Selection.activeGameObject=this.gameObject;
               if (!SRanipal_Eye_Framework.Instance.EnableEye)
               {
                  enabled = false;
                  return;
               }

               
               
               eyeWashV3 = new Vector3(eyeWash.x, eyeWash.y, 1.0f);

               m_armature = armature;

               EyeShapeTables = armature.eyeTablesList;
               EyesModels = new Transform[] {leftEye, rightEye};

               SetEyesModels(EyesModels[0], EyesModels[1]);
               SetEyeShapeTables(EyeShapeTables);

               AnimationCurve[] curves = new AnimationCurve[(int)EyeShape.Max];
               for (int i = 0; i < EyebrowAnimationCurves.Length; ++i)
               {
                  if (i == (int)EyeShape.Eye_Left_Up || i == (int)EyeShape.Eye_Right_Up)          curves[i] = EyebrowAnimationCurveUpper;
                  else if (i == (int)EyeShape.Eye_Left_Down || i == (int)EyeShape.Eye_Right_Down) curves[i] = EyebrowAnimationCurveLower;
                  else                                                                            curves[i] = EyebrowAnimationCurveHorizontal;
               }
               SetEyeShapeAnimationCurves(curves);

               SRanipal_Eye.GetEyeWeightings(out EyeWeightings);
               UpdateEyeShapes(EyeWeightings);
            }


            public Vector3 directionOverrideLeft = Vector3.forward;
            public Vector3 directionOverrideRight = Vector3.forward;

            bool isLeftEyeActiveLast = false;
            bool isRightEyeActiveLast = false;

            float isLeftEyeActiveTime = 0;
            float isRightEyeActiveTime = 0;

            const float blinkEyeDirectionBlendTime = .25f;

            Vector3 leftEyeDirectionLast = Vector3.zero;
            Vector3 rightEyeDirectionLast = Vector3.zero;

            public void PostUpdate() //must be done after vrik
            {
               if (SRanipal_Eye_Framework.Status != SRanipal_Eye_Framework.FrameworkStatus.WORKING &&
                  SRanipal_Eye_Framework.Status != SRanipal_Eye_Framework.FrameworkStatus.NOT_SUPPORT) return;
               
               if (NeededToGetData)
               {
                  EyeData eyeData = new EyeData();
                  SRanipal_Eye.GetEyeData(ref eyeData);

                  bool isLeftEyeActive = false;
                  bool isRightEyeAcitve = false;
                  if (SRanipal_Eye_Framework.Status == SRanipal_Eye_Framework.FrameworkStatus.WORKING)
                  {
                     isLeftEyeActive = eyeData.verbose_data.left.GetValidity(SingleEyeDataValidity.SINGLE_EYE_DATA_GAZE_ORIGIN_VALIDITY);
                     isRightEyeAcitve = eyeData.verbose_data.right.GetValidity(SingleEyeDataValidity.SINGLE_EYE_DATA_GAZE_ORIGIN_VALIDITY);
                  }
                  else if (SRanipal_Eye_Framework.Status == SRanipal_Eye_Framework.FrameworkStatus.NOT_SUPPORT)
                  {
                     isLeftEyeActive = true;
                     isRightEyeAcitve = true;
                  }

                  if (isLeftEyeActive || isRightEyeAcitve)
                  {
                     SRanipal_Eye.GetEyeWeightings(out EyeWeightings);
                     UpdateEyeShapes(EyeWeightings);
                  }
                  else
                  {
                     /*
                     for (int i = 0; i < (int)EyeShape.Max; ++i)
                     {
                        bool isBlink = ((EyeShape)i == EyeShape.Eye_Left_Blink || (EyeShape)i == EyeShape.Eye_Right_Blink);
                        EyeWeightings[(EyeShape)i] = isBlink ? 1 : 0;
                     }
                     */

                     //UpdateEyeShapes(EyeWeightings);
                     //UpdateEyeShapesManalOverride(EyeWeightings);



                     //Debug.Log ("UpdateGazeRayIndexed");
                     //UpdateGazeRayIndexed (directionOverrideLeft, EyeIndex.LEFT);
                     //UpdateGazeRayIndexed (directionOverrideRight, EyeIndex.RIGHT);


                     //return;
                  }

                  isLeftEyeActive = eyeData.verbose_data.left.eye_openness > 0.1f;
                  isRightEyeAcitve = eyeData.verbose_data.right.eye_openness > 0.1f;

                  if (UI_Panel_Tracking.Instance.seperateEyeTrackingToggle.isOn) {
                     //Get the rays for each eye seperately
                     Ray gazeRay = new Ray ();
                     foreach (int eyeIndex in Enum.GetValues(typeof(EyeIndex))) {
                        SRanipal_Eye.GetGazeRay ((GazeIndex)eyeIndex, out gazeRay);//, out GazeDirectionLocal);

                        Vector3 direction = Vector3.Scale (gazeRay.direction, eyeWashV3);

                        /*switch ((GazeIndex)eyeIndex) {
                        case GazeIndex.LEFT:
                           direction.x += divergence;
                           direction.y += kappaAngle;
                           break;
                        case GazeIndex.RIGHT:
                           direction.x -= divergence;
                           direction.y += kappaAngle;
                           break;
                        }*/
                        ApplyEyeDirection (eyeIndex, direction, isLeftEyeActive, isRightEyeAcitve);
                     }
                  } else {
                     Vector3 GazeOriginCombinedLocal, GazeDirectionCombinedLocal = Vector3.zero;
                     if (SRanipal_Eye.GetGazeRay (GazeIndex.COMBINE, out GazeOriginCombinedLocal, out GazeDirectionCombinedLocal)) {
                     } else if (SRanipal_Eye.GetGazeRay (GazeIndex.LEFT, out GazeOriginCombinedLocal, out GazeDirectionCombinedLocal)) {
                     } else if (SRanipal_Eye.GetGazeRay (GazeIndex.RIGHT, out GazeOriginCombinedLocal, out GazeDirectionCombinedLocal)) {
                     }

                     //Get the rays for each eye seperately
                     foreach (int eyeIndex in Enum.GetValues(typeof(EyeIndex))) {

                        var direction = Vector3.Scale (GazeDirectionCombinedLocal, eyeWashV3);  ;

                        ApplyEyeDirection (eyeIndex, direction, isLeftEyeActive, isRightEyeAcitve);
                              
                     }     
                  
                  }

                  isLeftEyeActiveLast = isLeftEyeActive;
                  isRightEyeActiveLast = isRightEyeAcitve;
               }
            }


            void ApplyEyeDirection(int eyeIndex, Vector3 direction, bool isLeftEyeActive, bool isRightEyeAcitve) {
               switch ((GazeIndex)eyeIndex) {
               case GazeIndex.LEFT:                           
                  if (isLeftEyeActiveLast != isLeftEyeActive) {
                     if (isLeftEyeActive) {
                        isLeftEyeActiveTime = Time.time + blinkEyeDirectionBlendTime;
                     }
                  }

                  direction.x += divergence;
                  direction.y += kappaAngle;

                  if (Time.time > isLeftEyeActiveTime) {
                     if (isLeftEyeActive) {
                        leftEyeDirectionLast = direction;
                     }
                  } else {
                     var delt = TrackerLinker.Instance.eyeDirectionBlendCurve.Evaluate( 1 - ((isLeftEyeActiveTime - Time.time) / blinkEyeDirectionBlendTime) );

                     leftEyeDirectionLast = Vector3.Lerp (leftEyeDirectionLast, direction, delt);
                  }

                  UpdateGazeRayIndexed (leftEyeDirectionLast, (EyeIndex)eyeIndex);
                  break;
               case GazeIndex.RIGHT:
                  if (isRightEyeActiveLast != isRightEyeAcitve) {
                     if (isRightEyeAcitve) {
                        isRightEyeActiveTime = Time.time + blinkEyeDirectionBlendTime;
                     }
                  }

                  direction.x -= divergence;
                  direction.y += kappaAngle;

                  if (Time.time > isRightEyeActiveTime) {
                     if (isRightEyeAcitve) {
                        rightEyeDirectionLast = direction;
                     }
                  } else {
                     var delt = TrackerLinker.Instance.eyeDirectionBlendCurve.Evaluate( 1 - ((isRightEyeActiveTime - Time.time) / blinkEyeDirectionBlendTime ));
                     rightEyeDirectionLast = Vector3.Lerp (rightEyeDirectionLast, direction, delt);
                  }

                  UpdateGazeRayIndexed (rightEyeDirectionLast, (EyeIndex)eyeIndex);
                  break;
               }
            }

            [Range (-1,1)]
            public float divergence = 0;

            [Range (-1,1)]
            public float kappaAngle = 0;

            public Vector2 eyeWash = new Vector2 (0.5f, 0.2f);
            public Vector3 eyeWashV3;
            
            public void UpdateGazeRayIndexed(Vector3 gazeDirectionLocal, EyeIndex eyesModelsIndex)
            {
               int i = (int)eyesModelsIndex;
               
               Vector3 target = EyeAnchors[i].transform.TransformPoint(gazeDirectionLocal);
               //Debug.DrawLine ( EyeAnchors[i].transform.position, target, Color.magenta);
               
               EyesModels [i].LookAt(target, m_armature.eyesForward.up);
               switch (eyesModelsIndex)
               {
                  case EyeIndex.LEFT:
                     m_armature.eyeL.localRotation *= m_armature.eyeL_InitialRotation;// m_armature.eyeL_InitialRotation;
                     break;
                  case EyeIndex.RIGHT:
                     m_armature.eyeR.localRotation *= m_armature.eyeR_InitialRotation;// m_armature.eyeR_InitialRotation;
                     break;
               }
               
            }




            private void OnDestroy()
            {
               DestroyEyeAnchors();
            }

            public void SetEyesModels(Transform leftEye, Transform rightEye)
            {
               if (leftEye != null && rightEye != null)
               {
                  EyesModels = new Transform[NUM_OF_EYES] { leftEye, rightEye };
                  DestroyEyeAnchors();
                  CreateEyeAnchors();
               }
            }

            public void SetEyeShapeTables(List<EyeShapeTable> eyeShapeTables)
            {
               bool valid = true;
               if (eyeShapeTables == null)
               {
                  valid = false;
               }
               else
               {
                  for (int table = 0; table < eyeShapeTables.Count; ++table)
                  {
                     if (eyeShapeTables[table].skinnedMeshRenderer == null)
                     {
                        valid = false;
                        break;
                     }
                     for (int shape = 0; shape < eyeShapeTables[table].eyeShapes.Length; ++shape)
                     {
                        EyeShape eyeShape = eyeShapeTables[table].eyeShapes[shape];
                        if (eyeShape > EyeShape.Max || eyeShape < 0)
                        {
                           valid = false;
                           break;
                        }
                     }
                  }
               }
               if (valid)
                  EyeShapeTables = eyeShapeTables;
            }

            public void SetEyeShapeAnimationCurves(AnimationCurve[] eyebrowAnimationCurves)
            {
               if (eyebrowAnimationCurves.Length == (int)EyeShape.Max)
                  EyebrowAnimationCurves = eyebrowAnimationCurves;
            }
               
            [SerializeField] Vector3 gazeDirectionCombinedLocalManualOverride = Vector3.forward;
            public void UpdateGazeRayManualOverride(Vector3 gazeDirectionCombinedLocal)
            {

               //Debug.Log ("UpdateGazeRay");
               for (int i = 0; i < EyesModels.Length; ++i)
               {
                  Vector3 target = EyeAnchors[i].transform.TransformPoint(gazeDirectionCombinedLocalManualOverride);
                  //Debug.DrawLine (EyeAnchors[i].transform.position, target, Color.cyan);
                  EyesModels[i].LookAt(target, transform.up);
                  EyesModels[i].Rotate(Vector3.right * -90);
               }
            }


            public void UpdateGazeRay(Vector3 gazeDirectionCombinedLocal)
                {
                    for (int i = 0; i < EyesModels.Length; ++i)
                    {
                        Vector3 target = EyeAnchors[i].transform.TransformPoint(gazeDirectionCombinedLocal);
                  //Debug.DrawLine (EyeAnchors[i].transform.position, target, Color.magenta);
                        EyesModels[i].LookAt(target);
                  EyesModels[i].Rotate(Vector3.right * 90);
                    }
                }

            [SerializeField][Range(0,1)] float blinkWeightLeftManalOverride = 0;
            [SerializeField][Range(0,1)] float blinkWeightRightManalOverride = 0;
            [SerializeField][Range(0,1)] float browWeightManalOverride = 0;
            public void UpdateEyeShapesManalOverride(Dictionary<EyeShape, float> eyeWeightings)
            {
               eyeWeightings [EyeShape.Eye_Left_Blink] = blinkWeightLeftManalOverride;
               eyeWeightings [EyeShape.Eye_Right_Blink] = blinkWeightRightManalOverride;

               foreach (var table in EyeShapeTables) {

                  for (int i = 0; i < table.eyeShapes.Length; ++i)
                  {
                     EyeShape eyeShape = table.eyeShapes[i];

                     if (eyeShape == EyeShape.Eye_Left_Blink) {
                        eyeWeightings[eyeShape] = blinkWeightLeftManalOverride;
                     } else if (eyeShape == EyeShape.Eye_Right_Blink) {
                        eyeWeightings[eyeShape] = blinkWeightRightManalOverride;
                     } else {
                        eyeWeightings[eyeShape] = browWeightManalOverride;
                     }

                  }

                  RenderModelEyeShape (table, eyeWeightings);
               }

               m_armature.audioFaceRigFFT.UpdateEyelidsLeft (eyeWeightings[EyeShape.Eye_Left_Blink]);
               m_armature.audioFaceRigFFT.UpdateEyelidsRight (eyeWeightings[EyeShape.Eye_Right_Blink]);
            }

            public void UpdateEyeShapes(Dictionary<EyeShape, float> eyeWeightings)
            {
               foreach (var table in EyeShapeTables) {
                  RenderModelEyeShape (table, eyeWeightings);
               }

               //Update eye lid bones (makehuman rigs use bones for eyelids)
               m_armature.audioFaceRigFFT.UpdateEyelidsLeft (eyeWeightings[EyeShape.Eye_Left_Blink]);
               m_armature.audioFaceRigFFT.UpdateEyelidsRight (eyeWeightings[EyeShape.Eye_Right_Blink]);
            }

            private void RenderModelEyeShape(EyeShapeTable eyeShapeTable, Dictionary<EyeShape, float> weighting)
            {
               
               for (int i = 0; i < eyeShapeTable.eyeShapes.Length; ++i)
               {
                  EyeShape eyeShape = eyeShapeTable.eyeShapes[i];
                  //if (eyeShape > EyeShape.Max || eyeShape < 0) continue;
                           
                  if (eyeShape == EyeShape.Eye_Left_Blink || eyeShape == EyeShape.Eye_Right_Blink) {
                     eyeShapeTable.skinnedMeshRenderer.SetBlendShapeWeight(i, weighting[eyeShape] * 100f);
                  }
                  //else
                  //{
                  // AnimationCurve curve = EyebrowAnimationCurves[(int)eyeShape];
                  //
                  // eyeShapeTable.skinnedMeshRenderer.SetBlendShapeWeight(i, curve.Evaluate(weighting[eyeShape]) * 100f); //Crashed here
                  //}
               }

               //Debug.Log("EYE WEIGHT " + weighting[EyeShape.Eye_Left_Blink]);
               //m_armature.audioFaceRigFFT.UpdateEyelidsLeft (weighting[EyeShape.Eye_Left_Blink]);
               //m_armature.audioFaceRigFFT.UpdateEyelidsRight (weighting[EyeShape.Eye_Right_Blink]);
            }

            private void CreateEyeAnchors()
            {

               EyeAnchors = new GameObject[NUM_OF_EYES];
               for (int i = 0; i < NUM_OF_EYES; ++i)
               {
                  EyeAnchors[i] = new GameObject();
                  EyeAnchors[i].name = "EyeAnchor_" + i;
                  EyeAnchors [i].transform.SetParent (gameObject.transform);
                  EyeAnchors[i].transform.localPosition = transform.InverseTransformPoint(EyesModels[i].position);
                  //EyeAnchors[i].transform.localRotation = Quaternion.Inverse(transform.rotation) * EyesModels[i].rotation;
                  EyeAnchors[i].transform.forward = Vector3.forward;
                  //EyeAnchors[i].transform.rotation = EyesModels[i].rotation;
                  EyeAnchors[i].transform.localScale = EyesModels[i].localScale;
               }


            }

            private void DestroyEyeAnchors()
            {
               if (EyeAnchors != null)
               {
                  foreach (var obj in EyeAnchors)
                     if (obj != null) Destroy(obj);
               }
            }
         }
      }
   }
}

SRanipal_AvatarEyeSample2.cs

Edited November 4, 2019 by AirMouse

[VIVE_Jason_Lu]

Hi @AirMouse

 

Could you let me know which version of SDK are you using now?

You can check it from SRanipal.dll's detail page.

I have test SR_Runtime 1.1.2.0 with both SDK 1.1.0.1 and 1.1.2.0 and they all works as expected.

 

Sincerely

Jason Lu

 

 

[AirMouse]

I looked through my entire Unity project but could not locate any SRanipal.dll's "detail page". Could you please explain how to reach it?

And I am pretty sure that I am using an old SDK in my project, but I can not find a download for the latest 1.1.2.0 sdk anywhere.

When I look at the official vive-sranipal-sdk page then I only see SRanipal SDK v1.1.0.1 (556 MB) available.

Where do I download the latest 1.1.2.0 sdk you mention???

I even tried this link but the file does not exist: https://hub.vive.com/en-US/download/SRanipal_SDK_1.1.2.0.zip

I can't wait to try the latest SDK in my project! And I am very hopeful it will fix!!

Thank you very much for your help.

Edited November 5, 2019 by AirMouse

[VIVE_Jason_Lu]

HI @AirMouse

It's in the dll's property page.

Regards,

Jason

[VIVE_Jason_Lu]

Another short question, Is the sample scene in our SDK broke too?

 

Jason

[AirMouse]

Hi Jason

 

Thanks for that picture, I feel kind of silly that I did not think to check there 😶

Here's what I saw:

So I think my version is a very old one. I will update to the 1.1.0.1 SDK tonight. 😊

 

But I have one question, you said you tested SR_Runtime 1.1.2.0 with both SDK 1.1.0.1 and SDK 1.1.2.0. But I was not able to find any mention of the 1.1.2.0 SDK? Is it available for download anywhere?

Thanks again for your help. And I am very sorry to keep bothering you.

Edited November 5, 2019 by AirMouse