Idea

          The Basic idea of this project was to develop an interface with a remotely physical controller and the GazeboSim, to control the motion and interact with a simulated Robot inside the Gazebo world. Ideal choice for this purpose was the Thalmic Labs Myo Armband. Myo has internal IMU sernsors which can track Arm motion and rotation and 8-EMG built in sensors, for Hand Gesture recognition.

          Also, for the Gazebo simulation I used the Robonaut 2, aka R2. Robonaut is a Humanoid Robot created by NASA and the car manufacturer General Motors. Robonaut has a head, torso, arms and hands like a person. So Robonaut can perform tasks designed to be done by human hands. As a matter of fact, the simulated R2 has 18 Hand Joints and 7 Arm Joints, approaching the hypothetic degrees of freedom of Human's Hand and Arm Joints!

          So this was a challenge for me to create this control interface between the Myo Armband and the Simulated Robonaut 2, to give me the abillity to control his Arms and Hands Motions wirelessly while I'm wearing the Myo on my Arm. Robonaut will mimic the Arm and Hand motions that receives from Myo's orientation and motion Raw data. Of course to succeed that, I used ROS to create a new interface ROS Package.   

Simulation Screenshots

Here are some screenshots I took, during the simulation time.

Each screenshot has a description below it, explaining the simulation state at that moment...

Robonaut's Arm and Hand Joints and Links.

Controlling only the Hand of Robonaut. (+ Myo Hand Gesture Classifier)

Setting the new Arm Limitations...

Controlling both Arm and Hand motions of Robonaut..

doing poses..

..and poses..

..and poses..

Interacting with Gazebo World. [Grabbing things]

This part needs some improvement , because to interact with something inside the Gazebo World, something virtual, the robot's movements must be as smooth as accurate can be. In this case, we have an Armband Controller that publishes a fixed amount of data in a period of time and it's worn on a Human's Arm! So you may notice that it's not going to be as smooth experience as expected. :) It's just a stability issue that I hope I will fix in the near future... ( and I'll upload a video, about that :) )

Below you can see two RQT Graphs during the simulation process, while all the ROS Nodes are enabled and Publish or Subscibe to different ROS Topics

And a more detailed graph..

More details about the ROS package Structure, as I mentioned before, you will find in the detailed report.

All the above pictures (+more) are concluded in the screenshot.zip, which can be found in Download section.

Videos

Here is a video of me controlling R2's Hand Gestures, wearing the Myo and using two different Joint stiffness parameters.

And the video with R2's Arm Control and User Arm's Joint Limitation setup in the start of the video.

Downloads