Robotic platform

• Fujitsu’s HOAP-3 humanoid robot
• HOAP = Humanoid for Open Architecture Platform
• 28 degrees of freedom, where each motor can be controlled at a rate of 1kHz

Robot simulator

• Allows offline development of control and behavioral components
• Includes the robot's specifications such as mass properties, body segments' length, ...
• Direct interface with the robot real-time controller

Motor controllers

• Postural control
• Hierarchical inverse kinematics (IK) robust to kinematic singularities and joint limits
• Adjustable kinematic chains (e.g. for tool use) while standing or seated
• Control of the position/orientation of both hands while handling postural stability
• Attentional mechanism (e.g. the robot looks at the object that is currently used)
• Online adaptation of trajectories in joint space

Vision system

• Stereoscopic vision with 2 webcams of 640 × 480 pixels
• Identification of objects through a set of specific B&W patterns
• Reconstruction of position and orientation (handling of occlusion)
• Intrinsic and extrinsic parameters estimation (use of the AR Toolkit)

Recording movements through motion sensors

• Use of a homemade suit composed of 6 X-Sens motion sensors
• 13 joint angles reconstructed from the absolute orientation matrices

Communication with the robot

• Speech synthesis
• A Wiimote is used as a wireless Bluetooth remote control to guide the interaction and to answer to the robot’s questions

Statistical learning system

• See Sylvain Calinon's research page for more information

The Dancing Robot

This application is basically a showcase of the capabilities of the motor controller, i.e. the hierarchical inverse kinematics system, to handle both the following of several simultaneous trajectories in cartesian space (position and orientation of the hands, legs, head and rump) as well as postural stability.

To make reseach funnier, the robot was brought to various places of Switzerland. (video)

Moreover, during an conference in California, as the result of a spontaneous and improvised collaboration with the Keepon team, the two robots developed a certain friendship, crashed a party together, got into a conference room and invaded mars... (video)

The Chief Cook Robot

This application demonstrates how the Chief Cook Robot is capable of learning to cook an omelet by whipping eggs, cutting ham and grating cheese.

The robot learns these skills by means of a probabilistic model using Gaussian Mixture Model (GMM) and Gaussian Mixture Regression (GMR), which allows it to learn to generalize the skill across various situations (See Sylvain Calinon's research page ). Moreover, being dynamical and driven online by nature, the robot can reproduce its skills by being robust to external perturbations such as a moving bowl.

Soon in all homes! (video)

Pong, the Robot Edition

This entertainment-centered application shows another type of Human-Robot Interaction. The robot gives a new gaming dimension to the old-school arcade game Pong. (video)

Rather than playing alone, the human player faces now a robot, which reacts according to the state of the game. Through appropriate speech dialogs and gestures, it engages the user in a fierce but competition. All disturbing technics, such as in-game speech are included.

Technically, the robot is connected to the game and hence knows the current game's state. However, to provide a real interaction experience, the robot effectively act on the joystick and buttons, and perform eyes-tracking of the ball on the screen by turning the head. Thus, a predictive model has been implemented to handle the latency due the robot's movement dynamics. This model is trained during a preliminary phase, where the robot plays the game alone to optimize its movements through self-calibration.

The Portraitist Robot

Salvador DaBot, so-called by its fathers due to fancy moustache and beret add-ons, is an artist who draws protraits of the persons it meets. Such an expensive printer has the advantage of using a real pencil, providing to its customers the impression of a real drawing experience, which it is! (video)

Technically, the robot captures a photo using its integrated webcams, extracts the contours of captured face, and then computes strokes useful to render nice shades of grey. Moreover, rather than drawing the lines like a normal printer usually does, Salvador draws surfaces, one after the others, which looks more natural.

Finally, because of its well-known artistic talents, Salvador DaBot was invited to the Google Zeitgeist'08 event, where it spent hours and used many pencils to realize several masterpieces of the guests invited to this event.