OneSwitch.org.uk asked me yesterday about the feasibility of a handsfree ps3 controller. This kind of controller is especially great for people with disabilities, and I’m sure everybody can take the fun out of it.
The sixaxis emulator receives events from “common” input devices (mouse, keyboard, game devices) and translates them into ps3 controls.
There’s nothing preventing us to use a head tracking system (more generally, a gesture recognition system) and a voice recognition system as “handfree” input devices.
And a combination of common and handfree input devices is even possible…
I googled about a Linux head tracking system. Many projects are using the OpenCV library. I finally found a great project called EHCI – Enhanced human computer interface through webcam image processing library.
A demo to get 3d position of the head is available there. Head position could be translated into thumbstick movements. For example, this could be used to aim in FPS games (right thumbstick).
I have plenty of functionalities to add to the sixaxis emulator… but as always, the fun ones come to the top of my list!
My next task will be to get a demo of head tracking used to aim in a FPS.
EDIT July 9:
In the youtube video demonstrating a handfree PS3 controller, there is no technical details about how it works, so it could be fake. The only alternative ways to control a PS3 are external emulators, like using a PC+bluetooth or a programmable usb controller. Running a piece of software on the PS3 itself (while gaming) hasn’t been performed yet.
EDIT August 5:
I tried the EHCI 6DOF head tracking, but I think it’s not and not stable and accurate enough for playing purposes. With my latest work about using the sixaxis emulator in windows, I’ll be able to test the freetrack 6DOF head tracking (unfortunately not available in Linux), which seems to be approved by many gamers (see videos on youtube). This system requires a webcam and a small additional hardware (quite simple = 3 IR diodes, some resistances and a battery). It can emulate a standard joystick with 6 axis. Events from this joystick can be easily grabbed by the sixaxis emulator, and mapped to some sixaxis controls (as I already did for a steering wheel). I probably will test that at the end of August.
Exciting stuff. Something else that would be fantastic would be if the PC could run gamer assist routines. Obvious example is rapid fire, but to be able to latch a control on/off (e.g. accellerate) or to toggle between two or more controls on one button can be fantstic. A bit like what 4Noah utilities can do already actually, but to have power over a PS3 / other console.
http://www.oneswitch.org.uk/2/I/4Noah/gaming-switch.htm
Some actual/future capabilities of the sixaxis emulator already do some user-assist:
– keyboard macros (actual)
– keymap switch (future)
Rapid fire and control latch are added on my list!
My webcam is not standard enough (although it is a logitech one). I ordered a USB Video Class compliant one. I will continue my investigation as soon as I receive it.
There are some software that translate head movements to mouse movements, such as eViacam. As the sixaxis emulator already uses mouse events, it could be quite easy to get something work quickly.