Bluetooth latency issues

As the PS4 only supports bluetooth controls, I decided to work on the latency issues that occur when controlling a PS3 over bluetooth with GIMX:

  • Latency when using a slim/superslim PS3 and GIMX in Ubuntu 12.04+.
  • Latency when using a bluetooth headset or several controllers at the same time.

As I only had a fat PS3, I had to get a slim PS3. I bought one with a defective blu-ray lens for 60€ (about 80$).

I discovered several problems:

  • A change in the Linux 3.1 kernel: the bluetooth transmitter was forced to go into the active mode instead of remaining in the sniff mode.
  • Bluetooth packets not properly synced: the IN report has to be sent immediately after receiving the OUT report.

These problems are both related to the bluetooth sniff mode, which is used to reduce the power consumption.
To simplify, each device in the piconet is active during a fixed window:

X+0 - the master sends a packet to slave 1
X+1 - slave 1 sends a packet to the master and reduces its power till its Tsniff period is over (11250µs)
X+2 - the master sends a packet to slave 2
X+3 - slave 2 sends a packet to the master and reduces its power till its Tsniff period is over (11250µs)
...
X+18 - the master sends a packet to slave 1
...

The master sends OUT reports, and the slaves IN reports.
If an IN report is not sent at the right time, it gets retransmitted later.
If the IN report period is slightly lower than the OUT report period, the IN reports will be gradually buffered, thus generating a latency that slowly increases.

The current GIMX design can’t guarantee that the IN reports are sent in the right slot, because the OS can interrupt the program and schedule another task.

The following improvements could be made:

  • Merge emu and emuclient: this would prevent the kernel from interrupting emu to run emuclient (this can happen just after receiving an OUT report and before sending the IN report…). This would improve GIMX performances especially on single core CPUs (like the Raspberry Pi). This would still not ensure a 100% perfect sync, but I believe this is the only possible improvement without adding extra hardware.
  • Handle the Bluetooth protocol with a DIY adapter: PC —USB— AVR USB —UART— BT module —BT— PS3.

Ubuntu 11.04

Ubuntu 11.04 is out! Some important changes for the sixaxis emulator:

  • the bluez version is now 4.91

The Ubuntu 10.10 bluez version (4.69) seems to cause some connection problems in some undefined cases.

  • the middle mouse button emulation is now off by default

This “feature” can cause a 50ms lag for left and right mouse buttons.

Download link.

A tool to estimate the latency

In a series of posts I am talking about the sixaxis emulator latency.

What kind of latency am I talking about? Basically, the time between event reception from PC peripherals, and event emission to the PS3. The most critical event to me is a mouse left clic in a FPS game.

The most accessible estimation is to measure the difference between usb timestamps. See this post for more details. I created a small utility to make this estimation easier. It has to be launched in a terminal, but I am thinking to add it some day in sixemugui.

matlo@matlo-hcpc:~/hack/test$ wget http://diyps3controller.googlecode.com/svn/trunk/tools/sixemulat.c
matlo@matlo-hcpc:~/hack/test$ gcc -lm -o sixemulat sixemulat.c

matlo@matlo-hcpc:~/hack/test$ lsusb
...
Bus 003 Device 008: ID 0a12:0001 Cambridge Silicon Radio, Ltd Bluetooth Dongle (HCI mode)
Bus 003 Device 005: ID 046d:c041 Logitech, Inc. G5 Laser Mouse

matlo@matlo-hcpc:~/hack/test$ ./sixemulat
usage: ./sixemulat <mouse bus id> <mouse device id> <dongle bus id> <dongle device id>
matlo@matlo-hcpc:~/hack/test$ ./sixemulat 3 5 3 8

Be sure to use the correct mouse/dongle bus/device ids, specifically if you have several mice and/or several dongles!

Sixemulat displays:
– the number of caught left clics
– the worst latency
– the average latency

About a hundred of samples should be enough to get significant values.

The average latency is an interesting value, but the most important value is the worst latency. Commercial products will tend to give you only the average latency, as it is a lower (generally half) value…

I only tested this tool with two logitech mice. If it doesn’t work for you, please get in touch with me.

Left and right mouse button lag

In my previous post I told that I was experiencing about 50ms latency for left and right mouse clics. After some investigations I discovered that the problem was located in xorg.

It appears it’s not a bug, it’s a functionality called “3rd button emulation” that is useful with a 2 button mouse. Xorg waits up to 50ms to see if both left and right buttons are pressed, and in case it is, it generates a middle button press.

It’s nothing to say that this implementation is lame and should be disabled by default.

Following bug reports were filled by people complaining about that behavior:

Although a real middle button press is enough to disable the functionality (not persistent), it’s possible to disable it by default (works for Ubuntu 10.10):

sudo gedit /usr/share/X11/xorg.conf.d/20-mouse.conf

Copy-paste to following lines:

Section “InputClass”
Identifier “Mouse0”
MatchIsPointer “true”
Option “Emulate3Buttons” “false”
EndSection

Save, and Restart!

Another post about latency

My previous work on that subject

I already worked on measuring how much latency the sixaxis emulator is adding compared to a real sixaxis. I did that using a microphone, measuring the overall response time for a gunshot.

Assuming we have:
R=C+G
R=overall response time
C=controller latency
G=game latency

For the real sixaxis:
R1=C1+G
For the sixaxis emulator:
R2=C2+G

Assuming G is constant:
R2-R1=C2-C1

To estimate the controller latency difference, which is C2-C1, I just had to subtract the overall response time.
The result was about 15ms, which is not that bad compared to the overall response time (>100ms) and the human response time (>100ms).

A way to estimate the latency of the sixaxis emulator

The basic idea is to measure the time between a usb packet coming from the mouse (or any other usb controller) and a packet sent to the usb bt dongle.

For ex, the time between the usb packet for a left clic and the usb packet that sends the R1 button press (via the bluetooth dongle).

A great tool provided with the Linux kernel is usbmon, which allows to sniff the usb traffic. It’s possible to use it with wireshark (with root user rights), but it’s better to play in a terminal to automate extraction of valuable information.

The usbmon documentation is available there: link.

First of all, what the lsusb output says:

matlo@matlo-hcpc:~/hack/test$ lsusb
Bus 004 Device 002: ID 050d:0200 Belkin Components
Bus 004 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 003 Device 009: ID 054c:0268 Sony Corp. Batoh Device
Bus 003 Device 008: ID 0a12:0001 Cambridge Silicon Radio, Ltd Bluetooth Dongle (HCI mode)
Bus 003 Device 006: ID 054c:0268 Sony Corp. Batoh Device
Bus 003 Device 005: ID 046d:c041 Logitech, Inc. G5 Laser Mouse
Bus 003 Device 003: ID 046d:c517 Logitech, Inc. LX710 Cordless Desktop Laser
Bus 003 Device 002: ID 058f:9254 Alcor Micro Corp. Hub
Bus 003 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 002 Device 003: ID 046d:0809 Logitech, Inc. Webcam Pro 9000
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

What is useful there is the bus and device ids. The G5 mouse is device 5 on usb bus 3, and the bt dongle is device 8 on usb bus 3.

To start the capture:

matlo@matlo-hcpc:~/hack/test$ sudo cat /sys/kernel/debug/usb/usbmon/0u > test

Generate left clics into the sixaxis emulator, and stop the capture (ctrl+c).

Then, to display the mouse packets:

matlo@matlo-hcpc:~/hack/test$ cat test.txt | grep “3:005”

And for the bt dongle packets:

matlo@matlo-hcpc:~/hack/test$ cat test.txt | grep “3:008”

Both these commands may give a huge amount of data.

To only display packets that correspond to left clicks:

matlo@matlo-hcpc:~/hack/test$ cat test.txt | grep “3:005:1 0:1 10 = 01000000 00000000 0000”
f6747a40 967102843 C Ii:3:005:1 0:1 10 = 01000000 00000000 0000
f6747a40 968852805 C Ii:3:005:1 0:1 10 = 01000000 00000000 0000
f6747a40 970341772 C Ii:3:005:1 0:1 10 = 01000000 00000000 0000
f6747a40 971609742 C Ii:3:005:1 0:1 10 = 01000000 00000000 0000
f6747a40 972760716 C Ii:3:005:1 0:1 10 = 01000000 00000000 0000

To only display packets that correspond to R1 button presses:

matlo@matlo-hcpc:~/hack/test$ cat test.txt | grep “3:008:2 -115 58 = 2b203600 32004300 a1010000 08000080 80808000 00000000 00000000 0000ff00”
f32e2640 967105205 S Bo:3:008:2 -115 58 = 2b203600 32004300 a1010000 08000080 80808000 00000000 00000000 0000ff00
f32e2640 968861926 S Bo:3:008:2 -115 58 = 2b203600 32004300 a1010000 08000080 80808000 00000000 00000000 0000ff00
f32e2640 970345964 S Bo:3:008:2 -115 58 = 2b203600 32004300 a1010000 08000080 80808000 00000000 00000000 0000ff00
f3a904c0 971618231 S Bo:3:008:2 -115 58 = 2b203600 32004300 a1010000 08000080 80808000 00000000 00000000 0000ff00
f3a90ac0 972769407 S Bo:3:008:2 -115 58 = 2b203600 32004300 a1010000 08000080 80808000 00000000 00000000 0000ff00

What is to be extracted from this is the timestamp in microseconds, which is the second column.

matlo@matlo-hcpc:~/hack/test$ cat test.txt | grep “3:005:1 0:1 10 = 01000000 00000000 0000” | awk ‘{ print $2 }’
967102843
968852805
970341772
971609742
972760716

matlo@matlo-hcpc:~/hack/test$ cat test.txt | grep “3:008:2 -115 58 = 2b203600 32004300 a1010000 08000080 80808000 00000000 00000000 0000ff00” | awk ‘{ print $2}’
967105205
968861926
970345964
971618231
972769407

Then, it’s easy to copy-paste this data into a spreadsheet, compute the difference, compute the average, and the standard deviation.

For this example (with 5 more samples), it gives a 7.6ms average difference, with a 2.9ms standard deviation. The max difference is about 12ms.

It has to be reminded that the event polling period is 10ms (equal to the sixaxis report period), so that the maximum latency added by the event polling is 10ms, and the average is 5ms. Thus, the processing time is in average 7.6-5=2.6ms. It has to be noticed that I’m using a Linux kernel with realtime/low latency capabilities.

It’s also possible that there is some more latency added by the usb stack and the bt dongle, but it’s probably less than the 7.6ms processing time.

Conclusion:

– it’s probably not possible to have a lower response time with a sixaxis emulation (bt or usb)
– to lower the controller latency, the only potential way is to use a usb device to usb device controller (a custom usb link between the PC and the PS3), with a report period lower than 10ms

Thanks to that work I found a latency issue (about 50ms) that only impacts left and right mouse clics. The work-around -I’m not joking!- is to press the middle button after the mouse is plugged to the PC (see next post).

Back to the latency: my audio system sucks

In my previous post, I warned that I was experiencing some measure inconsistency.

The reason is simple: my audio system sucks 🙁

For some measures, I recorded the LCD audio output, and for some others, I recorded the audio output of my digital audio system.

But my digital audio system is adding an audio latency that varies from about 40ms for PCM 2.0 to about 90ms for Dolby Digital! (the PS3 could introduce some of this latency, see EDIT2)

Speaking of stuffs that may add latency, a good advice I could give is not to use weak usb extension cable.

I changed the following stuffs so as to take better measures:

  • only record the LCD audio output
  • use a better microphone that makes a lot less noise
  • increase the LCD audio output level
  • take all measures within 10cm near the speaker (1m = 3.4ms)
  • only start the sixaxis emulator, and record the sound with another PC

New measures of the response time:

  • Wireless sixaxis: average 112ms, standard deviation 11ms
  • Wired sixaxis: average 108ms, standard deviation 10ms
  • Sixaxis emulator: average 123ms, standard deviation 8ms

Note that this response time probably includes the response time of the LCD audio output, which seems to be quite constant, since the standard deviation is about 10ms in each case.

The sixaxis emulator is adding 11ms latency compared to the wireless sixaxis, and 15ms compared to the wired sixaxis.
This actually shows the sixaxis emulator is doing quite a good job 🙂

EDIT: I took new measures for CSS in Win7 too, it gives a 158ms response time, with a 16ms standard deviation.

EDIT2: I should measure the response time for CSS with my digital audio system, so as to see if the PS3 is a source of audio latency for dolby digital…

EDIT3: all samples in a single archive file

Sixaxis emulator: the truth about latency

Warning: I need to work again on these measures. I took some more measures for the wireless sixaxis. These measures are not consistent with my earlier values :s For these new measures, the average game response time with the wireless sixaxis is about 150ms. All values in this post have to be confirmed. EDIT: See my next post.

No suspense: it’s not good.

I measured the average delay between a gunshot action and its corresponding gunshot sound (in BF:BC2).

  • Real wireless sixaxis: 125ms
  • Real wired sixaxis: 139ms
  • Sixaxis emulator: 168ms

Note that this isn’t the controller latency. This is the overall latency for a gunshot.
The difference between two values gives the controller latency difference.

The sixaxis emulator adds a 43ms latency. That’s more than 1 or 2 frames (40ms@25fps, 33ms@30fps, 17ms@60fps). Not good at all.
I should take some measures with a FPS on PC.
125ms seems to be very high for a descent game controller.
The wired sixaxis gives more latency than the wireless one… not as expected.

Methodology:

– plug a microphone on the PC
– start audacity
– start a FPS on the PS3
– put the microphone on the device to test (sixaxis or mouse)
– adjust the volume of the game so that gunshot actions/sounds are recorded
– start recording in audacity
– repeat the gunshot action (empty the cartridge of a pistol) with ~2s delays
– stop recording
– get measures between gunshot actions/sounds and compute the average

I don’t know how precise it is to measure the game latency, but it’s at least good enough to compute a latency difference.

Following images are screen-shot examples of the audio wave:


As you can see in the second screen-shot, there are two events: the gunshot action (mouse clic) and the gunshot sound. It shows a capture recorded with the sixaxis emulator. The delay was about 150ms for this gunshot.

EDIT:

I took measures for Counter Strike Source in Win7. I get a 188ms average response time for a gunshot.

Although all the measured values seem very high, it seems the real sixaxis and the sixaxis emulator are not that bad… compared to CSS results.