rt2x00 (rt61 and rt2560) 4-6X slower than rt2500 legacy

Live forum: http://rt2x00.serialmonkey.com/viewtopic.php?t=5169

itto

21-01-2009 17:53:48

Greetings rt2x00 Gurus -

I have a linux box running 2.6.27 recompiled from the git sources around New Years 2009. It also has one rt2500 and one rt61 card installed. I am running a small application that uses the wireless interface to boot several embedded devices with soft real-time requirements for synchronizing their boots. I am running the driver in monitor mode and using injection to send packets to my devices.

I have been running with the rt2500 legacy driver and encountered an issue with the driver not getting receive interrupts for several seconds, which caused some problems for my devices. (I posted a log about this issue on the rt2500 legacy board in December but am still in the queue.) I decided to try the latest (as of Christmas Eve) rt2x00 driver. I haven't been successful with getting the rt2500pci driver working, but have worked a little with the rt61/rt2x00 card and driver. I have been noticing some significant packet loss when exchanging fast, small packets (~35 - 90 bytes per packet with 10-15 byte ACKs) using short preambles with my devices. I did some very crude profiling of my application running on both the rt2500 legacy driver and the rt61 (it includes the send/recv calls) and am seeing that packet handling (round-trip performance to receive a packet, inject a reply, and go back to selecting on the socket) is taking 4X - 5X longer in the newer driver (and mac stack).

I did some searching on this forum and only found one other post that talked about performance issues in the new driver for rt61, though they were running WPA and trying to attach to another AP
viewtopic.php?f=5&t=5077&start=0&hilit=performance+profiling+profile

It mentioned that slow performance was a "known issue."

I'm afraid I don't have a log for the driver, as my profiling was done at application level. I was wondering if I can get more information about what is known about the slow performance - is there another way to configure the driver or the stack to minimize this? I know the new driver uses mac80211 instead of having 802.11 handling as part of the driver - should I be poking the mac80211 people instead?

Thank you for your assistance!

-itto

IvD

21-01-2009 18:46:45

Slow bitrates is a known issue, which has proven to be difficult to resolve. However, you mentioned using short preamble, and that is something I had suspected as possible problem some time ago. I did made a fix for it last week, could you give latest rt2x00.git a testrun to see if it improves things?

Also are you using the rate control module (default) or have you forced the bitrate to 54MB?

itto

21-01-2009 19:51:29

Greetings IvD -

I will download the latest git sources and see how they do for me.

I'm actually setting the rate down to 2M fixed in a script that calls iwconfig before my app starts up. My embedded devices are incapable of running with 802.11g rates. I was pretty surprised by the packet loss and long handling times given the slower rates that I'm using.

Thanks for getting back to me so quickly!

-itto

IvD

21-01-2009 21:02:40

Ok, setting to 2 Mb is fine as well. It means you are at least comparing the 2 drivers with the bitrate forced to a specific rate.
Let me know how the latest rt2x00.git is behaving, I'm quite curious if it has any improvements.

itto

23-01-2009 21:03:44

Greetings -

I grabbed the latest git sources, and compiled it. Since there was also a kernel change from 2.6.27 to 2.6.28, I did a whole kernel/header/module compile and install. Looks like the driver version changed to 2.3.0 as well.

I'm getting an error when I try to set the driver into monitor mode

> iwconfig wlan1 mode monitor

Error for wireless request "Set Mode" (8B06)
SET failed on device wlan1 ; Operation not supported.

Is monitor mode really not supported anymore? What did I do wrong?

Thanks!

-itto

IvD

23-01-2009 23:02:00

You can only change working mode while interface is down.

itto

24-01-2009 00:23:24

Greetings -

Yes, the interface is down. I'm still getting errors putting it into monitor mode.

-itto

IvD

24-01-2009 01:20:32

what is the output of ifconfig -a and iwconfig?

itto

24-01-2009 01:48:20

Greetings -

ifconfig (no -a) shows no wlan1 interface up, just my eth0.

ifconfig -a shows

wlan1 Link encapEthernet HWaddr 000E2EDCB11F
BROADCAST PROMISC MULTICAST MTU1500 Metric1
RX packets0 errors0 dropped0 overruns0 frame0
TX packets0 errors0 dropped0 overruns0 carrier0
collisions0 txqueuelen1000
RX bytes0 (0.0 b) TX bytes0 (0.0 b)

wmaster0 Link encapUNSPEC HWaddr 00-0E-2E-DC-B1-1F-65-74-00-00-00-00-00-00-00-00
BROADCAST MULTICAST MTU1500 Metric1
RX packets0 errors0 dropped0 overruns0 frame0
TX packets0 errors0 dropped0 overruns0 carrier0
collisions0 txqueuelen1000
RX bytes0 (0.0 b) TX bytes0 (0.0 b)


and iwconfig shows

wmaster0 no wireless extensions.

wlan1 IEEE 802.11bg ESSID""
ModeManaged Frequency2.442 GHz Access Point Not-Associated
Tx-Power=12 dBm
Retry limit1 RTS throff Fragment thr=2352 B
Encryption keyoff
Power Managementoff
Link Quality0 Signal level0 Noise level0
Rx invalid nwid0 Rx invalid crypt0 Rx invalid frag0
Tx excessive retries0 Invalid misc0 Missed beacon0


I also noticed that I can't seem to set any mode - Ad-Hoc also fails. Only Managed mode (which it's already set to) succeeds.

Thanks for the diagnostic help. It's sure painful running on the bleeding edge....

-itto

IvD

24-01-2009 11:07:52

Thats odd, is there anything strange 'dmesg'?

itto

25-01-2009 01:27:57

Greetings IvD -

The only rt2x00 things that show up in dmesg are

device wlan1 entered promiscuous mode
phy0 -> rt2xoolib_request_firmware Info - Loading firmware file 'rt2561s.bin'.
rt61pci 0000020a.0 firmware requesting rt2561s.bin
phy0 -> rt2x00lib_request_firmware Info - Firmware detected - version 0.8.
phy0 -> rt2x00mac_conf_tx Info - Coforured Tx queue 0 - CWmin 5, CWmax 10, Aifs 2, TXop 0.
phy0 -> rt2x00mac_conf_tx Info - Coforured Tx queue 1 - CWmin 5, CWmax 10, Aifs 2, TXop 0.
phy0 -> rt2x00mac_conf_tx Info - Coforured Tx queue 2 - CWmin 5, CWmax 10, Aifs 2, TXop 0.
phy0 -> rt2x00mac_conf_tx Info - Coforured Tx queue 3 - CWmin 5, CWmax 10, Aifs 2, TXop 0.
ADDRCONF(DETDEV_UP) wlan1 link is not ready


Thank you -
itto

itto

26-01-2009 17:05:39

Greetings rt2x00 Gurus -

One more tidbit of information that might help you

I started up the rt2500pci built with the latest kernel/source base (un-blacklist and reboot) to see if it exhibits the same problem going into monitor mode as the rt61 does. On boot, the driver crashes majestically. Neither the rt2500pci nor the rt61 devices are brought up. Dmesg reports

phy0 -> rt2500pci_validate_eeprom EEPROM recovery - NIC 0xfff0
phy0 -> rt2x00_set_chip Info - Chipset detected - rt 0201, rf 0003, rev 0000 0004.
BUG unable to handle kernel paging request at 7465735f
IP [<c0727cc3>] __regulatory_hint+0xd3/0x210
*pde = 00000000
Oops 0000 [#1] SMP
last sysfs file /sys/devices/pci000000/0000001d.7/usb1/idVendor
Modules linked in rt2500pci(+) rt2x00pci rt2x00lib led_class input_polldev mac80211 ata_generic iTCO_wdt iTCO_vendor_support eeprom_93cx6 skge pata_via shpchp intel_agp agpgart evdev
Pid 2282, comm modprobe Not tainted (2.6.28-rc9-wl-smp #5) To Be Filled By O.E.M.
EIP 0060[<c0727cc3>] EFLAGS 00010286 CPU 0
EIP is at __regulatory_hint+0xd3/0x210
EAX de978f60 EBX de978f60 ECX 00000000 EDX fffffff4
ESI 7465735f EDI 00000003 EBP de99ce40 ESP de90fc54
DS 007b ES 007b FS 00d8 GS 0033 SS 0068
Process modprobe (pid 2282, ti=de90e000 task=df87b730 task.ti=de90e000)
Stack
00000000 0000ffff 000001ff de99c040 009a116f de90fc8e c041889a 00000000
00000100 00000101 7465735f de99c040 00000000 de99ce40 c07281c2 00000000
00000000 00000003 00000004 e0e5801c e0e59b3c de99c0f8 e0e58ff6 e0e59a1c
Call Trace
[<c041889a>] delay_tsc_0x2a/0x50
[<c07281c2>] regulatory_hint+0x32/0x50
[<e0e5801c>] rt2500pci_probe_hw+0x1bc/0x560 [rt2500pci]
[<e0e585c0>] rt2500pci_eepromregister_read+0x0/0x30 {rt2500pci]
[<e0e58730>] rt2500pci_eepromregister_write+0x0/0x50 [rt2500pci]
[<c011a8c1>] __ioremap_caller+0x1b1/0x290
[<e0dbfa9e>] rt2x00lib_probe_dev+0x5e/03b0 [rt2x00lib]
[<e0e440ab>] rt2x00pci_alloc_reg+0x5b/0xc0 [rt2x00pci]
[<e0e44524>] rt2x00pci_probe+0x124/0x1e0 [rt2x00pci]
[<c04289a6>] pci_decive_probe+0x56/0x80
[<c04662f8>] driver_probe_device+0x88/0x180
[<c0428580>] pci_match_device+0x10/0xb0
[<c046646e>] __driver_attach+0x7e/0x80
[<c0465a9a>] bus_for_each_dev+0x3a/0x60
[<c0466176>] driver_attach+0x16/0x20
[<c04663f0>] __driver_attach+0x0/0x80
[<c0465edc>] bus_add_driver+0xac/0x220
[<c04284b0>] pci_device_shutdown+0x0/0x20
[<c04288f0>] pci_device_remove+0x0/0x40
[<c046667d>] driver_register+0x4d/0x120
[<e0e5e000>] rt2500pci_init+0x0/0x14 [rt2500pci]
[<c0428be7>] __pci_register_driver+0x47/0x90
[<e0e5e000>] rt2500pci_init+0x0/0x14 [rt2500pci]
[<c0101036>] _stext+0x36/0x1b0
[<c01bfaff>] sysfs_addrm_start+0x3f/0xb0
[<c016f0c9>] free_unmap_vmap_area_noflush+0x29/0x70
[<c016f248>] __vunmap+0x58/0xc0
[<c0150ae1>] load_module+0x15e1/0x1820
[<e0e448d0>] rt2x00pci_write_tx_data+0x0/0xa7 [rt2x00pci]
[<c0150da7>] sys_init_module+0x87/0x1a0
[<c017c511>] sys_read+0x41/0x70
[<c0103232>] syscall_call+0x7/0xb
[<c0750000>] io_schedule_0x10/0x30
Code ff 8d b4 26 00 00 00 00 c6 44 24 13 00 a1 c4 9e 92 c0 ba d0 80 00 00 e8 dc 05 a5 ff ba f4 ff ff ff 85 c0 89 c3 0f 84 6b ff ff ff <0f> b6 06 8d 6e 01 88 43 08 0f b6 46 01 89 7b 04 88 43 09 8b 44
EIP [<c0727cc3>] __regulatory_hint+0xd3/0x210 SSESP 0068de90fc54
---[ end trace 981ed02c0dc74a53 ]---

I'm going to boot back to the older driver and kernel set to keep working. I think this may be repeatable if you need me to boot back to this version and get more diagnostic information.

Thanks for your assistance -

-itto

IvD

26-01-2009 17:34:17

I guess you are using a GIT snapshort from 2 weeks ago. I suggest you update to the latest version.

itto

26-01-2009 17:48:49

This is compiled from the sources I picked up last tuesday (1/20).

-itto

IvD

26-01-2009 18:32:43

Thats odd because the function which is crashing doesn't exist anymore in rt2x00.git. It was only present for a few days 2 weeks ago. Could you try updating the tree to todays version?

itto

27-01-2009 00:54:13

Greetings IvD -

I got the latest sources this morning. I compiled and installed the kernel, modules, and headers. My uname -a shows 2.6.28-rc9-wl-smp #1 SMP Mon Jan 26 12726 2009.

I'm still seeing the rt2500pci crash on boot with the stack trace looking nearly the same as I posted, and rt61 doesn't go into any mode except managed mode.

When I got the sources, I went into the rt2x00 git directory that I set up the first time, and typed 'git fetch' at the command prompt. I see it updating many packages, and the rt2x00 directory is changed and all files have today's timestamp. Then I run the config/compile/install from there as well. Is this the best way to update? How do I know if I have the latest ones (aside from trusting git) or how can I detect if there are any problems with the packages that I'm getting or the repository that I'm getting from?

Thanks -

-itto

itto

29-01-2009 02:09:20

Greetings rt2x00 Gurus -

I cloned a fresh copy of 1/27 sources and built and installed fresh. When I ran the performance profiling with my application, the driver operates about the same as it did before with about 4x slowdown in round-trip message processing. So while the change to the short preamble code doesn't appear to break anything, it also didn't seem to improve anything either (for me).

Do you have any other suggestions for how I can configure to get faster round-trip performance?

Thanks so much -

-itto