Gutsy Network Manager 0.6.5 and Ndiswrapper fails to connect to hidden AP

Binary package hint: network-manager

After upgrading to gutsy I could not connect to a hidden wireless through Network Manager using the Ndiswrapper driver.

I am running Kubuntu on IBM laptop Z60t with Atheros wireless card AR5212
My wireless router is a Netgear WGR614.

The problem appears to be a patch that was applied to the original Network Manager 0.6.5 source.

I found a previous bug that relates to this patch.
https://bugs.launchpad.net/baltix/+source/knetworkmanager/+bug/50214
(I added a comment but as it is in 'Fix Released' status I guess it will not be looked at any more.)

Note: Prior to upgrading to gutsy I had download the source for:
wireless-tools 29
Network Manager 0.6.5 (unpatched)
latest ndiswrapper, wpa_supplicat

This was to fix wireless connectivity issues after a kernel upgrade.

Looking in the logs after the gutsy upgrade I noticed in the daemon.log that that AP_SCAN 1 was being used where previously it was AP_SCAN 2

Oct 23 09:48:31 localhost NetworkManager: <info> Activation (wlan0) started...
Oct 23 09:48:31 localhost NetworkManager: <info> Activation (wlan0) Stage 1 of 5 (Device Prepare) scheduled...
Oct 23 09:48:31 localhost NetworkManager: <info> Activation (wlan0) Stage 1 of 5 (Device Prepare) started...
Oct 23 09:48:31 localhost NetworkManager: <info> Activation (wlan0) Stage 2 of 5 (Device Configure) scheduled...
Oct 23 09:48:31 localhost NetworkManager: <info> Activation (wlan0) Stage 1 of 5 (Device Prepare) complete.
Oct 23 09:48:31 localhost NetworkManager: <info> Activation (wlan0) Stage 2 of 5 (Device Configure) starting...
Oct 23 09:48:31 localhost NetworkManager: <info> Activation (wlan0/wireless): access point 'ferguson-home' is encrypted, but NO valid key exists. New key needed.
Oct 23 09:48:31 localhost NetworkManager: <info> Activation (wlan0) New wireless user key requested for network 'ferguson-home'.
Oct 23 09:48:31 localhost NetworkManager: <info> Activation (wlan0) Stage 2 of 5 (Device Configure) complete.
Oct 23 09:48:32 localhost NetworkManager: <info> Activation (wlan0) New wireless user key for network 'ferguson-home' received.
Oct 23 09:48:32 localhost NetworkManager: <info> Activation (wlan0) Stage 1 of 5 (Device Prepare) scheduled...
Oct 23 09:48:32 localhost NetworkManager: <info> Activation (wlan0) Stage 1 of 5 (Device Prepare) started...
Oct 23 09:48:32 localhost NetworkManager: <info> Activation (wlan0) Stage 2 of 5 (Device Configure) scheduled...
Oct 23 09:48:32 localhost NetworkManager: <info> Activation (wlan0) Stage 1 of 5 (Device Prepare) complete.
Oct 23 09:48:32 localhost NetworkManager: <info> Activation (wlan0) Stage 2 of 5 (Device Configure) starting...
Oct 23 09:48:32 localhost NetworkManager: <info> Activation (wlan0/wireless): access point 'ferguson-home' is encrypted, and a key exists. No new key needed.
Oct 23 09:48:33 localhost NetworkManager: <info> supplicant_interface_init() - connect to global ctrl socket (0/10).
Oct 23 09:48:33 localhost NetworkManager: <info> supplicant_interface_init() - connect to global ctrl socket (1/10).
Oct 23 09:48:33 localhost NetworkManager: <info> SUP: sending command 'INTERFACE_ADD wlan0^I^Iwext^I/usr/local/var/run/wpa_supplicant3^I'
Oct 23 09:48:33 localhost NetworkManager: <info> SUP: response was 'OK'
Oct 23 09:48:33 localhost NetworkManager: <info> supplicant_init() - connect to device ctrl socket (1/10).
Oct 23 09:48:33 localhost NetworkManager: <info> SUP: sending command 'AP_SCAN 1'
Oct 23 09:48:33 localhost NetworkManager: <info> SUP: response was 'OK'
Oct 23 09:48:33 localhost NetworkManager: <info> SUP: sending command 'ADD_NETWORK'
Oct 23 09:48:33 localhost NetworkManager: <info> SUP: response was '0'
Oct 23 09:48:33 localhost NetworkManager: <info> SUP: sending command 'SET_NETWORK 0 ssid 6665726775736f6e2d686f6d65'
Oct 23 09:48:33 localhost NetworkManager: <info> SUP: response was 'OK'
Oct 23 09:48:33 localhost NetworkManager: <info> SUP: sending command 'SET_NETWORK 0 scan_ssid 1'
Oct 23 09:48:33 localhost NetworkManager: <info> SUP: response was 'OK'
Oct 23 09:48:33 localhost NetworkManager: <info> SUP: sending command 'SET_NETWORK 0 proto WPA'
Oct 23 09:48:33 localhost NetworkManager: <info> SUP: response was 'OK'
Oct 23 09:48:33 localhost NetworkManager: <info> SUP: sending command 'SET_NETWORK 0 key_mgmt WPA-PSK'
Oct 23 09:48:33 localhost NetworkManager: <info> SUP: response was 'OK'
Oct 23 09:48:33 localhost NetworkManager: <info> SUP: sending command 'SET_NETWORK 0 psk <key>'
Oct 23 09:48:33 localhost NetworkManager: <info> SUP: response was 'OK'
Oct 23 09:48:33 localhost NetworkManager: <info> SUP: sending command 'SET_NETWORK 0 pairwise TKIP'
Oct 23 09:48:33 localhost NetworkManager: <info> SUP: response was 'OK'
Oct 23 09:48:33 localhost NetworkManager: <info> SUP: sending command 'SET_NETWORK 0 group TKIP'
Oct 23 09:48:33 localhost NetworkManager: <info> SUP: response was 'OK'
Oct 23 09:48:33 localhost NetworkManager: <info> SUP: sending command 'ENABLE_NETWORK 0'
Oct 23 09:48:33 localhost NetworkManager: <info> SUP: response was 'OK'
Oct 23 09:48:33 localhost NetworkManager: <info> Activation (wlan0) Stage 2 of 5 (Device Configure) complete.
Oct 23 09:49:33 localhost NetworkManager: <info> Activation (wlan0/wireless): association took too long (>60s), failing activation.
Oct 23 09:49:33 localhost NetworkManager: <info> Activation (wlan0) failure scheduled...
Oct 23 09:49:34 localhost NetworkManager: <info> Activation (wlan0) failed for access point (ferguson-home)
Oct 23 09:49:34 localhost NetworkManager: <info> Activation (wlan0) failed.
Oct 23 09:49:34 localhost NetworkManager: <info> Deactivating device wlan0.
Oct 23 09:49:34 localhost NetworkManager: <info> SUP: sending command 'DISABLE_NETWORK 0'
Oct 23 09:49:34 localhost NetworkManager: <info> SUP: response was 'OK'
Oct 23 09:49:34 localhost NetworkManager: <info> SUP: sending command 'AP_SCAN 0'
Oct 23 09:49:34 localhost NetworkManager: <info> SUP: response was 'OK'
Oct 23 09:49:34 localhost NetworkManager: <info> SUP: sending command 'TERMINATE'

To quickly check if this was the problem I ran wpa_supplicant from the command line using the ndiswrapper driver.

wpa_supplicant -c/etc/wpa_supplicant.conf -iwlan0 -d

I set ap_scan = 1 in the wpa_supplicant.conf file and wpa_supplicant could not connect to the wireless router. When I set the value back to 2 it could connect.

Note: If I used the madwifi driver then Network Manager could connect fine to my hidden wireless network.

Running the same wpa_supplicant test as above for the madwifi driver I could connect to the wireless router using AP_SCAN 1 or 2

I downloaded the source package unpacked and applied the patches.
 network-manager_0.6.5-0ubuntu16.diff.gz
 network-manager_0.6.5-0ubuntu16.dsc
 network-manager_0.6.5.orig.tar.gz

This file nm-device-802-11-wireless.c is where the AP_SCAN command is set.

I changed this section of the code (approx line 2923):
 if (!strcmp (kernel_driver, "orinoco_cs"))
                 ap_scan = "AP_SCAN 2";
         else if (is_adhoc || !supports_wpa)
   ap_scan = "AP_SCAN 2";
  else
                 ap_scan = "AP_SCAN 1";
to:
 if (!strcmp (kernel_driver, "orinoco_cs"))
                 ap_scan = "AP_SCAN 2";
         else if (is_adhoc || !supports_wpa)
   ap_scan = "AP_SCAN 2";
         else if (!nm_ap_get_broadcast (ap) && !strcmp (kernel_driver, "ndiswrapper"))
   ap_scan = "AP_SCAN 2";
  else
                 ap_scan = "AP_SCAN 1";

I can now use the Ndiswrapper with Network Manager to connect to a hidden wireless network.

In the original non patched version of nm-device-802-11-wireless.c all non broadcast access points had ap_scan = "AP_SCAN 2"

Further down in the same file was this section of code and the comment that explained why AP_SCAN 1 was being used for hidden networks.

 /* For non-broadcast networks, we need to set "scan_ssid 1" to scan with probe request frames.
  * However, don't try to probe Ad-Hoc networks.
  */
 if (!nm_ap_get_broadcast (ap) && !is_adhoc)
 {
  /*
   * since using "AP_SCAN 1" for hidden networks, wpa_supplicant
   * does not seem to bring the essid to the device anymore...
   * perhaps this is a wpa_supplicant/wext/driver issue or perhaps this
   * is simply how it is. We set the ESSID here for now.
   */
  nm_device_802_11_wireless_set_essid(self, essid);

  if (!nm_utils_supplicant_request_with_check (ctrl, "OK", __func__, NULL,
    "SET_NETWORK %i scan_ssid 1", nwid))
   goto out;
 }

It seems that setting AP_SCAN 1 for all drivers connecting to hidden networks isn't correct though.

This is from the wpa_config Struct Reference

AP scanning/selection.

By default, wpa_supplicant requests driver to perform AP scanning and then uses the scan results to select a suitable AP. Another alternative is to allow the driver to take care of AP scanning and selection and use wpa_supplicant just to process EAPOL frames based on IEEE 802.11 association information from the driver.

1: wpa_supplicant initiates scanning and AP selection (default).

0: Driver takes care of scanning, AP selection, and IEEE 802.11 association parameters (e.g., WPA IE generation); this mode can also be used with non-WPA drivers when using IEEE 802.1X mode; do not try to associate with APs (i.e., external program needs to control association). This mode must also be used when using wired Ethernet drivers.

2: like 0, but associate with APs using security policy and SSID (but not BSSID); this can be used, e.g., with ndiswrapper and NDIS drivers to enable operation with hidden SSIDs and optimized roaming; in this mode, the network blocks in the configuration are tried one by one until the driver reports successful association; each network block should have explicit security policy (i.e., only one option in the lists) for key_mgmt, pairwise, group, proto variables.

My fix was based on how the original nm-device-802-11-wireless.c set AP_SCAN for hidden networks and restricted to the ndiswrapper driver.