Urfkill saved wrong WWAN state after enabling/disabling flight mode

phablet@ubuntu-phablet:/usr/share/ofono/scripts$ ./list-modems
[ /ril_1 ]
    Manufacturer = Fake Manufacturer
    Online = 0
    Revision = MOLY.WR8.W1315.MD.WG.MP.V37.P5, 2014/05/15 11:49
    Lockdown = 0
    Features =
    Interfaces = org.ofono.CallVolume org.ofono.VoiceCallManager org.ofono.NetworkTime
    Type = hardware
    Model = Fake Modem Model
    Emergency = 0
    Serial = 354142060305523
    Powered = 1
    [ org.ofono.CallVolume ]
        Muted = 0
        MicrophoneVolume = 0
        SpeakerVolume = 0
    [ org.ofono.VoiceCallManager ]
        EmergencyNumbers = 08 000 999 110 112 911 118 119
    [ org.ofono.NetworkTime ]

[ /ril_0 ]
    Manufacturer = Fake Manufacturer
    Online = 0
    Revision = MOLY.WR8.W1315.MD.WG.MP.V37.P5, 2014/05/15 11:49
    Lockdown = 0
    Features =
    Interfaces = org.ofono.CallVolume org.ofono.VoiceCallManager org.ofono.NetworkTime
    Type = hardware
    Model = Fake Modem Model
    Emergency = 0
    Serial = 354142060025527
    Powered = 1
    [ org.ofono.CallVolume ]
        Muted = 0
        MicrophoneVolume = 0
        SpeakerVolume = 0
    [ org.ofono.VoiceCallManager ]
        EmergencyNumbers = 08 000 999 110 112 911 118 119
    [ org.ofono.NetworkTime ]

phablet@ubuntu-phablet:/usr/share/urfkill/scripts$ ./flight-mode
flight-mode is: 0

phablet@ubuntu-phablet:/usr/share/urfkill/scripts$ ./enumerate
/org/freedesktop/URfkill/devices/0
Printing props for unspecialized device: /org/freedesktop/URfkill/devices/0
Printing props for Kernel device: /org/freedesktop/URfkill/devices/0
index: 0
platform: 0
urftype: org.freedesktop.URfkill.Device.Kernel
type: 2
name: hci0
hard: 0
soft: 0

/org/freedesktop/URfkill/devices/200
Printing props for unspecialized device: /org/freedesktop/URfkill/devices/200
Printing props for Hybris device: /org/freedesktop/URfkill/devices/200
index: 200
platform: 0
urftype: org.freedesktop.URfkill.Device.Hybris
type: 1
name: hybris_wifi
soft: 0

/org/freedesktop/URfkill/devices/100
Printing props for unspecialized device: /org/freedesktop/URfkill/devices/100
Printing props for Ofono device: /org/freedesktop/URfkill/devices/100
index: 100
platform: 0
urftype: org.freedesktop.URfkill.Device.Ofono
type: 5
name: Fake Manufacturer Fake Modem Model
soft: 1

/org/freedesktop/URfkill/devices/101
Printing props for unspecialized device: /org/freedesktop/URfkill/devices/101
Printing props for Ofono device: /org/freedesktop/URfkill/devices/101
index: 101
platform: 0
urftype: org.freedesktop.URfkill.Device.Ofono
type: 5
name: Fake Manufacturer Fake Modem Model
soft: 1

phablet@ubuntu-phablet:/var/lib$ cat urfkill/saved-states
[WLAN]
soft=false
prev-soft=false

[BLUETOOTH]
soft=false
prev-soft=false

[UWB]
prev-soft=false
soft=false

[WIMAX]
prev-soft=false
soft=false

[WWAN]
prev-soft=false
soft=true

[GPS]
prev-soft=false
soft=false

[FM]
prev-soft=false
soft=false

[NFC]
prev-soft=false
soft=false

[ALL]
soft=false

This might be a similar to bug 1321627

Stopping urfkill, removing state file and rebooting made it to work again

Tried for more than 20 times and can't reproduce the problem. Seems to be a hard one to reproduce.

I also assisted Julian Funk today with the same problem. This time on mako/RTM. Changed urfkill task to Confirmed accordingly.

https://github.com/alfonsosanchezbeato/urfkill/tree/strict-fm

is a solution to make sure that even when there are problems we can return to a valid state after a reboot.

It is difficult to reproduce this, I was able to arrive to a bad state following these steps:

1. Boot system with 2 SIMs, normal state
2. Offline both modems using ofono scripts
        $ cd /usr/share/ofono/scripts
        $ ./offline_modem /ril_0
        $ ./offline_modem /ril_1
3. Toggle on flight mode
4. Toggle off flight mode. The modems are not onlined, and soft=true for WWAN, but not for ALL in saved-states file.

urfkill is keeping the previous state of the modems (the state before FM was set). I wonder if for some reason/bug the modems were offline when the bug happened. The "searching" status could be for one modem, but the other could be offline, which is enough for triggering the bug. Or, the searching status was wrong and the modem was actually offline.

This PR makes WWAN state == flight mode (ALL) state, fixes the situation described in this comment and as a minimum would let us recover with a reboot in case we come across again with a "saved-states" with content as in comment #2:

https://github.com/cyphermox/urfkill/pull/19

Alfonso, the linked pull request has no regression test to confirm the fix for the problem you described. Can you write a test for it?

Also, from what you say about the searching status, it sounds like there should be better logging to know which modem is searching.

@Leo

What this change does is ensure that when "strict flight-mode" is enabled ( which is the default ), that the state of the modem will always follow the global state of flight-mode itself.

The only way to test this is to stop urfkill, and modify its saved-state file so that it contains "[WWAN] soft=true", reboot the phone, and ensure that the modem is active.

I discussed with Alfonso, and he'll add this as a new case to the exiting urfkill test plan.

I'd also like to note that this is a safeguard that will allow the user to recover their device if the original problem is hit again. It's not a total solution, so we should make sure not include the bug in the debian/changelog, as there's more work to do on this one...

Tony, a manual test is good, so thanks.

An automated test is even better. We are now working on the suite to test upgrades, and we learned about ways to do a reboot of the test bed using adt-run. So what you describe is automatable, but it's up to you to decide if it's worth it or it's better to keep it manual. Let us know if you are interested in adt-run reboots and we'll give you a hand.

Test 9 added to urfkill test plan:

https://wiki.ubuntu.com/Process/Merges/TestPlans/urfkill

We need to land this in both vivid and RTM.

Until the fix is released...

After running into this problem the following steps got my phone back up and running (Nexus 4, r214):

1) Open the terminal app on the phone and validate using password if configured

2) $ sudo stop urfkill

3) $ sudo rm /var/lib/urfkill/saved-states -rf

4) $ sudo reboot

The phone should come back up with the carrier re-enabled.

This bug was fixed in the package urfkill - 0.6.0~20150318.103828.5539c0d.1-0ubuntu1

---------------
urfkill (0.6.0~20150318.103828.5539c0d.1-0ubuntu1) vivid; urgency=medium

  * New upstream snapshot, bugfix only:
    - Guard against double call to set ofono's online property.
    - UrfConfig: make the strict flight mode a configuration parameter.
    - UrfConfig: Enforce strict flight mode when set. (LP: #1427439)
 -- Mathieu Trudel-Lapierre <email address hidden> Thu, 19 Mar 2015 20:50:13 -0400

tagging to land in the update

This bug was fixed in the package urfkill - 0.6.0~20150318.103828.5539c0d.1~rtm-0ubuntu1

---------------
urfkill (0.6.0~20150318.103828.5539c0d.1~rtm-0ubuntu1) 14.09; urgency=medium

  * New upstream snapshot, bugfix only:
    - Guard against double call to set ofono's online property.
    - UrfConfig: make the strict flight mode a configuration parameter.
    - UrfConfig: Enforce strict flight mode when set. (LP: #1427439)
 -- Mathieu Trudel-Lapierre <email address hidden> Thu, 19 Mar 2015 20:50:13 -0400

I've re-opened the urfkill (Ubuntu) task as the underlying bug can still occur, and the fix released is a workaround which will allow a user to recover their modem by rebooting the phone.

We still have not managed to reproduce the underlying problem. What's needed is a stress test script for urfkill which can be configured to run for hours. Something similar to the checkbox reboot, suspend, hibernate tests used by Canonical's mainstream HW enablement team.