Activity log for bug #557429

Date Who What changed Old value New value Message
2010-04-07 17:04:58 Jamie Strandboge bug added bug
2010-04-07 17:05:23 Jamie Strandboge nominated for series Ubuntu Lucid
2010-04-07 17:05:23 Jamie Strandboge bug task added linux (Ubuntu Lucid)
2010-04-07 17:06:16 Jamie Strandboge linux (Ubuntu Lucid): importance Undecided High
2010-04-07 17:07:31 Jamie Strandboge description Using the latest beta-2 server ISO and following http://testcases.qa.ubuntu.com/Install/ServerRAID1, booting out of sync RAID1 array fails with ext3 (comes up as syncd). Steps to reproduce: 1. in a kvm virtual machine, using 2 virtio qcow2 disks each 1768M in size, 768M ram and 2 VCPUs, in the installer I create the md devices: /dev/md0: 1.5G, ext3, / /dev/md1: ~350M, swap Choose to boot in degraded mode. All other installer options are defaults 2. reboot into Lucid install and check /proc/mdstat: ok, both disks show up and are in sync 3. shutdown VM. remove 2nd disk, power on the VM and check /proc/mdstat: ok, boots degraded and mdstat shows the disk 4. shutdown VM. reconnect 2nd disk and remove 1st disk, power on the VM and check /proc/mdstat: ok, boots degraded and mdstat shows the disk 5. shutdown VM. reconnect 1st disk (so now both disks are connected, but out of sync), power on the VM Expected results: At this point it should boot degraded with /proc/mdstat showing it is syncing (recovering). This is how it works with ext4. Actual results: Array comes up with both disks in the array and in sync. fsck notices this and complains a *lot*: /dev/md0 contains a filesystem with errors Duplicate or bad block in use Multiply-claimed block(s) in inode... ... /dev/md0: File /var/log/boot.log (inode #68710, mod time Wed Apr 7 11:35:59 2010) has multiply-claimed block(s), shared with 1 file(s): /dev/md0: /var/log/udev (inode #69925, mod time Wed Apr 7 11:35:59 2010) /dev/md0: /dev/mdo0: UNEXPECTED CONSISTENCY; RUN fsk MANUALLY. The boot loops infinitely on this because the mountall reports that fsck terminated with stats 4, then reports that '/' is a filesystem with errors, then tries again (and again, and again). See: http://iso.qa.ubuntu.com/qatracker/result/3918/286 Using the latest beta-2 server ISO and following http://testcases.qa.ubuntu.com/Install/ServerRAID1, booting out of sync RAID1 array fails with ext3 (comes up as syncd). Steps to reproduce: 1. in a kvm virtual machine, using 2 virtio qcow2 disks each 1768M in size, 768M ram and 2 VCPUs, in the installer I create the md devices: /dev/md0: 1.5G, ext3, / /dev/md1: ~350M, swap Choose to boot in degraded mode. All other installer options are defaults 2. reboot into Lucid install and check /proc/mdstat: ok, both disks show up and are in sync 3. shutdown VM. remove 2nd disk, power on the VM and check /proc/mdstat: ok, boots degraded and mdstat shows the disk 4. shutdown VM. reconnect 2nd disk and remove 1st disk, power on the VM and check /proc/mdstat: ok, boots degraded and mdstat shows the disk 5. shutdown VM. reconnect 1st disk (so now both disks are connected, but out of sync), power on the VM Expected results: At this point it should boot degraded with /proc/mdstat showing it is syncing (recovering). This is how it works with ext4. Actual results: Array comes up with both disks in the array and in sync. fsck notices this and complains a *lot*: /dev/md0 contains a filesystem with errors Duplicate or bad block in use Multiply-claimed block(s) in inode... ... /dev/md0: File /var/log/boot.log (inode #68710, mod time Wed Apr 7 11:35:59 2010) has multiply-claimed block(s), shared with 1 file(s):  /dev/md0: /var/log/udev (inode #69925, mod time Wed Apr 7 11:35:59 2010) /dev/md0: /dev/mdo0: UNEXPECTED CONSISTENCY; RUN fsk MANUALLY. The boot loops infinitely on this because the mountall reports that fsck terminated with stats 4, then reports that '/' is a filesystem with errors, then tries again (and again, and again). See: http://iso.qa.ubuntu.com/qatracker/result/3918/286 I filed this against 'linux'; please adjust as necessary.
2010-04-07 17:07:56 Jamie Strandboge summary booting out of sync RAID1 array fails with ext3 (comes up as syncd) booting out of sync RAID1 array fails with ext3 (comes up as already in sync)
2010-04-07 17:10:05 Ubuntu QA Website tags iso-testing
2010-04-07 17:44:14 Jamie Strandboge attachment added superblocks.txt http://launchpadlibrarian.net/43369032/superblocks.txt
2010-04-07 17:54:08 Jamie Strandboge description Using the latest beta-2 server ISO and following http://testcases.qa.ubuntu.com/Install/ServerRAID1, booting out of sync RAID1 array fails with ext3 (comes up as syncd). Steps to reproduce: 1. in a kvm virtual machine, using 2 virtio qcow2 disks each 1768M in size, 768M ram and 2 VCPUs, in the installer I create the md devices: /dev/md0: 1.5G, ext3, / /dev/md1: ~350M, swap Choose to boot in degraded mode. All other installer options are defaults 2. reboot into Lucid install and check /proc/mdstat: ok, both disks show up and are in sync 3. shutdown VM. remove 2nd disk, power on the VM and check /proc/mdstat: ok, boots degraded and mdstat shows the disk 4. shutdown VM. reconnect 2nd disk and remove 1st disk, power on the VM and check /proc/mdstat: ok, boots degraded and mdstat shows the disk 5. shutdown VM. reconnect 1st disk (so now both disks are connected, but out of sync), power on the VM Expected results: At this point it should boot degraded with /proc/mdstat showing it is syncing (recovering). This is how it works with ext4. Actual results: Array comes up with both disks in the array and in sync. fsck notices this and complains a *lot*: /dev/md0 contains a filesystem with errors Duplicate or bad block in use Multiply-claimed block(s) in inode... ... /dev/md0: File /var/log/boot.log (inode #68710, mod time Wed Apr 7 11:35:59 2010) has multiply-claimed block(s), shared with 1 file(s):  /dev/md0: /var/log/udev (inode #69925, mod time Wed Apr 7 11:35:59 2010) /dev/md0: /dev/mdo0: UNEXPECTED CONSISTENCY; RUN fsk MANUALLY. The boot loops infinitely on this because the mountall reports that fsck terminated with stats 4, then reports that '/' is a filesystem with errors, then tries again (and again, and again). See: http://iso.qa.ubuntu.com/qatracker/result/3918/286 I filed this against 'linux'; please adjust as necessary. Using the latest beta-2 server ISO and following http://testcases.qa.ubuntu.com/Install/ServerRAID1, booting out of sync RAID1 array fails with ext3 (comes up as syncd). Steps to reproduce: 1. in a kvm virtual machine, using 2 virtio qcow2 disks each 1768M in size, 768M ram and 2 VCPUs, in the installer I create the md devices: /dev/md0: 1.5G, ext3, / /dev/md1: ~350M, swap Choose to boot in degraded mode. All other installer options are defaults 2. reboot into Lucid install and check /proc/mdstat: ok, both disks show up and are in sync 3. shutdown VM. remove 2nd disk, power on the VM and check /proc/mdstat: ok, boots degraded and mdstat shows the disk 4. shutdown VM. reconnect 2nd disk and remove 1st disk, power on the VM and check /proc/mdstat: ok, boots degraded and mdstat shows the disk 5. shutdown VM. reconnect 1st disk (so now both disks are connected, but out of sync), power on the VM Expected results: At this point it should boot degraded with /proc/mdstat showing it is syncing (recovering). This is how it works with ext4. Note that in the past one would have to 'sudo mdadm -a /dev/md0 /dev/MISSING-DEVICE' before syncing would occur. This no longer seems to be required. Actual results: Array comes up with both disks in the array and in sync. fsck notices this and complains a *lot*: /dev/md0 contains a filesystem with errors Duplicate or bad block in use Multiply-claimed block(s) in inode... ... /dev/md0: File /var/log/boot.log (inode #68710, mod time Wed Apr 7 11:35:59 2010) has multiply-claimed block(s), shared with 1 file(s):  /dev/md0: /var/log/udev (inode #69925, mod time Wed Apr 7 11:35:59 2010) /dev/md0: /dev/mdo0: UNEXPECTED CONSISTENCY; RUN fsk MANUALLY. The boot loops infinitely on this because the mountall reports that fsck terminated with stats 4, then reports that '/' is a filesystem with errors, then tries again (and again, and again). See: http://iso.qa.ubuntu.com/qatracker/result/3918/286 I filed this against 'linux'; please adjust as necessary.
2010-04-07 18:03:13 Jamie Strandboge attachment added disk1_superblock_after_boot_degraded.txt http://launchpadlibrarian.net/43370398/disk1_superblock_after_boot_degraded.txt
2010-04-07 18:08:37 Jamie Strandboge attachment added disk2_superblock_after_boot_degraded.txt http://launchpadlibrarian.net/43370809/disk2_superblock_after_boot_degraded.txt
2010-04-07 18:24:33 Jamie Strandboge attachment added disk1_and_disk2_before_activate.txt http://launchpadlibrarian.net/43372051/disk1_and_disk2_before_activate.txt
2010-04-07 18:25:37 Jamie Strandboge attachment added disk1_and_disk2_after_autodetect.txt http://launchpadlibrarian.net/43372149/disk1_and_disk2_after_autodetect.txt
2010-04-07 18:39:18 Jamie Strandboge description Using the latest beta-2 server ISO and following http://testcases.qa.ubuntu.com/Install/ServerRAID1, booting out of sync RAID1 array fails with ext3 (comes up as syncd). Steps to reproduce: 1. in a kvm virtual machine, using 2 virtio qcow2 disks each 1768M in size, 768M ram and 2 VCPUs, in the installer I create the md devices: /dev/md0: 1.5G, ext3, / /dev/md1: ~350M, swap Choose to boot in degraded mode. All other installer options are defaults 2. reboot into Lucid install and check /proc/mdstat: ok, both disks show up and are in sync 3. shutdown VM. remove 2nd disk, power on the VM and check /proc/mdstat: ok, boots degraded and mdstat shows the disk 4. shutdown VM. reconnect 2nd disk and remove 1st disk, power on the VM and check /proc/mdstat: ok, boots degraded and mdstat shows the disk 5. shutdown VM. reconnect 1st disk (so now both disks are connected, but out of sync), power on the VM Expected results: At this point it should boot degraded with /proc/mdstat showing it is syncing (recovering). This is how it works with ext4. Note that in the past one would have to 'sudo mdadm -a /dev/md0 /dev/MISSING-DEVICE' before syncing would occur. This no longer seems to be required. Actual results: Array comes up with both disks in the array and in sync. fsck notices this and complains a *lot*: /dev/md0 contains a filesystem with errors Duplicate or bad block in use Multiply-claimed block(s) in inode... ... /dev/md0: File /var/log/boot.log (inode #68710, mod time Wed Apr 7 11:35:59 2010) has multiply-claimed block(s), shared with 1 file(s):  /dev/md0: /var/log/udev (inode #69925, mod time Wed Apr 7 11:35:59 2010) /dev/md0: /dev/mdo0: UNEXPECTED CONSISTENCY; RUN fsk MANUALLY. The boot loops infinitely on this because the mountall reports that fsck terminated with stats 4, then reports that '/' is a filesystem with errors, then tries again (and again, and again). See: http://iso.qa.ubuntu.com/qatracker/result/3918/286 I filed this against 'linux'; please adjust as necessary. Using the latest beta-2 server ISO and following http://testcases.qa.ubuntu.com/Install/ServerRAID1, booting out of sync RAID1 array fails with ext3 (comes up as syncd). Steps to reproduce: 1. in a kvm virtual machine, using 2 virtio qcow2 disks each 1768M in size, 768M ram and 2 VCPUs, in the installer I create the md devices: /dev/md0: 1.5G, ext3, / /dev/md1: ~350M, swap Choose to boot in degraded mode. All other installer options are defaults 2. reboot into Lucid install and check /proc/mdstat: ok, both disks show up and are in sync 3. shutdown VM. remove 2nd disk, power on the VM and check /proc/mdstat: ok, boots degraded and mdstat shows the disk 4. shutdown VM. reconnect 2nd disk and remove 1st disk, power on the VM and check /proc/mdstat: ok, boots degraded and mdstat shows the disk 5. shutdown VM. reconnect 1st disk (so now both disks are connected, but out of sync), power on the VM Expected results: At this point it should boot degraded with /proc/mdstat showing it is syncing (recovering). This is how it works with ext4. Note that in the past one would have to 'sudo mdadm -a /dev/md0 /dev/MISSING-DEVICE' before syncing would occur. This no longer seems to be required. Actual results: Array comes up with both disks in the array and in sync. Sometimes the are error messages saying that there are disk errors, and the boot continues to login, but root is mounted readonly and /proc/mdstat shows we are in sync. Sometimes fsck notices this and complains a *lot*: /dev/md0 contains a filesystem with errors Duplicate or bad block in use Multiply-claimed block(s) in inode... ... /dev/md0: File /var/log/boot.log (inode #68710, mod time Wed Apr 7 11:35:59 2010) has multiply-claimed block(s), shared with 1 file(s):  /dev/md0: /var/log/udev (inode #69925, mod time Wed Apr 7 11:35:59 2010) /dev/md0: /dev/mdo0: UNEXPECTED CONSISTENCY; RUN fsk MANUALLY. The boot loops infinitely on this because the mountall reports that fsck terminated with stats 4, then reports that '/' is a filesystem with errors, then tries again (and again, and again). See: http://iso.qa.ubuntu.com/qatracker/result/3918/286 I filed this against 'linux'; please adjust as necessary.
2010-04-07 19:21:47 Jamie Strandboge description Using the latest beta-2 server ISO and following http://testcases.qa.ubuntu.com/Install/ServerRAID1, booting out of sync RAID1 array fails with ext3 (comes up as syncd). Steps to reproduce: 1. in a kvm virtual machine, using 2 virtio qcow2 disks each 1768M in size, 768M ram and 2 VCPUs, in the installer I create the md devices: /dev/md0: 1.5G, ext3, / /dev/md1: ~350M, swap Choose to boot in degraded mode. All other installer options are defaults 2. reboot into Lucid install and check /proc/mdstat: ok, both disks show up and are in sync 3. shutdown VM. remove 2nd disk, power on the VM and check /proc/mdstat: ok, boots degraded and mdstat shows the disk 4. shutdown VM. reconnect 2nd disk and remove 1st disk, power on the VM and check /proc/mdstat: ok, boots degraded and mdstat shows the disk 5. shutdown VM. reconnect 1st disk (so now both disks are connected, but out of sync), power on the VM Expected results: At this point it should boot degraded with /proc/mdstat showing it is syncing (recovering). This is how it works with ext4. Note that in the past one would have to 'sudo mdadm -a /dev/md0 /dev/MISSING-DEVICE' before syncing would occur. This no longer seems to be required. Actual results: Array comes up with both disks in the array and in sync. Sometimes the are error messages saying that there are disk errors, and the boot continues to login, but root is mounted readonly and /proc/mdstat shows we are in sync. Sometimes fsck notices this and complains a *lot*: /dev/md0 contains a filesystem with errors Duplicate or bad block in use Multiply-claimed block(s) in inode... ... /dev/md0: File /var/log/boot.log (inode #68710, mod time Wed Apr 7 11:35:59 2010) has multiply-claimed block(s), shared with 1 file(s):  /dev/md0: /var/log/udev (inode #69925, mod time Wed Apr 7 11:35:59 2010) /dev/md0: /dev/mdo0: UNEXPECTED CONSISTENCY; RUN fsk MANUALLY. The boot loops infinitely on this because the mountall reports that fsck terminated with stats 4, then reports that '/' is a filesystem with errors, then tries again (and again, and again). See: http://iso.qa.ubuntu.com/qatracker/result/3918/286 I filed this against 'linux'; please adjust as necessary. Using the latest beta-2 server ISO and following http://testcases.qa.ubuntu.com/Install/ServerRAID1, booting out of sync RAID1 array fails with ext3 (comes up as syncd). Steps to reproduce: 1. in a kvm virtual machine, using 2 virtio qcow2 disks each 1768M in size, 768M ram and 2 VCPUs, in the installer I create the md devices: /dev/md0: 1.5G, ext3, / /dev/md1: ~350M, swap Choose to boot in degraded mode. All other installer options are defaults 2. reboot into Lucid install and check /proc/mdstat: ok, both disks show up and are in sync 3. shutdown VM. remove 2nd disk, power on the VM and check /proc/mdstat: ok, boots degraded and mdstat shows the disk 4. shutdown VM. reconnect 2nd disk and remove 1st disk, power on the VM and check /proc/mdstat: ok, boots degraded and mdstat shows the disk 5. shutdown VM. reconnect 1st disk (so now both disks are connected, but out of sync), power on the VM Expected results: At this point it should boot degraded with /proc/mdstat showing it is syncing (recovering). This is how it works with ext4. Note that in the past one would have to 'sudo mdadm -a /dev/md0 /dev/MISSING-DEVICE' before syncing would occur. This no longer seems to be required. Actual results: Array comes up with both disks in the array and in sync. Sometimes there are error messages saying there are disk errors, and the boot continues to login, but root is mounted readonly and /proc/mdstat shows we are in sync. Sometimes fsck notices this and complains a *lot*: /dev/md0 contains a filesystem with errors Duplicate or bad block in use Multiply-claimed block(s) in inode... ... /dev/md0: File /var/log/boot.log (inode #68710, mod time Wed Apr 7 11:35:59 2010) has multiply-claimed block(s), shared with 1 file(s):  /dev/md0: /var/log/udev (inode #69925, mod time Wed Apr 7 11:35:59 2010) /dev/md0: /dev/mdo0: UNEXPECTED CONSISTENCY; RUN fsck MANUALLY. The boot loops infinitely on this because the mountall reports that fsck terminated with status 4, then reports that '/' is a filesystem with errors, then tries again (and again, and again). See: http://iso.qa.ubuntu.com/qatracker/result/3918/286 I filed this against 'linux'; please adjust as necessary.
2010-04-07 20:43:29 Phillip Susi linux (Ubuntu Lucid): status New Confirmed
2010-04-08 20:52:02 ceg affects linux (Ubuntu Lucid) mdadm (Ubuntu Lucid)
2010-04-09 15:34:14 Leann Ogasawara mdadm (Ubuntu Lucid): status Confirmed Triaged
2010-04-09 15:34:14 Leann Ogasawara mdadm (Ubuntu Lucid): milestone ubuntu-10.04
2010-04-09 15:34:14 Leann Ogasawara mdadm (Ubuntu Lucid): assignee Canonical Kernel Team (canonical-kernel-team)
2010-04-09 16:02:18 Leann Ogasawara mdadm (Ubuntu Lucid): assignee Canonical Kernel Team (canonical-kernel-team)
2010-04-10 20:09:27 Dirk removed subscriber Pip
2010-04-14 13:23:20 ceg summary booting out of sync RAID1 array fails with ext3 (comes up as already in sync) booting out of sync RAID1 array comes up as already in sync (data-corruption)
2010-04-14 13:47:23 ceg description Using the latest beta-2 server ISO and following http://testcases.qa.ubuntu.com/Install/ServerRAID1, booting out of sync RAID1 array fails with ext3 (comes up as syncd). Steps to reproduce: 1. in a kvm virtual machine, using 2 virtio qcow2 disks each 1768M in size, 768M ram and 2 VCPUs, in the installer I create the md devices: /dev/md0: 1.5G, ext3, / /dev/md1: ~350M, swap Choose to boot in degraded mode. All other installer options are defaults 2. reboot into Lucid install and check /proc/mdstat: ok, both disks show up and are in sync 3. shutdown VM. remove 2nd disk, power on the VM and check /proc/mdstat: ok, boots degraded and mdstat shows the disk 4. shutdown VM. reconnect 2nd disk and remove 1st disk, power on the VM and check /proc/mdstat: ok, boots degraded and mdstat shows the disk 5. shutdown VM. reconnect 1st disk (so now both disks are connected, but out of sync), power on the VM Expected results: At this point it should boot degraded with /proc/mdstat showing it is syncing (recovering). This is how it works with ext4. Note that in the past one would have to 'sudo mdadm -a /dev/md0 /dev/MISSING-DEVICE' before syncing would occur. This no longer seems to be required. Actual results: Array comes up with both disks in the array and in sync. Sometimes there are error messages saying there are disk errors, and the boot continues to login, but root is mounted readonly and /proc/mdstat shows we are in sync. Sometimes fsck notices this and complains a *lot*: /dev/md0 contains a filesystem with errors Duplicate or bad block in use Multiply-claimed block(s) in inode... ... /dev/md0: File /var/log/boot.log (inode #68710, mod time Wed Apr 7 11:35:59 2010) has multiply-claimed block(s), shared with 1 file(s):  /dev/md0: /var/log/udev (inode #69925, mod time Wed Apr 7 11:35:59 2010) /dev/md0: /dev/mdo0: UNEXPECTED CONSISTENCY; RUN fsck MANUALLY. The boot loops infinitely on this because the mountall reports that fsck terminated with status 4, then reports that '/' is a filesystem with errors, then tries again (and again, and again). See: http://iso.qa.ubuntu.com/qatracker/result/3918/286 I filed this against 'linux'; please adjust as necessary. Using the latest beta-2 server ISO and following http://testcases.qa.ubuntu.com/Install/ServerRAID1 Booting out of sync RAID1 array fails with ext3: It comes up as synced, but is corrupted. (According to comment #18: ext3 vs ext4 seems to be mere happenstance.) Steps to reproduce: 1. in a kvm virtual machine, using 2 virtio qcow2 disks each 1768M in size, 768M ram and 2 VCPUs, in the installer I create the md devices: /dev/md0: 1.5G, ext3, / /dev/md1: ~350M, swap Choose to boot in degraded mode. All other installer options are defaults 2. reboot into Lucid install and check /proc/mdstat: ok, both disks show up and are in sync 3. shutdown VM. remove 2nd disk, power on the VM and check /proc/mdstat: ok, boots degraded and mdstat shows the disk 4. shutdown VM. reconnect 2nd disk and remove 1st disk, power on the VM and check /proc/mdstat: ok, boots degraded and mdstat shows the disk 5. shutdown VM. reconnect 1st disk (so now both disks are connected, but out of sync), power on the VM Expected results: At this point it should boot degraded with /proc/mdstat showing it is syncing (recovering). This is how it works with ext4. Note that in the past one would have to 'sudo mdadm -a /dev/md0 /dev/MISSING-DEVICE' before syncing would occur. This no longer seems to be required. Actual results: Array comes up with both disks in the array and in sync. Sometimes there are error messages saying there are disk errors, and the boot continues to login, but root is mounted readonly and /proc/mdstat shows we are in sync. Sometimes fsck notices this and complains a *lot*: /dev/md0 contains a filesystem with errors Duplicate or bad block in use Multiply-claimed block(s) in inode... ... /dev/md0: File /var/log/boot.log (inode #68710, mod time Wed Apr 7 11:35:59 2010) has multiply-claimed block(s), shared with 1 file(s):  /dev/md0: /var/log/udev (inode #69925, mod time Wed Apr 7 11:35:59 2010) /dev/md0: /dev/mdo0: UNEXPECTED CONSISTENCY; RUN fsck MANUALLY. The boot loops infinitely on this because the mountall reports that fsck terminated with status 4, then reports that '/' is a filesystem with errors, then tries again (and again, and again). See: http://iso.qa.ubuntu.com/qatracker/result/3918/286 I filed this against 'linux'; please adjust as necessary.
2010-04-14 14:59:19 ceg summary booting out of sync RAID1 array comes up as already in sync (data-corruption) array with conflicting changes is assembled with data corruption/silent loss
2010-04-14 15:08:22 ceg bug task added mdadm
2010-04-16 15:39:44 Thierry Carrez mdadm (Ubuntu Lucid): assignee Dustin Kirkland (kirkland)
2010-04-16 15:40:03 Thierry Carrez mdadm (Ubuntu Lucid): milestone ubuntu-10.04
2010-04-19 20:03:32 Dustin Kirkland  mdadm (Ubuntu Lucid): assignee Dustin Kirkland (kirkland)
2010-04-19 20:03:40 Dustin Kirkland  mdadm (Ubuntu Lucid): status Triaged Won't Fix
2010-04-19 20:26:13 Matija Polajnar removed subscriber Matija Polajnar
2010-04-20 10:32:19 Steve Langasek mdadm (Ubuntu Lucid): status Won't Fix Triaged
2010-04-20 19:49:00 ceg description Using the latest beta-2 server ISO and following http://testcases.qa.ubuntu.com/Install/ServerRAID1 Booting out of sync RAID1 array fails with ext3: It comes up as synced, but is corrupted. (According to comment #18: ext3 vs ext4 seems to be mere happenstance.) Steps to reproduce: 1. in a kvm virtual machine, using 2 virtio qcow2 disks each 1768M in size, 768M ram and 2 VCPUs, in the installer I create the md devices: /dev/md0: 1.5G, ext3, / /dev/md1: ~350M, swap Choose to boot in degraded mode. All other installer options are defaults 2. reboot into Lucid install and check /proc/mdstat: ok, both disks show up and are in sync 3. shutdown VM. remove 2nd disk, power on the VM and check /proc/mdstat: ok, boots degraded and mdstat shows the disk 4. shutdown VM. reconnect 2nd disk and remove 1st disk, power on the VM and check /proc/mdstat: ok, boots degraded and mdstat shows the disk 5. shutdown VM. reconnect 1st disk (so now both disks are connected, but out of sync), power on the VM Expected results: At this point it should boot degraded with /proc/mdstat showing it is syncing (recovering). This is how it works with ext4. Note that in the past one would have to 'sudo mdadm -a /dev/md0 /dev/MISSING-DEVICE' before syncing would occur. This no longer seems to be required. Actual results: Array comes up with both disks in the array and in sync. Sometimes there are error messages saying there are disk errors, and the boot continues to login, but root is mounted readonly and /proc/mdstat shows we are in sync. Sometimes fsck notices this and complains a *lot*: /dev/md0 contains a filesystem with errors Duplicate or bad block in use Multiply-claimed block(s) in inode... ... /dev/md0: File /var/log/boot.log (inode #68710, mod time Wed Apr 7 11:35:59 2010) has multiply-claimed block(s), shared with 1 file(s):  /dev/md0: /var/log/udev (inode #69925, mod time Wed Apr 7 11:35:59 2010) /dev/md0: /dev/mdo0: UNEXPECTED CONSISTENCY; RUN fsck MANUALLY. The boot loops infinitely on this because the mountall reports that fsck terminated with status 4, then reports that '/' is a filesystem with errors, then tries again (and again, and again). See: http://iso.qa.ubuntu.com/qatracker/result/3918/286 I filed this against 'linux'; please adjust as necessary. Re-attaching parts of an array that have been running degraded separately and contain the same amount and conflicting changes, results in the assembly of a corrupt array. ---- Using the latest beta-2 server ISO and following http://testcases.qa.ubuntu.com/Install/ServerRAID1 Booting out of sync RAID1 array fails with ext3: It comes up as synced, but is corrupted.      (According to comment #18: ext3 vs ext4 seems to be mere happenstance.) Steps to reproduce: 1. in a kvm virtual machine, using 2 virtio qcow2 disks each 1768M in size, 768M ram and 2 VCPUs, in the installer I create the md devices: /dev/md0: 1.5G, ext3, / /dev/md1: ~350M, swap Choose to boot in degraded mode. All other installer options are defaults 2. reboot into Lucid install and check /proc/mdstat: ok, both disks show up and are in sync 3. shutdown VM. remove 2nd disk, power on the VM and check /proc/mdstat: ok, boots degraded and mdstat shows the disk 4. shutdown VM. reconnect 2nd disk and remove 1st disk, power on the VM and check /proc/mdstat: ok, boots degraded and mdstat shows the disk 5. shutdown VM. reconnect 1st disk (so now both disks are connected, but out of sync), power on the VM Expected results: At this point it should boot degraded with /proc/mdstat showing it is syncing (recovering). This is how it works with ext4. Note that in the past one would have to 'sudo mdadm -a /dev/md0 /dev/MISSING-DEVICE' before syncing would occur. This no longer seems to be required. Actual results: Array comes up with both disks in the array and in sync. Sometimes there are error messages saying there are disk errors, and the boot continues to login, but root is mounted readonly and /proc/mdstat shows we are in sync. Sometimes fsck notices this and complains a *lot*: /dev/md0 contains a filesystem with errors Duplicate or bad block in use Multiply-claimed block(s) in inode... ... /dev/md0: File /var/log/boot.log (inode #68710, mod time Wed Apr 7 11:35:59 2010) has multiply-claimed block(s), shared with 1 file(s):  /dev/md0: /var/log/udev (inode #69925, mod time Wed Apr 7 11:35:59 2010) /dev/md0: /dev/mdo0: UNEXPECTED CONSISTENCY; RUN fsck MANUALLY. The boot loops infinitely on this because the mountall reports that fsck terminated with status 4, then reports that '/' is a filesystem with errors, then tries again (and again, and again). See: http://iso.qa.ubuntu.com/qatracker/result/3918/286 I filed this against 'linux'; please adjust as necessary. ----- Fixing this would support safe hot-pluggable segmentation of arrays: (arrays are only --run degraded manually or if required and incomplete during boot) * --incremental should stop auto re-adding "removed" members (so that --remove provides a manual means turn hot-plugging off) * When arrays are --run degraded missing members should be marked "failed" but not "removed". * Always check for conflicting "failed" states in superblocks, to detect conflicting changes. + always report (console and --monitor event) if conflicting changes are detected + require --force with --add for a manual re-sync of conflicting changes (unlike with resyncing an outdated device, in this case changes will get lost) * To facilitate inspection --incremental should assemble array components with conflicting changes into auxiliary devices with mangled UUIDs (safe and easy diffing, merging, etc. even on desktop level)
2010-04-22 09:20:37 VJK removed subscriber VJK
2010-04-23 12:52:27 Dustin Kirkland  removed subscriber Dustin Kirkland
2010-04-25 09:36:09 ceg description Re-attaching parts of an array that have been running degraded separately and contain the same amount and conflicting changes, results in the assembly of a corrupt array. ---- Using the latest beta-2 server ISO and following http://testcases.qa.ubuntu.com/Install/ServerRAID1 Booting out of sync RAID1 array fails with ext3: It comes up as synced, but is corrupted.      (According to comment #18: ext3 vs ext4 seems to be mere happenstance.) Steps to reproduce: 1. in a kvm virtual machine, using 2 virtio qcow2 disks each 1768M in size, 768M ram and 2 VCPUs, in the installer I create the md devices: /dev/md0: 1.5G, ext3, / /dev/md1: ~350M, swap Choose to boot in degraded mode. All other installer options are defaults 2. reboot into Lucid install and check /proc/mdstat: ok, both disks show up and are in sync 3. shutdown VM. remove 2nd disk, power on the VM and check /proc/mdstat: ok, boots degraded and mdstat shows the disk 4. shutdown VM. reconnect 2nd disk and remove 1st disk, power on the VM and check /proc/mdstat: ok, boots degraded and mdstat shows the disk 5. shutdown VM. reconnect 1st disk (so now both disks are connected, but out of sync), power on the VM Expected results: At this point it should boot degraded with /proc/mdstat showing it is syncing (recovering). This is how it works with ext4. Note that in the past one would have to 'sudo mdadm -a /dev/md0 /dev/MISSING-DEVICE' before syncing would occur. This no longer seems to be required. Actual results: Array comes up with both disks in the array and in sync. Sometimes there are error messages saying there are disk errors, and the boot continues to login, but root is mounted readonly and /proc/mdstat shows we are in sync. Sometimes fsck notices this and complains a *lot*: /dev/md0 contains a filesystem with errors Duplicate or bad block in use Multiply-claimed block(s) in inode... ... /dev/md0: File /var/log/boot.log (inode #68710, mod time Wed Apr 7 11:35:59 2010) has multiply-claimed block(s), shared with 1 file(s):  /dev/md0: /var/log/udev (inode #69925, mod time Wed Apr 7 11:35:59 2010) /dev/md0: /dev/mdo0: UNEXPECTED CONSISTENCY; RUN fsck MANUALLY. The boot loops infinitely on this because the mountall reports that fsck terminated with status 4, then reports that '/' is a filesystem with errors, then tries again (and again, and again). See: http://iso.qa.ubuntu.com/qatracker/result/3918/286 I filed this against 'linux'; please adjust as necessary. ----- Fixing this would support safe hot-pluggable segmentation of arrays: (arrays are only --run degraded manually or if required and incomplete during boot) * --incremental should stop auto re-adding "removed" members (so that --remove provides a manual means turn hot-plugging off) * When arrays are --run degraded missing members should be marked "failed" but not "removed". * Always check for conflicting "failed" states in superblocks, to detect conflicting changes. + always report (console and --monitor event) if conflicting changes are detected + require --force with --add for a manual re-sync of conflicting changes (unlike with resyncing an outdated device, in this case changes will get lost) * To facilitate inspection --incremental should assemble array components with conflicting changes into auxiliary devices with mangled UUIDs (safe and easy diffing, merging, etc. even on desktop level) Re-attaching parts of an array that have been running degraded separately and contain the same amount and conflicting changes, results in the assembly of a corrupt array. ---- Using the latest beta-2 server ISO and following http://testcases.qa.ubuntu.com/Install/ServerRAID1 Booting out of sync RAID1 array fails with ext3: It comes up as synced, but is corrupted.      (According to comment #18: ext3 vs ext4 seems to be mere happenstance.) Steps to reproduce: 1. in a kvm virtual machine, using 2 virtio qcow2 disks each 1768M in size, 768M ram and 2 VCPUs, in the installer I create the md devices: /dev/md0: 1.5G, ext3, / /dev/md1: ~350M, swap Choose to boot in degraded mode. All other installer options are defaults 2. reboot into Lucid install and check /proc/mdstat: ok, both disks show up and are in sync 3. shutdown VM. remove 2nd disk, power on the VM and check /proc/mdstat: ok, boots degraded and mdstat shows the disk 4. shutdown VM. reconnect 2nd disk and remove 1st disk, power on the VM and check /proc/mdstat: ok, boots degraded and mdstat shows the disk 5. shutdown VM. reconnect 1st disk (so now both disks are connected, but out of sync), power on the VM Expected results: At this point it should boot degraded with /proc/mdstat showing it is syncing (recovering). This is how it works with ext4. Note that in the past one would have to 'sudo mdadm -a /dev/md0 /dev/MISSING-DEVICE' before syncing would occur. This no longer seems to be required. Actual results: Array comes up with both disks in the array and in sync. Sometimes there are error messages saying there are disk errors, and the boot continues to login, but root is mounted readonly and /proc/mdstat shows we are in sync. Sometimes fsck notices this and complains a *lot*: /dev/md0 contains a filesystem with errors Duplicate or bad block in use Multiply-claimed block(s) in inode... ... /dev/md0: File /var/log/boot.log (inode #68710, mod time Wed Apr 7 11:35:59 2010) has multiply-claimed block(s), shared with 1 file(s):  /dev/md0: /var/log/udev (inode #69925, mod time Wed Apr 7 11:35:59 2010) /dev/md0: /dev/mdo0: UNEXPECTED CONSISTENCY; RUN fsck MANUALLY. The boot loops infinitely on this because the mountall reports that fsck terminated with status 4, then reports that '/' is a filesystem with errors, then tries again (and again, and again). See: http://iso.qa.ubuntu.com/qatracker/result/3918/286 I filed this against 'linux'; please adjust as necessary. ----- Fixing: * When assembling, mdadm could check for conflicting "failed" states in the superblocks of members to detect conflicting changes. On conflicts, i.e. if an additional member claims an allready running member has failed: + that member should not be added to the array + report (console and --monitor event) that an alternative version with conflicting changes has been detected "mdadm: not re-adding /dev/≤member> to /dev/≤array> because constitutes an alternative version containing conflicting changes" + require and support --force with --add for manual re-syncing of alternative versions (because unlike with re-syncing outdated devices/versions, in this case changes will get lost). Enhancement 1) To facilitate easy inspection of alternative versions (i.e. for safe and easy diffing, merging, etc.) --incremental could assemble array components that contain alternative versions into temporary auxiliary devices. (would require temporarily mangling the fs UUID to ensure there are no duplicates in the system) Enhancement 2) Those that want to be able to disable hot-plugging of segments with conflicting changes/alternative versions (after an incidence with multiple versions connected at the same time occured) will need some additional enhancements: + A way to mark some raid members (segments) as containing known alternative versions, and to mark them as such when an incident occurs in which they come up after another segment of the array is already running degraded. (possibly a superblock marking itself as failed) + An option like "AUTO -SINGLE_SEGMENTS_WITH_KNOWN_ALTERNATIVE_VERSIONS" to disable hotplug support for alternative versions once they came up after some other version and got marked as containig an alternative version.
2010-04-25 09:38:26 ceg description Re-attaching parts of an array that have been running degraded separately and contain the same amount and conflicting changes, results in the assembly of a corrupt array. ---- Using the latest beta-2 server ISO and following http://testcases.qa.ubuntu.com/Install/ServerRAID1 Booting out of sync RAID1 array fails with ext3: It comes up as synced, but is corrupted.      (According to comment #18: ext3 vs ext4 seems to be mere happenstance.) Steps to reproduce: 1. in a kvm virtual machine, using 2 virtio qcow2 disks each 1768M in size, 768M ram and 2 VCPUs, in the installer I create the md devices: /dev/md0: 1.5G, ext3, / /dev/md1: ~350M, swap Choose to boot in degraded mode. All other installer options are defaults 2. reboot into Lucid install and check /proc/mdstat: ok, both disks show up and are in sync 3. shutdown VM. remove 2nd disk, power on the VM and check /proc/mdstat: ok, boots degraded and mdstat shows the disk 4. shutdown VM. reconnect 2nd disk and remove 1st disk, power on the VM and check /proc/mdstat: ok, boots degraded and mdstat shows the disk 5. shutdown VM. reconnect 1st disk (so now both disks are connected, but out of sync), power on the VM Expected results: At this point it should boot degraded with /proc/mdstat showing it is syncing (recovering). This is how it works with ext4. Note that in the past one would have to 'sudo mdadm -a /dev/md0 /dev/MISSING-DEVICE' before syncing would occur. This no longer seems to be required. Actual results: Array comes up with both disks in the array and in sync. Sometimes there are error messages saying there are disk errors, and the boot continues to login, but root is mounted readonly and /proc/mdstat shows we are in sync. Sometimes fsck notices this and complains a *lot*: /dev/md0 contains a filesystem with errors Duplicate or bad block in use Multiply-claimed block(s) in inode... ... /dev/md0: File /var/log/boot.log (inode #68710, mod time Wed Apr 7 11:35:59 2010) has multiply-claimed block(s), shared with 1 file(s):  /dev/md0: /var/log/udev (inode #69925, mod time Wed Apr 7 11:35:59 2010) /dev/md0: /dev/mdo0: UNEXPECTED CONSISTENCY; RUN fsck MANUALLY. The boot loops infinitely on this because the mountall reports that fsck terminated with status 4, then reports that '/' is a filesystem with errors, then tries again (and again, and again). See: http://iso.qa.ubuntu.com/qatracker/result/3918/286 I filed this against 'linux'; please adjust as necessary. ----- Fixing: * When assembling, mdadm could check for conflicting "failed" states in the superblocks of members to detect conflicting changes. On conflicts, i.e. if an additional member claims an allready running member has failed: + that member should not be added to the array + report (console and --monitor event) that an alternative version with conflicting changes has been detected "mdadm: not re-adding /dev/≤member> to /dev/≤array> because constitutes an alternative version containing conflicting changes" + require and support --force with --add for manual re-syncing of alternative versions (because unlike with re-syncing outdated devices/versions, in this case changes will get lost). Enhancement 1) To facilitate easy inspection of alternative versions (i.e. for safe and easy diffing, merging, etc.) --incremental could assemble array components that contain alternative versions into temporary auxiliary devices. (would require temporarily mangling the fs UUID to ensure there are no duplicates in the system) Enhancement 2) Those that want to be able to disable hot-plugging of segments with conflicting changes/alternative versions (after an incidence with multiple versions connected at the same time occured) will need some additional enhancements: + A way to mark some raid members (segments) as containing known alternative versions, and to mark them as such when an incident occurs in which they come up after another segment of the array is already running degraded. (possibly a superblock marking itself as failed) + An option like "AUTO -SINGLE_SEGMENTS_WITH_KNOWN_ALTERNATIVE_VERSIONS" to disable hotplug support for alternative versions once they came up after some other version and got marked as containig an alternative version. Re-attaching parts of an array that have been running degraded separately and contain the same amount and conflicting changes or use a write intent bitmap, results in the assembly of a corrupt array. ---- Using the latest beta-2 server ISO and following http://testcases.qa.ubuntu.com/Install/ServerRAID1 Booting out of sync RAID1 array fails with ext3: It comes up as synced, but is corrupted.      (According to comment #18: ext3 vs ext4 seems to be mere happenstance.) Steps to reproduce: 1. in a kvm virtual machine, using 2 virtio qcow2 disks each 1768M in size, 768M ram and 2 VCPUs, in the installer I create the md devices: /dev/md0: 1.5G, ext3, / /dev/md1: ~350M, swap Choose to boot in degraded mode. All other installer options are defaults 2. reboot into Lucid install and check /proc/mdstat: ok, both disks show up and are in sync 3. shutdown VM. remove 2nd disk, power on the VM and check /proc/mdstat: ok, boots degraded and mdstat shows the disk 4. shutdown VM. reconnect 2nd disk and remove 1st disk, power on the VM and check /proc/mdstat: ok, boots degraded and mdstat shows the disk 5. shutdown VM. reconnect 1st disk (so now both disks are connected, but out of sync), power on the VM Expected results: At this point it should boot degraded with /proc/mdstat showing it is syncing (recovering). This is how it works with ext4. Note that in the past one would have to 'sudo mdadm -a /dev/md0 /dev/MISSING-DEVICE' before syncing would occur. This no longer seems to be required. Actual results: Array comes up with both disks in the array and in sync. Sometimes there are error messages saying there are disk errors, and the boot continues to login, but root is mounted readonly and /proc/mdstat shows we are in sync. Sometimes fsck notices this and complains a *lot*: /dev/md0 contains a filesystem with errors Duplicate or bad block in use Multiply-claimed block(s) in inode... ... /dev/md0: File /var/log/boot.log (inode #68710, mod time Wed Apr 7 11:35:59 2010) has multiply-claimed block(s), shared with 1 file(s):  /dev/md0: /var/log/udev (inode #69925, mod time Wed Apr 7 11:35:59 2010) /dev/md0: /dev/mdo0: UNEXPECTED CONSISTENCY; RUN fsck MANUALLY. The boot loops infinitely on this because the mountall reports that fsck terminated with status 4, then reports that '/' is a filesystem with errors, then tries again (and again, and again). See: http://iso.qa.ubuntu.com/qatracker/result/3918/286 I filed this against 'linux'; please adjust as necessary. ----- Fixing: * When assembling, mdadm could check for conflicting "failed" states in the   superblocks of members to detect conflicting changes. On conflicts, i.e. if an   additional member claims an allready running member has failed:    + that member should not be added to the array    + report (console and --monitor event) that an alternative      version with conflicting changes has been detected "mdadm: not      re-adding /dev/≤member> to /dev/≤array> because constitutes an      alternative version containing conflicting changes"    + require and support --force with --add for manual re-syncing of      alternative versions (because unlike with re-syncing outdated      devices/versions, in this case changes will get lost). Enhancement 1)   To facilitate easy inspection of alternative versions (i.e. for safe and   easy diffing, merging, etc.) --incremental could assemble array   components that contain alternative versions into temporary   auxiliary devices.   (would require temporarily mangling the fs UUID to ensure there are no   duplicates in the system) Enhancement 2)   Those that want to be able to disable hot-plugging of   segments with conflicting changes/alternative versions (after an   incidence with multiple versions connected at the same time occured)   will need some additional enhancements:    + A way to mark some raid members (segments) as containing      known alternative versions, and to mark them as such when an      incident occurs in which they come up after another      segment of the array is already running degraded.      (possibly a superblock marking itself as failed)    + An option like      "AUTO -SINGLE_SEGMENTS_WITH_KNOWN_ALTERNATIVE_VERSIONS"      to disable hotplug support for alternative versions once they came      up after some other version and got marked as containig an alternative version.
2010-04-26 00:38:53 ceg description Re-attaching parts of an array that have been running degraded separately and contain the same amount and conflicting changes or use a write intent bitmap, results in the assembly of a corrupt array. ---- Using the latest beta-2 server ISO and following http://testcases.qa.ubuntu.com/Install/ServerRAID1 Booting out of sync RAID1 array fails with ext3: It comes up as synced, but is corrupted.      (According to comment #18: ext3 vs ext4 seems to be mere happenstance.) Steps to reproduce: 1. in a kvm virtual machine, using 2 virtio qcow2 disks each 1768M in size, 768M ram and 2 VCPUs, in the installer I create the md devices: /dev/md0: 1.5G, ext3, / /dev/md1: ~350M, swap Choose to boot in degraded mode. All other installer options are defaults 2. reboot into Lucid install and check /proc/mdstat: ok, both disks show up and are in sync 3. shutdown VM. remove 2nd disk, power on the VM and check /proc/mdstat: ok, boots degraded and mdstat shows the disk 4. shutdown VM. reconnect 2nd disk and remove 1st disk, power on the VM and check /proc/mdstat: ok, boots degraded and mdstat shows the disk 5. shutdown VM. reconnect 1st disk (so now both disks are connected, but out of sync), power on the VM Expected results: At this point it should boot degraded with /proc/mdstat showing it is syncing (recovering). This is how it works with ext4. Note that in the past one would have to 'sudo mdadm -a /dev/md0 /dev/MISSING-DEVICE' before syncing would occur. This no longer seems to be required. Actual results: Array comes up with both disks in the array and in sync. Sometimes there are error messages saying there are disk errors, and the boot continues to login, but root is mounted readonly and /proc/mdstat shows we are in sync. Sometimes fsck notices this and complains a *lot*: /dev/md0 contains a filesystem with errors Duplicate or bad block in use Multiply-claimed block(s) in inode... ... /dev/md0: File /var/log/boot.log (inode #68710, mod time Wed Apr 7 11:35:59 2010) has multiply-claimed block(s), shared with 1 file(s):  /dev/md0: /var/log/udev (inode #69925, mod time Wed Apr 7 11:35:59 2010) /dev/md0: /dev/mdo0: UNEXPECTED CONSISTENCY; RUN fsck MANUALLY. The boot loops infinitely on this because the mountall reports that fsck terminated with status 4, then reports that '/' is a filesystem with errors, then tries again (and again, and again). See: http://iso.qa.ubuntu.com/qatracker/result/3918/286 I filed this against 'linux'; please adjust as necessary. ----- Fixing: * When assembling, mdadm could check for conflicting "failed" states in the   superblocks of members to detect conflicting changes. On conflicts, i.e. if an   additional member claims an allready running member has failed:    + that member should not be added to the array    + report (console and --monitor event) that an alternative      version with conflicting changes has been detected "mdadm: not      re-adding /dev/≤member> to /dev/≤array> because constitutes an      alternative version containing conflicting changes"    + require and support --force with --add for manual re-syncing of      alternative versions (because unlike with re-syncing outdated      devices/versions, in this case changes will get lost). Enhancement 1)   To facilitate easy inspection of alternative versions (i.e. for safe and   easy diffing, merging, etc.) --incremental could assemble array   components that contain alternative versions into temporary   auxiliary devices.   (would require temporarily mangling the fs UUID to ensure there are no   duplicates in the system) Enhancement 2)   Those that want to be able to disable hot-plugging of   segments with conflicting changes/alternative versions (after an   incidence with multiple versions connected at the same time occured)   will need some additional enhancements:    + A way to mark some raid members (segments) as containing      known alternative versions, and to mark them as such when an      incident occurs in which they come up after another      segment of the array is already running degraded.      (possibly a superblock marking itself as failed)    + An option like      "AUTO -SINGLE_SEGMENTS_WITH_KNOWN_ALTERNATIVE_VERSIONS"      to disable hotplug support for alternative versions once they came      up after some other version and got marked as containig an alternative version. Re-attaching parts of an array that have been running degraded separately and contain conflicting changes of the same amount, or within the range of a write intent bitmap, results in the assembly of a corrupt array. ---- Using the latest beta-2 server ISO and following http://testcases.qa.ubuntu.com/Install/ServerRAID1 Booting out of sync RAID1 array fails with ext3: It comes up as synced, but is corrupted.      (According to comment #18: ext3 vs ext4 seems to be mere happenstance.) Steps to reproduce: 1. in a kvm virtual machine, using 2 virtio qcow2 disks each 1768M in size, 768M ram and 2 VCPUs, in the installer I create the md devices: /dev/md0: 1.5G, ext3, / /dev/md1: ~350M, swap Choose to boot in degraded mode. All other installer options are defaults 2. reboot into Lucid install and check /proc/mdstat: ok, both disks show up and are in sync 3. shutdown VM. remove 2nd disk, power on the VM and check /proc/mdstat: ok, boots degraded and mdstat shows the disk 4. shutdown VM. reconnect 2nd disk and remove 1st disk, power on the VM and check /proc/mdstat: ok, boots degraded and mdstat shows the disk 5. shutdown VM. reconnect 1st disk (so now both disks are connected, but out of sync), power on the VM Expected results: At this point it should boot degraded with /proc/mdstat showing it is syncing (recovering). This is how it works with ext4. Note that in the past one would have to 'sudo mdadm -a /dev/md0 /dev/MISSING-DEVICE' before syncing would occur. This no longer seems to be required. Actual results: Array comes up with both disks in the array and in sync. Sometimes there are error messages saying there are disk errors, and the boot continues to login, but root is mounted readonly and /proc/mdstat shows we are in sync. Sometimes fsck notices this and complains a *lot*: /dev/md0 contains a filesystem with errors Duplicate or bad block in use Multiply-claimed block(s) in inode... ... /dev/md0: File /var/log/boot.log (inode #68710, mod time Wed Apr 7 11:35:59 2010) has multiply-claimed block(s), shared with 1 file(s):  /dev/md0: /var/log/udev (inode #69925, mod time Wed Apr 7 11:35:59 2010) /dev/md0: /dev/mdo0: UNEXPECTED CONSISTENCY; RUN fsck MANUALLY. The boot loops infinitely on this because the mountall reports that fsck terminated with status 4, then reports that '/' is a filesystem with errors, then tries again (and again, and again). See: http://iso.qa.ubuntu.com/qatracker/result/3918/286 I filed this against 'linux'; please adjust as necessary. ----- From linux-raid list: mdadm --incremental should only included both disks in the array if 1/ their event counts are the same, or +/- 1, or 2/ there is a write-intent bitmap and the older event count is within the range recorded in the write-intent bitmap. Fixing: * When assembling, mdadm could check for conflicting "failed" states in the   superblocks of members to detect conflicting changes. On conflicts, i.e. if an   additional member claims an already running member has failed:    + that member should not be added to the array    + report (console and --monitor event) that an alternative      version with conflicting changes has been detected "mdadm: not      re-adding /dev/≤member> to /dev/≤array> because constitutes an      alternative version containing conflicting changes"    + require and support --force with --add for manual re-syncing of      alternative versions (because unlike with re-syncing outdated      devices/versions, in this case changes will get lost). Enhancement 1)   To facilitate easy inspection of alternative versions (i.e. for safe and   easy diffing, merging, etc.) --incremental could assemble array   components that contain alternative versions into temporary   auxiliary devices.   (would require temporarily mangling the fs UUID to ensure there are no   duplicates in the system) Enhancement 2)   Those that want to be able to disable hot-plugging of   segments with conflicting changes/alternative versions (after an   incidence with multiple versions connected at the same time occured)   will need some additional enhancements:    + A way to mark some raid members (segments) as containing      known alternative versions, and to mark them as such when an      incident occurs in which they come up after another      segment of the array is already running degraded.      (possibly a superblock marking itself as failed)    + An option like      "AUTO -SINGLE_SEGMENTS_WITH_KNOWN_ALTERNATIVE_VERSIONS"      to disable hotplug support for alternative versions once they came      up after some other version and got marked as containig an alternative version.
2010-04-28 18:11:44 Steve Langasek ubuntu-release-notes: status New Fix Released
2010-09-28 20:53:15 Oli Wade bug added subscriber Oli Wade
2010-10-10 18:45:18 totya bug added subscriber totya
2010-10-11 08:28:04 Laurent Bonnaud bug added subscriber Laurent Bonnaud
2010-10-14 01:26:51 Simon Déziel bug added subscriber Simon Déziel
2010-10-20 23:07:47 Clint Byrum bug added subscriber Clint Byrum
2010-10-30 16:47:12 Colan Schwartz bug added subscriber Colan Schwartz
2011-02-21 05:03:46 Nataraj bug added subscriber Nataraj
2011-07-26 11:26:49 Marc D. removed subscriber Marc A. Donges
2012-02-10 10:05:16 Sergei Ianovich bug added subscriber Sergey Yanovich
2012-04-09 09:34:56 Scott Merrilees bug added subscriber Scott Merrilees
2013-07-06 11:32:03 Adolfo Jayme Barrientos bug task deleted mdadm (Ubuntu Lucid)
2016-01-05 19:34:23 Till Ulen removed subscriber Alexander Konovalenko
2019-04-09 18:38:22 Laurent Bonnaud removed subscriber Laurent Bonnaud