fusion mpt sas driver does not find a RAID1 disk during installation(Sun Galaxy X4200 and X4100, Dell SASR5/i)

I can confirm this bug. It also happens on my X4200 using Hardware RAID1. Now I've installed Dapper using software RAID1 and it works as expected.

Joan

Not sure what part of the system needs to support hardware level raid. It should not make a difference to the kernel.

Please attach dmesg for when this fails (entire dmesg, not just the scsi portions).

This week I have a spare X4200. I'll send full dmesg output. Do you need more tests? I'll try latest dapper install CD.

Joan

dmesg of a X4200 booting with Hardware RAID 1

lsmod of dapper install CD (030506 daily snapshot) on a X4200.

lspci of a Sun X4200 with Hardware RAID 1

/var/log/partman of Dapper install CD (040506 ds) on a X4200 with Hw RAID1

/var/log/syslog of Dapper install CD (030506 ds) on a X4200 with Hw RAID1

Attached some files. All captured from an install attempt on a Sun X4200 with Hw RAID1 configured (4 disks 2 Arrays of 2 disks in RAID 1). If more files/test are needed I can obtain/perfom them. Used Dapper install CD 030506 Daily Snapshot.

Upgrading kernel to 2.6.16.16 (possibly 2.6.16.x) solves this issue.

This is an issue with the LSI kernel drivers before 2.6.16. This issue exists due to the fact that the driver does not recognize the SAS bus with certain RAID sets. Check this post to see the details on the fixes: http://lwn.net/Articles/172503/ Search for fusion. The key part in the link is the section stating: [SCSI] fusion - mptctl - adding support for bus_type=SAS. There is a patch available to older kernels. This link explores the problems in detail and explains methods for obtaining the kernel patches: http://forums.gentoo.org/viewtopic-t-415813.html I have the patch if anyone needs it.

A patch for dapper would be nice.

Hi,

as this bug is currently in status "Needs Info" I would like to know which infos are manditory to support this kind of hardware raid?
It is enough to take the changes made on the fusion driver between 2.6.15.x and 2.6.16.0

Any more infos needed?

Ben: This should be the neccessary changes:
2006-02-16 Linus Torvalds Merge /linux/kernel/git/jejb/scsi-rc-fixes-2.6
http://www.kernel.org/git/?p=linux/kernel/git/stable/linux-2.6.16.y.git;a=shortlog;h=659eadfc20751c1e0784f4910d194fcd836b86db;pg=8
http://www.kernel.org/git/?p=linux/kernel/git/stable/linux-2.6.16.y.git;a=commitdiff;h=26d451b603e754ded83f0e5becab2a78253ad100

actually, it's probably enough to get
c7c82987b4844f555d309ccbd42abe95d46822ff
and
816aa907b909177bdf6e6e6b0d00c5e5a6e2be8c
from linus' tree. However, the former of those requires an ABI bump.
I'm testing a modified tree with these two patches cherry-picked and backported; will post results ASAP.

Is there any chance of this going into dapper? We would really like to deploy Ubuntu on our Sun servers here so we can use it instead of Solaris on our SunRay network.

Waiting for someone to tell me if the patches work.

Although I think the problem exists between keyboard and chair, I am not able to compile a new kernel with the changes posted by thombot.

I used linux-source-2.6.17 (2.6.15-26.46) and applied the following two patches (with errors)
http://www.kernel.org/git/?p=linux/kernel/git/stable/linux-2.6.17.y.git;a=commitdiff_plain;h=c7c82987b4844f555d309ccbd42abe95d46822ff;hp=f2ea8671a8376e09cf759aa8cb3de8b8d3bced9e
http://www.kernel.org/git/?p=linux/kernel/git/stable/linux-2.6.17.y.git;a=commitdiff_plain;h=f61c1c41b6b959d08d87eee876406bdbcac77b7c;hp=c7c82987b4844f555d309ccbd42abe95d46822ff

while running make-kpkg I get the following error:

make[3]: *** [drivers] Error 2
make[3]: *** Waiting for unfinished jobs....
  CC [M] fs/xfs/linux-2.6/xfs_vnode.o
  CC [M] fs/xfs/support/debug.o
  CC [M] fs/xfs/support/uuid.o
  CC [M] fs/xfs/support/move.o
  CC [M] fs/xfs/linux-2.6/xfs_vfs.o
  LD [M] fs/xfs/xfs.o
make[3]: Leaving directory `/usr/src/linux-source-2.6.15-2.6.15/debian/build/build-386'
make[2]: *** [stamp-build] Error 2
make[2]: Leaving directory `/usr/src/linux-source-2.6.15-2.6.15/debian/build/build-386'
make[1]: *** [build] Error 2
make[1]: Leaving directory `/usr/src/linux-source-2.6.15-2.6.15'
make: *** [stamp-buildpackage] Error 2

real 6m17.417s
user 20m44.020s
sys 2m27.760s

Maybe someone could help me building this kernel

As a workaround you can check out my little howto
http://jesusch.de/?page=ubuntu

I've been hacking my way around this problem much the same way jessy did ever since Dapper was released. It's not like lack of support for newer hardware isn't a common problem with most Linux distributions, so I've accepted that I had to do so. However, today I was made aware of the page at http://www.ubuntu.com/sun/. Select quotes:

«The combination of Ubuntu and Sun Microsystems hardware offers a knock-out combination.»
«The following servers and workstations are certified to run Ubuntu Linux [...list including X4100/X4200...]»
«[...] the Ubuntu release will make it easy for customers to install Linux on its UltraSPARC T1 and x64-based systems [...]»

I was very happy, believing this meant I didn't have go through that pain-in-the-arse process it has been install these boxes anymore (which I do quite often, being a Sun reseller). So I downloaded the latest amd64 netboot.tar.gz from dapper-updates, which was surprisingly enough just as unable to install my X4100 as earlier versions.

You might want to tune down the hyperbole a bit until it actually works as stated.

Tore

Hey Tore, did you read my mind this morning? Or did you
read UWN Issue 22 too ;)

I holeheartly agree with Tore. I2 weeks ago I've tried to
update netboot.tgz and Install new Galaxy server and was a disappointed too.

Nevertheless AFAIR Ben was only waiting on a confirmation
that the proposed fix really works. AFAIU you can confirm
it for several installation ;)

Ben, what more is needed to get this into the next kernel update
for dapper? (And downloadable netboot!)

Achim

* allee

> Hey Tore, did you read my mind this morning? Or did you
> read UWN Issue 22 too ;)

  Neither, just stumbled over the page, and immediately got great
 expectations...

> Nevertheless AFAIR Ben was only waiting on a confirmation
> that the proposed fix really works. AFAIU you can confirm
> it for several installation ;)

  As noted earlier the proposed diffs doesn't apply cleanly, and I'm not
 really much of a kernel developer so attempting any manual merges is
 bound to end in disaster. I don't know if Thom May had any success
 with his testing or not.

  This tree might be of interest, though:
 <http://www.kernel.org/git/?p=linux/kernel/git/jejb/scsi-fusion-raid-2.6.git;a=summary>

--
Tore Anderson

The simplest workaround that comes to mind, would be installing kernel image from Edgy. It works for me, but it is not clean (and might cause security upgrade problems in the production environment). Besides Dapper is supposed to be certified to work with Sun machines. One starts to wonder about quality of the certification process...

We seem to be seeing the same failure on a Dell Poweredge 860 with their SAS 5/IR RAID card
which I think is just a rebadged LSI.

(There really needs to be a defined way of stuffing update modules into the existing install CD; for example, hypothetically, imagine that one of these machines was 4000 miles away on the other end of a IP-KVM and you'd asked someone over there to put a dapper server CD in and you want to do the rest. Hypothetically speaking it would be a pain if there was no neat way to fix the fact that the kernel on the install CD couldn't cope; for a LTS solution this is important).

Dave

Non-hypothetically, we already have a spec for this called ubiquity-drivers, to let users insert a CD or floppy with extra drivers on it. Will be implemented in feisty.

I've put a backport of the fixed mptfusion version (v3.03.05) into ubuntu-dapper-updates.git which will make its way into dapper-proposed. Please test when it becomes available, sorry for the delay. -- Kyle

Hi,

I'm really looking forward (not to say I'm horny) to test your changes.
Unfortunatelly I don't have any experience using git - will there also
be some binary linux-image in a deb format?
Other point we have to keep in mind is how your changes can be
integrated into the dapper installer. Will there be an update towards
6.06.2?

Cheers
Bjoern

Kyle McMartin wrote:
> I've put a backport of the fixed mptfusion version (v3.03.05) into
> ubuntu-dapper-updates.git which will make its way into dapper-proposed.
> Please test when it becomes available, sorry for the delay. -- Kyle
>
> ** Changed in: linux-source-2.6.15 (Ubuntu)
> Status: Confirmed => Fix Committed
>
>

I finally got a Dell Poweredge 860 with SAS 5/IR running by backporting the mpt driver.

Those were the necessary commits from Linus' tree :

a9b2937a1eab2939d0eed3830ead88664ed7445d
f2ea8671a8376e09cf759aa8cb3de8b8d3bced9e
c7c82987b4844f555d309ccbd42abe95d46822ff
816aa907b909177bdf6e6e6b0d00c5e5a6e2be8c (slightly modified to get rid of 2.6.16's mutexes)

Kyle,

I can't find your updated driver in any public git repository. Should it
appear in
http://www.kernel.org/git/linux/kernel/git/kyle/ubuntu-dapper-updates.git or

http://www.kernel.org/git/linux/kernel/git/bcollins/ubuntu-dapper-updates.git?

Regards,
Cédric

Hi Cédric,

unfortunatelly Kyle has forgotten to push his changes to the repo :)
They made their way into http://www.kernel.org/git/linux/kernel/git/kyle/ubuntu-dapper-updates.git

After all I'm not able to compile that tree standalone using the following command inside that tree because of several missing items inside the debian/ directory.

`AUTOBUILD=1 fakeroot debian/rules binary-debs flavours=server`

Bjoern

I have successfully built the tree from http://www.kernel.org/pub/scm/linux/kernel/git/kyle/ubuntu-dapper-updates.git but it is lacking the last commit (816aa907b909177bdf6e6e6b0d00c5e5a6e2be8c) which introduce raid support in mptsas.

Applying my backported patch (http://librarian.launchpad.net/5702451/linux-2.6.git-816aa907b909177bdf6e6e6b0d00c5e5a6e2be8c-backported.patch) on top of it does the trick.

Hi Cedric,

I have been able to compile Kyle's tree including just your patch.
Unfortunatelly the X4100 still does not run using that kernel && HW-RAID :/

Any idea when 'fix commited' will change to 'fix released' ?

And does it imply that the netboot tar files are updated
too for dapper?

I have a X4200 and just tried the linux-source-2.6.15 2.6.15-50.61 with mptfusion: Backport v3.03.05, but I was not able to boot from my RAID1 disk. My Ubuntu 6.06.1TLS boot and works perfectly with the Edgy kernel...
Any ideas?
Thanks

Laurent

Ok, thanks for testing! I'll try another fix and see if I can get something uploaded by the weekend.

Cheers,
 Kyle

Remarking as In Progress. Committed fix did not help. Stay tuned for a forthcoming upload which will hopefully work better.

Pushed into kernel-team repo.

-EWRONGBUG, sorry.

Hi,

Is there any ongoing progress to overcome this bug?

The same experience happening to me, with
Dell PowerEdge SC1435 with SAS5iR
Raid Controller.

-Rully

Hi Rully,

Kyle already applied some patches but unfortunatelly those did not fix
the RAID issues :/

Bjoern

Hi Bjoern,

Thanks for the information.
It seems that we have to create our kernel ourself.
As in 2.6.20, there is no problem with RAID1.

Anyway, thanks

-Rully

On 3/5/07, jessy <email address hidden> wrote:
> Hi Rully,
>
> Kyle already applied some patches but unfortunatelly those did not fix
> the RAID issues :/
>
> Bjoern
>
> --
> fusion mpt sas driver does not find a RAID1 disk during installation(Sun Galaxy X4200 and X4100, Dell SASR5/i)
> https://launchpad.net/bugs/37452
>

Hi Rully,

you already get full RAID support starting from 2.6.16-RC4

Bjoern

rully schrieb:
> Hi Bjoern,
>
> Thanks for the information.
> It seems that we have to create our kernel ourself.
> As in 2.6.20, there is no problem with RAID1.
>
> Anyway, thanks
>
> -Rully
>
> On 3/5/07, jessy <email address hidden> wrote:
>> Hi Rully,
>>
>> Kyle already applied some patches but unfortunatelly those did not fix
>> the RAID issues :/
>>
>> Bjoern
>>
>> --
>> fusion mpt sas driver does not find a RAID1 disk during installation(Sun Galaxy X4200 and X4100, Dell SASR5/i)
>> https://launchpad.net/bugs/37452
>>
>

Any updates on this? Or any sort of work around? I am interested in anything that will allow me to install LTS on a Poweredge 860

Hi Zachary,

As mentioned earlier in this thread, I could successfully install LTS on a Poweredge 860 by applying a bunch of patches backported from 2.6.16 (see earlier my posts for the list)

Here is the path I've followed :
- patch and build the sources corresponding to the installer's kernel (2.6.15-26-386)
- patch and build the sources of latest LTS linux-server (2.6.15-61-server)
- launch the installer
- when the network is up and running, upload (via ssh) the mpt modules from the 2.6.15-26-386 patched kernel into the installer's filesystem
- depmod, modprobe
- finish the install
- before the reboot, upload and install (in the chroot) the 2.6.15-61-server linux-image package

I've setup a repository containing the latest packages with the patches applied (2.6.15-61sdrik) :
deb http://cedric.gabriello.fr/ubuntu dapper misc

Cédric

Hi Cedric,

do you know if your patches already made their way into
dapper-proposed-updates?

Cedric Schieli wrote:
>
> As mentioned earlier in this thread, I could successfully install LTS on a Poweredge 860 by applying a bunch of patches backported from 2.6.16 (see earlier my posts for the list)
>
>

Hi Jessy,

Yes, all of the patches, except the one attached to this bug report (
http://librarian.launchpad.net/5702451/linux-2.6.git-
816aa907b909177bdf6e6e6b0d00c5e5a6e2be8c-backported.patch), have been
applied to the proposed-updates branch.

Regards,
Cedric

2007/4/19, jessy <email address hidden>:
>
> Hi Cedric,
>
> do you know if your patches already made their way into
> dapper-proposed-updates?
>
> Cedric Schieli wrote:
> >
> > As mentioned earlier in this thread, I could successfully install LTS
> on a Poweredge 860 by applying a bunch of patches backported from 2.6.16(see earlier my posts for the list)
> >
> >
>
> --
> fusion mpt sas driver does not find a RAID1 disk during installation(Sun
> Galaxy X4200 and X4100, Dell SASR5/i)
> https://bugs.launchpad.net/bugs/37452
> You received this bug notification because you are a direct subscriber
> of the bug.
>

Can you please test the kernel found here:
http://people.ubuntu.com/~kyle/support/mptsas/linux-image-2.6.15-28-server_2.6.15-28.57_i386.deb

(other kernel flavours available in the same directory)

Thanks! Kyle.

Hi,

I reinstalled a SUN X4100 using 6.06.1 without RAID enabled.
Afterwards I installed your kernel http://people.ubuntu.com/~kyle/support/mptsas/linux-image-2.6.15-28-server_2.6.15-28.56_i386.deb which is working without RAID (mirror) enabled.

As soon as I enable HW RAID the system loads the kernel, gets stuck for several minutes and then stating that ther is no /dev/sda1 and falling back into busybox

You can find a screenshot from console output here:
http://jesusch.de/~jesusch/img/ubuntu-%2337452.png

Bjoern

Thx Kyle! Can you provide an updated netboot.tar.gz too? Or a pointer how to do it? The x4100s have only two disk slots, so I can't configure raid and additionally install kubuntu on a non-raid disk as non is left :(

There's no amd64-k8 flavour but it should not be too complicate to tweak the
netboot environment to use k7 instead

Hi Bjoern,
how did you do enable HW RAID? Can one migrate a 'simple' disk to a RAID1 mirror?
Achim

yes you can.
first install ubuntu on sda
remeber to keep some megs free at the end of the disk - they will be
needed for setting up the RAID later on
after you have finished you need to install a kernel image that supports
HW RAID for the LSI (starting from 2.6.16, e.g. stock kernel from edgy)
Unfortunatelly that's the point where Kyle's image did not solve the
missing support for my box.

After this you can go to the LSI tool, create a mirror RAID and the tool
will ask for the first disk if you want to keep the data
Add the second disk and voila.

Bjoern

allee schrieb:
> Hi Bjoern,
> how did you do enable HW RAID? Can one migrate a 'simple' disk to a RAID1 mirror?
> Achim
>

Kyle: Tested on the .57 on both a Sun X4100 (SAS1064) and a Dell Precision workstation (SAS1068), and .56 on the Dell Precision, and either do not see the hardware RAID 1 volume.

Jessy: when you say "some megs free", do you mean to leave unallocated space at the end of the drive ? When I use a single partition for the entire, it refuse to create the RAID 1 volume non-destructively, claiming lack of free space.

Kyle: Tested on the .57 on both a Sun X4100 (SAS1064) and a Dell Precision workstation (SAS1068), and .56 on the Dell Precision, and either do not see the hardware RAID 1 volume.

Jessy: when you say "some megs free", do you mean to leave unallocated space at the end of the drive ? When I use a single partition for the entire disk, it refuse to create the RAID 1 volume non-destructively, claiming lack of free space, even though I have plenty of free space in the file system.

Hi Etienne,

unfortunatelly I do not know how much space needs to be kept free at the
end of the drive - I always assumed 100megs which worked for me (36G drives)
free space on the filesystem does not do the job.

Bjoern

Etienne Goyer schrieb:
> Jessy: when you say "some megs free", do you mean to leave unallocated
> space at the end of the drive ? When I use a single partition for the
> entire disk, it refuse to create the RAID 1 volume non-destructively,
> claiming lack of free space, even though I have plenty of free space in
> the file system.
>

* Etienne Goyer

> Jessy: when you say "some megs free", do you mean to leave
> unallocated space at the end of the drive ? When I use a single
> partition for the entire disk, it refuse to create the RAID 1 volume
> non-destructively, claiming lack of free space, even though I have
> plenty of free space in the file system.

   It needs to be unpartitioned, and also it will complain if there is
  any partition of the LVM type (0x8e) anywhere on the drive. You can
  easily change that back once the kernel is upgraded and the RAID
  established, though. Then you can also extend the last partition so it
  uses all the free space (delete/recreate - as long as the new partition
  starts on the same cylinder boundary as the old one this is not
  destructive), and finally use pv/vg/lvextend and/or resize2fs to
  reclaim all the space.

--
Tore Anderson

After much hairpulling and reading, here is my workarround. Hope this helps someone

Ubuntu 6.06 install on Dell Poweredge 860 with 2.6.17-10 kernel

There is a problem with the SAS controller configured in RAID 1 with the kernel (2.6.15-26-386) that is used for the installation process. It is unable to see the RAID configured drives and “hangs” when trying to start partman to begin the partitioning process. In addition the kernel which will be installed by the normal installation process is also unable to access to see the drives in this configuration resulting in a indefinite wait after the “booting the kernel” message.
Workaround
• Remove the primary raid hard disk from the server and install it in a non raid machine.
• Start the server installation and install like you normally would (make sure the last partition is swap!)
• Before the installation wants to reboot, go to a console (ctrl + alt +F2 ).
• Copy a working kernel .deb package (http://packages.ubuntu.com/edgy/base/linux-image-2.6.17-10-server) on a usb stick and place it in the chroot environment (/target)
• Chroot to this environment
chroot /target
• and install the package
dpkg -i linux-image-2.6.17-10-server
• Make sure that the network interfaces that are used by the server are in /etc/network/interfaces. If there is only one interface listed while the target machine has two, none will work (maybe only in case they use the same driver)
• Shutdown the temp machine and place the disk back in the server

When the server boots go in to the SAS configuration and select synchronize disks in the RAID properties menu. (This will take quite a while…)

When the server boots, it will probably display an I/O error on sda5 (or the last partition number you use) if you used a normal installation this partition will be the swap partition. I suspect that by re-synchronizing the array some of the last sectors of the drive are no longer available.
The assumption is made that this partition is the swap partition in the following use caution and watch out and be careful and use at your own risk and so on and so on

• Deactivate the swap partition
swapoff /dev/sda? # replace by your partition number
• Use fdisk to delete the swap partition and the extended partition which is created by a standard install
• Recreate the partitions (they will be slightly smaller this time)
• Format the new partition
mkfs -c /dev/sda? # replace by your partition number
Reactivate the swap
mkswap /dev/sda? # replace by your partition number
swapon /dev/sda? # replace by your partition number

Done.

Backported driver for linux-backports-modules-2.6.15

Unfortunatelly this may only work if the root filesystem is on a non-raid device
I installed 2.6.15-29 as well as the backports-modules, but it seems that the backported modules are not taken into the initrd.
So the kernel is still not able to mount the root filesystem :/

Accepted into dapper-proposed:

 linux-backports-modules-2.6.15 (2.6.15-29.1) dapper-proposed; urgency=low
 .
   [Ben Collins]
 .
   * fusion: Added mptsas drivers from feisty tree
     - Bug #37452
   * fusion: Back down to edgy 2.6.17 version, and include
     scsi-transport-sas module
   * e1000: Added v7.6.9.1 module update
   * e1000: Added to build
   * ubuntu: Enable e1000

Please test once it is available, and give feedback here.

Using kernel 2.6.15-51-amd64-generic (from dapper-backports) as installer kernel on a Dell PowerEdge 1950 I get unresolved symbols while loading mptsas.ko and scsi_transport_sas.ko, e.g.

sas_phy_alloc
sas_phy_free
sas_end_device_alloc
...

I tried kernel 2.6.15-51-386 with linux-backports-modules-2.6.15-51-386 on Sun Fire 4100 M2 form dapper-proposed

When booting, I got the following:

ALERT! /dev/sda does not exist. Dropping to a shell
...
BusyBox...
...

# modprobe mptsas
WARNING: Error inserting scsi_transport_sas (/lib/modules/2.6.15-51-386/updates/storage//fusion/scsi_transport_sas.ko): Unknown symbol in module, or unknown parameter (see dmesg)
FATAL: Error inserting mptsas (/lib/modules/2.6.15-51-386/updates/storage//fusion/mptsas.ko): Unknown symbol in module, or unknown parameter (see dmesg)

This seems to be the same problem as in bug 164449. Volker, please upgrade to the latest linux-backports-modules in -proposed, that fixes the symbol problem.

So.... This seems to have been going on for a long time.... I also have some Dell PowerEdge 860s with Dell SASR5/i Raid controllers and I would like to run Dapper on them.... Does anyone have an installation porcess that can work using a normal Dapper server install CD?? Perhaps a brief HowTo would suffice... There are a number of workarounds above, but they all seem a bit daunting... I need a process that can be repeated on multiple servers by normal admins... You know... A boot-this-CD-and-follow-the-prompts sort of install...

Thanks,

Dave

Hi Dave,

unfortunatelly there is still no universal solution for dapper.
I'm not sure about the timeframe for 6.06.2 but in that release there might be a solution as long as the backported modules will go _into_ the installer kernel as well as _into_ the linux-image that is going to be installed.

Backing this out of 6.06.2, since it still does not work. Bjoern Boschmann kindly tested the 6.06.2 CD on an X4100 with this controller:

0000:02:03.0 SCSI storage controller: LSI Logic / Symbios Logic SAS1064 PCI-X Fusion-MPT SAS (rev 02)

I'll attach a screenshot from booting (which doesn't tell much, though).

From my point of view the situation is more critical.
Until now it was at least possible to boot and install an X4100 using Dapper.1 CD IF HW-RAID was disabled.

Using Dapper.2 CD from 10. Jan 2008 this is no longer possible as the machine freezes.

Martin Pitt told me that it was possible using some X4100 in their test-center - maybe that already was some X4100 M2??

Now it's official!

Using 6.06.2 installer it's no longer possible to install a SUN X4100 at all.

Considering that we bought 4 SUN servers x4100 a year ago because they were certified for Dapper, and considering that i need to install Dapper to get Zimbra support, I feel quite concerned to learn here that the hardwaire raid is not supported. On top of this it seems there is an issue in Dapper installing the root on a LVN partition which is over two soft raid. In the end I will have to rely over two soft raid 1, which is a pain as I would need one bigger partition.

Very sad this bug got forgotten, it seems quite critical to me :/

Achim,
     Please test this versus the latest Alpha of Lucid. I need to determine if this was resolved upstream. Should you encounter this in the latest build of Lucid, please run 'apport-collect -p linux 37452' so that logging can be attached to this report.

Thanks!

~JFo

marking invalid due to length of time since last update.