Ubiquity Installer doesn't recognize existing btrfs partitions

Here are the syslog and the partman log as requested

Here is proof that the btrfs volume can be mounted and contains data. Data that would be erased by the Ubuntu installer due to this bug.

btrfs is not intended to be used this way any more than any other filesystem, which is to say not at all. If you intend to boot from that drive it *must* have a partition table to boot. If you insist on using a data disk this way ( and there really is no reason to -- are you really going to miss that 1 mb of space the partition table takes up? ), I believe this problem was due to some bugs in parted that were fixed in 14.10. Can you try that and confirm whether it has fixed this?

I'll test in 14.10 and get back to you.

@ Phillip
As for your legacy comment.

"If you intend to boot from that drive it *must* have a partition table to boot."

I have been booting my BTRFS systems just fine on pure raw btrfs for two years now on my laptop and desktop.

As our good friends over at ARCH have also proven
https://bbs.archlinux.org/viewtopic.php?id=166591
https://wiki.archlinux.org/index.php/Partitioning#Btrfs_Partitioning

BTRFS reserves the first 64Kib in order to allow booting. GRUB2 already supports booting from BTRFS. As BTRFS is its own partition table, LVM and filesystem in one.

Partition tables are legacy from the 50's!! it's 2014! Technology has gotten past partition tables in both ZFS and BTRFS.
It's not about the 1mb that a partition table occupies but removing complexity and overhead (layers of the onionskin). This leads to a speed increase in btrfs especially on SSD's. Plus it's easier to manage storage when you have One common set of commands instead of several unrelated programs each with their own syntax. So in the end it's also faster to set up your system and manage it running raw btrfs.

Swap and that terrible EFI folder can be held on a SD card to counter any argument before it starts.

So please come to the future, where things are just a little less complex.

Yes it still happens in 14.10 please see screenshots with proof.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 10/28/2014 09:39 PM, Damiön la Bagh wrote:
> BTRFS reserves the first 64Kib in order to allow booting. GRUB2
> already supports booting from BTRFS. As BTRFS is its own partition
> table, LVM and filesystem in one.

Last I saw, when adding the btrfs module to the grub core, it grew
larger than 64k and so wouldn't fit in that reserved space, but if you
say it is working that's neat.

> Partition tables are legacy from the 50's!! it's 2014! Technology
> has gotten past partition tables in both ZFS and BTRFS.

ZFS and BTRFS have nothing to do with it. If you want to, you can
format an entire disk as ext4 or any other fs too; it just isn't smart.
With btrfs on the disk without a partition table, it becomes impossible
to share that disk with an OS that does not understand btrfs, even by
using eg. gparted to repartition the drive and add a small shared
partition. We also have a new partition table format in the last few
years that newer computers require to boot since the bios is finally
going away. Even if you can manage to get a bios based computer to boot
directly from a full btrfs volume ( and many bios implementations won't
boot the disk unless they recognize a valid dos partition table with an
active partition ), this won't fly with UEFI.

> It's not about the 1mb that a partition table occupies but
> removing complexity and overhead (layers of the onionskin). This
> leads to a speed increase in btrfs especially on SSD's. Plus it's
> easier to manage storage when you have One common set of commands
> instead of several unrelated programs each with their own syntax.
> So in the end it's also faster to set up your system and manage it
> running raw btrfs.

There is no overhead to placing a filesystem on a partition instead of
the raw device. Formatting the raw device does not make it one iota
faster.

> Swap and that terrible EFI folder can be held on a SD card to
> counter any argument before it starts.

Assuming the computer has one and that it's UEFI firmware recognizes it.

Some other scenarios that come to mind where you will regret not having
a partition table are if you ever add a second disk and want to expand
btrfs to span or raid across them both. I'm pretty sure the grub btrfs
code does not handle this so you would need to add a /boot partition.
Another is that if that disk is ever plugged into a Windows system, it
will helpfully offer to format the "blank" disk instead of recognizing
that it is occupied by another OS. A partition table is pretty cheap
insurance against this.

> Yes it still happens in 14.10 please see screenshots with proof.

Ok, can you post the output of parted -l? Does it recognize that it
is btrfs?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (MingW32)

iQEcBAEBAgAGBQJUUUZvAAoJEI5FoCIzSKrw0y0H/0arcwnwwvkLSu5HICaC+Puc
Got6A4TNDtgCdCALXK9cnfYc36IETu8JyUVJ3OEQFrQt2ZCx2zwJY19mnkl+RIFG
wD/eUQS+MBWRRL6xyUAUCOdadSS1xNZUCWHMO9mZi7E9l6bluZBjmTvl52IEHZDm
9GtWZnRUuN8TloszP9Q6/y+DMLJr5PeVAsiqC5s4TItze5f4GZSeJy/TFlrgzY6j
5VoEaC7eG2O2ooIZgvXGIPtPD3yGQhWRYEuSlkLe3ZOq3o5tRG7/OG5V6ApZXXu4
Iv/69TKsCFWbi0Malmu9MyoIcHZ/9Cb7TfR251PqOCY1ZdL07GR2Hbn5GPLBg7A=
=D8Yv
----...

>>if you ever add a second disk and want to expand
>>btrfs to span or raid across them both.

BTRFS is really easy to add or subtract a disk. Grub2 has no problem with it and I can still boot after the disk has been added and balanced.
https://btrfs.wiki.kernel.org/index.php/Using_Btrfs_with_Multiple_Devices
it's as easy as
btrfs device add /dev/sdc /mnt
btrfs filesystem balance /mnt

As for other operating systems. This is a moot point, the scenario is using Ubuntu+Btrfs with physical SSD's. If the disks need to be converted back to legacy partition table format it's as simple as writing a few 00's to the first 64kib with dd.
dd if=/dev/zero of=/dev/sdX count=64

For UEFI, I've already complained to the UEFI bord that they are holding back technology by insisting on using a 1980's file system to hold UEFI. It's the most ridiculous 'invention' in computers in the last 10 years. I understand the want and need to kill the BIOS but there are much more graceful solutions then defaulting to 30+ year old legacy tech.

As for /boot it's part of my normal file system and the 1st 64Kib points to /boot on boot for the rest of the boot information.
ubuntu@ubuntu:~$ sudo btrfs device scan
Scanning for Btrfs filesystems
ubuntu@ubuntu:~$ sudo mount /dev/sda /mnt
ubuntu@ubuntu:~$ ls /mnt/@/boot
abi-3.13.0-24-generic initrd.img-3.13.0-39-generic
abi-3.13.0-35-generic memtest86+.bin
...
initrd.img-3.13.0-36-generic vmlinuz-3.13.0-39-generic
initrd.img-3.13.0-37-generic
(truncated for brevity)

Parted is broken when it comes to btrfs, it doesn't handle it gracefully at all.
See the screenshot:
The top left corner shows that parted just defaults to MSDOS even though you can clearly see in the top right that there is no partition table written.
The bottom terminal is the Ubuntu 14.10 Live USB stick for comparison.
Commands used were
sudo parted -l

and

sudo fdisk /dev/sdX
x
d

where X represents ' a ' and ' c ' respectively.

Should I bring this up with the parted developers as well?

It looks like someone at least attempted to add better btrfs suport
http://lists.gnu.org/archive/html/bug-parted/2013-09/msg00018.html

...
+/* Located 64k inside the partition (start of the first btrfs superblock) */
+#define BTRFS_MAGIC 0x4D5F53665248425FULL /* ascii _BHRfS_M, no null */
+#define BTRFS_CSUM_SIZE 32
+#define BTRFS_FSID_SIZE 16
...

On 10/29/2014 07:04 PM, Damiön la Bagh wrote:
> BTRFS is really easy to add or subtract a disk. Grub2 has no
> problem with it and I can still boot after the disk has been added
> and balanced.

Wow... last I saw they had trouble fitting code to do a simple single
disk case into 64k and were asking about increasing that size. To
handle multiple disks and still fit in 64k is nothing short of miraculous.

> As for other operating systems. This is a moot point, the scenario
> is using Ubuntu+Btrfs with physical SSD's. If the disks need to be
> converted back to legacy partition table format it's as simple as
> writing a few 00's to the first 64kib with dd. dd if=/dev/zero
> of=/dev/sdX count=64

I meant sharing part of the drive, rather than blowing it all away.
With a partition table you can always fire up gparted and shrink down
the btrfs partition and make a new one if you need it. I even patched
things to do this while the volume is mounted a while back ;)

> For UEFI, I've already complained to the UEFI bord that they are
> holding back technology by insisting on using a 1980's file system
> to hold UEFI. It's the most ridiculous 'invention' in computers in
> the last 10 years. I understand the want and need to kill the BIOS
> but there are much more graceful solutions then defaulting to 30+
> year old legacy tech.

It's about as simple as filesystems get, everyone already supports it
and it doesn't really have any drawbacks that matter in a pre boot
environment, so it was a no brainer rather than inventing a whole new
filesystem or mandating that different parties adopt an existing
filesystem of another. Implementations are still free to support other
filesystems if they choose; it's just that fat was the lowest common
denominator and bios vendors never do any more than they absolutely have
to. I've heard that macs can use hfs for the esp though.

> Parted is broken when it comes to btrfs, it doesn't handle it
> gracefully at all. See the screenshot: The top left corner shows
> that parted just defaults to MSDOS even though you can clearly see
> in the top right that there is no partition table written. The
> bottom terminal is the Ubuntu 14.10 Live USB stick for comparison.
> Commands used were sudo parted -l

Looking at your screen shots, it appears that there is something that at
least tries to look like it is an MBR there in the boot sector. In
14.10, I fixed parted to correctly recognize filesystems on bare
devices, but had to take special steps to avoid confusing an ntfs/fat
boot sector for an MBR since they look very similar. My guess is that
grub is trying to mimic an MBR and fooling parted into thinking that it
is one. I probably just need to detect btrfs first and prefer that over
MBR like I did with fat and ntfs.

If you feel comfortable reinstalling grub ( and it sounds like you do ),
then you might conduct an experiment and zero out that first sector and
see if that fixes the problem, though obviously to boot you will then
need to reinstall grub.

> Should I bring this up with the parted developers as well?

You're talking to him ;)

The 1st 15 mintues of this presentation by the SUN engineers who wrote ZFS explains the design philosophy behind partitionless systems (BTRFS/ZFS).
If you can find 15 minutes I highly recommend watching.

"ZFS The Last Word in File Systems"
http://youtu.be/NRoUC9P1PmA

(other filesystems)
The whole point when choosing BTRFS on a setup is to not use any other file systems.
BTRFS/ZFS were designed to replace all other partitions+filesystems+LVM+RAID schemes (see the video)

(coding in small spaces)
The first 64Kib only needs to point to /@/boot how much code do you really need for a pointer?
note: @ refers to the root subvolume that Ubuntu creates automatically on an btrfs install.

(UEFI FAT)
They didn't need to change MBR they just needed to change it to a pointer to a place in an existing filesystem (like the way btrfs works)

I checked both my desktop and laptop BTRFS boot drives and they both show 0x000: EB 63 90 in the very first sector.
So I am guessing the EB 63 90 represents BTRFS's boot pointer area.

(wipe btrfs boot sector)
I'll try your experiment in a VM tomorrow (am on nightshift now and want to have my mind clear and focused when I'm wiping sectors)
This did spark me to do some more research. Apparently there is a utility to remove ZFS/BTRFS from a physical disk that is prefered over dd and doesn't have to destroy all of your data. wipefs. https://btrfs.wiki.kernel.org/index.php/Problem_FAQ#How_to_clean_up_old_superblock_.3F

(parted developer)
Thank you from the bottom of my heart! Your software has helped me and my colleagues in numerous ways over the years.
Thank you for making something so robust and easy to use. I've made donations to the gParted project in the past, I'd like to make another donation, is there a page for parted separately or are gParted and parted developed together?

(parted feature req)
where is the best place to add feature requests for parted? I've been secretly hoping that parted would one day shdisplay LVM volume groups and logical volumes, and would display BTRFS/ZFS subvolumes in a similar fashion to how MSDOS extended partitions are currently shown.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 10/31/2014 12:00 AM, Damiön la Bagh wrote:
> The 1st 15 mintues of this presentation by the SUN engineers who
> wrote ZFS explains the design philosophy behind partitionless
> systems (BTRFS/ZFS). If you can find 15 minutes I highly recommend
> watching.

Again, there is nothing inherently "partitionless" about btrfs or zfs.
You can go without partitions with any filesystem, and no filesystem
gets any benefit from not having a partition table.

> (other filesystems) The whole point when choosing BTRFS on a setup
> is to not use any other file systems. BTRFS/ZFS were designed to
> replace all other partitions+filesystems+LVM+RAID schemes (see the
> video)

No, the idea is to replace lvm and raid layers for dividing up and
adding redundancy to storage spaces. In other words, you used to
partition a disk into different pieces for different parts of the
filesystem, like / and /home, even when they each used the same
filesystem type, and you don't want or need to do this with btrfs.

A single btrfs partition per disk is not the same thing as not having
any partition table at all; do not confuse the two.

> (coding in small spaces) The first 64Kib only needs to point to
> /@/boot how much code do you really need for a pointer? note: @
> refers to the root subvolume that Ubuntu creates automatically on
> an btrfs install.

There is no "only points to". That 64k has to have all of the code
needed for the grub core, the disk driver, and the btrfs filesystem
code, which has to be capable of understanding any btrfs filesystem and
locating any file on it. That's a LOT of code for such a complex
filesystem.

> (UEFI FAT) They didn't need to change MBR they just needed to
> change it to a pointer to a place in an existing filesystem (like
> the way btrfs works)

I think you are confused here again. There are no "pointers". In
btrfs, they simply leave the first 64k of the disk untouched so that a
boot loader can put itself in the place the pc bios normally loads it
from. Blindly loading the first sector of a disk was one of the major
design flaws of the pc bios and the reason uefi switched to having an
actual filesystem on the disk that the firmware understands and can
locate and load files ( that identify which architecture they are for so
the correct one can be loaded ) from. The MBR -> GPT switch is really
just an added bonus that you may as well take advantage of at the same
time you switch to UEFI, but it is actually possible to just use MBR and
define an EFI system partition with that.

> I checked both my desktop and laptop BTRFS boot drives and they
> both show 0x000: EB 63 90 in the very first sector. So I am
> guessing the EB 63 90 represents BTRFS's boot pointer area.

Again, there is no pointer. The first 440 bytes of the mbr/boot sector
are executable 16 bit 8086 code. The first instruction of which is
usually a jump instruction. EB 63 90 decodes to JMP 0165. You are
looking at the first instruction of grub. The grub code goes on to read
the rest of itself from that first 64k, then on to opening the
filesystem, loading modules, the config file, etc.

> (wipe btrfs boot secto...