Comment 11 for bug 1269547

Tom and others affected by the bitmap file sparseness -

Here are the current options I see for addressing this, including your suggestions. What do you think?

    - Re. tracking two parallel sets of bitmaps, what would yo do with
      the first set - delete its files externally as they are rotated?
      This would also need a limit on the in-memory bitmap size with a
      forced rotation when it is reached for the second set. Also, how
      would the information of the two sets of files be merged
      if there is need to find the last tracked LSN or by XtraBackup
      if both are present?
    - The option of variable-length bitmap data pages seems to be
      easiest to implement and use (with some reservations re. crash
      recovery, but these should be possible work out), but the
      savings there will be less as only data inside a single
      checkpoint would be compressed whereas other approaches would
      compress arbitrarily many checkpoints. Would you be willing to
      test a prototype to measure space savings?
    - Re. the table skip option, the biggest concern is silently
      broken backups due to missing data as I wrote before. To make
      backups safe, the information of the skipped tables (and the LSN
      intervals for when the table skips were active) needs to be
      present in the bitmaps somehow. Then XtraBackup would only
      accept bitmaps if the skipped table information is consistent
      with XB --tables options. The skipped tables information could
      be provided by a special data page type in the bitmaps, but, if
      we need to change the format, might as well do the compact
      representation instead.
    - I also considered an option of providing an external utility to
      merge finished bitmap files. It would have to be a part of
      XtraBackup, which might not necessarily be acceptable because of
      other considerations. It would also have to work with
      arbitrarily large bitmap files.