Evergreen

extra options and a change in behavior for the command-line marc_export script

Bug #1404387 reported by Jason Etheridge
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Evergreen
Confirmed
Wishlist
Unassigned
Evergreen 3.next

Bug Description

Top 4 commits from http://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/collab/phasefx/marc_export

This changes some JOIN's to LEFT JOIN's so we can output holdings along with bibs without holdings, and adds options for excluding certain holdings and bibs. I haven't checked for an impact on performance.

commit 4c0a382bcb327514cc4b2555a4f0764391cf1cdb
Author: Jason Etheridge <email address hidden>
Date: Fri Dec 19 15:11:14 2014 -0500

    This changes the behavior of --items

    And adds an option for --exclude-itemless-bibs. Previously, bib records without
    items would not be exported at all with the --items argument. Now they will (*),
    but the --exclude-itemless-bibs argument will restore the original behavior.

    * - This change was accomplished by performing a LEFT JOIN against the call number
    and copy tables, but if the --library option is used, then the original vanilla
    JOIN is still used, so whether an itemless bib will export or not with that
    argument will depend on whether there is a non-deleted call number on the bib.

    Signed-off-by: Jason Etheridge <email address hidden>

commit f2aaaa5360aa4b0164dc65d55b5f1ec658ef02f1
Author: Jason Etheridge <email address hidden>
Date: Fri Dec 19 15:52:33 2014 -0500

    options for excluding certain items

    by internal item id's and by barcodes.

        --exclude-item-id-file=<file containing list of item ids, one per line>
        --exclude-item-barcodes-file=<file containing list of item barcodes, one per line>

    Works in conjunction with --exclude-itemless-bibs, so if that argument is in effect and
    all items on a given bib are effectively filtered out via an --exclude-item argument,
    then the bib itself will not be included in the output.

    Signed-off-by: Jason Etheridge <email address hidden>

commit 5fe597a36d79cc71e5c975edcb80fd5cadc5adcf
Author: Jason Etheridge <email address hidden>
Date: Fri Dec 19 16:02:33 2014 -0500

    another option for excluding certain items

    this time by circ_modifier

        --exclude-item-circmod-file=<file containing list of circ modifier codes, one per line>

    Works in conjunction with --exclude-itemless-bibs, so if that argument is in effect and
    all items on a given bib are effectively filtered out via an --exclude-item argument,
    then the bib itself will not be included in the output.

    Signed-off-by: Jason Etheridge <email address hidden>

commit 61fc1ecb23681a7ea7c50e5ba7984ee0f67376e3 (HEAD, working/collab/phasefx/marc_export, phasefx)
Author: Jason Etheridge <email address hidden>
Date: Fri Dec 19 16:29:04 2014 -0500

    Option to exclude specific records by id

    --exclude-record-id-file=<file containing list of record ids, one per line>

    Signed-off-by: Jason Etheridge <email address hidden>

Revision history for this message
Jason Stephenson (jstephenson) wrote :

I'll take a look at these additions on my development VM. Dunno if I'll really get to it until the week after Christmas, but I'm confident we can get this into master before 2.8 is released.

Changed in evergreen:
milestone: none → 2.next
importance: Undecided → Wishlist
status: New → In Progress
assignee: nobody → Jason Stephenson (jstephenson)
Revision history for this message
Jason Stephenson (jstephenson) wrote :

I finally took the time to look at this branch (at the last minute). I see one issue while checking it today.

Items without circ_modifiers (we have a lot of those) cause the following warning when --items is specified:

Use of uninitialized value in hash element at /openils/bin/marc_export line 540.

I'll bang at some of the other options tomorrow. It takes a couple of hours to dump all of our records to fully test the options with --all.

Revision history for this message
Jason Stephenson (jstephenson) wrote :

Using this command line:

 marc_export --items --exclude-itemless-bibs --all --format=XML --encoding=UTF-8 > all-items.xml

I got the following message 19 times:

Deep recursion on subroutine "Marque::Biblio::next" at /openils/bin/marc_export line 565.

Without --exclude-itemless-bibs, I did not get that message.

I do get more results without --exclude-itemless-bibs, so that part of it appears to be working.

Revision history for this message
Jason Stephenson (jstephenson) wrote :

I should add that would probably be line 564 in regular marc_export. I added a line when I threw in a workaround for the warning in comment #2.

Revision history for this message
Jason Stephenson (jstephenson) wrote :

So, after a little googling, the deep recursion message would come from having 100 or more itemless bibs in a row. The easiest "fix" is to add a "no warnings 'recursion';" in that block.

Also, we don't need to import MARC::File::XML at that point, either. We only need to do that when there is a parsing error with going from raw XML to MARC::Record.

Revision history for this message
Jason Stephenson (jstephenson) wrote :

Changing status from in progress and removing myself as assignee because I am not likely to get to work on this any time soon.

Changed in evergreen:
status: In Progress → Confirmed
assignee: Jason Stephenson (jstephenson) → nobody
Terran McCanna (tmccanna)
tags: added: cat-importexport
Jason Etheridge (phasefx)
Changed in evergreen:
assignee: nobody → Jason Etheridge (phasefx)
Jason Etheridge (phasefx)
Changed in evergreen:
assignee: Jason Etheridge (phasefx) → nobody
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.