marc_export should have a --pipe option to force acceptance of a list of ids on standard input

Branch is here: https://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/dyrcona/lp1940662-marc_export-pipe

In case it isn't clear from the bug description or commit message, this feature is useful if you want to export as subset of a single library's holdings and exclude holdings from other libraries.

One way that I plan to use this in productionon is like so:

/usr/bin/psql -U evergreen -h db2 -A -q -t -c '<some query to find the records I want>' | marc_export --pipe --items --library SHORTNAME > file.mrc

CW MARS is already using this patch in production.

Note: to the best of my knowledge, I do not think this can be tested from staff UI; not from record bucket and not from Cataloging > Marc Import/Export. If I am missing something, just let me know. I'm happy to test.

You are correct, Jennifer.

marc_export is a command line utility script to batch export MARC records that is run on an Evergreen server. There is no way to test this in the staff UI.

This applied cleanly to master.

I tested this out on a test server with a copy of our production data.

--pipe only, works fine.
--pipe with --library and --descendant. works fine. I was confused at first until I saw that it is working at the volume level, so owning library. We use floating so quite a few of our items end up at different circ_library locations than where we originally send them.

--pipe with --all reports an error like it should.

I'm running into an issue trying to use --pipe with --since though. It appears to me like --since gets skipped any time there is a list of IDs. --since isn't mentioned in this ticket, but is mentioned in the commit message. Should --since work with --pipe?

I can add in release notes and see if the docs/modules/development/pages/support_scripts.adoc should be updated also.

Josh

I don't recall why --since is in the commit message. I'm not sure what the combination of --pipe and --since should do. As you say, it ignores --since in that case.

The point of --pipe is to be able to export item information for a location or group of locations for a limited set of records. The idea is you would have a SQL script or some other source determine which records to export and pipe that into marc_export. With certain options, marc_export will not wait for standard input, so --pipe forces it to do that.

Perhaps it should be an error to use --pipe with --since?

Working signoff branch at user/stompro/lp1940662-marc_export-pipe-signoff

https://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/stompro/lp1940662-marc_export-pipe-signoff

I added in release notes and updated the support scripts documentation. I also added in a check for using --pipe with --since. That seemed fairly straight forward. I don't know if that merits needing a new signoff?

Josh

Josh's changes work for me. CWMARS has been using the original commit in production since the summer of 2021 without incident, so I've pushed the signed off commit with an amended commit message to master for including in 3.11.

Thanks, Josh!