perl-based tests against stock test data on running system

I'm going to go ahead and add a pullrequest to this. I have more tests coming, but I can push a new branch/ticket for them. There are also two tickets/branches that silence some warnings that these tests manage to trigger:

https://bugs.launchpad.net/evergreen/+bug/1207425

and

https://bugs.launchpad.net/opensrf/+bug/1207464

Note that there are 5 commits at the tip of collab/phasefx/livetests

Sneaked in two more commits since no one has grabbed this yet. 7 commits at tip. Feel free to squash.

I force-pushed a rebased and squashed branch to collab/phasefx/livetests. Just a single commit at the tip.

Pushed another commit for 05-pay_bills.t

First impression: the copy/paste code in the test cases (like osrf_connect(), etc) must be factored out into a common library. Heavy copy/paste is a bad code quality smell.

Ideally as much as possible of the common code should be in a common library that can be used by other actual production code, too, so that the tests can focus on specifics. Fortunately, there is such a library: OpenILS::Utils::Cronscript!

The remaining test-specific code could go into a OpenILS::Utils::TestHelper library, or something like that.

Thanks Dan; I did try to make use of oils_header.pl for this purpose, but had trouble pulling it in so that it worked both with make check and when running the tests directly out of the live_t/ directory. Doing the latter is probably not strictly needed, though it helps to be able to do that when developing the tests. I'll give it another go, but if anyone has any ideas in that regard, I'd appreciate any help. I also got the impression that the testing community shies away from pulling in external libraries with tests when not needed, but if the library is kept simple, maybe that's not a big deal for us.

I pushed two new commits to collab/phasefx/livetests (so 4 commits at the tip that aren't in master). Adds a method to Cronscript, adds a TestUtils module inheriting from Cronscript, and has the live_t/ tests making use of TestUtils for less code duplication. Let me know how it looks.

Jason, I gave this a run through. My only concern is that the tests can only be run once per newly minted database, since the tests fail when a workstation already exists. One solution that worked for me was to call register workstation in override mode:

open-ils.actor.workstation.register.override

This will work regardless of whether the workstation exists. Alternatively, we could delete each workstation we create.

Or we could add a set of stock workstations to the seed data and use those instead of creating new ones for each test.

Bill, either of those sound sane and I have no objection to them. But my concern with your concern is that in the long run, it may be difficult to keep these tests perfectly reversible, so I wonder if we should not have that expectation in order to make it easier to produce tests in the first place. The tactic I had in mind would be for an integration server to to use a pristine database (at minimum) for each test run. A bit onerous maybe for developers running these things by hand, but not for an automated system. What do you think?

Hmm, yeah, that may be an unreasonable expectation on my part. I'm not entirely sure why I think it's important. My first instinct was to run the tests multiple times, though. I guess I see it as another dimension to the tests. I.e. does running the test once break the system in some way? That may not be the right way to think about, though...

Erring on the side of *not* increasing the barrier to entry for testing, disregard my concerns.

Ran through another round of testing. Squashed and added a few more words to top two commits. Pushed to master. Thanks, Jason.