Comment 5 for bug 1543094

My recent findins about compate-and-swap used in internal ipam driver indicates,
that it scales worse than locking approach.
Compare-and-swap workflow in ipam driver:
A. Start transaction.
B. Do various db opeations(read/write), can be tens of db queries.
C. Read availability range.
D. Calculate updated availability range and try to update it
E. If count of updated rows is less than expected raise RetryRequest
F. Commit transaction

RetryRequest is handled outside db transaction, so retry has to start from step A.
It means cost of one retry is cost of each operation from A to E, and that is tens of db
queries done on step B.

Why do we need to restart entire transaction?
Because of transaction isolation level, which is often default to REPEATED_READ.
REPEATED_READ transaction isolation level means reading the same row returns
the same result.
So if we try to handle retry request without restarting transaction, then on C step
the same availability range would be read, even if another transaction has changed this range.

Locking workflow:
A. Start transaction.
B. Do various db opeations(read/write), can be tens of db queries.
C. Lock for update availability range row.
D. Do actual update.
F. Commit transaction.
In this case if two processes concurenly update row, one of them hangs up on step C.
When lock is released second process continues from step C, without having to restart
entire transaction.

Locking approach has issues in work with Galera multi master. In this case each master node
can ascquire lock on step C, but only the first master will be able to commit transaction.
For the second master Deadlock will be raised.
Currently Deadlock is handled in the same wrapper which handles RetryRequest and entire
transaction is restarted in this case.
So in rare case where the same row is updated on different masters entire transaction get restarted,
but for compare-and-swap each concurent update result in entire transaction restart.

To make compate-and-swap to work effectively it should repeat only steps C and D when RetryRequest is raised.
But in our case, where ipam driver is called inside big transaction it looks hardly achievable.
But still how it can be achieved?
- Use READ_COMMITTED isolation level for all backends.
 Not sure if it is something that is safe to do.
- Create new connection/transaction inside of ipam driver that is out of scope of big transaction.
 All my experiments in trying to get new connection inside of running big transaction finished with no success.
 But doesn't look impossible yet.
- Any other ideas?