Akiban Persistit 3.1.7

Milestone information

Project:
Akiban Persistit
Series:
trunk
Version:
3.1.7
Released:
2012-09-11  
Registrant:
Nathan Williams
Release registered:
2012-09-17
Active:
No. Drivers cannot target bugs and blueprints to this milestone.  

Download RDF metadata

Activities

Assigned to you:
No blueprints or bugs assigned to you.
Assignees:
5 Peter Beaman
Blueprints:
No blueprints are targeted to this milestone.
Bugs:
5 Fix Released

Download files for this release

File Description Downloads

Release notes 

************************************
Akiban Persistit - 3.1.7
************************************

Release Date
=============
September 11, 2012

Overview
=========
Fix several bugs, add buffer pool preload ("warm-up"), reformat code base.

See http://www.akiban.com/akiban-persistit for a summary of features and benefits, licensing information and how to get support.

Documentation
==============
Users Guide: http://www.akiban.com/ak-docs/admin/persistit
JavaDoc: http://www.akiban.com/sites/all/libraries/persistit-api/index.html

Building Akiban-Persistit
==========================
Use Maven (maven.apache.org) to build Persistit.

To build:

  mvn install

The resulting jar files are in the target directory. To build the Javadoc:

  mvn javadoc:javadoc

The resulting Javadoc HTML files are in target/site/apidocs.

Building and Running the Examples
---------------------------------

Small examples are located in the examples directory. Each has a short README file describing the example, and an Ant build
script (http://ant.apache.org). After building the main akiban-persisit jar file using Maven, you may run:

  ant run

in each of the examples subdirectories to build and run the examples.

Buffer Pool Configuration
==========================
For optimal performance, proper configuration of the Persistit buffer pool is required. See section "Configuring the Buffer
Pool" in the configuration document http://www.akiban.com/ak-docs/admin/persistit/Configuration.html

NOTE: Especially when used with multi-gigabyte heaps, the default Hotspot JVM server heuristics are be suboptimal for
Persistit applications. Persistit is usually configured to allocate a large fraction of the heap to Buffer instances that are
allocated at startup and held for the duration of the Persistit instance. For efficient operation, all of the Buffer instances
must fit in the tenured (old) generation of the heap to avoid very significant garbage collector overhead. Use either
-XX:NewSize or -Xmn to adjust the relative sizes of the new and old generations.

Resolved Issues
================
AssertionError in IntegrityCheckTest
https://launchpad.net/bugs/1044513
A bug in unit test code caused spurious assertion errors.

IntegrityCheck throws ArrayIndexOutOfBoundsException
https://launchpad.net/bugs/1044397
IntegrityCheck and the CLI pview command were not immune to
certain inconsistent byte patterns in a corrupt page, making them
inadequate for diagnosing a severely corrupted database.

Mismatched buffer pool sizes and volume page size results in NPE
https://launchpad.net/bugs/1041293
Attempting to open a Volume with a valid page size S resulted in a
NullPointerException if there was no buffer pool having page size
S specified. This same condition how throws an IllegalStateException.

Build fails: persistit-coverage hangs with deadlock
https://launchpad.net/bugs/1043536
A bug in a unit test left a deadlock condition under certain race
conditions. Speed variations caused by clover instrumentation
triggered frequent occurrences.

Changes and New Features
=========================

Persistit 3.1.7 - Code Base Reformatted
---------------------------------------

To simplify diffs and improve legibility, the entire code base was reformatted and "cleaned up" by the Eclipse code formatting
tool. The maven build now automatically formats all
source to ensure coherent diffs in the future. The settings for formatting and code style cleanup by Eclipse are found in the
src/etc directory.

Persistit 3.1.7 - Buffer Pool Preload
-------------------------------------

On a server with a large buffer pool (many gigabytes), a Persistit instance can run for a long time before the buffer pool
becomes populated with a suitable working set of database pages. Until then performance is degraded due to a potentially large
number of random reads. For a production server the result may be poor performance for minutes to hours after restart.

The preload facility periodically records an inventory of the pages currently in the buffer pool(s) and optionally reloads the
same set of pages when Persistit is restarted. During the preload process Persistit attempts to read pages in approximate
file-address order to accelerate reads. In one of our experiments Persistit preloads a buffer pool with over 800,000 16Kbyte
buffers in about a minute, which is orders of magnitude faster than the same process would take with reads performed
incrementally at random.

Two new configuration properties `setBufferInventoryEnabled(boolean)
<../apidocs/com/persistit/Configuration.html#setBufferInventoryEnabled(boolean)>`_ and com.persistit#setBufferPreloadEnabled
control this behavior. These settings are turned off by default in version 3.1.7.

Known Issues
=============

Transactional Tree Management
-----------------------------

All operations within Trees such as store, fetch, remove and traverse are correctly supported within transactions. However, the
operations to create and delete Tree instances currently do not respect transaction boundaries. For example, if a transaction
creates a new Tree, it is immediately visible within other Transactions and will continue to exist even if the original
transaction aborts. (However, records inserted or modified by the original transaction will not be visible until the transaction
commits.) Prior to creating/removing trees, transaction processing should be quiesced and allowed to complete.

Problems with Disk Full
------------------------------------

https://bugs.launchpad.net/akiban-persistit/+bug/916071

There are rare cases where Persistit will generate exceptions other than java.io.IOException: No space left on device when a disk
volume containing the journal or volume file fills up. The database will be intact upon recovery, but the application may receive
unexpected exceptions.

Out of Memory Error, Direct Memory Buffer
------------------------------------------------------

https://bugs.launchpad.net/akiban-persistit/+bug/985117

Out of Memory Error, Direct Memory Buffer. Can cause failed transactions under extreme load conditions as a result of threads
getting backed up writing to the journal file. However, this error is transient and recoverable by by retrying the failed
transaction.

* Workaround: Ensure your application has the ability to retry failed transactions

Tree#getChangeCount may return inaccurate result
-------------------------------------------------------------

https://bugs.launchpad.net/akiban-persistit/+bug/986465

The getChangeCount method may return inaccurate results as its not currently transactional. The primary consumer is the
PersistitMap. As a result of this bug Persistit may not generate java.util.ConcurrentModiciationException when it is supposed to.

Multi-Version-Values sometimes not fully pruned
-------------------------------------------------------------

https://bugs.launchpad.net/akiban-persistit/+bug/1000331

Multi-version values are not always pruned properly causing volume growth. The number of MVV records and their overhead size can
be obtaining by running the IntegrityCheck task.

* Workaround 1: Run the IntegrityCheck task (CLI command icheck) with the -P option which will prune the MVVs. This will remove
obsolete MVV instances and in many cases free up pages in which new data can be stored. However, it will not reduce the actual
size of the volume file.

* Workaround 2: To reduce the size of the volume you can use the CLI commands save and load to reload the data into a
newly created volume file. See http://www.akiban.com/ak-docs/admin/persistit/Management.html#management for more information
about these operations.

Changelog 

This release does not have a changelog.

0 blueprints and 5 bugs targeted

Bug report Importance Assignee Status
1043536 #1043536 Build fails: persistit-coverage hangs with deadlock 2 Critical Peter Beaman  10 Fix Released
1041293 #1041293 Mismatched buffer pool sizes and volume page size results in NPE 4 Medium Peter Beaman  10 Fix Released
1044397 #1044397 IntegrityCheck throws ArrayIndexOutOfBoundsException when analyzing a corrupt volume 4 Medium Peter Beaman  10 Fix Released
1044513 #1044513 AssertionError in IntegrityCheckTest 4 Medium Peter Beaman  10 Fix Released
1047973 #1047973 O(N*N) Loop in JournalManager#cleanupPageList 4 Medium Peter Beaman  10 Fix Released
This milestone contains Public information
Everyone can see this information.