to.etc.dbpool
Class ConnectionPool

java.lang.Object
  extended by to.etc.dbpool.ConnectionPool

public final class ConnectionPool
extends java.lang.Object

Working

This represents an actual pool for a single database. It collects a list of connections to the same database. The connection pool is a generic provider for database connections of several types. These can be pooled connections but the thing can also be used to provide connections without a pooled set being built.

To start a pool it needs to be defined first. Defining the pool means that the definePool() call in the PoolManager has been called, providing the pool's data like it's ID, the database driver to use, the userid/password etc etc. Defining a pool does not allocate a pool of connections. After definition all of the connection calls work but when a connection is closed it gets discarded as well, i.e. it is not returned to the pool.

To start pooling connections you need to call initializePool() after it's definition. This forces the pool to allocate it's minimum number of available connections, and new connections will be retrieved from here. When connections are closed they will be returned to the pool unless the max #of open connections is reached in which case the connection will be discarded.

Connections from the pool

Connections returned by this code are always wrapped by a Connection wrapper. We call these connections Connection proxies. To obtain the actual connection (for instance when you need to do driver-specific calls) you need to cast the Connection to the PooledConnection wrapper and call getRealConnection() on it.

All of the pools' connections do resource management: all resources (statements, result sets) obtained from the connection is kept in a list with the connection so that they can be released/closed as soon as the connection proxy is closed. This prevents reused real connections from having open statements. It also means that resource leaks are minimized: as long as the connection is closed it also closes all open cursors.

Main connection types

Two main types of connection exist: Pooled connections and Unpooled connections. The latter is a bit of a misnomer but the term is kept because too much code uses these terms.

Pooled connections

Pooled connections are connections that are used in server applications. These are obtained from the set of free connections if possible and are subject to several checks:

Unpooled connections

Unpooled connections are connections that do not have the above checks executed on them. As such they can execute database code that takes longer for instance for deamon-like tasks. If the pool operates in pooled mode the connections are allocated from the available connections in the pool so they will be provided quickly.

In addition, Unpooled connections are not counted as "used" connections from the pool's connection set. This means an unlimited amount of Unpooled connections can be allocated.

Closing connections

By default these connection types are closeable, meaning that calling close() on the connection will discard all resources and return the real connection back to the pool. It is however possible to set a connection proxy into uncloseable mode with a call to setCloseable(false). After this call all calls to close() are silently ignored. To actually close the proxy you need to call closeForced().

This is used in server code where connections are cached during a request. As a server executes many code sections during the handling of a request the overhead of allocating connections for every database action is large. Another problem is that calling code from other code can cause multiple connections to be allocated to the same database, reducing the ability of the pool to provide connections. To prevent this we usually want to "cache" connections once allocated during the scope of a request. This caching can easily be done by setting the connection to uncloseable at allocation time and closing it in a top-level code part just before the request terminates. Setting the connection to uncloseable ensures that if code calls close the connection stays alive for other code.

Thread connections

Besides explicit connection caching where a connection is obtained, set to uncloseable and reused by calls there is another way to cache connections during the lifetime of a request: by using ThreadConnections. A thread connection can be pooled or unpooled as usual. When a thread connection is allocated the connection is registered to belong to the thread which allocated it. This thread becomes the "owner" of that thread. The current ThreadConnection for any given pool and Thread can be quickly obtained by looking in a per-thread hashtable mapped by pool.

The first time a thread allocates a ThreadConnection it will not have a current one so a new connection of the requested base type (pooled, unpooled) is allocated and saved in the thread's map. This connection will then be set to uncloseable to prevent it from being closed inadvertedly.

The next time the thread requests a ThreadConnection this stored copy will be returned, allowing for reuse of a connection during the time it is allocated to the thread.

Since a ThreadConnection is uncloseable calling close() on it will not close the connection; this must be done either by calling closeForced() or by calling the closeThreadConnections() call on the poolmanager. This call will walk the connection list for the calling thread and discard all of the thread connections allocated therein. After this call a new call to allocate a thread connection will allocate a new connection from the poolset.

Interaction between ThreadConnections and the other shit

Unfortunately the "cache-it-myself" and the ThreadConnections method of connection caching do not mix that well. It would be best to select one method for a complete system and not use the other. If this is impossible (because multiple code bases each use their own approach like NEMA) there are some special semantics to remember.

Mixing both methods works best if the "cache-it-yourself" method uses a ThreadConnection and does not forcefully close it, leaving that to the closeThreadConnections() call. If the "cache-it-yourself" method allocates a pooled connection (not a thread connection) and another piece of code allocates a thread connection this will cause a single thread to use two connections (possibly to the same database). This cannot be avoided because forcing the pooled connection to check for a threadconnection first causes a shitload of trouble when that connection gets closed.

Version:
$Version$
Author:
jal

Nested Class Summary
static class ConnectionPool.ErrorEntry
           
 
Field Summary
static java.util.logging.Logger ALLOC
           
static java.util.logging.Logger JAN
           
protected  boolean m_dbg_stacktrace
          T if this pool has stack tracing enabled.
protected  long m_n_open_rs
          The #of resultsets opened by all statements in the pool
protected  int m_n_open_stmt
          The #of statements CURRENTLY allocated by the pool
protected  long m_n_rows
          Deprecated. 
protected  int m_peak_open_stmt
          The #of statements MAX allocated by the pool
protected  long m_statementTotalPrepareCount
          The #of prepare statements executed.
static java.util.logging.Logger MSG
           
 
Constructor Summary
ConnectionPool(PoolManager pm, java.lang.String id, PoolConfig config)
           
 
Method Summary
 PoolConfig c()
          Return the config parameter class.
 boolean dbgIsStackTraceEnabled()
          Returns T if stack tracking is enabled for debugging purposes.
 void dbgRelease(java.lang.String what, java.sql.Connection dbc)
           
 void dbgSetStacktrace(boolean on)
          Switches stack tracing ON.
 int getConnectionUsedTooLongWarningTimeout()
          Returns the #of seconds that a connection must have been USED before a warning and a stack dump is generated.
 java.lang.String getID()
           
 PoolManager getManager()
          Return the owner pool manager.
 javax.sql.DataSource getPooledDataSource()
           
 PoolStats getPoolStatistics()
          Copies all pool data into the poolStats structure.
 java.util.List<ConnectionPool.ErrorEntry> getSavedErrorList()
           
 java.sql.Connection getUnpooledConnection(java.lang.String username, java.lang.String password)
           
 javax.sql.DataSource getUnpooledDataSource()
           
 java.util.List<ConnectionProxy> getUsedConnections()
          Get a list of all ConnextionProxy's currently in use.
 int[] getUseTimeTable()
           
 java.lang.String getUseTimeTableStr()
          Returns a HTML-formatted table of connection usage times.
 boolean hasSavedErrors()
           
 void initialize()
          Tries to put the pool in "pooled mode".
 boolean isPooledMode()
           
 boolean isSavingErrors()
           
 void saveError(java.lang.String subject, java.lang.String msg)
           
 boolean scanExpiredConnections(int scaninterval_in_secs, boolean forcedisconnects)
          This function gets called from the broker's janitor thread, OR from the purgatory handler (the thing called when all connections are used).
 void setSaveErrors(boolean on)
           
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

MSG

public static final java.util.logging.Logger MSG

JAN

public static final java.util.logging.Logger JAN

ALLOC

public static final java.util.logging.Logger ALLOC

m_n_open_stmt

protected int m_n_open_stmt
The #of statements CURRENTLY allocated by the pool


m_peak_open_stmt

protected int m_peak_open_stmt
The #of statements MAX allocated by the pool


m_n_open_rs

protected long m_n_open_rs
The #of resultsets opened by all statements in the pool


m_statementTotalPrepareCount

protected long m_statementTotalPrepareCount
The #of prepare statements executed.


m_n_rows

@Deprecated
protected long m_n_rows
Deprecated. 

m_dbg_stacktrace

protected boolean m_dbg_stacktrace
T if this pool has stack tracing enabled.

Constructor Detail

ConnectionPool

public ConnectionPool(PoolManager pm,
                      java.lang.String id,
                      PoolConfig config)
               throws java.sql.SQLException
Throws:
java.sql.SQLException
Method Detail

c

public PoolConfig c()
Return the config parameter class.

Returns:

initialize

public void initialize()
                throws java.sql.SQLException
Tries to put the pool in "pooled mode". If the pool already is in pooled mode we are done before we knew it ;-)

Throws:
java.sql.SQLException

dbgRelease

public void dbgRelease(java.lang.String what,
                       java.sql.Connection dbc)

getUsedConnections

public java.util.List<ConnectionProxy> getUsedConnections()
Get a list of all ConnextionProxy's currently in use. It gets that list with "this" locked but locks nothing else. The entries returned are live, so by the time you are using them they can be dead (because they are closed/invalidated at that time).

We return the proxies, not the entries, because the proxies remain valid for a single use context. It means they are stable in time and have a single life cycle (life, dead).

Returns:

getUnpooledConnection

public java.sql.Connection getUnpooledConnection(java.lang.String username,
                                                 java.lang.String password)
                                          throws java.sql.SQLException
Parameters:
username -
password -
Returns:
Throws:
java.sql.SQLException

scanExpiredConnections

public boolean scanExpiredConnections(int scaninterval_in_secs,
                                      boolean forcedisconnects)
This function gets called from the broker's janitor thread, OR from the purgatory handler (the thing called when all connections are used).


getConnectionUsedTooLongWarningTimeout

public int getConnectionUsedTooLongWarningTimeout()
Returns the #of seconds that a connection must have been USED before a warning and a stack dump is generated.

Returns:

getUseTimeTable

public int[] getUseTimeTable()

getUseTimeTableStr

public java.lang.String getUseTimeTableStr()
Returns a HTML-formatted table of connection usage times.

Returns:

setSaveErrors

public void setSaveErrors(boolean on)

hasSavedErrors

public boolean hasSavedErrors()

isSavingErrors

public boolean isSavingErrors()

getSavedErrorList

public java.util.List<ConnectionPool.ErrorEntry> getSavedErrorList()

saveError

public void saveError(java.lang.String subject,
                      java.lang.String msg)

getID

public java.lang.String getID()

getManager

public final PoolManager getManager()
Return the owner pool manager.

Returns:

isPooledMode

public boolean isPooledMode()

getUnpooledDataSource

public javax.sql.DataSource getUnpooledDataSource()

getPooledDataSource

public javax.sql.DataSource getPooledDataSource()

dbgIsStackTraceEnabled

public boolean dbgIsStackTraceEnabled()
Returns T if stack tracking is enabled for debugging purposes.


dbgSetStacktrace

public void dbgSetStacktrace(boolean on)
Switches stack tracing ON. This is very expensive and should only be used in case of trouble. Switching on stack trace (done thru the /nema/ servlet path) causes the pool to remember the last 10 stack paths that accessed a connection.


getPoolStatistics

public PoolStats getPoolStatistics()
Copies all pool data into the poolStats structure. This provides a point-in-time copy of all pool statistics in proper relation.