path: root/src/site/fml/technical-faq.fml

<?xml version="1.0"?>
<faqs id="FAQ" title="Frequently Asked Technical Questions">
 <part id="General">

   <faq id="process-killed">
     <question>What happens when a Carbonado process is killed while in the middle of a transaction?</question>
     <answer>
       <p>
Carbonado uses shutdown hooks to make sure that all in progress transactions
are properly rolled back. If you hard-kill a process (kill -9), then the
shutdown won't get run. This can cause a problem when using BDB, and
db_recover must be run to prevent future data corruption. BDB-JE is not
affected, however, as it automatically runs recovery upon restart.
       </p>
     </answer>
   </faq>

   <faq id="replicated-bootstrap">
     <question> How do I bootstrap a replicated repository?</question>
     <answer>
<p>
By running a resync operation programmatically. The ReplicatedRepository has a
ResyncCapability which has a single method named "resync". It accepts a
Storable class type, a throttle parameter, and an optional filter. Consult the
<a href="http://carbonado.sourceforge.net/apidocs/com/amazon/carbonado/capability/ResyncCapability.html">Javadocs</a> for more info.
</p>
<p>
In your application you might find it convenient to add an administrative
command to invoke the resync operation. This makes it easy to repair any
inconsistencies that might arise over time.
</p>
     </answer>
   </faq>

   <faq id="deadlock">
     <question> I sometimes see lock timeout errors and deadlocks, what is going on?</question>
     <answer>
<p>
A lock timeout may be caused by an operation that for whatever reason took too
long, or it may also indicate a deadlock. By default, Carbonado uses a lock
timeout of 0.5 seconds for BDB based repositories. It can be changed by calling
setLockTimeout on the repository builder.
</p>
<p>
Deadlocks may be caused by:
</p>
<p>
<ol>
<li>Application lock acquisition order</li>
<li>BDB page split</li>
<li>Index update</li>
</ol>
</p>
<p>
In the first case, applications usually cause deadlock situations by updating a
record within the same transaction that previously read it. This causes the
read lock to be upgraded to a write lock, which is inherently deadlock
prone. To prevent this problem, switch the transaction to update mode. This
causes all acquired read locks to be upgradable, usually by acquiring a write
lock from the start.
</p>
<p>
Another cause of this deadlock is when you iterate over a cursor, updating
entries as you go. To resolve this, either copy the cursor entries to a list
first, or operate within a transaction which is in update mode.
</p>
<p>
The second case, BDB page split, is a problem in the regular BDB product.
It is not a problem with BDB-JE. When inserting records into a BDB,
it may need to rebalance the b-tree structure. It does this by splitting a leaf
node and updating the parent node. To update the parent node, a write lock must
be acquired but another thread might have a read lock on it while trying to
lock the leaf node being split.
</p>
<p>
There is no good solution to the BDB page split deadlock. Instead, your
application must be coded to catch deadlocks are retry transactions.  They are
more common when filling up a new BDB.
</p>
<p>
The third case, index updates, is caused by updating a record while another
thread is using the index for finding the record. Carbonado's indexing strategy
can be coded to defer index updates when this happens, but it currently does
not. In the meantime, there is no general solution.
</p>
<p>
Lock timeouts (or locks not granted) may be caused by:
</p>
<p>
<ol>
<li>Failing to exit all transactions</li>
<li>Open cursors with REPEATABLE_READ isolation</li>
<li>Heavy concurrency</li>
</ol>
</p>
<p>
If any transactions are left open, then any locks it acquired don't get
released. Over time the database lock table fills up. When using BDB, the
"db_stat -c" command can show information on the lock table. Running
"db_recover" can clear any stuck locks. To avoid this problem, always run
transactions within a try-finally statement and exit the transaction in the
finally section.
</p>
<p>
By default, BDB transactions have REPEATABLE_READ isolation level. This means
that all locks acquired when iterating cursors within the transaction are not
released until the transaction exits. This can cause the lock table to fill
up. To work around this, enter the transaction with an explicit isolation level
of READ_COMMITTED which releases read locks sooner.
</p>
<p>
Applications that have a high number of active threads can cause general lock
contention. BDB-JE uses row-level locks, and so lock timeouts caused by
contention are infrequent. The regular BDB product uses page-level locks, thus
increasing the likelyhood of lock contention.
</p>
     </answer>
   </faq>

   <faq id="subselect">
     <question>How do I perform a subselect?</question>
     <answer>
<p>
Carbonado query filters do not support subselects, athough it can be
emulated. Suppose the query you wish to execute looks something like this in
SQL:
</p>
<p><pre>
select * from foo where foo.name in (select name from bar where ...)
</pre></p>
<p>
This can be emulated by querying bar, and for each result, fetching foo.
</p>
<p><pre>
// Note that the query is ordered by name.
Cursor&lt;Bar&gt; barCursor = barStorage.query(...).orderBy("name").fetch();
String lastNameSeen = null;
while (barCursor.hasNext()) {
    Bar bar = barCursor.next();
    if (lastNameSeen != null &amp;&amp; lastNameSeen.equals(bar.getName()) {
        continue;
    }
    lastNameSeen = bar.getName();
    Foo foo = fooStorage.query("name = ?").with(lastNameSeen).tryLoadOne();
    if (foo != null) {
        // Got a result, do something with it.
        ...
    }
}
</pre></p>
<p>
For best performance, you might want to make sure that Foo has an index on its name property.
</p>
<p>
You may track the feature request <a href="http://sourceforge.net/tracker/index.php?func=detail&amp;aid=1578197&amp;group_id=171277&amp;atid=857357">here</a>.
</p>
    </answer>
   </faq>

   <faq id="table-generation">
     <question>Does Carbonado support generating SQL tables from Storables?</question>
     <answer>
<p>
No, it does not. Carbonado instead requires that your Storable definition
matches a table, if using the JDBC repository. When using a repository that has
no concept of tables, like the BDB repositories, the Storable is the canonical
definition. In that case, changes to the Storable effectively change the
"table". In addition, properties can be added and removed, and older records
can still be read.
</p>
<p>
Although it is technically feasible for Carbonado to support generating SQL
tables, Storable definitions are not expressive enough to cover all the
features that can go into a table. For example, you cannot currently define a
foreign key constraint in Carbonado.
</p>
     </answer>
   </faq>

   <faq id="isnull">
     <question>How do I query for "IS NULL"?</question>
     <answer>
<p>
Carbonado treats nulls as ordinary values wherever possible, so nothing special
needs to be done. That is, just search for null like any other value. The query
call might look like:
</p>
<p><pre>
Query&lt;MyType&gt; query = storage.query("value = ?").with(null);
Cursor&lt;MyType&gt; = query.fetch();
...
</pre></p>
<p>
When using the JDBC repository, the generated SQL will contain the "IS NULL"
phrase in the WHERE clause.
</p>
     </answer>
   </faq>

   <faq id="sql-debugging">
     <question>How do I see generated SQL?</question>
     <answer>
       <p>
To see the SQL statements generated by the JDBC repository, you can install a
JDBC DataSource that logs all activity. Provided in the JDBC repository package
is the LoggingDataSource class, which does this. As a convenience, it can be
installed simply by calling setDataSourceLogging(true) on the
JDBCRepositoryBuilder.
       </p>
     </answer>
   </faq>

   <faq id="jdbc-indexes">
     <question>What happens if JDBC repository cannot get index info?</question>
     <answer>
       <p>
The JDBC repository checks if the Storable alternate keys match those defined
in the database. To do this, it tries to get the index info. If the user
account does not have permissions, a message is logged and this check is
skipped. This should not cause any harm, unless the alternate keys don't
match. This can cause unexpected errors when using the replicated repository.
       </p>
     </answer>
   </faq>

   <faq id="mysql-increment">
     <question>How do I use MySQL auto-increment columns?</question>
     <answer>
       <p>
As of 2006-10-23, Carbonado MySQL support is very thin. The @Sequence
annotation is intended to be used for mapping to auto-increment columns, if the
database does not support proper sequences. Until support is added,
auto-increment columns will not work.
       </p>
     </answer>
   </faq>

   <faq id="unique">
     <question>Can I do the equivalent of a "unique" constraint?</question>
     <answer>
       <p>
The @AlternateKeys annotation is provided specifically for this purpose. Both
@PrimaryKey and @AlternateKeys define unique indexes. The only real difference
is that there can be only one primary, but many alternate keys are allowed.
       </p>
     </answer>
   </faq>

   <faq id="join-cache">
     <question>How does one manually flush the Carbonado join cache?</question>
     <answer>

       <p>
The Carbonado join cache is a lazy-read cache, local to a Storable instance. It
is not a global write-through cache, and so no flushing is necessary.
</p>
<p>
The first time a join property has been accessed, a reference is saved in the
master Storable instance. This optimization makes the most sense when filtering
based on a join property. The query loads the join property, and you'll likely
want it too. This avoids a double load.
       </p>

     </answer>
   </faq>

   <faq id="evolution">
     <question>How can schemas evolve?</question>
     <answer>
       <p>
Independent repositories like BDB support automatic schema evolution. You may
freely add or remove non-primary key properties and still load older
storables. Changes to primary key properties is not supported, since they
define a clustered index. Also, property data types cannot be changed except if
a boxed property is changed to a non-boxed primitive and vice versa.
</p>
<p>
Every storable persisted by Carbonado in BDB starts with a layout version,
which defines the set of properties encoded. Carbonado separately persists the
mapping from layout version to property set, such that when it decodes a
storable it knows what properties to expect.
</p>
<p>
When adding or removing properties, existing persisted storables are not
immediately modified. If you remove a property and add it back, you can recover
the property value still encoded in the existing storables. Property values are
not fully removed from an existing storable instance until it is explicitly
updated. At this time, the layout version used is the current one, and the
removed property values are lost.
</p>
<p>
When loading a storable which does not have a newly added property, the
property value is either null, 0, or false, depending on the data type. You can
call the isPropertyUninitialized method on the storable to determine if this
default property value is real or not.
</p>
<p>
In order to change a property type to something that cannot be automatically
converted, the change must be performed in phases. First, define a new
property, with a different name. Then load all the existing storables and
update them, setting the new property value. Next, remove the old property. To
potentially free up storage you can update all the storables again. If you wish
the newly added property to retain the original name, follow these steps again
in a similar fashion to change it.
</p>
     </answer>
   </faq>

   <faq id="iterate-all">
     <question>How do I iterate over all storable types in a repository?</question>
     <answer>
       <p>
Given a repository and an appropriately set classpath, can we iterate through
all the various storables held in the repository without actually knowing what
the repository might hold in advance?
</p>
<p>
Repositories that implement StorableInfoCapability provide this
functionality. The reason its a capability is that some repos (JDBC) don't have
a registry of storables. BDB based ones do, and so this capability works.
</p>
     </answer>
   </faq>

 </part>
</faqs>