Omgili, forum search, forums search, search forums, discussion search,discussions search, search discussions, board search, boards search, search boards
  Advanced Search

2 phase commit with external data

On Fri, 6 Nov 2009 10:59:24 -0500, Peter Keegan <...@gmail.com

I'm trying to use a two phase commit involving a Lucene index and an
external file derived from the index.
Here are the steps:

1. prepare commit on Lucene index
2. prepare commit on external file
3. commit Lucene index
4. commit external file

Step 2 requires an IndexReader with access to the 'prepared' Lucene index,
but I don't see any methods for this. Is there a way to read the prepared
index? I really only need access to a stored field. I'm using Lucene-2.9

Thanks,
Peter



On Fri, 6 Nov 2009 11:02:53 -0500, Michael McCandless <...@mikemccandless.com

Can you use IndexWriter.getReader() to get the reader for step 2?

Failing that, you could simply commit the change, but use a deletion
policy that keeps the old commit alive. Then open a normal reader and
read whatever you need for step 2, and commit the external file. If
an error happens and you need to rollback you can simply open a new
IndexWriter on the old commit point -- this lets you rollback even if
the commit has already happened.

Mike

On Fri, Nov 6, 2009 at 10:59 AM, Peter Keegan <...@gmail.com
---------------------------------------------------------------------
To unsubscribe, e-mail: java...@lucene.apache.org
For additional commands, e-mail: java...@lucene.apache.org

On Fri, 6 Nov 2009 11:22:05 -0500, Peter Keegan <...@gmail.com

Yes - perfect! I didn't think that would be different than refreshing or
recreating an IndexReader.

I don't need to keep the old commit alive. The goal is to keep the external
file in synch with the index, so a separate searcher process will see
consistent data. By postponing both commits, the window where they are out
of synch is very small (2 file renames). I record the Lucene index version
in the external file for checking synchcronization.

Thanks,
Peter

On Fri, Nov 6, 2009 at 11:02 AM, Michael McCandless <
luce...@mikemccandless.com

On Fri, 6 Nov 2009 11:33:12 -0500, Michael McCandless <...@mikemccandless.com

On Fri, Nov 6, 2009 at 11:22 AM, Peter Keegan <...@gmail.com
Great!

getReader() searches the full index, plus uncommitted changes.

OK.

Mike

---------------------------------------------------------------------
To unsubscribe, e-mail: java...@lucene.apache.org
For additional commands, e-mail: java...@lucene.apache.org

On Fri, 6 Nov 2009 11:35:27 -0500, Peter Keegan <...@gmail.com

Which version of the index will IndexWriter.getReader() return if there have
been updates, but no call to 'prepareCommit'?

On Fri, Nov 6, 2009 at 11:33 AM, Michael McCandless <
luce...@mikemccandless.com

On Fri, 6 Nov 2009 11:40:45 -0500, Michael McCandless <...@mikemccandless.com

It will always return a reader reflecting every change done with that
writer (plus, the index as it was when the writer was opened) before
getReader was called.

It's unaffected by the call to prepareCommit.

Mike

On Fri, Nov 6, 2009 at 11:35 AM, Peter Keegan <...@gmail.com
---------------------------------------------------------------------
To unsubscribe, e-mail: java...@lucene.apache.org
For additional commands, e-mail: java...@lucene.apache.org

Discussion Title: 2 phase commit with external data
Title Keywords: phase  commit  with  external  data