concept tuple in category lucene

appears as: tuple, tuples, The tuples, tuple
Lucene in Action, Second Edition

This is an excerpt from Manning's book Lucene in Action, Second Edition.

Given an entity, a way to represent its RDF description is to use two tables, each one having a tuple per RDF triple: in one table (outgoing relationships), the tuple key is the subject s (entity), the first cell is the predicate p (property), and the second one is the object o (value). In the second table (incoming relationships), the order is reversed; the key is the object o, and the second cell is the subject s. A generalized version of this accounting for multivalued predicates (such as multiple statements having the same s and p but different o) can be seen as two tuple tables composed by p and n values. This is shown in table 13.1. In SIREn, there’s no limit on the number of cells and null values are free.

Table 13.1. Representation of an entity tuple table

Outgoing relations

Subject

Predicate

Object 1

Object 2

 

Object n

id1 rdf:type Person Student ... Thing
id1 foaf:name Renaud Delbru R. Delbru ...  
id1 foaf:mailbox mail@deri.org   ...  
id1 foaf:made paper-9 paper-12 ... paper-n
... ...        
Incoming relations
Object Predicate Subject 1 Subject 2   Subject n
id1 foaf:knows giovanni nickolai ...  
id1 foaf:member DERI ø ...  
... ...        

Each cell can be searched independently or together as one unit using full-text search operations such as Boolean, proximity, or fuzzy operators. By combining multiple cells, you can query a tuple pattern to retrieve a list of entity identifiers. It’s also possible to combine tuples to query an entity description. The tuples listed in section 13.2 provide an example of combining query operators to match the entity described in table 13.1.

Lucene documents are sent to the index server. The document is analyzed before indexing. We created our own Lucene analyzer for fields containing tuples. The field content, of type tuple, has a special syntax derived from the N-Triples syntax:

http://www.w3.org/1999/02/22-rdf-syntax-ns#type
  <http://xmlns.com/foaf/0.1/Person> .
<http://xmlns.com/foaf/0.1/name> "Renaud Delbru" .
<http://xmlns.com/foaf/0.1/knows> <http://g1o.net#me>
  _:node4321 .

URIs are enclosed in angle brackets, literals are written using double quotes, blank nodes are written as _:nodeID, and a dot signifies the end of a tuple.

A TupleTokenizer, a grammar-based tokenizer generated with JFlex, breaks text using the tuple syntax into tokens. It generates tokens of different types, such as uri, bnode, and literal for the elements of a tuple, but also cell_delimiter and tuple_delimiter to inform the TupleAnalyzer downstream of the ending of a tuple element or tuple (section 4.2.4 describes the token’s type attribute). The TupleTokenizer embeds a Lucene analyzer for analysis of the literal tokens. Whenever a literal token is created, it’s sent to this second analyzer in order to tokenize the literals. This offers the flexibility of reusing Lucene components for analyzing the textual information of the literals.

The TupleScorer scores a Boolean combination of cells, and provides an iterator over the entities and tuples (eid, tid) matching the query. It’s based on the CellConjunctionScorer, CellDisjunctionScorer, and CellReqExclScorer to score a conjunction, disjunction, or exclusion of cells inside a tuple. Each walks through the iterators of the underlying scorers (CellScorer) and perform joins over entities and tuples.