concept transaction in category graphs

This is an excerpt from Manning's book Graph Powered Machine Learning MEAP V08. Login to get full access to this book.

Starting from such transaction dataset, let’s define a graph model for our transaction dataset. Each transaction involves two nodes: a person (the customer, the user) and a merchant. The nodes are linked by the transaction itself. A transaction has a date and a status. Legitimate transactions have the status "Undisputed". Fraudulent transactions are "Disputed". Figure  2.18, below, presents how the data looks as a graph.

Figure  2.18  A sample of a graph model for credit card fraud detection

In Figure  2.18 the red connections are disputed transactions while the other are regular transactions. The resulting graph is large but the kind of analysis that has to be performed will be not affected by the dimension of the graph. Starting from such a dataset the analytics steps to spot the source of fraud are:

Listing 9.1 Code for importing transactions

def import_transactions(self, directory):
    transactions = pd.read_csv(directory + "creditcard.csv") #A
    for k in range(50):#B
        writing_thread = threading.Thread(target = self.write_transaction)
        writing_thread.daemon = True
        writing_thread.start()
 
    j = 0; 
    for index, row in transactions.iterrows(): #C
        j += 1
        transaction = {
            'transactionId': j,
            'isFraud': row['Class'],
            'transactionDt': row['Time'],
            'transactionAmt': row['Amount']}
        vector = self.normalize(row, ['Time', 'Class']) #D
        transaction['vector'] = vector;
        self._transactions.put(transaction); #E
        if j % 10000 == 0:
            print(j, "lines processed") 
    print(j, "lines processed")
    self._transactions.join() #F
    print("Done")
 
def write_transaction(self): #G
    query = """
        WITH {row} as map
        CREATE (transaction:Transaction {transactionId: map.transactionId})
        SET transaction += map
    """
    i = 0
    while True:
        row = self._transactions.get()
        with self._driver.session() as session:
            try:
                session.run(query, {"row": row})
                i += 1
                if i % 2000 == 0:
                    with self._print_lock:
                        print(i, "lines processed on one thread")
            except Exception as e:
                print(e, row)
        self._transactions.task_done() 

copy

This query returns a count of 118, which means of the first 1,000 transactions returned ordered by descending score, 11.8% were fraudulent. That might seem like a pretty low value, but it isn’t. Recall that there are 492 fraudulent transactions in the dataset out of a total of 284,807, which means only 0.17% of the transactions are actually fraudulent. This result reinforces our intuition that the greater the distance a transaction is from its closest neighbor, the higher the chance is that the transaction is fraudulent. This distance is defined score in the previous and next queries and discussions. It will be computed in different ways, but in all cases the higher is the score the higher is the considered the chance that the transaction is fraudulent.

This is an excerpt from Manning's book Neo4j in Action. Login to get full access to this book.

It’s very important to note that Neo4j is a transactional database. Everything you know about transactions from the relational world applies to Neo4j as well. Transaction support is one the key differentiators between Neo4j and the other NoSQL databases we mentioned earlier, which don’t support all ACID properties.

First, let’s examine the idiomatic way of programmatically creating transactions in Neo4j, as shown in the following snippet:

try (Transaction tx = graphDatabaseService.beginTx()) {
    //do something with the database
    tx.success();
}

copy

This code begins by making use of the Java 7 try-with-resource statement to create a new transaction and define a block of code against which this transaction will apply. Any statements that interact with the database and are executed within this block will all operate against the same transaction. The call to the success method at the end of the try block signifies that a commit should occur when the transaction finishes (or the resource “closes,” in try-with-resource terminology). The transaction finishes (or closes) when the code block completes. Neo4j will then ensure that this transaction is committed (having detected that the success method was previously called). If an exception is thrown while “doing something with the database,” the success method won’t be called, and the transaction will be rolled back when it finishes. Neo4j’s decision logic regarding whether to commit or roll back a transaction is based on whether the success or failure method was previously called. If you want to roll back a transaction explicitly, such as from a conditional code block, you can invoke the failure method, and the transaction will be unconditionally rolled back at the end of the block. Calling neither the success nor failure method will also result in the transaction being rolled back (default).

Transactions can be a complicated topic, but hopefully you now understand how Neo4j allows you to take full advantage of ACID-based transactions to manage concurrent access to graph resources.

We’ve covered how transactions work in detail: the default transaction behavior is satisfactory for most common cases, but we’ve examined the implications of transactions trying to access the same graph resources at the same time and how higher isolation levels can be achieved by using read and write locks explicitly.

Listing 7.2. Attempting to update with a transaction

This listing represents the general pattern of starting a transaction, carrying out some operations, and assuming they complete without exception. After the success method is called and the code block completes, the changes are committed to the database.

Schema-based operations and transactions

If you’re performing any schema-related activities within the graph, this code will need to be in a separate transaction from all other Neo4j transactional code; otherwise, an exception of the form ConstraintViolationException will be thrown with the message “Cannot perform data updates in a transaction that has performed schema updates.”