performance - Handling large transactions: any time/memory tradeoffs? -


Text after "

In our system there is a (very common) case where the user's action involves setting action / on / label can trigger the removal of nodes and the sum of the order of thousands of relationships entities (labeled 100K node a remove label set B 80K for label, set property for [x, y, z] 20K nodes and so on). Of course, I could not squeeze in a transaction all of them, and thanks to the fact that these nodes can be easily divided into separate sets in large numbers, I work in a separate transaction, which of course, breaks extend all Seediti, but in terms of performance satisfies us that I, a large those transactions trying to suffocate in to control all of them, so Are the top-level transactions try to update all internal transactions dB, which is certainly a result of the

My config (well, the relevant parts):

  "org.neo4j.server.database.mode": "yes", "use_memory_mapped_buffers": "true", "neostore.nodestore.db.mapped_memory": "450 million", "neostore.relationshipstore .db.mapped_memory ":" 450 million "" neostore.propertystore.db.mapped_memory ":" 450 million "," neostore.propertystore.db.strings.mapped_memory ":" 300 "," neostore.propertystore.db.arrays. Mapped_memory ":" 50M "," cache_type ":" HPC "," dense_node_threshold ":" 15 "," query_cache_size ":" 150 " 

You are right to revise the thousands of institutions in the same transaction. Not a single user action program is going to show up in Neo4j. Nested transactions in Neo4j are simply "Placebo" transactions, as you can tell correctly.

I need to start thinking about alternative strategies to get my goals (which I do not know about) without the need to update many institutions.

If no option is possible, then I will say if the update after the user action is fine for a short time. If the answer is yes, then I store a message about user action in a continuous queue, which I will process on asynchronous. In this way, the user returns quickly and updates are eventually.

Finally, if the time between the action of the user is acceptable and to take larger updates more time, I will consider and "agent" he constantly crawls the graph and the transaction Update the labels of those entities in opposition to the updated updates, which it encounters. Take a look for inspiration.


Comments