java - fast way to execute multiple CREATE statements -

-

i access neo4j database via java , want create 1,3 million nodes. therefore create 1,3 million "create" statements. figured out, query way long. can execute ~100 create statements per query - otherwise query fails:

client client; webresource cypher; string request; clientresponse cypherresponse; string query = ""; int nrqueries = 0;  for(hashmap<string, string> entity : entities){     nrqueries++;     query += " create [...] ";      if(nrqueries%100==0){         client = client.create();         cypher = client.resource(server_root_uri + "cypher");         request = "{\"query\":\""+query+"\"}";         cypherresponse = cypher.accept(mediatype.application_json).post(clientresponse.class, request);         cypherresponse.close();         query = "";     } }

well, want execute 1,3 million queries , can combine 100 1 request, still have 13,000 requests, take long time. there way faster?

you have 2 other options should considering: import tool , load csv option.

the right question here "how put data neo4j fast" rather "how execute lot of create statements quickly". both of these options way faster doing individual create statements, wouldn't mess individual creates anymore.

michael hunger wrote great blog post describing multiple facets of importing data neo4j, should check out if want understand more why options, not options.

the load csv option going name suggests. you'll use cypher query language load data directly files, , goes substantially faster because commit records in "batches" (the documentation describes this). you're still using transactions data in, you're doing faster, in batches, , while being able create complex relationships along way.

the import tool similar, except it's high performance creates of large volumes of data. magic here (and why it's fast) skips transaction layer. both thing , bad thing, depending on perspective (michael hunger's blog post believe explains tradeoffs).

without knowing data, it's hard make specific recommendation - generality, i'd start load csv default, , move import tool if , if volume of data big, or insert performance requirements intense. reflects slight bias on part transactions thing, , staying @ cypher layer (rather using separate command line tool) thing, ymmv.


Comments

Post a Comment

Popular posts from this blog

Java 3D LWJGL collision -

-
i making 3d java game lwjgl library, , wondering how add collision detection, player not go through models. i using obj models. here objloader class, loads models: package renderengine; import java.io.bufferedreader; import java.io.file; import java.io.filenotfoundexception; import java.io.filereader; import java.util.arraylist; import java.util.list; import models.rawmodel; import org.lwjgl.util.vector.vector2f; import org.lwjgl.util.vector.vector3f; public class objloader { public static rawmodel loadobjmodel(string filename, loader loader){ filereader fr = null; try { fr = new filereader(new file("res/"+filename+".obj")); } catch (filenotfoundexception e) { system.err.println("couldn't load file!"); e.printstacktrace(); } bufferedreader reader = new bufferedreader(fr); string line; list<vector3f> vertices = new arraylist<vector3f...
Read more

spring - SubProtocolWebSocketHandler - No handlers -

-
i have ugly error during deploying spring app on jboss. 18:11:16,025 error [org.apache.catalina.core.containerbase.[jboss.web].[default-host].[/consumer]] (msc service thread 1-7) exception sending context initialized event listener instance of class org.springframework.web.context.contextloaderlistener: org.springframework.context.applicationcontextexception: failed start bean 'subprotocolwebsockethandler'; nested exception java.lang.illegalargumentexception: no handlers @ org.springframework.context.support.defaultlifecycleprocessor.dostart(defaultlifecycleprocessor.java:176) [spring-context-4.1.3.release.jar:4.1.3.release] @ org.springframework.context.support.defaultlifecycleprocessor.access$200(defaultlifecycleprocessor.java:51) [spring-context-4.1.3.release.jar:4.1.3.release] @ org.springframework.context.support.defaultlifecycleprocessor$lifecyclegroup.start(defaultlifecycleprocessor.java:346) [spring-context-4.1.3.release.jar:4.1.3.release] @ org.s...
Read more

methods - python can't use function in submodule -

-
my folder structure this: pythonstuff/ program.py modulea/ __init__.py foo.py bar.py here's code bar.py : def hello(): print("hello, world!") here's code program.py : #!/usr/bin/env python3 modulea import bar bar.hello() i run $ python3 program.py somehow error: file "program.py", line 3, in <module> bar.hello() attributeerror: 'module' object has no attribute 'hello' edit: __init__.py file empty. edit2: after trying realized had bar.py in root directory contained hello() method. bar.py in modulea/ directory empty. add it import os __all__ = [] module in os.listdir(os.path.dirname(__file__)): if module != '__init__.py' , module[-3:] == '.py': __all__.append(module[:-3]) the init .py files required make python treat directories containing packages; done prevent directories common name, such string, unintentionally hiding v...
Read more