Skip to content

Commit 889e61c

Browse files
Andy2003Andy2003
authored
Update README.md (#434)
Add HISTORY.md to document the evolution and key changes of the Neo4j Spatial library update README.md
1 parent 0da2289 commit 889e61c

File tree

3 files changed

+56
-223
lines changed

3 files changed

+56
-223
lines changed

‎HISTORY.md

+30
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
## History
2+
3+
This library began as a collaborative vision between Neo-Technology
4+
and [Craig Taverner](https://github.com/craigtaverner) in early 2010.
5+
The bulk of the initial work was done by [Davide Savazzi](https://github.com/svzdvd) as part of his 2010 Google Summer
6+
of Code (GSoC) project
7+
with Craig as mentor, as a project within the OSGeo GSoC program.
8+
In 2011 and 2012 two further GSoC projects contributed, the last of which saw Davide return as mentor.
9+
10+
The original vision for the library was a comprehensive suite of GIS capabilities somewhat inspired by PostGIS,
11+
while remaining aligned with the graph-nature of the underlying Neo4j database.
12+
To achieve this lofty goal the JTS and GeoTools libraries were used, giving a large suite of capabilities very early on.
13+
14+
However, back in 2010 Neo4j was an embedded database with deployments that saw a low level of concurrent operation.
15+
However, for many years now, Neo4j has been primarily deployed in high concurrent server environments.
16+
Over the years there have been various efforts to make the library more appropriate for use in these environments:
17+
18+
* REST API (early Neo4j 1.x servers had no Cypher)
19+
* IndexProvider mechanism (used during Neo4j 1.x and 2.x server, and removed for Neo4j 3.0)
20+
* 0.23: Addition of Cypher procedures for Neo4j 3.0
21+
* 0.24: Addition of a high performance bulk importer to the in-graph RTree index
22+
* 0.25: Addition of GeoHash indexes for point layers
23+
* 0.26: Support for native Neo4j point types
24+
* 0.27: Major port to Neo4j 4.x which deprecated many of the Neo4j API's the library depended on
25+
* 0.29: Port to Neo4j 5.13
26+
27+
However, despite all these improvements, the core of the library only exposes the rich capabilities of JTS and GeoTools
28+
if used in an embedded environment.
29+
The large effort required to port the library to Neo4j 4.0 resulted in many backwards incompatibilities and
30+
highlighted the need for a new approach to spatial libraries for Neo4j.

‎README.md

+26-218
Original file line numberDiff line numberDiff line change
@@ -5,119 +5,31 @@
55
Neo4j Spatial is a library facilitating the import, storage and querying of spatial data in
66
the [Neo4j open source graph database](http://neo4j.org/).
77

8-
This projects manual is deployed as part of the local build as
9-
the [Neo4j Spatial Manual](http://neo4j-contrib.github.io/spatial).
10-
11-
![Open Street Map](http://neo4j-contrib.github.io/spatial/0.24-neo4j-3.1/images/one-street.png "Open Street Map")
12-
13-
## History
14-
15-
This library began as a collaborative vision between Neo-Technology
16-
and [Craig Taverner](https://github.com/craigtaverner) in early 2010.
17-
The bulk of the initial work was done by [Davide Savazzi](https://github.com/svzdvd) as part of his 2010 Google Summer
18-
of Code (GSoC) project
19-
with Craig as mentor, as a project within the OSGeo GSoC program.
20-
In 2011 and 2012 two further GSoC projects contributed, the last of which saw Davide return as mentor.
21-
22-
The original vision for the library was a comprehensive suite of GIS capabilities somewhat inspired by PostGIS,
23-
while remaining aligned with the graph-nature of the underlying Neo4j database.
24-
To achieve this lofty goal the JTS and GeoTools libraries were used, giving a large suite of capabilities very early on.
25-
26-
However, back in 2010 Neo4j was an embedded database with deployments that saw a low level of concurrent operation.
27-
However, for many years now, Neo4j has been primarily deployed in high concurrent server environments.
28-
Over the years there have been various efforts to make the library more appropriate for use in these environments:
29-
30-
* REST API (early Neo4j 1.x servers had no Cypher)
31-
* IndexProvider mechanism (used during Neo4j 1.x and 2.x server, and removed for Neo4j 3.0)
32-
* 0.23: Addition of Cypher procedures for Neo4j 3.0
33-
* 0.24: Addition of a high performance bulk importer to the in-graph RTree index
34-
* 0.25: Addition of GeoHash indexes for point layers
35-
* 0.26: Support for native Neo4j point types
36-
* 0.27: Major port to Neo4j 4.x which deprecated many of the Neo4j API's the library depended on
37-
* 0.29: Port to Neo4j 5.13
38-
39-
However, despite all these improvements, the core of the library only exposes the rich capabilities of JTS and GeoTools
40-
if used in an embedded environment.
41-
The large effort required to port the library to Neo4j 4.0 resulted in many backwards incompatibilities and
42-
highlighted the need for a new approach to spatial libraries for Neo4j.
43-
44-
## Neo4j 4.x Support
45-
46-
This library was originally written in 2010 when Neo4j was still releasing early 1.x versions.
47-
This means it made use of internal Java API's that were deprecated over the years, some as early as Neo4j 2.x.
48-
When Neo4j 4.0 was released, many deprecated APIs were entirely removed.
49-
And the transaction API was changed in a fundamental way.
50-
This has meant that the spatial library needed a major refactoring to work with Neo4j 4.x:
51-
52-
* The library previously depended on nested transactions. This allowed code that was called
53-
within a transaction (like procedures) to be the same as code that was not (Java API).
54-
The removal of this support required that all internal API's needed to include parameters
55-
for the current transaction, and only the specific surface designed for embedded use not have that.
56-
* The library made use of Lucene based explicit indexes in many places.
57-
The removal of support for explicit indexes required completely new solutions in several places:
58-
* The `OSMImporter` will instead now use normal Neo4j schema indexes (introduced in 2.0).
59-
However, these can only be created in separate index transactions.
60-
Due to the new transaction model this requires stopping the import transaction,
61-
starting an index transaction, and then restarting the import transaction.
62-
All of this is incompatible with procedures, which already have an incoming, non-stoppable transaction.
63-
The solution to this second problem was to run the actual import in another thread.
64-
This has the additional benefit of retaining the original batch-processing capabilities.
65-
The negative consequence of this it that it requires modifying the security model of the procedure context.
66-
* The `ExplicitIndexBackedPointIndex` has been modified to instead use a schema index.
67-
This required similar tricks to those employed in the `OSMImporter` described above.
68-
* Neo4j 4.0 runs only in Java 11, and until recently GeoTools did not support newer Java versions.
69-
It was therefor necessary to upgrade the GeoTools libraries to version 24.2.
70-
This in turn required a re-write of the Neo4jDataStore interface since the older API had
71-
long been deprecated, and was entirely unavailable in newer versions.
72-
* Neo4j 4.1 was slightly stricter in regard to passing nodes as parameters, requiring the nodes
73-
objects to have been created in the current transaction.
74-
To work around this we added `.byId` versions of the `spatial.addNode` and `spatial.removeNode` procedures.
75-
We also changed the `spatial.removeNode` procedures to return `nodeId` instead of `node`.
76-
* The change to Neo4j 4.2 was more subtle. Mostly only internal API's around the use of `Path` instead of `File`.
77-
One change that could be noticed was the `IndexManager.IndexAccessMode` class.
78-
In the 0.27.0 and 0.27.1 versions we used `OverridenAccessMode` to take the users existing access mode
79-
and simply add on the rights to create tokens and indexes. In 0.27.2 we instead use `RestrictedAccessMode`
80-
to restrict the users access right to the built in `AccessModel.Static.SCHEMA` and then boost to enable
81-
index and token writes. The difference is subtle and should only be possible to notice in Enterprise Edition.
82-
* 0.28.0 tackles the ability to import multiple OSM files. The initial solution for Neo4j 4.x made use
83-
of schema indexes keyed by the label and property. However, that means that all OSM imports would share
84-
the same index. If they are completely disjointed data sets, this would not matter. But if you import
85-
overlapping OSM files or different versions of the same file, a mangled partial merger would result.
86-
0.28.0 solves this by using different indexes, and keeping all imports completely separate.
87-
The more complex problems of importing newer versions, and stitching together overlapping areas, are not
88-
yet solved.
89-
* Neo4j 4.3 has an issue with leaking RelationshipTraversalCursor, and we needed to do some workarounds to
90-
avoid this issue, usually by exhausting the iterator, which can have a higher performance cost in some cases.
91-
Version 0.28.1 includes this fix.
92-
93-
Consequences of the port to Neo4j 4.x:
94-
95-
* The large number of changes mean that the 0.27.x versions should be considered very alpha.
96-
* Many API's have changed and client code might need to be adapted to take the changes into account.
97-
* The new DataStore API is entirely untested in GeoServer, besides the existing unit and integration tests.
98-
* The need to manage threads and create schema indexes results in the procedures requiring
99-
unrestricted access to internal API's of Neo4j.
100-
101-
This last point means that you need to set the following in your `neo4j.conf` file:
102-
103-
```
104-
dbms.security.procedures.unrestricted=spatial.*
105-
```
106-
107-
If you are concerned about the security implications of unrestricted access, my best advice is to review
108-
the code and decide for yourself the level of risk you face. See, for example, the
109-
method `IndexAccessMode.withIndexCreate`,
110-
which adds index create capabilities to the security model.
111-
This means that users without index creation privileges will be able to create the necessary spatial support indexes
112-
described above.
113-
This code was not written because we wanted to allow for that case, it was written because in the Neo4j security model,
114-
procedures that can write data (mode=`WRITE`) are not allowed to create indexes.
115-
So this security-fix was required even in the Community Edition of Neo4j.
116-
117-
---
118-
119-
The rest of the README might have information that is no longer accurate for the current version of this library.
120-
Please report any mistakes as issues, or consider raising a pull-request with an appropriate fix.
8+
This project's manual is deployed to the [Neo4j Labs page](https://neo4j.com/labs/neo4j-spatial/5/).
9+
## Versioning
10+
11+
Since version 5.19.0, the versioning of Neo4j Spatial is aligned with the versioning of Neo4j itself.
12+
This means that Neo4j Spatial 5.19.0 is build against Neo4j 5.19, and so on.
13+
14+
## Installation
15+
16+
1. Copy the desired neo4j-spatial-x.x.x-server-plugin.jar from the [release page](https://github.com/neo4j-contrib/spatial/releases) to your neo4j plugin directory.
17+
2. set up your database with the following configuration in your `neo4j.conf` file:
18+
19+
```
20+
dbms.security.procedures.unrestricted=spatial.*
21+
```
22+
23+
NOTE: If you're concerned about the security risks of unrestricted access, we recommend reviewing the code to assess
24+
the level of risk for your use case. For example, the method
25+
[`IndexAccessMode.withIndexCreate`](/src/main/java/org/neo4j/gis/spatial/index/IndexManager.java#L42) grants index
26+
creation capabilities within the security model. This allows users without index creation privileges to generate the
27+
required spatial support indexes. This behavior was not intentionally designed to bypass security but was necessary
28+
due to Neo4j’s security model, where procedures with WRITE mode are not permitted to create indexes.
29+
As a result, this adjustment was required even in the Community Edition of Neo4j.
30+
31+
3. Restart your Neo4j server.
32+
4. Run `CALL spatial.procedures` in the Neo4j browser to see the available spatial procedures.
12133
12234
## Concept Overview
12335
@@ -178,7 +90,7 @@ mvn install
17890
This will download all dependencies, compiled the library, run the tests and install the artifact in your local
17991
repository.
18092
The spatial plugin will also be created in the `target` directory, and can be copied to your local server using
181-
instructions on the spatial server plugin below.
93+
instructions from the [installation section](#installation).
18294
18395
## Layers and GeometryEncoders ##
18496
@@ -377,72 +289,8 @@ at http://oss.infoscience.co.jp/neo4j/wiki.neo4j.org/content/Neo4j_Spatial_in_uD
377289
378290
## Using the Neo4j Spatial Server plugin ##
379291
380-
The Neo4j Spatial Plugin is available for inclusion in the server version of Neo4j 2.x, Neo4j 3.x and Neo4j 4.x.
381-
382-
* Using GeoTools 9.0 (for GeoServer 2.3.2):
383-
* [v0.12 for Neo4j 2.0.4](https://github.com/neo4j-contrib/m2/blob/master/releases/org/neo4j/neo4j-spatial/0.12-neo4j-2.0.4/neo4j-spatial-0.12-neo4j-2.0.4-server-plugin.zip?raw=true)
384-
* [v0.13 for Neo4j 2.1.8](https://github.com/neo4j-contrib/m2/blob/master/releases/org/neo4j/neo4j-spatial/0.13-neo4j-2.1.8/neo4j-spatial-0.13-neo4j-2.1.8-server-plugin.zip?raw=true)
385-
* [v0.14 for Neo4j 2.2.7](https://github.com/neo4j-contrib/m2/blob/master/releases/org/neo4j/neo4j-spatial/0.14-neo4j-2.2.7/neo4j-spatial-0.14-neo4j-2.2.7-server-plugin.zip?raw=true)
386-
* [v0.15.2 for Neo4j 2.3.4](https://github.com/neo4j-contrib/m2/blob/master/releases/org/neo4j/neo4j-spatial/0.15.2-neo4j-2.3.4/neo4j-spatial-0.15.2-neo4j-2.3.4-server-plugin.zip?raw=true)
387-
* [v0.19 for Neo4j 3.0.3](https://github.com/neo4j-contrib/m2/blob/master/releases/org/neo4j/neo4j-spatial/0.19-neo4j-3.0.3/neo4j-spatial-0.19-neo4j-3.0.3-server-plugin.jar?raw=true)
388-
* Using GeoTools 14.4 (for GeoServer 2.8.4):
389-
* [v0.25.1 for Neo4j 3.0.8](https://github.com/neo4j-contrib/m2/blob/master/releases/org/neo4j/neo4j-spatial/0.25.1-neo4j-3.0.8/neo4j-spatial-0.25.1-neo4j-3.0.8-server-plugin.jar?raw=true)
390-
* [v0.25.3 for Neo4j 3.1.4](https://github.com/neo4j-contrib/m2/blob/master/releases/org/neo4j/neo4j-spatial/0.25.3-neo4j-3.1.4/neo4j-spatial-0.25.3-neo4j-3.1.4-server-plugin.jar?raw=true)
391-
* [v0.25.4 for Neo4j 3.2.8](https://github.com/neo4j-contrib/m2/blob/master/releases/org/neo4j/neo4j-spatial/0.25.4-neo4j-3.2.8/neo4j-spatial-0.25.4-neo4j-3.2.8-server-plugin.jar?raw=true)
392-
* [v0.25.5 for Neo4j 3.3.5](https://github.com/neo4j-contrib/m2/blob/master/releases/org/neo4j/neo4j-spatial/0.25.5-neo4j-3.3.5/neo4j-spatial-0.25.5-neo4j-3.3.5-server-plugin.jar?raw=true)
393-
* [v0.26.2 for Neo4j 3.4.12](https://github.com/neo4j-contrib/m2/blob/master/releases/org/neo4j/neo4j-spatial/0.26.2-neo4j-3.4.12/neo4j-spatial-0.26.2-neo4j-3.4.12-server-plugin.jar?raw=true)
394-
* [v0.26.2 for Neo4j 3.5.2](https://github.com/neo4j-contrib/m2/blob/master/releases/org/neo4j/neo4j-spatial/0.26.2-neo4j-3.5.2/neo4j-spatial-0.26.2-neo4j-3.5.2-server-plugin.jar?raw=true)
395-
* Using GeoTools 24.2 (for GeoServer 2.18.x):
396-
* [v0.27.0 for Neo4j 4.0.3](https://github.com/neo4j-contrib/m2/blob/master/releases/org/neo4j/neo4j-spatial/0.27.0-neo4j-4.0.3/neo4j-spatial-0.27.0-neo4j-4.0.3-server-plugin.jar?raw=true)
397-
* [v0.27.1 for Neo4j 4.1.7](https://github.com/neo4j-contrib/m2/blob/master/releases/org/neo4j/neo4j-spatial/0.27.1-neo4j-4.1.7/neo4j-spatial-0.27.1-neo4j-4.1.7-server-plugin.jar?raw=true)
398-
* [v0.27.2 for Neo4j 4.2.3](https://github.com/neo4j-contrib/m2/blob/master/releases/org/neo4j/neo4j-spatial/0.27.2-neo4j-4.2.3/neo4j-spatial-0.27.2-neo4j-4.2.3-server-plugin.jar?raw=true)
399-
* [v0.28.0 for Neo4j 4.2.3](https://github.com/neo4j-contrib/m2/blob/master/releases/org/neo4j/neo4j-spatial/0.28.0-neo4j-4.2.3/neo4j-spatial-0.28.0-neo4j-4.2.3-server-plugin.jar?raw=true)
400-
* [v0.28.1 for Neo4j 4.3.10](https://github.com/neo4j-contrib/m2/blob/master/releases/org/neo4j/neo4j-spatial/0.28.1-neo4j-4.3.10/neo4j-spatial-0.28.1-neo4j-4.3.10-server-plugin.jar?raw=true)
401-
* [v0.28.1 for Neo4j 4.4.3](https://github.com/neo4j-contrib/m2/blob/master/releases/org/neo4j/neo4j-spatial/0.30.0-neo4j-5.13.0/neo4j-spatial-0.30.0-neo4j-5.13.0-server-plugin.jar?raw=true)
402-
403-
For versions up to 0.15-neo4j-2.3.4:
404-
405-
~~~bash
406-
#install the plugin
407-
unzip neo4j-spatial-XXXX-server-plugin.zip -d $NEO4J_HOME/plugins
408-
409-
#start the server
410-
$NEO4J_HOME/bin/neo4j start
411-
412-
#list REST API (edit to correct password)
413-
curl -u neo4j:neo4j http://localhost:7474/db/data/
414-
~~~
415-
416-
For versions for neo4j 3.0 and later:
417-
418-
~~~bash
419-
#install the plugin
420-
cp neo4j-spatial-XXXX-server-plugin.jar $NEO4J_HOME/plugins/
421-
422-
#start the server
423-
$NEO4J_HOME/bin/neo4j start
424-
425-
#list REST API (edit to correct password)
426-
curl -u neo4j:neo4j http://localhost:7474/db/data/
427-
428-
#list spatial procedures (edit to correct password)
429-
curl -u neo4j:neo4j -H "Content-Type: application/json" -X POST -d '{"query":"CALL spatial.procedures"}' http://localhost:7474/db/data/cypher
430-
~~~
431-
432292
The server plugin provides access to the internal spatial capabilities using three APIs:
433293
434-
* A REST API for creating layers and adding nodes or geometries to layers.
435-
* For usage information
436-
see [Neo4j Spatial Manual REST](http://neo4j-contrib.github.io/spatial/#spatial-server-plugin)
437-
* Note that this API provides only limited access to Spatial, with no access the GeoPipes or import utilities
438-
* This API was entirely removed when support for Neo4j 4.0 was added (version 0.27)
439-
* An IndexProvider API (2.x only) for Cypher access using START node=node:geom({query})
440-
* It is only possible to add nodes and query for nodes, and the resulting graph structure is not compatible with any
441-
other spatial API (not compatible with Java API, REST or Procedures), so if you use this approach, do not blend it
442-
with the other approaches.
443-
* There is some brief documentation
444-
at [Finding geometries within distance using cypher](http://neo4j-contrib.github.io/spatial/#rest-api-find-geometries-within--distance-using-cypher)
445-
* This API was removed for 3.0 releases, and so is only available for Neo4j 2.x
446294
* Procedures for much more comprehensive access to spatial from Cypher
447295
* Documentation is not yet available, but you can list the available procedures within Neo4j using the
448296
query `CALL spatial.procedures`
@@ -468,46 +316,6 @@ cd spatial
468316
mvn clean package
469317
~~~
470318
471-
### Building Neo4j Spatial Documentation ###
472-
473-
Add your GitHub credentials in your `~/.m2/settings.xml`
474-
475-
~~~xml
476-
<settings>
477-
<servers>
478-
<server>
479-
<id>github</id>
480-
<username>[email protected]</username>
481-
<password>secret</password>
482-
</server>
483-
</servers>
484-
</settings>
485-
~~~
486-
487-
To build and deploy:
488-
489-
~~~bash
490-
git clone https://github.com/neo4j/spatial.git
491-
cd spatial
492-
mvn clean install site -Pneo-docs-build
493-
~~~
494-
495-
## Using Neo4j spatial in your Java project with Maven ##
496-
497-
Add the following repositories and dependency to your project's pom.xml:
498-
499-
~~~xml
500-
<dependency>
501-
<groupId>org.neo4j</groupId>
502-
<artifactId>neo4j-spatial</artifactId>
503-
<version>5.26.0</version>
504-
</dependency>
505-
~~~
506-
507-
The version specified on the last version line can be changed to match the version you wish to work with (based on the
508-
version of Neo4j itself you are using). To see which versions are available see the list
509-
at [Neo4j Spatial Releases](https://github.com/neo4j-contrib/m2/tree/master/releases/org/neo4j/neo4j-spatial).
510-
511319
## Running Neo4j spatial code from the command-line ##
512320
513321
Some of the classes in Neo4j-Spatial include main() methods and can be run on the command-line.

‎pom.xml

-5
Original file line numberDiff line numberDiff line change
@@ -344,11 +344,6 @@
344344
</dependencies>
345345
</dependencyManagement>
346346
<repositories>
347-
<repository>
348-
<id>java.net</id>
349-
<name>Java.net repo</name>
350-
<url>https://maven.java.net/service/local/repositories/releases/content/</url>
351-
</repository>
352347
<repository>
353348
<id>osgeo</id>
354349
<name>OSGeo Release Repository</name>

0 commit comments

Comments
 (0)