The Neo4j Operations Manual v3 - we-yun.com 3.0 Docs/neo4j-operations... · Chapter 1. Introduction...

The Neo4j Operations Manualv3.0

Table of Contents1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.1. Neo4j editions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.2. Neo4j for the enterprise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.2.1. Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.2.2. Design considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Read scalability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

High availability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

Disaster recovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

Analytics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2. Deployment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2.1. System Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2.2. File locations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

2.2.1. Log Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

2.2.2. Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

2.2.3. Permissions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

2.3. Single instance install . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

2.3.1. Linux installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

Linux Packages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

Unix Console Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

Linux Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2.3.2. OSx installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

Mac OSx Installer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

Running Neo4j from the Terminal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

OSX Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2.3.3. Windows installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

Windows Installer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

Windows Console Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

Windows service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

Windows PowerShell module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

2.3.4. Multiple server instances on one machine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

2.4. Neo4j Cluster install . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

2.4.1. Setup and configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

Important configuration settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

2.4.2. Arbiter instances . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

2.4.3. Endpoints for status information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

The endpoints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

2.4.4. HAProxy for load balancing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

Configuring HAProxy for the Bolt Protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

Configuring HAProxy for the REST API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

Optimizing for reads and writes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

Cache-based sharding with HAProxy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

2.5. Post-install tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

2.5.1. Waiting for Neo4j to start . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

2.5.2. Setting the number of open files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

2.5.3. Setup for remote debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

2.5.4. Usage Data Collector . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

Technical Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

How to disable UDC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

2.6. Upgrading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

2.6.1. Single-instance upgrade. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

Supported upgrade paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

Upgrade instructions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

2.6.2. Neo4j cluster upgrade . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

Back up the Neo4j database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

Shut down the cluster. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

Upgrade the master . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

Upgrade the slaves . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

Restart the cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

2.7. Import tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

2.7.1. CSV file header format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

Nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

Relationships . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

ID spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

2.7.2. Command line usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

Output and statistics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

Verbose error information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

2.7.3. Import tool examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

Basic example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

Customizing configuration options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

Using separate header files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

Multiple input files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

Types and labels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

Property types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

ID handling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

Bad input data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

3. Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

3.1. Securing Neo4j Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

3.1.1. Secure the port and remote client connection accepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

3.1.2. Server authentication and authorization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

3.1.3. HTTPS support. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

3.1.4. Arbitrary code execution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

3.1.5. Server authorization rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

3.1.6. Enforcing Server Authorization Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

3.1.7. Using Wildcards to Target Security Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

3.1.8. Using Complex Wildcards to Target Security Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

3.1.9. Using a proxy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

3.1.10. LOAD CSV . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

4. Backup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

4.1. Introducing backup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

4.2. Performing backups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

4.2.1. Backup commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

4.2.2. Incremental backups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

4.3. Restoring a backup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

5. Monitoring. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

5.1. Adjusting remote JMX access to the Neo4j Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

5.2. How to connect to a Neo4j instance using JMX and JConsole . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

5.3. Reference of supported JMX MBeans . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

5.4. Metrics Reporting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

5.4.1. Introducing Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

5.4.2. Graphite Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

5.4.3. Export to CSV Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

5.4.4. Configuration settings reference for metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

5.4.5. Available metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

6. Performance tuning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

6.1. Modifying configuration settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

6.2. Cypher tuning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

6.3. Memory tuning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

6.3.1. OS memory sizing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

6.3.2. Page cache sizing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

6.3.3. Heap sizing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

6.3.4. Tuning of the garbage collector . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

6.4. Transaction logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

6.5. Compressed property value storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

6.6. Linux file system tuning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

6.7. Disks, RAM and other tips . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

7. Tutorials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

7.1. Set up a Neo4j cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

7.1.1. Important configuration settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

dbms.mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

ha.server_id . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

ha.host.coordination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

ha.initial_hosts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

ha.host.data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

7.1.2. Download and configure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

7.1.3. Start the Neo4j Servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

7.2. Set up a local cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

7.2.1. Download and configure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

Start the Neo4j Servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

8. Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

8.1. Configuration Settings Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

8.1.1. Configuring Bolt Connectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

8.1.2. Configuring HTTP Connectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86

8.2. JMX Beans . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86

8.3. Available metrics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

8.3.1. Java Virtual Machine Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

9. License . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

© 2016 Neo Technology

License: Creative Commons 3.0

1

Chapter 1. IntroductionThis is the operations manual for Neo4j version 3.0, authored by the Neo4j Team.

The main parts of the manual are:

• Introduction — Introducing Neo4j Community and Enterprise Editions.

• Deployment — Instructions on how to deploy Neo4j into production environments.

• Security — Instructions on setting up Neo4j security.

• Backup — Instructions on setting up Neo4j backups.

• Monitoring — Instructions on setting up Neo4j monitoring.

• Performance tuning — Instructions on how to go about performance tuning for Neo4j.

• Tutorials — Step-by-step instructions on various scenarios for setting up Neo4j.

• Configuration Settings Reference — Listings of all Neo4j configuration parameters.

Who should read this?

This manual is written for:

• the engineer performing the Neo4j production deployment.

• the operations engineer supporting and maintaining the Neo4j production database.

• the enterprise architect investigating database options.

• the infrastructure architect planning the Neo4j production deployment.

1.1. Neo4j editions

There are two editions of Neo4j to choose from: Community edition and Enterprise edition. The natureof the required solution will help decide which edition to select.

Community edition is a fully functional edition of Neo4j, suitable for single instance deployments. Ithas full support for key Neo4j features, such as ACID compliance, Cypher, and programming APIs. It isideal for smaller workgroup or do-it-yourself projects similar to:

• learning Neo4j and just getting started

• building a solution for an internal team that can tolerate downtime for support

• building a solution available to external users, but without guarantees on uptime or availability

• building a solution which does not have high demands for scalability or concurrent access

Enterprise edition extends the functionality of Community Edition to include key features forperformance and scalability, such as a clustering architecture for high availability and online backupfunctionality. It is the choice for production systems with availability requirements or needs for scalingup, for example:

• the ability to scale up your solution with the clustering architecture

• 24x7 availability capabilities

• ability to support disaster recovery

• provisioning for early stage load testing

• access to professional support from Neo Technology

Which is the right Neo4j edition for a particular deployment?

2

As a rule of thumb:

1. Both editions offer the same, great core graph database capabilities

2. Enterprise edition is the choice for a commercial solution, a critical or highly depended-on internalsolution, and when anticipate needing scalability, redundancy, or high availability.

Table 1. Features

Edition Enterprise Community

Property Graph Model X X

Native Graph Processing & Storage X X

ACID X X

Cypher - Graph Query Language X X

Language Drivers X X

Extensible REST API X X

High-Performance Native API X X

HTTPS X X

Table 2. Performance & Scalability

Edition Enterprise Community

Enterprise Lock Manager X -

High-Performance Cache X -

Clustering X -

Hot Backups X -

Advanced Monitoring X -

1.2. Neo4j for the enterprise

This section covers the major features of Neo4j Enterprise Edition.

1.2.1. Architecture

Figure 1. Neo4j cluster

Neo4j Clustering is comprised of a single master instance and zero or more slave instances. All

3

instances in the cluster have full copies of your data in their local database files. Each databaseinstance contains the logic needed in order to coordinate with the other members of the cluster fordata replication and election management.

When performing a write transaction on a slave each write operation will be synchronized with themaster. Locks will be acquired on both master and slave. When the transaction commits it will first becommitted on the master and then, if successful, on the slave. To ensure consistency, a slave has to beup to date with the master before performing a write operation. This is built into the communicationprotocol between the slave and master, so that updates will be applied to a slave communicating withits master automatically.

Write transactions performed directly through the master will execute in the same way as running innormal non-cluster mode. On success the transaction will be pushed out to a configurable number ofslaves. This is done optimistically, meaning that if the push fails, the transaction will still be successful.

Whenever a Neo4j database becomes unavailable, by means of for example hardware failure ornetwork outages, the other database instances in the cluster will detect that and mark it astemporarily failed. A database instance that becomes available after being unavailable willautomatically catch up with the cluster. If the master goes down another member will be elected andhave its role switched from slave to master after a quorum has been reached within the cluster. Whenthe new master has performed its role switch it will broadcast its availability to all the other membersof the cluster. Normally a new master is elected and started within just a few seconds and during thistime no writes can take place

A special case of a slave instance is the arbiter instance. The arbiter instance does not operate anydatabase, but can be regarded as cluster participants in that its role is to take part in master electionswith the single purpose of breaking ties in the election process. That makes possible a scenario whereyou have a cluster of two Neo4j database instances plus an arbiter instance, and still enjoy toleranceof a single failure of either of the three instances.

All this can be summarized as:

• Write transactions can be performed on any database instance in a cluster.

• Neo4j cluster is fault tolerant and can continue to operate from any number of machines down toa single machine.

• Slaves will be automatically synchronized with the master on write operations.

• If the master fails, a new master will be elected automatically.

• The cluster automatically handles instances becoming unavailable (for example due to networkissues), and also makes sure to accept them as members in the cluster when they are availableagain.

• Transactions are atomic, consistent and durable but eventually propagated out to other slaves.

• Updates to slaves are eventually consistent by nature but can be configured to be pushedoptimistically from master during commit.

• If the master goes down, any running write transaction will be rolled back and new transactionswill block or fail until a new master has become available.

• Reads are highly available and the ability to handle read load scales with more database instancesin the cluster.

1.2.2. Design considerations

When designing your solution, some of your first considerations will concern your functionalrequirements and the type of technology choices you make to meet them. Some of those functionalrequirements likely will include a need to scale to many concurrent users, maintain consistent uptime,or the ability to recover from a system failure and maintain availability. These are importantproduction related questions that help drive your technical decisions and can ultimately guide you to

4

choose to cluster Neo4j.

This covers four major advantages of using Neo4j clustering:

1. Read Scalability

2. High Availability

3. Disaster Recovery

4. Analytics

Read scalability

Clustering Neo4j allows you to distribute read workload across a number of Neo4j instances. You cantake two approaches to scaling your reads with Neo4j:

Distribute load balance reads to any slave instance in the cluster

Neo4j’s clustering architecture replicates the entire database to each instance in your cluster.Therefore you are able to direct any read from your application to any slave instance without muchconcern for data locality.

Figure 2. Distribute load balance reads to any slave instance in the cluster

When would you chose this method?

1. You need to scale up the number of concurrent read requests

2. Your data has no natural or obvious way of partitioning reads

3. A significant portion of the data that needs to be read can reasonably be expected to already be inmemory on any instance in the cluster.

5

Distribute direct reads to specific instances in the cluster

This is sometimes referred to as "cache-based partitioning". The strategy simply allows you to takeadvantage of natural partitions in your data to direct reads to particular instances where the systemwill already have those datasets in memory. This approach is significantly beneficial when your totalactive dataset is much larger than can fit in memory in any particular instance.

Figure 3. Cache-based partitioning

When would you choose this method?

1. Your total active data set is larger than can reasonably be expected to fit in memory in any singleinstance in your cluster.

2. A natural or obvious partition can be identified in your dataset

3. You have the application and operations ability to direct which instances are read from.

High availability

6

Figure 4. High availability cluster

A significant and fundamental functional requirement for any service or application is therequirements for overall availability. Very often this question is answered more by the demands of theusers, the times they would be interacting with the solution, the impact downtime would have on thebusiness or users of the system to complete their roles, or the financial impact of a system failure.These are not always customer-facing solutions and can be critical internal systems.

Availability can often be addressed with various strategies for recovery or mirroring. However, Neo4j’sclustering architecture is an automated solution for ensuring Neo4j is consistently available to yourapplication and end-users.

How do you know if you need Neo4j’s clustering for high availability reasons?

1. Neo4j is serving data for a critical business or consumer-facing solution that would impact theability for the company to conduct business if the component were down.

2. Global end-users with random access behavior are depending on the data stored in Neo4j.

3. Business continuity must be ensured by availability of disaster recovery features.

Disaster recovery

Disaster recovery, in general terms, defines your ability to recover from major outages of yourservices. The most common example is whole-datacenter outages where many services are disrupted.In these cases a disaster recovery strategy can define a failover datacenter along with a strategy forbringing services back online.

Neo4j clustering can accommodate disaster recovery strategies that require very short-windows ofdowntime or low tolerances for data loss in disaster scenarios. By deploying a cluster instance to analternate location, you have an active copy of your database up and available in your designateddisaster recovery location that is consistently keeping up with the transactions against your database.

Why would you choose Clustering in support of Disaster Recovery?

7

1. Minimize downtime: Your application availability demands are very high and you cannot sustainsignificant periods of downtime.

2. Require real-time: You already employ a disaster recovery strategy for other application or servicecomponents that are near real-time.

3. Minimize data loss: You have a significantly large database that changes frequently and have lowtolerance for data loss in a disaster scenario.

Analytics

Your application needs to access data for its' purposes. It reads data, writes data, and is generallykeeping your application service or end-users happy. Then comes the analytics team that wants tocollect and aggregate data for their reports. Next thing you know, you have a set of long-runningcompute queries running against your production databases and disrupting your service or end-users' happiness.

You can’t avoid servicing the needs of the analytics requests, but you can box in the impact theirqueries have on your service. Neo4j clustering can be used to include separate instances entirely insupport of query analytics, either from end users or from BI tools. Using clustering means the data isalways up to date for analytics queries as well.

When would you decide to use clustering to support analytics needs?

1. You have regular BI users that consistently need to run analytics against the most recent versionsof the data

2. Your analytics includes queries that aggregate over large or entire sets of data

3. Your analytics processes include complex compute algorithms for predictive or modeling purposes

8

Chapter 2. Deployment

2.1. System Requirements

CPU

Performance is generally memory or I/O bound for large graphs, and compute bound for graphs thatfit in memory.

Minimum

Intel Core i3

Recommended

Intel Core i7

IBM POWER8

Memory

More memory allows for larger graphs, but it needs to be configured properly to avoid disruptivegarbage collection operations. See Memory tuning for suggestions.

Minimum

2GB

Recommended

16—32GB or more

Disk

Aside from capacity, the performance characteristics of the disk are the most important whenselecting storage. Neo4j workloads tend significantly toward random reads. Select media with lowaverage seek time: SSD over spinning disks. Consult Disks, RAM and other tips for more details.

Minimum

10GB SATA

Recommended

SSD w/ SATA

Filesystem

For proper ACID behavior, the filesystem must support flush (fsync, fdatasync). See Linux file systemtuning for a discussion on how to configure the filesystem in Linux for optimal performance.

Minimum

ext4 (or similar)

Recommended

ext4, ZFS

Software

9

Neo4j requires a Java Virtual Machine to operate. Community Edition installers for Windows and Macinclude a JVM for convenience. Other distributions, including all distributions of Neo4j EnterpriseEdition, require a pre-installed JVM.

Java

OpenJDK 8 (http://openjdk.java.net/) or Oracle Java 8(http://www.oracle.com/technetwork/java/javase/downloads/index.html)

IBM Java 8 (http://www.ibm.com/developerworks/java/jdk/)

Operating Systems

Linux, HP-UX, Windows Server 2012 for production

Additionally, Windows XP and Mac OS X for development

Architectures

x86

OpenPOWER (POWER8)

2.2. File locations

This table shows where important files can be found by default in various Neo4j distribution packages.

Package Configuration

Data Logs Metrics Import Bin Lib Plugins

Linux or OSX tarball

<neo4j-home>/conf/neo4j.conf

<neo4j-home>/data

<neo4j-home>/logs

<neo4j-home>/metrics

<neo4j-home>/import

<neo4j-home>/bin

<neo4j-home>/lib

<neo4j-home>/plugins

Windowszip

<neo4j-home>\conf\neo4j.conf

<neo4j-home>\data

<neo4j-home>\logs

<neo4j-home>\metrics

<neo4j-home>\import

<neo4j-home>\bin

<neo4j-home>\lib

<neo4j-home>\plugins

Debian/Ubuntu .deb

/etc/neo4j/neo4j.conf

/var/lib/neo4j/data

/var/log/neo4j

/var/lib/neo4j/metrics

/var/lib/neo4j/import

/var/lib/neo4j/bin

/var/lib/neo4j/lib

/var/lib/neo4j/plugins

Windowsdesktop

%APPDATA%\Neo4jCommunityEdition\neo4j.conf

%APPDATA%\Neo4jCommunityEdition

%APPDATA%\Neo4jCommunityEdition\logs

%APPDATA%\Neo4jCommunityEdition\metrics

%APPDATA%\Neo4jCommunityEdition\import

%ProgramFiles%\Neo4jCE 3.0\bin

(in package) %ProgramFiles%\Neo4jCE3.0\plugins

OS Xdesktop

${HOME}/Documents/Neo4j/neo4j.conf

${HOME}/Documents/Neo4j

${HOME}/Documents/Neo4j/logs

${HOME}/Documents/Neo4j/metrics

${HOME}/Documents/Neo4j/import

(in package) (in package) (in package)

Please note that the data directory is internal to Neo4j and its structure subject to change betweenversions without notice.

2.2.1. Log Files

Filename Description

neo4j.log The standard log, where general information about Neo4jis written.

debug.log Information useful when debugging problems with Neo4j.

http.log Request log for the HTTP API.

gc.log Garbage Collection logging provided by the JVM.

10

http://openjdk.java.net/

http://www.oracle.com/technetwork/java/javase/downloads/index.html

http://www.ibm.com/developerworks/java/jdk/

Filename Description

query.log Log of executed queries that takes longer than a specifiedthreshold. (Enterprise only.)

2.2.2. Configuration

Some of these paths are configurable with dbms.directories.* settings; see Configuration SettingsReference for details.

The locations of <neo4j-home>, bin and conf can be configured using environment variables.

Location Default Environment variable Notes

<neo4j-home> parent of bin NEO4J_HOME Must be set explicitly if bin isnot a subdirectory.

bin directory where neo4j scriptis located

NEO4J_BIN Must be set explicitly if neo4jscript is invoked as asymlink.

conf <neo4j-home>/conf NEO4J_CONF Must be set explicitly if it isnot a subdirectory of <neo4j-home>.

2.2.3. Permissions

The user that Neo4j runs as must have the following permissions:

Read only

• conf

• import

• bin

• lib

• plugins

Read and write

• data

• logs

• metrics

Exectute

• all files in bin

2.3. Single instance install

2.3.1. Linux installation

Linux Packages

• For Debian packages, see the instructions at http://debian.neo4j.org/.

After installation you may have to do some platform specific configuration and performance tuning.For that, refer to Post-install tasks.

11

http://debian.neo4j.org/

Unix Console Application

1. Download the latest release from http://neo4j.com/download/.

• Select the appropriate tar.gz distribution for your platform.

2. Extract the contents of the archive, using: tar -xf <filename>

• Refer to the top-level extracted directory as: NEO4J_HOME

3. Change directory to: $NEO4J_HOME

• Run: ./bin/neo4j console

4. Stop the server by typing Ctrl-C in the console.

Linux Service

The neo4j command can also be used with start, stop, restart or status instead of console. By usingthese actions, you can create a Neo4j service. See the neo4j man page for further details.

This approach to running Neo4j as a service is deprecated. We strongly advise youto run Neo4j from a package where feasible.

You can build your own init.d script. See for instance the Linux Standard Base specification onsystem initialization (http://refspecs.linuxfoundation.org/LSB_3.1.0/LSB-Core-generic/LSB-Core-

generic/tocsysinit.html), or one of the many samples (https://gist.github.com/chrisvest/7673244) and tutorials(http://www.linux.com/learn/tutorials/442412-managing-linux-daemons-with-init-scripts).

2.3.2. OSx installation

Mac OSx Installer

1. Download the .dmg installer that you want from http://neo4j.com/download/.

2. Click the downloaded installer file.

3. Drag the Neo4j icon into the Applications folder.

If you install Neo4j using the Mac installer and already have an existing instance ofNeo4j the installer will ensure that both the old and new versions can co-exist onyour system.

Running Neo4j from the Terminal

The server can be started in the background from the terminal with the command neo4j start, andthen stopped again with neo4j stop. The server can also be started in the foreground with neo4jconsole — then it’s log output will be printed to the terminal.

OSX Service

Use the standard OSX system tools to create a service based on the neo4j command.

2.3.3. Windows installation

12

http://neo4j.com/download/

http://refspecs.linuxfoundation.org/LSB_3.1.0/LSB-Core-generic/LSB-Core-generic/tocsysinit.html

https://gist.github.com/chrisvest/7673244

http://www.linux.com/learn/tutorials/442412-managing-linux-daemons-with-init-scripts


Windows Installer

1. Download the version that you want from http://neo4j.com/download/.

• Select the appropriate version and architecture for your platform.

2. Double-click the downloaded installer file.

3. Follow the prompts.

The installer will prompt to be granted Administrator privileges. Newer versions ofWindows come with a SmartScreen feature that may prevent the installer fromrunning — you can make it run anyway by clicking "More info" on the "Windowsprotected your PC" screen.

If you install Neo4j using the windows installer and you already have an existinginstance of Neo4j the installer will select a new install directory by default. If youspecify the same directory it will ask if you want to upgrade. This should proceedwithout issue although some users have reported a JRE is damaged error. If you seethis error simply install Neo4j into a different location.

Windows Console Application

1. Download the latest release from http://neo4j.com/download/.

• Select the appropriate Zip distribution.

2. Right-click the downloaded file, click Extract All.

3. Change directory to top-level extracted directory.

• Run bin\neo4j console

4. Stop the server by typing Ctrl-C in the console.

Windows service

Neo4j can also be run as a Windows service. Install the service with bin\neo4j install-service andstart it with bin\neo4j start. Other commands available are stop, restart, status and uninstall-service.

Windows PowerShell module

The Neo4j PowerShell module allows administrators to:

• install, start and stop Neo4j Windows® Services

• and start tools, such as Neo4j Shell and Neo4j Import.

The PowerShell module is installed as part of the ZIP file (http://neo4j.com/download/) distributions ofNeo4j.

System Requirements

• Requires PowerShell v2.0 or above.

• Supported on either 32 or 64 bit operating systems.

13




Managing Neo4j on Windows

On Windows it is sometimes necessary to Unblock a downloaded zip file before you can import itscontents as a module. If you right-click on the zip file and choose "Properties" you will get a dialog.Bottom-right on that dialog you will find an "Unblock" button. Click that. Then you should be able toimport the module.

Running scripts has to be enabled on the system. This can for example be achieved by executing thefollowing from an elevated PowerShell prompt:

Set-ExecutionPolicy -ExecutionPolicy RemoteSigned

For more information see About execution policies (https://technet.microsoft.com/en-

us/library/hh847748.aspx).

The powershell module will display a warning if it detects that you do not have administrative rights.

How do I import the module?

The module file is located in the bin directory of your Neo4j installation, i.e. where you unzipped thedownloaded file. For example, if Neo4j was installed in C:\Neo4j then the module would be importedlike this:

Import-Module C:\Neo4j\bin\Neo4j-Management.psd1

This will add the module to the current session.

Once the module has been imported you can start an interactive console version of a Neo4j Serverlike this:

Invoke-Neo4j console

To stop the server, issue Ctrl-C in the console window that was created by the command.

How do I get help about the module?

Once the module is imported you can query the available commands like this:

Get-Command -Module Neo4j-Management

The output should be similar to the following:

CommandType Name Version Source----------- ---- ------- ------Function Invoke-Neo4j 3.0.0 Neo4j-ManagementFunction Invoke-Neo4jAdmin 3.0.0 Neo4j-ManagementFunction Invoke-Neo4jBackup 3.0.0 Neo4j-ManagementFunction Invoke-Neo4jImport 3.0.0 Neo4j-ManagementFunction Invoke-Neo4jShell 3.0.0 Neo4j-Management

The module also supports the standard PowerShell help commands.

Get-Help Invoke-Neo4j

14

https://technet.microsoft.com/en-us/library/hh847748.aspx

To see examples for a command, do like this:

Get-Help Invoke-Neo4j -examples

Example usage

• List of available commands:

Invoke-Neo4j

• Current status of the Neo4j service:

Invoke-Neo4j status

• Install the service with verbose output:

Invoke-Neo4j install-service -Verbose

• Available commands for administrative tasks:

Invoke-Neo4jAdmin

Common PowerShell parameters

The module commands support the common PowerShell parameter of Verbose.

2.3.4. Multiple server instances on one machine

Neo4j can be configured to run as several instances on one machine. This might be done to runseveral databases for testing or development. This is not recommended for a production deployment.

For how to set this up, see Set up a local cluster. Just use the Neo4j edition of your choice, follow theguide and remember to not set the servers to run in HA mode.

2.4. Neo4j Cluster install

2.4.1. Setup and configuration

Neo4j can be configured in cluster mode to accommodate differing requirements for load, faulttolerance and available hardware. Refer to design considerations for a discussion on different designoptions.

Follow these steps in order to configure a Neo4j cluster:

1. Download and install the Neo4j Enterprise Edition on each of the servers to be included in thecluster.

2. If applicable, decide which server(s) that are to be configured as arbiter instance(s).

3. Edit the Neo4j configuration file on each of the servers to accommodate the design decisions.

4. Follow installation instructions for a single instance install.

15

5. Modify the configuration files on each server as outlined in the section below. There are manyparameters that can be modified to achieve a certain behavior. However, the only ones mandatoryfor an initial cluster are: dbms.mode, ha.server_id and ha.initial_hosts.

Important configuration settings

Each instance in a Neo4j HA cluster must be assigned an integer ID, which serves as its uniqueidentifier. At startup, a Neo4j instance contacts the other instances specified in the ha.initial_hostsconfiguration option.

When an instance establishes a connection to any other, it determines the current state of the clusterand ensures that it is eligible to join. To be eligible the Neo4j instance must host the same databasestore as other members of the cluster (although it is allowed to be in an older state), or be a newdeployment without a database store.

Please note that IP Addresses or Hostnames should be explicitly configured for the machinesparticipating in the cluster. Neo4j will attempt to configure IP addresses for itself in the absence ofexplicit configuration.

dbms.mode

dbms.mode configures the operating mode of the database.

For cluster mode it is set to: dbms.mode=HA

ha.server_id

ha.server_id is the cluster identifier for each instance. It must be a positive integer and must beunique among all Neo4j instances in the cluster.

For example, ha.server_id=1.

ha.host.coordination

ha.host.coordination is an address/port setting that specifies where the Neo4j instance will listen forcluster communications (like hearbeat messages). The default port is 5001. In the absence of aspecified IP address, Neo4j will attempt to find a valid interface for binding. While this behaviortypically results in a well-behaved server, it is strongly recommended that users explicitly choose an IPaddress bound to the network interface of their choosing to ensure a coherent cluster deployment.

For example, ha.host.coordination=192.168.33.22:5001 will listen for cluster communications on thenetwork interface bound to the 192.168.33.0 subnet on port 5001.

ha.initial_hosts

ha.initial_hosts is a comma separated list of address/port pairs, which specify how to reach otherNeo4j instances in the cluster (as configured via their ha.host.coordination option). Thesehostname/ports will be used when the Neo4j instances start, to allow them to find and join thecluster. Specifying an instance’s own address is permitted. Do not use any whitespace in thisconfiguration option.

For example, ha.initial_hosts=192.168.33.22:5001,192.168.33.21:5001 will attempt to reach Neo4jinstances listening on 192.168.33.22 on port 5001 and 192.168.33.21 on port 5001 on the192.168.33.0 subnet.

16

ha.host.data

ha.host.data is an address/port setting that specifies where the Neo4j instance will listen fortransactions from the cluster master. The default port is 6001. In the absence of a specified IP address,Neo4j will attempt to find a valid interface for binding. While this behavior typically results in a well-behaved server, it is strongly recommended that users explicitly choose an IP address bound to thenetwork interface of their choosing to ensure a coherent cluster topology.

ha.host.data must use a different port to ha.host.coordination.

For example, ha.host.data=192.168.33.22:6001 will listen for transactions from the cluster master onthe network interface bound to the 192.168.33.0 subnet on port 6001.

Address and port formats

The ha.host.coordination and ha.host.data configuration options are specified as<IP address>:<port>.

For ha.host.data the IP address must be the address assigned to one of the host’snetwork interfaces.

For ha.host.coordination the IP address must be the address assigned to one of thehost’s network interfaces, or the value 0.0.0.0, which will cause Neo4j to listen onevery network interface.

Either the address or the port can be omitted, in which case the default for that partwill be used. If the address is omitted, then the port must be preceded with a colon(eg. :5001).

The syntax for setting the port range is: <hostname>:<first port>[-<second port>].In this case, Neo4j will test each port in sequence, and select the first that isunused. Note that this usage is not permitted when the hostname is specified as0.0.0.0 (the "all interfaces" address).

For a hands-on tutorial for setting up a Neo4j cluster, see Set up a Neo4j cluster.

Review the Configuration Settings Reference section for a list of all available configuration settings.

2.4.2. Arbiter instances

A typical deployment of Neo4j will use a cluster of 3 machines to provide fault-tolerance and readscalability. This setup is described in Set up a Neo4j cluster.

While having at least 3 instances is necessary for failover to happen in case the master becomesunavailable, it is not required for all instances to run the full Neo4j stack. Instead, something calledarbiter instances can be deployed. They are regarded as cluster participants in that their role is to takepart in master elections with the single purpose of breaking ties in the election process. That makespossible a scenario where you have a cluster of 2 Neo4j database instances and an additional arbiterinstance and still enjoy tolerance of a single failure of either of the 3 instances.

Arbiter instances are configured in neo4j.conf using the same settings as standard Neo4j clustermembers. The instance is configured to be an arbiter by setting the dbms.mode option to ARBITER.Settings that are not cluster specific are of course ignored, so you can easily start up an arbiterinstance in place of a properly configured Neo4j instance.

To start the arbiter instance, run neo4j as normal:

neo4j_home$ ./bin/neo4j start

17

You can stop, install and remove it as a service and ask for its status in exactly the same way as forother Neo4j instances.

2.4.3. Endpoints for status information

Introduction

A common use case for Neo4j HA clusters is to direct all write requests to the master while usingslaves for read operations, distributing the read load across the cluster and and gain failovercapabilities for your deployment. The most common way to achieve this is to place a load balancer infront of the HA cluster, an example being shown with HA Proxy. As you can see in that guide, it makesuse of a REST endpoint to discover which instance is the master and direct write load to it. In thissection, we’ll deal with this REST endpoint and explain its semantics.

The endpoints

Each HA instance comes with 3 endpoints regarding its HA status. They are complimentary but eachmay be used depending on your load balancing needs and your production setup. Those are:

• /db/manage/server/ha/master

• /db/manage/server/ha/slave

• /db/manage/server/ha/available

The /master and /slave endpoints can be used to direct write and non-write traffic respectively tospecific instances. This is the optimal way to take advantage of Neo4j’s scaling characteristics. The/available endpoint exists for the general case of directing arbitrary request types to instances that areavailable for transaction processing.

To use the endpoints, perform an HTTP GET operation on either and the following will be returned:

Table 3. HA REST endpoint responses

Endpoint Instance State Returned Code Body text

/db/manage/server/ha/master

Master 200 OK true

Slave 404 Not Found false

Unknown 404 Not Found UNKNOWN

/db/manage/server/ha/slave

Master 404 Not Found false

Slave 200 OK true


/db/manage/server/ha/available

Master 200 OK master

Slave 200 OK slave


Examples

From the command line, a common way to ask those endpoints is to use curl. With no arguments, curlwill do an HTTP GET on the URI provided and will output the body text, if any. If you also want to getthe response code, just add the -v flag for verbose output. Here are some examples:

• Requesting master endpoint on a running master with verbose output

18

#> curl -v localhost:7474/db/manage/server/ha/master* About to connect() to localhost port 7474 (#0)* Trying ::1...* connected* Connected to localhost (::1) port 7474 (#0)> GET /db/manage/server/ha/master HTTP/1.1> User-Agent: curl/7.24.0 (x86_64-apple-darwin12.0) libcurl/7.24.0 OpenSSL/0.9.8r zlib/1.2.5> Host: localhost:7474> Accept: */*>< HTTP/1.1 200 OK< Content-Type: text/plain< Access-Control-Allow-Origin: *< Transfer-Encoding: chunked< Server: Jetty(6.1.25)<* Connection #0 to host localhost left intacttrue* Closing connection #0

• Requesting slave endpoint on a running master without verbose output:

#> curl localhost:7474/db/manage/server/ha/slavefalse

• Finally, requesting the master endpoint on a slave with verbose output

#> curl -v localhost:7475/db/manage/server/ha/master* About to connect() to localhost port 7475 (#0)* Trying ::1...* connected* Connected to localhost (::1) port 7475 (#0)> GET /db/manage/server/ha/master HTTP/1.1> User-Agent: curl/7.24.0 (x86_64-apple-darwin12.0) libcurl/7.24.0 OpenSSL/0.9.8r zlib/1.2.5> Host: localhost:7475> Accept: */*>< HTTP/1.1 404 Not Found< Content-Type: text/plain< Access-Control-Allow-Origin: *< Transfer-Encoding: chunked< Server: Jetty(6.1.25)<* Connection #0 to host localhost left intactfalse* Closing connection #0

Unknown status

The UNKNOWN status exists to describe when a Neo4j instance is neither master norslave. For example, the instance could be transitioning between states (master toslave in a recovery scenario or slave being promoted to master in the event offailure), or the instance could be an arbiter instance. If the UNKNOWN status isreturned, the client should not treat the instance as a master or a slave and shouldinstead pick another instance in the cluster to use, wait for the instance to transitfrom the UNKNOWN state, or undertake restorative action via systems admin.

If the Neo4j server has Basic Security enabled, the HA status endpoints will also require authenticationcredentials. For some load balancers and proxy servers, providing this with the request is not anoption. For those situations, consider disabling authentication of the HA status endpoints by settingdbms.security.ha_status_auth_enabled=false in the neo4j.conf configuration file.

2.4.4. HAProxy for load balancing

In the Neo4j HA architecture, the cluster is typically fronted by a load balancer. In this section we willexplore how to set up HAProxy to perform load balancing across the HA cluster.

19

For this tutorial we will assume a Linux environment with HAProxy already installed. Seehttp://www.haproxy.org/ for downloads and installation instructions.

Configuring HAProxy for the Bolt Protocol

In a typical HA deployment, HAProxy will be configured with two open ports, one for routing writeoperations to the master and one for load balancing read operations over slaves. Each application willhave two driver instances, one connected to the master port for performing writes and one connectedto the slave port for performing reads.

Let’s first set up the mode and timeouts. The settings below will kill the connection if a server or aclient is idle for longer than two hours. Long-running queries may take longer time, but this can betaken care of by enabling HAProxy’s TCP heartbeat feature.

defaults mode tcp

timeout connect 30s

timeout client 2h timeout server 2h

Set up where drivers wanting to perform writes will connect:

frontend neo4j-write bind *:7680 default_backend current-master

Now, let’s set up the backend that points to the current master instance.

backend current-master option httpchk HEAD /db/manage/server/ha/master HTTP/1.0

server db01 10.0.1.10:7687 check port 7474 server db02 10.0.1.11:7687 check port 7474 server db03 10.0.1.12:7687 check port 7474

In the example above httpchk is configured in the way you would do it if authentication has beendisabled for Neo4j. By default however, authentication is enabled and you will need to pass in anauthentication header. This would be along the lines of option httpchk HEAD/db/manage/server/ha/master HTTP/1.0\r\nAuthorization:\ Basic\ bmVvNGo6bmVvNGo= where the lastpart has to be replaced with a base64 encoded value for your username and password.

Configure where drivers wanting to perform reads will connect:

frontend neo4j-read bind *:7681 default_backend slaves

Finally, configure a backend that points to slaves in a round-robin fashion:

backend slaves balance roundrobin option httpchk HEAD /db/manage/server/ha/slave HTTP/1.0

server db01 10.0.1.10:7687 check port 7474 server db02 10.0.1.11:7687 check port 7474 server db03 10.0.1.12:7687 check port 7474

20

http://www.haproxy.org/

Note that the servers in the slave backend are configured the same way as in the current-masterbackend.

Then by putting all the above configurations into one file, we get a basic workable HAProxyconfiguration to perform load balancing for applications using the Bolt Protocol.

By default, encryption is enabled between servers and drivers. With encryption turned on, theHAProxy configuration constructed above needs no change to work directly in TLS/SSL passthroughlayout for HAProxy. However depending on the driver authentication strategy adopted, some specialrequirements might apply to the server certificates.

For drivers using trust-on-first-use authentication strategy, each driver would register the HAProxyport it connects to with the first certificate received from the cluster. Then for all subsequentconnections, the driver would only establish connections with the server whose certificate is the sameas the one registered. Therefore, in order to make it possible for a driver to establish connections withall instances in the cluster, this mode requires all the instances in the cluster sharing the samecertificate.

If drivers are configured to run in trusted-certificate mode, then the certificate known to the driversshould be a root certificate to all the certificates installed on the servers in the cluster. Alternatively,for the drivers such as Java driver who supports registering multiple certificates as trusted certificates,the drivers also work well with a cluster if server certificates used in the cluster are all registered astrusted certificates.

To use HAProxy with other encryption layout, please refer to their full documentation at their website.

Configuring HAProxy for the REST API

HAProxy can be configured in many ways. The full documentation is available at their website.

For this example, we will configure HAProxy to load balance requests to three HA servers. Simply writethe following configuration to /etc/haproxy.cfg:

global daemon maxconn 256

defaults mode http timeout connect 5000ms timeout client 50000ms timeout server 50000ms

frontend http-in bind *:80 default_backend neo4j

backend neo4j option httpchk GET /db/manage/server/ha/available server s1 10.0.1.10:7474 maxconn 32 server s2 10.0.1.11:7474 maxconn 32 server s3 10.0.1.12:7474 maxconn 32

listen admin bind *:8080 stats enable

HAProxy can now be started by running:

/usr/sbin/haproxy -f /etc/haproxy.cfg

You can connect to http://<ha-proxy-ip>:8080/haproxy?stats to view the status dashboard. Thisdashboard can be moved to run on port 80, and authentication can also be added. See the HAProxy

21

http://<ha-proxy-ip>:8080/haproxy?stats

documentation for details on this.

Optimizing for reads and writes

Neo4j provides a catalogue of health check URLs (see Endpoints for status information) that HAProxy(or any load balancer for that matter) can use to distinguish machines using HTTP response codes. Inthe example above we used the /available endpoint, which directs requests to machines that aregenerally available for transaction processing (they are alive!).

However, it is possible to have requests directed to slaves only, or to the master only. If you are able todistinguish in your application between requests that write, and requests that only read, then you cantake advantage of two (logical) load balancers: one that sends all your writes to the master, and onethat sends all your read-only requests to a slave. In HAProxy you build logical load balancers by addingmultiple backends.

The trade-off here is that while Neo4j allows slaves to proxy writes for you, this indirectionunnecessarily ties up resources on the slave and adds latency to your write requests. Conversely, youdon’t particularly want read traffic to tie up resources on the master; Neo4j allows you to scale out forreads, but writes are still constrained to a single instance. If possible, that instance should exclusivelydo writes to ensure maximum write performance.

The following example excludes the master from the set of machines using the /slave endpoint.



frontend http-in bind *:80 default_backend neo4j-slaves

backend neo4j-slaves option httpchk GET /db/manage/server/ha/slave server s1 10.0.1.10:7474 maxconn 32 check server s2 10.0.1.11:7474 maxconn 32 check server s3 10.0.1.12:7474 maxconn 32 check


In practice, writing to a slave is uncommon. While writing to slaves has the benefitof ensuring that data is persisted in two places (the slave and the master), it comesat a cost. The cost is that the slave must immediately become consistent with themaster by applying any missing transactions and then synchronously apply the newtransaction with the master. This is a more expensive operation than writing to themaster and having the master push changes to one or more slaves.

Cache-based sharding with HAProxy

Neo4j HA enables what is called cache-based sharding. If the dataset is too big to fit into the cache ofany single machine, then by applying a consistent routing algorithm to requests, the caches on eachmachine will actually cache different parts of the graph. A typical routing key could be user ID.

In this example, the user ID is a query parameter in the URL being requested. This will route the sameuser to the same machine for each request.

22



frontend http-in bind *:80 default_backend neo4j-slaves

backend neo4j-slaves balance url_param user_id server s1 10.0.1.10:7474 maxconn 32 server s2 10.0.1.11:7474 maxconn 32 server s3 10.0.1.12:7474 maxconn 32


Naturally the health check and query parameter-based routing can be combined to only routerequests to slaves by user ID. Other load balancing algorithms are also available, such as routing bysource IP (source), the URI (uri) or HTTP headers(hdr()).

2.5. Post-install tasks

2.5.1. Waiting for Neo4j to start

After starting Neo4j it may take some time before the database is ready to serve requests. Systemsthat depend on the database should be able to retry if it is unavailable in order to cope with networkglitches and other brief outages. To specifically wait for Neo4j to be available after starting, poll theBolt or HTTP endpoint until it gives a successful response.

The details of how to poll depend:

• Whether the client uses HTTP or Bolt.

• Whether encryption or authentication are enabled.

It’s important to include a timeout in case Neo4j fails to start. Normally ten seconds should besufficient, but database recovery or upgrade may take much longer depending on the size of thestore. If the instance is part of a cluster then the endpoint will not be available until other instanceshave started up and the cluster has formed.

Here is an example of polling written in Bash using the HTTP endpoint, with encryption andauthentication disabled.

end="$((SECONDS+10))"while true; do [[ "200" = "$(curl --silent --write-out %{http_code} --output /dev/null http://localhost:7474)" ]] &&break [[ "${SECONDS}" -ge "${end}" ]] && exit 1 sleep 1done

2.5.2. Setting the number of open files

Linux platforms impose an upper limit on the number of concurrent files a user may have open. Thisnumber is reported for the current user and session with the ulimit -n command:

23

user@localhost:~$ ulimit -n1024

The usual default of 1024 is often not enough. This is especially true when many indexes are used or aserver installation sees too many connections. Network sockets count against the limit as well. Usersare therefore encouraged to increase the limit to a healthy value of 40 000 or more, depending onusage patterns. It is possible to set the limit with the ulimit command, but only for the root user, andit only affects the current session. To set the value system wide, follow the instructions for yourplatform.

What follows is the procedure to set the open file descriptor limit to 40 000 for user neo4j underUbuntu 10.04 and later.

If you opted to run the neo4j service as a different user, change the first field in step2 accordingly.

1. Become root, since all operations that follow require editing protected system files.

user@localhost:~$ sudo su -Password:root@localhost:~$

2. Edit /etc/security/limits.conf and add these two lines:

neo4j soft nofile 40000neo4j hard nofile 40000

3. Edit /etc/pam.d/su and uncomment or add the following line:

session required pam_limits.so

4. A restart is required for the settings to take effect.

After the above procedure, the neo4j user will have a limit of 40 000 simultaneous open files. Ifyou continue experiencing exceptions on Too many open files or Could not stat() directory, youmay have to raise the limit further.

2.5.3. Setup for remote debugging

In order to configure the Neo4j server for remote debugging sessions, the Java debugging parametersneed to be passed to the Java process through the configuration. They live in the conf/neo4j-wrapper.properties file.

In order to specify the parameters, add a line for the additional Java arguments like this:

dbms.jvm.additional=-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005

This configuration will start a Neo4j server ready for remote debugging attachement at localhost andport 5005. Use these parameters to attach to the process from Eclipse, IntelliJ or your remotedebugger of choice after starting the server.

24

2.5.4. Usage Data Collector

The Neo4j Usage Data Collector is a sub-system that gathers usage data, reporting it to the UDC-server at udc.neo4j.org. It is easy to disable, and does not collect any data that is confidential. Formore information about what is being sent, see below.

The Neo4j team uses this information as a form of automatic, effortless feedback from the Neo4jcommunity. We want to verify that we are doing the right thing by matching download statistics withusage statistics. After each release, we can see if there is a larger retention span of the serversoftware.

The data collected is clearly stated here. If any future versions of this system collect additional data,we will clearly announce those changes.

The Neo4j team is very concerned about your privacy. We do not disclose any personally identifiableinformation.

Technical Information

To gather good statistics about Neo4j usage, UDC collects this information:

• Kernel version: The build number, and if there are any modifications to the kernel.

• Store id: A randomized globally unique id created at the same time a database is created.

• Ping count: UDC holds an internal counter which is incremented for every ping, and reset for everyrestart of the kernel.

• Source: This is either "neo4j" or "maven". If you downloaded Neo4j from the Neo4j website, it’s"neo4j", if you are using Maven to get Neo4j, it will be "maven".

• Java version: The referrer string shows which version of Java is being used.

• Registration id: For registered server instances.

• Tags about the execution context (e.g. test, language, web-container, app-container, spring, ejb).

• Neo4j Edition (community, enterprise).

• A hash of the current cluster name (if any).

• Distribution information for Linux (rpm, dpkg, unknown).

• User-Agent header for tracking usage of REST client drivers

• MAC address to uniquely identify instances behind firewalls.

• The number of processors on the server.

• The amount of memory on the server.

• The JVM heap size.

• The number of nodes, relationships, labels and properties in the database.

After startup, UDC waits for ten minutes before sending the first ping. It does this for two reasons;first, we don’t want the startup to be slower because of UDC, and secondly, we want to keep pingsfrom automatic tests to a minimum. The ping to the UDC servers is done with a HTTP GET.

How to disable UDC

UDC is easily turned off by disabling it in the database configuration, in neo4j.conf for Neo4j server orin the configuration passed to the database in embedded mode. See UDC Configuration in theconfiguration section for details.

25

2.6. Upgrading

2.6.1. Single-instance upgrade

This section describes upgrading a single Neo4j instance. To upgrade a Neo4j HA cluster (Neo4jEnterprise), a very specific procedure must be followed. Please see Neo4j cluster upgrade.

Throughout this instruction, the files used to store the Neo4j data are referred to as database files.These files is are found in the directory specified by dbms.directories.data in neo4j.conf.

Disk space requirements

An upgrade requires substantial free disk space, as it makes an entire copy of thedatabase. The upgraded database may also require larger data files overall.

It is recommended to make available an extra 50% disk space on top of the existingdatabase files.

In addition to this, don’t forget to reserve the disk space needed for the pre-upgrade backup.

Supported upgrade paths

Before upgrading to a new major or minor release, the database must first be upgraded to the latestversion within the relevant release. The latest version is available at this page:http://neo4j.com/download/other-releases. The following Neo4j upgrade paths are supported:

• 2.0.latest → 3.0.1

• 2.1.latest → 3.0.1

• 2.2.latest → 3.0.1

• 2.3.latest → 3.0.1

• 3.0.any → 3.0.1

Upgrade instructions

Upgrade from 2.x

1. Cleanly shut down the database if it is running.

2. Make a backup copy of the database files. If using the online backup tool available with Neo4jEnterprise, ensure that backups have completed successfully.

3. Install Neo4j 3.0.1.

4. Review the settings in the configuration files of the previous installation and transfer any customsettings to the 3.0.1 installation. Since many settings have been changed between Neo4j 2.x and3.0.1, it is advisable to use the config-migrator to migrate the config files for you. The config-migrator can be found in the tools directory, and can be invoked with a command like: java -jarconfig-migrator.jar path/to/neo4j2.3 path/to/neo4j3.0. Take note of any warnings printed, andmanually review the edited config files produced.

5. Import your data from the old installation using neo4j-admin import --mode=database--database=<database-name> --from=<source-directory>.

6. If the database is not called graph.db, set dbms.active_database in neo4j.conf to the name of thedatabase.

7. Set dbms.allow_format_migration=true in neo4j.conf of the 3.0.1 installation. Neo4j will fail to startwithout this configuration.

26

http://neo4j.com/download/other-releases

8. Start up Neo4j 3.0.1.

9. The database upgrade will take place during startup.

10. Information about the upgrade and a progress indicator are logged into debug.log.

11. When upgrade has finished, the dbms.allow_format_migration should be set to false or beremoved.

12. It is good practice to make a full backup immediately after the upgrade.

Cypher compatibility

The Cypher language may evolve between Neo4j versions. For backwardcompatibility, Neo4j provides directives which allow explicitly selecting a previousCypher language version. This is possible to do globally or for individual statements,as described in the Neo4j Developer Manual (http://neo4j.com/docs/developer-

manual/3.0).

Upgrade from 3.x

1. Cleanly shut down the database if it is running.

2. Make a backup copy of the database files. If using the online backup tool available with Neo4jEnterprise, ensure that backups have completed successfully.


4. Review the settings in the configuration files of the previous installation and transfer any customsettings to the 3.0.1 installation.

5. Wen using the default data directory, copy it from the old installation to the new. If databases arestored in a custom location, configure dbms.directories.data for the new installation to point tothis custom location.


7. Set dbms.allow_format_migration=true in neo4j.conf of the 3.0.1 installation. Neo4j will fail to startwithout this configuration.

8. Start up Neo4j 3.0.1.

9. The database upgrade will take place during startup.

10. Information about the upgrade and a progress indicator are logged into debug.log.

11. When upgrade has finished, the dbms.allow_format_migration should be set to false or beremoved.

12. It is good practice to make a full backup immediately after the upgrade.

2.6.2. Neo4j cluster upgrade

Upgrading a Neo4j HA cluster to Neo4j 3.0.1 requires following a specific process in order to ensurethat the cluster remains consistent, and that all cluster instances are able to join and participate in thecluster following their upgrade. Neo4j 3.0.1 does not support rolling upgrades.

Back up the Neo4j database

• Before starting any upgrade procedure, it is very important to make a full backup of your database.

• For detailed instructions on backing up your Neo4j database, refer to the backup guide.

27

http://neo4j.com/docs/developer-manual/3.0

Shut down the cluster

• Shut down the slave instances one by one.

• Shut down the master last.

Upgrade the master

1. Install Neo4j 3.0.1 on the master, keeping the database files untouched.

2. Disable HA in the configuration, by setting dbms.mode=SINGLE in neo4j.conf.

3. Upgrade as described for a single instance of Neo4j

4. When upgrade has finished, shut down Neo4j again.

5. Re-enable HA in the configuration by setting dbms.mode=HA in neo4j.conf.

6. Make a full backup of the Neo4j database. Please note that backups from before the upgrade areno longer valid for update via the incremental online backup. Therefore it is important to performa full backup, using an empty target directory, at this point.

Upgrade the slaves

On each slave:

1. Remove all database files.


3. Review the settings in the configuration files in the previous installation, and transfer any customsettings to the 3.0.1 installation. Be aware of settings that have changed name between versions.


5. If applicable, copy the security configuration from the master, since this is not propagatedautomatically.

At this point it is an alternative to manually copy database files from the master tothe slaves. Doing so will avoid the need to sync from the master when starting. Thiscan save considerable time when upgrading large databases.

Restart the cluster

1. Start the master instance.

2. Start the slaves, one by one. Once a slave has joined the cluster, it will sync the database from themaster instance.

2.7. Import tool

The import tool is used to create a new Neo4j database from data in CSV files.

This chapter explains how to use the tool, format the input data and concludes with an examplebringing everything together.

These are some things you’ll need to keep in mind when creating your input files:

• Fields are comma separated by default but a different delimiter can be specified.

• All files must use the same delimiter.

28

• Multiple data sources can be used for both nodes and relationships.

• A data source can optionally be provided using multiple files.

• A header which provides information on the data fields must be on the first row of each datasource.

• Fields without corresponding information in the header will not be read.

• UTF-8 encoding is used.

Indexes are not created during the import. Instead you’ll need to add indexesafterwards (see [graphdb-neo4j-schema-indexes]).

Data cannot be imported into an existing database using this tool.

• If you want to load small to medium sized CSV files see [query-load-csv].

• If you want to bulk import into an existing database see [batchinsert].

2.7.1. CSV file header format

The header row of each data source specifies how the fields should be interpreted. The samedelimiter is used for the header row as for the rest of the data.

The header contains information for each field, with the format: <name>:<field_type>. The <name> isused as the property key for values, and ignored in other cases. The following <field_type> settingscan be used for both nodes and relationships:

Property value

Use one of int, long, float, double, boolean, byte, short, char, string to designate the data type. Ifno data type is given, this defaults to string. To define an array type, append [] to the type. Bydefault, array values are separated by ;. A different delimiter can be specified with --array-delimiter.

IGNORE

Ignore this field completely.

See below for the specifics of node and relationship data source headers.

Nodes

The following field types do additionally apply to node data sources:

ID

Each node must have a unique id which is used during the import. The ids are used to find thecorrect nodes when creating relationships. Note that the id has to be unique across all nodes in theimport, even nodes with different labels.

LABEL

Read one or more labels from this field. Like array values, multiple labels are separated by ;, or bythe character specified with --array-delimiter.

Relationships

For relationship data sources, there are three mandatory fields:

TYPE

29

The relationship type to use for the relationship.

START_ID

The id of the start node of the relationship to create.

END_ID

The id of the end node of the relationship to create.

ID spaces

The import tool assumes that node identifiers are unique across node files. If this isn’t the case thenwe can define an id space. Id spaces are defined in the ID field of node files.

For example, to specify the Person id space we would use the field type ID(Person) in our personsnode file. We also need to reference that id space in our relationships file i.e. START_ID(Person) orEND_ID(Person).

2.7.2. Command line usage

Linux

Under Unix/Linux/OSX, the command is named neo4j-import. Depending on the installation type, thetool is either available globally, or used by executing ./bin/neo4j-import from inside the installationdirectory.

Windows

Under Windows, used by executing bin\neo4j-import from inside the installation directory.

For help with running the import tool under Windows, see the reference in Windows.

Options

--into <store-dir>

Database directory to import into. Must not contain existing database.

--nodes[:Label1:Label2] "<file1>,<file2>,…"

Node CSV header and data. Multiple files will be logically seen as one big file from the perspectiveof the importer. The first line must contain the header. Multiple data sources like these can bespecified in one import, where each data source has its own header. Note that file groups must beenclosed in quotation marks.

--relationships[:RELATIONSHIP_TYPE] "<file1>,<file2>,…"

Relationship CSV header and data. Multiple files will be logically seen as one big file from theperspective of the importer. The first line must contain the header. Multiple data sources like thesecan be specified in one import, where each data source has its own header. Note that file groupsmust be enclosed in quotation marks.

--delimiter <delimiter-character>

Delimiter character, or 'TAB', between values in CSV data. The default option is ,.

--array-delimiter <array-delimiter-character>

Delimiter character, or 'TAB', between array elements within a value in CSV data. The default optionis ;.

30

--quote <quotation-character>

Character to treat as quotation character for values in CSV data. The default option is “. Quotesinside quotes escaped like `"""Go away"", he said." and "\"Go away\", he said." aresupported. If you have set "’” to be used as the quotation character, you could write the previousexample like this instead: '"Go away", he said.'

--multiline-fields <true/false>

Whether or not fields from input source can span multiple lines, i.e. contain newline characters.Default value: false

--input-encoding <character set>

Character set that input data is encoded in. Provided value must be one out of the availablecharacter sets in the JVM, as provided by Charset#availableCharsets(). If no input encoding isprovided, the default character set of the JVM will be used.

--ignore-empty-strings <true/false>

Whether or not empty string fields, i.e. "" from input source are ignored, i.e. treated as null. Defaultvalue: false

--id-type <id-type>

One out of [STRING, INTEGER, ACTUAL] and specifies how ids in node/relationship input files aretreated. STRING: arbitrary strings for identifying nodes. INTEGER: arbitrary integer values foridentifying nodes. ACTUAL: (advanced) actual node ids. The default option is STRING. Default value:STRING

--processors <max processor count>

(advanced) Max number of processors used by the importer. Defaults to the number of availableprocessors reported by the JVM. There is a certain amount of minimum threads needed so for thatreason there is no lower bound for this value. For optimal performance this value shouldn’t begreater than the number of available processors.

--stacktrace <true/false>

Enable printing of error stack traces.

--bad-tolerance <max number of bad entries>

Number of bad entries before the import is considered failed. This tolerance threshold is aboutrelationships refering to missing nodes. Format errors in input data are still treated as errors.Default value: 1000

--skip-bad-relationships <true/false>

Whether or not to skip importing relationships that refers to missing node ids, i.e. either start orend node id/group referring to node that wasn’t specified by the node input data. Skipped nodeswill be logged, containing at most number of entites specified by bad-tolerance. Default value: true

--skip-duplicate-nodes <true/false>

Whether or not to skip importing nodes that have the same id/group. In the event of multiplenodes within the same group having the same id, the first encountered will be imported whereasconsecutive such nodes will be skipped. Skipped nodes will be logged, containing at most numberof entities specified by bad-tolerance. Default value: false

--ignore-extra-columns <true/false>

Whether or not to ignore extra columns in the data not specified by the header. Skipped columnswill be logged, containing at most number of entities specified by bad-tolerance. Default value:false

--db-config <path/to/neo4j.properties>

(advanced) File specifying database-specific configuration. For more information consult manual

31

about available configuration options for a neo4j configuration file. Only configuration affectingstore at time of creation will be read. Examples of supported config are:dbms.relationship_grouping_threshold unsupported.dbms.block_size.stringsunsupported.dbms.block_size.array_properties

Output and statistics

While an import is running through its different stages, some statistics and figures are printed in theconsole. The general interpretation of that output is to look at the horizontal line, which is divided upinto sections, each section representing one type of work going on in parallel with the other sections.The wider a section is, the more time is spent there relative to the other sections, the widest being thebottleneck, also marked with *. If a section has a double line, instead of just a single line, it means thatmultiple threads are executing the work in that section. To the far right a number is displayed tellinghow many entities (nodes or relationships) have been processed by that stage.

As an example:

[*>:20,25 MB/s------------------|PREPARE(3)====================|RELATIONSHIP(2)===============]16M

Would be interpreted as:

• > data being read, and perhaps parsed, at 20,25 MB/s, data that is being passed on to …

• PREPARE preparing the data for …

• RELATIONSHIP creating actual relationship records and …

• v writing the relationships to the store. This step isn’t visible in this example, because it’s so cheapcompared to the other sections.

Observing the section sizes can give hints about where performance can be improved. In the exampleabove, the bottleneck is the data read section (marked with >), which might indicate that the disk isbeing slow, or is poorly handling simultaneous read and write operations (since the last section oftenrevolves around writing to disk).

Verbose error information

In some cases if an unexpected error occurs it might be useful to supply the command line option--stacktrace to the import (and rerun the import to actually see the additional information). This willhave the error printed with additional debug information, useful for both developers and issuereporting.

2.7.3. Import tool examples

Let’s look at a few examples. We’ll use a data set containing movies, actors and roles.

While you’ll usually want to store your node identifier as a property on the node forlooking it up later, it’s not mandatory. If you don’t want the identifier to be persistedthen don’t specify a property name in the :ID field.

Basic example

First we’ll look at the movies. Each movie has an id, which is used to refer to it in other data sources, atitle and a year Along with these properties we’ll also add the node labels Movie and Sequel.

By default the import tool expects CSV files to be comma delimited.

32

movies.csv

movieId:ID,title,year:int,:LABELtt0133093,"The Matrix",1999,Moviett0234215,"The Matrix Reloaded",2003,Movie;Sequeltt0242653,"The Matrix Revolutions",2003,Movie;Sequel

Next up are the actors. They have an id - in this case a shorthand - and a name and all have the Actorlabel.

actors.csv

personId:ID,name,:LABELkeanu,"Keanu Reeves",Actorlaurence,"Laurence Fishburne",Actorcarrieanne,"Carrie-Anne Moss",Actor

Finally we have the roles that an actor plays in a movie which will be represented by relationships inthe database. In order to create a relationship between nodes we refer to the ids used in actors.csvand movies.csv in the START_ID and END_ID fields. We also need to provide a relationship type (in thiscase ACTS_IN) in the :TYPE field.

roles.csv

:START_ID,role,:END_ID,:TYPEkeanu,"Neo",tt0133093,ACTED_INkeanu,"Neo",tt0234215,ACTED_INkeanu,"Neo",tt0242653,ACTED_INlaurence,"Morpheus",tt0133093,ACTED_INlaurence,"Morpheus",tt0234215,ACTED_INlaurence,"Morpheus",tt0242653,ACTED_INcarrieanne,"Trinity",tt0133093,ACTED_INcarrieanne,"Trinity",tt0234215,ACTED_INcarrieanne,"Trinity",tt0242653,ACTED_IN

With all data in place, we execute the following command:

neo4j-import --into path_to_target_directory --nodes movies.csv --nodes actors.csv --relationshipsroles.csv

We’re now ready to start up a database from the target directory. (see [server-installation])

Once we’ve got the database up and running we can add appropriate indexes. (see [cypherdoc-labels-constraints-and-indexes].)

It is possible to import only nodes using the import tool - just don’t specify arelationships file when calling neo4j-import. If you do this you’ll need to createrelationships later by another method - the import tool only works for initial graphpopulation.

Customizing configuration options

We can customize the configuration options that the import tool uses (see Options) if our data doesn’tfit the default format. The following CSV files are delimited by ;, use | as their array delimiter and use' for quotes.

33

movies2.csv

movieId:ID;title;year:int;:LABELtt0133093;'The Matrix';1999;Moviett0234215;'The Matrix Reloaded';2003;Movie|Sequeltt0242653;'The Matrix Revolutions';2003;Movie|Sequel

actors2.csv

personId:ID;name;:LABELkeanu;'Keanu Reeves';Actorlaurence;'Laurence Fishburne';Actorcarrieanne;'Carrie-Anne Moss';Actor

roles2.csv

:START_ID;role;:END_ID;:TYPEkeanu;'Neo';tt0133093;ACTED_INkeanu;'Neo';tt0234215;ACTED_INkeanu;'Neo';tt0242653;ACTED_INlaurence;'Morpheus';tt0133093;ACTED_INlaurence;'Morpheus';tt0234215;ACTED_INlaurence;'Morpheus';tt0242653;ACTED_INcarrieanne;'Trinity';tt0133093;ACTED_INcarrieanne;'Trinity';tt0234215;ACTED_INcarrieanne;'Trinity';tt0242653;ACTED_IN

We can then import these files with the following command line options:

neo4j-import --into path_to_target_directory --nodes movies2.csv --nodes actors2.csv --relationshipsroles2.csv --delimiter ";" --array-delimiter "|" --quote "'"

Using separate header files

When dealing with very large CSV files it’s more convenient to have the header in a separate file. Thismakes it easier to edit the header as you avoid having to open a huge data file just to change it.

import-tool can also process single file compressed archives. e.g. --nodesnodes.csv.gz or --relationships rels.zip

We’ll use the same data as in the previous example but put the headers in separate files.

movies3-header.csv

movieId:ID,title,year:int,:LABEL

movies3.csv

tt0133093,"The Matrix",1999,Moviett0234215,"The Matrix Reloaded",2003,Movie;Sequeltt0242653,"The Matrix Revolutions",2003,Movie;Sequel

actors3-header.csv

personId:ID,name,:LABEL

34

actors3.csv

keanu,"Keanu Reeves",Actorlaurence,"Laurence Fishburne",Actorcarrieanne,"Carrie-Anne Moss",Actor

roles3-header.csv

:START_ID,role,:END_ID,:TYPE

roles3.csv

keanu,"Neo",tt0133093,ACTED_INkeanu,"Neo",tt0234215,ACTED_INkeanu,"Neo",tt0242653,ACTED_INlaurence,"Morpheus",tt0133093,ACTED_INlaurence,"Morpheus",tt0234215,ACTED_INlaurence,"Morpheus",tt0242653,ACTED_INcarrieanne,"Trinity",tt0133093,ACTED_INcarrieanne,"Trinity",tt0234215,ACTED_INcarrieanne,"Trinity",tt0242653,ACTED_IN

Note how the file groups are enclosed in quotation marks in the command:

neo4j-import --into path_to_target_directory --nodes "movies3-header.csv,movies3.csv" --nodes "actors3-header.csv,actors3.csv" --relationships "roles3-header.csv,roles3.csv"

Multiple input files

As well as using a separate header file you can also provide multiple nodes or relationships files. Thismay be useful when processing the output from a Hadoop pipeline for example. Files within such aninput group can be specified with multiple match strings, delimited by ,, where each match string canbe either: the exact file name or a regular expression matching one or more files. Multiple matching fileswill be sorted according to their characters and their natural number sort order for file namescontaining numbers.

movies4-header.csv

movieId:ID,title,year:int,:LABEL

movies4-part1.csv

tt0133093,"The Matrix",1999,Moviett0234215,"The Matrix Reloaded",2003,Movie;Sequel

movies4-part2.csv

tt0242653,"The Matrix Revolutions",2003,Movie;Sequel

actors4-header.csv

personId:ID,name,:LABEL

actors4-part1.csv

keanu,"Keanu Reeves",Actorlaurence,"Laurence Fishburne",Actor

35

actors4-part2.csv

carrieanne,"Carrie-Anne Moss",Actor

roles4-header.csv

:START_ID,role,:END_ID,:TYPE

roles4-part1.csv

keanu,"Neo",tt0133093,ACTED_INkeanu,"Neo",tt0234215,ACTED_INkeanu,"Neo",tt0242653,ACTED_INlaurence,"Morpheus",tt0133093,ACTED_INlaurence,"Morpheus",tt0234215,ACTED_IN

roles4-part2.csv

laurence,"Morpheus",tt0242653,ACTED_INcarrieanne,"Trinity",tt0133093,ACTED_INcarrieanne,"Trinity",tt0234215,ACTED_INcarrieanne,"Trinity",tt0242653,ACTED_IN

The call to neo4j-import would look like this:

neo4j-import --into path_to_target_directory --nodes "movies4-header.csv,movies4-part1.csv,movies4-part2.csv" --nodes "actors4-header.csv,actors4-part1.csv,actors4-part2.csv" --relationships "roles4-header.csv,roles4-part1.csv,roles4-part2.csv"

Types and labels

Using the same label for every node

If you want to use the same node label(s) for every node in your nodes file you can do this byspecifying the appropriate value as an option to neo4j-import. In this example we’ll put the label Movieon every node specified in movies5.csv:

movies5.csv

movieId:ID,title,year:inttt0133093,"The Matrix",1999

There’s then no need to specify the :LABEL field in the node file if you pass it as acommand line option. If you do then both the label provided in the file and the oneprovided on the command line will be added to the node.

In this case, we’ll put the labels Movie and Sequel on the nodes specified in sequels5.csv.

sequels5.csv

movieId:ID,title,year:inttt0234215,"The Matrix Reloaded",2003tt0242653,"The Matrix Revolutions",2003

36

actors5.csv

personId:ID,namekeanu,"Keanu Reeves"laurence,"Laurence Fishburne"carrieanne,"Carrie-Anne Moss"

roles5.csv

:START_ID,role,:END_ID,:TYPEkeanu,"Neo",tt0133093,ACTED_INkeanu,"Neo",tt0234215,ACTED_INkeanu,"Neo",tt0242653,ACTED_INlaurence,"Morpheus",tt0133093,ACTED_INlaurence,"Morpheus",tt0234215,ACTED_INlaurence,"Morpheus",tt0242653,ACTED_INcarrieanne,"Trinity",tt0133093,ACTED_INcarrieanne,"Trinity",tt0234215,ACTED_INcarrieanne,"Trinity",tt0242653,ACTED_IN


neo4j-import --into path_to_target_directory --nodes:Movie movies5.csv --nodes:Movie:Sequel sequels5.csv--nodes:Actor actors5.csv --relationships roles5.csv

Using the same relationship type for every relationship

If you want to use the same relationship type for every relationship in your relationships file you cando this by specifying the appropriate value as an option to neo4j-import. In this example we’ll put therelationship type ACTS_IN on every relationship specified in roles6.csv:

movies6.csv


actors6.csv


roles6.csv

:START_ID,role,:END_IDkeanu,"Neo",tt0133093keanu,"Neo",tt0234215keanu,"Neo",tt0242653laurence,"Morpheus",tt0133093laurence,"Morpheus",tt0234215laurence,"Morpheus",tt0242653carrieanne,"Trinity",tt0133093carrieanne,"Trinity",tt0234215carrieanne,"Trinity",tt0242653

If you provide a relationship type on the command line and in the relationships filethe one in the file will be applied.


37

neo4j-import --into path_to_target_directory --nodes movies6.csv --nodes actors6.csv--relationships:ACTED_IN roles6.csv

Property types

The type for properties specified in nodes and relationships files is defined in the header row. (see CSVfile header format)

The following example creates a small graph containing one actor and one movie connected by anACTED_IN relationship. There is a roles property on the relationship which contains an array of thecharacters played by the actor in a movie.

movies10.csv

movieId:ID,title,year:int,:LABELtt0099892,"Joe Versus the Volcano",1990,Movie

actors10.csv

personId:ID,name,:LABELkeanu,"Keanu Reeves",Actorlaurence,"Laurence Fishburne",Actorcarrieanne,"Carrie-Anne Moss",Actorlaurence,"Laurence Harvey",Actor

roles10.csv

:START_ID,roles:string[],:END_ID,:TYPEmeg,"DeDe;Angelica Graynamore;Patricia Graynamore",tt0099892,ACTED_IN

The arguments to neo4j-import would be the following:

neo4j-import --into path_to_target_directory --nodes movies10.csv --nodes actors10.csv --relationshipsroles10.csv

ID handling

Each node processed by neo4j-import must provide a unique id. We use this id to find the correctnodes when creating relationships.

Working with sequential or auto incrementing identifiers

The import tool makes the assumption that identifiers are unique across node files. This may not bethe case for data sets which use sequential, auto incremented or otherwise colliding identifiers. Thosedata sets can define id spaces where identifiers are unique within their respective id space.

For example if movies and people both use sequential identifiers then we would define Movie andActor id spaces.

movies7.csv

movieId:ID(Movie),title,year:int,:LABEL1,"The Matrix",1999,Movie2,"The Matrix Reloaded",2003,Movie;Sequel3,"The Matrix Revolutions",2003,Movie;Sequel

38

actors7.csv

personId:ID(Actor),name,:LABEL1,"Keanu Reeves",Actor2,"Laurence Fishburne",Actor3,"Carrie-Anne Moss",Actor

We also need to reference the appropriate id space in our relationships file so it knows which nodes toconnect together:

roles7.csv

:START_ID(Actor),role,:END_ID(Movie)1,"Neo",11,"Neo",21,"Neo",32,"Morpheus",12,"Morpheus",22,"Morpheus",33,"Trinity",13,"Trinity",23,"Trinity",3

The command line arguments would remain the same as before:

neo4j-import --into path_to_target_directory --nodes movies7.csv --nodes actors7.csv--relationships:ACTED_IN roles7.csv

Bad input data

The import tool has a threshold of how many bad entities (nodes/relationships) to tolerate and skipbefore failing the import. By default 1000 bad entities are tolerated. A bad tolerance of 0 will as anexample fail the import on the first bad entity. For more information, see the <<import-tool-option-bad-tolerance, --bad-tolerance>> option.

There are different types of bad input, which we will look into.

Relationships referring to missing nodes

Relationships that refer to missing node ids, either for :START_ID or :END_ID are considered badrelationships. Whether or not such relationships are skipped is controlled with --skip-bad-relationships flag which can have the values true or false or no value, which means true. Specifyingfalse means that any bad relationship is considered an error and will fail the import. For moreinformation, see the <<import-tool-option-skip-bad-relationships, --skip-bad-relationships>> option.

In the following example there is a missing emil node referenced in the roles file.

movies9.csv


actors9.csv


39

roles9.csv

:START_ID,role,:END_ID,:TYPEkeanu,"Neo",tt0133093,ACTED_INkeanu,"Neo",tt0234215,ACTED_INkeanu,"Neo",tt0242653,ACTED_INlaurence,"Morpheus",tt0133093,ACTED_INlaurence,"Morpheus",tt0234215,ACTED_INlaurence,"Morpheus",tt0242653,ACTED_INcarrieanne,"Trinity",tt0133093,ACTED_INcarrieanne,"Trinity",tt0234215,ACTED_INcarrieanne,"Trinity",tt0242653,ACTED_INemil,"Emil",tt0133093,ACTED_IN

The command line arguments would remain the same as before:

neo4j-import --into path_to_target_directory --nodes movies9.csv --nodes actors9.csv --relationshipsroles9.csv

Since there was only one bad relationship the import process will complete successfully and a not-imported.bad file will be created and populated with the bad relationships.

not-imported.bad

InputRelationship: source: roles9.csv:11 properties: [role, Emil] startNode: emil endNode: tt0133093 type: ACTED_IN refering to missing node emil

Multiple nodes with same id within same id space

Nodes that specify :ID which has already been specified within the id space are considered badnodes. Whether or not such nodes are skipped is controlled with --skip-duplicate-nodes flag whichcan have the values true or false or no value, which means true. Specifying false means that anyduplicate node is considered an error and will fail the import. For more information, see the <<import-tool-option-skip-duplicate-nodes, --skip-duplicate-nodes>> option.

In the following example there is a node id that is specified twice within the same id space.

actors10.csv

personId:ID,name,:LABELkeanu,"Keanu Reeves",Actorlaurence,"Laurence Fishburne",Actorcarrieanne,"Carrie-Anne Moss",Actorlaurence,"Laurence Harvey",Actor

neo4j-import --into path_to_target_directory --nodes actors10.csv --skip-duplicate-nodes

Since there was only one bad node the import process will complete successfully and a not-imported.bad file will be created and populated with the bad node.

not-imported.bad

Id 'laurence' is defined more than once in global id space, at least at actors10.csv:3 and actors10.csv:5

40

Chapter 3. Security

3.1. Securing Neo4j Server

3.1.1. Secure the port and remote client connection accepts

By default, the Neo4j Server is bundled with a Web server that binds to host localhost on port 7474,answering only requests from the local machine.

This is configured in neo4j.conf:

# Let the webserver only listen on the specified IP. Default is localhost (only# accept local connections). Uncomment to allow any connection.dbms.connector.http.type=HTTPdbms.connector.http.enabled=true#dbms.connector.http.address=0.0.0.0:7474

If you want the server to listen to external hosts, configure the Web server in neo4j.conf by setting theproperty dbms.connector.http.address=0.0.0.0:7474 which will cause the server to bind to all availablenetwork interfaces. Note that firewalls et cetera have to be configured accordingly as well.

3.1.2. Server authentication and authorization

Neo4j requires clients to supply authentication credentials when accessing the REST API. Without validcredentials, access to the database will be forbidden.

The authentication and authorization data is stored under data/dbms/auth. If necessary, this file canbe copied over to other neo4j instances to ensure they share the same username/password.

When accessing Neo4j over unsecured networks, make sure HTTPS is configured and used for access(see HTTPS support).

If necessary, authentication may be disabled. This will allow any client to access the database withoutsupplying authentication credentials.

# Disable authorizationdbms.security.auth_enabled=false

Disabling authentication is not recommended, and should only be done if theoperator has a good understanding of their network security, including protectionagainst cross-site scripting (XSS) (http://en.wikipedia.org/wiki/Cross-site_scripting) attacksvia web browsers. Developers should not disable authentication if they have a localinstallation using the default listening ports.

3.1.3. HTTPS support

The Neo4j server includes built in support for SSL encrypted communication over HTTPS. The firsttime the server starts, it automatically generates a self-signed SSL certificate and a private key.Because the certificate is self signed, it is not safe to rely on for production use. Instead, you shouldprovide your own key and certificate for the server to use.

41

http://en.wikipedia.org/wiki/Cross-site_scripting

Using auto-generation of self-signed SSL certificates will not work if the Neo4jserver has been configured with multiple connectors that bind to different IPaddresses. If you need to use multiple IP addresses, please configure certificatesmanually and use multi-host or wildcard certificates instead.

To provide your own key and certificate, put the files neo4j.key and neo4j.cert in the certificatesdirectory. Note that the files must be named exactly neo4j.key and neo4j.cert. The location of thedirectory can be configured by setting dbms.directories.certificates in neo4j.conf.

# Certificates location (auto generated if the file does not exist)dbms.directories.certificates=certificates

Note that the key should be unencrypted. Make sure you set correct permissions on the private key,so that only the Neo4j server user can read/write it.

Neo4j also supports chained SSL certificates. This requires to have all certificates in PEM formatcombined in one file and the private key needs to be in DER format.

You can set what port the HTTPS connector should bind to in the same configuration file, as well asturn HTTPS on or off:

dbms.connector.https.type=HTTPdbms.connector.https.enabled=truedbms.connector.https.encryption=TLSdbms.connector.https.address=localhost:7473

3.1.4. Arbitrary code execution

The Neo4j server exposes remote scripting functionality by default that allow fullaccess to the underlying system. Exposing your server without implementing asecurity layer presents a substantial security vulnerability.

By default, the Neo4j database comes with some places where arbitrary code code execution canhappen. These are the REST endpoints. To secure these, either disable them completely by removingoffending plugins from the server classpath, or secure access to these URLs through proxies orAuthorization Rules. Also, the Java Security Manager, seehttp://docs.oracle.com/javase/7/docs/technotes/guides/security/index.html, can be used to secureparts of the codebase.

3.1.5. Server authorization rules

Administrators may require more fine-grained security policies in addition to the basic authorizationand/or IP-level restrictions on the Web server. Neo4j server supports administrators in allowing ordisallowing access the specific aspects of the database based on credentials that users or applicationsprovide.

To facilitate domain-specific authorization policies in Neo4j Server, security rules can be implementedand registered with the server. This makes scenarios like user and role based security andauthentication against external lookup services possible. Seeorg.neo4j.server.rest.security.SecurityRule in the javadocs downloadable from Maven Central(org.neo4j.app:neo4j-server)(http://search.maven.org/#search%7Cgav%7C1%7Cg%3A%22org.neo4j.app%22%20AND%20a%3A%22neo4j-

server%22).

42

http://docs.oracle.com/javase/7/docs/technotes/guides/security/index.html

http://search.maven.org/#search%7Cgav%7C1%7Cg%3A%22org.neo4j.app%22%20AND%20a%3A%22neo4j-server%22

http://search.maven.org/#search%7Cgav%7C1%7Cg%3A%22org.neo4j.app%22%20AND%20a%3A%22neo4j-server%22

The use of Server Authorization Rules may interact unexpectedly with the built-inauthentication and authorization (see Server authentication and authorization), ifenabled.

3.1.6. Enforcing Server Authorization Rules

In this example, a (dummy) failing security rule is registered to deny access to all URIs to the server bylisting the rules class in 'neo4j.conf':

org.neo4j.server.rest.security_rules=my.rules.PermanentlyFailingSecurityRule

with the rule source code of:

public class PermanentlyFailingSecurityRule implements SecurityRule{

public static final String REALM = "WallyWorld"; // as per RFC2617 :-)

@Override public boolean isAuthorized( HttpServletRequest request ) { return false; // always fails - a production implementation performs // deployment-specific authorization logic here }

@Override public String forUriPath() { return "/*"; }

@Override public String wwwAuthenticateHeader() { return SecurityFilter.basicAuthenticationResponse(REALM); }}

With this rule registered, any access to the server will be denied. In a production-qualityimplementation the rule will likely lookup credentials/claims in a 3rd-party directory service (e.g. LDAP)or in a local database of authorized users.

Example request

• POST http://localhost:7474/db/data/node

• Accept: application/json; charset=UTF-8

Example response

• 401: Unauthorized

• WWW-Authenticate: Basic realm="WallyWorld"

3.1.7. Using Wildcards to Target Security Rules

In this example, a security rule is registered to deny access to all URIs to the server by listing the rule(s)class(es) in 'neo4j.conf'. In this case, the rule is registered using a wildcard URI path (wherecharacters can be used to signify any part of the path). For example /users means the rule willbe bound to any resources under the /users root path. Similarly /users*type* will bind the rule toresources matching URIs like /users/fred/type/premium.

43

org.neo4j.server.rest.security_rules=my.rules.PermanentlyFailingSecurityRuleWithWildcardPath


public String forUriPath(){ return "/protected/*";}

With this rule registered, any access to URIs under /protected/ will be denied by the server. Usingwildcards allows flexible targeting of security rules to arbitrary parts of the server’s API, including anyunmanaged extensions or managed plugins that have been registered.

Example request

• GET http://localhost:7474/protected/tree/starts/here/dummy/more/stuff

• Accept: application/json

Example response



3.1.8. Using Complex Wildcards to Target Security Rules

In this example, a security rule is registered to deny access to all URIs matching a complex pattern.The config looks like this:

org.neo4j.server.rest.security_rules=my.rules.PermanentlyFailingSecurityRuleWithComplexWildcardPath


public class PermanentlyFailingSecurityRuleWithComplexWildcardPath implements SecurityRule{

public static final String REALM = "WallyWorld"; // as per RFC2617 :-)

@Override public boolean isAuthorized( HttpServletRequest request ) { return false; }

@Override public String forUriPath() { return "/protected/*/something/else/*/final/bit"; }

@Override public String wwwAuthenticateHeader() { return SecurityFilter.basicAuthenticationResponse(REALM); }}

Example request

• GEThttp://localhost:7474/protected/wildcard_replacement/x/y/z/something/else/more_wildcard_repla

44

cement/a/b/c/final/bit/more/stuff

• Accept: application/json

Example response



3.1.9. Using a proxy

Although the Neo4j server has a number of security features built-in (see the above chapters), forsensitive deployments it is often sensible to front against the outside world it with a proxy like Apachemod_proxy [1: http://httpd.apache.org/docs/2.2/mod/mod_proxy.html].

This provides a number of advantages:

• Control access to the Neo4j server to specific IP addresses, URL patterns and IP ranges. This canbe used to make for instance only the '/db/data' namespace accessible to non-local clients, whilethe '/db/admin' URLs only respond to a specific IP address.

<Proxy *> Order Deny,Allow Deny from all Allow from 192.168.0</Proxy>

While it is possible to develop plugins using Neo4j’s SecurityRule (see above), operationsprofessionals would often prefer to configure proxy servers such as Apache. However, it should benoted that in cases where both approaches are being used, they will work harmoniously providedthat the behavior is consistent across proxy server and SecurityRule plugins.

• Run Neo4j Server as a non-root user on a Linux/Unix system on a port < 1000 (e.g. port 80) using

ProxyPass /neo4jdb/data http://localhost:7474/db/dataProxyPassReverse /neo4jdb/data http://localhost:7474/db/data

• Simple load balancing in a clustered environment to load-balance read load using the Apachemod_proxy_balancer [2: http://httpd.apache.org/docs/2.2/mod/mod_proxy_balancer.html] plugin

<Proxy balancer://mycluster>BalancerMember http://192.168.1.50:80BalancerMember http://192.168.1.51:80</Proxy>ProxyPass /test balancer://mycluster

3.1.10. LOAD CSV

The Cypher LOAD CSV clause can be used to import CSV files over the network or from the local filesystem. When reading from the file system the file:/// URL that is used is resolved relative to thedirectory configured by dbms.directories.import. The default value is import. This is a securitymeasure which prevents the database from accessing files outside of the standard import directory.

To remove this security measure and allow access to any file on the system, setdbms.directories.import to be empty.

The related dbms.security.allow_csv_import_from_file_urls setting can be set to false to completely

45

http://httpd.apache.org/docs/2.2/mod/mod_proxy.html

http://httpd.apache.org/docs/2.2/mod/mod_proxy_balancer.html

disable access to the file system for LOAD CSV.

To review all security-related configuration settings see the Configuration Settings Reference.

46

Chapter 4. Backup

The backup features are only available in the Neo4j Enterprise Edition.

4.1. Introducing backup

Backups are performed over the network, from a running Neo4j server and into a local copy of thedatabase store (the backup). The backup is run using the neo4j-backup tool, which is providedalongside Neo4j Enterprise.

Neo4j Server must be configured to run a backup service. This is enabled via theconfiguration parameter dbms.backup.enabled, and is enabled by default. Theinterface and port the backup service listens on is configured via the parameterdbms.backup.address and defaults to the loopback interface and port 6362. It istypical to reconfigure this to listen on an external interface, by settingdbms.backup.address=<my-host-ip-address>:6362. It can also be configured to listenon all interfaces by setting dbms.backup.address=0.0.0.0:6362.

Performing a backup requires specifying the target host, an optional port, and the backup location.The backup tool will automatically select a full or incremental backup, based on whether an existingbackup is present at that location.

See the configuration reference section for detailed documentation on available configurationoptions.

4.2. Performing backups

4.2.1. Backup commands

# Performing a full backup: create a blank directory and run the backup toolmkdir /mnt/backup/neo4j-backup./bin/neo4j-backup -host 192.168.1.34 -to /mnt/backup/neo4j-backup

# Performing an incremental backup: just specify the location of your previous backup./bin/neo4j-backup -host 192.168.1.34 -to /mnt/backup/neo4j-backup

# Performing an incremental backup where the service is listening on a non-default port./bin/neo4j-backup -host 192.168.1.34 -port 9999 -to /mnt/backup/neo4j-backup

4.2.2. Incremental backups

An incremental backup is performed whenever an existing backup directory is specified and thetransaction logs are present since the last backup (see note below). The backup tool will then copy anynew transactions from the Neo4j server and apply them to the backup. The result will be an updatedbackup that is consistent with the current server state.

However, the incremental backup may fail for a number of reasons:

• If the existing directory doesn’t contain a valid backup.

• If the existing directory contains a backup of a different database store.

• If the existing directory contains a backup from a previous database version.

47

Note that when copying the outstanding transactions, the server needs access tothe transaction logs. These logs are kept by Neo4j and automatically removed aftera period of time, based on the parameter dbms.tx_log.rotation.retention_policy. Ifthe required transaction logs have already been removed, the backup tool will do afull backup instead.

4.3. Restoring a backup

The Neo4j backups are fully functional databases. To use a backup, simply shut down the databaseand replace all the files in the data directory with the backup. Then start the database.

To restore from backup in a clustered environment, follow these steps:

1. Shut down all database instances in the cluster.

2. Restore the backup to the individual database folders.

3. Start the database instances.

48

Chapter 5. Monitoring

Most of the monitoring features are only available in the Enterprise edition ofNeo4j.

In order to be able to continuously get an overview of the health of a Neo4j database, there aredifferent levels of monitoring facilities available. Most of these are exposed through JMX(http://www.oracle.com/technetwork/java/javase/tech/javamanagement-140525.html). Neo4j Enterprise also hasthe ability to automatically report metrics to commonly used monitoring systems.

5.1. Adjusting remote JMX access to the Neo4j Server

Per default, the Neo4j Enterprise Server edition does not allow remote JMX connections, since therelevant options in the 'conf/neo4j-wrapper.conf' configuration file are commented out. To enable thisfeature, you have to remove the # characters from the various com.sun.management.jmxremoteoptions there.

When commented in, the default values are set up to allow remote JMX connections with certain roles,refer to the 'conf/jmx.password', 'conf/jmx.access' and 'conf/neo4j-wrapper.conf' files for details.

Make sure that 'conf/jmx.password' has the correct file permissions. The owner of the file has to bethe user that will run the service, and the permissions should be read only for that user. On Unixsystems, this is 0600.

On Windows, follow the tutorial athttp://docs.oracle.com/javase/7/docs/technotes/guides/management/security-windows.html to setthe correct permissions. If you are running the service under the Local System Account, the user thatowns the file and has access to it should be SYSTEM.

With this setup, you should be able to connect to JMX monitoring of the Neo4j server using <IP-OF-SERVER>:3637, with the username monitor and the password Neo4j.

Note that it is possible that you have to update the permissions and/or ownership of the'conf/jmx.password' and 'conf/jmx.access' files — refer to the relevant section in 'conf/neo4j-wrapper.conf' for details.

For maximum security, please adjust at least the password settings in'conf/jmx.password' for a production installation.

For more details, see:http://docs.oracle.com/javase/7/docs/technotes/guides/management/agent.html.

5.2. How to connect to a Neo4j instance using JMX andJConsole

First, start your Neo4j database, for instance using

$NEO4j_HOME/bin/neo4j start

Now, start JConsole with

$JAVA_HOME/bin/jconsole

49

http://www.oracle.com/technetwork/java/javase/tech/javamanagement-140525.html

http://docs.oracle.com/javase/7/docs/technotes/guides/management/security-windows.html

http://docs.oracle.com/javase/7/docs/technotes/guides/management/agent.html

Connect to the process running your Neo4j database instance:

Figure 5. Connecting JConsole to the Neo4j Java process

Now, beside the MBeans exposed by the JVM, you will see an org.neo4j section in the MBeans tab.Under that, you will have access to all the monitoring information exposed by Neo4j.

For opening JMX to remote monitoring access, please see Adjusting remote JMX access to the Neo4jServer and the JMX documention(http://docs.oracle.com/javase/7/docs/technotes/guides/management/agent.html).

50

http://docs.oracle.com/javase/7/docs/technotes/guides/management/agent.html

Figure 6. Neo4j MBeans View

5.3. Reference of supported JMX MBeans

For a reference to all the parameters specific to MBeans exposed by Neo4j, see MBeans exposed byNeo4j.

5.4. Metrics Reporting

Metrics reporting is only available in the Neo4j Enterprise Edition.

5.4.1. Introducing Metrics

Neo4j Enterprise can be configured to continuously export Neo4j-specific metrics to Graphite or CSVfiles. This makes it easy to monitor the health of running Neo4j instances.

Neo4j Enterprise can expose metrics for the following parts of the database, and does so by default:

51

// default setting for enabling all supported metricsmetrics.enabled=true

// default setting for enabling all Neo4j specific metricsmetrics.neo4j.enabled=true

// setting for exposing metrics about transactions; number of transactions started, committed, etc.metrics.neo4j.tx.enabled=true

// setting for exposing metrics about the Neo4j page cache; page faults, evictions, flushes andexceptions, etc.metrics.neo4j.pagecache.enabled=true

// setting for exposing metrics about approximately entities are in the database; nodes, relationships,properties, etc.metrics.neo4j.counts.enabled=true

// setting for exposing metrics about the network usage of the HA cluster componentmetrics.neo4j.network.enabled=true

5.4.2. Graphite Configuration

For Graphite integration add the following settings to neo4j.conf:

metrics.graphite.enabled=true // default is 'false'metrics.graphite.server=<ip>:2003metrics.graphite.interval=<how often to send data, defaults to 3s>metrics.prefix=<Neo4j instance name, e.g. wwwneo1>

Start the Neo4j Server and connect to Graphite via a web browser in order to monitor your Neo4jmetrics.

5.4.3. Export to CSV Configuration

For storing metrics in local CSV files add the following settings to neo4j.conf:

metrics.csv.enabled=true // default is 'false'metrics.csv.path=<file or directory path, defaults to "metrics/" in the store directory>metrics.csv.interval=<how often to store data, defaults to 3s>

The CSV exporter does not automatically rotate the output files, so it isrecommended to also set up a CRON job to periodically archive the files.

5.4.4. Configuration settings reference for metrics

See the configuration reference section for detailed documentation on available configurationoptions.

5.4.5. Available metrics

For a reference to all the parameters specific to metrics, see Available metrics.

52

Chapter 6. Performance tuningThis section describes some of the internal workings of Neo4j memory settings and how to adjustthem for optimal performance.

6.1. Modifying configuration settings

6.2. Cypher tuning

The first thing to look at when Neo4j is not performing as expected is how the Cypher queries arebeing executed. Make sure that they don’t do more work than they have to. Some queries mayaccidentally be written in a way that generates a large cartesian product. Other queries may have toperform expensive label scans because an important index is missing. The Neo4j developer manualhas more information on how to investigate Cypher performance issues.

6.3. Memory tuning

Neo4j will automatically configure default values for memory-related configuration parameters thatare not explicitly defined within its configuration on startup. In doing so, it will assume that all of theRAM on the machine is available for running Neo4j.

There are three types of memory to consider: OS Memory, Page Cache and Heap Space.

Please notice that the OS memory is not explicitly configurable, but is "what is left" when donespecifying page cache and heap space. If configuring page cache and heap space equal to or greaterthan the available RAM, or if not leaving enough head room for the OS, the OS will start swapping todisk, which will heavily affect performance. Therefore, follow this checklist:

1. Plan OS memory sizing

2. Plan page cache sizing

3. Plan heap sizing

4. Do the sanity check:

Actual OS allocation = available RAM - (page cache + heap size)

Make sure that your system is configured such that it will never need to swap.

6.3.1. OS memory sizing

Some memory must be reserved for all activities on the server that are not Neo4j related. In addition,leave enough memory for the operating system file buffer cache to fit the contents of the index andschema directories, since it will impact index lookup performance if the indexes cannot fit in memory.1G is a good starting point for when Neo4j is the only server running on that machine.

OS Memory = 1GB + (size of graph.db/index) + (size of graph.db/schema)

6.3.2. Page cache sizing

The page cache is used to cache the Neo4j data as stored on disk. Ensuring that all, or at least most, ofthe graph data from disk is cached into memory will help avoid costly disk access and result in optimalperformance. You can determine the total memory needed for the page cache by summing up thesizes of the NEO4J_HOME/data/databases/graph.db/neostore.*.db files and adding 20% for growth.

53

The parameter for specifyig the page cache is: dbms.memory.pagecache.size. This specifies how muchmemory Neo4j is allowed to use for this cache.

If this is not explicitly defined on startup, Neo4j will look at how much available memory the machinehas, subtract the JVM max heap allocation from that, and then use 50% of what is left for the pagecache. This is considered the default configuration.

The following are two possible methods for estimating the page cache size:

1. For an existing Neo4j database, sum up the size of all the store.db files in your store file directory,to figure out how big a page cache you need to fit all your data. Add another 20% for growth. Forinstance, on a posix system you can look at the total of running $ du -hc store.db in thedata/databases/graph.db directory.

2. For a new Neo4j database, it is useful to run an import with a fraction (e.g. 1/100th) of the dataand then multiply the resulting store-size by that fraction (x 100). Add another 20% for growth.For example: import 1/100th of the data and sum up the sizes of the resulting database files. Thenmultiply by 120 for a total estimate of the database size, including 20% for growth.

Parameter Possible values Effect

dbms.memory.pagecache.size The maximum amount of memoryto use for the page cache, either inbytes, or greater byte-like units,such as 100m for 100 mega-bytes,or 4g for 4 giga-bytes.

The amount of memory to use formapping the store files, in a unit ofbytes. This will automatically berounded down to the nearest wholepage. This value cannot be zero. Forextremely small and memoryconstrained deployments, it isrecommended to still reserve at least acouple of megabytes for the page cache.

unsupported.dbms.report_configuration true or false If set to true the current configurationsettings will be written to the defaultsystem output, mostly the console orthe logfiles.

6.3.3. Heap sizing

The size of the available heap memory is an important aspect for the performance of Neo4j.

Generally speaking, it is beneficial to configure a large enough heap space to sustain concurrentoperations. For many setups, a heap size between 8G and 16G is large enough to run Neo4j reliably.

The heap memory size is determined by the parameters in NEO4J_HOME/conf/neo4j-wrapper.conf,namely dbms.memory.heap.initial_size and dbms.memory.heap.max_size providing the heap size inMegabytes, e.g. 16000. It is recommended to set these two parameters to the same value to avoidunwanted full garbage collection pauses.

6.3.4. Tuning of the garbage collector

The heap is separated into an old generation and a young generation. New objects are allocated in theyoung generation, and then later moved to the old generation, if they stay live (in use) for longenough. When a generation fills up, the garbage collector performs a collection, during which all otherthreads in the process are paused. The young generation is quick to collect since the pause timecorrelates with the live set of objects, and is independent of the size of the young generation. In theold generation, pause times roughly correlates with the size of the heap. For this reason, the heapshould ideally be sized and tuned such that transaction and query state never makes it to the oldgeneration.

The heap size is configured with the dbms.memory.heap.max_size (in MBs) setting in the neo4j-wrapper.conf file. The initial size of the heap is specified by the dbms.memory.heap.initial_size setting,or with the -Xms???m flag, or chosen heuristically by the JVM itself if left unspecified. The JVM will

54

automatically grow the heap as needed, up to the maximum size. The growing of the heap requires afull garbage collection cycle. It is recommended to set the initial heap size and the maximum heap sizeto the same value. This way the pause that happens when the garbage collector grows the heap canbe avoided.

The ratio of the size between the old generation and the new generation of the heap is controlled bythe -XX:NewRatio=N flag. N is typically between 2 and 8 by default. A ratio of 2 means that the oldgeneration size, divided by the new generation size, is equal to 2. In other words, two thirds of theheap memory will be dedicated to the old generation. A ratio of 3 will dedicate three quarters of theheap to the old generation, and a ratio of 1 will keep the two generations about the same size. A ratioof 1 is quite aggressive, but may be necessary if your transactions changes a lot of data. Having a largenew generation can also be important if you run Cypher queries that need to keep a lot of dataresident, for example when sorting big result sets.

If the new generation is too small, short-lived objects may be moved to the old generation too soon.This is called premature promotion and will slow the database down by increasing the frequency ofold generation garbage collection cycles. If the new generation is too big, the garbage collector maydecide that the old generation does not have enough space to fit all the objects it expects to promotefrom the new to the old generation. This turns new generation garbage collection cycles into oldgeneration garbage collection cycles, again slowing the database down. Running more concurrentthreads means that more allocations can take place in a given span of time, in turn increasing thepressure on the new generation in particular.

The Compressed OOPs feature in the JVM allows object references to be compressedto use only 32 bits. The feature saves a lot of memory, but is not enabled for heapslarger than 32 GB. Gains from increasing the heap size beyond 32 GB can thereforebe small or even negative, unless the increase is significant (64 GB or above).

Neo4j has a number of long-lived objects, that stay around in the old generation, effectively for thelifetime of the Java process. To process them efficiently, and without adversely affecting the garbagecollection pause time, we recommend using a concurrent garbage collector.

How to tune the specific garbage collection algorithm depends on both the JVM version and theworkload. It is recommended to test the garbage collection settings under realistic load for days orweeks. Problems like heap fragmentation can take a long time to surface.

To gain good performance, these are the things to look into first:

• Make sure the JVM is not spending too much time performing garbage collection. The goal is tohave a large enough heap to make sure that heavy/peak load will not result in so called GC-trashing. Performance can drop as much as two orders of magnitude when GC-trashing happens.Having too large heap may also hurt performance so you may have to try some different heapsizes.

• Use a concurrent garbage collector. We find that -XX:+UseG1GC works well in most use-cases.

• The Neo4j JVM needs enough heap memory for the transaction state and query processing,plus some head-room for the garbage collector. Because the heap memory needs are soworkload dependent, it is common to see configurations from 1 GB, up to 32 GBs of heapmemory.

• Start the JVM with the -server flag and a good sized heap.

• The operating system on a dedicated server can usually make do with 1 to 2 GBs of memory,but the more physical memory the machine has, the more memory the operating system willneed.

Edit the following properties:

Table 4. neo4j-wrapper.conf JVM tuning properties

55

Property Name Meaning

dbms.memory.heap.initial_size initial heap size (in MB)

dbms.memory.heap.max_size maximum heap size (in MB)

dbms.jvm.additional additional literal JVM parameter

6.4. Transaction logs

The transaction logs record all operations in the database. They are the source of truth in scenarioswhere the database needs to be recovered. Transaction logs are used to provide for incrementalbackups, as well as for cluster operations. For any given configuration at least the latest non-emptytransaction log will be kept.

By default, log switches happen when log sizes surpass 250 MB. This can be configured using theparameter dbms.tx_log.rotation.size.

There are several different means of controlling the amount of transaction logs that is kept, using theparameter dbms.tx_log.rotation.retention_policy. The format in which this is configured is:

dbms.tx_log.rotation.retention_policy=<true/false>dbms.tx_log.rotation.retention_policy=<amount> <type>

For example:

# Will keep logical logs indefinitelydbms.tx_log.rotation.retention_policy=true

# Will keep only the most recent non-empty logdbms.tx_log.rotation.retention_policy=false

# Will keep logical logs which contains any transaction committed within 30 daysdbms.tx_log.rotation.retention_policy=30 days

# Will keep logical logs which contains any of the most recent 500 000 transactionsdbms.tx_log.rotation.retention_policy=500k txs

Full list:

Type Description Example

files Number of most recent logical log files to keep "10 files"

size Max disk size to allow log files to occupy "300M size" or "1Gsize"

txs Number of latest transactions to keep Keep "250k txs" or "5Mtxs"

hours Keep logs which contains any transaction committedwithin N hours from current time

"10 hours"

days Keep logs which contains any transaction committedwithin N days from current time

"50 days"

6.5. Compressed property value storage

Neo4j can in many cases compress and inline the storage of property values, such as short arrays andstrings, with the purpose of saving disk space and possibly an I/O operation.

Compressed storage of short arrays

56

Neo4j will try to store your primitive arrays in a compressed way. To do that, it employs a "bit-shaving"algorithm that tries to reduce the number of bits required for storing the members of the array. Inparticular:

1. For each member of the array, it determines the position of leftmost set bit.

2. Determines the largest such position among all members of the array.

3. It reduces all members to that number of bits.

4. Stores those values, prefixed by a small header.

That means that when even a single negative value is included in the array then the original size of theprimitives will be used.

There is a possibility that the result can be inlined in the property record if:

• It is less than 24 bytes after compression.

• It has less than 64 members.

For example, an array long[] {0L, 1L, 2L, 4L} will be inlined, as the largest entry (4) will require 3 bitsto store so the whole array will be stored in 4 × 3 = 12 bits. The array long[] {-1L, 1L, 2L, 4L}however will require the whole 64 bits for the -1 entry so it needs 64 × 4 = 32 bytes and it will end up inthe dynamic store.

Compressed storage of short strings

Neo4j will try to classify your strings in a short string class and if it manages that it will treat itaccordingly. In that case, it will be stored without indirection in the property store, inlining it instead inthe property record, meaning that the dynamic string store will not be involved in storing that value,leading to reduced disk footprint. Additionally, when no string record is needed to store the property,it can be read and written in a single lookup, leading to performance improvements and less diskspace required.

The various classes for short strings are:

• Numerical, consisting of digits 0..9 and the punctuation space, period, dash, plus, comma andapostrophe.

• Date, consisting of digits 0..9 and the punctuation space dash, colon, slash, plus and comma.

• Hex (lower case), consisting of digits 0..9 and lower case letters a..f

• Hex (upper case), consisting of digits 0..9 and upper case letters a..f

• Upper case, consisting of upper case letters A..Z, and the punctuation space, underscore, period,dash, colon and slash.

• Lower case, like upper but with lower case letters a..z instead of upper case

• E-mail, consisting of lower case letters a..z and the punctuation comma, underscore, period, dash,plus and the at sign (@).

• URI, consisting of lower case letters a..z, digits 0..9 and most punctuation available.

• Alpha-numerical, consisting of both upper and lower case letters a..zA..z, digits 0..9 andpunctuation space and underscore.

• Alpha-symbolical, consisting of both upper and lower case letters a..zA..Z and the punctuationspace, underscore, period, dash, colon, slash, plus, comma, apostrophe, at sign, pipe andsemicolon.

• European, consisting of most accented european characters and digits plus punctuation space,dash, underscore and period — like latin1 but with less punctuation.

• Latin 1.

57

• UTF-8.

In addition to the string’s contents, the number of characters also determines if the string can beinlined or not. Each class has its own character count limits, which are

Table 5. Character count limits

String class Charactercount limit

Numerical, Date and Hex 54

Uppercase, Lowercase and E-mail 43

URI, Alphanumerical and Alphasymbolical 36

European 31

Latin1 27

UTF-8 14

That means that the largest inline-able string is 54 characters long and must be of the Numerical classand also that all Strings of size 14 or less will always be inlined.

Also note that the above limits are for the default 41 byte PropertyRecord layout — if that parameter ischanged via editing the source and recompiling, the above have to be recalculated.

6.6. Linux file system tuning

Databases often produce many small and random reads when querying data, and few sequentialwrites when committing changes.

By default, most Linux distributions schedule IO requests using the Completely Fair Queuing (CFQ)algorithm, which provides a good balance between throughput and latency. The particular IOworkload of a database, however, is better served by the Deadline scheduler. The Deadline schedulergives preference to read requests, and processes them as soon as possible. This tends to decrease thelatency of reads, while the latency of writes goes up. Since the writes are usually sequential, theirlingering in the IO queue increases the change of overlapping or adjacent write requests beingmerged together. This effectively reduces the number of writes that are sent to the drive.

On Linux, the IO scheduler for a drive, in this case sda, can be changed at runtime like this:

$ echo 'deadline' > /sys/block/sda/queue/scheduler$ cat /sys/block/sda/queue/schedulernoop [deadline] cfq

Another recommended practice is to disable file and directory access time updates. This way, the filesystem won’t have to issue writes that update this meta-data, thus improving write performance. Thiscan be accomplished by setting the noatime,nodiratime mount options in fstab, or when issuing thedisk mount command.

6.7. Disks, RAM and other tips

As with any persistence solution, performance depends a lot on the persistence media used. Betterdisks equals better performance.

If you have multiple disks or persistence media available it may be a good idea to divide the store filesand transaction logs across those disks. Keeping the store files on disks with low seek time can dowonders for read operations. Today a typical mechanical drive has an average seek time of about5ms. This can cause a query or traversal to be very slow when the amount of RAM assigned to the

58

page cache is too small. A new, good SATA enabled SSD has an average seek time of less than 100microseconds, meaning those scenarios will execute at least 50 times faster. However, this is still tensor hundreds of times slower than accessing RAM.

To avoid hitting disk you need more RAM. On a standard mechanical drive you can handle graphs witha few tens of millions of primitives (nodes, relationships and properties) with 2-3 GBs of RAM. A serverwith 8-16 GBs of RAM can handle graphs with hundreds of millions of primitives, and a good serverwith 16-64 GBs can handle billions of primitives. However, if you invest in a good SSD you will be ableto handle much larger graphs on less RAM.

Use tools like dstat or vmstat to gather information when your application is running. If the swap orpaging numbers are high, that is a sign that the Lucene indexes don’t quite fit in memory. In this case,queries that do index lookups will have high latencies.

When Neo4j starts up, its page cache is empty and needs to warm up. This can take a while, especiallyfor large stores. It is not uncommon to see a long period with many blocks being read from the drive,and high IO wait times.

Neo4j also flushes its page cache in the background, so it is not uncommon to see a steady trickle ofblocks being written to the drive during steady-state. This background flushing only produces a smallamount of IO wait, however. If the IO wait times are high during steady-state, it may be a sign thatNeo4j is bottle-necked on the random IO performance of the drive. The best drives for running Neo4jare fast SSDs that can take lots of random IOPS.

59

Chapter 7. Tutorials

7.1. Set up a Neo4j cluster

This guide will give step-by-step instructions for setting up a basic cluster of three separate machines.For a description of the clustering architecture and related design considerations, refer toIntroduction.

7.1.1. Important configuration settings

Each instance in a Neo4j HA cluster must be assigned an integer ID, which serves as its uniqueidentifier. At startup, a Neo4j instance contacts the other instances specified in the ha.initial_hostsconfiguration option.

When an instance establishes a connection to any other, it determines the current state of the clusterand ensures that it is eligible to join. To be eligible the Neo4j instance must host the same databasestore as other members of the cluster (although it is allowed to be in an older state), or be a newdeployment without a database store.

Please note that IP Addresses or Hostnames should be explicitly configured for the machinesparticipating in the cluster. Neo4j will attempt to configure IP addresses for itself in the absence ofexplicit configuration.

dbms.mode

dbms.mode configures the operating mode of the database.

For cluster mode it is set to: dbms.mode=HA

ha.server_id

ha.server_id is the cluster identifier for each instance. It must be a positive integer and must beunique among all Neo4j instances in the cluster.

For example, ha.server_id=1.

ha.host.coordination

ha.host.coordination is an address/port setting that specifies where the Neo4j instance will listen forcluster communications (like hearbeat messages). The default port is 5001. In the absence of aspecified IP address, Neo4j will attempt to find a valid interface for binding. While this behaviortypically results in a well-behaved server, it is strongly recommended that users explicitly choose an IPaddress bound to the network interface of their choosing to ensure a coherent cluster deployment.

For example, ha.host.coordination=192.168.33.22:5001 will listen for cluster communications on thenetwork interface bound to the 192.168.33.0 subnet on port 5001.

ha.initial_hosts

ha.initial_hosts is a comma separated list of address/port pairs, which specify how to reach otherNeo4j instances in the cluster (as configured via their ha.host.coordination option). Thesehostname/ports will be used when the Neo4j instances start, to allow them to find and join thecluster. Specifying an instance’s own address is permitted. Do not use any whitespace in thisconfiguration option.

60

For example, ha.initial_hosts=192.168.33.22:5001,192.168.33.21:5001 will attempt to reach Neo4jinstances listening on 192.168.33.22 on port 5001 and 192.168.33.21 on port 5001 on the192.168.33.0 subnet.

ha.host.data

ha.host.data is an address/port setting that specifies where the Neo4j instance will listen fortransactions from the cluster master. The default port is 6001. In the absence of a specified IP address,Neo4j will attempt to find a valid interface for binding. While this behavior typically results in a well-behaved server, it is strongly recommended that users explicitly choose an IP address bound to thenetwork interface of their choosing to ensure a coherent cluster topology.

ha.host.data must use a different port to ha.host.coordination.

For example, ha.host.data=192.168.33.22:6001 will listen for transactions from the cluster master onthe network interface bound to the 192.168.33.0 subnet on port 6001.

Address and port formats

The ha.host.coordination and ha.host.data configuration options are specified as<IP address>:<port>.

For ha.host.data the IP address must be the address assigned to one of the host’snetwork interfaces.

For ha.host.coordination the IP address must be the address assigned to one of thehost’s network interfaces, or the value 0.0.0.0, which will cause Neo4j to listen onevery network interface.

Either the address or the port can be omitted, in which case the default for that partwill be used. If the address is omitted, then the port must be preceded with a colon(eg. :5001).

The syntax for setting the port range is: <hostname>:<first port>[-<second port>].In this case, Neo4j will test each port in sequence, and select the first that isunused. Note that this usage is not permitted when the hostname is specified as0.0.0.0 (the "all interfaces" address).

7.1.2. Download and configure

• Download Neo4j Enterprise from the Neo4j download site (http://neo4j.com/download/), and unpackon three separate machines.

• Configure the HA related settings for each installation as outlined below. Note that all threeinstallations have the same configuration except for the ha.server_id property.

Neo4j instance #1 — neo4j-01.local

61


conf/neo4j.conf

# Unique server id for this Neo4j instance# can not be negative id and must be uniqueha.server_id = 1

# List of other known instances in this clusterha.initial_hosts = neo4j-01.local:5001,neo4j-02.local:5001,neo4j-03.local:5001# Alternatively, use IP addresses:#ha.initial_hosts = 192.168.0.20:5001,192.168.0.21:5001,192.168.0.22:5001

# HA - High Availability# SINGLE - Single mode, default.dbms.mode=HA

dbms.connector.http.type=HTTPdbms.connector.http.enabled=truedbms.connector.http.address=0.0.0.0:7474


conf/neo4j.conf






conf/neo4j.conf





7.1.3. Start the Neo4j Servers

Start the Neo4j servers as usual. Note that the startup order does not matter.

neo4j-01$ ./bin/neo4j start


62


Startup Time

When running in HA mode, the startup script returns immediately instead ofwaiting for the server to become available. This is because the instance does notaccept any requests until a cluster has been formed. In the example above thishappens when you start the second instance. To keep track of the startup state youcan follow the messages in neo4j.log — the path is printed before the startup scriptreturns.

Now, you should be able to access the three servers and check their HA status. Open the locationsbelow in a web browser and issue the following command in the editor after having set a passwordfor the database: :play sysinfo

• http://neo4j-01.local:7474/



You can replace database #3 with an 'arbiter' instance, see Arbiter instances.

That’s it! You now have a Neo4j HA cluster of three instances running. You can start by making achange on any instance and those changes will be propagated between them. For more HA relatedconfiguration options take a look at Setup and configuration.

7.2. Set up a local cluster

If you want to start a cluster similar to the one described above, but for development and testingpurposes, it is convenient to run all Neo4j instances on the same machine. This is easy to achieve,although it requires some additional configuration as the defaults will conflict with each other.Furthermore, the default dbms.memory.pagecache.size assumes that Neo4j has the machine to itself. Ifwe in this example assume that the machine has 4 gigabytes of memory, and that each JVM consumes500 megabytes of memory, then we can allocate 500 megabytes of memory to the page cache of eachserver.

7.2.1. Download and configure

1. Download Neo4j Enterprise from the Neo4j download site (http://neo4j.com/download/), and unpackinto three separate directories on your test machine.

2. Configure the HA related settings for each installation as outlined below.

Neo4j instance #1 — ~/neo4j-01

63

http://neo4j-01.local:7474/




conf/neo4j.conf

# Reduce the default page cache memory allocationdbms.memory.pagecache.size=500m

# Port to listen to for incoming backup requests.dbms.backup.address = 127.0.0.1:6366


# List of other known instances in this clusterha.initial_hosts = 127.0.0.1:5001,127.0.0.1:5002,127.0.0.1:5003

# IP and port for this instance to bind to for communicating cluster information# with the other neo4j instances in the cluster.ha.host.coordination = 127.0.0.1:5001

# IP and port for this instance to bind to for communicating data with the# other neo4j instances in the cluster.ha.host.data = 127.0.0.1:6363




conf/neo4j.conf










64

conf/neo4j.conf









Start the Neo4j Servers

Start the Neo4j servers as usual. Note that the startup order does not matter.

localhost:~/neo4j-01$ ./bin/neo4j start



Now, you should be able to access the three servers and check their HA status. Open the locationsbelow in a web browser and issue the following command in the editor after having set a passwordfor the database: :play sysinfo

• http://127.0.0.1:7474/

• http://127.0.0.1:7475/

• http://127.0.0.1:7476/

65

http://127.0.0.1:7474/

http://127.0.0.1:7475/

http://127.0.0.1:7476/

Chapter 8. Configuration

8.1. Configuration Settings Reference

This page documents Neo4j’s configuration settings. They can be set in neo4j.conf.

Table 6. Settings used by the server configuration

Name Description

browser.allow_outgoing_connections Configure the policy for outgoing Neo4j Browserconnections.

browser.credential_timeout Configure the Neo4j Browser to time out logged in usersafter this idle period.

browser.remote_content_hostname_whitelist Whitelist of hosts for the Neo4j Browser to be allowed tofetch content from.

browser.retain_connection_credentials Configure the Neo4j Browser to store or not store usercredentials.

cypher.default_language_version Set this to specify the default parser (language version).

cypher.forbid_exhaustive_shortestpath This setting is associated with performance optimization.

cypher.hints_error Set this to specify the behavior when Cypher planner orruntime hints cannot be fulfilled.

cypher.min_replan_interval The minimum lifetime of a query plan before a query isconsidered for replanning.

cypher.planner Set this to specify the default planner for the defaultlanguage version.

cypher.statistics_divergence_threshold The threshold when a plan is considered stale.

dbms.active_database Name of the database to load.

dbms.allow_format_migration Whether to allow a store upgrade in case the currentversion of the database starts against an older storeversion.

dbms.backup.address Listening server for online backups.

dbms.backup.enabled Enable support for running online backups.

dbms.checkpoint.interval.time Configures the time interval between check-points.

dbms.checkpoint.interval.tx Configures the transaction interval between check-points.

dbms.checkpoint.iops.limit Limit the number of IOs the background checkpointprocess will consume per second.

dbms.directories.certificates Directory for storing certificates to be used by Neo4j forTLS connections.

dbms.directories.data Path of the data directory.

dbms.directories.import Sets the root directory for file URLs used with the CypherLOAD CSV clause.

dbms.directories.lib Path of the lib directory.

dbms.directories.logs Path of the logs directory.

dbms.directories.metrics The target location of the CSV files: a path to a directorywherein a CSV file per reported field will be written.

dbms.directories.plugins Location of the database plugin directory.

dbms.directories.run Path of the run directory.

dbms.index_sampling.background_enabled Enable or disable background index sampling.

dbms.index_sampling.buffer_size Size of buffer used by index sampling.

66

Name Description

dbms.index_sampling.update_percentage Percentage of index updates of total index size requiredbefore sampling of a given index is triggered.

dbms.index_searcher_cache_size The maximum number of open Lucene index searchers.

dbms.logs.debug.level Debug log level threshold.

dbms.logs.debug.rotation.delay Minimum time interval after last rotation of the debug logbefore it may be rotated again.

dbms.logs.debug.rotation.keep_number Maximum number of history files for the debug log.

dbms.logs.debug.rotation.size Threshold for rotation of the debug log.

dbms.logs.gc.enabled Enable GC Logging.

dbms.logs.gc.options GC Logging Options.

dbms.logs.gc.rotation.keep_number Number of GC logs to keep.

dbms.logs.gc.rotation.size Size of each GC log that is kept.

dbms.logs.http.enabled Enable HTTP request logging.

dbms.logs.http.rotation.keep_number Number of HTTP logs to keep.

dbms.logs.http.rotation.size Size of each HTTP log that is kept.

dbms.logs.query.enabled Log executed queries that takes longer than theconfigured threshold.

dbms.logs.query.parameter_logging_enabled Log parameters for executed queries that took longer thanthe configured threshold.

dbms.logs.query.rotation.keep_number Maximum number of history files for the query log.

dbms.logs.query.rotation.size The file size in bytes at which the query log will auto-rotate.

dbms.logs.query.threshold If the execution of query takes more time than thisthreshold, the query is logged - provided query logging isenabled.

dbms.memory.pagecache.size The amount of memory to use for mapping the store files,in bytes (or kilobytes with the 'k' suffix, megabytes with 'm'and gigabytes with 'g').

dbms.memory.pagecache.swapper Specify which page swapper to use for doing paged IO.

dbms.mode Configure the operating mode of the database — 'SINGLE'for stand-alone operation, 'HA' for operating as a memberin a cluster or 'ARBITER' for an HA-only cluster memberwith no database.

dbms.query_cache_size The number of Cypher query execution plans that arecached.

dbms.read_only Only allow read operations from this Neo4j instance.

dbms.record_format Database record format.

dbms.relationship_grouping_threshold Relationship count threshold for considering a node to bedense.

dbms.security.allow_csv_import_from_file_urls Determines if Cypher will allow using file URLs whenloading data using LOAD CSV.

dbms.security.auth_enabled Enable auth requirement to access Neo4j.

dbms.security.ha_status_auth_enabled Require authorization for access to the HA statusendpoints.

dbms.security.http_authorization_classes Comma-seperated list of custom security rules for Neo4jto use.

dbms.shell.enabled Enable a remote shell server which Neo4j Shell clients canlog in to.

67

Name Description

dbms.shell.host Remote host for shell.

dbms.shell.port The port the shell will listen on.

dbms.shell.read_only Read only mode.

dbms.shell.rmi_name The name of the shell.

dbms.threads.worker_count Number of Neo4j worker threads, your OS might enforce alower limit than the maximum value specified here.

dbms.transaction_timeout Timeout for idle transactions.

dbms.tx_log.rotation.retention_policy Make Neo4j keep the logical transaction logs for being ableto backup the database.

dbms.tx_log.rotation.size Specifies at which file size the logical log will auto-rotate.

dbms.udc.enabled Enable the UDC extension.

dbms.unmanaged_extension_classes Comma-separated list of <classname>=<mount point> forunmanaged extensions.

ha.allow_init_cluster Whether to allow this instance to create a cluster if unableto join.

ha.branched_data_policy Policy for how to handle branched data.

ha.broadcast_timeout Timeout for broadcasting values in cluster.

ha.configuration_timeout Timeout for waiting for configuration from an existingcluster member during cluster join.

ha.data_chunk_size Max size of the data chunks that flows between masterand slaves in HA.

ha.default_timeout Default timeout used for clustering timeouts.

ha.election_timeout Timeout for waiting for other members to finish a roleelection.

ha.heartbeat_interval How often heartbeat messages should be sent.

ha.heartbeat_timeout Timeout for heartbeats between cluster members.

ha.host.coordination Host and port to bind the cluster managementcommunication.

ha.host.data Hostname and port to bind the HA server.

ha.initial_hosts A comma-separated list of other members of the cluster tojoin.

ha.internal_role_switch_timeout Timeout for waiting for internal conditions during stateswitch, like for transactions to complete, before switchingto master or slave.

ha.join_timeout Timeout for joining a cluster.

ha.learn_timeout Timeout for learning values.

ha.leave_timeout Timeout for waiting for cluster leave to finish.

ha.max_channels_per_slave Maximum number of connections a slave can have to themaster.

ha.paxos_timeout Default timeout for all Paxos timeouts.

ha.phase1_timeout Timeout for Paxos phase 1.

ha.phase2_timeout Timeout for Paxos phase 2.

ha.pull_batch_size Size of batches of transactions applied on slaves whenpulling from master.

ha.pull_interval Interval of pulling updates from master.

68

Name Description

ha.role_switch_timeout Timeout for request threads waiting for instance tobecome master or slave.

ha.server_id Id for a cluster instance.

ha.slave_lock_timeout Timeout for taking remote (write) locks on slaves.

ha.slave_only Whether this instance should only participate as slave incluster.

ha.slave_read_timeout How long a slave will wait for response from master beforegiving up.

ha.tx_push_factor The amount of slaves the master will ask to replicate acommitted transaction.

ha.tx_push_strategy Push strategy of a transaction to a slave during commit.

metrics.bolt.messages.enabled Enable reporting metrics about Bolt Protocol messageprocessing.

metrics.csv.enabled Set to true to enable exporting metrics to CSV files.

metrics.csv.interval The reporting interval for the CSV files.

metrics.cypher.replanning.enabled Enable reporting metrics about number of occurredreplanning events.

metrics.enabled The default enablement value for all the supportedmetrics.

metrics.graphite.enabled Set to true to enable exporting metrics to Graphite.

metrics.graphite.interval The reporting interval for Graphite.

metrics.graphite.server The hostname or IP address of the Graphite server.

metrics.jvm.buffers.enabled Enable reporting metrics about the buffer pools.

metrics.jvm.gc.enabled Enable reporting metrics about the duration of garbagecollections.

metrics.jvm.memory.enabled Enable reporting metrics about the memory usage.

metrics.jvm.threads.enabled Enable reporting metrics about the current number ofthreads running.

metrics.neo4j.checkpointing.enabled Enable reporting metrics about Neo4j check pointing.

metrics.neo4j.cluster.enabled Enable reporting metrics about HA cluster info.

metrics.neo4j.counts.enabled Enable reporting metrics about approximately how manyentities are in the database.

metrics.neo4j.enabled The default enablement value for all Neo4j specificsupport metrics.

metrics.neo4j.logrotation.enabled Enable reporting metrics about the Neo4j log rotation.

metrics.neo4j.network.enabled Enable reporting metrics about the network usage.

metrics.neo4j.pagecache.enabled Enable reporting metrics about the Neo4j page cache.

metrics.neo4j.server.enabled Enable reporting metrics about Server threading info.

metrics.neo4j.tx.enabled Enable reporting metrics about transactions.

metrics.prefix A common prefix for the reported metrics field names.

tools.consistency_checker.check_graph Perform checks between nodes, relationships, properties,types and tokens.

tools.consistency_checker.check_indexes Perform checks on indexes.

tools.consistency_checker.check_label_scan_store Perform checks on the label scan store.

tools.consistency_checker.check_property_owners Perform optional additional checking on propertyownership.

69

Table 7. browser.allow_outgoing_connections

Description Configure the policy for outgoing Neo4j Browser connections.

Valid values browser.allow_outgoing_connections is a boolean

Default value true

Table 8. browser.credential_timeout

Description Configure the Neo4j Browser to time out logged in users after this idle period. Setting this to 0indicates no limit.

Valid values browser.credential_timeout is a duration (valid units are ms, s, m)

Default value 0

Table 9. browser.remote_content_hostname_whitelist

Description Whitelist of hosts for the Neo4j Browser to be allowed to fetch content from.

Valid values browser.remote_content_hostname_whitelist is a string

Default value http://guides.neo4j.com,https://guides.neo4j.com,http://localhost,https://localhost

Table 10. browser.retain_connection_credentials

Description Configure the Neo4j Browser to store or not store user credentials.

Valid values browser.retain_connection_credentials is a boolean

Default value true

Table 11. cypher.default_language_version

Description Set this to specify the default parser (language version).

Valid values cypher.default_language_version is one of 2.3, 3.0, default

Default value default

Table 12. cypher.forbid_exhaustive_shortestpath

Description This setting is associated with performance optimization. Set this to true in situations where itis preferable to have any queries using the 'shortestPath' function terminate as soon aspossible with no answer, rather than potentially running for a long time attempting to find ananswer (even if there is no path to be found). For most queries, the 'shortestPath' algorithmwill return the correct answer very quickly. However there are some cases where it is possiblethat the fast bidirectional breadth-first search algorithm will find no results even if they exist.This can happen when the predicates in the WHERE clause applied to 'shortestPath' cannot beapplied to each step of the traversal, and can only be applied to the entire path. When thequery planner detects these special cases, it will plan to perform an exhaustive depth-firstsearch if the fast algorithm finds no paths. However, the exhaustive search may be orders ofmagnitude slower than the fast algorithm. If it is critical that queries terminate as soon aspossible, it is recommended that this option be set to true, which means that Neo4j will neverconsider using the exhaustive search for shortestPath queries. However, please note that if nopaths are found, an error will be thrown at run time, which will need to be handled by theapplication.

Valid values cypher.forbid_exhaustive_shortestpath is a boolean

Default value false

Table 13. cypher.hints_error

Description Set this to specify the behavior when Cypher planner or runtime hints cannot be fulfilled. Iftrue, then non-conformance will result in an error, otherwise only a warning is generated.

Valid values cypher.hints_error is a boolean

Default value false

Table 14. cypher.min_replan_interval

70

http://guides.neo4j.com,https://guides.neo4j.com,http://localhost,https://localhost

Description The minimum lifetime of a query plan before a query is considered for replanning.

Valid values cypher.min_replan_interval is a duration (valid units are ms, s, m)

Default value 1000

Table 15. cypher.planner

Description Set this to specify the default planner for the default language version.

Valid values cypher.planner is one of COST, RULE, default

Default value default

Table 16. cypher.statistics_divergence_threshold

Description The threshold when a plan is considered stale. If any of the underlying statistics used to createthe plan has changed more than this value, the plan is considered stale and will be replanned.A value of 0 means always replan, and 1 means never replan.

Valid values cypher.statistics_divergence_threshold is a double which is minimum 0.0, and is maximum 1.0

Default value 0.5

Table 17. dbms.active_database

Description Name of the database to load.

Valid values dbms.active_database is a string

Default value graph.db

Table 18. dbms.allow_format_migration

Description Whether to allow a store upgrade in case the current version of the database starts against anolder store version. Setting this to true does not guarantee successful upgrade, it just allows anupgrade to be performed.

Valid values dbms.allow_format_migration is a boolean

Default value false

Table 19. dbms.backup.address

Description Listening server for online backups.

Valid values dbms.backup.address is a hostname and port

Default value 127.0.0.1:6362-6372

Table 20. dbms.backup.enabled

Description Enable support for running online backups.

Valid values dbms.backup.enabled is a boolean

Default value true

Table 21. dbms.checkpoint.interval.time

Description Configures the time interval between check-points. The database will not check-point moreoften than this (unless check pointing is triggered by a different event), but might check-pointless often than this interval, if performing a check-point takes longer time than the configuredinterval. A check-point is a point in the transaction logs, from which recovery would start from.Longer check-point intervals typically means that recovery will take longer to complete in caseof a crash. On the other hand, a longer check-point interval can also reduce the I/O load thatthe database places on the system, as each check-point implies a flushing and forcing of all thestore files. The default is '5m' for a check-point every 5 minutes. Other supported units are 's'for seconds, and 'ms' for milliseconds.

Valid values dbms.checkpoint.interval.time is a duration (valid units are ms, s, m)

71

Default value 300000

Table 22. dbms.checkpoint.interval.tx

Description Configures the transaction interval between check-points. The database will not check-pointmore often than this (unless check pointing is triggered by a different event), but might check-point less often than this interval, if performing a check-point takes longer time than theconfigured interval. A check-point is a point in the transaction logs, from which recovery wouldstart from. Longer check-point intervals typically means that recovery will take longer tocomplete in case of a crash. On the other hand, a longer check-point interval can also reducethe I/O load that the database places on the system, as each check-point implies a flushing andforcing of all the store files. The default is '100000' for a check-point every 100000transactions.

Valid values dbms.checkpoint.interval.tx is an integer which is minimum 1


Table 23. dbms.checkpoint.iops.limit

Description Limit the number of IOs the background checkpoint process will consume per second. Thissetting is advisory, is ignored in Neo4j Community Edition, and is followed to best effort inEnterprise Edition. An IO is in this case a 8 KiB (mostly sequential) write. Limiting the write IO inthis way will leave more bandwidth in the IO subsystem to service random-read IOs, which isimportant for the response time of queries when the database cannot fit entirely in memory.The only drawback of this setting is that longer checkpoint times may lead to slightly longerrecovery times in case of a database or system crash. A lower number means lower IOpressure, and consequently longer checkpoint times. The configuration can also becommented out to remove the limitation entirely, and let the checkpointer flush data as fast asthe hardware will go. Set this to -1 to disable the IOPS limit.

Valid values dbms.checkpoint.iops.limit is an integer

Default value 1000

Table 24. dbms.directories.certificates

Description Directory for storing certificates to be used by Neo4j for TLS connections.

Valid values A filesystem path; relative paths are resolved against the installation root, <neo4j-home>

Default value certificates

Table 25. dbms.directories.data

Description Path of the data directory. You must not configure more than one Neo4j installation to use thesame data directory.


Default value data

Table 26. dbms.directories.import

Description Sets the root directory for file URLs used with the Cypher LOAD CSV clause. This must be set to asingle directory, restricting access to only those files within that directory and itssubdirectories.


Table 27. dbms.directories.lib

Description Path of the lib directory.


Default value lib

Table 28. dbms.directories.logs

Description Path of the logs directory.

72


Default value logs

Table 29. dbms.directories.metrics

Description The target location of the CSV files: a path to a directory wherein a CSV file per reported fieldwill be written.


Default value metrics

Table 30. dbms.directories.plugins

Description Location of the database plugin directory. Compiled Java JAR files that contain databaseprocedures will be loaded if they are placed in this directory.


Default value plugins

Table 31. dbms.directories.run

Description Path of the run directory.


Default value run

Table 32. dbms.index_sampling.background_enabled

Description Enable or disable background index sampling.

Valid values dbms.index_sampling.background_enabled is a boolean

Default value true

Table 33. dbms.index_sampling.buffer_size

Description Size of buffer used by index sampling.

Valid values dbms.index_sampling.buffer_size is a byte size (valid multipliers are k, m, g, K, M, G) which isminimum 1048576, and is maximum 2147483647


Table 34. dbms.index_sampling.update_percentage

Description Percentage of index updates of total index size required before sampling of a given index istriggered.

Valid values dbms.index_sampling.update_percentage is an integer which is minimum 0

Default value 5

Table 35. dbms.index_searcher_cache_size

Description The maximum number of open Lucene index searchers.

Valid values dbms.index_searcher_cache_size is an integer which is minimum 1


Table 36. dbms.logs.debug.level

Description Debug log level threshold.

Valid values dbms.logs.debug.level is one of DEBUG, INFO, WARN, ERROR, NONE

Default value INFO

73

Table 37. dbms.logs.debug.rotation.delay

Description Minimum time interval after last rotation of the debug log before it may be rotated again.

Valid values dbms.logs.debug.rotation.delay is a duration (valid units are ms, s, m)


Table 38. dbms.logs.debug.rotation.keep_number

Description Maximum number of history files for the debug log.

Valid values dbms.logs.debug.rotation.keep_number is an integer which is minimum 1

Default value 7

Table 39. dbms.logs.debug.rotation.size

Description Threshold for rotation of the debug log.

Valid values dbms.logs.debug.rotation.size is a byte size (valid multipliers are k, m, g, K, M, G) which isminimum 0, and is maximum 9223372036854775807


Table 40. dbms.logs.gc.enabled

Description Enable GC Logging.

Valid values dbms.logs.gc.enabled is a boolean

Default value false

Table 41. dbms.logs.gc.options

Description GC Logging Options.

Valid values dbms.logs.gc.options is a string

Default value -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCApplicationStoppedTime-XX:+PrintPromotionFailure -XX:+PrintTenuringDistribution

Table 42. dbms.logs.gc.rotation.keep_number

Description Number of GC logs to keep.

Valid values dbms.logs.gc.rotation.keep_number is an integer

Default value 5

Table 43. dbms.logs.gc.rotation.size

Description Size of each GC log that is kept.

Valid values dbms.logs.gc.rotation.size is a byte size (valid multipliers are k, m, g, K, M, G) which is minimum 0,and is maximum 9223372036854775807


Table 44. dbms.logs.http.enabled

Description Enable HTTP request logging.

Valid values dbms.logs.http.enabled is a boolean

Default value false

Table 45. dbms.logs.http.rotation.keep_number

Description Number of HTTP logs to keep.

Valid values dbms.logs.http.rotation.keep_number is an integer

74

Default value 5

Table 46. dbms.logs.http.rotation.size

Description Size of each HTTP log that is kept.

Valid values dbms.logs.http.rotation.size is a byte size (valid multipliers are k, m, g, K, M, G) which is minimum0, and is maximum 9223372036854775807


Table 47. dbms.logs.query.enabled

Description Log executed queries that takes longer than the configured threshold. NOTE: This feature is onlyavailable in the Neo4j Enterprise Edition.

Valid values dbms.logs.query.enabled is a boolean

Default value false

Table 48. dbms.logs.query.parameter_logging_enabled

Description Log parameters for executed queries that took longer than the configured threshold.

Valid values dbms.logs.query.parameter_logging_enabled is a boolean

Default value true

Table 49. dbms.logs.query.rotation.keep_number

Description Maximum number of history files for the query log.

Valid values dbms.logs.query.rotation.keep_number is an integer which is minimum 1

Default value 7

Table 50. dbms.logs.query.rotation.size

Description The file size in bytes at which the query log will auto-rotate. If set to zero then no rotation willoccur. Accepts a binary suffix k, m or g.

Valid values dbms.logs.query.rotation.size is a byte size (valid multipliers are k, m, g, K, M, G) which isminimum 0, and is maximum 9223372036854775807


Table 51. dbms.logs.query.threshold

Description If the execution of query takes more time than this threshold, the query is logged - providedquery logging is enabled. Defaults to 0 seconds, that is all queries are logged.

Valid values dbms.logs.query.threshold is a duration (valid units are ms, s, m)

Default value 0

Table 52. dbms.memory.pagecache.size

Description The amount of memory to use for mapping the store files, in bytes (or kilobytes with the 'k'suffix, megabytes with 'm' and gigabytes with 'g'). If Neo4j is running on a dedicated server,then it is generally recommended to leave about 2-4 gigabytes for the operating system, givethe JVM enough heap to hold all your transaction state and query context, and then leave therest for the page cache. The default page cache memory assumes the machine is dedicated torunning Neo4j, and is heuristically set to 50% of RAM minus the max Java heap size.

Valid values dbms.memory.pagecache.size is a byte size (valid multipliers are k, m, g, K, M, G) which isminimum 245760


Table 53. dbms.memory.pagecache.swapper

75

Description Specify which page swapper to use for doing paged IO. This is only used when integrating withproprietary storage technology.

Valid values dbms.memory.pagecache.swapper is a string

Table 54. dbms.mode

Description Configure the operating mode of the database — 'SINGLE' for stand-alone operation, 'HA' foroperating as a member in a cluster or 'ARBITER' for an HA-only cluster member with nodatabase.

Valid values dbms.mode is a string

Default value SINGLE

Table 55. dbms.query_cache_size

Description The number of Cypher query execution plans that are cached.

Valid values dbms.query_cache_size is an integer which is minimum 0

Default value 1000

Table 56. dbms.read_only

Description Only allow read operations from this Neo4j instance. This mode still requires write access tothe directory for lock purposes.

Valid values dbms.read_only is a boolean

Default value false

Table 57. dbms.record_format

Description Database record format. Enterprise edition only. Valid values: standard,<<config_high_limit,high_limit>>. Default value: standard.

Valid values dbms.record_format is a string

Default value

Table 58. dbms.relationship_grouping_threshold

Description Relationship count threshold for considering a node to be dense.

Valid values dbms.relationship_grouping_threshold is an integer which is minimum 1

Default value 50

Table 59. dbms.security.allow_csv_import_from_file_urls

Description Determines if Cypher will allow using file URLs when loading data using LOAD CSV. Setting thisvalue to false will cause Neo4j to fail LOAD CSV clauses that load data from the file system.

Valid values dbms.security.allow_csv_import_from_file_urls is a boolean

Default value true

Table 60. dbms.security.auth_enabled

Description Enable auth requirement to access Neo4j.

Valid values dbms.security.auth_enabled is a boolean

Default value false

Table 61. dbms.security.ha_status_auth_enabled

Description Require authorization for access to the HA status endpoints.

Valid values dbms.security.ha_status_auth_enabled is a boolean

76

Default value true

Table 62. dbms.security.http_authorization_classes

Description Comma-seperated list of custom security rules for Neo4j to use.

Valid values dbms.security.http_authorization_classes is a comma-seperated string

Default value []

Table 63. dbms.shell.enabled

Description Enable a remote shell server which Neo4j Shell clients can log in to.

Valid values dbms.shell.enabled is a boolean

Default value false

Table 64. dbms.shell.host

Description Remote host for shell. By default, the shell server listens only on the loopback interface, butyou can specify the IP address of any network interface or use 0.0.0.0 for all interfaces.

Valid values dbms.shell.host is a string which must be a valid name

Default value 127.0.0.1

Table 65. dbms.shell.port

Description The port the shell will listen on.

Valid values dbms.shell.port is an integer which must be a valid port number (is in the range 0 to 65535)

Default value 1337

Table 66. dbms.shell.read_only

Description Read only mode. Will only allow read operations.

Valid values dbms.shell.read_only is a boolean

Default value false

Table 67. dbms.shell.rmi_name

Description The name of the shell.

Valid values dbms.shell.rmi_name is a string which must be a valid name

Default value shell

Table 68. dbms.threads.worker_count

Description Number of Neo4j worker threads, your OS might enforce a lower limit than the maximumvalue specified here.

Valid values dbms.threads.worker_count is an integer which is in the range 1 to 44738

Default value 2

Table 69. dbms.transaction_timeout

Description Timeout for idle transactions.

Valid values dbms.transaction_timeout is a duration (valid units are ms, s, m)

Default value 60000

Table 70. dbms.tx_log.rotation.retention_policy

77

Description Make Neo4j keep the logical transaction logs for being able to backup the database. Can beused for specifying the threshold to prune logical logs after. For example "10 days" will prunelogical logs that only contains transactions older than 10 days from the current time, or "100ktxs" will keep the 100k latest transactions and prune any older transactions.

Valid values dbms.tx_log.rotation.retention_policy is a string which must be true/false or of format'<number><optional unit> <type>' for example 100M size for limiting logical log space on diskto 100Mb, or 200k txs for limiting the number of transactions to keep to 200 000

Default value 7 days

Table 71. dbms.tx_log.rotation.size

Description Specifies at which file size the logical log will auto-rotate. 0 means that no rotation willautomatically occur based on file size.

Valid values dbms.tx_log.rotation.size is a byte size (valid multipliers are k, m, g, K, M, G) which is minimum1048576


Table 72. dbms.udc.enabled

Description Enable the UDC extension.

Valid values dbms.udc.enabled is a boolean

Default value true

Table 73. dbms.unmanaged_extension_classes

Description Comma-separated list of <classname>=<mount point> for unmanaged extensions.

Valid values dbms.unmanaged_extension_classes is a comma-seperated list of <classname>=<mountpoint> strings

Default value []

Table 74. ha.allow_init_cluster

Description Whether to allow this instance to create a cluster if unable to join.

Valid values ha.allow_init_cluster is a boolean

Default value true

Table 75. ha.branched_data_policy

Description Policy for how to handle branched data.

Valid values ha.branched_data_policy is one of keep_all, keep_last, keep_none

Default value keep_all

Table 76. ha.broadcast_timeout

Description Timeout for broadcasting values in cluster. Must consider end-to-end duration of Paxosalgorithm. This value is the default value for the <<config_ha.join_timeout,ha.join_timeout>>and <<config_ha.leave_timeout,ha.leave_timeout>> settings.

Valid values ha.broadcast_timeout is a duration (valid units are ms, s, m)

Default value 30000

Table 77. ha.configuration_timeout

Description Timeout for waiting for configuration from an existing cluster member during cluster join.

Valid values ha.configuration_timeout is a duration (valid units are ms, s, m)

Default value 1000

78

Table 78. ha.data_chunk_size

Description Max size of the data chunks that flows between master and slaves in HA. Bigger size mayincrease throughput, but may also be more sensitive to variations in bandwidth, whereas lowersize increases tolerance for bandwidth variations.

Valid values ha.data_chunk_size is a byte size (valid multipliers are k, m, g, K, M, G) which is minimum 1024


Table 79. ha.default_timeout

Description Default timeout used for clustering timeouts. Override specific timeout settings with propervalues if necessary. This value is the default value for the<<config_ha.heartbeat_interval,ha.heartbeat_interval>>,<<config_ha.paxos_timeout,ha.paxos_timeout>> and<<config_ha.learn_timeout,ha.learn_timeout>> settings.

Valid values ha.default_timeout is a duration (valid units are ms, s, m)

Default value 5000

Table 80. ha.election_timeout

Description Timeout for waiting for other members to finish a role election. Defaults to<<config_ha.paxos_timeout,ha.paxos_timeout>>.

Valid values ha.election_timeout is a duration (valid units are ms, s, m)

Default value 5000

Table 81. ha.heartbeat_interval

Description How often heartbeat messages should be sent. Defaults to<<config_ha.default_timeout,ha.default_timeout>>.

Valid values ha.heartbeat_interval is a duration (valid units are ms, s, m)

Default value 5000

Table 82. ha.heartbeat_timeout

Description Timeout for heartbeats between cluster members. Should be at least twice that of<<config_ha.heartbeat_interval,ha.heartbeat_interval>>.

Valid values ha.heartbeat_timeout is a duration (valid units are ms, s, m)

Default value 11000

Table 83. ha.host.coordination

Description Host and port to bind the cluster management communication.

Valid values ha.host.coordination is a hostname and port

Default value 0.0.0.0:5001-5099

Table 84. ha.host.data

Description Hostname and port to bind the HA server.

Valid values ha.host.data is a hostname and port

Default value 0.0.0.0:6001-6011

Table 85. ha.initial_hosts

Description A comma-separated list of other members of the cluster to join.

Valid values ha.initial_hosts is a list separated by "," where items are a hostname and port

Mandatory The ha.initial_hosts configuration setting is mandatory.

79

Table 86. ha.internal_role_switch_timeout

Description Timeout for waiting for internal conditions during state switch, like for transactions tocomplete, before switching to master or slave.

Valid values ha.internal_role_switch_timeout is a duration (valid units are ms, s, m)

Default value 10000

Table 87. ha.join_timeout

Description Timeout for joining a cluster. Defaults to<<config_ha.broadcast_timeout,ha.broadcast_timeout>>.

Valid values ha.join_timeout is a duration (valid units are ms, s, m)

Default value 30000

Table 88. ha.learn_timeout

Description Timeout for learning values. Defaults to <<config_ha.default_timeout,ha.default_timeout>>.

Valid values ha.learn_timeout is a duration (valid units are ms, s, m)

Default value 5000

Table 89. ha.leave_timeout

Description Timeout for waiting for cluster leave to finish. Defaults to<<config_ha.broadcast_timeout,ha.broadcast_timeout>>.

Valid values ha.leave_timeout is a duration (valid units are ms, s, m)

Default value 30000

Table 90. ha.max_channels_per_slave

Description Maximum number of connections a slave can have to the master.

Valid values ha.max_channels_per_slave is an integer which is minimum 1

Default value 20

Table 91. ha.paxos_timeout

Description Default timeout for all Paxos timeouts. Defaults to<<config_ha.default_timeout,ha.default_timeout>>. This value is the default value for the<<config_ha.phase1_timeout,ha.phase1_timeout>>,<<config_ha.phase2_timeout,ha.phase2_timeout>> and<<config_ha.election_timeout,ha.election_timeout>> settings.

Valid values ha.paxos_timeout is a duration (valid units are ms, s, m)

Default value 5000

Table 92. ha.phase1_timeout

Description Timeout for Paxos phase 1. Defaults to <<config_ha.paxos_timeout,ha.paxos_timeout>>.

Valid values ha.phase1_timeout is a duration (valid units are ms, s, m)

Default value 5000

Table 93. ha.phase2_timeout

Description Timeout for Paxos phase 2. Defaults to <<config_ha.paxos_timeout,ha.paxos_timeout>>.

Valid values ha.phase2_timeout is a duration (valid units are ms, s, m)

Default value 5000

Table 94. ha.pull_batch_size

80

Description Size of batches of transactions applied on slaves when pulling from master.

Valid values ha.pull_batch_size is an integer

Default value 100

Table 95. ha.pull_interval

Description Interval of pulling updates from master.

Valid values ha.pull_interval is a duration (valid units are ms, s, m)

Default value 0

Table 96. ha.role_switch_timeout

Description Timeout for request threads waiting for instance to become master or slave.

Valid values ha.role_switch_timeout is a duration (valid units are ms, s, m)


Table 97. ha.server_id

Description Id for a cluster instance. Must be unique within the cluster.

Valid values ha.server_id is an instance id, which has to be a valid integer

Mandatory The ha.server_id configuration setting is mandatory.

Table 98. ha.slave_lock_timeout

Description Timeout for taking remote (write) locks on slaves. Defaults to<<config_ha.slave_read_timeout,ha.slave_read_timeout>>.

Valid values ha.slave_lock_timeout is a duration (valid units are ms, s, m)

Default value 20000

Table 99. ha.slave_only

Description Whether this instance should only participate as slave in cluster. If set to true, it will never beelected as master.

Valid values ha.slave_only is a boolean

Default value false

Table 100. ha.slave_read_timeout

Description How long a slave will wait for response from master before giving up.

Valid values ha.slave_read_timeout is a duration (valid units are ms, s, m)

Default value 20000

Table 101. ha.tx_push_factor

Description The amount of slaves the master will ask to replicate a committed transaction.

Valid values ha.tx_push_factor is an integer which is minimum 0

Default value 1

Table 102. ha.tx_push_strategy

Description Push strategy of a transaction to a slave during commit.

Valid values ha.tx_push_strategy is one of round_robin, fixed_descending, fixed_ascending

Default value fixed_ascending

81

Table 103. metrics.bolt.messages.enabled

Description Enable reporting metrics about Bolt Protocol message processing.

Valid values metrics.bolt.messages.enabled is a boolean

Default value false

Table 104. metrics.csv.enabled

Description Set to true to enable exporting metrics to CSV files.

Valid values metrics.csv.enabled is a boolean

Default value false

Table 105. metrics.csv.interval

Description The reporting interval for the CSV files. That is, how often new rows with numbers areappended to the CSV files.

Valid values metrics.csv.interval is a duration (valid units are ms, s, m)

Default value 3000

Table 106. metrics.cypher.replanning.enabled

Description Enable reporting metrics about number of occurred replanning events.

Valid values metrics.cypher.replanning.enabled is a boolean

Default value false

Table 107. metrics.enabled

Description The default enablement value for all the supported metrics. Set this to false to turn off allmetrics by default. The individual settings can then be used to selectively re-enable specificmetrics.

Valid values metrics.enabled is a boolean

Default value false

Table 108. metrics.graphite.enabled

Description Set to true to enable exporting metrics to Graphite.

Valid values metrics.graphite.enabled is a boolean

Default value false

Table 109. metrics.graphite.interval

Description The reporting interval for Graphite. That is, how often to send updated metrics to Graphite.

Valid values metrics.graphite.interval is a duration (valid units are ms, s, m)

Default value 3000

Table 110. metrics.graphite.server

Description The hostname or IP address of the Graphite server.

Valid values metrics.graphite.server is a hostname and port

Default value :2003

Table 111. metrics.jvm.buffers.enabled

Description Enable reporting metrics about the buffer pools.

82

Valid values metrics.jvm.buffers.enabled is a boolean

Default value false

Table 112. metrics.jvm.gc.enabled

Description Enable reporting metrics about the duration of garbage collections.

Valid values metrics.jvm.gc.enabled is a boolean

Default value false

Table 113. metrics.jvm.memory.enabled

Description Enable reporting metrics about the memory usage.

Valid values metrics.jvm.memory.enabled is a boolean

Default value false

Table 114. metrics.jvm.threads.enabled

Description Enable reporting metrics about the current number of threads running.

Valid values metrics.jvm.threads.enabled is a boolean

Default value false

Table 115. metrics.neo4j.checkpointing.enabled

Description Enable reporting metrics about Neo4j check pointing; when it occurs and how much time ittakes to complete.

Valid values metrics.neo4j.checkpointing.enabled is a boolean

Default value false

Table 116. metrics.neo4j.cluster.enabled

Description Enable reporting metrics about HA cluster info.

Valid values metrics.neo4j.cluster.enabled is a boolean

Default value false

Table 117. metrics.neo4j.counts.enabled

Description Enable reporting metrics about approximately how many entities are in the database; nodes,relationships, properties, etc.

Valid values metrics.neo4j.counts.enabled is a boolean

Default value false

Table 118. metrics.neo4j.enabled

Description The default enablement value for all Neo4j specific support metrics. Set this to false to turn offall Neo4j specific metrics by default. The individual metrics.neo4j.* metrics can then be turnedon selectively.

Valid values metrics.neo4j.enabled is a boolean

Default value false

Table 119. metrics.neo4j.logrotation.enabled

Description Enable reporting metrics about the Neo4j log rotation; when it occurs and how much time ittakes to complete.

Valid values metrics.neo4j.logrotation.enabled is a boolean

83

Default value false

Table 120. metrics.neo4j.network.enabled

Description Enable reporting metrics about the network usage.

Valid values metrics.neo4j.network.enabled is a boolean

Default value false

Table 121. metrics.neo4j.pagecache.enabled

Description Enable reporting metrics about the Neo4j page cache; page faults, evictions, flushes,exceptions, etc.

Valid values metrics.neo4j.pagecache.enabled is a boolean

Default value false

Table 122. metrics.neo4j.server.enabled

Description Enable reporting metrics about Server threading info.

Valid values metrics.neo4j.server.enabled is a boolean

Default value false

Table 123. metrics.neo4j.tx.enabled

Description Enable reporting metrics about transactions; number of transactions started, committed, etc.

Valid values metrics.neo4j.tx.enabled is a boolean

Default value false

Table 124. metrics.prefix

Description A common prefix for the reported metrics field names. By default, this is either be 'neo4j', or acomputed value based on the cluster and instance names, when running in an HAconfiguration.

Valid values metrics.prefix is a string

Default value neo4j

Table 125. tools.consistency_checker.check_graph

Description Perform checks between nodes, relationships, properties, types and tokens.

Valid values tools.consistency_checker.check_graph is a boolean

Default value true

Table 126. tools.consistency_checker.check_indexes

Description Perform checks on indexes. Checking indexes is more expensive than checking the nativestores, so it may be useful to turn off this check for very large databases.

Valid values tools.consistency_checker.check_indexes is a boolean

Default value true

Table 127. tools.consistency_checker.check_label_scan_store

Description Perform checks on the label scan store. Checking this store is more expensive than checkingthe native stores, so it may be useful to turn off this check for very large databases.

Valid values tools.consistency_checker.check_label_scan_store is a boolean

Default value true

84

Table 128. tools.consistency_checker.check_property_owners

Description Perform optional additional checking on property ownership. This can detect a theoreticalinconsistency where a property could be owned by multiple entities. However, the check is veryexpensive in time and memory, so it is skipped by default.

Valid values tools.consistency_checker.check_property_owners is a boolean

Default value false

8.1.1. Configuring Bolt Connectors

Bolt Connectors are ports that accept connections via the Bolt Database Protocol, which is theprotocol used by official Neo4j Driver Libraries. Neo4j can be configured with one or more Boltconnectors. This allows separate connectors to be configured for remote and local connections, withdifferent encryption requirements.

Each connector has a unique key to identify it, denoted (bolt-connector-key) in the listing below.

Table 129. Configuration options for Bolt connectors. "(bolt-connector-key)" is a placeholder for a uniquename for the connector, for instance "bolt-public" or some other name that describes what the connector isfor.

Name Description

dbms.connector.(bolt-connector-key).address Address the connector should bind to.

dbms.connector.(bolt-connector-key).enabled Enable this connector.

dbms.connector.(bolt-connector-key).tls_level Encryption level to require this connector to use.

dbms.connector.(bolt-connector-key).type Connector type.

Table 130. dbms.connector.(bolt-connector-key).address

Description Address the connector should bind to.

Valid values address is a hostname and port

Default value localhost:7687

Table 131. dbms.connector.(bolt-connector-key).enabled

Description Enable this connector.

Valid values enabled is a boolean

Default value false

Table 132. dbms.connector.(bolt-connector-key).tls_level

Description Encryption level to require this connector to use.

Valid values tls_level is one of REQUIRED, OPTIONAL, DISABLED

Default value OPTIONAL

Table 133. dbms.connector.(bolt-connector-key).type

Description Connector type. You should always set this to the connector type you want.

Valid values type is one of BOLT, HTTP

Default value BOLT

85

8.1.2. Configuring HTTP Connectors

HTTP Connectors expose Neo4j’s HTTP endpoints. HTTPS connectors are configured by setting aconnector to require encryption. There must be exactly one HTTP connector and zero or one HTTPSconnectors configured.

Each connector has a unique key to identify it, denoted (http-connector-key) in the listing below.

Table 134. Configuration options for HTTP connectors. "(http-connector-key)" is a placeholder for a uniquename for the connector, for instance "http-public" or some other name that describes what the connector isfor.

Name Description

dbms.connector.(http-connector-key).address Address the connector should bind to.

dbms.connector.(http-connector-key).enabled Enable this connector.

dbms.connector.(http-connector-key).encryption Enable TLS for this connector.

dbms.connector.(http-connector-key).type Connector type.

Table 135. dbms.connector.(http-connector-key).address

Description Address the connector should bind to.

Valid values address is a hostname and port

Default value localhost:7474

Table 136. dbms.connector.(http-connector-key).enabled

Description Enable this connector.

Valid values enabled is a boolean

Default value false

Table 137. dbms.connector.(http-connector-key).encryption

Description Enable TLS for this connector.

Valid values encryption is one of NONE, TLS

Default value NONE

Table 138. dbms.connector.(http-connector-key).type

Description Connector type. You should always set this to the connector type you want.

Valid values type is one of BOLT, HTTP

Default value HTTP

8.2. JMX Beans

Table 139. MBeans exposed by Neo4j

Name Description

Branched Store Information about the branched stores present in this HAcluster member.

Configuration The configuration parameters used to configure Neo4j.

Diagnostics Diagnostics provided by Neo4j.

High Availability Information about an instance participating in a HAcluster.

86

Name Description

Index sampler Handle index sampling.

Kernel Information about the Neo4j kernel.

Locking Information about the Neo4j lock status.

Memory Mapping The status of Neo4j memory mapping.

Page cache Information about the Neo4j page cache. All numbers arecounts and sums since the Neo4j instance was started.

Primitive count Estimates of the numbers of different kinds of Neo4jprimitives.

Store file sizes Information about the sizes of the different parts of theNeo4j graph store.

Transactions Information about the Neo4j transaction manager.

For additional information on the primitive datatypes (int, long etc.) used in the JMXattributes, please see [property-value-types] in [graphdb-neo4j-properties].

Table 140. MBean Branched Store (org.neo4j.management.BranchedStore) Attributes

Name Description Type Read Write

Information about the branched stores present in this HA cluster member

BranchedStores A list of the branched stores org.neo4j.management.BranchedStoreInfo(javadocs/org/neo4j/management/BranchedStoreInfo.html)[] as CompositeData(http://docs.oracle.com/javase/7/docs/api/javax/management/openmbean/CompositeData.html)[]

yes no

Table 141. MBean Configuration (org.neo4j.jmx.impl.ConfigurationBean) Attributes


The configuration parameters used to configure Neo4j

cypher.default_language_version

Set this to specify the default parser (languageversion).

String yes no

87

javadocs/org/neo4j/management/BranchedStoreInfo.html

javadocs/org/neo4j/management/BranchedStoreInfo.html

http://docs.oracle.com/javase/7/docs/api/javax/management/openmbean/CompositeData.html


cypher.forbid_exhaustive_shortestpath

This setting is associated with performanceoptimization. Set this to true in situationswhere it is preferable to have any queriesusing the 'shortestPath' function terminate assoon as possible with no answer, rather thanpotentially running for a long time attemptingto find an answer (even if there is no path tobe found). For most queries, the 'shortestPath'algorithm will return the correct answer veryquickly. However there are some cases whereit is possible that the fast bidirectionalbreadth-first search algorithm will find noresults even if they exist. This can happenwhen the predicates in the WHERE clauseapplied to 'shortestPath' cannot be applied toeach step of the traversal, and can only beapplied to the entire path. When the queryplanner detects these special cases, it will planto perform an exhaustive depth-first search ifthe fast algorithm finds no paths. However,the exhaustive search may be orders ofmagnitude slower than the fast algorithm. If itis critical that queries terminate as soon aspossible, it is recommended that this optionbe set to true, which means that Neo4j willnever consider using the exhaustive search forshortestPath queries. However, please notethat if no paths are found, an error will bethrown at run time, which will need to behandled by the application.

String yes no

cypher.hints_error Set this to specify the behavior when Cypherplanner or runtime hints cannot be fulfilled. Iftrue, then non-conformance will result in anerror, otherwise only a warning is generated.

String yes no

cypher.planner Set this to specify the default planner for thedefault language version.

String yes no

dbms.allow_format_migration

Whether to allow a store upgrade in case thecurrent version of the database starts againstan older store version. Setting this to truedoes not guarantee successful upgrade, it justallows an upgrade to be performed.

String yes no

dbms.auto_index.nodes.enabled

Controls the auto indexing feature for nodes.Setting it to false shuts it down, while trueenables it by default for properties listed in thedbms.auto_index.nodes.keys setting.

String yes no

dbms.auto_index.nodes.keys

A list of property names (comma separated)that will be indexed by default. This applies tonodes only.

String yes no

dbms.auto_index.relationships.enabled

Controls the auto indexing feature forrelationships. Setting it to false shuts it down,while true enables it by default for propertieslisted in thedbms.auto_index.relationships.keys setting.

String yes no

dbms.auto_index.relationships.keys

A list of property names (comma separated)that will be indexed by default. This applies torelationships only.

String yes no

dbms.backup.address Listening server for online backups String yes no

dbms.backup.enabled Enable support for running online backups String yes no

88


dbms.checkpoint.interval.time

Configures the time interval between check-points. The database will not check-point moreoften than this (unless check pointing istriggered by a different event), but mightcheck-point less often than this interval, ifperforming a check-point takes longer timethan the configured interval. A check-point is apoint in the transaction logs, from whichrecovery would start from. Longer check-pointintervals typically means that recovery will takelonger to complete in case of a crash. On theother hand, a longer check-point interval canalso reduce the I/O load that the databaseplaces on the system, as each check-pointimplies a flushing and forcing of all the storefiles. The default is '5m' for a check-point every5 minutes. Other supported units are 's' forseconds, and 'ms' for milliseconds.

String yes no

dbms.checkpoint.interval.tx

Configures the transaction interval betweencheck-points. The database will not check-point more often than this (unless checkpointing is triggered by a different event), butmight check-point less often than this interval,if performing a check-point takes longer timethan the configured interval. A check-point is apoint in the transaction logs, from whichrecovery would start from. Longer check-pointintervals typically means that recovery will takelonger to complete in case of a crash. On theother hand, a longer check-point interval canalso reduce the I/O load that the databaseplaces on the system, as each check-pointimplies a flushing and forcing of all the storefiles. The default is '100000' for a check-pointevery 100000 transactions.

String yes no

dbms.checkpoint.iops.limit

Limit the number of IOs the backgroundcheckpoint process will consume per second.This setting is advisory, is ignored in Neo4jCommunity Edition, and is followed to besteffort in Enterprise Edition. An IO is in this casea 8 KiB (mostly sequential) write. Limiting thewrite IO in this way will leave more bandwidthin the IO subsystem to service random-readIOs, which is important for the response timeof queries when the database cannot fitentirely in memory. The only drawback of thissetting is that longer checkpoint times maylead to slightly longer recovery times in case ofa database or system crash. A lower numbermeans lower IO pressure, and consequentlylonger checkpoint times. The configuration canalso be commented out to remove thelimitation entirely, and let the checkpointerflush data as fast as the hardware will go. Setthis to -1 to disable the IOPS limit.

String yes no

dbms.directories.logs Path of the logs directory String yes no

dbms.directories.plugins

Location of the database plugin directory.Compiled Java JAR files that contain databaseprocedures will be loaded if they are placed inthis directory.

String yes no

dbms.index_sampling.background_enabled

Enable or disable background index sampling String yes no

dbms.index_sampling.buffer_size

Size of buffer used by index sampling String yes no

dbms.index_sampling.update_percentage

Percentage of index updates of total index sizerequired before sampling of a given index istriggered

String yes no

dbms.logs.debug.level Debug log level threshold. String yes no

89


dbms.logs.debug.rotation.delay

Minimum time interval after last rotation ofthe debug log before it may be rotated again.

String yes no

dbms.logs.debug.rotation.keep_number

Maximum number of history files for thedebug log.

String yes no

dbms.logs.debug.rotation.size

Threshold for rotation of the debug log. String yes no

dbms.logs.query.enabled Log executed queries that takes longer thanthe configured threshold. NOTE: This feature isonly available in the Neo4j Enterprise Edition.

String yes no

dbms.logs.query.parameter_logging_enabled

Log parameters for executed queries that tooklonger than the configured threshold.

String yes no

dbms.logs.query.rotation.keep_number

Maximum number of history files for thequery log.

String yes no

dbms.logs.query.rotation.size

The file size in bytes at which the query log willauto-rotate. If set to zero then no rotation willoccur. Accepts a binary suffix k, m or g.

String yes no

dbms.logs.query.threshold

If the execution of query takes more time thanthis threshold, the query is logged - providedquery logging is enabled. Defaults to 0seconds, that is all queries are logged.

String yes no

dbms.memory.pagecache.size

The amount of memory to use for mappingthe store files, in bytes (or kilobytes with the 'k'suffix, megabytes with 'm' and gigabytes with'g'). If Neo4j is running on a dedicated server,then it is generally recommended to leaveabout 2-4 gigabytes for the operating system,give the JVM enough heap to hold all yourtransaction state and query context, and thenleave the rest for the page cache. The defaultpage cache memory assumes the machine isdedicated to running Neo4j, and isheuristically set to 50% of RAM minus the maxJava heap size.

String yes no

dbms.memory.pagecache.swapper

Specify which page swapper to use for doingpaged IO. This is only used when integratingwith proprietary storage technology.

String yes no

dbms.read_only Only allow read operations from this Neo4jinstance. This mode still requires write accessto the directory for lock purposes.

String yes no

dbms.record_format Database record format. Enterprise editiononly. Valid values: standard, high_limit.Default value: standard.

String yes no

dbms.relationship_grouping_threshold

Relationship count threshold for considering anode to be dense

String yes no

dbms.security.auth_enabled

Enable auth requirement to access Neo4j. String yes no

dbms.security.ha_status_auth_enabled

Require authorization for access to the HAstatus endpoints.

String yes no

dbms.shell.enabled Enable a remote shell server which Neo4j Shellclients can log in to.

String yes no

dbms.shell.host Remote host for shell. By default, the shellserver listens only on the loopback interface,but you can specify the IP address of anynetwork interface or use 0.0.0.0 for allinterfaces.

String yes no

dbms.shell.port The port the shell will listen on. String yes no

dbms.shell.read_only Read only mode. Will only allow readoperations.

String yes no

90


dbms.shell.rmi_name The name of the shell. String yes no

dbms.tx_log.rotation.retention_policy

Make Neo4j keep the logical transaction logsfor being able to backup the database. Can beused for specifying the threshold to prunelogical logs after. For example "10 days" willprune logical logs that only containstransactions older than 10 days from thecurrent time, or "100k txs" will keep the 100klatest transactions and prune any oldertransactions.

String yes no

dbms.tx_log.rotation.size

Specifies at which file size the logical log willauto-rotate. 0 means that no rotation willautomatically occur based on file size.

String yes no

ha.allow_init_cluster Whether to allow this instance to create acluster if unable to join.

String yes no

ha.branched_data_policy Policy for how to handle branched data. String yes no

ha.broadcast_timeout Timeout for broadcasting values in cluster.Must consider end-to-end duration of Paxosalgorithm. This value is the default value forthe ha.join_timeout and ha.leave_timeoutsettings.

String yes no

ha.configuration_timeout

Timeout for waiting for configuration from anexisting cluster member during cluster join.

String yes no

ha.data_chunk_size Max size of the data chunks that flowsbetween master and slaves in HA. Bigger sizemay increase throughput, but may also bemore sensitive to variations in bandwidth,whereas lower size increases tolerance forbandwidth variations.

String yes no

ha.default_timeout Default timeout used for clustering timeouts.Override specific timeout settings with propervalues if necessary. This value is the defaultvalue for the ha.heartbeat_interval,ha.paxos_timeout and ha.learn_timeoutsettings.

String yes no

ha.election_timeout Timeout for waiting for other members tofinish a role election. Defaults toha.paxos_timeout.

String yes no

ha.heartbeat_interval How often heartbeat messages should besent. Defaults to ha.default_timeout.

String yes no

ha.heartbeat_timeout Timeout for heartbeats between clustermembers. Should be at least twice that ofha.heartbeat_interval.

String yes no

ha.host.coordination Host and port to bind the cluster managementcommunication.

String yes no

ha.host.data Hostname and port to bind the HA server. String yes no

ha.initial_hosts A comma-separated list of other members ofthe cluster to join.

String yes no

ha.internal_role_switch_timeout

Timeout for waiting for internal conditionsduring state switch, like for transactions tocomplete, before switching to master or slave.

String yes no

ha.join_timeout Timeout for joining a cluster. Defaults toha.broadcast_timeout.

String yes no

ha.learn_timeout Timeout for learning values. Defaults toha.default_timeout.

String yes no

ha.leave_timeout Timeout for waiting for cluster leave to finish.Defaults to ha.broadcast_timeout.

String yes no

91


ha.max_channels_per_slave

Maximum number of connections a slave canhave to the master.

String yes no

ha.paxos_timeout Default timeout for all Paxos timeouts.Defaults to ha.default_timeout. This value isthe default value for the ha.phase1_timeout,ha.phase2_timeout and ha.election_timeoutsettings.

String yes no

ha.phase1_timeout Timeout for Paxos phase 1. Defaults toha.paxos_timeout.

String yes no

ha.phase2_timeout Timeout for Paxos phase 2. Defaults toha.paxos_timeout.

String yes no

ha.pull_batch_size Size of batches of transactions applied onslaves when pulling from master

String yes no

ha.pull_interval Interval of pulling updates from master. String yes no

ha.role_switch_timeout Timeout for request threads waiting forinstance to become master or slave.

String yes no

ha.server_id Id for a cluster instance. Must be uniquewithin the cluster.

String yes no

ha.slave_lock_timeout Timeout for taking remote (write) locks onslaves. Defaults to ha.slave_read_timeout.

String yes no

ha.slave_only Whether this instance should only participateas slave in cluster. If set to true, it will never beelected as master.

String yes no

ha.slave_read_timeout How long a slave will wait for response frommaster before giving up.

String yes no

ha.tx_push_factor The amount of slaves the master will ask toreplicate a committed transaction.

String yes no

ha.tx_push_strategy Push strategy of a transaction to a slave duringcommit.

String yes no

jmx.port Configuration attribute String yes no

unsupported.cypher.compiler_tracing

Enable tracing of compilation in cypher. String yes no

unsupported.cypher.runtime

Set this to specify the default runtime for thedefault language version.

String yes no

unsupported.dbms.block_size.array_properties

Specifies the block size for storing arrays. Thisparameter is only honored when the store iscreated, otherwise it is ignored. Also note thateach block carries a ~10B of overhead sorecord size on disk will be slightly larger thanthe configured block size

String yes no

unsupported.dbms.block_size.labels

Specifies the block size for storing labelsexceeding in-lined space in node record. Thisparameter is only honored when the store iscreated, otherwise it is ignored. Also note thateach block carries a ~10B of overhead sorecord size on disk will be slightly larger thanthe configured block size

String yes no

unsupported.dbms.block_size.strings

Specifies the block size for storing strings. Thisparameter is only honored when the store iscreated, otherwise it is ignored. Note that eachcharacter in a string occupies two bytes,meaning that e.g a block size of 120 will hold a60 character long string before overflowinginto a second block. Also note that each blockcarries a ~10B of overhead so record size ondisk will be slightly larger than the configuredblock size

String yes no

92


unsupported.dbms.counts_store_rotation_timeout

Maximum time to wait for active transactioncompletion when rotating counts store

String yes no

unsupported.dbms.directories.neo4j_home

Root relative to which directory settings areresolved. This is set in code and should neverbe configured explicitly.

String yes no

unsupported.dbms.disconnected

Disable all protocol connectors. String yes no

unsupported.dbms.edition

Configuration attribute String yes no

unsupported.dbms.ephemeral

Configuration attribute String yes no

unsupported.dbms.gc_monitor_threshold

The amount of time in ms the monitor threadhas to be blocked before logging a message itwas blocked.

String yes no

unsupported.dbms.gc_monitor_wait_time

Amount of time in ms the GC monitor threadwill wait before taking another measurement.

String yes no

unsupported.dbms.id_generator_fast_rebuild_enabled

Use a quick approach for rebuilding the IDgenerators. This give quicker recovery time,but will limit the ability to reuse the space ofdeleted entities.

String yes no

unsupported.dbms.kernel_id

An identifier that uniquely identifies this graphdatabase instance within this JVM. Defaults toan auto-generated number depending on howmany instance are started in this JVM.

String yes no

unsupported.dbms.logs.debug.debug_loggers

Debug log contexts that should output debuglevel logging

String yes no

unsupported.dbms.memory.pagecache.pagesize

Target size for pages of mapped memory. Ifset to 0, then a reasonable default is chosen,depending on the storage device used.

String yes no

unsupported.dbms.report_configuration

Print out the effective Neo4j configurationafter startup.

String yes no

unsupported.dbms.shutdown_transaction_end_timeout

The maximum amount of time to wait forrunning transactions to complete beforeallowing initiated database shutdown tocontinue

String yes no

unsupported.dbms.transaction_start_timeout

The maximum amount of time to wait for thedatabase to become available, when starting anew transaction.

String yes no

unsupported.ha.cluster_name

The name of a cluster. String yes no

unsupported.tools.batch_inserter.batch_size

Specifies number of operations that batchinserter will try to group into one batch beforeflushing data into underlying storage.

String yes no

Table 142. MBean Diagnostics (org.neo4j.management.Diagnostics) Attributes


Diagnostics provided by Neo4j

DiagnosticsProviders A list of the ids for the registered diagnosticsproviders.

List (java.util.List) yes no

Table 143. MBean Diagnostics (org.neo4j.management.Diagnostics) Operations

Name Description ReturnType Signature

dumpAll Dump diagnostics information to JMX String (no parameters)

dumpToLog Dump diagnostics information to the log. void (no parameters)

93


dumpToLog Dump diagnostics information to the log. void java.lang.String

extract Operation exposed for management String java.lang.String

Table 144. MBean High Availability (org.neo4j.management.HighAvailability) Attributes


Information about an instance participating in a HA cluster

Alive Whether this instance is alive or not boolean yes no

Available Whether this instance is available or not boolean yes no

InstanceId The identifier used to identify this server in theHA cluster

String yes no

InstancesInCluster Information about all instances in this cluster org.neo4j.management.ClusterMemberInfo(javadocs/org/neo4j/management/ClusterMemberInfo.html)[] as CompositeData(http://docs.oracle.com/javase/7/docs/api/javax/management/openmbean/CompositeData.html)[]

yes no

LastCommittedTxId The latest transaction id present in thisinstance’s store

long yes no

LastUpdateTime The time when the data on this instance waslast updated from the master

String yes no

Role The role this instance has in the cluster String yes no

Table 145. MBean High Availability (org.neo4j.management.HighAvailability) Operations


update (If this is a slave) Update the database onthis instance with the latest transactionsfrom the master

String (no parameters)

Table 146. MBean Kernel (org.neo4j.jmx.Kernel) Attributes


Information about the Neo4j kernel

DatabaseName The name of the mounted database String yes no

KernelStartTime The time from which this Neo4j instance wasin operational mode.

Date (java.util.Date) yes no

KernelVersion The version of Neo4j String yes no

MBeanQuery An ObjectName that can be used as a queryfor getting all management beans for thisNeo4j instance.

javax.management.ObjectName

yes no

ReadOnly Whether this is a read only instance boolean yes no

StoreCreationDate The time when this Neo4j graph store wascreated.

Date (java.util.Date) yes no

StoreId An identifier that, together with store creationtime, uniquely identifies this Neo4j graphstore.

String yes no

StoreLogVersion The current version of the Neo4j store logicallog.

long yes no

Table 147. MBean Locking (org.neo4j.management.LockManager) Attributes

94

javadocs/org/neo4j/management/ClusterMemberInfo.html

javadocs/org/neo4j/management/ClusterMemberInfo.html



Information about the Neo4j lock status

Locks Information about all locks held by Neo4j java.util.List<org.neo4j.kernel.info.LockInfo>as CompositeData(http://docs.oracle.com/javase/7/docs/api/javax/management/openmbean/CompositeData.html)[]

yes no

NumberOfAvertedDeadlocks

The number of lock sequences that wouldhave lead to a deadlock situation that Neo4jhas detected and averted (by throwingDeadlockDetectedException).

long yes no

Table 148. MBean Locking (org.neo4j.management.LockManager) Operations


getContendedLocks getContendedLocks java.util.List<org.neo4j.kernel.info.LockInfo> as CompositeData(http://docs.oracle.com/javase/7/docs/api/javax/management/openmbean/CompositeData.html)[]

long

Table 149. MBean Memory Mapping (org.neo4j.management.MemoryMapping) Attributes


The status of Neo4j memory mapping

MemoryPools Get information about each pool of memorymapped regions from store files with memorymapping enabled

org.neo4j.management.WindowPoolInfo(javadocs/org/neo4j/management/WindowPoolInfo.html)[] as CompositeData(http://docs.oracle.com/javase/7/docs/api/javax/management/openmbean/CompositeData.html)[]

yes no

Table 150. MBean Page cache (org.neo4j.management.PageCache) Attributes


Information about the Neo4j page cache. All numbers are counts and sums since the Neo4j instance was started

BytesRead Number of bytes read from durable storage. long yes no

BytesWritten Number of bytes written to durable storage. long yes no

EvictionExceptions Number of exceptions caught during pageeviction. This number should be zero, or atleast not growing, in a healthy database.Otherwise it could indicate drive failure,storage space, or permission problems.

long yes no

Evictions Number of page evictions. How many pageshave been removed from memory to makeroom for other pages.

long yes no

Faults Number of page faults. How often requesteddata was not found in memory and had to beloaded.

long yes no

FileMappings Number of files that have been mapped intothe page cache.

long yes no

FileUnmappings Number of files that have been unmappedfrom the page cache.

long yes no

Flushes Number of page flushes. How many dirtypages have been written to durable storage.

long yes no

95



javadocs/org/neo4j/management/WindowPoolInfo.html

javadocs/org/neo4j/management/WindowPoolInfo.html



Pins Number of page pins. How many pages havebeen accessed (monitoring must be enabledseparately).

long yes no

The page pin count metric is disabled by default for performance reasons, in whichcase the pin count value will always be zero. The page pin count metric can beenabled by adding this line to the neo4j-wrapper.conf file: dbms.jvm.additional=-Dorg.neo4j.io.pagecache.tracing.tracePinUnpin=true

Table 151. MBean Primitive count (org.neo4j.jmx.Primitives) Attributes


Estimates of the numbers of different kinds of Neo4j primitives

NumberOfNodeIdsInUse An estimation of the number of nodes used inthis Neo4j instance

long yes no

NumberOfPropertyIdsInUse

An estimation of the number of propertiesused in this Neo4j instance

long yes no

NumberOfRelationshipIdsInUse

An estimation of the number of relationshipsused in this Neo4j instance

long yes no

NumberOfRelationshipTypeIdsInUse

The number of relationship types used in thisNeo4j instance

long yes no

Table 152. MBean Store file sizes (org.neo4j.jmx.StoreFile) Attributes


Information about the sizes of the different parts of the Neo4j graph store

ArrayStoreSize The amount of disk space used to store arrayproperties, in bytes.

long yes no

LogicalLogSize The amount of disk space used by the currentNeo4j logical log, in bytes.

long yes no

NodeStoreSize The amount of disk space used to store nodes,in bytes.

long yes no

PropertyStoreSize The amount of disk space used to storeproperties (excluding string values and arrayvalues), in bytes.

long yes no

RelationshipStoreSize The amount of disk space used to storerelationships, in bytes.

long yes no

StringStoreSize The amount of disk space used to store stringproperties, in bytes.

long yes no

TotalStoreSize The total disk space used by this Neo4jinstance, in bytes.

long yes no

Table 153. MBean Transactions (org.neo4j.management.TransactionManager) Attributes


Information about the Neo4j transaction manager

LastCommittedTxId The id of the latest committed transaction long yes no

NumberOfCommittedTransactions

The total number of committed transactions long yes no

NumberOfOpenedTransactions

The total number started transactions long yes no

NumberOfOpenTransactions

The number of currently open transactions long yes no

96


NumberOfRolledBackTransactions

The total number of rolled back transactions long yes no

PeakNumberOfConcurrentTransactions

The highest number of transactions everopened concurrently

long yes no

Table 154. MBean Index sampler (org.neo4j.management.IndexSamplingManager) Operations


triggerIndexSampling triggerIndexSampling void java.lang.String,java.lang.String,boolean

8.3. Available metrics

Table 155. Database CheckPointing Metrics

Name Description

neo4j.check_point.events

The total number of check point events executed so far

neo4j.check_point.total_time

The total time spent in check pointing so far

neo4j.check_point.check_point_duration

The duration of the check point event

Table 156. Database Data Metrics

Name Description

neo4j.ids_in_use.relationship_type

The total number of different relationship types stored in the database

neo4j.ids_in_use.property

The total number of different property names used in the database

neo4j.ids_in_use.relationship

The total number of relationships stored in the database

neo4j.ids_in_use.node The total number of nodes stored in the database

Table 157. Database PageCache Metrics

Name Description

neo4j.page_cache.eviction_exceptions

The total number of exceptions seen during the eviction process in the page cache

neo4j.page_cache.flushes

The total number of flushes executed by the page cache

neo4j.page_cache.unpins

The total number of page unpins executed by the page cache

neo4j.page_cache.pins The total number of page pins executed by the page cache

neo4j.page_cache.evictions

The total number of page evictions executed by the page cache

neo4j.page_cache.page_faults

The total number of page faults happened in the page cache

Table 158. Database Transaction Metrics

Name Description

neo4j.transaction.started

The total number of started transactions

97

Name Description

neo4j.transaction.peak_concurrent

The highest peak of concurrent transactions ever seen on this machine

neo4j.transaction.active

The number of currently active transactions

neo4j.transaction.active_read

The number of currently active read transactions

neo4j.transaction.active_write

The number of currently active write transactions

neo4j.transaction.committed

The total number of committed transactions

neo4j.transaction.committed_read

The total number of committed read transactions

neo4j.transaction.committed_write

The total number of committed write transactions

neo4j.transaction.rollbacks

The total number of rolled back transactions

neo4j.transaction.rollbacks_read

The total number of rolled back read transactions

neo4j.transaction.rollbacks_write

The total number of rolled back write transactions

neo4j.transaction.terminated

The total number of terminated transactions

neo4j.transaction.terminated_read

The total number of terminated read transactions

neo4j.transaction.terminated_write

The total number of terminated write transactions

neo4j.transaction.last_committed_tx_id

The ID of the last committed transaction

neo4j.transaction.last_closed_tx_id

The ID of the last closed transaction

Table 159. Cypher Metrics

Name Description

neo4j.cypher.replan_events

The total number of times Cypher has decided to re-plan a query

Table 160. Database LogRotation Metrics

Name Description

neo4j.log_rotation.events

The total number of transaction log rotations executed so far

neo4j.log_rotation.total_time

The total time spent in rotating transaction logs so far

neo4j.log_rotation.log_rotation_duration

The duration of the log rotation event

Table 161. Network Metrics

Name Description

neo4j.network.slave_network_tx_writes

The amount of bytes transmitted on the network containing the transaction data from a slaveto the master in order to be committed

neo4j.network.master_network_store_writes

The amount of bytes transmitted on the network while copying stores from a machines toanother

98

Name Description

neo4j.network.master_network_tx_writes

The amount of bytes transmitted on the network containing the transaction data from amaster to the slaves in order to propagate committed transactions

Table 162. Cluster Metrics

Name Description

neo4j.cluster.slave_pull_updates

The total number of update pulls executed by this instance

neo4j.cluster.slave_pull_update_up_to_tx

The highest transaction id that has been pulled in the last pull updates by this instance

neo4j.cluster.is_master

Whether or not this instance is the master in the cluster

neo4j.cluster.is_available

Whether or not this instance is available in the cluster

8.3.1. Java Virtual Machine Metrics

These metrics are environment dependent and they may vary on different hardware and with JVMconfigurations. Typically these metrics will show information about garbage collections (for examplethe number of events and time spent collecting), memory pools and buffers, and finally the number ofactive threads running.

99

Chapter 9. LicenseCreative Commons 3.0

You are free to

Share

copy and redistribute the material in any medium or format

Adapt

remix, transform, and build upon the material

for any purpose, even commercially.

The licensor cannot revoke these freedoms as long as you follow the license terms.

Under the following terms

Attribution

You must give appropriate credit, provide a link to the license, and indicate if changes were made.You may do so in any reasonable manner, but not in any way that suggests the licensor endorsesyou or your use.

ShareAlike

If you remix, transform, or build upon the material, you must distribute your contributions underthe same license as the original.

No additional restrictions

You may not apply legal terms or technological measures that legally restrict others from doinganything the license permits.

Notices

You do not have to comply with the license for elements of the material in the public domain or whereyour use is permitted by an applicable exception or limitation.

No warranties are given. The license may not give you all of the permissions necessary for yourintended use. For example, other rights such as publicity, privacy, or moral rights may limit how youuse the material.

See http://creativecommons.org/licenses/by-sa/3.0/ for further details. The full license text is availableat http://creativecommons.org/licenses/by-sa/3.0/legalcode.

100

http://creativecommons.org/licenses/by-sa/3.0/

http://creativecommons.org/licenses/by-sa/3.0/legalcode

The Neo4j Operations Manual v3 - we-yun.com 3.0 Docs/neo4j-operations... · Chapter 1. Introduction...

Documents

Transcript of The Neo4j Operations Manual v3 - we-yun.com 3.0 Docs/neo4j-operations... · Chapter 1. Introduction...