kernel.impl.coreapijava
這個包包含核心API的直接實現。核心API是org.neo4j中定義的API。graphdb及其子包。 這裏的類是實現細節,可能會在不通知的狀況下更改。
IOnode
這是Neo4j IO抽象層。 它當前包含磁盤IO機制和文件分頁。
因爲歷史緣由,這個模塊包含了Neo4j的多個重要組件: - 嵌入的Java API - org.neo4j.graphdb - 嵌入的Java API實現 - org.neo4j.kernel.coreapi - org.neo4j.kernel.core - 嵌入的遍歷Java API - org.neo4j.graphdb.traversal - 嵌入的遍歷API實現 - org.neo4j.kernel.traversal - 批量導入 - org.neo4j.unsafe.impl.batchimport - 批處理插入器(遺留) - org.neo4j.unsafe.batchinsert - 事務狀態構建層(「內核API」) - org.neo4j.kernel.api - org.neo4j.kernel.impl.api - 存儲引擎 - org.neo4j.kernel.impl.store, - org.neo4j.kernel.impl.recovery - org.neo4j.kernel.impl.transaction - 配置 - org.neo4j.kernel.configuration - 常見的工具 - org.neo4j.helpers - org.neo4j.kernel.impl.util - org.neo4j.kernel.lifecycle - org.neo4j.kernel.monitoring - 鎖 - org.neo4j.kernel.impl.locking - 內核擴展 - org.neo4j.kernel.extension
內核API管理與Neo4j內核的全部交互,主要由Cypher runtime 和 Core API。api
Usage緩存
這個接口的入口點是連接:src/main/java/org/neo4j/impl/kernel/api/Kernel.java[Kernel] 經過這裏,咱們能夠開始一個連接:src/main/java/org/neo4j/impl/kernel/api/Session.java[Session], 它容許咱們開始連接:src/main/java/org/neo4j/impl/kernel api/Transaction.java[Transactions]. 一旦進入事務,咱們能夠直接訪問內核的全部主要功能:服務器
使用遊標讀取app
讀操做使用特殊的遊標,這容許在熱路徑中很是低的對象分配率。ide
設計說明 一般,遊標的設計方法是經過遊標直接公開底層表示包含的全部信息。 其思想是將暴露給運行時的遊標實現爲直接位於頁面數據之上的「視圖」或「投影」,提供對存儲的原始字節的解釋。 不過,數據的公開並不會一直進行下去,而是試圖在容許底層格式的將來更改之間取得平衡。 做爲一個思想實驗,我嘗試以這樣一種方式定義API,應該能夠沿着這些方面改變實現:函數
在大多數狀況下,經過使用不一樣的遊標類型顯式地訪問不一樣的存儲。 除了一個例外:link:src/main/java/ org/neo4j/impl/kernel/api/labelset。java[節點的標籤]。工具
整體設計方法是讓API的客戶機控制遊標的生命週期。 這比將遊標集中在引擎內更有效。單元測試
這樣作的緣由是,因爲客戶機是一個查詢,因此使用模式是徹底已知的,所以能夠顯式地從池中取出和返回到池中。 一樣,能夠顯式地在同一個查詢中重用。可是不容許客戶機提供本身的遊標類型,必須使用連接提供的類型: link:src/main/java/org/neo4j/impl/kernel/api/CursorFactory.java[CursorFactory].
客戶機控制遊標的生命週期後,就排除了「Cursor」之類的通用接口,由於咱們須要遊標是已知的具體類型。
「規範」用法是從其餘遊標初始化遊標,而不是從記錄中訪問指針值,而後顯式初始化遊標。 指針值也能夠訪問,由於向量化的操做管道更喜歡這樣的使用模式。
對底層表示的一個更改將使實現這個API更加容易,即便全部節點按照類型對它們的關係進行分組,而不只僅是密集節點。
此外,若是事務狀態以與存儲中相同的格式存儲在堆外緩衝區中,則實現將更容易和更有效。
Neo4j UDC
Usage Data Collector (UDC)是一個內核擴展,它收集關於Neo4j服務器的統計信息。 詳見: https://neo4j.com/docs/operations-manual/current/configuration/usage-data-collector/
procedure-compiler
Neo4j工具-過程|用戶功能編譯器 這是一個註釋處理器,它將在編譯時驗證存儲過程。 雖然能夠執行大多數基本檢查,可是仍然須要一些單元測試來驗證一些運行時行爲。 它是作什麼的? 一旦將存儲過程編譯器添加到項目類路徑中(請參閱下面的Maven/Gradle說明),若是不知足如下任何要求,它將觸發編譯失敗: - `@Context` 字段必須是' public '且非' final ' - 全部其餘字段必須是 `static` - `Map` 記錄字段/過程參數必須將鍵類型定義爲 `String` - `@Procedure`|`@UserFunction` 類必須定義一個沒有參數的公共構造函數 - `@Procedure` 方法必須返回一個流Stream - `@Procedure`|`@UserFunction` 參數和記錄類型必須支持 - `@Procedure`|`@UserFunction` 參數必須用 `@Name`標註 - `@UserFunction` 不能在根名稱空間中定義 - 全部訪問過的 `@Procedure`|`@UserFunction`名稱必須是惟一的 *已部署的Neo4j實例能夠聚合來自不一樣jar的存儲過程。 註釋處理器沒法檢測到jar之間的命名衝突。 根據定義,它一次只能檢查一個編譯單元。
====== ## class StoreType
StoreType.java public enum StoreType { NODE_LABEL( DatabaseFile.NODE_LABEL_STORE, true, false ) { @Override public CommonAbstractStore open( NeoStores neoStores ) { return neoStores.createNodeLabelStore(); } }, NODE( DatabaseFile.NODE_STORE, true, false ) { @Override public CommonAbstractStore open( NeoStores neoStores ) { return neoStores.createNodeStore(); } }, PROPERTY_KEY_TOKEN_NAME( DatabaseFile.PROPERTY_KEY_TOKEN_NAMES_STORE, true, true ) { @Override public CommonAbstractStore open( NeoStores neoStores ) { return neoStores.createPropertyKeyTokenNamesStore(); } }, PROPERTY_KEY_TOKEN( DatabaseFile.PROPERTY_KEY_TOKEN_STORE, true, true ) { @Override public CommonAbstractStore open( NeoStores neoStores ) { return neoStores.createPropertyKeyTokenStore(); } }, PROPERTY_STRING( DatabaseFile.PROPERTY_STRING_STORE, true, false ) { @Override public CommonAbstractStore open( NeoStores neoStores ) { return neoStores.createPropertyStringStore(); } }, PROPERTY_ARRAY( DatabaseFile.PROPERTY_ARRAY_STORE, true, false ) { @Override public CommonAbstractStore open( NeoStores neoStores ) { return neoStores.createPropertyArrayStore(); } }, PROPERTY( DatabaseFile.PROPERTY_STORE, true, false ) { @Override public CommonAbstractStore open( NeoStores neoStores ) { return neoStores.createPropertyStore(); } }, RELATIONSHIP( DatabaseFile.RELATIONSHIP_STORE, true, false ) { @Override public CommonAbstractStore open( NeoStores neoStores ) { return neoStores.createRelationshipStore(); } }, RELATIONSHIP_TYPE_TOKEN_NAME( DatabaseFile.RELATIONSHIP_TYPE_TOKEN_NAMES_STORE, true, true ) { @Override public CommonAbstractStore open( NeoStores neoStores ) { return neoStores.createRelationshipTypeTokenNamesStore(); } }, RELATIONSHIP_TYPE_TOKEN( DatabaseFile.RELATIONSHIP_TYPE_TOKEN_STORE, true, true ) { @Override public CommonAbstractStore open( NeoStores neoStores ) { return neoStores.createRelationshipTypeTokenStore(); } }, LABEL_TOKEN_NAME( DatabaseFile.LABEL_TOKEN_NAMES_STORE, true, true ) { @Override public CommonAbstractStore open( NeoStores neoStores ) { return neoStores.createLabelTokenNamesStore(); } }, LABEL_TOKEN( DatabaseFile.LABEL_TOKEN_STORE, true, true ) { @Override public CommonAbstractStore open( NeoStores neoStores ) { return neoStores.createLabelTokenStore(); } }, SCHEMA( DatabaseFile.SCHEMA_STORE, true, true ) { @Override public CommonAbstractStore open( NeoStores neoStores ) { return neoStores.createSchemaStore(); } }, RELATIONSHIP_GROUP( DatabaseFile.RELATIONSHIP_GROUP_STORE, true, false ) { @Override public CommonAbstractStore open( NeoStores neoStores ) { return neoStores.createRelationshipGroupStore(); } }, COUNTS( DatabaseFile.COUNTS_STORES, false, false ) { @Override public CountsTracker open( NeoStores neoStores ) { return neoStores.createCountStore(); } @Override void close( Object object ) { try { ((CountsTracker) object).shutdown(); } catch ( IOException e ) { throw new UnderlyingStorageException( e ); } } }, META_DATA( DatabaseFile.METADATA_STORE, true, true ) // Make sure this META store is last { @Override public CommonAbstractStore open( NeoStores neoStores ) { return neoStores.createMetadataStore(); } };
ENUM_YPE | CALL_FUNC |
---|---|
NODE_LABEL | createNodeLabelStore |
NODE | createNodeStore |
PROPERTY_KEY_TOKEN_NAME | createPropertyKeyTokenNamesStore |
… | … |
該類包含對「NodeStore、RelationshipStore、PropertyStore和RelationshipTypeStore」的引用。 實際上,NeoStores並不「存儲」任何東西,而是爲在其中執行的「類型和版本」驗證擴展了AbstractStore。
CommonAbstractStore createNodeStore() { return initialize( new NodeStore( layout.nodeStore(), layout.idNodeStore(), config, idGeneratorFactory, pageCache, logProvider, (DynamicArrayStore) getOrCreateStore( StoreType.NODE_LABEL ), recordFormats, openOptions ) ); } CommonAbstractStore createNodeLabelStore() { return createDynamicArrayStore( layout.nodeLabelStore(), layout.idNodeLabelStore(), IdType.NODE_LABELS, GraphDatabaseSettings.label_block_size ); } CommonAbstractStore createPropertyKeyTokenStore() { return initialize( new PropertyKeyTokenStore( layout.propertyKeyTokenStore(), layout.idPropertyKeyTokenStore(), config, idGeneratorFactory, pageCache, logProvider, (DynamicStringStore) getOrCreateStore( StoreType.PROPERTY_KEY_TOKEN_NAME ), recordFormats, openOptions ) ); } CommonAbstractStore createPropertyKeyTokenNamesStore() { return createDynamicStringStore( layout.propertyKeyTokenNamesStore(), layout.idPropertyKeyTokenNamesStore(), IdType.PROPERTY_KEY_TOKEN_NAME, TokenStore.NAME_STORE_BLOCK_SIZE ); } CommonAbstractStore createPropertyStore() { return initialize( new PropertyStore( layout.propertyStore(), layout.idPropertyStore(), config, idGeneratorFactory, pageCache, logProvider, (DynamicStringStore) getOrCreateStore( StoreType.PROPERTY_STRING ), (PropertyKeyTokenStore) getOrCreateStore( StoreType.PROPERTY_KEY_TOKEN ), (DynamicArrayStore) getOrCreateStore( StoreType.PROPERTY_ARRAY ), recordFormats, openOptions ) ); } CommonAbstractStore createPropertyStringStore() { return createDynamicStringStore( layout.propertyStringStore(), layout.idPropertyStringStore(), IdType.STRING_BLOCK, GraphDatabaseSettings.string_block_size ); } CommonAbstractStore createPropertyArrayStore() { return createDynamicArrayStore( layout.propertyArrayStore(), layout.idPropertyArrayStore(), IdType.ARRAY_BLOCK, GraphDatabaseSettings.array_block_size ); } CommonAbstractStore createRelationshipStore() { return initialize( new RelationshipStore( layout.relationshipStore(), layout.idRelationshipStore(), config, idGeneratorFactory, pageCache, logProvider, recordFormats, openOptions ) ); } CommonAbstractStore createRelationshipTypeTokenStore() { return initialize( new RelationshipTypeTokenStore( layout.relationshipTypeTokenStore(), layout.idRelationshipTypeTokenStore(), config, idGeneratorFactory, pageCache, logProvider, (DynamicStringStore) getOrCreateStore( StoreType.RELATIONSHIP_TYPE_TOKEN_NAME ), recordFormats, openOptions ) ); } CommonAbstractStore createRelationshipTypeTokenNamesStore() { return createDynamicStringStore( layout.relationshipTypeTokenNamesStore(), layout.idRelationshipTypeTokenNamesStore(), IdType.RELATIONSHIP_TYPE_TOKEN_NAME, TokenStore.NAME_STORE_BLOCK_SIZE ); } CommonAbstractStore createLabelTokenStore() { return initialize( new LabelTokenStore( layout.labelTokenStore(), layout.idLabelTokenStore(), config, idGeneratorFactory, pageCache, logProvider, (DynamicStringStore) getOrCreateStore( StoreType.LABEL_TOKEN_NAME ), recordFormats, openOptions ) ); } CommonAbstractStore createSchemaStore() { return initialize( new SchemaStore( layout.schemaStore(), layout.idSchemaStore(), config, IdType.SCHEMA, idGeneratorFactory, pageCache, logProvider, recordFormats, openOptions ) ); } CommonAbstractStore createRelationshipGroupStore() { return initialize( new RelationshipGroupStore( layout.relationshipGroupStore(), layout.idRelationshipGroupStore(), config, idGeneratorFactory, pageCache, logProvider, recordFormats, openOptions ) ); } CommonAbstractStore createLabelTokenNamesStore() { return createDynamicStringStore( layout.labelTokenNamesStore(), layout.idLabelTokenNamesStore(), IdType.LABEL_TOKEN_NAME, TokenStore.NAME_STORE_BLOCK_SIZE ); } CountsTracker createCountStore() { boolean readOnly = config.get( GraphDatabaseSettings.read_only ); CountsTracker counts = readOnly ? createReadOnlyCountsTracker( layout ) : createWritableCountsTracker( layout ); NeoStores neoStores = this; counts.setInitializer( new DataInitializer<CountsAccessor.Updater>() { private final Log log = logProvider.getLog( MetaDataStore.class ); @Override public void initialize( CountsAccessor.Updater updater ) { log.warn( "Missing counts store, rebuilding it." ); new CountsComputer( neoStores, pageCache, layout ).initialize( updater ); log.warn( "Counts store rebuild completed." ); } @Override public long initialVersion() { return ((MetaDataStore) getOrCreateStore( StoreType.META_DATA )).getLastCommittedTransactionId(); } } ); try { counts.init(); // TODO: move this to LifeCycle } catch ( IOException e ) { throw new UnderlyingStorageException( "Failed to initialize counts store", e ); } return counts; } CommonAbstractStore createMetadataStore() { return initialize( new MetaDataStore( metadataStore, layout.idMetadataStore(), config, idGeneratorFactory, pageCache, logProvider, recordFormats.metaData(), recordFormats.storeVersion(), openOptions ) ); } private CommonAbstractStore createDynamicStringStore( File storeFile, File idFile, IdType idType, Setting<Integer> blockSizeProperty ) { return createDynamicStringStore( storeFile, idFile, idType, config.get( blockSizeProperty ) ); } private CommonAbstractStore createDynamicStringStore( File storeFile, File idFile, IdType idType, int blockSize ) { return initialize( new DynamicStringStore( storeFile, idFile, config, idType, idGeneratorFactory, pageCache, logProvider, blockSize, recordFormats.dynamic(), recordFormats.storeVersion(), openOptions ) ); } private CommonAbstractStore createDynamicArrayStore( File storeFile, File idFile, IdType idType, Setting<Integer> blockSizeProperty ) { return createDynamicArrayStore( storeFile, idFile, idType, config.get( blockSizeProperty ) ); } CommonAbstractStore createDynamicArrayStore( File storeFile, File idFile, IdType idType, int blockSize ) { if ( blockSize <= 0 ) { throw new IllegalArgumentException( "Block size of dynamic array store should be positive integer." ); } return initialize( new DynamicArrayStore( storeFile, idFile, config, idType, idGeneratorFactory, pageCache, logProvider, blockSize, recordFormats, openOptions ) ); }
createCountStore() | createCountStore() |
… | CommonAbstractStore |
顯然,僅有createCountStore() 返回 createCountStore().其他均爲CommonAbstractStore.
createNodeStore | initialize(new NodeStore()) |
createPropertyKeyTokenStore | initialize( new PropertyKeyTokenStore()) |
createPropertyStore | initialize( new PropertyStore()) |
createRelationshipStore | initialize(new RelationshipStore()) |
createRelationshipTypeTokenStore | initialize(new RelationshipTypeTokenStore()) |
createLabelTokenStore | initialize(new LabelTokenStore()) |
createSchemaStore | initialize(new SchemaStore()) |
createRelationshipGroupStore | initialize( new RelationshipGroupStore()) |
createMetadataStore | initialize(new MetaDataStore()) |
createDynamicStringStore | initialize( new DynamicStringStore()) |
createDynamicArrayStore | initialize( new DynamicArrayStore()) private 檢查blockSize |
createNodeLabelStore | createDynamicArrayStore() |
createPropertyArrayStore | createDynamicArrayStore() |
createDynamicArrayStore | createDynamicArrayStore() |
createPropertyKeyTokenNamesStore | createDynamicStringStore() |
createPropertyStringStore | createDynamicStringStore() |
createRelationshipTypeTokenNamesStore | createDynamicStringStore() |
createLabelTokenNamesStore | createDynamicStringStore() |
createDynamicStringStore | createDynamicStringStore() |
createCountStore | special |
NeoStores.java private CommonAbstractStore createDynamicStringStore( File storeFile, File idFile, IdType idType, Setting<Integer> blockSizeProperty ) { return createDynamicStringStore( storeFile, idFile, idType, config.get( blockSizeProperty ) ); } private CommonAbstractStore createDynamicStringStore( File storeFile, File idFile, IdType idType, int blockSize ) { return initialize( new DynamicStringStore( storeFile, idFile, config, idType, idGeneratorFactory, pageCache, logProvider, blockSize, recordFormats.dynamic(), recordFormats.storeVersion(), openOptions ) ); } private CommonAbstractStore createDynamicArrayStore( File storeFile, File idFile, IdType idType, Setting<Integer> blockSizeProperty ) { return createDynamicArrayStore( storeFile, idFile, idType, config.get( blockSizeProperty ) ); } CommonAbstractStore createDynamicArrayStore( File storeFile, File idFile, IdType idType, int blockSize ) { if ( blockSize <= 0 ) { throw new IllegalArgumentException( "Block size of dynamic array store should be positive integer." ); } return initialize( new DynamicArrayStore( storeFile, idFile, config, idType, idGeneratorFactory, pageCache, logProvider, blockSize, recordFormats, openOptions ) ); }
可見,這裏都是直接或間接調用 initialize(new <type>(args…)) 實現的.
NeoStores.java private <T extends CommonAbstractStore> T initialize( T store ) { store.initialise( createIfNotExist ); return store; }
CommonAbstractStore.java void initialise( boolean createIfNotExists ) { try { checkAndLoadStorage( createIfNotExists ); } catch ( Exception e ) { closeAndThrow( e ); } }
NeoStores類包含對「NodeStore、RelationshipStore、PropertyStore和RelationshipTypeStore」的引用。 實際上,NeoStores並不「存儲」任何東西,而是爲在其中執行的「類型和版本」驗證擴展了AbstractStore。
CommonAbstractStore.java /** * This method is called by constructors. Checks the header record and loads the store. * <p> * Note: This method will map the file with the page cache. The store file must not * be accessed directly until it has been unmapped - the store file must only be * accessed through the page cache. * @param createIfNotExists If true, creates and initialises the store file if it does not exist already. If false, * this method will instead throw an exception in that situation. */ protected void checkAndLoadStorage( boolean createIfNotExists ) { int pageSize = pageCache.pageSize(); int filePageSize; try ( PagedFile pagedFile = pageCache.map( storageFile, pageSize, ANY_PAGE_SIZE ) ) { extractHeaderRecord( pagedFile ); filePageSize = pageCache.pageSize() - pageCache.pageSize() % getRecordSize(); } catch ( NoSuchFileException | StoreNotFoundException e ) { if ( createIfNotExists ) { try { createStore( pageSize ); return; } catch ( IOException e1 ) { e.addSuppressed( e1 ); } } if ( e instanceof StoreNotFoundException ) { throw (StoreNotFoundException) e; } throw new StoreNotFoundException( "Store file not found: " + storageFile, e ); } catch ( IOException e ) { throw new UnderlyingStorageException( "Unable to open store file: " + storageFile, e ); } loadStorage( filePageSize ); }
此方法由構造函數調用。檢查頭記錄並加載存儲。
注意:這個方法將用頁面緩存映射文件。在未映射以前,不能直接訪問存儲文件——只能經過頁面緩存訪問存儲文件。 @param createIfNotExists(若是爲真)建立並初始化存儲文件(若是它還不存在)。若是爲false,則此方法將在這種狀況下拋出異常。 這裏調用createStore(pageSize)建立存儲文件.
CommonAbstractStore.java private void createStore( int pageSize ) throws IOException { try ( PagedFile file = pageCache.map( storageFile, pageSize, StandardOpenOption.CREATE ) ) { initialiseNewStoreFile( file ); } checkAndLoadStorage( false ); } protected void initialiseNewStoreFile( PagedFile file ) throws IOException { if ( getNumberOfReservedLowIds() > 0 ) { try ( PageCursor pageCursor = file.io( 0, PF_SHARED_WRITE_LOCK ) ) { if ( pageCursor.next() ) { pageCursor.setOffset( 0 ); createHeaderRecord( pageCursor ); if ( pageCursor.checkAndClearBoundsFlag() ) { throw new UnderlyingStorageException( "Out of page bounds when writing header; page size too small: " + pageCache.pageSize() + " bytes." ); } } } } // Determine record size right after writing the header since some stores // use it when initializing their stores to write some records. recordSize = determineRecordSize(); idGeneratorFactory.create( idFile, getNumberOfReservedLowIds(), false ); } private void createHeaderRecord( PageCursor cursor ) { int offset = cursor.getOffset(); storeHeaderFormat.writeHeader( cursor ); cursor.setOffset( offset ); readHeaderAndInitializeRecordFormat( cursor ); } /** * This method is called when opening the store to extract header data and determine things like * record size of the specific record format for this store. Some formats rely on information * in the store header, that's why it happens at this stage. * * @param cursor {@link PageCursor} initialized at the start of the store header where header information * can be read if need be. This can be {@code null} if this store has no store header. The initialization * of the record format still happens in here. * @throws IOException if there were problems reading header information. */ private void readHeaderAndInitializeRecordFormat( PageCursor cursor ) { storeHeader = storeHeaderFormat.readHeader( cursor ); }
IntStoreHeaderFormat.java @Override public IntStoreHeader readHeader( PageCursor cursor ) { return new IntStoreHeader( cursor.getInt() ); }
[ ] 發現了 readme.md 或許有點用