mongodb批量處理

mongodb支持批量插入。

1.使用Java mongodb api

查看源碼com.mongodb.MongoCollectionImpl，有兩個方法

@Override public void insertMany(final List<? extends TDocument> documents) { insertMany(documents, new InsertManyOptions()); } @Override public void insertMany(final List<? extends TDocument> documents, final InsertManyOptions options) { notNull("documents", documents); List<InsertRequest> requests = new ArrayList<InsertRequest>(documents.size()); for (TDocument document : documents) { if (document == null) { throw new IllegalArgumentException("documents can not contain a null value"); } if (getCodec() instanceof CollectibleCodec) { document = ((CollectibleCodec<TDocument>) getCodec()).generateIdIfAbsentFromDocument(document); } requests.add(new InsertRequest(documentToBsonDocument(document))); } executor.execute(new MixedBulkWriteOperation(namespace, requests, options.isOrdered(), writeConcern) .bypassDocumentValidation(options.getBypassDocumentValidation())); }

insertMany(final List<? extends TDocument> documents) 默認使用 private boolean ordered = true;即有序插入。

insertMany(final List<? extends TDocument> documents, final InsertManyOptions options)調用者本身設置。

ordered屬性有什麼用？

/** * Gets whether the documents should be inserted in the order provided, stopping on the first failed insertion. The default is true. * If false, the server will attempt to insert all the documents regardless of an failures. * * @return whether the the documents should be inserted in order */
    public boolean isOrdered() { return ordered; }

大概意思是：

true：按提供的順序插入文檔，並在首次插入失敗時中止。 （按順序插入文檔，遇到失敗後中止。以前已經插入成功的不返回，失敗及失敗以後的不插入。）

false: 服務器將嘗試插入全部文檔，而不考慮失敗。（只要能插入成功的，都存儲進數據庫）

 

2.spring-data-mongodb

org.springframework.data.mongodb.core.MongoTemplate

如下是該方法的源碼和使用案例：

/* * (non-Javadoc) * @see org.springframework.data.mongodb.core.ExecutableInsertOperation#bulkOps(org.springframework.data.mongodb.core.BulkMode, java.lang.String) */
    public BulkOperations bulkOps(BulkMode bulkMode, String collectionName) { return bulkOps(bulkMode, null, collectionName); }

public int batchInsertStudents() {

 BulkWriteResult result = null;

try { 　　List<Student> documents = new ArrayList<>();
      　　String collectionName = "myTestCol"; 　　BulkOperations bulkOp = this.mongoTemplate.bulkOps(BulkMode.UNORDERED, collectionName);
      　　for(int i = 502610; i< 2000000; i++) { 　　Student student = new Student(); 　　student.setId(String.valueOf(i)); 　　student.setAge(i); 　　student.setGender("男"); 　　student.setName("李三"+ i); 　　documents.add(student); 　　} 　　bulkOp.insert(documents); result = bulkOp.execute();
 }catch (DuplicateKeyException e) { System.out.println("**********" + e.getMessage()); 　} return result; }

有兩種模式：

/** * Mode for bulk operation. **/
    enum BulkMode { /** Perform bulk operations in sequence. The first error will cancel processing. */ ORDERED, /** Perform bulk operations in parallel. Processing will continue on errors. */ UNORDERED };

與以前的order（true|false）相對應。