[MetalKit]38-Using-ARKit-with-Metal-part-2使用ARKit與Metal-2

本系列文章是對 metalkit.org 上面MetalKit內容的全面翻譯和學習.

MetalKit系統文章目錄

---

正如上次強調的那樣,在ARKit應用中共有三個層級:渲染,追蹤和場景理解.上次咱們詳細分析瞭如何用Metal在自定義view中實現渲染. ARKit使用了視覺慣性里程計來精確追蹤周圍的世界,並將相機傳感器數據和CoreMotion數據結合起來.即便當咱們在運動時,也無需額外調校來保持畫面穩定.本文中,咱們關注場景理解-用平面檢測,點擊測試和光線估計來描述場景屬性的方法.ARKit能分析相機中的場景,並找到相似地板那樣的水平面.首先,咱們須要啓用平面檢測功能(默認是off),只需在運行會話配置羊添加一行:

override func viewWillAppear(_ animated: Bool) {
    super.viewWillAppear(animated)
    let configuration = ARWorldTrackingConfiguration()
    configuration.planeDetection = .horizontal
    session.run(configuration)
}
複製代碼

注意:當前API版本下,只能檢測水平面

ARSessionObserver代理方法對處理會話錯誤,追蹤改變和打斷很是有用:

func session(_ session: ARSession, didFailWithError error: Error) {}
func session(_ session: ARSession, cameraDidChangeTrackingState camera: ARCamera) {}
func session(_ session: ARSession, didOutputAudioSampleBuffer audioSampleBuffer: CMSampleBuffer) {}
func sessionWasInterrupted(_ session: ARSession) {}
func sessionInterruptionEnded(_ session: ARSession) {}
複製代碼

還有另外一些代理方法屬於ARSessionDelegate協議(屬於ARSessionObserver擴展)能讓咱們處理錨點.在第一個裏面寫上print():

func session(_ session: ARSession, didAdd anchors: [ARAnchor]) {
    print(anchors)
}
func session(_ session: ARSession, didRemove anchors: [ARAnchor]) {}
func session(_ session: ARSession, didUpdate anchors: [ARAnchor]) {}
func session(_ session: ARSession, didUpdate frame: ARFrame) {}
複製代碼

讓咱們進入到Renderer.swift裏面.首先,建立一些須要的類屬性.這些變量將幫助咱們建立並在屏幕上顯示一個調試用的平面:

var debugUniformBuffer: MTLBuffer!
var debugPipelineState: MTLRenderPipelineState!
var debugDepthState: MTLDepthStencilState!var debugMesh: MTKMesh!
var debugUniformBufferOffset: Int = 0
var debugUniformBufferAddress: UnsafeMutableRawPointer!
var debugInstanceCount: Int = 0
複製代碼

下一步,在**setupPipeline()**咱們建立緩衝器:

debugUniformBuffer = device.makeBuffer(length: anchorUniformBufferSize, options: .storageModeShared)
複製代碼

咱們須要給咱們的平面建立新的頂點及片斷函數,還有新的渲染管線和深度模板狀態.在建立命令隊列的前面添加幾行:

let debugGeometryVertexFunction = defaultLibrary.makeFunction(name: "vertexDebugPlane")!
let debugGeometryFragmentFunction = defaultLibrary.makeFunction(name: "fragmentDebugPlane")!
anchorPipelineStateDescriptor.vertexFunction =  debugGeometryVertexFunction
anchorPipelineStateDescriptor.fragmentFunction = debugGeometryFragmentFunction
do { try debugPipelineState = device.makeRenderPipelineState(descriptor: anchorPipelineStateDescriptor)
} catch let error { print(error) }
debugDepthState = device.makeDepthStencilState(descriptor: anchorDepthStateDescriptor)
複製代碼

下一步,在**setupAssets()**裏咱們須要建立一個新的Model I/O平面網格,並用它建立Metal網格.在函數的末尾添加幾行:

mdlMesh = MDLMesh(planeWithExtent: vector3(0.1, 0.1, 0.1), segments: vector2(1, 1), geometryType: .triangles, allocator: metalAllocator)
mdlMesh.vertexDescriptor = vertexDescriptor
do { try debugMesh = MTKMesh(mesh: mdlMesh, device: device)
} catch let error { print(error) }
複製代碼

下一步,在**updateBufferStates()**中咱們須要更新平面所在緩衝器的地址.添加下面幾行:

debugUniformBufferOffset = alignedInstanceUniformSize * uniformBufferIndex
debugUniformBufferAddress = debugUniformBuffer.contents().advanced(by: debugUniformBufferOffset)
複製代碼

下一步,在**updateAnchors()**中咱們須要更新變換矩陣和錨點數.在循環以前添加下面幾行:

let count = frame.anchors.filter{ $0.isKind(of: ARPlaneAnchor.self) }.count
debugInstanceCount = min(count, maxAnchorInstanceCount - (anchorInstanceCount - count))
複製代碼

而後,在循環中用下面幾行替換最後的三行:

if anchor.isKind(of: ARPlaneAnchor.self) {
    let transform = anchor.transform * rotationMatrix(rotation: float3(0, 0, Float.pi/2))
    let modelMatrix = simd_mul(transform, coordinateSpaceTransform)
    let debugUniforms = debugUniformBufferAddress.assumingMemoryBound(to: InstanceUniforms.self).advanced(by: index)
    debugUniforms.pointee.modelMatrix = modelMatrix
} else {
    let modelMatrix = simd_mul(anchor.transform, coordinateSpaceTransform)
    let anchorUniforms = anchorUniformBufferAddress.assumingMemoryBound(to: InstanceUniforms.self).advanced(by: index)
    anchorUniforms.pointee.modelMatrix = modelMatrix
}
複製代碼

咱們必須將平面繞z軸旋轉90度,來讓它呈水平狀態.注意,咱們使用了一個自定義方法,名爲rotationMatrix(),讓咱們來定義它.咱們在之前的文章中,當第一次介紹3D矩陣時就見過了這個矩陣:

func rotationMatrix(rotation: float3) -> float4x4 {
    var matrix: float4x4 = matrix_identity_float4x4
    let x = rotation.x
    let y = rotation.y
    let z = rotation.z
    matrix.columns.0.x = cos(y) * cos(z)
    matrix.columns.0.y = cos(z) * sin(x) * sin(y) - cos(x) * sin(z)
    matrix.columns.0.z = cos(x) * cos(z) * sin(y) + sin(x) * sin(z)
    matrix.columns.1.x = cos(y) * sin(z)
    matrix.columns.1.y = cos(x) * cos(z) + sin(x) * sin(y) * sin(z)
    matrix.columns.1.z = -cos(z) * sin(x) + cos(x) * sin(y) * sin(z)
    matrix.columns.2.x = -sin(y)
    matrix.columns.2.y = cos(y) * sin(x)
    matrix.columns.2.z = cos(x) * cos(y)
    matrix.columns.3.w = 1.0
    return matrix
}
複製代碼

下一步,在drawAnchorGeometry() 中咱們須要確保咱們在繪製以前至少有一個錨點.將第一行替換爲下面這行:

guard anchorInstanceCount - debugInstanceCount > 0 else { return }
複製代碼

下一步,讓咱們建立drawDebugGeometry() 函數來繪製咱們的平面.它很是相似於錨點繪製函數:

func drawDebugGeometry(renderEncoder: MTLRenderCommandEncoder) {
    guard debugInstanceCount > 0 else { return }
    renderEncoder.pushDebugGroup("DrawDebugPlanes")
    renderEncoder.setCullMode(.back)
    renderEncoder.setRenderPipelineState(debugPipelineState)
    renderEncoder.setDepthStencilState(debugDepthState)
    renderEncoder.setVertexBuffer(debugUniformBuffer, offset: debugUniformBufferOffset, index: 2)
    renderEncoder.setVertexBuffer(sharedUniformBuffer, offset: sharedUniformBufferOffset, index: 3)
    renderEncoder.setFragmentBuffer(sharedUniformBuffer, offset: sharedUniformBufferOffset, index: 3)
    for bufferIndex in 0..<debugMesh.vertexBuffers.count {
        let vertexBuffer = debugMesh.vertexBuffers[bufferIndex]
        renderEncoder.setVertexBuffer(vertexBuffer.buffer, offset: vertexBuffer.offset, index:bufferIndex)
    }
    for submesh in debugMesh.submeshes {
        renderEncoder.drawIndexedPrimitives(type: submesh.primitiveType, indexCount: submesh.indexCount, indexType: submesh.indexType, indexBuffer: submesh.indexBuffer.buffer, indexBufferOffset: submesh.indexBuffer.offset, instanceCount: debugInstanceCount)
    }
    renderEncoder.popDebugGroup()
}
複製代碼

在Renderer中,還有一件須要完成,就是-在update() 中結束編碼前,調用這個函數:

drawDebugGeometry(renderEncoder: renderEncoder)
複製代碼

最後,讓咱們進入Shaders.metal文件中.咱們須要一個新的結構體,只包含從頂點描述符中傳遞過來的頂點位置:

typedef struct {
    float3 position [[attribute(0)]];
} DebugVertex;
複製代碼

在頂點着色器中咱們用模型-視圖矩陣來更新頂點位置:

vertex float4 vertexDebugPlane(DebugVertex in [[ stage_in]],
                               constant SharedUniforms &sharedUniforms [[ buffer(3) ]],
                               constant InstanceUniforms *instanceUniforms [[ buffer(2) ]],
                               ushort vid [[vertex_id]],
                               ushort iid [[instance_id]]) {
    float4 position = float4(in.position, 1.0);
    float4x4 modelMatrix = instanceUniforms[iid].modelMatrix;
    float4x4 modelViewMatrix = sharedUniforms.viewMatrix * modelMatrix;
    float4 outPosition = sharedUniforms.projectionMatrix * modelViewMatrix * position;
    return outPosition;
}
複製代碼

最後,在片斷着色器中,咱們給平面一個顯眼在顏色以便於在視圖中觀察到它:

fragment float4 fragmentDebugPlane() {
    return float4(0.99, 0.42, 0.62, 1.0);
}
複製代碼

若是你運行應用,當檢測到平面時,你將看到添加了一個矩形,像這樣:

接下來要作的是當咱們檢測到更多或從先前檢測到的平面上移開時,更新/移除平面.別一個代理方法能幫助咱們實現這個效果.接下來,咱們將研究碰撞和物理效果.只是對之後的思考.

我要感謝Caroline爲本文構造了平面檢測.

源代碼source code已發佈在Github上.

下次見!