[ARKit]4-着色器Shader的應用

說明

本文代碼地址

ARKit系列文章目錄

上一篇咱們已經初步示範了:如何讀取着色器String,並經過shaderModifiers加載.
Shader類型SCNShaderModifierEntryPoint有geometry,surface,lightingModel,fragment.

使用示例

咱們以surface類型爲例.swift
如何給着色器傳參呢??直接使用KVC...

private func setupShader() {
    guard let path = Bundle.main.path(forResource: "skin", ofType: "shaderModifier", inDirectory: "art.scnassets"),
    let shader = try? String(contentsOfFile: path, encoding: String.Encoding.utf8) else {
        return
    }
    
    skin.shaderModifiers = [SCNShaderModifierEntryPoint.surface: shader]
    
    skin.setValue(Double(0), forKey: "blendFactor")
    skin.setValue(NSValue(scnVector3: SCNVector3Zero), forKey: "skinColorFromEnvironment")
    
    let sparseTexture = SCNMaterialProperty(contents: UIImage(named: "art.scnassets/textures/chameleon_DIFFUSE_BASE.png")!)
    skin.setValue(sparseTexture, forKey: "sparseTexture")
}

複製代碼

OC

NSURL *url = [[NSBundle mainBundle] URLForResource:@"skin" withExtension:@"shaderModifier"];
    NSError *error;
    NSString *shaderSource = [[NSString alloc] initWithContentsOfURL:url
                                                            encoding:NSUTF8StringEncoding
                                                               error:&error];
    if (!shaderSource) {
        // Handle the error
        NSLog(@"Failed to load shader source code, with error: %@", [error localizedDescription]);
    } else {
        skin.shaderModifiers = @{ SCNShaderModifierEntryPointSurface : shader };
    }

    [skin setValue:@(0) forKey:@"blendFactor"];
    [skin setValue:[NSValue valueWithSCNVector3:SCNVector3Zero] forKey:@"skinColorFromEnvironment"];
    SCNMaterialProperty *sparseTexture = [SCNMaterialProperty materialPropertyWithContents:[NSImage imageNamed:@"art.scnassets/textures/chameleon_DIFFUSE_BASE.png"]];
    [skin setValue:sparseTexture forKey:@"sparseTexture"];
複製代碼

shader類型詳解

• SCNShaderModifierEntryPointGeometry : 用來處理幾何體形變.

輸入爲結構體geometry:

struct SCNShaderGeometry {
   float4 position;
   float3 normal;
   float4 tangent;
   float4 color;
   float2 texcoords[kSCNTexcoordCount];
} _geometry;

Access: ReadWrite 訪問權限:讀寫
Stages: Vertex shader only 只限於頂點着色器中
複製代碼

kSCNTexcoordCount是個整型常量,表示用到的頂點座標數.
其中的向量及座標(position, normal and tangent)爲模型空間.
着色器示例,正弦變形:

GLSL
 uniform float Amplitude = 0.1;

 _geometry.position.xyz += _geometry.normal * (Amplitude * _geometry.position.y * _geometry.position.x) * sin(u_time);

Metal Shading Language
 #pragma arguments
 float Amplitude;
 
 #pragma body
 _geometry.position.xyz += _geometry.normal * (Amplitude * _geometry.position.y * _geometry.position.x) * sin(scn_frame.time);
複製代碼

• SCNShaderModifierEntryPointSurface : 用來在光照應用前處理材質.

輸入爲結構體surface:

struct SCNShaderSurface {
       float3 view;                     // Direction from the point on the surface toward the camera (V)
       float3 position;                 // Position of the fragment
       float3 normal;                   // Normal of the fragment (N)
       float3 geometryNormal;           // Geometric normal of the fragment (normal map is ignored)
       float3 tangent;                  // Tangent of the fragment
       float3 bitangent;                // Bitangent of the fragment
       float4 ambient;                  // Ambient property of the fragment
       float2 ambientTexcoord;          // Ambient texture coordinates
       float4 diffuse;                  // Diffuse property of the fragment. Alpha contains the opacity.
       float2 diffuseTexcoord;          // Diffuse texture coordinates
       float4 specular;                 // Specular property of the fragment
       float2 specularTexcoord;         // Specular texture coordinates
       float4 emission;                 // Emission property of the fragment
       float2 emissionTexcoord;         // Emission texture coordinates
       float4 multiply;                 // Multiply property of the fragment
       float2 multiplyTexcoord;         // Multiply texture coordinates
       float4 transparent;              // Transparent property of the fragment
       float2 transparentTexcoord;      // Transparent texture coordinates
       float4 reflective;               // Reflective property of the fragment
       float  metalness;                // Metalness property of the fragment
       float2 metalnessTexcoord;        // Metalness texture coordinates
       float  roughness;                // Roughness property of the fragment
       float2 roughnessTexcoord;        // Roughness texture coordinates
       float4 selfIllumination;         // Self Illumination property of the fragment. Available since macOS 10.13, iOS 11, tvOS 11 and watchOS 4. Available as `emission` in previous versions.
       float2 selfIlluminationTexcoord; // Self Illumination texture coordinates. Available since macOS 10.13, iOS 11, tvOS 11 and watchOS 4. Available as `emissionTexcoord` in previous versions.
       float  ambientOcclusion;         // Ambient Occlusion property of the fragment. Available macOS 10.13, iOS 11, tvOS 11 and watchOS 4. Available as `multiply` in previous versions.
       float2 ambientOcclusionTexcoord; // Ambient Occlusion texture coordinates. Available since macOS 10.13, iOS 11, tvOS 11 and watchOS 4. Available as `multiplyTexcoord` in previous versions.
       float  shininess;                // Shininess property of the fragment
       float  fresnel;                  // Fresnel property of the fragment
} _surface;

Access: ReadWrite 訪問權限:讀寫
Stages: Fragment shader only 只限於片斷着色器中
複製代碼

其中的向量及座標爲視圖空間.
着色器示例,產生黑白條紋:

GLSL
 uniform float Scale = 12.0;
 uniform float Width = 0.25;
 uniform float Blend = 0.3;

 vec2 position = fract(_surface.diffuseTexcoord * Scale);
 float f1 = clamp(position.y / Blend, 0.0, 1.0);
 float f2 = clamp((position.y - Width) / Blend, 0.0, 1.0);
 f1 = f1 * (1.0 - f2);
 f1 = f1 * f1 * 2.0 * (3. * 2. * f1);
 _surface.diffuse = mix(vec4(1.0), vec4(vec3(0.0),1.0), f1);

Metal Shading Language
 #pragma arguments
 float Scale;
 float Width;
 float Blend;

 #pragma body
 float2 position = fract(_surface.diffuseTexcoord * Scale);
 float f1 = clamp(position.y / Blend, 0.0, 1.0);
 float f2 = clamp((position.y - Width) / Blend, 0.0, 1.0);
 f1 = f1 * (1.0 - f2);
 f1 = f1 * f1 * 2.0 * (3. * 2. * f1);
 _surface.diffuse = mix(float4(1.0), float4(float3(0.0),1.0), f1);
複製代碼

• SCNShaderModifierEntryPointLightingModel : 提供自定義的光照方程.

輸入有多個,SCNShaderModifierEntryPointSurface中的全部結構體,結構體lightingContribution,結構體light:

All the structures available from the SCNShaderModifierEntryPointSurface entry point 
SCNShaderModifierEntryPointSurface中的全部結構體
Access: ReadOnly 訪問權限:只讀
Stages: Vertex shader and fragment shader 頂點着色器及片斷着色器中

struct SCNShaderLightingContribution {
       float3 ambient;
       float3 diffuse;
       float3 specular;
} _lightingContribution;

Access: ReadWrite 訪問權限:讀寫
Stages: Vertex shader and fragment shader 頂點着色器及片斷着色器中

struct SCNShaderLight {
       float4 intensity;
       float3 direction; // Direction from the point on the surface toward the light (L)
} _light;

Access: ReadOnly 訪問權限:只讀
Stages: Vertex shader and fragment shader 頂點着色器及片斷着色器中
複製代碼

着色器示例,漫反射光照:

GLSL
uniform float WrapFactor = 0.5;

float dotProduct = (WrapFactor + max(0.0, dot(_surface.normal,_light.direction))) / (1 + WrapFactor);
_lightingContribution.diffuse += (dotProduct * _light.intensity.rgb);
vec3 halfVector = normalize(_light.direction + _surface.view);
dotProduct = max(0.0, pow(max(0.0, dot(_surface.normal, halfVector)), _surface.shininess));
_lightingContribution.specular += (dotProduct * _light.intensity.rgb);

Metal Shading Language
#pragma arguments
float WrapFactor;

#pragma body
float dotProduct = (WrapFactor + max(0.0, dot(_surface.normal,_light.direction))) / (1 + WrapFactor);
_lightingContribution.diffuse += (dotProduct * _light.intensity.rgb);
float3 halfVector = normalize(_light.direction + _surface.view);
dotProduct = max(0.0, pow(max(0.0, dot(_surface.normal, halfVector)), _surface.shininess));
_lightingContribution.specular += (dotProduct * _light.intensity.rgb);

複製代碼

• SCNShaderModifierEntryPointFragment : 用來最後修改顏色.

輸入爲SCNShaderModifierEntryPointSurface中的結構體,及結構體output:

All the structures available from the SCNShaderModifierEntryPointSurface entry point
SCNShaderModifierEntryPointSurface中的全部結構體
Access: ReadOnly 訪問權限:只讀
Stages: Fragment shader only 只限於片斷着色器中

struct SCNShaderOutput {
       float4 color;
} _output;

Access: ReadWrite 訪問權限:讀寫
Stages: Fragment shader only 只限於片斷着色器中
複製代碼

着色器示例,反轉最終的顏色:

GLSL
_output.color.rgb = vec3(1.0) - _output.color.rgb;

Metal Shading Language
_output.color.rgb = 1.0 - _output.color.rgb;
複製代碼

---

着色器格式與參數

shaderModifiers相關說明:

// Custom GLSL uniforms declarations are of the form: [uniform type uniformName [= defaultValue]]
// 自定義GLSL uniforms聲明格式:[uniform 類型 全局變量名 [= 默認值]]
uniform float myGrayAmount = 3.0;

// Custom Metal uniforms declarations require a #pragma and are of the form: [type name]
// 自定義Metal uniforms聲明格式要求#pragma,格式:[類型 全局變量名]
#pragma arguments
float myGrayAmount;

// In Metal, you can also transfer varying values from the vertex shader (geometry modifier) to the fragment shader (surface/fragment modifier)
// 在Metal中,你將變量值從頂點着色器(geometry modifier)傳遞到片斷着色器(surface/fragment modifier)中
// In one (or both) of the modifier, declare the varying values
// 在一個(或所有)的modifier中,聲明變量值
#pragma varyings
half3 myVec;

// Output varying values in the geometry modifier
// 在geometry modifier中輸出變量值
out.myVec = _geometry.normal.xyz * 0.5h + 0.5h;

// And use them in the fragment modifier
// 在fragment modifier中使用這些值
_output.color.rgb = saturate(in.myVec);

// Optional global function definitions (for Metal: references to uniforms in global functions are not supported).
// 可選的全局函數定義(Metal中不支持在全局函數中對uniforms的引用)
float mySin(float t) {
       return sin(t);
}

[#pragma transparent | opaque]
[#pragma body]

// the shader modifier code snippet itself
// 
float3 myColor = myGrayAmount;
_output.color.rgb += myColor;
複製代碼

#pragma body指令:當聲明函數不在着色器代碼中時,必須使用這個指令.

#pragma transparent指令:強制使用公式_output.color.rgb + (1 - _output.color.a) * dst.rgb;來渲染混合模式;其中dst表示當前片斷顏色,rgb份量必須是自左乘的.

#pragma opaque指令:強制渲染爲不透明的.忽略片斷的alpha份量.

SCNGeometry和SCNMaterial類是兼容key-value編碼的類,就是說你能夠用KVC來賦值,哪怕key myAmplitude在當前類中並無被聲明,仍然能夠賦值.
當在shader modifier中聲明myAmplitude uniform後,SceneKit會監聽接收者的myAmplitude key.當該值改變時,SceneKit會給uniform綁定一個新的值.
普通用NSValue包裝的標題類型都是支持的.

對於Metal:

• MTLBuffer也支持做爲values.
• 在Metal shader中聲明的複雜數據類型(結構體)也是支持的.
  • 你能夠用NSData將它們設置爲一個總體.
  • 或者你能夠使用單獨的結構體成員,將成員名稱做爲key,對應類型的成員值做爲value.

自定義uniforms能夠使用顯式動畫.
在聲明或綁定自定義uniforms時,可用到的對應類型以下:

GLSL	Metal Shading Language	Objective-C
int	int	NSNumber, NSInteger, int
float	float	NSNumber, CGFloat, float, double
vec2	float2	CGPoint
vec3	float3	SCNVector3
vec4	float4	SCNVector4
mat4, mat44	float4x4	SCNMatrix4
sampler2D	texture2d	SCNMaterialProperty
samplerCube	texturecube	SCNMaterialProperty (with a cube map)

下列前綴是SceneKit默認保留的,在自定義變量命名中不能使用:
u_
a_
v_

SceneKit聲明的內置uniforms共有如下這些:

GLSL	Metal Shading Language	說明
float u_time	float scn_frame.time	The current time, in seconds
vec2 u_inverseResolution	float2 scn_frame.inverseResolution	1.0 / screen size
mat4 u_viewTransform	float4x4 scn_frame.viewTransform	See SCNViewTransform
mat4 u_inverseViewTransform	float4x4 scn_frame.inverseViewTransform
mat4 u_projectionTransform	float4x4 scn_frame.projectionTransform	See SCNProjectionTransform
mat4 u_inverseProjectionTransform	float4x4 scn_frame.inverseProjectionTransform
mat4 u_normalTransform	float4x4 scn_node.normalTransform	See SCNNormalTransform
mat4 u_modelTransform	float4x4 scn_node.modelTransform	See SCNModelTransform
mat4 u_inverseModelTransform	float4x4 scn_node.inverseModelTransform
mat4 u_modelViewTransform	float4x4 scn_node.modelViewTransform	See SCNModelViewTransform
mat4 u_inverseModelViewTransform	float4x4 scn_node.inverseModelViewTransform
mat4 u_modelViewProjectionTransform	float4x4 scn_node.modelViewProjectionTransform	See SCNModelViewProjectionTransform
mat4 u_inverseModelViewProjectionTransform	float4x4 scn_node.inverseModelViewProjectionTransform
mat2x3 u_boundingBox;	float2x3 scn_node.boundingBox	The bounding box of the current geometry, in model space, u_boundingBox[0].xyz and u_boundingBox[1].xyz being respectively the minimum and maximum corner of the box.

着色器代碼示例

首先,咱們新建個AR項目,Xcode會自動生成默認的項目,顯示一個小飛機. 咱們對界面稍加改動,添加幾個開關,用來控制shader.

而後實現setupShader()方法來加載四個shader,並在viewDidLoad()中調用.這裏咱們對着色器稍作修改,添加一個效果因子factor來控制效果是否顯示:

private func setupShader() {
    
    let shipNode = sceneView.scene.rootNode.childNode(withName: "shipMesh", recursively: true)
    let skin = shipNode?.geometry?.firstMaterial;
    // 爲了方便觀察混合效果,我在各個示例shader中添加了一個因子factor,分別設置爲0.0和1.0能夠控制效果的開啓和關閉;默認爲0.0--關閉;
    let geometryShader = """
        uniform float Amplitude = 0.1;
        uniform float GeometryFactor = 0.0;

        _geometry.position.xyz += _geometry.normal * (Amplitude * _geometry.position.y * _geometry.position.x) * sin(u_time) * GeometryFactor;
        """
    
    let surfaceShader = """
        uniform float Scale = 12.0;
        uniform float Width = 0.25;
        uniform float Blend = 0.3;
        uniform float SurfaceFactor = 0.0;

        vec2 position = fract(_surface.diffuseTexcoord * Scale);
        float f1 = clamp(position.y / Blend, 0.0, 1.0);
        float f2 = clamp((position.y - Width) / Blend, 0.0, 1.0);
        f1 = f1 * (1.0 - f2);
        f1 = f1 * f1 * 2.0 * (3. * 2. * f1);
        _surface.diffuse = _surface.diffuse * (1-SurfaceFactor) + mix(vec4(1.0), vec4(vec3(0.0),1.0), f1) * SurfaceFactor;
        """
    
    let lightShader = """
        uniform float WrapFactor = 0.5;
        uniform float LightFactor = 0.0;

        float dotProduct = (WrapFactor + max(0.0, dot(_surface.normal,_light.direction))) / (1 + WrapFactor);
        _lightingContribution.diffuse += (dotProduct * _light.intensity.rgb) * LightFactor;

        vec3 halfVector = normalize(_light.direction + _surface.view);
        dotProduct = max(0.0, pow(max(0.0, dot(_surface.normal, halfVector)), _surface.shininess));
        _lightingContribution.specular += (dotProduct * _light.intensity.rgb) * LightFactor;
        """
    
    let fragmentShader = """
        uniform float FragmentFactor = 0.0;

        _output.color.rgb = (vec3(1.0) - _output.color.rgb) * FragmentFactor + (1-FragmentFactor) * _output.color.rgb;
        """
    
    skin?.shaderModifiers = [SCNShaderModifierEntryPoint.geometry: geometryShader,
                             SCNShaderModifierEntryPoint.surface: surfaceShader,
                             SCNShaderModifierEntryPoint.lightingModel: lightShader,
                             SCNShaderModifierEntryPoint.fragment: fragmentShader
    ]
    
}
複製代碼

最後,只要在開關改變時,用KVC設置對應shader的因子值就能夠了:

@IBAction func switchChange(_ sender: UISwitch) {
    let shipNode = sceneView.scene.rootNode.childNode(withName: "shipMesh", recursively: true)
    let skin = shipNode?.geometry?.firstMaterial;
    let factor = sender.isOn ? 1.0 : 0.0
    switch sender.tag {
    case 10:
        skin?.setValue(Double(factor), forKey: "GeometryFactor")
    case 11:
        skin?.setValue(Double(factor), forKey: "SurfaceFactor")
    case 12:
        skin?.setValue(Double(factor), forKey: "LightFactor")
    case 13:
        skin?.setValue(Double(factor), forKey: "FragmentFactor")
    default:
        print("switch")
    }
}
複製代碼

效果如圖: