以前曾經寫過用SurfaceView，TextureView+MediaPlayer 播放視頻，和 ffmpeg avi 解碼後SurfaceView播放視頻 ，今天再給你們來一篇 OpenGL ES+MediaPlayer 來播放視頻。java
當年也曾呆過camera開發組近一年時間，惋惜那時候沒寫博客的意識，沒能給本身給你們留下多少乾貨分享。android
上個效果圖吧：
用 OpenGL 着色器實現黑白（灰度圖）效果。web
即 0.299,0.587,0.114 CRT中轉灰度的模型算法
下面看具體實現的邏輯：
若是你曾用 OpenGL 實現過貼圖，那麼就容易理解多了。微信
和圖片不一樣的是，視頻須要不斷地刷新，每當有新的一幀來時，咱們都應該更新紋理，而後從新繪製。用 OpenGL 播放視頻就是把視頻貼到屏幕上。app
對openGL不熟的同窗先看這裏：學 OpenGL 必知道的圖形學知識編輯器
1.先寫頂點着色器和片斷着色器ide
頂點着色器：函數
attribute vec4 aPosition;//頂點位置
attribute vec4 aTexCoord;//S T 紋理座標
varying vec2 vTexCoord;
uniform mat4 uMatrix;
uniform mat4 uSTMatrix;
void main() {
vTexCoord = (uSTMatrix * aTexCoord).xy;
gl_Position = uMatrix*aPosition;
}
片斷着色器：oop
#extension GL_OES_EGL_image_external : require
precision mediump float;
varying vec2 vTexCoord;
uniform samplerExternalOES sTexture;
void main() {
gl_FragColor=texture2D(sTexture, vTexCoord);
}
對着色器語言不懂的同窗看這裏：http://blog.csdn.net/king1425/article/details/71425556
samplerExternalOES 代替貼圖片時的 sampler2D ,做用就是和s urfaceTexture 配合進行紋理更新和格式轉換。
2.MediaPlayer的輸出
在 GLVideoRenderer 的構造函數中初始化 MediaPlayer ：
mediaPlayer=new MediaPlayer();
try{
mediaPlayer.setDataSource(context, Uri.parse(videoPath));
}catch (IOException e){
e.printStackTrace();
}
mediaPlayer.setAudioStreamType(AudioManager.STREAM_MUSIC);
mediaPlayer.setLooping(true);
mediaPlayer.setOnVideoSizeChangedListener(this);
onSurfaceCreated函數中使用SurfaceTexture來設置MediaPlayer的輸出。
咱們要用SurfaceTexture 建立一個Surface，而後將這個Surface做爲MediaPlayer的輸出表面。
SurfaceTexture的主要做用就是，從視頻流和相機數據流獲取新一幀的數據，獲取新數據調用的方法是updateTexImage。
須要注意的是MediaPlayer的輸出每每不是RGB格式（通常是YUV），而GLSurfaceView須要RGB格式才能正常顯示。
因此咱們先在onSurfaceCreated中將生成紋理的代碼改爲這樣：
textureId = textures[0];
GLES20.glBindTexture(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, textureId);
ShaderUtils.checkGlError("ws-------glBindTexture mTextureID");
GLES20.glTexParameterf(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_MIN_FILTER,
GLES20.GL_NEAREST);
GLES20.glTexParameterf(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_MAG_FILTER,
GLES20.GL_LINEAR);
GLES11Ext.GL_TEXTURE_EXTERNAL_OES的用處是什麼？
以前提到視頻解碼的輸出格式是YUV的（YUV420sp，應該是），那麼這個擴展紋理的做用就是實現YUV格式到RGB的自動轉化，咱們就不須要再爲此寫YUV轉RGB的代碼了。
而後在onSurfaceCreated的最後加上以下代碼:
surfaceTexture = new SurfaceTexture(textureId);
surfaceTexture.setOnFrameAvailableListener(this);//監聽是否有新的一幀數據到來
Surface surface = new Surface(surfaceTexture);
mediaPlayer.setSurface(surface);
surface.release();
if (!playerPrepared){
try {
mediaPlayer.prepare();
playerPrepared=true;
} catch (IOException t) {
Log.e(TAG, "media player prepare failed");
}
mediaPlayer.start();
playerPrepared=true;
}
用SurfaceTexture 建立一個Surface，而後將這個Surface做爲MediaPlayer的輸出表面.
在onDrawFrame中
synchronized (this){
if (updateSurface){
surfaceTexture.updateTexImage();//獲取新數據
surfaceTexture.getTransformMatrix(mSTMatrix);//讓新的紋理和紋理座標系可以正確的對應,mSTMatrix的定義是和projectionMatrix徹底同樣的。
updateSurface = false;
}
}
在有新數據時，用updateTexImage來更新紋理，這個getTransformMatrix的目的，是讓新的紋理和紋理座標系可以正確的對應, mSTMatrix 的定義是和 projectionMatrix 徹底同樣的。
private final float[] vertexData = {
1f,-1f,0f,
-1f,-1f,0f,
1f,1f,0f,
-1f,1f,0f
};
private final float[] textureVertexData = {
1f,0f,
0f,0f,
1f,1f,
0f,1f
};
vertexData 表明要繪製的視口座標。textureVertexData 表明視頻紋理，與屏幕座標對應
而後咱們讀取座標，在此本身咱們先與着色器映射。
在onSurfaceCreated映射
aPositionLocation= GLES20.glGetAttribLocation(programId,"aPosition");
uMatrixLocation=GLES20.glGetUniformLocation(programId,"uMatrix");
uSTMMatrixHandle = GLES20.glGetUniformLocation(programId, "uSTMatrix");
uTextureSamplerLocation=GLES20.glGetUniformLocation(programId,"sTexture");
aTextureCoordLocation=GLES20.glGetAttribLocation(programId,"aTexCoord");
onDrawFrame中讀取：
GLES20.glUseProgram(programId);
GLES20.glUniformMatrix4fv(uMatrixLocation,1,false,projectionMatrix,0);
GLES20.glUniformMatrix4fv(uSTMMatrixHandle, 1, false, mSTMatrix, 0);
vertexBuffer.position(0);
GLES20.glEnableVertexAttribArray(aPositionLocation);
GLES20.glVertexAttribPointer(aPositionLocation, 3, GLES20.GL_FLOAT, false,
12, vertexBuffer);
textureVertexBuffer.position(0);
GLES20.glEnableVertexAttribArray(aTextureCoordLocation);
GLES20.glVertexAttribPointer(aTextureCoordLocation,2,GLES20.GL_FLOAT,false,8,textureVertexBuffer);
GLES20.glActiveTexture(GLES20.GL_TEXTURE0);
GLES20.glBindTexture(GLES11Ext.GL_TEXTURE_EXTERNAL_OES,textureId);
GLES20.glUniform1i(uTextureSamplerLocation,0);
GLES20.glViewport(0,0,screenWidth,screenHeight);
GLES20.glDrawArrays(GLES20.GL_TRIANGLE_STRIP, 0, 4);
GLVideoRenderer 所有代碼以下：
package com.ws.openglvideoplayer;
import android.content.Context;
import android.graphics.SurfaceTexture;
import android.media.AudioManager;
import android.media.MediaPlayer;
import android.net.Uri;
import android.opengl.GLES11Ext;
import android.opengl.GLES20;
import android.opengl.GLSurfaceView;
import android.opengl.Matrix;
import android.util.Log;
import android.view.Surface;
import java.io.IOException;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.nio.FloatBuffer;
import javax.microedition.khronos.egl.EGLConfig;
import javax.microedition.khronos.opengles.GL10;
/**
* Created by Shuo.Wang on 2017/3/19.
*/
public class GLVideoRenderer implements GLSurfaceView.Renderer
, SurfaceTexture.OnFrameAvailableListener, MediaPlayer.OnVideoSizeChangedListener {
private static final String TAG = "GLRenderer";
private Context context;
private int aPositionLocation;
private int programId;
private FloatBuffer vertexBuffer;
private final float[] vertexData = {
1f,-1f,0f,
-1f,-1f,0f,
1f,1f,0f,
-1f,1f,0f
};
private final float[] projectionMatrix=new float[16];
private int uMatrixLocation;
private final float[] textureVertexData = {
1f,0f,
0f,0f,
1f,1f,
0f,1f
};
private FloatBuffer textureVertexBuffer;
private int uTextureSamplerLocation;
private int aTextureCoordLocation;
private int textureId;
private SurfaceTexture surfaceTexture;
private MediaPlayer mediaPlayer;
private float[] mSTMatrix = new float[16];
private int uSTMMatrixHandle;
private boolean updateSurface;
private boolean playerPrepared;
private int screenWidth,screenHeight;
public GLVideoRenderer(Context context,String videoPath) {
this.context = context;
playerPrepared=false;
synchronized(this) {
updateSurface = false;
}
vertexBuffer = ByteBuffer.allocateDirect(vertexData.length * 4)
.order(ByteOrder.nativeOrder())
.asFloatBuffer()
.put(vertexData);
vertexBuffer.position(0);
textureVertexBuffer = ByteBuffer.allocateDirect(textureVertexData.length * 4)
.order(ByteOrder.nativeOrder())
.asFloatBuffer()
.put(textureVertexData);
textureVertexBuffer.position(0);
mediaPlayer=new MediaPlayer();
try{
mediaPlayer.setDataSource(context, Uri.parse(videoPath));
}catch (IOException e){
e.printStackTrace();
}
mediaPlayer.setAudioStreamType(AudioManager.STREAM_MUSIC);
mediaPlayer.setLooping(true);
mediaPlayer.setOnVideoSizeChangedListener(this);
}
@Override
public void onSurfaceCreated(GL10 gl, EGLConfig config) {
String vertexShader = ShaderUtils.readRawTextFile(context, R.raw.simple_vertex_shader);
String fragmentShader= ShaderUtils.readRawTextFile(context, R.raw.simple_fragment_shader);
programId=ShaderUtils.createProgram(vertexShader,fragmentShader);
aPositionLocation= GLES20.glGetAttribLocation(programId,"aPosition");
uMatrixLocation=GLES20.glGetUniformLocation(programId,"uMatrix");
uSTMMatrixHandle = GLES20.glGetUniformLocation(programId, "uSTMatrix");
uTextureSamplerLocation=GLES20.glGetUniformLocation(programId,"sTexture");
aTextureCoordLocation=GLES20.glGetAttribLocation(programId,"aTexCoord");
int[] textures = new int[1];
GLES20.glGenTextures(1, textures, 0);
textureId = textures[0];
GLES20.glBindTexture(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, textureId);
ShaderUtils.checkGlError("glBindTexture mTextureID");
/*GLES11Ext.GL_TEXTURE_EXTERNAL_OES的用處？
以前提到視頻解碼的輸出格式是YUV的（YUV420p，應該是），那麼這個擴展紋理的做用就是實現YUV格式到RGB的自動轉化，
咱們就不須要再爲此寫YUV轉RGB的代碼了*/
GLES20.glTexParameterf(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_MIN_FILTER,
GLES20.GL_NEAREST);
GLES20.glTexParameterf(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_MAG_FILTER,
GLES20.GL_LINEAR);
surfaceTexture = new SurfaceTexture(textureId);
surfaceTexture.setOnFrameAvailableListener(this);//監聽是否有新的一幀數據到來
Surface surface = new Surface(surfaceTexture);
mediaPlayer.setSurface(surface);
surface.release();
if (!playerPrepared){
try {
mediaPlayer.prepare();
playerPrepared=true;
} catch (IOException t) {
Log.e(TAG, "media player prepare failed");
}
mediaPlayer.start();
playerPrepared=true;
}
}
@Override
public void onSurfaceChanged(GL10 gl, int width, int height) {
Log.d(TAG, "onSurfaceChanged: "+width+" "+height);
screenWidth=width; screenHeight=height;
}
@Override
public void onDrawFrame(GL10 gl) {
GLES20.glClear( GLES20.GL_DEPTH_BUFFER_BIT | GLES20.GL_COLOR_BUFFER_BIT);
synchronized (this){
if (updateSurface){
surfaceTexture.updateTexImage();//獲取新數據
surfaceTexture.getTransformMatrix(mSTMatrix);//讓新的紋理和紋理座標系可以正確的對應,mSTMatrix的定義是和projectionMatrix徹底同樣的。
updateSurface = false;
}
}
GLES20.glUseProgram(programId);
GLES20.glUniformMatrix4fv(uMatrixLocation,1,false,projectionMatrix,0);
GLES20.glUniformMatrix4fv(uSTMMatrixHandle, 1, false, mSTMatrix, 0);
vertexBuffer.position(0);
GLES20.glEnableVertexAttribArray(aPositionLocation);
GLES20.glVertexAttribPointer(aPositionLocation, 3, GLES20.GL_FLOAT, false,
12, vertexBuffer);
textureVertexBuffer.position(0);
GLES20.glEnableVertexAttribArray(aTextureCoordLocation);
GLES20.glVertexAttribPointer(aTextureCoordLocation,2,GLES20.GL_FLOAT,false,8,textureVertexBuffer);
GLES20.glActiveTexture(GLES20.GL_TEXTURE0);
GLES20.glBindTexture(GLES11Ext.GL_TEXTURE_EXTERNAL_OES,textureId);
GLES20.glUniform1i(uTextureSamplerLocation,0);
GLES20.glViewport(0,0,screenWidth,screenHeight);
GLES20.glDrawArrays(GLES20.GL_TRIANGLE_STRIP, 0, 4);
}
@Override
synchronized public void onFrameAvailable(SurfaceTexture surface) {
updateSurface = true;
}
@Override
public void onVideoSizeChanged(MediaPlayer mp, int width, int height) {
Log.d(TAG, "onVideoSizeChanged: "+width+" "+height);
updateProjection(width,height);
}
private void updateProjection(int videoWidth, int videoHeight){
float screenRatio=(float)screenWidth/screenHeight;
float videoRatio=(float)videoWidth/videoHeight;
if (videoRatio>screenRatio){
Matrix.orthoM(projectionMatrix,0,-1f,1f,-videoRatio/screenRatio,videoRatio/screenRatio,-1f,1f);
}else Matrix.orthoM(projectionMatrix,0,-screenRatio/videoRatio,screenRatio/videoRatio,-1f,1f,-1f,1f);
}
public MediaPlayer getMediaPlayer() {
return mediaPlayer;
}
}
要實現上圖中的濾鏡視頻效果，只需用0.299,0.587,0.114 CRT中轉灰度的模型算法。（本身能夠網上搜尋更多效果，這裏只是拋磚引玉）
更改片斷着色器便可：
#extension GL_OES_EGL_image_external : require
precision mediump float;
varying vec2 vTexCoord;
uniform samplerExternalOES sTexture;
void main() {
//gl_FragColor=texture2D(sTexture, vTexCoord);
vec3 centralColor = texture2D(sTexture, vTexCoord).rgb;
gl_FragColor = vec4(0.299*centralColor.r+0.587*centralColor.g+0.114*centralColor.b);
}
到此結束，咱們已經實現了OpenGL ES+MediaPlayer 渲染播放視頻+濾鏡效果。後期將講述全景視頻原理及實現過程，敬請關注~
做者：小碼哥_WS
連接：https://www.jianshu.com/p/13320a8549db
-- END --
進技術交流羣，掃碼添加個人微信：Byte-Flow
獲取視頻教程和源碼
推薦：
FFmpeg + OpenGLES 實現視頻解碼播放和視頻濾鏡
FFmpeg + OpenGL ES 實現 3D 全景播放器
以爲不錯，點個在看唄~
本文分享自微信公衆號 - 字節流動（google_developer）。
若有侵權，請聯繫 support@oschina.cn 刪除。
本文參與「OSC源創計劃」，歡迎正在閱讀的你也加入，一塊兒分享。