越来越觉得放弃JavaCV FFmpeg native API，直接使用JavaCV二次封装的API开发是很明智的选择，使用JavaCV二次封装的API开发避免了各种内存操作不当引起的crash。
上一次介绍了 JavaCV 采集摄像头及桌面视频数据，这次介绍一下如何采集摄像头和麦克风数据推送到流媒体服务器。
引入依赖
跟上一次一样，这里使用的还是最新的JavaCV库（1.5.5）.
<properties>
<javacpp.version>1.5.5</javacpp.version>
</properties>
<dependencies>
<dependency>
<groupId>org.bytedeco</groupId>
<artifactId>javacv</artifactId>
<version>${javacpp.version}</version>
</dependency>
<dependency>
<groupId>org.bytedeco</groupId>
<artifactId>javacv-platform</artifactId>
<version>${javacpp.version}</version>
</dependency>
</dependencies>
视频采集录制
在 JavaCV 采集摄像头及桌面视频数据 介绍了两种采集摄像头数据的方法，这里采用第一种，即使用 OpencvFrameGrabber 采集摄像头数据。
public class VideoRecorder implements Runnable {
private static final int VIDEO_DEVICE_INDEX = 0;
private FFmpegFrameRecorder recorder;
private int width, height;
public VideoRecorder(FFmpegFrameRecorder recorder, int width, int height) {
this.recorder = recorder;
this.width = width;
this.height = height;
}
@Override
public void run() {
try {
OpenCVFrameGrabber grabber = new OpenCVFrameGrabber(VIDEO_DEVICE_INDEX);
grabber.setImageWidth(width);
grabber.setImageHeight(height);
grabber.start();
long startTS = 0, videoTS = 0;
Frame frame = null;
while (!Thread.interrupted() && (frame = grabber.grab()) != null) {
if (startTS == 0) {
startTS = System.currentTimeMillis();
}
videoTS = 1000 * (System.currentTimeMillis() - startTS);
if (videoTS > recorder.getTimestamp()) {
recorder.setTimestamp(videoTS);
}
recorder.record(frame);
}
grabber.close();
} catch (Exception e) {
e.printStackTrace();
}
}
}
音频采集录制
音频采集直接使用JDK的API即可，直接读麦克风的数据塞给录制器，采样率为44100，16bit，小端模式。
public class AudioRecoder implements Runnable {
private FFmpegFrameRecorder recorder;
private int channels;
private int sampleRate;
public AudioRecoder(FFmpegFrameRecorder recorder, int sampleRate, int channels) {
this.recorder = recorder;
this.sampleRate = sampleRate;
this.channels = channels;
}
@Override
public void run() {
try {
AudioFormat format = new AudioFormat(Float.valueOf(sampleRate), 16, channels, true, false);
TargetDataLine line = (TargetDataLine) AudioSystem.getLine(new DataLine.Info(TargetDataLine.class, format));
line.open(format);
line.start();
int sampleRate = (int) format.getSampleRate();
int numChannels = format.getChannels();
byte[] buffer = new byte[sampleRate * numChannels];
while (!Thread.interrupted()) {
int nBytesRead = 0;
while (nBytesRead == 0) {
nBytesRead = line.read(buffer, 0, line.available());
}
int nSamplesRead = nBytesRead / 2;
short[] samples = new short[nSamplesRead];
ByteBuffer.wrap(buffer).order(ByteOrder.LITTLE_ENDIAN).asShortBuffer().get(samples);
ShortBuffer sBuff = ShortBuffer.wrap(samples, 0, nSamplesRead);
recorder.recordSamples(sampleRate, numChannels, sBuff);
}
} catch (Exception e) {
e.printStackTrace();
}
}
}
录制推流
这里的录制推流涉及音频和视频，音频和视频分开线程采集并录制，这样可以降低延迟，在录制推流方面还可以通过设置录制参数降低延迟：
recorder.setVideoOption("tune", "zerolatency");
recorder.setVideoOption("preset", "ultrafast");
启动录制推流器，视频采集录制线程和音频采集录制线程，开始推流：
recorder.start();
Thread vt = new Thread(new VideoRecorder(recorder, width, height));
Thread at = new Thread(new AudioRecoder(recorder, sampleRate, channels));
vt.start();
at.start();
vt.join();
at.join();
效果展示
可以使用VLC拉取程序的推流查看效果：
拉流的音视频流信息：
录制推流完整源码可以在公众号上获取
=========================================================
关注公众号 “HiIT青年” 发送 “javacv-recoder” 获取。（如果没有收到回复，可能是你之前取消过关注。）
关注公众号，阅读更多文章。
