前言
最近看一部新番——《國王排名》非常的溫暖治愈乖篷,當(dāng)時在想,什么樣的作者能畫出這樣的漫畫擅笔?
于是去了解一下這部番的作者——十日草輔险胰。
- 23歲立志當(dāng)漫畫家汹押,多次向出版社投稿后沒有下文,最終因為生活所迫起便,不得已成為上班族棚贾。
- 41歲告別上班族生活窖维,再次挑戰(zhàn)成為漫畫家的夢想。
- 43歲在素人漫畫網(wǎng)站投稿一年半后妙痹,漫畫在推特成為話題铸史,點擊率迎來爆炸性成長,也收到正規(guī)出版社的聯(lián)系怯伊。
當(dāng)了20年的社畜琳轿,因為這一段經(jīng)歷,才能畫出來如此令人淚目的作品吧耿芹!
聊聊數(shù)據(jù)流:在我剛做攝像頭的時候崭篡,我就一直想弄清楚,數(shù)據(jù)流是怎么傳輸?shù)陌娠酰?jīng)過哪些模塊琉闪?
掌握這個流程,對于分析camera圖像異常問題砸彬,算法集成颠毙,大有裨益!畫圖工具:Visio
-
本文知識點:
- 一拿霉、數(shù)據(jù)流總體框圖
- 二吟秩、硬件層面的傳輸
- 三咱扣、軟件層面的傳輸
- 四绽淘、應(yīng)用以及debug技巧
介紹幾個概念
- ISP(Image Signal Processing) 圖像信號處理,在高通中相當(dāng)于就是Camera2 架構(gòu)中的 VFE,負責(zé)處理Sensor模塊的原數(shù)據(jù)流(RAW圖)并生成 ISP 輸出(YUV圖)闹伪。
- Channel :通道沪铭、管道的意思,捆綁多個圖像流的松散概念偏瓤,想象一下杀怠,數(shù)據(jù)流就類似于我們的水流,我們要把水送給千家萬戶厅克,是不是需要水管赔退。因此——Channel相當(dāng)于水管
- Stream :流,類似于現(xiàn)實世界的水流证舟。每個流只能有一種格式;它是在相機硬件和應(yīng)用程序之間交換捕獲圖像緩沖區(qū)的接口
一硕旗、數(shù)據(jù)流總體框圖
sensor模塊:獲取圖像,輸出raw圖女责。
ISPIF: ISP interface漆枚,VFE和Sensor模塊的中間層接口。
VFE(Video Front End):一個ISP-硬件圖像處理芯片抵知,用來處理相機幀數(shù)據(jù)墙基,高通的ISP算法就運行于這個模塊软族,例如:白平衡,LSC残制,gama校正立砸,AF,AWB,AE等。
CPP(Camera Postprocessor):camera后處理痘拆,用于處理VFE模塊輸出的YUV圖像仰禽,支持翻轉(zhuǎn)/旋轉(zhuǎn)、降噪纺蛆、平滑/銳化吐葵、裁剪功能。
HAL層:把YUV圖像送給LCD顯示或者給app桥氏,后續(xù)存儲成jpg温峭。
二、硬件層面的傳輸
- Sensor捕捉到 raw圖 通過MIPI傳輸?shù)?ISPIF(ISP interface)字支,
- 然后把 raw圖 傳輸給VFE對圖像數(shù)據(jù)進行處理(如白平衡凤藏,顏色校正,gama校正等)堕伪,輸出YUV圖揖庄。
數(shù)據(jù)到了VFE(即ISP)后怎么傳輸?shù)杰浖用妫?/strong>
- VFE(ISP)會把 【YUV圖像數(shù)據(jù)】 填充到buffer中,填充好了之后欠雌,會發(fā)送一個中斷信號
- 中斷處理程序msm_isp_process_axi_irq設(shè)置當(dāng)前buf的狀態(tài)是填充好的蹄梢,最終把buf添加到done buffers隊列里
調(diào)用流程如下:
msm_isp_process_axi_irq ->
msm_isp_process_axi_irq_stream ->
msm_isp_process_done_buf ->
msm_isp_buf_done ->
msm_vb2_buf_done ->
vb2_buffer_done
void msm_isp_process_axi_irq_stream(···)
{
struct msm_isp_buffer *done_buf = NULL;
···
//填充buf
done_buf = stream_info->buf[pingpong_bit];
···
if (stream_info->pending_buf_info.is_buf_done_pending != 1) {
//處理填充的buf
msm_isp_process_done_buf(vfe_dev, stream_info,
done_buf, time_stamp, frame_id);
}
}
最終調(diào)用到vb2_buffer_done
void vb2_buffer_done(struct vb2_buffer *vb, enum vb2_buffer_state state)
{
//buffer隊列
struct vb2_queue *q = vb->vb2_queue;
···
/* 同步 buffers */
for (plane = 0; plane < vb->num_planes; ++plane)
call_void_memop(vb, finish, vb->planes[plane].mem_priv);
spin_lock_irqsave(&q->done_lock, flags);
if (state == VB2_BUF_STATE_QUEUED ||
state == VB2_BUF_STATE_REQUEUEING) {
vb->state = VB2_BUF_STATE_QUEUED;
} else {
/* Add the buffer to the done buffers list */
list_add_tail(&vb->done_entry, &q->done_list);
vb->state = state;//設(shè)置buf狀態(tài)
}
atomic_dec(&q->owned_by_drv_count);
spin_unlock_irqrestore(&q->done_lock, flags);
···
}
- 將緩沖區(qū)添加到done buffers鏈表中
list_add_tail(&vb->done_entry, &q->done_list); - 設(shè)置buf狀態(tài):VB2_BUF_STATE_DONE
vb->state = state;
到這里,我們的數(shù)據(jù)就被填充到kernel的緩沖區(qū)(buffer)富俄,等待用戶空間取數(shù)據(jù)禁炒!
所有的buffer都由vb2_queue隊列管理
三、軟件層面的傳輸
數(shù)據(jù)流框圖如下:
3.1 kernel層
-
vb2 queue隊列的申請
kernel/msm-4.9/drivers/media/platform/msm/camera_v2/camera/camera.c
static int camera_v4l2_open(struct file *filep)
{
...
/* every stream has a vb2 queue */
//初始化vbuffer2_queue
rc = camera_v4l2_vb2_q_init(filep)
···
}
通過camera_v4l2_vb2_q_init為每一個stream申請vb2 queue霍比。
需要注意的是幕袱,這里申請的是 用來存放buffer的隊列。
-
buffer的申請
而buffer的在用戶空間申請的悠瞬,高通使用的是用戶空間緩沖區(qū)(V4L2_MEMORY_USERPTR)
hardware/qcom/camera/QCamera2/HAL/QCameraMem.cpp
int QCameraMemory::allocOneBuffer(QCameraMemInfo &memInfo,
unsigned int heap_id, size_t size, bool cached, bool secure_mode)
{
main_ion_fd = open("/dev/ion", O_RDONLY);
rc = ioctl(main_ion_fd, ION_IOC_ALLOC, &alloc);
}
為什么使用ION buffer呢们豌?
- ION 是當(dāng)前 Android 流行的內(nèi)存分配管理機制,在多媒體部分中使用的最多浅妆,例如從 Camera 到 Display望迎,
從 Mediaserver 到 Surfaceflinger,都會利用 ION 進行內(nèi)存分配管理狂打。 - ION-最顯著的特點是它可以被用戶空間的進程之間或者內(nèi)核空間的模塊之間進行內(nèi)存共享擂煞,而且這種共享可以是零拷貝的。
3.2 mm-camera-interface層
3.2.1 CAM_dataPoll線程的創(chuàng)建
這部分的代碼分析會有些繞趴乡,看下圖去理解:
hardware/qcom/camera/QCamera2/stack/mm-camera-interface/src/mm_camera_channel.c
int32_t mm_channel_init(mm_channel_t *my_obj,
mm_camera_channel_attr_t *attr,
mm_camera_buf_notify_t channel_cb,
void *userdata)
{
···
LOGD("Launch data poll thread in channel open");
snprintf(my_obj->poll_thread[0].threadName, THREAD_NAME_SIZE, "CAM_dataPoll");
mm_camera_poll_thread_launch(&my_obj->poll_thread[0],
MM_CAMERA_POLL_TYPE_DATA);
/* change state to stopped state */
my_obj->state = MM_CHANNEL_STATE_STOPPED;
return rc;
}
mm_camera_poll_thread_launch 會創(chuàng)建一個管道对省,讓線程進行通信蝗拿;
- 這里mm_camera_poll_thread_launch在主線程運行
- mm_camera_poll_thread在CAM_dataPoll線程運行
int32_t mm_camera_poll_thread_launch(mm_camera_poll_thread_t * poll_cb,
mm_camera_poll_thread_type_t poll_type)
{
//Initialize pipe fds
poll_cb->pfds[0] = -1;
poll_cb->pfds[1] = -1;
rc = pipe(poll_cb->pfds);
/* 創(chuàng)建一個data_poll線程 */
pthread_create(&poll_cb->pid, NULL, mm_camera_poll_thread, (void *)poll_cb);
<=================================>
static void *mm_camera_poll_thread(void *data)
{
mm_camera_poll_thread_t *poll_cb = (mm_camera_poll_thread_t *)data;
mm_camera_cmd_thread_name(poll_cb->threadName);
/* add pipe read fd into poll first */
poll_cb->poll_fds[poll_cb->num_fds++].fd = poll_cb->pfds[0];
mm_camera_poll_sig_done(poll_cb);
mm_camera_poll_set_state(poll_cb, MM_CAMERA_POLL_TASK_STATE_POLL);
return mm_camera_poll_fn(poll_cb);
}
<=================================>
return rc;
}
pthread_create會創(chuàng)建一個data_poll線程,該線程最終會調(diào)用mm_camera_poll_fn蒿涎。
static void *mm_camera_poll_fn(mm_camera_poll_thread_t *poll_cb)
{
int rc = 0, i;
···
do {
for(i = 0; i < poll_cb->num_fds; i++) {
poll_cb->poll_fds[i].events = POLLIN|POLLRDNORM|POLLPRI;
}
//輪詢監(jiān)聽fd
rc = poll(poll_cb->poll_fds, poll_cb->num_fds, poll_cb->timeoutms);
if(rc > 0) {
if ((poll_cb->poll_fds[0].revents & POLLIN) &&
(poll_cb->poll_fds[0].revents & POLLRDNORM)) {
//管道里面的消息
/* if we have data on pipe, we only process pipe in this iteration */
LOGD("cmd received on pipe\n");
mm_camera_poll_proc_pipe(poll_cb);
} else {//event 回調(diào)
for(i=1; i<poll_cb->num_fds; i++) {
/* Checking for ctrl events -event 回調(diào)*/
if ((poll_cb->poll_type == MM_CAMERA_POLL_TYPE_EVT) &&
(poll_cb->poll_fds[i].revents & POLLPRI)) {
LOGD("mm_camera_evt_notify\n");
if (NULL != poll_cb->poll_entries[i-1].notify_cb) {
poll_cb->poll_entries[i-1].notify_cb(poll_cb->poll_entries[i-1].user_data);
}
}
/*data 回調(diào)*/
if ((MM_CAMERA_POLL_TYPE_DATA == poll_cb->poll_type) &&
(poll_cb->poll_fds[i].revents & POLLIN) &&
(poll_cb->poll_fds[i].revents & POLLRDNORM)) {
LOGD("mm_stream_data_notify\n");
if (NULL != poll_cb->poll_entries[i-1].notify_cb) {
poll_cb->poll_entries[i-1].notify_cb(poll_cb->poll_entries[i-1].user_data);
}
}
}
}
} else {
/* in error case sleep 10 us and then continue. hard coded here */
usleep(10);
continue;
}
} while ((poll_cb != NULL) && (poll_cb->state == MM_CAMERA_POLL_TASK_STATE_POLL));
return NULL;
}
- 死循環(huán)- 輪詢(poll)的方式監(jiān)聽fd哀托,這里只監(jiān)聽了pipe的poll_cb->pfds[0],后續(xù)data和event都會監(jiān)聽
- 如果管道有消息劳秋,則回調(diào)mm_camera_poll_proc_pipe
- 如果有event事件仓手,調(diào)用相關(guān)回調(diào)mm_camera_evt_notify
- 如果有data數(shù)據(jù),調(diào)用相關(guān)回調(diào)mm_stream_data_notify
這里是通過poll_cb->poll_entries[i-1].notify_cb(poll_cb->poll_entries[i-1].user_data調(diào)用回調(diào)函數(shù)mm_stream_data_notify
mm_stream_data_notify是在哪注冊的玻淑?
hardware/qcom/camera/QCamera2/stack/mm-camera-interface/src/mm_camera_stream.c
int32_t mm_stream_qbuf(mm_stream_t *my_obj, mm_camera_buf_def_t *buf)
{
if (1 == my_obj->queued_buffer_count) {
uint8_t idx = mm_camera_util_get_index_by_num(
my_obj->ch_obj->cam_obj->my_num, my_obj->my_hdl);
/* Add fd to data poll thread */
LOGH("Add Poll FD %p type: %d idx = %d num = %d fd = %d",···);
rc = mm_camera_poll_thread_add_poll_fd(&my_obj->ch_obj->poll_thread[0],
idx, my_obj->my_hdl, my_obj->fd, mm_stream_data_notify,
(void*)my_obj, mm_camera_async_call);
}
}
在mm_stream_qbuf調(diào)用函數(shù)mm_camera_poll_thread_add_poll_fd函數(shù) 進行注冊
hardware/qcom/camera/QCamera2/stack/mm-camera-interface/src/mm_camera_thread.c
int32_t mm_camera_poll_thread_add_poll_fd(···)
{
int32_t rc = -1;
if (MAX_STREAM_NUM_IN_BUNDLE > idx) {
poll_cb->poll_entries[idx].fd = fd;//添加Poll FD
poll_cb->poll_entries[idx].handler = handler;
poll_cb->poll_entries[idx].notify_cb = notify_cb;//注冊回調(diào)函數(shù)
poll_cb->poll_entries[idx].user_data = userdata;//data數(shù)據(jù)
/* send poll entries updated signal to poll thread */
if (call_type == mm_camera_sync_call ) {
rc = mm_camera_poll_sig(poll_cb, MM_CAMERA_PIPE_CMD_POLL_ENTRIES_UPDATED);
} else {
rc = mm_camera_poll_sig_async(poll_cb,MM_CAMERA_PIPE_CMD_POLL_ENTRIES_UPDATED_ASYNC );
}
} else {
LOGE("invalid handler %d (%d)", handler, idx);
}
return rc;
}
該函數(shù)的作用如下:
- 添加Poll FD
- 注冊回調(diào)函數(shù)
- 調(diào)用mm_camera_poll_sig_async往管道里面寫cmd嗽冒,其實就去更新poll_fds
static int32_t mm_camera_poll_sig_async(mm_camera_poll_thread_t *poll_cb,
uint32_t cmd)
{
/* send through pipe */
cmd_evt.cmd = cmd;
/* reset the statue to false */
poll_cb->status = FALSE;
/* send cmd to worker */
ssize_t len = write(poll_cb->pfds[1], &cmd_evt, sizeof(cmd_evt));
}
一旦往poll_cb->pfds[1]寫數(shù)據(jù),
那么poll(poll_cb->poll_fds, poll_cb->num_fds, poll_cb->timeoutms)就會監(jiān)聽到fd變化
管道有變化补履,就會調(diào)用 mm_camera_poll_proc_pipe(poll_cb);
static void mm_camera_poll_proc_pipe(mm_camera_poll_thread_t *poll_cb)
{
···
if (MM_CAMERA_POLL_TYPE_DATA == poll_cb->poll_type &&
poll_cb->num_fds <= MAX_STREAM_NUM_IN_BUNDLE) {
for(i = 0; i < MAX_STREAM_NUM_IN_BUNDLE; i++) {
if(poll_cb->poll_entries[i].fd >= 0) {
/* fd is valid, we update poll_fds to this fd */
poll_cb->poll_fds[poll_cb->num_fds].fd = poll_cb->poll_entries[i].fd;
poll_cb->poll_fds[poll_cb->num_fds].events = POLLIN|POLLRDNORM|POLLPRI;
poll_cb->num_fds++;
}
}
}
···
}
這里其實就是更新poll_cb->poll_fds添坊,這樣就能輪詢data數(shù)據(jù)了,有數(shù)據(jù)就會調(diào)用相關(guān)回調(diào)箫锤。
poll(poll_cb->poll_fds, poll_cb->num_fds, poll_cb->timeoutms)
mm_stream_data_notify做了什么贬蛙?
static void mm_stream_data_notify(void* user_data)
{
rc = mm_stream_read_msm_frame(my_obj, &buf_info,(uint8_t)length);
}
int32_t mm_stream_read_msm_frame(mm_stream_t * my_obj,
mm_camera_buf_info_t* buf_info,
uint8_t num_planes)
{
//調(diào)用VIDIOC_DQBUF從kernel中把buffer取出來
rc = ioctl(my_obj->fd, VIDIOC_DQBUF, &vb);
if(buf_info->buf->buf_type == CAM_STREAM_BUF_TYPE_USERPTR) {
//把取出來buffer數(shù)據(jù)填充到stream_buf
mm_stream_read_user_buf(my_obj, buf_info);
}
mm_stream_handle_rcvd_buf(my_obj, &buf_info, has_cb);
}
- 調(diào)用VIDIOC_DQBUF從kernel中把buffer取出來。
- mm_stream_handle_rcvd_buf :把取出來buffer數(shù)據(jù)填充到stream_buf
- mm_stream_handle_rcvd_buf(my_obj, &buf_info, has_cb)喚醒cmd線程
super_buf的回調(diào)處理
如果stream綁定到channel里谚攒,就會調(diào)用2個回調(diào)函數(shù)阳准,這里篇幅問題,就不展開分析了馏臭。
- mm_channel_dispatch_super_buf
把super buffer傳給注冊的用戶 - mm_channel_process_stream_buf
處理super buffer野蝇。
3.2.2 CAM_StrmAppData線程的創(chuàng)建
hardware/qcom/camera/QCamera2/stack/mm-camera-interface/src/mm_camera_channel.c
int32_t mm_channel_start(mm_channel_t *my_obj)
{
/* start stream */
rc = mm_stream_fsm_fn(s_objs[i], MM_STREAM_EVT_START,NULL, NULL);
}
在mm_channel_start的時候,通過MM_STREAM_EVT_START命令去啟動流
hardware/qcom/camera/QCamera2/stack/mm-camera-interface/src/mm_camera_stream.c
int32_t mm_stream_fsm_reg(mm_stream_t * my_obj,
mm_stream_evt_type_t evt,
void * in_val,
void * out_val)
{
switch(evt) {
case MM_STREAM_EVT_START:
{
uint8_t has_cb = 0;
uint8_t i;
/* launch cmd thread if CB is not null */
pthread_mutex_lock(&my_obj->cb_lock);
for (i = 0; i < MM_CAMERA_STREAM_BUF_CB_MAX; i++) {
if((NULL != my_obj->buf_cb[i].cb) &&
(my_obj->buf_cb[i].cb_type != MM_CAMERA_STREAM_CB_TYPE_SYNC)) {
has_cb = 1;
break;
}
}
pthread_mutex_unlock(&my_obj->cb_lock);
pthread_mutex_lock(&my_obj->cmd_lock);
if (has_cb) {
snprintf(my_obj->cmd_thread.threadName, THREAD_NAME_SIZE, "CAM_StrmAppData");
//創(chuàng)建CAM_StrmAppData線程
mm_camera_cmd_thread_launch(&my_obj->cmd_thread,
mm_stream_dispatch_app_data,
(void *)my_obj);
}
pthread_mutex_unlock(&my_obj->cmd_lock);
//設(shè)置stream狀態(tài)
my_obj->state = MM_STREAM_STATE_ACTIVE;
//啟動stream
rc = mm_stream_streamon(my_obj);
}
break;
}
- 判斷stream是否有callback函數(shù)
- 如果有位喂,就創(chuàng)建CAM_StrmAppData線程浪耘,回調(diào)函數(shù)為:mm_stream_dispatch_app_data
mm_stream_dispatch_app_data做了什么乱灵?
- 調(diào)用hal層的stream注冊的回調(diào)函數(shù)
/* callback */
my_obj->buf_cb[i].cb(&super_buf,my_obj->buf_cb[i].user_data);
- 釋放buf:mm_stream_buf_done(my_obj, buf_info->buf);
到這里塑崖,我們可能會產(chǎn)生一個疑問,my_obj->buf_cb[i].cb指的是什么痛倚?
my_obj->buf_cb[i].cb在哪里進行了注冊规婆?
void QCameraStream::dataNotifyCB(mm_camera_super_buf_t *recvd_frame,
void *userdata)
{
LOGD("\n");
QCameraStream* stream = (QCameraStream *)userdata;
···
*frame = *recvd_frame;
stream->processDataNotify(frame);
return;
}
int32_t QCameraStream::processDataNotify(mm_camera_super_buf_t *frame)
{
//frame入隊
if (mDataQ.enqueue((void *)frame)) {
//發(fā)送一條cmd = CAMERA_CMD_TYPE_DO_NEXT_JOB
return mProcTh.sendCmd(CAMERA_CMD_TYPE_DO_NEXT_JOB, FALSE, FALSE);
}
}
- frame入隊
- 發(fā)送一條cmd = CAMERA_CMD_TYPE_DO_NEXT_JOB,并且喚醒cmd線程
分析到這里蝉稳,我們會有疑問抒蚜,cmd線程在哪里創(chuàng)建的,又是怎么處理的耘戚?
接下來看到hal層
3.3 Hal層
CAM_strmDatProc線程的創(chuàng)建
hardware/qcom/camera/QCamera2/HAL/QCameraStream.cpp
int32_t QCameraStream::start()
{
int32_t rc = 0;
mDataQ.init();
//創(chuàng)建一個回調(diào)函數(shù)為dataProcRoutine的線程: CAM_strmDatProc
rc = mProcTh.launch(dataProcRoutine, this);
<=====================>
int32_t QCameraCmdThread::launch(···)
{
/* launch the thread */
pthread_create(&cmd_pid,
NULL,
start_routine,
user_data);
return NO_ERROR;
}
<=====================>
···
return rc;
}
- 創(chuàng)建一個回調(diào)函數(shù)為dataProcRoutine的線程: CAM_strmDatProc嗡髓,
- 該線程的作用:回調(diào)hal層注冊的函數(shù)
void *QCameraStream::dataProcRoutine(void *data)
{
int running = 1;
int ret;
QCameraStream *pme = (QCameraStream *)data;
QCameraCmdThread *cmdThread = &pme->mProcTh;
cmdThread->setName("CAM_strmDatProc");
do {
do {
//線程等待
ret = cam_sem_wait(&cmdThread->cmd_sem);
if (ret != 0 && errno != EINVAL) {
LOGE("cam_sem_wait error (%s)",
strerror(errno));
return NULL;
}
} while (ret != 0);
// we got notified about new cmd avail in cmd queue
camera_cmd_type_t cmd = cmdThread->getCmd();
switch (cmd) {
case CAMERA_CMD_TYPE_DO_NEXT_JOB:
{
LOGD("Do next job");
mm_camera_super_buf_t *frame =
(mm_camera_super_buf_t *)pme->mDataQ.dequeue();
if (NULL != frame) {
if (pme->mDataCB != NULL) {
pme->mDataCB(frame, pme, pme->mUserData);
} else {
// no data cb routine, return buf here
pme->bufDone(frame);
free(frame);
}
}
}
break;
···
}
} while (running);
LOGH("X");
return NULL;
}
- 線程阻塞 cam_sem_wait(&cmdThread->cmd_sem);,等待被喚醒才能繼續(xù)執(zhí)行
- frame出隊:mm_camera_super_buf_t *frame = (mm_camera_super_buf_t *)pme->mDataQ.dequeue()
- 獲取cmd:cmdThread->getCmd()
- 如果收到 cmd = CAMERA_CMD_TYPE_DO_NEXT_JOB收津,調(diào)用pme->mDataCB
CAM_strmDatProc是在哪里被喚醒的饿这?
int32_t QCameraStream::processDataNotify(mm_camera_super_buf_t *frame)
{
//frame入隊
if (mDataQ.enqueue((void *)frame)) {
//發(fā)送一條cmd = CAMERA_CMD_TYPE_DO_NEXT_JOB
return mProcTh.sendCmd(CAMERA_CMD_TYPE_DO_NEXT_JOB, FALSE, FALSE);
}
}
pme->mDataCB在哪注冊的浊伙?
hardware/qcom/camera/QCamera2/HAL/QCamera2HWICallbacks.cpp
以元數(shù)據(jù)流為例子,調(diào)用流程如下:
addStreamToChannel(pChannel, CAM_STREAM_TYPE_METADATA,metadata_stream_cb_routine, this);
pChannel->addStream
pStream->init
在init函數(shù)中:
mDataCB = stream_cb;
mUserData = userdata;
- 元數(shù)據(jù)流cb = metadata_stream_cb_routine
- 預(yù)覽流cb = preview_stream_cb_routine 和 synchronous_stream_cb_routine
- zsl流cb = zsl_channel_cb
- 拍照流 cb = capture_channel_cb_routine
- video流 cb = video_stream_cb_routine
四长捧、應(yīng)用以及debug技巧
應(yīng)用1:算法集成
參考我之前寫的系列博文:
【Camera專題】HAL1- 實現(xiàn)第三方算法并集成到Android系統(tǒng)
應(yīng)用2:分析圖像異常問題
這里分享一個工作中遇到的問題:
現(xiàn)象:在視頻通話時嚣鄙,概率性某一幀圖像閃一下綠屏。
分析:這個問題單純看log是無法解決問題的串结,沒有報錯哑子。
解決:復(fù)現(xiàn)問題,在ISP或者hal層dump出相關(guān)圖像肌割,確定問題出現(xiàn)在那一層卧蜓,然后具體問題具體分析。
當(dāng)時我們dump出hal層圖像把敞,發(fā)現(xiàn)hal層圖像是ok的烦却,因為問題并非出現(xiàn)在底層,而是應(yīng)用層先巴,后來跟應(yīng)用同事溝通其爵,他們使用了OpenGl對圖像進行了處理,導(dǎo)致出現(xiàn)了此問題伸蚯!
如何dump圖像
- 圖像類型
#define QCAMERA_DUMP_FRM_PREVIEW 1
#define QCAMERA_DUMP_FRM_VIDEO (1<<1)
#define QCAMERA_DUMP_FRM_INPUT_JPEG (1<<2)
#define QCAMERA_DUMP_FRM_THUMBNAIL (1<<3)
#define QCAMERA_DUMP_FRM_RAW (1<<4)
#define QCAMERA_DUMP_FRM_OUTPUT_JPEG (1<<5)
#define QCAMERA_DUMP_FRM_INPUT_REPROCESS (1<<6)
-
1. 拍照raw
CAM_FORMAT_BAYER_MIPI_RAW_10BPP_GBRG,
CAM_FORMAT_BAYER_MIPI_RAW_10BPP_GRBG, //29
CAM_FORMAT_BAYER_MIPI_RAW_10BPP_RGGB, //30
CAM_FORMAT_BAYER_MIPI_RAW_10BPP_BGGR,
adb root
adb shell setprop persist.vendor.camera.raw_yuv 1
adb shell setprop persist.vendor.camera.snapshot_raw 1
adb shell setprop persist.vendor.camera.dumpimg 16
adb shell setprop persist.vendor.camera.raw.format 29
adb reboot
-
2. 預(yù)覽raw
524304 = 0x80010 :0x8代表raw圖
QCAMERA_DUMP_FRM_RAW : 0X10
adb root
adb remount
adb shell chmod 777 /data/vendor/camera
adb shell setprop persist.vendor.camera.raw_yuv 1
adb shell setprop persist.vendor.camera.preview_raw 1
adb shell setprop persist.vendor.camera.dumpimg 524304
adb shell setprop persist.vendor.camera.dumpimg_num 20//數(shù)量
- 3. ISP/VFE輸出的YUV圖
Preview: adb shell setprop persist.vendor.camera.isp.dump 2
Analysis: adb shell setprop persist.vendor.camera.isp.dump 2048
Snapshot: adb shell setprop persist.vendor.camera.isp.dump 8
Video: adb shell setprop persist.vendor.camera.isp.dump 16
adb shell setprop persist.vendor.camera.isp.dump_cnt 100;//dump yuv的數(shù)量
- 4. Hal輸出的YUV圖
- persist.vendor.camera.dumpimg =1 代表 preview
- persist.vendor.camera.dumpimg =2 代表 video
- persist.vendor.camera.dumpimg =3 代表 preview & video
adb root
adb shell chmod 777 /data/vendor/camera
adb shell setprop persist.vendor.camera.dumpimg 1
adb shell setprop persist.vendor.camera.dumpimg_num 100//數(shù)量
- 5.dump h264編碼前后的video
adb root
adb remount
adb shell chmod 777 /data/misc/media
adb shell setenforce 0
adb shell setprop vidc.enc.log.in 1
adb shell setprop vidc.enc.log.out 1
- Stay hungry摩渺,Stay Foolish!