1合瓢、前言
本文主要研究高通平臺(tái)Camera驅(qū)動(dòng)和HAL層代碼架構(gòu),熟悉高通Camera的控制流程杨凑。
平臺(tái):Qcom-高通平臺(tái)
Hal版本:【HAL1】
知識(shí)點(diǎn)如下:
從HAL層到driver層:研究Camera以下內(nèi)容
1.打開(open)流程
2.預(yù)覽(preview)流程
3.拍照(tackPicture)流程
2滤奈、Camera軟件架構(gòu)
由上圖可以看出,Android Camera 框架是 client/service 的架構(gòu),
-
1.有兩個(gè)進(jìn)程:
client 進(jìn)程:可以看成是 AP 端,主要包括 JAVA 代碼與一些 native c/c++代碼;
service 進(jìn) 程:”屬于服務(wù)端,是 native c/c++代碼,主要負(fù)責(zé)和 linux kernel 中的 camera driver 交互,搜集 linuxkernel 中 cameradriver 傳上來的數(shù)據(jù),并交給顯示系統(tǒng)SurfaceFlinger顯示撩满。
client 進(jìn)程與 service 進(jìn)程通過 Binder 機(jī)制通信, client 端通過調(diào)用 service 端的接口實(shí)現(xiàn)各個(gè)具體的功能蜒程。
2.最下面的是kernel層的驅(qū)動(dòng),其中按照V4L2架構(gòu)實(shí)現(xiàn)了camera sensor等驅(qū)動(dòng)鹦牛,向用戶空間提供/dev/video0節(jié)點(diǎn)搞糕,這些設(shè)備節(jié)點(diǎn)文件,把操作設(shè)備的接口暴露給用戶空間曼追。
3.在往上是HAL層窍仰,高通代碼實(shí)現(xiàn)了對(duì)/dev/video0的基本操作,對(duì)接了android的camera相關(guān)的interface礼殊。
2.1 Camera的open流程
2.1.1 Hal層
Android中Camera的調(diào)用流程, 基本是 Java -> JNI -> Service -> HAL -> 驅(qū)動(dòng)層驹吮。
frameworks/av/services/camera/libcameraservice/device1/CameraHardwareInterface.h
status_t initialize(CameraModule *module) {
···
rc = module->open(mName.string(), (hw_device_t **)&mDevice);
···
}
這里調(diào)用module->open開始調(diào)用到HAL層针史,那調(diào)用的是哪個(gè)方法呢?
我們繼續(xù)往下看:
hardware/qcom/camera/QCamera2/HAL/wrapper/QualcommCamera.cpp
static hw_module_methods_t camera_module_methods = {
open: camera_device_open,
};
實(shí)際上是調(diào)用了camera_device_open函數(shù)碟狞,為了對(duì)調(diào)用流程更加清晰的認(rèn)識(shí)啄枕,
我畫了一張流程圖(畫圖工具:processon):
open流程圖已經(jīng)很清晰明了,我們關(guān)注一些重點(diǎn)函數(shù):
在HAL層的 module->open(mName.string(), (hw_device_t **)&mDevice)層層調(diào)用族沃,最終會(huì)調(diào)用到函數(shù)mm_camera_open(cam_obj);
hardware/qcom/camera/QCamera2/HAL/core/src/QCameraHWI.cpp
QCameraHardwareInterface::QCameraHardwareInterface(int cameraId, int mode)
{
···
/* Open camera stack! */
mCameraHandle=camera_open(mCameraId, &mem_hooks);
//Preview
result = createPreview();
//Record
result = createRecord();
//Snapshot
result = createSnapshot();
/* launch jpeg notify thread and raw data proc thread */
mNotifyTh = new QCameraCmdThread();
mDataProcTh = new QCameraCmdThread();
···
}
分析:new QCameraHardwareInterface()進(jìn)行初始化:主要做了以下動(dòng)作:
- 1.打開camera
- 2.creat preview stream频祝、record stream、snapshot stream
- 3.創(chuàng)建2個(gè)線程(jpeg notify thread和raw data proc thread)
hardware/qcom/camera/QCamera2/stack/mm-camera-interface/src/mm_camera.c
int32_t mm_camera_open(mm_camera_obj_t *my_obj)
{
···
my_obj->ctrl_fd = open(dev_name, O_RDWR | O_NONBLOCK);
···
}
在V4L2框架中脆淹,Camera被看做一個(gè)視頻設(shè)備常空,使用open函數(shù)打開這個(gè)設(shè)備:這里以阻塞模式打開Camera。
1. 用非阻塞模式打開攝像頭設(shè)備
cameraFd = open("/dev/video0", O_RDWR | O_NONBLOCK);
2. 如果用阻塞模式打開攝像頭設(shè)備盖溺,上述代碼變?yōu)椋?
cameraFd = open("/dev/video0", O_RDWR);
ps:關(guān)于阻塞模式和非阻塞模式
應(yīng)用程序能夠使用阻塞模式或非阻塞模式打開視頻設(shè)備漓糙,如果使用非阻塞模式調(diào)用視頻設(shè)備,
即使尚未捕獲到信息烘嘱,驅(qū)動(dòng)依舊會(huì)把緩存(DQBUFF)里的東西返回給應(yīng)用程序昆禽。
那么,接下來就會(huì)調(diào)用到Kernel層的代碼
2.1.2Kernel層
kernel/drivers/media/platform/msm/camera_v2/msm.c
static struct v4l2_file_operations msm_fops = {
.owner = THIS_MODULE,
.open = msm_open,
.poll = msm_poll,
.release = msm_close,
.ioctl = video_ioctl2,
#ifdef CONFIG_COMPAT
.compat_ioctl32 = video_ioctl2,
#endif
};
實(shí)際上是調(diào)用了msm_open這個(gè)函數(shù)蝇庭,我們跟進(jìn)去看:
static int msm_open(struct file *filep)
{
···
/* !!! only ONE open is allowed !!! */
if (atomic_cmpxchg(&pvdev->opened, 0, 1))
return -EBUSY;
spin_lock_irqsave(&msm_pid_lock, flags);
msm_pid = get_pid(task_pid(current));
spin_unlock_irqrestore(&msm_pid_lock, flags);
/* create event queue */
rc = v4l2_fh_open(filep);
if (rc < 0)
return rc;
spin_lock_irqsave(&msm_eventq_lock, flags);
msm_eventq = filep->private_data;
spin_unlock_irqrestore(&msm_eventq_lock, flags);
/* register msm_v4l2_pm_qos_request */
msm_pm_qos_add_request();
···
}
分析:
通過調(diào)用v4l2_fh_open函數(shù)打開Camera,該函數(shù)會(huì)創(chuàng)建event隊(duì)列等進(jìn)行一些其他操作醉鳖。
接下來我們跟著log去看:
camera open log
<3>[ 12.526811] msm_camera_power_up type 1
<3>[ 12.526818] msm_camera_power_up:1303 gpio set val 33
<3>[ 12.528873] msm_camera_power_up index 6
<3>[ 12.528885] msm_camera_power_up type 1
<3>[ 12.528893] msm_camera_power_up:1303 gpio set val 33
<3>[ 12.534954] msm_camera_power_up index 7
<3>[ 12.534969] msm_camera_power_up type 1
<3>[ 12.534977] msm_camera_power_up:1303 gpio set val 28
<3>[ 12.540162] msm_camera_power_up index 8
<3>[ 12.540177] msm_camera_power_up type 1
<3>[ ·
<3>[ ·
<3>[ ·
<3>[ 12.562753] msm_sensor_match_id: read id: 0x5675 expected id 0x5675:
<3>[ 12.562763] ov5675_back probe succeeded
<3>[ 12.562771] msm_sensor_driver_create_i2c_v4l_subdev camera I2c probe succeeded
<3>[ 12.564930] msm_sensor_driver_create_i2c_v4l_subdev rc 0 session_id 1
<3>[ 12.565495] msm_sensor_driver_create_i2c_v4l_subdev:120
<3>[ 12.565507] msm_camera_power_down:1455
<3>[ 12.565514] msm_camera_power_down index 0
分析:
最終就是調(diào)用msm_camera_power_up上電,msm_sensor_match_id識(shí)別sensor id遗契,調(diào)用ov5675_back probe()探測(cè)函數(shù)去完成匹配設(shè)備和驅(qū)動(dòng)的工作辐棒,msm_camera_power_down下電病曾!
到此 我們的open流程就結(jié)束了k狗洹!泰涂!
2.2 Camera的preview流程
2.2.1 Hal層
hardware/qcom/camera/QCamera2/HAL/QCamera2HWI.cpp
int QCamera2HardwareInterface::startPreview()
{
···
int32_t rc = NO_ERROR;
···
rc = startChannel(QCAMERA_CH_TYPE_PREVIEW);
···
}
這里調(diào)用startChannel(QCAMERA_CH_TYPE_PREVIEW)鲫竞,開啟preview流。
接來下看我畫的一張流程圖:(Hal層)
關(guān)注一些重點(diǎn)函數(shù):
hardware/qcom/camera/QCamera2/HAL/QCameraChannel.cpp
int32_t QCameraChannel::start()
{
···
mStreams[i]->start();//流程1
···
rc = m_camOps->start_channel(m_camHandle, m_handle);//流程2
···
}
進(jìn)入QCameraChannel::start()函數(shù)開始執(zhí)行兩個(gè)流程逼蒙,分別是
mStreams[i]->start()和m_camOps->start_channel(m_camHandle, m_handle);
流程1:mStreams[i]->start()
1.通過mProcTh.launch(dataProcRoutine, this)開啟新線程
2.執(zhí)行CAMERA_CMD_TYPE_DO_NEXT_JOB分支从绘,
3.從mDataQ隊(duì)列中取出數(shù)據(jù)并放入mDataCB中,等待數(shù)據(jù)返回到對(duì)應(yīng)的stream回調(diào)中去,
4.最后向kernel請(qǐng)求數(shù)據(jù)是牢;
流程2:m_camOps->start_channel(m_camHandle, m_handle);
通過流程圖僵井,我們可以清晰的看到,經(jīng)過一系列復(fù)雜的調(diào)用用驳棱,
最后在mm_camera_channel.c中
調(diào)用mm_channel_start(mm_channel_t *my_obj)函數(shù)批什,
來看mm_channel_start做了什么事情:
hardware/qcom/camera/QCamera2/stack/mm-camera-interface/src/mm_camera_channel.c
int32_t mm_channel_start(mm_channel_t *my_obj)
{
···
/* 需要發(fā)送cb,因此啟動(dòng)線程 */
/* 初始化superbuf隊(duì)列 */
mm_channel_superbuf_queue_init(&my_obj->bundle.superbuf_queue);
/* 啟動(dòng)cb線程社搅,通過cb調(diào)度superbuf中 */
snprintf(my_obj->cb_thread.threadName, THREAD_NAME_SIZE, "CAM_SuperBuf");
mm_camera_cmd_thread_launch(&my_obj->cb_thread,
mm_channel_dispatch_super_buf,
(void*)my_obj);
/* 啟動(dòng) cmd 線程驻债,作為superbuf接收數(shù)據(jù)的回調(diào)函數(shù)*/
snprintf(my_obj->cmd_thread.threadName, THREAD_NAME_SIZE, "CAM_SuperBufCB");
mm_camera_cmd_thread_launch(&my_obj->cmd_thread,
mm_channel_process_stream_buf,
(void*)my_obj);
/* 為每個(gè)strean分配 buf */
/*allocate buf*/
rc = mm_stream_fsm_fn(s_objs[i],
MM_STREAM_EVT_GET_BUF,
NULL,
NULL);
/* reg buf */
rc = mm_stream_fsm_fn(s_objs[i],
MM_STREAM_EVT_REG_BUF,
NULL,
NULL);
/* 開啟 stream */
rc = mm_stream_fsm_fn(s_objs[i],
MM_STREAM_EVT_START,
NULL,
NULL);
···
}
過程包括:
- 1.創(chuàng)建cb thread乳规,cmd thread線程以及
- 2.為每個(gè)stream分配buf
- 3.開啟stream;
我們繼續(xù)關(guān)注開啟stream后的流程:
rc = mm_stream_fsm_fn(s_objs[i],MM_STREAM_EVT_START,NULL,NULL);
調(diào)用到
rc = mm_stream_fsm_reg(my_obj, evt, in_val, out_val)
hardware/qcom/camera/QCamera2/stack/mm-camera-interface/src/mm_camera_stream.c
int32_t mm_stream_fsm_reg(···)
{
···
case MM_STREAM_EVT_START:
rc = mm_stream_streamon(my_obj);
···
}
在mm_camera_stream.c中調(diào)用mm_stream_streamon(mm_stream_t *my_obj)函數(shù).
向kernel發(fā)送v4l2請(qǐng)求合呐,等待數(shù)據(jù)回調(diào)
int32_t mm_stream_streamon(mm_stream_t *my_obj)
{
···
enum v4l2_buf_type buf_type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
···
rc = ioctl(my_obj->fd, VIDIOC_STREAMON, &buf_type);
···
}
2.2.2 Kernel層
kernel/drivers/media/platform/msm/camera_v2/camera/camera.c
通過ioctl的方式暮的,經(jīng)過層層調(diào)用,最后調(diào)用到camera_v4l2_streamon();
static int camera_v4l2_streamon(struct file *filep, void *fh,
enum v4l2_buf_type buf_type)
{
struct v4l2_event event;
int rc;
struct camera_v4l2_private *sp = fh_to_private(fh);
rc = vb2_streamon(&sp->vb2_q, buf_type);
camera_pack_event(filep, MSM_CAMERA_SET_PARM,
MSM_CAMERA_PRIV_STREAM_ON, -1, &event);
rc = msm_post_event(&event, MSM_POST_EVT_TIMEOUT);
···
rc = camera_check_event_status(&event);
return rc;
}
分析:通過msm_post_event發(fā)生數(shù)據(jù)請(qǐng)求淌实,等待數(shù)據(jù)回調(diào)冻辩。
Preview完整流程圖
到此,preview預(yù)覽流程結(jié)束
2.3 Camera的takePicture流程
事實(shí)上拆祈,tackPicture流程和preview的流程很類似微猖!
以ZSL模式(零延遲模式)為切入點(diǎn):
2.3.1 Hal層
hardware/qcom/camera/QCamera2/HAL/QCamera2HWI.cpp
int QCamera2HardwareInterface::takePicture()
{
···
//流程1
mCameraHandle->ops->start_zsl_snapshot(mCameraHandle->camera_handle,
pZSLChannel->getMyHandle());
···
//流程2
rc = pZSLChannel->takePicture(numSnapshots);
···
}
進(jìn)入QCamera2HardwareInterface::takePicture后,會(huì)走2個(gè)流程:
1.mCameraHandle->ops->start_zsl_snapshot(···);
2.pZSLChannel->takePicture(numSnapshots);
流程1:
經(jīng)過層層調(diào)用缘屹,最終會(huì)調(diào)用到mm_channel_start_zsl_snapshot
hardware/qcom/camera/QCamera2/stack/mm-camera-interface/src/mm_camera_channel.c
int32_t mm_channel_start_zsl_snapshot(mm_channel_t *my_obj)
{
int32_t rc = 0;
mm_camera_cmdcb_t* node = NULL;
node = (mm_camera_cmdcb_t *)malloc(sizeof(mm_camera_cmdcb_t));
if (NULL != node) {
memset(node, 0, sizeof(mm_camera_cmdcb_t));
node->cmd_type = MM_CAMERA_CMD_TYPE_START_ZSL;
/* enqueue to cmd thread */
cam_queue_enq(&(my_obj->cmd_thread.cmd_queue), node);
/* wake up cmd thread */
cam_sem_post(&(my_obj->cmd_thread.cmd_sem));
} else {
CDBG_ERROR("%s: No memory for mm_camera_node_t", __func__);
rc = -1;
}
return rc;
}
分析:
該函數(shù)主要做了2件事情:
- 1 cam_queue_enq(&(my_obj->cmd_thread.cmd_queue), node);入隊(duì)
- 2 通過cam_sem_post(&(my_obj->cmd_thread.cmd_sem));喚醒cmd線程
這里的node->cmd_type=MM_CAMERA_CMD_TYPE_START_ZSL
hardware/qcom/camera/QCamera2/stack/mm-camera-interface/src/mm_camera_thread.c
static void *mm_camera_cmd_thread(void *data)
{
···
case MM_CAMERA_CMD_TYPE_START_ZSL:
cmd_thread->cb(node, cmd_thread->user_data);
···
}
這里cmd_thread->cb是回調(diào)函數(shù):
cmd_thread->cb = mm_channel_process_stream_buf凛剥,經(jīng)過層層復(fù)雜的回調(diào)
最終:
mm_channel_superbuf_skip(ch_obj, &ch_obj->bundle.superbuf_queue);
super_buf = (mm_channel_queue_node_t*)node->data;
將buffer 取出 且釋放list中的node,最終將buffer queue給kernel進(jìn)行下一次填充.
流程2:
同樣轻姿,經(jīng)過層層調(diào)用犁珠,最終調(diào)用到mm_channel_request_super_buf
hardware/qcom/camera/QCamera2/stack/mm-camera-interface/src/mm_camera_channel.c
int32_t mm_channel_request_super_buf(mm_channel_t *my_obj, uint32_t num_buf_requested)
{
/* set pending_cnt
* will trigger dispatching super frames if pending_cnt > 0 */
/* send cam_sem_post to wake up cmd thread to dispatch super buffer */
node = (mm_camera_cmdcb_t *)malloc(sizeof(mm_camera_cmdcb_t));
if (NULL != node) {
memset(node, 0, sizeof(mm_camera_cmdcb_t));
node->cmd_type = MM_CAMERA_CMD_TYPE_REQ_DATA_CB;
node->u.req_buf.num_buf_requested = num_buf_requested;
/* enqueue to cmd thread */
cam_queue_enq(&(my_obj->cmd_thread.cmd_queue), node);
/* wake up cmd thread */
cam_sem_post(&(my_obj->cmd_thread.cmd_sem));
} else {
CDBG_ERROR("%s: No memory for mm_camera_node_t", __func__);
rc = -1;
}
return rc;
}
分析:該函數(shù)和流程1一樣:
- 1 cam_queue_enq(&(my_obj->cmd_thread.cmd_queue), node);入隊(duì)
- 2 通過cam_sem_post(&(my_obj->cmd_thread.cmd_sem));喚醒cmd線程
static void *mm_camera_cmd_thread(void *data)
{
···
case MM_CAMERA_CMD_TYPE_START_ZSL:
case MM_CAMERA_CMD_TYPE_REQ_DATA_CB:
cmd_thread->cb(node, cmd_thread->user_data);
···
}
這里和流程1一樣,就不再贅述!
2.3.2 Kernel層
int32_t mm_camera_start_zsl_snapshot(mm_camera_obj_t *my_obj)
{
···
rc = mm_camera_util_s_ctrl(my_obj->ctrl_fd,
CAM_PRIV_START_ZSL_SNAPSHOT, &value);
···
}
int32_t mm_camera_util_s_ctrl(int32_t fd, uint32_t id, int32_t *value)
{
···
rc = ioctl(fd, VIDIOC_S_CTRL, &control);
···
}
kernel/drivers/media/v4l2-core/v4l2-subdev.c
static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg)
{
···
case VIDIOC_S_CTRL:
return v4l2_s_ctrl(vfh, vfh->ctrl_handler, arg);
···
}
通過ioctl(fd, VIDIOC_S_CTRL, &control)的方式互亮,借助V4L2框架犁享,調(diào)用到kernel層,
最終buffer queue給kernel進(jìn)行下一次填充豹休。
takePicture完整流程圖