系列文章

高通msm-V4L2-Camera驅(qū)動(dòng)淺析1-初識(shí)
高通msm-V4L2-Camera驅(qū)動(dòng)淺析2-框架詳解
高通msm-V4L2-Camera驅(qū)動(dòng)淺析3-session
高通msm-V4L2-Camera驅(qū)動(dòng)淺析4-stream
高通msm-V4L2-Camera驅(qū)動(dòng)淺析5-buffer

前言

一些推薦的文章
深入理解Android相機(jī)體系結(jié)構(gòu)之八
msm8996平臺(tái)的 camera 框架筆記
Camera Kernel 驅(qū)動(dòng)層代碼邏輯分析

上一篇文章結(jié)合V4L2框架的知識(shí)狞谱，對(duì)高通v4l2 camera驅(qū)動(dòng)進(jìn)行簡(jiǎn)單分析。
本文聊聊qcom的camera內(nèi)核框架設(shè)計(jì)和一些相關(guān)概念蛹磺。

一、Camera內(nèi)核驅(qū)動(dòng)框架

msm框架把Camera系統(tǒng)分為 msm-config 和 msm-sensor 兩部分：

• msm-config：用于創(chuàng)建v4l2_device购岗，用于管理所有的子設(shè)備嗤形，同時(shí)生成/dev/video0設(shè)備節(jié)點(diǎn)，暴露控制接口給用戶空間使用蜒谤，主要包括了session/stream的創(chuàng)建與銷毀，event事件的分發(fā)至扰，另外鳍徽，還創(chuàng)建了media_device，用于暴露枚舉接口給用戶來輪詢查找整個(gè)內(nèi)核的子設(shè)備敢课。

相關(guān)源碼：kernel/msm-4.9/drivers/media/platform/msm/camera_v2/msm.c

• msm-sensor：創(chuàng)建一個(gè)v412_device,用于管理sensor阶祭，比如sensor 的初始化，啟動(dòng)流直秆，關(guān)閉流濒募，圖像buf的管理等，同時(shí)創(chuàng)建video1/video2設(shè)備節(jié)點(diǎn)暴露接口給用戶空間進(jìn)行訪問圾结。

相關(guān)源碼：kernel/msm-4.9/drivers/media/platform/msm/camera_v2/camera/camera.c

ps:我當(dāng)前的機(jī)器有前攝和后攝瑰剃，因此就有2個(gè)節(jié)點(diǎn)：video1，video2筝野。

框架源碼路徑：kernel/msm-4.9/drivers/media/platform/msm/camera_v2/

camera/      fd/          ispif/       jpeg_dma/    Makefile     msm.c        msm_sd.h     pproc/       
common/      isp/         jpeg_10/     Kconfig      msm_buf_mgr/ msm.h        msm_vb2/     sensor/

我們主要關(guān)注以下模塊：

• msm.c： 所有qcom camera相關(guān)設(shè)備的管理,提供session晌姚、stream粤剧、event、vb2_queue等接口的創(chuàng)建和刪除
• camera/：用于控制sensor挥唠，提供了ioctl_ops抵恋，如啟動(dòng)流，qbuf和dqbuf宝磨，subscribe_event等
• sensor/：具體的硬件驅(qū)動(dòng)弧关，例如sensor、eeprom懊烤、actuator梯醒、ois等

其他的模塊，如fd-人臉識(shí)別相關(guān)腌紧，jpeg編碼相關(guān)等，未來有機(jī)會(huì)再分析畜隶！

二壁肋、msm.c

kernel/msm-4.9/drivers/media/platform/msm/camera_v2/msm.c

2.1 開機(jī)時(shí)加載msm模塊

static int __init msm_init(void)
{
    return platform_driver_register(&msm_driver);
}

static void __exit msm_exit(void)
{
    platform_driver_unregister(&msm_driver);
}
module_init(msm_init);
module_exit(msm_exit);
MODULE_DESCRIPTION("MSM V4L2 Camera");
MODULE_LICENSE("GPL v2");

開機(jī)時(shí)，會(huì)加載msm模塊籽慢，內(nèi)核匹配設(shè)備和驅(qū)動(dòng)浸遗，當(dāng)設(shè)備和驅(qū)動(dòng)匹配時(shí)，就會(huì)調(diào)用msm_probe函數(shù)箱亿。

2.2 msm_probe

static int msm_probe(struct platform_device *pdev)
{
···省略部分源碼
    //初始化media_device
    media_device_init(msm_v4l2_dev->mdev);
    strlcpy(msm_v4l2_dev->mdev->model, "msm-config",sizeof(msm_v4l2_dev->mdev->model));    
     //注冊(cè)一個(gè)媒體設(shè)備,其名稱(model)為:msm-config
    rc = media_device_register(msm_v4l2_dev->mdev);
    //注冊(cè)一個(gè)v4l2_device實(shí)例
    rc = v4l2_device_register(&(pdev->dev), pvdev->vdev->v4l2_dev);
    //綁定fops
    pvdev->vdev->fops     = &msm_fops;
    //綁定ioctl_ops
    pvdev->vdev->ioctl_ops = &g_msm_ioctl_ops;
    //v4l2_device實(shí)例名稱為"msm-config"
    strlcpy(pvdev->vdev->name, "msm-config", sizeof(pvdev->vdev->name));
   //注冊(cè)一個(gè)視頻設(shè)備
    rc = video_register_device(pvdev->vdev,VFL_TYPE_GRABBER, -1);
    //初始化一個(gè)會(huì)話隊(duì)列
    msm_init_queue(msm_session_q);
}

• 注冊(cè)了一個(gè)媒體設(shè)備跛锌，創(chuàng)建/dev/media0設(shè)備節(jié)點(diǎn)，這里我們注冊(cè)的媒體設(shè)備名稱(model)為：msm_config届惋。
  上層可以通過IOCTL指令：MEDIA_IOC_ENUM_ENTITIES髓帽，枚舉所有的media設(shè)備，然后根據(jù)設(shè)備名稱msm_config找到的/dev/video0節(jié)點(diǎn)脑豹。
  

• 注冊(cè)了v4l2_device實(shí)例郑藏，該結(jié)構(gòu)體里面有一個(gè)鏈表：struct list_head subdevs，所有的子設(shè)備都會(huì)添加都該subdevs鏈表里面瘩欺，方便維護(hù)必盖。

• 注冊(cè)一個(gè)視頻設(shè)備，創(chuàng)建/dev/video0設(shè)備節(jié)點(diǎn)俱饿，給用戶空間使用歌粥，其本質(zhì)上是一個(gè)字符設(shè)備，提供fops和ioctl_fops給上層使用拍埠。
  

• 初始化一個(gè)session隊(duì)列失驶，所有的session都添加到這個(gè)隊(duì)列中，方便管理維護(hù)械拍。

2.3 提供fops給用戶空間使用

static struct v4l2_file_operations msm_fops = {
    .owner  = THIS_MODULE,
    .open   = msm_open,
    .poll   = msm_poll,
    .release = msm_close,
    .unlocked_ioctl   = video_ioctl2,
#ifdef CONFIG_COMPAT
    .compat_ioctl32 = video_ioctl2,
#endif
};

static const struct v4l2_ioctl_ops g_msm_ioctl_ops = {
    .vidioc_subscribe_event = msm_subscribe_event,
    .vidioc_unsubscribe_event = msm_unsubscribe_event,
    .vidioc_default = msm_private_ioctl,
};

v4l2_file_operations：

• msm_open：創(chuàng)建event隊(duì)列
• msm_poll：輪詢event隊(duì)列
• msm_close：釋放相關(guān)資源

v4l2_ioctl_ops：

• msm_subscribe_event：訂閱事件
• msm_unsubscribe_event：注銷事件

2.4 msm_open

在msm_probe中突勇，注冊(cè)了一個(gè)媒體設(shè)備/dev/media0,
上層可以通過IOCTL指令：MEDIA_IOC_ENUM_ENTITIES装盯，枚舉所有的media設(shè)備，
然后根據(jù)設(shè)備名稱msm_config找到的/dev/video0節(jié)點(diǎn)甲馋。
具體邏輯代碼如下：

vendor/qcom/proprietary/mm-camera/mm-camera2/media-controller/mct_shim_layer/mct_shim_layer.c

int mct_shimlayer_process_module_init(mm_camera_shim_ops_t
  *shim_ops_tbl)
{
    //該函數(shù)通過MEDIA_IOC_ENUM_ENTITIES指令枚舉所有媒體設(shè)備埂奈，
    //然后通過媒體設(shè)備名稱msm_config找到/dev/video0節(jié)點(diǎn)
    get_config_node_name(config_node_name);

    snprintf(dev_name, sizeof(dev_name), "/dev/%s", config_node_name);
    //用戶空間調(diào)用了open方法，dev_name = "/dev/video0"
    config_fd = open(dev_name, O_RDWR | O_NONBLOCK);
}

用戶空間調(diào)用open(dev_name, O_RDWR | O_NONBLOCK)打開設(shè)備節(jié)點(diǎn)/dev/video0,通過VFS文件系統(tǒng)定躏，最終會(huì)調(diào)用到msm_open函數(shù)账磺。

static int msm_open(struct file *filep)
{
    int rc = -1;

    /* create event queue */
    rc = v4l2_fh_open(filep);
<=========================>
        int v4l2_fh_open(struct file *filp)
        {
        struct video_device *vdev = video_devdata(filp);
        struct v4l2_fh *fh = kzalloc(sizeof(*fh), GFP_KERNEL);

        filp->private_data = fh;
        if (fh == NULL)
            return -ENOMEM;
        v4l2_fh_init(fh, vdev);//初始化v4l2_fh
        v4l2_fh_add(fh);//添加v4l2_fh到video_device
        return 0;
        }
<=========================>

    /* register msm_v4l2_pm_qos_request */
    msm_pm_qos_add_request();

    return rc;
}

msm_open中，主要完成了v4l2_fh的初始化工作痊远，后續(xù)所有的event都由v4l2_fh管理垮抗。

v4l2_fh結(jié)構(gòu)體是用來描述event隊(duì)列,通常在打開設(shè)備節(jié)點(diǎn)時(shí)進(jìn)行v4l2_fh注冊(cè)。

v4l2_fh_init(fh, vdev)：初始化v4l2_fh碧聪，添加v4l2_ctrl_handler到v4l2_fh：fh->ctrl_handler = vdev->ctrl_handler;

v4l2_fh_add(fh)：添加v4l2_fh到video_device冒版，方便核心層調(diào)用到。

關(guān)于v4l2_fh逞姿，可以參考以下文章：
linux v4l2學(xué)習(xí)之-消息機(jī)制

2.5 流的管理

• msm_create_stream：創(chuàng)建流
• msm_delete_stream：刪除流
• msm_get_stream ：獲取流
• __msm_queue_find_stream：查找流

當(dāng)然還有其他流的操作辞嗡，就不全部一一列出來了。

三滞造、camera.c分析

kernel/msm-4.9/drivers/media/platform/msm/camera_v2/camera/camera.c

3.1 sensor_probe

vendor/qcom/proprietary/mm-camera/mm-camera2/media-controller/modules/sensors/module/sensor_init.c

static boolean sensor_probe(···)
{
    struct sensor_init_cfg_data          cfg;
  /* Pass slave information to kernel and probe */
  memset(&cfg, 0, sizeof(cfg));
  cfg.cfgtype = CFG_SINIT_PROBE;
  cfg.cfg.setting = slave_info;
  if (ioctl(fd, VIDIOC_MSM_SENSOR_INIT_CFG, &cfg) < 0) {
    SINFO("[%s]CFG_SINIT_PROBE failed",sensor_name);
    ret = FALSE;
    goto ERROR;
  }
}

• 通過IOCTL命令：VIDIOC_MSM_SENSOR_INIT_CFG续室，調(diào)用到msm_sensor_driver_cmd(s_init, arg)，
• 通過CFG_SINIT_PROBE調(diào)用到msm_sensor_driver_probe(···）谒养；
• msm_sensor_driver_probe函數(shù)會(huì)調(diào)用camera_init_v4l2進(jìn)行初始化

3.2 camera_init_v4l2

int camera_init_v4l2(struct device *dev, unsigned int *session)
{
    struct msm_video_device *pvdev;
    struct v4l2_device *v4l2_dev;
    int rc = 0;
···省略部分源碼
#if defined(CONFIG_MEDIA_CONTROLLER)
    //初始化media_device
    media_device_init(v4l2_dev->mdev);
    strlcpy(v4l2_dev->mdev->model, "msm_camera",
            sizeof(v4l2_dev->mdev->model));

    v4l2_dev->mdev->dev = dev;
    //注冊(cè)一個(gè)媒體設(shè)備,其名稱(model)為:msm_camera
    rc = media_device_register(v4l2_dev->mdev);
    rc = media_entity_pads_init(&pvdev->vdev->entity, 0, NULL);
#endif

    pvdev->vdev->v4l2_dev = v4l2_dev;
    //注冊(cè)一個(gè)v4l2_device實(shí)例
    rc = v4l2_device_register(dev, pvdev->vdev->v4l2_dev);
    strlcpy(pvdev->vdev->name, "msm-sensor", sizeof(pvdev->vdev->name));

    //綁定fops
    pvdev->vdev->fops     = &camera_v4l2_fops;
    //綁定ioctl_ops
    pvdev->vdev->ioctl_ops = &camera_v4l2_ioctl_ops;
    //設(shè)置視頻設(shè)備的能力
    pvdev->vdev->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING;
    //注冊(cè)一個(gè)視頻設(shè)備挺狰，名稱為:msm-sensor
    rc = video_register_device(pvdev->vdev,VFL_TYPE_GRABBER, -1);
    return rc;
}

• 注冊(cè)一個(gè)媒體設(shè)備,生成節(jié)點(diǎn)/dev/media1,其名稱(model)為:msm-camera
  

• 注冊(cè)一個(gè)v4l2_device實(shí)例

• 綁定fops和ioctl_ops

• 注冊(cè)一個(gè)視頻設(shè)備，生成節(jié)點(diǎn)/dev/video1,其名稱為:msm-sensor
  

3.3 提供fops給用戶空間使用

static struct v4l2_file_operations camera_v4l2_fops = {
    .owner   = THIS_MODULE,
    .open   = camera_v4l2_open,
    .poll   = camera_v4l2_poll,
    .release = camera_v4l2_close,
    .unlocked_ioctl   = video_ioctl2,
#ifdef CONFIG_COMPAT
    .compat_ioctl32 = camera_v4l2_compat_ioctl,
#endif
};

static const struct v4l2_ioctl_ops camera_v4l2_ioctl_ops = {
    .vidioc_querycap = camera_v4l2_querycap,
    .vidioc_s_crop = camera_v4l2_s_crop,
    .vidioc_g_crop = camera_v4l2_g_crop,
    .vidioc_queryctrl = camera_v4l2_queryctrl,
    .vidioc_g_ctrl = camera_v4l2_g_ctrl,
    .vidioc_s_ctrl = camera_v4l2_s_ctrl,
    .vidioc_reqbufs = camera_v4l2_reqbufs,
    .vidioc_querybuf = camera_v4l2_querybuf,
    .vidioc_qbuf = camera_v4l2_qbuf,
    .vidioc_dqbuf = camera_v4l2_dqbuf,
    .vidioc_streamon =  camera_v4l2_streamon,
    .vidioc_streamoff = camera_v4l2_streamoff,
    .vidioc_g_fmt_vid_cap_mplane = camera_v4l2_g_fmt_vid_cap_mplane,
    .vidioc_s_fmt_vid_cap_mplane = camera_v4l2_s_fmt_vid_cap_mplane,
    .vidioc_try_fmt_vid_cap_mplane = camera_v4l2_try_fmt_vid_cap_mplane,

    /* Stream type-dependent parameter ioctls */
    .vidioc_g_parm = camera_v4l2_g_parm,
    .vidioc_s_parm = camera_v4l2_s_parm,

    /* event subscribe/unsubscribe */
    .vidioc_subscribe_event = camera_v4l2_subscribe_event,
    .vidioc_unsubscribe_event = camera_v4l2_unsubscribe_event,
    .vidioc_default = camera_v4l2_vidioc_private_ioctl,
};

3.4 會(huì)話的創(chuàng)建

應(yīng)用會(huì)調(diào)用my_obj->ctrl_fd = open(dev_name, O_RDWR | O_NONBLOCK);
通過VFS文件系統(tǒng)买窟，最終會(huì)調(diào)用到camera_v4l2_open丰泊。

static int camera_v4l2_open(struct file *filep)
{
...省略部分源碼
    //v4l2_fh的初始化工作，后續(xù)所有的event都由v4l2_fh管理
    camera_v4l2_fh_open(filep);
    /* every stream has a vb2 queue */
    //初始化vbuffer2_queue
    rc = camera_v4l2_vb2_q_init(filep)
    if (!atomic_read(&pvdev->opened)) {
        /* create a new session when first opened */
        //創(chuàng)建一個(gè)會(huì)話蔑祟，添加到會(huì)話隊(duì)列中
        rc = msm_create_session(pvdev->vdev->num, pvdev->vdev);
    //創(chuàng)建一個(gè)cmd隊(duì)列
        rc = msm_create_command_ack_q(pvdev->vdev->num,
            find_first_zero_bit((const unsigned long *)&opn_idx,
                MSM_CAMERA_STREAM_CNT_BITS));
    }
}

• 初始化v4l2_fh趁耗，所有的event都由v4l2_fh管理
• 始化vbuffer2_queue
• 創(chuàng)建一個(gè)會(huì)話，添加到會(huì)話隊(duì)列中
• 創(chuàng)建一個(gè)cmd隊(duì)列

3.5 stream的創(chuàng)建

static int camera_v4l2_s_parm(struct file *filep, void *fh,
    struct v4l2_streamparm *parm)
{
    int rc = 0;
    struct v4l2_event event;
  ···省略部分源碼
    rc = msm_create_stream(event_data->session_id,event_data->stream_id, &sp->vb2_q);

    return rc;
}

Stay Hungry疆虚，Stay Foolish!