系列文章

高通msm-V4L2-Camera驅(qū)動淺析1-初識
高通msm-V4L2-Camera驅(qū)動淺析2-框架詳解
高通msm-V4L2-Camera驅(qū)動淺析3-session
高通msm-V4L2-Camera驅(qū)動淺析4-stream
高通msm-V4L2-Camera驅(qū)動淺析5-buffer

一颈墅、stream(流)的設(shè)計(jì)思路

為什么要使用stream這個設(shè)計(jì)呢恤筛？
如果你做為開發(fā)者，現(xiàn)在你有個任務(wù)：假設(shè)camera像素為500W毒坛，你需要把每一幀500w的圖像傳輸給用戶使用，怎么辦屯伞？

• 方法1：通過【幀IO】訪問方式
  使用read和write的方式豪直，通過read讀取每一幀數(shù)據(jù)，數(shù)據(jù)需要在內(nèi)核和用戶之間拷貝末融。但問題是：這種方式訪問速度可能會非常慢暇韧。
  一幀500W的數(shù)據(jù)量，一般場景下懈玻，我們的幀率都至少有20多幀涂乌，這種訪問速度肯定不能接受。

  那怎么辦呢骂倘？可不可以這樣巴席，我在內(nèi)核申請一片內(nèi)存(buf),然后把它映射(mmap)到用戶空間，這樣用戶可以像直接讀內(nèi)存一樣荧库，訪問到我們內(nèi)核空間的數(shù)據(jù)赵刑。

• 方法2：通過【流IO】訪問方式：

  • 內(nèi)存映射緩沖區(qū)(V4L2_MEMORY_MMAP)：在內(nèi)核空間開辟緩沖區(qū)，應(yīng)用通過mmap()系統(tǒng)調(diào)用映射到用戶地址空間
  • 用戶空間緩沖區(qū)(V4L2_MEMORY_USERPTR)：在用戶空間的應(yīng)用中開辟緩沖區(qū)蚪战，用戶與內(nèi)核空間之間交換緩沖區(qū)指針。

因此stream(流)的概念就誕生了!
stream又涉及到圖像的vb2_buffer瞎疼，vb2_buffer又由vb2_queue 隊(duì)列管理

關(guān)于vb2_buffer與vb2_queue在下一篇文章在進(jìn)行討論壁畸。

二、stream(流)的創(chuàng)建

2.1 stream(流)數(shù)據(jù)結(jié)構(gòu)

struct msm_stream {
    struct list_head list;//鏈表頭

    /* stream index per session, same
     * as stream_id but set through s_parm
     */
    unsigned int stream_id;//流id
    /* vb2 buffer handling */
    struct vb2_queue *vb2_q;//buf隊(duì)列
    spinlock_t stream_lock;
    struct list_head queued_list;
};

2.2 session太抓、stream令杈、vb2_buffer的關(guān)系

上一篇文章里面有講到，session中包含了stream的隊(duì)列：

struct msm_session {
    struct msm_queue_head stream_q;
}

stream里面又嵌入了struct vb2_queue *vb2_q隊(duì)列;
而vb2_queue里面又包含用來保存和傳輸圖像的buf——struct vb2_buffer *bufs[VB2_MAX_FRAME];

• session管理stream
• steam管理vb2_queue
• vb2_queue管理vb2_buffer

2.3 流的創(chuàng)建

int msm_create_stream(unsigned int session_id,
    unsigned int stream_id, struct vb2_queue *q)
{
    struct msm_session *session;
    struct msm_stream  *stream;
    //根據(jù)session_id判斷會話是否已經(jīng)創(chuàng)建
    session = msm_queue_find(msm_session_q, struct msm_session,
        list, __msm_queue_find_session, &session_id);
    if (!session)
        return -EINVAL;
    //申請內(nèi)存
    stream = kzalloc(sizeof(*stream), GFP_KERNEL);
    if (!stream)
        return -ENOMEM;
    //賦值stream_id
    stream->stream_id = stream_id;
    //賦值vb2_queue隊(duì)列
    stream->vb2_q = q;
    //初始化自旋鎖
    spin_lock_init(&stream->stream_lock);
    //將新建立的流加入session管理的stream隊(duì)列中
    msm_enqueue(&session->stream_q, &stream->list);
    //流的隊(duì)列長度+1
    session->stream_q.len++;

    INIT_LIST_HEAD(&stream->queued_list);

    return 0;

流包括很多悔常，比如預(yù)覽流给赞，拍照流片迅，元數(shù)據(jù)流等待，這些流都由統(tǒng)一的stream_q隊(duì)列管理柑蛇，每創(chuàng)建一個流，
就會入隊(duì)空免。

2.4 流的創(chuàng)建流程-從hal到kernel

細(xì)致的調(diào)用流程盆耽，參考【Camera專題】HAL層-addChannel和startChannel簡析
這里寫一下關(guān)鍵的接口

用戶空間：HAL層
hardware/qcom/camera/QCamera2/HAL/QCamera2HWI.cpp

int32_t QCamera2HardwareInterface::addPreviewChannel()
{
`
    CDBG_HIGH("%s :zcf camera_handle=%d ops=%p", __func__,mCameraHandle->camera_handle,mCameraHandle->ops);
    pChannel = new QCameraChannel(mCameraHandle->camera_handle,
                                  mCameraHandle->ops);
···

    // preview only channel, don't need bundle attr and cb
    rc = pChannel->init(NULL, NULL, NULL);
··· 

    // meta data stream always coexists with preview if applicable
    rc = addStreamToChannel(pChannel, CAM_STREAM_TYPE_METADATA,
                            metadata_stream_cb_routine, this);
···

    if (isNoDisplayMode()) {
        rc = addStreamToChannel(pChannel, CAM_STREAM_TYPE_PREVIEW,
                                nodisplay_preview_stream_cb_routine, this);
    } else {
        rc = addStreamToChannel(pChannel, CAM_STREAM_TYPE_PREVIEW,
                                preview_stream_cb_routine, this);
    }
···
    if(···)
         rc = addStreamToChannel(pChannel, CAM_STREAM_TYPE_ANALYSIS,
                NULL, this);

    m_channels[QCAMERA_CH_TYPE_PREVIEW] = pChannel;
    return rc;
}

在啟動預(yù)覽的時(shí)候摄杂，添加了3路數(shù)據(jù)流，分別是：

• CAM_STREAM_TYPE_METADATA ： 元數(shù)據(jù)流
• CAM_STREAM_TYPE_PREVIEW：預(yù)覽流
• CAM_STREAM_TYPE_ANALYSIS：人臉分析流

用戶空間：HAL-mm-camera-interface
hardware/qcom/camera/QCamera2/stack/mm-camera-interface/src/mm_camera_stream.c

調(diào)用流程：
mm_channel_init
mm_camera_intf_add_channel
mm_camera_intf_add_stream
mm_channel_fsm_fn
mm_channel_fsm_fn_stopped
mm_channel_add_stream
mm_stream_fsm_fn/*獲取流 */
mm_stream_fsm_inited
mm_stream_init
mm_stream_set_ext_mode

最終是調(diào)用到mm_stream_set_ext_mode

int32_t mm_stream_set_ext_mode(mm_stream_t * my_obj)
{

    s_parm.type =  V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
    //通過IOCTL的方式墨坚，調(diào)用到內(nèi)核映挂。
    rc = ioctl(my_obj->fd, VIDIOC_S_PARM, &s_parm);

    return rc;
}

通過IOCTL的方式盗尸，調(diào)用到內(nèi)核咪辱。
ioctl(my_obj->fd, VIDIOC_S_PARM, &s_parm);

內(nèi)核空間

kernel/msm-4.9/drivers/media/v4l2-core/v4l2-ioctl.c

static struct v4l2_ioctl_info v4l2_ioctls[] = {
···
IOCTL_INFO_FNC(VIDIOC_S_PARM, v4l_s_parm, v4l_print_streamparm, INFO_FL_PRIO),
···
}

static int v4l_s_parm(const struct v4l2_ioctl_ops *ops,
                struct file *file, void *fh, void *arg)
{
    //調(diào)用vidioc_s_parm
    return ops->vidioc_s_parm(file, fh, p);
}

kernel/msm-4.9/drivers/media/platform/msm/camera_v2/camera/camera.c

static const struct v4l2_ioctl_ops camera_v4l2_ioctl_ops = {
···
    /* Stream type-dependent parameter ioctls */
    .vidioc_g_parm = camera_v4l2_g_parm,
    .vidioc_s_parm = camera_v4l2_s_parm,
···
};

在前面的文章油狂，有說到ioctl_ops這個操作集合：
其中.vidioc_s_parm = camera_v4l2_s_parm。

static int camera_v4l2_s_parm(struct file *filep, void *fh,
    struct v4l2_streamparm *parm)
{
    //創(chuàng)建流
    rc = msm_create_stream(event_data->session_id,
        event_data->stream_id, &sp->vb2_q);

    return rc;
}

三专筷、log

HAL層關(guān)于數(shù)據(jù)流的定義：

typedef enum {
    CAM_STREAM_TYPE_DEFAULT,       /* default stream type */
    CAM_STREAM_TYPE_PREVIEW,       /* preview */
    CAM_STREAM_TYPE_POSTVIEW,      /* postview */
    CAM_STREAM_TYPE_SNAPSHOT,      /* snapshot */
    CAM_STREAM_TYPE_VIDEO,         /* video */
    CAM_STREAM_TYPE_CALLBACK,      /* app requested callback */
    CAM_STREAM_TYPE_IMPL_DEFINED, /* opaque format: could be display, video enc, ZSL YUV */
    CAM_STREAM_TYPE_METADATA,      /* meta data */
    CAM_STREAM_TYPE_RAW,           /* raw dump from camif */
    CAM_STREAM_TYPE_OFFLINE_PROC,  /* offline process */
    CAM_STREAM_TYPE_PARM,         /* mct internal stream */
    CAM_STREAM_TYPE_ANALYSIS,     /* analysis stream */
    CAM_STREAM_TYPE_DEPTH,        /* Depth stream for depth sensor*/
    CAM_STREAM_TYPE_MAX,
} cam_stream_type_t;

• CAM_STREAM_TYPE_PREVIEW = 1
• CAM_STREAM_TYPE_METADATA = 7
• CAM_STREAM_TYPE_ANALYSIS = 11

//hal 層
D QCamera : <MCI><DBG> mm_stream_fsm_inited: 596: open dev name =/dev/video1 , fd = 78
D QCamera : <MCI><DBG> mm_stream_set_ext_mode: 1894: E, my_handle = 0x900, fd = 78, state = 1
D QCamera : <MCI><DBG> mm_stream_set_ext_mode: 1901: stream fd=78, rc=0, extended_mode=1

//元數(shù)據(jù)流
D QCamera : <MCI><HIGH> mm_stream_calc_offset: 4876: Stream type 7 num_planes 1
D QCamera : <MCI><HIGH> mm_stream_calc_offset: 4883: Plane 0, stride 764376, scanline 1, width 764376, height 1,length 764416

//預(yù)覽流
D QCamera : <MCI><HIGH> mm_stream_calc_offset: 4876: Stream type 1 num_planes 2
D QCamera : <MCI><HIGH> mm_stream_calc_offset: 4883: Plane 0, stride 384, scanline 320, width 360, height 320, length 122880
D QCamera : <MCI><HIGH> mm_stream_calc_offset: 4883: Plane 1, stride 384, scanline 160, width 360, height 160, length 61440

//人臉分析流
D QCamera : <MCI><HIGH> mm_stream_calc_offset: 4876: Stream type 11 num_planes 1
D QCamera : <MCI><HIGH> mm_stream_calc_offset: 4883: Plane 0, stride 544, scanline 480, width 540, height 480,length 261120

//kernel層
camera_v4l2_s_parm:session_id = 1,event_data->stream_id=1,sp->stream_id=1
camera_v4l2_s_parm:session_id = 1,event_data->stream_id=2,sp->stream_id=2
camera_v4l2_s_parm:session_id = 1,event_data->stream_id=3,sp->stream_id=3

Stay Hungry吮旅，Stay Foolish!