單細(xì)胞數(shù)據(jù)爆發(fā)，現(xiàn)在細(xì)胞數(shù)據(jù)隨便就是萬和百萬級(jí)別纹腌，除了有許多專門開發(fā)出來快速處理百萬級(jí)的工具包（例如Cumulus等）和down sample的思想外，為了減輕個(gè)人PC內(nèi)存有限的問題滞磺，其實(shí)方法還蠻多的壶笼，例如ArchR處理scATAC數(shù)據(jù)一樣，就是將數(shù)據(jù)放在硬盤里面調(diào)用雁刷，而不是全部一股腦讀入內(nèi)存，這種思想在大數(shù)據(jù)處理時(shí)還蠻常見的保礼。
以SCE（SingleCellExperiment）對(duì)象為例沛励，主要實(shí)現(xiàn)方法就是HDF5Array包，將數(shù)據(jù)以h5的格式存儲(chǔ)在硬盤上炮障，然后sce對(duì)象靈活訪問和調(diào)用目派，減輕內(nèi)存負(fù)荷。
In the context of a SingleCellExperiment object, the file path linked with the assays can be used to efficiently manage large datasets by storing the assay data on disk rather than in memory. This is particularly useful when dealing with large single-cell RNA sequencing datasets.
The HDF5Array package in Bioconductor allows you to store assay data in HDF5 files, which can then be linked to a SingleCellExperiment object. This approach helps manage memory usage efficiently. Here's how you can create a SingleCellExperiment object with assay data stored in an HDF5 file:
1. 工具包配置:
if (!requireNamespace("BiocManager", quietly = TRUE))
install.packages("BiocManager")
BiocManager::install(c("SingleCellExperiment", "HDF5Array"))
library(SingleCellExperiment)
library(HDF5Array)
SingleCellExperiment對(duì)象本質(zhì)也是R的S4類對(duì)象胁赢，和Suerat object還有scanpy的anndata都是可以靈活轉(zhuǎn)化的企蹭，非常簡(jiǎn)潔。
SCE對(duì)象的官方教程：Chapter 4 The SingleCellExperiment class | Introduction to Single-Cell Analysis with Bioconductor
2. 創(chuàng)建基因x細(xì)胞的表達(dá)矩陣智末，且存為h5文件
# Create some example data
# 如果你有 這里換成讀入
counts <- matrix(rpois(100, lambda = 10), nrow = 10, ncol = 10)
rownames(counts) <- paste0("Gene", 1:10)
colnames(counts) <- paste0("Cell", 1:10)
# Save the counts matrix to an HDF5 file
h5file <- tempfile(fileext = ".h5")
writeHDF5Array(counts, filepath = h5file, name = "counts")
如果是10X之類的數(shù)據(jù)谅摄，對(duì)應(yīng)讀入就好了，不需要自己手動(dòng)構(gòu)建系馆。
3. 創(chuàng)建SCE對(duì)象
# Load the HDF5 file as an HDF5Array
counts_h5 <- HDF5Array(filepath = h5file, name = "counts")
# Create the SingleCellExperiment object
sce <- SingleCellExperiment(assays = list(counts = counts_h5))
# Add row data (gene metadata)
rowData(sce) <- DataFrame(GeneID = rownames(counts))
# Add column data (cell metadata)
colData(sce) <- DataFrame(CellID = colnames(counts))
# Accessing the data
counts(sce)
rowData(sce)
colData(sce)
In this example, the counts matrix is stored in an HDF5 file, and the SingleCellExperiment object links to this file for the assay data. This approach is particularly useful for large datasets, as it avoids loading the entire dataset into memory.
You can access and manipulate the assay data in the SingleCellExperiment object just like any other assay, but the data will be read from and written to the HDF5 file on disk as needed.
然后你就會(huì)發(fā)現(xiàn)送漠，你可以隨意訪問counts矩陣，但是這個(gè)sce對(duì)象卻非常小由蘑，具體來看闽寡，其實(shí)會(huì)發(fā)現(xiàn)它內(nèi)部的sce@assays@data@listData[["counts"]]@seed@seed@filepath其實(shí)只是一個(gè)矩陣路徑，而不是真實(shí)的數(shù)據(jù)尼酿。
