scx --versionscx 0.6.2 (+net +zarr)
scx --help for every subcommand, captured from the binary when the site is built.
Each section below is the output of scx <command> --help, run when this page is built, so it always matches the released binary.
Single-cell streaming interop engine
Usage: scx <COMMAND>
Commands:
convert Convert between single-cell formats
inspect Inspect a single-cell file
validate Validate a single-cell file against a YAML schema
diff Compare two datasets slot by slot
snapshot Dump a materialised IR snapshot to a directory of NPY files
export Export a tabular slot to CSV or Parquet
concat Concatenate datasets along the obs (cell) axis
merge Assemble slots from multiple files onto a base h5ad (slot-patch merge)
help Print this message or the help of the given subcommand(s)
Options:
-h, --help Print help
-V, --version Print version
Convert between single-cell formats
Input auto-detected by content:
.h5ad — AnnData H5AD
.h5mu — MuData (one modality, picked with --assay)
.h5seurat — SeuratDisk H5Seurat (dgCMatrix or BPCells-backed)
.rds/.rda — Seurat object, SingleCellExperiment or SpatialExperiment
.h5 — 10x HDF5 (Cell Ranger output)
.mtx[.gz] — MatrixMarket, or a 10x MEX directory
.zarr — AnnData Zarr store, v2 or v3 (directory), or the one
table of a SpatialData store
outs/ — 10x Visium (Space Ranger); for Visium HD, one bin
directory, e.g. binned_outputs/square_008um
BPCells matrix directory; Parquet with the `net` feature
Output format selected by extension (any other is an error):
.h5ad — AnnData H5AD
.h5seurat — SeuratDisk H5Seurat
.bpcells — BPCells matrix directory (X + dimnames only)
.zarr — AnnData Zarr v3 store (directory)
Usage: scx convert [OPTIONS] <INPUT> <OUTPUT>
Arguments:
<INPUT>
Input file, a URL (with the `net` feature), or `-` to read from stdin (copied to a temporary file first, as a URL is)
<OUTPUT>
Output file (.h5ad or .h5seurat), a directory: `.zarr` (AnnData Zarr v3) or `.bpcells` (X only), or `-` to write an h5ad to stdout (written to a temporary file first; the log goes to stderr and no provenance sidecar is written), e.g. `ssh host scx convert big.h5ad - --where donor=d1 | scx convert - d1.zarr`
Options:
--chunk-size <CHUNK_SIZE>
Cells per streaming chunk
[default: 5000]
--n-vars <N_VARS>
Number of genes (var axis) for Parquet input. Required for the per-cell-list layout (e.g. Tahoe-100M: the gene_metadata row count, ~62710); derived automatically for dense (one column per gene) or when --genes is given
--genes <GENES>
Gene dictionary for per-cell-list Parquet input: a path or URL to the gene_metadata Parquet (columns token_id, ensembl_id, gene_symbol; Orion's gene_token_id/gene_name are accepted too). Remaps `genes` token ids to real columns, drops the marker token, fills var, and derives n_vars. For Tahoe-100M this is the gene_metadata subset alongside expression_data
--dtype <DTYPE>
Output data type [f32, f64, i32, u32]
[default: f32]
--assay <ASSAY>
Seurat assay [default: RNA for H5Seurat; the object's active assay for .rds]; for .h5mu, the modality [default: the only one, else rna]
--layer <LAYER>
Seurat layer to convert (H5Seurat only)
[default: counts]
--object <OBJECT>
For an .rds/.rda that holds several objects (a saved list of Seurat objects), the one to read: list names joined by `$`, e.g. `--object 'seus$Control'`. The error for such a file lists them
--dgcmatrix
Write legacy `.h5seurat` output using dgCMatrix storage instead of BPCells
--x-slot <X_SLOT>
Where to write the matrix. H5Seurat: counts, data, or auto (default). H5AD: X (= auto) or layers/<name>, which writes the matrix there and leaves /X absent.
auto (H5Seurat): if the source has a layer named "counts", or an AnnData raw.X with X's cells and genes, X is assumed to be normalised and goes into the "data" slot, and that layer or raw.X goes into "counts"; otherwise X goes into "counts". A source layer with the target's name is skipped.
[default: auto]
--project <PROJECT>
Seurat project name written into the H5Seurat root (H5Seurat output only)
[default: SeuratProject]
--seuratdisk-compat
Write SeuratDisk-compatible H5Seurat (adds version/project attrs and empty scaffold groups required by LoadH5Seurat). Default: lean output for Seurat v5 + BPCells direct loading
--source-url <SOURCE_URL>
Canonical URL of the source file (baked into uns["scx_provenance"])
--source-sha256 <SOURCE_SHA256>
Pre-computed SHA-256 of the source file (64 hex chars, optionally prefixed `sha256:`). Skips rehashing — intended for pipelines like hapiq that already compute the hash on download
--source-hash <SOURCE_HASH>
Hash of the source recorded in the provenance records (`source.hash`), computed while the conversion runs: `blake3` (default, multithreaded: ~0.15 s for 3 GB), `sha256` (~1.6 s on one core; what repositories publish) or `none`. `--source-sha256` records a hash you already have instead
[default: blake3]
[possible values: blake3, sha256, none]
--output-hash <OUTPUT_HASH>
Hash of the output recorded in the provenance sidecar (`output.hash`). Hashing reads the whole output once more after conversion: for 3.5 GB, ~0.1 s with blake3 (multithreaded) vs ~1.7 s with sha256. `none` leaves it out
[default: blake3]
[possible values: blake3, sha256, none]
--compress [<COMPRESS>]
gzip-compress `.h5ad` output. Pass `--compress` for the default level 6, or `--compress N` for level N (0..=9). Off by default. Ignored for `.h5seurat` output
--gene-major
Write X gene-major (each gene's cells contiguous, stored as anndata `csc_matrix`; `.h5ad` output only), for per-gene consumers. X is read once to count, then once per block of genes that fits --transpose-mem
--transpose-mem <GB>
Memory budget in GB for transposing X: the nonzeros held at a time when --gene-major writes X by gene, or when a gene-major (CSC) input is written by cell (outputs other than `.h5ad`, which keep it gene-major). Less memory means more reads of X
[default: 4]
--only <ONLY>
Write only these comma-separated auxiliary slots (e.g. `--only obsp` or `--only obsm:X_pca,layers`). Applies to obsm/varm/uns/layers/obsp; X/obs/var are always written. Excluding obsm also skips reading it (lowers peak RAM). `obs:COL` / `var:COL` select columns: naming any keeps only the named columns of that table (`--only obs:donor,obs:cell_type` also drops every auxiliary slot; add them back by name). Mutually exclusive with --exclude
--exclude <EXCLUDE>
Drop these comma-separated auxiliary slots (e.g. `--exclude obsp,obsm,uns`). Applies to obsm/varm/uns/layers/obsp; X/obs/var are always written; `obs:COL` / `var:COL` drop one column. Mutually exclusive with --only
--where <COL=VALUE>
Keep only the cells whose obs column COL equals VALUE. Repeat it: values given for the same column OR together, different columns AND (`--where cell_type=B --where cell_type=T --where donor=d1` keeps B and T cells of donor d1). Values compare as text (booleans as True/False); NA never matches. The text up to the first `=` is the column, so a value may contain `=`; quote it for the shell when it has spaces (`--where "cell_type=CD4 T"`).
X, layers, raw.X, obs and obsm keep the matching rows, and obsp those rows and columns; var, varm and uns are unchanged. h5ad, Zarr, dgCMatrix H5Seurat and 10x HDF5 read only the matching rows of X; other inputs read all of it. No match writes a valid file with 0 cells.
-h, --help
Print help (see a summary with '-h')
Inspect a single-cell file
Prints format, shape, and a summary of every slot (obs, var, obsm, layers, obsp, varp, varm, uns) without converting.
For 10x HDF5 files (Cell Ranger output), shows barcode/feature counts and feature-type breakdown instead of obs/var slots.
For any other valid HDF5 file (plain/unrecognized), prints a depth-2 tree of groups and datasets with shapes and dtypes — useful for exploring intermediate files saved in HDF5.
Usage: scx inspect [OPTIONS] <INPUT>
Arguments:
<INPUT>
Input file
Options:
--assay <ASSAY>
Seurat assay [default: RNA for H5Seurat; the object's active assay for .rds]; for .h5mu, the modality [default: the only one, else rna]
--layer <LAYER>
Matrix layer to read: the Seurat layer (H5Seurat), or an H5AD `/layers/<name>` used in place of `/X`. Defaults to `/X` for H5AD (auto-falling-back to a sole layer when `/X` is absent) and to `counts` for H5Seurat
--object <OBJECT>
For an .rds/.rda that holds several objects (a saved list of Seurat objects), the one to read: list names joined by `$`, e.g. `--object 'seus$Control'`. The error for such a file lists them
-y, --yes
Download a URL input without asking, however large. Without it, inspect asks before downloading more than 1 GB, and stops when it can't ask (stdin is not a terminal)
-h, --help
Print help (see a summary with '-h')
Validate a single-cell file against a YAML schema
Checks shape, slot presence, dtypes, and index uniqueness without
loading any matrix data. Exits 0 on pass, 1 on validation failure,
2 on file or schema parse error.
Example schema (schemas/after_normalization.yaml):
obs: ">= 1000"
vars: ">= 500"
layers: [normalized]
obsm: [X_pca]
x_dtype: f32
obs_index_unique: true
Usage: scx validate [OPTIONS] --schema <SCHEMA> <INPUT>
Arguments:
<INPUT>
Input file or directory, in any format scx reads
Options:
--schema <SCHEMA>
Path to YAML schema file
--json
Emit results as JSON instead of the default human-readable report
-h, --help
Print help (see a summary with '-h')
Compare two datasets slot by slot
Reads A and B (any formats scx reads) and reports, per slot, whether
they are the same, the same within tolerance, different, or present
on one side only. Matrices are streamed. What conversion legitimately
changes (storage dtype, categorical level order) is a NOTE, not a
difference; empty and absent slots compare equal.
Exits 0 when everything compared is the same, 1 when something
differs or couldn't be compared, 2 when the comparison couldn't run.
See docs/diff.md.
Usage: scx diff [OPTIONS] <A> <B>
Arguments:
<A>
First dataset (A)
<B>
Second dataset (B)
Options:
--rtol <RTOL>
Relative tolerance for floats: |a - b| <= atol + rtol * |b|
[default: 0.000001]
--atol <ATOL>
Absolute tolerance for floats
[default: 0]
--exact
Require floats to match exactly (rtol = atol = 0)
--only <ONLY>
Compare only these slots (same syntax as convert --only)
--exclude <EXCLUDE>
Skip these slots (same syntax as convert --exclude)
--map <A=B>
Pair matrix slots whose names differ, as A=B (repeatable), e.g. --map X=layers/data --map raw=X. Names: X, raw, layers/<n>, obsp/<n>
--by <BY>
Match cells and genes by index name or by position
[default: name]
[possible values: name, position]
--max-examples <MAX_EXAMPLES>
Mismatching entries listed per slot
[default: 5]
--chunk-size <CHUNK_SIZE>
Rows per streamed chunk
[default: 5000]
--json
Emit the report as JSON
-h, --help
Print help (see a summary with '-h')
Export a tabular slot to CSV or Parquet
Reads the specified slot from the input file and writes it as a flat
table. The output format is determined by the file extension (any
other is an error):
.csv — comma-separated values
.parquet — Apache Parquet
Supported slots:
obs — cell metadata (index + all columns)
var — gene/feature metadata (index + all columns)
obsm/<name> — embedding matrix (index + dim_0, dim_1, …)
Examples:
scx export merged.h5ad --slot obs --output cells.csv
scx export merged.h5ad --slot var --output genes.parquet
scx export merged.h5ad --slot obsm/X_pca --output pca.csv
Usage: scx export [OPTIONS] --output <OUTPUT> <INPUT>
Arguments:
<INPUT>
Input file (h5ad or h5seurat)
Options:
--slot <SLOT>
Slot to export: obs, var, or obsm/<name>
[default: obs]
-o, --output <OUTPUT>
Output path (.csv or .parquet)
--assay <ASSAY>
Seurat assay [default: RNA for H5Seurat; the object's active assay for .rds]; for .h5mu, the modality [default: the only one, else rna]
--layer <LAYER>
Seurat layer (H5Seurat only)
[default: counts]
-h, --help
Print help (see a summary with '-h')
Concatenate datasets along the obs (cell) axis
Follows `anndata.concat()` semantics: cells are stacked, genes are
aligned by name (inner join by default), and obs columns missing from an
input are NA-filled (float NaN, int 0, bool false, string "",
categorical "NA"). X and layers stream chunk-by-chunk; obs/var/obsm are
materialised. Output is always .h5ad.
Not carried over: obsp/varp (anndata's `pairwise`, off by default),
varm, and uns — the output's uns holds only the concat provenance record.
Examples:
scx concat a.h5ad b.h5ad -o all.h5ad --label sample
scx concat s*.h5ad -o all.h5ad --join outer --index-unique -
Usage: scx concat [OPTIONS] --output <OUTPUT> <INPUTS>...
Arguments:
<INPUTS>...
Input files (two or more)
Options:
-o, --output <OUTPUT>
Output .h5ad path
--join <JOIN>
Gene-axis join: inner (shared genes) or outer (union, zero-filled)
[default: inner]
--label <LABEL>
Add an obs column with this name recording each cell's source
--keys <KEYS>
Comma-separated source names, one per input (default: file stems). Used as --label values and as the --index-unique suffix
--index-unique <INDEX_UNIQUE>
Separator appended to obs_names to keep them unique (e.g. `-`)
--merge <MERGE>
How to carry var columns over [none, same, unique, first, only]
[default: none]
--chunk-size <CHUNK_SIZE>
Cells per streaming chunk
[default: 5000]
--dtype <DTYPE>
Output data type [f32, f64, i32, u32]
[default: f32]
--compress [<COMPRESS>]
gzip-compress the output (`--compress` for level 6, `--compress N` for 0..=9)
-h, --help
Print help (see a summary with '-h')
Assemble slots from multiple files onto a base h5ad (slot-patch merge)
Two modes:
Create: --base source.h5ad --patch ... --output merged.h5ad
Append: --into merged.h5ad --patch ...
Patch format: file.h5ad:slot/name[,slot/name...]
Supported slot groups: layers, obs, var, obsm, varm, obsp, uns
Provenance: uns["scx_provenance"] records the base and each patched
slot's source and SHA-256. When each slot was added goes in the sidecar
<output>.prov.json (extended by --into), so the same merge gives the
same bytes.
Files written by h5py/anndata carry HDF5 group timestamps that merge
would update, so merge refuses them as a base or --into target. Rewrite
the file first with `scx convert in.h5ad out.h5ad`, or pass
--allow-nonreproducible.
Examples:
scx merge --base raw.h5ad --patch norm.h5ad:layers/normalized --output merged.h5ad
scx merge --config merge.yaml
Usage: scx merge [OPTIONS]
Options:
--base <BASE>
Base file to copy (create mode; mutually exclusive with --into)
--output <OUTPUT>
Output path for the new merged file (create mode)
--into <INTO>
Existing merged file to append into (append mode)
--patch <SPEC>
Patch spec: file.h5ad:slot/name[,slot/name...]
--on-conflict <ON_CONFLICT>
How to handle a slot that already exists [error, skip, overwrite]
[default: error]
--tag <KEY=VALUE>
Embed a tag in the provenance record (key=value)
--config <CONFIG>
Path to a YAML config file (supersedes all other flags)
--chunk-size <CHUNK_SIZE>
Cells per streaming chunk
[default: 5000]
--allow-nonreproducible
Merge into an h5py/anndata-written file even though its HDF5 group timestamps make the output differ on every run
-h, --help
Print help (see a summary with '-h')
snapshot dumps SCX’s internal representation of a file as a directory of NPY arrays. scx convert can read the snapshot back, which isolates write cost from read cost in benchmarks. It is not a format to exchange data in.
Dump a materialised IR snapshot to a directory of NPY files
Reads the input file and writes raw-binary NPY arrays plus a
meta.json manifest to OUTPUT_DIR. The snapshot can then be fed
back into `scx convert` for benchmarking isolated from I/O.
Examples:
scx snapshot pbmc.h5seurat ir/ # everything
scx snapshot pbmc.h5seurat ir/ --only X,obs_index
scx snapshot pbmc.h5seurat ir/ --exclude layers,obsp
Slot specifiers for --only / --exclude:
X, obs_index, var_index, uns
obs, obs:col_name, var, var:col_name
obsm, obsm:key, varm, varm:key
layers, layers:key, obsp, obsp:key, varp, varp:key
Usage: scx snapshot [OPTIONS] <INPUT> <OUTPUT_DIR>
Arguments:
<INPUT>
Input file (.h5seurat, .h5ad, …)
<OUTPUT_DIR>
Output directory (created if absent)
Options:
--only <ONLY>
Include only these comma-separated slot specifiers
--exclude <EXCLUDE>
Exclude these comma-separated slot specifiers
--chunk-size <CHUNK_SIZE>
Cells per streaming chunk (for reading the input)
[default: 5000]
--assay <ASSAY>
Seurat assay [default: RNA for H5Seurat; the object's active assay for .rds]; for .h5mu, the modality [default: the only one, else rna]
--layer <LAYER>
Seurat layer (H5Seurat only)
[default: counts]
-h, --help
Print help (see a summary with '-h')