2. Archive MERFISH Experiment#

2.1. Archive steps#

  1. Prepare a tar.gz file for single MERFISH experiment

  2. Compress active csv files

  3. Save transcripts in HDF5 format

  4. Save TIFF image in zarr format with proper chunks

  5. After successful archive, delete the original raw data files.

2.2. Archive Process#

from merfishing import ArchiveMerfishExperiment
# this is a small test dataset
experiment_dir = 'dummy_experiment/'

Caution

The code below will instantly start archive process, data in the experiment_dir will be modified.

# This step take ~16 hours to run on a real 500-gene 1cm2 experiment
ArchiveMerfishExperiment(experiment_dir)
dummy_experiment: Archive Raw and Output Data: /gale/netapp/cemba3c/merfish/example/dummy_experiment/dummy_experiment.tar.gz
/gale/netapp/cemba3c/merfish/example/dummy_experiment/output/region_0 does not contain cellpose results or the results are incomplete, using watershed results from vizgen pipeline instead.
region_0: Converted TIF files to Zarr
region_0: Saved transcripts to HDF5
region_0: Compressed vizgen output
dummy_experiment: Deleted raw data
<merfishing.core.archive_data.ArchiveMerfishExperiment at 0x7f62a86062e0>

2.3. After Archive Process Finished#

The archive code above will generate a tar.gz file located in {experiment_dir}/{experiment_name}.tar.gz. Archive this file for long term data storage.

Important

Once the tar.gz file is successfully generated, the raw data will be deleted automatically to save space. Make sure you archive the tar.gz file properly.