--- name: binary-path-detection-and-validation description: Use when when setting up a bioinformatics pipeline (particularly Hi-C data processing) that depends on multiple external binaries with version constraints, and you need to configure the environment in a way that is both portable across systems and reproducible across runs. license: CC-BY-4.0 metadata: edam_operation: http://edamontology.org/operation_0004 edam_topics: - http://edamontology.org/topic_0084 tools: - MultiQC 1.8 - bowtie2 - samtools (>=1.9) - R - samtools - Python - iced - pysam derived_from: - doi: 10.1186/s13059-015-0831-x title: hicpro evidence_spans: - A couple of tools such as `bowtie2` and `samtools` (>=1.9) can be automatically installed if not detected. - A couple of tools such as `bowtie2` and `samtools` (>=1.9) can be automatically installed if not detected - samtools (>=1.9) can be automatically installed if not detected - R (http://www.r-project.org/) with the following packages claims: [] provenance: collection: https://w3id.org/holobiomicslab/asb-skill/collection/epigenomics/v1 assembled_by: scripts/collect_metabolomics_collection.py sources: - build: coll_hicpro doi: 10.1186/s13059-015-0831-x title: hicpro dedup_kept_from: coll_hicpro schema_version: 0.2.0 --- # binary-path-detection-and-validation ## Summary A configuration-driven mechanism for detecting and validating required binary tool installations (bowtie2, samtools, R, Python) by querying the system PATH and version constraints, then generating an environment-specific locked configuration file. This skill ensures reproducibility by recording exact tool locations and validating minimum version requirements before pipeline execution. ## When to use When setting up a bioinformatics pipeline (particularly Hi-C data processing) that depends on multiple external binaries with version constraints, and you need to configure the environment in a way that is both portable across systems and reproducible across runs. Specifically: (1) users provide a configuration template with optional explicit paths, (2) missing paths must be auto-detected from $PATH, (3) detected tools must meet minimum version thresholds (e.g., samtools ≥1.9, Python >3.7), and (4) the final configuration must be locked to prevent accidental runtime modification. ## When NOT to use - When all tool paths are already hardcoded in pipeline scripts or environment modules—this skill is unnecessary if binaries are already resolved. - If the pipeline does not have version constraints or does not require specific binaries—the overhead of path detection and validation adds complexity without benefit. - When deploying via containerized images (Docker, Singularity) where dependencies are pre-installed and environment is fixed—detection is redundant in a sealed container. ## Inputs - config-install.txt template file (plaintext with placeholder entries for tool paths and cluster scheduler) - System environment variables ($PATH) - User-specified tool paths (optional, in config-install.txt) ## Outputs - config-system.txt (read-only locked configuration file with resolved paths for all dependencies) - Installed or verified binary tools (bowtie2, samtools, R, Python) at specified locations - Installation logs and version validation records ## How to apply First, users edit a config-install.txt template to optionally specify explicit paths for PREFIX, BOWTIE2_PATH, SAMTOOLS_PATH, R_PATH, PYTHON_PATH, and CLUSTER_SYS; any unset entries trigger automated detection. For each undetected tool, query the system PATH using the 'which' command to locate the binary; if not found, attempt automatic installation (bowtie2 and samtools ≥1.9 are auto-installable). Once located (whether user-specified or auto-detected), validate that the installed version meets minimum requirements—samtools ≥1.9 and Python >3.7 are mandatory. Compile all validated paths and system parameters into a structured config-system.txt file with locked read-only permissions to prevent user modification during pipeline execution. Run 'make configure' followed by 'make CONFIG_SYS=config-install.txt install' to execute the full setup workflow. ## Related tools - **bowtie2** (Short-read DNA sequence aligner; auto-installed if not detected during binary-path validation) — http://bowtie-bio.sourceforge.net/bowtie2/index.shtml - **samtools** (SAM/BAM file manipulation and coordinate sorting; version ≥1.9 required and auto-installable if missing) - **Python** (Runtime for iced ICE normalization module and Hi-C data processing scripts; version >3.7 required) - **R** (Statistical computing environment for ggplot2 and RColorBrewer visualization packages) — http://www.r-project.org/ - **iced** (Python module implementing iterative correction and eigenvector decomposition (ICE) normalization of Hi-C contact matrices; must be independently installed) — https://github.com/hiclib/iced - **pysam** (Python wrapper for samtools C-API to read and manipulate SAM/BAM alignments; version ≥0.15.4 required) — https://github.com/pysam-developers/pysam ## Examples ``` tar -zxvf HiC-Pro-master.tar.gz && cd HiC-Pro-master && make configure && make CONFIG_SYS=config-install.txt install ``` ## Evaluation signals - config-system.txt file exists, is readable, and contains non-empty valid paths for all mandatory dependencies (BOWTIE2_PATH, SAMTOOLS_PATH, R_PATH, PYTHON_PATH) - Version validation passes for all detected binaries: samtools --version reports ≥1.9; python --version reports >3.7 - config-system.txt has read-only permissions (mode 444 or equivalent) to prevent accidental user modification - No 'tool not found' or 'version mismatch' errors appear in installation logs when running 'make install' - Subsequent pipeline invocations use the locked config-system.txt without re-detecting or re-validating paths, ensuring reproducibility ## Limitations - Automatic installation is limited to bowtie2 and samtools ≥1.9; other dependencies (R, Python, iced) must be pre-installed or manually specified in config-install.txt - The 'which' command used for PATH queries may behave differently across Unix shells and OS distributions, potentially causing false negatives on non-standard systems - Version detection relies on tool-specific version flag semantics (e.g., 'samtools --version', 'python --version'); tools with non-standard version output may fail validation - Once config-system.txt is locked as read-only, correcting a path error requires manual unlock or reinstall—no in-place recovery mechanism is documented - The skill does not validate that detected tools are functional beyond version checking (e.g., corrupt binaries, missing shared libraries, or insufficient file permissions are not caught) ## Evidence - [methods] Edit the config-install.txt file and set the paths. If not set, the dependencies will be sought in the $PATH: "Edit the config-install.txt file and set the paths. If not set, the dependencies will be sought in the $PATH" - [methods] A couple of tools such as `bowtie2` and `samtools` (>=1.9) can be automatically installed if not detected.: "A couple of tools such as `bowtie2` and `samtools` (>=1.9) can be automatically installed if not detected." - [methods] make CONFIG_SYS=config-install.txt install: "make CONFIG_SYS=config-install.txt install" - [readme] Python (>3.7) with *pysam (>=0.15.4)*, *bx-python(>=0.8.8)*, *numpy(>=1.18.1)*, and *scipy(>=1.4.1)* libraries: "Python (>3.7) with *pysam (>=0.15.4)*, *bx-python(>=0.8.8)*, *numpy(>=1.18.1)*, and *scipy(>=1.4.1)* libraries" - [readme] samtools (>1.9). Unix sort (**which support -V option**) is required!: "samtools (>1.9). Unix sort (**which support -V option**) is required!"