GERONIMO
Introduction
GERONIMO is a bioinformatics pipeline designed to conduct high-throughput homology searches of structural genes using covariance models. These models are based on the alignment of sequences and the consensus of secondary structures. The pipeline is built using Snakemake, a workflow management tool that allows for the reproducible execution of analyses on various computational platforms.
The idea for developing GERONIMO emerged from a comprehensive search for telomerase RNA in lower plants and was subsequently refined through an expanded search of telomerase RNA across Insecta. GERONIMO can test hundreds of genomes and ensures the stability and reproducibility of the analyses performed.
Scope
The GERONIMO tool utilises covariance models (CMs) to conduct homology searches of RNA sequences across a wide range of gene families in a broad evolutionary context. Specifically, it can be utilised to:
- Detect RNA sequences that share a common evolutionary ancestor
- Identify and align orthologous RNA sequences among closely related species, as well as paralogous sequences within a single species
- Identify conserved non-coding RNAs in a genome, and extract upstream genomic regions to characterise potential promoter regions.
It is important to note that GERONIMO is a computational tool, and as such, it is intended to be run on a computer with a small amount of data. Appropriate computational infrastructure is necessary for analysing hundreds of genomes.
Although GERONIMO was primarily designed for Telomerase RNA identification, its functionality extends to include the detection and alignment of other RNA gene families, including rRNA, tRNA, snRNA, miRNA, and lncRNA. This can aid in identifying paralogs and orthologs across different species that may carry specific functions, making it useful for phylogenetic analyses.
It is crucial to remember that some gene families may exhibit similar characteristics but different functions. Therefore, analysing the data and functional annotation after conducting the search is essential to characterise the sequences properly.
Pipeline overview
By default, the GERONIMO pipeline conducts high-throughput searches of homology sequences in downloaded genomes utilizing covariance models. If a significant similarity is detected between the model and genome sequence, the pipeline extracts the upstream region, making it convenient to identify the promoter of the discovered gene. In brief, the pipeline:
- Compiles a list of genomes using the NCBI's Entrez database based on a specified query, e.g. "Rhodophyta"[Organism]
- Downloads and decompresses the requested genomes using rsync and gunzip, respectively
- Optionally, generates a covariance model based on a provided alignment using Infernal
- Conducts searches among the genomes using the covariance model Infernal
- Supplements genome information with taxonomy data using rentrez
- Expands the significant hits sequence by extracting upstream genomic regions using blastcmd
- Compiles the results, organizes them into a tabular format, and generates a visual summary of the performed analysis.
Quick start
The GERONIMO is available as a snakemake pipeline
running on Linux and Windows operating systems.
Windows 10
Instal Linux on Windows 10 (WSL) according to instructions, which bottling down to opening PowerShell or Windows Command Prompt in administrator mode and pasting the following:
wsl --install
wsl.exe --install UBUNTU
Then restart the machine and follow the instructions for setting up the Linux environment.
Linux:
Check whether the conda is installed:
conda -V
GERONIMO was tested on conda 23.3.1
1) If you do not have installed conda
, please install miniconda
Please follow the instructions for installing miniconda
2) Continue with installing mamba
(recommended but optional)
conda install -n base -c conda-forge mamba
3) Install snakemake
conda activate base
mamba create -p env_snakemake -c conda-forge -c bioconda snakemake
mamba activate env_snakemake
snakemake --help
In case of complications, please check the section Questions & Answers
below or follow the official documentation for troubleshooting.
Clone the GERONIMO repository
Go to the path in which you want to run the analysis and clone the repository:
cd
git clone https://github.com/amkilar/GERONIMO.git
Run sample analysis to ensure GERONIMO installation was successful
All files are prepared for the sample analysis as a default. Please execute the line below:
snakemake -s GERONIMO.sm --cores 1 --use-conda results/summary_table.xlsx
This will prompt GERONIMO to quickly scan all modules, verifying the correct setup of the pipeline without executing any analysis.
You should see the message Building DAG of jobs...
, followed by Nothing to be done (all requested files are present and up to date).
, when successfully completed.
If you want to run the sample analysis fully, please remove the folder results
from the GERONIMO directory and execute GERONIMO again with:
snakemake -s GERONIMO.sm --cores 1 --use-conda results/summary_table.xlsx
You might consider allowing more cores to speed up the analysis, which might take up to several hours.
You might want to clean GERONIMO/
directory from the files produced by the example analysis. You can safely remove the following:
GERONIMO/results
GERONIMO/database
GERONIMO/taxonomy
GERONIMO/temp
.create_genome_list.touch
list_of_genomes.txt
Setup the inputs
1) Prepare the covariance models
:
Browse the collection of available covariance models
at Rfam (You can find the covariance model in the tab Curation
.)
Paste the covariance model to the folder GERONIMO/models
and ensure its name follows the convention: cov_model_
OR
Prepare your own covariance model
using LocARNA
-
Paste or upload your sequences to the web server and download the
.stk
file with the alignment result.Please note that the
.stk
file format is crucial for the analysis, containing sequence alignment and secondary structure consensus.The LocARNA web service allows you to align 30 sequences at once - if you need to align more sequences, please use the standalone version available here
After installation run:mlocarna my_fasta_sequences.fasta
-
Paste the
.stk
alignment file to the folderGERONIMO/model_to_build
and ensure its name follows the convention:.stk
Please check the example
heterotrichea.stk
format inGERONIMO/models_to_built
for reference
2) Adjust the config.yaml
file
Please adjust the analysis specifications, as in the following example:
database: ' [Organism]' (in case of difficulties with defining the database query, please follow the instructions below)
extract_genomic_region-length: (here you can determine how long the upstream genomic region should be extracted; tested for 200)
models: ["", ""] (here specify the names of models that should be used to perform analysis)
Here you can also insert the name of the covariance model you want to build with GERONIMO - just be sure you placed
.stk
file inGERONIMO/models_to_build
before starting analysisCPU_for_model_building: (specify the number of available CPUs devoted to the process of building model (cannot exceed the CPU number allowed to snakemake with
--cores
)You might ignore this parameter when you do not need to create a new covariance model
Keep in mind that the covariance models and alignments must be present in the respective GERONIMO folders.
3) Remove folder results
, which contains example analysis output
4) Please ensure you have enough storage capacity to download all the requested genomes (in the GERONIMO/
directory)
Run GERONIMO
mamba activate env_snakemake
cd ~/GERONIMO
snakemake -s GERONIMO.sm --cores --use-conda results/summary_table.xlsx
Example results
Outputs characterisation
A) Summary table
The Excel table contains the results arranged by taxonomy information and hit significance. The specific columns include:
- family, organism_name, class, order, phylum (taxonomy context)
- GCA_id - corresponds to the genome assembly in the NCBI database
- model - describes which covariance model identified the result
- label - follows the Infernal convention of categorizing hits
- number - the counter of the result
- e_value - indicates the significance level of the hit
- HIT_sequence - the exact HIT sequence found by Infernal, which corresponds to the covariance model
- HIT_ID - describes in which part of the genome assembly the hit was found, which may help publish novel sequences
- extended_genomic_region - upstream sequence, which may contain a possible promoter sequence
- secondary_structure - the secondary structure consensus of the covariance model
B) Significant Hits Distribution Across Taxonomy Families
The plot provides an overview of the number of genomes in which at least one significant hit was identified, grouped by family. The bold black line corresponds to the number of genomes present in each family, helping to minimize bias regarding unequal data representation across the taxonomy.
C) Hits Distribution in Genomes Across Families
The heatmap provides information about the most significant hits from the genome, identified by a specific covariance model. Genomes are grouped by families (on the right). Hits are classified into three categories based on their e-values. Generally, these categories correspond to hit classifications ("HIT," "MAYBE," "NO HIT"). The "HIT" category is further divided to distinguish between highly significant hits and moderately significant ones.
GERONIMO directory structure
The GERONIMO directory structure is designed to produce files in a highly structured manner, ensuring clear insight and facilitating the analysis of results. During a successful run, GERONIMO produces the following folders:
-
/database
- which contains genome assemblies that were downloaded from the NCBI database and grouped in subfolders -
/taxonomy
- where taxonomy information is gathered and stored in the form of tables -
/results
- the main folder containing all produced results:/infernal_raw
- contains the raw results produced by Infernal/infernal
- contains restructured results of Infernal in table format/cmdBLAST
- contains results of cmdblast, which extracts the extended genomic region/summary
- contains summary files that join results from Infernal, cmdblast, and attach taxonomy context/plots
- contains two types of summary plots
-
/temp
- folder contains the information necessary to download genome assemblies from NCBI database -
/env
- stores instructions for dependency installation -
/models
- where calibrated covariance models can be pasted, for example, from the Rfam database -
/modes_to_built
- where multiple alignments in .stk format can be pasted -
/scripts
- contains developed scripts that perform results structurization
The example GERONIMO directory structure:
GERONIMO
├── database
│ ├── GCA_000091205.1_ASM9120v1_genomic
│ ├── GCA_000341285.1_ASM34128v1_genomic
│ ├── GCA_000350225.2_ASM35022v2_genomic
│ └── ...
├── env
├── models
├── model_to_build
├── results
│ ├── cmdBLAST
│ │ ├── MRP
│ │ │ ├── GCA_000091205.1_ASM9120v1_genomic
│ │ │ │ ├── extended
│ │ │ │ └── filtered
│ │ │ ├── GCA_000341285.1_ASM34128v1_genomic
│ │ │ │ ├── extended
│ │ │ │ └── filtered
│ │ │ ├── GCA_000350225.2_ASM35022v2_genomic
│ │ │ │ ├── extended
│ │ │ │ └── filtered
│ │ │ └── ...
│ │ ├── SRP
│ │ │ ├── GCA_000091205.1_ASM9120v1_genomic
│ │ │ │ ├── extended
│ │ │ │ └── filtered
│ │ │ ├── GCA_000341285.1_ASM34128v1_genomic
│ │ │ │ ├── extended
│ │ │ │ └── filtered
│ │ │ ├── GCA_000350225.2_ASM35022v2_genomic
│ │ │ │ ├── extended
│ │ │ │ └── filtered
│ │ │ └── ...
│ │ ├── ...
│ ├── infernal
│ │ ├── MRP
│ │ │ ├── GCA_000091205.1_ASM9120v1_genomic
│ │ │ ├── GCA_000341285.1_ASM34128v1_genomic
│ │ │ ├── GCA_000350225.2_ASM35022v2_genomic
│ │ │ ├── ...
│ │ ├── SRP
│ │ │ ├── GCA_000091205.1_ASM9120v1_genomic
│ │ │ ├── GCA_000341285.1_ASM34128v1_genomic
│ │ │ ├── GCA_000350225.2_ASM35022v2_genomic
│ │ │ ├── ...
│ ├── plots
│ ├── raw_infernal
│ │ ├── MRP
│ │ │ ├── GCA_000091205.1_ASM9120v1_genomic
│ │ │ ├── GCA_000341285.1_ASM34128v1_genomic
│ │ │ ├── GCA_000350225.2_ASM35022v2_genomic
│ │ │ ├── ...
│ │ ├── SRP
│ │ │ ├── GCA_000091205.1_ASM9120v1_genomic
│ │ │ ├── GCA_000341285.1_ASM34128v1_genomic
│ │ │ ├── GCA_000350225.2_ASM35022v2_genomic
│ │ │ ├── ...
│ └── summary
│ ├── GCA_000091205.1_ASM9120v1_genomic
│ ├── GCA_000341285.1_ASM34128v1_genomic
│ ├── GCA_000350225.2_ASM35022v2_genomic
│ ├── ...
├── scripts
├── taxonomy
└── temp
GERONIMO applicability
Expanding the evolutionary context
To add new genomes or database queries to an existing analysis, please follow the instructions:
- Rename the
list_of_genomes.txt
file toprevious_list_of_genomes.txt
or any other preferred name. - Modify the
config.yaml
file by replacing the previous database query with the new one. - Delete:
summary_table.xlsx
,part_summary_table.csv
,summary_table_models.xlsx
files located in theGERONIMO\results
directory.create_genome_list.touch
file
- Run GERONIMO to calculate new results using the command:
snakemake -s GERONIMO.sm --cores --use-conda results/summary_table.xlsx
- Once the new results are generated, reviewing them before merging them with the original results is recommended.
- Copy the contents of the
previous_list_of_genomes.txt
file and paste them into the currentlist_of_genomes.txt
. - Delete:
summary_table.xlsx
located in theGERONIMO\results
directory.create_genome_list.touch
file
- Run GERONIMO to merge the results from both analyses using the command:
snakemake -s GERONIMO.sm --cores 1 --use-conda results/summary_table.xlsx
Incorporating new covariance models into existing analysis
- Copy the new covariance model to
GERONIMO/models
- Modify the
config.yaml
file by adding the name of the new model to the linemodels: [...]
- Run GERONIMO to see the updated analysis outcome
Building a new covariance model
With GERONIMO, building a new covariance model from multiple sequence alignment in the .stk
format is possible.
To do so, simply paste .stk
file to GERONIMO/models_to_build
and paste the name of the new covariance model to config.yaml
file to the line models: [""]
and run GERONIMO.
Questions & Answers
How to specify the database query?
- Visit the NCBI Assemblies website.
- Follow the instruction on the graphic below:
WSL: problem with creating snakemake_env
In the case of an error similar to the one below:
CondaError: Unable to create prefix directory '/mnt/c/Windows/system32/env_snakemake'. Check that you have sufficient permissions.
You might try to delete the cache with: rm -r ~/.cache/
and try again.
When snakemake
does not seem to be installed properly
In the case of the following error:
Command 'snakemake' not found ...
Check whether the env_snakemake
is activated.
It should result in a change from (base) to (env_snakemake) before your login name in the command line window.
If you still see (base)
before your login name, please try to activate the environment with conda:
conda activate env_snakemake
Please note that you might need to specify the full path to the env_snakemake
, like /home/your user name/env_snakemake
How to browse GERONIMO results obtained in WSL?
You can easily access the results obtained on WSL from your Windows environment by opening File Explorer
and pasting the following line into the search bar: \\wsl.localhost\Ubuntu\home\
. This will reveal a folder with your username, as specified during the configuration of your Ubuntu system. To locate the GERONIMO results, simply navigate to the folder with your username and then to the home
folder. (\\wsl.localhost\Ubuntu\home\\home\GERONIMO
)
GERONIMO occupies a lot of storage space
Through genome downloads, GERONIMO can potentially consume storage space, rapidly leading to a shortage. Currently, downloading genomes is an essential step for optimal GERONIMO performance.
Regrettably, if the analysis is rerun without the /database
folder, it will result in the need to redownload genomes, which is a highly time-consuming process.
Nevertheless, if you do not intend to repeat the analysis and have no requirement for additional genomes or models, you are welcome to retain your results tables and plots while removing the remaining files.
It is strongly advised against using local machines for extensive analyses. If you lack access to external storage space, it is recommended to divide the analysis into smaller segments, which can be later merged, as explained in the section titled Expanding the evolutionary context
.
Considering this limitation, I am currently working on implementing a solution that will help circumvent the need for redundant genome downloads without compromising GERONIMO performance in the future.
You might consider deleting the .snakemake
folder to free up storage space. However, please note that deleting this folder will require the reinstallation of GERONIMO dependencies when the analysis is rerun.
License
Copyright (c) 2023 Agata M. Kilar
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
Contact
mgr inż. Agata Magdalena Kilar, PhD (agata.kilar@ceitec.muni.cz)
Version History
main @ 7ad0050 (earliest) Created 1st Aug 2023 at 02:34 by Agata Kilar
Update README.md
Frozen
main
7ad0050
Creator
Submitter
Views: 1775 Downloads: 222
Created: 1st Aug 2023 at 02:34
Last updated: 3rd Aug 2023 at 19:15
None