prothint.py:503: SyntaxWarning: invalid escape sequence '\.'

[peterjc]

I'm trying to follow https://funannotate2.readthedocs.io/en/latest/usage.html with a fungal genome, with GeneMark installed and on the $PATH. Using funannotate2 version 26.6.21 installed with conda and pip as per #82.

$ funannotate2 --version
funannotate2 v26.6.21
$ funannotate2 train -f CpEU46.pilon-ill.clean.fasta -o CpEU46.pilon-ill_f2 --cpus 8 --species Cryphonectria parasitica --strain CpEU46
...
[Aug 03 03:15 PM] /home/pcock/GeneMark/gmes_linux_64/ProtHint/bin/prothint.py:503: SyntaxWarning: invalid escape sequence '\.'
  systemCall('sed \"s/\.//\" ' + args.proteins + ' | sed \"s/|/_/g\" > ' +

Details

$ funannotate2 train -f CpEU46.pilon-ill.clean.fasta -o CpEU46.pilon-ill_f2 --cpus 8 --species Cryphonectria parasitica --strain CpEU46
[Aug 03 02:38 PM] Python v3.13.14; funannotate2 v26.6.21; gfftk v26.5.22; buscolite v26.6.21; annorefine v2026.2.22
[Aug 03 02:38 PM] Loading genome assembly and running QC checks
[Aug 03 02:38 PM] Genome stats:
{
  "n_contigs": 48,
  "size": 43704490,
  "n50": 4117409,
  "n90": 1682561,
  "l50": 4,
  "l90": 10,
  "avg_length": 910510
}
[Aug 03 02:38 PM] Filtered genome for training: kept 24 contigs (43,651,106 bp), filtered 24 contigs (53,384 bp) shorter than 10,000 bp
[Aug 03 02:39 PM] Getting taxonomy information
{
  "superkingdom": "Eukaryota",
  "kingdom": "Fungi",
  "phylum": "Ascomycota",
  "class": "Sordariomycetes",
  "order": "Diaporthales",
  "family": "Cryphonectriaceae",
  "genus": "Cryphonectria",
  "species": "Cryphonectria parasitica"
}
[Aug 03 02:39 PM] Choosing best augustus species based on taxonomy: neurospora
[Aug 03 02:39 PM] Choosing best busco species based on taxonomy: sordariomycetes
[Aug 03 02:39 PM] Running buscolite to generate training set using filtered genome
[Aug 03 02:39 PM] sordariomycetes_odb12.2 lineage contains 3876 BUSCO models
[Aug 03 02:39 PM] Prefiltering predictions using miniprot of ancestral sequences
[Aug 03 02:47 PM] Found 1630 complete models from miniprot, now launching 5405 augustus/pyhmmer [species=neurospora] jobs for 3234 BUSCO models
[Aug 03 02:58 PM] Analysis complete:
 single-copy=3709
 fragmented=98
 duplicated=0
 total=3876
[Aug 03 02:59 PM] Training set [/mnt/shared/scratch/pcock/funannotate2_CpEU46.pilon-ill/CpEU46.pilon-ill_f2/train_misc/busco_training_set.gff3] loaded with 3807 gene models
[Aug 03 02:59 PM] 3,062 of 3,807 models pass training parameters
[Aug 03 02:59 PM] 3,062 gene models selected for training, now splitting into test [n=200] and train [n=2862]
[Aug 03 02:59 PM] Training augustus using filtered training set
[Aug 03 03:00 PM] Initial training completed in 00:00:57s
{
  "tool": "augustus",
  "model": "a4fc13b6-9edd-4f9f-984a-57fde7d9982b",
  "n_test_genes": 200,
  "ref_genes_found": 200,
  "ref_genes_missed": 0,
  "extra_query_genes": 102,
  "average_aed": 0.058457833314996134,
  "nucleotide_sensitivity": 0.9553113687865541,
  "nucleotide_precision": 0.9603446186581475,
  "exon_sensitivity": 0.7512562814070352,
  "exon_precision": 0.7748743718592964,
  "gene_sensitivity": 1.0,
  "gene_precision": 0.5048543689320388
}
[Aug 03 03:00 PM] Training snap using filtered training set
[Aug 03 03:00 PM] Initial training completed in 00:00:07s
{
  "tool": "snap",
  "model": "snap-trained.hmm",
  "n_test_genes": 200,
  "ref_genes_found": 200,
  "ref_genes_missed": 0,
  "extra_query_genes": 129,
  "average_aed": 0.06516239447240182,
  "nucleotide_sensitivity": 0.9450143847763707,
  "nucleotide_precision": 0.95280493321487,
  "exon_sensitivity": 0.6979695431472082,
  "exon_precision": 0.7495769881556683,
  "gene_sensitivity": 1.0,
  "gene_precision": 0.397196261682243
}
[Aug 03 03:00 PM] Training glimmerHMM using filtered training set
[Aug 03 03:13 PM] Initial training completed in 00:03:50 and parameter optimization completed in 00:08:53s
{
  "tool": "glimmerhmm",
  "model": "train",
  "n_test_genes": 200,
  "ref_genes_found": 197,
  "ref_genes_missed": 3,
  "extra_query_genes": 41,
  "average_aed": 0.08293110704527754,
  "nucleotide_sensitivity": 0.9354106423124983,
  "nucleotide_precision": 0.9398699856405379,
  "exon_sensitivity": 0.6328125,
  "exon_precision": 0.6887152777777777,
  "gene_sensitivity": 0.9594594594594594,
  "gene_precision": 0.6339285714285714
}
[Aug 03 03:13 PM] Training GeneMark-ES using fast mode
[Aug 03 03:13 PM] Creating optimized training genome
[Aug 03 03:13 PM] Loaded 2862 training models from 14 contigs
[Aug 03 03:13 PM] Created training subset: 924 regions, 21,804,136 bp total
[Aug 03 03:13 PM] Fast GeneMark training on subset genome (hints disabled for coordinate mapping)
[Aug 03 03:15 PM] CMD ERROR: gmes_petap.pl --ES --cores 8 --training --sequence training_subset.fasta --fungus
[Aug 03 03:15 PM] warning on: /home/pcock/GeneMark/gmes_linux_64/gmhmme3  -m /mnt/shared/scratch/pcock/funannotate2_CpEU46.pilon-ill/CpEU46.pilon-ill_f2/train_misc/genemark/run/ES_ini/es_ini.mod   -f tr  -k -0.03  -o /mnt/shared/scratch/pcock/funannotate2_CpEU46.pilon-ill/CpEU46.pilon-ill_f2/train_misc/genemark/run/ES_A_1/hmmout/dna.fa_1.out  /mnt/shared/scratch/pcock/funannotate2_CpEU46.pilon-ill/CpEU46.pilon-ill_f2/train_misc/genemark/data/training/dna.fa_1
warning on: /home/pcock/GeneMark/gmes_linux_64/gmhmme3  -m /mnt/shared/scratch/pcock/funannotate2_CpEU46.pilon-ill/CpEU46.pilon-ill_f2/train_misc/genemark/run/ES_ini/es_ini.mod   -f tr  -k -0.03  -o /mnt/shared/scratch/pcock/funannotate2_CpEU46.pilon-ill/CpEU46.pilon-ill_f2/train_misc/genemark/run/ES_A_1/hmmout/dna.fa_3.out  /mnt/shared/scratch/pcock/funannotate2_CpEU46.pilon-ill/CpEU46.pilon-ill_f2/train_misc/genemark/data/training/dna.fa_3
...
warning on: /home/pcock/GeneMark/gmes_linux_64/gmhmme3  -m /mnt/shared/scratch/pcock/funannotate2_CpEU46.pilon-ill/CpEU46.pilon-ill_f2/train_misc/genemark/run/ES_ini/es_ini.mod   -f tr  -k -0.03  -o /mnt/shared/scratch/pcock/funannotate2_CpEU46.pilon-ill/CpEU46.pilon-ill_f2/train_misc/genemark/run/ES_A_1/hmmout/dna.fa_922.out  /mnt/shared/scratch/pcock/funannotate2_CpEU46.pilon-ill/CpEU46.pilon-ill_f2/train_misc/genemark/data/training/dna.fa_922
warning on: /home/pcock/GeneMark/gmes_linux_64/gmhmme3  -m /mnt/shared/scratch/pcock/funannotate2_CpEU46.pilon-ill/CpEU46.pilon-ill_f2/train_misc/genemark/run/ES_ini/es_ini.mod   -f tr  -k -0.03  -o /mnt/shared/scratch/pcock/funannotate2_CpEU46.pilon-ill/CpEU46.pilon-ill_f2/train_misc/genemark/run/ES_A_1/hmmout/dna.fa_924.out  /mnt/shared/scratch/pcock/funannotate2_CpEU46.pilon-ill/CpEU46.pilon-ill_f2/train_misc/genemark/data/training/dna.fa_924
error, file not found /home/pcock/GeneMark/gmes_linux_64/parse_set.pl: set.out
error on call: /home/pcock/GeneMark/gmes_linux_64/parse_set.pl --section ES_A --cfg  /mnt/shared/scratch/pcock/funannotate2_CpEU46.pilon-ill/CpEU46.pilon-ill_f2/train_misc/genemark/run.cfg  --v

[Aug 03 03:15 PM] /home/pcock/GeneMark/gmes_linux_64/ProtHint/bin/prothint.py:503: SyntaxWarning: invalid escape sequence '\.'
  systemCall('sed \"s/\.//\" ' + args.proteins + ' | sed \"s/|/_/g\" > ' +

I had not been able to get funannotate v1 to work with GeneMark either.

[nextgenusfs]

I'm not much help here. I use this hack solution for GeneMark for development https://github.com/nextgenusfs/dockerized-genemark. But at work I don't have a license so I just don't use it. Instead, for fungi I add this: https://github.com/nextgenusfs/helixerlite. I didn't include helixerlite in the f2 code because of dependencies.

[peterjc]

Thank you. I'm not 100% sure if my current employer qualifies for the free GeneMark license.

I'd had helixer recommended to me, so trying helixerlite seems worthwhile.

Would this feed into f2 something like this:

$ helixerlite --fasta genome.fasta --lineage fungi --out helixer.gff3
$ funannotate2 train -f genome.fasta -t helixer.gff ...

[nextgenusfs]

Helixer uses a generalized model (one for fungi, one for land plants, etc), so you don't need to train. helixerlite was my attempt to unify the complicated install and multiple tools (there was a separate rust HelixerPost application to generate the GFF3), the output is fully formatted GFF3. So you would pass the results to funannotate2 predict via the -e,--external parameter, ie https://funannotate2.readthedocs.io/en/latest/predict.html#using-external-gene-models. It will then use those models in the initial auto-weighting algorithm, typically it finds that the helixerlite are quite good.