--- id: "claim:verifying-llm-extracted-text-with-token-alignment-2026-02-10-biorxiv:8d016d1e5ff2aaf9af8b" kind: "claim" title: "General-domain tokenizers do not reliably capture gene names and chemical identifiers." description: "" url: "https://sina.bio/claim/general-domain-tokenizers-do-not-reliably-capture-gene-names-and-chemic-3d597e68" --- # General-domain tokenizers do not reliably capture gene names and chemical identifiers. ## Content General-domain tokenizers do not reliably capture gene names and chemical identifiers. ## Metadata ```json { "claim_position": 94, "source_publication_id": "version:verifying-llm-extracted-text-with-token-alignment--2026-02-10-biorxiv", "source_work_id": "work:verifying-llm-extracted-text-with-token-alignment" } ``` ## Connections (3) ### Verifying LLM-extracted text with token alignment - ID: `version:verifying-llm-extracted-text-with-token-alignment--2026-02-10-biorxiv` - Kind: `publication` - Direction: `incoming` - URL: This publication contains this grounded claim. > The slightly reduced alignment on BIO-BOAT likely reflects the complexity of biomedical vocabulary, where gene names and chemical identifiers are often split incon- sistently and therefore cannot be reliably captured by a tokenizer trained on general-domain text. Source: 2026-02-10-biorxiv.pdf — ### Higher alignment recall can reduce specificity, particularly for complex biomedical terminology, motivating domain-speci - ID: `result:verifying-llm-extracted-text-with-token-alignment-2026-02-10-biorxiv:8bfa98ff193505765f03` - Kind: `result` - Direction: `incoming` - URL: This claim supports this result. ### On biomedical BIO-BOAT targets, subword tokenization recovers about 89% of non-contiguous phrases versus 45-46% for word - ID: `result:verifying-llm-extracted-text-with-token-alignment-2026-02-10-biorxiv:21582eadcfbee0c2803d` - Kind: `result` - Direction: `incoming` - URL: This claim supports this result.