Keywords
domain shift
selective prediction
abstention
confidence calibration
entity extraction
Abstract
What evidence is needed to determine which confidence signals allow a financial NER system to abstain safely when textual register changes? This review answers by treating confidence-aware entity extraction as a property of a sociotechnical workflow rather than a feature that can be read from average accuracy. The focal setting is financial named-entity recognition across filings, news, and highly shifted social-media text, where a fluent answer or entity label can rest on stale, temporally invalid, or systematically omitted evidence. Evidence from the assigned publications is synthesized with foundational studies of calibration, distribution shift, causal structure, and responsible deployment. Four requirements follow: preserve the lineage of documents, timestamps, lexical matches, repository or market graphs, model confidence, and matured feedback; measure stability across relevant perturbations; connect confidence to a specific action; and maintain a route for human challenge and correction. The framework distinguishes descriptive performance from decision utility and separates uncertainty about the world from uncertainty created by the model and its evaluator. It also shows why faster inference or richer reasoning is valuable only when it improves a defined decision under a transparent resource budget. The article is a literature review and research agenda, not a report of a newly completed trial.
References
Zheng, Zihao, et al. “Reliable Financial Named Entity Recognition under Domain Shift.” arXiv, version 1, 20 Aug. 2026, https://arxiv.org/abs/2608.19558v1. Preprint.
Hu, Yuntong, et al. “LARGER: Lexically Anchored Repository Graph Exploration and Retrieval.” arXiv, version 1, 8 May 2026, https://arxiv.org/abs/2605.16352v1. Preprint.
Chen, Yiwei, et al. “One-Step Generative Distillation.” ICASSP 2026 - 2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2026, pp. 2866–70, https://doi.org/10.1109/icassp55912.2026.11461982.
Hamilton, William L., Rex Ying, and Jure Leskovec. “Inductive Representation Learning on Large Graphs.” Advances in Neural Information Processing Systems, vol. 30, 2017.
Thakur, Nandan, et al. “BEIR: A Heterogeneous Benchmark for Zero-Shot Evaluation of Information Retrieval Models.” Advances in Neural Information Processing Systems, vol. 34, 2021, pp. 5370–82.
Lin, Jimmy, Rodrigo Nogueira, and Andrew Yates. Pretrained Transformers for Text Ranking: BERT and Beyond. Morgan & Claypool, 2021.
Gururangan, Suchin, et al. “Don’t Stop Pretraining: Adapt Language Models to Domains and Tasks.” Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020, pp. 8342–60.
Loughran, Tim, and Bill McDonald. “When Is a Liability Not a Liability? Textual Analysis, Dictionaries, and 10-Ks.” Journal of Finance, vol. 66, no. 1, 2011, pp. 35–65.
Araci, Dogu. “FinBERT: Financial Sentiment Analysis with Pre-Trained Language Models.” arXiv, 2019, arXiv:1908.10063.
Guo, Chuan, et al. “On Calibration of Modern Neural Networks.” Proceedings of the 34th International Conference on Machine Learning, 2017, pp. 1321–30.
Geifman, Yonatan, and Ran El-Yaniv. “Selective Classification for Deep Neural Networks.” Advances in Neural Information Processing Systems, vol. 30, 2017.
Sculley, D., et al. “Hidden Technical Debt in Machine Learning Systems.” Advances in Neural Information Processing Systems, vol. 28, 2015.
Robertson, Stephen, and Hugo Zaragoza. “The Probabilistic Relevance Framework: BM25 and Beyond.” Foundations and Trends in Information Retrieval, vol. 3, no. 4, 2009, pp. 333–89.