TY  - JOUR
AB  - In the increasingly competitive proofing tools market, it is becoming ever more important to find evaluation methods and metrics that provide stable, invariable measurements. This article focuses on the evaluation metrics currently used in spelling checker evaluation. We commence with a brief explication of some presuppositions relevant to the evaluation of spelling checkers. We then give a detailed description of metrics currently used in various evaluations of proofing tools, discussing and demonstrating their shortcomings. Subsequently, we illustrate that, by adjusting some of these metrics, by incorporating additional information (such as the percentage of errors in the text, and a measurement of the suggestion adequacy), and by adapting the evaluation methodology, a single metric can be designed that evaluates the overall linguistic performance of spelling checkers effectively. In conclusion, we suggest how these metrics can be further improved in future research, in order to open up the possibility to set a single benchmark that can be used for any spelling checker, irrespective of the language being evaluated, or the nature of the text used for the evaluation.
AU  - Van Huyssteen, Gerhard B.
AU  - Eiselen, E. Roald
AU  - Puttkammer, Martin J.
C1  - https://gerhard.pro/publications/VanHuyssteenEiselenPuttkammer2004
KW  - evaluation
human language technology
proofing tools
spelling checker
LA  - In: English | On: Afrikaans
LB  - PublicationVanHuyssteenEiselenPuttkammer2004
N1  - evaluering, mensetaaltegnologie, skryfhulpmiddels, speltoetser
PY  - 2004
SP  - 91-99
ST  - Re-evaluating evaluation metrics for spelling checker evaluations
T2  - Proceedings of First Workshop on International Proofing Tools and Language Technologies. Patras: University of Patras.
TI  - Re-evaluating evaluation metrics for spelling checker evaluations
ID  - 4923
ER  -