EdIE-Viz is an interactive natural language processing demo to mine information from brain imaging reports written by radiologists. Given the raw text of a radiology report for a brainscan, the demo creates output for two systems that automatically extract medical findings and their locations related to the brain. Both systems also differentiate between definite positive findings and ones which turn out to be negative when inspecting the scan image. A finding is considered negative if it is negated in the text (e.g. no ischaemic stroke).
The demo compares EdIE-R, a rule-based system developed for this type of data, to EdIE-N, a neural network based algorithm to do the same tasks. Both systems were developed using the same data annotated by domain experts and have been validated on the same unseen test data. The systems and demo were developed by staff at the Language Technology Group at the University of Edinburgh as part of an ongoing collaboration with neurologists, radiologists and medical researchers at the University of Edinburgh.
Both systems have different strengths and weaknesses. EdIE-R performs highly accurately for observations for which it has rules and for which it contains examples in its in-built lexicon. EdIE-N is trained on far less training data than neural network systems tend to be trained on given the limitations of data access for this domain and the challenges involved to get such data annotated with ground truth by domain experts. However, it is still able to recognise the majority of observations and is also much more robust to spelling errors or spelling variations than the rule-based system. You can try that out by introducing a few errors into the text. For example, change “infarct” to “infact” or “small vessel disease” to “small vessl disease” and see how the two systems compare.
How to use EdIE-Viz
- The demo provides a synthetic example of a realistic brain imaging report so you can try it out yourself.
- You can get the predictions of the models for the input text by clicking on the button.
- You can reset any changes you made to the text by clicking on the button.
- If you are feeling adventurous, you can modify the text to investigate the strengths and weaknesses of the two models.
Philip John Gorinski, Honghan Wu, Claire Grover, Richard Tobin, Conn Talbot, Heather Whalley, Cathie Sudlow, William Whiteley and Beatrice Alex (2019). Named Entity Recognition for Electronic Health Records: A Comparison of Rule-based and Machine Learning Approaches, accepted for presentation at the HealTAC 2019 Conference, 24-25th of April 2019.
Dominic Sykes, Andreas Grivas, Claire Grover, Richard Tobin, Cathie Sudlow, William Whiteley, Andrew McIntosh, Heather Whalley, Beatrice Alex (2020), Comparison of Rule-based and Neural Network Models for Negation Detection in Radiology Reports, Journal of Natural Language Engineering.
Andreas Grivas, Beatrice Alex, Claire Grover, Richard Tobin, William Whiteley (2020), Not a cute stroke: Analysis of Rule- and Neural Network-based Information Extraction Systems for Brain Radiology Reports, LOUHI workshop, EMNLP 2020.