Tag Archives: Computerlinguistik

Information

Information Reduction 1: Coding

Leave a comment

Two types of coding

In a previous post, I described two fundamentally different types of coding. In the first, the intention is to carry all the information contained in the source over into the encoded version. In the second, on the other hand, we deliberately refrain from doing this. It is the second – the information-losing – type that is of particular interest to us.

When I highlighted this difference in my presentations twenty years ago and the phrase ‘information reduction’ appeared prominently in my slides, my project partners pointed out that this might not go down too well with the audience. After all, everyone wants to win; nobody wants to lose. How can I promote a product for which loss is a quality feature?

Well, sometimes we have to face the fact that the thing we have been trying to avoid at all costs is actually of great value. And that’s certainly the case for information-losing coding.

Medical coding

Our company specialised in the encoding of free-text medical diagnoses. Our program read the diagnoses that doctors write in free text in their patients’ medical records and automatically assigned them a code based upon a standard coding system (ICD-10) with about 15,000 codes (Switzerland, 2019). This sounds like a lot, but the number is small considering the billions of distinguishable diagnoses and diagnostic formulations that occur in the field of medicine (see article). Of course, the individual code cannot contain more information than the standard code is able to discern for the case in question. The full-text diagnoses usually contained more information than this and our task was to automatically extract the relevant parts from the free texts in order to assign the correct code. We were fairly successful in this attempt.

Coding is part of a longer chain

But coding is only one step in a bigger process. Firstly, the information-processing chain extends from codes to flat rates per case (Diagnosis Related Groups = DRGs). Secondly, the free texts to be coded in the medical record are themselves the result of a multi-stage chain of information processing and reduction that has already been performed. Overall, a hospital case involves a chain made up of the following stages from patient examination to flat rate per case:

  • Patient: amount of information contained in the patient.
  • Doctor: amount of information about the patient that the doctor recognises.
  • Medical record: amount of information documented by the doctor.
  • Diagnoses: amount of information contained in the texts regarding the diagnoses.
  • Codes: amount of information contained in the diagnosis codes.
  • Flat rate per case: amount of information contained in the flat rate per case.

The information is reduced at every step, usually quite dramatically. The question is, how does this process work? Can the reduction be automated. And is it a determinate process, or one in which multiple options exist?

This is a page about information reduction — see also overview.

Translation: Tony Häfliger and Vivien Blandford

Logic

Non-Monotonic Reasoning (NMR)

Leave a comment

Concept Molecules and NMR

In the article Two types of coding 1, I described the challenge of getting computers to ‘understand’ the incredibly diverse range of medical diagnoses that may crop up in a text. To meet this challenge, the computer has to convert the various diagnostic formulations encountered into a consistent format that represents all the semantic details in an easily retrievable form.

With concept molecules we have succeeded in doing this. We were aided here by two properties of the concept molecules method:
a) the consistently composite representation of semantics, and
b) a non-monotonic reasoner.
At the time, the use of a non-monotonic reasoner was very much out of vogue. Most research groups in the field of medical computational linguistics were in the process of switching from First Order Logic (FOL) to Description Logic (DL), believing that DL is the best way to get computers to interpret complex semantics. As it turned out, however, it was us – a small private research company without state support – that was successful. Instead of the accepted doctrine of FOL and DL based upon a monotonic approach, we used a non-monotonic method.

What is monotonic logic?

In logic, monotony means that the truth of statements does not change even if new contradictory information subsequently crops up. Thus, what has been recognised within the system as true remains true, and what has been recognised as false remains false. Under non-monotony, on the other hand, conclusions drawn by the system can be called into question on the basis of additional information.

So, what’s the problem with non-monotony?

It is clear that proof is only possible in a monotonic system. In a non-monotonic system, on the other hand, there is always the possibility of another argument cropping up that leads to completely different conclusions. Since proof is essential in mathematics, it is obvious that mathematical logic relies on monotony.

Computational linguistics, however, is not about proof, but about the correct assignment of words to concepts. Thus, the advantage of being able to supply proof – as important as it clearly is for mathematics – is irrelevant to our task.

And the problem with monotony?

A system that cannot change its conclusions is not able to learn in any real sense. The human brain, for example, is in no way monotonic.

Moreover, a monotonic system must also be closed, whereas in practice scientific ontologies are not closed, but grow as knowledge progresses. Progress of this kind is also evident in the development of an interpretation program with its complex algorithms:  here, too, there is continuous improvement and expansion that poses problems for monotonic systems.

In addition, monotonic systems are not particularly efficient when it comes to dealing with exceptions. It is well known that there are exceptions to every rule, and a non-monotonic system can handle these in a much more effective and straightforward way.

Non-monotony in practice

If we compare rules-based systems, I believe that non-monotonic systems are clearly preferable to monotonic ones for our purposes. Non-monotony is by no means the easy option and has a few pitfalls and knotty issues of its own, but the ease with which even detailed and complex fields can be modelled decides the issue in its favour.