Home › Categories › Health & Wellness› PLM-ICD: Leveraging Pretrained Language Models For Automatic ICD Coding

PLM-ICD: Leveraging Pretrained Language Models For Automatic ICD Coding.pdf

2022.clinicalnlp-1.2.pdf
Preview of PLM-ICD: Leveraging Pretrained Language Models for Automatic ICD Coding
🔗 Source: aclanthology.org
📊 Size: 323 KB
📄 Pages: 11 pages
⬇️ Downloads: 46

Summary

Despite PLMs' impressive performance on natural language understanding tasks, they have underperformed on automatic ICD coding compared to traditional models. The authors identify three main issues hindering their effectiveness:


Large label space: The vast number of possible ICD codes makes it difficult for models to learn effective representations for each one.
 Long input sequences: Clinical notes often exceed the maximum sequence length limitations of PLMs.
* Domain mismatch: PLMs are typically pretrained on general-domain text, while clinical notes have a highly specialized medical vocabulary and context.

To address these challenges, the authors propose PLM-ICD, a framework that incorporates three key strategies:

1. Domain-specific pretraining: Fine-tuning PLMs on a domain-specific corpus mimicking EHRs to adapt them to the medical domain.
2. Segment pooling: Splitting long clinical notes into smaller segments to fit within the PLM's sequence length limit and aggregating representations from these segments.
3. Label attention: Using a label attention mechanism to weigh the contributions of different ICD codes during training, addressing the large label set problem.

Experimentation on benchmark MIMIC datasets demonstrates that PLM-ICD outperforms previous models based solely on PLMs, achieving state-of-the-art performance in terms of multiple metrics. The paper further analyzes factors influencing PLM performance, including pretraining method, corpus, vocabulary construction, and optimization schedules.

Download File

Description

It explores potential reasons for poor performance of pretrained models in this task and proposes a novel fine-tuning strategy.

Technical Information

  • File Format: PDF
  • File Size: 323 KB
  • Pages: 11
  • Language: EN
  • Total Downloads: 46
  • Last Updated: 6 days ago

Document Overview

This PDF document about PLM-ICD: Leveraging Pretrained Language Models for Automatic ICD Coding provides comprehensive information and guidance. Whether you're a beginner or advanced user, this resource offers valuable insights into PLM-ICD: Leveraging Pretrained Language Models for Automatic ICD Coding.

Related Topics

If you're interested in PLM-ICD: Leveraging Pretrained Language Models for Automatic ICD Coding, you might also want to explore:

PLM-ICD: guidePLM-ICD: tutorialLeveraging guideLeveraging tutorialPretrained guidePretrained tutorialLanguage guideLanguage tutorialModels guideModels tutorialAutomatic guideAutomatic tutorialCoding guideCoding tutorial

Related Categories

Business & Finance
41204 documents
Education & Learning
22711 documents
Technology
4783 documents

Related Documents

Preview of NCD Coding Updates
PDF file

NCD Coding Updates

118 KB-EN16 pages
CMS updates National Coverage Determinations (NCDs) with ICD-10 conversions and other coding revisions, effective April 1, 2018, with implementation...
Download
Preview of DÉCODEURS ICD
PDF file

DÉCODEURS ICD

159 KBFR1 page
les décodeurs ICD de Hunter sont compatibles avec les programmateurs ACC2 et offrent une flexibilité maximale avec des versions à 1, 2, 4 et 6...
Download
Preview of Nested State Clouds: Distilling Knowledge Graphs from Contextual Embeddings
PDF file

Nested State Clouds: Distilling Knowledge Graphs from Contextual Embeddings

2.63 MBEN17 pages
This thesis proposes a novel interpretability technique called Nested State Clouds (NSC) to distill sparse knowledge graphs from a model's...
Download
Preview of Deep Language Models
PDF file

Deep Language Models

4.91 MBEN160 pages
This thesis proposes three novel approaches to automate software testing and optimisation using deep language models, addressing the challenges of...
Download
Preview of Native Language ID
PDF file

Native Language ID

167 KBJulian Brooke ; Graeme HirstEN18 pages
Researchers developed a robust native language identification system using lexical features, which were previously avoided due to topic confounds,...
Download

Download PLM-ICD: Leveraging Pretrained Language Models for Automatic ICD Coding eBooks for free and learn more about PLM-ICD: Leveraging Pretrained Language Models for Automatic ICD Coding. These books contain exercises and tutorials to improve your practical skills, at all levels!

Not satisfied with this document? We have related documents to PLM-ICD: Leveraging Pretrained Language Models for Automatic ICD Coding, try searching with similar keywords: PLM-ICD: Leveraging Pretrained Language Models for Automatic ICD Coding, "Hinder och möjliggörare för 1.5°-livsstilar: Ytliga och djupgående strukturella faktorer som påverkar potentialen för hållbar k, Ändring av genomföranderam för en europeisk plattform för utbyte av balansenergi från frekvensåterställn ingsreserver med manuell, Rekommendationer för vaccination mot covid-19 för särskilda grupper av barn -, förstudie för att utvärdera förutsättningarna att genom en innovationsupphandli ng utveckla en drifttjänst för geoenergilager, Självkänsla och KBT ‐ Påverkas självkänslan vid KBT för depression och ångesttillstånd?Se lf‐esteem and CBT ‐ How does CBT for de, Matglädje för alla: en guide till rätt konsistens för olika behov, Visual Language Pretrained Multiple Instance Zero-Shot Transfer for Histopathology Images

You can download PDF versions of the user's guide, manuals and ebooks about PLM-ICD: Leveraging Pretrained Language Models for Automatic ICD Coding, you can also find and download for free A free online manual (notices) with beginner and intermediate, Downloads Documentation, You can download PDF files (or DOC and PPT) about PLM-ICD: Leveraging Pretrained Language Models for Automatic ICD Coding for free, but please respect copyrighted ebooks.