Home › Categories › Technology› Relating Neural Text Degeneration

Relating Neural Text Degeneration.pdf

2021.blackboxnlp-1.16.pdf
Preview of Relating Neural Text Degeneration
🔗 Source: aclanthology.org
📊 Size: 3.04 MB
👤 Author: Ting-Rui Chiang ; Yun-Nung Chen
⬇️ Downloads: 230

Summary

This work relates two issues in neural-based text generation: exposure bias and text degeneration. Despite numerous studies, the impact of exposure bias on text generation remains unclear. Text degeneration, a problem recently found in the pre-trained language model GPT-2, refers to the generation of bland or repetitive texts. The authors investigate the connection between these two issues, finding that text degeneration is likely partly caused by exposure bias. They identify mistakes made by GPT-2 before text degeneration occurs and show that the hidden states of GPT-2 deviate from those generated by encoding real text. The results provide a foundation for further investigation on exposure bias and text degeneration problems.

Download File

Description

This work relates two issues in neural-based text generation: exposure bias and text degeneration.

Technical Information

  • File Format: PDF
  • File Size: 3.04 MB
  • Pages: 12
  • Language: EN
  • Author: Ting-Rui Chiang ; Yun-Nung Chen
  • Total Downloads: 230
  • Last Updated: 1 week ago

Document Overview

This PDF document about Relating Neural Text Degeneration provides comprehensive information and guidance. Whether you're a beginner or advanced user, this resource offers valuable insights into Relating Neural Text Degeneration.

Related Topics

If you're interested in Relating Neural Text Degeneration, you might also want to explore:

Relating guideRelating tutorialNeural guideNeural tutorialText guideText tutorialDegeneration guideDegeneration tutorial

Related Categories

Business & Finance
41204 documents
Education & Learning
22711 documents
Health & Wellness
8956 documents

Related Documents

Preview of Towards Text Generation with Adversarially Learned Neural Outlines
PDF file

Towards Text Generation with Adversarially Learned Neural Outlines

386 KBSandeep Subramanian, Sai Rajeswar Mudumba, Alessandro Sordoni, Adam Trischler, Aaron C. Courville, CEN13 pages
Document en en
Download
Preview of Table-to-Text Generation
PDF file

Table-to-Text Generation

603 KBWenting Zhao ; Ye Liu ; Yao Wan ; Philip YuEN12 pages
The proposed approach, Attend, Memorize and Generate (AMG), aims to improve few-shot table-to-text generation by preserving table structure and...
Download
Preview of Next Generation
PDF file

Next Generation

2 MBEN88 pages
Next Generation South Africa research by Singizi Consulting Africa and the British Council explores the voices of young people, their demographics,...
Download
Preview of ACCUPLACER Next-Generation Reading Sample Questions
PDF file

ACCUPLACER Next-Generation Reading Sample Questions

593 KBEN14 pages
The College Board is a non-profit organization that connects students to college success and opportunity, providing programs and services to prepare...
Download
Preview of Text content missing for title generation.
PDF file

Text content missing for title generation.

545 KBEN2 pages
Document de 2 pages
Download

Download Relating Neural Text Degeneration eBooks for free and learn more about Relating Neural Text Degeneration. These books contain exercises and tutorials to improve your practical skills, at all levels!

Not satisfied with this document? We have related documents to Relating Neural Text Degeneration, try searching with similar keywords: Relating Neural Text Degeneration, Models Of Neural Networks Physics Of Neural Networ, Neural Networks Evolutionary Swarm Neural Network , Neural Networks R Rojas Neural Networks Springer V, Dictionary Of Technical Terms Relating To The Text, Timothy J Sandoval Carleen Mandolfo Martin J Buss Relating To The Text, Age Related Macular Degeneration, Age Related Macular Degeneration A Clinical Perspe

You can download PDF versions of the user's guide, manuals and ebooks about Relating Neural Text Degeneration, you can also find and download for free A free online manual (notices) with beginner and intermediate, Downloads Documentation, You can download PDF files (or DOC and PPT) about Relating Neural Text Degeneration for free, but please respect copyrighted ebooks.