Table of Links
A. Mathematical Formulation of WMLE
C Qualitative Bias Analysis Framework and Example of Bias Amplification Across Generations
D Distribution of Text Quality Index Across Generations
E Average Perplexity Across Generations
F Example of Quality Deterioration Across Generations
D Distribution of Text Quality Index Across Generations
D.1 Deterministic Setup
D.2 Beam Search Setup
D.3 Nucleus Sampling Setup
E Average Perplexity Across Generations
F Example of Quality Deterioration Across Generations
Examples of GPT-2 text outputs affected by iterative synthetic (Example articles are truncated for brevity, as the full articles are lengthy but exhibit consistent quality degradation throughout; see the dataset for details).
F.1 Example of Deterministic Setup
• Original Article: The world’s eyes are on President Obama this week as he rallies a coalition of nations to “destroy” the extremist Islamic State (IS) and its terrorist-led territory. Yet if the Arab world is ever to throw off its many forms of tyranny – from theocracy to autocracy to monarchy – it also needs a model to emulate. Text Quality Index: 2.81
• Generation 0: , the Syrian government has launched a series of airstrikes on the militant group’s stronghold of Raqqa, the capital of the self-proclaimed Islamic State. The strikes have targeted the Islamic State’s military headquarters, the Al-Bab, a military training center, and the Al-Baba, a military training center in it will need to confront the Islamic State’s growing influence in the region. Text Quality Index: 2.58; Perplexity: 6.68
• Generation 1: , says US official, says US military is ready to go, says Pentagon official. Says US military is ready to go, says Pentagon official. Says US military is ready to go, says Pentagon official. Says US military is ready to go, says Pentagon official. Says US military is ready to go, says Pentagon official the world is not so simple. Text Quality Index: 2.44; Perplexity: 5.77
• Generation 5: in Iraq and Syria (ISIS) group Read more The Iraqi army has been fighting the Islamic State since the group seized large swaths of territory in Iraq and Syria (ISIS) group in 2014. The Iraqi army has been fighting the Islamic State the Iraqi army. The move comes as the U.S. the Iraqi the the the the holiest places in the world. Text Quality Index: 2.01; Perplexity: 3.17
• Generation 10: the Iraqi the Iraqi army. The move comes as the United States and its allies are ramping up their military campaign against the Islamic State, the Iraqi the Iraqi the Iraqi the Iraqi the Iraqi the Iraqi the Iraqi the Iraqi the Iraqi the Iraqi the Iraqi the Iraqi the Iraqi the Iraqi the Iraqi army. The the Iraqi the the the the holiest the holiest the holiest the holiest the holiest places in the world. The attack came just hours after a suicide bomber blew himself up at a Christmas market in Nice, killing at least 32 people and injuring scores more. Text Quality Index: 1.24; Perplexity: 4.23
F.2 Example of Beam Search Setup
• Original Article: Senate approval will be a much higher bar to clear: There’s no shortage of bipartisan criticism from the upper chamber right now, and the GOP holds a considerably smaller majority margin on that half of the Hill. Text Quality Index: 2.70
• Generation 0: on Thursday, with Senate Majority Leader Mitch McConnell (R-Ky.) announcing that the bill would be headed to the House for a vote. Senate Majority Leader Harry Reid (D-Nev.) said in a statement that the Senate will vote on the bill ""as soon as possible." Text Quality Index: 2.26; Perplexity: 7.99
• Generation 1: Read more “It’s going to take a lot of work,” House Speaker John Boehner (R-Ohio) said on ABC’s “This Week.” “It’s going to take a lot of work,” House Speaker John Boehner (R-Ohio. “It’s going to take a lot of work,” said Sen. John McCain, R-Ariz., chairman of the Senate Armed Services Committee. “It’s going to take a lot of patience,” he added. “It’s going to take Recovery and Reinvestment Act of 2009.Text Quality Index: 2.26; Perplexity: 5.42
• Generation 5: committee MORE (R-Ohio) and Senate Majority Leader Mitch McConnell Addison (Mitch) Mitchell McConnellSchumer: Nunes gave Trump’secretly altered’ version of memo Davis: ‘Deep state’ existed in “ House Speaker John A. Boehner House Speaker House Speaker House Speaker John A. Boehner of the House Speaker House Speaker House Speaker John A. Boehner (ROhio) and Senate Majority Leader Mitch McConnell Addison (Mitch) Mitchell McConnellSchumer: Nunes gave Trump’secretly altered’ version of memo Davis: ‘Deep state’ existed in ’16 – but it elected Trump Health Care’s “I don’t know. Text Quality Index: 1.07; Perplexity: 3.33
• Generation 10: vote House Speaker John A. Boehner (R-Ohio) and Senate Majority Leader Mitch McConnell Addison (Mitch) Mitchell McConnellSchumer: Nunes gave Trump’secretly “ House Speaker John A. Boehner (R-Ohio) House Speaker House Speaker House of “ House Speaker House Speaker House Speaker House of Senate Senate Majority Senate Majority Senate Majority Senate Majority Senate Majority Senate Majority Senate Majority Leader Senate Majority Leader Mitch McConnell Addison (Mitch) Mitchell McConnellSchumer Text Quality Index: 0.93; Perplexity: 3.45
F.3 Example of Nucleus Sampling Setup
• Original Article: Senate approval will be a much higher bar to clear: There’s no shortage of bipartisan criticism from the upper chamber right now, and the GOP holds a considerably smaller majority margin on that half of the Hill. Text Quality Index: 2.70
• Generation 0: , but no major votes on amendments Republicans said would be needed after a day of debate. Senate GOP senators announced Thursday they were giving the bill to the House for a vote on the Senate floor before heading to Washington for a vote on the bill, according to reports. The vote is expected to be postponed amid a GOP filibuster is about to vote. Text Quality Index: 2.47; Perplexity: 19.13
• Generation 1: , House, after GOP leaders failed to reach a deal on funding legislation The House voted Thursday to proceed on a measure to overhaul the nation’s health care system, with the final vote coming in the upper chamber. But Senate Republicans, angered by the failure of to reach a deal on funding bill, are expected to is now set to. Text Quality Index: 2.31; Perplexity: 14.21
• Generation 5: , White House says MORE (R-Neb.) said Monday that President Obama’s health care law is on track to pass the Senate with bipartisan support. The White House on Tuesday signaled the vote was a vote of 216-189. Text Quality Index: 2.20; Perplexity: 18.64
• Generation 10: vote Thursday afternoon, sending bill on track to head to President Trump for final vote Thursday afternoon, sending bill on track to head to President Trump for final vote Thursday afternoon, sending bill on track to. The bill to move forward with the bill. ""This is a. ""The the law is the. Text Quality Index: 1.59; Perplexity: 27.51
Authors:
(1) Ze Wang, Holistic AI and University College London;
(2) Zekun Wu, Holistic AI and University College London;
(3) Jeremy Zhang, Emory University;
(4) Navya Jain, University College London;
(5) Xin Guan, Holistic AI;
(6) Adriano Koshiyama.
This paper is available on arxiv under CC BY-NC-SA 4.0 license.