Guilty Until Proven Human

Someone pastes your text into a box. The box returns a number: 98% AI. Now you are in an argument you cannot win, because you have been asked to prove that you wrote your own sentences, and there is nothing in the sentences that can prove it.

The best detectors are good, they do serve a certain purpose, and the state-of-the-art algorithms they use are not to be thrown away lightly. However, the point they are trying to prove is far from saying anything about the actual authorship of the text. A number that is probably right is still not evidence of what someone did.

The experiment

To detect AI writing, we will use Pangram, one of the most popular tools for this purpose.

Pangram will flag which parts of the text it finds are AI written, and which are human written. I will mark the AI written parts in red, and the human written parts in green.

Human text

Let's try to write a fast and dirty paragraph without using any assistance. I will not try to make it beautiful or perfect, but keep it simple and natural.

LLM text

Now let's try to write a paragraph using an LLM. Let's mention the main points that we want to include in the paragraph, so the text resembles the human written one. Here is the prompt:

Write a paragraph about a dog beach and mention the following:
- The beach bar with dog ice cream
- My dog enjoys the other dogs and people there
- She doesn not like to swim but likes to cool off in the water
- Its a great way to unwind after work
- I miss the beach in the winter

Here is the result:

LLM as an editor

Now let's make it more interesting, and let's take the human written paragraph and ask the LLM to fix the style a bit. Here is the prompt:

Lets fix the style of the following paragraph: [paragraph]. Try to improve it and make it more readable while retaining the same information.

Here is the result:

The text reads better, doesn't change any of the information, and rephrases some of the sentences a bit to make them sound more natural.

Human as the plagiator

Now let's try to humanize the fully AI written paragraph from the LLM text section a bit by rephrasing some of the sentences.

Before continuing with the topic of AI assisted writing, we have to mention dubious circumstances and the origin of the training materials that have been used to train most of the mainstream AI models. It has been widely reported that the training materials have been sourced from the internet, and that some of that vast amount of text includes copyrighted material. This is a serious issue, and it is not to be ignored, but this is not the topic of this blog post. Let's assume we are using an ethical LLM that is not trained on copyrighted material.

The breakdown

Now what did we learn from this experiment?

Writing without assistance

This is the way we have been writing for thousands of years, and it has been widely regarded as a craft. For a journalist, writing serves the means of communicating the news in the best way possible. For a fiction writer, writing means creating a distinct style and voice of the author, with the beauty of words put on paper often being more important than the story itself. For a lawyer, writing ties to the legal meaning of the words, and the context in which they are used. For a business person, writing is a means of communicating professionalism and clarity of thought.

Grammar checkers

Before we got these, a text full of grammatical errors meant that the author has a poor grasp of the language, and is probably not able to communicate effectively.

When we got the ability to easily and quickly check our text for grammatical correctness, it was considered unprofessional to share a text that contains any errors, since that came off as a sign of laziness or lack of care.

AI as an editor

These days, we have the ability to use AI to edit our text and make it more readable and natural. Much as grammar checkers, sharing a text that is structured poorly is considered unprofessional and disrespectful to the reader.

No one complains when they receive a clear, understandable text that is well structured and communicates the message effectively.

AI as a plagiator

Here is where things get interesting. A fully AI generated story, that has been rewritten by a human, sentence by sentence, will not trigger the AI writing detection tools.

A fully human written story, that has been revised for clarity and readability using AI, will.

What a score actually tells you

The only thing we can conclude when we see a high AI assisted writing score on one of these tools, is that AI has been used to assist in the writing process. Even that is not always the case, since the tools are not always accurate.

Our judgement, when it comes to AI assisted writing, should depend on the context and purpose of the written text. There are books with great stories and poor writing styles, and there are books with poor stories and great writing styles. Passing the AI writing as one's own style is a clear violation of the intent of the writing, and should be considered as such. But if the writing is transparent and the quality of the writing is not the focus, it seems like a shame to not use the tools at our disposal.

In some ways, using an LLM to brainstorm and revise the perfect sentence can be exemplary, since it may mean that the author went to great lengths to make the text as good as possible, and as understandable as possible to the reader.

If our intent is to offer our readers a clear and understandable text that communicates the message effectively, it almost seems disrespectful to our readers not to use AI to iterate on the text, see possible improvements, select the best words, and make the text as good as possible.

I know this sounds super far-fetched and possibly like a pure excuse, but let's take a step back and think about it for a moment.

A parallel from music

Let's draw a parallel with another art form that includes writing as a key ingredient as well.

Thriller by Michael Jackson was released in 1982 and sold more than 65 million copies worldwide, making it the best selling album of all time. By Quincy Jones's own account, they considered around 700 songs before recording it, some written by Michael, most of them by other professional songwriters. Out of 9 songs that made it to the album, only 4 were written by Michael Jackson. His role was to select the best songs for the album, and considering the album's massive success, he did a great job.

Michael and Quincy did not have to write the songs themselves, to create the best selling album of all time. Still, the arrangement, the execution, the musicianship and the overall production of the album are a direct result of their work.

There are many musical artists that write, arrange, play instruments and produce their own songs, executing the entire production process themselves. There are also many artists that hire other professionals to write, arrange, play instruments and produce their own songs. It seems that the public opinion on the topic is forgiving of the latter, and appreciation of the former became a niche market.

It is only reasonable to assume that the same will happen with writing. One could argue it has already happened, with ghostwriters and book editors being hired to make the text as good as possible.

Here are some well-known books where the writing was done, in whole or in large part, by someone other than the author on the cover. A few are openly credited, which is part of the point.

The conclusion

The ability to craft beautiful text did not, and will not die with the advent of AI, and will in many ways still be valuable. What has changed with the coming of AI is the separation between the ability to craft beautiful text and the ability to come up with ideas and concepts that grasp human emotions and attention. Inability to write good prose is no longer a blocker to being a successful writer, opening the doors to many more people to write and express their ideas and thoughts.

Similar to Quincy and Michael, who selected the best songs for the album without having to write them themselves, similar to The Prodigy, Moby, Wu-Tang, Public Enemy, DJ Shadow, Daft Punk, Fatboy Slim and many more who could not have made the records they did without the help of technology that allowed them to replace a master of an instrument with a sampler and a crate of records, it is only reasonable to assume that the same will happen with writing. A new kind of writing will emerge, and I feel it is great that it will, because our collective culture will be richer for another form of expression.

The commodity of having the choice of words and sentences at an affordable price, will produce the same effect that the commodity of having instrument samples without having to learn to play an instrument had on music.

A quote from Pixar Animation Studios leader John Lasseter, who at the time was facing scrutiny for challenging traditional hand-drawn animation with 3D computer animation, provides a perfect conclusion to this blog post:

art challenges technology, and technology inspires art