Review of Draftsmith 2.0

By Dr Linda Nix AE

Draftsmith is an artificial intelligence (AI) assistant intended for professional writers and editors, from Intelligent Editing – the same team behind PerfectIt. That alone explains Intelligent Editing’s design for Draftsmith to be “different to other AI”, in that it “doesn’t write for you” but keeps the writer or editor “in control” while improving or editing existing text. The other important difference is their claim it “never uses your data for AI training or any other purpose” – which is important if, like me, you share the ethical concerns held by publishers and authors worldwide about the unauthorised content harvesting that has gone into training most other AI tools. A review of version 1 suggested it is built on ChatGPT, though, so I still have some reservations.

This review evaluates Draftsmith 2.0 from the perspective of an experienced professional editor who works on long, complex documents on a daily basis. It looks at how easy Draftsmith is to get up and running, where it works well and where it has limitations, and assesses its overall usefulness in an editing workflow. It does not assess it from a writing perspective and does not go into the detail of its features or functionality, which were covered in the review of version 1.

Installation and getting started

Draftsmith 2.0 was easy to install from Microsoft AppSource, which detected both versions of Word on my desktop computer’s parallel operating systems. I chose my Windows system for the installation because Draftsmith requires Office 365 and my Mac uses an older version of Word. I keep this older version quarantined from various updates and add-ins that I find disruptive for everyday writing and editing, and use the up-to-date Windows version for running newer, Windows-only tools over the edited document. 

Once installed, Draftsmith has its own tab in Word from which you select “Launch Draftsmith” and log into your Draftsmith account. At first I could not get any of the tools to work. After consulting the user manual, I discovered my privacy settings needed to have Microsoft’s “connected experiences” enabled. Unfortunately, Microsoft does not allow you to choose which connected experiences are on – it’s all or nothing. That has implications for workflow: there is no offline mode for using Draftsmith. While that will not bother many (most) users, it means that Draftsmith becomes another Windows-only tool I use at the final stages of a job. 

Enabling this setting still did not get Draftsmith to work, so I contacted their support team. They responded promptly: I needed to set the add-in cache to clear. But there was also an issue with my free review account, which kept expiring, until eventually this was resolved too. As a result, I have not tested Draftsmith as fully as I intended. 

After watching the demonstration videos, I found using Draftsmith straightforward. It lives up to the claims of keeping you in control, allowing you to review its suggestions and decide whether you want them or not. Just remember that if you want Draftsmith’s suggestions to be tracked, turn on Track Changes first – the videos fail to mention this, implying that tracking will happen automatically. 

Testing and results

Improving the text

In my limited testing, I found Draftsmith superior to similar tools like Word’s built-in Editor or Copilot in the Simplify, Redraft and Polish options. Its suggestions for improving each sentence were mostly intelligent and helpful. For example, in the opening sentence above, I originally had “the same people who developed PerfectIt”, but I preferred its suggestion “the same team behind PerfectIt”. Interestingly, Draftsmith inserted “that” in places it is grammatically optional – contrary to tools like Grammarly which are training writers to delete “that” as being redundant. Since I now have to educate many authors on the difference between an optional and a necessary “that”, this is a point in Draftsmith’s favour. 

However, I found many suggested edits unnecessary or neutral at best. No matter which tool I used, Draftsmith wanted to either delete “quarantined” in the earlier paragraph about installation or replace it with “set aside”. Every edit completely changed the meaning. Once I adjusted settings to a college-level audience and the style to either “formal”, “professional” or “academic”, Draftsmith stopped trying to delete “quarantined” but instead suggested “restricted” or “isolated” – better than “set aside”, but still an unnecessary change. 

Reducing word count

Where Draftsmith particularly failed, for me, was in “Reduce Word Count”. Frequently the words it wanted to remove or replace were essential – there was rarely an instance in which the actual meaning, let alone nuance, was retained. In the screenshot below, I cannot see the rationale in changing “wanted” to “sought” – both have the problem of attributing volition to an AI and the number of words (one) is the same.

Limitations of the sentence-by-sentence approach

Draftsmith’s approach to reviewing existing text sentence by sentence can sometimes be a problem for lists, because it tends to review the lead-in sentence and each list item in isolation. Suggested changes to the lead-in sentence included changing the colon before the list to a full stop (easy to reject) but also changing the sentence structure so that it no longer led into the list. Draftsmith’s suggestions for “improving” each list item as a sentence in its own right were not careful to keep parallel structure. For example, in a list structured with leading verbs, it suggested removing the verb in some list items but not others. However, some tweaks to settings may be enough to resolve these issues – Draftsmith was happier to leave list structure alone when set to academic and professional style, despite me running it over lists taken from their own web copy. 

Draftsmith also does not pick up repetition, not even in close proximity, happily re-editing the same sentence pasted verbatim, as shown in the following screenshot. 

Rather than options for expressing the same idea in slightly different words, I would prefer a warning about the repetition. Verbatim repetition should be easy to detect. A professional editor will tell an author when an idea is repeated, even if that idea has been expressed in different words, and will do so even when there are pages or even chapters between the two expressions of the idea. That ability relies on the editor to have a very good memory – and is an area where I would expect automation tools that have infallible memories to be of more assistance (and where PerfectIt excels). 

The manual edit warnings

If the sentence under review contains a footnote, formatted characters such as bold or italics, or a hyperlink, Draftsmith warns you to apply its suggestions manually or risk losing data or formatting. This is a positive feature for those who work on text with minimal or in-text referencing, and only occasional character formatting. But for the kind of books, journal articles and reports that I edit, which are heavily footnoted and formatted, this warning will apply to almost every sentence. Having to review and manually apply any suggestions I agree with does not seem quicker than simply editing the text directly myself. 

These warnings should also apply to quotations. Draftsmith frequently wanted to revise words within quotation marks and sentences in paragraphs formatted as a block quote (for example, using Word’s default Quote style) – both editorial no-nos. Below are its suggestions using the “Polish” tool – that is, for a light copyedit – on a simple block quote. 

I also tested it on some more complex text: the next screenshot shows the text of a judgment (Kwu v R [2024] NSWCCA 199 at [34]) containing block quotes and quotes within quotes. Not only did it suggest changing text within the block quotes and quote marks, but it also removed quote marks for no reason, replaced non-breaking spaces with an ordinary space (why?), and suggested other changes that are incorrect – a “statement of principle” is not the same as a “principle statement”. 

In some quotes I tested it on, it helpfully picked up and corrected some transcription errors, but a simple spell-check would have done that. 

The careful editor will select “skip” on quotations, but a warning would help prevent inadvertent acceptance of the suggested edits. 

The verdict

Draftsmith is better than most AI-powered revision tools available but – unlike PerfectIt – is not sufficiently developed to be a time-saver for most professional editors. In particular, it needs an experienced editor to understand when to accept its suggestions and when to reject them. An important editorial skill that is specifically tested in the IPEd accreditation exam is the ability to leave text alone. Reviewing (and rejecting) Draftsmith’s unnecessary or incorrect suggestions where the text is reasonable to begin with, or accepting the suggestions but applying them manually where the text does need editing, only adds time to the editing process. In its current form, Draftsmith will likely be most useful for short and simple text such as marketing copy that does not contain footnotes or much formatting, and where the outcome is more important than the process. 

I mention process because that’s where professional editors are most valuable. An editor reads the author’s words in entirety and in context, and then works out if a change is necessary, and if so, what the edit should be. With experience, this process is very quick. I have to disagree with the previous reviewer that using Draftsmith to “unpack” a dense or complex sentence, instead of spending 15 minutes working it out yourself, makes it worth the money. If an editor does not go through that thought process, they cannot explain to the author why they are making an edit at all, nor why they are suggesting that particular edit not only within the context of the sentence but also within the context of the paragraph, section or document as a whole. 

That is, professional editors are always in conversation with their authors, explaining how and why their suggestions improve the author’s work. Getting a machine to do the editor’s thinking robs them of this process: the editor instead must work out why Draftsmith suggested the edit (it does not explain its “thinking”) and still write a comment to the author to explain it. For example, I would never suggest such a neutral change as “wanted” to “sought” without an explanation to persuade the author to accept the suggestion. Time spent trying to work out a reason for that suggestion is time wasted. 

While I struggle to see how Draftsmith can help an experienced editor, I think it could be a very useful tool for training early career editors. An inexperienced editor could work through each suggestion and explain (to themselves or a tutor) their own reasons for accepting or rejecting the edit, until that process is second-nature and they no longer need a tool for the prompts. 

Will I use it? Possibly. From what I can see, with enough experimentation, I might be able to tweak the settings enough for Draftsmith to give me the kind of edits I would find useful and timesaving (whether I can find the time to do that is another matter). And I might use it on my own writing for ideas and suggestions, but only where I know I will not have the benefit of a human professional editor to review my work.