I promise that this column was written by me and not by artificial intelligence. The fact that I have to write such a sentence says volumes about where we are at the moment.
Here’s another one of those volumes: You can get a weekly email summarizing what happened at the latest Concord City Council meeting that was, to use an advertising phrase from my youth, untouched by human hands.
The free newsletter is from a Maine startup called Civic Sunlight that uses Large Language Models like ChatGPT to either save local journalism, expand local journalism or destroy local journalism, I can’t decide which. The company just launched in October and is still mostly an experiment by two tech guys who moved to coastal Maine from the big city because of COVID.
The two men, Tom Cochran and David Mortlock, met at Phillips Exeter Academy in the mid-1990s but lost touch and then bumped into each other by accident in Maine. As they re-connected they found that they shared a common problem: It was hard to know what the government in their small town was doing – not because the information was hidden but because there was too much of it scat tered all around.
“Transparency is not the problem. Accessibility to this abundant information is the problem,” Cochran said.
Even though governments now do things like video-record important meetings and put them online, he said, who has time to sit through them and figure out what’s important to you? Almost nobody, as you can tell from statistics about the tiny number of people who actually click on them.
This is where news reporters, like me, come in.
Shrinking journalism
I have spent a big chunk of my adult life in town halls and schools listening to people talk about easements and budget items and attendance patterns and other stuff that would make you fall asleep, which I then turn into glittering prose poems about the inner workings of democracy.
But as you know, dear reader, the internet has throttled the business model which supports local journalism, and the number of reporters who are paid to sift through all those government meetings is shrinking in New Hampshire and everywhere else.
“I was talking to the town council in Belfast (Maine) and I asked the guys there, do reporters cover this? They said 15 years ago we’d have two or three, 25 years ago they’d have four,” Cochran said. “With consolidation, the number of media went down to one. Then came streaming, COVID. Now it’s zero.”
Civic Sunlight aims to fill that gap, sort of.
It scrapes the transcripts of meetings of city councils, select boards and any other bodies posted on municipal websites and runs the text through a series of LLMs to produce an email newsletter summarizing what happened at the meeting. The project started in Camden, Maine, and is expanding fast. It’s available for free for government meetings from Concord and Nashua, 24 towns and cities in Maine and a handful of places in Massachusetts, Vermont, New York, New Jersey and Pennsylvania.
The result is what anybody who has played with ChatGPT would expect: Readable, concise, workmanlike. And sometimes dead wrong.
“Our first experiment we assumed ‘Oh this is simple, just take a transcript and run it through Chat GPT,’” Cochran said. “But it would be riddled with incredible amounts of errors and hallucinations, and you realize it’s pretty different than we expected.”
Database limitations
A quick tutorial: At their heart, Large Language Models are just extremely sophisticated autocorrect machines. Based on what has been written or spoken in all the material placed in their database, they use probability tables and other mathematical models to calculate what word a human would be likely to use next.
The output depends both on their math models and, importantly, what’s in their database. Since LLMs don’t actually know anything they don’t realize what part of their data is relevant and don’t have any context from outside the database. This can result in some extreme errors.
A case in point: Civic Sunlight’s Concord newsletter on Nov. 12 said the City Council “has approved funding for Memorial Field and the Penacook Library Activity Center.” Sorry, no. The council merely discussed the field, which has yet to be funded, and mentioned the library, which was funded years ago and recently opened.
“This is a prime example of the A.I. making things up when it doesn’t know what to say. Both of those topics were featured at the meeting but (Civic Sunlight), not knowing any context, clearly wasn’t sure how to summarize and so it just made something up,” said Catherine McLaughlin, our city reporter who covered that meeting.
Cochran said much of the work getting Civic Sunlight ready involved reducing such errors, partly by feeding the scraped transcripts through a series of specialized LLMs.
“We use eight different (LLMs) and proprietary code. It’s it’s kind of like cooking; we have different ingredients for different results,” Cochran said. He admits this is still a work in progress.
Is this a business?
Interesting as this might be, Cochran and Mortlock aren’t spending their own money creating Civic Sunlight for the love of it. They’re looking for a business model.
The use of A.I. means that scaling up in size, often a weak point for start-ups, is pretty easy. Cochran said it only costs about $25 per meeting to add a summary to a newsletter, far cheaper than covering it with a human presence. So they wouldn’t need a lot of income to justify adding more towns to the coverage area, assuming they can figure out how to make any income.
“We’re still debating on a business plan. Are we selling a public-facing newsletter that we can monetize or are we selling a wire service?” Cochran said. “It would be nice to have a sustainable business, at least in all of Maine and a couple of other states. Our market is not big cities. Our market is small communities where there’s a dearth of coverage.”
So far their only customer is a Maine newspaper that is using Civic Sunlight as a wire service, scanning through the newsletter for story ideas. Newspapers of New England, the Monitor’s family-owned parent company, is talking about whether A.I. might have a similar role for us down the road, to help keep track of what’s happening in the dozens of communities we cover.
The help would be limited since we would never publish the output of a software that hallucinates so much. I’m also nervous about A.I.’s massive energy consumption. But it’s not out of the question that LLMs or other iterations of A.I. might have a role in our work at some point.
Which leads to a question about using software to sort of do what reporters actually do. Is that good or bad for local journalism and, indirectly, for democracy?
Speaking as somebody who has watched newsrooms hollowed out to feed corporate profit I can definitely see A.I. becoming a “not good but good enough” replacement for human reporters. There are already online pseudo-news sites that use A.I. to churn out cheap, fast clickbait, sometimes with a side helping of deliberate misinformation.
But I can also see A.I. boosting local newsrooms by doing some of the info-gathering grunt work that takes up so much time, allowing people to do more journalism of the sort that can’t be captured by algorithm.
In a nation increasingly filled with “news deserts” – places with no local media coverage of any sort – that might not be bad. Cochran says it’s one of the possibilities that has spurred development of Civic Sunlight.
“We know that this or any other technology is not going to save things. The goal is to make it a little bit better than it was, to make a sizable number of people slightly more informed on the things that matter to them, based on facts rather than whatever rumor they heard from their neighbors.”
Return to the Concord Monitor