Turn a Story into a Comic with Lore Machine

Twelve years ago, author Will Douglas Heaven wrote a humorous short story about a man who was obsessed with luxury cat food in his spare time. Now, a generative AI platform called Lore Machine has turned this interesting story into a comic.

The author inserted his story into the text box and received a message: “ We are identifying scenes, locations, characters, and emotions. This may take up to 2 minutes .” Lore Machine analyzed the text, extracted descriptions of the characters and locations mentioned, and then passed that information to an image-generating model.

An illustrated storyboard appears on the screen. As the author clicks through the vivid comic illustrations of his almost forgotten characters, it's a wonder.

After more than a year of development, Lore Machine is now available to the public for the first time. For $10 a month, you can upload 100,000 words (up to 30,000 words per upload) and create 80 images for short stories, scripts, podcasts, and more. There are also tiers for power users, including a $160/month enterprise plan with 2.24 million words and 1,792 images. The illustrations come in a variety of preset styles, from manga to watercolor to ’80s TV.

Zac Ryder, founder of creative agency Modern Arts, has been using the early access version of the tool since Lore Machine founder Thobey Campion first showed Ryder what it could do. Ryder submitted a short film script to the system and Lore Machine turned it into a 16-page comic book overnight.

“I remember Thobey sharing his screen. We were all completely blown away,” Ryder said. “It wasn’t just about the visual aspect, but the storytelling: From the story line to the emotions of the characters, everything was perfect from the start.”

Modern Arts is currently using Lore Machine to develop a fictional universe for a comic book series — a work based on a script written by the creator of Netflix's Love, Death & Robots .

Lore Machine is made from familiar components:

• A large language model (LLM) scans your text, identifying descriptions of people, places, as well as general sentiment.
• A Stable Diffusion version will generate the image.

What sets Lore Machine apart, however, is how easy it is to use. Uploading your story and downloading the results takes just a few clicks.

That makes Lore Machine a prime example of a new wave of tools: user-friendly yet hiding incredible power behind an incredibly simple web interface.

“There’s a lot of work to be done to update new AI tools, and the interfaces and workflows are different for each tool ,” said Ben Palmer, CEO of New Computer Corporation, a content creation company. “There’s a lot of appeal to using one big tool with a consistent user interface. I feel like that’s where the industry will find its footing.”

Campion founded the company behind Lore Machine two years ago to develop a blockchain version of Wikipedia. But Campion changed direction when he saw how people were applying generative models. Campion used the free text-to-image model Midjourney to create a comic version of Samuel Taylor Coleridge’s The Rime of the Ancient Mariner . He says the work went viral, but the process of making it was no fun.

“My wife hated that project ,” Campion says. “I was up until 4 a.m. every night trying to get the images perfect .” The problem is that text-to-image models like Midjourney create each image one by one. That makes it difficult to maintain consistency across different images of the same character. Even locking in a specific style across multiple images can be difficult.

The experience made him realize that the technology needed to be much easier to use. Campion wouldn’t say exactly how Lore Machine manages to keep its imagery and style consistent across a series of illustrations. It’s pretty good, but not perfect: In one scene in one of his stories, a short-haired character suddenly gets bangs, and in another scene, the same character appears twice—in a different place.

The illustrations also started to feel generic. The author was disappointed that many of the character descriptions were very specific, but not shown in the illustrations. However, compared to doing each panel by hand, Lore Machine was a big step forward.

“The consistency is remarkable,” says Ryder . It gave Ryder’s company Modern Arts enough confidence to use Lore Machine on a project with a client. “ If we kept having to go back and fix consistency issues, there was no way we could deliver on time,” he says .

Like all generative models, the technology behind Lore Machine can generate harmful content on demand. Campion says they were trying to prevent the creation of images depicting violence or hate — but at the same time, they didn’t want to limit the creative expression of artists.

Much of the initial interest in Lore Machine has come from marketing companies. But Campion hopes that a wider release will encourage more people to try it. Six months ago, Campion said, he got a call from the principal of a school for children with disabilities in Manhattan. The principal wanted to run his textbooks through the tool so the kids could see the images. “ I hadn’t even thought about it. I was so stuck in the Hollywood mindset ,” Campion said.