sia.hackernoon.com

I've been thinking about making a multiplayer RPG web platform since my days as a Flash dev. There are plenty of online TTRPG options out there (many great!) but none of them felt quite right for how I'd like to play.

Have you seen the video where Deborah Ann Woll shows Jon Bernthal how to play D&D? (it has 2M views!)

https://www.youtube.com/watch?v=JpVJZrabMQE&embedable=true

That was so cool. What if I could make something that feels like that?

The Idea

A narrative-driven, multiplayer online rpg platform, light on rules but high on shared storytelling, with outcomes determined by the roll of a D20.

I've always loved D&D modules. The art. The exciting introduction. The detailed encounters and epic story plans for the players to uncover as their characters make their way through to a climactic finale.

What if I could create a way for people to make their own adventure modules, for any RPG genre, then let players run their characters through those adventures?

Who would run the games though? Game masters are hard to find. What if I could train an AI to be a game master? To actually run a well structured adventure plan (designed by a human!) that would be fun for players and not just a bunch of AI slop?

Getting Started

I've been having fun building lots of things with AI (see my project starter). I'll be using my preferred stack to make D20Adventures.com, a Next.js web app with Tailwind for UI (need it even be said?) powered by the AI SDK utilizing Gemini and a Convex database deployed to Vercel.

And I've decided to build it in the open, publishing the code on Github.

The Prototype

For a prototype, I'm literally starting from the scenario laid out on the podcast, a ranger walking through the woods at night hears a crack in the distance, which turns out to be an owlbear.

My aim is to build a short one-shot adventure and see if I can train an AI DM to actually run a small series of turns and encounters for this adventure. I fleshed out the scenario a bit with a backstory:

The Midnight Summons A mysterious summons from an old druid friend draws a reclusive ranger into the wilds of the Valkarr forest.

After a lot of trial and error, I was finally able to do a full play through and published it to YouTube:

https://youtu.be/9wzNVmhVMlg?si=qaNEJ8wcF4bzZJK9&embedable=true

How It Works

Landing Page

The landing page is a big hero image (generated on Midjourney with a prompt of “The Power of the D20”). I added some simple fade in animation using the new CSS @starting-style rule:

.fade-in {
  @apply opacity-100 transition-opacity duration-1000 ease-in-out;
  @starting-style {
    opacity: 0;
  }
}

Authentication

To play the Quick Start, I require a user account. This is to avoid getting charged lots of money because of anonymous people or bots using my APIs. Clerk makes it very simple to add user management, and I use them on all my projects.

Additionally, I have token usage tracking where I will limit usage with a token system, where you start out with enough tokens to do one play through of the demo, then can buy more as you go.

The First Turn

When the user lands on the adventure page for the quick start, the first thing that happens is we load the data for the demo adventure, which is just a simple JSON file (like this) on S3. An adventure module or plan in my system is made up of a series of encounters, that are linked to each other with instructions for the LLM:

"encounters": [
  {
    "id": "broken-silence",
    "title": "Broken Silence",
    "intro": "Thalbern, a solitary ranger of the Valkarr woods, has always trusted the silence of the wilds more than the promises of men. Orphaned by border raiders and raised by the elves of the Valkrarr Forest, he has spent years living on the edge of Kordavos, guiding travelers, hunting for his own survival, and keeping his distance from the tangled politics of the city.\n\nYet on this night, a message delivered by a red squirrel bearing the unmistakable script of Wollandora, a trusted elven friend and druid, has drawn him from his hidden home. The note was simple and urgent: Meet me at the Old Standing Stones at midnight. The balance of the forest could depend on it.\n\nNow, as midnight approaches, Thalbern moves quietly through the dense undergrowth, guided by memory and instinct. It is dark with almost no moonlight coming through the forest canopy.\n\nSuddenly, the hush of the night is broken by a sharp crack. Something large has just stepped on a branch somewhere off in the distance.",
    "instructions": "A perception check is appropriate if Thalbern investigates (low difficulty with a plus 3 modifier). If successful, he will determine it is a large creature that is approaching quickly. With a high roll (18+), he will determine it is an Owlbear. If combat ensues and Thalbern is below 25% health, Wollandora will intervene. If Thalbern avoids or defeats the Owlbear, or if Wollandora saves him, he proceeds to the Old Standing Stones.",
    "image": "images/settings/realm-of-myr/the-midnight-summons/broken-silence-2.png",
    "transitions": [
      {
        "condition": "If Thalbern successfully uses stealth to evade and proceeds cautiously towards the Standing Stones, go to meeting-at-stones.",
        "encounter": "meeting-at-stones"
      },
      {
        "condition": "If Thalbern fails a perception check, advance to owlbear-confrontation.",
        "encounter": "owlbear-confrontation"
      },
      {
        "condition": "If Thalbern fails any dice roll (including stealth, perception, or any other check), advance to owlbear-confrontation.",
        "encounter": "owlbear-confrontation"
      },
      {
        "condition": "If Thalbern does NOT successfully use stealth to evade, go to owlbear-confrontation.",
        "encounter": "owlbear-confrontation"
      },
      {
        "condition": "If Thalbern does nothing or takes no action, go to owlbear-confrontation.",
        "encounter": "owlbear-confrontation"
      },
      {
        "condition": "If Thalbern has a healthPercent of less than 50%, go to wollandora-intervention.",
        "encounter": "wollandora-intervention"
      }
    ]
  },
  {
    "id": "owlbear-confrontation",
    "title": "Owlbear Confrontation",
    "intro": "From the direction of the sound, a little bit of eye shine glints in the shadows of the tree line. A hulking fifteen foot tall monster with the body of a giant bear and the head of an owl. As it crashes out from the undergrowth, it lets out a guttural squawk, clearly agitated and territorial.",
    "instructions": "The Owlbear will attack. If Thalbern attempts an animal handling check (high difficulty) and succeeds, he can move past the Owlbear. If Thalbern wins initiative and attempts to hide, he can move past the Owlbear if he passes a medium difficulty stealth check. If Thalbern's health drops to a critical level, Wollandora appears and drives off the Owlbear, transitioning to 'wollandora-intervention'. If Thalbern defeats the Owlbear, describe his victory and transition to 'meeting-at-stones'.",
    "image": "images/settings/realm-of-myr/the-midnight-summons/owlbear-confrontation.png",
    "npc": [
      {
        "id": "owlbear",
        "behavior": "Aggressively attacks any perceived threat. Will fight until heavily wounded or driven off.",
        "initialInitiative": 1
      }
    ],
    "transitions": [
      {
        "condition": "Thalbern defeats the Owlbear, manages to evade it, successfully uses Animal Handling to pacify and move past it, or successfully rolls any other way to move past it.",
        "encounter": "meeting-at-stones"
      },
      {
        "condition": "Thalbern is reduced to critical health by the Owlbear.",
        "encounter": "timely-rescue"
      }
    ]
  },
  ...
]

The First Reply

Since we are in demo mode, there is no actual adventure created yet on the backend. That happens when the player makes the first reply. With every reply, we run it though a formatNarrativeAction function that uses AI to evaluate the reply to make sure it is grammatically correct, in the third person and adds dialogue or other prose to make it work well in a literary narrative style.

After the formatting is applied, the reply is sent to a server action. Because this is a demo, we are creating the adventure in the database when this first reply is received. After this is done, it will follow the normal flow for processing a player reply.

Processing Player Responses

The processTurnReply function loads the current turn from Convex, the adventure data from S3, and identifies the specific encounter and the character performing the action.

With this context, it then uses AI to determine whether the action is plausible (my son when play testing had the ranger launch a nuke at the owlbear) and, if so, whether a dice roll is mechanically required (e.g., an "Attack Roll" or a "Stealth Check"), including the type of roll and its difficulty.

We can do this with a function call, where we can specify to the AI that we want structured data returned, in this case a rollRequirementSchema:

import { z } from "zod";

export const rollRequirementSchema = z.union([
  z.object({
    rollType: z.string().describe("The type of roll required, e.g. 'Stealth Check'"),
    difficulty: z.number().describe("The difficulty class (DC) for the roll"),
    modifier: z.number().optional().describe("Bonus or penalty to the roll, e.g. +2 or -1"),
  }),
  z.null()
]);

export type RollRequirement = z.infer<typeof rollRequirementSchema>;

Then we have a function that sends a detailed prompt and the schema to generateObject:

export async function getRollRequirementForAction(action: string) {
  const prompt = `
Given the following player or NPC action, determine if a dice roll is required for the character to attempt the action. If a roll is required, return a JSON object with "rollType" (choose the most appropriate from the list below) and "difficulty" (a number between 5 and 25). If no roll is required, return the JSON value null (not a string).

Possible roll types:
- Perception Check
- Investigation Check
- Insight Check
- Stealth Check
...

Examples:
Action: "Try to sneak past the guards."
Result: { "rollType": "Stealth Check", "difficulty": 15 }

Action: "Attack the goblin."
Result: { "rollType": "Attack Roll", "difficulty": 12 }

Action: "Try to determine what the sound is."
Result: { "rollType": "Perception Check", "difficulty": 10 }

Action: "Say hello."
Result: null

Now, given the following action, determine the roll requirement.

Action: "${action}"
`;
  try {
    const result = await generateObject({
      schema: rollRequirementSchema,
      prompt,
    });
    if (
      result.object &&
      typeof result.object === "object" &&
      "rollType" in result.object &&
      (result.object.rollType === "null" || result.object.rollType === "none" || result.object.rollType === "")
    ) {
      return null;
    }
    return result.object ?? null;
  } catch (error) {
    throw error;
  }
}

After appending the the player’s response to the turn's story, if a dice roll is required we update the character's state in the Convex turn data to reflect that they've replied but their turn isn't complete. This update causes a realtime update to the UI showing the details of the roll and give the player a button to roll a D20.

Once the user rolls, we have a resolvePlayerRollResult server action that updates the narrative with a visual display of the result and writes prose describing the outcome. We also have another AI function call to update health and status for all the characters in the turn.

NPC Actions

When it is an NPCs turn to reply, we follow a similar pattern as a player, except in this case the AI will write the response. Every encounter includes information about the NPCs motivation and when combined with the context of the adventure narrative so far, hopefully the AI can generate a good reply, then generate its own dice roll and outcome update.

Training AI to Run RPGs

If you've ever tried to do a game session with AI in a chat, you know how quickly it can go off the rails.

Hopefully by providing the appropriate context, specific prompt instructions and structured data function calls, we can get an experience that is enjoyable.

There is definitely a lot of trial and error, console logging and prompt tweaking. For example, when asking why the AI did not follow the encounter instructions in one of my test runs, this is what Cursor chat had to say to me:

In summary, the LLM did not follow its explicit instructions to transition when a character fails a perception check, despite being provided with clear evidence of such a failure and a transition rule for that specific scenario. This seems to be a misapplication of the provided logic by the LLM.

I guess the randomness of the AI not doing exactly what is expected could be part of the fun. I suppose we'll find out!

Here is an example of one of the play sessions.

You can see the full source code for this project at github.com/johnpolacek/d20adventures.com

Can AI Run Your Next D&D Game? This Project Aims to Find Out

The Idea

Getting Started

The Prototype

How It Works