Group Chat

Put several AI characters in one conversation in Unreal Engine 5 with the Neocortex Group Director, each with its own voice, joining, leaving and talking to each other.

Pro or Team plan

A cast of two or more characters needs a Pro or Team plan. A cast of one works like a normal single character chat on any plan.

UNeocortexGroupDirector runs one conversation across several characters. Each character keeps its own Smart Agent, with its own voice, animation and events. The director sends one turn for the whole cast and plays every reply through the character that said it, one after the other. The cast can be up to three characters.

How the server runs a group scene is covered in Web API Group Chat.

Setting up

Add the characters

Give each character actor its own Neocortex Smart Agent and pick a character with the Pick button next to its Project ID. Each agent plays its own lines on its own Audio Component, so the voices come from the right place.

Add the director

Add a Neocortex Group Director component to any actor and add the agents to its Agents array, or call AddAgent for each one.

Add the UI (optional)

UNeocortexGroupChatUI is the chat for a group: on the left over a fading backdrop, each speaker's turn a bubble with the character's name, and a text field and voice controls under it.

UNeocortexGroupChatUI* Chat = CreateWidget<UNeocortexGroupChatUI>(PlayerController, UNeocortexGroupChatUI::StaticClass());
Chat->SetDirector(Director);
Chat->SetMicrophone(Microphone); // optional: adds the voice controls
Chat->AddToViewport();

Use one group chat for the whole cast. Don't also give each character a UNeocortexChatUI, or every reply shows up twice.

Talking to the group

Call	Who answers
`Send(Message)`	The cast decides who answers, up to `MaxTurns` speakers.
`SendTo(Agent, Message)`	That character.
`SendToCharacter(CharacterId, Message)`	The character with that id.
`ContinueScene()`	Nobody is addressed: the characters talk among themselves.
`SendAudio(WavData)`	Transcribes the player's recording, raises `OnPlayerSpeech` with the text, then runs it like `Send`.

GroupScene.cpp

void AGroupScene::BeginPlay()
{
    Super::BeginPlay();
    Director->OnTurnStarted.AddDynamic(this, &AGroupScene::LockInput);
    Director->OnTurnFinished.AddDynamic(this, &AGroupScene::UnlockInput);
    Director->OnSpeaker.AddDynamic(this, &AGroupScene::ShowSpeaker);
    Director->OnRequestError.AddDynamic(this, &AGroupScene::HandleError);

    Director->Send(TEXT("Hi everyone, introduce yourselves"));
}

void AGroupScene::ShowSpeaker(const FNeocortexGroupMessage& Message)
{
    FString Text;
    for (const FNeocortexChatLine& Line : Message.Lines) Text += Line.Text;
    Subtitles->ShowLine(Message.Name, Text);
}

void AGroupScene::HandleError(const FNeocortexRequestError& Error)
{
    if (Error.IsForbidden()) { UE_LOG(LogTemp, Warning, TEXT("Group chat needs a Pro or Team plan")); }
}

void AGroupScene::AskAria()
{
    Director->SendTo(Aria, TEXT("Aria, what do you think?"));
}

A turn plays out in order. OnTurnStarted fires first, then OnGroupResponse with the whole turn, then OnSpeaker before each character speaks its part. Each agent raises its own OnChatLineStarted, OnEmotionChanged and OnReplyFinished as it speaks, so your animation code works the same as for a single character. OnTurnFinished fires once everyone is done, and also when a turn could not run.

Input sent while a turn is running is ignored with a warning in the log. IsBusy() tells you whether a turn is running: lock your input on OnTurnStarted and open it again on OnTurnFinished. Abort() cancels the turn and silences every character.

MaxTurns (1 to 5, default 5) caps how many characters speak in one turn.

Actions in a group

Actions are registered on each agent, not on the director. When a character asks for an action in its part of the turn, its own agent runs the handler.

Per-character actions

Aria->RegisterAction(TEXT("WAVE"), [this](const FNeocortexChatAction& Action, TFunction<void()> Finish)
{
    PlayWave(AriaMesh, MoveTemp(Finish));
});

Changing the cast

Characters can join and leave while the conversation runs:

A character walks in

void AGroupScene::OnBramArrived()
{
    Director->AddAgent(Bram);
    Director->ContinueScene(); // the others notice the newcomer
}

RemoveAgent(Agent) takes a character out of the cast. An ambient turn with ContinueScene() right after a change lets the cast react straight away.

History and sessions

The director keeps one shared session in SessionId. It starts empty and is set by the first turn. It lives in memory only, so store it yourself if you want to resume the scene later. ClearSession() starts a new conversation on the next turn.

RequestHistory(Limit, Before) loads a page of the shared transcript, every speaker oldest first, on OnHistory. Pass the previous page's cursor as Before to page back.

The full list of members and events is in the API Reference. The Group Chat sample shows characters joining and leaving.

On this page