Interactables
Tell characters about your scene and act on what they decide to do
Send the scene with each turn; actions come back with the targetId of the entity they apply to. Fields and limits: Web API Interactables.
Context from the server
Recommended. Works with every element and agent.
createNeocortexHandler({
// ...
buildContext: async (user, turn) => ({
metadata: (await shop.stockFor(user.id)).map((item) => ({
id: item.sku,
type: "OBJECT",
name: item.name,
properties: [{ name: "price", value: `${item.price} coins` }],
})),
events: [{ priority: 1, content: `Customer has ${await wallet.balance(user.id)} coins` }],
}),
});turn is { characterIds, message, request }.
Context from the browser
For worlds that only exist in the page. Requires allowClientContext: true on the proxy, otherwise it is silently dropped. Anything sent from the browser can be edited by the player.
import { EventLogger, Interactables, SmartAgent } from "@neocortex-link/web/client";
const scene = new Interactables();
const events = new EventLogger();
const agent = new SmartAgent({ endpoint: "/api/neocortex", characterId: "guard-id", interactables: scene, eventLogger: events });
scene
.set({ id: "guard", type: "CHARACTER", name: "Guard", characterId: "guard-id", position: guard.position }) // itself
.set({ id: "door-1", type: "OBJECT", name: "Cellar door", position: door.position, properties: [{ name: "locked", value: "true" }] })
.focus("door-1"); // what the player is looking at
events.push("The player picked the vault lock", 2);Both are read when each turn is sent. Keep ids stable, and send only what's nearby: the proxy caps a turn at 40 entities and 20 events, keeping server context first.
Actions
agent.registerAction("OPEN_DOOR", (action) => doors.get(action.targetId)?.open());
chat.agent.registerAction("GO_TO", (action) => npc.walkTo(world.get(action.targetId))); // on an elementNames are case-insensitive. targetId is "" for untargeted actions. Every action, handled or not, also fires action on the agent and nx-action on elements.
When handlers run is set by actionTrigger:
| Value | Runs |
|---|---|
"whenResponseReceived" | When the reply arrives. Default. |
"whenSpeechStarts" | When the first line starts. |
"afterReplySpoken" | After the last line. |
Emotions
agent.on("emotionChanged", ({ emotion }) => avatar.play(emotion));The emotion list is under SmartAgent.