API Reference

All Neocortex components are Blueprint-accessible. This reference covers the main components of the plugin; every function and event shown here can be used from both C++ and Blueprints.

Neocortex Smart Agent

UNeocortexSmartAgent is the primary component for AI character interactions. Add it to an actor, set the Project ID, and call its send functions. It automatically manages session state and service connections.

Properties

Functions

SendMessage

Sends a text message to the agent and receives a text response via OnChat.

• Parameters:
  • Message: The message to send.

SendMessageForAudio

Sends a text message to the agent and receives both text (OnChat) and audio (OnAudio) responses.

• Parameters:
  • Message: The message to send.

TranscribeBytes

Transcribes audio data to text, delivered via OnTranscribed.

• Parameters:
  • Data: Audio data in WAV format.

SendAudioForAudio

Transcribes audio, then sends the result as a message, receiving both text and audio responses, the full voice-in / voice-out flow. Equivalent to Unity's AudioToAudio.

• Parameters:
  • WavData: Audio data in WAV format.

GetChatHistory

Retrieves the chat history for this agent's character via OnChatHistory.

• Parameters:
  • Limit: Maximum number of messages to retrieve (default 10).

ClearSessionId

Clears the session ID to start a new conversation.

IsRequestPending

Returns true while a request is in flight or a beats reply is still playing.

IsSpeaking

Returns true while a reply is pending or its beats are still being played back (beats modes).

Events

Chat Beats

Set Beats Mode to have replies delivered as ordered per-emotion "beats" instead of one block of text:

| Mode | Behavior | | ---- | -------- | ---- | | Off | Normal single reply. Default. | Unchanged | | TextAndEmotion | Reveal each beat as its own message and change emotion per beat. | | ExpressiveAudio | Also voice each beat in its own emotion. |

In a beats mode, input arriving while the character is still speaking is queued and submitted once the current reply finishes, there is no barge-in.

Emotions

The Emotion values received in chat responses and beat events cover 16 states, organized by intensity quadrants (calm/energetic and positive/negative):

Neutral, Happy, Pleased, Disappointed, Upset, Amazed, Curious, Confused, Alarmed, Fascinated, Impressed, Annoyed, Angry, Confident, Reassured, Concerned, Scared

---

Neocortex Microphone Recorder

UNeocortexMicrophoneRecorderComponent is a Blueprint-accessible component for capturing microphone input and optionally transcribing speech. It can automatically send recordings to a Smart Agent for transcription.

Properties

Functions

StartRecording

Begins capturing audio from the default microphone device. Returns true if recording started successfully, false if permission was denied or the device is unavailable.

StopRecording

Stops capturing audio and optionally sends the result to the Smart Agent for transcription.

• Parameters:
  • bAutoTranscribe: If true and Smart Agent is set, automatically transcribe the recording (default true).

IsRecording

Returns true if currently capturing audio.

GetAmplitude

Returns the RMS amplitude of the most recent audio chunk (0–1). Useful for driving amplitude UI.

GetAvailableDeviceNames

Returns display names of all available audio input devices.

SetPreferredDevice

Sets the preferred input device by name (must match a name from GetAvailableDeviceNames). Takes effect immediately, stops and restarts the capture device if recording.

Events

---

Neocortex Interactable

UNeocortexInteractableComponent marks an actor as interactable within the Neocortex world state. It automatically registers with the subsystem on BeginPlay and unregisters on EndPlay, providing contextual information to AI agents about objects in the world.

Properties

Enable Include All Interactables on the Smart Agent to send registered interactables (optionally filtered by Search Radius) with each chat request.

---

Neocortex Event Logger

UNeocortexEventLogger is a Blueprint function library for recording game events that give the character context about what recently happened in your game. The event log is sent with each chat request and consumed (cleared) automatically when sent.

Storage lives in the Neocortex subsystem, so it is isolated per GameInstance (PIE-safe). It holds at most 20 events with a maximum of 64 characters each, sorted by priority (High first, newest-first within a priority).

Functions

PushEvent

Records a game event. Content is silently truncated to 64 characters; ignored when the log is full.

• Parameters:
  • Priority: Low, Medium, or High.
  • Content: A short description of the event.

ClearEvents

Clears all recorded events.

---

Neocortex Session Manager

UNeocortexSessionManager manages persistent session IDs for characters across gameplay sessions. Session data is cached in memory, persisted to configuration files, and thread-safe for concurrent access.

Functions

Get

Retrieves the session ID for a given character. Returns the stored session ID, or an empty string if not found.

• Parameters:
  • ProjectId: Unique identifier for the Neocortex Smart NPC character.

Set

Stores or updates the session ID for a character.

• Parameters:
  • ProjectId: Unique identifier for the character.
  • SessionId: Session ID to associate with the character.

Clear

Removes the session ID for a character, effectively starting a new conversation history.

• Parameters:
  • ProjectId: Unique identifier for the character.

---

UI Widgets

The plugin ships with ready-made UMG widgets used by the sample maps, which you can reuse in your own UI:

Sample Maps

Sample maps are located at Plugins/Neocortex/Content/Samples: