Neocortex Smart Agent

The Neocortex Smart Agent component is the primary interface for interacting with your Neocortex project. It allows you to send and receive text and audio messages, manage chat sessions, and retrieve chat history.

Methods

TextToText

Send a text message to the Neocortex project, and expect a text response.

• Parameters:
  • message: The text message to send.

  var smartAgent = GetComponent<NeocortexSmartAgent>();
  smartAgent.OnChatResponseReceived.AddListener((response) =>
  {
      Debug.Log($"Message: {response.message}");
      Debug.Log($"Action: {response.action}");
      Debug.Log($"Emotion: {response.emotion.ToString()}");
      Debug.Log($"Conversation Flow State: {response.flowState}");
  });
  smartAgent.TextToText("Hello, Neocortex!");

TextToAudio

Send a text message to the Neocortex project, and expect a audio response.

• Parameters:
  • message: The text message to send.

  var audioSource = GetComponent<AudioSource>();
  var smartAgent = GetComponent<NeocortexSmartAgent>();
  smartAgent.OnChatResponseReceived.AddListener((response) =>
  {
      Debug.Log($"Message: {response.message}");
      Debug.Log($"Action: {response.action}");
      Debug.Log($"Emotion: {response.emotion.ToString()}");
      Debug.Log($"Conversation Flow State: {response.flowState}");
  });
  smartAgent.OnAudioResponseReceived.AddListener((audioClip) =>
  {
      audioSource.clip = audioClip;
      audioSource.Play();
  });
  
  smartAgent.TextToAudio("Hello, Neocortex!");

AudioToText

Sends an audio clip to the Neocortex project. This method is used with NeocortexAudioReceiver component to send audio data.

• Parameters:
  • audioClip: The audio clip to send.

  var smartAgent = GetComponent<NeocortexSmartAgent>();
  smartAgent.OnTranscriptionReceived.AddListener((message) =>
  {
      Debug.Log($"You: {message}");
  });
  
  var audioReceiver = GetComponent<NeocortexAudioReceiver>();
  audioReceiver.OnAudioRecorded.AddListener((audioClip) =>
  {
      Debug.Log($"Audio Data Length: {audioClip.samples}");
      smartAgent.AudioToText(audioClip);
  });
  
  // Start recording audio for 3 seconds
  audioReceiver.StartMicrophone();
  await Task.Delay(3000);
  audioReceiver.StopMicrophone();

AudioToAudio

Sends an audio clip to the Neocortex project and expects an audio response. This method is used with NeocortexAudioReceiver component to send audio data.

• Parameters:
  • audioClip: The audio clip to send.

  var audioSource = GetComponent<AudioSource>();
  var smartAgent = GetComponent<NeocortexSmartAgent>();
  smartAgent.OnAudioResponseReceived.AddListener((audioClip) =>
  {
      audioSource.clip = audioClip;
      audioSource.Play();
  });
  smartAgent.OnTranscriptionReceived.AddListener((message) =>
  {
      Debug.Log($"You: {message}");
  });
  smartAgent.OnChatResponseReceived.AddListener((response) =>
  {
      Debug.Log($"Message: {response.message}");
      Debug.Log($"Action: {response.action}");
      Debug.Log($"Emotion: {response.emotion.ToString()}");
  });
  
  var audioReceiver = GetComponent<NeocortexAudioReceiver>();
  audioReceiver.OnAudioRecorded.AddListener((audioClip) =>
  {
      Debug.Log($"Audio Data Length: {audioClip.samples}");
      smartAgent.AudioToAudio(audioClip);
  });
  
  // Start recording audio for 3 seconds
  audioReceiver.StartMicrophone();
  await Task.Delay(3000);
  audioReceiver.StopMicrophone();

GetChatHistory

Returns the last chat messages with given limit if there was a chat session on user device.

• Parameters:
  • limit: The number of last messages to fetch.

  var smartAgent = GetComponent<NeocortexSmartAgent>();
  smartAgent.OnChatHistoryReceived.AddListener((messages) =>
  {
      foreach(var message in messages){
        Debug.Log($"{message.sender}: {message.content}");
      }
  });
  smartAgent.GetChatHistory(5);

GetSessionID

Returns the string Chat Session ID generated for the current or previous chat. This string is saved in PlayerPrefs with neocortex-session-id key. If there was no previous chat session, or the session id is cleaned it will return empty string.

var smartAgent = GetComponent<NeocortexSmartAgent>();
var sessionId = smartAgent.GetSessionID();

CleanSessionID

Cleans the session id which essentially helps start a new chat with no previous history.

var smartAgent = GetComponent<NeocortexSmartAgent>();
smartAgent.CleanSessionID();

Events

OnChatResponseReceived

Event that is triggered when the Neocortex project responds to a text message.

• Arguements:
  • response: The response from the Neocortex project.

  var smartAgent = GetComponent<NeocortexSmartAgent>();
  smartAgent.OnChatResponseReceived += (response) =>
  {
      Debug.Log($"Message: {response.message}");
      Debug.Log($"Action: {response.action}");
      Debug.Log($"Emotion: {response.emotion.ToString()}");
  };

OnTranscriptionReceived

Event that is triggered when the Neocortex project transcribes an audio message to text.

• Arguements:
  • message: The transcribed audio message.

  var smartAgent = GetComponent<NeocortexSmartAgent>();
  smartAgent.OnTranscriptionReceived += (message) =>
  {
      Debug.Log($"You: {message}");
  };

OnAudioResponseReceived

Event that is triggered when the Neocortex project responds with an audio message.

• Arguements:
  • audioClip: The audio clip received from the Neocortex project.

  var audioSource = GetComponent<AudioSource>();
  var smartAgent = GetComponent<NeocortexSmartAgent>();
  smartAgent.OnAudioResponseReceived += (audioClip) =>
  {
      audioSource.clip = audioClip;
      audioSource.Play();
  };

OnChatHistoryReceived

Event that is triggered when a chat history of the current or previous session is received.

• Arguements:
  • messages: An array of past messages.

  var smartAgent = GetComponent<NeocortexSmartAgent>();
  smartAgent.OnChatHistoryReceived += (messages) =>
  {
      foreach(var message in messages){
        Debug.Log($"{message.sender}: {message.content}");
      }
  };

OnRequestFailed

Event that is triggered when a request to the Neocortex project fails.

• Arguements:
  • error: The error message.

  var smartAgent = GetComponent<NeocortexSmartAgent>();
  smartAgent.OnRequestFailed += (error) =>
  {
      Debug.LogError(error);
  };