Basic Concepts

This page will help you get started with most of the basic functionality of the CUE Engine, as well as provide guidance through common problems you might encounter.

Sending a CUETrigger

Enable transmission for Android.

Before you start sending triggers, you will need to enable transmitting on the engine for Android. This should be done just after you authenticate with your API key. For iOS it's handled automatically.

Kotlin

CUEEngine.getInstance().isTransmittingEnabled = true;

Java

CUEEngine.getInstance().setTransmittingEnabled(true);

Transmission Modes

The engine supports three different modes for transmitting a CUE Trigger, sending a number or a string, sending low latency live triggers, and data mode for sending ASCII strings.

Sending a Number or Trigger String

A CUE Trigger can be queued as a number or a string. While they can be used interchangeably, it may be more convenient to use one over the other depending on your use case.

String triggers are of the form X.X.X where X is an integer from 0 - 461, and number triggers are integers between 0 and 98,611,127.

Kotlin

// Sending a trigger represented as a string 
val input: String = 123.123.123
CUEEngine.getInstance().queueTrigger(input)
  
// Sending a trigger as a number
val input: Long = 123456
CUEEngine.getInstance().queueTriggerAsNumber(input)

Java

// Sending a trigger represented as a string 
String input = "123.123.123";
CUEEngine.getInstance().queueTrigger(input);

// Sending a trigger represented as a number
long input = 12345678;
CUEEngine.getInstance().queueTriggerAsNumber(input);

Swift

// Sending a trigger represented as a string 
let input: String = "123.123.123"
CUEEngine.sharedInstance().queueTrigger(input)

// Sending a trigger represented as a number
let input: Int = 12345678
CUEEngine.sharedInstance().queueTrigger(asNumber: input)

Objective-C

// Sending a trigger represented as a string 
NSString *input = @"123.123.123";
[CUEEngine.sharedInstance queueTrigger:input];

// Sending a trigger represented as a number
long input = 12345678;
[CUEEngine.sharedInstance queueTriggerAsNumber:input];

Sending an ASCII String

ASCII strings can be transmitted in data mode, and one message cannot be longer than 64 characters. Data mode first transmits a header trigger, signaling to the receiver that it's about to receive a data mode string. There is a config setting that causes the engine to start in data mode by default, removing the need for the header trigger, increasing the bandwidth.

Kotlin

CUEEngine.getInstance().queueMessage(input)

Java

CUEEngine.getInstance().queueMessage(input);

Swift

CUEEngine.sharedInstance().queueMessage(input)

Objective-C

[CUEEngine.sharedInstance queueMessage:input];

Multi-Trigger

Using multi-trigger mode allows you to send larger numbers without sacrificing speed. This mode works by sending two CUE Triggers at once within two different frequency bands, and increases the largest integer that can be sent within a 1.0 second duration to 9,724,154,368,210,128. To send a multi-trigger, you first need to modify the engine config to use multi_trigger_only mode.

{
    "version"                 : "1.0.0",
    "info"                    : "",

    "mode"                    : "multi_trigger_only",
    "generation"              : 2,
    "freq_channel_0"          : 18500.0,
    "multi_trigger_carriers"  : [ 18500.0, 15500.0 ]
}

It's important to set the frequencies in multi_trigger_carriers at least 2.5KHz apart from one another to avoid self-interference. To use this modified config in your application, add it as an argument to setupWithAPIKeyAndConfig after microphone permission is given, just like you would normally. While this example has the config hardcoded in the application, we recommend hosting the config somewhere server-side providing you more flexibility if the config needs to change for one reason or another.

Kotlin

val config String = """
    {
        "version"                 : "1.23.0",
        "info"                    : "",

        "mode"                    : "multi_trigger_only",
        "generation"              : 2,
        "multi_trigger_carriers"  : [ 18500.0, 15500.0 ]
    } 
"""

// Call it after microphone permission is given
CUEEngine.getInstance().setupWithAPIKeyAndConfig(this, API_KEY, config);

Java

String config = "{\n" +
                "    \"version\"                 : \"1.0.0\",\n" +
                "    \"info\"                    : \"\",\n" +
                "    \"mode\"                    : \"multi_trigger_only\",\n" +
                "    \"generation\"              : 2,\n" +
                "    \"freq_channel_0\"          : 18500.0,\n" +
                "    \"multi_trigger_carriers\"  : [ 18500.0, 15500.0 ]\n" +
                "}";

// Call it after microphone permission is given
CUEEngine.getInstance().setupWithAPIKeyAndConfig(this, API_KEY, config);

Swift

let config: String = """
    {
        "version"                 : "1.23.0",
        "info"                    : "",

        "mode"                    : "multi_trigger_only",
        "generation"              : 2,
        "multi_trigger_carriers"  : [ 18500.0, 15500.0 ]
    }
"""

CUEEngine.sharedInstance().setup(withAPIKey: API_KEY, andWithConfig: config)

Objective-C

NSString* config = @"{\n"
                   @"    \"version\"                 : \"1.0.0\",\n"
                   @"    \"info\"                    : \"\",\n"
                   @"    \"mode\"                    : \"multi_trigger_only\",\n"
                   @"    \"generation\"              : 2,\n"
                   @"    \"freq_channel_0\"          : 18500.0,\n"
                   @"    \"multi_trigger_carriers\"  : [ 18500.0, 15500.0 ]\n"
                   @"}";

[CUEEngine.sharedInstance setupWithAPIKey:API_KEY andWithConfig:config];

Sending a Number

In most cases, you will likely be sending multi-triggers that represent long numbers.

Kotlin

// Sending a number as a multi-trigger, where your_number is between 0 and 9,724,154,368,210,128
val number Long = <your_number>;
CUEEngine.getInstance().queueMultiTriggerAsNumber(number);

Java

// Sending a number as a multi-trigger, where your_number is between 0 and 9,724,154,368,210,128
long number = <your_number>;
CUEEngine.getInstance().queueMultiTriggerAsNumber(number);

Swift

// Sending a number as a multi-trigger, where your_number is between 0 and 9,724,154,368,210,128
let number: Int64 = <your number>
CUEEngine.sharedInstance().queueMultiTrigger(asNumber: number)

Objective-C

// Sending a number as a multi-trigger, where your_number is between 0 and 9,724,154,368,210,128
long long number = <your number>;
[CUEEngine.sharedInstance queueMultiTriggerAsNumber:number];

Sending a Trigger String

However, there may be cases where you want to have more control over which symbols are sent. In this case, you can send a six symbol trigger string.

Kotlin

// Sending a multi-trigger trigger string, where X is between 0 and 461
val input String = "XXX.XXX.XXX.XXX.XXX.XXX"
CUEEngine.getInstance().queueMultiTrigger(input)

Java

// Sending a multi-trigger trigger string, where X is between 0 and 461
String input = "XXX.XXX.XXX.XXX.XXX.XXX";
CUEEngine.getInstance().queueMultiTrigger(input);

Swift

// Sending a multi-trigger trigger string, where X is between 0 and 461
let input: String = "XXX.XXX.XXX.XXX.XXX.XXX"
CUEEngine.sharedInstance().queueMultiTrigger(input)

Objective-C

// Sending a multi-trigger trigger string, where X is between 0 and 461
NSString* input = @"XXX.XXX.XXX.XXX.XXX.XXX";
[CUEEngine.sharedInstance queueMultiTrigger:input];

Receiving a CUETrigger

To receive a CUE Trigger, first and foremost microphone permissions need to be granted to the app before anything can happen. Next, you must implement the Receiver Callback provided by the engine interface. Finally, when you are ready to receive a trigger, the engine should be set to the listening state.

Kotlin

// Call it after microphone permission is given
CUEEngine.getInstance().setReceiverCallback { json: String ->

}

CUEEngine.getInstance().startListening()

...
...

// When you are done recording, it's good practice to stop listening
CUEEngine.getInstance().stopListening()

Java

// Call it after microphone permission is given
CUEEngine.getInstance().setReceiverCallback(new CUEReceiverCallbackInterface() {
  @Override
  public void run(@NonNull String json) {

  }
});

CUEEngine.getInstance().startListening();

...
...

// When you are done recording, it's good practice to stop listening
CUEEngine.getInstance().stopListening();

Swift

CUEEngine.sharedInstance().setReceiverCallback() { (json: String) -> () in

}

CUEEnigne.sharedInstance().startListening()

...
...

// When you are done recording, it's good practice to stop listening
CUEEnigne.sharedInstance().stopListening()

Objective-C

[CUEEngine.sharedInstance setReceiverCallback:
 ^void( NSString* jsonString ) {

}];

[CUEEngine.sharedInstance startListening];

...
...

// When you are done recording, it's good practice to stop listening
[CUEEnigne.sharedInstance stopListening];

Structure of a CUETrigger

Basic terms

Each time a CUE audio signal is detected by the device's audio input or microphone, a JSON payload is returned to your application within the ReceiverCallback block you provided to the CUEEngine shared instance.

There are two types of audio signal formats: a trigger message and a data message. The type of audio signal is stated in the mode field of the returned JSON.

Within a trigger audio signal, the id is encoded in the raw-indices as three symbols separated by a . character. E.g. "42.21.43", "1.2.34", etc., where each symbol is an integer between 0 and 461.

The data audio signal is encoded in the message field as a byte stream represented by a JSON string.

Example payloads are provided below.

Example Payloads

Trigger Payload

JSON

{
    "generation": 2,
    "latency_ms": 960.0,
    "mode": "trigger",
    "noise": 163.40673828125,
    "payload": {"myKey":"myValue"},
    "power": 69682.734375,
    "raw-indices": "1.32.45",
    "trigger-as-number": 228273,
    "winner-indices": "1.32.45"
}

Data Payload

JSON

{
    "generation": 2,
    "latency_ms": 6375.0,
    "message": "hello world",
    "mode": "data",
    "payload": {...}
}

Description

Basic Parameters

• "mode":

  1. "trigger":

     Intended for transmitting content IDs or small amounts of data over short to long distances. Each trigger consists of a three-symbol ID (each symbol ranging from 0 to 461).

     • payload size: 26 bits
     • latency: ~1.0 seconds

  2. "multi_trigger":

     This is multiple triggers played simultaneously, if you have modified your config to increase the bandwidth to support a higher throughput. With a larger bandwidth, multi_trigger mode tends to be more audible than the default ultrasonic trigger mode.

  3. "data":

     For transmitting arbitrarily large data payloads over short to medium distances. Transmission occurs at a rate of 26bps (or 52bps using multi-triggers). Payload size varies according to signal duration.

     bandwidth: 26 bits/sec
     latency: N/A (varies according to number of packets in message)

• "latency_ms":

  Time in milliseconds since start of the CUE message decoding process

• "raw-indices":

  The "symbol string" or "indices" of the detected trigger (e.g., "1.2.3").

• "winner-indices":

  The same as "raw-indices"

• "trigger-as-number":

The default format of triggers (i.e., XXX.XXX.XXX where XXX is an integer between 0 and 461) can be non-intuitive for some use cases (e.g., audio ticketing, where each ticket is a 10-digit number). That is why each CUETrigger object contains a method that returns the trigger in an integer format. You can convert a trigger to an integer by calling getTriggerAsNumber() . You can transmit a trigger as an integer by calling queueTriggerAsNumber(number). See examples here:

Kotlin

// Firstly we set a callback to handle a detected `trigger`.
// And only after that we we will do an actual sending.
CUEEngine.getInstance().setReceiverCallback { json: String ->
		
    // When a `trigger` is detected, convert a `trigger` to a number
    val CUETrigger trigger = CUETrigger.parse(json)
    val triggerNum: Long = trigger.getTriggerAsNumber

    ...
    ...

}

// Send a trigger representing a number
// which will be handled in a callback defined upper
CUEEngine.getInstance().queueTriggerAsNumber(number)

Java

// Firstly we set a callback to handle a detected `trigger`.
// And only after that we we will do an actual sending.
CUEEngine.getInstance().setReceiverCallback(new CUEReceiverCallbackInterface() {
    @Override
    public void run(@NonNull String json) {
        // When a `trigger` is detected, convert a `trigger` to a number
        final CUETrigger model = CUETrigger.parse(json);
        final long triggerNum = model.getTriggerAsNumber();
    }
});

...
...

// Send a trigger representing a number
// which will be handled in a callback (defined upper)
CUEEngine.getInstance().queueTriggerAsNumber(number);

Swift

// Firstly we set a callback to handle a detected `trigger`.
// And only after that we we will do an actual sending.
CUEEngine.sharedInstance().setReceiverCallback() { (json: String) -> () in

    // When a `trigger` is detected, convert a `trigger` to a number
    let trigger: CUETrigger = CUETrigger.init(jsonString: json)
    let triggerNum: Int64 = trigger.triggerAsNumber;
}

...
...

// Send a trigger representing a number
// which will be handled in a callback defined upper
[CUEEngine.sharedInstance().queueTrigger(asNumber:number];

Objective-C

// Firstly we set a callback to handle a detected `trigger`.
// And only after that we we will do an actual sending.
[CUEEngine.sharedInstance setReceiverCallback:
 ^void( NSString* jsonString ) {
    // When a `trigger` is detected, convert a `trigger` to a number
    CUETrigger *trigger = [[CUETrigger alloc] initWithJsonString:jsonString];
    long long triggerNum = [trigger triggerAsNumber];
}];

...
...

// Send a trigger representing a number
// which will be handled in a callback defined upper
[CUEEngine.sharedInstance queueTriggerAsNumber:number];

Advanced Parameters

The following parameters are only needed for advanced metrics, such as estimating distance from the audio source.

• "power":

  Log (base 10) of the median channel strength and noise level

• "noise":

  A measure of the noise not contributing to the signal or boosting the SNR.