TrimbleCloud.Events

Events Client library is a lightweight module for Producing and Consuming events through a websocket connection. It is capable of:

• Sending and receiving events through a websocket connection at higher TPS.
• Retrying if the connection is lost or interrupted and ensuring events are not getting lost.
• Sending statistics (total events produced and consumed, time taken to publish…etc) to Datadog and other metrics systems.
• Supporting Authentication with TID V4.
• Pause polling based on response queue size.

Getting Started

Installation

dotnet add package trimblecloud.events

This library is available in Trimble Artifactory.

Prerequisites

For an application to access the Events server:

1. The application must be registered in the Trimble Cloud Console.
2. You must authorize the application to publish to the event topic.
3. Use the Client Credential Grant Type to get token.

Compatibility

Compatible with .NET standard 2.1 and above.

Client Configuration

EventsClientConfig

Key	Description	Value	Mandatory/Optional
Endpoint	Provide Events web socket (publisher/consumer) url	String Example: wss://TBU:1000	Mandatory
ConnectionRetry	Provide maximum retry count	int	Mandatory
AuthUrl	Provide TID authorization url	Staging: https://stage.id.trimblecloud.com/oauth/token Production: https://id.trimble.com/oauth/token| Mandatory
ClientID	Provide client Id	<CLIENT_ID>	Mandatory
ClientSecret	Provide client secrets	<CLIENT_SECRET>	Mandatory
TidScope	Set the value of TID Scope for the application. Applicable for clients that require TID scope for authentication.	String	Mandatory
Workflow	Specify workflow type	Workflow.PUBLISHER / Workflow.CONSUMER	Mandatory
EnableBatch	To enable internal batching. By default it is false. If it is set as False, batching and batch retry has to be handled by the client.	boolean	Required only if Workflow is PUBLISHER
BatchBufferSize	Specify batch size. Minimum value is 1.	int	Required only if EnableBatch is true
BatchBufferMS	Specify batch buffering time in milliseconds. Minimum value is 1.	int	Required only if EnableBatch is true
BatchExpiryMS	Specify batch cache expiry in milliseconds. Minimum value is 1.	int	Required only if EnableBatch is true
AckWaitMS	Specify maximum wait time to receive ACK from the server. After which,the client resends the event to the server. Minimum value is 1.	int	Required only if EnableBatch is true
AckRetry	Specify retry count if ACK is not received from the server. Minimum value is 1.	int	Required only if EnableBatch is true
ClientAcknowledge	Configure this to True when acknowledging a consuming message explicitly.	boolean	Only for Consumer. Optional. Defaulted to false
MetricRegistry	Configure any metric client of your own choice. The registry should be implemented through IMetric interface collection. As default, events SDK provides default DDMetric collector.	IMetric	Optional
CallBack	Provide a websocket listener method that receives response and exceptions from the Events server.	public void CallBack(object sender, Response response)	Optional

Connection Retry

• Provide the ClientConfig.CONNECTION_RETRIES in the configuration.
• If the connection could not be opened, or if abnormal closure occurred due to transient / non-transient error, the client retries internally for 15 times and resends the undelivered events.
• When maximum retry counts (ClientConfig.CONNECTION_RETRIES + 15 internal retries) are reached, the client throws a websocket exception and sends back the undelivered events. Ensure you are handling/resending the undelivered events.
• The Client uses an exponential backoff mechanism, when trying to reconnect . Internally the exponential retry wait time is set to 5 seconds.

NOTE: Exponential backoff algorithm means if the first reconnect attempt fails, subsequent reconnect attempts are delayed by increasingly longer amounts of time. For example, retry.wait.time is configured as 5 seconds, it may retry the connection after 5 seconds, 10 seconds, 15 seconds, and so on.

Client Properties

Name	Type	Description	Syntax	Value
Active	Boolean	To check if the client is active (connection has been successfully established)	client.Active	true/false
ConnectionStatus	Enum	To check the state of the client	client.ConnectionStatus	None - Not set Connecting - Connection establishment in progress Open - Connection is established and active Reconnecting - Retrying connection on anomalies Closing - Connection closing either due to close call or connection retry limit reached Closed - Connection closed either due to close call or connection retry limit reached

Publisher

A publisher sends events to Events server through a websocket connection. To publish events, publishers must need a:

• NameSpace configured
• An event Schema configured
• An Event with the event schema grouped under a namespace

Publishing events

Follow the below steps to publish events,

1. Configure EventsClientConfig and create EventsClient

Provide the required client configuration to establish a new Publisher websocket connection with the Events server as below.

using TrimbleCloud.Events;

var config = new EventsClientConfig()
{
    Endpoint = new Uri("wss://<<publisher_endpoint>>"),
    ConnectionRetry = 5,
    AuthUrl = "https://stage.id.trimblecloud.com/oauth/token",
    ClientID = "<CLIENT_ID>",
    ClientSecret = "<CLIENT_SECRET>",
    TidScope = "<SCOPE>",
    Workflow = Workflow.PUBLISHER,
    #...
};

var client = new EventsClient(config);
client.CallBack += CallBack;

2. Create a PublishEvent instance

Create an instance of PublishEvent with namespace, event, event version and payload.

PublishEvent

Field	Description	Type	Example
Namespace	Namespace name. It should follow the pattern of “com.trimble.division.servicename”.	String	com.trimble.tcp.sample
Event	Name of the Event. It should contain only letters and numbers. No special characters are allowed.	String	SampleEvent
EventVersion	Version of the event. It should follow the pattern v[0-9].[0-9].	String	v1.0, v2.0
Payload	Publisher Payload. It should be JSON encoded and should match the Avro Schema of the respective Event.	String	"{\"id\":\"1214\",\"specversion\":\"1.0\",\"source\":\"\\id\\/<apiid>\"," +"\"type\":\"delete.v1\",\"time\":\"{{isoTimestamp}}\",\"subject\":\"\\/id\\/UserDelete\"," +"\"data\":{\"uuid\":\"121212\",\"application\":\"02cf0a0c-19db-45eb-8b19-90f93c2703ce\"," +"\"subject\":\"02cf0a0c-19db-45eb-8b19-90f93c2703ce1\"}}"
EntityID (optional)	To send sequential events use entity_id. For example, user profile id to maintain ordering in edits on user information.	String	profileid

Sample

var publishEvent = new PublishEvent()
{
    Namespace = "<NAMESPACE>",
    Event = "<EVENT>",
    EventVersion = "<EVENT_VERSION>",
    EntityID = "<ENTITY_ID>",
    Payload = JsonConvert.SerializeObject("<PAYLOAD>")
};

3. Publish the Event

There are two ways to publish.

Method 1

We can publish by using the PublishMessage method when EnableBatch property is set to True.

try
{
    client.PublishMessage(publishEvent);
}
catch (Exception ex)
{
    Console.WriteLine("Exception while sending batch: " + ex.Message);
}

Method 2

We can publish by using the Send method when EnableBatch property is set to False. Send method takes EventsBatch as parameter which contains ID and list of PublishEvent.

Events Batch

Field	Description	Type	Example
ID	Batch Id. Batch id cannot be empty.	String	***********
Events	List of Events. Can contain more than one event of type PublishEvent. Cannot be empty.	List<PublishEvent>	new List<PublishEvent>() { publishEvent1, publishEvent2 }

var batch = new EventsBatch()
{
    ID = Guid.NewGuid(),
    Events = new List<PublishEvent>() { publishEvent }
};

// Check for client status before sending
if (client.ConnectionStatus != WebSocketClientState.Open)
{
    client.Send(batch);
}
else
{
    // Wait for connection and retry
}

Exceptions

Name	Description
PublisherException	Publisher Exception is thrown in the following cases: Error while connecting to server Connection disconnected Authentication failure Exceeded maximum Retry connection Batch expired Batch, ACK, Event Request cannot be sent Header missing (Contact Events team for such Exception) If the Publish_message method is called when enable_batch is set False during client creation. Batch with empty events is passed in the send method.
ValidationException	ValidationException is thrown in following cases: Mandatory parameter missing Parameter type mismatch Parameter value not within limits
WebsocketException	Websocket Exception is thrown if Connection retry fails.
ConnectionClosedException	ConnectionClosedException is thrown in following cases: If publish is called, after connection close If the send method is called when connection is closed or during connection retry.
ServiceException	ServiceException is thrown in following cases: Error occurred during TID oauth token request TID Token generation failed Unsupported recorder for Datadog

Consumer

A Consumer consumes events from Events server through a websocket connection. It’s recommended that you review the schema of an event to decide whether an event is relevant to you. For an application to consume from Events server:

• Review the Events that you want to Consume.
• The application must be registered in the - Trimble Cloud Console.
• Submit the following Onboarding request to get access to the namespace and the event that you wish to access.

Consuming events

Follow the below steps to consume events,

1. Configure EventsClientConfig and create EventsClient

Provide the required client configuration to establish a new Consumer websocket connection with the Events server as below.

using TrimbleCloud.Events;

var config = new EventsClientConfig()
{
    Endpoint = new Uri("wss://<<consumer_endpoint>>"),
    ConnectionRetry = 5,
    AuthUrl = "https://stage.id.trimblecloud.com/oauth/token",
    ClientID = "<CLIENT_ID>",
    ClientSecret = "<CLIENT_SECRET>",
    TidScope = "<SCOPE>",
    Workflow = Workflow.CONSUMER,
    # ...
};

client = new EventsClient(config);
client.CallBack += CallBack;

2. Create a ConsumeEvent instance

Create an instance of ConsumeEvent with namespace, event and event version.

ConsumeEvent

The ConsumeEvent is asynchronous. Response will be sent via call back.

Field	Description	Type	Example
Namespace	Namespace name. It should follow the pattern of “com.trimble.division.servicename”.	String	com.trimble.tcp.sample
Event	Name of the Event. It should contain only letters and numbers. No special characters are allowed.	String	SampleEvent
EventVersion	Version of the event. It should follow the pattern v[0-9].[0-9].	String	v1.0, v2.0
VisibilityTimeout (optional)	Provide the duration the server should wait to receive acknowledgment for the message before resending the message. If the value is not provided, the default value is set to 30000 ms. Minimum value = 3000 ms Maximum value = 3600000 ms	int	3600

Sample

var consumeEvent = new ConsumeEvent()
{
    Namespace = "<NAMESPACE>",
    Event = "<EVENT>",
    EventVersion = "<EVENT_VERSION>",
};

3. Poll for Events

Consumes events from the Events server. It takes ConsumeEvent as an argument.

try
{
    client.Poll(consumeEvent);
}
catch (Exception e)
{
    Console.WriteLine("Exception occurred while polling:  " + e.Message);
}

Exceptions

Name	Description
ConsumerException	ConsumerException is thrown in the following cases: Error while connecting to server Connection disconnected Authentication failure Exceeded maximum Retry connection On receiving error messages from server on consume
ValidationException	Thrown when configuration object is missing one or more mandatory parameters.
WebsocketException	WebsocketException is thrown when Connection retry failed.
ConnectionClosedException	ConnectionClosedException is thrown if Poll method is called after closing the connection.
ServiceException	ServiceException is thrown in following cases: Error occurred during TID oauth token request TID Token generation failed Unsupported recorder for Datadog

Connection Close

Once the events are published/consumed, you have to close the connection by calling CloseAsync method.

List<PublishEvent> events = client.CloseAsync().Result;

After the close is called,

• The Events client will not accept any new events. If you try to send, it throws ConnectionClosedException.
• The undelivered events/not acknowledged events will be returned as a list.
• It is recommended for the Client application to stop sending messages and wait for ack with timeout before Close, if required.

Advanced Settings

Long Poll

Applicable for low volume events. Allowed poll duration for a long polling request is 60 seconds only. Any other value will be rejected by the server. Requests with AdditionalFields dictionary with the key long.poll.duration and value as 60000 will be considered as a long poll request.

Only one long poll request is allowed for a particular Event for 60 seconds till the response is received for the same. Others will be rejected with the exception "Long poll is in progress for the event : ${eventName}".

Once the response has been received for a long poll request for a particular Event, we will be able to send a new long poll request for the same Event.

If there is already a long poll request from another client instance is in progress for a particular Event at the server side and if we send another long poll request for the same Event then we will get the exception, "Already long poll is in progress" from the server.

var consumeEvent = new ConsumeEvent()
{
    Namespace = "<NAMESPACE>",
    Event = "<EVENT>",
    EventVersion = "<EVENT_VERSION>",
};
var additionalFields = new Dictionary<string, string>();
additionalFields.Add("long.poll.duration", "60000");

try
{
    client.Poll(consumeEvent);
}
catch (Exception e)
{
    Console.WriteLine("Exception occurred while polling:  " + e.Message);
}

Client Acknowledgement

For polling, if we need to disable the auto acknowledgement of the Events client and manually send acknowledgement, we need to set the EventsClientConfig.ClientAcknowledge property to “true” and call acknowledge() method. The message can be acknowledged as a whole or each event in the message can be acknowledged.

private static void ResponseHandler(object sender, Response response)
{
    if (response.ServiceResponse.ResponseMessages != null)
    {
        foreach (var kafkaResponse in response.ServiceResponse.ResponseMessages)
            Console.WriteLine(kafkaResponse.Message);

        Console.WriteLine("Sending acknowledgment...");
        eventsClient.Acknowledge(response.ServiceResponse.AcknowledgmentId);
    }
    else
    {
        Console.WriteLine(response.ServiceException.Message);
    }
}

Acknowledge method throws the following exception

• Connection Already Closed
• Connecting (initial connection or retry in progress)
• Any other client Exception on sending ack
• When tried to send acknowledgement with EventsClientConfig.ClientAcknowledge set to “false”.

Acknowledgment not received within 30 seconds is considered as failure and events will be resent again in the subsequent polls.

Event Ordering

Events Servers take the best efforts to deliver the events in order in which it is received. Clients should have logics within messages to manage strict ordering. To send sequential events, use EntityID.

private static void ResponseHandler(object sender, Response response)
{
    if (response.ServiceResponse.ResponseMessages != null)
    {
        foreach (var kafkaResponse in response.ServiceResponse.ResponseMessages)
            Console.WriteLine(kafkaResponse.EntityID);
    }
    else
    {
        Console.WriteLine(response.ServiceException.Message);
    }
}

Support

Send email to cloudplatform_support@trimble.com