Subscriptions for Messages and Notifications

Subscriptions allow you to be notified of new messages or changes via a Message Queue of your choice

Subscriptions are used to trigger an asynchronous background process in response to an event on the commercetools platform. Common use cases include sending an Order Confirmation Email, charging a Credit Card after the delivery has been made, or synchronizing customer accounts to a CRM system.

As a payload, a Subscription delivers one of the predefined Messages or any Change to a resource.

The first part of this document explains how to setup a subscription, while the second part details the payload and delivery guarantees.

Representations

Subscription

  • id - String
  • version - Number
  • key - String - Optional - User-specific unique identifier for the subscription
  • destination - Destination - The Message Queue into which the notifications are to be sent
  • messages - Array of MessageSubscription - The messages subscribed to
  • changes - Array of ChangeSubscription - The change notifications subscribed to
  • format - Format - The format in which the payload is delivered.
  • createdAt - DateTime
  • lastModifiedAt - DateTime

SubscriptionDraft

  • key - String - Optional - User-specific unique identifier for the subscription
  • destination - Destination - The Message Queue into which the notifications are to be sent
  • messages - Array of MessageSubscription - Optional - The messages to be subscribed to.
  • changes - Array of ChangeSubscription - Optional - The change notifications to be subscribed to.
  • format - Format - Optional - The format in which the payload is delivered.

Either messages or changes need to be set.

Destination

A destination contains all info necessary for the commercetools platform to deliver a message onto your Message Queue. Message Queues can be differentiated by the type field.

Currently ↗ AWS SQS , ↗ AWS SNS , ↗ Azure Service Bus, ↗ Azure Event Grid and ↗ IronMQ are supported.

AWS SQS Destination

↗ AWS SQS is a pull-queue on AWS. We support the Standard queue type (the FIFO queue type is not supported).

  • type - String - "SQS"
  • queueUrl - String
  • accessKey - String
  • accessSecret - String
  • region - String

The queue needs to exist beforehand. It is recommended to create an IAM user with an accessKey and accessSecret pair specifically for each subscription that only has the sqs:SendMessage permission on this queue.

AWS SNS Destination

↗ AWS SNS can be used to push messages to AWS Lambda, HTTP endpoints (webhooks) or fan-out messages to SQS queues.

  • type - String - "SNS"
  • topicArn - String
  • accessKey - String
  • accessSecret - String

The topic needs to exist beforehand. It is recommended to create an IAM user with an accessKey and accessSecret pair specifically for each subscription that only has the sns:Publish permission on this topic.

Azure Service Bus Destination

↗ Azure Service Bus can be used as a pull-queue with ↗ Queues or to fan-out messages with ↗ Topics and Subscriptions .

  • type - String - "AzureServiceBus"
  • connectionString - String

To setup a subscription with Azure Service Bus you first need to create a Queue/Topic in the ↗ Azure Portal . To grant access to resources Azure Service Bus uses the concept of ↗ Shared Access Policies . A shared access policy is defined on a scope and includes certain permissions. To allow the commercetools platform to push messages to your Queue/Topic you need to create a Shared Access Policy on your Queue/Topic and include the Send permission. When you open the policy in the Azure Portal you will find that Azure generated two so called connection strings for your policy. A connection string might look like this:

Endpoint=sb://<namespace>.servicebus.windows.net/;SharedAccessKeyName=<key-name>;SharedAccessKey=<key>;EntityPath=<queue-name>

You can pass either of the two connection strings as argument for the connectionString parameter of an Azure Service Bus Destination.
Make sure that the entity path points to your Queue/Topic.

Azure Event Grid Destination

↗ Azure Event Grid can be used to push messages to Azure Functions, HTTP endpoints (webhooks), and several other Azure tools.

Event Grid can only be used with the CloudEvents Format (Preview).

  • type - String - "AzureServiceBus"
  • uri - String - The URI of the Topic
  • accessKey - String

To setup a subscription with Azure Event Grid you first need to create a Topic in the ↗ Azure Portal . To allow the commercetools platform to push messages to your Topic, you need to provide an access key. These can also be found in the ↗ Azure Portal after creating the Topic in the section “Access Keys”.

Google Cloud Pub/Sub Destination

↗ Google Cloud Pub/Sub can be used both as a pull-queue, and to push messages to e.g. Google Cloud Functions or HTTP endpoints (webhooks).

  • type - String - "GoogleCloudPubSub"
  • projectId - String - The Id of the project that contains the Pub/Sub topic
  • topic - String - Name of the topic

The topic needs to exist beforehand. The topic has to give the pubsub.topics.publish permission to our service account subscriptions@commercetools-platform.iam.gserviceaccount.com.

IronMQ Destination

↗ IronMQ can be used both as a pull-queue, and to push messages to IronWorkers or HTTP endpoints (webhooks) or fan-out messages to other IronMQ queues.

  • type - String - "IronMQ"
  • uri - String - The webhook URI of your IronMQ.

The webhook URI must contain a valid OAuth token. It roughly looks like this: https://...iron.io/.../queues/.../webhook?oauth=....

MessageSubscription

  • resourceTypeId - String
  • types - Array of String - Optional

types must contain valid message types for this resource, e.g. for resource type product the message type ProductPublished is valid. If no types of messages are given, the subscription is valid for all messages of this resource.

For supported resources and message types, please refer to the Messages Types Documentation. Messages will be delivered even if the Messages REST API is disabled.

The Message Subscription Payload is delivered.

ChangeSubscription

  • resourceTypeId - String

The following list of resourceTypeId are currently supported:

  • cart
  • category
  • customer
  • customer-group
  • inventory-entry
  • order
  • payment
  • product
  • product-type
  • review

Different payloads for resource creation, update and deletion are delivered.

Format

The format in which the payload is delivered.

Platform Format

The Platform Format uses constructs that are similar to the ones used in the REST API, e.g. on the Messages Query endpoint

  • type - String - "Platform"

CloudEvents Format (Preview)

CloudEvents is a specification for describing event data in a common way. CloudEvents seeks to ease event declaration and delivery across services, platforms and beyond. It is hosted by the Cloud Native Computing Foundation.

CloudEvents is a new effort and it’s still under active development. The format is versioned, and we do not plan to offer long-term support for outdated versions yet. Therefore, CloudEvents is currently only offered in Preview, with the goal to give feedback to the CloudEvents working group.

The CloudEvents format can be used with any Destination, and the payload delivered in the JSON Event Format. Azure Event Grid offers native support to filter and route CloudEvents.

  • type - String - "CloudEvents"
  • cloudEventsVersion - String - "0.1"

Get a Subscription

Get a Subscription by ID

Retrieves the representation of a subscription by its id.

Endpoint: /{projectKey}/subscriptions/{id}
Method: GET
OAuth2 Scopes: manage_subscriptions:{projectKey}
Response Representation: Subscription

Get a Subscription by Key

Retrieves the representation of a subscription by its key.

Endpoint: /{projectKey}/subscriptions/key={key}
Method: GET
OAuth2 Scopes: manage_subscriptions:{projectKey}
Response Representation: Subscription

Query Subscriptions

Endpoint: /{projectKey}/subscriptions
Method: GET
OAuth2 Scopes: manage_subscriptions:{projectKey}
Response Representation: PagedQueryResult with the results array of Subscription
Query Parameters:

Create a Subscription

Endpoint: /{projectKey}/subscriptions
Method: POST
OAuth2 Scopes: manage_subscriptions:{projectKey}
Request Representation: SubscriptionDraft
Response Representation: Subscription

The creation of a Subscription is eventually consistent, it may take up to a minute before it becomes fully active.

In order to test that the destination is correctly configured, a test message will be put into the queue. If the message could not be delivered, the subscription will not be created. The payload of the test message is a notification of type ResourceCreated for the resourceTypeId subscription.

Currently, a maximum of 25 subscriptions can be created per project.

Update Subscription

Updates of a Subscription are eventually consistent, it may take up to a minute before changes becomes fully active.

Update Subscription by ID

Endpoint: /{projectKey}/subscriptions/{id}
Method: POST
OAuth2 Scopes: manage_subscriptions:{projectKey}
Response Representation: Subscription
Fields:

  • version - Number - Required
    The expected version of the subscription on which the changes should be applied. If the expected version does not match the actual version, a 409 Conflict will be returned.
  • actions - Array of UpdateAction - Required
    The list of update actions to be performed on the subscription.

Update Subscription by Key

Endpoint: /{projectKey}/subscriptions/key={key}
Method: POST
OAuth2 Scopes: manage_subscriptions:{projectKey}
Response Representation: Subscription
Fields:

  • version - Number - Required
    The expected version of the subscription on which the changes should be applied. If the expected version does not match the actual version, a 409 Conflict will be returned.
  • actions - Array of UpdateAction - Required
    The list of update actions to be performed on the subscription.

Update Actions
Please find below the individual update actions provided on this endpoint.


Set Key

  • action - String - "setKey"
  • key - String - Optional
    If key is absent or null, this field will be removed if it exists.

Set Messages

  • action - String - "setMessages"
  • messages - Array of MessageSubscription - Optional - The messages to be subscribed to.

messages can only be unset if changes is set.

Set Changes

  • action - String - "setChanges"
  • changes - Array of ChangeSubscription - Optional - The change notifications to be subscribed to.

changes can only be unset if messages is set.

Delete Subscription

The deletion of a Subscription is eventually consistent, it may take up to a minute before it becomes fully deactived.

Delete Subscription by ID

Endpoint: /{projectKey}/subscriptions/{id}
Method: DELETE
OAuth2 Scopes: manage_subscriptions:{projectKey}
Query Parameters:

  • version - Number - Required

Delete Subscription by Key

Endpoint: /{projectKey}/subscriptions/key={key}
Method: DELETE
OAuth2 Scopes: manage_subscriptions:{projectKey}
Query Parameters:

  • version - Number - Required

Delivery

The Delivery Payload depends on the chosen Format.

Delivery Payloads for the Platform Format

All payloads for the Platform Format share these common fields:

  • projectKey - String - The key of the project. Useful if the same queue is filled from multiple projects.
  • notificationType - String - Identifies the payload
  • resource - Reference to an object
    A reference to the resource that triggered this delivery.

Message Subscription Payload

This payload will be sent for a MessageSubscription.

  • notificationType - String - "Message"
  • payloadNotIncluded - PaylaodNotIncluded - Optional

The payload will always contain the common fields id, version, sequenceNumber, resourceVersion, createdAt and lastModifiedAt for any message.

If the payload fits within the size limit of your message queue (the limit is often 256kb), all additional fields for the specific message are included as well (along with the type field). If the payload did not fit, it can be retrieved from the Messages HTTP API if messages are enabled.

If the payload does not fit within the size limit of your message queue, the field payloadNotIncluded is present.

PayloadNotIncluded
  • reason - String
  • payloadType - String - The value of the type field in the original payload.

ResourceCreated Payload

This payload will be sent for a ChangeSubscription if a resource was created.

  • notificationType - String - "ResourceCreated"
  • version - Number - The version of the resource after it was created
  • modifiedAt - DateTime - The lastModifiedAt of the resource after it was created

ResourceUpdated Payload

This payload will be sent for a ChangeSubscription if a resource was updated.

  • notificationType - String - "ResourceUpdated"
  • version - Number - The version of the resource after the update
  • oldVersion - Number - The version of the resource before the update
  • modifiedAt - DateTime - The lastModifiedAt of the resource after it was updated

ResourceDeleted Payload

This payload will be sent for a ChangeSubscription if a resource was deleted.

  • notificationType - String - "ResourceDeleted"
  • version - Number - The version of the resource at the time of deletion
  • modifiedAt - DateTime - The point in time when the resource was deleted

Delivery Payloads for the CloudEvents Format (Preview)

All payloads for the CloudEvents Format (Preview) share these common fields:

  • cloudEventsVersion - String
  • eventId - String
  • eventType - String
    The eventType is namespaced with com.commercetools, followed by the Resource Type Id.
    The final entry for a MessageSubscription is the Messages Type, e.g. com.commercetools.product.ProductPublished.
    The final entry for a ChangeSubscription is either ResourceCreated, ResourceUpdated or ResourceDeleted, e.g. com.commercetools.order.ResourceCreated.
  • source - String - The default REST URI of resource that triggered this event, including the project key. E.g. /my-project/products/4aa81de5-45d3-4fca-96e2-860df4389d09.
  • eventTime - DateTime - Corresponds to the lastModifiedAt of the resource at the time the event was triggered.
  • extensions - Object - The fields for each payload type are defined below.
  • data - Object - Optional - Currently, only set for MessageSubscriptions.

Message Subscription Payload

These addition fields inside extensions will be sent for a MessageSubscription:

  • version - Number - The version of the resource after the update
  • sequenceNumber - Number - The sequenceNumber of the Message

Additionally, the data field will be sent (at the top level of the JSON):

Message Data Payload

If the complete payload fits within the size limit of your message queue (the limit is often 256kb), all fields for the specific message type are included (along with the type field). However, the default message fields are not present, as their values are already in the CloudEvents context: id is present in eventId, sequenceNumber and resourceVersion inside the extensions, resource can be constructed from source, and both createdAt and lastModifiedAt are present in eventTime.

If the payload does not fit, it can be retrieved from the Messages HTTP API if messages are enabled. In this case, the data will contain the field:

ResourceCreated Payload

These addition fields inside extensions will be sent for a ChangeSubscription if a resource was created.

  • version - Number - The version of the resource after it was created

ResourceUpdated Payload

These addition fields inside extensions will be sent for a ChangeSubscription if a resource was updated.

  • version - Number - The version of the resource after the update
  • oldVersion - Number - The version of the resource before the update

ResourceDeleted Payload

These addition fields inside extensions will be sent for a ChangeSubscription if a resource was deleted.

  • version - Number - The version of the resource at the time of deletion

Delivery Guarantees

At-least-once Delivery

If the commercetools platform did not get a positive acknowledgement that messages have been accepted by the destination, it will retry to deliver the messages for at least 48 hours. After that, messages may be dropped.

A side effect of the retry is that the same message may be put several times into the message queue. An idempotent message processor that doesn’t process the same message twice can check whether the message was already processed. For notificationType "Message" use the fields resource.id and sequenceNumber, or for other types the fields resource.id and version.

We guarantee that all message payloads we try to deliver to a destination are valid according to its specification. Therefore, the only cases where messages can not be delivered are due to an ongoing incident, or due to a misconfiguration.

No guarantee on order

Messages are not guaranteed to be delivered in their natural order (e.g. with ascending sequenceNumber or ascending version). This is especially true in the case of retries.

For notificationType "Message", a message processor can use the fields resource.id and sequenceNumber to process messages in the correct order (e.g. if the last processed sequenceNumber was 2, and the current message is 4, the current message can be put back into the queue for processing at a later point in time).

For other notificationType, the fields resource.id, version and (in case of update) oldVersion can be used. Note that version is not sequential.

Deletion of Subscription in case of continuous failure

If all deliveries for a subscription fail for several days (e.g. if the Message Queue was deleted, or the access credentials saved in the destination became invalid), we reserve the right to delete the subscription. (In case of a production project, we will try to get in touch with you!)