Subscriptions for Messages and Notifications

Subscriptions allow you to be notified of new messages or changes via a Message Queue of your choice

Subscriptions are used to trigger an asynchronous background process in response to an event on the commercetools platform. Common use cases include sending an Order Confirmation Email, charging a Credit Card after the delivery has been made, or synchronizing customer accounts to a CRM system.

As a payload, a Subscription delivers one of the predefined Messages or any Change to a resource.

The first part of this document explains how to setup a subscription, while the second part details the payload and delivery guarantees.

Representations

Subscription

  • id - String
  • version - Number
  • key - String - Optional - User-specific unique identifier for the subscription
  • destination - Destination - The Message Queue into which the notifications are to be sent
  • messages - Array of MessageSubscription - The messages subscribed to
  • changes - Array of ChangeSubscription - The change notifications subscribed to
  • format - Format - The format in which the payload is delivered.
  • status - SubscriptionHealthStatus
  • createdAt - DateTime
  • lastModifiedAt - DateTime

SubscriptionDraft

  • key - String - Optional - User-specific unique identifier for the subscription
  • destination - Destination - The Message Queue into which the notifications are to be sent
  • messages - Array of MessageSubscription - Optional - The messages to be subscribed to.
  • changes - Array of ChangeSubscription - Optional - The change notifications to be subscribed to.
  • format - Format - Optional - The format in which the payload is delivered.

Either messages or changes need to be set.

Destination

A destination contains all info necessary for the commercetools platform to deliver a message onto your Message Queue. Message Queues can be differentiated by the type field.

Currently ↗ AWS SQS , ↗ AWS SNS , ↗ Azure Service Bus, ↗ Azure Event Grid and ↗ Google Cloud Pub/Sub are supported.

When retrieving a Destination, secrets or access keys are partially hidden for security reasons.

AWS SQS Destination

↗ AWS SQS is a pull-queue on AWS. We support the Standard queue type (the FIFO queue type is not supported).

  • type - String - "SQS"
  • queueUrl - String
  • accessKey - String
  • accessSecret - String - Partially hidden on retrieval
  • region - String

The queue needs to exist beforehand. It is recommended to create an IAM user with an accessKey and accessSecret pair specifically for each subscription that only has the sqs:SendMessage permission on this queue.

AWS SNS Destination

↗ AWS SNS can be used to push messages to AWS Lambda, HTTP endpoints (webhooks) or fan-out messages to SQS queues.

  • type - String - "SNS"
  • topicArn - String
  • accessKey - String
  • accessSecret - String - Partially hidden on retrieval

The topic needs to exist beforehand. It is recommended to create an IAM user with an accessKey and accessSecret pair specifically for each subscription that only has the sns:Publish permission on this topic.

Azure Service Bus Destination

↗ Azure Service Bus can be used as a pull-queue with ↗ Queues or to fan-out messages with ↗ Topics and Subscriptions .

  • type - String - "AzureServiceBus"
  • connectionString - String - The SharedAccessKey is partially hidden on retrieval.

To setup a subscription with Azure Service Bus you first need to create a Queue/Topic in the ↗ Azure Portal . To grant access to resources Azure Service Bus uses the concept of ↗ Shared Access Policies . A shared access policy is defined on a scope and includes certain permissions. To allow the commercetools platform to push messages to your Queue/Topic you need to create a Shared Access Policy on your Queue/Topic and include the Send permission. When you open the policy in the Azure Portal you will find that Azure generated two so called connection strings for your policy. A connection string might look like this:

Endpoint=sb://<namespace>.servicebus.windows.net/;SharedAccessKeyName=<key-name>;SharedAccessKey=<key>;EntityPath=<queue-name>

You can pass either of the two connection strings as argument for the connectionString parameter of an Azure Service Bus Destination.
Make sure that the entity path points to your Queue/Topic.

Azure Event Grid Destination

↗ Azure Event Grid can be used to push messages to Azure Functions, HTTP endpoints (webhooks), and several other Azure tools.

Event Grid can only be used with the CloudEvents Format (Preview).

  • type - String - "EventGrid"
  • uri - String - The URI of the Topic
  • accessKey - String - Partially hidden on retrieval

To setup a subscription with Azure Event Grid you first need to create a Topic in the ↗ Azure Portal . To allow the commercetools platform to push messages to your Topic, you need to provide an access key. These can also be found in the ↗ Azure Portal after creating the Topic in the section “Access Keys”.

Google Cloud Pub/Sub Destination

↗ Google Cloud Pub/Sub can be used both as a pull-queue, and to push messages to e.g. Google Cloud Functions or HTTP endpoints (webhooks).

  • type - String - "GoogleCloudPubSub"
  • projectId - String - The Id of the project that contains the Pub/Sub topic
  • topic - String - Name of the topic

The topic needs to exist beforehand. The topic has to give the pubsub.topics.publish permission to our service account subscriptions@commercetools-platform.iam.gserviceaccount.com.

MessageSubscription

  • resourceTypeId - String
  • types - Array of String - Optional

types must contain valid message types for this resource, e.g. for resource type product the message type ProductPublished is valid. If no types of messages are given, the subscription is valid for all messages of this resource.

For supported resources and message types, please refer to the Messages Types Documentation. Messages will be delivered even if the Messages Query HTTP API is disabled.

The Message Subscription Payload is delivered.

ChangeSubscription

  • resourceTypeId - String

The following list of resourceTypeId are currently supported:

  • cart
  • cart-discount
  • category
  • channel
  • customer
  • customer-group
  • discount-code
  • extension
  • inventory-entry
  • order
  • payment
  • product
  • product-discount
  • product-type
  • review
  • shopping-list
  • subscription
  • state
  • tax-category
  • type

Different payloads for resource creation, update and deletion are delivered.

Format

The format in which the payload is delivered.

Platform Format

The Platform Format uses constructs that are similar to the ones used in the REST API, e.g. on the Messages Query HTTP API

  • type - String - "Platform"

CloudEvents Format (Preview)

CloudEvents is a specification for describing event data in a common way. CloudEvents seeks to ease event declaration and delivery across services, platforms and beyond. It is hosted by the Cloud Native Computing Foundation.

CloudEvents is a new effort and it’s still under active development. The format is versioned, and we do not plan to offer long-term support for outdated versions yet. Therefore, CloudEvents is currently only offered in Preview, with the goal to give feedback to the CloudEvents working group.

The CloudEvents format can be used with any Destination, and the payload delivered in the JSON Event Format. Azure Event Grid offers native support to filter and route CloudEvents.

  • type - String - "CloudEvents"
  • cloudEventsVersion - String - "0.1"

Subscription Health Status

Values of the SubscriptionHealthStatus enumeration:

  • Healthy - The subscription is delivering messages as expected
  • ConfigurationError - Messages can not be delivered with the current configuration. If the configuration is fixed in this health status, all messages that haven’t been delivered yet will still be delivered and the status will change to Healthy.
  • ConfigurationErrorDeliveryStopped - Messages can not be delivered with the current configuration and delivery of those is not attempted anymore. Undelivered messages are not retained and will not be delivered if the configuration is fixed in this status, but the status will change to Healthy as soon as new messages can be delivered successfully again.
  • TemporaryError - Messages can not be delivered currently, but not due to the configuration. E.g. the destination has a temporary outage.

Common causes for ConfigurationErrors are deleting the destination queue, deleting the access credentials, or removing the permission to push messages into the queue.

ConfigurationErrors are automatically turned into ConfigurationErrorDeliveryStopped after some time. See the Delivery Guarantees section for more details.

The configuration can be fixed by e.g. by re-creating the configuration on the destination side, or by setting a new configuration with the update action Change Destination.

Please note that the health of our delivery infrastructure is independent of the SubscriptionHealthStatus, and can be checked on our status page (it is part of the Background Services).

Get a Subscription

Get a Subscription by ID

Retrieves the representation of a subscription by its id.

Endpoint: /{projectKey}/subscriptions/{id}
Method: GET
OAuth2 Scopes: manage_subscriptions:{projectKey}
Response Representation: Subscription

Get a Subscription by Key

Retrieves the representation of a subscription by its key.

Endpoint: /{projectKey}/subscriptions/key={key}
Method: GET
OAuth2 Scopes: manage_subscriptions:{projectKey}
Response Representation: Subscription

Get the Health Status of a Subscription by ID

Retrieves the SubscriptionHealthStatus of a subscription by its id. This endpoint can be polled by a monitoring or alerting system, in order to automatically inform you if your configuration is not working anymore, or in case of any other error.

Endpoint: /{projectKey}/subscriptions/{id}/health
Method: GET

This endpoint is publicly visible.

No Body is returned. The SubscriptionHealthStatus is mapped onto HTTP status codes as follows:

  • Healthy - 200
  • ConfigurationError and ConfigurationErrorDeliveryStopped - 400
  • TemporaryError - 503

Query Subscriptions

Endpoint: /{projectKey}/subscriptions
Method: GET
OAuth2 Scopes: manage_subscriptions:{projectKey}
Response Representation: PagedQueryResult with the results array of Subscription
Query Parameters:

Create a Subscription

Endpoint: /{projectKey}/subscriptions
Method: POST
OAuth2 Scopes: manage_subscriptions:{projectKey}
Request Representation: SubscriptionDraft
Response Representation: Subscription

The creation of a Subscription is eventually consistent, it may take up to a minute before it becomes fully active.

In order to test that the destination is correctly configured, a test message will be put into the queue. If the message could not be delivered, the subscription will not be created. The payload of the test message is a notification of type ResourceCreated for the resourceTypeId subscription.

Currently, a maximum of 25 subscriptions can be created per project.

Update Subscription

Updates of a Subscription are eventually consistent, it may take up to a minute before changes becomes fully active.

Update Subscription by ID

Endpoint: /{projectKey}/subscriptions/{id}
Method: POST
OAuth2 Scopes: manage_subscriptions:{projectKey}
Response Representation: Subscription
Fields:

  • version - Number - Required
    The expected version of the subscription on which the changes should be applied. If the expected version does not match the actual version, a 409 Conflict will be returned.
  • actions - Array of UpdateAction - Required
    The list of update actions to be performed on the subscription.

Update Subscription by Key

Endpoint: /{projectKey}/subscriptions/key={key}
Method: POST
OAuth2 Scopes: manage_subscriptions:{projectKey}
Response Representation: Subscription
Fields:

  • version - Number - Required
    The expected version of the subscription on which the changes should be applied. If the expected version does not match the actual version, a 409 Conflict will be returned.
  • actions - Array of UpdateAction - Required
    The list of update actions to be performed on the subscription.

Update Actions
Please find below the individual update actions provided on this endpoint.


Set Key

  • action - String - "setKey"
  • key - String - Optional
    If key is absent or null, this field will be removed if it exists.

Set Messages

  • action - String - "setMessages"
  • messages - Array of MessageSubscription - Optional - The messages to be subscribed to.

messages can only be unset if changes is set.

Set Changes

  • action - String - "setChanges"
  • changes - Array of ChangeSubscription - Optional - The change notifications to be subscribed to.

changes can only be unset if messages is set.

Change Destination

  • action - String - "changeDestination"
  • destination - Destination

Updating the destination is eventually consistent, it may take up to a minute before it becomes fully active. During this time, messages may be delivered simultaneously to both the old and the new destination.

In order to test that the destination is correctly configured, a test message will be put into the queue. If the message could not be delivered, the update will fail. The payload of the test message is a notification of type ResourceCreated for the resourceTypeId subscription. The status will change to Healthy, if it wasn’t already.

Delete Subscription

The deletion of a Subscription is eventually consistent, it may take up to a minute before it becomes fully deactived.

Delete Subscription by ID

Endpoint: /{projectKey}/subscriptions/{id}
Method: DELETE
OAuth2 Scopes: manage_subscriptions:{projectKey}
Query Parameters:

  • version - Number - Required

Delete Subscription by Key

Endpoint: /{projectKey}/subscriptions/key={key}
Method: DELETE
OAuth2 Scopes: manage_subscriptions:{projectKey}
Query Parameters:

  • version - Number - Required

Delivery

The Delivery Payload depends on the chosen Format.

Delivery Payloads for the Platform Format

All payloads for the Platform Format share these common fields:

  • projectKey - String - The key of the project. Useful if the same queue is filled from multiple projects.
  • notificationType - String - Identifies the payload
  • resource - Reference to an object
    A reference to the resource that triggered this delivery.
  • resourceUserProvidedIdentifiers - UserProvidedIdentifiers - Optional

Message Subscription Payload

This payload will be sent for a MessageSubscription.

  • notificationType - String - "Message"
  • payloadNotIncluded - PaylaodNotIncluded - Optional

The payload will always contain these common fields for any message:

If the payload fits into the size limit of your message queue (the limit is often 256kb), all additional fields for the specific message are included as well (along with the type field). If the payload did not fit, it can be retrieved from the Messages Query HTTP API if messages are enabled.

If the payload does not fit into the size limit of your message queue, the field payloadNotIncluded is present.

PayloadNotIncluded
  • reason - String
  • payloadType - String - The value of the type field in the original payload.

ResourceCreated Payload

This payload will be sent for a ChangeSubscription if a resource was created.

  • notificationType - String - "ResourceCreated"
  • version - Number - The version of the resource after it was created
  • modifiedAt - DateTime - The lastModifiedAt of the resource after it was created

ResourceUpdated Payload

This payload will be sent for a ChangeSubscription if a resource was updated.

  • notificationType - String - "ResourceUpdated"
  • version - Number - The version of the resource after the update
  • oldVersion - Number - The version of the resource before the update
  • modifiedAt - DateTime - The lastModifiedAt of the resource after it was updated

ResourceDeleted Payload

This payload will be sent for a ChangeSubscription if a resource was deleted.

  • notificationType - String - "ResourceDeleted"
  • version - Number - The version of the resource at the time of deletion
  • modifiedAt - DateTime - The point in time when the resource was deleted

Delivery Payloads for the CloudEvents Format (Preview)

All payloads for the CloudEvents Format (Preview) share these common fields:

  • cloudEventsVersion - String
  • eventId - String
  • eventType - String
    The eventType is namespaced with com.commercetools, followed by the Resource Type Id and the type of subscription (either message or change).
    The final entry for a MessageSubscription is the Messages Type, e.g. com.commercetools.product.message.ProductPublished.
    The final entry for a ChangeSubscription is either ResourceCreated, ResourceUpdated or ResourceDeleted, e.g. com.commercetools.order.change.ResourceCreated.
  • source - String - The default REST URI of resource that triggered this event, including the project key. E.g. /my-project/products/4aa81de5-45d3-4fca-96e2-860df4389d09.
  • eventTime - DateTime - Corresponds to the lastModifiedAt of the resource at the time the event was triggered.
  • extensions - Object - The fields for each payload type are defined below.
  • data - Object - Optional - Currently, only set for MessageSubscriptions.

Message Subscription Payload

These addition fields inside extensions will be sent for a MessageSubscription:

  • version - Number - The version of the resource after the update
  • sequenceNumber - Number - The sequenceNumber of the Message
  • dataOmitted - Boolean - True if the complete payload did not fit into the size limit of your message queue.

Additionally, the data field will be sent (at the top level of the JSON):

Message Data Payload

If the complete payload fits into the size limit of your message queue (the limit is often 256kb), all fields for the specific message type are included (along with the type field). However, the default message fields are not present, as their values are already in the CloudEvents context: id is present in eventId, sequenceNumber and resourceVersion inside the extensions, resource can be constructed from source, and both createdAt and lastModifiedAt are present in eventTime.

If the payload does not fit, it can be retrieved from the Messages Query HTTP API if messages are enabled. In this case, the data will contain the field:

ResourceCreated Payload

These addition fields inside extensions will be sent for a ChangeSubscription if a resource was created.

  • version - Number - The version of the resource after it was created

ResourceUpdated Payload

These addition fields inside extensions will be sent for a ChangeSubscription if a resource was updated.

  • version - Number - The version of the resource after the update
  • oldVersion - Number - The version of the resource before the update

ResourceDeleted Payload

These addition fields inside extensions will be sent for a ChangeSubscription if a resource was deleted.

  • version - Number - The version of the resource at the time of deletion

Delivery Guarantees

At-least-once Delivery

If the commercetools platform did not get a positive acknowledgement that messages have been accepted by the destination, it will retry to deliver the messages. Our Retry Policy depends on the Subscrption Heatlh Status:

  • TemporaryError- Retry for up to 48 hours, afterwards messages may be dropped.
  • ConfigurationError - Retry for up to 24 hours (for production projects) or up to 1 hour (development or staging projects), afterwards the status changes to ConfigurationErrorDeliveryStopped and messages are dropped.

A side effect of the retry is that the same message may be put several times into the message queue. An idempotent message processor that doesn’t process the same message twice can check whether the message was already processed. For notificationType "Message" use the fields resource.id and sequenceNumber, or for other types the fields resource.id and version.

We guarantee that all message payloads we try to deliver to a destination are valid according to its specification. Therefore, the only cases where messages can not be delivered are due to an ongoing incident, or due to a misconfiguration.

You can monitor the health of a subscription with a tool of your choice. For production-critical queues, we recommend to set up an automatic alert.

No guarantee on order

Messages are not guaranteed to be delivered in their natural order (e.g. with ascending sequenceNumber or ascending version). This is especially true in the case of retries.

For notificationType "Message", a message processor can use the fields resource.id and sequenceNumber to process messages in the correct order (e.g. if the last processed sequenceNumber was 2, and the current message is 4, the current message can be put back into the queue for processing at a later point in time).

For other notificationType, the fields resource.id, version and (in case of update) oldVersion can be used. Note that version is not sequential.