Amazon pub/sub in the cloud

Amazon keeps extending its cloud offering. They have just added Amazon Simple Notification Service (SNS). SNS is a publish/subscribe mechanism.

As explained in earlier posts, I expect Integration-As-A-Service to become more important. One of the larger players (Amazon, Google, EMC, Cisco, Microsoft, …) may one day come up with a wonderful solution for Business-2-Business communication between organizations.

When I first learned about Simple Queuing Service of Amazon back in 2006, I intially thought that SQS could serve as a transport mechanism for B2B communication. But that didn’t work out. As the message size of SQS was very limited, data first had to be stored on S3. Authentication and authorization were also very limited.

So I looked around in the SNS documentation to see what SNS actually is and see if it can serve as a basis for B2B communication. Amazon thinks SNS is usable for B2B or application integration:

Application integration: Amazon SNS can be used in workflow systems to relay events among distributed computer applications, move data between data stores, or update records in business systems. For example, in an order processing application, notification messages may be sent whenever a transaction occurs; a customer places an order, the transaction is forwarded to a payment processor for approval, and an order confirmation message is published to an Amazon SNS topic.Some facts

  • Messages can be published over HTTP, HTTPS, E-mail or SQS
  • Proprietary solution/mechanism, not based on any standard (no AS1, AS2, SFTP, WS-Notification, WS-Eventing, …)
  • Messages are (again) limited to 8KB. Just like SQS: too small.
  • Authentication is based on AWS accounts, so also every subscriber requires an AWS account, hindering factor.
  • Messages are pushed, not polled. This is good for performance. For polling, use SQS.
  • But when pushing, the subscriber must expose a web service or mail account. How to secure this: no authentication from Amazon to endpoint receiving notifications; no basic auth, no support for client certs, …
  • Messages are signed by Amazon. This is good, very good. Signing is based on HmacSHA256.

Nice and interesting, but not good enough… In particular the message size remains a blocking factor.

Questions left:

  • What happens if messages cannot be delivered for a longer periode of time? E.g. when a subscriber disappears?
  • How does a message that is published over HTTP exactly look like (signed, JSON)? What parameters are passed in the URL?
  • Can an SSL endpoint with self-signed cert receive notifications?
  • What if SSL cert of endpoint is expired?
  • Are mail messages signed and if yes, how?
  • How and when are messages actually persisted?
  • The publish service isn’t idempotent it seems?

PS: all based on reading the docs, must confess that I didn’t actually test it

Authored by: Guy



Curious to know more about this topic?

Working at i8c

i8c is a system integrator that strives for an informal atmosphere between its employees, who have an average age of approx 30 years old. We invest a lot of effort in the professional development of each individual, through a direct connection between the consultants and the management (no multiple layers of middle management). We are based in Kontich, near Antwerp, but our customers are mainly located in the triangle Ghent-Antwerp-Brussels and belong to the top 500 companies in Belgium (Securex, Electrabel, UCB, etc…).

Quality Assurance

i8c is committed to delivering quality services and providing customer satisfaction. That’s why we invested in the introduction of a Quality Management System, which resulted in our ISO9001:2000 certification. This guarantees that we will meet your expectations, as a reliable, efficient and mature partner for your SOA & integration projects.

i8c - ISO9001-2015

Also worth reading

AWS AppFlow: Streamlining SaaS Integrations with AWS Services

In today’s digital world, organizations are constantly looking for ways to streamline their workflows and improve their data management processes. One of the key challenges that organizations face is integrating their various software as a service (SaaS) applications with their data management systems. This is

Read More »

Apigee Scope Validation using OpenAPI Specification

In API security and management, we often use a lot of different security mechanisms to protect the requested resource behind the API Gateway. One of these mechanisms is the validation of scopes to authorize a client on a specific sub-resource of the API. Most of

Read More »