Edges

Edges transmit messages of a single Message Type between two Nodes.

Edges:

• can store an unlimited number of messages for delivery. This is critical when the Node that the Edge is delivering messages to is unreliable.
• guarantee delivery of messages to the target Node.
• guarantee the order of message delivery to the target Node.
• encrypt messages both in flight and at rest.
• can optionally move messages to the Dead Letter Emitter after a configurable number of delivery tries.
  • Edges default to attempting delivery forever.
• can be moved to other Nodes that are sending/receiving the Edge's Message Type.
• can be purged of messages that are currently enqueued.
• send alerts when they become backed up.

Edges are implemented in EchoStream using AWS SQS FIFO queues. Unless you are using a language other than Python to implement External Nodes you should never have to interact with the underlying queue directly.

Message Type

Edges are created between two Nodes. To create the Edge, the source Node's send Message Type and the target Node's receive Message Type must be identical; upon creation, the Edge assumes that Message Type.

Once created the Message Type of the Edge cannot be changed.

Dead Letters

By default an Edge will attempt to deliver a message to it's target Node forever. Errors that occur during the processing of a message within the target Node will cause the Edge to block until that message is successfully delivered. In cases where the loss of a message would affect critical business requirements (e.g. - the loss of an invoice) this is the desired behavior from the Edge.

However, there are many business use cases where the loss of one of more messages is not so catastrophic. If your use case is tolerant of such loss, you may set the maximum number of delivery attempts via the maxRecieveCount parameter on Edge creation. If set, this parameter must be 1 or greater. If the edge attempts delivery that many times and delivery fails, the failing message will be removed from the Edge and sent to the Dead Letter Emitter Node.

For example, let's assume that we are sending a message every 1 minute that contains the latest state of a customer's account. Since it is the latest state being sent, the loss of one or more of these messages is unimportant to the overall business process. In this case, we may set the maxReceiveCount to 1 when we create the Edge. This will cause the Edge to only attempt delivery once prior to sending the failed message to the Dead Letter Emitter Node.

Alerts

Every Edge that you create has an alert that is attached to it. This alert watches the Edge for the age of the oldest message. If the oldest message exceeds 60 minutes the Edge will emit an alert via the Alert Emitter Node. You can process these alerts via alert monitoring.

When the Edge successfully delivers enough messages to reduce the oldest message below the 60 minute threshold a subsequent OK will be sent by the Alert Emitter Node letting you know that processing is back within specifications.

Metrics

You can query an Edge for metrics on the number of messages it is currently transmitting by accessing the messageCounts attribute. This will return three metrics that allow you to monitor your Edge.

• approximateNumberOfMessages: The approximate number of messages that the Edge is currently transmitting.
• approximateNumberOfMessagesDelayed: The number of messages that are on the Edge that have explicitly had their delivery delayed. Normally this will be 0.
• approximateNumberOfMessagesNotVisible: Often referred to as "in flight" messages, this is the number of messages that have been received by the target Node but have not been deleted from the Edge. Most of the time this will be 10 or less, but can climb much higher if the target Node is an External or Managed Node. A number greater than 10 generally indicates that there is a software logic error in the target Node.