May 2, 2018
Serverless idempotency with Azure Functions
Azure Functions and Durable Functions said that “functions should be idempotent.” However, what does it mean for the Azure Functions context?
What is idempotence?
According to the Wikipedia,
Idempotence is the property of certain operations in mathematics and computer science that they can be applied multiple times without changing the result beyond the initial application.
I can imagine the idempotence for the Infrastructure as Code tools and functional programming. However, what is that with Azure Functions?
For example, We write database, send messages like queue, notification, e-mails. It looks challenging to keep idempotency.
Why is idempotence important?
My colleague answer the question. In short, it is important for the cloud environment for keeping the resilient to retries. Especially it is very important for the durable functions. Your activity could be called multiple times by the replay.
By meriting of running in the cloud, assuming the worst case scenario: the underlying hardware might have a power outage at any time. That could be in the middle of your function execution of even after your function gets to the last line of your user code (so it looks like it was successful). This failure will lead to a retry of your function. This operation happens naturally for triggers like QueueTrigger, and even an HTTP client may resend the message if it has a retry loop. This means your function could get executed multiple times and needs to be idempotent so that it’s resilient to retries.
REST-API
This blog post with the movie is a useful resource for the idempotence for REST-API.
Database
Read is idempotent. Upsert or Update is also idempotent. Delete is not idempotent, however, if there is no resource, just return an error. You can compromise it. Create is not idempotent. However, if the database support Upsert it is idempotent. Sometimes, Database state could be changed during the call. However, it is ok. For the update, you can use optimistic lock strategy to conquer that.
Send e-mail
In case of the e-mail, you can compromise. If you receive two e-mails, it might not be a big deal. However, if you want to implement idempotency, you can save the state if the e-mail has been sent or not. This strategy is similar to the Queue strategy. Please see the next section.
Queue
We use queue frequently with Azure Functions. Imagine that if you use HTTP trigger with Queue output bindings, how we can make it idempotent? If you call HTTP trigger multiple times with the same parameter, obviously, it resends a queue.
We can’t achieve the idempotency for a single function. However, we can accomplish that in the whole system. If the queue trigger receives the queue with the same id, do nothing.
Code sample
I create a sample code for this. These are the node functions with HTTP and Queue trigger. I use Storage Table bindings to make sure the execution has occurred in the past.
HTTP trigger
Queue trigger
Usage
Using postman, send POST request to this body. You can change the id.
{ “id”: 8,
“name”: “ushio”
}The HTTP trigger function sends this message to the Queue trigger function. The second function executes something if the id is new to them.
2018–05–02T07:58:01 Welcome, you are now connected to log-streaming service.
2018–05–02T07:58:22.866 [Info] Function started (Id=4e8ac2da-773e-4b8c-b0ef-7e527073c52a)
2018–05–02T07:58:23.162 [Info] JavaScript queue trigger function processed work item: { RowKey: 8, name: ‘ushio’ }
2018–05–02T07:58:23.162 [Info] new id
2018–05–02T07:58:23.303 [Info] Function completed (Success, Id=4e8ac2da-773e-4b8c-b0ef-7e527073c52a, Duration=444ms)
2018–05–02T07:58:37.414 [Info] Function started (Id=46d4b3b1–51f0–4927-bd52–750ca849966e)If you send the same request, you will see
2018–05–02T07:58:37.445 [Info] JavaScript queue trigger function processed work item: { RowKey: 8, name: ‘ushio’ }
2018–05–02T07:58:37.445 [Info] already done it
2018–05–02T07:58:37.445 [Info] Function completed (Success, Id=46d4b3b1–51f0–4927-bd52–750ca849966e, Duration=28ms)
What happens if the Queue trigger causes an exception?
It seems that the queue has been consumed, but the execution has not done yet. Don’t worry about that. Azure Functions Queue trigger looks after that for you. Once the error happens, Azure Functions leave the queue. The Queue is consumed only after the successful execution of the Queue trigger functions. Then it retries to consume the queue. If it fails five times, it is going to the dead queue.
See this article
I also include some logic for proving that.
When I repeatedly send, sometimes, you can see the error. Just after that, you can see the retry happens.
2018–05–02T08:02:03.799 [Info] Function started (Id=6d04eed3–048e-440d-bb1b-09e04868563b)
2018–05–02T08:02:03.799 [Info] JavaScript queue trigger function processed work item: { RowKey: 8, name: ‘ushio’ }
2018–05–02T08:02:03.946 [Error] Exception while executing function: Functions.QueueTriggerJS1. mscorlib: Something happens exception.
2018–05–02T08:02:04.009 [Error] Function completed (Failure, Id=6d04eed3–048e-440d-bb1b-09e04868563b, Duration=206ms)
2018–05–02T08:02:04.135 [Info] Function started (Id=a6438c66–05b9–4c5f-90b6–7b84328b6f40)
2018–05–02T08:02:04.151 [Info] JavaScript queue trigger function processed work item: { RowKey: 8, name: ‘ushio’ }
2018–05–02T08:02:04.291 [Error] Exception while executing function: Functions.QueueTriggerJS1. mscorlib: Something happens exception.
2018–05–02T08:02:04.434 [Error] Function completed (Failure, Id=a6438c66–05b9–4c5f-90b6–7b84328b6f40, Duration=294ms)
Detail Logic
You can see the Queue Listener which is a part of the Queue Trigger bindings. It make the queue invisible for a while, then if it is completed the function, it will consume the queue.
Conclusion
To achieve the idempotency with Azure Functions, you don’t need to meet it in a single function. You can do it with the whole system. Sometimes, you can compromise it, however, always think the possibility of the retry. It will gain resiliency of your app.
Retrospectives
Since this code is very simple, however, I struggled for a couple of hours. I can’t find the correct way to configure the Storage Table input bindings.
The answer is like this. “rowKey” was very difficult.
{
"type": "table",
"name": "inputTable",
"tableName": "inTable",
"partitionKey": "Test",
"rowKey": "{rowKey}",
"take": 50,
"connection": "AzureWebJobsDashboard",
"direction": "in"
},I try to find the answer by this document, however I can’t clearly understand the spec.
Eventually, I dump the context object, and realized that the parameter. The specification says that directly specify the data. however, RowKey didn’t work. :)
context.log(context);It help me to find out the solution. The output is like this
{
invocationId: '73e4503a-f033-41ab-9155-0f58e1955371', executionContext:
{ invocationId: '73e4503a-f033-41ab-9155-0f58e1955371',
functionName: 'QueueTriggerJS1',
functionDirectory: 'D:\\home\\site\\wwwroot\\QueueTriggerJS1' },
bindings:
{ myQueueItem: { RowKey: 8, name: 'ushio' },
inputTable: { Name: 'ushio', PartitionKey: 'Test', RowKey: '8' } },
log:
{ [Function]
error: [Function],
warn: [Function],
info: [Function],
verbose: [Function],
metric: [Function] },
bindingData:
{ queueTrigger: '{"RowKey":8,"name":"ushio"}', dequeueCount: 3,
expirationTime: '5/9/2018 8:48:35 AM +00:00',
id: 'c7b3c3af-6979-4cda-aaff-79845ede184f',
insertionTime: '5/2/2018 8:48:35 AM +00:00',
nextVisibleTime: '5/2/2018 8:58:36 AM +00:00',
popReceipt: 'AgAAAAMAAAAAAAAAgv50wfPh0wE=',
sys:
{ methodName: 'QueueTriggerJS1',
utcNow: 2018-05-02T08:48:36.273Z },
rowKey: '8',
name: 'ushio',
invocationId: '73e4503a-f033-41ab-9155-0f58e1955371' }, done: [Function] }I should have dumped from the start.
