Reactive Programming with Event Grid and Azure Functions
I play with the Azure Event Grid . Azure Event Grid is a fully-managed intelligent event routing service that allows for uniform event consumption using a publish-subscribe model. Azure Functions have some integration with Event Grid. I’d like to share some of the tips which you to enable Event Grid.
1. Value
At the first time, you might think what you can use it for? I also thought the same thing. Now I gradually understand several benefits.
- Low cost / serverless
- Low latency
- Message Filtering
- Reliable
Low cost / serverless
According to the Pricing page .
It costs 0.3$ / millions operations with 100, 000 operations for free for a month. It cost around 5 USD per month. Compared with Service Bus Standard plan, which has Publish-Subscribe service, is $10 for month for 5 million operations. For the Event Grid, it is $2.97. Service Bus is very good for highly reliable system like banking system. Event Grid is fit for Event Driven Architecture with scalability. It supports millions events per second.
Low Latency
Event Driven Architecture is not good at Low Latency scenario. If you use Queue for messaging method, it requires Polling/Long Polling. It aims for consistency rather than low latency. However, sometimes, You want to build event-driven serverless system with Low latency. If you create a printing system with event-driven architecture with Queue, it is difficult to print less than 1 min from the request. Also, If you want to move from Microservices architecture to Serverless architecture, you might need Low Latency scenario.
Message Filtering
Since Event Grid using Pub-Sub model, Event Handlers need to subscribe a Topic. When The Topic gets an message, message is routed event Handlers, Event Grid allow you to filter the message. You can chose which event you receive. For example, If you use a blob storage, you can filter via directory/file patterns.
Reliability
Once an Event Grid Topic accept a message and if it fails to send message to event handler, it will retry with exponential back off method. It is much reliable than just use http request. You can substitute some use cases that you use queue for sending message for asynchronous messaging.
2. Azure Functions Integration
You can use Event Grid with Azure Functions. Azure Functions support EventGrid Trigger. However, I can’t find explicit document until now. So, I’d like to share some learning.
Location
Currently Event Grid is preview. The Location which you can use with Azure Functions is limited. Now West US 2 and West Central US is for the Blob Integration. You can check which location you can use in here.
Setup Event Grid Trigger
Create Custom Topic
To understand the behavior of EventGrid, let’s start with Custom Topic. On the Azure Portal, create Event Grid Topic.
Create an Event Grid Topic from the Azure Portal.
Get the Topic Endpoint and Topic Key for sending messages from the Event Topic.
Now ready to use Topic.
Create Azure Functions
Let’s write code for Event Grid Trigger. I wrote an sample with C#. You can find whole code in here. You need to install this nuget package.
You might want to know the EventGridEvent definition. You can get a lot of meta data from this. You can get the message from “Data” attribute from the senders.
Then just upload your code to a Function App.
Create Event Subscriptions
You can create an Event Subscription from the Azure Portal. Go to the Function App which you deploy your code. Just click this link.
You will see this screen. Choose the Event Grid Topic name which you want to subscribe. Then save.
Now you are ready to accept Event Grid message at the Azure Functions.
Sending a message to the Topic
Sending message is also simple. Just install Microsoft.Azure.EventGrid package as nuget package. You need topicHostName and topicKey which you can get already. One tips, you can pass Json Serializable Poco as the data object. I don’t know why but if you pass Anonymous Json Serializable Object, it will cause an error.
Additional NOTE (30th Jan, 2018):
The data object is not restricted to JObject by EventGrid runtime. It should be cast to JToken as the API level. Now have some issue for Event Grid Trigger. see https://github.com/Azure/azure-functions-eventgrid-extension/issues/25.
Send message using this method. Then you will see the Event Grid Trigger works.
2018-01-28T05:54:11.744 Function started (Id=a9e4dfb8-c553-49c3-bbb3-940dd49f25c2)2018-01-28T05:54:11.744 The SomeFunc : {"topic":"/subscriptions/YOUR_SUBSCIPTION/resourceGroups/RemoveEventGrid/providers/Microsoft.EventGrid/topics/sometopic","subject":"some/func1","data":{"Id":"100","Name":"Tsuyoshi"},"eventType":"func-event","publishTime":"0001-01-01T00:00:00","eventTime":"2018-01-28T05:54:11.4957888Z","id":"7d2aeb69-716c-49cf-ba1c-c2f721580acb"}2018-01-28T05:54:11.744 Function completed (Success, Id=a9e4dfb8-c553-49c3-bbb3-940dd49f25c2, Duration=11ms)
Also, you can filter by Event Type and Subject. When you configure Event Subscription like this, it will trig by only “some/” prefix subject. This feature is handy.
Trigger is almost the same as calling a webhook. If you create a simple webhook function with Event Subscription like this,
Open the Event Grid Topic page, then click Event Subscription
Then create webhook with the HttpTrigger function’s URL.
Then send message to the topic, the HttpTrigger function is also triggered.
2018-01-28T05:55:55.731 Function started (Id=2739a4c7-a28a-420e-8080-a4dcc0e1acc0)2018-01-28T05:55:55.731 HttpHook hooked: [{"id": "68572a72-e4d3-411c-971d-962871a8e3b1","subject": "some/func1","data": {"Id": "100","Name": "Tsuyoshi"},"eventType": "func-event","eventTime": "2018-01-28T05:55:55.5907116Z","dataVersion": "1.0.0","metadataVersion": "1","topic": "/subscriptions/YOUR_SUBSCRIPTION/resourceGroups/RemoveEventGrid/providers/Microsoft.EventGrid/topics/sometopic"}]
Event Grid Trigger vs HttpTrigger
I recommend to use EventGrid Trigger. It provides the endpoint url verfication with Event Grid. If you use HttpTrigger, you probably will see 3 initial requests, they are verfication errors, in preview they are just ignored.
Debugging
You can check if your messages are routed as expected. on the Event Grid Topic page. You can see the Metrics page for each Event Grid Subscription.
One of the difficulty of the debugging is, if you’ve got an error during messaging, sometimes you can’t see the error messages. It looks just not delivered the messages since this is still preview. I encounter that issue three times. Once it was just an shortage of Azure specific region.
Use Microsoft.Azure.EventGrid package
The second was I tried to send EventGridEvent with passing non-json-serializable object to the data field. At that time, I didn’t use the Microsoft.Azure.EventGrid package. Just send message with HttpClient. Since now you can use EventGrid package, I recommend to use this. It prevent these issue.
Use the same Function Name with your method.
The third was, The medhod name is not match with Function Name. This is the WRONG sample. :)
Also if you encounter the issue which you can’t see an error on your portal and Metrics page, you can ask support request on Azure Portal.
Conclusion
Event Gird is new way to Reactive Programming world. Enjoy the Event Grid with Azure Functions.