Thursday, January 18, 2018

How to architect long running flows which exceed the 30 day timeout limit in Microsoft Flow

Brown snail in its shell slowly inches across the asphalt

Photo by Alex Blăjan on Unsplash

Microsoft Flow has a built in timeout of 30 days for any running Flow (Logic Apps has 90 days), which for most scenarios is not a problem.

However, there are business processes that might run longer than 30 days, and I’ll address two common scenarios and how to overcome them.

See https://docs.microsoft.com/en-us/flow/limits-and-config for details on Flow limit and see https://docs.microsoft.com/en-us/azure/logic-apps/logic-apps-limits-and-config for limits on Logic Apps.

Scenario 1: Any single action will never last 30 days, but the total Flow might

Let’s imagine you have 5 consecutive actions in your flow, where there is no chance that any of them will run for more than 30 days. For example 5 consecutive approval steps. The way to work around this is to create one flow per approval, and then have them call each other as they go along.

To illustrate the steps I have created two flows. One named “Flow – Initial” and the other “Flow – 2”.

I’ll start with the second flow, as this will be called from the first.

This takes a HTTP request as input. In this case two parameters, a title and description, which is used in the an approval action. After you save this flow, copy the POST URL, which you will need in the initial flow.

image

The initial flow takes input from the user, and then calls the second flow, a pattern you can use to continue calling the third flow etc..

image

Scenario 2: A single action might take more than 30 days

Again, let’s take a look at an approval flow. You start an approval task i a Flow, but it could take more than 30 days before someone accepts or rejects it.

Again you need one initial flow which kicks it all off. This can be a Flow button, or any other trigger which grabs the initial input.

The key is to generate a JSON object with all the needed data which you can use in the flow with the actual business logic.

I’ll use Flow – 2 as my staring point, and the idea is to have this Flow call itself as long as needed until the approval is accepted or rejected.

First I’ll access the settings for the Approval action.

image

As I mentioned at the beginning, the default max timeout is 30 days in Microsoft Flow. The idea is to restart the flow if it times out within this boundary. You don’t have the possibility to perform actions if the flow itself times out, but we can act on when a single action times out.

If you set the action timeout to 29 days it’s guaranteed to timeout before the whole flow times out. To test it out you can set the timeout lower, for example 15 minutes (PT15M).

image

After the approval action add an HTTP action with the same URL as the HTTP trigger and the same Body – which is the data you passed in with a title and description. Next open the Configure run after option.

image

Uncheck the is successful option, and check the has timed out option.

image

The effect of this is that if the Approval was approved or rejected within 29 days, then all is good. If not, the approval request will actually be removed and a new one is issued by re-running the flow.

Of course, you might want to add e-mail alerts or other logic when an action times out before you re-try it. You could also add a parameter in the initial JSON data saying which run this is, and then increment per run if you want to re-try X number of times. Either way you should have a staring point to continue building out the logic you need for long running flow.

Summary

Even though a Microsoft Flow times out after 30 days, there are ways to architect yourself around this by using multiple flows which either call on each other, or call on themselves. Or if a timeout within 90 days works for you, you could move your Microsoft Flow to run as a Logic App instead.

10 comments:

  1. I'm curious if a potential corner case can be a recursion issue since the initial flow is still running while invoking itself? Have you tested if there are any limits on recursion depth or flow stack depth?

    ReplyDelete
    Replies
    1. Nope, take a look at http://www.techmikael.com/2017/10/blocking-vs-non-blocking-flows.html

      Tested and works just fine :)

      Delete
  2. Hi Mikael,

    Great article! Is it possible though that instead of the trigger as HTTP, I'd use SharePoint's "When an item is created or modified" to monitor requests? And for the last one, instead of an HTTP, I'll do an "Update Item" for SharePoint as well?

    Reason being is that the flow we deployed is being used by our organization right now, and there are approvals already running.

    I'd appreciate your prompt response. Thanks!
    David

    ReplyDelete
    Replies
    1. I guess an item update should retrigger the flow just fine. Just make sure you don't have more flows triggering so you get in a loop of sorts.

      Delete
  3. Milkael,
    I need to setup an auto approval workflow to go through 3 consecutive approval levels. When a request is submitted, it goes to the first approver for approving. After the approver is approved, update the SP list, then it goes to the second approver, then third approver. After the third approver approves, the approval process is complete. I am testing using one workflow for the approval process. I know that the approval will take months. With one flow, how can I reset the flow again and again until all 3 approvers approve?

    Thanks for your prompt response.

    ReplyDelete
    Replies
    1. You need to keep state and pass that to the new Flow you start if it times out. Draw it out on a white board, then implement :)

      Delete
  4. Which way do you interprete Chris McNultys announcement concerning the licensing change of HTTP connectors?
    https://techcommunity.microsoft.com/t5/Office-Retirement-Blog/UPDATED-Updates-to-Microsoft-Flow-and-PowerApps-for-Office-365/ba-p/289589

    If the HTTP connector really is going to be a premium connector, then your proposed architecture would require at least a Flow Plan 1. I.e. an Office 365 Plan is not sufficient to apply this architecture (which would be a real harm to Flow developers)

    ReplyDelete
    Replies
    1. For every user which is involved in a Flow using HTTP connectors, you need a license for them. And I wrote this before the change - so someone will have to rethink this :) Or move to LogicApps and calculate what the cost would be there.

      Delete
  5. Does it mean either Flow Plan1 or Plan2 is required for the Flow developers only? Is Office 365 Plan sufficient enough for end users who use the app?

    ReplyDelete
    Replies
    1. To be fully license compliant (as far as I can understand) you need a P1 for the dev, and a P1 for anyone involved in the running of the Flow - triggering it via some user action somewhere.

      Delete