Implementing good DevOps practices with the LUIS Cognitive Service has been traditionally hard. Largely because the initial focus of the service has been on the underlying performance and on the portal experience - making a complex machine learning problem easy to use by any developer. In fact, now the portal is so easy to use, it can be used by non-developers too, such as business folk and even content editors. The downside to this initial engineering focus, was that there was limited public API capability that you could leverage in order to automate some of the more ardous tasks typcially faced by enterprise organisations such as:

  • Limited support for different environments
  • No/fragmented CLI tooling
  • Limited testing capability

But, over the last 12+ months things have steady improved and much of these challenges have been addressed

LUIS Multiple API Endpoints for authoring & predictions:

LUIS now offers a much richer public API, including separate endpoints:

Convergence of Bot Framework/LUIS CLI Tooling

What was once a fragmented toolset has now been converged into a single Bot Framework CLI tool:

DevOps Focus

A set of DevOps tools for NLU services, including easy to use Azure DevOps Pipelines Tasks that do the heavy lifting for training/testing LUIS models:

LUIS DevOps with Github

Now, there is guidance and best practice advice for settingup Github Actions: https://github.com/Azure-Samples/LUIS-DevOps-Template which includes:

  • Documentation
  • End-to-end walkthrough

Covering:

  • Github flow branch management
  • Setting up build agent dependencies
  • Handling secrets/environment variables
  • Azure authentication
  • Leveraging bf & NLU.DevOps cli tools
  • Testing models
  • Analyzing model performance
  • Publishing models

Continuous Integration Workflow architecture

For the complete CI workflow you can look in the repo, but it’s useful to understand the key stages within the process:

Note here, the initial publish of the model uses curl post rather than the bf luis:application:publish - at the time of writing this blog, there is is an issue with the bf cli tool when using the --direct publish switch. However, this has now been addressed via this PR and will be in the next release of the tooling.

Pull Request Workflow architecture

The PR workflow is much simpler:

So, if you’re interested in setting up Github Actions with LUIS - go check out the new resource: https://github.com/Azure-Samples/LUIS-DevOps-Template