• mobile_menu_1_icon


    • Certifications

    • Programs

  • mobile_menu_2_icon


  • mobile_menu_3_icon


  • mobile_menu_4_icon


Mobile Header Background

Deploying ML and AI in the Real World Using Rendezvous Architecture

By John White Last Updated on Jun 11, 2021

There’s Fire behind That Smoke

Machine learning and AI are getting a lot of press recently, but it is more than just propaganda. Applications and systems that use ML/AI are creating serious amounts of value, often in very amazing applications. But using these new techniques to build systems doesn’t get you a free permit on basic engineering issues. To get an applied and sustainable value you will have to connect these systems to real problems whose solution has actual business value. To prosper in this, you have to have a repeatable engineering process that can organize these ML/AI models reliably.

In many ways, there are strong similarities with software development processes, but with important differences. You have to have version control (but it is different with data), nonstop deployment (but this is riskier with models), and automated testing (but that is tougher when we are learning a model somewhat than coding a program).

In over-all, building ML/AI systems breaks down into three major actions. These are 1) gathering data to train a model with, 2) training the model, and 3) organizing the model. Training the model is what most people and maximum courses focus on, but gathering data and deploying (and monitoring) models are at least as central. Let’s start with a look at the deployment process.

Suggested Read: History of artificial intelligence

Rendezvous with Destiny

What are the experiments that need to be met for actual deployment in real business settings?

With the normal software, you deploy new services after testing them, but you naturally only have one version running at a time (excluding possibly during the promotion process itself). That doesn’t work with services grounded on machine learning for several reasons, the chief one being that it is main to monitor the operation of new models and associate them to existing models before obliging to them fully.


This is usually called a champion/challenger process. Even after a new contestant model has been accepted as the new champion, it is shared to keep the old champion around for a bit “just in case”. It is also mutual to keep a stable version of the model known as a canary model around for a very long time for judgment resolves. We don’t just want to keep these contestants, old champs and such around, we want to keep them actively appraising all requests so that we can associate what how they are doing on every demand that we get.

Also read: moving deeper into AI

The result of using rendezvous architecture is that you can bring up new models at any time and watch how they perform for as long as you like. You can compare the conduct of challenger models against the behavior of the winner or older models in a truthful production environment. Once you are content that you want the new challenger to step up, you don’t have to change anything excluding the formation of the rendezvous server; factually, all that happens is that the new champion’s results are no longer discounted.

The importance of using a rendezvous architecture is that you get the welfares of continuous deployment, but with some crinkles to address the special needs compulsory by working with machine learning models. The rendezvous architecture also gives you the aptitude to monitor the process of models (by associating them to each other), or to notice changes in the kinds of queries being acknowledged (by looking for changes in the delivery of the outputs of the canary over time).

Featured article: How do machines learn?


Even though the rendezvous architecture focuses typically on the deployment problem, it can help with data collection as well. Since all models get all the similar requests, you can add a special model (known as a decoy) that produces no results, but just archives every request that it sees. This archived data is guaranteed to be a correct record of what the other models have seen, and receiving that precisely precise is a big part of the data collection problem. This means that by rendezvous to crack the deployment problem for ML/AI systems goes a good ways toward solving the data collection as well. If you are planning to deploy a system that uses machine learning or AI, a rendezvous architecture may well be what you need.