Many organizations face two inherent problems with AI: the need to automate at least some processes so humans can do innovative work, and the fact that dozens of existing chatbots are either insufficient or error-prone.
Hybrid AI is not a new concept. The most common definition of hybrid AI is technology that combines symbolic AI (human intelligence) with non-symbolic AI (machine intelligence) to deliver better results.
Usama Fayyad, president of technology and strategy consulting firm Open Insights, described machine learning as an iterative improvement of adaptive algorithms based on training data.
“Whether you are building deep learning or higher-level models [such as] probabilistic Bayesian models, you need a way to get the right training data, which usually comes from humans providing the right classification or interpretation,” Fayyad said. “Human interpretation and labeling are essential for learning systems ranging from machine-learned ranking in a core web search engine to training in autonomous vehicles.”
As for the fusion of deep learning methods and symbolic methods, Fayyad made distinctions between procedural knowledge and declarative knowledge. Procedural knowledge means that humans know how to do something without being able to explain it, whereas Declarative knowledge can be verbalized. For example, many speech recognition and vision problems are procedural in nature, as they are difficult for humans to explain; therefore, they lend themselves more to black-box approaches or those that lack transparency.
“I consider hybrid solutions to be very important, both for dealing with procedural tasks and for filling current knowledge gaps,” Fayyad said. “In my opinion, hybrid solutions are the right approach in almost all cases, especially if we want to explain and understand what AI does.”
The resources needed for effective hybrid AI
Examples of successful hybrid AIs demonstrate both domain knowledge and AI expertise in solving real-world problems. Without domain knowledge, the solution tends not to match the problem. Without AI expertise, it can be difficult to understand the challenges and what to do to solve them.
“End-users who are the intended consumers of certain predictions may be assigned an active role in a hybrid AI system as end-decision makers on those predictions and may accept, invalidate, or modify each prediction based on their own personal knowledge. and contextual,” said Fabio Pirovano, chief technology officer at Docebo, a provider of AI-powered learning suites. “To be effective, however, the AI system must fulfill a ‘contract’ with the end user by making its predictions available to experts in a timely enough manner.”
The most needed technology support is the ability to record final decisions made by experts, either for offline analysis by the data scientists responsible for the original AI model, or for use as benchmark data. additional training that fundamentally improves the models.
While there are many technology building blocks available, creating a cohesive end-to-end solution tends to be a patchwork endeavor. Pirovano said he considers the most practical example of hybrid AI today to be the human-in-the-loop, because the technology tools needed to take advantage of symbolic reasoning and statistical learning are relatively small. immature from a business perspective.
Having the right mindset is also important, and that starts with identifying a business problem and then using the right technology to solve it, which may or may not include hybrid AI.
“The most important mindset is one where we have a deep understanding of not only the limitations of algorithms, but also the deep dependence on data quality, availability, and issues,” Fayyad said. “Most importantly, an understanding of the solution we will deliver will require continuous feedback and rebuilding as the data, domain environment, and requirements change.”
Common benefits and challenges
Examples of hybrid AIs today are most effective when humans and machines respectively do what they do best.
“Humans are good at making judgments, while machines are good at processing,” said Adnan Masood, Ph.D., chief AI/ML architect at digital transformation company UST. “The machine can process 5 million videos in 10 seconds, but I can’t. So let the machine [to] do their job, and if anyone smokes in those videos, I’ll be the judge of how that smoking is portrayed.”
Fundamentally, the effectiveness of hybrid AI depends on human judgment for training and optimization in most use cases. Otherwise, a chatbot could degrade the customer experience, for example. Therefore, the first important challenge is to equip hybrid AI projects with the appropriate technical expertise. The second is to overcome both the lack of industry best practices on how hybrid AI systems look and the lack of tools and frameworks to implement those best practices.
“The goal should be to understand when and how symbolic AI can best be applied and fruitfully combined with statistical learning models,” said Docebo’s Pirovano.