Without contexts will you trust a data science model prescription?
It is increasingly becoming a common practice where enterprises are deriving values from optimization and machine learning driven data science models. Business units focused on marketing, customer experience management, equipment maintenance, services, scheduling, etc., have been building several standalone data science models and in some cases have attained sufficient maturity. In a typical business operation, many functions are connected, dependent and even synchronized. For example, a decision on an equipment maintenance issue combined with customer support during that issue resolution may impact customer experience. What happens when more and more data science models make isolated decisions, sometimes at rapid rates, with minimal or no explanations? Can multiple data science model responses be made to be consistent, collaborative, transparent and yet be responsive? This paper considers a need defined in National Academies of Sciences, Engineering, and Medicine 2012. Guidebook for Airport Irregular Operations (IROPS) Contingency Planning. Washington, DC: TheNational Academies Press. https://doi.org/10.17226/14667. The guidebook mentions that more than one party being involved in taking care of passengers during IROPS events, and with the advent of digital technologies, airlines can be quite efficient in providing necessary support to their customers. Due to the networked nature of air transportation system, the severity of these impacts can be high when they cascade. Airlines tend to rely on a suite of optimization and data science applications, often developed by multiple teams or vendors due to the complex nature of both business and compliance needs. When not solved for consistency, collaboration and transparency needs, airline operators are forced to use as much information as they can gather from piece wise recommendations, and use their judgment and prior knowledge to handle customer needs, such as re-accommodations, etc., in an artful and humane manner as possible. In this paper we showcase how we can tackle the consistency need to begin with. Subsequent papers will build on the consistency framework to show case collaborative and transparent behavior without compromising responsiveness among a set of collaborating data science models. described.
A perspective of Collaborating Data Science Models and Contexts
It is increasingly becoming a common practice where enterprises are deriving values from optimization and machine learning driven data science models. Availability of data, big data platforms, robust machine learning frameworks, cloud native platforms, etc., have enabled talented data science teams to bring in opportunities for business units to experiment, fine tune and excel. Business units focused on needs such as marketing, customer experience management, equipment maintenance, services, scheduling, etc., have been building several standalone data science models and in some cases have attained sufficient maturity.
In a typical business operation, many functions are connected, dependent and even synchronized. For example, a decision on equipment maintenance issue combined with customer support during that issue resolution may impact customer experience. While this is a common scenario, enterprises still have disconnects. Such disconnects get widespread, as more and more data science models make isolated recommendations, with employees feeling they are unable to offer the best information and/or service. This situation is wide spread irrespective of the industry, such as airlines, retail stores, telecommunication services, etc.
So, how to make data science models to be consistent, collaborative, transparent and yet be responsive? This paper addresses the first need where multiple data science models benefit from consistent and up-to-date information. Subsequent papers will build on the consistence framework and demonstrate collaboration and transparency characteristics.
Business Problem Definition
We will study the IROPS case to illustrate the data science model needs defined earlier, from National Academies of Sciences, Engineering, and Medicine 2012. Guidebook for Airport Irregular Operations (IROPS) Contingency Planning. Washington, DC: The National Academies Press. https://doi.org/10.17226/14667, .
While the scope of IROPS is large, it nevertheless is a good use case to illustrate the value of enabling collaborative behavior among data science models. The guidebook for IROPS planning provides a passenger centrist schematic shown. More than one party is involved in taking care of passengers, and with the advent of digital means, airlines can be quite efficient in providing necessary IROPS support to customers and employees. Due to the networked nature of air transportation system, the severity of these impacts can be high when they cascade. Airlines use several vertical or narrowly focused optimization solutions to minimizing costs, maximizing passengers’ and employees’ satisfaction, etc. The vertical optimization solutions are very useful and effective. And yet comprehensive cost functions that span the breadth of airlines' operations and beyond, do not exist and so they often combine multiple outcomes for optimal Re-accommodation solutions. The step of combining multiple optimization outcomes is handled at best in a semi-automated manner with partial or no knowledge on the drivers for the individual optimization outcomes.
in addition, the Guidebook, also identifies three actions that are critical: communication, coordination, and collaboration on a timescale.
Many of the solutions used in tackling an IROPS scenario are often developed by different vendors therefore requiring distinct inputs, perhaps even through different mechanisms, and hence enterprises find it harder to provide consistent inputs to them. When inputs are not coordinated or consistent, solutions from vertical data science or optimization solutions may suffer from race conditions, thereby making the semi-automated combining of piece-wise conclusions prohibitive. Coordinating inputs can be tricky, where information tends to flow and update at varying speeds. In such circumstances, operators choose to use as much information as they can gather and use their judgment and prior knowledge to handle Reaccom situations in an artful and humane manner as feasible. For a simplified example, consider an eco system where there are a few vertical problem solvers focused on applying advanced optimization strategies in respective domains, assisting business functions during IROPS event.
CUSTOMER VALUES SYSTEM: A comprehensive solution that leverages all interactions, perceptions, and patterns’ based derivation of operational dimensions to assess value of customers so that businesses may tailor their services, interactions and remediations with a goal to improve overall value for both customers and airline
OPERATIONAL COMPLIANCE & OPTIMIZATION SYSTEMS: These are usually a set of highly advanced applications that employ domain and compliance requirements in optimizing actions or event responses. Following lists a few key focus areas detail.
AIRCRAFT MAINTENANCE AND COMPLIANCE
CREW, TERMINAL & GROUND OPERATIONS OPTIMIZATION
CARGO ROUTING OPTIMIZATION
Consistency Framework - Characteristics of contexts for data science models
Of the three characteristics mentioned, consistency is the fundamental requirement. This characteristic must be addressed in terms of all that is exchanged between collaborating data science models. Keep in mind that one model's output could be consumed by another. Since most data science models used in operations need to be aware of up-to-date information, it is necessary that not only same information is provided to all of them, but they all should be the timely information as well, irrespective of the frequency of changes. We term all such information that is provided to any data science model as contexts. Here are some characteristics that a facility that manages contexts must uphold.
Platform options for shared contexts
Several options exist that may be used to implement a robust context service/facility. The three options discussed below have been in use at many enterprises for a while now, although not directly for the purpose we are after.
Architecture for consistency framework
We use the 4+1 view model of software architecture to illustrate concerns from various stakeholders of the architecture. https://www.cs.ubc.ca/~gregor/teaching/papers/4+1view-architecture.pdf
The rough class diagram references the “things” that play key roles at an architecture level. In a data centrist application, the classes give an idea of information on shareable context candidates. For our IROPS example, we have Passengers and Employees dealing with the objects listed, such as Flights, etc. Employees also use a few special purpose standalone applications. The interactions are mentioned between the objects.
We will roughly group the process view into two – one that directly deals with managing contexts and everything else that will be used are bundled as IROPS apps, while leaving the Collaboration and Transparency managers for later. Not all processes are shown in the demonstration section.
The consistency manager follows a simple equation – initialize, keep-up & retire. Obviously we need to support failure recovery and logging, etc. Only a subset of the development view is shown on the right.
The consistency components are purposely kept simplistic as we expect resilient performance from them. Since we are dealing with highly connected and yet geographically dispersed environment, in order to solve for holistic options, consistency manager needs to have a purview on entire operations. Some impacts are localized and some may be system wide. This is a critical requirement that is managed by simplistic design of the consistency manager.
The collaboration manager (to be discussed later) transforms base contexts into useful information to enable collaborative behavior.
The demonstration environment uses commodity and heterogeneous hardware, thanks to open source environment. The physical environment scales due to several Apache data ecosystem frameworks. In a real production environment, managed infrastructure may be required. In addition, the physical view provides an idea of what may be needed for subsequent material.
We will use scenarios that only concern with consistency manager in this paper.
Consistency Manager Demo
Four of the components listed in the Development View are shown below in 4 distinct windows. The upper left blue window shows outputs from handler interacting with Aircraft Maintenance model. The upper right black window deals with Crew Allocation model. The lower left white window deals with Cargo Routing model. The bottom right red window processes passenger activities and it is where we will add inputs. Data shown in red window is manually typed, but we might as well stream in real life. All of the handlers pay attention to the inputs, enabled by Kafka. It is important to pay attention to Kafka delivery mechanisms, such as consumer groups etc. We do implement idempotent code in our handlers so that we handle messages exactly once. Kafka delivers at least once and we make sure to manage occasional duplicates.
We have created the topics shown on the left, where acmaint, cargoroute & crewalloc are designed for our model handlers and the other three paxact, fltact, iropact represent activities from operational systems. However, in consistency manager demo, we will use just the paxact events for now. You may see below how the acmaint topic is defined. Keep in mind that in a true production scenario we will have a few additional configurations.
Subsequent demo screens show how each handler maintains consistent and unique states. In the bottom red window we will create passenger check ins and each handler states can be followed using the annotations added to the screens. In the red window each line represents a passenger check in for a flight at an airport.
In the following screen, the Aircraft Maintenance Recommend context shows three states that correspond to the last three passenger activities as indicated by the dotted lines. Choices exist for the handlers as to when a model needs to be invoked. in this demonstration we are simply focused on do we have consistent and up to date information so that whenever handlers need they all have the same information.
In the next paper we will use specific AI techniques to implement the collaborative and transparent manager.
We will use facilities like knowledge representation, predicate logic, production rules, etc., to build and drive on operational agenda.