Recent Publications

Research papers that are using SEE Data – Technion and Out-of-Technion Publications starting from 2013 year, listed chronologically and alphabetically

Recent Publications (research papers from 2013-2020 years listed chronologically and alphabetically)
catalog keyyear paper typedata type
rp12020Altman D., Yom-Tov G., Olivares M., Ashtar S., Rafaeli A. Do Customer Emotions Affect Worker Speed? An Empirical Study of Emotional Load in Online Customer Contact Centers, Forthcoming in MSOM, 2020, online appendix.

Problem Definition: Research in Operations Management has focused mainly on system-level load, ignoring the fact that service agents and customers express a variety of emotions that may impact service processes and outcomes. We introduce the concept of emotional load—the emotional demands that customer behaviors impose on service agents—to analyze how customer emotions affect service worker’s behavior.
Academic / Practical Relevance: Most theories in Organizational Behavior literature predict that emotions expressed by customers reduce agent’s cognitive abilities and therefore should reduce the agent’s speed (e.g. by increasing the service time required to serve an angry customer). We aim to shed light on the magnitude of that phenomenon while addressing important econometric challenges. We also investigate an important mechanism that drives this relation, namely, agent effort. We discuss practical opportunities that arise from measuring emotional load, and how it can be used to enhance productivity.
Methodology: We measure the emotional load of agents using sentiment analysis tools that quantify positive/negative customer emotion expressions in an online chat-type contact center, and link it to agent behavior: response time, and the length and number of messages required to complete a service request. Identifying a causal effect of customer emotion on agent behavior using observational data is challenging because there are confounding factors associated to the complexity of service requests, which are related to both customer emotions and agent behavior. Our identification strategy uses panel data and exploits the variation across messages within a focal request, using fixed effects to control for unobserved factors associated to case complexity. Instrumental variables are also used to address issues of measurement error and other endogeneity problems; the instruments are based on exogenous shocks to agent performance indicators that have been studied in the service operations literature.
Results: Analyses show that emotional load created by negative customer emotions increases agent response time (RT), the length of the agent messages (a measure of effort) and the required number of messages needed to complete a service request. Emotional load and agent RT reciprocally effect each other, with long agent RTs and a high number of messages producing more negative customer emotion.
Managerial Implications: We suggest that the emotional content in customer communications should be an important factor to consider when assigning workload to agents in a service system. Our study provides a rigorous methodology to measure the emotional content from customer text messages and objectively evaluate its associated workload. We discuss how this can be used to improve staffing decisions and dynamic workload routing through real-time monitoring of emotional load.

conference paperonline chats
rp22020Berkenstadt G., Gal A., Senderovich A., Shraga R., Weidlich M. Queueing Inference for Process Performance Analysis with Missing Life-Cycle Data.ICPM 2020: 57-64.

Abstract: Measuring key performance indicators, such as queue lengths and waiting times, using event logs serve for improvement of resource-driven business processes. However, existing techniques assume the availability of complete life cycle information, including the time a case was scheduled for execution (aka arrival times). Yet, in practice, such information may be missing for a large portion of the recorded cases. In this paper, we propose a methodology to address missing life-cycle data by incorporating predicted information in business processes performance analysis. Our approach builds upon techniques from queueing theory and leverages supervised learning to accurately predict performance indicators based on an event log with missing data. Our experimental results using both synthetic and real-world data demonstrate the effectiveness of our approach.

conference paperhospitals
rp32020Carmeli N., Yom-Tov G.B., Boxma O. State-Dependent Estimation of Delay Distributions in Fork-Join Networks

Problem definition: Developing estimators for delay distributions in a queueing network with a Fork-Join structure.
Academic / Practical Relevance: Delay announcements have become an essential tool in service system operations; they influence customer behavior and network efficiency. Most current delay announcement methods are designed for relatively simple call-center environments. Here we focus on a complex Fork-Join network of an Emergency Department. Such systems are mostly in a transient state and although queues for each station are fairly short, delays are long. These delays are the result of both resource scarcity and synchronization delays.
Methodology: We develop an exact analysis of the system. The analysis is based on recursively constructing the LaplaceStieltjes transform of the joint distributions, conditioning on customers’ movements in the network. We then examine the accuracy of the proposed approach on data of an Emergency Department.
Results: Estimations are provided throughout the customer’s stay in the network under Markovian assumptions; we then discuss (and provide) relaxations of both service time distribution and structural model assumptions. We provide evidence that the methodology developed is better than Last-to-Enter-Service (LES) estimators (which are based on snapshot principle arguments), reducing Root-Mean-Squared-Error (RMSE) by 25–30%. Our use case demonstrates the importance of incorporating speedup effects of service rates into delay estimators, which are naturally captured by our model, improving accuracy by 10%.
Managerial Implications: The results of our study allow management to implement delay announcements in complex Fork-Join networks. We also highlight the power of using exact approaches instead of relying on approximations, though the latter might be necessary for larger systems.

working paperhospitals
rp42020Castellanos A., Yom-Tov G.B., Goldberg Y. Silent Abandonment in Contact Centers: Estimating Customer Patience from Uncertain Data

Abstract: In the quest to improve services, companies offer customers the opportunity to interact with agents through contact centers, where the communication is mainly text-based. This has become one of the favorite channels of communication with companies in recent years. However, contact centers face operational challenges, since the measurement of common proxies for customer experience, such as knowledge of whether customers have abandoned the queue and their willingness to wait for service (patience), are subject to information uncertainty. We focus this research on the impact of a main source of such uncertainty: silent abandonment by customers. These customers leave the system while waiting for a reply to their inquiry, but give no indication of doing so, such as closing the mobile app of the interaction. As a result, the system is unaware that they have left and waste agent time and capacity until this fact is realized. In this paper, we show that 30%–67% of the abandoning customers abandon the system silently, and that such customer behavior reduces system efficiency by 5%–15%. To do so, we develop methodologies to identify silent-abandonment customers in two types of contact centers: chat and messaging systems. We first use text analysis and an SVM model to obtain the actual abandonment level. We then use a parametric estimator and develop an expectation-maximization algorithm to estimate customer patience accurately, as customer patience is an important parameter for fitting queueing models to the data. We show how accounting for silent abandonment in a queueing model improves dramatically the estimation accuracy of key measures of performance. Finally, we suggest strategiesto op erationally cope with the phenomenon of silent abandonment.

working paperonline chats; online messaging
rp332020Chen M., Baron O., Mandelbaum A., Wang J., Yom-Tov G.B., Arber N. Waiting Time Management in a Healthcare Open-Shop Network, Manlu Chen video INFORMS 2020 Virtual Conference.

Abstract: Motivated by a large healthcare clinic that provides periodic health screening services, we study how to improve the system performance of an open-shop service system using priority and routing policies combined with information technologies. We consider the system performances from both macro system-level perspective, such as total wait, and micro station-level perspective, such as excessive waits at single stations. We empirically evaluate automation efforts done by the clinic to improve the streamline of patients using automatic routing and queue communication using SMS notifications and find that such practices have negligible impacts on system performance. We, therefore, use a stylized queueing model and simulations to study priority policies for open-shop networks that consider customers' profiles. We demonstrate that such policies may outperform the station-level FCFS policy by simultaneously improving both the macro- and micro-level performance measures. One of the challenges we uncover is the tension between pooling consid erations that promote last-minute decisions and behavioral considerations that promote early decisions to enable customers to smooth transfer from one station to another. To overcome this tension, we propose a buffer strategy that postpones the routing decisions, which leaves the system more exibility and gathers more information for decision making. We demonstrate the efficacy of this policy in improving and balancing these performance measures.

working paperhealthcare clinic
rp52020Daw A., Castellanos A., Yom-Tov G.B., Pender J., Gruendlinger L. The Co-Production of Service: Modeling Service Times in Contact Centers Using Hawkes Processes

Abstract: In customer support centers, a successful service interaction involves a dialogue between a customer and an agent. Both parties depend on one another for information and problem solving, and this interaction defines a co-produced service process. In this paper, we propose, develop, and compare new stochastic models for the co-production of service in a contact center. To this end, we develop and examine alternative Hawkes process models. More specifically, our models incorporate both dynamic busyness factors that depend on the agent workload (e.g. concurrency) as well as dynamic factors that depend on the inner-mechanics of the interaction (e.g. number of words each party wrote). To understand how well our Hawkes models describe the message-timestamps, we compare the goodness-of-fit of these models on contact center data from industry.
We show that the word-count bivariate Hawkes model, which takes into account the mutual interaction and the amount of information provided by each party, fits the data the best. In addition to a great goodness-of-fit, the Hawkes models allow us to construct explicit expressions for the relationship between the correspondence rates of each party and the conversation progress. These formulae illustrate that the agent is more dominant in pacing the service along in the short term, but that the customer has a more profound effect on the duration of the conversation in the long run. Finally, we use our models to predict the future level of activity within a given conversation, through which we find that the bivariate Hawkes processes that incorporate the amount of information provided by each party or the sentiment expressed by the customer give us the most accurate predictions.

working paperonline messaging
rp152020Mandelbaum A., Momcilovic P., Trichakis N., Kadish S., Leib R., Bunnell C.A. (2020) Data-Driven Appointment-Scheduling Under Uncertainty: The Case of an Infusion Unit in a Cancer Center. Management Science 66(1):243-270.

Abstract: Service systems are often stochastic and preplanned by appointments, yet implementations of their appointment systems are prevalently deterministic. At the planning stage of healthcare services, for example, customer punctuality and service durations are often assumed equal their means - and this gap, between planned and reality, motivated our research. Specifically, we consider appointment scheduling and sequencing under a time-varying number of servers, in a data-rich environment where service durations and punctuality are uncertain. Our data-driven approach, based on infinite-server queues, yields tractable and scalable solutions that accommodate hundreds of jobs and servers. We successfully test our approach against near-optimal algorithms (which exist for merely single servers). This entails the development of a data-driven robust optimization approach with novel uncertainty sets. To test for practical performance, we leverage a unique data set from a cancer center that combines real-time locations, electronic health records, and appointments log. Focusing on one of the center’s infusion units (roughly 90 daily appointments, 25+ infusion chairs), we reduce cost (waiting plus overtime) on the order of 15%–40% consistently, under a wide range of experimental setups.

journal articlehospitals
rp72020Westphal M., Yom-Tov G.B., Parush A., Carmeli N., Shaulov A., Shapira C., Rafaeli A. A Patient-Centered Information System (myED) for Emergency Care Journeys: Design, Development, and Initial Adoption, JMIR Form Res 2020;4(2):e16410. (DOI:10.2196/16410)

Background: Medical care is highly complex in that it addresses patient-centered health goals that require the coordination of multiple care providers. Emergency department (ED) patients currently lack a sense of predictability about ED procedures. This increases frustration and aggression. Herein, we describe a system for providing real-time information to ED patients regarding the procedures in their ED medical journey.
Objective: This study aimed to develop a system that provides patients with dynamically updated information about the specific procedures and expected waiting times in their personal ED journey, and to report initial evaluations of this system.
Methods: To develop the myED system, we extracted information from hospital databases and translated it using process mining and user interface design into a language that is accessible and comprehensible to patients. We evaluated the system using a mixed methods approach that combined observations, interviews, and online records.
Results: Interviews with patients, accompanying family members, and health care providers (HCPs) confirmed patients’ needs for information about their personal ED journey. The system developed enables patients to access this information on their personal mobile phones through a responsive website. In the third month after deployment, 492 of 1614 (30.48%) patients used myED. Patients' understanding of their ED journey improved significantly (F8,299=2.519; P=.01), and patients showed positive reactions to the system. We identified future challenges, including achieving quick engagement without delaying medical care. Salient reasons for poor system adoption were patients' medical state and technological illiteracy. HCPs confirmed the potential of myED and identified means that could improve patient experience and staff cooperation.
Conclusions: Our iterative work with ED patients, HCPs, and a multidisciplinary team of developers yielded a system that provides personal information to patients about their ED journey in a secure, effective, and user-friendly way. MyED communicates this information through mobile technology. This improves health care by addressing patient psychological needs for information and understanding, which are often overlooked. We continue to test and refine the system and expect to find positive effects of myED on patients' ED experience and hospital operations.

journal articlehospitals
rp62020Yom-Tov G.B., Yedidsion L., Xie Y. An Invitation Control Policy for Proactive Service Systems: Balancing Efficiency, Value and Service Level, M&SOM, 2020 (DOI: 10.1287/msom.2019.0852) (Link to the manuscript:Proactive_service_systems_2_2_2019).

Problem definition: We study the problem of designing a dynamic invitation policy for proactive service systems with finite customer patience under scarce capacity. In such systems, prior knowledge regarding customer value or importance is used to decide whether the company should offer service or not. Academic/practical relevance: Proactive service systems are becoming more popular, as data availability and machine-learning techniques are developed to forecast customer needs. However, very little is known about the efficient use of such tools to promote and manage service systems. Methodology: We use fluid approximation and the Filippov convex method to analyze system dynamics and develop approximations for important performance measures. Results: We show that whereas prioritizing customers in descending order of their r-µ ranking (as long as there are idle servers in the system) is optimal on the fluid level, refinements are necessary in the presence of abandonment on the stochastic level. We propose an r-µ-N policy to account for customer patience. Managerial implications: Our policy can be used to promote service effectivenes and allow decision makers the means to trade off service level against costs in such systems explicitly. Using a case study of a transportation service provider, we show that such a policy can triple revenue compared with random arrivals (nonproactive policy).

conference paperonline chats
rp252020Yu Q. When Providing Wait Times, It Pays to Underpromise and Overdeliver,  Harvard Business Review. (Digital Article), 2020.

Summary: Virtual queues, or systems that allow you to hold your place without physically standing in line, have become commonplace in restaurants, call centers, and many other businesses — and how you build those systems can have a major impact on the customer’s experience. In this piece, the author shares key takeaways from over a decade’s worth of research on how companies can optimize their virtual queuing systems. Specifically, she suggests that providing wait time estimates can reduce customers’ average wait time, and that providing pessimistic estimates (i.e., telling customers they will have to wait longer than they actually will) can improve the customer experience. In addition, the research shows that providing more frequent progress updates also improves the customer experience, and that customers who wait for longer than expected will take longer when their turn finally arrives (suggesting that pessimistic estimates can also help businesses to increase throughput).

digital articlecall centers
rp242020Yu Q., Allon G., Bassamboo A. The Reference Effect of Delay Announcements: A Field Experiment, accepted, Management Science, 2020.

Abstract: We explore whether customers are loss averse in time and how delay information may impact such reference-dependent behavior using observational and field-experiment data from two call centers of an Israeli bank. We consider settings with no announcements and announcements of different accuracy levels. We face two key challenges: (1) we do not directly observe the reference points customer use as any other studies using field data; and (2) it is difficult to separate the reference-dependent behavior from the potential non-linear waiting cost of customers. To address these challenges, we develop a dynamic decision model with consumer learning, through which we infer the reference point each customer used during any given call. The reference points may be different across different customers and evolve across different calls of the same customers. We also exclude the alternative explanation by showing that our main reference-dependent models better explain the observed customer abandonment than models where customers have non-linear waiting cost. Our results indicate that customers are loss averse regardless of the availability or accuracy of the announcements, when their waiting time is relatively long (90s or longer). While delay announcements do not alter the nature that customers are loss averse, accurate announcements may affect customers' belief about the offered waiting time and thus impact the reference points. Through counterfactual studies, we demonstrate that providing delay announcements improves the call center performance given the loss aversion behavior observed in our data. Interestingly, as customers become more loss averse, the value of providing delay announcements decreases.

journal articlecall centers
rp82019Carmen R., Yom-Tov G., Van Nieuwenhuyse I., Joubert B., Ofran Y. The role of specialized hospital units in infection and mortality risk reduction among patients with hematological cancers, PLoS ONE 14(3): e0211694. (

Patients with hematological malignancies are susceptible to life-threatening infections after chemotherapy. The current study aimed to evaluate whether management of such patients in dedicated inpatient and emergency wards could provide superior infection prevention and outcome.
We have developed an approach allowing to retrieve infection-related information from unstructured electronic medical records of a tertiary center. Data on 2,330 adults receiving 13,529 chemotherapy treatments for hematological malignancies were identified and assessed. Infection and mortality hazard rates were calculated with multivariate models. Patients were randomly divided into 80:20 training and validation cohorts. To develop patient-tailored risk-prediction models, several machine-learning methods were compared using area under the curve (AUC).
Of the tested algorithms, the probit model was found to most accurately predict the evaluated hazards and was implemented in an online calculator. The infection-prediction model identified risk factors for infection based on patient characteristics, treatment and history. Observation of patients with a high predicted infection risk in general wards appeared to increase their infection hazard (p = 0.009) compared to similar patients observed in hematology units. The mortality-risk model demonstrated that for infection events starting at home, admission through hematology services was associated with a lower mortality hazard compared to admission through the general emergency department (p = 0.007). Both models show that dedicated hematological facilities and emergency services improve patient outcome post-chemotherapy. The calculated numbers needed to treat were 30.27 and 31.08 for the dedicated emergency and observation facilities, respectively. Infection hazard risks were found to be non-monotonic in time.
The accuracy of the proposed mortality and infection risk-prediction models was high, with the AUC of 0.74 and 0.83, respectively. Our results demonstrate that temporal assessment of patient risks is feasible. This may enable physicians to move from one-point decision-making to a continuous dynamic observation, allowing a more flexible and patient-tailored admission policy

journal articlehospitals
rp92019Rafaeli A., Altman D., Yom-Tov G.B. Cognitive and Emotional Load Influence Response Time of Customer-Service Agents: A Large Scale Analysis of Chat Service Conversations. Proceedings of the 52nd Hawaii International Conference on System Sciences (HICSS-52), January 8-11, 2019, Hawaii, US, 10 pages. (DOI:10.24251/hicss.2019.231) (Link to the manuscript: HICSS-52_0190)

Abstract: We highlight two psychological aspects of the load in service work – cognitive load (amount of information customers present) and emotional load (emotions customers present), and examine their effects on response time of service agents, in service conversations conducted using text-based chats. Using operational data of 145,995 chat service conversations, we show that cognitive load and emotional load increase agent response time both between and within service conversations. Our analyses unpack common assumptions that number of customers is identical to amount of work load, and shed light on customer-agent dynamics both between and within service conversations. In studying text-based service communication, which is rapidly expanding and insufficiently studied, we open up exciting opportunities for further research.

conference paperonline chats
rp102019Senderovich A., Beck J.C., Gal A., Weidlich M. Congestion Graphs for Automated Time Predictions. AAAI 2019: 4854-4861.

Abstract: Time prediction is an essential component of decision making in various Artificial Intelligence application areas, including transportation systems, healthcare, and manufacturing. Predictions are required for efficient resource allocation and scheduling, optimized routing, and temporal action planning. In this work, we focus on time prediction in congested systems, where entities share scarce resources. To achieve accurate and explainable time prediction in this setting, features describing system congestion (e.g., workload and resource availability), must be considered. These features are typically gathered using process knowledge, (i.e., insights on the interplay of a system’s entities). Such knowledge is expensive to gather and may be completely unavailable. In order to automatically extract such features from data without prior process knowledge, we propose the model of congestion graphs, which are grounded in queueing theory. We show how congestion graphs are mined from raw event data using queueing theory based assumptions on the information contained in these logs. We evaluate our approach on two real-world datasets from healthcare systems where scarce resources prevail: an emergency department and an outpatient cancer clinic. Our experimental results show that using automatic generation of congestion features, we get an up to 23% improvement in terms of relative error in time prediction, compared to common baseline methods. We also detail how congestion graphs can be used to explain delays in the system.

conference paperhospitals
rp112019Senderovich A., Booth Kyle E.C., Beck J.C. Learning Scheduling Models from Event Data. ICAPS 2019: 401-409.

Abstract: A significant challenge in declarative approaches to scheduling is the creation of a model: the set of resources and their capacities and the types of activities and their temporal and resource requirements. In practice, such models are developed manually by skilled consultants and used repeatedly to solve different problem instances. For example, in a factory, the model may be used each day to schedule the current customer orders. In this work, we aim to automate the creation of such models by learning them from event data. We introduce a novel methodology that combines process mining, timed Petri nets (TPNs), and constraint programming (CP). The approach learns a sub-class of TPN from event logs of executions of past schedules and maps the TPN to a broad class of scheduling problems. We show how any problem of the scheduling class can be converted to a CP model. With new instance data (e.g., the day’s orders), the CP model can then be solved by an off-the-shelf solver. Our approach provides an end-to-end solution, going from event logs to model-based optimal schedules. To demonstrate the value of the methodology we conduct experiments in which we learn and solve scheduling models from two types of data: logs generated from job-shop scheduling benchmarks and real-world event logs from an outpatient hospital.

conference paperhospitals
rp122019Senderovich A., Di Francescomarino C., Maggi F.M. From knowledge-driven to data-driven inter-case feature encoding in predictive process monitoring. Inf. Syst. 84: 255-264 (2019).

Abstract: Predictive process monitoring (PPM) is a research area that focuses on predicting measures of interest (e.g., the completion time) for running cases based on event logs. State-of-the-art PPM techniques only consider intra-case information that comes from the case whose measures of interest one wishes to predict. However, in many systems, the outcome of a running case depends on the interplay of all cases that are being executed concurrently, or can be derived from the characteristics of cases that are executed in the same period of time. For example, in many situations, running cases compete over scarce resources, and the completion time of a running case can be derived from the number of similar cases running concurrently. In this work, we present a general framework for feature encoding that relies on a bi-dimensional state space representation. The first dimension corresponds to intra-case dependencies and utilizes existing feature encoding techniques. The second dimension encodes inter-case features using two approaches: (1) a knowledge-driven encoding (KDE), which assumes prior knowledge on case types, and (2) a data-driven encoding (DDE), which automatically identifies case types from data using case proximity metrics. Both approaches partition the event log into sets of cases that share common characteristics, and derive features according to these commonalities. We demonstrate the usefulness of the proposed framework with an empirical evaluation carried out against two real-life datasets coming from an outpatient hospital process and a manufacturing process.

journal articlehospitals
rp132019Senderovich A., Weidlich M., Gal A. Context-aware temporal network representation of event logs: Model and methods for process performance analysis. Inf. Syst. 84: 240-254.

Abstract: Analysing performance of business processes is an important vehicle to improve their operation. Specifically, an accurate assessment of sojourn times and remaining times enables bottleneck analysis and resource planning. Recently, methods to create respective performance models from event logs have been proposed. These works have several limitations, though: They either consider control-flow and performance information separately, or rely on an ad-hoc selection of temporal relations between events. In this paper, we introduce the Temporal Network Representation (TNR) of a log. It is based on Allen’s interval algebra, comprises the pairwise temporal relations for activity executions, and potentially incorporates the context in which these relations have been observed. We demonstrate the usefulness of the TNR for detecting (unrecorded) delays and for probabilistic mining of variants when modelling the performance of a process. In order to compare different models from the performance perspective, we further develop a framework for measuring performance fitness. Under this framework, TNR-based process discovery is guaranteed to dominate existing techniques in measuring performance characteristics of a process. In addition, we show how contextual information in terms of the congestion levels of the process can be mined in order to further improve capabilities for performance analysis. To illustrate the practical value of the proposed models, we evaluate our approaches with three real-life datasets. Our experiments show that the TNR yields an improvement in performance fitness over state-of-the-art algorithms, while congestion learning is able to accurately reconstruct congestion levels from event data.

journal articlehospitals
rp142019Shraga R., Gal A., Schumacher D., Senderovich A., Weidlich M. Inductive Context-aware Process Discovery. ICPM 2019: 33-40.

Abstract: Discovery plays a key role in data-driven analysis of business processes. The vast majority of contemporary discovery algorithms aims at the identification of control-flow constructs. The increase in data richness, however, enables discovery that incorporates the context of process execution beyond the control-flow perspective. A "control-flow first" approach, where context data serves for refinement and annotation, is limited and fails to detect fundamental changes in the control-flow that depend on context data. In this work, we thus propose a novel approach for combining the control-flow and data perspectives under a single roof by extending inductive process discovery. Our approach provides criteria under which context data, handled through unsupervised learning, take priority over control-flow in guiding process discovery. The resulting model is a process tree, in which some operators carry data semantics instead of control-flow semantics. We evaluate the approach using synthetic and real-world datasets and show that the resulting models are superior to state-of-the-art discovery methods in terms of measures that are based on multi perspective alignments.

conference paperhospitals
rp272019Webb E., Yu Q., Bretthauer K. Linking Delay Announcements, Abandonment, and Service Time, under revision for the special issue at OR.

Abstract: Using call center field data, we examine the reference-dependent impact of delay announcements on both customer abandonment behavior and time spent in service. Delay announcements induce a reference point telling customers how long they must wait for service. Customers behave differently before and after this reference point. Our novel contributions include demonstrating the effects of delay announcements on service time, as well as simultaneously considering both reference-dependent abandonment behavior and reference- dependent service times. One additional minute spent waiting after the reference point increases the hazard rate of abandonment by up to 52%, a much larger effect than the hazard rate decrease from having one additional minute to go before the reference point. For those customers who do not abandon, we also observe reference-dependent behavior during service. Conditional on the same total wait in queue, one additional minute spent waiting after the reference point triggers up to 21 extra seconds in service. While longer total waiting time in queue also leads to longer service time, its effect is much smaller than this reference-dependent effect. Reference-dependent behaviors in abandonment and service impact system-level congestion in opposite directions. Therefore, managers should consider both effects when considering operational decisions, including staffing and customer prioritization.

journal articlecall centers
rp282019Whitt W., Zhang X. Periodic Little’s Law, OPERATIONS RESEARCH Vol. 67, No. 1, January–February 2019, pp. 267–280.

Abstract: Motivated by our recent study of patient flow data from an Israeli emergency department (ED), we establish a sample path periodic Little’s law (PLL), which extends the sample path Little’s law (LL). The ED data analysis led us to propose a periodic stochastic process to represent the aggregate ED occupancy level, with the length of a periodic cycle being 1 week. Because we conducted the ED data analysis over successive hours, we construct our PLL in discrete time. The PLL helps explain the remarkable similarities between the simulation estimates of the average hourly ED occupancy level over a week using our proposed stochastic model fit to the data and direct estimates of the ED occupancy level from the data. We also establish a steady-state stochastic PLL, similar to the time-varying LL.

journal articlehospitals
rp302018Bastianin A., Galeotti M., Manera M. Statistical and economic evaluation of time series models for forecasting arrivals at call centers, Empirical Economics, 2018.

Abstract: Call centers’ managers are interested in obtaining accurate point and distributional forecasts of call arrivals in order to achieve an optimal balance between service quality and operating costs. We present a strategy for selecting forecast models of call arrivals which is based on three pillars: (i) flexibility of the loss function; (ii) statistical evaluation of forecast accuracy; and (iii) economic evaluation of forecast performance using money metrics. We implement fourteen time series models and seven forecast combination schemes on three series of daily call arrivals. Although we focus mainly on point forecasts, we also analyze density forecast evaluation. We show that second-moment modeling is important for both point and density forecasting and that the simple seasonal random walk model is always outperformed by more general specifications. Our results suggest that call center managers should invest in the use of forecast models which describe both first and second moments of call arrivals.

journal articlecall centers
rp162018Senderovich A., Shleyfman A., Weidlich M., Gal A., Mandelbaum A. To aggregate or to eliminate? Optimal model simplification for improved process performance prediction. Inf. Syst. 78: 96-111 (2018).

Abstract: Operational process models such as generalised stochastic Petri nets (GSPNs) are useful when answering performance questions about business processes (e.g. ‘how long will it take for a case to finish?’). Recently, methods for process mining have been developed to discover and enrich operational models based on a log of recorded executions of processes, which enables evidence-based process analysis. To avoid a bias due to infrequent execution paths, discovery algorithms strive for a balance between over-fitting and under-fitting regarding the originating log. However, state-of-the-art discovery algorithms address this balance solely for the control-flow dimension, neglecting the impact of their design choices in terms of performance measures. In this work, we thus offer a technique for controlled performance-driven model reduction of GSPNs, using structural simplification rules, namely foldings. We propose a set of foldings that aggregate or eliminate performance information. We further prove the soundness of these foldings in terms of stability preservation and provide bounds on the error that they introduce with respect to the original model. Furthermore, we show how to find an optimal sequence of simplification rules, such that their application yields a minimal model under a given error budget for performance estimation. We evaluate the approach with two real-world datasets from the healthcare and telecommunication domains, showing that model simplification indeed enables a controlled reduction of model size, while preserving performance metrics with respect to the original model. Moreover, we show that aggregation dominates elimination when abstracting performance models by preventing under-fitting due to information loss.

journal articlehospitals
rp172018Yom-Tov G.B., Gurtz T. Delay Guarantee Planning of Call-Back Options in Time-Varying Service Systems. In Winter Simulation Conference, December 9-12, 2018, Gothenburg, Sweden (DOI:10.1109/WSC.2018.8632519). (Link to the manuscript call_back).

Abstract: Many service centers offer a “call-back” option, in which customers entering the queue are informed of the anticipated (on-line) wait and can choose to wait either online or off-line till an agent contacts them. We show that such a policy has the potential of both improving service performance and server utilization, by balancing the load between overloaded and underloaded periods. However, our analysis suggests that companies need not offer that service at all times, and that the delay guarantees proposed should be planned according to the anticipated load throughout the day. In order to optimize the operation of such a system,we develop an Iterative Simulation Algorithm to determine what delay guarantees the company should offer in a time-varying environment. Those guarantees depend on service level targets the company wishes to provide and the delay sensitivity of the online vs. off-line customers.

conference papercall centers
rp182017Senderovich A., Weidlich M., Gal A. Temporal Network Representation of Event Logs for Improved Performance Modelling in Business Processes. BPM 2017: 3-21 (best paper award!)

Abstract: Analysing performance of business processes is an important vehicle to improve their operation. Specifically, an accurate assessment of sojourn times and remaining times enables bottleneck analysis and resource planning. Recently, methods to create respective performance models from event logs have been proposed. These works are severely limited, though: They either consider control-flow and performance information separately, or rely on an ad-hoc selection of temporal relations between events. In this paper, we introduce the Temporal Network Representation (TNR) of a log, based on Allen’s interval algebra, as a complete temporal representation of a log, which enables simultaneous discovery of control-flow and performance information. We demonstrate the usefulness of the TNR for detecting (unrecorded) delays and for probabilistic mining of variants when modelling the performance of a process. In order to compare different models from the performance perspective, we develop a framework for measuring performance fitness. Under this framework, we provide guarantees that TNR-based process discovery dominates existing techniques in measuring performance characteristics of a process. To illustrate the practical value of the TNR, we evaluate the approach against three real-life datasets. Our experiments show that the TNR yields an improvement in performance fitness over state-of-the-art algorithms.

conference paperhospitals
rp292017Whitt W., Zhang X. A data-driven model of an emergency department, Operations Research for Health Care, 12 (2017) 1–15.

Abstract: This paper develops an aggregate stochastic model of an emergency department (ED) based on a careful study of data on individual patient arrival times and length of stay in the ED of the Rambam Hospital in Israel, which was used in a large-scale exploratory data analysis by Armony et al. (2015). This data set is of special interest because it has been made publicly available, so that the experiments are reproducible. Our analysis confirms the previous conclusions about the time-varying arrival rate and its consequences, but we also find that the probability of admission to an internal ward from the ED and the patient length-of-stay distribution should be time varying as well. Our analysis culminates in a new time-varying infinite-server aggregate stochastic model of the ED, where both the length-of-stay distribution and the arrival rate are periodic over a week.

journal articlehospitals
rp262017Yu Q., Allon G., Bassamboo A. How do Delay Announcements Shape Customer Behavior? An Empirical Study, Management Science, 63(1): 1-20, January 2017.

Abstract: In this paper, we explore the impact of delay announcements using an empirical approach by analyzing the data from a medium-sized call center. We first explore the question of whether delay announcements impact customers’ behavior using a nonparametric approach. The answer to this question appears to be ambiguous. We thus turn to investigate the fundamental mechanism by which delay announcements impact customer behavior, by constructing a dynamic structural model. In contrast to the implicit assumption made in the literature that announcements do not directly impact customers’ waiting costs, our key insights show that delay announcements not only impact customers’ beliefs about the system but also directly impact customers’ waiting costs. In particular, customers’ per-unit waiting cost decreases with the offered waiting times associated with the announcements. The results of our counterfactual analysis show that it may not be necessary to provide announcements with very fine granularity.

journal articlecall centers
rp192016Senderovich A., Weidlich M., Yedidsion L., Gal A., Mandelbaum A., Kadish S., Bunnell C.A. Conformance checking and performance improvement in scheduled processes: A queueing-network perspective. Inf. Syst. 62: 185-206 (2016).

Abstract: Service processes, for example in transportation, telecommunications or the health sector, are the backbone of today׳s economies. Conceptual models of service processes enable operational analysis that supports, e.g., resource provisioning or delay prediction. In the presence of event logs containing recorded traces of process execution, such operational models can be mined automatically.
In this work, we target the analysis of resource-driven, scheduled processes based on event logs. We focus on processes for which there exists a pre-defined assignment of activity instances to resources that execute activities. Specifically, we approach the questions of conformance checking (how to assess the conformance of the schedule and the actual process execution) and performance improvement (how to improve the operational process performance). The first question is addressed based on a queueing network for both the schedule and the actual process execution. Based on these models, we detect operational deviations and then apply statistical inference and similarity measures to validate the scheduling assumptions, thereby identifying root-causes for these deviations. These results are the starting point for our technique to improve the operational performance. It suggests adaptations of the scheduling policy of the service process to decrease the tardiness (non-punctuality) and lower the flow time. We demonstrate the value of our approach based on a real-world dataset comprising clinical pathways of an outpatient clinic that have been recorded by a real-time location system (RTLS). Our results indicate that the presented technique enables localization of operational bottlenecks along with their root-causes, while our improvement technique yields a decrease in median tardiness and flow time by more than 20%.

journal articlehospitals
rp202015Armony M., Israelit S., Mandelbaum A., Marmor Y.N., Tseytlin Y., Yom-Tov G.B. On Patient Flow in Hospitals: A Data-Based Queueing-Science Perspective, Stochastic Systems, Vol. 5, No. 1, 146-194, 2015 (DOI: 10.1214/14-SSY153). (Link to the manuscript: Patient flow (May 2015) Extended Version

Abstract: Hospitals are complex systems with essential societal benefits and huge mounting costs. These costs are exacerbated by inefficiencies in hospital processes, which are often manifested by congestion and long delays in patient care. Thus, a queueing-network view of patient flow in hospitals is natural for studying and improving its performance. The goal of our research is to explore patient flow data through the lens of a queueing scientist. The means is exploratory data analysis (EDA) in a large Israeli hospital, which reveals important features that are not readily explainable by existing models.
Questions raised by our EDA include: Can a simple (parsimonious) queueing model usefully capture the complex operational reality of the Emergency Department (ED)? What time scales and operational regimes are relevant for modeling patient length of stay in the Internal Wards (IWs)? How do protocols of patient transfer between the ED and the IWs influence patient delay, workload division and fairness? EDA also underscores the importance of an integrative view of hospital units by, for example, relating ED bottlenecks to IW physician protocols. The significance of such questions and our related findings raise the need for novel queueing models and theory, which we present here as research opportunities.
Hospital data, and specifically patient flow data at the level of the individual patient, is increasingly collected but is typically confidential and/or proprietary. We have been fortunate to partner with a hospital that allowed us to open up its data for everyone to access. This enables reproducibility of our findings, through a user-friendly platform that is accessible via the Technion SEELab

journal articlehospitals
rp212015Senderovich A., Weidlich M., Gal A., Mandelbaum A. Queue mining for delay prediction in multi-class service processes. Inf. Syst. 53: 278-295.

Abstract: Information systems have been widely adopted to support service processes in various domains, e.g., in the telecommunication, finance, and health sectors. Information recorded by systems during the operation of these processes provides an angle for operational process analysis, commonly referred to as process mining. In this work, we establish a queueing perspective in process mining to address the online delay prediction problem, which refers to the time that the execution of an activity for a running instance of a service process is delayed due to queueing effects. We present predictors that treat queues as first-class citizens and either enhance existing regression-based techniques for process mining or are directly grounded in queueing theory. In particular, our predictors target multi-class service processes, in which requests are classified by a type that influences their processing. Further, we introduce queue mining techniques that derive the predictors from event logs recorded by an information system during process execution. Our evaluation based on large real-world datasets, from the telecommunications and financial sectors, shows that our techniques yield accurate online predictions of case delay and drastically improve over predictors neglecting the queueing perspective.

journal articlecall centers
rp312014Goldberg Y., Ritov Y., Mandelbaum A. Predicting the Continuation of a Function with Applications to Call Center Data. Journal of Statistical Planning and Inference 147 (2014) 53–65.

Abstract: We show how to construct the best linear unbiased predictor (BLUP) for the continuation of a curve, and apply the proposed estimator to real-world call center data. Using the BLUP, we demonstrate prediction of the workload process, both directly and based on prediction of the arrival counts. The Matlab code and all data sets in the presented examples are available in the supplementary material.

journal articlecall centers
rp222014Senderovich A., Weidlich M., Gal A., Mandelbaum A. Queue Mining - Predicting Delays in Service Processes. CAiSE 2014: 42-57.

Abstract: Information systems have been widely adopted to support service processes in various domains, e.g., in the telecommunication, finance, and health sectors. Recently, work on process mining showed how management of these processes, and engineering of supporting systems, can be guided by models extracted from the event logs that are recorded during process operation. In this work, we establish a queueing perspective in operational process mining. We propose to consider queues as first-class citizens and use queueing theory as a basis for queue mining techniques. To demonstrate the value of queue mining, we revisit the specific operational problem of online delay prediction: using event data, we show that queue mining yields accurate online predictions of case delay.

conference papercall centers
rp232014Senderovich A., Weidlich M., Gal A., Mandelbaum A. Mining Resource Scheduling Protocols. BPM 2014: 200-216.

Abstract: In service processes, as found in the telecommunications, financial, or healthcare sector, customers compete for the scarce capacity of service providers. For such processes, performance analysis is important and it often targets the time that customers are delayed prior to service. However, this wait time cannot be fully explained by the load imposed on service providers. Indeed, it also depends on resource scheduling protocols, which determine the order of activities that a service provider decides to follow when serving customers. This work focuses on automatically learning resource decisions from events. We hypothesize that queueing information serves as an essential element in mining such protocols and hence, we utilize the queueing perspective of customers in the mining process. We propose two types of mining techniques: advanced classification methods from data mining that include queueing information in their explanatory features and heuristics that originate in queueing theory. Empirical evaluation shows that incorporating the queueing perspective into mining of scheduling protocols improves predictive power.

conference papercall centers
rp322013Aksin Z., Ata B., Emadi S., Su C. Structural estimation of callers' delay sensitivity in call centers. Management Science, 59(12), December 2013, 2727-2746.

Abstract: We model the decision-making process of callers in call centers as an optimal stopping problem. After each waiting period, a caller decides whether to abandon a call or continue to wait. The utility of a caller is modeled as a function of her waiting cost and reward for service. We use a random-coefficients model to capture the heterogeneity of the callers and estimate the cost and reward parameters of the callers using the data from individual calls made to an Israeli call center. We also conduct a series of counterfactual analyses that explore the effects of changes in service discipline on resulting waiting times and abandonment rates. Our analysis reveals that modeling endogenous caller behavior can be important when major changes (such as a change in service discipline) are implemented and that using a model with an exogenously specified abandonment distribution may be misleading.

journal articlecall centers