TheModernCallCenter:AMulti-,PRODUCTION

科沃斯 5
ANDOPERATIONSMANAGEMENT Vol.16,No.6,November-December2007,pp.665–688issn1059-1478͉07͉1606͉665$1.25 POMS doi10.3401/poms.©2007ProductionandOperationsManagementSociety TheModernCallCenter:AMulti- DisciplinaryPerspectiveonOperations ManagementResearch ZeynepAksin•MorArmony•VijayMehrotra CollegeofAdministrativeSciencesandEconomics,KocUniversity,RumeliFeneriYolu,34450Sariyer-Istanbul,Turkey LeonardN.SternSchoolofBusiness,NewYorkUniversity,West4thStreet,KMC8–62,NewYork,NewYork10012,USA DepartmentofDecisionSciences,CollegeofBusiness,SanFranciscoStateUniversity,1600HollowayAvenue,SanFrancisco,California94132-1722,USA zaksin@ku.edu.tr•marmony@stern.nyu.edu•vjm@sfsu.edu Callcentersareanincreasinglyimportantpartoftoday’sbusinessworld,employingmillionsofagentsacrosstheglobeandservingasaprimarycustomer-facingchannelforfirmsinmanydifferentindustries.Callcentershavebeenafertileareaforoperationsmanagementresearchersinseveraldomains,includingforecasting,capacityplanning,queueing,andpersonnelscheduling.Inaddition,asmunicationsandinformationtechnologyhaveadvancedoverthepastseveralyears,theoperationalchallengesfacedbycallcentermanagershaveeplicated.Issuesassociatedwithhumanresourcesmanagement,sales,andmarketinghavealsoeincreasinglyrelevanttocallcenteroperationsandassociatedacademicresearch. Inthispaper,weprovideasurveyoftherecentliteratureoncallcenteroperationsmanagement.Alongwithtraditionalresearchareas,wepayspecialattentiontonewmanagementchallengesthathavebeencausedbyemergingtechnologies,tobehavioralissuesassociatedwithbothcallcenteragentsandcustomers,andtotheinterfacebetweencallcenteroperationsandsalesandmarketing.Weidentifyahandfulofbroadthemesforfutureinvestigationwhilealsopointingoutseveralveryspecificresearchopportunities. Keywords:callcenters;staffing;skill-basedrouting;personnelscheduling;outsourcingSubmissionsandeptance:Submissionsandeptance:ReceivedApril2007;revisionreceivedOctober 2007;eptedOctober2007.
1.Introduction Virtuallyallbusinessesareinterestedinprovidinginformationandassistancetoexistingandprospectivecustomers.Inrecentyears,thedecreasedcostsofmunicationsandinformationtechnologyhavemadeitincreasinglyeconomicaltoconsolidatesuchinformationdeliveryfunctions,whichledtotheemergenceofgroupsthatspecializeinhandlingcustomerphonecalls.Forthevastmajorityofthesegroups,theirprimaryfunctionistoreceivetelephonecallsthathavebeeninitiatedbycustomers.Suchoperations,knownas“inbound”callcenters,aretheicofthispaper. Inboundcallcentersareverylabor-intensiveoperations,withthecostofstaffmemberswhohandle phonecalls(alsoknownas“agents”)prising60–80%oftheoveralloperatingbudget.Inboundcallcentersmaybephysicallyhousedacrossseveraldifferentlocations,timezones,orcountries. Inboundcallcentersmakeupalargeandgrowingpartoftheglobaleconomy.Althoughreliableindustrystatisticsarenotoriouslyhardeby,theingCallManagementInstitute(ICMI),ahighlyreputableindustryassociation,regularlytrackspublishedindustrystatisticsfromseveralsources(/statistics/demographics.aspx).By2008,variousstudiescitedbyICMIpredictthefollowing: •TheUnitedStateswillhaveover47,000callcentersand2.7millionagents. 665 666 Aksin,Armony,andMehrotra:TheModernCallCenterProductionandOperationsManagement16
(6),pp.665–688,©2007ProductionandOperationsManagementSociety •Europe,theMiddleEast,andAfricatogetherwillhave45,000callcentersand2.1millionagents. •CanadaandLatinAmericawillhaveanestimated305,500and730,000agents,respectively. Meanwhile,thedemandforcallcenteragentsinIndiahasgrownsofastthatthelaborsupplyhasbeenunabletokeepupwithit:by2009,thedemandforagentsinIndiaisprojectedtobeover1million,andmorethan20%ofthosepositionswillbeunfilledbecauseofashortageofavailableskilledlabor. Whenacustomercallsaninboundcallcenter,variouscallhandlingandroutingtechnologieswillattempttoroutethecalltoanavailableagent.However,thereisoftennoagentavailabletoimmediatelyanswerthephonecall,inwhichcasethecustomeristypicallyputonholdandplacedinaqueue.Thecustomer,inturn,mayabandonthequeuebyhangingup,eitherimmediatelyafterbeingplacedonholdorafterwaitingforsomeamountoftimewithoutreceivingservice.Onceconnectedtoanagent,acustomerwillspeakwiththatagentforsomerandomtime,afterwhicheitherthecallwillpletedorthecustomerwillbe“handedoff”toanotheragentorqueueforfurtherassistance.Thequalityoftheserviceistypicallyviewedasafunctionofbothhowlongthecustomermustwaittoreceiveserviceandthevaluethatthecustomerattributestotheinformationandservicethatisreceived. Callcentermanagersareincreasinglyexpectedtodeliverbothlowoperatingcostsandhighservicequality.Tomeetthesepotentiallyconflictingobjectives,callcentermanagersarechallengedwithdeployingtherightnumberofstaffmemberswiththerightskillstotherightschedulesinordertomeetanuncertain,time-varyingdemandforservice.Traditionally,meetingthischallengehasrequiredcallcentermanagerstowrestlewithclassicaloperationsmanagementdecisionsaboutforecastingtraffic,acquiringcapacity,deployingresources,andmanagingservicedelivery. Inrecentyears,thecallcenterlandscapehasbeenalteredbyawidevarietyofmanagerialandtechnologicaladvances.Reducedinformationtechnologyandmunicationscosts—thesameforcesthatcontributedsignificantlytothegrowthofthecallcenterindustry—havealsoledtorapiddisaggregationofinformation-intensiveactivities(ApteandMason1995).Forcallcenters,thistranslatedintoincreasedcontractingofcallcenterservicestothirdpartiesmonlyreferredtoas“outsourcing”)andthedispersionofservicedeliverytolocationsacrosstheglobe(“offshoring”).Inaddition,advancesinmunicationstechnologiesenabledrichercallcenterworkflow,includingincreasinglyintelligentroutingofcallsacrossagentsandphysicalsites,automatedinteractionwithcustomerswhileonhold,andcallmessaging thatresultsinautomaticcallbackstocustomersonceanagentisavailable. Also,ascallcentersnowserveasthe“publicface”formanyfirms,thereisincreasingexecutiveconsiderationoftheirvitalroleincustomeracquisitionandretention.Similarly,themanagerialawarenessofcallcenters’potentialtogeneratesignificantincrementalrevenuebyaugmentingserviceencounterswithpotentialsalesopportunitieshasalsobeengrowingrapidly:forexample,arecentMcKinseystudyrevealedthatcreditpaniesgenerateupto25%ofnewrevenuefrominboundcallcenters(Eichfeld,Morse,andScott2006).However,forcallcentermanagers,thereissignificantplexityassociatedwithmanagingthisdualservice-and-salesrolepromisingresponsetimes,servicequality,andcustomersatisfaction. Finally,everycallcentermanagerisacutelyawarethatphoneconversationsbetweencustomersandagentsareinteractionsbetweenhumanbeings.Thissuggeststhatthepsychologicalissuesassociatedwiththeagents’experiencecanhaveamajorimpactonbothcustomersatisfactionandoverallsystemperformance.Althoughthesetypesofissueshavebeenresearchedextensivelybybehavioralscientists,operationsmanagementresearchershaveonlyrecentlybeguntoexplicitlyincludesuchfactorsinricheranalyticmodels. Giventhesizeofthecallcenterindustryandplexityassociatedwithitsoperations,callcentershaveemergedasafertilegroundforacademicresearch.Arelativelyrecentsurveypaper(Gans,Koole,andMandbelbaum2003)cites164papersassociatedwithcallcenter-relatedproblems,andanexpandedon-linebibliography(Mandbelbaum2004)includesover450papersalongwithdozensofcasestudiesandbooks.Inaddition,therehavebeenseveralmorespecializedsurveysassociatedwithcallcenteroperations,includingthatofKooleandMandelbaum(2002),whofocusedonqueueingmodelsforcallcenters;L’Ecuyer(2006),whofocusedonoptimizationproblemsforcallcenters;andKooleandPot(2006)andAksin,Karaesmen,andOrmeci(2007),whobothfocusedonmultiskillcallcenters. Thissurveyseekstoprovideabroadperspectiveonbothtraditionalandemergingcallcentermanagementchallengesandtheassociatedacademicresearch.Thespecificobjectivesandmajorcontributionsofthispaperareasfollows:
1.Toprovideasurveyoftheacademicliteratureassociatedwithtraditionalcallcenterproblemareassuchasforecasting,queueing,capacityplanning,andagentschedulingoverthepastfewyears;
2.Toidentifyseveralkeyemergingphenomenonthataffectcallcentermanagersandtocatalogthe Aksin,Armony,andMehrotra:TheModernCallCenterProductionandOperationsManagement16
(6),pp.665–688,©2007ProductionandOperationsManagementSociety 667 academicresearchthathasbeendoneinresponsetothesedevelopments;
3.Torecognizenewcallcenteroperationsmanagementparadigmsthatconsidertheroleofthecallcenterinhelpingfirmstoattract,retain,andgeneraterevenuefromcustomersandtoproposesomeimportantimplicationsofthesenewparadigmsonfutureresearch;
4.Tochronicleresearchonpsychologicalaspectsofcallcenteragentexperience,surveyrecentoperationsmanagementpapersthathaveincorporatedsomeoftheseideasintotheirmodeling,andsuggestwaysinwhichsuchworkcanbeincorporatedintofutureoperationsmanagementresearch;and
5.Tohighlightgapsinthecurrentliteratureoncallcenteroperationsmanagementandopportunitiesareasforfutureresearch.Theremainderofthepaperanizedasfollows.InSection2,wesurveyrecentworkontraditionalcallcenteroperationsmanagementproblems.Section3reviewsresearchthatconsidersdemandmodulationasanalternativetosupplysidemanagement.InSection4,welookattheresearchliteraturethatemergedasaresultoftechnology-driveninnovations,includingmulti-siteroutingandpooling,thedesignofmulti-skillcallcenters,theblendingofinboundcallswithothertypesofworkflowsuchasoutboundcallsandemails,andincreasedcallcenteroutsourcing.InSection5,weexamineseveralkeyhumanresourcesissuesthataffectcallcentersandchroniclerecentoperationsmanagementresearchthatsoughttoincorporatesomeofthesefactorsintotheirmodels.InSection6,weexploreresearchthatintegratescallcenteroperationswithsalesandmarketingobjectives,focusingoncross-sellingandlong-termcustomerrelationshipmanagement.Ineachoftheabovesections,wesuggestspecificopportunitiesforfutureresearch.mentsareprovidedinSection7.
2.ManagingCallCenterOperations:TheTraditionalView Traditionaloperationsmanagementchallengesforcallcentermanagersincludethedeterminationofhowmanyagentstohireatwhattimesbasedonalongtermforecastofdemandforservices(“resourceacquisition”)andtheschedulingofanavailablepoolofagentsforagiventimeperiodbasedondetailedshorttermforecastsforagiventimeperiod(“resourcedeployment”).Inaddition,onceinitialresourcedeploymentdecisionshavebeenmade,theremaybeadditionalshorter-termdecisionstobemade,includingforecastupdating,scheduleupdating,andrealtimecallrouting. Resourceacquisitiondecisionsmustbemadeseveralweeksandsometimesmonthsaheadoftimebe- causeofleadtimesforhiringandtrainingagents.Also,becausemostcallcentershavefairlyhighemployeeturnoverandabsenteeismlevels,modelsthatsupportresourceacquisitiondecisionsmustexplicitlyountforrandomattritionandabsenteeism. Resourcedeploymentdecisionsaretypicallymade1ormoreweeksinadvanceofwhenthecallsactuallyarrive.Acost-effectiveresourcedeploymentplanattemptstocloselymatchthesupplyofagentresourceswiththeuncertaindemandforservices.The(highlyvariable)demandforresourcesisexpressedintermsofcallforecasts,whichareposedofcallarrivaldistributionsandservicetimedistributions,bothofwhichvaryovertime.Thisvariabilitymeansthatbothforecastingandqueueingmodelsplayanimportantroleinmodelingresourcedeploymentdecisions.Fromaschedulingperspective,agentscantypicallybeassignedtoarangeofshiftpatterns,andtheprocessofdetermininganoptimal(ornear-optimal)schedulehasasignifiplexity. Inaddition,asnewdataaboutforecastsandagentavailabilityesavailableforagivendayorweek,thisinformationcanbeusedtomodifyboththeneartermcallarrivalforecastsandtheagentschedulesthataredrivenbythem.Finally,ascallsactuallyarrive,theremaybespecificdecisionstobemadeaboutqueuingpoliciesorcallrouting. Inthissection,webeginoursurveybylookingatrecentworkonthesecallcenteroperationsmanagementproblems.WefocusoncallforecastinginSection2.1,resourceacquisitioninSection2.2,andperformanceevaluation,staffing,scheduling,androutinginSection2.3.Next,weconsiderthebasicproblemsofstaffing,scheduling,androutingwhenarrivalratesarerandominSection2.4.Finally,Section2.5providesabriefoverviewofdevelopmentsinperformanceevaluationmodelsforcallcenters,reflectingsomeofthenewercharacteristicsofmoderncallcenters. 2.1.CallForecastingCallforecastsaredefinedby(a)thespecificqueueorcalltypeassociatedwiththeforecast;(b)thetimebetweenthecreationoftheforecastandtheactualtimeperiodforwhichtheforecastwascreated(oftenreferredtoastheforecasting“leadtime”);and(c)thedurationofthetimeperiodsforwhichtheforecastsarecreated,whichcanrangefrommonthly(tosupportresourceacquisitiondecisions)toshorttimeframes,suchas15-,30-,or60-minuteperiods(tosupportresourcedeploymentdecisions).Overtheyears,therehavebeenrelativelyfewpapersthatfocusedonforecastingcallvolumes,promptingGansetal.(2003)toassertthatcallforecastingwas“stillinitsinfancy.” However,inthepastfewyears,therehavebeenahandfulofimportantdevelopmentsinthecallfore- 668 Aksin,Armony,andMehrotra:TheModernCallCenterProductionandOperationsManagement16
(6),pp.665–688,©2007ProductionandOperationsManagementSociety castingfield,drivenbyincreasedavailabilityofhistoricaldatabasesofcallvolumesandbyutilizationandadaptationofnewtechniquesthathavebeenappliedtosimilarforecastingproblemsinotherapplicationareas. Weinberg,Brown,andStroud(2007)proposeamultiplicativeeffectsmodelforforecastingPoissonarrivalratesforshortintervals,typically15,30,or60minutesinlength,witha1-dayleadtime.Intheirsetting,thecallarrivalrateforagiventimeintervalofaparticulardayoftheweekismodeledastheproductoftheforecastedvolumeforthatdayoftheweekandtheproportionofcallsthatarriveinthattimeintervalplusarandomerrorterm.Toestimatethemodel’sparameters,theauthorsadoptaBayesianframework,proposingasetofpriordistributions,andusingaMonteCarloMarkovchainmodeltoestimatetheparametersoftheposteriordistribution. putationallyintensive,themethodologyproposedbyWeinberg,Brown,andStroud(2007)isquitevaluablefromanoperationalperspective.Inparticular,becausethemodelproducesforecastsofPoissonarrivalratesonanintra-dayintervalbasis,theseresultscanbeusedinconjunctionwithperformancemodelsandagentschedulingalgorithms.Inaddition,theauthorsproposeamodificationofthismethodtoallowforintra-dayforecastupdating,whichcaninturnbeusedtosupportintra-dayagentscheduleupdating.ThepaperincludesaforecastingcasestudyinwhichdatafromalargeNorthmercialbank’scallcentersareusedtotestboththe1-day-aheadforecastsandintra-dayforecastupdates,withverypromisingresults. SoyerandTarimcilar(2007)introduceanewmethodologyforcallforecastingthatdrawsonideasfromsurvivalanalysisandmarketingmodelsofcustomerheterogeneity.Specifically,thispapermodelscallarrivalsasamodulatedPoissonprocess,wherethearrivalratesaredrivenbyadvertisementsthatareintendedtostimulatecustomerstocontactthecallcenter.TheparametersforthecallintensityassociatedwitheachparticulartypeofadvertisementandfuturetimeintervalaremodeledbyaBayesianframework,usingaGibbssampler(DellaportesandSmith1993)toapproximatetheposteriordistributions.Theauthorsalsotesttheirmethodologybyconductingnumericalexperimentsusingcallvolumedatafromacallcenterforwhichallcallscanbetraceddirectlytospecificadvertisements,withtheforecastsbeingcreatedforsingle-andmulti-daytimeperiods. ShenandHuang(2007)developastatisticalmodelforforecastingcallvolumesforeachintervalofagivendayandalsoprovideanextensionoftheircoremodelingframeworktoountforintradayforecastupdating.Theirmodelisbasedontheuseofsingularvaluepositiontoachieveasubstantialdimen- sionalityreduction,andtheirapproachalsoposespredictivefactorsintointer-andintra-dayfeatures.Fortheempiricalcasespresented,themethodologyproducesforecaststhataremoreuratethanboththe(highlyunsophisticated)standardindustrypracticeandtheresultsfromWeinberg,Brown,andStroud(2007);themethodologyisalsosignificantlyputationallyintensivethantheMonteCarloMarkovchainmethodsofWeinberg,Brown,andStroud(2007). Taylor(2007)presentsanempiricalstudyparestheperformanceofawiderangeofunivariatemethodsinforecastingcallvolumesforseveralUKbankcallcentersaswellasfortheIsraelibankcallcenterdatafromBrownetal.(2005),consideringleadtimesrangingfrom1dayto2weeks.Taylor’sparisonincludesmethodsthathaveappearedpreviouslyinthecallcenterliterature,suchasseasonalAutoRegressiveMovingAveragemodeling(AndrewsandCunningham1995)anddynamicharmonicregression(Tychetal.2002),aswellasseveralothermodelsthathavenotpreviouslybeenusedforcallcenterforecasting.Thelattergroupincludesanexponentialsmoothingmodelfordoubleseasonalitythatwasoriginallydevelopedforforecastingshorttermelectricutilitydemand(Taylor2003);aperiodicAutoRegressivemodel;andamodelbasedonrobustexponentialsmoothingbasedonexponentiallyweightedleastabsolutedeviations(Cipra1992).Theparisonshowednoclear“winner,”becausedifferentmethodsprovedtobemoreeffectiveunderdifferentleadtimesanddifferentworkloads. 2.2.PersonnelPlanning:ResourceAcquisitionThecallcenterresourceacquisitionproblemhasbeenstudiedbyahandfulofresearchers.GansandZhou(2002)modelaprocessinwhichagentsarehiredandexperiencebothlearningandattritionovertime,demonstratingthatathresholdpolicyforhiringagentsisoptimalintheirsetting.Ahn,Righter,andShanthikumar(2005)lookatageneralclassofservicesystemsanddemonstratethatundertheassumptionofcontinuousnumberofagentswhocanbehiredandfiredatwill,theoptimalpolicyisofa“hire-up-to/firedown-to”form.Bordoloi(2004)binescontroltheoryandchance-constrainedprogrammingtechniquestoderivesteady-stateworkforcelevelsfordifferentknowledgegroupsandahiringstrategytoachievethesetargets.Bhandari,Harchol-Balter,andSchellerWolf(2007)considerboththehiringofregularworkersandthecontractingofpart-timeworkersalongwiththeoperationalproblemofdetermininghowmanypart-timeworkerstodeployunderdifferentloadconditions.Ryder,Ross,andhio(2008)examinetheimpactofdifferentroutingstrategiesonemployeelearninginamulti-skillenvironmentinan Aksin,Armony,andMehrotra:TheModernCallCenterProductionandOperationsManagement16
(6),pp.665–688,©2007ProductionandOperationsManagementSociety 669 attempttounderstandtheconnectionbetweenrouting,learning,andoverallstaffingneeds. Giventheimportanceoftheresourceacquisitiondecision,thereissignificantneedforadditionalresearchinthisarea,includingmodelsforlong-termforecasting,personnelplanningforgeneralmulti-skillcallcenters,andresourceacquisitionplanningforworksofserviceproviders(asdescribedbyKeblisandChen2006,forexample). 2.3.PersonnelPlanning:Staffing,Scheduling,andRouting Thetraditionalapproachtocallcenterresourcedeploymentdecisionsistoattempttobuildanagentschedulethatminimizescostswhileachievingsomecustomerwaitingtimedistributionobjectives.Assuch,targetedstaffinglevelsforeachperiodoftheschedulinghorizonaretypicallykeyinputstotheschedulingandrosteringproblems.Thesetargetsdependonbothhowmuchworkisarrivingintothecallcenteratwhattimes(asestimatedbythecallvolumeforecastsandtheforecastedmeanservicetimes)andhowquicklythecallcenterseekstoservethesecustomers(estimatedbysomefunctionofthecustomerwaitingtimedistribution).Oncetheforecastsandwaitingtimegoalshavebeenestablished,queueingperformanceevaluationmodelsareusedtodeterminethetargetednumberofserviceresourcestobedeployed.Theactualperformanceobtainedfromthedeployedresourcesalsodependsontheoperationalproblemofallocatingingcallstotheseresourcesdynamically,knownasthecallroutingproblem.Ourreviewfollowsthesamehierarchicalorderthatwouldbefollowedintheresourcedeploymentproblemforcallcenters:wefirstreviewstaffingproblems,thenprovideanoverviewofschedulingandrosteringproblems,andfinallydemonstratehowthecallroutingprobleminteractswiththem. 2.3.1.StaffingProblems.Simulationmodelsandanalyticqueueingmodelsarethetwoalternativestoperformanceevaluation.MehrotraandFama(2003)providesanoverviewoftheinputsrequiredforbuildingacallcentersimulationmodel,whileKooleandMandelbaum(2002),andMandelbaumandZeltyn(2006)aregoodsourcesforadetailedoverviewofqueueingmodelsofcallcenters. ThesimplestqueueingmodelofacallcenteristheM/M/squeue,alsoknownasanErlang-Csystem.Thismodelignoresblockingandcustomerabandonments.TheErlang-Bsystemincorporatesblockingofcustomers.TheErlang-CmodelisfurtherdevelopedtoincorporatecustomerimpatienceintheErlang-Asystem(t,Mandelbaum,andReiman2002).PerformancemeasuresandapproximationsfortheErlang-AsystemarediscussedbyMandelbaumandZeltyn(2007b).Sensitivityofthismodeltochangesin itsparametersisanalyzedbyWhitt(2006c),whereitisdemonstratedthatperformanceisrelativelyinsensitivetosmallchangesinabandonmentrates. Formostinboundcallcenters,themanagementobjectiveistoachieverelativelyshortmeanwaitingtimesandrelativelyhighagentutilizationrates.Gansetal.(2003)refertosuchanenvironmentasa“QualityandEfficiencyDriven”regime.Inthiscontext,letRbethesystem-offeredloadmeasuredintermsofthemeanarrivalratetimesandthemeanservicetime.Theso-called“square-rootsafety-staffingrule”stipulatesthatifRislargeenoughthenstaffingthesystemwithRϩ␤͌Rservers(forsomeparameter␤)willachievebothshortcustomerwaitingtimesandhighserverutilization. ThisrulewasfirstobservedbyErlang(1948)andwaslaterformalizedbyHalfinandWhitt(1981)fortheErlang-Cmodel(i.e.,anM/M/squeue).ItspracticaluracywastestedforservicesystemsbyKolesarandGreen(1998).ThisrulewasfurthersupportedbyBorst,Mandelbaum,andReiman(2004)andMaglarasandZeevi(2003)undervariouseconomicconsiderations.Thisrulehassincebeendemonstratedtoberobustwithrespecttomodelassumptionssuchascustomerabandonment(t,Mandelbaum,andReiman2002;ZeltynandMandelbaum2005),aninboundcallcenterwithacall-backoption(ArmonyandMaglaras2004a,b),andcallcenterswithmultiplequeuesandagentskills(Gurvich,ArmonyandMandelbaum2006,ArmonyandMandelbaum2004),whichwillbediscussedinmoredetailbelow. Borst,Mandelbaum,andReiman(2004)havealsoidentifiedtwootheroperatingregimes:thequalitydrivenandtheefficiencydriven(ED)regimes,whicharerationaloperatingregimesundercertaincostsstructures.IntheEDregimeserverutilizationisemphasizedoverservicequality;however,withcustomerabandonment,thisregimecanalsoresultinreasonableperformanceasmeasuredbyexpectedwaitingtimeandfractionofcustomerabandonment(Whitt2004b).WhitthasproposedfluidmodelsforsystemapproximationundertheEDregime(Whitt2006a,b)andhasshownitsapplicabilityinstaffingdecisionsunderuncertainarrivalrateandagentabsenteeism. Mostoftheearlyliteratureonstaffingdealswiththeseproblemsinsettingswithasinglepoolofhomogenousagents(seereferencesinGansetal.2003;t,Mandelbaum,andReiman2002;Borst,Mandelbaum,andReiman2004;Atlason,Epelman,andHenderson2004;andMasseyandWallace2006).Recentliteratureonstaffingmodelsfocusesonmultiskillsettings,thatis,incallcenterswherecallsofdifferenttypesareservedusingservicerepresentativeswithdifferentskills(Pot,Bhulai,Koole,2007;Bhulai,Koole,andPot,2007;CezikandL’Ecuyer, 670 Aksin,Armony,andMehrotra:TheModernCallCenterProductionandOperationsManagement16
(6),pp.665–688,©2007ProductionandOperationsManagementSociety 2006;ChevalierandVandenSchrieck,2006;HarrisonandZeevi,2004,WallaceandWhitt,2005,Armony,2005,Bassamboo,Harrison,andZeevi2005,2006).AdifferentsettingwithhomogeneousagentsservingvariouscustomertypestowhomdifferentiatedserviceisprovidedisanalyzedbyGurvich,Armony,andMandelbaum(2006).Aksin,Karaesmen,andOrmeci(2007),KooleandPot(2006),andL’Ecuyer(2006)surveyrecentresearchonmulti-skillcallcenterproblems. Typicallystaffingformulationsseektodeterminethenumberoffull-timeequivalentemployeesneededgivenanobjectivefunctionandsomeconstraints.Themostwidelyusedisastaffingcostminimizationobjectivewithservicelevelconstraints(see,forexample,Atlason,Epelman,andHenderson2004;CezikandL’Ecuyer,2006;Bhulai,Koole,andPot2007;JagermanandMelamed,2004;MandelbaumandZeltyn2007a),althoughstaffingproblemswithprofitmaximizationobjectiveshavealsobeenproposed(AksinandHarker,2003;KooleandPot,2005;Helber,Stolletz,andBothe2005;BaronandMilner,2006).Armonyetal.(2007)establishconvexitypropertiesparativestaticsforanM/M/squeuewithimpatience,demonstratingtherelationshipbetweenabandonmentsandoptimalstaffing.KooleandPot(2005a)showthattheseconvexitypropertiesfailtoholdwhenthebuffersizeisalsoadecisionvariable.Canonetal.(2005)formulatethestaffingproblemasadeterministicschedulingproblem. 2.3.2.ShiftSchedulingandRostering.Takingtheresultsfromthestaffingproblemasinputs,typicallyonaninterval-by-intervalbasis,theshiftschedulingproblemdeterminesanoptimalcollectionofshiftstobeworked,seekingtominimizecostswhileachievingservicelevelsorotherlaborrequirements.Closelyrelatedtotheschedulingproblem,therosteringbinesshiftsintorostersandprovidestheactualmatchingbetweenemployeesandrosters.Theschedulingproblemandtherosteringproblemhavebeenstudiedextensively,bothinthecontextofcallcenters(seereferencesinGansetal.2003)andinmoregeneralcontexts(Ernstetal.2004chroniclesover700papersonics).Inthissection,ratherthanattemptanextensivesurveyoftheschedulingandrosteringliterature,weinsteaddescribeseveraldifferentapproachestotheseproblems,alongwithillustrativerecentpapersandsomefruitfuldirectionsforfutureresearch. Thetraditionalapproachtotheschedulingproblemistoformulateandsolveamathematicalprogramtoidentifyaminimumcostschedule.Althoughvariantsofthisapproachhavebeenwidelyutilized,bothintheresearchliteratureandinindustrialapplications,overtheyearsseveralissueshavealsobeenidentifiedwiththisbasicmethod.Forlargecallcenterswithasingle queueofcallarrivalsandahomogeneouspoolofagents,eachwithseveralpossibleshiftandbinationsandassociatedrestrictions,thesizeofthemathematicalprogramgrowsveryrapidly.Thisissueisaddressedbyseveralresearchers,mostnotablyAykin(1996,2000),whomodelsflexiblebreakconstraintsforeachshiftandteststheproposedmethodologywithseverallargetestproblems. Anotherproblemwiththetraditionalmathematicalprogrammingapproachisthatitrequiresasinputatargetagentstaffinglevelforeachtimeinterval.Thisconceptoftargetstaffinglevelisinturnbasedontheassumptionthatallagentsareabletohandleallingcalls.However,inamulti-queue/multi-skillenvironment,thisassumptionisclearlyviolated,andmuchoftheworkinrecentyearshassoughttoaddressthisspecificingofthetraditionalmethodology.Fukunagaetal.(2002)proposeahybridmethodbinesschedulingheuristicswithsimulationtosimultaneouslysolveboththeschedulingandtherosteringproblemanddiscussmercialimplementationofthismethodthatisusedbyover1,000callcenterstoday.Similarly,CezikandL’Ecuyer(2006)proposeamethodologybineslinearprogrammingwithsimulationtodetermineaschedule.Avramidisetal.(2007)developsearchmethodsthatusequeueingperformanceapproximationstoproduceagentschedulesforamulti-skillcallcenter. Anotherstreamofresearchintheareaofcallcenterschedulingfocusesoneliminatingapproximationsthatresultfromthetraditionalseparationbetweenthestaffingandtheschedulingproblemsdescribedabove.Motivatedbythedependencyofadjacenttimeintervals’waitingtimedistributions,whichisignoredbytraditionalschedulingalgorithms,Atlason,Epelman,andHenderson(2004)usesubgradientinformationfortheobjectivefunctionalongwithsimulationinordertodetermineagentschedules.Inasimilarspirit,notingtraditionalmethodsassumethatservicelevelgoalsare“hardconstraints”thatmustbemetduringeachinterval,KooleandvanderSluis(2003)insteaddevelopaschedulingmethodologythatseekstomeetonlyanoverallservicelevelobjectiveoverthecourseofanentireschedulingperiod(typicallyadayoraweek).Ingolfsson,Cabral,andWu(2003)notethatthetraditionalstaffingmethodsusesteady-statestaffingmodelsforindividualintervalsandseektoeliminateerrorsinducedbythisapproximationbyusingtransientresultsonaperiod-by-periodbasis,whichtheyrefertoasthe“randomizationmethod,”alongwithintegerprogrammingtocreateagentschedules.Motivatedbythepotentialimpactofunderstaffingoncallabandonment,Saltzman(2005)andSaltzmanandMehrotra(2007)developandtestaschedulingmethodologybineslinearprogramming,tabusearch,andsimulationwhileincludingcoststostaff, Aksin,Armony,andMehrotra:TheModernCallCenterProductionandOperationsManagement16
(6),pp.665–688,©2007ProductionandOperationsManagementSociety 671 waitingtimes,andabandonedcallsintheobjectivefunction. Theseparationofshiftschedulingfromtheactualrosteringprocesspresentsanotherpotentialproblemwiththetraditionalapproach.Inpractice,themismatchbetweenthe(ideal)optimalshiftsandthe(actual)assignmentofshiftstoindividualagentscanhaveamajornegativeimpactontheoverallperformanceofthecallcenter,andthisimpactisoftenexacerbatedbyupdatestocallforecastsandschedulesthatresultfromnewinformationbeingobtainedaftertheinitialschedulehasbeencreated.Becauseofplexityassociatedwiththecoordinationofindividualagents’preferencesandrestrictions,manylargecallcentersandmulti-sitecallcenteroperationsrequireagentsto“bid”onparticularshiftssequentially,withtheorderofbiddingbasedonfactorssuchasseniorityandpreviousqualityofservicedelivered.Buildingonthispractice(knowninthecallcenterindustryas“shiftbidding”),Keblis,Li,andStein(2007)investigateanauction-basedapproachtotheproblemofmatchinglaborsupplywithlabordemandinacallcenter,allowingagentstopetitivelyfordifferentshifts.Inparticular,thistypeofbiddingmechanismsuggestsamethodforpricingservicesforpart-time“workathome”agents,whilealsofacilitatingreal-timescheduleadjustmentsasaresultofupdatedcallforecasts.Theissueofreal-timescheduleadjustmentsinserviceoperationshasalsobeenaddressedbyHur,Mabert,andBretthauer(2004),EastonandGoodale(2005),andMehrotra,Ozluk,andSaltzman(2006). 2.3.3.TheCallRoutingProblem.Theroutingproblemisacontrolproblemthatinvolvesassigningingcallstospecificagentsorpoolsofagentsandthenschedulingcallswhenseveralarewaitingforthesameagentpool.Thisproblemhasattractedalotofattentionasacallcenterapplicationandmoregenerallyasachallengingqueueingcontrolproblem(Ormeci,as,andEmmons2002;Ormeci,2004;GansandZhou,2003;Koole,Pot,andTalim2003;Atar,Mandelbaum,andReiman2004a,b;MandelbaumandStolyar,2004;HarrisonandZeevi,2004b;Armony,2005;deVericourtandZhou,2006;Bhulai,2005;KooleandPot,2006;Bassamboo,Harrison,andZeevi2005;Tezcan,2005;Atar,2005a,2005b;Jouinietal.2006,TezcanandDai,2006,GurvichandWhitt,2007). Theproblemsofstaffing,scheduling,androutingexhibithierarchicaldependency.Thecallroutingprobleminmulti-skillcallcentersisalsoknownasskills-basedrouting.Inmulti-skillsettings,howwellcallsarerouteddeterminestheeffectivenessofstaffusage,whilethestaffingproblemconstrainstheroutingdecision.Theseproblemsinteract,asexplainedviaexamplesinAksin,Karaesmen,andOrmeci(2007) andKooleandPot(2006),andfurtherinteractwiththeflexibilitydesignproblem(AksinandKaraesmen2003;Aksin,Karaesmen,andOrmeci2007).Thehierarchicaldependency,aswellasthecloseinteractionbetweenstaffingandrouting,maketheseproblemschallengingfromanoperationsresearchperspective(CezikandL’Ecuyer2006;HarrisonandZeevi2005;ArmonyandMaglaras2004a,WallaceandWhitt2005;Bhulai,Koole,andPot2007;Gurvich,Armony,andMandelbaum2006,Bassamboo,Harrison,andZeevi2006;ChevalierandVandenSchrieck2006).Evenwhentreatedinisolationandignoringimportantinterdependencies,obtainingoptimalsolutionsposesachallenge.Deterministiclinearprogramming,diffusion,orfluidapproximationshavebeenproposedtoethisprobleminlarge-scalecenters(ArmonyandMaglaras2004a,b;ArmonyandMandelbaum2004;HarrisonandZeevi2004,2005;Bassamboo,Harrison,andZeevi2006;Whitt2006a,b;TezcanandDai2006;GurvichandWhitt2007).Otherpapersusesimulationbinationwithoptimization(Atlason,Epelman,andHenderson2003;Atlason,Epelman,andHenderson2004;CezikandL’Ecuyer2006),losssystem,orotherapproximations(KooleandTalim,2000,ChevalierandTabordon,2003;Koole,Pot,andTalim2003;Shumsky2004;Chevalier,Shumsky,andTabordon2004;KooleandPot2005b;ChevalierandVandenSchrieck2006;Franx,Koole,andPot2006;Avramidisetal.2006)toenableanalysis. Despitethelargenumberofpapersdiscussedinthissection,webelievethattherearesignificantresearchopportunitieswiththeseclassicalproblems.Inparticular,capturingmoreofthedependencyandinteractionamongstaffing,scheduling,androutingisapromisingdirectionforfurtherresearch. 2.4.PersonnelPlanningunderArrivalRateUncertainty Historically,mostofthepapersinthecallcenterliteraturehavemodeledthearrivalprocesstobeatimeinhomogeneousPoissonprocessand,thus,forecastingcallvolumesisinmostcases(implicitlyorexplicitly)equivalenttoestimatingthetime-dependentPoissonarrivalrates.Thisassumptionisinmanycasesquitereasonable.Forexample,Brownetal.(2005)conductedanextensiveempiricalstudyofhistoricaldatafromanIsraelibank’scallcenteroperationsandconclusivelyfailedtorejectthehypothesisthatthecallarrivalsfollowatime-inhomogeneousPoissonprocess;however,inthesamestudy,afterusingcalltype,timeofday,anddayofweektobuildanempiricalmodeltoforecastthecallarrivalratesforshorttimeintervals,theauthorsconcludedthatthePoissonarrivalratesarenoteasilypredictable. Becauseofthedifficultyofuratelyforecastingcallarrivalrates,severalresearchershaveexploredthe 672 Aksin,Armony,andMehrotra:TheModernCallCenterProductionandOperationsManagement16
(6),pp.665–688,©2007ProductionandOperationsManagementSociety implicationsofmodelingcallarrivalswitharandomarrivalrate.Whitt(1999b)suggestsaparticularformofarandomarrivalrateforcapturingforecastuncertainty.ChenandHenderson(2001),Avramidis,Deslauriers,andL’Ecuyer(2004),Brownetal.(2005),andSteckley,Henderson,andMehrotra(2005)pointouttherandomnessofarrivalsinrealcallcenters,afeaturethatisignoredinmostoftheliterature.Steckley,Henderson,andMehrotra(2005),HarrisonandZeevi(2005),Robbinsetal.(2006),andTorzhkovandArmony(2007)analyzecallcenterperformanceunderrandomarrivals.Thompson(1999)andJongbloedandKoole(2001)providemethodsfordeterminingtargetstaffingwhenthearrivalrateisrandom.Ross(2001,Chapter4)offersextensionstothesquare-rootstaffingruletoountforarandomarrivalrate.Robbinsetal.(2007)considerthequestionofcross-trainingasubsetofagentsfromdifferentqueuestomeetdemandinthepresenceofuncertainarrivalrates.OtherrecentpapersthatfocusonplanningproblemsinthepresenceofrandomarrivalsarethosebySteckley,Henderson,andMehrotra(2007),Whitt(2006e),BaronandMilner(2006),BassambooandZeevi(2007),andAldorNoiman(2006). Anothertraditionalcallcentermodelingassumptionisthatthearrivalsduringonetimeperiodwithinaplanninghorizonareindependentofthearrivalsintheothertimeperiodsforpurposesofdeterminingstaffinglevelsandagentschedules.Green,Kolesar,andSoares(2001,2003)havedubbedthisthestationary,independent,periodbyperiodmethod.However,severalempiricalstudieshavedemonstratedthatformanycallcentersthereissignificantcorrelationincallvolumesacrosstimeperiods.Brownetal.(2005)developanon-linearleastsquaresmodelinwhichapreviousday’scallvolumeisanindependentvariableinpredictingthesubsequentday’scallvolume,producingroughlya50%reductioninthevariabilityoftheforecasteddailyvolumes.Motivatedbyempiricalanalysisofalargemunicationfirm’scallcentersthatdemonstratesbothgreater-than-Poissonvariabilityandstrongcorrelationacrosstimeperiodswithinthesameday,Avramidis,Deslauriers,andL’Ecuyer(2004)developandtestseveralanalyticmodelsinwhichthearrivalrateforeachintervalofthedayisarandomvariablethatiscorrelatedwiththearrivalratesoftheotherintervals.Steckley,Henderson,andMehrotra(2005)analyzedatafromseveralcallcentersandidentifysignificantcross-periodcorrelationincallvolumes;motivatedbytheseresults,Mehrotra,Ozluk,andSaltzman(2006)presentaframeworkforintradayforecastandscheduleupdatingthatutilizesthecallarrivalmodelofWhitt(1999b)tomodelcrossperiodcorrelation. Webelievethatthispointstoatleasttwointerestingandimportantareasforfutureresearch.First,thereis aneedforresearchintoadditionalperformanceanalysismodelsunderdifferentarrivalratevariabilityassumptions,aswellasformorevalidationofsuchassumptionswithoperationaldata.Second,reconsideringtheschedulingandrosteringproblemsunderthemoregeneralassumptionthatarrivalratesarerandomvariablesisanotherverypromisingareathatisjustnowbeginningtoreceiveattentionfromresearchers.Forexample,RobbinsandHarrison(2007)viewarrivalratevariabilityasaponentoftheagentschedulingproblemandproposeastochasticprogrammingsolutiontodeterminethebinationofagentsandshiftsthatexplicitlyountsfortheriskinherentinthearrivalrateuncertainty. 2.5.PerformanceEvaluationforModernCallCenters Ascallcentershaveevolvedintermsofsizeandconfiguration,andasmoreempiricalanalysishasshedlightonthefeaturesoftypicalqueueingmodelprimitiveslikearrivals,abandonment,andservicetimesinthesecenters,newperformanceevaluationmodelshavebeendevelopedandanalyzed.Thesemodelsaremotivatedbydifferentfeaturesofmoderncallcenters,aswellasempiricallyobservedcharacteristicsofqueueingmodelprimitives.ThelatteranalysishasbeeninitiatedbyaresearchcollaborationbetweenresearchersatTheTechnionandTheWhartonSchoolthathasprovidedacleansourceofcustomercallbasedcallcenterdatafromseveralsources,whichhassubsequentlybeendevelopedintopleteplatformfordata-basedanalysisofcallcenterproblems(adescriptionoftheDataMOCCAProjectcanbeobtainedfromhttp://iew3.technion.ac.il/serveng/References/DataMOCCA).Theimportantdistinctionofthedataprovidedinthisprojectisthatunliketypicalcallcenterdatathataveragesdataovertimeintervals,thesedataareonaper-callbasis,thusenablingdeeperanalysisaswellasamorenaturaltietomarketing-orhumanresource-relatedanalyses.Furtheruseofthistypeofdatatoexplorethelinksbetweencallcenteroperationalproblemsandhumanresourceandcustomerrelatedissuesisapromisingdirectionforfutureresearch. Largecallcentershavemotivatedtheanalysisofheavytrafficlimitsasusefulapproximationsofqueueingmodels(see,forexample,HalfinandWhitt1981,t,Mandelbaum,andReiman2002,Jenningsetal.1996,Whitt2004a,b).Motivatedbyrecentempiricalstudiesdemonstratingthatservicetimesandabandonmenttimesarenotnecessarilyexponentiallydistributed(Mandelbaum,Sakov,andZeltyn2000;Brownetal.2005),modelswithgeneralservicetimesandgeneralabandonmenttimeshavebeenanalyzedandapproximationsfortheirperformancedeveloped Aksin,Armony,andMehrotra:TheModernCallCenterProductionandOperationsManagement16
(6),pp.665–688,©2007ProductionandOperationsManagementSociety 673 (Whitt2004b,2005,2006c;Reed2005,ZeltynandMandelbaum2005,Jelenkovic,Mandelbaum,andMomcilovic2004,MandelbaumandMomcilovic2007,GamarnikandMomcilovic2007,KaspiandRamanan2007).MandelbaumandZeltyn(2004)explorealinearrelationshipbetweentheprobabilitytoabandonandthewaitingtimeinqueueinanErlang-Amodel.Althoughsuchlinearityshouldnotexistinthepresenceofgeneralimpatiencedistributions,empiricalevidencebyBrownetal.(2005)suggestsasimilarlinearrelationship.MandelbaumandZeltyn(2004)analyzetheproblemboththeoreticallyandempiricallyanddemonstratethat,overrealisticparametervalues,generalimpatiencedistributionsresultinperformancethatresemblestheErlang-Amodel.Thisisanimportantresult,supportingtherobustnessoftheErlang-Amodel,eveninsettingswithnon-exponentialimpatiencetimes.Similarly,asreviewedinmoredetailinSection2.4,Steckley,Henderson,andMehrotra(2005),HarrisonandZeevi(2005)andTorzhkovandArmony(2007)analyzecallcenterperformanceunderrandomarrivals. Blockedorabandonedcallsmayrediallater,whichisafeatureignoredinmostmodels.ThistypeofretrialbehavioranditsinfluenceonperformanceismodeledbyMandelbaumetal.(1999)andAguiretal.(2004).Approximations,inparticularafluidapproximation,performverywellforsuchsystems.Theuseoffluidapproximationsinthepresenceoftime-varyingparametersisalsosupportedbyRidley,Fu,andMassey(2003)andJimenezandKoole(2004).Theneedtomanagemulti-skillcallcentershasledtoperformanceevaluationmodelsforsystemswithflexibleservers(ChevalierandTabordon2003;Shumsky2004;StolletzandHelber2004;Whitt2006a;Franx,Koole,andPot2006). Webelievethatperformanceevaluationwillcontinuetoprovideresearchopportunities,particularlyinlightofthedevelopmentsdescribedinSections3and4below.
3.DemandModulation Manycallcentersfacehighlyunpredictabledemandthatisalsotime-varying.Thetime-varyingelementisrelativelyeasytohandlebyadjustingstaffinglevels.PapersbyJenningsetal.(1996),Massey(2002),Ridley,Fu,andMassey(2003),Feldmanetal.(2005),andGreen,Kolesar,andWhitt(2007)areexamplesofpapersthatconsiderthestaffingproblemundertimevaryingdemand.Butwhencallvolumeisunpredictable,limitedflexibilityinadjustingstaffinglevelsmayleadtosituationsofover-orunder-staffing,atleasttemporarily.Thissectiondealswithmeansofmodulatingdemandasawayofensuringloadbalancingandhigherlevelofpredictability.Demandmodula- tionisalsousedtoreduceoperatingcostsbyencouragingcallerstoobtainservicethroughotherchannels,suchasthe,thataremorescalableorlessexpensive. Thesimplestformofdemandmodulationthatmaybeusedincallcentersiscalladmissions.Themostprimitiveformofcalladmissionisabusysignalthatcustomersencountereverytimealllinesarebusy.Givencostsofinfrastructure,suchbusysignalsareveryrareinmediumtolargecallcentersandnonburstycallvolume.Amoresophisticatedformofcalladmissioncanbedonebyselectivelyadmittingcallsordingtotheirrelativeimportancetoanization(Ormeci,2004).Thispracticeisalsoveryunusualincallcenters.Bassamboo,Harrison,andZeevi(2006)demonstratethatundersomecircumstancesitisbeneficialnottoadmitlessprofitablecustomerssoastoreducethechancesoflosingmoreprofitableoneslateron. Regardlessofwhetheracallcenterregulatesitscallsthroughanadmissioncontrolmechanism,onefactthatcallcentermanagersmustfaceisthatcallersareinherentlyimpatient.Ifacustomercallisnotansweredwithinacertaintime,thecustomerwillhangup(abandon)andsubsequentlymayeitherretrylaterornot.Generally,callcentermanagersstrivetominimizethenumberofabandonments,becauseofthepremisethatabandonmentsareassociatedwithanegativewaitingexperienceandmightleadtolossofgoodwillandeventochurn.However,abandonmentsalsohaveaponentassociatedwiththem,becausetheyprovideanaturalmechanismforloadbalancing.Towit,whenthesystemisheavilyloadedimpatientcustomerstendtoabandon,alleviatingtheworkloadandhenceshorteningthewaitingtimesofthemorepatientcallers. Becauseoftheimportanceofabandonmentindeterminingstaffinglevels,therehasbeenastreamofliteraturethatfocusesonunderstandingcustomerabandonment(HassinandHaviv1995;MandelbaumandShimkin2000;Zohar,Mandelbaum,andShimkin2002;ShimkinandMandelbaum2004)anditsimpactonsystemperformance(t,Mandelbaum,andReiman2002;MandelbaumandZeltyn2004;ZeltynandMandelbaum2005;Armony,Plambeck,andSeshadri2007;MandelbaumandZeltyn2006;BaronandMilner2006;MandelbaumandZeltyn2007b). Acknowledgingthatoverloadedsituationsandabandonmentswillalwaysexist,researchershaveproposedthatnotifyingcallersoftheiranticipateddelayassoonastheycallwouldcauseimpatientcustomerstoleaverightaway(balk),whereasthemorepatientcustomersarelikelytowaituntiltheircallisanswered.Whitt(1999a)hasdemonstratedthattheoverallaveragewaitingtimeofallcustomersisreducedifdelayannouncementisurate.GuoandZipkin 674 Aksin,Armony,andMehrotra:TheModernCallCenterProductionandOperationsManagement16
(6),pp.665–688,©2007ProductionandOperationsManagementSociety (2006,2007a)haveidentifiedcasesinwhichinformationimprovesperformance,buthavealsodemonstratedthatsuchinformationcanactuallyhurttheserviceprovidersorthecustomersunderexponentialservicetimeandmoregeneralphase-typedistributions.GuoandZipkin(2007b)notedthattheeffectofinformationontotalthroughputdependsontheshapeofthedistributiondescribingthecustomers’sens

标签: #含金量 #cpu #英语 #培训班 #公路 #前景 #java #会议