Automatic and optimal shift scheduling using Reinforcement Learning (Part 5)
Using Sequential Decision Analytics to find ongoing optimal decisions
Retail Industry
Scheduling
Powell Unified Framework
Reinforcement Learning
Python
Author
Kobus Esterhuysen
Published
November 4, 2023
Modified
November 4, 2023
0 INTRODUCTION
In the previous part we found that the agent exploited the situation by having thCumShifts and thSickProb near their max values so that it can use the complete pool. So, instead of using thresholds below which resources are included as candidates, we always had all (available) resources in the candidates pool. The agent earned rewards or incurred penalties to discourage allocation of less ideal resources. The had the parameters:
\(\theta^{CumShifts}\)
\(\theta^{SickProb}\)
Each of these operated as weighting factors to weigh the associated component of the cumulative reward \(C^{cum}\):
In this part we add one more weighting parameters so that we now have:
\(\theta^{CumShifts}\)
\(\theta^{SickProb}\)
\(\theta^{CumMerits}\)
The cumulative reward is now calculated: \(C^{cum} = \theta^{CumShifts}.C^{cum}_{CumShifts} + \theta^{SickProb}.C^{cum}_{SickProb} + \theta^{CumMerits}.C^{cum}_{CumMerits}\)
In addition, we add a fourth (categorical) parameter called \(\theta^{Select}\) which can have one of two values:
‘random’
‘ranked_CumMerits’
Once the list of available candidates are identified this list is either randomized (‘random’), or it is ranked by the resources’ cumulative merits (‘ranked_CumMerits’).
To keep the visualizations manageable, we will still only have the resources:
Courtesy Clerk (7 resources)
Stocker (3 resources)
In a later part we will also add:
Cleaner (2 resources)
Curbsider (4 resources)
Daily demands are provided by a stochastic demand simulator based on past needs and trends. We also add a merit simulator.
The overall structure of this project and report follows the traditional CRISP-DM format. However, instead of the CRISP-DM’S “4 Modeling” section, we inserted the “6 step modeling process” of Dr. Warren Powell in section 4 of this document. Dr Powell’s universal framework shows great promise for unifying the formalisms of at least a dozen different fields. Using his framework enables easier access to thinking patterns in these other fields that might be beneficial and informative to the sequential decision problem at hand. Traditionally, this kind of problem would be approached from the reinforcement learning perspective. However, using Dr. Powell’s wider and more comprehensive perspective almost certainly provides additional value.
In order to make a strong mapping between the code in this notebook and the mathematics in the Powell Universal Framework (PUF), we follow the following convention for naming Python identifier names:
How to read/say
var name & flavor first
at t/n
for entity OR of/with attribute
\(\hat{R}^{fail}_{t+1,a}\) has code Rhat__fail_tt1_a which is read: “Rhatfail at t+1 of/with (attribute) a”
Superscripts
variable names have a double underscore to indicate a superscript
\(X^{\pi}\): has code X__pi, is read X pi
when there is a ‘natural’ distinction between the variable symbol and the superscript (e.g. a change in case), the double underscore is sometimes omitted: Xpi instead of X__pi, or MSpend_t instead of M__Spend_t
Subscripts
variable names have a single underscore to indicate a subscript
\(S_t\): has code S_t, is read ‘S at t’
\(M^{Spend}_t\) has code M__Spend_t which is read: “MSpend at t”
\(\hat{R}^{fail}_{t+1,a}\) has code Rhat__fail_tt1_a which is read: “Rhatfail at t+1 of/with (attribute) a” [RLSO-p436]
Arguments
collection variable names may have argument information added
\(X^{\pi}(S_t)\): has code X__piIS_tI, is read ‘X pi in S at t’
the surrounding I’s are used to imitate the parentheses around the argument
Next time/iteration
variable names that indicate one step in the future are quite common
\(R_{t+1}\): has code R_tt1, is read ‘R at t+1’
\(R^{n+1}\): has code R__nt1, is read ‘R at n+1’
Rewards
State-independent terminal reward and cumulative reward
\(F\): has code F for terminal reward
\(\sum_{n}F\): has code cumF for cumulative reward
State-dependent terminal reward and cumulative reward
\(C\): has code C for terminal reward
\(\sum_{t}C\): has code cumC for cumulative reward
Vectors where components use different names
\(S_t(R_t, p_t)\): has code S_t.R_t and S_t.p_t, is read ‘S at t in R at t, and, S at t in p at t’
the code implementation is by means of a named tuple
self.State = namedtuple('State', SVarNames) for the ‘class’ of the vector
self.S_t for the ‘instance’ of the vector
Vectors where components reuse names
\(x_t(x_{t,GB}, x_{t,BL})\): has code x_t.x_t_GB and x_t.x_t_BL, is read ‘x at t in x at t for GB, and, x at t in x at t for BL’
the code implementation is by means of a named tuple
self.Decision = namedtuple('Decision', xVarNames) for the ‘class’ of the vector
self.x_t for the ‘instance’ of the vector
Use of mixed-case variable names
to reduce confusion, sometimes the use of mixed-case variable names are preferred (even though it is not a best practice in the Python community), reserving the use of underscores and double underscores for math-related variables
1 BUSINESS UNDERSTANDING
The HR manager has to schedule resources for each day. This schedule will be automated and optimized by the AI agent.
The number of resources of each type for each schedule slots for each day will be provided by the simulator. Only two resource types will be handled:
Courtesy
Stocker
The HR manager typically runs the AI Shift Scheduler 2 weeks into the future to produce a tentative schedule to publish for the team.
As demands for shift slot allocations come in, they are handled in the following way:
the candidates for the resource type must have their:
\(R^{Avail}_t\) be True
the specific resources are then marked for allocation considering the number of resources needed for the type
the state of the resources are then updated including the number of accumulated shifts
at the end of the shift all resources are made available again
The overall objective will be to maximize the cumulative reward.
2 DATA UNDERSTANDING
Based on recent market research, the demand may be modeled by a Poisson distribution for each resource type: \[
\begin{aligned}
\mu^{ResourceType} &= \mathrm{SIM\_MU\_D[RESOURCE\_TYPE]}
\end{aligned}
\]
So we have: \[
D^{ResourceType}_{t+1} \sim Pois(\mu^{ResourceType})
\]
For each decision window, a MeritProb and DemeritProb unique to each resource is used to simulate whether that resource earned a merit (+1) and/or demerit (-1). The two values are summed to get a net merit. These merit values are also accumulated.
The decision window is 1 day and these simulations are for the daily demands for Shift1.
DeprecationWarning: Please use `shift` from the `scipy.ndimage` namespace, the `scipy.ndimage.interpolation` namespace is deprecated.
from scipy.ndimage.interpolation import shift
mer_sim = MeritSimulator(seed=1234)MeritData = []for i inrange(100): mer =list(mer_sim.simulate().values()) MeritData.append(mer)labels = [f'{an}_merit'for an in aNAMES]df = pd.DataFrame.from_records(data=MeritData, columns=labels); df[:10]
We will use the data provided by the simulator directly. There is no need to perform additional data preparation.
4 MODELING
4.1 Narrative
Please review the narrative in section 1. The next figure is a representation of the solution to the problem:
AIShiftScheduler1
4.2 Core Elements
This section attempts to answer three important questions:
What metrics are we going to track?
What decisions do we intend to make?
What are the sources of uncertainty?
For this problem, the only metric we are interested in is the cumulative reward after each horizon. The only source of uncertainty is the levels of demand and the merits/demerits for each of the resource types.
4.3 Mathematical Model | SUS Design
A Python class is used to implement the model for the SUS (System Under Steer):
class Model():
def __init__(self, S_0_info):
...
...
4.3.1 State variables
The state variables represent what we need to know.
\(x_{tab}\) is a boolean vector that indicates whether a specific resource is to be allocated to a demand
Decisions are made with a policy (TBD below):
\(X^{\pi}(S_t)\)
4.3.3 Exogenous information variables
The exogenous information variables represent what we did not know (when we made a decision). These are the variables that we cannot control directly. The information in these variables become available after we make the decision \(x_t\).
When we assume that the demand in each time period is revealed, without any model to predict the demand based on past demands, we have, using approach 1:
The transition function describe how the state variables evolve over time. We have the equations:
\[
R^{Avail}_{t+1} =
\begin{cases}
1 & \text{if resource with attribute $a$ has not been allocated} \\
0 & \text{if resource with attribute $a$ has been allocated }
\end{cases}
\]
\[
R^{CumShifts}_{t+1} =
\begin{cases}
R^{CumShifts}_{t} + 1 & \text{if resource was allocated} \\
R^{CumShifts}_{t} & \text{if resource was not allocated }
\end{cases}
\]
4.3.6 Implementation of the System Under Steer (SUS) Model
class Model():def__init__(self, W_fn=None, S__M_fn=None, C_fn=None):self.S_t = {'R_t': pd.DataFrame({'ResourceId': RESOURCE_IDS,'Type': TYPES,'RAvail_t': AVAILABILITIES_DOW['0_Sunday'],'RCumShifts_t': [0]*len(TYPES), ##cumulative allocs (for T)'RCumMerits_t': [0]*len(TYPES), }),'D_t': pd.DataFrame({'Type': RESOURCE_TYPES,'DShift_t': [1]*len(RESOURCE_TYPES), }),'M_t': pd.DataFrame({'ResourceId': RESOURCE_IDS,'Type': TYPES,'nMerits_t': [0]*len(TYPES), }), }self.x_t = {'xAlloc_t': pd.DataFrame({'Comb': abNAMES, ##Combination'Allocd_t': [False]*len(abNAMES), ##Allocated }), }self.Ccum =0.0##cumulative rewardself.Ccum_CumShifts =0.0self.Ccum_SickProb =0.0self.Ccum_CumMerits =0.0self.Ucum_Total =0##cumulative unallocated/unmet demands##cumulative unallocated/unmet demandsself.Ucum = {rt: 0for rt in RESOURCE_TYPES}## def reset(self):# self.Ccum = 0.0# self.Ucum = 0## exogenous information, dependent on a random process,## the directly observed demandsdef W_fn(self, t):return {'demands': DEM.simulate(),'merits': MER.simulate(), }def update_Ccum(self, dt, S_t, x_t, theta): m1 = x_t['xAlloc_t'].merge(S_t['R_t'][['RCumShifts_t', 'RAvail_t', 'RCumMerits_t']], left_index=True, right_index=True) m2 = m1.merge(SICK_PROBS[[dt.month_name()]], left_index=True, right_index=True) m2.rename(columns={dt.month_name(): 'SickProb'}, inplace=True) m2['Capacity'] = CAPACITIES.loc[:, ['Capacity']] m2['Ccum_CumShifts'] =-(m2['RCumShifts_t'] - m2['Capacity'])## m2['Ccum_SickProb'] = -100*m2['SickProb']## m2['Ccum_SickProb'] = -100*(m2['SickProb'] - m2['SickProb'].mean()) ##100 makes values more comparable with other component of Ccum m2['Ccum_SickProb'] =-1*(m2['SickProb'] - m2['SickProb'].mean()) ##100 makes values more comparable with other component of Ccum## m2['Ccum_Avail'] = m2['RAvail_t'].replace({True: 0, False: -10}) m2['Ccum_CumMerits'] = m2['RCumMerits_t'] summables = m2.loc[ m2['Allocd_t'] ==True,## ['Ccum_CumShifts', 'Ccum_SickProb', 'Ccum_Avail'] ['Ccum_CumShifts', 'Ccum_SickProb', 'Ccum_CumMerits'], ]## Ccum_CumShifts, Ccum_SickProb, Ccum_Avail = summables.sum(axis=0) Ccum_CumShifts, Ccum_SickProb, Ccum_CumMerits = summables.sum(axis=0) Ccum =\ theta.thCumShifts*Ccum_CumShifts +\ theta.thSickProb*Ccum_SickProb +\ theta.thCumMerits*Ccum_CumMerits## theta.thAvail*Ccum_Availself.Ccum_CumShifts += Ccum_CumShiftsself.Ccum_SickProb += Ccum_SickProbself.Ccum_CumMerits += Ccum_CumMeritsself.Ccum += Ccumdef performAllocDecision(self, S_t, x_t, theta):## find list of ResourceIds for allocs from x_t resourceIds = x_t['xAlloc_t'].loc[ x_t['xAlloc_t']['Allocd_t']==True, ['Comb'] ]['Comb'].str.split('_').str[:1].tolist();##print(f'{resourceIds=}') resourceIds_flat = [e[0] for e in resourceIds];##print(f'{resourceIds_flat=}')## update state of allocs S_t['R_t'].loc[ S_t['R_t']['ResourceId'].isin(resourceIds_flat), ['RAvail_t'] ] =False S_t['R_t'].loc[ S_t['R_t']['ResourceId'].isin(resourceIds_flat), ['RCumShifts_t'] ] +=1## update Ccum with allocations## self.Ccum += len(resourceIds_flat) #number of allocationsdef S__M_fn(self, t, dt, S_t, x_t, W_tt1, theta):## perform decision taken this morningself.performAllocDecision(S_t, x_t, theta)## D_t #direct approachfor rt in RESOURCE_TYPES: sh_demands = W_tt1['demands'][rt] S_t['D_t'].loc[S_t['D_t']['Type']==rt, 'DShift_t'] = sh_demandsfor an in aNAMES: resId = an.split('_')[0] merits = W_tt1['merits'][an] S_t['M_t'].loc[S_t['M_t']['ResourceId'] == resId, 'nMerits_t'] = merits## Update cumulative merits of all resources S_t['R_t']['RCumMerits_t'] += S_t['M_t']['nMerits_t']## Update availabilities of all resources dow = (t +1)%7 mask = AVAILABILITIES_DOW.columns.str.contains(str(dow) +'_') dow_col = AVAILABILITIES_DOW.loc[:, mask] S_t['R_t']['RAvail_t'] = AVAILABILITIES_DOW[dow_col.columns[0]] record_t = [t, dt] +\list(S_t['R_t']['RAvail_t']) +\list(S_t['R_t']['RCumShifts_t']) +\list(S_t['R_t']['RCumMerits_t']) +\list(S_t['D_t']['DShift_t']) +\ [self.Ucum[rt] for rt in RESOURCE_TYPES] +\ [self.Ucum_Total] +\ [self.Ccum_CumShifts] +\ [self.Ccum_SickProb] +\ [self.Ccum_CumMerits] +\ [self.Ccum] +\list(x_t['xAlloc_t']['Allocd_t'])return record_tdef C_fn(self, S_t, x_t, W_tt1, theta):returndef step(self, t, dt, theta):## IND = '\t\t'## print(f"{IND}..... M. step() .....\n{t=}\n{theta=}") W_tt1 =self.W_fn(t);##print(f'{W_tt1=}')## update state & reward record_t =self.S__M_fn(t, dt, self.S_t, self.x_t, W_tt1, theta)return record_t
4.4 Uncertainty Model
We will simulate the shift demand vector \(D^{Shift}_{t+1}\), and number of merits vector \(n^{Merits}_{t+1}\) as described in section 2.
4.5 Policy Design
There are two main meta-classes of policy design. Each of these has two subclasses: - Policy Search - Policy Function Approximations (PFAs) - Cost Function Approximations (CFAs) - Lookahead - Value Function Approximations (VFAs) - Direct Lookaheads (DLAs)
In this project we will only use one approach: - A simple allocate parameterized policy (from the PFA class), called X__Alloc.
4.5.1 Implementation of Policy Design
import randomclass Policy():def__init__(self, model):self.model = modelself.Policy = namedtuple('Policy', piNAMES) # 'class'self.Theta = namedtuple('Theta', thNAMES) # 'class'def build_policy(self, info):returnself.Policy(*[info[pin] for pin in piNAMES])def build_theta(self, info):returnself.Theta(*[info[thn] for thn in thNAMES])def X__Alloc(self, t, dt, S_t, x_t, theta):## print(f"\n..... Policy.X__Alloc() .....\n{t=}") demandsToService = []for rt in RESOURCE_TYPES: number = S_t['D_t'].loc[ S_t['D_t']['Type']==rt, ['DShift_t'] ].squeeze() demandsToService.append((rt, number))## print(f"demandsToService:\n{demandsToService}")for demand in demandsToService: resourceType, number = demand;##print(f'{resourceType=}, {number=}') candidates = S_t['R_t'].loc[ (S_t['R_t']['Type'] == resourceType) &\ (S_t['R_t']['RAvail_t'] ==True), ['ResourceId', 'Type', 'RAvail_t', 'RCumShifts_t', 'RCumMerits_t'] ];##print(f'candidates BEFORE arrangement for selection=\n{candidates}')if theta.thSelect =='random': candidates = candidates.sample(frac =1) #shuffle rows; now opt/non have different x'selif theta.thSelect =='ranked_CumMerits': candidates = candidates.sort_values(by='RCumMerits_t', ascending=False)else:print(f'ERROR: Invalid value for theta.thSelect: {theta.thSelect}')break## print(f'candidates AFTER arrangement for selection=\n{candidates}')iflen(candidates) >0and number >0:iflen(candidates) >= number: allocs = candidates.iloc[0:number, :];##print(f'allocs=\n{allocs}') #pick first 'number' availseliflen(candidates) < number:##print('%%% NOT ENOUGH candidates') allocs = copy(candidates);##print(f'allocs=\n{allocs}') #pick all candidates unallocated_demands = number -len(candidates) ##unmet demands## self.model.Ccum -= unallocated_demandsself.model.Ucum_Total += unallocated_demandsself.model.Ucum[resourceType] += unallocated_demandsiflen(allocs) ==0: x_t['xAlloc_t'].loc[ x_t['xAlloc_t']['Comb'].str.contains(resourceType), ['Allocd_t'] ] =Falseelse: x_t['xAlloc_t'].loc[ ##remove all allocs before adding new allocs x_t['xAlloc_t']['Comb'].str.contains(resourceType), ['Allocd_t'] ] =False x_t['xAlloc_t'].loc[ x_t['xAlloc_t']['Comb'].apply(lambda x: x.split("_")[0]).isin(allocs['ResourceId']), ['Allocd_t'] ] =Trueself.model.update_Ccum(dt, S_t, x_t, theta)else:## print(f'%%% Shift has no resource for demand {demand}, or demand is 0.') x_t['xAlloc_t'].loc[ x_t['xAlloc_t']['Comb'].str.contains(resourceType), ['Allocd_t'] ] =False## print(f"x_t['xAlloc_t']:\n{x_t['xAlloc_t']}")def run_grid_sample_paths(self, theta, piName, record): CcumIomega__lI = []for l inrange(1, L +1): ## for each sample-path M = Model()## P = Policy(M) #NO!, overwrite existing global Pself.model = M record_l = [piName, theta, l] dt = pd.to_datetime(START_DATE_TIME) dt_delta = pd.Timedelta(DATE_TIME_DELTA)for t inrange(T): ## for each transition/step## print(f'\t%%% {t=}')## >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>getattr(self, piName)(t, dt, self.model.S_t, self.model.x_t, theta) #lookup (new) today's decision## sit in post-decision state until end of cycle (evening)## S_t, Ccum, x_t = self.model.step(t, x_t, theta) record_t =self.model.step(t, dt, theta) #change from today to tomorrow## >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> record.append(record_l + record_t) dt = dt + dt_delta CcumIomega__lI.append(self.model.Ccum) #just above (SDAM-eq2.9)return CcumIomega__lIdef perform_grid_search_sample_paths(self, piName, thetas): Cbarcum = defaultdict(float) Ctilcum = defaultdict(float) expCbarcum = defaultdict(float) expCtilcum = defaultdict(float) numThetas =len(thetas) record = []print(f'{numThetas=:,}') nth =1 i =0;print(f'... printing every {nth}th theta (if considered) ...')for theta in thetas:ifTrue: ##in case relationships between thetas can be exploited## a dict cannot be used as a key, so we define theta_key, e.g.## theta_key = ((168.0, 72.0), (200.0, 90.0)):## theta_key = tuple(tuple(itm.values()) for itm in theta) theta_key = theta ##if theta is not a dictif i%nth ==0: print(f'{i:,}/{numThetas:,}, {theta}')## >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CcumIomega__lI =self.run_grid_sample_paths(theta, piName, record)## >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Cbarcum_tmp = np.array(CcumIomega__lI).mean() #(SDAM-eq2.9) Ctilcum_tmp = np.sum(np.square(np.array(CcumIomega__lI) - Cbarcum_tmp))/(L -1) Cbarcum[theta_key] = Cbarcum_tmp Ctilcum[theta_key] = np.sqrt(Ctilcum_tmp/L) expCbarcum_tmp = pd.Series(CcumIomega__lI).expanding().mean() expCbarcum[theta_key] = expCbarcum_tmp expCtilcum_tmp = pd.Series(CcumIomega__lI).expanding().std() expCtilcum[theta_key] = expCtilcum_tmp i +=1##endif best_theta =max(Cbarcum, key=Cbarcum.get) worst_theta =min(Cbarcum, key=Cbarcum.get) best_Cbarcum = Cbarcum[best_theta] best_Ctilcum = Ctilcum[best_theta] worst_Cbarcum = Cbarcum[worst_theta] worst_Ctilcum = Ctilcum[worst_theta] thetaStar_expCbarcum = expCbarcum[best_theta] thetaStar_expCtilcum = expCtilcum[best_theta] thetaStar_expCtilcum[0] =0##set NaN to 0## best_theta_w_names = tuple((# ({# a1NAMES[0]: subvec[0],# a1NAMES[1]: subvec[1]# })) for subvec in best_theta)## best_theta_n = self.build_theta({'thAdj': best_theta_w_names[0]})## best_theta_n = self.build_theta({'thAdj1': best_theta_w_names[0], 'thAdj3': best_theta_w_names[1]})## print(f'best_theta_n:\n{best_theta_n}\n{best_Cbarcum=:.2f}\n{best_Ctilcum=:.2f}')## worst_theta_w_names = tuple((# ({# a1NAMES[0]: subvec[0],# a1NAMES[1]: subvec[1]})) for subvec in worst_theta)## worst_theta_n = self.build_theta({'thAdj': worst_theta_w_names[0]})## worst_theta_n = self.build_theta({'thAdj1': worst_theta_w_names[0], 'thAdj3': worst_theta_w_names[1]})## print(f'worst_theta_n:\n{worst_theta_n}\n{worst_Cbarcum=:.2f}\n{worst_Ctilcum=:.2f}')return\ thetaStar_expCbarcum, thetaStar_expCtilcum, \ Cbarcum, Ctilcum, \ best_theta, worst_theta, \ best_Cbarcum, worst_Cbarcum, \ best_Ctilcum, worst_Ctilcum, \ record## EXAMPLE:## thetasA: Buy## thetasA_name: 'thBuy'## names: ELA## 1_1: 1 theta sub-vectors, each having 1 theta## thetas = grid_search_thetas_1_2(thetasBuy 'thBuy', CAR_TYPES)def grid_search_thetas_1_1(self, thetasA, thetasA_name, names): thetas = [self.build_theta({thetasA_name: {names[0]: thA0}})for thA0 in thetasA[names[0]] ]return thetas## EXAMPLE:## thetasA: Buy## thetasA_name: 'thBuy'## names: ELA, SON## 1_2: 1 theta sub-vectors, each having 2 thetas## thetas = grid_search_thetas_1_2(thetasBuy 'thBuy', CAR_TYPES)def grid_search_thetas_1_2(self, thetasA, thetasA_name, names): thetas = [self.build_theta({thetasA_name: {names[0]: thA0, names[1]: thA1}})for thA0 in thetasA[names[0]]for thA1 in thetasA[names[1]] ]return thetas## EXAMPLE:## thetasA: Adj## thetasA_name: 'thAdj'## names: ELA, SON## 1_4: 1 theta sub-vectors, each having 4 thetas## thetas = grid_search_thetas_1_4(thetasBuy 'thAdj', bNAMES)def grid_search_thetas_1_4(self, thetasA, thetasA_name, names): thetas = [self.build_theta({thetasA_name: {names[0]: thA0, names[1]: thA1, names[2]: thA2, names[3]: thA3}})for thA0 in thetasA[names[0]]for thA1 in thetasA[names[1]]for thA2 in thetasA[names[2]]for thA3 in thetasA[names[3]] ]return thetas## EXAMPLE:## thetasA: Buy## thetasB: Max## thetasA_name: 'thBuy'## thetasB_name: 'thMax'## names: ELA## 2_1: 2 theta sub-vectors, each having 1 theta## thetas = grid_search_thetas_2_1(thetasBuy, thetasMax, 'thBuy', 'thMax', CAR_TYPES)def grid_search_thetas_2_1(self, thetasA, thetasB, thetasA_name, thetasB_name, names): thetas = [self.build_theta({thetasA_name: {names[0]: thA0}, thetasB_name: {names[0]: thB0}})for thA0 in thetasA[names[0]]for thB0 in thetasB[names[0]] ]return thetas## EXAMPLE:## thetasA: Buy## thetasB: Max## thetasA_name: 'thBuy'## thetasB_name: 'thMax'## names: ELA, SON## 2_2: 2 theta sub-vectors, each having 2 thetas## thetas = grid_search_thetas_4(thetasBuy, thetasMax, 'thBuy', 'thMax', CAR_TYPES)def grid_search_thetas_2_2(self, thetasA, thetasB, thetasA_name, thetasB_name, names): thetas = [self.build_theta({thetasA_name: {names[0]: thA0, names[1]: thA1}, thetasB_name: {names[0]: thB0, names[1]: thB1}})for thA0 in thetasA[names[0]]for thA1 in thetasA[names[1]]for thB0 in thetasB[names[0]]for thB1 in thetasB[names[1]] ]return thetas############################################################################### PLOTTING############################################################################def round_theta(self, complex_theta): thetas_rounded = []for theta in complex_theta: evalues_rounded = []for _, evalue in theta.items(): evalues_rounded.append(float(f"{evalue:f}")) thetas_rounded.append(tuple(evalues_rounded))returnstr(tuple(thetas_rounded))def plot_Fhat_map_2(self, FhatI_theta_I, thetasX, thetasY, labelX, labelY, title): Fhat_values = [ FhatI_theta_I[ (thetaX,thetaY)## ((thetaX,),(thetaY,)) ]for thetaY in thetasY for thetaX in thetasX ] Fhats = np.array(Fhat_values) increment_count =len(thetasX) Fhats = np.reshape(Fhats, (-1, increment_count))#. fig, ax = plt.subplots() im = ax.imshow(Fhats, cmap='hot', origin='lower', aspect='auto')## create colorbar cbar = ax.figure.colorbar(im, ax=ax)## cbar.ax.set_ylabel(cbarlabel, rotation=-90, va="bottom") ax.set_xticks(np.arange(0, len(thetasX), 5))#.## ax.set_xticks(np.arange(len(thetasX))) ax.set_yticks(np.arange(0, len(thetasY), 5))#.## ax.set_yticks(np.arange(len(thetasY)))## NOTE: round tick labels, else very messy## function round() does not work, have to do this way thetasX_form = [f'{th:.0f}'for th in thetasX] thetasY_form = [f'{th:.0f}'for th in thetasY] ax.set_xticklabels(thetasX[::5])## ax.set_xticklabels(thetasX); ax.set_xticklabels(thetasX_form) ax.set_yticklabels(thetasY[::5])## ax.set_yticklabels(thetasY); ax.set_yticklabels(thetasY_form)## rotate the tick labels and set their alignment.## plt.setp(ax.get_xticklabels(), rotation=45, ha="right",rotation_mode="anchor") ax.set_title(title) ax.set_xlabel(labelX) ax.set_ylabel(labelY)## fig.tight_layout() plt.show()returnTruedef plot_Fhat_map_3(self, FhatI_theta_I, thetasX, thetasY, labelX, labelY, title, thetaFixed1):## Fhat_values = [FhatI_theta_I[(thetaX,thetaY)] for thetaY in thetasY for thetaX in thetasX] Fhat_values = [ FhatI_theta_I[(thetaX,thetaY, thetaFixed1)]for thetaY in thetasYfor thetaX in thetasX] Fhats = np.array(Fhat_values) increment_count =len(thetasX) Fhats = np.reshape(Fhats, (-1, increment_count))#. fig, ax = plt.subplots() im = ax.imshow(Fhats, cmap='hot', origin='lower', aspect='auto')## create colorbar cbar = ax.figure.colorbar(im, ax=ax)## cbar.ax.set_ylabel(cbarlabel, rotation=-90, va="bottom") ax.set_xticks(np.arange(0, len(thetasX), 5))#.## ax.set_xticks(np.arange(len(thetasX))) ax.set_yticks(np.arange(0, len(thetasY), 5))#.## ax.set_yticks(np.arange(len(thetasY)))## NOTE: round tick labels, else very messy## function round() does not work, have to do this way## thetasX_form = [f'{th:.1f}' for th in thetasX]## thetasY_form = [f'{th:.1f}' for th in thetasY] ax.set_xticklabels(thetasX[::5])#.## ax.set_xticklabels(thetasX)## ax.set_xticklabels(thetasX_form) ax.set_yticklabels(thetasY[::5])#.## ax.set_yticklabels(thetasY)## ax.set_yticklabels(thetasY_form)## rotate the tick labels and set their alignment.## plt.setp(ax.get_xticklabels(), rotation=45, ha="right",rotation_mode="anchor") ax.set_title(title) ax.set_xlabel(labelX) ax.set_ylabel(labelY)## fig.tight_layout() plt.show()returnTrue## def plot_Fhat_map_4(self, ## sub-vectors# FhatI_theta_I,# thetasX, thetasY, labelX, labelY, title,# thetaFixed1, thetaFixed2):# ## Fhat_values = [FhatI_theta_I[(thetaX,thetaY)] for thetaY in thetasY for thetaX in thetasX]# Fhat_values = [# FhatI_theta_I[((thetaX,thetaY), (thetaFixed1,thetaFixed2))]# for thetaY in thetasY# for thetaX in thetasX]# Fhats = np.array(Fhat_values)# increment_count = len(thetasX)# Fhats = np.reshape(Fhats, (-1, increment_count))#.# fig, ax = plt.subplots()# im = ax.imshow(Fhats, cmap='hot', origin='lower', aspect='auto')# ## create colorbar# cbar = ax.figure.colorbar(im, ax=ax)# ## cbar.ax.set_ylabel(cbarlabel, rotation=-90, va="bottom")# ax.set_xticks(np.arange(0, len(thetasX), 5))#.# ## ax.set_xticks(np.arange(len(thetasX)))# ax.set_yticks(np.arange(0, len(thetasY), 5))#.# ## ax.set_yticks(np.arange(len(thetasY)))# ## NOTE: round tick labels, else very messy# ## function round() does not work, have to do this way# ## thetasX_form = [f'{th:.1f}' for th in thetasX]# ## thetasY_form = [f'{th:.1f}' for th in thetasY]# ax.set_xticklabels(thetasX[::5])#.# ## ax.set_xticklabels(thetasX)# ## ax.set_xticklabels(thetasX_form)# ax.set_yticklabels(thetasY[::5])#.# ## ax.set_yticklabels(thetasY)# ## ax.set_yticklabels(thetasY_form)# ## rotate the tick labels and set their alignment.# ## plt.setp(ax.get_xticklabels(), rotation=45, ha="right",rotation_mode="anchor")# ax.set_title(title)# ax.set_xlabel(labelX)# ax.set_ylabel(labelY)# ## fig.tight_layout()# plt.show()# return Truedef plot_Fhat_map_4(self, ## no sub-vectors FhatI_theta_I, thetasX, thetasY, labelX, labelY, title, thetaFixed1, thetaFixed2):## Fhat_values = [FhatI_theta_I[(thetaX,thetaY)] for thetaY in thetasY for thetaX in thetasX] Fhat_values = [ FhatI_theta_I[(thetaX,thetaY, thetaFixed1,thetaFixed2)]for thetaY in thetasYfor thetaX in thetasX] Fhats = np.array(Fhat_values) increment_count =len(thetasX) Fhats = np.reshape(Fhats, (-1, increment_count))#. fig, ax = plt.subplots() im = ax.imshow(Fhats, cmap='hot', origin='lower', aspect='auto')## create colorbar cbar = ax.figure.colorbar(im, ax=ax)## cbar.ax.set_ylabel(cbarlabel, rotation=-90, va="bottom") ax.set_xticks(np.arange(0, len(thetasX), 5))#.## ax.set_xticks(np.arange(len(thetasX))) ax.set_yticks(np.arange(0, len(thetasY), 5))#.## ax.set_yticks(np.arange(len(thetasY)))## NOTE: round tick labels, else very messy## function round() does not work, have to do this way## thetasX_form = [f'{th:.1f}' for th in thetasX]## thetasY_form = [f'{th:.1f}' for th in thetasY] ax.set_xticklabels(thetasX[::5])#.## ax.set_xticklabels(thetasX)## ax.set_xticklabels(thetasX_form) ax.set_yticklabels(thetasY[::5])#.## ax.set_yticklabels(thetasY)## ax.set_yticklabels(thetasY_form)## rotate the tick labels and set their alignment.## plt.setp(ax.get_xticklabels(), rotation=45, ha="right",rotation_mode="anchor") ax.set_title(title) ax.set_xlabel(labelX) ax.set_ylabel(labelY)## fig.tight_layout() plt.show()returnTrue## color_style examples: 'r-', 'b:', 'g--'def plot_Fhat_chart(self, FhatI_theta_I, thetasX, labelX, labelY, title, color_style, thetaStar): mpl.rcParams['lines.linewidth'] =1.2 xylabelsize =16## plt.figure(figsize=(13, 9)) plt.figure(figsize=(13, 4)) plt.title(title, fontsize=20) Fhats = FhatI_theta_I.values() plt.plot(thetasX, Fhats, color_style) plt.axvline(x=thetaStar, color='k', linestyle=':') plt.xlabel(labelX, rotation=0, labelpad=10, ha='right', va='center', fontweight='bold', size=xylabelsize) plt.ylabel(labelY, rotation=0, labelpad=1, ha='right', va='center', fontweight='normal', size=xylabelsize) plt.show()## expanding Fhat chartdef plot_expFhat_chart(self, df, labelX, labelY, title, color_style): mpl.rcParams['lines.linewidth'] =1.2 xylabelsize =16 plt.figure(figsize=(13, 4)) plt.title(title, fontsize=20) plt.plot(df, color_style) plt.xlabel(labelX, rotation=0, labelpad=10, ha='right', va='center', fontweight='bold', size=xylabelsize) plt.ylabel(labelY, rotation=0, labelpad=1, ha='right', va='center', fontweight='normal', size=xylabelsize) plt.show()## expanding Fhat chartsdef plot_expFhat_charts(self, means, stdvs, labelX, labelY, suptitle, pars=defaultdict(str)): n_charts =2 xlabelsize =14 ylabelsize =14 mpl.rcParams['lines.linewidth'] =1.2 default_colors = plt.rcParams['axes.prop_cycle'].by_key()['color'] fig, axs = plt.subplots(n_charts, sharex=True) fig.set_figwidth(13); fig.set_figheight(9) fig.suptitle(suptitle, fontsize=18) xi =0 legendlabels = [] axs[xi].set_title(r"$exp\bar{C}^{cum}(\theta^*)$", loc='right', fontsize=16)for i,itm inenumerate(means.items()): axs[xi].spines['top'].set_visible(False); axs[xi].spines['right'].set_visible(True); axs[xi].spines['bottom'].set_visible(False) leg = axs[xi].plot(itm[1], color=pars['colors'][i]) legendlabels.append(itm[0]) axs[xi].set_ylabel(labelY, rotation=0, ha='right', va='center', fontweight='normal', size=ylabelsize) xi =1 axs[xi].set_title(r"$exp\tilde{C}^{cum}(\theta^*)$", loc='right', fontsize=16)for i,itm inenumerate(stdvs.items()): axs[xi].spines['top'].set_visible(False); axs[xi].spines['right'].set_visible(True); axs[xi].spines['bottom'].set_visible(False)## leg = axs[xi].plot(itm[1], default_colors[i], linestyle='--') leg = axs[xi].plot(itm[1], pars['colors'][i], linestyle='--') axs[xi].set_ylabel(labelY, rotation=0, ha='right', va='center', fontweight='normal', size=ylabelsize) fig.legend(## [leg], labels=legendlabels, title="Policies", loc='upper right', fancybox=True, shadow=True, ncol=1) plt.xlabel(labelX, rotation=0, labelpad=10, ha='right', va='center', fontweight='normal', size=xlabelsize) plt.show()def plot_records(self, df, df_non, pars=defaultdict(str)): n_a =len(aNAMES) n_b =len(bNAMES) n_ab =len(abNAMES) n_x =1 n_charts = n_ab + n_b + n_a + n_b +1+1+3 ylabelsize =14 mpl.rcParams['lines.linewidth'] =1.2 mycolors = ['g', 'b'] fig, axs = plt.subplots(n_charts, sharex=True)## fig.set_figwidth(13); fig.set_figheight(9) fig.set_figwidth(13); fig.set_figheight(20) fig.suptitle(pars['suptitle'], fontsize=14) WHERE ='post'#'pre' #'post' xi =0for i,ab inenumerate(abNAMES): axs[xi+i].set_ylim(auto=True); axs[xi+i].spines['top'].set_visible(False); axs[xi+i].spines['right'].set_visible(True); axs[xi+i].spines['bottom'].set_visible(False) axs[xi+i].step(df[f'Allocd_t_{ab}'], 'm-', where=WHERE)ifnot df_non isNone: axs[xi+i].step(df_non[f'Allocd_t_{ab}'], 'c-.', where=WHERE) axs[xi+i].axhline(y=0, color='k', linestyle=':') abl = ab.split("___") al = abl[0].split('_'); al = al[0]+'\_'+al[1]; bl = abl[1] y1ab ='$x^{Allocd}_{t,'+f'{",".join([al, bl])}'+'}$' axs[xi+i].set_ylabel(y1ab, rotation=0, ha='right', va='center', fontweight='bold', size=ylabelsize)for j inrange(df.shape[0]//T): axs[xi+i].axvline(x=(j+1)*T, color='grey', ls=':') xi = n_abfor i,b inenumerate(bNAMES): y1ab ='$D^{Shift}_{t,'+f'{b}'+'}$' axs[xi+i].set_ylim(auto=True); axs[xi+i].spines['top'].set_visible(False); axs[xi+i].spines['right'].set_visible(True); axs[xi+i].spines['bottom'].set_visible(False) axs[xi+i].step(df[f'DShift_t_{b}'], mycolors[xi%len(mycolors)], where=WHERE)ifnot df_non isNone: axs[xi+i].step(df_non[f'DShift_t_{b}'], 'c-.', where=WHERE) axs[xi+i].axhline(y=DEM.muD[b], color='k', linestyle=':') axs[xi+i].set_ylabel(y1ab, rotation=0, ha='right', va='center', fontweight='bold', size=ylabelsize)for j inrange(df.shape[0]//T): axs[xi+i].axvline(x=(j+1)*T, color='grey', ls=':') xi = n_ab + n_bfor i,a inenumerate(aNAMES): axs[xi+i].set_ylim(auto=True); axs[xi+i].spines['top'].set_visible(False); axs[xi+i].spines['right'].set_visible(True); axs[xi+i].spines['bottom'].set_visible(False) axs[xi+i].step(df[f'RAvail_t_{a}'], 'r:', where=WHERE) axs[xi+i].step(df[f'RCumShifts_t_{a}'], 'm-', where=WHERE) axs[xi+i].step(df[f'RCumMerits_t_{a}'], 'b:', where=WHERE)## if not df_non is None: axs[xi+i].step(df_non[f'RAvail_t_{a}'], 'c-.', where=WHERE)ifnot df_non isNone: axs[xi+i].step(df_non[f'RCumShifts_t_{a}'], 'c-.', where=WHERE) axs[xi+i].axhline(y=0, color='k', linestyle=':') cap = CAPACITIES.loc[CAPACITIES['ResourceId'] == a.split('_')[0], 'Capacity'].values[0] axs[xi+i].axhline(y=cap, color='g', linestyle=':') al = a.split('_'); al = al[0]+'\_'+al[1]; y1ab ='$R^{CumShifts}_{t,'+f'{al}'+'}$' axs[xi+i].set_ylabel(y1ab, rotation=0, ha='right', va='center', fontweight='bold', size=ylabelsize)for j inrange(df.shape[0]//T): axs[xi+i].axvline(x=(j+1)*T, color='grey', ls=':') xi = n_ab + n_b + n_a Ucum_cols = ['Ucum_'+bn for bn in bNAMES] ymax =max(list(df[Ucum_cols].max()))for i,b inenumerate(bNAMES): y1ab ='$U^{cum}_{'+f'{b}'+'}$' axs[xi+i].set_ylim(0, ymax); axs[xi+i].spines['top'].set_visible(False); axs[xi+i].spines['right'].set_visible(True); axs[xi+i].spines['bottom'].set_visible(False) axs[xi+i].step(df[f'Ucum_{b}'], 'r-', where=WHERE)ifnot df_non isNone: axs[xi+i].step(df_non[f'Ucum_{b}'], 'c-.', where=WHERE) axs[xi+i].axhline(y=ymax, color='r', linestyle=':') axs[xi+i].set_ylabel(y1ab, rotation=0, ha='right', va='center', fontweight='bold', size=ylabelsize)for j inrange(df.shape[0]//T): axs[xi+i].axvline(x=(j+1)*T, color='grey', ls=':') xi = n_ab + n_b + n_a + n_b axs[xi].set_ylim(auto=True); axs[xi].spines['top'].set_visible(False); axs[xi].spines['right'].set_visible(True); axs[xi].spines['bottom'].set_visible(False) axs[xi].step(df['Ucum_Total'], 'k-', where=WHERE)ifnot df_non isNone: axs[xi].step(df_non['Ucum_Total'], 'c-.', where=WHERE) axs[xi].axhline(y=0, color='k', linestyle=':') axs[xi].set_ylabel('$U^{cum}_{Total}$'+'\n'+''+'$\mathrm{[Allocs]}$', rotation=0, ha='right', va='center', fontweight='bold', size=ylabelsize);for j inrange(df.shape[0]//T): axs[xi].axvline(x=(j+1)*T, color='grey', ls=':') xi = n_ab + n_b + n_a + n_b +1 axs[xi].set_ylim(auto=True); axs[xi].spines['top'].set_visible(False); axs[xi].spines['right'].set_visible(True); axs[xi].spines['bottom'].set_visible(False) axs[xi].step(df['Ccum_CumShifts'], 'm-', where=WHERE)ifnot df_non isNone: axs[xi].step(df_non['Ccum_CumShifts'], 'c-.', where=WHERE) axs[xi].axhline(y=0, color='k', linestyle=':') axs[xi].set_ylabel('$C^{cum}_{CumShifts}$', rotation=0, ha='right', va='center', fontweight='bold', size=ylabelsize);for j inrange(df.shape[0]//T): axs[xi].axvline(x=(j+1)*T, color='grey', ls=':') xi = n_ab + n_b + n_a + n_b +2 axs[xi].set_ylim(auto=True); axs[xi].spines['top'].set_visible(False); axs[xi].spines['right'].set_visible(True); axs[xi].spines['bottom'].set_visible(False) axs[xi].step(df['Ccum_SickProb'], 'm-', where=WHERE)ifnot df_non isNone: axs[xi].step(df_non['Ccum_SickProb'], 'c-.', where=WHERE) axs[xi].axhline(y=0, color='k', linestyle=':') axs[xi].set_ylabel('$C^{cum}_{SickProb}$', rotation=0, ha='right', va='center', fontweight='bold', size=ylabelsize);for j inrange(df.shape[0]//T): axs[xi].axvline(x=(j+1)*T, color='grey', ls=':') xi = n_ab + n_b + n_a + n_b +3 axs[xi].set_ylim(auto=True); axs[xi].spines['top'].set_visible(False); axs[xi].spines['right'].set_visible(True); axs[xi].spines['bottom'].set_visible(False) axs[xi].step(df['Ccum_CumMerits'], 'm-', where=WHERE)ifnot df_non isNone: axs[xi].step(df_non['Ccum_CumMerits'], 'c-.', where=WHERE) axs[xi].axhline(y=0, color='k', linestyle=':') axs[xi].set_ylabel('$C^{cum}_{CumMerits}$', rotation=0, ha='right', va='center', fontweight='bold', size=ylabelsize);for j inrange(df.shape[0]//T): axs[xi].axvline(x=(j+1)*T, color='grey', ls=':') xi = n_ab + n_b + n_a + n_b +4 axs[xi].set_ylim(auto=True); axs[xi].spines['top'].set_visible(False); axs[xi].spines['right'].set_visible(True); axs[xi].spines['bottom'].set_visible(False) axs[xi].step(df['Ccum'], 'm-', where=WHERE)## axs[xi].step(df['Ucum_Total'], 'k-', where=WHERE)ifnot df_non isNone: axs[xi].step(df_non['Ccum'], 'c-.', where=WHERE) axs[xi].axhline(y=0, color='k', linestyle=':') axs[xi].set_ylabel('$C^{cum}$', rotation=0, ha='right', va='center', fontweight='bold', size=ylabelsize);for j inrange(df.shape[0]//T): axs[xi].axvline(x=(j+1)*T, color='grey', ls=':') axs[xi].set_xlabel('$t\ \mathrm{[decision\ windows]}$', rotation=0, ha='right', va='center', fontweight='bold', size=ylabelsize);if(pars['legendLabels']): fig.legend(labels=pars['legendLabels'], loc='lower left', fontsize=16)def plot_evalu_comparison(self, df1, df2, df3, pars=defaultdict(str)): legendlabels = ['X__BuyBelow', 'X__Bellman'] n_charts =5 ylabelsize =14 mpl.rcParams['lines.linewidth'] =1.2 fig, axs = plt.subplots(n_charts, sharex=True) fig.set_figwidth(13); fig.set_figheight(9) thetaStarStr = []forcmpin pars["thetaStar"]: thetaStarStr.append(f'{cmp:.1f}') thetaStarStr ='('+', '.join(thetaStarStr) +')' fig.suptitle(pars['suptitle'], fontsize=14) xi =0 axs[xi].set_ylim(auto=True); axs[xi].spines['top'].set_visible(False); axs[xi].spines['right'].set_visible(True); axs[xi].spines['bottom'].set_visible(False) axs[xi].step(df1[f'x_t'], 'r-', where='post') axs[xi].step(df2[f'x_t'], 'g-.', where='post') axs[xi].step(df3[f'x_t'], 'b:', where='post') axs[xi].axhline(y=0, color='k', linestyle=':') y1ab ='$x_{t}$' axs[xi].set_ylabel(y1ab, rotation=0, ha='right', va='center', fontweight='bold', size=ylabelsize)for j inrange(df1.shape[0]//T): axs[xi].axvline(x=(j+1)*T, color='grey', ls=':') xi =1 axs[xi].set_ylim(auto=True); axs[xi].spines['top'].set_visible(False); axs[xi].spines['right'].set_visible(True); axs[xi].spines['bottom'].set_visible(False) axs[xi].step(df1[f'R_t'], 'r-', where='post') axs[xi].step(df2[f'R_t'], 'g-.', where='post') axs[xi].step(df3[f'R_t'], 'b:', where='post') axs[xi].axhline(y=0, color='k', linestyle=':') y1ab ='$R_{t}$' axs[xi].set_ylabel(y1ab, rotation=0, ha='right', va='center', fontweight='bold', size=ylabelsize)for j inrange(df1.shape[0]//T): axs[xi].axvline(x=(j+1)*T, color='grey', ls=':') xi =2 axs[xi].set_ylim(auto=True); axs[xi].spines['top'].set_visible(False); axs[xi].spines['right'].set_visible(True); axs[xi].spines['bottom'].set_visible(False) axs[xi].step(df1[f'p_t'], 'r-', where='post') axs[xi].step(df2[f'p_t'], 'g-.', where='post') axs[xi].step(df3[f'p_t'], 'b:', where='post') axs[xi].axhline(y=0, color='k', linestyle=':')if(pars['lower_non']): axs[xi].text(-4, pars['lower_non'], r'$\theta^{lower}$'+f"={pars['lower_non']:.1f}", size=10, color='c')if(pars['lower_non']): axs[xi].axhline(y=pars['lower_non'], color='c', linestyle=':')if(pars['upper_non']): axs[xi].text(-4, pars['upper_non'], r'$\theta^{upper}$'+f"={pars['upper_non']:.1f}", size=10, color='c')if(pars['upper_non']): axs[xi].axhline(y=pars['upper_non'], color='c', linestyle=':')if(pars['lower']): axs[xi].text(-4, pars['lower'], r'$\theta^{lower}$'+f"={pars['lower']:.1f}", size=10, color='m')if(pars['lower']): axs[xi].axhline(y=pars['lower'], color='m', linestyle=':')if(pars['upper']): axs[xi].text(-4, pars['upper'], r'$\theta^{upper}$'+f"={pars['upper']:.1f}", size=10, color='m')if(pars['upper']): axs[xi].axhline(y=pars['upper'], color='m', linestyle=':')if(pars['alpha_non']): axs[xi].text(-4, pars['alpha_non'], r'$\theta^{alpha}$'+f"={pars['alpha_non']:.1f}", size=10, color='c')if(pars['alpha_non']): axs[xi].axhline(y=pars['alpha_non'], color='c', linestyle=':')if(pars['trackSignal_non']): axs[xi].text(-4, pars['trackSignal_non'], r'$\theta^{trackSignal}$'+f"={pars['trackSignal_non']:.1f}", size=10, color='c')if(pars['trackSignal_non']): axs[xi].axhline(y=pars['trackSignal_non'], color='c', linestyle=':')if(pars['alpha']): axs[xi].text(-4, pars['alpha'], r'$\theta^{alpha}$'+f"={pars['alpha']:.1f}", size=10, color='m')if(pars['alpha']): axs[xi].axhline(y=pars['alpha'], color='m', linestyle=':')if(pars['trackSignal']): axs[xi].text(-4, pars['trackSignal'], r'$\theta^{trackSignal}$'+f"={pars['trackSignal']:.1f}", size=10, color='m')if(pars['trackSignal']): axs[xi].axhline(y=pars['trackSignal'], color='m', linestyle=':') y1ab ='$p_{t}$' axs[xi].set_ylabel(y1ab, rotation=0, ha='right', va='center', fontweight='bold', size=ylabelsize)for j inrange(df1.shape[0]//T): axs[xi].axvline(x=(j+1)*T, color='grey', ls=':') xi =3 axs[xi].set_ylim(auto=True); axs[xi].spines['top'].set_visible(False); axs[xi].spines['right'].set_visible(True); axs[xi].spines['bottom'].set_visible(False) axs[xi].step(df1['b_t_val'], 'r-', where='post') axs[xi].step(df2['b_t_val'], 'g-.', where='post') axs[xi].step(df3['b_t_val'], 'b:', where='post') axs[xi].axhline(y=0, color='k', linestyle=':') y1ab ='$b_{t,val}$' axs[xi].set_ylabel(y1ab, rotation=0, ha='right', va='center', fontweight='bold', size=ylabelsize)for j inrange(df1.shape[0]//T): axs[xi].axvline(x=(j+1)*T, color='grey', ls=':') xi =4 axs[xi].set_ylim(auto=True); axs[xi].spines['top'].set_visible(False); axs[xi].spines['right'].set_visible(True); axs[xi].spines['bottom'].set_visible(False) axs[xi].step(df1['Ccum'], 'r-', where='post') axs[xi].step(df2['Ccum'], 'g-.', where='post') axs[xi].step(df3['Ccum'], 'b:', where='post') axs[xi].axhline(y=0, color='k', linestyle=':') axs[xi].set_ylabel('$\mathrm{cumC}$'+'\n'+'$\mathrm{(Profit)}$'+'\n'+''+'$\mathrm{[\$]}$', rotation=0, ha='right', va='center', fontweight='bold', size=ylabelsize); axs[xi].set_xlabel('$t\ \mathrm{[decision\ windows]}$', rotation=0, ha='right', va='center', fontweight='bold', size=ylabelsize);for j inrange(df1.shape[0]//T): axs[xi].axvline(x=(j+1)*T, color='grey', ls=':') fig.legend(## [leg], labels=legendlabels, title="Policies", loc='upper right', fontsize=16, fancybox=True, shadow=True, ncol=1)
4.6 Policy Evaluation
4.6.1 Training/Tuning
## setup labels to plot infoRAvail_t_labels = ['RAvail_t_'+an for an in aNAMES]RCumShifts_t_labels = ['RCumShifts_t_'+an for an in aNAMES]RCumMerits_t_labels = ['RCumMerits_t_'+an for an in aNAMES]DShift_t_labels = ['DShift_t_'+rt for rt in RESOURCE_TYPES]xAlloc_t_labels = ['Allocd_t_'+abn for abn in abNAMES]labels = ['piName', 'theta', 'l'] +\ ['t', 'dt'] +\ RAvail_t_labels + RCumShifts_t_labels + RCumMerits_t_labels +\ DShift_t_labels +\ ['Ucum_'+rt for rt in RESOURCE_TYPES] +\ ['Ucum_Total'] +\ ['Ccum_CumShifts'] +\ ['Ccum_SickProb'] +\ ['Ccum_CumMerits'] +\ ['Ccum'] +\ xAlloc_t_labels
## def grid_search_thetas(thetas1, thetas2, thetas1_name, thetas2_name):# thetas = [# P.build_theta({thetas1_name: thA0, thetas2_name: thB0})# for thA0 in thetas1# for thB0 in thetas2# ]# return thetas## def grid_search_thetas(thetas1, thetas1_name):# thetas = [# P.build_theta({thetas1_name: thA0})# for thA0 in thetas1# ]# return thetas
## print out the 14-day (2-week) scheduleprint(f"SCHEDULE:")for res_alloc in resource_allocs: _,_,id,resType,_,_,_ = res_alloc.split('_') resName =id+'_'+resTypeprint(f'\n{resName}:') sched_list =list(schedule.loc[ schedule[res_alloc] ==True, ['dt', res_alloc] ]['dt'])print(f"{[ts.strftime('%a %b %d %Y') for ts in sched_list]}")
SCHEDULE:
1_Courtesy:
['Sun Oct 29 2023', 'Sun Nov 05 2023', 'Tue Nov 07 2023']
2_Courtesy:
['Tue Oct 31 2023', 'Wed Nov 01 2023', 'Thu Nov 02 2023', 'Fri Nov 03 2023', 'Mon Nov 06 2023', 'Tue Nov 07 2023', 'Wed Nov 08 2023', 'Thu Nov 09 2023', 'Fri Nov 10 2023']
3_Courtesy:
['Tue Oct 31 2023', 'Wed Nov 01 2023', 'Thu Nov 02 2023', 'Sat Nov 04 2023', 'Mon Nov 06 2023', 'Tue Nov 07 2023', 'Wed Nov 08 2023', 'Sat Nov 11 2023']
4_Courtesy:
['Mon Oct 30 2023', 'Tue Oct 31 2023', 'Wed Nov 01 2023', 'Sat Nov 04 2023', 'Tue Nov 07 2023', 'Sat Nov 11 2023']
5_Courtesy:
['Fri Nov 03 2023', 'Fri Nov 10 2023']
6_Courtesy:
['Fri Nov 03 2023', 'Sun Nov 05 2023', 'Tue Nov 07 2023', 'Wed Nov 08 2023', 'Fri Nov 10 2023']
7_Courtesy:
['Sat Nov 04 2023', 'Mon Nov 06 2023', 'Tue Nov 07 2023', 'Wed Nov 08 2023', 'Sat Nov 11 2023']
8_Stocker:
['Sun Oct 29 2023', 'Mon Oct 30 2023', 'Tue Oct 31 2023', 'Wed Nov 01 2023', 'Fri Nov 03 2023', 'Sat Nov 04 2023', 'Tue Nov 07 2023', 'Sat Nov 11 2023']
9_Stocker:
['Mon Oct 30 2023', 'Tue Oct 31 2023', 'Wed Nov 01 2023', 'Thu Nov 02 2023', 'Fri Nov 03 2023', 'Sun Nov 05 2023', 'Mon Nov 06 2023', 'Tue Nov 07 2023', 'Wed Nov 08 2023', 'Thu Nov 09 2023', 'Fri Nov 10 2023']
10_Stocker:
['Sat Nov 04 2023', 'Sun Nov 05 2023', 'Thu Nov 09 2023', 'Sat Nov 11 2023']
