Admin-Website-Consumer Multi-Agent System Using PyMDP (Part 2)

Back to Portfolio of Projects |  LearnableLoopAI.com |  Blog | 

In Part 1 we setup a three-entity multi-agent system that use PyMDP. In Part 2 we implement the first version of the client’s required proof-of-concept (POC) wherein the agents of a 3-entity system interact to strive for the efficient management of marketing and advertising effort making use of Google Ads, in particular, a search campaign.

import os
import sys
import pathlib
import numpy as np
## import seaborn as sns
import matplotlib.pyplot as plt
import copy
## from pprint import pprint ##.

path = pathlib.Path(os.getcwd())
module_path = str(path.parent) + '/'
sys.path.append(module_path)

from pymdp.agent import Agent
from pymdp import utils
from pymdp.maths import softmax

import matplotlib.pyplot as plt
from matplotlib.gridspec import GridSpec
from matplotlib.lines import Line2D
from matplotlib.ticker import MaxNLocator

## ?utils.get_model_dimensions_from_labels
##. rewrite method to allow for shorter names so that the _lab dict can be used more
##    easily when setting up the matrices
def get_model_dimensions_from_labels(model_labels):
    ## modalities = model_labels['observations']
    modalities = model_labels['y'] ##.
    num_modalities = len(modalities.keys())
    num_obs = [len(modalities[modality]) for modality in modalities.keys()]

    ## factors = model_labels['states']
    factors = model_labels['s'] ##.
    num_factors = len(factors.keys())
    num_states = [len(factors[factor]) for factor in factors.keys()]

    ## if 'actions' in model_labels.keys():
    if 'a' in model_labels.keys(): ##.        
        ## controls = model_labels['actions']
        controls = model_labels['a'] ##.
        num_control_fac = len(controls.keys())
        num_controls = [len(controls[cfac]) for cfac in controls.keys()]
        return num_obs, num_modalities, num_states, num_factors, num_controls, num_control_fac
    else:
        return num_obs, num_modalities, num_states, num_factors

1 Business Understanding

The client has a need for a proof-of-concept (POC) wherein the agents of a 3-entity system interact to strive for efficient management of marketing and advertising effort making use of Google Ads, in particular, a search campaign.

Running a successful Google Ads search campaign requires making a range of decisions. Here is a structured list of key decisions that can be made:

• Campaign Setup Decisions

  • Goal selection:
    Choose a primary objective for the campaign:

    • Sales
      
    • Leads
      
    • Website traffic
      
    • Product and brand consideration

  • Campaign type:
    Select “Search Campaign” as the type. Optionally, enable the Display Network to expand reach.

  • Bidding strategy:
    Choose how you’ll bid on keywords:

    • Manual CPC (Cost-per-click)
    • Enhanced CPC
    • Automated bidding strategies:
      • Maximize Clicks
        
      • Maximize Conversions
        
      • Target CPA (Cost-per-acquisition)
        
      • Target ROAS (Return on ad spend)
        
• Targeting Decisions

  • Location targeting:
    Choose specific geographic areas (countries, cities, or a radius around a location).
    Options: People in or interested in targeted locations.

  • Language targeting:
    Select languages spoken by your audience.

  • Device targeting:
    Adjust bids or exclude devices (mobile, desktop, tablet) based on performance.

  • Ad schedule:
    Choose specific days or times to show ads, such as business hours.

• Budgeting Decisions

  • Daily budget:
    Set a daily budget to control overall spend.

  • Bid adjustments:
    Increase or decrease bids for devices, locations, or demographics based on performance.

• Keyword Strategy Decisions

  • Keyword selection:
    Choose keywords relevant to your products/services. Use tools like the Google Keyword Planner.

  • Match type selection:
    Define how closely a search query should match your keywords:

    • Broad Match
      
    • Broad Match Modifier
      
    • Phrase Match
      
    • Exact Match
      
    • Negative Keywords (to prevent irrelevant clicks)
• Ad Copy Decisions
  • Ad type:
    Choose between:
    • Expanded Text Ads (ETA)
      
    • Responsive Search Ads (RSA)
  • Ad copy creation:
    Focus on:
    • Headline variations (up to 15 for RSA)
      
    • Description lines
      
    • Display URL
      
    • Call-to-action (CTA)
  • Ad extensions:
    Select ad extensions to enhance visibility:
    • Sitelinks
      
    • Callouts
      
    • Structured snippets
      
    • Call extension
      
    • Location extension
      
    • Price extension
      
    • App extension
      
• Conversion Tracking Decisions
  • Conversion actions:
    Decide which user actions to track as conversions:
    • Form submissions
      
    • Phone calls
      
    • Downloads
      
    • Purchases
  • Attribution model:
    Select an attribution model:
    • Last click
      
    • First click
      
    • Linear
      
    • Time decay
      
    • Position-based
      
    • Data-driven
      
• Performance Optimization Decisions

  • Quality Score improvement:
    Focus on improving ad relevance, landing page experience, and expected CTR.

  • A/B testing:
    Continuously test different ad copies, extensions, and bidding strategies.

  • Bid management:
    Adjust bids for high-performing keywords and reduce bids for low-performing ones.

  • Search term analysis:
    Regularly review actual search terms triggering ads and update negative keywords.

Next we will assign each of these categories of decisions to a specific entity.

1.1 Administrator entity

The Administrator entity will administer the operation of the campaign and focuses on strategic decisions, budget management, and overall campaign setup. It is responsible for high-level strategy and resource allocation decisions. Of the above list of decisions, we assign the following to this entity:

• Campaign Setup Decisions

  • Goal selection:
    Choose a primary objective for the campaign:

    • Sales
      
    • Leads
      
    • Website traffic
      
    • Product and brand consideration

  • Campaign type:
    Select “Search Campaign” as the type. Optionally, enable the Display Network to expand reach.

  • Bidding strategy:
    Choose how you’ll bid on keywords:

    • Manual CPC (Cost-per-click)
    • Enhanced CPC
    • Automated bidding strategies:
      • Maximize Clicks
        
      • Maximize Conversions
        
      • Target CPA (Cost-per-acquisition)
        
      • Target ROAS (Return on ad spend)
        
• Budgeting Decisions

  • Daily budget:
    Set a daily budget to control overall spend.

  • Bid adjustments:
    Increase or decrease bids for devices, locations, or demographics based on performance.

• Conversion Tracking Decisions
  • Conversion actions:
    Decide which user actions to track as conversions:
    • Form submissions
      
    • Phone calls
      
    • Downloads
      
    • Purchases
  • Attribution model:
    Select an attribution model:
    • Last click
      
    • First click
      
    • Linear
      
    • Time decay
      
    • Position-based
      
    • Data-driven

1.2 Website entity

The Website entity is responsible for ensuring the ad relevance and user experience by focusing on keyword strategy, ad content, and extensions. The Website entity focuses on the technical aspects of ad creation and ensuring relevance. The Website agent ensures that the ads match user intent and are relevant to the content and structure of the site, impacting both Quality Score and click-through rates.

• Keyword Strategy Decisions

  • Keyword selection:
    Choose keywords relevant to your products/services. Use tools like the Google Keyword Planner.

  • Match type selection:
    Define how closely a search query should match your keywords:

    • Broad Match
    • Broad Match Modifier
      
    • Phrase Match
    • Exact Match
    • Negative Keywords (to prevent irrelevant clicks)
• Ad Copy Decisions
  • Ad type:
    Choose between:
    • Expanded Text Ads (ETA)
      
    • Responsive Search Ads (RSA)
  • Ad copy creation:
    Focus on:
    • Headline variations (up to 15 for RSA)
      
    • Description lines
      
    • Display URL
      
    • Call-to-action (CTA)
  • Ad extensions:
    Select ad extensions to enhance visibility:
    • Sitelinks
      
    • Callouts
      
    • Structured snippets
      
    • Call extension
      
    • Location extension
      
    • Price extension
      
    • App extension

1.3 Consumer entity

The Consumer entity represents user behavior, preferences, and demand patterns, focusing on targeting and performance optimization. The Consumer entity optimizes the campaign based on user behavior and feedback. The Consumer agent represents how the campaign should target real users (customers) and optimizes the campaign based on user behavior and feedback.

• Targeting Decisions

  • Location targeting:
    Choose specific geographic areas (countries, cities, or a radius around a location).
    Options: People in or interested in targeted locations.

  • Language targeting:
    Select languages spoken by your audience.

  • Device targeting:
    Adjust bids or exclude devices (mobile, desktop, tablet) based on performance.

  • Ad schedule:
    Choose specific days or times to show ads, such as business hours.

• Performance Optimization Decisions

  • Quality Score improvement:
    Focus on improving ad relevance, landing page experience, and expected CTR.

  • A/B testing:
    Continuously test different ad copies, extensions, and bidding strategies.

  • Bid management:
    Adjust bids for high-performing keywords and reduce bids for low-performing ones.

  • Search term analysis:
    Regularly review actual search terms triggering ads and update negative keywords.

2 Entity interactions

Only a minimal set of the above decisions will be implemented in this version of the POC. The following diagram is a representation of the interactions between the entities in this project.

NOTE ON MULTI-AGENT GRAPHICS

• Each entity has
  • an agent
  • an environment
• The name/identifier of an entity always starts with a single capital letter, e.g.
  • ‘Administrator’, or in abbreviated form, say ‘Adm’
  • ‘Website’, or in abbreviated form, say ‘Web’ or ‘Wsi’
  • ‘Consumer’, or in abbreviated form, say ‘Csr’ or ‘Con’
• In general, the environment for a specific entity agent consists of multiple environment segments
• Each entity has one colocated environment segment, e.g.
  • An ant entity’s agent (inside its brain) is colocated with the ant’s own environment segment which contains, for example, a state variable capturing the ant’s location. This ant agent may also be interested in influencing other environment segments.
• An entity’s environment consists of its colocated environment segment as well as (potentially) all other entities’ colocated environment segments
• An entity’s agent may be interested in emitted observations from a subset of other entities’ environment segments
• An entity’s agent may be interested in emitting actions to a subset of other entities’ environment segments
• The graphic of an ent has a vertical dotted line separating its agt from its colocated environment
• If an entity mainly interacts with one other entity, their graphics are usually placed next to each other horizontally
• All actions flow at the top
• All observations flow at the bottom
• When too many actions and observations need to be shown, one action busbar is drawn at the top, and one observation busbar at the bottom
• An entity’s agent usually attempts to steer or influence at least one environment segment
• An entity’s agent is usually informed by observations from at least one environment segment
• An entity may be impacted by exogenous information/signals, indicated by $W^{\mathrm{Ent}}$
• All variables/signals $x$ are identified by $x^{\mathrm{Ent}}$
• The standard symbols for the identification of variables/signals are:
  • observation $y^{\mathrm{Ent}}$ where Ent is the emitter of the observation
  • action $a^{\mathrm{Ent}}$ where Ent is the emitter of the action
  • state $s^{\mathrm{Ent}}$ where Ent is the owner of the inferred state
  • parameter ${\theta }^{\mathrm{Ent}}$ where Ent is the owner of the inferred parameter
  • control $u^{\mathrm{Ent}}$ where Ent is the owner of the inferred control
• “True” (i.e. non-inferred) symbols have a breve symbol on top, e.g.
  • ${\breve{\theta }}^{\mathrm{Web}}$ is the true parameters of the Website entity
  • ${\breve{s}}^{\mathrm{Web}}$ is the true state of the Website entity
  • ${\breve{\theta }}^{\mathrm{Csr}}$ is the true parameters of the Consumer entity
  • ${\breve{s}}^{\mathrm{Csr}}$ is the true state of the Consumer entity

The Administrator agent has symbols:

• $u^{\mathrm{Web}}$ is the inferred control states of the Website entity
• ${\theta }^{\mathrm{Web}}$ is the inferred parameters of the Website entity
• $s^{\mathrm{Web}}$ is the inferred state of the Website entity
• $y^{\mathrm{Web}}$ is the observation from the Website entity

The Website environment has symbols:

• $a^{\mathrm{Adm}}$ is the action from the Administrator entity
• ${\breve{\theta }}^{\mathrm{Web}}$ is the true parameters of the Website entity
• ${\breve{s}}^{\mathrm{Web}}$ is the true state of the Website entity
• $W^{\mathrm{Web}}$ is the exogenous information impacting the Website entity
• $y^{\mathrm{Web}}$ is the observation from the Website entity

The Website agent has symbols:

• $u^{\mathrm{Csr}}$ is the inferred control states of the Consumer entity
• ${\theta }^{\mathrm{Csr}}$ is the inferred parameters of the Consumer entity
• $s^{\mathrm{Csr}}$ is the inferred state of the Consumer entity
• $y^{\mathrm{Csr}}$ is the observation from the Consumer entity

The Consumer environment has symbols:

• $a^{\mathrm{Web}}$ is the action from the Website entity
• ${\breve{\theta }}^{\mathrm{Csr}}$ is the true parameters of the Consumer entity
• ${\breve{s}}^{\mathrm{Csr}}$ is the true state of the Consumer entity
• $W^{\mathrm{Csr}}$ is the exogenous information impacting the Consumer entity
• $y^{\mathrm{Csr}}$ is the observation from the Consumer entity

2.1 Administrator agent

This is the Administrator agent’s generative model for its environment.

2.1.1 State factors

The inferred state factors from the Administrator’s environment are:

• $s_1^{\mathrm{Web}}$ (AD_TYPE)
• $s_2^{\mathrm{Web}}$ (AD_COPY_CREATION)
• $s_3^{\mathrm{Web}}$ (AD_EXTENSIONS) [LATER]
• $s_1^{\mathrm{Csr}}$ (FEEDBACK) [LATER]

2.1.1.1 $s_1^{\mathrm{Web}}$ (AD_TYPE)

It has possible values/levels:

• $s_1^{\mathrm{Web}}=0$, EXPANDED_TEXT_ADS
• $s_1^{\mathrm{Web}}=1$, RESPONSIVE_SEARCH_ADS

2.1.1.2 $s_2^{\mathrm{Web}}$ (AD_COPY_CREATION)

It has possible values/levels:

• $s_2^{\mathrm{Web}}=0$, DESCRIPTION_LINES
• $s_2^{\mathrm{Web}}=1$, DISPLAY_URL
• $s_2^{\mathrm{Web}}=2$, CALL_TO_ACTION

2.1.1.3 $s_3^{\mathrm{Web}}$ (AD_EXTENSIONS) [LATER]

It has possible values/levels:

• $s_3^{\mathrm{Web}}=0$, SITE_LINKS
• $s_3^{\mathrm{Web}}=1$, CALLOUTS
• $s_3^{\mathrm{Web}}=2$, STRUCTURED_SNIPPETS

2.1.1.4 $s_1^{\mathrm{Csr}}$ (FEEDBACK) [LATER]

It has possible values/levels:

• $s_1^{\mathrm{Csr}}=0$, POSITIVE
• $s_1^{\mathrm{Csr}}=1$, NEGATIVE

2.1.2 Observation modalities

The observed observation modalities from the Administrator’s environment are:

• $y_1^{\mathrm{Web}}$ (AD_TYPE_OBS)
• $y_2^{\mathrm{Web}}$ (AD_COPY_CREATION_OBS)
• $y_3^{\mathrm{Web}}$ (AD_EXTENSIONS_OBS)
• $y_1^{\mathrm{Csr}}$ (FEEDBACK_OBS)

2.1.2.1 $y_1^{\mathrm{Web}}$ (AD_TYPE_OBS)

It has possible values/levels:

• $y_1^{\mathrm{Web}}=0$, EXPANDED_TEXT_ADS_OBS
• $y_1^{\mathrm{Web}}=1$, RESPONSIVE_SEARCH_ADS_OBS
• $y_1^{\mathrm{Web}}=2$, UNKNOWN_OBS

2.1.2.2 $y_2^{\mathrm{Web}}$ (AD_COPY_CREATION_OBS)

It has possible values/levels:

• $y_2^{\mathrm{Web}}=0$, DESCRIPTION_LINES_OBS
• $y_2^{\mathrm{Web}}=1$, DISPLAY_URL_OBS
• $y_2^{\mathrm{Web}}=2$, CALL_TO_ACTION_OBS

2.1.2.3 $y_3^{\mathrm{Web}}$ (AD_EXTENSIONS_OBS)

It has possible values/levels:

• $y_3^{\mathrm{Web}}=0$, SITE_LINKS_OBS
• $y_3^{\mathrm{Web}}=1$, CALLOUTS_OBS
• $y_3^{\mathrm{Web}}=2$, STRUCTURED_SNIPPETS_OBS

2.1.2.4 $y_1^{\mathrm{Csr}}$ (FEEDBACK_OBS)

It has possible values/levels:

• $y_1^{\mathrm{Csr}}=0$, POSITIVE_OBS
• $y_1^{\mathrm{Csr}}=1$, NEGATIVE_OBS

2.1.3 Control factors

The ‘control state’ factors are the agent’s representation of the control states (or actions) that it believes can influence the dynamics of the hidden states - i.e. hidden state factors that are under the influence of control states are are ‘controllable’. In practice, we often encode every hidden state factor as being under the influence of control states, but the ‘uncontrollable’ hidden state factors are driven by a trivially-1-dimensional control state or action-affordance. This trivial action simply ‘maintains the default environmental dynamics as they are’ i.e. does nothing. This will become more clear when we set up the transition model (the B matrices) below.

The action control factors to the Administrator’s environment are:

• $a_1^{\mathrm{Adm}}$ (NULL)
• $a_2^{\mathrm{Adm}}$ (SET_MATCH_TYPE_ACTION)

2.1.3.1 $a_1^{\mathrm{Adm}}$ (NULL)

It has possible values/levels:

• $a_1^{\mathrm{Adm}}=0$, NULL_ACT

2.1.3.2 $a_2^{\mathrm{Adm}}$ (SET_MATCH_TYPE_ACTION)

It has possible values/levels:

• $a_1^{\mathrm{Adm}}=0$, BROAD_MATCH_ACT
• $a_2^{\mathrm{Adm}}=1$, PHRASE_MATCH_ACT
• $a_3^{\mathrm{Adm}}=2$, EXACT_MATCH_ACT

(Controllable-) Transition Dynamics

Importantly, some hidden state factors are controllable by the agent, meaning that the probability of being in state $i$ at $t+1$ isn’t merely a function of the state at $t$, but also of actions (or from the generative model’s perspective, control states ). So each transition likelihood or B matrix encodes conditional probability distributions over states at $t+1$, where the conditioning variables are both the states at $t-1$ and the actions at $t-1$. This extra conditioning on control states is encoded by a third, lagging dimension on each factor-specific B matrix. So they are technically B “tensors” or an array of action-conditioned B matrices.

For example, in our case the 2nd hidden state factor $s_2^{\mathrm{Web}}$ (AD_COPY_CREATION) is under the control of the agent, which means the corresponding transition likelihoods B[1] are indexable by both previous state and action.

_labAdm = { ## labels for Administrator
    "a": {
        "aᴬᵈᵐ₁": [ ## "NULL"
            "NULL_ACT", 
        ],
        "aᴬᵈᵐ₂": [ ## "SET_MATCH_TYPE_ACTION"
            "BROAD_MATCH_ACT", 
            "PHRASE_MATCH_ACT", 
            "EXACT_MATCH_ACT"
        ],
    },
    "s": {
        "sᵂᵉᵇ₁": [ ## "AD_TYPE"
            "EXPANDED_TEXT_ADS", 
            "RESPONSIVE_SEARCH_ADS",
        ],
        "sᵂᵉᵇ₂": [ ## "AD_COPY_CREATION"
            "DESCRIPTION_LINES", 
            "DISPLAY_URL", 
            "CALL_TO_ACTION"
        ],
        ## LATER:
        # "sᵂᵉᵇ₃": [ ## "AD_EXTENSIONS"
        #     "SITE_LINKS", 
        #     "CALLOUTS",             
        #     "STRUCTURED_SNIPPETS"
        # ],
        # "sᶜˢʳ₁": [ ## "FEEDBACK"
        #     "POSITIVE", 
        #     "NEGATIVE"
        # ],
    },
    "s̆": {
        "s̆ᵂᵉᵇ₁": [ ## "AD_TYPE"
            "EXPANDED_TEXT_ADS", 
            "RESPONSIVE_SEARCH_ADS",
        ],
        "s̆ᵂᵉᵇ₂": [ ## "AD_COPY_CREATION"
            "DESCRIPTION_LINES", 
            "DISPLAY_URL", 
            "CALL_TO_ACTION"
        ],
    },    
    "y": {
        "yᵂᵉᵇ₁": [ ## "AD_TYPE_OBS"
            "EXPANDED_TEXT_ADS_OBS",
            "RESPONSIVE_SEARCH_ADS_OBS",
            "UNKNOWN_OBS"
        ],
        "yᵂᵉᵇ₂": [ ## "AD_COPY_CREATION_OBS"
            "DESCRIPTION_LINES_OBS",
            "DISPLAY_URL_OBS",
            "CALL_TO_ACTION_OBS"
        ],
        "yᵂᵉᵇ₃": [ ## "AD_EXTENSIONS_OBS"
            "SITE_LINKS_OBS",
            "CALLOUTS_OBS",
            "STRUCTURED_SNIPPETS_OBS"
        ],
        ## LATER:
        # "yᶜˢʳ₁": [ ## "FEEDBACK_OBS"
        #     "POSITIVE_OBS",
        #     "NEGATIVE_OBS"
        # ]
    },
}
_yWeb_car,_yWeb_num, _sWeb_car,_sWeb_num, _aAdm_car,_aAdm_num = get_model_dimensions_from_labels(_labAdm) ##.
_yWeb_car,_yWeb_num, _sWeb_car,_sWeb_num, _aAdm_car,_aAdm_num

([3, 3, 3], 3, [2, 3], 2, [1, 3], 2)

print(f'{_aAdm_car=}') ## cardinality of control factors
print(f'{_aAdm_num=}') ## number of control factors

print(f'{_sWeb_car=}') ## cardinality of state factors
print(f'{_sWeb_num=}') ## number of state factors

print(f'{_yWeb_car=}') ## cardinality of observation modalities
print(f'{_yWeb_num=}') ## number of observation modalities

_aAdm_car=[1, 3]
_aAdm_num=2
_sWeb_car=[2, 3]
_sWeb_num=2
_yWeb_car=[3, 3, 3]
_yWeb_num=3

_aAdm_fac_names = list(_labAdm['a'].keys()); print(f'{_aAdm_fac_names=}') ## control factor names
_sWeb_fac_names = list(_labAdm['s'].keys()); print(f'{_sWeb_fac_names=}') ## state factor names
_s̆Web_fac_names = list(_labAdm['s̆'].keys()); print(f'{_s̆Web_fac_names=}') ## state factor names
_yWeb_mod_names = list(_labAdm['y'].keys()); print(f'{_yWeb_mod_names=}') ## observation modality names

_aAdm_fac_names=['aᴬᵈᵐ₁', 'aᴬᵈᵐ₂']
_sWeb_fac_names=['sᵂᵉᵇ₁', 'sᵂᵉᵇ₂']
_s̆Web_fac_names=['s̆ᵂᵉᵇ₁', 's̆ᵂᵉᵇ₂']
_yWeb_mod_names=['yᵂᵉᵇ₁', 'yᵂᵉᵇ₂', 'yᵂᵉᵇ₃']

2.1.4 Observation likelihood matrix, $\mathbf{A}$

We now setup the observation likelihood matrix which is the first main component of generative model.

## A = utils.obj_array_zeros([[o] + _car_state for _, o in enumerate(_car_obser)]) ##.
_Aᴬᵈᵐ = utils.obj_array_zeros([[y_car] + _sWeb_car for y_car in _yWeb_car])
print(f'{len(_Aᴬᵈᵐ)=}')
_Aᴬᵈᵐ

len(_Aᴬᵈᵐ)=3

array([array([[[0., 0., 0.],
               [0., 0., 0.]],

              [[0., 0., 0.],
               [0., 0., 0.]],

              [[0., 0., 0.],
               [0., 0., 0.]]]), array([[[0., 0., 0.],
                                        [0., 0., 0.]],

                                       [[0., 0., 0.],
                                        [0., 0., 0.]],

                                       [[0., 0., 0.],
                                        [0., 0., 0.]]]),
       array([[[0., 0., 0.],
               [0., 0., 0.]],

              [[0., 0., 0.],
               [0., 0., 0.]],

              [[0., 0., 0.],
               [0., 0., 0.]]])], dtype=object)

2.1.4.1 Observation modality $y_1^{\mathrm{Web}}$ (AD_TYPE_OBS)

We setup this modality’s likelihood mapping, corresponding to how $y_1^{\mathrm{Web}}$ (AD_TYPE_OBS) is related to hidden states.

_Aᴬᵈᵐ[0][
    _labAdm['y']['yᵂᵉᵇ₁'].index('UNKNOWN_OBS'), 
    :, 
    _labAdm['s']['sᵂᵉᵇ₂'].index('DESCRIPTION_LINES')
] = 1.0
_Aᴬᵈᵐ[0][
    _labAdm['y']['yᵂᵉᵇ₁'].index('UNKNOWN_OBS'), 
    :, 
    _labAdm['s']['sᵂᵉᵇ₂'].index('DISPLAY_URL')
] = 1.0
_Aᴬᵈᵐ[0][
    _labAdm['y']['yᵂᵉᵇ₁'].index('EXPANDED_TEXT_ADS_OBS'),
    _labAdm['s']['sᵂᵉᵇ₁'].index('EXPANDED_TEXT_ADS'), 
    _labAdm['s']['sᵂᵉᵇ₂'].index('CALL_TO_ACTION')
] = 0.8
_Aᴬᵈᵐ[0][
    _labAdm['y']['yᵂᵉᵇ₁'].index('RESPONSIVE_SEARCH_ADS_OBS'),
    _labAdm['s']['sᵂᵉᵇ₁'].index('EXPANDED_TEXT_ADS'), 
    _labAdm['s']['sᵂᵉᵇ₂'].index('CALL_TO_ACTION')
] = 0.2

_Aᴬᵈᵐ[0][
    _labAdm['y']['yᵂᵉᵇ₁'].index('RESPONSIVE_SEARCH_ADS_OBS'),
    _labAdm['s']['sᵂᵉᵇ₁'].index('RESPONSIVE_SEARCH_ADS'), 
    _labAdm['s']['sᵂᵉᵇ₂'].index('CALL_TO_ACTION')
] = 0.8
_Aᴬᵈᵐ[0][
    _labAdm['y']['yᵂᵉᵇ₁'].index('EXPANDED_TEXT_ADS_OBS'),
    _labAdm['s']['sᵂᵉᵇ₁'].index('RESPONSIVE_SEARCH_ADS'), 
    _labAdm['s']['sᵂᵉᵇ₂'].index('CALL_TO_ACTION')
] = 0.2

_Aᴬᵈᵐ[0]

array([[[0. , 0. , 0.8],
        [0. , 0. , 0.2]],

       [[0. , 0. , 0.2],
        [0. , 0. , 0.8]],

       [[1. , 1. , 0. ],
        [1. , 1. , 0. ]]])

2.1.4.2 Observation modality $y_2^{\mathrm{Web}}$ (AD_COPY_CREATION_OBS)

We setup this modality’s likelihood mapping, corresponding to how $y_2^{\mathrm{Web}}$ (AD_COPY_CREATION_OBS) is related to hidden states.

_Aᴬᵈᵐ[1][
    _labAdm['y']['yᵂᵉᵇ₂'].index('CALL_TO_ACTION_OBS'), 
    :, 
    _labAdm['s']['sᵂᵉᵇ₂'].index('DESCRIPTION_LINES')
] = 1.0

_Aᴬᵈᵐ[1][
    _labAdm['y']['yᵂᵉᵇ₂'].index('CALL_TO_ACTION_OBS'), 
    :, 
    _labAdm['s']['sᵂᵉᵇ₂'].index('CALL_TO_ACTION')
] = 1.0

_EXPANDED_TEXT_ADS_MAPPING_ADM = softmax(np.array([1.0, 0]))
_RESPONSIVE_SEARCH_ADS_MAPPING_ADM = softmax(np.array([0.0, 1.0]))

_Aᴬᵈᵐ[1][
    _labAdm['y']['yᵂᵉᵇ₂'].index('DESCRIPTION_LINES_OBS'),
    _labAdm['s']['sᵂᵉᵇ₁'].index('EXPANDED_TEXT_ADS'), 
    _labAdm['s']['sᵂᵉᵇ₂'].index('DISPLAY_URL')
] = _EXPANDED_TEXT_ADS_MAPPING_ADM[0]
_Aᴬᵈᵐ[1][
    _labAdm['y']['yᵂᵉᵇ₂'].index('DISPLAY_URL_OBS'),
    _labAdm['s']['sᵂᵉᵇ₁'].index('EXPANDED_TEXT_ADS'), 
    _labAdm['s']['sᵂᵉᵇ₂'].index('DISPLAY_URL')
] = _EXPANDED_TEXT_ADS_MAPPING_ADM[1]

_Aᴬᵈᵐ[1][
    _labAdm['y']['yᵂᵉᵇ₂'].index('DESCRIPTION_LINES_OBS'),
    _labAdm['s']['sᵂᵉᵇ₁'].index('RESPONSIVE_SEARCH_ADS'), 
    _labAdm['s']['sᵂᵉᵇ₂'].index('DISPLAY_URL')
] = _RESPONSIVE_SEARCH_ADS_MAPPING_ADM[0]
_Aᴬᵈᵐ[1][
    _labAdm['y']['yᵂᵉᵇ₂'].index('DISPLAY_URL_OBS'),
    _labAdm['s']['sᵂᵉᵇ₁'].index('RESPONSIVE_SEARCH_ADS'), 
    _labAdm['s']['sᵂᵉᵇ₂'].index('DISPLAY_URL')
] = _RESPONSIVE_SEARCH_ADS_MAPPING_ADM[1]

_Aᴬᵈᵐ[1]

array([[[0.        , 0.73105858, 0.        ],
        [0.        , 0.26894142, 0.        ]],

       [[0.        , 0.26894142, 0.        ],
        [0.        , 0.73105858, 0.        ]],

       [[1.        , 0.        , 1.        ],
        [1.        , 0.        , 1.        ]]])

2.1.4.3 Observation modality $y_3^{\mathrm{Web}}$ (AD_EXTENSIONS_OBS)

We setup this modality’s likelihood mapping, corresponding to how $y_3^{\mathrm{Web}}$ (AD_EXTENSIONS_OBS) is related to hidden states.

_Aᴬᵈᵐ[2][
    _labAdm['y']['yᵂᵉᵇ₃'].index('SITE_LINKS_OBS'), 
    :, 
    _labAdm['s']['sᵂᵉᵇ₂'].index('DESCRIPTION_LINES')
] = 1.0
_Aᴬᵈᵐ[2][
    _labAdm['y']['yᵂᵉᵇ₃'].index('CALLOUTS_OBS'), 
    :, 
    _labAdm['s']['sᵂᵉᵇ₂'].index('DISPLAY_URL')
] = 1.0
_Aᴬᵈᵐ[2][
    _labAdm['y']['yᵂᵉᵇ₃'].index('STRUCTURED_SNIPPETS_OBS'), 
    :, 
    _labAdm['s']['sᵂᵉᵇ₂'].index('CALL_TO_ACTION')
] = 1.0

_Aᴬᵈᵐ[2]

array([[[1., 0., 0.],
        [1., 0., 0.]],

       [[0., 1., 0.],
        [0., 1., 0.]],

       [[0., 0., 1.],
        [0., 0., 1.]]])

print(f'=== _sWeb_car:\n{_sWeb_car}')
print(f'=== _yWeb_car:\n{_yWeb_car}')
_Aᴬᵈᵐ

=== _sWeb_car:
[2, 3]
=== _yWeb_car:
[3, 3, 3]

array([array([[[0. , 0. , 0.8],
               [0. , 0. , 0.2]],

              [[0. , 0. , 0.2],
               [0. , 0. , 0.8]],

              [[1. , 1. , 0. ],
               [1. , 1. , 0. ]]]),
       array([[[0.        , 0.73105858, 0.        ],
               [0.        , 0.26894142, 0.        ]],

              [[0.        , 0.26894142, 0.        ],
               [0.        , 0.73105858, 0.        ]],

              [[1.        , 0.        , 1.        ],
               [1.        , 0.        , 1.        ]]]),
       array([[[1., 0., 0.],
               [1., 0., 0.]],

              [[0., 1., 0.],
               [0., 1., 0.]],

              [[0., 0., 1.],
               [0., 0., 1.]]])], dtype=object)

2.1.5 State transition matrix, $\mathbf{B}$

_control_fac_idx_Adm = [1] ## used in Agent constructor

_Bᴬᵈᵐ = utils.obj_array(_sWeb_num); print(f'{_sWeb_num=}')
print(f'{len(_Bᴬᵈᵐ)=}')

_Bᴬᵈᵐ

_sWeb_num=2
len(_Bᴬᵈᵐ)=2

array([None, None], dtype=object)

2.1.5.1 Control factor $a_1^{\mathrm{Adm}}$ (NULL)

We setup this factor’s mapping, corresponding to how $a_1^{\mathrm{Adm}}$ (NULL) is related to hidden states.

_Bᴬᵈᵐ[0] = np.zeros((_sWeb_car[0], _sWeb_car[0], _aAdm_car[0])); print(f'{_sWeb_car[0]=}, {_sWeb_car[0]=}, {_aAdm_car[0]=}')
_Bᴬᵈᵐ[0]

_sWeb_car[0]=2, _sWeb_car[0]=2, _aAdm_car[0]=1

array([[[0.],
        [0.]],

       [[0.],
        [0.]]])

_p_stochAdm = 0.0
## we cannot influence factor zero, set up the 'default' stationary dynamics - 
## one state just maps to itself at the next timestep with very high probability, 
## by default. So this means the AD_TYPE state can change from one to another with 
## some low probability (p_stoch)
_Bᴬᵈᵐ[0][
    _labAdm['s']['sᵂᵉᵇ₁'].index('EXPANDED_TEXT_ADS'),
    _labAdm['s']['sᵂᵉᵇ₁'].index('EXPANDED_TEXT_ADS'), 
    _labAdm['a']['aᴬᵈᵐ₁'].index('NULL_ACT')
] = 1.0 - _p_stochAdm
_Bᴬᵈᵐ[0][
    _labAdm['s']['sᵂᵉᵇ₁'].index('RESPONSIVE_SEARCH_ADS'),
    _labAdm['s']['sᵂᵉᵇ₁'].index('EXPANDED_TEXT_ADS'), 
    _labAdm['a']['aᴬᵈᵐ₁'].index('NULL_ACT')
] = _p_stochAdm

_Bᴬᵈᵐ[0][
    _labAdm['s']['sᵂᵉᵇ₁'].index('RESPONSIVE_SEARCH_ADS'),
    _labAdm['s']['sᵂᵉᵇ₁'].index('RESPONSIVE_SEARCH_ADS'), 
    _labAdm['a']['aᴬᵈᵐ₁'].index('NULL_ACT')
] = 1.0 - _p_stochAdm
_Bᴬᵈᵐ[0][
    _labAdm['s']['sᵂᵉᵇ₁'].index('EXPANDED_TEXT_ADS'),
    _labAdm['s']['sᵂᵉᵇ₁'].index('RESPONSIVE_SEARCH_ADS'), 
    _labAdm['a']['aᴬᵈᵐ₁'].index('NULL_ACT')
] = _p_stochAdm

_Bᴬᵈᵐ[0]

array([[[1.],
        [0.]],

       [[0.],
        [1.]]])

2.1.5.2 Control factor $a_2^{\mathrm{Adm}}$ (SET_MATCH_TYPE_ACTION)

We setup this factor’s mapping, corresponding to how $a_2^{\mathrm{Adm}}$ (SET_MATCH_TYPE_ACTION) is related to hidden states.

_Bᴬᵈᵐ[1] = np.zeros((_sWeb_car[1], _sWeb_car[1], _aAdm_car[1]))
_Bᴬᵈᵐ[1][
    _labAdm['s']['sᵂᵉᵇ₂'].index('DESCRIPTION_LINES'), 
    :, 
    _labAdm['a']['aᴬᵈᵐ₂'].index('BROAD_MATCH_ACT')
] = 1.0
_Bᴬᵈᵐ[1][
    _labAdm['s']['sᵂᵉᵇ₂'].index('DISPLAY_URL'), 
    :, 
    _labAdm['a']['aᴬᵈᵐ₂'].index('PHRASE_MATCH_ACT')
] = 1.0
_Bᴬᵈᵐ[1][
    _labAdm['s']['sᵂᵉᵇ₂'].index('CALL_TO_ACTION'), 
    :, 
    _labAdm['a']['aᴬᵈᵐ₂'].index('EXACT_MATCH_ACT')
] = 1.0

_Bᴬᵈᵐ[1]

array([[[1., 0., 0.],
        [1., 0., 0.],
        [1., 0., 0.]],

       [[0., 1., 0.],
        [0., 1., 0.],
        [0., 1., 0.]],

       [[0., 0., 1.],
        [0., 0., 1.],
        [0., 0., 1.]]])

print(f'=== _aAdm_car:\n{_aAdm_car}')
print(f'=== _sWeb_car:\n{_sWeb_car}')
_Bᴬᵈᵐ

=== _aAdm_car:
[1, 3]
=== _sWeb_car:
[2, 3]

array([array([[[1.],
               [0.]],

              [[0.],
               [1.]]]), array([[[1., 0., 0.],
                                [1., 0., 0.],
                                [1., 0., 0.]],

                               [[0., 1., 0.],
                                [0., 1., 0.],
                                [0., 1., 0.]],

                               [[0., 0., 1.],
                                [0., 0., 1.],
                                [0., 0., 1.]]])], dtype=object)

2.1.6 Prior preferences vector, $\mathbf{C}$

Now we parameterise the C vector, or the prior beliefs about observations. This will be used in the expression of the prior over actions, which is technically a softmax function of the negative expected free energy of each action. It is the equivalent of the exponentiated reward function in reinforcement learning treatments.

_Cᴬᵈᵐ = utils.obj_array_zeros([y_car for y_car in _yWeb_car])
_Cᴬᵈᵐ

array([array([0., 0., 0.]), array([0., 0., 0.]), array([0., 0., 0.])],
      dtype=object)

_Cᴬᵈᵐ[1][
    _labAdm['y']['yᵂᵉᵇ₂'].index('DESCRIPTION_LINES_OBS'),
] = 1.0
_Cᴬᵈᵐ[1][
    _labAdm['y']['yᵂᵉᵇ₂'].index('DISPLAY_URL_OBS'),
] = -1.0
_Cᴬᵈᵐ[1][
    _labAdm['y']['yᵂᵉᵇ₂'].index('CALL_TO_ACTION_OBS'),
] = 0.0

_Cᴬᵈᵐ[1]

array([ 1., -1.,  0.])

2.1.7 Initialise an instance of the Agent() class:

All you have to do is call Agent(generative_model_params...) where generative_model_params are your A, B, C’s… and whatever parameters of the generative model you want to specify

_agtAdm = Agent(
    A=_Aᴬᵈᵐ, 
    B=_Bᴬᵈᵐ, 
    C=_Cᴬᵈᵐ, 
    control_fac_idx=_control_fac_idx_Adm
)
_agtAdm

<pymdp.agent.Agent at 0x7f513842fe50>

2.2 Administrator environment

This is the Administrator agent’s generative process for its environment.

It is important to note that the generative process doesn’t have to be described by A and B matrices - it can just be the arbitrary ‘rules of the game’ that you ‘write in’ as a modeller. But here we just use the same transition/likelihood matrices to make the sampling process straightforward.

## observation/transition matrices characterising the generative process

## currently only true values of Website
##   should be _Ăᴬᵈᵐ to include true values of complete Administrator env
_Ăᵂᵉᵇ = copy.deepcopy(_Aᴬᵈᵐ)

## True next-state may be calculated without a B matrix
## currently only true values of Website
##   should be _B̆ᴬᵈᵐ to include true values of complete Administrator env
_B̆ᵂᵉᵇ = copy.deepcopy(_Bᴬᵈᵐ)

3.1 Website agent

This is the Website agent’s generative model for its environment.

3.1.1 State factors

The inferred state factors from the Website’s environment are:

• $s_1^{\mathrm{Csr}}$ (AD_SCHEDULE)
• $s_2^{\mathrm{Csr}}$ (LOCATION_TARGET)
• $s_3^{\mathrm{Csr}}$ (LANGUAGE) [LATER]

3.1.1.1 $s_1^{\mathrm{Csr}}$ (AD_SCHEDULE)

It has possible values/levels:

• $s_1^{\mathrm{Csr}}=0$, BUSINESS_HOURS
• $s_1^{\mathrm{Csr}}=1$, AFTER_HOURS

3.1.1.2 $s_2^{\mathrm{Csr}}$ (LOCATION_TARGET)

It has possible values/levels:

• $s_2^{\mathrm{Csr}}=0$, EAST
• $s_2^{\mathrm{Csr}}=1$, CENTRAL
• $s_2^{\mathrm{Csr}}=2$, WEST

3.1.1.3 $s_3^{\mathrm{Csr}}$ (LANGUAGE) [LATER]

It has possible values/levels:

• $s_3^{\mathrm{Csr}}=0$, ENGLISH
• $s_3^{\mathrm{Csr}}=1$, SPANISH
• $s_3^{\mathrm{Csr}}=2$, CHINESE

3.1.2 Observation modalities

The observed observation modalities from the Artifact’s environment are:

• $y_1^{\mathrm{Csr}}$ (AD_SCHEDULE_OBS)
• $y_2^{\mathrm{Csr}}$ (LOCATION_TARGET_OBS)
• $y_3^{\mathrm{Csr}}$ (LANGUAGE_OBS)

3.1.2.1 $y_1^{\mathrm{Csr}}$ (AD_SCHEDULE_OBS)

It has possible values/levels:

• $y_1^{\mathrm{Csr}}=0$, BUSINESS_HOURS_OBS
• $y_1^{\mathrm{Csr}}=1$, AFTER_HOURS_OBS

3.1.2.2 $y_1^{\mathrm{Csr}}$ (LOCATION_TARGET_OBS)

It has possible values/levels:

• $y_1^{\mathrm{Csr}}=0$, EAST_OBS
• $y_1^{\mathrm{Csr}}=1$, CENTRAL_OBS
• $y_1^{\mathrm{Csr}}=2$, WEST_OBS

3.1.1.3 $s_1^{\mathrm{Csr}}$ (LANGUAGE_OBS)

It has three possible values/levels:

• $y_3^{\mathrm{Csr}}=0$, ENGLISH_OBS
• $y_3^{\mathrm{Csr}}=1$, SPANISH_OBS
• $y_3^{\mathrm{Csr}}=2$, CHINESE_OBS

3.1.3 Control factors

The ‘control state’ factors are the agent’s representation of the control states (or actions) that it believes can influence the dynamics of the hidden states - i.e. hidden state factors that are under the influence of control states are are ‘controllable’. In practice, we often encode every hidden state factor as being under the influence of control states, but the ‘uncontrollable’ hidden state factors are driven by a trivially-1-dimensional control state or action-affordance. This trivial action simply ‘maintains the default environmental dynamics as they are’ i.e. does nothing. This will become more clear when we set up the transition model (the B matrices) below.

The action control factors to the Website’s environment are:

• $a_1^{\mathrm{Web}}$ (NULL)
• $a_2^{\mathrm{Web}}$ (SET_DEVICE_TARGET_ACTION)

3.1.3.1 $a_1^{\mathrm{Web}}$ (NULL)

It has possible values/levels:

• $a_1^{\mathrm{Web}}=0$, NULL_ACT

3.1.3.2 $a_2^{\mathrm{Web}}$ (SET_DEVICE_TARGET_ACTION)

It has possible values/levels:

• $a_1^{\mathrm{Web}}=0$, MOBILE_ACT
• $a_2^{\mathrm{Web}}=1$, DESKTOP_ACT
• $a_3^{\mathrm{Web}}=2$, TABLET_ACT

_labWeb = { ## labels for Website interaction
    "a": {
        "aᵂᵉᵇ₁": [ ## "NULL"
            "NULL_ACT",
        ],
        "aᵂᵉᵇ₂": [ ## "SET_DEVICE_TARGET_ACTION"
            "MOBILE_ACT",
            "DESKTOP_ACT",
            "TABLET_ACT"
        ],
    },
    "s": {
        "sᶜˢʳ₁": [ ## "AD_SCHEDULE"
            "BUSINESS_HOURS", 
            "AFTER_HOURS",
        ],
        "sᶜˢʳ₂": [ ## "LOCATION_TARGET"
            "EAST", 
            "CENTRAL",
            "WEST"
        ],
    },
    "s̆": {
        "s̆ᶜˢʳ₁": [ ## "AD_SCHEDULE"
            "BUSINESS_HOURS", 
            "AFTER_HOURS",
        ],
        "s̆ᶜˢʳ₂": [ ## "LOCATION_TARGET"
            "EAST", 
            "CENTRAL",
            "WEST"
        ],
    },
    "y": {
        "yᶜˢʳ₁": [ ## "AD_SCHEDULE_OBS"
            "BUSINESS_HOURS_OBS",
            "OVERLAP_HOURS_OBS",
            "AFTER_HOURS_OBS"
        ],
        "yᶜˢʳ₂": [ ## "LOCATION_TARGET_OBS"
            "EAST_OBS",
            "CENTRAL_OBS",
            "WEST_OBS"
        ],
        "yᶜˢʳ₃": [ ## "LANGUAGE_OBS"
            "ENGLISH_OBS",
            "SPANISH_OBS",
            "CHINESE_OBS"
        ],
    },
}
_yCsr_car,_yCsr_num, _sCsr_car,_sCsr_num, _aWeb_car,_aWeb_num = get_model_dimensions_from_labels(_labWeb)
_yCsr_car,_yCsr_num, _sCsr_car,_sCsr_num, _aWeb_car,_aWeb_num

([3, 3, 3], 3, [2, 3], 2, [1, 3], 2)

print(f'{_aWeb_car=}') ## cardinality of control factors
print(f'{_aWeb_num=}') ## number of control factors

print(f'{_sCsr_car=}') ## cardinality of state factors
print(f'{_sCsr_num=}') ## number of state factors

print(f'{_yCsr_car=}') ## cardinality of observation modalities
print(f'{_yCsr_num=}') ## number of observation modalities

_aWeb_car=[1, 3]
_aWeb_num=2
_sCsr_car=[2, 3]
_sCsr_num=2
_yCsr_car=[3, 3, 3]
_yCsr_num=3

_aWeb_fac_names = list(_labWeb['a'].keys()); print(f'{_aWeb_fac_names=}') ## control factor names
_sCsr_fac_names = list(_labWeb['s'].keys()); print(f'{_sCsr_fac_names=}') ## state factor names
_s̆Csr_fac_names = list(_labWeb['s̆'].keys()); print(f'{_s̆Csr_fac_names=}') ## state factor names
_yCsr_mod_names = list(_labWeb['y'].keys()); print(f'{_yCsr_mod_names=}') ## observation modality names

_aWeb_fac_names=['aᵂᵉᵇ₁', 'aᵂᵉᵇ₂']
_sCsr_fac_names=['sᶜˢʳ₁', 'sᶜˢʳ₂']
_s̆Csr_fac_names=['s̆ᶜˢʳ₁', 's̆ᶜˢʳ₂']
_yCsr_mod_names=['yᶜˢʳ₁', 'yᶜˢʳ₂', 'yᶜˢʳ₃']

3.1.4 Observation likelihood matrix, $\mathbf{A}$

We now setup the observation likelihood matrix which is the first main component of generative model.

_Aᵂᵉᵇ = utils.obj_array_zeros([[y_car] + _sCsr_car for y_car in _yCsr_car])
print(f'{len(_Aᵂᵉᵇ)=}')
_Aᵂᵉᵇ

len(_Aᵂᵉᵇ)=3

array([array([[[0., 0., 0.],
               [0., 0., 0.]],

              [[0., 0., 0.],
               [0., 0., 0.]],

              [[0., 0., 0.],
               [0., 0., 0.]]]), array([[[0., 0., 0.],
                                        [0., 0., 0.]],

                                       [[0., 0., 0.],
                                        [0., 0., 0.]],

                                       [[0., 0., 0.],
                                        [0., 0., 0.]]]),
       array([[[0., 0., 0.],
               [0., 0., 0.]],

              [[0., 0., 0.],
               [0., 0., 0.]],

              [[0., 0., 0.],
               [0., 0., 0.]]])], dtype=object)

3.1.4.1 Observation modality $y_1^{\mathrm{Csr}}$ (AD_SCHEDULE_OBS)

We setup this modality’s likelihood mapping, corresponding to how $y_1^{\mathrm{Csr}}$ (AD_SCHEDULE_OBS) is related to hidden states.

_Aᵂᵉᵇ[0][
    _labWeb['y']['yᶜˢʳ₁'].index('AFTER_HOURS_OBS'), 
    :, 
    _labWeb['s']['sᶜˢʳ₂'].index('EAST')
] = 1.0
_Aᵂᵉᵇ[0][
    _labWeb['y']['yᶜˢʳ₁'].index('AFTER_HOURS_OBS'), 
    :, 
    _labWeb['s']['sᶜˢʳ₂'].index('CENTRAL')
] = 1.0
_Aᵂᵉᵇ[0][
    _labWeb['y']['yᶜˢʳ₁'].index('BUSINESS_HOURS_OBS'),
    _labWeb['s']['sᶜˢʳ₁'].index('BUSINESS_HOURS'), 
    _labWeb['s']['sᶜˢʳ₂'].index('WEST')
] = 0.8
_Aᵂᵉᵇ[0][
    _labWeb['y']['yᶜˢʳ₁'].index('OVERLAP_HOURS_OBS'),
    _labWeb['s']['sᶜˢʳ₁'].index('BUSINESS_HOURS'), 
    _labWeb['s']['sᶜˢʳ₂'].index('WEST')
] = 0.2

_Aᵂᵉᵇ[0][
    _labWeb['y']['yᶜˢʳ₁'].index('OVERLAP_HOURS_OBS'),
    _labWeb['s']['sᶜˢʳ₁'].index('AFTER_HOURS'), 
    _labWeb['s']['sᶜˢʳ₂'].index('WEST')
] = 0.8
_Aᵂᵉᵇ[0][
    _labWeb['y']['yᶜˢʳ₁'].index('BUSINESS_HOURS_OBS'),
    _labWeb['s']['sᶜˢʳ₁'].index('AFTER_HOURS'), 
    _labWeb['s']['sᶜˢʳ₂'].index('WEST')
] = 0.2

_Aᵂᵉᵇ[0]

array([[[0. , 0. , 0.8],
        [0. , 0. , 0.2]],

       [[0. , 0. , 0.2],
        [0. , 0. , 0.8]],

       [[1. , 1. , 0. ],
        [1. , 1. , 0. ]]])

3.1.4.2 Observation modality $y_2^{\mathrm{Csr}}$ (LOCATION_TARGET_OBS)

We setup this modality’s likelihood mapping, corresponding to how $y_2^{\mathrm{Csr}}$ (LOCATION_TARGET_OBS) is related to hidden states.

_Aᵂᵉᵇ[1][
    _labWeb['y']['yᶜˢʳ₂'].index('WEST_OBS'), 
    :, 
    _labWeb['s']['sᶜˢʳ₂'].index('EAST')
] = 1.0

_Aᵂᵉᵇ[1][
    _labWeb['y']['yᶜˢʳ₂'].index('WEST_OBS'), 
    :, 
    _labWeb['s']['sᶜˢʳ₂'].index('WEST')
] = 1.0

_BUSINESS_HOURS_MAPPING_WEB = softmax(np.array([1.0, 0]))
_AFTER_HOURS_MAPPING_WEB = softmax(np.array([0.0, 1.0]))

_Aᵂᵉᵇ[1][
    _labWeb['y']['yᶜˢʳ₂'].index('EAST_OBS'),
    _labWeb['s']['sᶜˢʳ₁'].index('BUSINESS_HOURS'), 
    _labWeb['s']['sᶜˢʳ₂'].index('CENTRAL')
] = _BUSINESS_HOURS_MAPPING_WEB[0]
_Aᵂᵉᵇ[1][
    _labWeb['y']['yᶜˢʳ₂'].index('CENTRAL_OBS'),
    _labWeb['s']['sᶜˢʳ₁'].index('BUSINESS_HOURS'), 
    _labWeb['s']['sᶜˢʳ₂'].index('CENTRAL')
] = _BUSINESS_HOURS_MAPPING_WEB[1]

_Aᵂᵉᵇ[1][
    _labWeb['y']['yᶜˢʳ₂'].index('EAST_OBS'),
    _labWeb['s']['sᶜˢʳ₁'].index('AFTER_HOURS'), 
    _labWeb['s']['sᶜˢʳ₂'].index('CENTRAL')
] = _AFTER_HOURS_MAPPING_WEB[0]
_Aᵂᵉᵇ[1][
    _labWeb['y']['yᶜˢʳ₂'].index('CENTRAL_OBS'),
    _labWeb['s']['sᶜˢʳ₁'].index('AFTER_HOURS'), 
    _labWeb['s']['sᶜˢʳ₂'].index('CENTRAL')
] = _AFTER_HOURS_MAPPING_WEB[1]

_Aᵂᵉᵇ[1]

array([[[0.        , 0.73105858, 0.        ],
        [0.        , 0.26894142, 0.        ]],

       [[0.        , 0.26894142, 0.        ],
        [0.        , 0.73105858, 0.        ]],

       [[1.        , 0.        , 1.        ],
        [1.        , 0.        , 1.        ]]])

3.1.4.3 Observation modality $y_3^{\mathrm{Csr}}$ (LANGUAGE_OBS)

We setup this modality’s likelihood mapping, corresponding to how $y_3^{\mathrm{Csr}}$ (LANGUAGE_OBS) is related to hidden states.

_Aᵂᵉᵇ[2][
    _labWeb['y']['yᶜˢʳ₃'].index('ENGLISH_OBS'), 
    :, 
    _labWeb['s']['sᶜˢʳ₂'].index('EAST')
] = 1.0
_Aᵂᵉᵇ[2][
    _labWeb['y']['yᶜˢʳ₃'].index('SPANISH_OBS'), 
    :, 
    _labWeb['s']['sᶜˢʳ₂'].index('CENTRAL')
] = 1.0
_Aᵂᵉᵇ[2][
    _labWeb['y']['yᶜˢʳ₃'].index('CHINESE_OBS'), 
    :, 
    _labWeb['s']['sᶜˢʳ₂'].index('WEST')
] = 1.0

_Aᵂᵉᵇ[2]

array([[[1., 0., 0.],
        [1., 0., 0.]],

       [[0., 1., 0.],
        [0., 1., 0.]],

       [[0., 0., 1.],
        [0., 0., 1.]]])

print(f'=== _sCsr_car:\n{_sCsr_car}')
print(f'=== _yCsr_car:\n{_yCsr_car}')
_Aᵂᵉᵇ

=== _sCsr_car:
[2, 3]
=== _yCsr_car:
[3, 3, 3]

array([array([[[0. , 0. , 0.8],
               [0. , 0. , 0.2]],

              [[0. , 0. , 0.2],
               [0. , 0. , 0.8]],

              [[1. , 1. , 0. ],
               [1. , 1. , 0. ]]]),
       array([[[0.        , 0.73105858, 0.        ],
               [0.        , 0.26894142, 0.        ]],

              [[0.        , 0.26894142, 0.        ],
               [0.        , 0.73105858, 0.        ]],

              [[1.        , 0.        , 1.        ],
               [1.        , 0.        , 1.        ]]]),
       array([[[1., 0., 0.],
               [1., 0., 0.]],

              [[0., 1., 0.],
               [0., 1., 0.]],

              [[0., 0., 1.],
               [0., 0., 1.]]])], dtype=object)

3.1.5 State transition matrix, $\mathbf{B}$

_Bᵂᵉᵇ = utils.obj_array(_sCsr_num); print(f'{_sCsr_num=}')
print(f'{len(_Bᵂᵉᵇ)=}')
_Bᵂᵉᵇ

_sCsr_num=2
len(_Bᵂᵉᵇ)=2

array([None, None], dtype=object)

3.1.5.1 Control factor $a_1^{\mathrm{Web}}$ (NULL)

_Bᵂᵉᵇ[0] = np.zeros((_sCsr_car[0], _sCsr_car[0], _aWeb_car[0])); print(f'{_sCsr_car[0]=}, {_sCsr_car[0]=}, {_aWeb_car[0]=}')
_Bᵂᵉᵇ[0]

_sCsr_car[0]=2, _sCsr_car[0]=2, _aWeb_car[0]=1

array([[[0.],
        [0.]],

       [[0.],
        [0.]]])

_p_stochWeb = 0.0

## we cannot influence factor zero, set up the 'default' stationary dynamics - 
## one state just maps to itself at the next timestep with very high probability, 
## by default. So this means the AD_SCHEDULE state can change from one to another with 
## some low probability (p_stoch)

_Bᵂᵉᵇ[0][
    _labWeb['s']['sᶜˢʳ₁'].index('BUSINESS_HOURS'),
    _labWeb['s']['sᶜˢʳ₁'].index('BUSINESS_HOURS'), 
    _labWeb['a']['aᵂᵉᵇ₁'].index('NULL_ACT')
] = 1.0 - _p_stochWeb

_Bᵂᵉᵇ[0][
    _labWeb['s']['sᶜˢʳ₁'].index('AFTER_HOURS'),
    _labWeb['s']['sᶜˢʳ₁'].index('BUSINESS_HOURS'), 
    _labWeb['a']['aᵂᵉᵇ₁'].index('NULL_ACT')
] = _p_stochWeb

_Bᵂᵉᵇ[0][
    _labWeb['s']['sᶜˢʳ₁'].index('AFTER_HOURS'),
    _labWeb['s']['sᶜˢʳ₁'].index('AFTER_HOURS'), 
    _labWeb['a']['aᵂᵉᵇ₁'].index('NULL_ACT')
] = 1.0 - _p_stochWeb

_Bᵂᵉᵇ[0][
    _labWeb['s']['sᶜˢʳ₁'].index('BUSINESS_HOURS'),
    _labWeb['s']['sᶜˢʳ₁'].index('AFTER_HOURS'), 
    _labWeb['a']['aᵂᵉᵇ₁'].index('NULL_ACT')
] = _p_stochWeb

_Bᵂᵉᵇ[0]

array([[[1.],
        [0.]],

       [[0.],
        [1.]]])

3.1.5.2 Control factor $a_2^{\mathrm{Web}}$ (SET_DEVICE_TARGET_ACTION)

We setup this factor’s mapping, corresponding to how $a_2^{\mathrm{Web}}$ (SET_DEVICE_TARGET_ACTION) is related to hidden states.

_Bᵂᵉᵇ[1] = np.zeros((_sCsr_car[1], _sCsr_car[1], _aWeb_car[1]))

_Bᵂᵉᵇ[1][
    _labWeb['s']['sᶜˢʳ₂'].index('EAST'), 
    :, 
    _labWeb['a']['aᵂᵉᵇ₂'].index('MOBILE_ACT')
] = 1.0
_Bᵂᵉᵇ[1][
    _labWeb['s']['sᶜˢʳ₂'].index('CENTRAL'), 
    :, 
    _labWeb['a']['aᵂᵉᵇ₂'].index('DESKTOP_ACT')
] = 1.0
_Bᵂᵉᵇ[1][
    _labWeb['s']['sᶜˢʳ₂'].index('WEST'), 
    :, 
    _labWeb['a']['aᵂᵉᵇ₂'].index('TABLET_ACT')
] = 1.0

_Bᵂᵉᵇ[1]

array([[[1., 0., 0.],
        [1., 0., 0.],
        [1., 0., 0.]],

       [[0., 1., 0.],
        [0., 1., 0.],
        [0., 1., 0.]],

       [[0., 0., 1.],
        [0., 0., 1.],
        [0., 0., 1.]]])

print(f'=== _aWeb_car:\n{_aWeb_car}')
print(f'=== _sCsr_car:\n{_sCsr_car}')
_Bᵂᵉᵇ

=== _aWeb_car:
[1, 3]
=== _sCsr_car:
[2, 3]

array([array([[[1.],
               [0.]],

              [[0.],
               [1.]]]), array([[[1., 0., 0.],
                                [1., 0., 0.],
                                [1., 0., 0.]],

                               [[0., 1., 0.],
                                [0., 1., 0.],
                                [0., 1., 0.]],

                               [[0., 0., 1.],
                                [0., 0., 1.],
                                [0., 0., 1.]]])], dtype=object)

3.1.6 Prior preferences vector, $\mathbf{C}$

Now we parameterise the C vector, or the prior beliefs about observations. This will be used in the expression of the prior over actions, which is technically a softmax function of the negative expected free energy of each action. It is the equivalent of the exponentiated reward function in reinforcement learning treatments.

_Cᵂᵉᵇ = utils.obj_array_zeros([y_car for y_car in _yCsr_car])
_Cᵂᵉᵇ

array([array([0., 0., 0.]), array([0., 0., 0.]), array([0., 0., 0.])],
      dtype=object)

_Cᵂᵉᵇ[1][
    _labWeb['y']['yᶜˢʳ₂'].index('EAST_OBS'),
] = 1.0
_Cᵂᵉᵇ[1][
    _labWeb['y']['yᶜˢʳ₂'].index('CENTRAL_OBS'),
] = -1.0
_Cᵂᵉᵇ[1][
    _labWeb['y']['yᶜˢʳ₂'].index('WEST_OBS'),
] = 0.0

_Cᵂᵉᵇ[1]

array([ 1., -1.,  0.])

3.1.7 Initialise an instance of the Agent() class:

All you have to do is call Agent(generative_model_params...) where generative_model_params are your A, B, C’s… and whatever parameters of the generative model you want to specify

_agtWeb = Agent(
    A=_Aᵂᵉᵇ, 
    B=_Bᵂᵉᵇ, 
    C=_Cᵂᵉᵇ, 
)
_agtWeb

<pymdp.agent.Agent at 0x7f513842cfd0>

3.2 Website environment

This is the Website agent’s generative process for its environment.

It is important to note that the generative process doesn’t have to be described by A and B matrices - it can just be the arbitrary ‘rules of the game’ that you ‘write in’ as a modeller. But here we just use the same transition/likelihood matrices to make the sampling process straightforward.

## observation/transition matrices characterising the generative process
_Ăᶜˢʳ = copy.deepcopy(_Aᵂᵉᵇ)
_B̆ᶜˢʳ = copy.deepcopy(_Bᵂᵉᵇ)

4 Run simulation

_T = 25 ## timesteps in the simulation

## Administrator
_s̆ᵂᵉᵇ = [ ## initial (true) state
    _labAdm['s̆']['s̆ᵂᵉᵇ₁'].index('EXPANDED_TEXT_ADS'), 
    _labAdm['s̆']['s̆ᵂᵉᵇ₂'].index('DESCRIPTION_LINES')
]; print(f'{_s̆ᵂᵉᵇ=}')

_yᵂᵉᵇ = [ ## initial observation
    _labAdm['y']['yᵂᵉᵇ₁'].index('UNKNOWN_OBS'), 
    _labAdm['y']['yᵂᵉᵇ₂'].index('CALL_TO_ACTION_OBS'),
    _labAdm['y']['yᵂᵉᵇ₃'].index('SITE_LINKS_OBS')
]; print(f'{_yᵂᵉᵇ=}')

_s̆ᵂᵉᵇ=[0, 0]
_yᵂᵉᵇ=[2, 2, 0]

## Website
_s̆ᶜˢʳ = [ ## initial (true) state
    _labWeb['s̆']['s̆ᶜˢʳ₁'].index('BUSINESS_HOURS'),
    _labWeb['s̆']['s̆ᶜˢʳ₂'].index('EAST')
]; print(f'{_s̆ᶜˢʳ=}')

_yᶜˢʳ = [ ## initial observation
    _labWeb['y']['yᶜˢʳ₁'].index('BUSINESS_HOURS_OBS'),
    _labWeb['y']['yᶜˢʳ₂'].index('EAST_OBS'),
    _labWeb['y']['yᶜˢʳ₃'].index('ENGLISH_OBS')
]; print(f'{_yᶜˢʳ=}')

_s̆ᶜˢʳ=[0, 0]
_yᶜˢʳ=[0, 0, 0]

Create some string names for the state, observation, and action indices to help with print statements

## Administrator
_aAdm_val_names = [_labAdm['a'][cfn] for cfn in _aAdm_fac_names]; print(f'{_aAdm_val_names=}')
_sWeb_val_names = [_labAdm['s'][sfn] for sfn in _sWeb_fac_names]; print(f'{_sWeb_val_names=}')
_s̆Web_val_names = [_labAdm['s̆'][sfn] for sfn in _s̆Web_fac_names]; print(f'{_s̆Web_val_names=}')
_yWeb_val_names = [_labAdm['y'][omn] for omn in _yWeb_mod_names]; print(f'{_yWeb_val_names=}')

_aAdm_val_names=[['NULL_ACT'], ['BROAD_MATCH_ACT', 'PHRASE_MATCH_ACT', 'EXACT_MATCH_ACT']]
_sWeb_val_names=[['EXPANDED_TEXT_ADS', 'RESPONSIVE_SEARCH_ADS'], ['DESCRIPTION_LINES', 'DISPLAY_URL', 'CALL_TO_ACTION']]
_s̆Web_val_names=[['EXPANDED_TEXT_ADS', 'RESPONSIVE_SEARCH_ADS'], ['DESCRIPTION_LINES', 'DISPLAY_URL', 'CALL_TO_ACTION']]
_yWeb_val_names=[['EXPANDED_TEXT_ADS_OBS', 'RESPONSIVE_SEARCH_ADS_OBS', 'UNKNOWN_OBS'], ['DESCRIPTION_LINES_OBS', 'DISPLAY_URL_OBS', 'CALL_TO_ACTION_OBS'], ['SITE_LINKS_OBS', 'CALLOUTS_OBS', 'STRUCTURED_SNIPPETS_OBS']]

## Website
_aWeb_val_names = [_labWeb['a'][cfn] for cfn in _aWeb_fac_names]; print(f'{_aWeb_val_names=}')
_sCsr_val_names = [_labWeb['s'][sfn] for sfn in _sCsr_fac_names]; print(f'{_sCsr_val_names=}')
_s̆Csr_val_names = [_labWeb['s̆'][sfn] for sfn in _s̆Csr_fac_names]; print(f'{_s̆Csr_val_names=}')
_yCsr_val_names = [_labWeb['y'][omn] for omn in _yCsr_mod_names]; print(f'{_yCsr_val_names=}')

_aWeb_val_names=[['NULL_ACT'], ['MOBILE_ACT', 'DESKTOP_ACT', 'TABLET_ACT']]
_sCsr_val_names=[['BUSINESS_HOURS', 'AFTER_HOURS'], ['EAST', 'CENTRAL', 'WEST']]
_s̆Csr_val_names=[['BUSINESS_HOURS', 'AFTER_HOURS'], ['EAST', 'CENTRAL', 'WEST']]
_yCsr_val_names=[['BUSINESS_HOURS_OBS', 'OVERLAP_HOURS_OBS', 'AFTER_HOURS_OBS'], ['EAST_OBS', 'CENTRAL_OBS', 'WEST_OBS'], ['ENGLISH_OBS', 'SPANISH_OBS', 'CHINESE_OBS']]

def act(agt, a_facs, a_fac_names, a_val_names, s̆_fac_names, t):
    if(t == 0): ## at t=0 agent has no q_pi yet, so no .sample_action()
        action = np.array([0.0, 0.0]) ##.
        print(f"_a: {[(a_fac_names[a], a_val_names[a][int(action[a])]) for a in range(len(s̆_fac_names))]}")
    else: ## t > 0
        action = agt.sample_action()
        ## min_F.append(np.min(_agent.F)) ## does not have .F
        print(f"_a: {[(a_fac_names[a], a_val_names[a][int(action[a])]) for a in range(len(s̆_fac_names))]}") ##.
        for afi, afn in enumerate(a_fac_names):
            a_facs[afn].append(a_val_names[afi][int(action[afi])])
    return action

def future(agt, qIpiIs, GNegs):
    ## _agent.infer_policies()
    qIpiI, GNeg = agt.infer_policies() ## posterior over policies and negative EFE
    print(f'{qIpiI=}')
    print(f'{GNeg=}')
    qIpiIs.append(qIpiI)
    GNegs.append(GNeg)

def next(s̆_facs, action, s̆, B̆, s̆_fac_names, s̆_val_names):
    for sfi, sf in enumerate(s̆):
        s̆[sfi] = utils.sample(B̆[sfi][:, sf, int(action[sfi])])
    print(f"_s̆: {[(s̆_fac_names[sfi], s̆_val_names[sfi][s̆[sfi]]) for sfi in range(len(s̆_fac_names))]}") ##.
    for sfi, sfn in enumerate(s̆_fac_names):
        s̆_facs[sfn].append(s̆_val_names[sfi][s̆[sfi]])

## def nextWithoutB(s̆_facs, s̆, s̆_fac_names, s̆_val_names, yCfg_mods):
#     ## for sfi, sf in enumerate(s̆):
#     ##     s̆[sfi] = utils.sample(B̆[sfi][:, sf, int(action[sfi])])
#     print(f'!!! BEFORE: {s̆=}')
#     if len(yCfg_mods['yᶜᶠᵍ₁']) > 0:
#         yCfg = yCfg_mods['yᶜᶠᵍ₁'][-1]
#         print(f"!!! {yCfg=}")
#         if yCfg == "REW_INVERSION_EVIDENCE":
#             ## flip value of s̆ᴳᵃᵐ₁
#             s̆_0_new = (s̆[0] + 1) % 2
#             # s̆_1_new = (s̆[1] + 1) % 2
#             s̆_1_new = s̆[1]
#             s̆ = [s̆_0_new, s̆_1_new]
#         print(f'!!! AFTER: {s̆=}')
#     print(f"_s̆: {[(s̆_fac_names[sfi], s̆_val_names[sfi][s̆[sfi]]) for sfi in range(len(s̆_fac_names))]}") ##.
#     for sfi, sfn in enumerate(s̆_fac_names):
#         s̆_facs[sfn].append(s̆_val_names[sfi][s̆[sfi]])

def observe(y_mods, y, Ă, s̆, _y_mod_names, _y_val_names):
    for omi, _ in enumerate(y):
        if len(s̆) == 1:
            y[omi] = utils.sample(Ă[omi][:, s̆[0]])
        elif len(s̆) == 2:
            y[omi] = utils.sample(Ă[omi][:, s̆[0], s̆[1]])
        else:
            print(f'ERROR: {len(s̆)=} not handled!')
    print(f"_y: {[(_y_mod_names[omi], _y_val_names[omi][y[omi]]) for omi in range(len(_y_mod_names))]}") ##.
    for ymi, ymn in enumerate(_y_mod_names):
        y_mods[ymn].append(_y_val_names[ymi][y[ymi]])

def infer(agt, s_facs, y, s_fac_names, lab):
    belief_state = agt.infer_states(y)
    print(f"Beliefs: {[(s_fac_names[sfi], belief_state[sfi].round(3).T) for sfi in range(len(s_fac_names))]}") ##.
    for sfi, sfn in enumerate(s_fac_names):
        s_facs[sfn].append( lab['s'][sfn][int(np.argmax(belief_state[sfi].round(3).T))] )

_aAdm_facs = {'aᴬᵈᵐ₁': [], 'aᴬᵈᵐ₂': []}
_sAdm_facs = {'sᵂᵉᵇ₁': [], 'sᵂᵉᵇ₂': []}
_s̆Web_facs = {'s̆ᵂᵉᵇ₁': [], 's̆ᵂᵉᵇ₂': []}
_yWeb_mods = {'yᵂᵉᵇ₁': [], 'yᵂᵉᵇ₂': [], 'yᵂᵉᵇ₃': []}

_aWeb_facs = {'aᵂᵉᵇ₁': [], 'aᵂᵉᵇ₂': []}
_sWeb_facs = {'sᶜˢʳ₁': [], 'sᶜˢʳ₂': []}
_s̆Csr_facs = {'s̆ᶜˢʳ₁': [], 's̆ᶜˢʳ₂': []}
_yCsr_mods = {'yᶜˢʳ₁': [], 'yᶜˢʳ₂': [], 'yᶜˢʳ₃': []}
## min_F = []
_qAdmIpiIs = []
_GAdmNegs = []

_qWebIpiIs = []
_GWebNegs = []

for t in range(_T):
    print(f"\nTime {t}:")

    ### act
    print('___act___')
    actionAdm = act(_agtAdm, _aAdm_facs, _aAdm_fac_names, _aAdm_val_names, _s̆Web_fac_names, t)
    actionWeb = act(_agtWeb, _aWeb_facs, _aWeb_fac_names, _aWeb_val_names, _s̆Csr_fac_names, t)

    ### future
    print('___future___')
    future(_agtAdm, _qAdmIpiIs, _GAdmNegs)
    future(_agtWeb, _qWebIpiIs, _GWebNegs)

    ### next
    print('___next___')
    next(_s̆Web_facs, actionAdm, _s̆ᵂᵉᵇ, _B̆ᵂᵉᵇ, _s̆Web_fac_names, _s̆Web_val_names)
    ## nextWithoutB(_s̆Gam_facs, _s̆ᴳᵃᵐ, _s̆Gam_fac_names, _s̆Gam_val_names, _yCfg_mods)
    next(_s̆Csr_facs, actionWeb, _s̆ᶜˢʳ, _B̆ᶜˢʳ, _s̆Csr_fac_names, _s̆Csr_val_names)

    ### observe
    print('___observe___')
    observe(_yWeb_mods, _yᵂᵉᵇ, _Ăᵂᵉᵇ, _s̆ᵂᵉᵇ, _yWeb_mod_names, _yWeb_val_names)
    observe(_yCsr_mods, _yᶜˢʳ, _Ăᶜˢʳ, _s̆ᶜˢʳ, _yCsr_mod_names, _yCsr_val_names)

    ### infer
    print('___infer___')
    infer(_agtAdm, _sAdm_facs, _yᵂᵉᵇ, _sWeb_fac_names, _labAdm)
    infer(_agtWeb, _sWeb_facs, _yᶜˢʳ, _sCsr_fac_names, _labWeb)

Time 0:
___act___
_a: [('aᴬᵈᵐ₁', 'NULL_ACT'), ('aᴬᵈᵐ₂', 'BROAD_MATCH_ACT')]
_a: [('aᵂᵉᵇ₁', 'NULL_ACT'), ('aᵂᵉᵇ₂', 'MOBILE_ACT')]
___future___
qIpiI=array([0.03478899, 0.20528675, 0.75992426])
GNeg=array([-3.60483043, -3.49388625, -3.41208556])
qIpiI=array([0.03478899, 0.20528675, 0.75992426])
GNeg=array([-3.60483043, -3.49388625, -3.41208556])
___next___
_s̆: [('s̆ᵂᵉᵇ₁', 'EXPANDED_TEXT_ADS'), ('s̆ᵂᵉᵇ₂', 'DESCRIPTION_LINES')]
_s̆: [('s̆ᶜˢʳ₁', 'BUSINESS_HOURS'), ('s̆ᶜˢʳ₂', 'EAST')]
___observe___
_y: [('yᵂᵉᵇ₁', 'UNKNOWN_OBS'), ('yᵂᵉᵇ₂', 'CALL_TO_ACTION_OBS'), ('yᵂᵉᵇ₃', 'SITE_LINKS_OBS')]
_y: [('yᶜˢʳ₁', 'AFTER_HOURS_OBS'), ('yᶜˢʳ₂', 'WEST_OBS'), ('yᶜˢʳ₃', 'ENGLISH_OBS')]
___infer___
Beliefs: [('sᵂᵉᵇ₁', array([0.5, 0.5])), ('sᵂᵉᵇ₂', array([1., 0., 0.]))]
Beliefs: [('sᶜˢʳ₁', array([0.5, 0.5])), ('sᶜˢʳ₂', array([1., 0., 0.]))]

Time 1:
___act___
_a: [('aᴬᵈᵐ₁', 'NULL_ACT'), ('aᴬᵈᵐ₂', 'EXACT_MATCH_ACT')]
_a: [('aᵂᵉᵇ₁', 'NULL_ACT'), ('aᵂᵉᵇ₂', 'TABLET_ACT')]
___future___
qIpiI=array([0.03478899, 0.20528675, 0.75992426])
GNeg=array([-3.60483043, -3.49388625, -3.41208556])
qIpiI=array([0.03478899, 0.20528675, 0.75992426])
GNeg=array([-3.60483043, -3.49388625, -3.41208556])
___next___
_s̆: [('s̆ᵂᵉᵇ₁', 'EXPANDED_TEXT_ADS'), ('s̆ᵂᵉᵇ₂', 'CALL_TO_ACTION')]
_s̆: [('s̆ᶜˢʳ₁', 'BUSINESS_HOURS'), ('s̆ᶜˢʳ₂', 'WEST')]
___observe___
_y: [('yᵂᵉᵇ₁', 'EXPANDED_TEXT_ADS_OBS'), ('yᵂᵉᵇ₂', 'CALL_TO_ACTION_OBS'), ('yᵂᵉᵇ₃', 'STRUCTURED_SNIPPETS_OBS')]
_y: [('yᶜˢʳ₁', 'BUSINESS_HOURS_OBS'), ('yᶜˢʳ₂', 'WEST_OBS'), ('yᶜˢʳ₃', 'CHINESE_OBS')]
___infer___
Beliefs: [('sᵂᵉᵇ₁', array([0.8, 0.2])), ('sᵂᵉᵇ₂', array([0., 0., 1.]))]
Beliefs: [('sᶜˢʳ₁', array([0.8, 0.2])), ('sᶜˢʳ₂', array([0., 0., 1.]))]

Time 2:
___act___
_a: [('aᴬᵈᵐ₁', 'NULL_ACT'), ('aᴬᵈᵐ₂', 'EXACT_MATCH_ACT')]
_a: [('aᵂᵉᵇ₁', 'NULL_ACT'), ('aᵂᵉᵇ₂', 'TABLET_ACT')]
___future___
qIpiI=array([0.00362533, 0.96895062, 0.02742404])
GNeg=array([-3.60483043, -3.25556369, -3.47836328])
qIpiI=array([0.00362533, 0.96895062, 0.02742404])
GNeg=array([-3.60483043, -3.25556369, -3.47836328])
___next___
_s̆: [('s̆ᵂᵉᵇ₁', 'EXPANDED_TEXT_ADS'), ('s̆ᵂᵉᵇ₂', 'CALL_TO_ACTION')]
_s̆: [('s̆ᶜˢʳ₁', 'BUSINESS_HOURS'), ('s̆ᶜˢʳ₂', 'WEST')]
___observe___
_y: [('yᵂᵉᵇ₁', 'EXPANDED_TEXT_ADS_OBS'), ('yᵂᵉᵇ₂', 'CALL_TO_ACTION_OBS'), ('yᵂᵉᵇ₃', 'STRUCTURED_SNIPPETS_OBS')]
_y: [('yᶜˢʳ₁', 'BUSINESS_HOURS_OBS'), ('yᶜˢʳ₂', 'WEST_OBS'), ('yᶜˢʳ₃', 'CHINESE_OBS')]
___infer___
Beliefs: [('sᵂᵉᵇ₁', array([0.941, 0.059])), ('sᵂᵉᵇ₂', array([0., 0., 1.]))]
Beliefs: [('sᶜˢʳ₁', array([0.941, 0.059])), ('sᶜˢʳ₂', array([0., 0., 1.]))]

Time 3:
___act___
_a: [('aᴬᵈᵐ₁', 'NULL_ACT'), ('aᴬᵈᵐ₂', 'PHRASE_MATCH_ACT')]
_a: [('aᵂᵉᵇ₁', 'NULL_ACT'), ('aᵂᵉᵇ₂', 'DESKTOP_ACT')]
___future___
qIpiI=array([9.75588752e-04, 9.97013954e-01, 2.01045701e-03])
GNeg=array([-3.60483043, -3.171738  , -3.55963817])
qIpiI=array([9.75588752e-04, 9.97013954e-01, 2.01045701e-03])
GNeg=array([-3.60483043, -3.171738  , -3.55963817])
___next___
_s̆: [('s̆ᵂᵉᵇ₁', 'EXPANDED_TEXT_ADS'), ('s̆ᵂᵉᵇ₂', 'DISPLAY_URL')]
_s̆: [('s̆ᶜˢʳ₁', 'BUSINESS_HOURS'), ('s̆ᶜˢʳ₂', 'CENTRAL')]
___observe___
_y: [('yᵂᵉᵇ₁', 'UNKNOWN_OBS'), ('yᵂᵉᵇ₂', 'DISPLAY_URL_OBS'), ('yᵂᵉᵇ₃', 'CALLOUTS_OBS')]
_y: [('yᶜˢʳ₁', 'AFTER_HOURS_OBS'), ('yᶜˢʳ₂', 'EAST_OBS'), ('yᶜˢʳ₃', 'SPANISH_OBS')]
___infer___
Beliefs: [('sᵂᵉᵇ₁', array([0.855, 0.145])), ('sᵂᵉᵇ₂', array([0., 1., 0.]))]
Beliefs: [('sᶜˢʳ₁', array([0.978, 0.022])), ('sᶜˢʳ₂', array([0., 1., 0.]))]

Time 4:
___act___
_a: [('aᴬᵈᵐ₁', 'NULL_ACT'), ('aᴬᵈᵐ₂', 'PHRASE_MATCH_ACT')]
_a: [('aᵂᵉᵇ₁', 'NULL_ACT'), ('aᵂᵉᵇ₂', 'DESKTOP_ACT')]
___future___
qIpiI=array([0.00211696, 0.98753736, 0.01034568])
GNeg=array([-3.60483043, -3.22075331, -3.50566866])
qIpiI=array([7.28121866e-04, 9.98298628e-01, 9.73250403e-04])
GNeg=array([-3.60483043, -3.15337172, -3.58669462])
___next___
_s̆: [('s̆ᵂᵉᵇ₁', 'EXPANDED_TEXT_ADS'), ('s̆ᵂᵉᵇ₂', 'DISPLAY_URL')]
_s̆: [('s̆ᶜˢʳ₁', 'BUSINESS_HOURS'), ('s̆ᶜˢʳ₂', 'CENTRAL')]
___observe___
_y: [('yᵂᵉᵇ₁', 'UNKNOWN_OBS'), ('yᵂᵉᵇ₂', 'DISPLAY_URL_OBS'), ('yᵂᵉᵇ₃', 'CALLOUTS_OBS')]
_y: [('yᶜˢʳ₁', 'AFTER_HOURS_OBS'), ('yᶜˢʳ₂', 'EAST_OBS'), ('yᶜˢʳ₃', 'SPANISH_OBS')]
___infer___
Beliefs: [('sᵂᵉᵇ₁', array([0.684, 0.316])), ('sᵂᵉᵇ₂', array([0., 1., 0.]))]
Beliefs: [('sᶜˢʳ₁', array([0.992, 0.008])), ('sᶜˢʳ₂', array([0., 1., 0.]))]

Time 5:
___act___
_a: [('aᴬᵈᵐ₁', 'NULL_ACT'), ('aᴬᵈᵐ₂', 'PHRASE_MATCH_ACT')]
_a: [('aᵂᵉᵇ₁', 'NULL_ACT'), ('aᵂᵉᵇ₂', 'DESKTOP_ACT')]
___future___
qIpiI=array([0.01151154, 0.81885042, 0.16963804])
GNeg=array([-3.60483043, -3.33829599, -3.43668564])
qIpiI=array([6.53848020e-04, 9.98616007e-01, 7.30144782e-04])
GNeg=array([-3.60483043, -3.14662726, -3.59793244])
___next___
_s̆: [('s̆ᵂᵉᵇ₁', 'EXPANDED_TEXT_ADS'), ('s̆ᵂᵉᵇ₂', 'DISPLAY_URL')]
_s̆: [('s̆ᶜˢʳ₁', 'BUSINESS_HOURS'), ('s̆ᶜˢʳ₂', 'CENTRAL')]
___observe___
_y: [('yᵂᵉᵇ₁', 'UNKNOWN_OBS'), ('yᵂᵉᵇ₂', 'DESCRIPTION_LINES_OBS'), ('yᵂᵉᵇ₃', 'CALLOUTS_OBS')]
_y: [('yᶜˢʳ₁', 'AFTER_HOURS_OBS'), ('yᶜˢʳ₂', 'EAST_OBS'), ('yᶜˢʳ₃', 'SPANISH_OBS')]
___infer___
Beliefs: [('sᵂᵉᵇ₁', array([0.855, 0.145])), ('sᵂᵉᵇ₂', array([0., 1., 0.]))]
Beliefs: [('sᶜˢʳ₁', array([0.997, 0.003])), ('sᶜˢʳ₂', array([0., 1., 0.]))]

Time 6:
___act___
_a: [('aᴬᵈᵐ₁', 'NULL_ACT'), ('aᴬᵈᵐ₂', 'PHRASE_MATCH_ACT')]
_a: [('aᵂᵉᵇ₁', 'NULL_ACT'), ('aᵂᵉᵇ₂', 'DESKTOP_ACT')]
___future___
qIpiI=array([0.00211696, 0.98753736, 0.01034568])
GNeg=array([-3.60483043, -3.22075331, -3.50566866])
qIpiI=array([6.28521104e-04, 9.98716579e-01, 6.54899545e-04])
GNeg=array([-3.60483043, -3.14415189, -3.60226091])
___next___
_s̆: [('s̆ᵂᵉᵇ₁', 'EXPANDED_TEXT_ADS'), ('s̆ᵂᵉᵇ₂', 'DISPLAY_URL')]
_s̆: [('s̆ᶜˢʳ₁', 'BUSINESS_HOURS'), ('s̆ᶜˢʳ₂', 'CENTRAL')]
___observe___
_y: [('yᵂᵉᵇ₁', 'UNKNOWN_OBS'), ('yᵂᵉᵇ₂', 'DESCRIPTION_LINES_OBS'), ('yᵂᵉᵇ₃', 'CALLOUTS_OBS')]
_y: [('yᶜˢʳ₁', 'AFTER_HOURS_OBS'), ('yᶜˢʳ₂', 'EAST_OBS'), ('yᶜˢʳ₃', 'SPANISH_OBS')]
___infer___
Beliefs: [('sᵂᵉᵇ₁', array([0.941, 0.059])), ('sᵂᵉᵇ₂', array([0., 1., 0.]))]
Beliefs: [('sᶜˢʳ₁', array([0.999, 0.001])), ('sᶜˢʳ₂', array([0., 1., 0.]))]

Time 7:
___act___
_a: [('aᴬᵈᵐ₁', 'NULL_ACT'), ('aᴬᵈᵐ₂', 'PHRASE_MATCH_ACT')]
_a: [('aᵂᵉᵇ₁', 'NULL_ACT'), ('aᵂᵉᵇ₂', 'DESKTOP_ACT')]
___future___
qIpiI=array([9.75588752e-04, 9.97013954e-01, 2.01045701e-03])
GNeg=array([-3.60483043, -3.171738  , -3.55963817])
qIpiI=array([6.19461593e-04, 9.98751592e-01, 6.28946433e-04])
GNeg=array([-3.60483043, -3.14324226, -3.60388072])
___next___
_s̆: [('s̆ᵂᵉᵇ₁', 'EXPANDED_TEXT_ADS'), ('s̆ᵂᵉᵇ₂', 'DISPLAY_URL')]
_s̆: [('s̆ᶜˢʳ₁', 'BUSINESS_HOURS'), ('s̆ᶜˢʳ₂', 'CENTRAL')]
___observe___
_y: [('yᵂᵉᵇ₁', 'UNKNOWN_OBS'), ('yᵂᵉᵇ₂', 'DESCRIPTION_LINES_OBS'), ('yᵂᵉᵇ₃', 'CALLOUTS_OBS')]
_y: [('yᶜˢʳ₁', 'AFTER_HOURS_OBS'), ('yᶜˢʳ₂', 'CENTRAL_OBS'), ('yᶜˢʳ₃', 'SPANISH_OBS')]
___infer___
Beliefs: [('sᵂᵉᵇ₁', array([0.978, 0.022])), ('sᵂᵉᵇ₂', array([0., 1., 0.]))]
Beliefs: [('sᶜˢʳ₁', array([0.997, 0.003])), ('sᶜˢʳ₂', array([0., 1., 0.]))]

Time 8:
___act___
_a: [('aᴬᵈᵐ₁', 'NULL_ACT'), ('aᴬᵈᵐ₂', 'PHRASE_MATCH_ACT')]
_a: [('aᵂᵉᵇ₁', 'NULL_ACT'), ('aᵂᵉᵇ₂', 'DESKTOP_ACT')]
___future___
qIpiI=array([7.28121866e-04, 9.98298628e-01, 9.73250403e-04])
GNeg=array([-3.60483043, -3.15337172, -3.58669462])
qIpiI=array([6.28521104e-04, 9.98716579e-01, 6.54899545e-04])
GNeg=array([-3.60483043, -3.14415189, -3.60226091])
___next___
_s̆: [('s̆ᵂᵉᵇ₁', 'EXPANDED_TEXT_ADS'), ('s̆ᵂᵉᵇ₂', 'DISPLAY_URL')]
_s̆: [('s̆ᶜˢʳ₁', 'BUSINESS_HOURS'), ('s̆ᶜˢʳ₂', 'CENTRAL')]
___observe___
_y: [('yᵂᵉᵇ₁', 'UNKNOWN_OBS'), ('yᵂᵉᵇ₂', 'DISPLAY_URL_OBS'), ('yᵂᵉᵇ₃', 'CALLOUTS_OBS')]
_y: [('yᶜˢʳ₁', 'AFTER_HOURS_OBS'), ('yᶜˢʳ₂', 'EAST_OBS'), ('yᶜˢʳ₃', 'SPANISH_OBS')]
___infer___
Beliefs: [('sᵂᵉᵇ₁', array([0.941, 0.059])), ('sᵂᵉᵇ₂', array([0., 1., 0.]))]
Beliefs: [('sᶜˢʳ₁', array([0.999, 0.001])), ('sᶜˢʳ₂', array([0., 1., 0.]))]

Time 9:
___act___
_a: [('aᴬᵈᵐ₁', 'NULL_ACT'), ('aᴬᵈᵐ₂', 'PHRASE_MATCH_ACT')]
_a: [('aᵂᵉᵇ₁', 'NULL_ACT'), ('aᵂᵉᵇ₂', 'DESKTOP_ACT')]
___future___
qIpiI=array([9.75588752e-04, 9.97013954e-01, 2.01045701e-03])
GNeg=array([-3.60483043, -3.171738  , -3.55963817])
qIpiI=array([6.19461593e-04, 9.98751592e-01, 6.28946433e-04])
GNeg=array([-3.60483043, -3.14324226, -3.60388072])
___next___
_s̆: [('s̆ᵂᵉᵇ₁', 'EXPANDED_TEXT_ADS'), ('s̆ᵂᵉᵇ₂', 'DISPLAY_URL')]
_s̆: [('s̆ᶜˢʳ₁', 'BUSINESS_HOURS'), ('s̆ᶜˢʳ₂', 'CENTRAL')]
___observe___
_y: [('yᵂᵉᵇ₁', 'UNKNOWN_OBS'), ('yᵂᵉᵇ₂', 'DISPLAY_URL_OBS'), ('yᵂᵉᵇ₃', 'CALLOUTS_OBS')]
_y: [('yᶜˢʳ₁', 'AFTER_HOURS_OBS'), ('yᶜˢʳ₂', 'EAST_OBS'), ('yᶜˢʳ₃', 'SPANISH_OBS')]
___infer___
Beliefs: [('sᵂᵉᵇ₁', array([0.855, 0.145])), ('sᵂᵉᵇ₂', array([0., 1., 0.]))]
Beliefs: [('sᶜˢʳ₁', array([1., 0.])), ('sᶜˢʳ₂', array([0., 1., 0.]))]

Time 10:
___act___
_a: [('aᴬᵈᵐ₁', 'NULL_ACT'), ('aᴬᵈᵐ₂', 'PHRASE_MATCH_ACT')]
_a: [('aᵂᵉᵇ₁', 'NULL_ACT'), ('aᵂᵉᵇ₂', 'DESKTOP_ACT')]
___future___
qIpiI=array([0.00211696, 0.98753736, 0.01034568])
GNeg=array([-3.60483043, -3.22075331, -3.50566866])
qIpiI=array([6.16163062e-04, 9.98764213e-01, 6.19623701e-04])
GNeg=array([-3.60483043, -3.14290778, -3.60448038])
___next___
_s̆: [('s̆ᵂᵉᵇ₁', 'EXPANDED_TEXT_ADS'), ('s̆ᵂᵉᵇ₂', 'DISPLAY_URL')]
_s̆: [('s̆ᶜˢʳ₁', 'BUSINESS_HOURS'), ('s̆ᶜˢʳ₂', 'CENTRAL')]
___observe___
_y: [('yᵂᵉᵇ₁', 'UNKNOWN_OBS'), ('yᵂᵉᵇ₂', 'DESCRIPTION_LINES_OBS'), ('yᵂᵉᵇ₃', 'CALLOUTS_OBS')]
_y: [('yᶜˢʳ₁', 'AFTER_HOURS_OBS'), ('yᶜˢʳ₂', 'EAST_OBS'), ('yᶜˢʳ₃', 'SPANISH_OBS')]
___infer___
Beliefs: [('sᵂᵉᵇ₁', array([0.941, 0.059])), ('sᵂᵉᵇ₂', array([0., 1., 0.]))]
Beliefs: [('sᶜˢʳ₁', array([1., 0.])), ('sᶜˢʳ₂', array([0., 1., 0.]))]

Time 11:
___act___
_a: [('aᴬᵈᵐ₁', 'NULL_ACT'), ('aᴬᵈᵐ₂', 'PHRASE_MATCH_ACT')]
_a: [('aᵂᵉᵇ₁', 'NULL_ACT'), ('aᵂᵉᵇ₂', 'DESKTOP_ACT')]
___future___
qIpiI=array([9.75588752e-04, 9.97013954e-01, 2.01045701e-03])
GNeg=array([-3.60483043, -3.171738  , -3.55963817])
qIpiI=array([6.14954210e-04, 9.98768822e-01, 6.16224057e-04])
GNeg=array([-3.60483043, -3.14278475, -3.6047015 ])
___next___
_s̆: [('s̆ᵂᵉᵇ₁', 'EXPANDED_TEXT_ADS'), ('s̆ᵂᵉᵇ₂', 'DISPLAY_URL')]
_s̆: [('s̆ᶜˢʳ₁', 'BUSINESS_HOURS'), ('s̆ᶜˢʳ₂', 'CENTRAL')]
___observe___
_y: [('yᵂᵉᵇ₁', 'UNKNOWN_OBS'), ('yᵂᵉᵇ₂', 'DESCRIPTION_LINES_OBS'), ('yᵂᵉᵇ₃', 'CALLOUTS_OBS')]
_y: [('yᶜˢʳ₁', 'AFTER_HOURS_OBS'), ('yᶜˢʳ₂', 'EAST_OBS'), ('yᶜˢʳ₃', 'SPANISH_OBS')]
___infer___
Beliefs: [('sᵂᵉᵇ₁', array([0.978, 0.022])), ('sᵂᵉᵇ₂', array([0., 1., 0.]))]
Beliefs: [('sᶜˢʳ₁', array([1., 0.])), ('sᶜˢʳ₂', array([0., 1., 0.]))]

Time 12:
___act___
_a: [('aᴬᵈᵐ₁', 'NULL_ACT'), ('aᴬᵈᵐ₂', 'PHRASE_MATCH_ACT')]
_a: [('aᵂᵉᵇ₁', 'NULL_ACT'), ('aᵂᵉᵇ₂', 'DESKTOP_ACT')]
___future___
qIpiI=array([7.28121866e-04, 9.98298628e-01, 9.73250403e-04])
GNeg=array([-3.60483043, -3.15337172, -3.58669462])
qIpiI=array([6.14510120e-04, 9.98770512e-01, 6.14977437e-04])
GNeg=array([-3.60483043, -3.1427395 , -3.60478292])
___next___
_s̆: [('s̆ᵂᵉᵇ₁', 'EXPANDED_TEXT_ADS'), ('s̆ᵂᵉᵇ₂', 'DISPLAY_URL')]
_s̆: [('s̆ᶜˢʳ₁', 'BUSINESS_HOURS'), ('s̆ᶜˢʳ₂', 'CENTRAL')]
___observe___
_y: [('yᵂᵉᵇ₁', 'UNKNOWN_OBS'), ('yᵂᵉᵇ₂', 'DISPLAY_URL_OBS'), ('yᵂᵉᵇ₃', 'CALLOUTS_OBS')]
_y: [('yᶜˢʳ₁', 'AFTER_HOURS_OBS'), ('yᶜˢʳ₂', 'EAST_OBS'), ('yᶜˢʳ₃', 'SPANISH_OBS')]
___infer___
Beliefs: [('sᵂᵉᵇ₁', array([0.941, 0.059])), ('sᵂᵉᵇ₂', array([0., 1., 0.]))]
Beliefs: [('sᶜˢʳ₁', array([1., 0.])), ('sᶜˢʳ₂', array([0., 1., 0.]))]

Time 13:
___act___
_a: [('aᴬᵈᵐ₁', 'NULL_ACT'), ('aᴬᵈᵐ₂', 'PHRASE_MATCH_ACT')]
_a: [('aᵂᵉᵇ₁', 'NULL_ACT'), ('aᵂᵉᵇ₂', 'DESKTOP_ACT')]
___future___
qIpiI=array([9.75588752e-04, 9.97013954e-01, 2.01045701e-03])
GNeg=array([-3.60483043, -3.171738  , -3.55963817])
qIpiI=array([6.14346833e-04, 9.98771134e-01, 6.14519376e-04])
GNeg=array([-3.60483043, -3.14272285, -3.60481288])
___next___
_s̆: [('s̆ᵂᵉᵇ₁', 'EXPANDED_TEXT_ADS'), ('s̆ᵂᵉᵇ₂', 'DISPLAY_URL')]
_s̆: [('s̆ᶜˢʳ₁', 'BUSINESS_HOURS'), ('s̆ᶜˢʳ₂', 'CENTRAL')]
___observe___
_y: [('yᵂᵉᵇ₁', 'UNKNOWN_OBS'), ('yᵂᵉᵇ₂', 'DISPLAY_URL_OBS'), ('yᵂᵉᵇ₃', 'CALLOUTS_OBS')]
_y: [('yᶜˢʳ₁', 'AFTER_HOURS_OBS'), ('yᶜˢʳ₂', 'EAST_OBS'), ('yᶜˢʳ₃', 'SPANISH_OBS')]
___infer___
Beliefs: [('sᵂᵉᵇ₁', array([0.855, 0.145])), ('sᵂᵉᵇ₂', array([0., 1., 0.]))]
Beliefs: [('sᶜˢʳ₁', array([1., 0.])), ('sᶜˢʳ₂', array([0., 1., 0.]))]

Time 14:
___act___
_a: [('aᴬᵈᵐ₁', 'NULL_ACT'), ('aᴬᵈᵐ₂', 'PHRASE_MATCH_ACT')]
_a: [('aᵂᵉᵇ₁', 'NULL_ACT'), ('aᵂᵉᵇ₂', 'DESKTOP_ACT')]
___future___
qIpiI=array([0.00211696, 0.98753736, 0.01034568])
GNeg=array([-3.60483043, -3.22075331, -3.50566866])
qIpiI=array([6.14286774e-04, 9.98771362e-01, 6.14350939e-04])
GNeg=array([-3.60483043, -3.14271672, -3.6048239 ])
___next___
_s̆: [('s̆ᵂᵉᵇ₁', 'EXPANDED_TEXT_ADS'), ('s̆ᵂᵉᵇ₂', 'DISPLAY_URL')]
_s̆: [('s̆ᶜˢʳ₁', 'BUSINESS_HOURS'), ('s̆ᶜˢʳ₂', 'CENTRAL')]
___observe___
_y: [('yᵂᵉᵇ₁', 'UNKNOWN_OBS'), ('yᵂᵉᵇ₂', 'DESCRIPTION_LINES_OBS'), ('yᵂᵉᵇ₃', 'CALLOUTS_OBS')]
_y: [('yᶜˢʳ₁', 'AFTER_HOURS_OBS'), ('yᶜˢʳ₂', 'CENTRAL_OBS'), ('yᶜˢʳ₃', 'SPANISH_OBS')]
___infer___
Beliefs: [('sᵂᵉᵇ₁', array([0.941, 0.059])), ('sᵂᵉᵇ₂', array([0., 1., 0.]))]
Beliefs: [('sᶜˢʳ₁', array([1., 0.])), ('sᶜˢʳ₂', array([0., 1., 0.]))]

Time 15:
___act___
_a: [('aᴬᵈᵐ₁', 'NULL_ACT'), ('aᴬᵈᵐ₂', 'PHRASE_MATCH_ACT')]
_a: [('aᵂᵉᵇ₁', 'NULL_ACT'), ('aᵂᵉᵇ₂', 'DESKTOP_ACT')]
___future___
qIpiI=array([9.75588752e-04, 9.97013954e-01, 2.01045701e-03])
GNeg=array([-3.60483043, -3.171738  , -3.55963817])
qIpiI=array([6.14346833e-04, 9.98771134e-01, 6.14519376e-04])
GNeg=array([-3.60483043, -3.14272285, -3.60481288])
___next___
_s̆: [('s̆ᵂᵉᵇ₁', 'EXPANDED_TEXT_ADS'), ('s̆ᵂᵉᵇ₂', 'DISPLAY_URL')]
_s̆: [('s̆ᶜˢʳ₁', 'BUSINESS_HOURS'), ('s̆ᶜˢʳ₂', 'CENTRAL')]
___observe___
_y: [('yᵂᵉᵇ₁', 'UNKNOWN_OBS'), ('yᵂᵉᵇ₂', 'DISPLAY_URL_OBS'), ('yᵂᵉᵇ₃', 'CALLOUTS_OBS')]
_y: [('yᶜˢʳ₁', 'AFTER_HOURS_OBS'), ('yᶜˢʳ₂', 'CENTRAL_OBS'), ('yᶜˢʳ₃', 'SPANISH_OBS')]
___infer___
Beliefs: [('sᵂᵉᵇ₁', array([0.855, 0.145])), ('sᵂᵉᵇ₂', array([0., 1., 0.]))]
Beliefs: [('sᶜˢʳ₁', array([1., 0.])), ('sᶜˢʳ₂', array([0., 1., 0.]))]

Time 16:
___act___
_a: [('aᴬᵈᵐ₁', 'NULL_ACT'), ('aᴬᵈᵐ₂', 'PHRASE_MATCH_ACT')]
_a: [('aᵂᵉᵇ₁', 'NULL_ACT'), ('aᵂᵉᵇ₂', 'DESKTOP_ACT')]
___future___
qIpiI=array([0.00211696, 0.98753736, 0.01034568])
GNeg=array([-3.60483043, -3.22075331, -3.50566866])
qIpiI=array([6.14510120e-04, 9.98770512e-01, 6.14977437e-04])
GNeg=array([-3.60483043, -3.1427395 , -3.60478292])
___next___
_s̆: [('s̆ᵂᵉᵇ₁', 'EXPANDED_TEXT_ADS'), ('s̆ᵂᵉᵇ₂', 'DISPLAY_URL')]
_s̆: [('s̆ᶜˢʳ₁', 'BUSINESS_HOURS'), ('s̆ᶜˢʳ₂', 'CENTRAL')]
___observe___
_y: [('yᵂᵉᵇ₁', 'UNKNOWN_OBS'), ('yᵂᵉᵇ₂', 'DESCRIPTION_LINES_OBS'), ('yᵂᵉᵇ₃', 'CALLOUTS_OBS')]
_y: [('yᶜˢʳ₁', 'AFTER_HOURS_OBS'), ('yᶜˢʳ₂', 'EAST_OBS'), ('yᶜˢʳ₃', 'SPANISH_OBS')]
___infer___
Beliefs: [('sᵂᵉᵇ₁', array([0.941, 0.059])), ('sᵂᵉᵇ₂', array([0., 1., 0.]))]
Beliefs: [('sᶜˢʳ₁', array([1., 0.])), ('sᶜˢʳ₂', array([0., 1., 0.]))]

Time 17:
___act___
_a: [('aᴬᵈᵐ₁', 'NULL_ACT'), ('aᴬᵈᵐ₂', 'PHRASE_MATCH_ACT')]
_a: [('aᵂᵉᵇ₁', 'NULL_ACT'), ('aᵂᵉᵇ₂', 'DESKTOP_ACT')]
___future___
qIpiI=array([9.75588752e-04, 9.97013954e-01, 2.01045701e-03])
GNeg=array([-3.60483043, -3.171738  , -3.55963817])
qIpiI=array([6.14346833e-04, 9.98771134e-01, 6.14519376e-04])
GNeg=array([-3.60483043, -3.14272285, -3.60481288])
___next___
_s̆: [('s̆ᵂᵉᵇ₁', 'EXPANDED_TEXT_ADS'), ('s̆ᵂᵉᵇ₂', 'DISPLAY_URL')]
_s̆: [('s̆ᶜˢʳ₁', 'BUSINESS_HOURS'), ('s̆ᶜˢʳ₂', 'CENTRAL')]
___observe___
_y: [('yᵂᵉᵇ₁', 'UNKNOWN_OBS'), ('yᵂᵉᵇ₂', 'DISPLAY_URL_OBS'), ('yᵂᵉᵇ₃', 'CALLOUTS_OBS')]
_y: [('yᶜˢʳ₁', 'AFTER_HOURS_OBS'), ('yᶜˢʳ₂', 'EAST_OBS'), ('yᶜˢʳ₃', 'SPANISH_OBS')]
___infer___
Beliefs: [('sᵂᵉᵇ₁', array([0.855, 0.145])), ('sᵂᵉᵇ₂', array([0., 1., 0.]))]
Beliefs: [('sᶜˢʳ₁', array([1., 0.])), ('sᶜˢʳ₂', array([0., 1., 0.]))]

Time 18:
___act___
_a: [('aᴬᵈᵐ₁', 'NULL_ACT'), ('aᴬᵈᵐ₂', 'PHRASE_MATCH_ACT')]
_a: [('aᵂᵉᵇ₁', 'NULL_ACT'), ('aᵂᵉᵇ₂', 'DESKTOP_ACT')]
___future___
qIpiI=array([0.00211696, 0.98753736, 0.01034568])
GNeg=array([-3.60483043, -3.22075331, -3.50566866])
qIpiI=array([6.14286774e-04, 9.98771362e-01, 6.14350939e-04])
GNeg=array([-3.60483043, -3.14271672, -3.6048239 ])
___next___
_s̆: [('s̆ᵂᵉᵇ₁', 'EXPANDED_TEXT_ADS'), ('s̆ᵂᵉᵇ₂', 'DISPLAY_URL')]
_s̆: [('s̆ᶜˢʳ₁', 'BUSINESS_HOURS'), ('s̆ᶜˢʳ₂', 'CENTRAL')]
___observe___
_y: [('yᵂᵉᵇ₁', 'UNKNOWN_OBS'), ('yᵂᵉᵇ₂', 'DESCRIPTION_LINES_OBS'), ('yᵂᵉᵇ₃', 'CALLOUTS_OBS')]
_y: [('yᶜˢʳ₁', 'AFTER_HOURS_OBS'), ('yᶜˢʳ₂', 'CENTRAL_OBS'), ('yᶜˢʳ₃', 'SPANISH_OBS')]
___infer___
Beliefs: [('sᵂᵉᵇ₁', array([0.941, 0.059])), ('sᵂᵉᵇ₂', array([0., 1., 0.]))]
Beliefs: [('sᶜˢʳ₁', array([1., 0.])), ('sᶜˢʳ₂', array([0., 1., 0.]))]

Time 19:
___act___
_a: [('aᴬᵈᵐ₁', 'NULL_ACT'), ('aᴬᵈᵐ₂', 'PHRASE_MATCH_ACT')]
_a: [('aᵂᵉᵇ₁', 'NULL_ACT'), ('aᵂᵉᵇ₂', 'DESKTOP_ACT')]
___future___
qIpiI=array([9.75588752e-04, 9.97013954e-01, 2.01045701e-03])
GNeg=array([-3.60483043, -3.171738  , -3.55963817])
qIpiI=array([6.14346833e-04, 9.98771134e-01, 6.14519376e-04])
GNeg=array([-3.60483043, -3.14272285, -3.60481288])
___next___
_s̆: [('s̆ᵂᵉᵇ₁', 'EXPANDED_TEXT_ADS'), ('s̆ᵂᵉᵇ₂', 'DISPLAY_URL')]
_s̆: [('s̆ᶜˢʳ₁', 'BUSINESS_HOURS'), ('s̆ᶜˢʳ₂', 'CENTRAL')]
___observe___
_y: [('yᵂᵉᵇ₁', 'UNKNOWN_OBS'), ('yᵂᵉᵇ₂', 'DESCRIPTION_LINES_OBS'), ('yᵂᵉᵇ₃', 'CALLOUTS_OBS')]
_y: [('yᶜˢʳ₁', 'AFTER_HOURS_OBS'), ('yᶜˢʳ₂', 'CENTRAL_OBS'), ('yᶜˢʳ₃', 'SPANISH_OBS')]
___infer___
Beliefs: [('sᵂᵉᵇ₁', array([0.978, 0.022])), ('sᵂᵉᵇ₂', array([0., 1., 0.]))]
Beliefs: [('sᶜˢʳ₁', array([1., 0.])), ('sᶜˢʳ₂', array([0., 1., 0.]))]

Time 20:
___act___
_a: [('aᴬᵈᵐ₁', 'NULL_ACT'), ('aᴬᵈᵐ₂', 'PHRASE_MATCH_ACT')]
_a: [('aᵂᵉᵇ₁', 'NULL_ACT'), ('aᵂᵉᵇ₂', 'DESKTOP_ACT')]
___future___
qIpiI=array([7.28121866e-04, 9.98298628e-01, 9.73250403e-04])
GNeg=array([-3.60483043, -3.15337172, -3.58669462])
qIpiI=array([6.14510120e-04, 9.98770512e-01, 6.14977437e-04])
GNeg=array([-3.60483043, -3.1427395 , -3.60478292])
___next___
_s̆: [('s̆ᵂᵉᵇ₁', 'EXPANDED_TEXT_ADS'), ('s̆ᵂᵉᵇ₂', 'DISPLAY_URL')]
_s̆: [('s̆ᶜˢʳ₁', 'BUSINESS_HOURS'), ('s̆ᶜˢʳ₂', 'CENTRAL')]
___observe___
_y: [('yᵂᵉᵇ₁', 'UNKNOWN_OBS'), ('yᵂᵉᵇ₂', 'DESCRIPTION_LINES_OBS'), ('yᵂᵉᵇ₃', 'CALLOUTS_OBS')]
_y: [('yᶜˢʳ₁', 'AFTER_HOURS_OBS'), ('yᶜˢʳ₂', 'EAST_OBS'), ('yᶜˢʳ₃', 'SPANISH_OBS')]
___infer___
Beliefs: [('sᵂᵉᵇ₁', array([0.992, 0.008])), ('sᵂᵉᵇ₂', array([0., 1., 0.]))]
Beliefs: [('sᶜˢʳ₁', array([1., 0.])), ('sᶜˢʳ₂', array([0., 1., 0.]))]

Time 21:
___act___
_a: [('aᴬᵈᵐ₁', 'NULL_ACT'), ('aᴬᵈᵐ₂', 'PHRASE_MATCH_ACT')]
_a: [('aᵂᵉᵇ₁', 'NULL_ACT'), ('aᵂᵉᵇ₂', 'DESKTOP_ACT')]
___future___
qIpiI=array([6.53848020e-04, 9.98616007e-01, 7.30144782e-04])
GNeg=array([-3.60483043, -3.14662726, -3.59793244])
qIpiI=array([6.14346833e-04, 9.98771134e-01, 6.14519376e-04])
GNeg=array([-3.60483043, -3.14272285, -3.60481288])
___next___
_s̆: [('s̆ᵂᵉᵇ₁', 'EXPANDED_TEXT_ADS'), ('s̆ᵂᵉᵇ₂', 'DISPLAY_URL')]
_s̆: [('s̆ᶜˢʳ₁', 'BUSINESS_HOURS'), ('s̆ᶜˢʳ₂', 'CENTRAL')]
___observe___
_y: [('yᵂᵉᵇ₁', 'UNKNOWN_OBS'), ('yᵂᵉᵇ₂', 'DESCRIPTION_LINES_OBS'), ('yᵂᵉᵇ₃', 'CALLOUTS_OBS')]
_y: [('yᶜˢʳ₁', 'AFTER_HOURS_OBS'), ('yᶜˢʳ₂', 'CENTRAL_OBS'), ('yᶜˢʳ₃', 'SPANISH_OBS')]
___infer___
Beliefs: [('sᵂᵉᵇ₁', array([0.997, 0.003])), ('sᵂᵉᵇ₂', array([0., 1., 0.]))]
Beliefs: [('sᶜˢʳ₁', array([1., 0.])), ('sᶜˢʳ₂', array([0., 1., 0.]))]

Time 22:
___act___
_a: [('aᴬᵈᵐ₁', 'NULL_ACT'), ('aᴬᵈᵐ₂', 'PHRASE_MATCH_ACT')]
_a: [('aᵂᵉᵇ₁', 'NULL_ACT'), ('aᵂᵉᵇ₂', 'DESKTOP_ACT')]
___future___
qIpiI=array([6.28521104e-04, 9.98716579e-01, 6.54899545e-04])
GNeg=array([-3.60483043, -3.14415189, -3.60226091])
qIpiI=array([6.14510120e-04, 9.98770512e-01, 6.14977437e-04])
GNeg=array([-3.60483043, -3.1427395 , -3.60478292])
___next___
_s̆: [('s̆ᵂᵉᵇ₁', 'EXPANDED_TEXT_ADS'), ('s̆ᵂᵉᵇ₂', 'DISPLAY_URL')]
_s̆: [('s̆ᶜˢʳ₁', 'BUSINESS_HOURS'), ('s̆ᶜˢʳ₂', 'CENTRAL')]
___observe___
_y: [('yᵂᵉᵇ₁', 'UNKNOWN_OBS'), ('yᵂᵉᵇ₂', 'DESCRIPTION_LINES_OBS'), ('yᵂᵉᵇ₃', 'CALLOUTS_OBS')]
_y: [('yᶜˢʳ₁', 'AFTER_HOURS_OBS'), ('yᶜˢʳ₂', 'EAST_OBS'), ('yᶜˢʳ₃', 'SPANISH_OBS')]
___infer___
Beliefs: [('sᵂᵉᵇ₁', array([0.999, 0.001])), ('sᵂᵉᵇ₂', array([0., 1., 0.]))]
Beliefs: [('sᶜˢʳ₁', array([1., 0.])), ('sᶜˢʳ₂', array([0., 1., 0.]))]

Time 23:
___act___
_a: [('aᴬᵈᵐ₁', 'NULL_ACT'), ('aᴬᵈᵐ₂', 'PHRASE_MATCH_ACT')]
_a: [('aᵂᵉᵇ₁', 'NULL_ACT'), ('aᵂᵉᵇ₂', 'DESKTOP_ACT')]
___future___
qIpiI=array([6.19461593e-04, 9.98751592e-01, 6.28946433e-04])
GNeg=array([-3.60483043, -3.14324226, -3.60388072])
qIpiI=array([6.14346833e-04, 9.98771134e-01, 6.14519376e-04])
GNeg=array([-3.60483043, -3.14272285, -3.60481288])
___next___
_s̆: [('s̆ᵂᵉᵇ₁', 'EXPANDED_TEXT_ADS'), ('s̆ᵂᵉᵇ₂', 'DISPLAY_URL')]
_s̆: [('s̆ᶜˢʳ₁', 'BUSINESS_HOURS'), ('s̆ᶜˢʳ₂', 'CENTRAL')]
___observe___
_y: [('yᵂᵉᵇ₁', 'UNKNOWN_OBS'), ('yᵂᵉᵇ₂', 'DESCRIPTION_LINES_OBS'), ('yᵂᵉᵇ₃', 'CALLOUTS_OBS')]
_y: [('yᶜˢʳ₁', 'AFTER_HOURS_OBS'), ('yᶜˢʳ₂', 'EAST_OBS'), ('yᶜˢʳ₃', 'SPANISH_OBS')]
___infer___
Beliefs: [('sᵂᵉᵇ₁', array([1., 0.])), ('sᵂᵉᵇ₂', array([0., 1., 0.]))]
Beliefs: [('sᶜˢʳ₁', array([1., 0.])), ('sᶜˢʳ₂', array([0., 1., 0.]))]

Time 24:
___act___
_a: [('aᴬᵈᵐ₁', 'NULL_ACT'), ('aᴬᵈᵐ₂', 'PHRASE_MATCH_ACT')]
_a: [('aᵂᵉᵇ₁', 'NULL_ACT'), ('aᵂᵉᵇ₂', 'DESKTOP_ACT')]
___future___
qIpiI=array([6.16163062e-04, 9.98764213e-01, 6.19623701e-04])
GNeg=array([-3.60483043, -3.14290778, -3.60448038])
qIpiI=array([6.14286774e-04, 9.98771362e-01, 6.14350939e-04])
GNeg=array([-3.60483043, -3.14271672, -3.6048239 ])
___next___
_s̆: [('s̆ᵂᵉᵇ₁', 'EXPANDED_TEXT_ADS'), ('s̆ᵂᵉᵇ₂', 'DISPLAY_URL')]
_s̆: [('s̆ᶜˢʳ₁', 'BUSINESS_HOURS'), ('s̆ᶜˢʳ₂', 'CENTRAL')]
___observe___
_y: [('yᵂᵉᵇ₁', 'UNKNOWN_OBS'), ('yᵂᵉᵇ₂', 'DESCRIPTION_LINES_OBS'), ('yᵂᵉᵇ₃', 'CALLOUTS_OBS')]
_y: [('yᶜˢʳ₁', 'AFTER_HOURS_OBS'), ('yᶜˢʳ₂', 'CENTRAL_OBS'), ('yᶜˢʳ₃', 'SPANISH_OBS')]
___infer___
Beliefs: [('sᵂᵉᵇ₁', array([1., 0.])), ('sᵂᵉᵇ₂', array([0., 1., 0.]))]
Beliefs: [('sᶜˢʳ₁', array([1., 0.])), ('sᶜˢʳ₂', array([0., 1., 0.]))]

colors = [
{'NULL_ACT':'black'}, ## aAdm_1
{'BROAD_MATCH_ACT':'red', 'PHRASE_MATCH_ACT':'green', 'EXACT_MATCH_ACT': 'blue'}, ## aAdm_2

{'EXPANDED_TEXT_ADS':'orange', 'RESPONSIVE_SEARCH_ADS':'purple'}, ## sWeb_1
{'DESCRIPTION_LINES':'red', 'DISPLAY_URL':'green', 'CALL_TO_ACTION': 'blue'}, ## sWeb_2

{'EXPANDED_TEXT_ADS':'orange', 'RESPONSIVE_SEARCH_ADS':'purple'}, ## qIsIWeb_1
{'DESCRIPTION_LINES':'red', 'DISPLAY_URL':'green', 'CALL_TO_ACTION': 'blue'}, ## qIsIWeb_2

{'EXPANDED_TEXT_ADS_OBS':'orange', 'RESPONSIVE_SEARCH_ADS_OBS':'purple', 'UNKNOWN_OBS':'pink'}, ## yWeb_1
{'DESCRIPTION_LINES_OBS':'red', 'DISPLAY_URL_OBS':'green', 'CALL_TO_ACTION_OBS': 'blue'}, ## yWeb_2
{'SITE_LINKS_OBS':'red', 'CALLOUTS_OBS':'green', 'STRUCTURED_SNIPPETS_OBS': 'blue'} ## yWeb_3
]

ylabel_size = 12
msi = 7 ## markersize for Line2D, diameter in points
siz = (msi/2)**2 * np.pi ## size for scatter, area of marker in points squared

fig = plt.figure(figsize=(9, 6))
## gs = GridSpec(6, 1, figure=fig, height_ratios=[1, 3, 1, 3, 3, 1])
gs = GridSpec(9, 1, figure=fig, height_ratios=[1, 1, 1, 1, 1, 1, 1, 1, 1])
ax = [fig.add_subplot(gs[i]) for i in range(9)]

i = 0
ax[i].set_title(f'Administrator/Website interaction', fontweight='bold',fontsize=14)
y_pos = 0
for t, s in zip(range(_T), _aAdm_facs['aᴬᵈᵐ₁']): ## 'NULL'
    ax[i].scatter(t, y_pos, color=colors[i][s], s=siz)
ax[i].set_ylabel('$a^{\mathrm{Adm}}_{1t}, \mathrm{NULL}$', rotation=0, fontweight='bold', fontsize=ylabel_size)
ax[i].set_yticks([])
ax[i].set_xticklabels([])
leg_items = [
    Line2D([0],[0],marker='o',color='w',markerfacecolor='black',markersize=msi,label='NULL_ACT')]
ax[i].legend(handles=leg_items, bbox_to_anchor=(1.05, 0.5), loc='center left', borderaxespad=0, labelspacing=0.1)
ax[i].spines['top'].set_visible(False); ax[i].spines['right'].set_visible(False)

i = 1
y_pos = 0
for t, s in zip(range(_T), _aAdm_facs['aᴬᵈᵐ₂']): ## 'SET_MATCH_TYPE_ACTION'
    ax[i].scatter(t, y_pos, color=colors[i][s], s=siz, label=s)
ax[i].set_ylabel('$a^{\mathrm{Adm}}_{2t}, \mathrm{SET\_MATCH\_TYPE\_ACTION}$', rotation=0, fontweight='bold', fontsize=ylabel_size)
ax[i].set_yticks([])
leg_items = [
    Line2D([0],[0],marker='o',color='w',markerfacecolor=colors[i]['BROAD_MATCH_ACT'],markersize=msi,label='BROAD_MATCH_ACT'),
    Line2D([0],[0],marker='o',color='w',markerfacecolor=colors[i]['PHRASE_MATCH_ACT'],markersize=msi,label='PHRASE_MATCH_ACT'),
    Line2D([0],[0],marker='o',color='w',markerfacecolor=colors[i]['EXACT_MATCH_ACT'],markersize=msi,label='EXACT_MATCH_ACT')]
ax[i].legend(handles=leg_items, bbox_to_anchor=(1.05, 0.5), loc='center left', borderaxespad=0, labelspacing=0.1)
ax[i].spines['top'].set_visible(False); ax[i].spines['right'].set_visible(False)

i = 2
y_pos = 0
for t, s in zip(range(_T), _sAdm_facs['sᵂᵉᵇ₁']): ## 'AD_TYPE'
    ax[i].scatter(t, y_pos, color=colors[i][s], s=siz)
ax[i].set_ylabel('$s^{\mathrm{Web}}_{1t}, \mathrm{AD\_TYPE}$', rotation=0, fontweight='bold', fontsize=ylabel_size)
ax[i].set_yticks([])
ax[i].set_xticklabels([])
leg_items = [
    Line2D([0],[0],marker='o',color='w',markerfacecolor=colors[i]['EXPANDED_TEXT_ADS'],markersize=msi,label='EXPANDED_TEXT_ADS'),
    Line2D([0],[0],marker='o',color='w',markerfacecolor=colors[i]['RESPONSIVE_SEARCH_ADS'],markersize=msi,label='RESPONSIVE_SEARCH_ADS')]
ax[i].legend(handles=leg_items, bbox_to_anchor=(1.05, 0.5), loc='center left', borderaxespad=0, labelspacing=0.1)
ax[i].spines['top'].set_visible(False); ax[i].spines['right'].set_visible(False)

i = 3
y_pos = 0
for t, s in zip(range(_T), _sAdm_facs['sᵂᵉᵇ₂']): ## 'AD_COPY_CREATION'
    ax[i].scatter(t, y_pos, color=colors[i][s], s=siz, label=s)
ax[i].set_ylabel('$s^{\mathrm{Web}}_{2t}, \mathrm{AD\_COPY\_CREATION}$', rotation=0, fontweight='bold', fontsize=ylabel_size)
ax[i].set_yticks([])
leg_items = [
    Line2D([0],[0],marker='o',color='w',markerfacecolor=colors[i]['DESCRIPTION_LINES'],markersize=msi,label='DESCRIPTION_LINES'),
    Line2D([0],[0],marker='o',color='w',markerfacecolor=colors[i]['DISPLAY_URL'],markersize=msi,label='DISPLAY_URL'),
    Line2D([0],[0],marker='o',color='w',markerfacecolor=colors[i]['CALL_TO_ACTION'],markersize=msi,label='CALL_TO_ACTION')]
ax[i].legend(handles=leg_items, bbox_to_anchor=(1.05, 0.5), loc='center left', borderaxespad=0, labelspacing=0.1)
ax[i].spines['top'].set_visible(False); ax[i].spines['right'].set_visible(False)

i = 4
y_pos = 0
for t, s in zip(range(_T), _sAdm_facs['sᵂᵉᵇ₁']): ## 'AD_TYPE'
    ax[i].scatter(t, y_pos, color=colors[i][s], s=siz)
ax[i].set_ylabel('$q(s)^{\mathrm{Web}}_{1t}, \mathrm{AD\_TYPE}$', rotation=0, fontweight='bold', fontsize=ylabel_size)
ax[i].set_yticks([])
ax[i].set_xticklabels([])
leg_items = [
    Line2D([0],[0],marker='o',color='w',markerfacecolor=colors[i]['EXPANDED_TEXT_ADS'],markersize=msi,label='EXPANDED_TEXT_ADS'),
    Line2D([0],[0],marker='o',color='w',markerfacecolor=colors[i]['RESPONSIVE_SEARCH_ADS'],markersize=msi,label='RESPONSIVE_SEARCH_ADS')]
ax[i].legend(handles=leg_items, bbox_to_anchor=(1.05, 0.5), loc='center left', borderaxespad=0, labelspacing=0.1)
ax[i].spines['top'].set_visible(False); ax[i].spines['right'].set_visible(False)

i = 5
y_pos = 0
for t, s in zip(range(_T), _sAdm_facs['sᵂᵉᵇ₂']): ## 'AD_COPY_CREATION'
    ax[i].scatter(t, y_pos, color=colors[i][s], s=siz, label=s)
ax[i].set_ylabel('$q(s)^{\mathrm{Web}}_{2t}, \mathrm{AD\_COPY\_CREATION}$', rotation=0, fontweight='bold', fontsize=ylabel_size)
ax[i].set_yticks([])
leg_items = [
    Line2D([0],[0],marker='o',color='w',markerfacecolor=colors[i]['DESCRIPTION_LINES'],markersize=msi,label='DESCRIPTION_LINES'),
    Line2D([0],[0],marker='o',color='w',markerfacecolor=colors[i]['DISPLAY_URL'],markersize=msi,label='DISPLAY_URL'),
    Line2D([0],[0],marker='o',color='w',markerfacecolor=colors[i]['CALL_TO_ACTION'],markersize=msi,label='CALL_TO_ACTION')]
ax[i].legend(handles=leg_items, bbox_to_anchor=(1.05, 0.5), loc='center left', borderaxespad=0, labelspacing=0.1)
ax[i].spines['top'].set_visible(False); ax[i].spines['right'].set_visible(False)

i = 6
y_pos = 0
for t, s in zip(range(_T), _yWeb_mods['yᵂᵉᵇ₁']): ## 'AD_TYPE_OBS'
    ax[i].scatter(t, y_pos, color=colors[i][s], s=siz, label=s)
ax[i].set_ylabel('$y^{\mathrm{Web}}_{1t}, \mathrm{AD\_TYPE\_OBS}$', rotation=0, fontweight='bold', fontsize=ylabel_size)
ax[i].set_yticks([])
leg_items = [
    Line2D([0],[0],marker='o',color='w',markerfacecolor=colors[i]['EXPANDED_TEXT_ADS_OBS'],markersize=msi,label='EXPANDED_TEXT_ADS_OBS'),
    Line2D([0],[0],marker='o',color='w',markerfacecolor=colors[i]['RESPONSIVE_SEARCH_ADS_OBS'],markersize=msi,label='RESPONSIVE_SEARCH_ADS_OBS'),
    Line2D([0],[0],marker='o',color='w',markerfacecolor=colors[i]['UNKNOWN_OBS'],markersize=msi,label='UNKNOWN_OBS')]
ax[i].legend(handles=leg_items, bbox_to_anchor=(1.05, 0.5), loc='center left', borderaxespad=0, labelspacing=0.1)
ax[i].spines['top'].set_visible(False); ax[i].spines['right'].set_visible(False)

i = 7
y_pos = 0
for t, s in zip(range(_T), _yWeb_mods['yᵂᵉᵇ₂']): ## 'AD_COPY_CREATION_OBS'
    ax[i].scatter(t, y_pos, color=colors[i][s], s=siz, label=s)
ax[i].set_ylabel('$y^{\mathrm{Web}}_{2t}, \mathrm{AD\_COPY\_CREATION\_OBS}$', rotation=0, fontweight='bold', fontsize=ylabel_size)
ax[i].set_yticks([])
leg_items = [
    Line2D([0],[0],marker='o',color='w',markerfacecolor=colors[i]['DESCRIPTION_LINES_OBS'],markersize=msi,label='DESCRIPTION_LINES_OBS'),
    Line2D([0],[0],marker='o',color='w',markerfacecolor=colors[i]['DISPLAY_URL_OBS'],markersize=msi,label='DISPLAY_URL_OBS'),
    Line2D([0],[0],marker='o',color='w',markerfacecolor=colors[i]['CALL_TO_ACTION_OBS'],markersize=msi,label='CALL_TO_ACTION_OBS')]
ax[i].legend(handles=leg_items, bbox_to_anchor=(1.05, 0.5), loc='center left', borderaxespad=0, labelspacing=0.1)
ax[i].spines['top'].set_visible(False); ax[i].spines['right'].set_visible(False)

i = 8
y_pos = 0
for t, s in zip(range(_T), _yWeb_mods['yᵂᵉᵇ₃']): ## 'AD_EXTENSIONS_OBS'
    ax[i].scatter(t, y_pos, color=colors[i][s], s=siz, label=s)
ax[i].set_ylabel('$y^{\mathrm{Web}}_{3t}, \mathrm{AD\_EXTENSIONS\_OBS}$', rotation=0, fontweight='bold', fontsize=ylabel_size)
ax[i].set_yticks([])
leg_items = [
    Line2D([0],[0],marker='o',color='w',markerfacecolor=colors[i]['SITE_LINKS_OBS'],markersize=msi,label='SITE_LINKS_OBS'),
    Line2D([0],[0],marker='o',color='w',markerfacecolor=colors[i]['CALLOUTS_OBS'],markersize=msi,label='CALLOUTS_OBS'),
    Line2D([0],[0],marker='o',color='w',markerfacecolor=colors[i]['STRUCTURED_SNIPPETS_OBS'],markersize=msi,label='STRUCTURED_SNIPPETS_OBS')]
ax[i].legend(handles=leg_items, bbox_to_anchor=(1.05, 0.5), loc='center left', borderaxespad=0, labelspacing=0.1)
ax[i].spines['top'].set_visible(False); ax[i].spines['right'].set_visible(False)

ax[i].xaxis.set_major_locator(MaxNLocator(integer=True))
## ax[i].xaxis.set_major_locator(MaxNLocator(nbins=10, integer=True))
ax[i].set_xlabel('$\mathrm{time,}\ t$', fontweight='bold', fontsize=12)

plt.tight_layout()
plt.subplots_adjust(hspace=0.1) ## Adjust this value as needed
plt.show()

colors = [
{'NULL_ACT':'black'}, ## aWeb_1
{'MOBILE_ACT':'red', 'DESKTOP_ACT':'green', 'TABLET_ACT': 'blue'}, ## aWeb_2

{'BUSINESS_HOURS':'orange', 'AFTER_HOURS':'purple'}, ## sCsr_1
{'EAST':'red', 'CENTRAL':'green', 'WEST': 'blue'}, ## sCsr_2

{'BUSINESS_HOURS':'orange', 'AFTER_HOURS':'purple'}, ## qIsICsr_1
{'EAST':'red', 'CENTRAL':'green', 'WEST': 'blue'}, ## qIsICsr_2

{'BUSINESS_HOURS_OBS':'orange', 'OVERLAP_HOURS_OBS':'purple', 'AFTER_HOURS_OBS':'pink'}, ## yCsr_1
{'EAST_OBS':'red', 'CENTRAL_OBS':'green', 'WEST_OBS': 'blue'}, ## yCsr_2
{'ENGLISH_OBS':'red', 'SPANISH_OBS':'green', 'CHINESE_OBS': 'blue'} ## yCsr_3
]

ylabel_size = 12
msi = 7 ## markersize for Line2D, diameter in points
siz = (msi/2)**2 * np.pi ## size for scatter, area of marker in points squared

fig = plt.figure(figsize=(9, 6))
## gs = GridSpec(6, 1, figure=fig, height_ratios=[1, 3, 1, 3, 3, 1])
gs = GridSpec(9, 1, figure=fig, height_ratios=[1, 1, 1, 1, 1, 1, 1, 1, 1])
ax = [fig.add_subplot(gs[i]) for i in range(9)]

i = 0
ax[i].set_title(f'Website/Consumer interaction', fontweight='bold',fontsize=14)
y_pos = 0
for t, s in zip(range(_T), _aWeb_facs['aᵂᵉᵇ₁']): ## 'NULL'
    ax[i].scatter(t, y_pos, color=colors[i][s], s=siz)
ax[i].set_ylabel('$a^{\mathrm{Web}}_{1t}, \mathrm{NULL}$', rotation=0, fontweight='bold', fontsize=ylabel_size)
ax[i].set_yticks([])
ax[i].set_xticklabels([])
leg_items = [
    Line2D([0],[0],marker='o',color='w',markerfacecolor='black',markersize=msi,label='NULL_ACT')]
ax[i].legend(handles=leg_items, bbox_to_anchor=(1.05, 0.5), loc='center left', borderaxespad=0, labelspacing=0.1)
ax[i].spines['top'].set_visible(False); ax[i].spines['right'].set_visible(False)

i = 1
y_pos = 0
for t, s in zip(range(_T), _aWeb_facs['aᵂᵉᵇ₂']): ## 'SET_DEVICE_TARGET_ACTION'
    ax[i].scatter(t, y_pos, color=colors[i][s], s=siz, label=s)
ax[i].set_ylabel('$a^{\mathrm{Web}}_{2t}, \mathrm{SET\_DEVICE\_TARGET\_ACTION}$', rotation=0, fontweight='bold', fontsize=ylabel_size)
ax[i].set_yticks([])
leg_items = [
    Line2D([0],[0],marker='o',color='w',markerfacecolor=colors[i]['MOBILE_ACT'],markersize=msi,label='MOBILE_ACT'),
    Line2D([0],[0],marker='o',color='w',markerfacecolor=colors[i]['DESKTOP_ACT'],markersize=msi,label='DESKTOP_ACT'),
    Line2D([0],[0],marker='o',color='w',markerfacecolor=colors[i]['TABLET_ACT'],markersize=msi,label='TABLET_ACT')]
ax[i].legend(handles=leg_items, bbox_to_anchor=(1.05, 0.5), loc='center left', borderaxespad=0, labelspacing=0.1)
ax[i].spines['top'].set_visible(False); ax[i].spines['right'].set_visible(False)

i = 2
y_pos = 0
for t, s in zip(range(_T), _sWeb_facs['sᶜˢʳ₁']): ## 'AD_SCHEDULE'
    ax[i].scatter(t, y_pos, color=colors[i][s], s=siz)
ax[i].set_ylabel('$s^{\mathrm{Csr}}_{1t}, \mathrm{AD\_SCHEDULE}$', rotation=0, fontweight='bold', fontsize=ylabel_size)
ax[i].set_yticks([])
ax[i].set_xticklabels([])
leg_items = [
    Line2D([0],[0],marker='o',color='w',markerfacecolor=colors[i]['BUSINESS_HOURS'],markersize=msi,label='BUSINESS_HOURS'),
    Line2D([0],[0],marker='o',color='w',markerfacecolor=colors[i]['AFTER_HOURS'],markersize=msi,label='AFTER_HOURS')]
ax[i].legend(handles=leg_items, bbox_to_anchor=(1.05, 0.5), loc='center left', borderaxespad=0, labelspacing=0.1)
ax[i].spines['top'].set_visible(False); ax[i].spines['right'].set_visible(False)

i = 3
y_pos = 0
for t, s in zip(range(_T), _sWeb_facs['sᶜˢʳ₂']): ## 'LOCATION_TARGET'
    ax[i].scatter(t, y_pos, color=colors[i][s], s=siz, label=s)
ax[i].set_ylabel('$s^{\mathrm{Csr}}_{2t}, \mathrm{LOCATION\_TARGET}$', rotation=0, fontweight='bold', fontsize=ylabel_size)
ax[i].set_yticks([])
leg_items = [
    Line2D([0],[0],marker='o',color='w',markerfacecolor=colors[i]['EAST'],markersize=msi,label='EAST'),
    Line2D([0],[0],marker='o',color='w',markerfacecolor=colors[i]['CENTRAL'],markersize=msi,label='CENTRAL'),
    Line2D([0],[0],marker='o',color='w',markerfacecolor=colors[i]['WEST'],markersize=msi,label='WEST')]
ax[i].legend(handles=leg_items, bbox_to_anchor=(1.05, 0.5), loc='center left', borderaxespad=0, labelspacing=0.1)
ax[i].spines['top'].set_visible(False); ax[i].spines['right'].set_visible(False)

i = 4
y_pos = 0
for t, s in zip(range(_T), _sWeb_facs['sᶜˢʳ₁']): ## 'AD_SCHEDULE'
    ax[i].scatter(t, y_pos, color=colors[i][s], s=siz)
ax[i].set_ylabel('$q(s)^{\mathrm{Csr}}_{1t}, \mathrm{AD\_SCHEDULE}$', rotation=0, fontweight='bold', fontsize=ylabel_size)
ax[i].set_yticks([])
ax[i].set_xticklabels([])
leg_items = [
    Line2D([0],[0],marker='o',color='w',markerfacecolor=colors[i]['BUSINESS_HOURS'],markersize=msi,label='BUSINESS_HOURS'),
    Line2D([0],[0],marker='o',color='w',markerfacecolor=colors[i]['AFTER_HOURS'],markersize=msi,label='AFTER_HOURS')]
ax[i].legend(handles=leg_items, bbox_to_anchor=(1.05, 0.5), loc='center left', borderaxespad=0, labelspacing=0.1)
ax[i].spines['top'].set_visible(False); ax[i].spines['right'].set_visible(False)

i = 5
y_pos = 0
for t, s in zip(range(_T), _sWeb_facs['sᶜˢʳ₂']): ## 'LOCATION_TARGET'
    ax[i].scatter(t, y_pos, color=colors[i][s], s=siz, label=s)
ax[i].set_ylabel('$q(s)^{\mathrm{Csr}}_{2t}, \mathrm{LOCATION\_TARGET}$', rotation=0, fontweight='bold', fontsize=ylabel_size)
ax[i].set_yticks([])
leg_items = [
    Line2D([0],[0],marker='o',color='w',markerfacecolor=colors[i]['EAST'],markersize=msi,label='EAST'),
    Line2D([0],[0],marker='o',color='w',markerfacecolor=colors[i]['CENTRAL'],markersize=msi,label='CENTRAL'),
    Line2D([0],[0],marker='o',color='w',markerfacecolor=colors[i]['WEST'],markersize=msi,label='WEST')]
ax[i].legend(handles=leg_items, bbox_to_anchor=(1.05, 0.5), loc='center left', borderaxespad=0, labelspacing=0.1)
ax[i].spines['top'].set_visible(False); ax[i].spines['right'].set_visible(False)

i = 6
y_pos = 0
for t, s in zip(range(_T), _yCsr_mods['yᶜˢʳ₁']): ## 'AD_SCHEDULE_OBS'
    ax[i].scatter(t, y_pos, color=colors[i][s], s=siz, label=s)
ax[i].set_ylabel('$y^{\mathrm{Csr}}_{1t}, \mathrm{AD\_SCHEDULE\_OBS}$', rotation=0, fontweight='bold', fontsize=ylabel_size)
ax[i].set_yticks([])
leg_items = [
    Line2D([0],[0],marker='o',color='w',markerfacecolor=colors[i]['BUSINESS_HOURS_OBS'],markersize=msi,label='BUSINESS_HOURS_OBS'),
    Line2D([0],[0],marker='o',color='w',markerfacecolor=colors[i]['OVERLAP_HOURS_OBS'],markersize=msi,label='OVERLAP_HOURS_OBS'),
    Line2D([0],[0],marker='o',color='w',markerfacecolor=colors[i]['AFTER_HOURS_OBS'],markersize=msi,label='AFTER_HOURS_OBS')]
ax[i].legend(handles=leg_items, bbox_to_anchor=(1.05, 0.5), loc='center left', borderaxespad=0, labelspacing=0.1)
ax[i].spines['top'].set_visible(False); ax[i].spines['right'].set_visible(False)

i = 7
y_pos = 0
for t, s in zip(range(_T), _yCsr_mods['yᶜˢʳ₂']): ## 'LOCATION_TARGET_OBS'
    ax[i].scatter(t, y_pos, color=colors[i][s], s=siz, label=s)
ax[i].set_ylabel('$y^{\mathrm{Csr}}_{2t}, \mathrm{LOCATION\_TARGET\_OBS}$', rotation=0, fontweight='bold', fontsize=ylabel_size)
ax[i].set_yticks([])
leg_items = [
    Line2D([0],[0],marker='o',color='w',markerfacecolor=colors[i]['EAST_OBS'],markersize=msi,label='EAST_OBS'),
    Line2D([0],[0],marker='o',color='w',markerfacecolor=colors[i]['CENTRAL_OBS'],markersize=msi,label='CENTRAL_OBS'),
    Line2D([0],[0],marker='o',color='w',markerfacecolor=colors[i]['WEST_OBS'],markersize=msi,label='WEST_OBS')]
ax[i].legend(handles=leg_items, bbox_to_anchor=(1.05, 0.5), loc='center left', borderaxespad=0, labelspacing=0.1)
ax[i].spines['top'].set_visible(False); ax[i].spines['right'].set_visible(False)

i = 8
y_pos = 0
for t, s in zip(range(_T), _yCsr_mods['yᶜˢʳ₃']): ## 'LANGUAGE_OBS'
    ax[i].scatter(t, y_pos, color=colors[i][s], s=siz, label=s)
ax[i].set_ylabel('$y^{\mathrm{Csr}}_{3t}, \mathrm{LANGUAGE\_OBS}$', rotation=0, fontweight='bold', fontsize=ylabel_size)
ax[i].set_yticks([])
leg_items = [
    Line2D([0],[0],marker='o',color='w',markerfacecolor=colors[i]['ENGLISH_OBS'],markersize=msi,label='ENGLISH_OBS'),
    Line2D([0],[0],marker='o',color='w',markerfacecolor=colors[i]['SPANISH_OBS'],markersize=msi,label='SPANISH_OBS'),
    Line2D([0],[0],marker='o',color='w',markerfacecolor=colors[i]['CHINESE_OBS'],markersize=msi,label='CHINESE_OBS')]
ax[i].legend(handles=leg_items, bbox_to_anchor=(1.05, 0.5), loc='center left', borderaxespad=0, labelspacing=0.1)
ax[i].spines['top'].set_visible(False); ax[i].spines['right'].set_visible(False)

ax[i].xaxis.set_major_locator(MaxNLocator(integer=True))
## ax[i].xaxis.set_major_locator(MaxNLocator(nbins=10, integer=True))
ax[i].set_xlabel('$\mathrm{time,}\ t$', fontweight='bold', fontsize=12)

plt.tight_layout()
plt.subplots_adjust(hspace=0.1) ## Adjust this value as needed
plt.show()