Visualizing Forney Factor Graphs when using RxInfer (Part 1)

In this project we lay a foundation for the visualization of Forney Factor Graphs (FFGs) when using RxInfer. Followup projects will refine the approach.

versioninfo() ## Julia version

Julia Version 1.10.0
Commit 3120989f39b (2023-12-25 18:01 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 12 × Intel(R) Core(TM) i7-8700B CPU @ 3.20GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-15.0.7 (ORCJIT, skylake)
  Threads: 1 on 12 virtual cores
Environment:
  JULIA_NUM_THREADS = 

import Pkg; Pkg.instantiate(); Pkg.activate()
# Pkg.add(Pkg.PackageSpec(;name="RxInfer", version="3.0.0"))
# Pkg.add(Pkg.PackageSpec(;name="Distributions", version="0.25.108"))
# Pkg.add(Pkg.PackageSpec(;name="GraphPlot", version="0.5.2"))
# Pkg.add(Pkg.PackageSpec(;name="Cairo", version="1.0.5"))
# Pkg.add(Pkg.PackageSpec(;name="Graphs", version="1.9.0"))
# Pkg.add(Pkg.PackageSpec(;name="MetaGraphsNext", version="0.7.0"))
# Pkg.add(Pkg.PackageSpec(;name="GraphPPL", version="4.0.2"))
# Pkg.add(Pkg.PackageSpec(;name="GraphViz", version="0.2.0"))
# Pkg.add(Pkg.PackageSpec(;name="Dictionaries", version="0.4.2"))
# Pkg.add(Pkg.PackageSpec(;name="Plots", version="1.40.4"))
# Pkg.add(Pkg.PackageSpec(;name="StableRNGs", version="1.0.2"))
# Pkg.add(Pkg.PackageSpec(;name="StatsPlots", version="0.15.7"))
# Pkg.add(Pkg.PackageSpec(;name="LaTeXStrings", version="1.3.1"))
# Pkg.add(Pkg.PackageSpec(;name="DataFrames", version="1.6.1"))
# Pkg.add(Pkg.PackageSpec(;name="CSV", version="0.10.14"))
# Pkg.add(Pkg.PackageSpec(;name="GLM", version="1.9.0"))

  Activating project at `~/.julia/environments/v1.10`

using RxInfer, Distributions, Random, GraphPlot, Cairo, Graphs, MetaGraphsNext, GraphPPL, GraphViz, Dictionaries, Plots, StableRNGs, LinearAlgebra, StatsPlots, LaTeXStrings, DataFrames, CSV, GLM

Pkg.status()

Status `~/.julia/environments/v1.10/Project.toml`
  [336ed68f] CSV v0.10.14
  [159f3aea] Cairo v1.0.5
  [a93c6f00] DataFrames v1.6.1
  [85a47980] Dictionaries v0.4.2
⌃ [31c24e10] Distributions v0.25.108
  [38e38edf] GLM v1.9.0
⌅ [b3f8163a] GraphPPL v4.0.2
⌃ [a2cc645c] GraphPlot v0.5.2
  [f526b714] GraphViz v0.2.0
⌃ [86223c79] Graphs v1.9.0
  [b964fa9f] LaTeXStrings v1.3.1
  [fa8bd995] MetaGraphsNext v0.7.0
  [91a5bcdd] Plots v1.40.4
⌃ [86711068] RxInfer v3.0.0
  [860ef19b] StableRNGs v1.0.2
  [f3b207a7] StatsPlots v0.15.7
Info Packages marked with ⌃ and ⌅ have new versions available. Those with ⌃ may be upgradable, but those with ⌅ are restricted by compatibility constraints from upgrading. To see why use `status --outdated`

Visualization code

This project was inspired by the work from

https://coda.io/d/RxInfer-2024-Active-Inference-Institute_ddtS-XZ4BJb/Developing-RxInfer-jl_sueIa#_lu1tw.

Some of the code has also been adopted/modified.

One of the distinguishing features of the RxInfer Julia package is that it uses reactive message passing (RMP). This means node updates in the factor graph happens asynchronously. A user usually tends to gloss over these details due to the increased complexity. When node updates happen in the more traditional way, i.e. synchronously (for example as with the PyMDP Python package), users tend to have the option of being aware of which node is currently being udpated. I is quite beneficial to conceptualize the workings of factor graphs in general and the visualization of the factor graph itself is often helpful. The existing visualization code is rather limited and that is the reason this project was undertaken. The ultimate goal is reach a visualization which conforms to the Constrained Forney Factor Graph (CFFG) as defined in

https://biaslab.github.io/pdf/arxiv/2306.08014.pdf

https://biaslab.github.io/pdf/arxiv/2306.02733.pdf

GraphPPL.jl exports the @model macro for model specification allowing the acceptance of a regular Julia function and then builds an FFG under the hood. This graph is bipartite where vertices belong to one of two sets: one for factors and one for variables. We will render these verices as follows:

• factor vertex is rendered by a rectangle
• variable vertex is renderer by a
  • point if connected to a single factor
  • double line if (typically) connected to multiple factors

For each of the examples in this project the following steps happen:

• A @model is created

• The @model is conditioned on some data

  • ususally a few points so that the visualization does not become unwieldy

• A RxInfer model is created (rxi_model)

• A GraphPPL model is obtained (gppl_model)

• A meta graph is obtained (meta_graph)

• The graph is rendered by the existing implementation (GraphPlot.gplot())

• Some vertex labels are inspected for the sake of interest

• The dot string is generated (generate_dot)

  • graph or digraph

  • layout using neato or dot

  • rank true or false

  • ranksep to separate the ranks

  • splines for spline, line, none, polyline, ortho, curved

  • param_prior_factor

  • paramvar

  • state_prior_factor

  • statevar

  • datafactor

  • datavar

  • busvars

  • deltas true/false

• The dot string is printed

• The graph is rendered with GraphViz (show_gv())

"""
Takes the DOT string given by generate_dot and 
executes it to produce a GraphViz visualisation. 
"""
function show_gv(dot_code_graph::String)
    eval(Meta.parse(dot_code_graph))
end

show_gv

"""
 Retain canonical chars of a label string.
"""
function retain_can(s::String)
    m = match(r"^[a-zA-Zγθs₀\*]*", s)
    return m.match
end

retain_can

"""
Given a user specified canonical factor label string, 
returns all the GraphPPL associated labels from the context's factor_nodes.
"""
function str_factors(gppl_model, factor::String)
        context = GraphPPL.getcontext(gppl_model)
        vec = []
        for (factor_ID, factor_label) in pairs(context.factor_nodes)
            ## println("$(factor_ID), $(factor_label)")
            ## println("$(typeof(factor_ID)), $(typeof(factor_label))")
            node_data = gppl_model[factor_label]
            ## println("$(node_data.properties.fform)\n")
            if retain_can(string(node_data.properties.fform)) == factor
                append!(vec, factor_label)
            end
        end
        result = join(string.(vec), "; ")
        return result
    end

str_factors

"""
Given a user specified canonical variable label string, 
returns all the GraphPPL associated labels from the context's individual_variables.
"""
function str_individual_vars(gppl_model, var::String)
    context = GraphPPL.getcontext(gppl_model)
    vec = []
    for (node_ID, node_label) in pairs(context.individual_variables)
        ## println("$(node_ID), $(node_label)")
        ## println("$(typeof(node_ID)), $(typeof(node_label))")
        if string(node_ID) == var
            append!(vec, node_label)
        end
    end
    result = join(string.(vec), "; ")
    return result
end

str_individual_vars

"""
Given a user specified canonical variable label string, 
returns all the GraphPPL associated labels from the context's vector_variables.
"""
function str_vector_vars(gppl_model, var::String)
    context = GraphPPL.getcontext(gppl_model)
    vec = []
    for (node_ID, node_label) in pairs(context.vector_variables)
        ## println("$(node_ID), $(node_label)")
        ## println("$(typeof(node_ID)), $(typeof(node_label))")
        if string(node_ID) == var
            append!(vec, node_label)
        end
    end
    result = join(string.(vec), "; ")
    return result
end

str_vector_vars

"""
Parses the input GraphPPL.Model and uses Base.IOBuffer to 
iteratively construct a Julia string, containing the DOT 
code for the GraphViz visualisation of the graph. This 
string is then returned. 
Currently isomorphic to the display given by GraphPlot.gplot.
"""
function generate_dot(; 
        Model::GraphPPL.Model, 
        digraph::Bool=false,
        font_size::Int, 
        edge_length::Float64 = 1.0, 
        layout::String="neato", 
        ranksep=1.0, 
        rank::Bool=false,
        splines="line",
     
        paramvar="θ", ## inferred param
        param_prior_factor="",

        statevar="s", ## inferred state
        state_prior_factor="",

        datafactor="", ## factors connecting to datavar
        datavar="x", 
     
        busvars=[], ## variables on a 'bus', usually connecting to more than 2 factors
        deltas::Bool=false, ## to add delta clamps as black squares
        width::Float64=10.0, height::Float64=10.0,
    )
    if digraph==true 
        edge_symbol = "->" 
        graph = "digraph"
    else 
        edge_symbol = "--" 
        graph = "graph"
    end
    meta_graph = Model.graph ## underlying MetaGraphsNext graph
    io_buffer = IOBuffer() ## Base.IOBuffer rather than successive string concatenation
    write(io_buffer, "dot\"\"\"\n$(graph) G {\n")
    write(io_buffer, "    layout=$(layout);\n")
    write(io_buffer, "    rankdir=TB;\n")
    # write(io_buffer, "    ranksep=.1;\n")
    write(io_buffer, "    ranksep=$(ranksep);\n")
    write(io_buffer, "    splines=$(splines);\n")
    write(io_buffer, "    size=\"$(width),$(height)!\";\n")
    write(io_buffer, ## write initial graph DOT code to the Base.IOBuffer
    """
        node [
            shape=circle, 
            fontsize=$(font_size), 
            ## style=filled, 
            ## fillcolor=lightgray, 
            ## margin=.02, 
            margin=.01, 
            ## width=.10, height=.10, //minimum size (inches), grows to contain label
            ## width=.10, height=.10, fixedsize=true, //fixed size (inches), does not grow to contain label
            width=.30, height=.30, fixedsize=true, //fixed size (inches), does not grow to contain label
            color=black,
            penwidth=0.2, 
        ]
        edge[
            color=black, 
            penwidth=0.2, 
            len=$(edge_length), 
        ];\n
    """)
    ## Vertices/Nodes
    for vertex in MetaGraphsNext.vertices(meta_graph)
        label = MetaGraphsNext.label_for(meta_graph, vertex)
        properties = Model[label].properties ## for distinguishing factor/variable nodes
        if isa(properties, GraphPPL.FactorNodeProperties) ## Factor node
            can_label = retain_can(string(properties.fform))
            ## if can_label == "*" can_label = "STAR" end
            if can_label == "*" 
                write(io_buffer, 
                """
                    \"$(label)\" [
                        label="*",
                        shape=box, 
                        style=filled, 
                        fillcolor=lightgray, 
                    ];\n
                """) 
            else
                write(io_buffer, 
                """
                    \"$(label)\" [
                        label=$(can_label),
                        shape=box, 
                        style=filled, 
                        fillcolor=lightgray, 
                    ];\n
                """) #.
            end
        elseif isa(properties, GraphPPL.VariableNodeProperties) ## Variable Node
            if string(properties.name) == paramvar #.
                can_label = string(Model[label].properties.name) #.
                write(io_buffer, "    \"$(label)\" [label=\"\", xlabel=$(can_label), shape=box, width=1.5, height=.01, fixedsize=true,];\n")
            elseif string(properties.name) == statevar #.
                can_label = string(properties.name)*"_"*string(properties.index) #.
                write(io_buffer, "    \"$(label)\" [label=\"\", xlabel=$(can_label), shape=box, width=1.5, height=.01, fixedsize=true,];\n")
            elseif string(Model[label].properties.name) in busvars #.
                can_label = string(properties.name) #.
                write(io_buffer, "    \"$(label)\" [label=\"\", xlabel=$(can_label), shape=box, width=1.5, height=.01, fixedsize=true,];\n")
            else
                if string(properties.name) == datavar #.
                    can_label = string(properties.name)*"_"*string(properties.index) #.
                    ## write(io_buffer, "    \"$(label)\" [label=\"\", xlabel=$(can_label), shape=circle, width=.01, height=.01, fixedsize=true];\n")
                    write(io_buffer, "    \"$(label)\" [label=$(can_label), shape=circle, width=.01, height=.01, fixedsize=true, color=none];\n")
                    if deltas==true
                        clamp_label = "clamp_"*can_label #.
                        write(io_buffer, "    \"$(clamp_label)\" [label=\"\", shape=box, style=filled, fillcolor=black, width=.05, height=.05, fixedsize=true,];\n") #.
                        write(io_buffer, "    \"$(label)\" $(edge_symbol) \"$(clamp_label)\";\n") #.CURRENT
                    end
                else
                    can_label = string(properties.name) #.
                    write(io_buffer, "    \"$(label)\" [label=$(can_label), shape=circle, width=.01, height=.01, fixedsize=true, color=none];\n") #.
                    if deltas==true
                        clamp_label = "clamp_"*can_label #.
                        write(io_buffer, "    \"$(clamp_label)\" [label=\"\", shape=box, style=filled, fillcolor=black, width=.05, height=.05, fixedsize=true,];\n") #.
                        write(io_buffer, "    \"$(label)\" $(edge_symbol) \"$(clamp_label)\";\n") #.CURRENT
                    end
                end
                ## connect each non-state variable node to a clamp
            end
        else
            println("ERROR: Unknown node type for label $(label)")
        end
    end
    ## Edges
    for edge in MetaGraphsNext.edges(meta_graph)
        source_vertex = MetaGraphsNext.label_for(meta_graph, edge.src)
        dest_vertex = MetaGraphsNext.label_for(meta_graph, edge.dst)
        ## write(io_buffer, "    \"$(source_vertex)\" -- \"$(dest_vertex)\" [len=$(edge_length)];\n")
        write(io_buffer, "    \"$(source_vertex)\" $(edge_symbol) \"$(dest_vertex)\";\n") #.CURRENT
    end
    if rank==true ## ranking with dot layout
        ## No layer for constvars, they will appear close to their connections

        ## Param layer 
        ## write(io_buffer, "subraph cluster_0 {rankdir=LR; rank=same; Beta_6; θ_1;};\n")
        ## write(io_buffer, "{ rank=same; $(get_factors(Model, infer_prior_factor)); $(get_individual_vars(Model, infervar)) };\n")
        if isempty(str_individual_vars(Model, paramvar))
            write(io_buffer, "{ rank=same; $(str_vector_vars(Model, paramvar)) };\n")
        else
            write(io_buffer, "{ rank=same; $(str_individual_vars(Model, paramvar)) };\n")
        end

        ## State layer 
        ## write(io_buffer, "subraph cluster_0 {rankdir=LR; rank=same; Beta_6; θ_1;};\n")
        ## write(io_buffer, "{ rank=same; $(get_factors(Model, infer_prior_factor)); $(get_individual_vars(Model, infervar)) };\n")
        write(io_buffer, "{ rank=same; $(str_vector_vars(Model, statevar)) };\n") #.CURRENT

        ## Data factor layer 
        write(io_buffer, "{ rank=same; $(str_factors(Model, datafactor)) };\n")
        
        ## Data layer
        write(io_buffer, "{ rank=max; $(str_vector_vars(Model, datavar)) };\n") #.CURRENT
        if deltas==true
            write(io_buffer, "{rank=max; clamp_y_1; clamp_y_2; clamp_y_3};\n")
        end

        ## Bus layer
        for i in busvars
            write(io_buffer, "{ rank=max; $(str_individual_vars(Model, i)) };\n")
        end
    end
    write(io_buffer, "}\n\"\"\"")
    final_dot_code = String(take!(io_buffer))
    return final_dot_code
end

generate_dot

Coin-toss Model

In this example, we are going to perform an exact inference for a coin-toss model that can be represented as:

\[\begin{aligned} p(\theta) &= \mathrm{Beta}(\theta|a, b),\\ p(y_i|\theta) &= \mathrm{Bern}(y_i|\theta),\\ \end{aligned}\]

where \(y_i \in \{0, 1\}\) is a binary observation induced by Bernoulli likelihood while \(\theta\) is a Beta prior distribution on the parameter of Bernoulli. We are interested in inferring the posterior distribution of \(\theta\).

The generative model is:

\[ \begin{aligned} p(\boldsymbol{y},\theta) &= p(\boldsymbol{y} \mid \theta) \cdot p(\theta) \\ &= \prod_{i=1}^N{p(y_i \mid \theta)} \cdot p(\theta) \\ &= \prod_{i=1}^N{\mathrm{Bern}(y_i \mid \theta)} \cdot \mathrm{Beta}(\theta \mid a,b) \end{aligned} \]

See Fig 4.1 in 2023_Bagaev_RPPfSBI_Thesis. So, \(N\) \(\delta\) factors, 1 Beta factor, \(N\) Ber factors.

@model function coin_model(y, a, b)
    ## We endow θ parameter of our model with some prior
    θ ~ Beta(a, b)

    ## We assume that outcome of each coin flip is governed by the Bernoulli distribution
    for i in eachindex(y)
        y[i] ~ Bernoulli(θ)
    end
end

coin_conditioned = coin_model(a = 2.0, b = 7.0) | (y = [ true, false, true ], )

coin_model(a = 2.0, b = 7.0) conditioned on: 
  y = Bool[1, 0, 1]

coin_rxi_model = RxInfer.create_model(coin_conditioned)

ProbabilisticModel{GraphPPL.Model{MetaGraph{Int64, SimpleGraph{Int64}, GraphPPL.NodeLabel, GraphPPL.NodeData, GraphPPL.EdgeLabel, GraphPPL.Context, MetaGraphsNext.var"#4#8", Float64}, GraphPPL.PluginsCollection{Tuple{}}, RxInfer.ReactiveMPGraphPPLBackend}}(GraphPPL.Model{MetaGraph{Int64, SimpleGraph{Int64}, GraphPPL.NodeLabel, GraphPPL.NodeData, GraphPPL.EdgeLabel, GraphPPL.Context, MetaGraphsNext.var"#4#8", Float64}, GraphPPL.PluginsCollection{Tuple{}}, RxInfer.ReactiveMPGraphPPLBackend}(Meta graph based on a SimpleGraph{Int64} with vertex labels of type GraphPPL.NodeLabel, vertex metadata of type GraphPPL.NodeData, edge metadata of type GraphPPL.EdgeLabel, graph metadata given by GraphPPL.Context(0, coin_model, "", nothing, {Beta = 1, Bernoulli = 3}, {}, {(Beta, 1) = Beta_6, (Bernoulli, 2) = Bernoulli_10, (Bernoulli, 3) = Bernoulli_12, (Bernoulli, 1) = Bernoulli_8}, {:constvar_2 = constvar_2_3, :θ = θ_1, :constvar_4 = constvar_4_5}, {:y = ResizableArray{GraphPPL.NodeLabel,1}(GraphPPL.NodeLabel[y_7, y_9, y_11])}, {}, {}, Base.RefValue{Any}(nothing)), and default weight 1.0, GraphPPL.PluginsCollection{Tuple{}}(()), RxInfer.ReactiveMPGraphPPLBackend(), Base.RefValue{Int64}(12)))

coin_gppl_model = RxInfer.getmodel(coin_rxi_model)

GraphPPL.Model{MetaGraph{Int64, SimpleGraph{Int64}, GraphPPL.NodeLabel, GraphPPL.NodeData, GraphPPL.EdgeLabel, GraphPPL.Context, MetaGraphsNext.var"#4#8", Float64}, GraphPPL.PluginsCollection{Tuple{}}, RxInfer.ReactiveMPGraphPPLBackend}(Meta graph based on a SimpleGraph{Int64} with vertex labels of type GraphPPL.NodeLabel, vertex metadata of type GraphPPL.NodeData, edge metadata of type GraphPPL.EdgeLabel, graph metadata given by GraphPPL.Context(0, coin_model, "", nothing, {Beta = 1, Bernoulli = 3}, {}, {(Beta, 1) = Beta_6, (Bernoulli, 2) = Bernoulli_10, (Bernoulli, 3) = Bernoulli_12, (Bernoulli, 1) = Bernoulli_8}, {:constvar_2 = constvar_2_3, :θ = θ_1, :constvar_4 = constvar_4_5}, {:y = ResizableArray{GraphPPL.NodeLabel,1}(GraphPPL.NodeLabel[y_7, y_9, y_11])}, {}, {}, Base.RefValue{Any}(nothing)), and default weight 1.0, GraphPPL.PluginsCollection{Tuple{}}(()), RxInfer.ReactiveMPGraphPPLBackend(), Base.RefValue{Int64}(12))

coin_meta_graph = coin_gppl_model.graph

Meta graph based on a SimpleGraph{Int64} with vertex labels of type GraphPPL.NodeLabel, vertex metadata of type GraphPPL.NodeData, edge metadata of type GraphPPL.EdgeLabel, graph metadata given by GraphPPL.Context(0, coin_model, "", nothing, {Beta = 1, Bernoulli = 3}, {}, {(Beta, 1) = Beta_6, (Bernoulli, 2) = Bernoulli_10, (Bernoulli, 3) = Bernoulli_12, (Bernoulli, 1) = Bernoulli_8}, {:constvar_2 = constvar_2_3, :θ = θ_1, :constvar_4 = constvar_4_5}, {:y = ResizableArray{GraphPPL.NodeLabel,1}(GraphPPL.NodeLabel[y_7, y_9, y_11])}, {}, {}, Base.RefValue{Any}(nothing)), and default weight 1.0

meta_graph = coin_meta_graph
GraphPlot.gplot( ## existing plotting functionality
    meta_graph,
    layout=spring_layout,
    nodelabel=collect(labels(meta_graph)),
    nodelabelsize=1.0,
    NODESIZE=0.12, ## diameter of the nodes
    nodelabelc="red", #.
    nodelabeldist=0.0,
    nodefillc="cyan",
)

for i in MetaGraphsNext.vertices(coin_meta_graph)
    label = MetaGraphsNext.label_for(coin_meta_graph, i)
    println("$i: $label")
end

1: θ_1
2: constvar_2_3
3: constvar_4_5
4: Beta_6
5: y_7
6: Bernoulli_8
7: y_9
8: Bernoulli_10
9: y_11
10: Bernoulli_12

## fieldnames(GraphPPL.Context)
coin_gppl_model_context = GraphPPL.getcontext(coin_gppl_model)
## coin_gppl_model_context.individual_variables
coin_gppl_model_context.vector_variables
## coin_gppl_model_context.factor_nodes

1-element UnorderedDictionary{Symbol, GraphPPL.ResizableArray{GraphPPL.NodeLabel, Vector{GraphPPL.NodeLabel}, 1}}
 :y │ ResizableArray{GraphPPL.NodeLabel,1}(GraphPPL.NodeLabel[y_7, y_9, y_11])

str_factors(coin_gppl_model, "Bernoulli")
str_individual_vars(coin_gppl_model, "θ")
str_individual_vars(coin_gppl_model, "constvar_2")
str_vector_vars(coin_gppl_model, "y")

"y_7; y_9; y_11"

coin_dot = generate_dot(
    Model = coin_gppl_model, 
    digraph = false,
    font_size = 4,
    edge_length = .5, #1.0,
    layout = "neato",
    # layout = "dot",
    rank = true,
    # rank = false,
    ranksep = .5,
    splines = "ortho", ## spline, line, none, polyline, ortho, curved
    
    param_prior_factor = "Beta",
    paramvar = "θ", #.

    # data_factors = "Bernoulli",
    datafactor = "Bernoulli",
    datavar = "y",
    
    busvars = [],
    
    # deltas = true,
    deltas = false,
    width = 6.0,
    height = 6.0
)
print(coin_dot) #.
show_gv(coin_dot)

dot"""
graph G {
    layout=neato;
    rankdir=TB;
    ranksep=0.5;
    splines=ortho;
    size="6.0,6.0!";
    node [
        shape=circle, 
        fontsize=4, 
        ## style=filled, 
        ## fillcolor=lightgray, 
        ## margin=.02, 
        margin=.01, 
        ## width=.10, height=.10, //minimum size (inches), grows to contain label
        ## width=.10, height=.10, fixedsize=true, //fixed size (inches), does not grow to contain label
        width=.30, height=.30, fixedsize=true, //fixed size (inches), does not grow to contain label
        color=black,
        penwidth=0.2, 
    ]
    edge[
        color=black, 
        penwidth=0.2, 
        len=0.5, 
    ];

    "θ_1" [label="", xlabel=θ, shape=box, width=1.5, height=.01, fixedsize=true,];
    "constvar_2_3" [label=constvar_2, shape=circle, width=.01, height=.01, fixedsize=true, color=none];
    "constvar_4_5" [label=constvar_4, shape=circle, width=.01, height=.01, fixedsize=true, color=none];
    "Beta_6" [
        label=Beta,
        shape=box, 
        style=filled, 
        fillcolor=lightgray, 
    ];

    "y_7" [label=y_1, shape=circle, width=.01, height=.01, fixedsize=true, color=none];
    "Bernoulli_8" [
        label=Bernoulli,
        shape=box, 
        style=filled, 
        fillcolor=lightgray, 
    ];

    "y_9" [label=y_2, shape=circle, width=.01, height=.01, fixedsize=true, color=none];
    "Bernoulli_10" [
        label=Bernoulli,
        shape=box, 
        style=filled, 
        fillcolor=lightgray, 
    ];

    "y_11" [label=y_3, shape=circle, width=.01, height=.01, fixedsize=true, color=none];
    "Bernoulli_12" [
        label=Bernoulli,
        shape=box, 
        style=filled, 
        fillcolor=lightgray, 
    ];

    "θ_1" -- "Beta_6";
    "θ_1" -- "Bernoulli_8";
    "θ_1" -- "Bernoulli_10";
    "θ_1" -- "Bernoulli_12";
    "constvar_2_3" -- "Beta_6";
    "constvar_4_5" -- "Beta_6";
    "y_7" -- "Bernoulli_8";
    "y_9" -- "Bernoulli_10";
    "y_11" -- "Bernoulli_12";
{ rank=same; θ_1 };
{ rank=same;  };
{ rank=same; Bernoulli_10; Bernoulli_12; Bernoulli_8 };
{ rank=max; y_7; y_9; y_11 };
}
"""

Warning: node 'constvar_2_3', graph 'G' size too small for label
Warning: node 'constvar_4_5', graph 'G' size too small for label
Warning: node 'Beta_6', graph 'G' size too small for label
Warning: node 'y_7', graph 'G' size too small for label
Warning: node 'Bernoulli_8', graph 'G' size too small for label
Warning: node 'y_9', graph 'G' size too small for label
Warning: node 'Bernoulli_10', graph 'G' size too small for label
Warning: node 'y_11', graph 'G' size too small for label
Warning: node 'Bernoulli_12', graph 'G' size too small for label

coin_dot = generate_dot(
    Model = coin_gppl_model, 
    digraph = false,
    font_size = 4,
    edge_length = .5, #1.0,
    # layout = "neato",
    layout = "dot",
    rank = true,
    ranksep = .5,
    # rank = false,
    splines = "ortho", ## spline, line, none, polyline, ortho, curved
    
    param_prior_factor = "Beta",
    paramvar = "θ", #.

    # data_factors = "Bernoulli",
    datafactor = "Bernoulli",
    datavar = "y",
    
    busvars = [],
    
    # deltas = true,
    deltas = false,
    width = 6.0,
    height = 6.0
)
print(coin_dot) #.
show_gv(coin_dot)

dot"""
graph G {
    layout=dot;
    rankdir=TB;
    ranksep=0.5;
    splines=ortho;
    size="6.0,6.0!";
    node [
        shape=circle, 
        fontsize=4, 
        ## style=filled, 
        ## fillcolor=lightgray, 
        ## margin=.02, 
        margin=.01, 
        ## width=.10, height=.10, //minimum size (inches), grows to contain label
        ## width=.10, height=.10, fixedsize=true, //fixed size (inches), does not grow to contain label
        width=.30, height=.30, fixedsize=true, //fixed size (inches), does not grow to contain label
        color=black,
        penwidth=0.2, 
    ]
    edge[
        color=black, 
        penwidth=0.2, 
        len=0.5, 
    ];

    "θ_1" [label="", xlabel=θ, shape=box, width=1.5, height=.01, fixedsize=true,];
    "constvar_2_3" [label=constvar_2, shape=circle, width=.01, height=.01, fixedsize=true, color=none];
    "constvar_4_5" [label=constvar_4, shape=circle, width=.01, height=.01, fixedsize=true, color=none];
    "Beta_6" [
        label=Beta,
        shape=box, 
        style=filled, 
        fillcolor=lightgray, 
    ];

    "y_7" [label=y_1, shape=circle, width=.01, height=.01, fixedsize=true, color=none];
    "Bernoulli_8" [
        label=Bernoulli,
        shape=box, 
        style=filled, 
        fillcolor=lightgray, 
    ];

    "y_9" [label=y_2, shape=circle, width=.01, height=.01, fixedsize=true, color=none];
    "Bernoulli_10" [
        label=Bernoulli,
        shape=box, 
        style=filled, 
        fillcolor=lightgray, 
    ];

    "y_11" [label=y_3, shape=circle, width=.01, height=.01, fixedsize=true, color=none];
    "Bernoulli_12" [
        label=Bernoulli,
        shape=box, 
        style=filled, 
        fillcolor=lightgray, 
    ];

    "θ_1" -- "Beta_6";
    "θ_1" -- "Bernoulli_8";
    "θ_1" -- "Bernoulli_10";
    "θ_1" -- "Bernoulli_12";
    "constvar_2_3" -- "Beta_6";
    "constvar_4_5" -- "Beta_6";
    "y_7" -- "Bernoulli_8";
    "y_9" -- "Bernoulli_10";
    "y_11" -- "Bernoulli_12";
{ rank=same; θ_1 };
{ rank=same;  };
{ rank=same; Bernoulli_10; Bernoulli_12; Bernoulli_8 };
{ rank=max; y_7; y_9; y_11 };
}
"""

Warning: node 'constvar_2_3', graph 'G' size too small for label
Warning: node 'constvar_4_5', graph 'G' size too small for label
Warning: node 'Beta_6', graph 'G' size too small for label
Warning: node 'y_7', graph 'G' size too small for label
Warning: node 'Bernoulli_8', graph 'G' size too small for label
Warning: node 'y_9', graph 'G' size too small for label
Warning: node 'Bernoulli_10', graph 'G' size too small for label
Warning: node 'y_11', graph 'G' size too small for label
Warning: node 'Bernoulli_12', graph 'G' size too small for label

Hidden Markov Model with Control (Part 1)

There are 1-hot encodings on all random variables. Using regular bold font to indicate the 1-hot encodings, and also setting \(t=1\), we have:

\[\begin{aligned} p(\boldsymbol{x}, \boldsymbol{s}) &\propto \underbrace{p(\mathbf s_0)}_{\substack{\text{Initial} \\ \text{state} \\ \text{prior}}} \prod_{k=1}^{T+1} \underbrace{p(\mathbf x_k \mid \mathbf s_k)}_{\substack{\text{Observation} \\ \text{model}}} \, \underbrace{p(\mathbf s_k \mid \mathbf s_{k-1}}_{\substack{\text{Transition} \\ \text{model}}}) \end{aligned}\]

where

\[\begin{aligned} p(\mathbf s_{0}) &= \mathcal{Cat}(\mathbf s_{0} ∣ \mathbf d) \\ p(\mathbf x_k ∣ \mathbf s_k) &= \mathcal{Cat}(\mathbf x_k ∣ \mathbf A \mathbf s_k) \\ p(\mathbf s_k ∣ \mathbf s_{k-1}) &= \mathcal{Cat}(\mathbf s_k ∣ \mathbf B \mathbf s_{k-1}) \\ \end{aligned}\]

@model function hidden_markov_model(x)
    B ~ MatrixDirichlet(ones(3, 3))
    A ~ MatrixDirichlet([10.0 1.0 1.0; 
                         1.0 10.0 1.0; 
                         1.0 1.0 10.0 ])    
    s₀ ~ Categorical(fill(1.0/3.0, 3))
    
    sₖ₋₁ = s₀
    for k in eachindex(x)
        s[k] ~ Transition(sₖ₋₁, B)
        x[k] ~ Transition(s[k], A)
        sₖ₋₁ = s[k]
    end
end

hmm_conditioned = hidden_markov_model() | (x = [[1.0, 0.0, 0.0], [0.0, 0.0, 1.0]],)

hidden_markov_model() conditioned on: 
  x = [[1.0, 0.0, 0.0], [0.0, 0.0, 1.0]]

hmm_rxi_model = RxInfer.create_model(hmm_conditioned)

ProbabilisticModel{GraphPPL.Model{MetaGraph{Int64, SimpleGraph{Int64}, GraphPPL.NodeLabel, GraphPPL.NodeData, GraphPPL.EdgeLabel, GraphPPL.Context, MetaGraphsNext.var"#4#8", Float64}, GraphPPL.PluginsCollection{Tuple{}}, RxInfer.ReactiveMPGraphPPLBackend}}(GraphPPL.Model{MetaGraph{Int64, SimpleGraph{Int64}, GraphPPL.NodeLabel, GraphPPL.NodeData, GraphPPL.EdgeLabel, GraphPPL.Context, MetaGraphsNext.var"#4#8", Float64}, GraphPPL.PluginsCollection{Tuple{}}, RxInfer.ReactiveMPGraphPPLBackend}(Meta graph based on a SimpleGraph{Int64} with vertex labels of type GraphPPL.NodeLabel, vertex metadata of type GraphPPL.NodeData, edge metadata of type GraphPPL.EdgeLabel, graph metadata given by GraphPPL.Context(0, hidden_markov_model, "", nothing, {MatrixDirichlet = 2, Transition = 4, Categorical{P} where P<:Real = 1}, {}, {(Transition, 4) = Transition_20, (MatrixDirichlet, 1) = MatrixDirichlet_4, (Categorical{P} where P<:Real, 1) = Categorical{P} where P<:Real_12, (MatrixDirichlet, 2) = MatrixDirichlet_8, (Transition, 3) = Transition_18, (Transition, 1) = Transition_14, (Transition, 2) = Transition_16}, {:A = A_5, :s₀ = s₀_9, :B = B_1, :constvar_6 = constvar_6_7, :constvar_10 = constvar_10_11, :constvar_2 = constvar_2_3}, {:s = ResizableArray{GraphPPL.NodeLabel,1}(GraphPPL.NodeLabel[s_13, s_17]), :x = ResizableArray{GraphPPL.NodeLabel,1}(GraphPPL.NodeLabel[x_15, x_19])}, {}, {}, Base.RefValue{Any}(nothing)), and default weight 1.0, GraphPPL.PluginsCollection{Tuple{}}(()), RxInfer.ReactiveMPGraphPPLBackend(), Base.RefValue{Int64}(20)))

hmm_gppl_model = RxInfer.getmodel(hmm_rxi_model)

GraphPPL.Model{MetaGraph{Int64, SimpleGraph{Int64}, GraphPPL.NodeLabel, GraphPPL.NodeData, GraphPPL.EdgeLabel, GraphPPL.Context, MetaGraphsNext.var"#4#8", Float64}, GraphPPL.PluginsCollection{Tuple{}}, RxInfer.ReactiveMPGraphPPLBackend}(Meta graph based on a SimpleGraph{Int64} with vertex labels of type GraphPPL.NodeLabel, vertex metadata of type GraphPPL.NodeData, edge metadata of type GraphPPL.EdgeLabel, graph metadata given by GraphPPL.Context(0, hidden_markov_model, "", nothing, {MatrixDirichlet = 2, Transition = 4, Categorical{P} where P<:Real = 1}, {}, {(Transition, 4) = Transition_20, (MatrixDirichlet, 1) = MatrixDirichlet_4, (Categorical{P} where P<:Real, 1) = Categorical{P} where P<:Real_12, (MatrixDirichlet, 2) = MatrixDirichlet_8, (Transition, 3) = Transition_18, (Transition, 1) = Transition_14, (Transition, 2) = Transition_16}, {:A = A_5, :s₀ = s₀_9, :B = B_1, :constvar_6 = constvar_6_7, :constvar_10 = constvar_10_11, :constvar_2 = constvar_2_3}, {:s = ResizableArray{GraphPPL.NodeLabel,1}(GraphPPL.NodeLabel[s_13, s_17]), :x = ResizableArray{GraphPPL.NodeLabel,1}(GraphPPL.NodeLabel[x_15, x_19])}, {}, {}, Base.RefValue{Any}(nothing)), and default weight 1.0, GraphPPL.PluginsCollection{Tuple{}}(()), RxInfer.ReactiveMPGraphPPLBackend(), Base.RefValue{Int64}(20))

hmm_meta_graph = hmm_gppl_model.graph

Meta graph based on a SimpleGraph{Int64} with vertex labels of type GraphPPL.NodeLabel, vertex metadata of type GraphPPL.NodeData, edge metadata of type GraphPPL.EdgeLabel, graph metadata given by GraphPPL.Context(0, hidden_markov_model, "", nothing, {MatrixDirichlet = 2, Transition = 4, Categorical{P} where P<:Real = 1}, {}, {(Transition, 4) = Transition_20, (MatrixDirichlet, 1) = MatrixDirichlet_4, (Categorical{P} where P<:Real, 1) = Categorical{P} where P<:Real_12, (MatrixDirichlet, 2) = MatrixDirichlet_8, (Transition, 3) = Transition_18, (Transition, 1) = Transition_14, (Transition, 2) = Transition_16}, {:A = A_5, :s₀ = s₀_9, :B = B_1, :constvar_6 = constvar_6_7, :constvar_10 = constvar_10_11, :constvar_2 = constvar_2_3}, {:s = ResizableArray{GraphPPL.NodeLabel,1}(GraphPPL.NodeLabel[s_13, s_17]), :x = ResizableArray{GraphPPL.NodeLabel,1}(GraphPPL.NodeLabel[x_15, x_19])}, {}, {}, Base.RefValue{Any}(nothing)), and default weight 1.0

meta_graph = hmm_meta_graph
GraphPlot.gplot( ## existing plotting functionality
    meta_graph,
    layout=spring_layout,
    nodelabel=collect(labels(meta_graph)),
    nodelabelsize=1.0,
    NODESIZE=0.07, ## diameter of the nodes
    nodelabelc="red", #.
    nodelabeldist=0.0,
    nodefillc="cyan",
)

(process:552): Pango-WARNING **: 20:54:19.753: pango_layout_set_markup_with_accel: Error on line 1 char 42: “:Real_12<” is not a valid name: “<”

for i in MetaGraphsNext.vertices(hmm_meta_graph)
    label = MetaGraphsNext.label_for(hmm_meta_graph, i)
    println("$i: $label")
end

1: B_1
2: constvar_2_3
3: MatrixDirichlet_4
4: A_5
5: constvar_6_7
6: MatrixDirichlet_8
7: s₀_9
8: constvar_10_11
9: Categorical{P} where P<:Real_12
10: s_13
11: Transition_14
12: x_15
13: Transition_16
14: s_17
15: Transition_18
16: x_19
17: Transition_20

## fieldnames(GraphPPL.Context)
hmm_gppl_model_context = GraphPPL.getcontext(hmm_gppl_model)
hmm_gppl_model_context.individual_variables
## hmm_gppl_model_context.vector_variables
## hmm_gppl_model_context.factor_nodes

6-element UnorderedDictionary{Symbol, GraphPPL.NodeLabel}
           :A │ A_5
          :s₀ │ s₀_9
           :B │ B_1
  :constvar_6 │ constvar_6_7
 :constvar_10 │ constvar_10_11
  :constvar_2 │ constvar_2_3

str_factors(hmm_gppl_model, "Categorical")
str_individual_vars(hmm_gppl_model, "s₀")
str_individual_vars(hmm_gppl_model, "constvar_6")
str_vector_vars(hmm_gppl_model, "s")

"s_13; s_17"

hmm_dot = generate_dot(
    Model = hmm_gppl_model, 
    digraph = false,
    font_size = 7,
    edge_length = .8,
    layout = "neato",
    # layout = "dot",
    rank = true,
    # rank = false,
    ranksep = .5,
    splines = "polyline", ## spline, line, none, polyline, ortho, curved

    ## param_prior_factor = "Categorical",
    ## paramvar = "s",

    state_prior_factor = "Categorical",
    statevar = "s",

    datafactor = "Transition",
    datavar = "x",

    busvars = ["A", "B"],
    deltas = false,
    # deltas = true,
    width = 6.0,
    height = 12.0
)
print(hmm_dot) #.
show_gv(hmm_dot)

dot"""
graph G {
    layout=neato;
    rankdir=TB;
    ranksep=0.5;
    splines=polyline;
    size="6.0,12.0!";
    node [
        shape=circle, 
        fontsize=7, 
        ## style=filled, 
        ## fillcolor=lightgray, 
        ## margin=.02, 
        margin=.01, 
        ## width=.10, height=.10, //minimum size (inches), grows to contain label
        ## width=.10, height=.10, fixedsize=true, //fixed size (inches), does not grow to contain label
        width=.30, height=.30, fixedsize=true, //fixed size (inches), does not grow to contain label
        color=black,
        penwidth=0.2, 
    ]
    edge[
        color=black, 
        penwidth=0.2, 
        len=0.8, 
    ];

    "B_1" [label="", xlabel=B, shape=box, width=1.5, height=.01, fixedsize=true,];
    "constvar_2_3" [label=constvar_2, shape=circle, width=.01, height=.01, fixedsize=true, color=none];
    "MatrixDirichlet_4" [
        label=MatrixDirichlet,
        shape=box, 
        style=filled, 
        fillcolor=lightgray, 
    ];

    "A_5" [label="", xlabel=A, shape=box, width=1.5, height=.01, fixedsize=true,];
    "constvar_6_7" [label=constvar_6, shape=circle, width=.01, height=.01, fixedsize=true, color=none];
    "MatrixDirichlet_8" [
        label=MatrixDirichlet,
        shape=box, 
        style=filled, 
        fillcolor=lightgray, 
    ];

    "s₀_9" [label=s₀, shape=circle, width=.01, height=.01, fixedsize=true, color=none];
    "constvar_10_11" [label=constvar_10, shape=circle, width=.01, height=.01, fixedsize=true, color=none];
    "Categorical{P} where P<:Real_12" [
        label=Categorical,
        shape=box, 
        style=filled, 
        fillcolor=lightgray, 
    ];

    "s_13" [label="", xlabel=s_1, shape=box, width=1.5, height=.01, fixedsize=true,];
    "Transition_14" [
        label=Transition,
        shape=box, 
        style=filled, 
        fillcolor=lightgray, 
    ];

    "x_15" [label=x_1, shape=circle, width=.01, height=.01, fixedsize=true, color=none];
    "Transition_16" [
        label=Transition,
        shape=box, 
        style=filled, 
        fillcolor=lightgray, 
    ];

    "s_17" [label="", xlabel=s_2, shape=box, width=1.5, height=.01, fixedsize=true,];
    "Transition_18" [
        label=Transition,
        shape=box, 
        style=filled, 
        fillcolor=lightgray, 
    ];

    "x_19" [label=x_2, shape=circle, width=.01, height=.01, fixedsize=true, color=none];
    "Transition_20" [
        label=Transition,
        shape=box, 
        style=filled, 
        fillcolor=lightgray, 
    ];

    "B_1" -- "MatrixDirichlet_4";
    "B_1" -- "Transition_14";
    "B_1" -- "Transition_18";
    "constvar_2_3" -- "MatrixDirichlet_4";
    "A_5" -- "MatrixDirichlet_8";
    "A_5" -- "Transition_16";
    "A_5" -- "Transition_20";
    "constvar_6_7" -- "MatrixDirichlet_8";
    "s₀_9" -- "Categorical{P} where P<:Real_12";
    "s₀_9" -- "Transition_14";
    "constvar_10_11" -- "Categorical{P} where P<:Real_12";
    "s_13" -- "Transition_14";
    "s_13" -- "Transition_16";
    "s_13" -- "Transition_18";
    "x_15" -- "Transition_16";
    "s_17" -- "Transition_18";
    "s_17" -- "Transition_20";
    "x_19" -- "Transition_20";
{ rank=same;  };
{ rank=same; s_13; s_17 };
{ rank=same; Transition_20; Transition_18; Transition_14; Transition_16 };
{ rank=max; x_15; x_19 };
{ rank=max; A_5 };
{ rank=max; B_1 };
}
"""

Warning: node 'constvar_2_3', graph 'G' size too small for label
Warning: node 'MatrixDirichlet_4', graph 'G' size too small for label
Warning: node 'constvar_6_7', graph 'G' size too small for label
Warning: node 'MatrixDirichlet_8', graph 'G' size too small for label
Warning: node 's₀_9', graph 'G' size too small for label
Warning: node 'constvar_10_11', graph 'G' size too small for label
Warning: node 'Categorical{P} where P<:Real_12', graph 'G' size too small for label
Warning: node 'Transition_14', graph 'G' size too small for label
Warning: node 'x_15', graph 'G' size too small for label
Warning: node 'Transition_16', graph 'G' size too small for label
Warning: node 'Transition_18', graph 'G' size too small for label
Warning: node 'x_19', graph 'G' size too small for label
Warning: node 'Transition_20', graph 'G' size too small for label
Warning: some nodes with margin (3.20,3.20) touch - falling back to straight line edges

hmm_dot = generate_dot(
    Model = hmm_gppl_model, 
    digraph = false,
    font_size = 7,
    edge_length = .8,
    # layout = "neato",
    layout = "dot",
    rank = true,
    # rank = false,
    ranksep = 1.,
    splines = "ortho", ## spline, line, none, polyline, ortho, curved

    ## param_prior_factor = "Categorical",
    ## paramvar = "s",

    state_prior_factor = "Categorical",
    statevar = "s",

    datafactor = "Transition",
    datavar = "x",

    busvars = ["A", "B"],
    deltas = false,
    # deltas = true,
    width = 6.0,
    height = 12.0
)
print(hmm_dot) #.
show_gv(hmm_dot)

dot"""
graph G {
    layout=dot;
    rankdir=TB;
    ranksep=1.0;
    splines=ortho;
    size="6.0,12.0!";
    node [
        shape=circle, 
        fontsize=7, 
        ## style=filled, 
        ## fillcolor=lightgray, 
        ## margin=.02, 
        margin=.01, 
        ## width=.10, height=.10, //minimum size (inches), grows to contain label
        ## width=.10, height=.10, fixedsize=true, //fixed size (inches), does not grow to contain label
        width=.30, height=.30, fixedsize=true, //fixed size (inches), does not grow to contain label
        color=black,
        penwidth=0.2, 
    ]
    edge[
        color=black, 
        penwidth=0.2, 
        len=0.8, 
    ];

    "B_1" [label="", xlabel=B, shape=box, width=1.5, height=.01, fixedsize=true,];
    "constvar_2_3" [label=constvar_2, shape=circle, width=.01, height=.01, fixedsize=true, color=none];
    "MatrixDirichlet_4" [
        label=MatrixDirichlet,
        shape=box, 
        style=filled, 
        fillcolor=lightgray, 
    ];

    "A_5" [label="", xlabel=A, shape=box, width=1.5, height=.01, fixedsize=true,];
    "constvar_6_7" [label=constvar_6, shape=circle, width=.01, height=.01, fixedsize=true, color=none];
    "MatrixDirichlet_8" [
        label=MatrixDirichlet,
        shape=box, 
        style=filled, 
        fillcolor=lightgray, 
    ];

    "s₀_9" [label=s₀, shape=circle, width=.01, height=.01, fixedsize=true, color=none];
    "constvar_10_11" [label=constvar_10, shape=circle, width=.01, height=.01, fixedsize=true, color=none];
    "Categorical{P} where P<:Real_12" [
        label=Categorical,
        shape=box, 
        style=filled, 
        fillcolor=lightgray, 
    ];

    "s_13" [label="", xlabel=s_1, shape=box, width=1.5, height=.01, fixedsize=true,];
    "Transition_14" [
        label=Transition,
        shape=box, 
        style=filled, 
        fillcolor=lightgray, 
    ];

    "x_15" [label=x_1, shape=circle, width=.01, height=.01, fixedsize=true, color=none];
    "Transition_16" [
        label=Transition,
        shape=box, 
        style=filled, 
        fillcolor=lightgray, 
    ];

    "s_17" [label="", xlabel=s_2, shape=box, width=1.5, height=.01, fixedsize=true,];
    "Transition_18" [
        label=Transition,
        shape=box, 
        style=filled, 
        fillcolor=lightgray, 
    ];

    "x_19" [label=x_2, shape=circle, width=.01, height=.01, fixedsize=true, color=none];
    "Transition_20" [
        label=Transition,
        shape=box, 
        style=filled, 
        fillcolor=lightgray, 
    ];

    "B_1" -- "MatrixDirichlet_4";
    "B_1" -- "Transition_14";
    "B_1" -- "Transition_18";
    "constvar_2_3" -- "MatrixDirichlet_4";
    "A_5" -- "MatrixDirichlet_8";
    "A_5" -- "Transition_16";
    "A_5" -- "Transition_20";
    "constvar_6_7" -- "MatrixDirichlet_8";
    "s₀_9" -- "Categorical{P} where P<:Real_12";
    "s₀_9" -- "Transition_14";
    "constvar_10_11" -- "Categorical{P} where P<:Real_12";
    "s_13" -- "Transition_14";
    "s_13" -- "Transition_16";
    "s_13" -- "Transition_18";
    "x_15" -- "Transition_16";
    "s_17" -- "Transition_18";
    "s_17" -- "Transition_20";
    "x_19" -- "Transition_20";
{ rank=same;  };
{ rank=same; s_13; s_17 };
{ rank=same; Transition_20; Transition_18; Transition_14; Transition_16 };
{ rank=max; x_15; x_19 };
{ rank=max; A_5 };
{ rank=max; B_1 };
}
"""

Warning: node 'constvar_2_3', graph 'G' size too small for label
Warning: node 'MatrixDirichlet_4', graph 'G' size too small for label
Warning: node 'constvar_6_7', graph 'G' size too small for label
Warning: node 'MatrixDirichlet_8', graph 'G' size too small for label
Warning: node 's₀_9', graph 'G' size too small for label
Warning: node 'constvar_10_11', graph 'G' size too small for label
Warning: node 'Categorical{P} where P<:Real_12', graph 'G' size too small for label
Warning: node 'Transition_14', graph 'G' size too small for label
Warning: node 'x_15', graph 'G' size too small for label
Warning: node 'Transition_16', graph 'G' size too small for label
Warning: node 'Transition_18', graph 'G' size too small for label
Warning: node 'x_19', graph 'G' size too small for label
Warning: node 'Transition_20', graph 'G' size too small for label

Sales Forecasting with Time-Varying Autoregressive Models (Part 4)

Our model for the Variational Bayesian Inference of a latent autoregressive model can be represented mathematically as follows:

\[\begin{aligned} p(\gamma) &= \mathcal{Gam}(\gamma|a, b)\\ p(\boldsymbol{\theta}) &= \mathcal{N}(\boldsymbol{\theta}|\mathbf{m}_{\theta}, \mathbf{V}_{\theta})\\ p(\mathbf{s}_t \mid \mathbf{s}_{t-1}, \boldsymbol{\theta}) &= \mathcal{N}\left(\mathbf{s}_t \mid \mathbf{B}\left(\boldsymbol{\theta}\right) \mathbf{s}_{t-1},\, \mathbf{V}(\gamma)\right)\\ p(x_t \mid \mathbf{s}_t) &= \mathcal{N}(x_t \mid \mathbf{s}_t, \tau^{-1}) \end{aligned}\]

where

• \(M^{AR}\) is the order of the autoregression process

• \(\boldsymbol{\theta}=(\theta_1, \theta_2, ..., \theta_{M^{AR}})^{\top}\) is a vector of transition coefficients

• \(\mathbf{s}_{t-1} = (s_{t-1}, s_{t-2}, ..., s_{t-M^{AR}})^\top\) is the previous state

• \(\mathbf{s}_t = (s_{t}, s_{t-1}, ..., s_{t-M^{AR}+1})^\top\) is the current state

• \(\gamma\) is the transition precision

• \(\tau\) is the observation precision

• \(\mathbf{v}_u = (1,0,...,0)^\top\) is an \(M^{AR}\)-dimensional unit vector selecting the first component of the vector that follows

• \(\mathbf{V}(\gamma) = (1/ \gamma)\mathbf{v}_u\mathbf{v}_u^\top\) is the transition covariance

• \(\mathbf{B}(\boldsymbol{\theta}) = \begin{bmatrix} \boldsymbol{\qquad\theta}^\top \\ \mathbf{I}_{M^{AR}-1} & \mathbf{0} \end{bmatrix}\) is the transition matrix

The following diagram represents an autoregressive process:

Autoregressive process

@model function lar_model(
    x, ##. data/observations 
    𝚃ᴬᴿ, ##. Uni/Multi variate 
    Mᴬᴿ, ##. AR order
    vᵤ, ##. unit vector 
    τ) ##. observation precision     
        ## Priors
        γ  ~ Gamma(α = 1.0, β = 1.0) ##. for transition precision    
        if 𝚃ᴬᴿ === Multivariate
            θ  ~ MvNormal(μ = zeros(Mᴬᴿ), Λ = diageye(Mᴬᴿ)) ##.kw μ,Λ only work inside macro
            s₀ ~ MvNormal(μ = zeros(Mᴬᴿ), Λ = diageye(Mᴬᴿ)) ##.kw μ,Λ only work inside macro
        else ## Univariate
            θ  ~ Normal(μ = 0.0, γ = 1.0)
            s₀ ~ Normal(μ = 0.0, γ = 1.0)
        end
        sₜ₋₁ = s₀
        for t in eachindex(x)
            s[t] ~ AR(sₜ₋₁, θ, γ) #.Eq (2b)
            if 𝚃ᴬᴿ === Multivariate
                x[t] ~ Normal(μ = dot(vᵤ, s[t]), γ = τ) #.Eq (2c)
            else
                x[t] ~ Normal(μ = vᵤ*s[t], γ = τ) #.Eq (2c)
            end
            sₜ₋₁ = s[t]
        end
    end

𝚃ᴬᴿ = Univariate
m = 1
τ̃ = 0.001 ## assumed observation precision
lar_conditioned = lar_model(
    𝚃ᴬᴿ=𝚃ᴬᴿ, 
    Mᴬᴿ=m, 
    vᵤ=ReactiveMP.ar_unit(𝚃ᴬᴿ, m), 
    τ=τ̃
) | (x = [266.0, 145.0, 183.0],)

lar_model(𝚃ᴬᴿ = Univariate, Mᴬᴿ = 1, vᵤ = 1.0, τ = 0.001) conditioned on: 
  x = [266.0, 145.0, 183.0]

lar_rxi_model = RxInfer.create_model(lar_conditioned)

ProbabilisticModel{GraphPPL.Model{MetaGraph{Int64, SimpleGraph{Int64}, GraphPPL.NodeLabel, GraphPPL.NodeData, GraphPPL.EdgeLabel, GraphPPL.Context, MetaGraphsNext.var"#4#8", Float64}, GraphPPL.PluginsCollection{Tuple{}}, RxInfer.ReactiveMPGraphPPLBackend}}(GraphPPL.Model{MetaGraph{Int64, SimpleGraph{Int64}, GraphPPL.NodeLabel, GraphPPL.NodeData, GraphPPL.EdgeLabel, GraphPPL.Context, MetaGraphsNext.var"#4#8", Float64}, GraphPPL.PluginsCollection{Tuple{}}, RxInfer.ReactiveMPGraphPPLBackend}(Meta graph based on a SimpleGraph{Int64} with vertex labels of type GraphPPL.NodeLabel, vertex metadata of type GraphPPL.NodeData, edge metadata of type GraphPPL.EdgeLabel, graph metadata given by GraphPPL.Context(0, lar_model, "", nothing, {AR = 3, * = 3, GammaShapeRate = 1, NormalMeanPrecision = 5}, {}, {(*, 2) = *_34, (NormalMeanPrecision, 4) = NormalMeanPrecision_38, (NormalMeanPrecision, 5) = NormalMeanPrecision_48, (GammaShapeRate, 1) = GammaShapeRate_6, (AR, 1) = AR_20, (NormalMeanPrecision, 1) = NormalMeanPrecision_12, (NormalMeanPrecision, 3) = NormalMeanPrecision_28, (NormalMeanPrecision, 2) = NormalMeanPrecision_18, (AR, 2) = AR_30, (AR, 3) = AR_40, (*, 1) = *_24, (*, 3) = *_44}, {:γ = γ_1, :constvar_42 = constvar_42_43, :constvar_14 = constvar_14_15, :constvar_16 = constvar_16_17, :anonymous_var_graphppl = anonymous_var_graphppl_41, :constvar_22 = constvar_22_23, :constvar_26 = constvar_26_27, :s₀ = s₀_13, :constvar_32 = constvar_32_33, :constvar_36 = constvar_36_37, :constvar_46 = constvar_46_47, :constvar_8 = constvar_8_9, :constvar_2 = constvar_2_3, :θ = θ_7, :constvar_4 = constvar_4_5, :constvar_10 = constvar_10_11}, {:s = ResizableArray{GraphPPL.NodeLabel,1}(GraphPPL.NodeLabel[s_19, s_29, s_39]), :x = ResizableArray{GraphPPL.NodeLabel,1}(GraphPPL.NodeLabel[x_25, x_35, x_45])}, {}, {}, Base.RefValue{Any}(nothing)), and default weight 1.0, GraphPPL.PluginsCollection{Tuple{}}(()), RxInfer.ReactiveMPGraphPPLBackend(), Base.RefValue{Int64}(48)))

lar_gppl_model = RxInfer.getmodel(lar_rxi_model)

GraphPPL.Model{MetaGraph{Int64, SimpleGraph{Int64}, GraphPPL.NodeLabel, GraphPPL.NodeData, GraphPPL.EdgeLabel, GraphPPL.Context, MetaGraphsNext.var"#4#8", Float64}, GraphPPL.PluginsCollection{Tuple{}}, RxInfer.ReactiveMPGraphPPLBackend}(Meta graph based on a SimpleGraph{Int64} with vertex labels of type GraphPPL.NodeLabel, vertex metadata of type GraphPPL.NodeData, edge metadata of type GraphPPL.EdgeLabel, graph metadata given by GraphPPL.Context(0, lar_model, "", nothing, {AR = 3, * = 3, GammaShapeRate = 1, NormalMeanPrecision = 5}, {}, {(*, 2) = *_34, (NormalMeanPrecision, 4) = NormalMeanPrecision_38, (NormalMeanPrecision, 5) = NormalMeanPrecision_48, (GammaShapeRate, 1) = GammaShapeRate_6, (AR, 1) = AR_20, (NormalMeanPrecision, 1) = NormalMeanPrecision_12, (NormalMeanPrecision, 3) = NormalMeanPrecision_28, (NormalMeanPrecision, 2) = NormalMeanPrecision_18, (AR, 2) = AR_30, (AR, 3) = AR_40, (*, 1) = *_24, (*, 3) = *_44}, {:γ = γ_1, :constvar_42 = constvar_42_43, :constvar_14 = constvar_14_15, :constvar_16 = constvar_16_17, :anonymous_var_graphppl = anonymous_var_graphppl_41, :constvar_22 = constvar_22_23, :constvar_26 = constvar_26_27, :s₀ = s₀_13, :constvar_32 = constvar_32_33, :constvar_36 = constvar_36_37, :constvar_46 = constvar_46_47, :constvar_8 = constvar_8_9, :constvar_2 = constvar_2_3, :θ = θ_7, :constvar_4 = constvar_4_5, :constvar_10 = constvar_10_11}, {:s = ResizableArray{GraphPPL.NodeLabel,1}(GraphPPL.NodeLabel[s_19, s_29, s_39]), :x = ResizableArray{GraphPPL.NodeLabel,1}(GraphPPL.NodeLabel[x_25, x_35, x_45])}, {}, {}, Base.RefValue{Any}(nothing)), and default weight 1.0, GraphPPL.PluginsCollection{Tuple{}}(()), RxInfer.ReactiveMPGraphPPLBackend(), Base.RefValue{Int64}(48))

lar_meta_graph = lar_gppl_model.graph

Meta graph based on a SimpleGraph{Int64} with vertex labels of type GraphPPL.NodeLabel, vertex metadata of type GraphPPL.NodeData, edge metadata of type GraphPPL.EdgeLabel, graph metadata given by GraphPPL.Context(0, lar_model, "", nothing, {AR = 3, * = 3, GammaShapeRate = 1, NormalMeanPrecision = 5}, {}, {(*, 2) = *_34, (NormalMeanPrecision, 4) = NormalMeanPrecision_38, (NormalMeanPrecision, 5) = NormalMeanPrecision_48, (GammaShapeRate, 1) = GammaShapeRate_6, (AR, 1) = AR_20, (NormalMeanPrecision, 1) = NormalMeanPrecision_12, (NormalMeanPrecision, 3) = NormalMeanPrecision_28, (NormalMeanPrecision, 2) = NormalMeanPrecision_18, (AR, 2) = AR_30, (AR, 3) = AR_40, (*, 1) = *_24, (*, 3) = *_44}, {:γ = γ_1, :constvar_42 = constvar_42_43, :constvar_14 = constvar_14_15, :constvar_16 = constvar_16_17, :anonymous_var_graphppl = anonymous_var_graphppl_41, :constvar_22 = constvar_22_23, :constvar_26 = constvar_26_27, :s₀ = s₀_13, :constvar_32 = constvar_32_33, :constvar_36 = constvar_36_37, :constvar_46 = constvar_46_47, :constvar_8 = constvar_8_9, :constvar_2 = constvar_2_3, :θ = θ_7, :constvar_4 = constvar_4_5, :constvar_10 = constvar_10_11}, {:s = ResizableArray{GraphPPL.NodeLabel,1}(GraphPPL.NodeLabel[s_19, s_29, s_39]), :x = ResizableArray{GraphPPL.NodeLabel,1}(GraphPPL.NodeLabel[x_25, x_35, x_45])}, {}, {}, Base.RefValue{Any}(nothing)), and default weight 1.0

meta_graph = lar_meta_graph
GraphPlot.gplot( ## existing plotting functionality
    meta_graph,
    layout=spring_layout,
    nodelabel=collect(labels(meta_graph)),
    nodelabelsize=1.0,
    NODESIZE=0.05, ## diameter of the nodes
    nodelabelc="red", #.
    nodelabeldist=0.0,
    nodefillc="cyan",
)

for i in MetaGraphsNext.vertices(lar_meta_graph)
    label = MetaGraphsNext.label_for(lar_meta_graph, i)
    println("$i: $label")
end

1: γ_1
2: constvar_2_3
3: constvar_4_5
4: GammaShapeRate_6
5: θ_7
6: constvar_8_9
7: constvar_10_11
8: NormalMeanPrecision_12
9: s₀_13
10: constvar_14_15
11: constvar_16_17
12: NormalMeanPrecision_18
13: s_19
14: AR_20
15: anonymous_var_graphppl_21
16: constvar_22_23
17: *_24
18: x_25
19: constvar_26_27
20: NormalMeanPrecision_28
21: s_29
22: AR_30
23: anonymous_var_graphppl_31
24: constvar_32_33
25: *_34
26: x_35
27: constvar_36_37
28: NormalMeanPrecision_38
29: s_39
30: AR_40
31: anonymous_var_graphppl_41
32: constvar_42_43
33: *_44
34: x_45
35: constvar_46_47
36: NormalMeanPrecision_48

## fieldnames(GraphPPL.Context)
lar_gppl_model_context = GraphPPL.getcontext(lar_gppl_model)
lar_gppl_model_context.individual_variables
# lar_gppl_model_context.vector_variables
# lar_gppl_model_context.factor_nodes

16-element UnorderedDictionary{Symbol, GraphPPL.NodeLabel}
                      :γ │ γ_1
            :constvar_42 │ constvar_42_43
            :constvar_14 │ constvar_14_15
            :constvar_16 │ constvar_16_17
 :anonymous_var_graphppl │ anonymous_var_graphppl_41
            :constvar_22 │ constvar_22_23
            :constvar_26 │ constvar_26_27
                     :s₀ │ s₀_13
            :constvar_32 │ constvar_32_33
            :constvar_36 │ constvar_36_37
            :constvar_46 │ constvar_46_47
             :constvar_8 │ constvar_8_9
             :constvar_2 │ constvar_2_3
                      :θ │ θ_7
             :constvar_4 │ constvar_4_5
            :constvar_10 │ constvar_10_11

str_factors(lar_gppl_model, "GammaShapeRate")
str_factors(lar_gppl_model, "AR")
str_factors(lar_gppl_model, "*")

str_individual_vars(lar_gppl_model, "γ")
str_individual_vars(lar_gppl_model, "constvar_42")
str_individual_vars(lar_gppl_model, "anonymous_var_graphppl")
str_individual_vars(lar_gppl_model, "s₀")
str_individual_vars(lar_gppl_model, "θ")

str_vector_vars(lar_gppl_model, "s")
str_vector_vars(lar_gppl_model, "x")

"x_25; x_35; x_45"

lar_dot = generate_dot(
    Model = lar_gppl_model, 
    digraph = false,
    font_size = 10,
    edge_length = 1.0,
    layout = "neato",
    # layout = "dot",
    rank = true,
    # rank = false,
    ranksep = .5,
    splines = "polyline", ## spline, line, none, polyline, ortho, curved

    ## param_prior_factor = "Categorical",
    ## paramvar = "s",

    state_prior_factor = "Categorical",
    statevar = "s",

    datafactor = "Transition",
    datavar = "x",

    busvars = ["A", "B"],
    deltas = false,
    # deltas = true,
    width = 8.0,
    height = 12.0
)
print(lar_dot) #.
show_gv(lar_dot)

dot"""
graph G {
    layout=neato;
    rankdir=TB;
    ranksep=0.5;
    splines=polyline;
    size="8.0,12.0!";
    node [
        shape=circle, 
        fontsize=10, 
        ## style=filled, 
        ## fillcolor=lightgray, 
        ## margin=.02, 
        margin=.01, 
        ## width=.10, height=.10, //minimum size (inches), grows to contain label
        ## width=.10, height=.10, fixedsize=true, //fixed size (inches), does not grow to contain label
        width=.30, height=.30, fixedsize=true, //fixed size (inches), does not grow to contain label
        color=black,
        penwidth=0.2, 
    ]
    edge[
        color=black, 
        penwidth=0.2, 
        len=1.0, 
    ];

    "γ_1" [label=γ, shape=circle, width=.01, height=.01, fixedsize=true, color=none];
    "constvar_2_3" [label=constvar_2, shape=circle, width=.01, height=.01, fixedsize=true, color=none];
    "constvar_4_5" [label=constvar_4, shape=circle, width=.01, height=.01, fixedsize=true, color=none];
    "GammaShapeRate_6" [
        label=GammaShapeRate,
        shape=box, 
        style=filled, 
        fillcolor=lightgray, 
    ];

    "θ_7" [label="", xlabel=θ, shape=box, width=1.5, height=.01, fixedsize=true,];
    "constvar_8_9" [label=constvar_8, shape=circle, width=.01, height=.01, fixedsize=true, color=none];
    "constvar_10_11" [label=constvar_10, shape=circle, width=.01, height=.01, fixedsize=true, color=none];
    "NormalMeanPrecision_12" [
        label=NormalMeanPrecision,
        shape=box, 
        style=filled, 
        fillcolor=lightgray, 
    ];

    "s₀_13" [label=s₀, shape=circle, width=.01, height=.01, fixedsize=true, color=none];
    "constvar_14_15" [label=constvar_14, shape=circle, width=.01, height=.01, fixedsize=true, color=none];
    "constvar_16_17" [label=constvar_16, shape=circle, width=.01, height=.01, fixedsize=true, color=none];
    "NormalMeanPrecision_18" [
        label=NormalMeanPrecision,
        shape=box, 
        style=filled, 
        fillcolor=lightgray, 
    ];

    "s_19" [label="", xlabel=s_1, shape=box, width=1.5, height=.01, fixedsize=true,];
    "AR_20" [
        label=AR,
        shape=box, 
        style=filled, 
        fillcolor=lightgray, 
    ];

    "anonymous_var_graphppl_21" [label=anonymous_var_graphppl, shape=circle, width=.01, height=.01, fixedsize=true, color=none];
    "constvar_22_23" [label=constvar_22, shape=circle, width=.01, height=.01, fixedsize=true, color=none];
    "*_24" [
        label="*",
        shape=box, 
        style=filled, 
        fillcolor=lightgray, 
    ];

    "x_25" [label=x_1, shape=circle, width=.01, height=.01, fixedsize=true, color=none];
    "constvar_26_27" [label=constvar_26, shape=circle, width=.01, height=.01, fixedsize=true, color=none];
    "NormalMeanPrecision_28" [
        label=NormalMeanPrecision,
        shape=box, 
        style=filled, 
        fillcolor=lightgray, 
    ];

    "s_29" [label="", xlabel=s_2, shape=box, width=1.5, height=.01, fixedsize=true,];
    "AR_30" [
        label=AR,
        shape=box, 
        style=filled, 
        fillcolor=lightgray, 
    ];

    "anonymous_var_graphppl_31" [label=anonymous_var_graphppl, shape=circle, width=.01, height=.01, fixedsize=true, color=none];
    "constvar_32_33" [label=constvar_32, shape=circle, width=.01, height=.01, fixedsize=true, color=none];
    "*_34" [
        label="*",
        shape=box, 
        style=filled, 
        fillcolor=lightgray, 
    ];

    "x_35" [label=x_2, shape=circle, width=.01, height=.01, fixedsize=true, color=none];
    "constvar_36_37" [label=constvar_36, shape=circle, width=.01, height=.01, fixedsize=true, color=none];
    "NormalMeanPrecision_38" [
        label=NormalMeanPrecision,
        shape=box, 
        style=filled, 
        fillcolor=lightgray, 
    ];

    "s_39" [label="", xlabel=s_3, shape=box, width=1.5, height=.01, fixedsize=true,];
    "AR_40" [
        label=AR,
        shape=box, 
        style=filled, 
        fillcolor=lightgray, 
    ];

    "anonymous_var_graphppl_41" [label=anonymous_var_graphppl, shape=circle, width=.01, height=.01, fixedsize=true, color=none];
    "constvar_42_43" [label=constvar_42, shape=circle, width=.01, height=.01, fixedsize=true, color=none];
    "*_44" [
        label="*",
        shape=box, 
        style=filled, 
        fillcolor=lightgray, 
    ];

    "x_45" [label=x_3, shape=circle, width=.01, height=.01, fixedsize=true, color=none];
    "constvar_46_47" [label=constvar_46, shape=circle, width=.01, height=.01, fixedsize=true, color=none];
    "NormalMeanPrecision_48" [
        label=NormalMeanPrecision,
        shape=box, 
        style=filled, 
        fillcolor=lightgray, 
    ];

    "γ_1" -- "GammaShapeRate_6";
    "γ_1" -- "AR_20";
    "γ_1" -- "AR_30";
    "γ_1" -- "AR_40";
    "constvar_2_3" -- "GammaShapeRate_6";
    "constvar_4_5" -- "GammaShapeRate_6";
    "θ_7" -- "NormalMeanPrecision_12";
    "θ_7" -- "AR_20";
    "θ_7" -- "AR_30";
    "θ_7" -- "AR_40";
    "constvar_8_9" -- "NormalMeanPrecision_12";
    "constvar_10_11" -- "NormalMeanPrecision_12";
    "s₀_13" -- "NormalMeanPrecision_18";
    "s₀_13" -- "AR_20";
    "constvar_14_15" -- "NormalMeanPrecision_18";
    "constvar_16_17" -- "NormalMeanPrecision_18";
    "s_19" -- "AR_20";
    "s_19" -- "*_24";
    "s_19" -- "AR_30";
    "anonymous_var_graphppl_21" -- "*_24";
    "anonymous_var_graphppl_21" -- "NormalMeanPrecision_28";
    "constvar_22_23" -- "*_24";
    "x_25" -- "NormalMeanPrecision_28";
    "constvar_26_27" -- "NormalMeanPrecision_28";
    "s_29" -- "AR_30";
    "s_29" -- "*_34";
    "s_29" -- "AR_40";
    "anonymous_var_graphppl_31" -- "*_34";
    "anonymous_var_graphppl_31" -- "NormalMeanPrecision_38";
    "constvar_32_33" -- "*_34";
    "x_35" -- "NormalMeanPrecision_38";
    "constvar_36_37" -- "NormalMeanPrecision_38";
    "s_39" -- "AR_40";
    "s_39" -- "*_44";
    "anonymous_var_graphppl_41" -- "*_44";
    "anonymous_var_graphppl_41" -- "NormalMeanPrecision_48";
    "constvar_42_43" -- "*_44";
    "x_45" -- "NormalMeanPrecision_48";
    "constvar_46_47" -- "NormalMeanPrecision_48";
{ rank=same; θ_7 };
{ rank=same; s_19; s_29; s_39 };
{ rank=same;  };
{ rank=max; x_25; x_35; x_45 };
{ rank=max;  };
{ rank=max;  };
}
"""

Warning: node 'γ_1', graph 'G' size too small for label
Warning: node 'constvar_2_3', graph 'G' size too small for label
Warning: node 'constvar_4_5', graph 'G' size too small for label
Warning: node 'GammaShapeRate_6', graph 'G' size too small for label
Warning: node 'constvar_8_9', graph 'G' size too small for label
Warning: node 'constvar_10_11', graph 'G' size too small for label
Warning: node 'NormalMeanPrecision_12', graph 'G' size too small for label
Warning: node 's₀_13', graph 'G' size too small for label
Warning: node 'constvar_14_15', graph 'G' size too small for label
Warning: node 'constvar_16_17', graph 'G' size too small for label
Warning: node 'NormalMeanPrecision_18', graph 'G' size too small for label
Warning: node 'anonymous_var_graphppl_21', graph 'G' size too small for label
Warning: node 'constvar_22_23', graph 'G' size too small for label
Warning: node 'x_25', graph 'G' size too small for label
Warning: node 'constvar_26_27', graph 'G' size too small for label
Warning: node 'NormalMeanPrecision_28', graph 'G' size too small for label
Warning: node 'anonymous_var_graphppl_31', graph 'G' size too small for label
Warning: node 'constvar_32_33', graph 'G' size too small for label
Warning: node 'x_35', graph 'G' size too small for label
Warning: node 'constvar_36_37', graph 'G' size too small for label
Warning: node 'NormalMeanPrecision_38', graph 'G' size too small for label
Warning: node 'anonymous_var_graphppl_41', graph 'G' size too small for label
Warning: node 'constvar_42_43', graph 'G' size too small for label
Warning: node 'x_45', graph 'G' size too small for label
Warning: node 'constvar_46_47', graph 'G' size too small for label
Warning: node 'NormalMeanPrecision_48', graph 'G' size too small for label
Warning: some nodes with margin (3.20,3.20) touch - falling back to straight line edges