Navigating a drone to a specified target is a commonly encountered problem. In the previous project an active inference agent placed a unit far enough from a radar source to become invisible. In this project, the idea is to send a drone to the location of the radar source to do reconnaissance.
In recent years, the landscape of machine learning has undergone a profound transformation with the emergence of active inference, a novel paradigm that draws inspiration from the principles of biological systems to inform intelligent decision-making processes. Unlike traditional approaches to machine learning, which often passively receive data and adjust internal parameters to optimize performance, active inference represents a dynamic and interactive framework where agents actively engage with their environment to gather information and make decisions in real-time.
At its core, active inference is rooted in the notion of agents as embodied entities situated within their environments, constantly interacting with and influencing their surroundings. This perspective mirrors the fundamental processes observed in living organisms, where perception, action, and cognition are deeply intertwined to facilitate adaptive behavior. By leveraging this holistic view of intelligence, active inference offers a unified framework that seamlessly integrates perception, decision-making, and action, thereby enabling agents to navigate complex and uncertain environments more effectively.
One of the defining features of active inference is its emphasis on the active acquisition of information. Rather than waiting passively for sensory inputs, agents proactively select actions that are expected to yield the most informative outcomes, thus guiding their interactions with the environment. This active exploration not only enables agents to reduce uncertainty and make more informed decisions but also allows them to actively shape their environments to better suit their goals and objectives.
Furthermore, active inference places a strong emphasis on the hierarchical organization of decision-making processes, recognizing that complex behaviors often emerge from the interaction of multiple levels of abstraction. At each level, agents engage in a continuous cycle of prediction, inference, and action, where higher-level representations guide lower-level processes while simultaneously being refined and updated based on incoming sensory information.
The applications of active inference span a wide range of domains, including robotics, autonomous systems, neuroscience, and cognitive science. In robotics, active inference offers a promising approach for developing robots that can adapt and learn in real-time, even in unpredictable and dynamic environments. In neuroscience and cognitive science, active inference provides a theoretical framework for understanding the computational principles underlying perception, action, and decision-making in biological systems.
In conclusion, active inference represents a paradigm shift in machine learning, offering a principled and unified framework for understanding and implementing intelligent behavior in artificial systems. By drawing inspiration from the principles of biological systems, active inference holds the promise of revolutionizing our approach to building intelligent machines and understanding the nature of intelligence itself.
An often encountered problem is to navigate a drone to a specified target. In the previous project Under the Radar with Active Inference, an active inference agent placed a unit just beyond the reach of an enemy radar source to hopefully become invisible. In this project, we will setup an active inference agent to send a drone to the location of the radar source to do reconnaissance. The operator of the drone will first maneuver it to a suitable hight. Then the agent will take over to guide it to the specified target (using a 2-dimensional approach for simplicity).
This problem is, of course, not limited to a military application. It is widely applicable - just think of the delivering of packages by logistics companies.
versioninfo() ## Julia version
# VERSION ## Julia versionJulia Version 1.10.4
Commit 48d4fd48430 (2024-06-04 10:41 UTC)
Build Info:
Official https://julialang.org/ release
Platform Info:
OS: Linux (x86_64-linux-gnu)
CPU: 12 × Intel(R) Core(TM) i7-8700B CPU @ 3.20GHz
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-15.0.7 (ORCJIT, skylake)
Threads: 1 default, 0 interactive, 1 GC (on 12 virtual cores)
Environment:
JULIA_NUM_THREADS = import Pkg
# Pkg.add(Pkg.PackageSpec(;name="RxInfer", version="3.0.0"))
Pkg.add(Pkg.PackageSpec(;name="RxInfer", version="3.6.0"))
Pkg.add("Plots")
using RxInfer
using Plots
default(label="", margin=10Plots.pt)
import RxInfer.ReactiveMP: getrecent, messageout Resolving package versions...
No Changes to `~/.julia/environments/v1.10/Project.toml`
No Changes to `~/.julia/environments/v1.10/Manifest.toml`
Resolving package versions...
No Changes to `~/.julia/environments/v1.10/Project.toml`
No Changes to `~/.julia/environments/v1.10/Manifest.toml`Pkg.status()Status `~/.julia/environments/v1.10/Project.toml`
[91a5bcdd] Plots v1.40.8
[86711068] RxInfer v3.6.0There is no pre-existing data to be analyzed.
There is no pre-existing data to be prepared.
Please review the narrative in section 1.
This section attempts to answer three important questions:
For this problem, we will only track:
Decisions will be in the form of agent-prescribed turn actions.
The sources of uncertainty relating to the environment will be
The system-under-steer/environment/generative process is a drone with a 4-dimensional state vector:
The drone will be steered by means of turn/yaw actions.
_s̃₀ = [-0.0, 0.0, -0.1, 3.0] ## initial state4-element Vector{Float64}:
-0.0
0.0
-0.1
3.0The state at time \(t\) of the system-under-steer (sustr), also referred to as the environment (envir), or the generative process (genpr) will be given by:
\[ \tilde{\mathbf{s}}_t = (x_t, y_t, v_{at}, v_{rt}) \] where
## Function to find the updates in the x & y components due to the velocity
function Aᵃ(s̃, δt)
a = zeros(4)
a[1] = s̃[4]*cos(s̃[3])*δt
a[2] = s̃[4]*sin(s̃[3])*δt
return a
end
Aᵃ(_s̃₀, 1) ## x & y components of velocity4-element Vector{Float64}:
2.9850124958340776
-0.29950024994048446
0.0
0.0Decisions are in the form of turn/yaw actions which adjust the angle of the velocity. Actions are limited to the interval \((-F^{EngLimit}, F^{EngLimit})\). It is given by:
\[ \begin{align} R^a &= F^{EngLimit} ⋅ \mathrm{tanh}(a) \\ &= 0.1 ⋅ \mathrm{tanh}(a) \end{align} \] where \(a_t\) is the action on the drone at time \(t\).
_Fᴱⁿᵍᴸⁱᵐⁱᵗ = 0.1
function Rᵃ(a::Real) ## turn/yaw rate
b = [ 0.0, 0.0, 1.0, 0.0 ]
return b*_Fᴱⁿᵍᴸⁱᵐⁱᵗ*tanh(a)
end
Rᵃ(0.25)4-element Vector{Float64}:
0.0
0.0
0.024491866240370915
0.0There are no exogenous information variables for this problem.
The transition function captures the dynamics of the environment/system-under-steer/generative process:
\[ \tilde{\mathbf{s}}_t = g(\tilde{\mathbf{s}}_{t-1}, a_t) \]
The evolution of the state, \(\tilde{\mathbf{s}}_t = (x_t, y_t, v_{at}, v_{rt})\) will now be described.
\[\begin{aligned} x_t &= x_{t-1} + v_{r,t-1}⋅ \mathrm{cos}(v_{a,t-1}) ⋅ δ t \\ y_t &= y_{t-1} + v_{r,t-1}⋅ \mathrm{sin}(v_{a,t-1}) ⋅ δ t \\ v_{at} &= v_{a,t-1} + R^a(a_{t-1}) \end{aligned}\]
The environment generates outcomes as noisy observations of the current state with an observation noise variance
\[ \begin{aligned} \mathbf\Theta &= \vartheta⋅\mathbf{I} \\ &= \begin{bmatrix} \vartheta & 0 & 0 & 0\\ 0 & ϑ & 0 & 0\\ 0 & 0 & \vartheta & 0\\ 0 & 0 & 0 & \vartheta\\ \end{bmatrix} \\ &= \begin{bmatrix} 10^{-4} & 0 & 0 & 0\\ 0 & 10^{-4} & 0 & 0\\ 0 & 0 & 10^{-4} & 0\\ 0 & 0 & 0 & 10^{-4}\\ \end{bmatrix} \\ \end{aligned} \] where \(\mathbf I\) is the \(4 × 4\) identity matrix.
The observation function can be represented by: \[ \mathbf{y}_t \sim \cal N(\tilde{\mathbf{s}}_t, \mathbf\Theta) \]
_γ = 1e4 ## transition precision (system noise)
_ϑ = 1e-4 ## observation variance (observation noise)0.0001The objective function is such that the Bethe free energy is minimized. This aspect will be handled by the RxInfer Julia package.
The agent and the environment interact through a Markov blanket. Because states of the agent are unknown to the world, we wrap them in a comprehension that only returns functions for interacting with the agent. Internal beliefs cannot be directly observed, and interaction is only allowed through the Markov blanket of the agent (i.e. the sensors and actuators).
function create_envir(; Rᵃ, s̃₀=[0.5, 0.5, 0.1, 3.0])
s̃ₜ₋₁ = s̃₀
s̃ₜ = s̃ₜ₋₁
yₜ = s̃ₜ
execute = (aₜ::Float64) -> begin ## aₜ is turn/yaw action
s̃ₜ = s̃ₜ₋₁ + Aᵃ(s̃ₜ₋₁, 1) + Rᵃ(aₜ) ##Compute next state
yₜ = s̃ₜ
s̃ₜ₋₁ = s̃ₜ ##Reset state
end
observe = () -> begin
return yₜ
end
return (execute, observe)
endcreate_envir (generic function with 1 method)As noted above, the sources of uncertainty relating to the environment will be:
According to the agent the state of the system-under-steer/environment/generative process will be \(s_t\), rather than \(s̃_t\), which will then be given by
\[ \mathbf{s}_t = (x_t, y_t, v_{at}, v_{rt}) \]
According to the agent the action on the environment at time \(t\) will be represented by \(u_t\), also known as the control state of the agent.
We start by specifying a probabilistic model for the agent that describes the agent’s internal beliefs over the external dynamics of the environment.
The generative model is defined as follows:
\[\begin{aligned} p'(\mathbf{x}_:, \mathbf{s}_:, \mathbf{u}_:) = \underbrace{p(\mathbf{s}_{t-1})}_{\substack{ \text{Initial} \\ \text{state} \\ \text{prior}}} \prod_{k=t}^{t+(T-1)} \underbrace{p(\mathbf{x}_k \mid \mathbf{s}_k)}_{\substack{ \text{Observation} \\ \text{model}}} \, \underbrace{p(\mathbf{s}_k \mid \mathbf{s}_{k-1}, \mathbf{u}_k}_{\substack{\text{Transition} \\ \text{model}}}) \, \underbrace{p(\mathbf{u}_k)}_{\substack { \text{Control} \\ \text{prior}}} \ \nonumber \qquad \mathrm{(2019\_VanDeLaar,\_SAIPbMP\ eq9)} \end{aligned}\]
The generative model includes future time steps.
To infer goal-driven (i.e. purposeful) behavior, we add prior beliefs \(p^+(\mathbf{x})\) about desired future observations. This leads to an extended agent model:
\[\begin{aligned} p(\mathbf{x}_:, \mathbf{s}_:, \mathbf{u}_:) &= \frac{p'(\mathbf{x}, \mathbf{s}, \mathbf{u}) p^+(\mathbf{x})}{\int_x p'(\mathbf{x}, \mathbf{s}, \mathbf{u})p^+(\mathbf{x}) \mathrm{d}\mathbf{x}} \\ &\propto \underbrace{p(\mathbf{s}_{t-1})\prod_{k=t}^{t+(T-1)} p(\mathbf{x}_k \mid \mathbf{s}_k)p(\mathbf{s}_k \mid \mathbf{s}_{k-1}, \mathbf{u}_k)p(\mathbf{u}_k)}_{\text{original generative model}} \, \underbrace{p^+(\mathbf{x}_k)}_{\substack { \text{extension} \\ \text{Goal} \\ \text{prior}}} \\ &\propto \underbrace{p(\mathbf{s}_{t-1})}_{\substack{ \text{Initial} \\ \text{state} \\ \text{prior}}} \prod_{k=t}^{t+(T-1)} \underbrace{p(\mathbf{x}_k \mid \mathbf{s}_k)}_{\substack{ \text{Observation} \\ \text{model}}} \, \underbrace{p(\mathbf{s}_k \mid \mathbf{s}_{k-1}, \mathbf{u}_k}_{\substack{\text{Transition} \\ \text{model}}}) \, \underbrace{p(\mathbf{u}_k)}_{\substack { \text{Control} \\ \text{prior}}} \, \underbrace{p^+(\mathbf{x}_k)}_{\substack { \text{Goal} \\ \text{prior}}} \nonumber \end{aligned}\]
The factors are defined as:
\[p(\mathbf{x}_k \mid \mathbf{s}_k) = \mathcal{N}(\mathbf{x}_k \mid \mathbf{s}_k,\,\mathbf\Theta)\] where \(\mathbf{x}_k = (\chi_{1k}, \chi_{2k}, ...)\) denotes observations of the agent after interacting with the environment.
\[ \begin{aligned} p(\mathbf{s}_k \mid \mathbf{s}_{k-1}, \mathbf{u}_k) &= \mathcal{N}(\mathbf{s}_k \mid \mathbf{s}_{k-1} + \mathbf{u}_k, ϑ) \\ p(\mathbf{s}_{t-1}) &= \mathcal{N}(\mathbf{s}_{t-1} \mid \mathbf{m}_{t-1},\, \mathbf{V}_{t-1}) \end{aligned} \]
The current state is a linear function of the previous state and action. We have endowed the agent with an accurate model of the system dynamics.
\[ \begin{aligned} p(\mathbf{u}_t) &= \prod_{k=t}^{t+(T-1)} \mathcal{N}(\mathbf{u}_k \mid \mathbf{0},\,\mathbf\Xi) \\ &= \prod_{k=t}^{t+(T-1)} \mathcal{N}(\mathbf{u}_k \mid \mathbf{0},\,ξ ⋅ \mathbf{I}) \\ &= \prod_{k=t}^{t+(T-1)} \mathcal{N}(\mathbf{u}_k \mid \mathbf{0},\,1.0) \\ &= \mathcal{N}(\mathbf{u}_t \mid \mathbf{m}_u,\, \mathbf{V}_u)\\ \end{aligned} \]
This represents the control priors.
\[ \begin{aligned} p^+(\mathbf{x}_t) &= \prod_{k=t}^{t+(T-1)} \mathcal{N}(\mathbf{x}_k \mid \mathbf{0},\,\sigma^{huge}) \cdot \mathcal{N}(\mathbf{x}_T \mid \mathbf{x}_+, \, \mathbf\Sigma)\\ &= \prod_{k=t}^{t+(T-1)} \mathcal{N}(\mathbf{x}_k \mid \mathbf{0},\,\sigma^{huge}) \cdot \mathcal{N}(\mathbf{x}_T \mid \mathbf{x}_+, \, \sigma ⋅ \mathbf I)\\ &= \prod_{k=t}^{t+(T-1)} \mathcal{N}(\mathbf{x}_k \mid \mathbf{0},\,10^{12}) \cdot \mathcal{N}(\mathbf{x}_T \mid [0.0, 0.0, 0.0*π, 0.1], \, 10^{-4} ⋅ \mathbf I)\\ &= \mathcal{N}(\mathbf{x}_t \mid \mathbf{m}_x,\, \mathbf{V}_x) \\ \end{aligned} \]
This represents the goal/target/preference priors and encodes a belief about a preferred target/goal \(\mathbf{x}₊\).
Setting \(t=1\), \[ p(\mathbf{s}_0) = \mathcal{N}(\mathbf{s}_0 \mid 0, 10^{12}) \]
This means we set a vague prior for the initial state.
The code in the next block defines the agent’s internal beliefs over the external dynamics and its probabilistic model of the environment, which correspond accurately by directly using the functions defined above. We use the @model macro from RxInfer to define the probabilistic model and the meta block to define approximation methods for the nonlinear state-transition functions.
In the model specification we in addition to the current state of the agent we include the beliefs over its future states (up to T steps ahead):
@model function dronenav_model(mᵤ, Vᵤ, mₓ, Vₓ, mₛ₍ₜ₋₁₎, Vₛ₍ₜ₋₁₎, T, Rᵃ)
## Transition function
g = (sₜ₋₁::AbstractVector) -> begin
sₜ = similar(sₜ₋₁) ## Next state
sₜ = Aᵃ(sₜ₋₁, 1.0) + sₜ₋₁
return sₜ
end
## Function for modeling turn/yaw control
h = (u::AbstractVector) -> Rᵃ(u[1])
Γ = _γ*diageye(4) ## Transition precision
𝚯 = _ϑ*diageye(4) ## Observation variance
sₜ₋₁ ~ MvNormal(mean=mₛ₍ₜ₋₁₎, cov=Vₛ₍ₜ₋₁₎)
sₖ₋₁ = sₜ₋₁
local s
## subtract t-1 from both sides of range k = t : t+(T-1)
## k used for future times in series, i used for sets
## T used for number of time steps in lookahead time horizon
for k in 1:T
## Control
u[k] ~ MvNormal(mean=mᵤ[k], cov=Vᵤ[k])
hIuI[k] ~ h(u[k]) where { meta=DeltaMeta(method=Unscented()) }
## State transition
gIsI[k] ~ g(sₖ₋₁) where { meta=DeltaMeta(method=Unscented()) }
ghSum[k] ~ gIsI[k] + hIuI[k]#.
s[k] ~ MvNormal(mean=ghSum[k], precision=Γ)
## Likelihood of future observations
x[k] ~ MvNormal(mean=s[k], cov=𝚯)
## Target/Goal prior
x[k] ~ MvNormal(mean=mₓ[k], cov=Vₓ[k])
sₖ₋₁ = s[k]
end
return (s, )
endNext, we define the agent:
function create_agent(; T=20, Rᵃ, x₊, s₀, ξ=0.1, σ=1e-4)
## Set control priors
Ξ = fill(ξ, 1, 1) ##Control prior variance
mᵤ = Vector{Float64}[ [0.0] for k=1:T ] ##Set control priors
Vᵤ = Matrix{Float64}[ Ξ for k=1:T ]
## Set target/goal priors
Σ = σ*diageye(4) ##Target/Goal prior variance
Σ[3, 3] = 1e4
Σ[4, 4] = 1e4
mₓ = [zeros(4) for k=1:T]
mₓ[end] = x₊ ##Set prior mean to reach target/goal at t=T
Vₓ = [huge*diageye(4) for k=1:T]
Vₓ[end] = Σ ##Set prior variance to reach target/goal at t=T
## Set initial brain state prior
mₛ₍ₜ₋₁₎ = s₀
Vₛ₍ₜ₋₁₎ = tiny*diageye(4)
## Set current inference results
result = nothing
## Bayesian inference by message passing
## The `infer` function is the heart of the agent
## It calls the `RxInfer.infer` function to perform Bayesian inference by message passing
compute = (υₜ::Float64, ŷₜ::Vector{Float64}) -> begin
mᵤ[1] = [υₜ] ## Register action with the generative model
Vᵤ[1] = fill(tiny, 1, 1) ## Clamp control prior to performed action
mₓ[1] = ŷₜ ## Register observation with the generative model
Vₓ[1] = tiny*diageye(4) ## Clamp target/goal prior to observation
result = infer(
model=dronenav_model(T=T, Rᵃ=Rᵃ),
data=Dict(
:mᵤ => mᵤ,
:Vᵤ => Vᵤ,
:mₓ => mₓ,
:Vₓ => Vₓ,
:mₛ₍ₜ₋₁₎ => mₛ₍ₜ₋₁₎,
:Vₛ₍ₜ₋₁₎ => Vₛ₍ₜ₋₁₎))
end
## The `act` function returns the inferred best possible action
act = () -> begin
if result !== nothing
return mode(result.posteriors[:u][3])[1]
else
return 0.0 ## Without inference result we return some 'random' action
end
end
## The `future` function returns the inferred future states
future = () -> begin
if result !== nothing
return getindex.(mode.(result.posteriors[:s]), 1)
else
return zeros(T)
end
end
## The `slide` function modifies the `(mₛ₍ₜ₋₁₎, Vₛ₍ₜ₋₁₎` for the next step
## and shifts (or slides) the array of future goals `(mₓ, Vₓ)`
## and inferred actions `(mᵤ, Vᵤ)`
slide = () -> begin
model = RxInfer.getmodel(result.model)
(s, ) = RxInfer.getreturnval(model)
varref = RxInfer.getvarref(model, s)
var = RxInfer.getvariable(varref)
slide_msg_idx = 3 ## This index is model dependent
(mₛ₍ₜ₋₁₎, Vₛ₍ₜ₋₁₎) = mean_cov(getrecent(messageout(var[2], slide_msg_idx)))
mᵤ = circshift(mᵤ, -1)
mᵤ[end] = [0.0]
Vᵤ = circshift(Vᵤ, -1)
Vᵤ[end] = Ξ
mₓ = circshift(mₓ, -1)
mₓ[end] = x₊
Vₓ = circshift(Vₓ, -1)
Vₓ[end] = Σ
end
return (act, future, compute, slide)
endcreate_agent (generic function with 1 method)In this simulation we are going to perform a naive action policy for a tight full-turn only. In this case the agent should not be able to achieve its goal:
_Nⁿᵃⁱᵛᵉ = 100 ## Total simulation time
_πⁿᵃⁱᵛᵉ = -0.1 ## Naive policy for full right turn action only
_s̃₀ = [8.0, 8.0, -0.1, 0.1]
(execute_naive, observe_naive) = create_envir(; ## Let there be a world
Rᵃ=Rᵃ,
s̃₀=_s̃₀
);
_yⁿᵃⁱᵛᵉ = Vector{Vector{Float64}}(undef, _Nⁿᵃⁱᵛᵉ)
for t = 1:_Nⁿᵃⁱᵛᵉ
execute_naive(_πⁿᵃⁱᵛᵉ) ## Execute environmental process
_yⁿᵃⁱᵛᵉ[t] = observe_naive() ## Observe external states
end_yⁿᵃⁱᵛᵉ100-element Vector{Vector{Float64}}:
[8.099500416527803, 7.990016658335318, -0.10996679946249559, 0.1]
[8.198896390738552, 7.979042128245321, -0.11993359892499117, 0.1]
[8.298178049006994, 7.967077499898885, -0.12990039838748674, 0.1]
[8.397335529063596, 7.954123961817553, -0.1398671978499823, 0.1]
[8.496358980974227, 7.94018280075747, -0.14983399731247787, 0.1]
[8.595238568118608, 7.925255401581565, -0.15980079677497344, 0.1]
[8.693964468167453, 7.909343247121981, -0.169767596237469, 0.1]
[8.792526874058176, 7.892447918032778, -0.17973439569996458, 0.1]
[8.890915994969092, 7.874571092632912, -0.18970119516246015, 0.1]
[8.989122057291999, 7.855714546739526, -0.1996679946249557, 0.1]
⋮
[15.555115664145777, 3.3315689878275765, -1.0169455505495926, 0.1]
[15.607712286813, 3.246518442361943, -1.0269123500120882, 0.1]
[15.65945862940756, 3.1609479098867594, -1.036879149474584, 0.1]
[15.710349551640839, 3.074865890659476, -1.0468459489370796, 0.1]
[15.760379998198484, 2.988280935746728, -1.0568127483995753, 0.1]
[15.80954499924259, 2.901201646174907, -1.066779547862071, 0.1]
[15.857839670905376, 2.8136366720757633, -1.0767463473245666, 0.1]
[15.905259215774336, 2.7255947118271378, -1.0867131467870623, 0.1]
[15.951798923368797, 2.6370845111888945, -1.096679946249558, 0.1]_x₊ = [0.0, 0.0, 0.0*π, 0.0]
animation_naive = @animate for i in 1:_Nⁿᵃⁱᵛᵉ
scatter([_x₊[1]], [_x₊[2]],
xlims=(0, 1.5*_s̃₀[1]),
ylims=(0, 1.5*_s̃₀[2]),
label="Target", markersize=10)
scatter!(
[_yⁿᵃⁱᵛᵉ[i][1]], [_yⁿᵃⁱᵛᵉ[i][2]],
xlims=(0, 1.5*_s̃₀[1]),
ylims=(0, 1.5*_s̃₀[2]),
label="Drone", markersize=6)
plot!(
[_yⁿᵃⁱᵛᵉ[i][1], _yⁿᵃⁱᵛᵉ[i][1] + 10*_yⁿᵃⁱᵛᵉ[i][4]*cos(_yⁿᵃⁱᵛᵉ[i][3])],
[_yⁿᵃⁱᵛᵉ[i][2], _yⁿᵃⁱᵛᵉ[i][2] + 10*_yⁿᵃⁱᵛᵉ[i][4]*sin(_yⁿᵃⁱᵛᵉ[i][3])],
arrow=true, color=:black, label="Velocity",
title="Move to Target/Goal", legend=:topleft,
aspect_ratio=:equal,
size=(600, 600)
)
end
gif(animation_naive, "shipnav-naive.gif", fps=24, show_msg=false) #.
In the active inference approach we are going to create an agent that models the environment around itself as well as the best possible actions in a probabilistic manner.
### Simulation parameters
## Total simulation time
_Nᵃⁱ = 200 ## Total simulation time
## Lookahead time horizon
_Tᵃⁱ = 100
## Initial state
_s₀ = [8.0, 8.0, -0.1, 0.1]
## Control prior variance value
_ξ = 1.0
## Target prior variance value
_σ = 1e-6
## Target/Goal state
_x₊ = [0.0, 0.0, 0.0*π, 0.1]4-element Vector{Float64}:
0.0
0.0
0.0
0.1(execute_ai, observe_ai) = create_envir(; ## Let there be a world
Rᵃ=Rᵃ,
s̃₀=_s₀
)
(act_ai, future_ai, compute_ai, slide_ai) = create_agent(; ## Let there be an agent
T =_Tᵃⁱ,
Rᵃ=Rᵃ,
x₊=_x₊,
s₀=_s₀,
ξ = _ξ,
σ = _σ
)
## Step through experimental protocol
_as = Vector{Float64}(undef, _Nᵃⁱ) ## Actions
_fs = Vector{Vector{Float64}}(undef, _Nᵃⁱ) ## Predicted future
_ys = Vector{Vector{Float64}}(undef, _Nᵃⁱ) ## Observations
## t used for general times in series
## N used for number of time steps in experiment
## for each t=1:N there are k=1:T future time steps (in _fs)
for t = 1:_Nᵃⁱ
## 1. Act-Execute-Observe: #.execute() & observe() from create_envir()
_as[t] = act_ai() ## Invoke an action from the agent
_fs[t] = future_ai() ## Fetch the predicted future states
execute_ai(_as[t]) ## The action influences hidden external states
_ys[t] = observe_ai() ## Observe the current environmental outcome (update p)
## 2. Infer:
compute_ai(_as[t], _ys[t]) ## Infer beliefs from current model state (update q)
## 3. Slide:
slide_ai() ## Prepare for next iteration
end_animation_ai = @animate for i in 1:_Nᵃⁱ
pls = scatter(
[_x₊[1]], [_x₊[2]],
xlims=(0, 1.5*_s₀[1]),
ylims=(0, 1.5*_s₀[2]),
label="Target", markersize=10)
pls = scatter!(
pls, [_ys[i][1]], [_ys[i][2]],
xlims=(0, 1.5*_s₀[1]),
ylims=(0, 1.5*_s₀[2]),
label="Drone", markersize=6)
pls = plot!(pls,
[_ys[i][1], _ys[i][1] + 10*_ys[i][4]*cos(_ys[i][3])],
[_ys[i][2], _ys[i][2] + 10*_ys[i][4]*sin(_ys[i][3])],
arrow=true, color=:black, label="Velocity",
title="Move to Target/Goal", legend=:topleft,
## xlabel="miles", ylabel="miles",
aspect_ratio=:equal)
plot!(pls, size=(600, 600))
end
gif(_animation_ai, "./shipnav-aif.gif", fps=24, show_msg=false)