reachable_and_absorbing function

Reachable and Absorbing States

Find reachable and absorbing states in the transition model.

reachable_states(x, states = NULL)

absorbing_states(x, states = NULL)

remove_unreachable_states(x)

Arguments

• x: a MDP pr POMDP object.
• states: a character vector specifying the names of the states to be checked. NULL checks all states.

Returns

reachable_states() returns a logical vector indicating if the states are reachable.

absorbing_states() returns a logical vector indicating if the states are absorbing (terminal).

the model with all unreachable states removed

Details

The function reachable_states() checks if states are reachable using the transition model.

The function absorbing_states() checks if a state or a set of states are absorbing (terminal states) with a zero reward (or -Inf for unavailable actions). If no states are specified (states = NULL), then all model states are checked. This information can be used in simulations to end an episode.

The function remove_unreachable_states() simplifies a model by removing unreachable states.

Examples

data(Maze)

gridworld_matrix(Maze, what = "label")

# the states marked with +1 and -1 are absorbing
absorbing_states(Maze)
which(absorbing_states(Maze))

# all states in the model are reachable
reachable_states(Maze)
which(!reachable_states(Maze))

See Also

Other MDP: MDP(), MDP2POMDP, MDP_policy_functions, accessors, actions(), add_policy(), gridworld, regret(), simulate_MDP(), solve_MDP(), transition_graph(), value_function()

Other POMDP: MDP2POMDP, POMDP(), accessors, actions(), add_policy(), plot_belief_space(), projection(), regret(), sample_belief_space(), simulate_POMDP(), solve_POMDP(), solve_SARSOP(), transition_graph(), update_belief(), value_function(), write_POMDP()

Author(s)

Michael Hahsler