Designing the Agent for the Task

1. Analyze the Problem / Task (PEAS)
2. Select Agent Architecture
3. Select Internal Representations
4. Apply Corresponding Algorithms

PEAS

In order to start the agent design process we need to specify / define:

• The Performance measure
• The Environment in which the agent will operate
• The Actuators that the agent will use to affect the environment
• The Sensors that the agent will use to perceive the environment

ex)

Agent type	Performance Measure	Environment	Actuators	Sensors
Taxi driver	Safe, fast, legal, comfortable trip, maximize profits, minimize impact on other road users	Roads, other traffic, police, pedestrians, customers, weather	Steering, accelerator, brake, signal, horn, display, speech	Cameras, radar, speedometer, GPS, engine sensors, accelerometer, microphones, touchscreen

Task Environment Properties

Fully vs partially observable (can be unobservable too)

• Where an agent sensor is capable to sense or access the complete state of an agent at each point in time, it is said to be a fully observable environment else it is partially observable.
• Maintaining a fully observable environment is easy as there is no need to keep track of the history of the surrounding.
• An environment is called unobservable when the agent has no sensors in all environments.

Example

Chess: Fully Driving: partially because what’s around the corner is not known

Single-agent vs multi-agent

multi-agent: competitive vs cooperative

• An environment consisting of only one agent is said to be a single-agent environment.
• A person left alone in a maze is an example of the single-agent system.
• An environment involving more than one agent is a multi-agent environment.
• The game of football is multi-agent as it involves 11 players in each team.

Example

Crossword - single agent Chess - multi-agent

Deterministic vs nondeterministic (stochastic)

Deterministic environment

• next state is completely determined by the current state and agent action
• Deterministic AND fully observable environment: no need to worry about uncertainty
• Deterministic AND partially observable may appear nondeterministic
• Output value is determined by the input value
• When a uniqueness in the agent’s current state completely determines the next state of the agent.

Nondeterministic (stochastic) environment

• Next state is NOT completely determined by the current state and agent action.
• It is impossible to objectively predict the results or results of the process because the knowledge of the cause-effect relationship is insufficient or the initial conditions are unknown.
• Random in nature which is not unique and cannot be completely determined by the agent

Example

Chess - there would be only a few possible moues for a coin at the current state and these moves can be determined. Self-driving cars - the actions of a self-driving car are not unique, it varies time to time.

Episodic vs sequential

Episodic environment

• agent experience is divided into individual, independent, and atomic episodes
• One percept - one action
• Next action is not a function of the previous action: not necessary to memorize it
• In an Episodic task environment, each of the agent’s actions is divided into atomic incidents or episodes. There is no dependency between current and previous incidents. In each incident, an agent receives input from the environment and then performs the corresponding action
• Example: Consider an example of Pick and Place robot, which is used to detect defective parts from the conveyor belts. Here, every time robot(agent) will make the decision on the current part i.e. there is no dependency between current and previous decisions.

Sequential environment

• Current decision / action COULD affect all future decisions / actions
• Better keep track of it.
• In a Sequential environment, the previous decisions can affect all future decisions. The next action of the agent depends on what action he has taken previously and what action he is supposed to take in the future.
• Example: Checkers - Where the previous move can affect all the following moves.

Static vs Dynamic

Static environment

• Environment CANNOT change while the agent is taking its time to decide -> Easy to deal with - Nor need it to worry about the passage of time.
• An idle environment with no change in its state is called a static environment.
• An empty house is static as there’s no change in the surroundings when an agent enters.

Dynamic environment

• Environment CAN change while the agent is taking its time to decide -> decision / action may be dated
• Are continuously asking the agent what it wants to do.
• If it hasn’t been decided yet, that counts as deciding to do nothing.
• Speed is important.
• An environment that keeps constantly changing itself when the agent is up with some action is said to be dynamic.
• A roller coaster ride is dynamic as it is set in motion and the environment keeps changing every instant.

Semi-dynamic

• If the environment itself does not change with the passage of time but the agent’s performance score does.

Discrete vs Continuous

Discrete environment

• State changes are discrete
• Time changes are discrete
• Percepts are discrete
• If an environment consists of a finite number of actions that can be deliberated in the environment to obtain the output, it is said to be a discrete environment.
• The game of chess is discrete as it has only a finite number of moves. The number of moves might vary with every game, but still, it’s fine.

Continuous environment

• State changes are continuous (“fluid”)
• Time changes are continuous*
• Percepts / Actions can be continuous
• The environment in which the actions are performed cannot be numbered i.e. is not discrete, is said to be continuous.
• Self-driving cars are an example of continuous environments as their actions are driving, parking, etc, which cannot be numbered.

*Discrete_time_and_continuous_time

Known vs Unknown (to the agent)

Known environment

• Agent knows all outcomes to its actions (or their probabilities)
• Agent “knows how the environment works”
• In a known environment, the output for all probable actions is given.

Unknown environment

• Agent “doesn’t know all the details about the inner workings of the environment”
• learning and exploration can be necessary
• Obviously, in the case of unknown evironment, for an agent to make a decision, it has to gain knowledge about how the environment works.

Task Environment Characteristics

Task Environment	Observable	Agent	Deterministic	Episodic	Static	Discrete
Crossword Puzzle Chess with a clock	Fully Fully	Single Multi	Deterministic Deterministic	Sequential Sequential	Static Semi	Discrete Discrete
Poker Backgammon	Partially Fully	Multi Multi	Stochastic Stochastic	Sequential Sequential	Static Static	Discrete Discrete
Taxi driving Medical diagnosis	Partially Partially	Multi Single	Stochastic Stochastic	Sequential Sequential	Dynamic Dynamic	Continuous Continuous
Image analysis Part-picking robot	Fully Partially	Single Single	Deterministic Stochastic	Episodic Episodic	Semi Dynamic	Continuous Continuous
Refinery controller English tutor	Partially Partially	Single Multi	Stochastic Stochastic	Sequential Sequential	Dynamic Dynamic	Continuous Discrete

Hardest Case / Problem

Partially observable (incomplete information, uncertainty) Multi-agent (complex interactions) Nondeterministic (uncertainty) Sequential (planning usually necessary) Dynamic (changing environment, uncertainty) Continuous (infinite number of states) Unknown (agent needs to learn / explore, uncertainty)

Reference

• Illinois Institue of Technology CS480 (Intro Artificial Inteligence) Fall 2022 prof. Hacek Dzilkowski