Agent Based Testing Report
Agent Based Testing Report
Hafsah
Qaisar Afzal
Ahsan Azam
Tahir Mehmood
Junaid Kiani
SP14-RSE-012
SP14-RSE-013
SP14-RSE-014
SP14-RSE-016
SP14-RSE-017
1. Reactivity: agents are able to sense contextual changes and react appropriately;
2. Pro-activity: agents are autonomous, so they are able to select which actions to take in
order to reach their goals in given situations;
3. Social ability: that is, agents are interacting entities, which cooperate, share
knowledge, or compete for goal achievement
OR
A software agent is defined as a software program that can perform specific tasks for a user and
possessing a degree of intelligence that permits it to perform parts of its tasks autonomously and
to interact with its environment in a useful manner.
Page 2
Sensor Input
Agent
Action Output
Environment
1.2.1.1.
Autonomy: The ability to take goal directed autonomous decisions without intervening with
human or other agents.
Goal Oriented: Does not simply act in response of environment.
Communicative: communicate with other agents including human.
Learning: changes its behavior based on previous experience.
Flexible: Actions are not scripts
COMSATS institute of Information Technology
Page 3
Pro-activity: The ability to initiate actions to accomplish the goals rather than responding to the
unpredictable environment.
Reactivity: The ability to respond in selective and timely fashion by perceiving the environment.
Social ability: The ability to work in collaboration with other agents and human in order to
achieve the goal.
Mobility: The ability to move from one platform/environment to another in self-directed way.
Page 4
Figure depicts the functioning of interface agents. Essentially, interface agents support and
provide assistance, typically to a user learning to use a particular application such as a
spreadsheet or an operating system. The agent observes and monitors the actions taken by the
user in the interface), learns new short-cuts, and suggests better ways of doing the task. Thus, the
interface agent acts as an autonomous personal assistant which cooperates with the user in
accomplishing some task in the application. As for learning, interface agents learn typically to
better assist its user in four ways all shown in Figure:
By observing and imitating the user (i.e. learning from the user);
Through receiving positive and negative feedback from the user (learning from
the user);
By receiving explicit instructions from the user (learning from the user);
By asking other agents for advice (i.e. learning from peers)
Their cooperation with other agents, if any, is limited typically to asking for advice, and not in
getting into protracted negotiation deals with them as is the case with collaborative agents. The
learning modes are typically by rote (memory-based learning) or parametric, though other
techniques such as evolutionary learning are also being introduced.
Page 5
UAVs for military and security applications are in use. Most of them are to a large
degree remotely controlled.
b) Apples Siri:
Siri is the intelligent personal assistant that helps you get things done just by asking. It allows
you to use your voice to send messages, schedule meetings, place phone calls, and more. But Siri
isnt like traditional voice recognition software that requires you to remember keywords and
speak specific commands. Siri understands your natural speech, and it asks you questions if it
needs more information to complete a task.
2. Multi-Agent System:
Page 6
Page 7
Increased complexity, since there are several distributed processes that run
autonomously and concurrently;
Amount of data, since systems can be made up by thousands of agents, each owning its
own data;
Irreproducibility effect, since we cant ensure that two executions of the systems will
lead to the same state, even if the same input is used. As a consequence, looking for a
particular error can be difficult if it is impossible to reproduce it each time [2].
They are also non-deterministic, since it is not possible to determine a priori all
interactions of an agent during its execution.
Agents are autonomous and cooperate with other agents, so they may run correctly by
themselves but incorrectly in a community or vice versa.
Page 8
4. Testing Techniques
4.1.
Role is an important attribute of an agent that has a huge scope and support for the analysis,
design and implementation of Multi-Agent System (MAS). A role can be defined as the
capability enabler that exposes to the agent that plays it a set of actions. Roles are created to do
something and it has the responsibility of achieving specific system goals and sub goals. Roles
provide a well-defined interface between agents and cooperative processes.
Role Oriented Testing
Testing the responsibilities corresponding to their respective role ensures that the agent plays its
role as per the intended goal. As long as the goals are getting satisfied, it is understood that the
individual agent is performing well that suits the system.
An = Gi ( Rj ( Rek ) ) where i = j = k = x = 1
Following are the steps involved in role-oriented testing.
Page 9
Protocol, which indicates how an agent playing such role can interact with agents playing
other role
<Agent>
<Goal>
<Role1>
<Responsibilities 1>
<Protocol>
.
<Responsibilities n>
<Protocols>
</Role1>
<Role n>
<Responsibilities 1>
<Protocol>
.
<Responsibilities n>
<Protocol>
</Role n>
</Goal>
</Agent>
Role Oriented Integration Testing
Integration testing in conventional software development is that to progressively integrate the
tested units (module, program, procedure, function).After ensuring that the individual agent in
isolation is working as per the requirement; the next immediate step is to integrate the agents
involved in the MAS so as to test the interaction, and communication among agents.
Role Oriented System Testing
System testing verifies that all elements (hardware, people, and databases) are integrated
properly so as to ensure whether the overall product met its requirement and achieved the
expected performance. System testing also deals with non-functional requirements of the
software such as recovery testing, security testing, stress testing and performance testing. System
testing in agent oriented approach will test the complete functionality and test the system as a
whole.
Role Schema
Role schema provides a well-defined interface between agents and cooperative processes. This
allows an agent to read and follow, normative rules established by the cooperation process even
if not previously known by the agent. Their major motivation to introduce such roles is to
increase the agent systems adaptability to structural changes. Role schema involves role name,
COMSATS institute of Information Technology
Page 10
agent name, goal to be achieved, description of the role, protocol and related activities,
permissions and responsibilities.
Role Schema
Page 11
all the agents involved in the system. Testing an agent is based on its own mental attributes such
as goal, plan, role and action. Test cases are to be generated so as to ensure the following,
1. To test whether the goal of an agent is achieved.
2. To test whether the plan get triggered by the event that is supposed to handle.
3. To test whether the agent takes appropriate role as expected by the environment
4. To test whether the agent performs the activity as per the functionality.
The primary aim of unit testing is to uncover errors within a given unit
A Mock Agent is a regular agent that communicates with just one agent: the AUT. It has just one
plan to test the AUT. The Mock Agents plan is equivalent to a test script, since it defines the
messages that should be sent to the AUT and the messages that should be received from it. By
testing an agent in isolation using Mock Agents the programmer is forced to consider the agents
interactions with its collaborators (or competitors), possibly before those collaborators (or
competitors) exist.
Agent unit test approach that is composed of five participants:
1- Test Suite: which consists in a set of Test Cases and a set of operations performed to
prepare the test environment before a Test Case starts.
2- Test Case: defines a scenario a set of conditions to which an Agent under Test is
exposed, and verifies whether this agent obeys its specification under such conditions.
3- Agent under Test (AUT): is the agent whose behavior is verified by a Test Case.
4- Mock Agent: consists in a fake implementation of a real agent that interacts with the
AUT. Its purpose is to simulate a real agent strictly for testing the AUT.
5- Agent Monitor: is responsible for monitoring agents life cycle in order to notify the TestCase about agents states.
Page 12
Each agent unit test follows the common structure depicted in Figure.
In step 1, the Test Suite creates the agents platform and whatever element needed to set up the
test environment.
After that, a Test Case is started. The Test Case creates Mock Agents to every role that interacts
with the Agent under Test - in the scenario defined by the Test Case (step 2).
Next, it creates the Agent under Test (step 3) and asks the Agent Monitor to be notified when the
interaction between the AUT and the Mock Agents finishes (step 4).
At this point, the AUT and the Mock Agent start to interact. The Mock Agent sends a message to
the AUT, and it replies (steps 5 and 6) or vice-versa. They can repeat steps 5 and 6 as many times
as necessary to perform the test codified in the Mock Agents plan (for instance, the Mock
Agent can reply three messages before finalizing its test activity).
During all this interaction process, the Agent Monitor keeps track of changes in agents life
cycle. In order to do that it uses three lists as illustrated in Figure : (i) Created Agents List:
maintains IDs of the agents that have been created, but are not running yet an ID is any
information that uniquely identifies an agent.(ii) Running Agents List: maintains IDs of the
agents in the running state. (iii) Work Done Agents List: maintains IDs of the Mock Agents that
have completed their plan (equivalent to a test script).When a Mock Agent concludes its plan, the
Agent Monitor includes the Mock Agents ID in the Work done list.
And then notifies the Test Case that the interaction between the Mock Agent and the AUT have
concluded (step 7).
Lastly, the Test Case asks the Mock Agent whether or not AUT acted as expected (step 8).
This agent unit testing approach has two main concerns: (i) the design of a Test Case based on
the use of Mock Agents; (ii) and the Test Case execution which relies on the Agent Monitor to
notify when the test script (codified in Mock Agents plan) was concluded. As each Mock Agent
exercises just one role of the AUT, rather than the wide interface that comprises all the features
provided it, we call this approach Role Driven Unit Testing.
Pros:
1- In MASs testing approach Unit Testing aims to remove faults along the
application development.
2- Unit testing is independent of any development life cycle used.
3- Unit testing in MAS verifies whether each agent in isolation respects its
specifications under normal and abnormal conditions.
COMSATS institute of Information Technology
Page 13
5. Conclusion:
Most of the existing research work on testing software agent and MAS focuses mainly on agent
and integration level. Basic issues of testing software agents like message passing,
distributed/asynchronous have been considered; testing frameworks have been proposed to
facilitate testing process. And yet, there is still much room for further investigations, for instance
[5]:
A complete and comprehensive testing process for software agents and MAS.
Testing MAS at system and acceptance level: how do the developers and the end-users
build confidence in autonomous agents?
Test inputs definition and generation to deal with open and dynamic nature of software
agents and MAS.
Test oracles, how to judge an autonomous behavior? How to evaluate agents that have
their own goals from human tester's subjective perspectives?
Deriving metrics to assess the qualities of the MAS under test, such as safety, efficiency,
and openness.
Page 14
References:
1. F. Bergenti, M. Gleizes, and F. Zambonelli, Methodologies and Software Engineering
for Agent Systems, The Agent-Oriented Software Engineering Handbook, Springer, Vol.
11, 2004.
2. I. Sommerville, Software Engineering, 9th edition, Addison Wesley, 2011.
3.
Z. Houhamdi, Test Suite Generation Process for Agent Testing, Indian Journal of
Computer Science and Engineering, Vol. 2, 2, 2011.
4. J. Botia, A. Lopez-Acosta, and G. Skarmeta, ACLAnalyser: A tool for debugging multiagent systems, Proceeding of the 16th European Conference on Artificial Intelligence,
pp. 967-968, IOS Press 2004.
5. Multi-Agent System Testing: A Survey
6. Sivakumar, N., and K. Vivekanandan. "Exploring the Need for Specialized Testing
Technique for an Agent-Based Software." PROMETHEUS 2002.7 (2002).
7. Agent Oriented Software Testing Role Oriented approach (IJACSA) International
Page 15
Page 16
9. Paydar, S., & Kahani, M. (2011). An agent-based framework for automated testing of
web-based systems. Journal of Software Engineering and Applications, 4(02), 86.
Web Links:
1.
2.
3.
4.
http://www.agentbuilder.com/Documentation/whyAgents.html
http://www.sce.carleton.ca/netmanage/docs/AgentsOverview/ao.html
http://www.ai.mit.edu/projects/humanoid-robotics-group/cog/capabilities.html
http://www.slideshare.net/ToniMorenoURV/introduction-to-agents-and-multiagentsystems
5. http://www.agentbuilder.com/Documentation/product.html
6. www.askjeeves.com
7. http://ti.arc.nasa.gov/tech/asr/planning-and-scheduling/remote-agent/
8. http://www.agentbuilder.com/AgentTechnology/agentApplications.html
9. http://www.agentbuilder.com/AgentTools/index.html
10. http://en.wikipedia.org/wiki/Distributed_artificial_intelligence
Page 17