Automatic testing. Training

Содержание

Слайд 2

Plan Development Goals Architecture Structure Agents Engine Gateway testing

Plan

Development Goals
Architecture
Structure
Agents
Engine
Gateway testing

Слайд 3

Development Goals QA of developed products Several pure-testing projects Socket API

Development Goals

QA of developed products
Several pure-testing projects
Socket API
iSCSI
Ethernet NIC firmware and

drivers
IPv6
...
Strong regression detection tool
Слайд 4

Key features Black and white box testing Distributed testing Automatic testing

Key features

Black and white box testing
Distributed testing
Automatic testing
Test coverage estimation
Regression tracking

tools
Cross-platform testing
Networking protocols support
Strong scenarios parametrization system
Слайд 5

Used in IPv4/IPv6 Socket API (more than 8000 tests); Ethernet switch

Used in

IPv4/IPv6 Socket API (more than 8000 tests);
Ethernet switch testing;
IEEE 802.2

(LLC), IEEE 802.1Q (VLAN);
IEEE 802.1D (STP);
ATM/AAL5 testing;
Elements of 802.1X, WPA, WPA2, RADIUS client on Access Point;
iSCSI testing;
CLI testing;
Testing SNMP MIBs;
DHCP Client testing;
CableHome1.0 some portal testing;
Hyperlan2 MAC layer;
Слайд 6

Base components Test Engine Run on Linux Build all components of

Base components

Test Engine
Run on Linux
Build all components of TE
Control of Test

Agents
Keep track of configuration changes
Run tests
Centralized log gathering
Слайд 7

Test Agent Target-specific application Linux FreeBSD NetBSD Windows OpenSolaris Run on

Test Agent

Target-specific application
Linux
FreeBSD
NetBSD
Windows
OpenSolaris
Run on an auxiliary station or DUT
Configuring station (interfaces,routes,etc.)‏

Слайд 8

Test Agent Generates/Handles network traffic: Packets representation in ASN.1 format Packet

Test Agent

Generates/Handles network traffic:
Packets representation in ASN.1 format
Packet iteration, inter

packet interval, number of packets
Sophisticated incoming traffic matching
Supported packet formats:
IEEE-802.3, IEEE-802.1D, IEEE-802.1Q, IEEE-802.2
ATM, AAL5
ARP, IPv4, IPv6, PPP, ICMPv4, UDP, TCP, DHCP, SNMP
...
Слайд 9

Test Agent Sun RPC functionality socket API remote command line Curl

Test Agent

Sun RPC functionality
socket API
remote command line
Curl
File operations
...
Daemons configuration:
DHCP, DNS, FTP,

TFTP, SNTP, LDAP, Radius, VTUN, 802.1X supplicant
Слайд 10

Auxiliary Agent Black Box testing: Network protocols Interoperability External Interfaces (SNMP,

Auxiliary Agent

Black Box testing:
Network protocols
Interoperability
External Interfaces (SNMP, CLI)‏
Access to external equipment:
Sophisticated

traffic generators
Media-specific condition emulators
Слайд 11

Embedded Agent White Box testing: Module testing API testing DUT-specific management

Embedded Agent

White Box testing:
Module testing
API testing
DUT-specific management

Слайд 12

Proxy Agent Configure DUT Access to external equipment

Proxy Agent

Configure DUT
Access to external equipment

Слайд 13

Test Engine Components Dispatcher Runs/terminates all modules Handles user interrupt events

Test Engine Components

Dispatcher
Runs/terminates all modules
Handles user interrupt events
Builder
Based on autoconf/automake/m4
Cross compilation

of Test Agents
Logger
RCF (Remote Control Facility)‏
Configurator
Tester
Слайд 14

Logger Provide unified logging interface for all TE components: Test Engine

Logger

Provide unified logging interface for all TE components: Test Engine and

Test Agents
Time synchronization between Agents and Engine
Sort out logs by timestamps
Define a number of log levels
Gather log dumps from Test Agents
Extended format qualifiers (to log files, memory dumps, packets)‏
Save log messages into a binary file
Слайд 15

Remote Control Facility Starts Test Agents according to configuration file or

Remote Control Facility

Starts Test Agents according to configuration file or on-fly

requests
Provides access to services exported by Test Agents
Controls concurrent access to Test Agents
Слайд 16

Configurator Loads start-up configuration of Agents Keeps track of configuration changes

Configurator

Loads start-up configuration of Agents
Keeps track of configuration changes
Rolls back configuration

to a specified point
Restores configuration after Test Agent crash
Save configurations in XML format
Слайд 17

Configuration Tree /agent:Agt_A/interface:eth0 = /agent:Agt_A/interface:eth0/index: = 2 /agent:Agt_A/interface:eth0/link_addr: = 00:16:d3:26:98:91 /agent:Agt_A/interface:eth0/mtu:

Configuration Tree

/agent:Agt_A/interface:eth0 =
/agent:Agt_A/interface:eth0/index: = 2
/agent:Agt_A/interface:eth0/link_addr: = 00:16:d3:26:98:91
/agent:Agt_A/interface:eth0/mtu:

= 1500
/agent:Agt_A/interface:eth0/net_addr:192.168.56.1 = 24
/agent:Agt_A/interface:eth0/net_addr:192.168.56.1/broadcast: = 192.168.56.255
/agent:Agt_A/interface:eth0/net_addr:fe80::216:d3ff:fe26:9891 = 64
/agent:Agt_A/interface:eth0/net_addr:fec0::1 = 48
/agent:Agt_A/interface:eth0/status: = 1

/agent:Agt_A/route:192.168.56.0|24 = 0.0.0.0
/agent:Agt_A/route:192.168.56.0|24/dev: = eth0
/agent:Agt_A/route:192.168.56.0|24/irtt: = 0
/agent:Agt_A/route:192.168.56.0|24/mtu: = 0
/agent:Agt_A/route:192.168.56.0|24/win: = 0

Слайд 18

Tester Structured test suite organization Test iteration by parameters Test Agents

Tester

Structured test suite organization
Test iteration by parameters
Test Agents can be automatically

preconfigured before test run
Requirement validation
Слайд 19

TE auxiliary tools Test Coverage Estimation Report Generator Test Result Comparator

TE auxiliary tools

Test Coverage Estimation
Report Generator
Test Result Comparator

Слайд 20

HG Testing Proxy agents Parametrization Networking capabilities Serial console logging Management interfaces VCM WebUI cutils

HG Testing

Proxy agents
Parametrization
Networking capabilities
Serial console logging
Management interfaces
VCM
WebUI
cutils

Слайд 21

Configuration 1

Configuration 1

Слайд 22

Configuration 2

Configuration 2