Software Engineer in Test

Since Mike Kirby (one of our Whitebox test team leads) doesn't have a blog, I thought I'd post some of our "Software Engineer in Test" training materials.

These are the subject areas and techniques we expect our Test Engineers to understand and the skills we want them to have to effectively perform whitebox testing in our group.

Software Engineer in Test level 1:

Week 1 Introduction
Read the following chapters from _Testing Object Oriented Systems_:
Chapter 3 "Testing: A Brief Introduction"
Chapter 4 "Testing Object-oriented Software" sections 4.1.2 -4.1.3 "Fault Model"
Chapter 4 sections 4.2.1 - 4.2.5 "Effects of the Paradigm"
Chapter 4 sections 4.4 "Coverage Models"
Chapter 5 "Test Models"
Chapter 8 "A Tester's Guide to UML"
http://dn.codegear.com/article/31863
Chapter 10.1 "Class Test and Integration"
Chapter 10.2 "Preliminaries" (Code coverage, Path Sensitization, Domain Testing)
Chapter 16 "Test Automation" 4pgs
Chapter 17 sections 1-3 & 6-7 "Assertions"
Chapter 19 sections 1-3 "Test Harness Design"

Week 2 Source Control Management
Create a SVN repository on your linux box, and check out sample the sample code to work through the examples.
Read chapter 2 "Basic Usage" and chapter 4 "Branching and Merging" of the subversion book.
http://svnbook.red-bean.com/en/1.4/svn-book.pdf
Test Development repository structure:
(links to our internal wiki)

You will be asked to give a live demonstration of the following repository activities:
How to create branches
How to rollback changes
How to merge changes
How to resolve conflicts
How to create and apply patches
Demonstrate a 3 way merging using KDiff3

Week 3 Application Runtime Analysis
Read Valgrind "Quick Start Guide"
http://valgrind.org/docs/manual/QuickStart.html

Read chapters 1-3 "Intro", "Using Valgrind" , "Using Valgrind - Advanced Topics"
Valgrind - http://valgrind.org/docs/manual/manual.html

Read chapter 4 on Valgrind-memcheck
Build, run sampeles and find the memory leaks
Create/load Valgrind-memcheck supressions files

Read chapter 6 on Valgrind-callgrind
Build, run sampeles and generate callgraphs
Convert callgraphs to images

Read chapter 5 on Valgrind-cachegrind
Generate cachegrind output files
Review cachegrind output with KCachegrind

Week 4 Quality characteristics for Whitebox
Reliability
Maintainability
Efficiency
Functionality

Week 5 Non-Whitebox Essentials
Monitoring
Syslog and logger and Syslog-ng
http://linux.die.net/man/8/syslogd
http://linux.die.net/man/5/syslog.conf
http://linux.die.net/man/1/logger

SNMP + Cacti
Probing the PROC filesystem

PostgreSQL
(link to our internal wiki)

Pelco Package utilities
(link to our internal wiki)

Using Packet Sniffers
tcpdump
whireshark

Pelco/Endura:
Syseng + Pulse + Crucible + Fisheye + Jira + Pelcoverse + PPM
(link to our internal wiki)

Endura Developer Reference
(link to our internal wiki)



Software Engineer in Test level 2:

Read the following chapters from _Testing Object Oriented Systems_:
Chapter 10.3 "Method Scope Test Design Patterns" (Recursive,Polymorphic)
Chapter 10.4 "Class Scope Test Design Patterns" (Invariant Boundaries, non/Modal Class Test)
Chapter 10.5 "Flattend Class Scope Test Design" (Hierarch Test)
Chapter 13.1 "Integration in Object-oriented Development"
Chapter 13.2 "Integration Patterns"
Chapter 14.3 "Implementation-specific Capabilities" (Configuration,Compatability,Performance,Fault Tolerance)
Chapter 15 "Regression Testing"
Chapter 17 section 17.4.5 "Assertions - C++"
Chapter 17 section 17.5 "Assertions - Depolyment"
Chapter 17 section 17.8 "Assertions - Notes"

Presenting Quality sub-characteristic:
Security
Time behavior
Resource utilization
Maturity
Fault tolerance
Analyzability
Testability
Stability

Fundamental Design Principles
Maximize cohesion, minimize coupling
Subclasses should be substitutable
Favor association over inheritance
Base the design on the invariants
Encapsulate design decisions
Minimize overall complexity

High Leverage Patterns
Factory method
Abstract factory
Model-View-Controller
Bridge / Strategy
Composite

How to write Evil, untestable code
http://googletesting.blogspot.com/2008/07/how-to-write-3v1l-untestable-code.html

Writing Testable Code
http://googletesting.blogspot.com/2008/08/by-miko-hevery-so-you-decided-to.html

The Difference Between Mocks And Stubs
http://googletesting.blogspot.com/2008/06/tott-friends-you-can-depend-on.html
http://martinfowler.com/articles/mocksArentStubs.html


Choosing Between The Differences
http://martinfowler.com/articles/mocksArentStubs.html#ChoosingBetweenTheDifferences
(Fixture setup, Test isolateion, Coupling, Design style)

Test Driven Development
http://googletesting.blogspot.com/2008/09/test-first-is-fun_08.html

Factories
http://googletesting.blogspot.com/2008/09/where-have-all-new-operators-gone.html

Dependency Injection | Seperate application instantiation from application logic
http://googletesting.blogspot.com/2008/07/how-to-think-about-new-operator-with.html

Singletons
http://googletesting.blogspot.com/2008/05/tott-using-dependancy-injection-to.html
http://misko.hevery.com/2008/08/17/singletons-are-pathological-liars/
http://misko.hevery.com/2008/08/25/root-cause-of-singletons/

The Invisible Branch: Differences between statement coverage and branch coverage
http://googletesting.blogspot.com/2008/05/tott-invisible-branch.html

Google C++ Testing Framework
http://googletesting.blogspot.com/2008/07/tott-expect-vs-assert.html

Must Pass and Would be nice to Pass Tests
http://googletesting.blogspot.com/2008/08/progressive-developer-knows-that-in.html

Code Coverage
http://googletesting.blogspot.com/2008/03/tott-understanding-your-coverage-data.html

Too Many Tests
http://googletesting.blogspot.com/2008/02/in-movie-amadeus-austrian-emperor.html

Test Messages
http://googletesting.blogspot.com/2008/02/tott-stroop-effect.html

Automating tests vs. test-automation
http://googletesting.blogspot.com/2007/10/automating-tests-vs-test-automation.html

Performance Testing
http://googletesting.blogspot.com/2007/10/performance-testing.html

Testing Applications and APIs
http://googletesting.blogspot.com/2007/09/testing-applications-and-apis.html

Copeland’s GTAC Video
http://youtube.com/watch?v=pDtfMYyaTQY

Stubs Speed up Your Unit Tests
http://googletesting.blogspot.com/2007/04/tott-stubs-speed-up-your-unit-tests.html

Better Stubbing in Python
http://googletesting.blogspot.com/2007/01/better-stubbing-in-python.html

Non-Whitebox Essentials
Static and Dynamic Libraries
http://www.yolinux.com/TUTORIALS/LibraryArchives-StaticAndDynamic.html

Endura
(links to internal wiki)

SQLite
http://www.sqlite.org

Tools and Frameworks
C++ http://www.boost.org/
Python + SWIG http://www.swig.org/
NoBug http://freshmeat.net/projects/nobug/
GUPnP http://freshmeat.net/projects/gupnp/
Java UPNPLib http://freshmeat.net/projects/upnplib/
Memcheck http://freshmeat.net/projects/memcheck/
MPatrol http://freshmeat.net/projects/mpatrol
http://www.cbmamiga.demon.co.uk/mpatrol/mpatrol_44.html
Demalloc http://freshmeat.net/projects/dmalloc/
http://dmalloc.com/docs/latest/online/dmalloc_18.html
SWIG http://www.swig.org/
GProf
Gcov + Lcov

Static Analysis
Using Understand
Review complexity
Generate static analysis reports
Coupling + Cohesion

Doxygen Commands
http://www.stack.nl/~dimitri/doxygen/commands.html
mainpage,subpage,brief,section,subsection,paragraph,details,li
bug,exception,test,todo,dot,image

Doxygen Call and Caller Graphs
Doxygen Dependency Graph

GNU Binary Utilities
http://sourceware.org/binutils/docs/binutils/index.html#Top
addr2line
nm
c++filt

RPM Packaging
(links to internal wiki)

Pulse personal builds
(links to internal wiki)

Syslog-ng
(links to internal wiki)

Sustaining Log Parser
(links to internal wiki)

Stream Monitor
(links to internal wiki)

Dynamically Linked Application Profiling
(links to internal wiki)

Integrating Existing Code
(links to internal wiki)

Engineering Process Improvement
(links to internal wiki)

ISO9126 Quality Characteristics
(links to internal wiki)

Multicast Tunneling
(links to internal wiki)

Test API
(links to internal wiki)