转自 Engineering Long-Lasting Software

1 class MyClass 2 def foo(x,y,z) 3 if x 4 if (y && z) then bar(0) end 5 else 6 bar(1) 7 end 8 end 9 def bar(x) ; @w = x ; end 10 end http://pastebin.com/aNhQjhbT

How much testing is enough? A poor but unfortunately widely-given answer is “As much as you can do before the shipping deadline.” A very coarse-grained alternative is the code-to-test ratio, the number of non-comment lines of code divided by number of lines of tests of all types. In production systems, this ratio is usually less than 1, that is, there are more lines of test than lines of app code. The command rake stats issued in the root directory of a Rails app computes this ratio based on the number of lines of RSpec tests and Cucumber scenarios.

A more precise way to approach the question is in terms of code coverage. Since the goal of testing is to exercise the subject code in at least the same ways it would be exercised in production, what fraction of those possibilities is actually exercised by the test suite? Surprisingly, measuring coverage is not as straightforward as you might suspect. Here is a simple fragment of code and the definitions of several commonly-used coverage terms as they apply to the example.

S0 or Method coverage: Is every method executed at least once by the test suite? Satisfying S0 requires calling foo and bar at least once each. Sometimes written with a subscript, S0.

S1 or Call coverage or Entry/Exit coverage: Has each method been called from every place it could be called? Satisfying S1 requires calling bar from both line 4 and line 6. C0 or Statement coverage: Is every statement of the source code executed at least once by the test suite, counting both branches of a conditional as a single statement? In addition to calling bar, satisfying C0 would require calling foo at least once with x true (otherwise the statement in line 4 will never be executed), and at least once with y false.

C1 or Branch coverage: Has each branch been taken in each direction at least once? Satisfying C1 would require calling foo with both false and true values of x and with values of y and z such that y && z in line 4 evaluates once to true and once to false. A more stringent condition, decision coverage, requires that each subexpression that independently affects a conditional expression be evaluated to true and false. In this example, a test would additionally have to separately set y and z so that the condition y && z fails once for y being false and once for z being false. C2 or Path coverage: Has every possible route through the code been executed? In this simple example, where x,y,z are treated as booleans, there are 8 possible paths.

Modified Condition/Decision Coverage (MCDC) combines a subset of the above levels: Every point of entry and exit in the program has been invoked at least once, every decision in the program has taken all possible outcomes at least once, and each condition in a decision has been shown to independently affect that decision’s outcome.

Achieving C0 coverage is relatively straightforward, and a goal of 100% C0 coverage is not unreasonable. Achieving C1 coverage is more difficult since test cases must be constructed more carefully to ensure each branch is taken at least once in each direction. C2 coverage is most difficult of all, and not all testing experts agree on the additional value of achieving 100% path coverage. Therefore, code coverage statistics are most valuable to the extent that they highlight undertested or untested parts of the code and show the overall comprehensiveness of your test suite. The next screencast shows how to use the SimpleCov Ruby gem (included in the bookware) to quickly check the C0 coverage of your RSpec tests.

顺便简单介绍下这本书，伯克利的教材。作者之一是那个计算机软硬件接口的 DavidPatterson。质量应该是完全可以保证的。覆盖很广（但只讲最关键的地方），ruby，rails，BDD，TDD，Refactoring,Design Pattern....但讲的很到位，就是太难了。。。