python-testing/reference.html at gh-pages · swcarpentry/python-testing · GitHub

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8">
    <meta name="generator" content="pandoc">
    <title>Software Carpentry: Testing</title>
    <link rel="shortcut icon" type="image/x-icon" href="/favicon.ico" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <link rel="stylesheet" type="text/css" href="css/bootstrap/bootstrap.css" />
    <link rel="stylesheet" type="text/css" href="css/bootstrap/bootstrap-theme.css" />
    <link rel="stylesheet" type="text/css" href="css/swc.css" />
    <link rel="alternate" type="application/rss+xml" title="Software Carpentry Blog" href="http://software-carpentry.org/feed.xml"/>
    <meta charset="UTF-8" />
    <!-- HTML5 shim, for IE6-8 support of HTML5 elements -->
    <!--[if lt IE 9]>
      <script src="http://html5shim.googlecode.com/svn/trunk/html5.js"></script>
    <![endif]-->
  </head>
  <body class="lesson">
    <div class="container card">
      <div class="banner">
        <a href="http://software-carpentry.org" title="Software Carpentry">
          <img alt="Software Carpentry banner" src="img/software-carpentry-banner.png" />
        </a>
      </div>
      <article>
      <div class="row">
        <div class="col-md-10 col-md-offset-1">
                    <a href="index.html"><h1 class="title">Testing</h1></a>
          <h2 class="subtitle">Reference</h2>
          <h2 id="basics-of-testing"><a href="01-basics.html">Basics of Testing</a></h2>
<ul>
<li>Tests compare that the result observed from running code is the same as what was expected ahead of time.</li>
<li>Tests should be written at the same time as the code they are testing is written.</li>
<li>Assertions and exceptions are like alarm systems embedded in the software, guarding against exceptional bahavior.</li>
<li>Unit tests try to test the smallest pieces of code possible, usually functions and methods.</li>
<li>Integration tests make sure that code units work together properly.</li>
<li>Regression tests ensure that everything works the same today as it did yesterday.</li>
</ul>
<h2 id="assertions"><a href="02-assertions.html">Assertions</a></h2>
<ul>
<li>Assertions are one line tests embedded in code.</li>
<li>The <code>assert</code> keyword is used to set an assertion.</li>
<li>Assertions halt execution if the argument is false.</li>
<li>Assertions do nothing if the argument is true.</li>
<li>The <code>numpy.testing</code> module provides tools numeric testing.</li>
<li>Assertions are the building blocks of tests.</li>
</ul>
<h2 id="exceptions"><a href="03-exceptions.html">Exceptions</a></h2>
<ul>
<li>Exceptions are effectively specialized runtime tests</li>
<li>Exceptions can be caught and handled with a try-except block</li>
<li>Many built-in Exception types are available</li>
</ul>
<h2 id="unit-tests"><a href="04-units.html">Unit Tests</a></h2>
<ul>
<li>Functions are the atomistic unit of software.</li>
<li>Simpler units are easier to test than complex ones.</li>
<li>A single unit test is a function containing assertions.</li>
<li>Such a unit test is run just like any other function.</li>
<li><strong>Running tests one at a time is pretty tedious.</strong> (let’s use a framework instead)</li>
</ul>
<h2 id="running-tests-with-pytest"><a href="05-pytest.html">Running Tests with Pytest</a></h2>
<ul>
<li>The <code>py.test</code> command collects and runs tests starting with <code>Test</code> or <code>test_</code>.</li>
<li><code>.</code> means the test passed</li>
<li><code>F</code> means the test failed or erred</li>
<li><code>x</code> is a known failure</li>
<li><code>s</code> is a purposefully skipped test</li>
</ul>
<h2 id="edge-and-corner-cases"><a href="06-edges.html">Edge and Corner Cases</a></h2>
<ul>
<li>Functions often fail at the edge of their range of validity</li>
<li>Edge case tests query the limits of a function’s behavior</li>
<li>Corner cases are where two edge cases meet</li>
</ul>
<h2 id="integration-and-regression-tests"><a href="07-integration.html">Integration and Regression Tests</a></h2>
<ul>
<li>Integration tests interrogate the coopration of pieces of the software</li>
<li>Regression tests use past behavior as the expected result</li>
</ul>
<h2 id="continuous-integration"><a href="08-ci.html">Continuous Integration</a></h2>
<ul>
<li>Servers exist for automatically running your tests</li>
<li>Running the tests can be triggered by a GitHub pull request</li>
<li>CI allows cross platform build testing</li>
<li>A <code>.travis.yml</code> file configures a build on the travis-ci servers</li>
<li>Many free CI servers are available</li>
</ul>
<h2 id="test-driven-development"><a href="09-tdd.html">Test Driven Development</a></h2>
<ul>
<li>Test driven development is a common software development technique</li>
<li>By writing the tests first, the function requirements are very explicit</li>
<li>TDD is not for everyone</li>
<li>TDD requires vigilance for success (cheating leads to failure)</li>
</ul>
<h2 id="fixtures"><a href="10-fixtures.html">Fixtures</a></h2>
<ul>
<li>It may be necessary to set up “fixtures” composing the test environment.</li>
</ul>
<h2 id="glossary">Glossary</h2>
<ul>
<li><code>assert</code> is a keyword that halts code execution when its argument is false</li>
<li><code>Exceptions</code> are customizeable cousin of assertions.</li>
<li><code>try</code> is a keyword that guards a piece of code which may throw an exception.</li>
<li><code>except</code> is a keyword used to catch and carefully handle that exception</li>
<li><code>pytest</code> is a python package with testing utilities</li>
<li><code>py.test</code> is a command-line program that collects and runs unit tests.</li>
<li><em>Unit Tests</em> investigate the behavior of units of code (such as functions, classes, or data structures).</li>
<li><em>Regression Tests</em> defend against new bugs, or regressions, which might appear due to new software and updates.</li>
<li><em>Integration Tests</em> check that various pieces of the software work together as expected.</li>
<li><em>Continuous Integration</em> automates the building and testing process accross platforms.</li>
<li><em>Test-driven Development</em> is a software development strategy in which the tests are written before the code.</li>
</ul>
        </div>
      </div>
      </article>
      <div class="footer">
        <a class="label swc-blue-bg" href="http://software-carpentry.org">Software Carpentry</a>
        <a class="label swc-blue-bg" href="https://github.com/swcarpentry/lesson-template">Source</a>
        <a class="label swc-blue-bg" href="mailto:admin@software-carpentry.org">Contact</a>
        <a class="label swc-blue-bg" href="LICENSE.html">License</a>
      </div>
    </div>
    <!-- Javascript placed at the end of the document so the pages load faster -->
    <script src="http://software-carpentry.org/v5/js/jquery-1.9.1.min.js"></script>
    <script src="css/bootstrap/bootstrap-js/bootstrap.js"></script>
  </body>
</html>