Check within same timespan

README.md

@@ -1,5 +1,5 @@
-# robotframeworkNL - the oneliner
-robotframeworkNL is a proving ground to boost Robot framework closer to Natural Language.
+# RobotNL - the oneliner
+RobotNL is a proving ground to boost Robot framework closer to Natural Language.
 
 ## Introduction
 This project is an extension to [Robot framework](https://robotframework.org/) and although [Robot framework](https://robotframework.org/) made a very good step towards the goals of [keyword-driven testing ](https://en.wikipedia.org/wiki/Keyword-driven_testing) to make it readable for all stakeholders, there is still quite a lot of syntax involved that keeps test cases from really staying concise and to-the-point. In this project we will be introducing concepts to lift [Robot framework](https://robotframework.org/) to an even higher level.
@@ -55,17 +55,20 @@ Using *Check that* keywords offers a large reduction in the need for variables i
 
 ### Time constraints
 
-*Check that* offers support for executing checks that may take some time to complete. When using the optional `within` argument, followed by a time duration, *Check that* will apply *smart polling* to re-evaluate the expression and the keywords during the given period. Specifying the time limit is done using the standard [Robot Framework time format](https://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html#toc-entry-176). It is advised to use a realistic time duration. This sets the correct expectation for the reader and helps robotnl optimise its polling algorithm.
+*Check that* offers support for executing checks that may take some time to complete. When using the optional `within` argument, followed by a time duration, *Check that* will apply *smart polling* to re-evaluate the expression and the keywords during the given period. Specifying the time limit is done using the standard [Robot Framework time format](https://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html#toc-entry-176). It is advised to use a realistic time duration. This sets the correct expectation for the reader and helps RobotNL optimise its polling algorithm.
 
 |**Example using time constaints**||||||
 |---|---|---|---|---|---|
 | Request elevator at floor | 3 |||||
 | Check that | elevator doors are closed | within | 20 seconds ||
 | Check that | current elevator floor | equals | 3 | within | 1 minute |
+| Check that | elevator doors are opened | within | same timespan ||
+
+The last line in the example uses the special argument `same timespan` to indicate that this check is linked to the same time constraint as the previous check. So, once the doors are closed, the elevator should not only reach the requested floor within 1 minute, but also the doors should be opened within that same 1-minute timespan. To use `within    same timespan` you must have executed a check with a time constraint before. This is not necessarily the directly preceding line. The most recent time constraint set within scope is used as starting point. For test suites, that is the current test suite, for keywords that is the current keyword.
 
 ### Hybrid manual testing
 
-To manually interact with your automated test run during testing or test case development, robotnl offers the *Check manual* and *Check interactive* keywords. These keywords can be included at any point in the test case to suspend the test run at the current position for user input.
+To manually interact with your automated test run during testing or test case development, RobotNL offers the *Check manual* and *Check interactive* keywords. These keywords can be included at any point in the test case to suspend the test run at the current position for user input.
 
 ***Check Manual*** allows asking the tester a question. The question typically requests manual verification of an expected outcome. The answer will PASS or FAIL the test case, which is also reflected in the test report.
 

atest/robotNL tests/01__Check that/Time constraints/01__Check within.robot

@@ -0,0 +1,35 @@
+*** Settings ***
+Resource          base.resource
+
+
+*** Test Cases ***
+Single action passes within time span
+    Check that    a delay of 10ms completes    within    1 second
+
+Polling action passes within time span
+    Check that    pass at the third attempt    within    2 seconds
+
+Keyword passes, but too late
+    Run keyword and expect error    *too late*
+    ...                             Check that    a delay of 50ms completes    within    10ms
+
+Single action fails after time span
+    ${message}=    Run keyword and expect error    CheckFailed*
+    ...            Check that    a delay of 200ms completes    equals    ${False}    within    100ms
+    Check that     ${message}    does not contain text    too late
+
+Polling action keeps failing within time span
+    Run keyword and expect error    CheckFailed*
+    ...                             Check that    Apple    equals    Pear    within    100ms
+
+Time constraint from keyword
+    Run keyword and expect error    *within 1/100th of a second [[]10 milliseconds[]] (too late)*
+    ...                             Check that    a delay of 100ms completes    within    1/100th of a second
+
+Invalid timespan text
+    Run keyword and expect error    *Invalid time string 'the blink of an eye'.
+    ...                             Check that    True    within    the blink of an eye
+
+Invalid timespan from keyword
+    Run keyword and expect error    *Invalid time string 'My object'.
+    ...                             Check that    True    within    not a timespan

atest/robotNL tests/01__Check that/Time constraints/02__Check within same timespan.robot

@@ -0,0 +1,182 @@
+*** Settings ***
+Resource          base.resource
+
+
+*** Test Cases ***
+The first check in a suite cannot refer to 'same timespan'
+    TRY
+        Check that    True    within    same timespan
+    EXCEPT    Joint timespan expected, but was not set*    type=GLOB
+        No Operation
+    ELSE
+        Fail    Expected fail did not occur
+    END
+
+Two checks pass within joint time constraint
+    Check that    a delay of 10ms completes    within    100ms
+    Check that    a delay of 10ms completes    within    same timespan
+
+Second check passes, but not within the joint time constraint
+    Check that    a delay of 10ms completes    within    100ms
+    TRY
+        Check postcondition    a delay of 100ms completes    within    same timespan
+    EXCEPT    *too late*    type=GLOB
+        No Operation
+    ELSE
+        Fail    Expected fail did not occur
+    END
+
+Setup new timespan to carry over to next test
+    Comment    Setup only. Actual checks are executed in the next test.
+    Check that    a delay of 10ms completes    within    100ms
+
+A timespan can carry over to the next Test
+    [Documentation]    Carry over of a timespan to the next test within a suite is
+    ...    useful when you have an expensive action that has multiple asynchronous
+    ...    results that you want to report and tag independently.
+    Comment    Expects the running timespan from the previous test
+    Check that    a delay of 10ms completes    within    same timespan
+    TRY
+        Check that    a delay of 90ms completes    within    same timespan
+    EXCEPT    *too late*    type=GLOB
+        Comment    Total delays add up to 110ms for the alotted 100ms.
+    ELSE
+        Fail    Expected fail did not occur
+    END
+
+A check without time constraint does not end the running timespan
+    Check that    a delay of 10ms completes    within    100ms
+    Check that    True
+    Check that    a delay of 10ms completes    within    same timespan
+
+Timespans can be reused inside control structures
+    Check that    a delay of 10ms completes    within    100ms
+    FOR    ${i}    IN RANGE    2
+        Check that    a delay of 10ms completes    within    same timespan
+    END
+
+Checks after the timespan expired are executed exactly once
+    Reset counter
+    TRY
+        Check that    ${False}    within    10ms
+    EXCEPT    CheckFailed*    type=GLOB
+        No Operation
+    ELSE
+        Fail    Expected fail did not occur
+    END
+    TRY
+        Check that    counter increases, then pass    within    same timespan
+    EXCEPT    *too late*    type=GLOB
+        No Operation
+    ELSE
+        Fail    Expected fail did not occur
+    END
+    Check that    Counter value    equals    1
+
+Keywords do not affect the timespan of the Test
+    [Documentation]    The keyword that separates the check that sets the timespan
+    ...    from the one using it, sets a new timnespan internally of just 10ms. If
+    ...    If the timespans are not properly separated, the second check will fail
+    ...    as 'too late'.
+    Check that    a delay of 10ms completes    within    100ms
+    Keyword that uses a shorter check within
+    Check that    a delay of 20ms completes    within    same timespan
+
+Keywords do not have access to the timespan of the calling test
+    Check that    True    within    100ms
+    TRY
+        Keyword that tries to use existing timespan
+    EXCEPT    Joint timespan expected, but was not set*    type=GLOB
+        No Operation
+    ELSE
+        Fail    Expected fail did not occur
+    END
+
+Keywords do not affect the timespan of other keywords
+    Check that    True    within    50ms
+    Nested keyword that uses check within
+    TRY
+        Check that    True    within    same timespan
+    EXCEPT    *too late*    type=GLOB
+        Comment    Timespan already passed during the nested keyword
+    ELSE
+        Fail    Expected fail did not occur
+    END
+
+Reporting shows the original timespan
+    Check that    True    within    10ms
+    TRY
+        Check that    False    within    same timespan
+    EXCEPT    *within 10ms*    type=GLOB
+        Comment    Note that the original text is kept, so '10ms' instead of the formatted '10 milliseconds'
+    ELSE
+        Fail    Expected fail did not occur
+    END
+
+Reporting shows the original timespan including the keyword that produced it
+    Check that    True    within    1/100th of a second
+    TRY
+        Check that    False    within    same timespan
+    EXCEPT    *within 1/100th of a second [[]10 milliseconds[]]*    type=GLOB
+        Comment    Reports the keyword with its resulting value formatted between brackets
+    ELSE
+        Fail    Expected fail did not occur
+    END
+
+Errors in timespan cancel the running timespan
+    Check that    True    within    10ms
+    TRY
+        Check that    True    within    not a timespan
+    EXCEPT    *Invalid time string 'My object'.*    type=GLOB
+        No Operation
+    ELSE
+        Fail    Expected fail did not occur
+    END
+    TRY
+        Check that    True    within    same timespan
+    EXCEPT    Joint timespan expected, but was not set*    type=GLOB
+        No Operation
+    ELSE
+        Fail    Expected fail did not occur
+    END
+
+Timespan is kept from the first instance
+    Check that    a delay of 50ms completes    within    100ms
+    Check that    a delay of 10ms completes    within    same timespan
+    TRY
+        Check postcondition    a delay of 50ms completes    within    same timespan
+    EXCEPT    *too late*    type=GLOB
+        Comment    Would not have failed if the timer were reset at the first 'same timespan'
+    ELSE
+        Fail    Expected fail did not occur
+    END
+
+
+*** Keywords ***
+Keyword that uses a shorter check within
+    Check that    True    within    10ms
+
+Keyword that tries to use existing timespan
+    Check that    a delay of 10ms completes    within    same timespan
+
+Nested keyword that uses check within
+    Check that    a delay of 10ms completes    within    50ms
+    Keyword that uses a longer check within
+    TRY
+        Check that    a delay of 50ms completes    within    same timespan
+    EXCEPT    *too late**    type=GLOB
+        Comment    Would not have failed if the longer timespan were set
+    ELSE
+        Fail    Expected fail did not occur
+    END
+
+Keyword that uses a longer check within
+    TRY
+        Check that    a delay of 10ms completes    within    same timespan
+    EXCEPT    Joint timespan expected, but was not set*    type=GLOB
+        Comment    Would not have failed if the calling keyword's timespan were accessable
+    ELSE
+        Fail    Expected fail did not occur
+    END
+    Check that    a delay of 10ms completes    within    1 second
+    Check that    a delay of 10ms completes    within    same timespan

atest/robotNL tests/01__Check that/Time constraints/base.resource

@@ -0,0 +1,9 @@
+*** Settings ***
+Resource          ../base.resource
+Library           helper_keywords.py
+
+
+*** Keywords ***
+a delay of ${timespan} completes
+    Sleep    ${timespan}
+    RETURN    ${True}

atest/robotNL tests/01__Check that/Time constraints/helper_keywords.py

@@ -0,0 +1,42 @@
+from robot.api.deco import library, keyword
+
+
+@library
+class helper_keywords:
+    def __init__(self):
+        self.gen = self._fail_fail_pass_generator()
+        self.counter = 0
+
+    @staticmethod
+    def _fail_fail_pass_generator():
+        yield False
+        yield False
+        yield True
+
+    @keyword("Pass at the third attempt")
+    def pass_at_third_attempt(self):
+        return next(self.gen)
+
+    @keyword("Reset counter")
+    def reset_counter(self):
+        self.counter = 0
+
+    @keyword("counter increases, then pass")
+    def plus_counter(self):
+        self.counter += 1
+        return True
+
+    @keyword("counter value")
+    def counter_value(self):
+        return self.counter
+
+    @keyword("1/100th of a second")
+    def timespan_keyword(self):
+        return '0.01 sec'
+
+    @keyword("Not a timespan")
+    def return_object(self):
+        class MyObject:
+            def __str__(self):
+                return 'My object'
+        return MyObject()