[Question] Steps in the pdf report

[olw-medcom]

Hello there,

I'm noticing a weird behaviour in the testcase pdf report (and the conformance report) where it generates an empty step?

This is my testcase code:

<steps>
      <interact id="step1UserData" desc="Step 1 - Provide CapabilityStatement">
         <request name="capabilityStatement"
                  inputType="MULTILINE_TEXT"
                  size="5"
                  required="true"
                  desc="Please supply the EDS Server Capability Statement"/>
      </interact>
      <group title="MedCom review step 1 asynchronously">
         <interact id="medcomReview1" desc="MedCom review" admin="true">
            <instruct desc="Please make sure the following requirements are addressed in the user input:"
                      showControls="false"
                      level="INFO"/>
            <instruct desc="Requirement 1" forceDisplay="true" showControls="false">
                "Please check whether the version is at least 8.2. You should be able to find it under 'software.version'"
              </instruct>
            <instruct desc="Requirement 2" forceDisplay="true" showControls="false">
                "Please ensure that updates, patches and deletes to the AuditEvent resources are disallowed. You should be able to find nothing under 'rest.resource.where(type = 'AuditEvent').interaction.where(code = 'update' or code = 'patch' or code = 'delete')'"
              </instruct>
            <request name="verdict"
                     desc="Step 1 - Verdict"
                     inputType="SELECT_SINGLE"
                     options="F1,F2,F3,F4,OK,Ej Relevant"
                     required="true"/>
            <request name="comment" desc="Step 1 - Comment" inputType="TEXT"/>
         </interact>
         <assign to="anyF1F2OrF3Results">not($medcomReview1{verdict} = 'F1' or $medcomReview1{verdict} = 'F2' or $medcomReview1{verdict} = 'F3')</assign>
         <verify id="medcomErrorCheck1"
                 handler="ExpressionValidator"
                 desc="Fail if any F1, F2 or F3 MedCom results"
                 stopOnError="true"
                 skipped="$anyF1F2OrF3Results">
            <input name="expression">$anyF1F2OrF3Results</input>
            <input name="failureMessage" asTemplate="true">"You may find additional information by clicking into the MedCom review step above"</input>
         </verify>
         <assign to="anyF4Results">not($medcomReview1{verdict} = 'F4')</assign>
         <verify id="medcomCheck1"
                 level="WARNING"
                 handler="ExpressionValidator"
                 desc="Warn if any F4 MedCom results"
                 skipped="$anyF4Results">
            <input name="expression">$anyF4Results</input>
            <input name="failureMessage" asTemplate="true">"You may find additional information by clicking into the MedCom review step above"</input>
         </verify>
      </group>
   </steps>

And this is how the generated PDF looks like:

OR:

As you can see there is this empty Step 2? Where does that come from and why has it been rendered?

Also, bonus question:
Is it possible somehow to include the instructions in the test case report? It seems it is only the descriptions on the request elements that is included and not the elements inside an element.

[costas80]

Hi @olw-medcom. The empty step is your group and it appears empty because it lacks a desc (description) element. As a general point, any non-hidden step is included in the report but I see how this could be confusing.

As a potential improvement here I can foresee the following approaches:

• Option 1: Exclude from the report steps that don't have a description.
• Option 2: When steps lack a description, default to using the step's title (the one displayed on the UI, whether explicitly provided or not). If a step defines no description or title, then this will still be included in the report as empty (basically matching what you see on the UI). This latter case would only come up if you e.g. specify something like a verify without a description (which would be odd).
• Option 3: Foresee a new attribute that allows you to specify if certain steps are included or not in the report.
• Option 4: Keep things unchanged.

I think I options 1 or 2 make most sense as option 3 is likely overkill (and something we could add later on if we really need it). Between these two options I think that option 2 makes most sense as it gives us fallbacks as opposed to excluding things. In addition, I would prefer keeping such container steps (group, if, foreach etc.) in the report as they add extra context and can serve as a means of pinpointing issues when a failure is due to a non-visible step (e.g. a bad assign or a hidden send). To summarise, I'd suggest option 2 - do you agree @olw-medcom?

Also, bonus question:
Is it possible somehow to include the instructions in the test case report? It seems it is only the descriptions on the request elements that is included and not the elements inside an element.

Would you mind creating this as a separate issue? It's a separate point and would be clearer to track as such (I'll add my comments when you create the issue).

[olw-medcom]

Yes I agree with option 2 @costas80 ^_^

PS: I will create the bonus question as a seperate issue.

[costas80]

Yes I agree with option 2 @costas80 ^_^

Thanks for confirming! I'll let you know when this is ready to pull.

[costas80]

Ready to pull @olw-medcom!

As discussed the title (where available) is now used as a fallback in case the desc is missing. For example, this test case where a group is used without a description ...

... will show the title in the report:

I trust this covers you @olw-medcom right?

[olw-medcom]

That works! Thanks @costas80!
I do wonder though.. Why is a group a "Step" in the report?

[costas80]

Why is a group a "Step" in the report?

Well, it is similar to an if or foreach or flow. Such steps have child steps but are also themselves steps. Granted, these could potentially be shown as "Group" etc. but we decided to refer to all of them as "Steps" for a simpler and more consistent presentation.

In case you are more asking on why are these included to begin with, it is because they add context, and also because they may be needed to convey issues that are not otherwise visible (imagine if a assign within the group failed).

[olw-medcom]

Ah right, that makes sense. Thanks @costas80