Leakage of warnings about supporting modules

[jvoigtlaender]

Specifically, in this task: https://autotool.fmi.uni-due.de/route/aufgabe/571, the IOTestHelper uses tail.

If the student submits a compiling main module, the presence of tail in the helper module has no impact.

But if the student submits a non-compiling main module, then in addition to the compiler error message about their own code, they also get this message about IOTestHelper:

IOTestHelper.hs:11:66: warning: [GHC-63394] [-Wx-partial (in -Wextended-warnings)]
    In the use of ‘tail’
    (imported from Prelude, but defined in GHC.List):
    "This is a partial function, it throws an error on empty lists. Replace it with drop 1, or use pattern matching or Data.List.uncons instead. Consider refactoring to use Data.List.NonEmpty."

That's of course confusing.

Is there a way to systematically make sure only complaints about the student code surface?

Or to make sure that at configuration time the task type checks that the modules are compile-error-and-warning-free?

[jvoigtlaender]

Would

First, upon upload of the configuration, part of the configuration validity checking is whether the contained sample solution would itself pass all the checks that will be performed on student submissions eventually.

from #24 address this?

If that compilation run would be done with -Werror, so that warnings from the test suite module would surface as errors?

[jvoigtlaender]

@patritzenfeld, could the

Or to make sure that at configuration time the task type checks that the modules are compile-error-and-warning-free?

from above now be achieved by somehow tweaking this call:

haskell-template-task/src/Haskell/Template/Task.hs

Line 387 in 1a726f0

sequence_ $ testPhases reject inform s solutionFile modules config exts content path

?

Essentially moving all the configGhcWarnings to configGhcErrors in the config passed onwards there.

And maybe even doing the same for configHlintSuggestion and configHlintError?

That would make the checking of the provided sample solution more strict, demanding that it do not only trigger no errors, but also no warnings according to the task config.

[patritzenfeld]

Sure, that seems simple enough.

[jvoigtlaender]

Actually, while I still think the proposal with propagating warnings to errors during the sample solution checking is a good one, it probably does not address the original issue in the first comment here.

There, the problem was an "unexpected" (for the student) -Wx-partial warning due to an occurrence of tail in the test code. That warning was certainly not one that came from, or was affected by, settings in the config.

Likely it came from something like the analog (at the time) of one of these calls:

haskell-template-task/src/Haskell/Template/Task.hs

Lines 910 to 922 in 1a726f0

	do
	-- Reject if submission does not compile with provided hidden modules,
	-- but without Test module.
	compilation <- liftIO $ runInterpreter (compiler dirname config noTest)
	checkResult reject compilation reject Nothing $ const $ return ()
	-- Reject if submission does not compile with provided hidden modules.
	-- This only runs when allowModifying is set to True in the config
	-- and displays a custom message telling students not to change type signatures.
	when (runIdentity $ allowModifying config) $ do
	compilationWithTests <- liftIO $ runInterpreter $
	compiler dirname config modules
	checkResult reject compilationWithTests signatureError Nothing $ const $ return ()

haskell-template-task/src/Haskell/Template/Task.hs

Lines 933 to 936 in 1a726f0

	do
	-- Reject if test suite fails for submission.
	result <- liftIO $ runInterpreter (interpreter dirname config modules)
	checkResult reject result reject Nothing $ handleCounts reject inform

that appear (unlike compileWithArgsAndCheck) not to be affected by config settings about warnings vs. errors, instead being run with whatever the default ghc config options are (such as -Wx-partial).

To address that issue, maybe indeed a Bool needs to be propagated to prepareInterpreter which lets the check function make sure that at least the last of the above calls at "config upload time" is run with -Werror.