Download files gives an error

1

Hi,

I’m trying to download a finished workflow: “reana-client download --workflow recast-399158_N2C1_WZ_300_0_3L”

gives

==> ERROR: Workflow recast-399158_N2C1_WZ_300_0_3L could not be retrieved: None is not of type ‘object’

Failed validating ‘type’ in schema[‘properties’][‘parameters’]:
{‘type’: ‘object’}

On instance[‘parameters’]:
None

Cheers,
Eric

2

Hi @eballabe

I assume the downloading of individual files works well? For example, doing the following:

$ reana-client ls -w recast-399158_N2C1_WZ_300_0_3L | grep pdf
$ reana-client download -w recast-399158_N2C1_WZ_300_0_3L _yadage/yadage_workflow_instance.pdf

should download for you the workflow execution graph.

If this works well, but the bare download command is leading to the above exception, then it looks like there is something wrong with the definition of outputs in your REANA specification.

Note that in reana.yaml the final outputs could be defined as a list of files, see recast example.

How does your reana.yaml looks like?

3

Hi Tibor,

yes, the download of individual files works and I get the list of pdf files in the workspace with the first command and the workflow execution graph with the second command.

This is my recast.yml. I just modified the input DAODs from the ATLAS pmssm-collection.

#define the name to execute SusySusx recasting
name: atlas/ana-susy-2019-09-onshell

metadata:
  author: maparo
  short_description: 'Full RECAST of SUSY EWK 3L onShell analysis'

spec:
  workflow: workflow.yml

example_inputs:
  default:
    initdata:
      dxaod_mc16a: 'root://eosuser.cern.ch//eos/user/e/eballabe/ewk-samples/mc16_13TeV.399158.MGPy8EG_A14N23LO_C1N2_WZ_300p0_0p0_3L_2L7.deriv.DAOD_SUSY2.e8533_a875_r10201_p3703'
      dxaod_mc16d: 'root://eosuser.cern.ch//eos/user/e/eballabe/ewk-samples/mc16_13TeV.399158.MGPy8EG_A14N23LO_C1N2_WZ_300p0_0p0_3L_2L7.deriv.DAOD_SUSY2.e8533_a875_r10724_p3712'
      dxaod_mc16e: 'root://eosuser.cern.ch//eos/user/e/eballabe/ewk-samples/mc16_13TeV.399158.MGPy8EG_A14N23LO_C1N2_WZ_300p0_0p0_3L_2L7.deriv.DAOD_SUSY2.e8533_a875_r9364_p3703'
      extra: '--isAtlfast --nevents -1 --doSyst'
      analysis: ['WZ']
      mX: '300' # only used for naming
      mY: '0' # only used for naming
4

Thanks for the example. If RECAST does not create any particular outputs clause at the top level of reana.yaml for you, which I suspect it does not, then nothing will be downloaded when you use a bare reana-client download.

You could therefore do some “manual” downloads of desired files and directories such as:

$ reana-client download -w myanalysis.42 myfile1 myfile2
$ reana-client download -w myanalysis.42 mydir

See the reana-client download command documentation.

Note that you could also script automated downloads from your workspace by combining ls and download commands. For example, to download all PDF files produced by the workflow in your workspace, you could do:

$ reana-client ls -w recast-399158_N2C1_WZ_300_0_3L --filter name=pdf | \
      awk 'NR>1 {print " " $1}' | \
      xargs reana-client download -w recast-399158_N2C1_WZ_300_0_3L

P.S. We’ll look at fixing the error that the client gave you in these cases when reana.yaml does not contain proper global outputs keywords. I just opened an issue in our tracker to improve this.

1 Like