Skip to content

Commit 0d0dc25

Browse files
authored
Merge pull request #215 from nf-core/dev
Release 2.1.0
2 parents d1218d2 + 1282615 commit 0d0dc25

File tree

100 files changed

+2391
-494
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

100 files changed

+2391
-494
lines changed

.editorconfig

Lines changed: 0 additions & 33 deletions
This file was deleted.

.github/CONTRIBUTING.md

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# nf-core/oncoanalyser: Contributing Guidelines
1+
# `nf-core/oncoanalyser`: Contributing Guidelines
22

33
Hi there!
44
Many thanks for taking an interest in improving nf-core/oncoanalyser.
@@ -55,36 +55,36 @@ These tests are run both with the latest available version of `Nextflow` and als
5555

5656
:warning: Only in the unlikely and regretful event of a release happening with a bug.
5757

58-
- On your own fork, make a new branch `patch` based on `upstream/master`.
58+
- On your own fork, make a new branch `patch` based on `upstream/main` or `upstream/master`.
5959
- Fix the bug, and bump version (X.Y.Z+1).
60-
- A PR should be made on `master` from patch to directly this particular bug.
60+
- Open a pull-request from `patch` to `main`/`master` with the changes.
6161

6262
## Getting help
6363

6464
For further information/help, please consult the [nf-core/oncoanalyser documentation](https://nf-co.re/oncoanalyser/usage) and don't hesitate to get in touch on the nf-core Slack [#oncoanalyser](https://nfcore.slack.com/channels/oncoanalyser) channel ([join our Slack here](https://nf-co.re/join/slack)).
6565

6666
## Pipeline contribution conventions
6767

68-
To make the nf-core/oncoanalyser code and processing logic more understandable for new contributors and to ensure quality, we semi-standardise the way the code and other contributions are written.
68+
To make the `nf-core/oncoanalyser` code and processing logic more understandable for new contributors and to ensure quality, we semi-standardise the way the code and other contributions are written.
6969

7070
### Adding a new step
7171

7272
If you wish to contribute a new step, please use the following coding standards:
7373

74-
1. Define the corresponding input channel into your new process from the expected previous process channel
74+
1. Define the corresponding input channel into your new process from the expected previous process channel.
7575
2. Write the process block (see below).
7676
3. Define the output channel if needed (see below).
7777
4. Add any new parameters to `nextflow.config` with a default (see below).
7878
5. Add any new parameters to `nextflow_schema.json` with help text (via the `nf-core pipelines schema build` tool).
7979
6. Add sanity checks and validation for all relevant parameters.
8080
7. Perform local tests to validate that the new code works as expected.
81-
8. If applicable, add a new test command in `.github/workflow/ci.yml`.
81+
8. If applicable, add a new test in the `tests` directory.
8282
9. Update MultiQC config `assets/multiqc_config.yml` so relevant suffixes, file name clean up and module plots are in the appropriate order. If applicable, add a [MultiQC](https://https://multiqc.info/) module.
8383
10. Add a description of the output files and if relevant any appropriate images from the MultiQC report to `docs/output.md`.
8484

8585
### Default values
8686

87-
Parameters should be initialised / defined with default values in `nextflow.config` under the `params` scope.
87+
Parameters should be initialised / defined with default values within the `params` scope in `nextflow.config`.
8888

8989
Once there, use `nf-core pipelines schema build` to add to `nextflow_schema.json`.
9090

.github/ISSUE_TEMPLATE/bug_report.yml

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,6 @@ body:
99
1010
- [nf-core website: troubleshooting](https://nf-co.re/usage/troubleshooting)
1111
- [nf-core/oncoanalyser pipeline documentation](https://nf-co.re/oncoanalyser/usage)
12-
1312
- type: textarea
1413
id: description
1514
attributes:

.github/actions/get-shards/action.yml

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
name: "Get number of shards"
2+
description: "Get the number of nf-test shards for the current CI job"
3+
inputs:
4+
max_shards:
5+
description: "Maximum number of shards allowed"
6+
required: true
7+
paths:
8+
description: "Component paths to test"
9+
required: false
10+
tags:
11+
description: "Tags to pass as argument for nf-test --tag parameter"
12+
required: false
13+
outputs:
14+
shard:
15+
description: "Array of shard numbers"
16+
value: ${{ steps.shards.outputs.shard }}
17+
total_shards:
18+
description: "Total number of shards"
19+
value: ${{ steps.shards.outputs.total_shards }}
20+
runs:
21+
using: "composite"
22+
steps:
23+
- name: Install nf-test
24+
uses: nf-core/setup-nf-test@v1
25+
with:
26+
version: ${{ env.NFT_VER }}
27+
- name: Get number of shards
28+
id: shards
29+
shell: bash
30+
run: |
31+
# Run nf-test with dynamic parameter
32+
nftest_output=$(nf-test test \
33+
--profile +docker \
34+
$(if [ -n "${{ inputs.tags }}" ]; then echo "--tag ${{ inputs.tags }}"; fi) \
35+
--dry-run \
36+
--ci \
37+
--changed-since HEAD^) || {
38+
echo "nf-test command failed with exit code $?"
39+
echo "Full output: $nftest_output"
40+
exit 1
41+
}
42+
echo "nf-test dry-run output: $nftest_output"
43+
44+
# Default values for shard and total_shards
45+
shard="[]"
46+
total_shards=0
47+
48+
# Check if there are related tests
49+
if echo "$nftest_output" | grep -q 'No tests to execute'; then
50+
echo "No related tests found."
51+
else
52+
# Extract the number of related tests
53+
number_of_shards=$(echo "$nftest_output" | sed -n 's|.*Executed \([0-9]*\) tests.*|\1|p')
54+
if [[ -n "$number_of_shards" && "$number_of_shards" -gt 0 ]]; then
55+
shards_to_run=$(( $number_of_shards < ${{ inputs.max_shards }} ? $number_of_shards : ${{ inputs.max_shards }} ))
56+
shard=$(seq 1 "$shards_to_run" | jq -R . | jq -c -s .)
57+
total_shards="$shards_to_run"
58+
else
59+
echo "Unexpected output format. Falling back to default values."
60+
fi
61+
fi
62+
63+
# Write to GitHub Actions outputs
64+
echo "shard=$shard" >> $GITHUB_OUTPUT
65+
echo "total_shards=$total_shards" >> $GITHUB_OUTPUT
66+
67+
# Debugging output
68+
echo "Final shard array: $shard"
69+
echo "Total number of shards: $total_shards"

.github/actions/nf-test/action.yml

Lines changed: 113 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,113 @@
1+
name: "nf-test Action"
2+
description: "Runs nf-test with common setup steps"
3+
inputs:
4+
profile:
5+
description: "Profile to use"
6+
required: true
7+
shard:
8+
description: "Shard number for this CI job"
9+
required: true
10+
total_shards:
11+
description: "Total number of test shards(NOT the total number of matrix jobs)"
12+
required: true
13+
paths:
14+
description: "Test paths"
15+
required: true
16+
tags:
17+
description: "Tags to pass as argument for nf-test --tag parameter"
18+
required: false
19+
runs:
20+
using: "composite"
21+
steps:
22+
- name: Setup Nextflow
23+
uses: nf-core/setup-nextflow@v2
24+
with:
25+
version: "${{ env.NXF_VERSION }}"
26+
27+
- name: Set up Python
28+
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5
29+
with:
30+
python-version: "3.13"
31+
32+
- name: Install nf-test
33+
uses: nf-core/setup-nf-test@v1
34+
with:
35+
version: "${{ env.NFT_VER }}"
36+
install-pdiff: true
37+
38+
- name: Setup apptainer
39+
if: contains(inputs.profile, 'singularity')
40+
uses: eWaterCycle/setup-apptainer@main
41+
42+
- name: Set up Singularity
43+
if: contains(inputs.profile, 'singularity')
44+
shell: bash
45+
run: |
46+
mkdir -p $NXF_SINGULARITY_CACHEDIR
47+
mkdir -p $NXF_SINGULARITY_LIBRARYDIR
48+
49+
- name: Conda setup
50+
if: contains(inputs.profile, 'conda')
51+
uses: conda-incubator/setup-miniconda@505e6394dae86d6a5c7fbb6e3fb8938e3e863830 # v3
52+
with:
53+
auto-update-conda: true
54+
conda-solver: libmamba
55+
conda-remove-defaults: true
56+
57+
# TODO Skip failing conda tests and document their failures
58+
# https://github.com/nf-core/modules/issues/7017
59+
- name: Run nf-test
60+
shell: bash
61+
env:
62+
NFT_DIFF: ${{ env.NFT_DIFF }}
63+
NFT_DIFF_ARGS: ${{ env.NFT_DIFF_ARGS }}
64+
NFT_WORKDIR: ${{ env.NFT_WORKDIR }}
65+
run: |
66+
nf-test test \
67+
--profile=+${{ inputs.profile }} \
68+
$(if [ -n "${{ inputs.tags }}" ]; then echo "--tag ${{ inputs.tags }}"; fi) \
69+
--ci \
70+
--changed-since HEAD^ \
71+
--verbose \
72+
--tap=test.tap \
73+
--shard ${{ inputs.shard }}/${{ inputs.total_shards }}
74+
75+
# Save the absolute path of the test.tap file to the output
76+
echo "tap_file_path=$(realpath test.tap)" >> $GITHUB_OUTPUT
77+
78+
- name: Generate test summary
79+
if: always()
80+
shell: bash
81+
run: |
82+
# Add header if it doesn't exist (using a token file to track this)
83+
if [ ! -f ".summary_header" ]; then
84+
echo "# 🚀 nf-test results" >> $GITHUB_STEP_SUMMARY
85+
echo "" >> $GITHUB_STEP_SUMMARY
86+
echo "| Status | Test Name | Profile | Shard |" >> $GITHUB_STEP_SUMMARY
87+
echo "|:------:|-----------|---------|-------|" >> $GITHUB_STEP_SUMMARY
88+
touch .summary_header
89+
fi
90+
91+
if [ -f test.tap ]; then
92+
while IFS= read -r line; do
93+
if [[ $line =~ ^ok ]]; then
94+
test_name="${line#ok }"
95+
# Remove the test number from the beginning
96+
test_name="${test_name#* }"
97+
echo "| ✅ | ${test_name} | ${{ inputs.profile }} | ${{ inputs.shard }}/${{ inputs.total_shards }} |" >> $GITHUB_STEP_SUMMARY
98+
elif [[ $line =~ ^not\ ok ]]; then
99+
test_name="${line#not ok }"
100+
# Remove the test number from the beginning
101+
test_name="${test_name#* }"
102+
echo "| ❌ | ${test_name} | ${{ inputs.profile }} | ${{ inputs.shard }}/${{ inputs.total_shards }} |" >> $GITHUB_STEP_SUMMARY
103+
fi
104+
done < test.tap
105+
else
106+
echo "| ⚠️ | No test results found | ${{ inputs.profile }} | ${{ inputs.shard }}/${{ inputs.total_shards }} |" >> $GITHUB_STEP_SUMMARY
107+
fi
108+
109+
- name: Clean up
110+
if: always()
111+
shell: bash
112+
run: |
113+
sudo rm -rf /home/ubuntu/tests/

.github/workflows/awsfulltest.yml

Lines changed: 13 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -1,35 +1,27 @@
11
name: nf-core AWS full size tests
2-
# This workflow is triggered on PRs opened against the master branch.
2+
# This workflow is triggered on PRs opened against the main/master branch.
33
# It can be additionally triggered manually with GitHub actions workflow dispatch button.
44
# It runs the -profile 'test_full' on AWS batch
55

66
on:
7-
pull_request:
8-
branches:
9-
- master
107
workflow_dispatch:
118
pull_request_review:
129
types: [submitted]
10+
release:
11+
types: [published]
1312

1413
jobs:
1514
run-platform:
1615
name: Run AWS full tests
17-
# run only if the PR is approved by at least 2 reviewers and against the master branch or manually triggered
18-
if: github.repository == 'nf-core/oncoanalyser' && github.event.review.state == 'approved' && github.event.pull_request.base.ref == 'master' || github.event_name == 'workflow_dispatch'
16+
# run only if the PR is approved by at least 2 reviewers and against the master/main branch or manually triggered
17+
if: github.repository == 'nf-core/oncoanalyser' && github.event.review.state == 'approved' && (github.event.pull_request.base.ref == 'master' || github.event.pull_request.base.ref == 'main') || github.event_name == 'workflow_dispatch' || github.event_name == 'release'
1918
runs-on: ubuntu-latest
2019
steps:
21-
- uses: octokit/request-action@v2.x
22-
id: check_approvals
23-
with:
24-
route: GET /repos/${{ github.repository }}/pulls/${{ github.event.pull_request.number }}/reviews
25-
env:
26-
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
27-
- id: test_variables
28-
if: github.event_name != 'workflow_dispatch'
20+
- name: Set revision variable
21+
id: revision
2922
run: |
30-
JSON_RESPONSE='${{ steps.check_approvals.outputs.data }}'
31-
CURRENT_APPROVALS_COUNT=$(echo $JSON_RESPONSE | jq -c '[.[] | select(.state | contains("APPROVED")) ] | length')
32-
test $CURRENT_APPROVALS_COUNT -ge 2 || exit 1 # At least 2 approvals are required
23+
echo "revision=${{ (github.event_name == 'workflow_dispatch' || github.event_name == 'release') && github.sha || 'dev' }}" >> "$GITHUB_OUTPUT"
24+
3325
- name: Launch workflow via Seqera Platform
3426
uses: seqeralabs/action-tower-launch@v2
3527
# TODO nf-core: You can customise AWS full pipeline tests as required
@@ -39,16 +31,16 @@ jobs:
3931
workspace_id: ${{ secrets.TOWER_WORKSPACE_ID }}
4032
access_token: ${{ secrets.TOWER_ACCESS_TOKEN }}
4133
compute_env: ${{ secrets.TOWER_COMPUTE_ENV }}
42-
revision: ${{ github.sha }}
43-
workdir: s3://${{ secrets.AWS_S3_BUCKET }}/work/oncoanalyser/work-${{ github.sha }}
34+
revision: ${{ steps.revision.outputs.revision }}
35+
workdir: s3://${{ secrets.AWS_S3_BUCKET }}/work/oncoanalyser/work-${{ steps.revision.outputs.revision }}
4436
parameters: |
4537
{
4638
"hook_url": "${{ secrets.MEGATESTS_ALERTS_SLACK_HOOK_URL }}",
47-
"outdir": "s3://${{ secrets.AWS_S3_BUCKET }}/oncoanalyser/results-${{ github.sha }}"
39+
"outdir": "s3://${{ secrets.AWS_S3_BUCKET }}/oncoanalyser/results-${{ steps.revision.outputs.revision }}"
4840
}
4941
profiles: test_full
5042

51-
- uses: actions/upload-artifact@v4
43+
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
5244
with:
5345
name: Seqera Platform debug log file
5446
path: |

.github/workflows/awstest.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@ jobs:
2525
}
2626
profiles: test
2727

28-
- uses: actions/upload-artifact@v4
28+
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
2929
with:
3030
name: Seqera Platform debug log file
3131
path: |

.github/workflows/branch.yml

Lines changed: 10 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,17 @@
11
name: nf-core branch protection
2-
# This workflow is triggered on PRs to master branch on the repository
3-
# It fails when someone tries to make a PR against the nf-core `master` branch instead of `dev`
2+
# This workflow is triggered on PRs to `main`/`master` branch on the repository
3+
# It fails when someone tries to make a PR against the nf-core `main`/`master` branch instead of `dev`
44
on:
55
pull_request_target:
6-
branches: [master]
6+
branches:
7+
- main
8+
- master
79

810
jobs:
911
test:
1012
runs-on: ubuntu-latest
1113
steps:
12-
# PRs to the nf-core repo master branch are only ok if coming from the nf-core repo `dev` or any `patch` branches
14+
# PRs to the nf-core repo main/master branch are only ok if coming from the nf-core repo `dev` or any `patch` branches
1315
- name: Check PRs
1416
if: github.repository == 'nf-core/oncoanalyser'
1517
run: |
@@ -22,7 +24,7 @@ jobs:
2224
uses: mshick/add-pr-comment@b8f338c590a895d50bcbfa6c5859251edc8952fc # v2
2325
with:
2426
message: |
25-
## This PR is against the `master` branch :x:
27+
## This PR is against the `${{github.event.pull_request.base.ref}}` branch :x:
2628
2729
* Do not close this PR
2830
* Click _Edit_ and change the `base` to `dev`
@@ -32,9 +34,9 @@ jobs:
3234
3335
Hi @${{ github.event.pull_request.user.login }},
3436
35-
It looks like this pull-request is has been made against the [${{github.event.pull_request.head.repo.full_name }}](https://github.com/${{github.event.pull_request.head.repo.full_name }}) `master` branch.
36-
The `master` branch on nf-core repositories should always contain code from the latest release.
37-
Because of this, PRs to `master` are only allowed if they come from the [${{github.event.pull_request.head.repo.full_name }}](https://github.com/${{github.event.pull_request.head.repo.full_name }}) `dev` branch.
37+
It looks like this pull-request is has been made against the [${{github.event.pull_request.head.repo.full_name }}](https://github.com/${{github.event.pull_request.head.repo.full_name }}) ${{github.event.pull_request.base.ref}} branch.
38+
The ${{github.event.pull_request.base.ref}} branch on nf-core repositories should always contain code from the latest release.
39+
Because of this, PRs to ${{github.event.pull_request.base.ref}} are only allowed if they come from the [${{github.event.pull_request.head.repo.full_name }}](https://github.com/${{github.event.pull_request.head.repo.full_name }}) `dev` branch.
3840
3941
You do not need to close this PR, you can change the target branch to `dev` by clicking the _"Edit"_ button at the top of this page.
4042
Note that even after this, the test will continue to show as failing until you push a new commit.

0 commit comments

Comments
 (0)