Skip to content

Commit 2706782

Browse files
ejseqeraejseqera
authored
Merge pull request #159 from nf-core/dev
Dev -> Master for 1.10.0 release
2 parents 084e5ef + 1b9d187 commit 2706782

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

48 files changed

+588
-343
lines changed

‎.editorconfig

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ trim_trailing_whitespace = true
88
indent_size = 4
99
indent_style = space
1010

11-
[*.{md,yml,yaml,html,css,scss,js,cff}]
11+
[*.{md,yml,yaml,html,css,scss,js}]
1212
indent_size = 2
1313

1414
# These files are edited and tested upstream in nf-core/modules

‎.github/ISSUE_TEMPLATE/bug_report.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,6 @@ body:
4545
* Nextflow version _(eg. 22.10.1)_
4646
* Hardware _(eg. HPC, Desktop, Cloud)_
4747
* Executor _(eg. slurm, local, awsbatch)_
48-
* Container engine: _(e.g. Docker, Singularity, Conda, Podman, Shifter or Charliecloud)_
48+
* Container engine: _(e.g. Docker, Singularity, Conda, Podman, Shifter, Charliecloud, or Apptainer)_
4949
* OS _(eg. CentOS Linux, macOS, Linux Mint)_
5050
* Version of nf-core/fetchngs _(eg. 1.1, 1.5, 1.8.2)_

‎.github/PULL_REQUEST_TEMPLATE.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,8 @@ Learn more about contributing: [CONTRIBUTING.md](https://github.com/nf-core/fetc
1515

1616
- [ ] This comment contains a description of changes (with reason).
1717
- [ ] If you've fixed a bug or added code that should be tested, add tests!
18-
- [ ] If you've added a new tool - have you followed the pipeline conventions in the [contribution docs](https://github.com/nf-core/fetchngs/tree/master/.github/CONTRIBUTING.md)- [ ] If necessary, also make a PR on the nf-core/fetchngs _branch_ on the [nf-core/test-datasets](https://github.com/nf-core/test-datasets) repository.
18+
- [ ] If you've added a new tool - have you followed the pipeline conventions in the [contribution docs](https://github.com/nf-core/fetchngs/tree/master/.github/CONTRIBUTING.md)
19+
- [ ] If necessary, also make a PR on the nf-core/fetchngs _branch_ on the [nf-core/test-datasets](https://github.com/nf-core/test-datasets) repository.
1920
- [ ] Make sure your code lints (`nf-core lint`).
2021
- [ ] Ensure the test suite passes (`nextflow run . -profile test,docker --outdir <OUTDIR>`).
2122
- [ ] Usage Documentation in `docs/usage.md` is updated.

‎.github/workflows/awsfulltest.yml

Lines changed: 0 additions & 31 deletions
This file was deleted.

‎.github/workflows/awstest.yml

Lines changed: 0 additions & 29 deletions
This file was deleted.

‎.github/workflows/branch.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ jobs:
1313
- name: Check PRs
1414
if: github.repository == 'nf-core/fetchngs'
1515
run: |
16-
{ [[ ${{github.event.pull_request.head.repo.full_name }} == nf-core/fetchngs ]] && [[ $GITHUB_HEAD_REF = "dev" ]]; } || [[ $GITHUB_HEAD_REF == "patch" ]]
16+
{ [[ ${{github.event.pull_request.head.repo.full_name }} == nf-core/fetchngs ]] && [[ $GITHUB_HEAD_REF == "dev" ]]; } || [[ $GITHUB_HEAD_REF == "patch" ]]
1717
1818
# If the above check failed, post a comment on the PR explaining the failure
1919
# NOTE - this doesn't currently work if the PR is coming from a fork, due to limitations in GitHub actions secrets

‎.github/workflows/clean-up.yml

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
name: "Close user-tagged issues and PRs"
2+
on:
3+
schedule:
4+
- cron: "0 0 * * 0" # Once a week
5+
6+
jobs:
7+
clean-up:
8+
runs-on: ubuntu-latest
9+
permissions:
10+
issues: write
11+
pull-requests: write
12+
steps:
13+
- uses: actions/stale@v7
14+
with:
15+
stale-issue-message: "This issue has been tagged as awaiting-changes or awaiting-feedback by an nf-core contributor. Remove stale label or add a comment otherwise this issue will be closed in 20 days."
16+
stale-pr-message: "This PR has been tagged as awaiting-changes or awaiting-feedback by an nf-core contributor. Remove stale label or add a comment if it is still useful."
17+
close-issue-message: "This issue was closed because it has been tagged as awaiting-changes or awaiting-feedback by an nf-core contributor and then staled for 20 days with no activity."
18+
days-before-stale: 30
19+
days-before-close: 20
20+
days-before-pr-close: -1
21+
any-of-labels: "awaiting-changes,awaiting-feedback"
22+
exempt-issue-labels: "WIP"
23+
exempt-pr-labels: "WIP"
24+
repo-token: "${{ secrets.GITHUB_TOKEN }}"

‎.github/workflows/cloud_tests_full.yml

Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
name: full-sized tests on cloud providers
2+
run-name: Submitting workflow to all cloud providers using full sized data
3+
on:
4+
release:
5+
types: [published]
6+
workflow_dispatch:
7+
inputs:
8+
platform:
9+
description: "Platform to run test"
10+
required: true
11+
default: "all"
12+
type: choice
13+
options:
14+
- all
15+
- aws
16+
- azure
17+
- gcp
18+
jobs:
19+
run-full-tests-on-aws:
20+
if: ${{ github.event.inputs.platform == 'all' || github.event.inputs.platform == 'aws' || !github.event.inputs }}
21+
runs-on: ubuntu-latest
22+
steps:
23+
- uses: seqeralabs/action-tower-launch@v1
24+
with:
25+
workspace_id: ${{ secrets.TOWER_WORKSPACE_ID }}
26+
access_token: ${{ secrets.TOWER_ACCESS_TOKEN }}
27+
compute_env: ${{ secrets.TOWER_CE_AWS_CPU }}
28+
workdir: "${{ secrets.TOWER_BUCKET_AWS }}/work/fetchngs/work-${{ github.sha }}"
29+
run_name: "aws_fetchngs_full"
30+
profiles: test_full,public_aws_ecr
31+
parameters: |
32+
{
33+
"hook_url": "${{ secrets.MEGATESTS_ALERTS_SLACK_HOOK_URL }}",
34+
"outdir": "${{ secrets.TOWER_BUCKET_AWS }}/fetchngs/results-${{ github.sha }}"
35+
}
36+
- uses: actions/upload-artifact@v3
37+
with:
38+
name: Tower debug log file
39+
path: tower_action_*.log
40+
run-full-tests-on-gcp:
41+
if: ${{ github.event.inputs.platform == 'all' || github.event.inputs.platform == 'gcp' || !github.event.inputs }}
42+
runs-on: ubuntu-latest
43+
steps:
44+
- uses: seqeralabs/action-tower-launch@v1
45+
with:
46+
workspace_id: ${{ secrets.TOWER_WORKSPACE_ID }}
47+
access_token: ${{ secrets.TOWER_ACCESS_TOKEN }}
48+
compute_env: ${{ secrets.TOWER_CE_GCP_CPU }}
49+
workdir: "${{ secrets.TOWER_BUCKET_GCP }}/work/fetchngs/work-${{ github.sha }}"
50+
run_name: "gcp_fetchngs_full"
51+
profiles: test_full
52+
parameters: |
53+
{
54+
"hook_url": "${{ secrets.MEGATESTS_ALERTS_SLACK_HOOK_URL }}",
55+
"outdir": "${{ secrets.TOWER_BUCKET_GCP }}/fetchngs/results-${{ github.sha }}"
56+
}
57+
- uses: actions/upload-artifact@v3
58+
with:
59+
name: Tower debug log file
60+
path: tower_action_*.log
61+
run-full-tests-on-azure:
62+
if: ${{ github.event.inputs.platform == 'all' || github.event.inputs.platform == 'azure' || !github.event.inputs }}
63+
runs-on: ubuntu-latest
64+
steps:
65+
- uses: seqeralabs/action-tower-launch@v1
66+
with:
67+
workspace_id: ${{ secrets.TOWER_WORKSPACE_ID }}
68+
access_token: ${{ secrets.TOWER_ACCESS_TOKEN }}
69+
compute_env: ${{ secrets.TOWER_CE_AZURE_CPU }}
70+
workdir: "${{ secrets.TOWER_BUCKET_AZURE }}/work/fetchngs/work-${{ github.sha }}"
71+
run_name: "azure_fetchngs_full"
72+
profiles: test_full
73+
parameters: |
74+
{
75+
"hook_url": "${{ secrets.MEGATESTS_ALERTS_SLACK_HOOK_URL }}",
76+
"outdir": "${{ secrets.TOWER_BUCKET_AZURE }}/fetchngs/results-${{ github.sha }}"
77+
}
78+
- uses: actions/upload-artifact@v3
79+
with:
80+
name: Tower debug log file
81+
path: tower_action_*.log

‎.github/workflows/cloud_tests_small.yml

Lines changed: 76 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,76 @@
1+
name: small-sized tests on cloud providers
2+
run-name: Submitting workflow to all cloud providers using small sized data
3+
on:
4+
workflow_dispatch:
5+
inputs:
6+
platform:
7+
description: "Platform to run test"
8+
required: true
9+
default: "all"
10+
type: choice
11+
options:
12+
- all
13+
- aws
14+
- azure
15+
- gcp
16+
jobs:
17+
run-small-tests-on-aws:
18+
if: ${{ github.event.inputs.platform == 'all' || github.event.inputs.platform == 'aws' }}
19+
runs-on: ubuntu-latest
20+
steps:
21+
- uses: seqeralabs/action-tower-launch@v1
22+
with:
23+
workspace_id: ${{ secrets.TOWER_WORKSPACE_ID }}
24+
access_token: ${{ secrets.TOWER_ACCESS_TOKEN }}
25+
compute_env: ${{ secrets.TOWER_CE_AWS_CPU }}
26+
workdir: "${{ secrets.TOWER_BUCKET_AWS }}/work/fetchngs/work-${{ github.sha }}"
27+
run_name: "aws_fetchngs_small"
28+
profiles: test,public_aws_ecr
29+
parameters: |
30+
{
31+
"outdir": "${{ secrets.TOWER_BUCKET_AWS }}/fetchngs/results-test-${{ github.sha }}"
32+
}
33+
- uses: actions/upload-artifact@v3
34+
with:
35+
name: Tower debug log file
36+
path: tower_action_*.log
37+
run-small-tests-on-gcp:
38+
if: ${{ github.event.inputs.platform == 'all' || github.event.inputs.platform == 'gcp' }}
39+
runs-on: ubuntu-latest
40+
steps:
41+
- uses: seqeralabs/action-tower-launch@v1
42+
with:
43+
workspace_id: ${{ secrets.TOWER_WORKSPACE_ID }}
44+
access_token: ${{ secrets.TOWER_ACCESS_TOKEN }}
45+
compute_env: ${{ secrets.TOWER_CE_GCP_CPU }}
46+
workdir: "${{ secrets.TOWER_BUCKET_GCP }}/work/fetchngs/work-${{ github.sha }}"
47+
run_name: "gcp_fetchngs_small"
48+
profiles: test
49+
parameters: |
50+
{
51+
"outdir": "${{ secrets.TOWER_BUCKET_GCP }}/fetchngs/results-test-${{ github.sha }}"
52+
}
53+
- uses: actions/upload-artifact@v3
54+
with:
55+
name: Tower debug log file
56+
path: tower_action_*.log
57+
run-small-tests-on-azure:
58+
if: ${{ github.event.inputs.platform == 'all' || github.event.inputs.platform == 'azure' }}
59+
runs-on: ubuntu-latest
60+
steps:
61+
- uses: seqeralabs/action-tower-launch@v1
62+
with:
63+
workspace_id: ${{ secrets.TOWER_WORKSPACE_ID }}
64+
access_token: ${{ secrets.TOWER_ACCESS_TOKEN }}
65+
compute_env: ${{ secrets.TOWER_CE_AZURE_CPU }}
66+
workdir: "${{ secrets.TOWER_BUCKET_AZURE }}/work/fetchngs/work-${{ github.sha }}"
67+
run_name: "azure_fetchngs_small"
68+
profiles: test
69+
parameters: |
70+
{
71+
"outdir": "${{ secrets.TOWER_BUCKET_AZURE }}/fetchngs/results-test-${{ github.sha }}"
72+
}
73+
- uses: actions/upload-artifact@v3
74+
with:
75+
name: Tower debug log file
76+
path: tower_action_*.log

‎.github/workflows/linting.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -78,7 +78,7 @@ jobs:
7878

7979
- uses: actions/setup-python@v4
8080
with:
81-
python-version: "3.7"
81+
python-version: "3.8"
8282
architecture: "x64"
8383

8484
- name: Install dependencies

‎.pre-commit-config.yaml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
repos:
2+
- repo: https://github.com/pre-commit/mirrors-prettier
3+
rev: "v2.7.1"
4+
hooks:
5+
- id: prettier

‎CHANGELOG.md

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,46 @@
33
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
44
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
55

6+
## [[1.10.0](https://github.com/nf-core/fetchngs/releases/tag/1.10.0)] - 2023-05-16
7+
8+
### Credits
9+
10+
Special thanks to the following for their contributions to the release:
11+
12+
- [Adam Talbot](https://github.com/adamrtalbot)
13+
- [Esha Joshi](https://github.com/ejseqera)
14+
- [Maxime Garcia](https://github.com/maxulysse)
15+
- [Moritz E. Beber](https://github.com/Midnighter)
16+
- [Rob Syme](https://github.com/robsyme)
17+
- [sirclockalot](https://github.com/sirclockalot)
18+
19+
Thank you to everyone else that has contributed by reporting bugs, enhancements or in any other way, shape or form.
20+
21+
### Enhancements & fixes
22+
23+
- [#85](https://github.com/nf-core/fetchngs/issues/85) - Not able to fetch metadata for ERR ids associated with ArrayExpress
24+
- [#104](https://github.com/nf-core/fetchngs/issues/104) - Add support back in for [GEO IDs](https://www.ncbi.nlm.nih.gov/geo) (removed in v1.7)
25+
- [#129](https://github.com/nf-core/fetchngs/issues/129) - Pipeline is working with SRA run ids but failing with corresponding Biosample ids
26+
- [#138](https://github.com/nf-core/fetchngs/issues/138) - Add support for downloading protected dbGAP data using a JWT file
27+
- [#144](https://github.com/nf-core/fetchngs/issues/144) - Add support to download 10X Genomics data
28+
- [PR #140](https://github.com/nf-core/fetchngs/pull/140) - Bumped modules version to allow for sratools download of sralite format files
29+
- [PR #147](https://github.com/nf-core/fetchngs/pull/147) - Updated pipeline template to [nf-core/tools 2.8](https://github.com/nf-core/tools/releases/tag/2.8)
30+
- [PR #148](https://github.com/nf-core/fetchngs/pull/148) - Fix default metadata fields for ENA API v2.0
31+
- [PR #150](https://github.com/nf-core/fetchngs/pull/150) - Add infrastructure and CI for multi-cloud full-sized tests run via Nextflow Tower
32+
- [PR #157](https://github.com/nf-core/fetchngs/pull/157) - Add `public_aws_ecr.config` to source mulled containers when using `public.ecr.aws` Docker Biocontainer registry
33+
34+
### Software dependencies
35+
36+
| Dependency | Old version | New version |
37+
| --------------- | ----------- | ----------- |
38+
| `synapseclient` | 2.6.0 | 2.7.1 |
39+
40+
> **NB:** Dependency has been **updated** if both old and new version information is present.
41+
>
42+
> **NB:** Dependency has been **added** if just the new version information is present.
43+
>
44+
> **NB:** Dependency has been **removed** if new version information isn't present.
45+
646
## [[1.9](https://github.com/nf-core/fetchngs/releases/tag/1.9)] - 2022-12-21
747

848
### Enhancements & fixes

0 commit comments

Comments
 (0)