Skip to content

Commit 6e191eb

Browse files
amrit110amrit110
committed
Initial commit
0 parents  commit 6e191eb

File tree

302 files changed

+52472
-0
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

302 files changed

+52472
-0
lines changed

‎.gitattributes

+2
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
# Exclude jupyter notebooks for language stats.
2+
*.ipynb linguist-vendored

‎.github/pull_request_template.md

+7
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
# PR Type ([Feature | Fix | Documentation])
2+
3+
## Short Description
4+
...
5+
6+
## Tests Added
7+
...

‎.github/workflows/code_checks.yml

+47
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
name: Code checks
2+
3+
on:
4+
push:
5+
branches:
6+
- master
7+
paths:
8+
- .pre-commit-config.yaml
9+
- .github/workflows/code_checks.yml
10+
- '**.py'
11+
- poetry.lock
12+
- pyproject.toml
13+
- requirements.txt
14+
- '**.ipynb'
15+
pull_request:
16+
branches:
17+
- master
18+
paths:
19+
- .pre-commit-config.yaml
20+
- .github/workflows/code_checks.yml
21+
- '**.py'
22+
- poetry.lock
23+
- pyproject.toml
24+
- requirements.txt
25+
- '**.ipynb'
26+
27+
jobs:
28+
run-code-check:
29+
runs-on: ubuntu-latest
30+
steps:
31+
- name: Install apt dependencies
32+
run: |
33+
sudo apt-get update
34+
sudo apt-get install libcurl4-openssl-dev libssl-dev
35+
- uses: actions/checkout@v3
36+
- name: Install poetry
37+
run: pipx install poetry
38+
- uses: actions/[email protected]
39+
with:
40+
python-version: 3.9.7
41+
cache: 'poetry'
42+
- name: Install dependencies and check code
43+
run: |
44+
poetry env use 3.9.7
45+
source $(poetry env info --path)/bin/activate
46+
poetry install
47+
pre-commit run --all-files

‎.github/workflows/docs_build.yml

+39
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
name: Build documentation and coverage report
2+
3+
on:
4+
pull_request:
5+
branches:
6+
- master
7+
paths:
8+
- .pre-commit-config.yaml
9+
- .github/workflows/docs_build.yml
10+
- '**.py'
11+
- poetry.lock
12+
- pyproject.toml
13+
- requirements.txt
14+
- '**.rst'
15+
- '**.md'
16+
17+
jobs:
18+
build:
19+
runs-on: ubuntu-latest
20+
steps:
21+
- name: Install apt dependencies
22+
run: |
23+
sudo apt-get update
24+
sudo apt-get install libcurl4-openssl-dev libssl-dev pandoc
25+
- uses: actions/checkout@v3
26+
- name: Install dependencies, build docs and coverage report
27+
run: pipx install poetry
28+
- uses: actions/[email protected]
29+
with:
30+
python-version: 3.9.7
31+
cache: 'poetry'
32+
- run: |
33+
poetry env use 3.9.7
34+
source $(poetry env info --path)/bin/activate
35+
poetry install
36+
pandoc README.md -f markdown -t rst -s -o docs/source/intro.rst
37+
cd docs && make html
38+
cd .. && coverage run -m pytest && coverage html && cp htmlcov/* docs/build/html/_static/
39+
mv docs/build/html/_static/index.html docs/build/html/_static/coverage_report.html

‎.github/workflows/docs_deploy.yml

+45
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
name: Deploy documentation
2+
3+
on:
4+
push:
5+
branches:
6+
- master
7+
paths:
8+
- .pre-commit-config.yaml
9+
- .github/workflows/docs_build.yml
10+
- .github/workflows/docs_deploy.yml
11+
- '**.py'
12+
- poetry.lock
13+
- pyproject.toml
14+
- requirements.txt
15+
- '**.rst'
16+
- '**.md'
17+
18+
jobs:
19+
deploy:
20+
runs-on: ubuntu-latest
21+
steps:
22+
- name: Install apt dependencies
23+
run: |
24+
sudo apt-get update
25+
sudo apt-get install libcurl4-openssl-dev libssl-dev pandoc
26+
- uses: actions/checkout@v3
27+
- name: Install dependencies, build docs and coverage report
28+
run: pipx install poetry
29+
- uses: actions/[email protected]
30+
with:
31+
python-version: 3.9.7
32+
cache: 'poetry'
33+
- run: |
34+
poetry env use 3.9.7
35+
source $(poetry env info --path)/bin/activate
36+
poetry install
37+
pandoc README.md -f markdown -t rst -s -o docs/source/intro.rst
38+
cd docs && make html
39+
cd .. && coverage run -m pytest && coverage html && cp htmlcov/* docs/build/html/_static/
40+
mv docs/build/html/_static/index.html docs/build/html/_static/coverage_report.html
41+
- name: Deploy to Github pages
42+
uses: JamesIves/[email protected]
43+
with:
44+
branch: github_pages
45+
folder: docs/build/html

‎.gitignore

+24
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
*__pycache__*
2+
*.log
3+
docs/build
4+
.python-version
5+
*.DS_Store
6+
htmlcov
7+
.coverage
8+
venv
9+
.ipynb_checkpoints
10+
*.pt
11+
*.csv
12+
mlruns
13+
.profile
14+
.env
15+
*.html
16+
_extract
17+
*.gzip
18+
*checkpoint*
19+
*.parquet
20+
*.npy
21+
*.pkl
22+
*.npz
23+
*.csv
24+
*.pickle

‎.pre-commit-config.yaml

+91
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,91 @@
1+
exclude: |
2+
(?x)(
3+
^drift_detection/|
4+
^notebooks/misc/batching_explore.ipynb
5+
)
6+
repos:
7+
- repo: https://github.com/pre-commit/pre-commit-hooks
8+
rev: v4.1.0 # Use the ref you want to point at
9+
hooks:
10+
- id: trailing-whitespace
11+
- id: check-ast
12+
- id: check-builtin-literals
13+
- id: check-docstring-first
14+
- id: check-executables-have-shebangs
15+
- id: debug-statements
16+
- id: end-of-file-fixer
17+
- id: mixed-line-ending
18+
args: [--fix=lf]
19+
- id: requirements-txt-fixer
20+
- id: trailing-whitespace
21+
- id: check-yaml
22+
23+
- repo: https://github.com/ambv/black
24+
rev: 22.3.0
25+
hooks:
26+
- id: black
27+
28+
- repo: https://github.com/PyCQA/isort
29+
rev: 5.5.2
30+
hooks:
31+
- id: isort
32+
33+
- repo: https://github.com/myint/docformatter
34+
rev: v1.3.1
35+
hooks:
36+
- id: docformatter
37+
args: [--in-place, --wrap-summaries=88, --wrap-descriptions=88, --blank]
38+
files: ".*.py$"
39+
40+
- repo: local
41+
hooks:
42+
- id: flake8
43+
name: flake8
44+
language: python
45+
entry: pflake8
46+
files: ".*.py$"
47+
48+
- repo: https://github.com/PyCQA/pydocstyle
49+
rev: 6.1.1
50+
hooks:
51+
- id: pydocstyle
52+
args: [--convention=numpy]
53+
additional_dependencies: [toml]
54+
55+
- repo: https://github.com/pre-commit/mirrors-mypy
56+
rev: v0.942
57+
hooks:
58+
- id: mypy
59+
args: [--namespace-packages, --explicit-package-bases]
60+
additional_dependencies: [types-python-dateutil==2.8.8, types-PyYAML==6.0.5]
61+
62+
- repo: local
63+
hooks:
64+
- id: pylint
65+
name: pylint
66+
language: python
67+
entry: pylint
68+
files: ".*.py$"
69+
70+
- repo: local
71+
hooks:
72+
- id: pytest
73+
name: pytest
74+
entry: pytest
75+
language: system
76+
pass_filenames: false
77+
always_run: true
78+
79+
- repo: https://github.com/nbQA-dev/nbQA
80+
rev: 1.3.1
81+
hooks:
82+
- id: nbqa-black
83+
- id: nbqa-isort
84+
- id: nbqa-check-ast
85+
- id: nbqa-flake8
86+
# - id: nbqa-mypy
87+
88+
- repo: https://github.com/kynan/nbstripout
89+
rev: 0.6.0
90+
hooks:
91+
- id: nbstripout

‎CHANGELOG.md

+34
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
# Changelog
2+
3+
All notable changes to this project will be documented in this file.
4+
5+
Currently this project is early in development, so the semantic versioning is more to just keep track of development changes, not releases.
6+
7+
## [0.0.1] - 2022-01-13 (6e5b1a00cddfe4cdaf58b41d82ad948a204dbc1d)
8+
This is first development version after Maria's handover of codebase.
9+
10+
### Added
11+
- Code formatting using black
12+
- Cleanup README
13+
- Installation of dependencies using both pip and Conda
14+
15+
### Fixed
16+
- Jupyter notebooks sourcing environment variables using python-dotenv package
17+
- postgres username automatically sourced using $USER env variable
18+
19+
20+
## [0.0.2] - 2022-02-01 (d397ae701a2b67ad023f0d4fd4c9d6d015e55d29)
21+
22+
### Added
23+
- Add pre-commit hooks for static code analysis, unit tests
24+
- Add docs generation using Sphinx, just uses README now, API docs yet to add
25+
- Move luigi pipeline stuff to cyclops.workflow (yet to be integrated, so
26+
this feature is broken ATM.
27+
- Add basic SQL querier, works for admin+diagnosis data
28+
- Move data-pipeline utils functions to cyclops.processors, clean up some
29+
functions, and add tests for them.
30+
- Add logging, wherever it makes sense. Freeze requirements.
31+
32+
### Changed
33+
- Split config files, so it becomes easier to manage.
34+
- Move model implementation, and training/validation specific scripts to models.

‎CONTRIBUTING.md

+31
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
# Contributing to cyclops
2+
3+
Thanks for your interest in contributing to cyclops!
4+
5+
To submit PRs, please fill out the PR template along with the PR. If the PR
6+
fixes an issue, don't forget to link the PR to the issue!
7+
8+
## Pre-commit hooks
9+
10+
Once the python virtual environment is setup, you can run pre-commit hooks using:
11+
12+
```bash
13+
pre-commit run --all-files
14+
```
15+
16+
## Coding guidelines
17+
18+
For code style, we recommend the [google style guide](https://google.github.io/styleguide/pyguide.html).
19+
20+
Pre-commit hooks apply the [black](https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html)
21+
code formatting.
22+
23+
For docstrings we use [numpy format](https://numpydoc.readthedocs.io/en/latest/format.html).
24+
25+
We also use [flake8](https://flake8.pycqa.org/en/latest/) and [pylint](https://pylint.pycqa.org/en/stable/)
26+
for further static code analysis. The pre-commit hooks show errors which you need
27+
to fix before submitting a PR.
28+
29+
Last but not the least, we use type hints in our code which is then checked using
30+
[mypy](https://mypy.readthedocs.io/en/stable/). Currently, mypy checks are not
31+
strict, but will be enforced more as the API code becomes more stable.

‎Dockerfile

+20
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
FROM python:3.9.7
2+
3+
4+
WORKDIR /app/cyclops
5+
ARG DEBIAN_FRONTEND=noninteractive
6+
ENV LANG C.UTF-8
7+
8+
9+
RUN apt-get update \
10+
&& apt-get install -y git software-properties-common \
11+
&& rm -rf /var/lib/apt/lists/*
12+
13+
14+
RUN python3 -m pip install --upgrade pip
15+
RUN pip install poetry
16+
17+
18+
COPY * /app/cyclops/
19+
RUN poetry config virtualenvs.create false \
20+
&& poetry install --no-interaction --no-ansi

0 commit comments

Comments
 (0)