CCL development proceeds via pull requests on GitHub. In short, the code base is forked on GitHub, a new change is pushed to a new branch, and then a pull request is issued by the developer. Once a pull request is issued, CCL uses Travis-CI to automatically run the benchmark/unit tests and the linter. Please read the sections below for guidance on how to modify the source code, test, and then document your modifications.
Working on the Python library¶
After completing the Developer Installation, you can change the
in your local copy of CCL. Any changes you make will be visible in
after you restart your interpreter. A typical workflow is to make some change,
rerun a failing benchmark or unit test, etc. until you’ve completed your new
feature and/or fixed a bug. See Writing and Running Benchmarks and Writing and Running Unit Tests for instructions
on how to run these test suites. Please add unit tests and benchmarks as appropriate for
any new functionality. You should also make sure to run the linter by following
the instructions in Linting with flake8. Finally, after you are satisfied with your new
code, make sure to follow Updating the Changelog and Documentation in order to the properly document your
Adding New C-level Features to CCL¶
Before adding new features to the CCL
C code, please read Understanding the Python-C Interface in
order to understand how the
C codes interact.
The following steps are needed to add a new source file
ccl_newfile.h to the library.
- Make sure to add a C-level include of your new header to
- Add the new source file
- Write a
SWIGinterface file for the new functions. This file gets placed in the
pyccldirectory. You may need to consult another CCL developer for help with the
- Add your new
ccl_newfile.ito the list of includes in the
- Write the public
PythonAPIs for your new function in a file called
Pythonpackage should access the
Ccode you wrote from the new functions in
- Import your new functions into the
pyccl/__init__.pyfile as needed.
- Add unit tests and benchmarks as appropriate for the new code. See Writing and Running Benchmarks and Writing and Running Unit Tests for details.
- Make sure to run the linter by following the instructions in Linting with flake8.
- Follow Updating the Changelog and Documentation in order to the properly document your new additions.
Occasionally, modifications made correctly as described above will still not
function properly. This might be due to multiple
pyccl installation files
not being properly overwritten, and the
Python interpreter getting confused.
At this point it is often best to resort to the “nuclear option” of
Python files related to
pyccl and starting from scratch. The
procedure is as follows:
pip uninstall pyccl(possibly multiple times) to remove all installed versions of
- Delete the compiled
rm -r build/to delete the build directory.
rm pyccl/ccl_wrap.c pyccl/ccllib.pyto delete the autogenerated
- Check various locations for the
site-packagesdirectory for any
pycclfiles and delete them.
- Reinstall CCL according to the Developer Installation instructions.
Updating the Changelog and Documentation¶
Please follow the following guidelines for documenting changes to CCL:
- New features and bug fixes should be documented in the
CHANGELOG.mdwith a brief description and the GitHub pull request number.
- Any new derivations/math essential to understanding a new CCL feature should be
documented in the CCL note. See :ref:
cclnotefor instructions on how to modify and compile it.
- All additions to the
Pythonpublic API should have
Pythondocstrings. These are written in
Sphinxcompatible format so that they can be incorporated into the CCL
Read the Docspages. See the current
Pythondocstrings in the
Pythonsource for examples.
- Additions to the
Ccode should be documented/commented so that other CCL developers can understand the code.
flake8 to ensure that the
Python code has a consistent style.
flake8 is available via
[pip|conda] install flake8.
You can run this tool locally by executing
$ flake8 pyccl
Any problems will be printed to
STDOUT. No output indicates that
Debug mode in Python¶
Because of the way the
Python wrapper handles exceptions that occur inside
C code, by default users will only see error messages for the most recent
error that occurs. If multiple errors occurred during a CCL function call, all but
the most recent error message will be overwritten. This convention can make it
difficult to debug the root cause of a problem.
To help with debugging this kind of issue, you can enable debug mode in the
Python wrapper. To do
so, simply call
pyccl.debug_mode(True). This will cause the
Python wrapper to print all
error messages to
STDERR whenever they occur.
Python exceptions will only be raised for the most
recent error, as before. (Note that Jupyter notebooks do not print
STDERR messages by default.)
Continuous Integration with Travis-CI¶
Travis-CI is a continuous integration service that reads the file
file in the repository and then runs the benchmarks/unit tests. More details on
Travis-CI can be found here: https://docs.travis-ci.com/user/getting-started/.
Every time you push a commit, Travis-CI will automatically try to build the
libraries with your new changes and run the benchmark/unit tests. You can check the
status of your builds by following the links from the pull request page. If your
build errors or fails, you can scroll through the log to find out what went wrong.
flake8 will result in the tests not passing. If your additions
require new dependencies, make sure that you include them in the
Deploying a New Release¶
When cutting a new release, the procedure is as follows:
Make sure any API changes are documented in
Commit to master
Create a new release from the GitHub interface here: https://github.com/LSSTDESC/CCL/releases/new
Manually create a source distribution from the root CCL folder:
$ python setup.py sdist
This command will create a
.tar.gzfile in the
distfolder. CAREFUL!!! Only build and upload the source distribution, not a binary wheel!
Upload source distribution to PyPi using
twine(can be installed using
$ twine upload dist/pyccl-x.x.x.tar.gz
Make sure your
setuptoolspackages are up to date, otherwise the markdown formatting of the
README.mdwill not be correctly processed on the CCL PyPi page.
conda-forgeautomated release bots will detect the new PyPi release and automatically make a pull request on the CCL feedstock. Once this pull request is merged, the new CCL release will be available on
condaafter a few hours.
Rebuild and redeploy the
Read the Docspages per the instructions in Building the Read the Docs Documentation. Note that you may need to adjust the major version number in
readthedocs/conf.pyif the new version has a major version number bump.
A CCL administrators will need to complete the steps above.
Read the Docs Documentation¶
To build the
Read the Docs documentation, follow the following steps:
$ cd readthedocs $ make clean $ make html
You can then inspect the outputs in
readthedocs/_build/index.html to make
sure the formatting is correct. Finally, contact the CCL
administrators to redeploy
the live documentation.
Building the CCL Note¶
The CCL note is a latex’ed documented located in
doc/0000-ccl_note. It is
used to document the scientific content of the CCL library. Note that documentation
of the actual APIs and code should reside in the
Python doc strings and
other code comments.
To compile the CCL note, type
make in the
If you need to modify the note, the files to modify are:
authors.csv: To document your contribution.
main.tex: To detail the changes to the library.
main.bib: To add new references.