GitHub notes for pyGIMLi developers

Contributing to the code

Note

These steps are only relevant if you have write privileges to the main GitHub repository. The general contribution guidelines can be found in _sec:contributing:.

1. Clone the GIMLi repository:

   git clone https://github.com/gimli-org/gimli gimli-src && cd gimli-src
   

You should be on the master branch by default. Check by calling git status. GIMLi encourages developers to keep the master branch clean. So please use the default development branch called dev. Save this script in .git/hooks for automatic warnings in case you are about to push to the master branch.

2. Checkout the dev branch:

   git checkout dev
   

3. Make changes in your favorite editor.

4. Add new files to the staging area:

   git add new_file1 new_file2
   

5. Make a commit with a meaningful message:

   git commit -a -m "ADD: important new feature."
   

   Make use of the prefixes in the table below to categorize commits.

6. Pull the latest developments from GitHub using automatic rebase:

For more info see: http://kernowsoul.com/blog/2012/06/20/4-ways-to-avoid-merge-commits-in-git/

git pull --rebase

You can set the rebase behavior on default with:

git config --global branch.autosetuprebase always

7. Push to the origin development branch:

   git push origin dev
   

Note that if you are on the dev branch, a git push should suffice, but the command above is more explicit about the branch which should be pushed.

8. Show available tags and branches:

   git tag
   git branch
   

9. If you plan bigger changes or developments create an own branch for it:

   git branch my_new_test_branch
   git checkout my_new_test_branch
   

Work on your code in branch my_new_test_branch and test it. If you are happy with your results merge them back into the dev branch and delete the branch.

git checkout dev
git merge my_new_test_branch
git branch -d my_new_test_branch

Branches

Branch name	Description	Purpose	CI (Jenkins) Current behavior	CI (Jenkins) Wishlist	Rules
dev	Main development branch	This is where new versions are developed and pull requests are merged to.	Build and test, on success generate http://dev.pygimli.org and merge to master	Test pull requests, build nightly conda packages for test channel	Run pg.test() and make doc before pushing. Tag related issues in commit message.
release	Latest release	This is where releases are “staged”. This usually means creating a git tag and manually merging dev into release. Hot fixes and website typos can be directly committed here.	Start in empty workspace, build, test and create documentation after each push. On success, merge into master and update http://www.pygimli.org.	Test “backward compatibility” (e.g., run example scripts from last release with this version). Test on Mac and Windows, too.	Make sure the tag is annotated and the version string is following the principles described below.
master	Latest tested dev	Make sure that if people checkout the repository, they always have a working version.	(represented by http://dev.pygimli.org)	Build pgcore (if necessary) and pygimli conda packages for release.	Never push anything to master!
	Feature branches	Larger endeavors and major restructuring should happen in dedicated feature branches (or forks), which are eventually merged to dev. This can also be useful if you want to give write access to others to jointly work on a feature.	Automatic testing can be requested (florian@pygimli.org).		Start feature branch from dev. Inform other developers about your develpment (to avoid conflicts and redundant work).

Commit messages

Please prepend your commit messages with one of the following prefixes depending on the nature of the commit:

Prefix	Meaning
ADD:	Addition of new functionality
API:	API change (argument orders or renames)
BLD:	Changes to pyGIMLi’s build pipeline
CI:	Continuous integration (i.e., Jenkins related)
CLN:	Clean up, refactoring or typo correction
DEF:	Change of default behavior
DEP:	Deprecation
DOC:	Improve documentation (i.e., docstrings or website)
ENH:	Enhancement, e.g. due to more flexibility by new (keyword) arguments
FIX:	Fixing a bug
REL:	Release (setting tags, updating version strings, etc.)
STY:	Coding style improvement (PEP8, PEP257)
TST:	New or improved test

Note

The tags API, ENH, ADD, FIX are relevant for creating the changelog later.

Version numbering

Following the PEP conventions, pygimli holds a __version__ string.

import pygimli as pg
print(pg.__version__)
v1.4.1-0-g76a4848-modified

This string consists of:

<Last git tag>-<Number of commits since last tag>-<last commit hash>

The string has a -modified suffix if pygimli has uncommitted changes.

To produce a new version, type:

git tag -a -m "First official release" "v1.0.x" # tags last commit as v1.0.x
git push --tags # pushes tags to GitHub

To see the commits since the last tag/release, you can use:

git log v1.3.0...HEAD --oneline

Or to see commits between specific versions:

git log v1.2.5...v1.2.6 --oneline

Alternatively, this information can also be obtained via GitHub.

Testing

Run specific API examples from shell:

python -c "import pygimli as pg; pg.test(pg.meshtools.createCircle, show=True)"

Run a specific test from shell.

python -c "import pygimli; from pygimli.physics.petro.resistivity import *; test_Archie()"

Run all tests

python -c "import pygimli; pygimli.test(show=True)"

Run pylint from shell to check code:

pylint --rcfile $GIMLIROOT/.pylintrc file.py

Run prospector to check code like landscape.io do:

prospector --profile=$GIMLIROOT/.prospector.yml file.py

Read API documentation from shell:

python -c "import pygimli as pg; help(pg.test)"

More information on pyGIMLi’s native testing function:

pygimli.test(target=None, show=False, onlydoctests=False, coverage=False, htmlreport=False, abort=False, verbose=True, devTests=False)

Run docstring examples and additional tests.

Examples

>>> import pygimli as pg
>>> # Run the whole test suite.
>>> pg.test() 
>>> # Test a single function by a string.
>>> pg.test("utils.boxprint", verbose=False) 
>>> # The target argument can also be the function directly
>>> from pygimli.utils import boxprint
>>> pg.test(boxprint, verbose=False) 
>>> # Use some logical expressions
>>> pg.test("draw and not drawMesh") 

Parameters:

• target (function or string or pattern (-k flag in pytest), optional) – Function or method to test. By default everything is tested.

• show (boolean, optional) – Show viewer windows during test run. They will be closed automatically.

• onlydoctests (boolean, optional) – Run test files in testing as well.

• coverage (boolean, optional) – Create a coverage report. Requires the pytest-cov plugin.

• htmlreport (str, optional) – Filename for HTML report such as www.pygimli.org/build_tests.html. Requires pytest-html plugin.

• abort (boolean, optional) – Return correct exit code, e.g. abort documentation build when a test fails.

• devTests (boolean[False]) – Don’t skip special tests marked for development, only with the @pg.skipOnDefaultTest decorator. Can be overwritten by env DEVTESTS.

Adding an example to the paper carousel

1. Put a catchy figure in doc/_static/usecases. It should be 750X380 px. You can use this command:

mkdir -p converted
mogrify -path converted -format jpg -resize "750x380^" -gravity North -crop 750x380+0+0 +repage -quality 95 *.jpg

2. Add a new dictionary with additional information into doc/paper_carousel.py. The format should be like:

dict(img="jordi2018.jpg",
     title="Improved geophysical images through geostatistical regularization",
     subtitle="Jordi et al. (2018), Geophysical Journal International",
     link="https://doi.org/10.1093/gji/ggy055")

Coding Guidelines

• Generally we try to use PEP 8 https://www.python.org/dev/peps/pep-0008/?

• All names should be literally and in CamelShape style.

• Classes starts with Upper-case Letters.

• Members and Methods always starts with Lower-case Letters.

• All class members (self.member) need to be initialized in the init.

• (ugly²) Do not use multi initialize in one line, e.g., a, b, c = 0, 0, 0

• check for data types with ‘if isinstance(var, type):’ instead ‘if type(var) == type:’

Use pylint or prospector to improve code quality.

We use: (exceptions in .landscape.yml and .pylintrc)

• pep8

• pep257

• pylint

• pyflakes

Behaviour by name for global functions:

createFOO(...)
    """Always needs to return an instance of FOO.""

showFOO(Bar, ...)
    """Always open a window or optionally use a given Axes to show Bar as Foo."""
    return ax, cbar

drawFOO(ax, Bar...)
    """Always needs an Axes ax to draw Bar as Foo""
    return graphics_object

readFOO(fileName, *args):
    """Read object from disc."""
    return obj

importFOO(fileName, obj, *args):
    """Import object from disc into an existing object."""
    return obj

exportFOO(obj, fileName):
    """Export object to disc in foreign (FOOs) format."""
    return true

convertFOO(fooObj):
    """Convert Foo obj into gimli Obj"""
    return gimliObj

API Documentation and doctests

Use the following documentation syntax or see: https://www.sphinx-doc.org/en/master/usage/extensions/napoleon.html

def foo(arg1, arg2):
"""Short description, i.e., one line to explain what foo does. [DOT_AT_END]
  [ONE BLANKLINE]
  Explain a little more verbose was foo does. Use references :cite:`Archie1942`
  Use links to pygimli api :gimliapi:`GIMLI::Mesh`, `:py:mod:pygimli.manager`
  for modules, `:py:func:pygimli.solver.solveFiniteElements` for functions

  Use math.
  .. math :
      a + \sigma \cdot \rho

  Explain all parameters.

  Args
  ----
  arg1: type | use links to :gimliapi:`GIMLI::Mesh`
      Describe arg1.
  arg2: type
      Describe arg2.

  Keyword Args
  ------------
  args: type
      Description.

  Attributes
  ----------
  For members

  Returns
  -------
  type:

  Examples
  --------
  >>> import foo
  >>>
  >>>

  See Also
  --------
      average : Weighted average,
      e.g., Link to tutorials :ref:`tut:Modelling_BC` assuming there
      has been set an appropriate label in the tutorial.

  References
  ----------
  if not in global bib file
  """