namespace portions documentation
welcome to the documentation of the portions (app/service modules and sub-packages) of this freely extendable aedev namespace (PEP 420).
features and use-cases
the portions of this namespace are simplifying your Python application or service in the areas/domains:
continuous integration
continuous deployment
outsourced text file (maintained centrally)
code and config file templates
git repository management (locally and remotely)
software development
software testing and QA
code maintenance guidelines
portions code requirements
pure python
fully typed (PEP 526)
fully documented
100 % test coverage
multi thread save
code checks (using pylint and flake8)
design pattern and software principles
contributing
we want to make it as easy and fun as possible for you to contribute to this project.
reporting bugs
before you create a new issue, please check to see if you are using the latest version of this project; the bug may already be resolved.
also search for similar problems in the issue tracker; it may already be an identified problem.
include as much information as possible into the issue description, at least:
version numbers (of Python and any involved packages).
small self-contained code example that reproduces the bug.
steps to reproduce the error.
any traceback/error/log messages shown.
requesting new features
on the git repository host server create new issue, providing a clear and detailed explanation of the feature you want and why it’s important to add.
if you are able to implement the feature yourself (refer to the contribution steps section below).
contribution steps
thanks for your contribution – we’ll get your merge request reviewed. you could also review other merge requests, just like other developers will review yours and comment on them. based on the comments, you should address them. once the reviewers approve, the maintainers will merge.
before you start make sure you have a GitLab account.
contribution can be done either with the git-repo-manager tool
or directly by using
the git
command and the Gitlab
server.
using the git repository manager grm
fork and clone the repository of this project to your computer
in your console change the working directory to your project’s parent folder. then run the following command with the
new_feature_or_fix
part replaced by an appropriate branch name, describing shortly your contribution:grm -b new_feature_or_fix fork aedev-group/aedev_aedev
Note
the
grm fork
action will also add the forked repository as the remoteupstream
to your local repository.after the repository fork you change your current working directory to the new working tree root folder, created by the
grm fork
action, and execute thegrm renew
action. this will prepare a new package version of the project and upgrade the project files created from templates to its latest version.code and check
now use your favorite IDE/Editor to implement the new feature or code the bug fix. don’t forget to amend the project with new unit and integrity tests, and ensure they pass, by executin from time to time the
grm check
action.publish your changes
before you initiate a push/merge request against the Gitlab server, execute the
grm prepare
action, which will create, with the help of thegit diff
command, a .commit_msg.txt file in the working tree root of your project, containing a short summary in the first line followed with a blank line and a list of the project files that got added, changed or deleted.Hint
the .commit_msg.txt file can be amended by any text editor before you run the
grm commit
action. for changes initiated by an issue please include the issue number (in the formatfixes #<issue-number>
) into this file. you may useMarkdown
syntax in this file for simple styling.to finally commit and upload your changes run the following three grm actions in the root folder of your project:
grm commit grm push grm request
the
grm commit
command is first executing agrm check
action to do a finally check of the project resources and to run the unit and integrity tests. if all these checks pass then a new git commit will be created, including your changes to the project.grm push``will then push the commit to your ``origin
remote repository (your fork) andgrm request
will finally create a bew merge/pull request against theupstream
remote repository (the forked one).Hint
to complete the workflow a maintainer of the project has to execute the
grm release
action. this will merge your changes into the main branch develop of theupstream
repository and then release a new version of the project onto PyPI.
more detailed information of the features of the grm
tool are available within the grm user manual.
using git and Gitlab
alternatively to the grm
tool you could directly use the git command suite and the
Gitlab website to achieve the same (with a lot more of typing and fiddling ;-):
fork the upstream repository into your user account.
clone your forked repo as
origin
remote to your computer, and add anupstream
remote for the destination repo by running the following commands in the console of your local machine:git clone https://gitlab.com/<YourGitLabUserName>/aedev_aedev.git git remote add upstream https://gitlab.com/aedev-group/aedev_aedev.git
checkout out a new local feature branch and update it to the latest version of the
develop
branch:git checkout -b <new_feature_or_fix_branch_name> develop git pull --rebase upstream develop
please keep your code clean by staying current with the
develop
branch, where code will be merged. if you find another bug, please fix it in a separated branch instead.push the branch to your fork. treat it as a backup:
git push origin <new_feature_or_fix_branch_name>
code
implement the new feature or the bug fix; include tests, and ensure they pass.
check
run the basic code style and typing checks locally (pylint, mypy and flake8) before you commit.
commit
for every commit please write a short summary in the first line followed with a blank line and then more detailed descriptions of the change. for bug fixes please include any issue number (in the format #nnn) in your summary:
git commit -m "issue #123: put change summary here (can be a issue title)"
Note
never leave the commit message blank! provide a detailed, clear, and complete description of your changes!
publish your changes (prepare a Merge Request)
before submitting a merge request, update your branch to the latest code:
git pull --rebase upstream develop
if you have made many commits, we ask you to squash them into atomic units of work. most issues should have one commit only, especially bug fixes, which makes them easier to back port:
git checkout develop git pull --rebase upstream develop git checkout <new_feature_or_fix_branch_name> git rebase -i develop
push changes to your fork:
git push -f
issue/make a GitLab Merge Request:
navigate to your fork where you just pushed to
click Merge Request
in the branch field write your feature branch name (this is filled with your default branch name)
click Update Commit Range
ensure the changes you implemented are included in the Commits tab
ensure that the Files Changed tab incorporate all of your changes
fill in some details about your potential patch including a meaningful title
click New merge request.
release to PyPI
the release of a new/changed project will automatically be initiated by the GitLab CI, using the two
protected vars PYPI_USERNAME
and PYPI_PASSWORD
(marked as masked) from the users group of this namespace, in
order to provide the user name and password of the maintainers PyPI account (on Gitlab.com at Settings/CI_CD/Variables).
useful links and resources
grm (git repository manager) module
project repository
and user manual
create new namespace
a PEP 420 namespace splits the codebase of a library or framework into multiple project repositories, called portions (of the namespace).
Hint
the aedev namespace is providing the grm tool to create and maintain any namespace and its portions.
the id of a new namespace consists of letters only and has to be available on PYPI. the group-name name gets by default
generated from the namespace name plus the suffix '-group'
, so best choose an id that results in a group name that
is available on your repository server.
register a new namespace portion
follow the steps underneath to add and register a new module as portion onto the aedev namespace:
open a console window and change the current directory to the parent directory of your projects root folders.
choose a not-existing/unique name for the new portion (referred as <portion-name> in the next steps).
run
grm --namespace=aedev --project=<portion_name> new-module
to register the portion name within the namespace, to create a new project folder aedev_<portion-name> (providing initial project files created from templates) and to get a pre-configured git repository (with the remote already set and the initial files unstaged, to be extended, staged and finally committed).run
cd aedev_<portion-name>
to change the current to the working tree root of the new portion project.run pyenv local venv_name (or any other similar tool) to create/prepare a local virtual environment.
fans of TDD are then coding unit tests in the prepared test module test_aedev_<portion-name>.py, situated within the tests sub-folder of your new code project folder.
extend the file <portion_name>.py situated in the aedev sub-folder to implement the new portion.
run
grm check-integrity
to run the linting and unit tests (if they fail go one or two steps back).run
grm prepare
, then amend the commit message within the file .commit_msg.txt, then rungrm commit
andgrm push
to commit and upload your new portion to your personal remote/server repository fork, and finally rungrm request
to request the merge/pull into the forked/upstream repository in the users group aedev-group (at https://gitlab.com/aedev-group).
the registration of a new portion to the aedev namespace has to be done by a namespace maintainer.
registered portions will automatically be included into the aedev namespace documentation, available at ReadTheDocs.
registered namespace package portions
the following list contains all registered portions of the aedev namespace, plus additional modules of each portion.
Hint
a not on the ordering: portions with no dependencies are at the begin of the following list. the portions that are depending on other portions of the aedev namespace are listed more to the end.
project setup helper functions |
|
outsourced Python project files templates |
|
templates and outsourced files for namespace root projects. |
|
aedev_tpl_add module main module |
|
create and maintain local/remote git repositories of Python projects |
|
main module of git_repo_manager. |
|
web api for www.pyanywhere.com and eu.pyanywhere.com |
|
individually configurable setup hook |
manuals and tutorials
indices and tables
ae
namespace projects and documentationaedev
namespace projects and documentation