Developer’s Guide
The Python environment
The first part of setting up your development environment is to ensure you have Python installed. pydecipher requires Python 3.8 or newer, and we recommend using the latest stable version of Python that has been released. Ultimately, how you set up your development environment is up to you, but the following instructions can serve as a guideline for someone who doesn’t already have their own preferences.
Attention
The following instructions apply to macOS/*nix platforms. If you will be developing on Windows, you’re on your own. And if you’d like to share your workflow, please feel free to contribute a Windows environment set-up guide!
Install pyenv to manage your system’s python environment more easily and reliably than using apt/brew/whatever package manager your OS uses.
Install pyenv-virtualenv to create and manage virtual environments within pyenv.
Run
pyenv install 3.8.2, or whatever is the latest stable release of Python.Create a virtual environment for pydecipher development with
pyenv virtualenv 3.8.2 pydecipher
Installing the package for development
Clone the repository:
$ git clone <pydecipher-url>
(Optional) If you are using pyenv-virtualenv to manage a pydecipher development virtual environment, enter the code directory you just cloned and run:
$ pyenv local pydecipher # or whatever you named your virtual environment
This will drop a
.python-versionfile in the root level of your pydecipher directory, enabling pyenv to automatically activate the environment whenever your current working directory is set to the pydecipher directory, or any of its subdirectories.
Ensure you are in the correct python environment, and then run the
pip installcommand with the –editable flag:$ pip install -e .[dev]
Check that pydecipher installed correctly by running:
$ pydecipher -V pydecipher 1.0.0
Install pre-commit:
$ pre-commit install
Building the Docker container
In order to build the Docker image, you will need to first clone the repository. After cloning the repository, change directory into the root directory of your working copy of the repository (the directory containing the Dockerfile), and run the following command to build the Docker image.
1$ docker build -t pydecipher .
After the Docker image is built, pydecipher can be run from within a Docker container by following the directions in Using the pydecipher Docker container. If changes to the code have been made, but are seemingly not appearing in the container, rebuilding the image using the --no-cache argument will build the image from scratch without using any cached image layers.
Style and formatting
All code should be run through black before submitting a PR. See the pyproject.toml file in the root of the directory to see our black configuration. For things that black doesn’t account for (naming conventions, import conventions, etc), please attempt to follow existing conventions in the codebase and PEP8 to the best of your ability (in that order).
Versioning
We follow PEP 440-versioning (by way of bump2version) to maintain the version number through all code and documentation. We differ from PEP 440 guidelines in that our pre-release versions don’t have an alpha build, and we don’t use X.Y as shorthand for X.Y.0. Our versions go as follows:
Version Format |
Release Type |
X.Y.Zb |
Beta release |
X.Y.Zrc |
Release Candidate |
X.Y.Z |
Final release |
Generally, releases should go from beta to release candidate, and release candidate to final. The following bump2version commands can be used to follow this format.
$ bump2version release # 1.0.0b → 1.0.0 $ bump2version patch # 1.0.0 → 1.0.1b $ bump2version minor # 1.0.1b → 1.1.0b $ bump2version patch --new-version 1.1.0rc # 1.1.0b → 1.1.0rc $ bump2version patch --new-version 1.1.0 # 1.1.0rc → 1.1.0 $ bump2version major # 1.1.0 → 2.0.0b
The bump2version command will tag a commit, but you can use the --verbose and --dry-run flags to prevent this and see what exactly will be changed before deciding if you actually want to run the bump2version command.
$ bump2version --verbose --dry-run patch ['patch'] current_version=1.0.0 commit=True tag=False files=pydecipher/__init__.py parse=(?P<major>\d+)\.(?P<minor>\d+)(\.(?P<patch>\d+))?(?P<release>[a-z]+)? serialize= {major}.{minor}.{patch}{release} {major}.{minor}.{patch} {major}.{minor} new_version=1.0.1b
For more on versioning, read this Medium article about PEP440 and bumpversion.
Git
Commits
Commits should have a short, descriptive title, and a body that explains the why behind the commit. Each logical change in the code should be placed in its own commit, without extraneous changes (fixing a typo in a totally unrelated file). For more on writing good commit messages, see the PyInstaller developer’s guide.
Branches
We follow Vincent Driessen’s git branching model as described in this blog post.
- develop branch:
origin/develop is the main development branch where HEAD’s source code is at a semi-stable state, waiting to be included in the next release.
- master branch:
origin/master is the release branch, where each commit considered a new release version. This includes beta and release candidates versions.
- release/ branches:
These branches only appear when a new version is in the process of being released. They serve as a staging ground for the release workflow.
- hotfix/ branches:
These branches are for urgent, unplanned production releases.
- feature/ branches:
Feature branches are used to develop new features, and upon completion, get merged into develop.
Merge/Pull Requests
To create a pull request, first fork the repository and clone the fork’s code:
$ git clone <pydecipher_repo_url> $ cd pydecipher
Now, create a branch:
$ git checkout -b feature/my-new-feature
Make your changes! Upon completion, please make sure you are still passing the tests. If possible, test on all platforms. Additionally, make sure to add an appropriate change-file to the docs/changes directory. See the changelog guidelines for more details. After adequate testing and documentation, synchronize your fork with the pydecipher upstream repository through a rebase or merge.
Rebase your changes on the current development head.
$ git remote add upstream <pydecipher_repo_url> $ git checkout feature/my-new-feature $ git pull --rebase upstream develop
Merge the current development head into your changes:
$ git remote add upstream <pydecipher_repo_url> $ git fetch upstream develop $ git checkout feature/my-new-feature $ git merge upstream/develop
Push your changes up to your fork:
$ git push
Lastly, open the Merge Requests page at <pydecipher_repo_url> and click “new merge request”.
Changelog and towncrier
We use towncrier to keep track of our changelog. With each pull request, please include a reStructuredText file of the format issue_number.category.rst. The issue number corresponds to the issue on GitHub, and the category is one of the following standard towncrier categories:
File Ext. |
Description |
.feature |
New features |
.bugfix |
Bug fix |
.doc |
Documentation improvement |
.removal |
Deprecation/removal of public API |
.misc |
Issue closed, but not of interest to users |
For example, if you were submitting a pull request for a new feature that adds support for FooBar-frozen Python artifacts (issue #1337 on GitHub), your file 1337.feature.rst would have the following contents:
Added capability to extract source code from FooBar-frozen Python binaries.
Documentation
New modules should be documented with numpy style doc-strings.
Testing
Before merging new code, ensure that all the integration tests pass by using pytest in the tests/ directory. As xdis/uncompyle6 improve and get better at decompiling Python bytecode, some of the tests may fail because they are expecting a certain exact amount of files to be decompiled. If the tests fail because more files are present, then those numbers in the tests should be increased due to reflect the new counts.
To run some of the integration tests, test files that are not included in this repository are required. The test files and their respective SHA256 hashes are shown in the table below.