diff -Nru jpylyzer-1.17.0/BUILD_HOWTO_LINUX jpylyzer-1.18.0/BUILD_HOWTO_LINUX --- jpylyzer-1.17.0/BUILD_HOWTO_LINUX 2016-01-05 16:26:23.000000000 +0000 +++ jpylyzer-1.18.0/BUILD_HOWTO_LINUX 1970-01-01 00:00:00.000000000 +0000 @@ -1,63 +0,0 @@ -This howto explains how to build 'jpylyzer' as an linux executable so that python is no longer a dependency. -This was done because there was an incompatibility between 'jpylyzer' and python version below 2.7. - -## INSTRUCTIONS (APPLIES TO PYINSTALLER 2) - -1. Download 'pyinstaller' from: - -http://www.pyinstaller.org/ - -2. Unpack it: - -tar -xjf pyinstaller-2.0.tar.bz2 - -3. compile (in below example result as one single file): - -python pyinstaller.py --onefile ./jpylyzer/jpylyzer.py - -4. look in the directory 'dist'. - -Voila! - -## INSTRUCTIONS (APPLIES TO PYINSTALLER 1.5) - -With PyInstaller 1.5 (which I think is the version that is still used on the Amazon Webcloud setup) -you first need to make a .spec file, so the compilation involves two steps: - -python Makespec.py --onefile ./jpylyzer/jpylyzer.py -python pyinstaller.py jpylyzer.spec - -## NOTE ON JPYLYZER MAKEFILE - -The current version of the makefile uses the following commands: - -pymakespec --onefile ./jpylyzer/jpylyzer.py -pyinstaller jpylyzer.spec - -Apparently pymakespec/pyinstaller are helper scripts that call Makespec.py and pyinstaller.py, respectively. - -This works on the Amazon Webcloud setup, but for local builds you will need to change these lines -so that the actual Python scripts are called. To make things slightly more complicated PyInstaller -has no default installation location, so you'll have to insert the file paths that apply to you -system, e.g.: - -python /home/johan/pyinstall/Makespec.py --onefile ./jpylyzer/jpylyzer.py -python /home/johan/pyinstall/pyinstaller.py jpylyzer.spec - -## BUILDING THE DEBIAN PACKAGES LOCALLY - -You can build jpylyzer locally using: - -dpkg-buildpackage -tc - -However this will result in an error because of a missing changelog file. Apparently it is generated -automatically on the Amazon Webcloud setup, but I'm not exactly sure how! For testing purposes you may -use a dummy changelog as a workaround. - -In addition you will need to edit the makefile as described in the previous section. - -More info on all this here: - -http://www.openplanetsfoundation.org/blogs/2013-04-23-adventures-debian-packaging - -Updated 2/5/2013 by JvdK \ No newline at end of file diff -Nru jpylyzer-1.17.0/BUILD_HOWTO_LINUX.md jpylyzer-1.18.0/BUILD_HOWTO_LINUX.md --- jpylyzer-1.17.0/BUILD_HOWTO_LINUX.md 1970-01-01 00:00:00.000000000 +0000 +++ jpylyzer-1.18.0/BUILD_HOWTO_LINUX.md 2017-08-23 11:42:58.000000000 +0000 @@ -0,0 +1,11 @@ +# How to build stand-alone Linux binaries + +To build stand-alone Linux binaries, simply run the *build-with-pyinstaller.sh* script from the jpylyzer root directory: + + ./build-with-pyinstaller.sh + +This will create the binaries under *pyi-build/dist*. The build script requires PyInstaller. If PyInstaller is not installled, it can be installed using: + + (sudo) pip install pyinstaller + +(If you don't know if you have PyInstaller just run the build script, which will display a message id it cannot find PyInstaller.) diff -Nru jpylyzer-1.17.0/BUILD_HOWTO_WIN32 jpylyzer-1.18.0/BUILD_HOWTO_WIN32 --- jpylyzer-1.17.0/BUILD_HOWTO_WIN32 2016-01-05 16:26:23.000000000 +0000 +++ jpylyzer-1.18.0/BUILD_HOWTO_WIN32 1970-01-01 00:00:00.000000000 +0000 @@ -1,38 +0,0 @@ -This howto explains how to build 'jpylyzer' as a 32-bit Windows -executable so that Python is no longer a dependency. Procedure described -here will also pack binaries in a distribution-ready ZIP file. - -This was tested under Windows 7 using PyInstaller 2.0. (I haven't found -a way to build actual Win64 binaries, since the build packages don't -appear to support this yet). For older PyInstaller versions (1.5) things -are a bit more complicated, so use version 2 if possible. - -SET-UP AND CONFIGURATION (YOU ONLY NEED TO GO THROUGH THESE STEPS ONCE): - -1. Download and install 'PyWin32' (Python for Windows extensions) from: - -http://sourceforge.net/projects/pywin32/files/ - -2. Download 'pyinstaller' from - -http://www.pyinstaller.org/ - -3. Unpack/unzip it with your favourite file archive manager. PyInstaller -is now ready for use. - -4. Open the batch file 'buildWin32.bat' which is part of the jpylyzer -distribution, and update the configuration variables 'python' and -'pathPyInstaller' according to your own system - -BUILDING THE WIN32 DISTRIBUTION: - -1. Run the batch file from the repo's root dir ( i.e. the one in which -it is located): - -buildWin32.bat - -2. Look at the zip file in the directory 'dist_win32'. - -Voila! - - diff -Nru jpylyzer-1.17.0/BUILD_HOWTO_WINDOWS.md jpylyzer-1.18.0/BUILD_HOWTO_WINDOWS.md --- jpylyzer-1.17.0/BUILD_HOWTO_WINDOWS.md 1970-01-01 00:00:00.000000000 +0000 +++ jpylyzer-1.18.0/BUILD_HOWTO_WINDOWS.md 2017-08-23 11:42:58.000000000 +0000 @@ -0,0 +1,35 @@ +# How to build the Windows binaries + + +## Prerequisites + +Windows binaries are now built under Linux using [Wine](https://www.winehq.org/). Make sure you have a recent version of Wine installed. Also check if (a recent version of) the *winbind* package is installed. If not, install it using: + + sudo apt-get install winbind + +## Building the binaries + +In your console window, go to the root of the jpylyzer directory. Then run: + + ./buildwin.sh + +The script first checks for the presence of (portable) 64 and 32 versions of Python 2.7 under *Wine*, and installs them if they are not found. Note that the installers need some manual input. Most importantly, make sure you enter the following installation paths: + +* `C:\Python27_64` for the 64-bit version; + +* `C:\Python27_32` for the 32-bit version. + +The script also automatically installs PyInstaller ifit is not there already. + +Once the above dependencies are installed, 64 and 32 bit binaries are built automatically. The (zipped) binaries can be found in the *dist* directory. + +## Troubleshooting + +If the output of the build script includes this error: + + err:winediag:SECUR32_initNTLMSP ntlm_auth was not found or is outdated. Make sure that ntlm_auth >= 3.0.25 is in your path. Usually, you can find it in the winbind package of your distribution. + +Fix this by installing *winbind* (see the top of this page). + + + \ No newline at end of file diff -Nru jpylyzer-1.17.0/buildWin32.bat jpylyzer-1.18.0/buildWin32.bat --- jpylyzer-1.17.0/buildWin32.bat 2016-01-05 16:26:23.000000000 +0000 +++ jpylyzer-1.18.0/buildWin32.bat 1970-01-01 00:00:00.000000000 +0000 @@ -1,68 +0,0 @@ -: -:: Build 32 bit Windows jpylyzer binaries from Python script, and pack them in ZIP file -:: -:: ZIP file includes license file, PDF User Manual and example files -:: -:: Johan van der Knijff, 25 april 2013 -:: -:: Dependencies: -:: -:: - Python 2.7, 32-bit version (PyInstaller doesn't work with Python 3 yet!) -:: - PyInstaller 2: http://www.pyinstaller.org/ -:: - PyWin32 (needed by PyInstaller): http://sourceforge.net/projects/pywin32/files/ -:: - a spec file with -:: -@echo off -setlocal - -::::::::: CONFIGURATION :::::::::: - -:: Python -:: Note that to produce a 32-bit binary we need a 32-bit Python version! -::set python=c:\python27\python -set python=c:\python27_32bit\python - -:: Path to PyInstaller -set pathPyInstaller=c:\pyinstall\ - -:: Script base name (i.e. script name minus .py extension) -set scriptBaseName=jpylyzer - -:: PyInstaller spec file that defines build options -set specFile=jpylyzer_win32.spec - -:: Directory where build is created (should be identical to 'name' in 'coll' in spec file!!) -set distDir=.\dist_win32\ - -:: Executes jpylyzer with -v option and stores output to -:: env variable 'version' -set vCommand=%python% .\%scriptBaseName%\%scriptBaseName%.py -v -%vCommand% 2> temp.txt -set /p version= < temp.txt -del temp.txt - -::::::::: BUILD :::::::::::::::::: - -:: Build binaries -%python% %pathPyInstaller%\pyinstaller.py %specFile% - -:: Generate name for ZIP file -set zipName=%scriptBaseName%_%version%_win32.zip - -:: Create ZIP file -%python% zipdir.py %distDir%\jpylyzer %distDir%\%zipName% - -::::::::: CLEANUP ::::::::::::::::: - -:: Delete build directory -rmdir build /S /Q - -:: Delete jpylyzer directory in distdir -rmdir %distDir%\jpylyzer /S /Q - -::::::::: PARTY TIME! ::::::::::::::::: - -echo / -echo Done! Created %zipName% in directory %distDir%! -echo / - diff -Nru jpylyzer-1.17.0/buildWin64.bat jpylyzer-1.18.0/buildWin64.bat --- jpylyzer-1.17.0/buildWin64.bat 2016-01-05 16:26:23.000000000 +0000 +++ jpylyzer-1.18.0/buildWin64.bat 1970-01-01 00:00:00.000000000 +0000 @@ -1,68 +0,0 @@ -: -:: Build 64 bit Windows jpylyzer binaries from Python script, and pack them in ZIP file -:: -:: ZIP file includes license file, PDF User Manual and example files -:: -:: Johan van der Knijff, 25 april 2013 -:: -:: Dependencies: -:: -:: - Python 2.7, 64-bit version (PyInstaller doesn't work with Python 3 yet!) -:: - PyInstaller 2: http://www.pyinstaller.org/ -:: - PyWin32 (needed by PyInstaller): http://sourceforge.net/projects/pywin32/files/ -:: - a spec file with -:: -@echo off -setlocal - -::::::::: CONFIGURATION :::::::::: - -:: Python -:: Note that to produce a 32-bit binary we need a 32-bit Python version! -set python=c:\python27\python -::set python=c:\python27_32bit\python - -:: Path to PyInstaller -set pathPyInstaller=c:\pyinstall\ - -:: Script base name (i.e. script name minus .py extension) -set scriptBaseName=jpylyzer - -:: PyInstaller spec file that defines build options -set specFile=jpylyzer_win64.spec - -:: Directory where build is created (should be identical to 'name' in 'coll' in spec file!!) -set distDir=.\dist_win64\ - -:: Executes jpylyzer with -v option and stores output to -:: env variable 'version' -set vCommand=%python% .\%scriptBaseName%\%scriptBaseName%.py -v -%vCommand% 2> temp.txt -set /p version= < temp.txt -del temp.txt - -::::::::: BUILD :::::::::::::::::: - -:: Build binaries -%python% %pathPyInstaller%\pyinstaller.py %specFile% - -:: Generate name for ZIP file -set zipName=%scriptBaseName%_%version%_win64.zip - -:: Create ZIP file -%python% zipdir.py %distDir%\jpylyzer %distDir%\%zipName% - -::::::::: CLEANUP ::::::::::::::::: - -:: Delete build directory -rmdir build /S /Q - -:: Delete jpylyzer directory in distdir -rmdir %distDir%\jpylyzer /S /Q - -::::::::: PARTY TIME! ::::::::::::::::: - -echo / -echo Done! Created %zipName% in directory %distDir%! -echo / - diff -Nru jpylyzer-1.17.0/buildwin.sh jpylyzer-1.18.0/buildwin.sh --- jpylyzer-1.17.0/buildwin.sh 1970-01-01 00:00:00.000000000 +0000 +++ jpylyzer-1.18.0/buildwin.sh 2017-08-23 11:42:58.000000000 +0000 @@ -0,0 +1,138 @@ +#!/bin/bash + +# Build 64 and 32 bit Windows binaries using Wine and WinPython. If the required Wine environment +# (WinPython + PyInstaller) cannot be found it is set up quasi-automatically (the WinPython installer +# needs some manual input) +# Precondition: 64-bit version of Wine is already installed. + +# WinPython download URLS +downloadURL64bit=https://sourceforge.net/projects/winpython/files/WinPython_2.7/2.7.13.1/WinPython-64bit-2.7.13.1Zero.exe/download +downloadURL32bit=https://sourceforge.net/projects/winpython/files/WinPython_2.7/2.7.13.1/WinPython-32bit-2.7.13.1Zero.exe/download + +# PyInstaller spec files that defines build options +specFile64bit=jpylyzer_win64.spec +specFile32bit=jpylyzer_win32.spec + +# Script base name (i.e. script name minus .py extension) +scriptBaseName=jpylyzer + +# Wine debug variable (suppresses garbage debugging messages) +WineDebug="-msvcrt" +#WineDebug="fixme-all" + +installPython(){ + # Installs Python. Arguments: + # - $1: bitness (32 or 64) + # - $2: download URL + echo "Downloading installer" + wget $2 -O pyTemp.exe + echo "" + echo "Installing Python. This requires some user input" + echo "" + echo "Follow installer instructions. For Destination Folder replace default path with C:\Python27_"$1 + echo "" + + WINEDEBUG=$WineDebug wine pyTemp.exe + + echo "Removing installer" + rm pyTemp.exe +} + +installPyInstaller(){ + # Installs pyInstaller if it is not installed already. Argument: + # - $1: python (full path of python interpreter) + echo "Checking for pyinstaller" + WINEDEBUG=$WineDebug wine $1 -m pip show pyinstaller + + if [ $? -eq 0 ]; then + echo "Pyinstaller already installed" + else + echo "Installing pyinstaller" + WINEDEBUG=$WineDebug wine $1 -m pip install pyinstaller + fi +} + +buildBinaries(){ + # Builds Windows binaries. + + # Read arguments: + bitness=$1 + pyRoot=$2 + pyInstallerWine=$pyRoot"/Scripts/pyinstaller.exe" + pythonWine=$3 + specFile=$4 + + # Working directory + workDir=$PWD + + # Directory where build is created (should be identical to 'name' in 'coll' in spec file!!) + distDir=$workDir"/dist/win"$bitness"/" + + # Executes jpylyzer with -v option and stores output to + # env variable 'version' + # Also trim trailing EOL character and replace '.' by '_' + WINEDEBUG=$WineDebug wine $pythonWine -m $scriptBaseName -v 2> temp.txt + version=$(head -n 1 temp.txt | tr -d '\r' |tr '.' '_' ) + rm temp.txt + + echo "Building binaries" + WINEDEBUG=$WineDebug wine $pyInstallerWine $specFile --distpath=$distDir + + # Generate name for ZIP file + zipName=$scriptBaseName"_"$version"_win"$bitness".zip" + echo zipName + + echo "Creating ZIP file" + cd $distDir + zip -r $zipName $scriptBaseName + cd $workDir + + echo "Deleting build directory" + rm -r $workDir"/build" + rm -r $distDir/$scriptBaseName +} + +echo "64 bit Python" + +if [ -d $HOME"/.wine/drive_c/Python27_64" ]; then + echo "Python (64 bit) already installed" +else + echo "Python (64 bit) not yet installed, installing now" + echo "" + installPython 64 $downloadURL64bit +fi + +# Get path to Python root +pyRoot64=$(ls -d ~/.wine/drive_c/Python27_64/python-*) + +# Python interpreter +python64=$pyRoot64"/python.exe" + +# Install PyInstaller (if not installed already) +installPyInstaller $python64 + +echo "32 bit Python" + +if [ -d $HOME"/.wine/drive_c/Python27_32" ]; then + echo "Python (32 bit) already installed" +else + echo "Python (32 bit) not yet installed, installing now" + echo "" + installPython 32 $downloadURL32bit +fi + +# Get path to Python root +pyRoot32=$(ls -d ~/.wine/drive_c/Python27_32/python-*) + +# Python interpreter +python32=$pyRoot32"/python.exe" + +# Install PyInstaller (if not installed already) +installPyInstaller $python32 + +echo "Building binaries, 64 bit" +buildBinaries 64 $pyRoot64 $python64 $specFile64bit + +echo "Building binaries, 32 bit" +buildBinaries 32 $pyRoot32 $python32 $specFile32bit + diff -Nru jpylyzer-1.17.0/build-with-pyinstaller.sh jpylyzer-1.18.0/build-with-pyinstaller.sh --- jpylyzer-1.17.0/build-with-pyinstaller.sh 2016-01-05 16:26:23.000000000 +0000 +++ jpylyzer-1.18.0/build-with-pyinstaller.sh 2017-08-23 11:42:58.000000000 +0000 @@ -2,10 +2,14 @@ # Bash script to build jpylyzer using PyInstaller +# Script base name (i.e. script name minus .py extension) +scriptBaseName=jpylyzer + # First check for PyInstaller command -v pyinstaller >/dev/null 2>&1 || { - echo >&2 "http://www.pyinstaller.org/ is required to build the Jpylyzer executable."; - echo >&2 "Please install PyInstaller http://pythonhosted.org/PyInstaller/#installing-pyinstaller."; + echo >&2 "PyInstaller is required to build the executable."; + echo >&2 "Please install PyInstaller with:"; + echo >&2 " (sudo) pip install pyinstaller" exit 1; } @@ -16,12 +20,12 @@ if [ $originalUserId == 0 ] then uname=$(getent passwd 1000 | cut -d: -f1) - sudo -u $uname "pyi-makespec --strip --onefile --paths=jpylyzer --specpath=pyi-build ./jpylyzer/jpylyzer.py" - sudo -u $uname "pyinstaller --strip --clean --paths=jpylyzer --distpath=pyi-build/dist --workpath=pyi-build/build ./pyi-build/jpylyzer.spec" + sudo -u $uname "pyi-makespec --strip --onefile --paths=$scriptBaseName --name=$scriptBaseName --specpath=pyi-build ./cli.py" + sudo -u $uname "pyinstaller --strip --clean --paths=$scriptBaseName --distpath=pyi-build/dist --workpath=pyi-build/build ./pyi-build/$scriptBaseName.spec" else # So making stripped binaries for debian packaging - pyi-makespec --strip --onefile --paths=jpylyzer --specpath=pyi-build ./jpylyzer/jpylyzer.py - pyinstaller --strip --clean --paths=jpylyzer --distpath=pyi-build/dist --workpath=pyi-build/build ./pyi-build/jpylyzer.spec + pyi-makespec --strip --onefile --paths=$scriptBaseName --name=$scriptBaseName --specpath=pyi-build ./cli.py + pyinstaller --strip --clean --paths=$scriptBaseName --distpath=pyi-build/dist --workpath=pyi-build/build ./pyi-build/$scriptBaseName.spec fi -./pyi-build/dist/jpylyzer --version; +./pyi-build/dist/$scriptBaseName --version; diff -Nru jpylyzer-1.17.0/cli.py jpylyzer-1.18.0/cli.py --- jpylyzer-1.17.0/cli.py 1970-01-01 00:00:00.000000000 +0000 +++ jpylyzer-1.18.0/cli.py 2017-08-23 11:42:58.000000000 +0000 @@ -0,0 +1,8 @@ +#! /usr/bin/env python +# +"""CLI wrapper script, ensures that relative imports work correctly in a PyInstaller build""" + +from jpylyzer.jpylyzer import main + +if __name__ == '__main__': + main() diff -Nru jpylyzer-1.17.0/debian/changelog jpylyzer-1.18.0/debian/changelog --- jpylyzer-1.17.0/debian/changelog 2016-02-09 10:18:10.000000000 +0000 +++ jpylyzer-1.18.0/debian/changelog 2017-09-21 10:05:42.000000000 +0000 @@ -1,3 +1,16 @@ +jpylyzer (1.18.0-2) unstable; urgency=medium + + * Add missing build-deps: python-setuptools + + -- Mathieu Malaterre Thu, 21 Sep 2017 12:05:42 +0200 + +jpylyzer (1.18.0-1) unstable; urgency=medium + + * New upstream release. + * Bump Std-Vers to 4.1.0, no changes needed + + -- Mathieu Malaterre Wed, 20 Sep 2017 20:53:12 +0200 + jpylyzer (1.17.0-1) unstable; urgency=medium * New upstream release diff -Nru jpylyzer-1.17.0/debian/control jpylyzer-1.18.0/debian/control --- jpylyzer-1.17.0/debian/control 2016-02-09 10:17:55.000000000 +0000 +++ jpylyzer-1.18.0/debian/control 2017-09-21 10:05:22.000000000 +0000 @@ -7,10 +7,10 @@ dh-python, lmodern, pandoc, - python (>= 2.6.6-3~), + python (>= 2.6.6-3~), python-setuptools, texlive-fonts-recommended, texlive-xetex -Standards-Version: 3.9.6 +Standards-Version: 4.1.0 X-Python-Version: >= 2.7 Homepage: https://github.com/openpreserve/jpylyzer diff -Nru jpylyzer-1.17.0/debian/rules jpylyzer-1.18.0/debian/rules --- jpylyzer-1.17.0/debian/rules 2016-02-09 10:07:54.000000000 +0000 +++ jpylyzer-1.18.0/debian/rules 2017-09-20 18:52:19.000000000 +0000 @@ -14,6 +14,7 @@ dh_clean jpylyzer.1 dh_clean doc/jpylyzerUserManual.pdf dh_clean doc/jpylyzerUserManual.txt + dh_clean jpylyzer.egg-info/ dh_clean override_dh_auto_build: diff -Nru jpylyzer-1.17.0/doc/jpylyzer.css jpylyzer-1.18.0/doc/jpylyzer.css --- jpylyzer-1.17.0/doc/jpylyzer.css 2016-01-05 16:26:23.000000000 +0000 +++ jpylyzer-1.18.0/doc/jpylyzer.css 2017-08-23 11:42:58.000000000 +0000 @@ -103,12 +103,16 @@ text-align: center; } +/* + math, math[mode="inline"] { display: inline; font-family: CMSY10, CMEX10, Symbol, Times; font-style: normal; } +*/ + math[mode="display"] { display: block; text-align: center; diff -Nru jpylyzer-1.17.0/doc/jpylyzerUserManual.html jpylyzer-1.18.0/doc/jpylyzerUserManual.html --- jpylyzer-1.17.0/doc/jpylyzerUserManual.html 2016-01-05 16:26:23.000000000 +0000 +++ jpylyzer-1.18.0/doc/jpylyzerUserManual.html 2017-08-23 11:42:58.000000000 +0000 @@ -6,7 +6,7 @@ Jpylyzer User Manual - + @@ -26,7 +26,7 @@
  • 2 Installation and set-up
    • @@ -139,51 +139,63 @@

    http://jpylyzer.openpreservation.org/

    You have three options:

      -

    1. Use the Python source code. This allows you to run the software as a Python script on most popular platforms (Windows, Linux, Mac, etc.). However, this requires that you have a recent version of the Python interpreter available on your system.

      2. +

    2. Install the software with the Pip package manager. This works on all platforms (Windows, Linux, Mac, etc.), but you need to have the Python interpreter available on your system. Jpylyzer is compatible with Python 2.7, and Python 3.2 and more recent (Python 3.0 and 3.1 are not supported).

    3. Alternatively, for Windows users there is also a set of stand-alone binaries1. These allow you to run jpylyzer as an executable Windows application, without any need for installing Python. This option is particularly useful for Windows users who cannot (or don’t want to) install software on their system.

    4. For Linux users Debian packages are available. These allow you to run jpylyzer without any need for installing Python.

    These options are described in the following sections.

    -

    2.2 Installation of Python script (Linux/Unix, Windows, Mac OS X)

    -

    First, download the source files using one of the ‘Source Code Downloads’ links on the OPF jpylyzer page.

    -

    Then unzip the contents of the ZIP file to an empty directory. If you are working on a Linux/Unix based system you may need to make the scripts executable, and convert any line breaks to Unix-style ones. To do this, use the following commands:

    -
    chmod 755 *.py
-dos2unix *.py
    -

    In order to run the script you will need either Python 2.7, or Python 3.2 (or more recent)2. Python can be downloaded from:

    -

    http://python.org/

    -

    2.2.1 Testing the installation

    -

    To test your installation, open a console window (or command prompt) and type:

    -
    %jpylyzerPath%/jpylyzer.py -h
    -

    In the above command, replace %jpylyzerPath% with the full path to the jpylyzer installation directory (i.e. the directory that contains ‘jpylyzer.py’ and its associated files). For example, if you extracted the files to directory ‘/home/jpylyzer’, the command would become:

    -
    /home/jpylyzer/jpylyzer.py -h
    -

    Executing this command should result in the following screen output:

    -
    usage: jpylyzer.py [-h] [--verbose] [--recurse] [--wrapper] [--nullxml]
-                   [--nopretty] [--version] jp2In [jp2In ...]
    -

    2.2.2 Troubleshooting

    -

    If the above test didn’t run successfully, first verify the following possible causes:

    - -
    #! /usr/bin/env python
    -

    into:

    -
    #! /usr/bin/env python27
    +

    2.2 Installation with Pip (Linux/Unix, Windows, Mac OS X)

    +

    2.2.1 General installation procedure

    +

    First make sure you have a recent version of pip. Then install jpylyzer with the following command:

    +
    pip install jpylyzer
    +

    2.2.2 Single user installation (Linux)

    +

    On most Linux systems the above command needs to be run as super user (see below). If you don't want this use the below command for a single-user install:

    +
    pip install jpylyzer --user
    +

    This will install the software to the .local folder (hidden by default!) in your home directory (~/.local). Next try to run jpylyzer by entering:

    +
    jpylyzer
    +

    Most likely this will result in:

    +
    jpylyzer: command not found
    +

    If this happens, add the directory ~/.local/bin (which is where the jpylyzer command-line tool is installed) to the PATH environment variable (you only need to do this once). To do this, locate the (hidden) file .profile in you home directory (~/), and open it in a text editor. Then add the following lines at the end of the file:

    +
    # set PATH so it includes the user's .local bin if it exists
+if [ -d "$HOME/.local/bin" ] ; then
+    PATH="$HOME/.local/bin:$PATH"
+fi
    +

    Save the file, log out of your session and then log in again. Open a command terminal and type:

    +
     jpylyzer
    +

    If all went well you now see this:

    +
    usage: jpylyzer [-h] [--verbose] [--recurse] [--wrapper] [--nullxml]
+                [--nopretty] [--version]
+                jp2In [jp2In ...]
+jpylyzer: error: the following arguments are required: jp2In
    +

    Which means that the installation was successful!

    +

    2.2.3 Global installation (Linux)

    +

    Simply enter:

    +
    sudo -H pip install isolyzer
    +

    No further configuration is needed in this case.

    +

    2.2.4 Note on pre-releases

    +

    The above command lines will only install stable versions of jpylyzer. In order to install the latest pre-release, add the --pre switch. For example:

    +
    sudo -H pip install jpylyzer --pre

    2.3 Installation of Windows binaries (Windows only)

    Download the binary using the link on the jpylyzer homepage. Unzip the contents of this file to an empty folder on your PC. Jpylyzer should now be ready for use.

    -

    2.3.1 Testing the installation

    +

    2.3.1 Testing the installation

    To test your installation, open a Command Prompt (‘DOS prompt’) and type:

    -
    %jpylyzerPath%\jpylyzer -h
    +
    %jpylyzerPath%\jpylyzer

    In the above command, replace %jpylyzerPath% with the full path to the jpylyzer installation directory (i.e. the directory that contains ‘jpylyzer.exe’ and its associated files). For example, if you extracted the files to directory c:\tools\jpylyzer, the command would become:

    -
    c:\tools\jpylyzer\jpylyzer -h
    +
    c:\tools\jpylyzer\jpylyzer

    Executing this command should result in the following screen output:

    -
    usage: jpylyzer.py [-h] [--verbose] [--recurse] [--wrapper] [--nullxml]
-                   [--nopretty] [--version] jp2In [jp2In ...]
    +
    usage: jpylyzer [-h] [--verbose] [--recurse] [--wrapper] [--nullxml]
+                [--nopretty] [--version]
+                jp2In [jp2In ...]
+jpylyzer: error: the following arguments are required: jp2In

    2.3.2 Running jpylyzer without typing the full path

    Optionally, you may also want to add the full path of the jpylyzer installation directory to the Windows ’Path’ environment variable. Doing so allows you to run jpylyzer from any directory on your PC without having to type the full path. In Windows 7 you can do this by selecting ‘settings’ from the ‘Start’ menu; then go to ‘control panel’/’system’ and go to the ‘advanced’ tab. Click on the ‘environment variables’ button. Finally, locate the ‘Path’ variable in the ‘system variables’ window, click on ‘Edit’ and add the full jpylyzer path (this requires local Administrator privileges). The settings take effect on any newly opened command prompt.

    2.4 Installation of Debian packages (Ubuntu/Linux)

    For a number of Linux architectures Debian packages of jpylyzer exist. To install, simply download the .deb file, double-click on it and select Install Package. Alternatively you can also do this in the command terminal by typing:

    sudo dpkg -i jpylyzer_1.13.0_i386.deb

    In both cases you need to have administrative privileges.

    +

    For Ubuntu and Debian alternative packages are available in the official release channels. To install simply run the following commands:

    +
    sudo apt-get update
+sudo apt-get install python-jpylyzer

    3 Using jpylyzer

    3.1 Overview

    This chapter describes the general use of jpylyzer. The first sections cover the use of jpylyzer as a command-line tool and as an importable Python module.

    @@ -289,15 +301,23 @@ 
    User warning: ignoring unknown box

    This happens if jpylyzer encounters a box that is not defined by JPEG 2000 Part 1. It should be noted that, to a large extent, JPEG 2000 Part 1 permits the presence of boxes that are defined outside the standard. Again, jpylyzer will simply ignore these and process all other boxes normally.

    3.3 Using jpylyzer as a Python module

    -

    Instead of using jpylyzer from the command-line, you can also import it as a module in your own Python programs. To do so, put all the jpylyzer source files in the same directory as your own code. Then import jpylyzer into your code by adding:

    -
    import jpylyzer
    -

    Subsequently you can call any function that is defined in jpylyzer.py. In practice you will most likely only need the checkOneFile function, which can be called in the following way:

    -
    jpylyzer.checkOneFile(file)
    -

    Here, file is the path to a file object. The function returns an element object that can either be used directly, or converted to XML using the ElementTree module3. The structure of the element object follows the XML output that described in Chapter 5.

    -

    Alternatively, you may only want to import the checkOneFile function, in which case the import statement becomes:

    -
    from jpylyzer import checkOneFile
    -

    This will allow you to call the function as follows:

    -
    checkOneFile(file)
    +

    Instead of using jpylyzer from the command-line, you can also import it as a module in your own Python programs. To do so, install jpylyzer with pip. Then import jpylyzer into your code by adding:

    +
    from jpylyzer import jpylyzer
    +

    Subsequently you can call any function that is defined in jpylyzer.py. In practice you will most likely only need the checkOneFile function. The following minimal script shows how this works:

    +
    #! /usr/bin/env python
+
+from jpylyzer import jpylyzer
+
+# Define JP2
+myFile = "/home/johan/jpylyzer-test-files/aware.jp2"
+
+# Analyse with jpylyzer, result to Element object
+myResult = jpylyzer.checkOneFile(myFile)
+
+# Return image height value
+imageHeight = myResult.findtext('./properties/jp2HeaderBox/imageHeaderBox/height')
+print(imageHeight)
    +

    Here, myResult is an Element object that can either be used directly, or converted to XML using the ElementTree module2. The structure of the element object follows the XML output that described in Chapter 5.

    4 Structure of a JP2 file

    4.1 Scope of this chapter

    This chapter gives a brief overview of the JP2 file format. A basic understanding of the general structure of JP2 is helpful for appreciating how jpylyzer performs its validation. It will also make it easier to understand jpylyzer‘s extracted properties, as these are reported as a hierarchical tree that corresponds to the internal structure of JP2.

    @@ -507,7 +527,7 @@

    If a file passed all tests, this is an indication that it is most likely valid JP2. In that case, the isValidJP2 element (section 5.4) has a value of “True” (and “False” in all other cases). These tests are all explained in chapters 6 and 7.

    5.6.1 Default and verbose reporting of test results

    -

    By default, jpylyzer only reports any tests that failed (i.e. returned “False”), including the corresponding part of the box structure. For a valid JP2 the tests element will be empty. If the --verbose flag is used, the results of all tests are included (including those that returned “True”)4.

    +

    By default, jpylyzer only reports any tests that failed (i.e. returned “False”), including the corresponding part of the box structure. For a valid JP2 the tests element will be empty. If the --verbose flag is used, the results of all tests are included (including those that returned “True”)3.

    5.7 properties element

    This element contains the extracted image properties, which are organised in a hierarchical tree that corresponds to JP2’s box structure. See chapters 6 and 7 for a description of the reported properties.

    6 JP2: box by box

    @@ -860,7 +880,7 @@ Enumerated colourspace (as descriptive text string) -icc (if meth equals “Restricted ICC” or “Any ICC”5) +icc (if meth equals “Restricted ICC” or “Any ICC”4) Properties of ICC profile as child element (see below) @@ -1008,7 +1028,7 @@ iccPermittedProfileClass (if meth equals “Restricted ICC”) -ICC profile class is “input device” or “display device”6 +ICC profile class is “input device” or “display device”5 iccNoLUTBasedProfile (if meth equals “Restricted ICC”) @@ -1273,19 +1293,19 @@ vRescInPixelsPerMeter -Vertical grid resolution, expressed in pixels per meter7 +Vertical grid resolution, expressed in pixels per meter6 hRescInPixelsPerMeter -Horizontal grid resolution, expressed in pixels per meter8 +Horizontal grid resolution, expressed in pixels per meter7 vRescInPixelsPerInch -Vertical grid resolution, expressed in pixels per inch9 +Vertical grid resolution, expressed in pixels per inch8 hRescInPixelsPerInch -Horizontal grid resolution, expressed in pixels per inch10 +Horizontal grid resolution, expressed in pixels per inch9 @@ -1368,19 +1388,19 @@ vResdInPixelsPerMeter -Vertical grid resolution, expressed in pixels per meter11 +Vertical grid resolution, expressed in pixels per meter10 hResdInPixelsPerMeter -Horizontal grid resolution, expressed in pixels per meter12 +Horizontal grid resolution, expressed in pixels per meter11 vResdInPixelsPerInch -Vertical grid resolution, expressed in pixels per inch13 +Vertical grid resolution, expressed in pixels per inch12 hResdInPixelsPerInch -Horizontal grid resolution, expressed in pixels per inch14 +Horizontal grid resolution, expressed in pixels per inch13 @@ -1450,7 +1470,7 @@

    Note that jpylyzer does not check whether the XML is valid, as this is not required by the standard. Besides, doing so would make jpylyzer significantly slower for XML that contains references to external schemas and DTDs.

    6.17 UUID box

    -

    This (optional) box contains additional (binary) information, which may be vendor-specific. Some applications (e.g. Kakadu and ExifTool) also use this box for storing XMP metadata (see Section 1.1.4 in Part 3 of the XMP specification15).

    +

    This (optional) box contains additional (binary) information, which may be vendor-specific. Some applications (e.g. Kakadu and ExifTool) also use this box for storing XMP metadata (see Section 1.1.4 in Part 3 of the XMP specification14).

    6.17.1 Element name

    uuidBox

    6.17.2 Reported properties

    @@ -1870,7 +1890,7 @@

    These marker segments (which are marked with an asterisk) are only minimally supported at this stage: if jpylyzer encounters any of them, it will include the corresponding element in the properties element of the output. However, jpylyzer currently does not analyse their contents, and the respective elements in the output will be empty.

    7.2.3 Bit streams

    -

    In addition to the above limitations, jpylyzer can not be used to establish whether the data in the bitstream are correct (this would require decoding the compressed image data, which is completely out of jpylyzer’s scope)16. As a result, if jpylyzer is used as part of a quality assurance workflow, it is recommended to also include an additional check on the image contents17. Also, jpylyzer does not perform any checks on marker segments within the bit-stream: start-of packet (SOP) and end-of-packet (EPH) markers.

    +

    In addition to the above limitations, jpylyzer can not be used to establish whether the data in the bitstream are correct (this would require decoding the compressed image data, which is completely out of jpylyzer’s scope)15. As a result, if jpylyzer is used as part of a quality assurance workflow, it is recommended to also include an additional check on the image contents16. Also, jpylyzer does not perform any checks on marker segments within the bit-stream: start-of packet (SOP) and end-of-packet (EPH) markers.

    7.2.4 Detection of incomplete or truncated codestreams

    A JP2’s tile part header contains information that makes it possible to detect incomplete and truncated codestreams in most cases. Depending on the encoder software used, this method may fail for images that only contain one single tile part (i.e. images that do not contain tiling).

    7.2.5 Current limitations of comment extraction

    @@ -1943,7 +1963,7 @@ quantizationConsistentWithLevels -Values of quantization parameters from QCD marker segment are consistent with levels from COD marker segment18 +Values of quantization parameters from QCD marker segment are consistent with levels from COD marker segment17 foundExpectedNumberOfTiles @@ -2013,7 +2033,7 @@ numberOfTiles -Number of tiles19 +Number of tiles18 csiz @@ -2887,24 +2907,23 @@

    1. The jpylyzer binaries were created using the PyInstaller package: http://www.pyinstaller.org/↩

      2. -

    2. Note that jpylyzer will not work under Python versions 3.0-3.1!↩

      3. -

    3. Note that jpylyzer versions 1.8 and earlier returned a formatted XML string instead of an element object!↩

      4. -

    4. Note that jpylyzer versions 1.4 and earlier used the verbose output format by default. This behaviour has changed in version 1.5 onwards, as the lengthy output turned out to be slightly confusing to some users.↩

      5. -

    5. The “Any ICC” method is defined in ISO/IEC 15444-2 (the JPX format), and is not allowed in JP2. However, jpylyzer offers limited support for JPX here by also reporting the properties of ICC profiles that were embedded using this method. Note that any file that uses this method will fail the “methIsValid” test (and thereby the validation).↩

      6. -

    6. Originally ISO/IEC 15444-1 only allowed “input device” profiles. Support of “display device” profiles was added through an amendment to the standard in 2013. The behaviour of jpylyzer is consistent with this amendment.↩

      7. -

    7. Calculated as: vRcN vRcD • 10 vRcE ↩

      8. -

    8. Calculated as: hRcN hRcD • 10 hRcE ↩

      9. -

    9. Calculated as: vRescInPixelsPerMeter • 25.4 • 10 -3 ↩

      10. -

    10. Calculated as: hRescInPixelsPerMeter • 25.4 • 10 -3 ↩

      11. -

    11. Calculated as: vRdN vRdD • 10 vRdE ↩

      12. -

    12. Calculated as: hRdN hRdD • 10 hRdE ↩

      13. -

    13. Calculated as: vResdInPixelsPerMeter • 25.4 • 10 -3 ↩

      14. -

    14. Calculated as: hResdInPixelsPerMeter • 25.4 • 10 -3 ↩

      15. -

    15. Link: http://wwwimages.adobe.com/www.adobe.com/content/dam/Adobe/en/devnet/xmp/pdfs/cs6/XMPSpecificationPart3.pdf↩

      16. -

    16. However, support for start of packet (SOP) and end of packet (EPH) markers may be included in future versions.↩

      17. -

    17. For example, in a TIFF to JP2 conversion workflow one could include a pixel-by-pixel comparison of the values in the TIFF and the JP2.↩

      18. -

    18. The consistency check verifies if the length of the quantization default marker segment (lqcd from qcd) is consistent with the quantization style (qStyle from qcd) and the number of decomposition levels (levels from cod). They are consistent if the following equation is true:↩

      19. -

    19. Calculated as: numberOfTiles = [ xsiz - xOsiz xTsiz ] • [ ysiz - yOsiz yTsiz ] ↩

      20. +

    20. Note that jpylyzer versions 1.8 and earlier returned a formatted XML string instead of an element object!↩

      21. +

    21. Note that jpylyzer versions 1.4 and earlier used the verbose output format by default. This behaviour has changed in version 1.5 onwards, as the lengthy output turned out to be slightly confusing to some users.↩

      22. +

    22. The “Any ICC” method is defined in ISO/IEC 15444-2 (the JPX format), and is not allowed in JP2. However, jpylyzer offers limited support for JPX here by also reporting the properties of ICC profiles that were embedded using this method. Note that any file that uses this method will fail the “methIsValid” test (and thereby the validation).↩

      23. +

    23. Originally ISO/IEC 15444-1 only allowed “input device” profiles. Support of “display device” profiles was added through an amendment to the standard in 2013. The behaviour of jpylyzer is consistent with this amendment.↩

      24. +

    24. Calculated as: vRcN vRcD • 10 vRcE ↩

      25. +

    25. Calculated as: hRcN hRcD • 10 hRcE ↩

      26. +

    26. Calculated as: vRescInPixelsPerMeter • 25.4 • 10 -3 ↩

      27. +

    27. Calculated as: hRescInPixelsPerMeter • 25.4 • 10 -3 ↩

      28. +

    28. Calculated as: vRdN vRdD • 10 vRdE ↩

      29. +

    29. Calculated as: hRdN hRdD • 10 hRdE ↩

      30. +

    30. Calculated as: vResdInPixelsPerMeter • 25.4 • 10 -3 ↩

      31. +

    31. Calculated as: hResdInPixelsPerMeter • 25.4 • 10 -3 ↩

      32. +

    32. Link: http://wwwimages.adobe.com/www.adobe.com/content/dam/Adobe/en/devnet/xmp/pdfs/cs6/XMPSpecificationPart3.pdf↩

      33. +

    33. However, support for start of packet (SOP) and end of packet (EPH) markers may be included in future versions.↩

      34. +

    34. For example, in a TIFF to JP2 conversion workflow one could include a pixel-by-pixel comparison of the values in the TIFF and the JP2.↩

      35. +

    35. The consistency check verifies if the length of the quantization default marker segment (lqcd from qcd) is consistent with the quantization style (qStyle from qcd) and the number of decomposition levels (levels from cod). They are consistent if the following equation is true:↩

      36. +

    36. Calculated as: numberOfTiles = [ xsiz - xOsiz xTsiz ] • [ ysiz - yOsiz yTsiz ] ↩

    diff -Nru jpylyzer-1.17.0/doc/jpylyzerUserManual.md jpylyzer-1.18.0/doc/jpylyzerUserManual.md --- jpylyzer-1.17.0/doc/jpylyzerUserManual.md 2016-01-05 16:26:23.000000000 +0000 +++ jpylyzer-1.18.0/doc/jpylyzerUserManual.md 2017-08-23 11:42:58.000000000 +0000 @@ -141,10 +141,11 @@ You have three options: -1. Use the Python source code. This allows you to run the software as a -Python script on most popular platforms (Windows, Linux, Mac, etc.). -However, this requires that you have a recent version of the Python -interpreter available on your system. +1. Install the software with the *Pip* package manager. This works on +all platforms (Windows, Linux, Mac, etc.), but you need to have +the Python interpreter available on your system. Jpylyzer is compatible with +Python 2.7, and Python 3.2 and more recent (Python 3.0 and 3.1 are not +supported). 2. Alternatively, for Windows users there is also a set of stand-alone binaries[^1]. These allow you to run *jpylyzer* as an @@ -157,67 +158,68 @@ These options are described in the following sections. -Installation of Python script (Linux/Unix, Windows, Mac OS X) {#installation-python} +Installation with Pip (Linux/Unix, Windows, Mac OS X) {#installation-pip} ----------------------------------------------------------------- -First, download the source files using one of the ‘Source Code -Downloads’ links on the OPF *jpylyzer* page. +### General installation procedure -Then unzip the contents of the ZIP file to an empty directory. If you -are working on a Linux/Unix based system you may need to make the -scripts executable, and convert any line breaks to Unix-style ones. To -do this, use the following commands: +First make sure you have a recent version of *pip*. Then install *jpylyzer* +with the following command: - chmod 755 *.py - dos2unix *.py + pip install jpylyzer -In order to run the script you will need either Python 2.7, or Python -3.2 (or more recent)[^2]. Python can be downloaded from: +### Single user installation (Linux) -[http://python.org/](http://python.org/) +On most Linux systems the above command needs to be run as super user (see below). +If you don't want this use the below command for a single-user install: -### Testing the installation + pip install jpylyzer --user -To test your installation, open a console window (or command prompt) and -type: +This will install the software to the `.local` folder (hidden by default!) in your +home directory (`~/.local`). Next try to run *jpylyzer* by entering: - %jpylyzerPath%/jpylyzer.py -h + jpylyzer -In the above command, replace *%jpylyzerPath%* with the full path to the -*jpylyzer* installation directory (i.e. the directory that contains -‘jpylyzer.py’ and its associated files). For example, if you extracted -the files to directory ‘/home/jpylyzer’, the command would become: +Most likely this will result in: - /home/jpylyzer/jpylyzer.py -h + jpylyzer: command not found -Executing this command should result in the following screen output: +If this happens, add the directory `~/.local/bin` (which is where the jpylyzer command-line +tool is installed) to the `PATH` environment variable (you only need to do this once). +To do this, locate the (hidden) file `.profile` in you home directory (`~/`), and open it in +a text editor. Then add the following lines at the end of the file: - usage: jpylyzer.py [-h] [--verbose] [--recurse] [--wrapper] [--nullxml] - [--nopretty] [--version] jp2In [jp2In ...] + # set PATH so it includes the user's .local bin if it exists + if [ -d "$HOME/.local/bin" ] ; then + PATH="$HOME/.local/bin:$PATH" + fi + +Save the file, log out of your session and then log in again. Open a command terminal and type: + + jpylyzer + +If all went well you now see this: + usage: jpylyzer [-h] [--verbose] [--recurse] [--wrapper] [--nullxml] + [--nopretty] [--version] + jp2In [jp2In ...] + jpylyzer: error: the following arguments are required: jp2In -### Troubleshooting +Which means that the installation was successful! -If the above test didn’t run successfully, first verify the following -possible causes: +### Global installation (Linux) -* On Windows: check if files with a *.py* extension are associated with -the Python interpreter. If you have multiple versions of Python on your -system, make sure that the association does not link to a Python version -that is incompatible with *jpylyzer* (e.g. Python 2.6 or older, or -Python 3.0/3.1). +Simply enter: -* On Unix/Linux: by default, *jpylyzer* uses the command interpreter -that is defined by the ‘python’ environment variable. If this is linked -to some (very) old version of Python, things may not work as expected. -If you run into problems because of this, update the command interpreter -references in *jpylyzer.py*, i.e. change: + sudo -H pip install isolyzer -
    #! /usr/bin/env python
    +No further configuration is needed in this case. -into: +### Note on pre-releases -
    #! /usr/bin/env python27
    +The above command lines will only install stable versions of jpylyzer. In order to install the latest pre-release, add the `--pre` switch. For example: + + sudo -H pip install jpylyzer --pre Installation of Windows binaries (Windows only) {#installation-windows} @@ -232,20 +234,21 @@ To test your installation, open a Command Prompt (‘DOS prompt’) and type: - %jpylyzerPath%\jpylyzer -h + %jpylyzerPath%\jpylyzer In the above command, replace *%jpylyzerPath%* with the full path to the *jpylyzer* installation directory (i.e. the directory that contains ‘jpylyzer.exe’ and its associated files). For example, if you extracted the files to directory `c:\tools\jpylyzer`, the command would become: - c:\tools\jpylyzer\jpylyzer -h + c:\tools\jpylyzer\jpylyzer Executing this command should result in the following screen output: - usage: jpylyzer.py [-h] [--verbose] [--recurse] [--wrapper] [--nullxml] - [--nopretty] [--version] jp2In [jp2In ...] - + usage: jpylyzer [-h] [--verbose] [--recurse] [--wrapper] [--nullxml] + [--nopretty] [--version] + jp2In [jp2In ...] + jpylyzer: error: the following arguments are required: jp2In ### Running jpylyzer without typing the full path @@ -272,6 +275,12 @@ In both cases you need to have administrative privileges. +For *Ubuntu* and *Debian* alternative packages are available in the +official release channels. To install simply run the following commands: + + sudo apt-get update + sudo apt-get install python-jpylyzer + Using *jpylyzer* {#using-jpylyzer} ================== @@ -476,32 +485,32 @@ --------------------------------------- Instead of using *jpylyzer* from the command-line, you can also import -it as a module in your own Python programs. To do so, put all the -*jpylyzer* source files in the same directory as your own code. Then -import *jpylyzer* into your code by adding: +it as a module in your own Python programs. To do so, install jpylyzer +with *pip*. Then import *jpylyzer* into your code by adding: - import jpylyzer + from jpylyzer import jpylyzer Subsequently you can call any function that is defined in *jpylyzer.py*. -In practice you will most likely only need the *checkOneFile* function, -which can be called in the following way: - - jpylyzer.checkOneFile(file) +In practice you will most likely only need the *checkOneFile* function. +The following minimal script shows how this works: -Here, *file* is the path to a file object. The function returns an -element object that can either be used directly, or converted to XML -using the *ElementTree* module[^3]. The structure of the -element object follows the XML output that described in [Chapter 5](#output-format). + #! /usr/bin/env python -Alternatively, you may only want to import the *checkOneFile* function, -in which case the import statement becomes: + from jpylyzer import jpylyzer - from jpylyzer import checkOneFile + # Define JP2 + myFile = "/home/johan/jpylyzer-test-files/aware.jp2" -This will allow you to call the function as follows: + # Analyse with jpylyzer, result to Element object + myResult = jpylyzer.checkOneFile(myFile) - checkOneFile(file) + # Return image height value + imageHeight = myResult.findtext('./properties/jp2HeaderBox/imageHeaderBox/height') + print(imageHeight) +Here, *myResult* is an *Element* object that can either be used directly, +or converted to XML using the *ElementTree* module[^3]. The structure of the +element object follows the XML output that described in [Chapter 5](#output-format). Structure of a JP2 file {#structure-jp2} ========================= diff -Nru jpylyzer-1.17.0/doc/mdToDeliveryFormats.sh jpylyzer-1.18.0/doc/mdToDeliveryFormats.sh --- jpylyzer-1.17.0/doc/mdToDeliveryFormats.sh 2016-01-05 16:26:23.000000000 +0000 +++ jpylyzer-1.18.0/doc/mdToDeliveryFormats.sh 2017-08-23 11:42:58.000000000 +0000 @@ -24,7 +24,7 @@ # Self-contained HTML outHtmlSC=${baseName}.html -# Create file that replacess userManual page on website +# Create file that replaces userManual page on website # Step 1: convert to HTML pandoc -s --columns 1000 --toc --toc-depth=2 --ascii -N -c $styleSheet -w html5 -o tmp.html $mdSource diff -Nru jpylyzer-1.17.0/doc/userManual.html jpylyzer-1.18.0/doc/userManual.html --- jpylyzer-1.17.0/doc/userManual.html 2016-01-05 16:26:23.000000000 +0000 +++ jpylyzer-1.18.0/doc/userManual.html 2017-08-23 11:42:58.000000000 +0000 @@ -18,7 +18,7 @@
  • 2 Installation and set-up
    • @@ -131,51 +131,63 @@

    http://jpylyzer.openpreservation.org/

    You have three options:

      -

    1. Use the Python source code. This allows you to run the software as a Python script on most popular platforms (Windows, Linux, Mac, etc.). However, this requires that you have a recent version of the Python interpreter available on your system.

      2. +

    2. Install the software with the Pip package manager. This works on all platforms (Windows, Linux, Mac, etc.), but you need to have the Python interpreter available on your system. Jpylyzer is compatible with Python 2.7, and Python 3.2 and more recent (Python 3.0 and 3.1 are not supported).

    3. Alternatively, for Windows users there is also a set of stand-alone binaries1. These allow you to run jpylyzer as an executable Windows application, without any need for installing Python. This option is particularly useful for Windows users who cannot (or don’t want to) install software on their system.

    4. For Linux users Debian packages are available. These allow you to run jpylyzer without any need for installing Python.

    These options are described in the following sections.

    -

    2.2 Installation of Python script (Linux/Unix, Windows, Mac OS X)

    -

    First, download the source files using one of the ‘Source Code Downloads’ links on the OPF jpylyzer page.

    -

    Then unzip the contents of the ZIP file to an empty directory. If you are working on a Linux/Unix based system you may need to make the scripts executable, and convert any line breaks to Unix-style ones. To do this, use the following commands:

    -
    chmod 755 *.py
-dos2unix *.py
    -

    In order to run the script you will need either Python 2.7, or Python 3.2 (or more recent)2. Python can be downloaded from:

    -

    http://python.org/

    -

    2.2.1 Testing the installation

    -

    To test your installation, open a console window (or command prompt) and type:

    -
    %jpylyzerPath%/jpylyzer.py -h
    -

    In the above command, replace %jpylyzerPath% with the full path to the jpylyzer installation directory (i.e. the directory that contains ‘jpylyzer.py’ and its associated files). For example, if you extracted the files to directory ‘/home/jpylyzer’, the command would become:

    -
    /home/jpylyzer/jpylyzer.py -h
    -

    Executing this command should result in the following screen output:

    -
    usage: jpylyzer.py [-h] [--verbose] [--recurse] [--wrapper] [--nullxml]
-                   [--nopretty] [--version] jp2In [jp2In ...]
    -

    2.2.2 Troubleshooting

    -

    If the above test didn’t run successfully, first verify the following possible causes:

    - -
    #! /usr/bin/env python
    -

    into:

    -
    #! /usr/bin/env python27
    +

    2.2 Installation with Pip (Linux/Unix, Windows, Mac OS X)

    +

    2.2.1 General installation procedure

    +

    First make sure you have a recent version of pip. Then install jpylyzer with the following command:

    +
    pip install jpylyzer
    +

    2.2.2 Single user installation (Linux)

    +

    On most Linux systems the above command needs to be run as super user (see below). If you don't want this use the below command for a single-user install:

    +
    pip install jpylyzer --user
    +

    This will install the software to the .local folder (hidden by default!) in your home directory (~/.local). Next try to run jpylyzer by entering:

    +
    jpylyzer
    +

    Most likely this will result in:

    +
    jpylyzer: command not found
    +

    If this happens, add the directory ~/.local/bin (which is where the jpylyzer command-line tool is installed) to the PATH environment variable (you only need to do this once). To do this, locate the (hidden) file .profile in you home directory (~/), and open it in a text editor. Then add the following lines at the end of the file:

    +
    # set PATH so it includes the user's .local bin if it exists
+if [ -d "$HOME/.local/bin" ] ; then
+    PATH="$HOME/.local/bin:$PATH"
+fi
    +

    Save the file, log out of your session and then log in again. Open a command terminal and type:

    +
     jpylyzer
    +

    If all went well you now see this:

    +
    usage: jpylyzer [-h] [--verbose] [--recurse] [--wrapper] [--nullxml]
+                [--nopretty] [--version]
+                jp2In [jp2In ...]
+jpylyzer: error: the following arguments are required: jp2In
    +

    Which means that the installation was successful!

    +

    2.2.3 Global installation (Linux)

    +

    Simply enter:

    +
    sudo -H pip install isolyzer
    +

    No further configuration is needed in this case.

    +

    2.2.4 Note on pre-releases

    +

    The above command lines will only install stable versions of jpylyzer. In order to install the latest pre-release, add the --pre switch. For example:

    +
    sudo -H pip install jpylyzer --pre

    2.3 Installation of Windows binaries (Windows only)

    Download the binary using the link on the jpylyzer homepage. Unzip the contents of this file to an empty folder on your PC. Jpylyzer should now be ready for use.

    -

    2.3.1 Testing the installation

    +

    2.3.1 Testing the installation

    To test your installation, open a Command Prompt (‘DOS prompt’) and type:

    -
    %jpylyzerPath%\jpylyzer -h
    +
    %jpylyzerPath%\jpylyzer

    In the above command, replace %jpylyzerPath% with the full path to the jpylyzer installation directory (i.e. the directory that contains ‘jpylyzer.exe’ and its associated files). For example, if you extracted the files to directory c:\tools\jpylyzer, the command would become:

    -
    c:\tools\jpylyzer\jpylyzer -h
    +
    c:\tools\jpylyzer\jpylyzer

    Executing this command should result in the following screen output:

    -
    usage: jpylyzer.py [-h] [--verbose] [--recurse] [--wrapper] [--nullxml]
-                   [--nopretty] [--version] jp2In [jp2In ...]
    +
    usage: jpylyzer [-h] [--verbose] [--recurse] [--wrapper] [--nullxml]
+                [--nopretty] [--version]
+                jp2In [jp2In ...]
+jpylyzer: error: the following arguments are required: jp2In

    2.3.2 Running jpylyzer without typing the full path

    Optionally, you may also want to add the full path of the jpylyzer installation directory to the Windows ’Path’ environment variable. Doing so allows you to run jpylyzer from any directory on your PC without having to type the full path. In Windows 7 you can do this by selecting ‘settings’ from the ‘Start’ menu; then go to ‘control panel’/’system’ and go to the ‘advanced’ tab. Click on the ‘environment variables’ button. Finally, locate the ‘Path’ variable in the ‘system variables’ window, click on ‘Edit’ and add the full jpylyzer path (this requires local Administrator privileges). The settings take effect on any newly opened command prompt.

    2.4 Installation of Debian packages (Ubuntu/Linux)

    For a number of Linux architectures Debian packages of jpylyzer exist. To install, simply download the .deb file, double-click on it and select Install Package. Alternatively you can also do this in the command terminal by typing:

    sudo dpkg -i jpylyzer_1.13.0_i386.deb

    In both cases you need to have administrative privileges.

    +

    For Ubuntu and Debian alternative packages are available in the official release channels. To install simply run the following commands:

    +
    sudo apt-get update
+sudo apt-get install python-jpylyzer

    3 Using jpylyzer

    3.1 Overview

    This chapter describes the general use of jpylyzer. The first sections cover the use of jpylyzer as a command-line tool and as an importable Python module.

    @@ -281,15 +293,23 @@ 
    User warning: ignoring unknown box

    This happens if jpylyzer encounters a box that is not defined by JPEG 2000 Part 1. It should be noted that, to a large extent, JPEG 2000 Part 1 permits the presence of boxes that are defined outside the standard. Again, jpylyzer will simply ignore these and process all other boxes normally.

    3.3 Using jpylyzer as a Python module

    -

    Instead of using jpylyzer from the command-line, you can also import it as a module in your own Python programs. To do so, put all the jpylyzer source files in the same directory as your own code. Then import jpylyzer into your code by adding:

    -
    import jpylyzer
    -

    Subsequently you can call any function that is defined in jpylyzer.py. In practice you will most likely only need the checkOneFile function, which can be called in the following way:

    -
    jpylyzer.checkOneFile(file)
    -

    Here, file is the path to a file object. The function returns an element object that can either be used directly, or converted to XML using the ElementTree module3. The structure of the element object follows the XML output that described in Chapter 5.

    -

    Alternatively, you may only want to import the checkOneFile function, in which case the import statement becomes:

    -
    from jpylyzer import checkOneFile
    -

    This will allow you to call the function as follows:

    -
    checkOneFile(file)
    +

    Instead of using jpylyzer from the command-line, you can also import it as a module in your own Python programs. To do so, install jpylyzer with pip. Then import jpylyzer into your code by adding:

    +
    from jpylyzer import jpylyzer
    +

    Subsequently you can call any function that is defined in jpylyzer.py. In practice you will most likely only need the checkOneFile function. The following minimal script shows how this works:

    +
    #! /usr/bin/env python
+
+from jpylyzer import jpylyzer
+
+# Define JP2
+myFile = "/home/johan/jpylyzer-test-files/aware.jp2"
+
+# Analyse with jpylyzer, result to Element object
+myResult = jpylyzer.checkOneFile(myFile)
+
+# Return image height value
+imageHeight = myResult.findtext('./properties/jp2HeaderBox/imageHeaderBox/height')
+print(imageHeight)
    +

    Here, myResult is an Element object that can either be used directly, or converted to XML using the ElementTree module2. The structure of the element object follows the XML output that described in Chapter 5.

    4 Structure of a JP2 file

    4.1 Scope of this chapter

    This chapter gives a brief overview of the JP2 file format. A basic understanding of the general structure of JP2 is helpful for appreciating how jpylyzer performs its validation. It will also make it easier to understand jpylyzer‘s extracted properties, as these are reported as a hierarchical tree that corresponds to the internal structure of JP2.

    @@ -499,7 +519,7 @@

    If a file passed all tests, this is an indication that it is most likely valid JP2. In that case, the isValidJP2 element (section 5.4) has a value of “True” (and “False” in all other cases). These tests are all explained in chapters 6 and 7.

    5.6.1 Default and verbose reporting of test results

    -

    By default, jpylyzer only reports any tests that failed (i.e. returned “False”), including the corresponding part of the box structure. For a valid JP2 the tests element will be empty. If the --verbose flag is used, the results of all tests are included (including those that returned “True”)4.

    +

    By default, jpylyzer only reports any tests that failed (i.e. returned “False”), including the corresponding part of the box structure. For a valid JP2 the tests element will be empty. If the --verbose flag is used, the results of all tests are included (including those that returned “True”)3.

    5.7 properties element

    This element contains the extracted image properties, which are organised in a hierarchical tree that corresponds to JP2’s box structure. See chapters 6 and 7 for a description of the reported properties.

    6 JP2: box by box

    @@ -852,7 +872,7 @@ Enumerated colourspace (as descriptive text string) -icc (if meth equals “Restricted ICC” or “Any ICC”5) +icc (if meth equals “Restricted ICC” or “Any ICC”4) Properties of ICC profile as child element (see below) @@ -1000,7 +1020,7 @@ iccPermittedProfileClass (if meth equals “Restricted ICC”) -ICC profile class is “input device” or “display device”6 +ICC profile class is “input device” or “display device”5 iccNoLUTBasedProfile (if meth equals “Restricted ICC”) @@ -1265,19 +1285,19 @@ vRescInPixelsPerMeter -Vertical grid resolution, expressed in pixels per meter7 +Vertical grid resolution, expressed in pixels per meter6 hRescInPixelsPerMeter -Horizontal grid resolution, expressed in pixels per meter8 +Horizontal grid resolution, expressed in pixels per meter7 vRescInPixelsPerInch -Vertical grid resolution, expressed in pixels per inch9 +Vertical grid resolution, expressed in pixels per inch8 hRescInPixelsPerInch -Horizontal grid resolution, expressed in pixels per inch10 +Horizontal grid resolution, expressed in pixels per inch9 @@ -1360,19 +1380,19 @@ vResdInPixelsPerMeter -Vertical grid resolution, expressed in pixels per meter11 +Vertical grid resolution, expressed in pixels per meter10 hResdInPixelsPerMeter -Horizontal grid resolution, expressed in pixels per meter12 +Horizontal grid resolution, expressed in pixels per meter11 vResdInPixelsPerInch -Vertical grid resolution, expressed in pixels per inch13 +Vertical grid resolution, expressed in pixels per inch12 hResdInPixelsPerInch -Horizontal grid resolution, expressed in pixels per inch14 +Horizontal grid resolution, expressed in pixels per inch13 @@ -1442,7 +1462,7 @@

    Note that jpylyzer does not check whether the XML is valid, as this is not required by the standard. Besides, doing so would make jpylyzer significantly slower for XML that contains references to external schemas and DTDs.

    6.17 UUID box

    -

    This (optional) box contains additional (binary) information, which may be vendor-specific. Some applications (e.g. Kakadu and ExifTool) also use this box for storing XMP metadata (see Section 1.1.4 in Part 3 of the XMP specification15).

    +

    This (optional) box contains additional (binary) information, which may be vendor-specific. Some applications (e.g. Kakadu and ExifTool) also use this box for storing XMP metadata (see Section 1.1.4 in Part 3 of the XMP specification14).

    6.17.1 Element name

    uuidBox

    6.17.2 Reported properties

    @@ -1862,7 +1882,7 @@

    These marker segments (which are marked with an asterisk) are only minimally supported at this stage: if jpylyzer encounters any of them, it will include the corresponding element in the properties element of the output. However, jpylyzer currently does not analyse their contents, and the respective elements in the output will be empty.

    7.2.3 Bit streams

    -

    In addition to the above limitations, jpylyzer can not be used to establish whether the data in the bitstream are correct (this would require decoding the compressed image data, which is completely out of jpylyzer’s scope)16. As a result, if jpylyzer is used as part of a quality assurance workflow, it is recommended to also include an additional check on the image contents17. Also, jpylyzer does not perform any checks on marker segments within the bit-stream: start-of packet (SOP) and end-of-packet (EPH) markers.

    +

    In addition to the above limitations, jpylyzer can not be used to establish whether the data in the bitstream are correct (this would require decoding the compressed image data, which is completely out of jpylyzer’s scope)15. As a result, if jpylyzer is used as part of a quality assurance workflow, it is recommended to also include an additional check on the image contents16. Also, jpylyzer does not perform any checks on marker segments within the bit-stream: start-of packet (SOP) and end-of-packet (EPH) markers.

    7.2.4 Detection of incomplete or truncated codestreams

    A JP2’s tile part header contains information that makes it possible to detect incomplete and truncated codestreams in most cases. Depending on the encoder software used, this method may fail for images that only contain one single tile part (i.e. images that do not contain tiling).

    7.2.5 Current limitations of comment extraction

    @@ -1935,7 +1955,7 @@ quantizationConsistentWithLevels -Values of quantization parameters from QCD marker segment are consistent with levels from COD marker segment18 +Values of quantization parameters from QCD marker segment are consistent with levels from COD marker segment17 foundExpectedNumberOfTiles @@ -2005,7 +2025,7 @@ numberOfTiles -Number of tiles19 +Number of tiles18 csiz @@ -2879,23 +2899,22 @@

    1. The jpylyzer binaries were created using the PyInstaller package: http://www.pyinstaller.org/↩

      2. -

    2. Note that jpylyzer will not work under Python versions 3.0-3.1!↩

      3. -

    3. Note that jpylyzer versions 1.8 and earlier returned a formatted XML string instead of an element object!↩

      4. -

    4. Note that jpylyzer versions 1.4 and earlier used the verbose output format by default. This behaviour has changed in version 1.5 onwards, as the lengthy output turned out to be slightly confusing to some users.↩

      5. -

    5. The “Any ICC” method is defined in ISO/IEC 15444-2 (the JPX format), and is not allowed in JP2. However, jpylyzer offers limited support for JPX here by also reporting the properties of ICC profiles that were embedded using this method. Note that any file that uses this method will fail the “methIsValid” test (and thereby the validation).↩

      6. -

    6. Originally ISO/IEC 15444-1 only allowed “input device” profiles. Support of “display device” profiles was added through an amendment to the standard in 2013. The behaviour of jpylyzer is consistent with this amendment.↩

      7. -

    7. Calculated as: vRcN vRcD • 10 vRcE ↩

      8. -

    8. Calculated as: hRcN hRcD • 10 hRcE ↩

      9. -

    9. Calculated as: vRescInPixelsPerMeter • 25.4 • 10 -3 ↩

      10. -

    10. Calculated as: hRescInPixelsPerMeter • 25.4 • 10 -3 ↩

      11. -

    11. Calculated as: vRdN vRdD • 10 vRdE ↩

      12. -

    12. Calculated as: hRdN hRdD • 10 hRdE ↩

      13. -

    13. Calculated as: vResdInPixelsPerMeter • 25.4 • 10 -3 ↩

      14. -

    14. Calculated as: hResdInPixelsPerMeter • 25.4 • 10 -3 ↩

      15. -

    15. Link: http://wwwimages.adobe.com/www.adobe.com/content/dam/Adobe/en/devnet/xmp/pdfs/cs6/XMPSpecificationPart3.pdf↩

      16. -

    16. However, support for start of packet (SOP) and end of packet (EPH) markers may be included in future versions.↩

      17. -

    17. For example, in a TIFF to JP2 conversion workflow one could include a pixel-by-pixel comparison of the values in the TIFF and the JP2.↩

      18. -

    18. The consistency check verifies if the length of the quantization default marker segment (lqcd from qcd) is consistent with the quantization style (qStyle from qcd) and the number of decomposition levels (levels from cod). They are consistent if the following equation is true:↩

      19. -

    19. Calculated as: numberOfTiles = [ xsiz - xOsiz xTsiz ] • [ ysiz - yOsiz yTsiz ] ↩

      20. +

    20. Note that jpylyzer versions 1.8 and earlier returned a formatted XML string instead of an element object!↩

      21. +

    21. Note that jpylyzer versions 1.4 and earlier used the verbose output format by default. This behaviour has changed in version 1.5 onwards, as the lengthy output turned out to be slightly confusing to some users.↩

      22. +

    22. The “Any ICC” method is defined in ISO/IEC 15444-2 (the JPX format), and is not allowed in JP2. However, jpylyzer offers limited support for JPX here by also reporting the properties of ICC profiles that were embedded using this method. Note that any file that uses this method will fail the “methIsValid” test (and thereby the validation).↩

      23. +

    23. Originally ISO/IEC 15444-1 only allowed “input device” profiles. Support of “display device” profiles was added through an amendment to the standard in 2013. The behaviour of jpylyzer is consistent with this amendment.↩

      24. +

    24. Calculated as: vRcN vRcD • 10 vRcE ↩

      25. +

    25. Calculated as: hRcN hRcD • 10 hRcE ↩

      26. +

    26. Calculated as: vRescInPixelsPerMeter • 25.4 • 10 -3 ↩

      27. +

    27. Calculated as: hRescInPixelsPerMeter • 25.4 • 10 -3 ↩

      28. +

    28. Calculated as: vRdN vRdD • 10 vRdE ↩

      29. +

    29. Calculated as: hRdN hRdD • 10 hRdE ↩

      30. +

    30. Calculated as: vResdInPixelsPerMeter • 25.4 • 10 -3 ↩

      31. +

    31. Calculated as: hResdInPixelsPerMeter • 25.4 • 10 -3 ↩

      32. +

    32. Link: http://wwwimages.adobe.com/www.adobe.com/content/dam/Adobe/en/devnet/xmp/pdfs/cs6/XMPSpecificationPart3.pdf↩

      33. +

    33. However, support for start of packet (SOP) and end of packet (EPH) markers may be included in future versions.↩

      34. +

    34. For example, in a TIFF to JP2 conversion workflow one could include a pixel-by-pixel comparison of the values in the TIFF and the JP2.↩

      35. +

    35. The consistency check verifies if the length of the quantization default marker segment (lqcd from qcd) is consistent with the quantization style (qStyle from qcd) and the number of decomposition levels (levels from cod). They are consistent if the following equation is true:↩

      36. +

    36. Calculated as: numberOfTiles = [ xsiz - xOsiz xTsiz ] • [ ysiz - yOsiz yTsiz ] ↩

    diff -Nru jpylyzer-1.17.0/.gitignore jpylyzer-1.18.0/.gitignore --- jpylyzer-1.17.0/.gitignore 2016-01-05 16:26:23.000000000 +0000 +++ jpylyzer-1.18.0/.gitignore 2017-08-23 11:42:58.000000000 +0000 @@ -48,7 +48,7 @@ # PyInstaller bits # #################### -*.spec +# *.spec # Vagrant bits # #################### @@ -70,6 +70,7 @@ *.dsc *.deb *.changes +*.egg-info # Logs and databases # ###################### diff -Nru jpylyzer-1.17.0/jpylyzer/boxvalidator.py jpylyzer-1.18.0/jpylyzer/boxvalidator.py --- jpylyzer-1.17.0/jpylyzer/boxvalidator.py 2016-01-05 16:26:23.000000000 +0000 +++ jpylyzer-1.18.0/jpylyzer/boxvalidator.py 2017-08-23 11:42:58.000000000 +0000 @@ -1,3 +1,4 @@ +"""Validator class for all boxes in JP2""" # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU Lesser General Public License as published by @@ -16,16 +17,15 @@ from __future__ import division import uuid import math -import config -import etpatch as ET -import byteconv as bc -from shared import listOccurrencesAreContiguous -from shared import printWarning +from . import config as config +from . import etpatch as ET +from . import byteconv as bc +from . import shared as shared class BoxValidator: - # Marker tags/codes that identify all sub-boxes as hexadecimal strings - #(Correspond to "Box Type" values, see ISO/IEC 15444-1 Section I.4) + """Marker tags/codes that identify all sub-boxes as hexadecimal strings + (Correspond to "Box Type" values, see ISO/IEC 15444-1 Section I.4)""" typeMap = { b'\x6a\x70\x32\x69': "intellectualPropertyBox", b'\x78\x6d\x6c\x20': "xmlBox", @@ -89,10 +89,11 @@ self.bTypeString = bType def validate(self): + """Generic box validation function""" try: to_call = getattr(self, "validate_" + self.boxType) except AttributeError: - printWarning( + shared.printWarning( "ignoring '" + self.boxType + "' (validator function not yet implemented)") else: @@ -107,15 +108,15 @@ def _isValid(self): for elt in self.tests.iter(): - if elt.text == False: + if elt.text is False: # File didn't pass this test, so not valid - return(False) - return(True) + return False + return True def _getBox(self, byteStart, noBytes): - # Parse JP2 box and return information on its - # size, type and contents - + """Parse JP2 box and return information on its + size, type and contents + """ # Box length (4 byte unsigned integer) boxLengthValue = bc.bytesToUInt( self.boxContents[byteStart:byteStart + 4]) @@ -149,8 +150,9 @@ return (boxLengthValue, boxType, byteEnd, boxContents) def _getMarkerSegment(self, offset): - # Read marker segment that starts at offset and return marker, size, - # contents and start offset of next marker + """Read marker segment that starts at offset and return marker, size, + contents and start offset of next marker + """ # First 2 bytes: 16 bit marker marker = self.boxContents[offset:offset + 2] @@ -180,10 +182,11 @@ return(marker, length, contents, offsetNext) def _calculateCompressionRatio(self, noBytes, bPCDepthValues, height, width): - # Computes compression ratio - # noBytes: size of compressed image in bytes - # bPCDepthValues: list with bits per component for each component - # height, width: image height, width + """Computes compression ratio + - noBytes: size of compressed image in bytes + - bPCDepthValues: list with bits per component for each component + - height, width: image height, width + """ # Total bits per pixel bitsPerPixel = 0 @@ -204,12 +207,13 @@ # Obviously something going wrong here ... compressionRatio = -9999 - return(compressionRatio) + return compressionRatio def _getBitValue(self, n, p): - # Get the bit value of denary (base 10) number n at the equivalent binary - # position p (binary count starts at position 1 from the left) - # Only works if n can be expressed as 8 bits !!! + """Get the bit value of denary (base 10) number n at the equivalent binary + position p (binary count starts at position 1 from the left) + Only works if n can be expressed as 8 bits !!! + """ # Word length in bits wordLength = 8 @@ -220,13 +224,11 @@ return (n >> shift) & 1 def testFor(self, testType, testResult): - # Add testResult node to tests element tree - - # print(config.outputVerboseFlag) + """Add testResult node to tests element tree""" - if config.outputVerboseFlag == False: + if config.outputVerboseFlag is False: # Non-verbose output: only add results of tests that failed - if testResult == False: + if testResult is False: self.tests.appendChildTagWithText(testType, testResult) else: @@ -234,39 +236,27 @@ self.tests.appendChildTagWithText(testType, testResult) def addCharacteristic(self, characteristic, charValue): - # Add characteristic node to characteristics element tree + """Add characteristic node to characteristics element tree""" self.characteristics.appendChildTagWithText(characteristic, charValue) # Validator functions for boxes def validate_unknownBox(self): - - # Although jpylyzer doesn't "know" anything about this box, we - # can at least report the 4 characters from the Box Type field - # (TBox) here + """Process 'unknown'box. Although jpylyzer doesn't know anything about this box, we + can at least report the 4 characters from the Box Type field (TBox) here + """ boxType = self.bTypeString - """" - # If boxType contains any device control characters (e.g. because of - # file corruption), replace them with printable character - if bc.containsControlCharacters(boxType): - boxType=bc.replaceControlCharacters(boxType) - - # Decode to string with Latin encoding - # Elementtree will deal with any non-ASCII characters by replacing - # them with numeric entity references - boxType=boxType.decode("iso-8859-15","strict") - """ - # Add (cleaned up) boxType string to output + # Add boxType string to output self.addCharacteristic("boxType", boxType) # Print warning message to screen - printWarning("ignoring unknown box") + shared.printWarning("ignoring unknown box") def validate_signatureBox(self): - # Signature box (ISO/IEC 15444-1 Section I.5.2) + """Signature box (ISO/IEC 15444-1 Section I.5.2)""" # Check box size, which should be 4 bytes self.testFor("boxLengthIsValid", len(self.boxContents) == 4) @@ -277,7 +267,7 @@ "signatureIsValid", self.boxContents[0:4] == b'\x0d\x0a\x87\x0a') def validate_fileTypeBox(self): - # File type box (ISO/IEC 15444-1 Section I.5.2) + """File type box (ISO/IEC 15444-1 Section I.5.2)""" # Determine number of compatibility fields from box length numberOfCompatibilityFields = (len(self.boxContents) - 8) / 4 @@ -319,13 +309,12 @@ self.testFor("compatibilityListIsValid", b'\x6a\x70\x32\x20' in cLList) def validate_jp2HeaderBox(self): - # JP2 header box (superbox) (ISO/IEC 15444-1 Section I.5.3) + """JP2 header box (superbox) (ISO/IEC 15444-1 Section I.5.3)""" # List for storing box type identifiers subBoxTypes = [] noBytes = len(self.boxContents) byteStart = 0 - bytesTotal = 0 # Dummy value boxLengthValue = 10 @@ -353,7 +342,7 @@ self.testFor( "containsImageHeaderBox", self.boxTagMap['imageHeaderBox'] in subBoxTypes) self.testFor("containsColourSpecificationBox", self.boxTagMap[ - 'colourSpecificationBox'] in subBoxTypes) + 'colourSpecificationBox'] in subBoxTypes) # If bPCSign equals 1 and bPCDepth equals 128 (equivalent to bPC field being # 255), this box should contain a Bits Per Components box @@ -362,13 +351,13 @@ if sign == 1 and depth == 128: self.testFor("containsBitsPerComponentBox", self.boxTagMap[ - 'bitsPerComponentBox'] in subBoxTypes) + 'bitsPerComponentBox'] in subBoxTypes) # Is the first box an Image Header Box? try: firstJP2HeaderBoxIsImageHeaderBox = subBoxTypes[ 0] == self.boxTagMap['imageHeaderBox'] - except: + except Exception: firstJP2HeaderBoxIsImageHeaderBox = False self.testFor( @@ -376,7 +365,7 @@ # Some boxes can have multiple instances, whereas for others only one # is allowed - self.testFor("noMoreThanOneImageHeaderBox", subBoxTypes.count( + self.testFor("noMoreThanOneImageHeaderBox", subBoxTypes.count( self.boxTagMap['imageHeaderBox']) <= 1) self.testFor("noMoreThanOneBitsPerComponentBox", subBoxTypes.count( self.boxTagMap['bitsPerComponentBox']) <= 1) @@ -391,15 +380,16 @@ # In case of multiple colour specification boxes, they should appear contiguously # within the header box - colourSpecificationBoxesAreContiguous = listOccurrencesAreContiguous( + colourSpecificationBoxesAreContiguous = shared.listOccurrencesAreContiguous( subBoxTypes, self.boxTagMap['colourSpecificationBox']) self.testFor("colourSpecificationBoxesAreContiguous", colourSpecificationBoxesAreContiguous) # If JP2 Header box contains a Palette Box, it should also contain a component # mapping box, and vice versa - if (self.boxTagMap['paletteBox'] in subBoxTypes and self.boxTagMap['componentMappingBox'] not in subBoxTypes) \ - or (self.boxTagMap['componentMappingBox'] in subBoxTypes and self.boxTagMap['paletteBox'] not in subBoxTypes): + if ((self.boxTagMap['paletteBox'] in subBoxTypes and self.boxTagMap['componentMappingBox'] + not in subBoxTypes) or (self.boxTagMap['componentMappingBox'] in subBoxTypes and + self.boxTagMap['paletteBox'] not in subBoxTypes)): paletteAndComponentMappingBoxesOnlyTogether = False else: paletteAndComponentMappingBoxesOnlyTogether = True @@ -409,8 +399,9 @@ # Validator functions for boxes in JP2 Header superbox def validate_imageHeaderBox(self): - # Image header box (ISO/IEC 15444-1 Section I.5.3.1) - # This is a fixed-length box that contains generic image info. + """Image header box (ISO/IEC 15444-1 Section I.5.3.1) + This is a fixed-length box that contains generic image info. + """ # Check box length (14 bytes, excluding box length/type fields) self.testFor("boxLengthIsValid", len(self.boxContents) == 14) @@ -451,7 +442,7 @@ bPCDepthIsWithinAllowedRange = 1 <= bPCDepth <= 38 bitDepthIsVariable = 1 <= bPC <= 255 - if bPCDepthIsWithinAllowedRange == True or bitDepthIsVariable == True: + if bPCDepthIsWithinAllowedRange or bitDepthIsVariable: bPCIsValid = True else: bPCIsValid = False @@ -480,8 +471,8 @@ self.testFor("iPRIsValid", 0 <= iPR <= 1) def validate_bitsPerComponentBox(self): - # bits per component box (ISO/IEC 15444-1 Section I.5.3.2) - # Optional box that specifies bit depth of each component + """bits per component box (ISO/IEC 15444-1 Section I.5.3.2) + Optional box that specifies bit depth of each component""" # Number of bPC field (each field is 1 byte) numberOfBPFields = len(self.boxContents) @@ -508,9 +499,9 @@ self.testFor("bPCIsValid", 1 <= bPCDepth <= 38) def validate_colourSpecificationBox(self): - # Colour specification box (ISO/IEC 15444-1 Section I.5.3.3) - # This box defines one method for interpreting colourspace of decompressed - # image data + """Colour specification box (ISO/IEC 15444-1 Section I.5.3.3) + This box defines one method for interpreting colourspace of decompressed + image data""" # Length of this box length = len(self.boxContents) @@ -596,11 +587,12 @@ self.characteristics.append(iccCharacteristics) def validate_icc(self): - # Extracts characteristics (property-value pairs) of ICC profile - # Note that although values are stored in 'text' property of sub-elements, - # they may have a type other than 'text' (binary string, integers, lists) - # This means that some post-processing (conversion to text) is needed to - # write these property-value pairs to XML + """Extracts characteristics (property-value pairs) of ICC profile + Note that although values are stored in 'text' property of sub-elements, + they may have a type other than 'text' (binary string, integers, lists) + This means that some post-processing (conversion to text) is needed to + write these property-value pairs to XML + """ # Profile header properties (note: incomplete at this stage!) @@ -791,13 +783,14 @@ # Description as binary string (excluding terminating null char) description = descTag[12:12 + descriptionLength - 1] - except: + except Exception: description = "" self.addCharacteristic("description", description) def validate_paletteBox(self): - # Palette box (ISO/IEC 15444-1 Section I.5.3.4) - # Optional box that specifies a palette + """Palette box (ISO/IEC 15444-1 Section I.5.3.4) + Optional box that specifies a palette + """ # Number of entries in the table (each field is 2 bytes) nE = bc.bytesToUShortInt(self.boxContents[0:2]) @@ -855,9 +848,10 @@ offset += bytesPadded def validate_componentMappingBox(self): - # Component mapping box (ISO/IEC 15444-1 Section I.5.3.5) - # This box defines how image channels are identified from actual - # components + """Component mapping box (ISO/IEC 15444-1 Section I.5.3.5) + This box defines how image channels are identified from actual + components + """ # Determine number of channels from box length numberOfChannels = int(len(self.boxContents) / 4) @@ -898,9 +892,10 @@ offset += 4 def validate_channelDefinitionBox(self): - # Channel definition box (ISO/IEC 15444-1 Section I.5.3.6) - # This box specifies the meaning of the samples in each channel in the - # image + """Channel definition box (ISO/IEC 15444-1 Section I.5.3.6) + This box specifies the meaning of the samples in each channel in the + image + """ # Number of channel descriptions (short integer) n = bc.bytesToUShortInt(self.boxContents[0:2]) @@ -942,9 +937,10 @@ offset += 6 def validate_resolutionBox(self): - # Resolution box (superbox)(ISO/IEC 15444-1 Section I.5.3.7 - # Specifies the capture and/or default display grid resolutions of - # the image. + """Resolution box (superbox)(ISO/IEC 15444-1 Section I.5.3.7 + Specifies the capture and/or default display grid resolutions of + the image. + """ # Marker tags/codes that identify all sub-boxes as hexadecimal strings tagCaptureResolutionBox = b'\x72\x65\x73\x63' @@ -955,7 +951,6 @@ noBytes = len(self.boxContents) byteStart = 0 - bytesTotal = 0 # Dummy value boxLengthValue = 10 @@ -983,7 +978,8 @@ # This box contains either one Capture Resolution box, one Default Display # resolution box, or one of both self.testFor("containsCaptureOrDisplayResolutionBox", - tagCaptureResolutionBox in subBoxTypes or tagDisplayResolutionBox in subBoxTypes) + tagCaptureResolutionBox in subBoxTypes or + tagDisplayResolutionBox in subBoxTypes) self.testFor("noMoreThanOneCaptureResolutionBox", subBoxTypes.count(tagCaptureResolutionBox) <= 1) self.testFor("noMoreThanOneDisplayResolutionBox", @@ -992,7 +988,7 @@ # Validator functions for boxes in Resolution box def validate_captureResolutionBox(self): - # Capture Resolution Box (ISO/IEC 15444-1 Section I.5.3.7.1) + """Capture Resolution Box (ISO/IEC 15444-1 Section I.5.3.7.1)""" # Check box size, which should be 10 bytes self.testFor("boxLengthIsValid", len(self.boxContents) == 10) @@ -1052,7 +1048,7 @@ "hRescInPixelsPerInch", round(hRescInPixelsPerInch, 2)) def validate_displayResolutionBox(self): - # Default Display Resolution Box (ISO/IEC 15444-1 Section I.5.3.7.2) + """Default Display Resolution Box (ISO/IEC 15444-1 Section I.5.3.7.2)""" # Check box size, which should be 10 bytes self.testFor("boxLengthIsValid", len(self.boxContents) == 10) @@ -1112,7 +1108,7 @@ "hResdInPixelsPerInch", round(hResdInPixelsPerInch, 2)) def validate_contiguousCodestreamBox(self): - # Contiguous codestream box (ISO/IEC 15444-1 Section I.5.4) + """Contiguous codestream box (ISO/IEC 15444-1 Section I.5.4)""" # Codestream length length = len(self.boxContents) @@ -1264,9 +1260,9 @@ if not numberOfTilesExpected: numberOfTilesExpected = 0 - # Impose upper limit on numberOfTilesExpected to avoid misbehaviour in case of corrupted files - # Value of 65535 equals upper value imposed by Kakadu (can't find this - # anywhere in the standard though) + # Impose upper limit on numberOfTilesExpected to avoid misbehaviour in case of corrupted + # files. Value of 65535 equals upper value imposed by Kakadu (can't find this anywhere + # in the standard though) numberOfTilesExpected = min(numberOfTilesExpected, 65535) # Create list with one entry for each tile @@ -1303,8 +1299,6 @@ tilePartCharacteristics.append(characteristicsTilePart) tileIndex = characteristicsTilePart.findElementText('sot/isot') - tilePartIndex = characteristicsTilePart.findElementText( - 'sor/tpsot') tilePartsOfTile = characteristicsTilePart.findElementText( 'sot/tnsot') @@ -1347,8 +1341,9 @@ # Validator functions for codestream elements def validate_siz(self): - # Image and tile size (SIZ) header fields (ISO/IEC 15444-1 Section - # A.5.1) + """Image and tile size (SIZ) header fields (ISO/IEC 15444-1 Section + A.5.1) + """ # Length of main image header lsiz = bc.bytesToUShortInt(self.boxContents[0:2]) @@ -1489,8 +1484,9 @@ offset += 3 def validate_cod(self): - # Coding style default (COD) header fields (ISO/IEC 15444-1 Section - # A.6.1) + """Coding style default (COD) header fields (ISO/IEC 15444-1 Section + A.6.1) + """ # Length of COD marker lcod = bc.bytesToUShortInt(self.boxContents[0:2]) @@ -1681,8 +1677,9 @@ offset += 1 def validate_qcd(self): - # Quantization default (QCD) header fields (ISO/IEC 15444-1 Section - # A.6.4) + """Quantization default (QCD) header fields (ISO/IEC 15444-1 Section + A.6.4) + """ # Length of QCD marker lqcd = bc.bytesToUShortInt(self.boxContents[0:2]) @@ -1760,7 +1757,7 @@ # of corresponding equations (need Annex E from standard for that) def validate_com(self): - # Codestream comment (COM) (ISO/IEC 15444-1 Section A.9.2) + """Codestream comment (COM) (ISO/IEC 15444-1 Section A.9.2)""" # Length of COM marker lcom = bc.bytesToUShortInt(self.boxContents[0:2]) @@ -1782,33 +1779,41 @@ # Contents (multiples of Ccom) comment = self.boxContents[4:lcom] - # Decode to string with Latin encoding, determine if valid ISO 8859-15 - - try: - comment = comment.decode("iso-8859-15", "strict") - except: - # Empty string in case of decode error - comment = "" - - # Ideally decode above should raise exception if comment is not valid - # ISO 8859-15, but this doesn't work. So instead we do this indirectly - # by looking for control characters (tab, newline and carriage return - # are OK) - if bc.removeControlCharacters(comment) == comment: + if rcom == 0: + # no validation of binary comment content commentIsValid = True - else: - commentIsValid = False + comment = bc.bytesToHex(comment) + + elif rcom == 1: + + # Decode to string with Latin encoding, determine if valid ISO + # 8859-15 + + try: + comment = comment.decode("iso-8859-15", "strict") + except UnicodeError: + # Empty string in case of decode error + comment = "" + + # Ideally decode above should raise exception if comment is not + # valid ISO 8859-15, but this doesn't work. So instead we do this + # indirectly by looking for control characters (tab, newline and + # carriage return are OK) + if bc.removeControlCharacters(comment) == comment: + commentIsValid = True + else: + commentIsValid = False self.testFor("commentIsValid", commentIsValid) - # Only add comment to characteristics if text (may contain binary data - # if rcom is 0!) - if rcom == 1: + # any non-printable data should have been removed. + if commentIsValid: self.addCharacteristic("comment", comment) def validate_sot(self): - # Start of tile-part (SOT) marker segment (ISO/IEC 15444-1 Section - # A.4.2) + """Start of tile-part (SOT) marker segment (ISO/IEC 15444-1 Section + A.4.2) + """ # Note that unlike other marker validation functions this one returns a # third result, which is the total tile-part length (psot)! @@ -1835,7 +1840,7 @@ # psot equals 0 (for last tile part) or greater than 14 (so range 1-13 # is illegal) - psotIsValid = not(1 <= psot <= 13) + psotIsValid = not 1 <= psot <= 13 self.testFor("psotIsValid", psotIsValid) # Tile part index @@ -1861,52 +1866,55 @@ # because there will be either lots of them or none at all!). def validate_coc(self): - # Empty function + """Empty function""" pass def validate_rgn(self): - # Empty function + """Empty function""" pass def validate_qcc(self): - # Empty function + """Empty function""" pass def validate_poc(self): - # Empty function + """Empty function""" pass def validate_tlm(self): - # Empty function + """Empty function""" pass def validate_plm(self): - # Empty function + """Empty function""" pass def validate_plt(self): - # Empty function + """Empty function""" pass def validate_ppm(self): - # Empty function + """Empty function""" pass def validate_ppt(self): - # Empty function + """Empty function""" pass def validate_crg(self): - # Empty function + """Empty function""" pass def validate_tilePart(self): - # Analyse tile part that starts at offsetStart and perform cursory validation - # Precondition: offsetStart points to SOT marker - # - # Limitations: - # - COD, COC, QCD, QCC and RGN are markers only allowed in first tile-part - # of a tile; there is currently no check on this (may be added later) + """Analyse tile part that starts at offsetStart and perform cursory validation + + Precondition: offsetStart points to SOT marker + + Limitations: + + - COD, COC, QCD, QCC and RGN are markers only allowed in first tile-part + of a tile; there is currently no check on this (may be added later) + """ offset = self.startOffset @@ -1929,17 +1937,14 @@ offset = offsetNext - # Last marker in every tile-part should be a start of data marker - foundSODMarker = False - # Loop through remaining tile part marker segments; extract properties of # and validate COD, QCD and COM marker segments. Also test for presence of # SOD marker # NOTE 1: limited testing so far because of unavailability of test images with these # markers at tile-part level!! - # NOTE 2: check for offsetNext !=-9999 was included after encountering image with corruption - # that resulted in nonsensical lsot values, ultimatelty leading to an infinite loop. Shouldn't happen - # anymore (although this may not be the most elegant way of handling + # NOTE 2: check for offsetNext !=-9999 was included after encountering image with + # corruption that resulted in nonsensical lsot values, ultimatelty leading to an infinite + # loop. Shouldn't happen anymore (although this may not be the most elegant way of handling # this) while marker != b'\xff\x93' and offsetNext != -9999: @@ -2035,7 +2040,7 @@ self.returnOffset = offsetNextTilePart def validate_xmlBox(self): - # XML Box (ISO/IEC 15444-1 Section I.7.1) + """XML Box (ISO/IEC 15444-1 Section I.7.1)""" data = self.boxContents @@ -2050,30 +2055,31 @@ # If no exception was raised data contains well-formed XML containsWellformedXML = True - except: + except Exception: # If parse raised error this is not well-formed XML containsWellformedXML = False # Useful for extracting null-terminated XML (older Kakadu versions) - if config.extractNullTerminatedXMLFlag == True: + if config.extractNullTerminatedXMLFlag: try: data = bc.removeNullTerminator(data) dataAsElement = ET.fromstring(data) self.characteristics.append(dataAsElement) - except: + except Exception: pass self.testFor("containsWellformedXML", containsWellformedXML) def validate_uuidBox(self): - # UUID Box (ISO/IEC 15444-1 Section I.7.2) - # For details on UUIDs see: http://tools.ietf.org/html/rfc4122.html + """UUID Box (ISO/IEC 15444-1 Section I.7.2) + For details on UUIDs see: http://tools.ietf.org/html/rfc4122.html - # Box contains 16-byte identifier, followed by block of data. - # Format of data is defined outside of the scope of JPEG 2000, - # so in most cases there's not much to validate here. Exception: - # if uuid = be7acfcb-97a9-42e8-9c71-999491e3afac this indicates - # presence of XMP metadata + Box contains 16-byte identifier, followed by block of data. + Format of data is defined outside of the scope of JPEG 2000, + so in most cases there's not much to validate here. Exception: + if uuid = be7acfcb-97a9-42e8-9c71-999491e3afac this indicates + presence of XMP metadata + """ boxLength = len(self.boxContents) @@ -2082,9 +2088,9 @@ # First 16 bytes contain UUID, convert to string of hex digits # in standard form - id = str(uuid.UUID(bytes=self.boxContents[0:16])) + boxUUID = str(uuid.UUID(bytes=self.boxContents[0:16])) - if id == "be7acfcb-97a9-42e8-9c71-999491e3afac": + if boxUUID == "be7acfcb-97a9-42e8-9c71-999491e3afac": # XMP packet data = self.boxContents[16:boxLength] @@ -2105,7 +2111,7 @@ # Useful for extracting null-terminated XML (older Kakadu # versions) - if config.extractNullTerminatedXMLFlag == True: + if config.extractNullTerminatedXMLFlag: try: data = bc.removeNullTerminator(data) dataAsElement = ET.fromstring(data) @@ -2116,13 +2122,12 @@ self.testFor("containsWellformedXML", containsWellformedXML) else: # Only add to UUID to characteristics tree - self.addCharacteristic("uuid", id) + self.addCharacteristic("uuid", boxUUID) def validate_uuidInfoBox(self): - # UUID Info box (superbox)(ISO/IEC 15444-1 Section I.7.3) - # Provides additional information on vendor-specific UUIDs - - # NOTE: Untested at this stage due to lack of suitable test files!!! + """UUID Info box (superbox)(ISO/IEC 15444-1 Section I.7.3) + Provides additional information on vendor-specific UUIDs + """ # Marker tags/codes that identify sub-boxes as hexadecimal strings tagListBox = b'\x75\x6c\x73\x74' @@ -2133,7 +2138,6 @@ noBytes = len(self.boxContents) byteStart = 0 - bytesTotal = 0 # Dummy value boxLengthValue = 10 @@ -2163,10 +2167,9 @@ self.testFor("containsOneURLBox", subBoxTypes.count(tagURLBox) == 1) def validate_uuidListBox(self): - # UUID List box (ISO/IEC 15444-1 Section I.7.3.1) - # Contains a list of UUIDs - - # NOTE: Untested at this stage due to lack of suitable test files!!! + """UUID List box (ISO/IEC 15444-1 Section I.7.3.1) + Contains a list of UUIDs + """ # Number of UUIDs nU = bc.bytesToUShortInt(self.boxContents[0:2]) @@ -2178,14 +2181,15 @@ # Loop through all UUIDs offset = 2 for i in range(nU): - id = str(uuid.UUID(bytes=self.boxContents[offset:offset + 16])) - self.addCharacteristic("uuid", id) + boxUUID = str(uuid.UUID(bytes=self.boxContents[offset:offset + 16])) + self.addCharacteristic("uuid", boxUUID) offset += 16 def validate_urlBox(self): - # Data Entry URL box (ISO/IEC 15444-1 Section I.7.3.2) - # Contains URL that can be used to obtain more information - # about UUIDs in UUID List box + """Data Entry URL box (ISO/IEC 15444-1 Section I.7.3.2) + Contains URL that can be used to obtain more information + about UUIDs in UUID List box + """ # Version number (1 byte unsigned integer) version = bc.bytesToUnsignedChar(self.boxContents[0:1]) @@ -2211,16 +2215,9 @@ # Remove null character as this cannot be represented as XML loc = bc.removeNullTerminator(loc) - """ - # If loc contains any device control characters (e.g. because of - # file corruption), replace them with printable character - if bc.containsControlCharacters(loc): - loc=bc.replaceControlCharacters(loc) - """ - # Try decode to UTF-8 try: - tmp = loc.decode("utf-8", "strict") + loc.decode("utf-8", "strict") self.testFor("locIsUTF8", True) except UnicodeDecodeError: self.testFor("locIsUTF8", False) @@ -2228,19 +2225,21 @@ self.addCharacteristic("loc", loc) def validate_JP2(self): - # Top-level function for JP2 validation: - # - # 1. Parses all top-level boxes in JP2 byte object, and calls separate validator - # function for each of these - # 2. Checks for presence of all required top-level boxes - # 3. Checks if JP2 header properties are consistent with corresponding properties - # in codestream header + """Top-level function for JP2 validation: + + 1. Parses all top-level boxes in JP2 byte object, and calls separate validator + function for each of these + 2. Checks for presence of all required top-level boxes + 3. Checks if JP2 header properties are consistent with corresponding properties + in codestream header + """ # Marker tags/codes that identify all top level boxes as hexadecimal strings - #(Correspond to "Box Type" values, see ISO/IEC 15444-1 Section I.4) + # (Correspond to "Box Type" values, see ISO/IEC 15444-1 Section I.4) tagSignatureBox = b'\x6a\x50\x20\x20' tagFileTypeBox = b'\x66\x74\x79\x70' tagJP2HeaderBox = b'\x6a\x70\x32\x68' + tagIntellectualPropertyBox = b'\x6a\x70\x32\x69' tagContiguousCodestreamBox = b'\x6a\x70\x32\x63' # List for storing box type identifiers @@ -2248,7 +2247,6 @@ noBytes = len(self.boxContents) byteStart = 0 - bytesTotal = 0 # Dummy value boxLengthValue = 10 @@ -2298,17 +2296,17 @@ # Is the first box a Signature Box (ISO/IEC 15444-1 Section I.5.1)? try: firstBoxIsSignatureBox = boxTypes[0] == tagSignatureBox - except: + except Exception: firstBoxIsSignatureBox = False # Is the second box a File Type Box (ISO/IEC 15444-1 Section I.5.2)? try: secondBoxIsFileTypeBox = boxTypes[1] == tagFileTypeBox - except: + except Exception: secondBoxIsFileTypeBox = False # JP2 Header Box: after File Type box, before (first) contiguous codestream box - #(ISO/IEC 15444-1 Section I.5.3)? + # (ISO/IEC 15444-1 Section I.5.3)? try: positionJP2HeaderBox = boxTypes.index(tagJP2HeaderBox) positionFirstContiguousCodestreamBox = boxTypes.index( @@ -2318,7 +2316,7 @@ locationJP2HeaderBoxIsValid = True else: locationJP2HeaderBoxIsValid = False - except: + except Exception: locationJP2HeaderBoxIsValid = False self.testFor("firstBoxIsSignatureBox", firstBoxIsSignatureBox) @@ -2356,7 +2354,7 @@ # integer. Probably a bug in ET. Using 'find' + text property does work # as expected - if jp2ImageHeader != None and sizHeader != None: + if jp2ImageHeader is not None and sizHeader is not None: # Height should be equal to ysiz -yOsiz @@ -2389,7 +2387,7 @@ # # 1. bPCSign and bPCDepth same for all components --> use values from image header # 2. bPCSign and bPCDepth vary across components --> use values from Bits Per - # Components box + # -- Components box # # Situation 1 is the most common one. Situation 2 can be identified by a value # of 255 of bPC in the image header, which corresponds to bPCSign = 1 diff -Nru jpylyzer-1.17.0/jpylyzer/byteconv.py jpylyzer-1.18.0/jpylyzer/byteconv.py --- jpylyzer-1.17.0/jpylyzer/byteconv.py 2016-01-05 16:26:23.000000000 +0000 +++ jpylyzer-1.18.0/jpylyzer/byteconv.py 2017-08-23 11:42:58.000000000 +0000 @@ -1,3 +1,4 @@ +"""Various functions for converting and manipulating bytes objects""" # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU Lesser General Public License as published by @@ -17,109 +18,111 @@ import binascii import unicodedata -# Convert byte object of bOrder byteorder to format using formatCharacter -# Return -9999 if unpack raised an error - def _doConv(bytestr, bOrder, formatCharacter): + """Convert byte object of bOrder byteorder to format using formatCharacter + Return -9999 if unpack raised an error + """ # Format string for unpack formatStr = bOrder + formatCharacter try: result = struct.unpack(formatStr, bytestr)[0] except: result = -9999 - return(result) + return result + +def bytesToULongLong(bytestring): + """Unpack 8 byte string to unsigned long long integer, assuming big-endian + byte order + """ + return _doConv(bytestring, ">", "Q") -def bytesToULongLong(bytes): - # Unpack 8 byte string to unsigned long long integer, assuming big-endian - # byte order. - return _doConv(bytes, ">", "Q") +def bytesToUInt(bytestring): + """Unpack 4 byte string to unsigned integer, assuming big-endian byte order""" + return _doConv(bytestring, ">", "I") -def bytesToUInt(bytes): - # Unpack 4 byte string to unsigned integer, assuming big-endian byte order. - return _doConv(bytes, ">", "I") +def bytesToUShortInt(bytestring): + """Unpack 2 byte string to unsigned short integer, assuming big-endian + byte order""" + return _doConv(bytestring, ">", "H") -def bytesToUShortInt(bytes): - # Unpack 2 byte string to unsigned short integer, assuming big-endian - # byte order - return _doConv(bytes, ">", "H") +def bytesToUnsignedChar(bytestring): + """Unpack 1 byte string to unsigned character/integer, assuming big-endian + byte order""" + return _doConv(bytestring, ">", "B") -def bytesToUnsignedChar(bytes): - # Unpack 1 byte string to unsigned character/integer, assuming big-endian - # byte order. - return _doConv(bytes, ">", "B") +def bytesToSignedChar(bytestring): + """Unpack 1 byte string to signed character/integer, assuming big-endian + byte order""" + return _doConv(bytestring, ">", "b") -def bytesToSignedChar(bytes): - # Unpack 1 byte string to signed character/integer, assuming big-endian - # byte order. - return _doConv(bytes, ">", "b") +def bytesToInteger(bytestring): + """Unpack byte string of any length to integer. + Taken from: + http://stackoverflow.com/questions/4358285/ -def bytesToInteger(bytes): - # Unpack byte string of any length to integer. - # - # Taken from: - # http://stackoverflow.com/questions/4358285/ - # - # JvdK: what endianness is assumed here? Could go wrong on some systems? + JvdK: what endianness is assumed here? Could go wrong on some systems? - # binascii.hexlify will be obsolete in python3 soon - # They will add a .tohex() method to bytes class - # Issue 3532 bugs.python.org + binascii.hexlify will be obsolete in python3 soon + They will add a .tohex() method to bytes class + Issue 3532 bugs.python.org + """ try: - result = int(binascii.hexlify(bytes), 16) + result = int(binascii.hexlify(bytestring), 16) except: result = -9999 - return (result) + return result def isctrl(c): - # Returns True if byte corresponds to device control character - # (See also: http://www.w3schools.com/tags/ref_ascii.asp) - return (ord(c) < 32 or ord(c) == 127) - # return (0 <= ord(c) <= 8) or (ord(c) == 12) or (14 <= ord(c) < 32) + """Returns True if byte corresponds to device control character + (See also: http://www.w3schools.com/tags/ref_ascii.asp)""" + return ord(c) < 32 or ord(c) == 127 -def bytesToHex(bytes): - # Return hexadecimal ascii representation of bytes - return binascii.hexlify(bytes) +def bytesToHex(bytestring): + """Return hexadecimal ascii representation of bytestring""" + return binascii.hexlify(bytestring) -def containsControlCharacters(bytes): - # Returns True if bytes object contains control characters +def containsControlCharacters(bytestring): + """Returns True if bytestring object contains control characters""" - for i in range(len(bytes)): - if isctrl(bytes[i:i + 1]): - return(True) - return(False) + for i in range(len(bytestring)): + if isctrl(bytestring[i:i + 1]): + return True + return False def removeControlCharacters(string): - # Remove control characters from string - # Adapted from: http://stackoverflow.com/a/19016117/1209004 + """Remove control characters from string + Adapted from: http://stackoverflow.com/a/19016117/1209004 + """ # Tab, newline and return are part of C0, but are allowed in XML allowedChars = [u'\t', u'\n', u'\r'] return "".join(ch for ch in string if unicodedata.category(ch)[0] != "C" or ch in allowedChars) -def removeNullTerminator(bytes): - # Remove null terminator from bytes +def removeNullTerminator(bytestring): + """Remove null terminator from bytestring""" - bytesOut = bytes.rstrip(b'\x00') - return(bytesOut) + bytesOut = bytestring.rstrip(b'\x00') + return bytesOut -def bytesToText(bytes): - # Unpack byte object to text string, assuming big-endian - # byte order. +def bytesToText(bytestring): + """Unpack byte object to text string, assuming big-endian + byte order + """ # Set encoding and error mode enc = "utf-8" @@ -127,13 +130,13 @@ try: # Decode to utf-8 - string = bytes.decode(encoding=enc, errors=errorMode) + string = bytestring.decode(encoding=enc, errors=errorMode) # Remove control characters result = removeControlCharacters(string) except: - # Return empty string if bytes cannot be decoded + # Return empty string if bytestring cannot be decoded result = "" - return(result) + return result diff -Nru jpylyzer-1.17.0/jpylyzer/config.py jpylyzer-1.18.0/jpylyzer/config.py --- jpylyzer-1.17.0/jpylyzer/config.py 2016-01-05 16:26:23.000000000 +0000 +++ jpylyzer-1.18.0/jpylyzer/config.py 2017-08-23 11:42:58.000000000 +0000 @@ -1,3 +1,4 @@ +"""Jpylyzer configuration settings that are shared between sub-modules""" # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU Lesser General Public License as published by diff -Nru jpylyzer-1.17.0/jpylyzer/etpatch.py jpylyzer-1.18.0/jpylyzer/etpatch.py --- jpylyzer-1.17.0/jpylyzer/etpatch.py 2016-01-05 16:26:23.000000000 +0000 +++ jpylyzer-1.18.0/jpylyzer/etpatch.py 2017-08-23 11:42:58.000000000 +0000 @@ -1,3 +1,8 @@ +"""Patch for 'findtext' bug in ElementTree +TODO: +1) Find out whether these patches are necessary +2) learn how to write and test patches properly +""" # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU Lesser General Public License as published by @@ -14,44 +19,44 @@ # import xml.etree.ElementTree as ET -from byteconv import bytesToText -from byteconv import removeControlCharacters -import config +from . import byteconv as bc +from . import config as config -# TODO: -# 1) Find out whether these patches are necessary -# 2) learn how to write and test patches properly - -# I don't want to mess with ANYthing :) def tostring(elem, enc, meth): + """Return string representation of Element object with user-defined encoding and method""" return ET.tostring(elem, enc, meth) def fromstring(text): + """Convert string to Element object""" return ET.fromstring(text) def SubElement(parent, tag): + """Return sub-element from parent element""" return ET.SubElement(parent, tag) class Element(ET.Element): + """Element class""" - # Replacement for ET's 'findtext' function, which has a bug - # that will return empty string if text field contains integer with - # value of zero (0); If there is no match, return None def findElementText(self, match): + """Replacement for ET's 'findtext' function, which has a bug + that will return empty string if text field contains integer with + value of zero (0); If there is no match, return None + """ elt = self.find(match) if elt is not None: - return(elt.text) + return elt.text else: - return(None) + return None def findAllText(self, match): - # Searches element and returns list that contains 'Text' attribute - # of all matching sub-elements. Returns empty list if element - # does not exist + """Searches element and returns list that contains 'Text' attribute + of all matching sub-elements. Returns empty list if element + does not exist + """ try: return [result.text for result in self.findall(match)] @@ -59,24 +64,25 @@ return [] def appendChildTagWithText(self, tag, text): - # Append childnode with text + """Append childnode with text""" el = ET.SubElement(self, tag) el.text = text def appendIfNotEmpty(self, subelement): - # Append sub-element, but only if subelement is not empty + """Append sub-element, but only if subelement is not empty""" if len(subelement) != 0: self.append(subelement) def makeHumanReadable(self, remapTable={}): - # Takes element object, and returns a modified version in which all - # non-printable 'text' fields (which may contain numeric data or binary strings) - # are replaced by printable strings - # - # Property values in original tree may be mapped to alternative (more user-friendly) - # reportable values using a remapTable, which is a nested dictionary. + """Takes element object, and returns a modified version in which all + non-printable 'text' fields (which may contain numeric data or binary strings) + are replaced by printable strings + + Property values in original tree may be mapped to alternative (more user-friendly) + reportable values using a remapTable, which is a nested dictionary. + """ for elt in self.iter(): # Text field of this element @@ -116,25 +122,21 @@ # Convert - if remappedValue != None: + if remappedValue is not None: # Data type textType = type(remappedValue) # Convert text field, depending on type if textType == bytes: - textOut = bytesToText(remappedValue) + textOut = bc.bytesToText(remappedValue) elif textType in numericTypes: textOut = str(remappedValue) else: - textOut = removeControlCharacters(remappedValue) + textOut = bc.removeControlCharacters(remappedValue) # Update output tree elt.text = textOut def toxml(self, indent=" "): - return(ET.tostring(self, 'UTF-8', 'xml')) - - # Disabled pretty-printing for now as minidom appears to choke on - # entity references, i.e. code below will go wrong: - # - # return minidom.parseString(selfAsString).toprettyxml(indent) + """Convert Element object to XML""" + return ET.tostring(self, 'UTF-8', 'xml') diff -Nru jpylyzer-1.17.0/jpylyzer/jpylyzer.py jpylyzer-1.18.0/jpylyzer/jpylyzer.py --- jpylyzer-1.17.0/jpylyzer/jpylyzer.py 2016-01-05 16:26:23.000000000 +0000 +++ jpylyzer-1.18.0/jpylyzer/jpylyzer.py 2017-08-23 11:42:58.000000000 +0000 @@ -1,24 +1,24 @@ #! /usr/bin/env python # -# -# -# jpylyzer -# -# Requires: Python 2.7 (older versions won't work) OR Python 3.2 or more recent -# (Python 3.0 and 3.1 won't work either!) -# -# Copyright (C) 2011 - 2014 Johan van der Knijff, Koninklijke Bibliotheek - -# National Library of the Netherlands -# -# Contributors: -# Rene van der Ark (refactoring of original code) -# Lars Buitinck -# Adam Retter, The National Archives, UK. -# Jaishree Davey, The National Archives, UK. -# Laura Damian, The National Archives, UK. -# Carl Wilson, Open Planets Foundation, UK. -# Stefan Weil, UB Mannheim, DE. -# +"""Jpylyzer validator for JPEG 200 Part 1 (JP2) images + + Requires: Python 2.7 (older versions won't work) OR Python 3.2 or more recent + (Python 3.0 and 3.1 won't work either!) + + Copyright (C) 2011 - 2017 Johan van der Knijff, Koninklijke Bibliotheek - + National Library of the Netherlands + +Contributors: + Rene van der Ark, NL (refactoring of original code). + Lars Buitinck, NL. + Adam Retter, The National Archives, UK. + Jaishree Davey, The National Archives, UK. + Laura Damian, The National Archives, UK. + Carl Wilson, Open Planets Foundation, UK. + Stefan Weil, UB Mannheim, DE. + Adam Fritzler, Planet Labs, USA. +""" + # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU Lesser General Public License as published by # the Free Software Foundation, either version 3 of the License, or @@ -36,28 +36,26 @@ import mmap import os import time -import imp import glob -import struct import argparse -import config import codecs import re -import etpatch as ET -import fnmatch -import xml.etree.ElementTree as ETree -from boxvalidator import BoxValidator from xml.dom import minidom -from byteconv import bytesToText -from shared import printWarning -scriptPath, scriptName = os.path.split(sys.argv[0]) +import xml.etree.ElementTree as ETree from six import u +from . import config as config +from . import etpatch as ET +from . import boxvalidator as bv +from . import shared as shared + + +scriptPath, scriptName = os.path.split(sys.argv[0]) # scriptName is empty when called from Java/Jython, so this needs a fix if len(scriptName) == 0: scriptName = 'jpylyzer' -__version__ = "1.17.0" +__version__ = "1.18.0" # Create parser parser = argparse.ArgumentParser( @@ -66,35 +64,17 @@ # list of existing files to be analysed existingFiles = [] +# Name space and schema strings +nsString = 'http://openpreservation.org/ns/jpylyzer/' +xsiNsString = 'http://www.w3.org/2001/XMLSchema-instance' +locSchemaString = 'http://openpreservation.org/ns/jpylyzer/ \ +http://jpylyzer.openpreservation.org/jpylyzer-v-1-1.xsd' -def main_is_frozen(): - return (hasattr(sys, "frozen") or # new py2exe - hasattr(sys, "importers") # old py2exe - or imp.is_frozen("__main__")) # tools/freeze - - -def get_main_dir(): - if main_is_frozen(): - return os.path.dirname(sys.executable) - return os.path.dirname(sys.argv[0]) - - -def readFileBytes(file): - # Read file, return contents as a byte object - - # Open file - f = open(file, "rb") - - # Put contents of file into a byte object. - fileData = f.read() - f.close() - - return(fileData) def generatePropertiesRemapTable(): - - # Generates nested dictionary which is used to map 'raw' property values - # (mostly integer values) to corresponding text descriptions + """Generates nested dictionary which is used to map 'raw' property values + (mostly integer values) to corresponding text descriptions + """ # Master dictionary for mapping of text descriptions to enumerated values # Key: corresponds to parameter tag name @@ -259,13 +239,14 @@ enumerationsMap['qStyle'] = qStyleMap enumerationsMap['rcom'] = registrationMap - return(enumerationsMap) + return enumerationsMap + -def fileToMemoryMap(file): - # Read contents of file to memory map object +def fileToMemoryMap(filename): + """Read contents of filename to memory map object""" - # Open file - f = open(file, "rb") + # Open filename + f = open(filename, "rb") # Call to mmap is different on Linux and Windows, so we need to know # the current platform @@ -282,12 +263,13 @@ except ValueError: # mmap fails on empty files. fileData = "" - - f.close() - return(fileData) -def checkOneFile(file): - # Process one file and return analysis result as element object + f.close() + return fileData + + +def checkOneFile(path): + """Process one file and return analysis result as element object""" # Create output elementtree object @@ -297,34 +279,33 @@ root = ET.Element('jpylyzer') else: root = ET.Element( - 'jpylyzer', {'xmlns': 'http://openpreservation.org/ns/jpylyzer/', - 'xmlns:xsi': 'http://www.w3.org/2001/XMLSchema-instance', - 'xsi:schemaLocation': 'http://openpreservation.org/ns/jpylyzer/ http://jpylyzer.openpreservation.org/jpylyzer-v-1-1.xsd'}) + 'jpylyzer', {'xmlns': nsString, + 'xmlns:xsi': xsiNsString, + 'xsi:schemaLocation': locSchemaString}) # Create elements for storing tool, file and status meta info toolInfo = ET.Element('toolInfo') fileInfo = ET.Element('fileInfo') statusInfo = ET.Element('statusInfo') - # File name and path - fileName = os.path.basename(file) - filePath = os.path.abspath(file) + # File name and path + fileName = os.path.basename(path) + filePath = os.path.abspath(path) # If file name / path contain any surrogate pairs, remove them to # avoid problems when writing to XML fileNameCleaned = stripSurrogatePairs(fileName) filePathCleaned = stripSurrogatePairs(filePath) - # Produce some general tool and file meta info toolInfo.appendChildTagWithText("toolName", scriptName) toolInfo.appendChildTagWithText("toolVersion", __version__) fileInfo.appendChildTagWithText("fileName", fileNameCleaned) fileInfo.appendChildTagWithText("filePath", filePathCleaned) fileInfo.appendChildTagWithText( - "fileSizeInBytes", str(os.path.getsize(file))) + "fileSizeInBytes", str(os.path.getsize(path))) try: - lastModifiedDate = time.ctime(os.path.getmtime(file)) + lastModifiedDate = time.ctime(os.path.getmtime(path)) except ValueError: # Dates earlier than 1 Jan 1970 can raise ValueError on Windows # Workaround: replace by lowest possible value (typically 1 Jan 1970) @@ -334,12 +315,12 @@ # Initialise success flag success = True - + try: # Contents of file to memory map object - fileData = fileToMemoryMap(file) - isValidJP2, tests, characteristics = BoxValidator("JP2", fileData).validate() - + fileData = fileToMemoryMap(path) + isValidJP2, tests, characteristics = bv.BoxValidator("JP2", fileData).validate() + if fileData != "": fileData.close() @@ -349,7 +330,7 @@ # Create printable version of tests and characteristics tree tests.makeHumanReadable() characteristics.makeHumanReadable(remapTable) - except Exception as ex: + except Exception as ex: isValidJP2 = False success = False exceptionType = type(ex) @@ -363,15 +344,15 @@ else: failureMessage = "unknown error (please report to developers)" - printWarning(failureMessage) + shared.printWarning(failureMessage) tests = ET.Element("tests") characteristics = ET.Element('properties') - + # Add status info statusInfo.appendChildTagWithText("success", str(success)) - if success == False: - statusInfo.appendChildTagWithText("failureMessage",failureMessage) - + if not success: + statusInfo.appendChildTagWithText("failureMessage", failureMessage) + # Append all results to root root.append(toolInfo) root.append(fileInfo) @@ -380,11 +361,13 @@ root.append(tests) root.append(characteristics) - return(root) + return root + def checkNullArgs(args): - # This method checks if the input arguments list and exits program if - # invalid or no input argument is supplied. + """This method checks if the input arguments list and exits program if + invalid or no input argument is supplied. + """ if len(args) == 0: @@ -394,23 +377,25 @@ def checkNoInput(files): - # Check if input arguments list results in any existing input files at all - # (and exits if not) + """Check if input arguments list results in any existing input files at all + (and exits if not) + """ if len(files) == 0: - printWarning("no images to check!") + shared.printWarning("no images to check!") sys.exit(config.ERR_CODE_NO_IMAGES) def printHelpAndExit(): - # Print help message and exit + """Print help message and exit""" print('') parser.print_help() sys.exit() + def stripSurrogatePairs(ustring): - # Removes surrogate pairs from a Unicode string + """Removes surrogate pairs from a Unicode string""" # This works for Python 3.x, but not for 2.x! # Source: http://stackoverflow.com/q/19649463/1209004 @@ -420,7 +405,7 @@ ustring.encode('utf-8') except UnicodeEncodeError: # Strip away surrogate pairs - tmp = ustring.encode('utf-8', 'surrogateescape') + tmp = ustring.encode('utf-8', 'replace') ustring = tmp.decode('utf-8', 'ignore') # In Python 2.x we need to use regex @@ -441,21 +426,16 @@ [\udc00-\udfff] # match trailing surrogate ) # end group """)) - - # Remove surrogates (i.e. replace by empty string) - tmp = lone.sub(r'',ustring).encode('utf-8') - ustring = tmp.decode('utf-8') - return(ustring) + # Remove surrogates (i.e. replace by empty string) + tmp = lone.sub(r'', ustring).encode('utf-8') + ustring = tmp.decode('utf-8') -def getFilesFromDir(dirpath): - for fp in os.listdir(dirpath): - filepath = os.path.join(dirpath, fp) - if os.path.isfile(filepath): - existingFiles.append(filepath) + return ustring def getFiles(searchpattern): + """Append paths of all files that match search pattern to existingFiles""" results = glob.glob(searchpattern) for f in results: if os.path.isfile(f): @@ -463,9 +443,9 @@ def getFilesWithPatternFromTree(rootDir, pattern): - # Recurse into directory tree and return list of all files - # NOTE: directory names are disabled here!! - + """Recurse into directory tree and return list of all files + NOTE: directory names are disabled here!! + """ for dirname, dirnames, filenames in os.walk(rootDir): # Suppress directory names for subdirname in dirnames: @@ -474,9 +454,11 @@ searchpattern = os.path.join(thisDirectory, pattern) getFiles(searchpattern) + def getFilesFromTree(rootDir): - # Recurse into directory tree and return list of all files - # NOTE: directory names are disabled here!! + """Recurse into directory tree and return list of all files + NOTE: directory names are disabled here!! + """ for dirname, dirnames, filenames in os.walk(rootDir): # Suppress directory names @@ -489,12 +471,13 @@ def findFiles(recurse, paths): + """Find all files that match a wildcard expression and add their paths to existingFiles""" WILDCARD = "*" # process the list of input paths for root in paths: - + if config.PYTHON_VERSION.startswith(config.PYTHON_2): # Convert root to UTF-8 (only needed for Python 2.x) root = unicode(root, 'utf-8') @@ -507,7 +490,7 @@ # Find files in the input path and add to list elif WILDCARD in root: # get the absolute path if not given - if not(os.path.isabs(root)): + if not os.path.isabs(root): root = os.path.abspath(root) # Expand wildcard in the input path. Returns a list of files, @@ -521,10 +504,6 @@ root = filesList[0] # get files from directory - """ Disabled JvdK: if enabled all files in direct child directories are analysed - do we really want that? - if os.path.isdir(root) and not recurse: - getFilesFromDir(root) - """ # If the input path returned files list, add files to List @@ -536,22 +515,16 @@ if os.path.isfile(f): existingFiles.append(f) - elif os.path.isdir(root) == False and os.path.isfile(root) == False: + elif not os.path.isdir(root) and not os.path.isfile(root): # One or more (but not all) paths do no exist - print a warning msg = root + " does not exist" - printWarning(msg) - - """ Disabled JvdK: - elif os.path.isdir(root) and not recurse: - #input path is a directory and is not recursive - getFilesFromDir(root) - """ + shared.printWarning(msg) # RECURSION and WILDCARD IN RECURSION # Check if recurse in the input path if recurse: # get absolute input path if not given - if not(os.path.isabs(root)): + if not os.path.isabs(root): root = os.path.abspath(root) if WILDCARD in root: @@ -584,7 +557,7 @@ def writeElement(elt, codec): - # Writes element as XML to stdout using defined codec + """Writes element as XML to stdout using defined codec""" # Element to string if config.PYTHON_VERSION.startswith(config.PYTHON_2): @@ -592,7 +565,7 @@ if config.PYTHON_VERSION.startswith(config.PYTHON_3): xmlOut = ET.tostring(elt, 'unicode', 'xml') - if config.noPrettyXMLFlag == False: + if not config.noPrettyXMLFlag: # Make xml pretty xmlPretty = minidom.parseString(xmlOut).toprettyxml(' ') @@ -619,15 +592,16 @@ def checkFiles(recurse, wrap, paths): - # This method checks the input argument path(s) for existing files and - # analyses them + """This method checks the input argument path(s) for existing files and + analyses them + """ # Find existing files in the given input path(s) findFiles(recurse, paths) # If there are no valid input files then exit program checkNoInput(existingFiles) - + # Set encoding of the terminal to UTF-8 if config.PYTHON_VERSION.startswith(config.PYTHON_2): out = codecs.getwriter(config.UTF8_ENCODING)(sys.stdout) @@ -637,9 +611,9 @@ # Wrap the xml output in element, if wrapper flag is true if wrap or recurse: xmlHead = "\n" - xmlHead += "\n" + xmlHead += "\n" else: xmlHead = "\n" out.write(xmlHead) @@ -649,7 +623,7 @@ # Analyse file xmlElement = checkOneFile(path) - + # Write output to stdout writeElement(xmlElement, out) @@ -659,6 +633,8 @@ def parseCommandLine(): + """Parse command line arguments""" + # Add arguments parser.add_argument('--verbose', action="store_true", @@ -670,7 +646,8 @@ action="store_true", dest="inputRecursiveFlag", default=False, - help="when analysing a directory, recurse into subdirectories (implies --wrapper)") + help="when analysing a directory, recurse into subdirectories \ + (implies --wrapper)") parser.add_argument('--wrapper', '-w', action="store_true", dest="inputWrapperFlag", @@ -680,7 +657,8 @@ action="store_true", dest="extractNullTerminatedXMLFlag", default=False, - help="extract null-terminated XML content from XML and UUID boxes(doesn't affect validation)") + help="extract null-terminated XML content from XML and UUID boxes \ + (doesn't affect validation)") parser.add_argument('--nopretty', action="store_true", dest="noPrettyXMLFlag", @@ -690,7 +668,8 @@ action="store", type=str, nargs='+', - help="input JP2 image(s), may be one or more (whitespace-separated) path expressions; prefix wildcard (*) with backslash (\\) in Linux") + help="input JP2 image(s), may be one or more (whitespace-separated) path \ + expressions; prefix wildcard (*) with backslash (\\) in Linux") parser.add_argument('--version', '-v', action='version', version=__version__) @@ -698,10 +677,12 @@ # Parse arguments args = parser.parse_args() - return(args) + return args def main(): + """Main command line application""" + # Get input from command line args = parseCommandLine() diff -Nru jpylyzer-1.17.0/jpylyzer/__main__.py jpylyzer-1.18.0/jpylyzer/__main__.py --- jpylyzer-1.17.0/jpylyzer/__main__.py 1970-01-01 00:00:00.000000000 +0000 +++ jpylyzer-1.18.0/jpylyzer/__main__.py 2017-08-23 11:42:58.000000000 +0000 @@ -0,0 +1,8 @@ +#! /usr/bin/env python + + +"""jpylyzer.__main__: executed when jpylyzer directory is called as script.""" + + +from .jpylyzer import main +main() diff -Nru jpylyzer-1.17.0/jpylyzer/shared.py jpylyzer-1.18.0/jpylyzer/shared.py --- jpylyzer-1.17.0/jpylyzer/shared.py 2016-01-05 16:26:23.000000000 +0000 +++ jpylyzer-1.18.0/jpylyzer/shared.py 2017-08-23 11:42:58.000000000 +0000 @@ -1,3 +1,4 @@ +"""Shared functions for jpylyzer sub-modules""" # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU Lesser General Public License as published by @@ -19,19 +20,20 @@ def printWarning(msg): + """Print warning to stderr""" msgString = ("User warning: " + msg + "\n") sys.stderr.write(msgString) def consecutive(lst): - # Returns True if items in lst are consecutive numbers + """Returns True if items in lst are consecutive numbers""" for i in range(1, len(lst)): if lst[i] - lst[i - 1] != 1: - return(False) - return(True) + return False + return True def listOccurrencesAreContiguous(lst, value): - # True if all occurrences of value in lst are at contiguous positions + """Returns True if all occurrences of value in lst are at contiguous positions""" indices_of_value = [i for i in range(len(lst)) if lst[i] == value] return consecutive(indices_of_value) diff -Nru jpylyzer-1.17.0/jpylyzer/six.py jpylyzer-1.18.0/jpylyzer/six.py --- jpylyzer-1.17.0/jpylyzer/six.py 2016-01-05 16:26:23.000000000 +0000 +++ jpylyzer-1.18.0/jpylyzer/six.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,868 +0,0 @@ -"""Utilities for writing code that runs on Python 2 and 3""" - -# Copyright (c) 2010-2015 Benjamin Peterson -# -# Permission is hereby granted, free of charge, to any person obtaining a copy -# of this software and associated documentation files (the "Software"), to deal -# in the Software without restriction, including without limitation the rights -# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell -# copies of the Software, and to permit persons to whom the Software is -# furnished to do so, subject to the following conditions: -# -# The above copyright notice and this permission notice shall be included in all -# copies or substantial portions of the Software. -# -# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE -# SOFTWARE. - -from __future__ import absolute_import - -import functools -import itertools -import operator -import sys -import types - -__author__ = "Benjamin Peterson " -__version__ = "1.10.0" - - -# Useful for very coarse version differentiation. -PY2 = sys.version_info[0] == 2 -PY3 = sys.version_info[0] == 3 -PY34 = sys.version_info[0:2] >= (3, 4) - -if PY3: - string_types = str, - integer_types = int, - class_types = type, - text_type = str - binary_type = bytes - - MAXSIZE = sys.maxsize -else: - string_types = basestring, - integer_types = (int, long) - class_types = (type, types.ClassType) - text_type = unicode - binary_type = str - - if sys.platform.startswith("java"): - # Jython always uses 32 bits. - MAXSIZE = int((1 << 31) - 1) - else: - # It's possible to have sizeof(long) != sizeof(Py_ssize_t). - class X(object): - - def __len__(self): - return 1 << 31 - try: - len(X()) - except OverflowError: - # 32-bit - MAXSIZE = int((1 << 31) - 1) - else: - # 64-bit - MAXSIZE = int((1 << 63) - 1) - del X - - -def _add_doc(func, doc): - """Add documentation to a function.""" - func.__doc__ = doc - - -def _import_module(name): - """Import module, returning the module after the last dot.""" - __import__(name) - return sys.modules[name] - - -class _LazyDescr(object): - - def __init__(self, name): - self.name = name - - def __get__(self, obj, tp): - result = self._resolve() - setattr(obj, self.name, result) # Invokes __set__. - try: - # This is a bit ugly, but it avoids running this again by - # removing this descriptor. - delattr(obj.__class__, self.name) - except AttributeError: - pass - return result - - -class MovedModule(_LazyDescr): - - def __init__(self, name, old, new=None): - super(MovedModule, self).__init__(name) - if PY3: - if new is None: - new = name - self.mod = new - else: - self.mod = old - - def _resolve(self): - return _import_module(self.mod) - - def __getattr__(self, attr): - _module = self._resolve() - value = getattr(_module, attr) - setattr(self, attr, value) - return value - - -class _LazyModule(types.ModuleType): - - def __init__(self, name): - super(_LazyModule, self).__init__(name) - self.__doc__ = self.__class__.__doc__ - - def __dir__(self): - attrs = ["__doc__", "__name__"] - attrs += [attr.name for attr in self._moved_attributes] - return attrs - - # Subclasses should override this - _moved_attributes = [] - - -class MovedAttribute(_LazyDescr): - - def __init__(self, name, old_mod, new_mod, old_attr=None, new_attr=None): - super(MovedAttribute, self).__init__(name) - if PY3: - if new_mod is None: - new_mod = name - self.mod = new_mod - if new_attr is None: - if old_attr is None: - new_attr = name - else: - new_attr = old_attr - self.attr = new_attr - else: - self.mod = old_mod - if old_attr is None: - old_attr = name - self.attr = old_attr - - def _resolve(self): - module = _import_module(self.mod) - return getattr(module, self.attr) - - -class _SixMetaPathImporter(object): - - """ - A meta path importer to import six.moves and its submodules. - - This class implements a PEP302 finder and loader. It should be compatible - with Python 2.5 and all existing versions of Python3 - """ - - def __init__(self, six_module_name): - self.name = six_module_name - self.known_modules = {} - - def _add_module(self, mod, *fullnames): - for fullname in fullnames: - self.known_modules[self.name + "." + fullname] = mod - - def _get_module(self, fullname): - return self.known_modules[self.name + "." + fullname] - - def find_module(self, fullname, path=None): - if fullname in self.known_modules: - return self - return None - - def __get_module(self, fullname): - try: - return self.known_modules[fullname] - except KeyError: - raise ImportError("This loader does not know module " + fullname) - - def load_module(self, fullname): - try: - # in case of a reload - return sys.modules[fullname] - except KeyError: - pass - mod = self.__get_module(fullname) - if isinstance(mod, MovedModule): - mod = mod._resolve() - else: - mod.__loader__ = self - sys.modules[fullname] = mod - return mod - - def is_package(self, fullname): - """ - Return true, if the named module is a package. - - We need this method to get correct spec objects with - Python 3.4 (see PEP451) - """ - return hasattr(self.__get_module(fullname), "__path__") - - def get_code(self, fullname): - """Return None - - Required, if is_package is implemented""" - self.__get_module(fullname) # eventually raises ImportError - return None - get_source = get_code # same as get_code - -_importer = _SixMetaPathImporter(__name__) - - -class _MovedItems(_LazyModule): - - """Lazy loading of moved objects""" - __path__ = [] # mark as package - - -_moved_attributes = [ - MovedAttribute("cStringIO", "cStringIO", "io", "StringIO"), - MovedAttribute("filter", "itertools", "builtins", "ifilter", "filter"), - MovedAttribute("filterfalse", "itertools", "itertools", "ifilterfalse", "filterfalse"), - MovedAttribute("input", "__builtin__", "builtins", "raw_input", "input"), - MovedAttribute("intern", "__builtin__", "sys"), - MovedAttribute("map", "itertools", "builtins", "imap", "map"), - MovedAttribute("getcwd", "os", "os", "getcwdu", "getcwd"), - MovedAttribute("getcwdb", "os", "os", "getcwd", "getcwdb"), - MovedAttribute("range", "__builtin__", "builtins", "xrange", "range"), - MovedAttribute("reload_module", "__builtin__", "importlib" if PY34 else "imp", "reload"), - MovedAttribute("reduce", "__builtin__", "functools"), - MovedAttribute("shlex_quote", "pipes", "shlex", "quote"), - MovedAttribute("StringIO", "StringIO", "io"), - MovedAttribute("UserDict", "UserDict", "collections"), - MovedAttribute("UserList", "UserList", "collections"), - MovedAttribute("UserString", "UserString", "collections"), - MovedAttribute("xrange", "__builtin__", "builtins", "xrange", "range"), - MovedAttribute("zip", "itertools", "builtins", "izip", "zip"), - MovedAttribute("zip_longest", "itertools", "itertools", "izip_longest", "zip_longest"), - MovedModule("builtins", "__builtin__"), - MovedModule("configparser", "ConfigParser"), - MovedModule("copyreg", "copy_reg"), - MovedModule("dbm_gnu", "gdbm", "dbm.gnu"), - MovedModule("_dummy_thread", "dummy_thread", "_dummy_thread"), - MovedModule("http_cookiejar", "cookielib", "http.cookiejar"), - MovedModule("http_cookies", "Cookie", "http.cookies"), - MovedModule("html_entities", "htmlentitydefs", "html.entities"), - MovedModule("html_parser", "HTMLParser", "html.parser"), - MovedModule("http_client", "httplib", "http.client"), - MovedModule("email_mime_multipart", "email.MIMEMultipart", "email.mime.multipart"), - MovedModule("email_mime_nonmultipart", "email.MIMENonMultipart", "email.mime.nonmultipart"), - MovedModule("email_mime_text", "email.MIMEText", "email.mime.text"), - MovedModule("email_mime_base", "email.MIMEBase", "email.mime.base"), - MovedModule("BaseHTTPServer", "BaseHTTPServer", "http.server"), - MovedModule("CGIHTTPServer", "CGIHTTPServer", "http.server"), - MovedModule("SimpleHTTPServer", "SimpleHTTPServer", "http.server"), - MovedModule("cPickle", "cPickle", "pickle"), - MovedModule("queue", "Queue"), - MovedModule("reprlib", "repr"), - MovedModule("socketserver", "SocketServer"), - MovedModule("_thread", "thread", "_thread"), - MovedModule("tkinter", "Tkinter"), - MovedModule("tkinter_dialog", "Dialog", "tkinter.dialog"), - MovedModule("tkinter_filedialog", "FileDialog", "tkinter.filedialog"), - MovedModule("tkinter_scrolledtext", "ScrolledText", "tkinter.scrolledtext"), - MovedModule("tkinter_simpledialog", "SimpleDialog", "tkinter.simpledialog"), - MovedModule("tkinter_tix", "Tix", "tkinter.tix"), - MovedModule("tkinter_ttk", "ttk", "tkinter.ttk"), - MovedModule("tkinter_constants", "Tkconstants", "tkinter.constants"), - MovedModule("tkinter_dnd", "Tkdnd", "tkinter.dnd"), - MovedModule("tkinter_colorchooser", "tkColorChooser", - "tkinter.colorchooser"), - MovedModule("tkinter_commondialog", "tkCommonDialog", - "tkinter.commondialog"), - MovedModule("tkinter_tkfiledialog", "tkFileDialog", "tkinter.filedialog"), - MovedModule("tkinter_font", "tkFont", "tkinter.font"), - MovedModule("tkinter_messagebox", "tkMessageBox", "tkinter.messagebox"), - MovedModule("tkinter_tksimpledialog", "tkSimpleDialog", - "tkinter.simpledialog"), - MovedModule("urllib_parse", __name__ + ".moves.urllib_parse", "urllib.parse"), - MovedModule("urllib_error", __name__ + ".moves.urllib_error", "urllib.error"), - MovedModule("urllib", __name__ + ".moves.urllib", __name__ + ".moves.urllib"), - MovedModule("urllib_robotparser", "robotparser", "urllib.robotparser"), - MovedModule("xmlrpc_client", "xmlrpclib", "xmlrpc.client"), - MovedModule("xmlrpc_server", "SimpleXMLRPCServer", "xmlrpc.server"), -] -# Add windows specific modules. -if sys.platform == "win32": - _moved_attributes += [ - MovedModule("winreg", "_winreg"), - ] - -for attr in _moved_attributes: - setattr(_MovedItems, attr.name, attr) - if isinstance(attr, MovedModule): - _importer._add_module(attr, "moves." + attr.name) -del attr - -_MovedItems._moved_attributes = _moved_attributes - -moves = _MovedItems(__name__ + ".moves") -_importer._add_module(moves, "moves") - - -class Module_six_moves_urllib_parse(_LazyModule): - - """Lazy loading of moved objects in six.moves.urllib_parse""" - - -_urllib_parse_moved_attributes = [ - MovedAttribute("ParseResult", "urlparse", "urllib.parse"), - MovedAttribute("SplitResult", "urlparse", "urllib.parse"), - MovedAttribute("parse_qs", "urlparse", "urllib.parse"), - MovedAttribute("parse_qsl", "urlparse", "urllib.parse"), - MovedAttribute("urldefrag", "urlparse", "urllib.parse"), - MovedAttribute("urljoin", "urlparse", "urllib.parse"), - MovedAttribute("urlparse", "urlparse", "urllib.parse"), - MovedAttribute("urlsplit", "urlparse", "urllib.parse"), - MovedAttribute("urlunparse", "urlparse", "urllib.parse"), - MovedAttribute("urlunsplit", "urlparse", "urllib.parse"), - MovedAttribute("quote", "urllib", "urllib.parse"), - MovedAttribute("quote_plus", "urllib", "urllib.parse"), - MovedAttribute("unquote", "urllib", "urllib.parse"), - MovedAttribute("unquote_plus", "urllib", "urllib.parse"), - MovedAttribute("urlencode", "urllib", "urllib.parse"), - MovedAttribute("splitquery", "urllib", "urllib.parse"), - MovedAttribute("splittag", "urllib", "urllib.parse"), - MovedAttribute("splituser", "urllib", "urllib.parse"), - MovedAttribute("uses_fragment", "urlparse", "urllib.parse"), - MovedAttribute("uses_netloc", "urlparse", "urllib.parse"), - MovedAttribute("uses_params", "urlparse", "urllib.parse"), - MovedAttribute("uses_query", "urlparse", "urllib.parse"), - MovedAttribute("uses_relative", "urlparse", "urllib.parse"), -] -for attr in _urllib_parse_moved_attributes: - setattr(Module_six_moves_urllib_parse, attr.name, attr) -del attr - -Module_six_moves_urllib_parse._moved_attributes = _urllib_parse_moved_attributes - -_importer._add_module(Module_six_moves_urllib_parse(__name__ + ".moves.urllib_parse"), - "moves.urllib_parse", "moves.urllib.parse") - - -class Module_six_moves_urllib_error(_LazyModule): - - """Lazy loading of moved objects in six.moves.urllib_error""" - - -_urllib_error_moved_attributes = [ - MovedAttribute("URLError", "urllib2", "urllib.error"), - MovedAttribute("HTTPError", "urllib2", "urllib.error"), - MovedAttribute("ContentTooShortError", "urllib", "urllib.error"), -] -for attr in _urllib_error_moved_attributes: - setattr(Module_six_moves_urllib_error, attr.name, attr) -del attr - -Module_six_moves_urllib_error._moved_attributes = _urllib_error_moved_attributes - -_importer._add_module(Module_six_moves_urllib_error(__name__ + ".moves.urllib.error"), - "moves.urllib_error", "moves.urllib.error") - - -class Module_six_moves_urllib_request(_LazyModule): - - """Lazy loading of moved objects in six.moves.urllib_request""" - - -_urllib_request_moved_attributes = [ - MovedAttribute("urlopen", "urllib2", "urllib.request"), - MovedAttribute("install_opener", "urllib2", "urllib.request"), - MovedAttribute("build_opener", "urllib2", "urllib.request"), - MovedAttribute("pathname2url", "urllib", "urllib.request"), - MovedAttribute("url2pathname", "urllib", "urllib.request"), - MovedAttribute("getproxies", "urllib", "urllib.request"), - MovedAttribute("Request", "urllib2", "urllib.request"), - MovedAttribute("OpenerDirector", "urllib2", "urllib.request"), - MovedAttribute("HTTPDefaultErrorHandler", "urllib2", "urllib.request"), - MovedAttribute("HTTPRedirectHandler", "urllib2", "urllib.request"), - MovedAttribute("HTTPCookieProcessor", "urllib2", "urllib.request"), - MovedAttribute("ProxyHandler", "urllib2", "urllib.request"), - MovedAttribute("BaseHandler", "urllib2", "urllib.request"), - MovedAttribute("HTTPPasswordMgr", "urllib2", "urllib.request"), - MovedAttribute("HTTPPasswordMgrWithDefaultRealm", "urllib2", "urllib.request"), - MovedAttribute("AbstractBasicAuthHandler", "urllib2", "urllib.request"), - MovedAttribute("HTTPBasicAuthHandler", "urllib2", "urllib.request"), - MovedAttribute("ProxyBasicAuthHandler", "urllib2", "urllib.request"), - MovedAttribute("AbstractDigestAuthHandler", "urllib2", "urllib.request"), - MovedAttribute("HTTPDigestAuthHandler", "urllib2", "urllib.request"), - MovedAttribute("ProxyDigestAuthHandler", "urllib2", "urllib.request"), - MovedAttribute("HTTPHandler", "urllib2", "urllib.request"), - MovedAttribute("HTTPSHandler", "urllib2", "urllib.request"), - MovedAttribute("FileHandler", "urllib2", "urllib.request"), - MovedAttribute("FTPHandler", "urllib2", "urllib.request"), - MovedAttribute("CacheFTPHandler", "urllib2", "urllib.request"), - MovedAttribute("UnknownHandler", "urllib2", "urllib.request"), - MovedAttribute("HTTPErrorProcessor", "urllib2", "urllib.request"), - MovedAttribute("urlretrieve", "urllib", "urllib.request"), - MovedAttribute("urlcleanup", "urllib", "urllib.request"), - MovedAttribute("URLopener", "urllib", "urllib.request"), - MovedAttribute("FancyURLopener", "urllib", "urllib.request"), - MovedAttribute("proxy_bypass", "urllib", "urllib.request"), -] -for attr in _urllib_request_moved_attributes: - setattr(Module_six_moves_urllib_request, attr.name, attr) -del attr - -Module_six_moves_urllib_request._moved_attributes = _urllib_request_moved_attributes - -_importer._add_module(Module_six_moves_urllib_request(__name__ + ".moves.urllib.request"), - "moves.urllib_request", "moves.urllib.request") - - -class Module_six_moves_urllib_response(_LazyModule): - - """Lazy loading of moved objects in six.moves.urllib_response""" - - -_urllib_response_moved_attributes = [ - MovedAttribute("addbase", "urllib", "urllib.response"), - MovedAttribute("addclosehook", "urllib", "urllib.response"), - MovedAttribute("addinfo", "urllib", "urllib.response"), - MovedAttribute("addinfourl", "urllib", "urllib.response"), -] -for attr in _urllib_response_moved_attributes: - setattr(Module_six_moves_urllib_response, attr.name, attr) -del attr - -Module_six_moves_urllib_response._moved_attributes = _urllib_response_moved_attributes - -_importer._add_module(Module_six_moves_urllib_response(__name__ + ".moves.urllib.response"), - "moves.urllib_response", "moves.urllib.response") - - -class Module_six_moves_urllib_robotparser(_LazyModule): - - """Lazy loading of moved objects in six.moves.urllib_robotparser""" - - -_urllib_robotparser_moved_attributes = [ - MovedAttribute("RobotFileParser", "robotparser", "urllib.robotparser"), -] -for attr in _urllib_robotparser_moved_attributes: - setattr(Module_six_moves_urllib_robotparser, attr.name, attr) -del attr - -Module_six_moves_urllib_robotparser._moved_attributes = _urllib_robotparser_moved_attributes - -_importer._add_module(Module_six_moves_urllib_robotparser(__name__ + ".moves.urllib.robotparser"), - "moves.urllib_robotparser", "moves.urllib.robotparser") - - -class Module_six_moves_urllib(types.ModuleType): - - """Create a six.moves.urllib namespace that resembles the Python 3 namespace""" - __path__ = [] # mark as package - parse = _importer._get_module("moves.urllib_parse") - error = _importer._get_module("moves.urllib_error") - request = _importer._get_module("moves.urllib_request") - response = _importer._get_module("moves.urllib_response") - robotparser = _importer._get_module("moves.urllib_robotparser") - - def __dir__(self): - return ['parse', 'error', 'request', 'response', 'robotparser'] - -_importer._add_module(Module_six_moves_urllib(__name__ + ".moves.urllib"), - "moves.urllib") - - -def add_move(move): - """Add an item to six.moves.""" - setattr(_MovedItems, move.name, move) - - -def remove_move(name): - """Remove item from six.moves.""" - try: - delattr(_MovedItems, name) - except AttributeError: - try: - del moves.__dict__[name] - except KeyError: - raise AttributeError("no such move, %r" % (name,)) - - -if PY3: - _meth_func = "__func__" - _meth_self = "__self__" - - _func_closure = "__closure__" - _func_code = "__code__" - _func_defaults = "__defaults__" - _func_globals = "__globals__" -else: - _meth_func = "im_func" - _meth_self = "im_self" - - _func_closure = "func_closure" - _func_code = "func_code" - _func_defaults = "func_defaults" - _func_globals = "func_globals" - - -try: - advance_iterator = next -except NameError: - def advance_iterator(it): - return it.next() -next = advance_iterator - - -try: - callable = callable -except NameError: - def callable(obj): - return any("__call__" in klass.__dict__ for klass in type(obj).__mro__) - - -if PY3: - def get_unbound_function(unbound): - return unbound - - create_bound_method = types.MethodType - - def create_unbound_method(func, cls): - return func - - Iterator = object -else: - def get_unbound_function(unbound): - return unbound.im_func - - def create_bound_method(func, obj): - return types.MethodType(func, obj, obj.__class__) - - def create_unbound_method(func, cls): - return types.MethodType(func, None, cls) - - class Iterator(object): - - def next(self): - return type(self).__next__(self) - - callable = callable -_add_doc(get_unbound_function, - """Get the function out of a possibly unbound function""") - - -get_method_function = operator.attrgetter(_meth_func) -get_method_self = operator.attrgetter(_meth_self) -get_function_closure = operator.attrgetter(_func_closure) -get_function_code = operator.attrgetter(_func_code) -get_function_defaults = operator.attrgetter(_func_defaults) -get_function_globals = operator.attrgetter(_func_globals) - - -if PY3: - def iterkeys(d, **kw): - return iter(d.keys(**kw)) - - def itervalues(d, **kw): - return iter(d.values(**kw)) - - def iteritems(d, **kw): - return iter(d.items(**kw)) - - def iterlists(d, **kw): - return iter(d.lists(**kw)) - - viewkeys = operator.methodcaller("keys") - - viewvalues = operator.methodcaller("values") - - viewitems = operator.methodcaller("items") -else: - def iterkeys(d, **kw): - return d.iterkeys(**kw) - - def itervalues(d, **kw): - return d.itervalues(**kw) - - def iteritems(d, **kw): - return d.iteritems(**kw) - - def iterlists(d, **kw): - return d.iterlists(**kw) - - viewkeys = operator.methodcaller("viewkeys") - - viewvalues = operator.methodcaller("viewvalues") - - viewitems = operator.methodcaller("viewitems") - -_add_doc(iterkeys, "Return an iterator over the keys of a dictionary.") -_add_doc(itervalues, "Return an iterator over the values of a dictionary.") -_add_doc(iteritems, - "Return an iterator over the (key, value) pairs of a dictionary.") -_add_doc(iterlists, - "Return an iterator over the (key, [values]) pairs of a dictionary.") - - -if PY3: - def b(s): - return s.encode("latin-1") - - def u(s): - return s - unichr = chr - import struct - int2byte = struct.Struct(">B").pack - del struct - byte2int = operator.itemgetter(0) - indexbytes = operator.getitem - iterbytes = iter - import io - StringIO = io.StringIO - BytesIO = io.BytesIO - _assertCountEqual = "assertCountEqual" - if sys.version_info[1] <= 1: - _assertRaisesRegex = "assertRaisesRegexp" - _assertRegex = "assertRegexpMatches" - else: - _assertRaisesRegex = "assertRaisesRegex" - _assertRegex = "assertRegex" -else: - def b(s): - return s - # Workaround for standalone backslash - - def u(s): - return unicode(s.replace(r'\\', r'\\\\'), "unicode_escape") - unichr = unichr - int2byte = chr - - def byte2int(bs): - return ord(bs[0]) - - def indexbytes(buf, i): - return ord(buf[i]) - iterbytes = functools.partial(itertools.imap, ord) - import StringIO - StringIO = BytesIO = StringIO.StringIO - _assertCountEqual = "assertItemsEqual" - _assertRaisesRegex = "assertRaisesRegexp" - _assertRegex = "assertRegexpMatches" -_add_doc(b, """Byte literal""") -_add_doc(u, """Text literal""") - - -def assertCountEqual(self, *args, **kwargs): - return getattr(self, _assertCountEqual)(*args, **kwargs) - - -def assertRaisesRegex(self, *args, **kwargs): - return getattr(self, _assertRaisesRegex)(*args, **kwargs) - - -def assertRegex(self, *args, **kwargs): - return getattr(self, _assertRegex)(*args, **kwargs) - - -if PY3: - exec_ = getattr(moves.builtins, "exec") - - def reraise(tp, value, tb=None): - if value is None: - value = tp() - if value.__traceback__ is not tb: - raise value.with_traceback(tb) - raise value - -else: - def exec_(_code_, _globs_=None, _locs_=None): - """Execute code in a namespace.""" - if _globs_ is None: - frame = sys._getframe(1) - _globs_ = frame.f_globals - if _locs_ is None: - _locs_ = frame.f_locals - del frame - elif _locs_ is None: - _locs_ = _globs_ - exec("""exec _code_ in _globs_, _locs_""") - - exec_("""def reraise(tp, value, tb=None): - raise tp, value, tb -""") - - -if sys.version_info[:2] == (3, 2): - exec_("""def raise_from(value, from_value): - if from_value is None: - raise value - raise value from from_value -""") -elif sys.version_info[:2] > (3, 2): - exec_("""def raise_from(value, from_value): - raise value from from_value -""") -else: - def raise_from(value, from_value): - raise value - - -print_ = getattr(moves.builtins, "print", None) -if print_ is None: - def print_(*args, **kwargs): - """The new-style print function for Python 2.4 and 2.5.""" - fp = kwargs.pop("file", sys.stdout) - if fp is None: - return - - def write(data): - if not isinstance(data, basestring): - data = str(data) - # If the file has an encoding, encode unicode with it. - if (isinstance(fp, file) and - isinstance(data, unicode) and - fp.encoding is not None): - errors = getattr(fp, "errors", None) - if errors is None: - errors = "strict" - data = data.encode(fp.encoding, errors) - fp.write(data) - want_unicode = False - sep = kwargs.pop("sep", None) - if sep is not None: - if isinstance(sep, unicode): - want_unicode = True - elif not isinstance(sep, str): - raise TypeError("sep must be None or a string") - end = kwargs.pop("end", None) - if end is not None: - if isinstance(end, unicode): - want_unicode = True - elif not isinstance(end, str): - raise TypeError("end must be None or a string") - if kwargs: - raise TypeError("invalid keyword arguments to print()") - if not want_unicode: - for arg in args: - if isinstance(arg, unicode): - want_unicode = True - break - if want_unicode: - newline = unicode("\n") - space = unicode(" ") - else: - newline = "\n" - space = " " - if sep is None: - sep = space - if end is None: - end = newline - for i, arg in enumerate(args): - if i: - write(sep) - write(arg) - write(end) -if sys.version_info[:2] < (3, 3): - _print = print_ - - def print_(*args, **kwargs): - fp = kwargs.get("file", sys.stdout) - flush = kwargs.pop("flush", False) - _print(*args, **kwargs) - if flush and fp is not None: - fp.flush() - -_add_doc(reraise, """Reraise an exception.""") - -if sys.version_info[0:2] < (3, 4): - def wraps(wrapped, assigned=functools.WRAPPER_ASSIGNMENTS, - updated=functools.WRAPPER_UPDATES): - def wrapper(f): - f = functools.wraps(wrapped, assigned, updated)(f) - f.__wrapped__ = wrapped - return f - return wrapper -else: - wraps = functools.wraps - - -def with_metaclass(meta, *bases): - """Create a base class with a metaclass.""" - # This requires a bit of explanation: the basic idea is to make a dummy - # metaclass for one level of class instantiation that replaces itself with - # the actual metaclass. - class metaclass(meta): - - def __new__(cls, name, this_bases, d): - return meta(name, bases, d) - return type.__new__(metaclass, 'temporary_class', (), {}) - - -def add_metaclass(metaclass): - """Class decorator for creating a class with a metaclass.""" - def wrapper(cls): - orig_vars = cls.__dict__.copy() - slots = orig_vars.get('__slots__') - if slots is not None: - if isinstance(slots, str): - slots = [slots] - for slots_var in slots: - orig_vars.pop(slots_var) - orig_vars.pop('__dict__', None) - orig_vars.pop('__weakref__', None) - return metaclass(cls.__name__, cls.__bases__, orig_vars) - return wrapper - - -def python_2_unicode_compatible(klass): - """ - A decorator that defines __unicode__ and __str__ methods under Python 2. - Under Python 3 it does nothing. - - To support Python 2 and 3 with a single code base, define a __str__ method - returning text and apply this decorator to the class. - """ - if PY2: - if '__str__' not in klass.__dict__: - raise ValueError("@python_2_unicode_compatible cannot be applied " - "to %s because it doesn't define __str__()." % - klass.__name__) - klass.__unicode__ = klass.__str__ - klass.__str__ = lambda self: self.__unicode__().encode('utf-8') - return klass - - -# Complete the moves implementation. -# This code is at the end of this module to speed up module loading. -# Turn this module into a package. -__path__ = [] # required for PEP 302 and PEP 451 -__package__ = __name__ # see PEP 366 @ReservedAssignment -if globals().get("__spec__") is not None: - __spec__.submodule_search_locations = [] # PEP 451 @UndefinedVariable -# Remove other six meta path importers, since they cause problems. This can -# happen if six is removed from sys.modules and then reloaded. (Setuptools does -# this for some reason.) -if sys.meta_path: - for i, importer in enumerate(sys.meta_path): - # Here's some real nastiness: Another "instance" of the six module might - # be floating around. Therefore, we can't use isinstance() to check for - # the six meta path importer, since the other six instance will have - # inserted an importer with different class. - if (type(importer).__name__ == "_SixMetaPathImporter" and - importer.name == __name__): - del sys.meta_path[i] - break - del i, importer -# Finally, add the importer to the meta path import hook. -sys.meta_path.append(_importer) diff -Nru jpylyzer-1.17.0/jpylyzer_win32.spec jpylyzer-1.18.0/jpylyzer_win32.spec --- jpylyzer-1.17.0/jpylyzer_win32.spec 2016-01-05 16:26:23.000000000 +0000 +++ jpylyzer-1.18.0/jpylyzer_win32.spec 2017-08-23 11:42:58.000000000 +0000 @@ -1,5 +1,5 @@ # -*- mode: python -*- -a = Analysis(['.\jpylyzer\jpylyzer.py'], +a = Analysis(['.\cli.py'], pathex=['.\jpylyzer'], hiddenimports=[], hookspath=None) diff -Nru jpylyzer-1.17.0/jpylyzer_win64.spec jpylyzer-1.18.0/jpylyzer_win64.spec --- jpylyzer-1.17.0/jpylyzer_win64.spec 1970-01-01 00:00:00.000000000 +0000 +++ jpylyzer-1.18.0/jpylyzer_win64.spec 2017-08-23 11:42:58.000000000 +0000 @@ -0,0 +1,28 @@ +# -*- mode: python -*- +a = Analysis(['.\cli.py'], + pathex=['.\jpylyzer'], + hiddenimports=[], + hookspath=None) +pyz = PYZ(a.pure) +exe = EXE(pyz, + a.scripts, + exclude_binaries=1, + name=os.path.join('build\\pyi.win64\\jpylyzer', 'jpylyzer.exe'), + debug=False, + strip=None, + upx=True, + console=True ) +coll = COLLECT(exe, + a.binaries + + [('./license/LICENSE.txt','LICENSE','DATA')], + [('./doc/jpylyzerUserManual.html','./doc/jpylyzerUserManual.html','DATA')], + [('./example_files/balloon.jp2','./example_files/balloon.jp2','DATA')], + [('./example_files/balloon_trunc1.jp2','./example_files/balloon_trunc1.jp2','DATA')], + [('./example_files/balloon_trunc2.jp2','./example_files/balloon_trunc2.jp2','DATA')], + [('./example_files/balloon_trunc3.jp2','./example_files/balloon_trunc3.jp2','DATA')], + [('./example_files/readme.txt','./example_files/readme.txt','DATA')], + a.zipfiles, + a.datas, + strip=None, + upx=True, + name=os.path.join('dist_win64', 'jpylyzer')) diff -Nru jpylyzer-1.17.0/package-pypi.sh jpylyzer-1.18.0/package-pypi.sh --- jpylyzer-1.17.0/package-pypi.sh 1970-01-01 00:00:00.000000000 +0000 +++ jpylyzer-1.18.0/package-pypi.sh 2017-08-23 11:42:58.000000000 +0000 @@ -0,0 +1,36 @@ +#!/bin/bash + +# This script creates a wheel distribution and uploads it to PyPi +# +# Requirements: +# +# twine https://pypi.python.org/pypi/twine/1.9.1 (pip install twine) +# wheel https://pypi.python.org/pypi/wheel (pip install wheel) + +# Repository: this is usually pypi; for testing use testpypi +# The corresponding repository URLS are defined in config file ~/.pypirc +#repository=testpypi +repository=pypi + +# Working directory +workDir=$PWD + +# Dist directory +distDir=$workDir"/dist/" + +# Clear contents of dist dir if it exists +if [ -d "$distDir" ]; then + rm -r "$distDir" +fi + +# Create wheel +python setup.py sdist bdist_wheel --universal + +# Upload package if wheel build was successful; if not show error message +if [ $? -eq 0 ]; then + twine upload --repository $repository dist/* +else + echo "Wheel build not successful quitting now ..." +fi + + diff -Nru jpylyzer-1.17.0/README.md jpylyzer-1.18.0/README.md --- jpylyzer-1.17.0/README.md 2016-01-05 16:26:23.000000000 +0000 +++ jpylyzer-1.18.0/README.md 2017-08-23 11:42:58.000000000 +0000 @@ -1,4 +1,4 @@ -#jpylyzer +# jpylyzer ## About *Jpylyzer* is a JP2 [(JPEG 2000 Part 1)][2] image validator and properties extractor. Its development was partially supported by the [SCAPE][4] Project. The SCAPE project is co-funded by the European Union under FP7 ICT-2009.4.1 (Grant Agreement number 270137). @@ -26,12 +26,12 @@ ### Usage - usage: jpylyzer.py [-h] [--verbose] [--recurse] [--wrapper] [--nullxml] + usage: jpylyzer [-h] [--verbose] [--recurse] [--wrapper] [--nullxml] [--nopretty] [--version] jp2In [jp2In ...] ### Positional arguments -`jp2In` : input JP2 image(s), may be one or more (whitespace-separated) path expressions; prefix wildcard (\*) with backslash (\\) in Linux.. +`jp2In` : input JP2 image(s), may be one or more (whitespace-separated) path expressions; prefix wildcard (\*) with backslash (\\) in Linux. ### Optional arguments @@ -50,6 +50,7 @@ `--nopretty` : suppress pretty-printing of XML output ## Output + Output is directed to the standard output device (*stdout*). ### Example @@ -77,21 +78,22 @@ (Outline only, this section is under development and needs more detail!). 1. Make changes to code -1. Update version number in *jpylyzer.py*, *setup.py* and *sonar-project.properties* (do we really need last file?) +1. Update version number in *jpylyzer.py*, and *sonar-project.properties* (do we really need last file?) 1. In case of changes to command-line interface, update [jpylyzer.pod](debian/jpylyzer.pod) file in *Debian* folder. 1. Create new entry in changelog using `dch -i`; then manually update version number, and create list of changes. 1. Update [User Manual](doc/jpylyzerUserManual.md) if necessary 1. Commit all changes 1. Add tag and commit +1. Create and upload PyPi packages by running [`package-pypi.sh`](./package-pypi.sh) 1. Build Linux packages using [instructions here](vagrant) -1. Build Windows binaries -1. Upload Linux/Windows packages to BinTray -1. Website: update *binVersion* in *_config.yml* (this updates the links to all packages on BinTray to the correct version) +1. Build Windows binaries using [instructions here](./BUILD_HOWTO_WINDOWS.md) +1. Go to [*Latest Release*](https://github.com/openpreserve/jpylyzer/releases/latest) and click on the *Edit* button +1. Upload Linux/Windows packages to the release by dragging them to the *Attach Binaries* field at the bottom +1. Website: update *binVersion* in *_config.yml* (this updates the links to all packages to the correct version) 1. Website: write short release note 1. Commit changes to website 1. Spread the word! - [1]: http://jpylyzer.openpreservation.org//jpylyzerUserManual.html [2]: http://www.jpeg.org/public/15444-1annexi.pdf [3]: http://www.itu.int/rec/T-REC-T.800/en diff -Nru jpylyzer-1.17.0/setup.py jpylyzer-1.18.0/setup.py --- jpylyzer-1.17.0/setup.py 2016-01-05 16:26:23.000000000 +0000 +++ jpylyzer-1.18.0/setup.py 2017-08-23 11:42:58.000000000 +0000 @@ -1,17 +1,37 @@ #!/usr/bin/env python +"""Setup script for jpylyzer""" +import codecs +import os +import re +from setuptools import setup, find_packages -from distutils.core import setup +def read(*parts): + """Read file and return contents""" + path = os.path.join(os.path.dirname(__file__), *parts) + with codecs.open(path, encoding='utf-8') as fobj: + return fobj.read() -# TO DO: figure out how to import version number automatically from code! +def find_version(*file_paths): + """Find and return version number""" + version_file = read(*file_paths) + version_match = re.search(r"^__version__ = ['\"]([^'\"]*)['\"]", version_file, re.M) + if version_match: + return version_match.group(1) + raise RuntimeError("Unable to find version string.") -readme = open('README.md', 'r') -README_TEXT = readme.read() -readme.close() +INSTALL_REQUIRES = ['setuptools', 'six'] +PYTHON_REQUIRES = '>=2.7, !=3.0.*, !=3.1.*, <4' + +README = open('README.md', 'r') +README_TEXT = README.read() +README.close() setup(name='jpylyzer', - packages=['jpylyzer'], - version='1.17.0', + packages=find_packages(), + version=find_version('jpylyzer', 'jpylyzer.py'), license='LGPL', + install_requires=INSTALL_REQUIRES, + python_requires=PYTHON_REQUIRES, platforms=['POSIX', 'Windows'], description='JP2 (JPEG 2000 Part 1) image validator and properties extractor', long_description=README_TEXT, @@ -19,5 +39,18 @@ author_email='johan.vanderknijff@kb.nl', maintainer='Johan van der Knijff', maintainer_email='johan.vanderknijff@kb.nl', - url='http://jpylyzer.openpreservation.org/' - ) + url='http://jpylyzer.openpreservation.org/', + download_url='https://github.com/openpreserve/jpylyzer/archive/' \ + + find_version('jpylyzer', 'jpylyzer.py') + '.tar.gz', + package_data={'jpylyzer': ['*.*']}, + entry_points={'console_scripts': [ + 'jpylyzer = jpylyzer.jpylyzer:main', + ]}, + classifiers=[ + 'Environment :: Console', + 'Programming Language :: Python :: 2', + 'Programming Language :: Python :: 2.7', + 'Programming Language :: Python :: 3', + ] + ) + diff -Nru jpylyzer-1.17.0/sonar-project.properties jpylyzer-1.18.0/sonar-project.properties --- jpylyzer-1.17.0/sonar-project.properties 2016-01-05 16:26:23.000000000 +0000 +++ jpylyzer-1.18.0/sonar-project.properties 2017-08-23 11:42:58.000000000 +0000 @@ -1,6 +1,6 @@ sonar.projectKey=org.opf-labs:jpylyzer sonar.projectName=jpylyzer -sonar.projectVersion=1.17.0 +sonar.projectVersion=1.18.0 sonar.sources=jpylyzer diff -Nru jpylyzer-1.17.0/travis-build.sh jpylyzer-1.18.0/travis-build.sh --- jpylyzer-1.17.0/travis-build.sh 2016-01-05 16:26:23.000000000 +0000 +++ jpylyzer-1.18.0/travis-build.sh 2017-08-23 11:42:58.000000000 +0000 @@ -1,3 +1,4 @@ #!/bin/sh -python Makespec.py --onefile ./jpylyzer/jpylyzer.py -python pyinstaller.py jpylyzer.spec +pyi-makespec --onefile --name=jpylyzer ./cli.py +pyinstaller jpylyzer.spec + diff -Nru jpylyzer-1.17.0/vagrant/precise32/bootstrap.sh jpylyzer-1.18.0/vagrant/precise32/bootstrap.sh --- jpylyzer-1.17.0/vagrant/precise32/bootstrap.sh 2016-01-05 16:26:23.000000000 +0000 +++ jpylyzer-1.18.0/vagrant/precise32/bootstrap.sh 2017-08-23 11:42:58.000000000 +0000 @@ -7,4 +7,5 @@ sudo apt-get install -y python-pip sudo apt-get install -y python-dev sudo pip install --use-mirrors pyinstaller +sudo pip install --use-mirrors six diff -Nru jpylyzer-1.17.0/vagrant/precise32/buildjpylyzer.sh jpylyzer-1.18.0/vagrant/precise32/buildjpylyzer.sh --- jpylyzer-1.17.0/vagrant/precise32/buildjpylyzer.sh 2016-01-05 16:26:23.000000000 +0000 +++ jpylyzer-1.18.0/vagrant/precise32/buildjpylyzer.sh 2017-08-23 11:42:58.000000000 +0000 @@ -1,7 +1,7 @@ #!/bin/bash ## Fetch and build jpylyzer -git clone https://github.com/openpreserve/jpylyzer.git +git clone -b master https://github.com/openpreserve/jpylyzer.git cd jpylyzer dpkg-buildpackage -tc lintian ../jpylyzer_*.deb diff -Nru jpylyzer-1.17.0/vagrant/precise64/bootstrap.sh jpylyzer-1.18.0/vagrant/precise64/bootstrap.sh --- jpylyzer-1.17.0/vagrant/precise64/bootstrap.sh 2016-01-05 16:26:23.000000000 +0000 +++ jpylyzer-1.18.0/vagrant/precise64/bootstrap.sh 2017-08-23 11:42:58.000000000 +0000 @@ -7,4 +7,5 @@ sudo apt-get install -y python-pip sudo apt-get install -y python-dev sudo pip install --use-mirrors pyinstaller +sudo pip install --use-mirrors six diff -Nru jpylyzer-1.17.0/vagrant/precise64/buildjpylyzer.sh jpylyzer-1.18.0/vagrant/precise64/buildjpylyzer.sh --- jpylyzer-1.17.0/vagrant/precise64/buildjpylyzer.sh 2016-01-05 16:26:23.000000000 +0000 +++ jpylyzer-1.18.0/vagrant/precise64/buildjpylyzer.sh 2017-08-23 11:42:58.000000000 +0000 @@ -1,7 +1,7 @@ #!/bin/bash ## Fetch and build jpylyzer -git clone https://github.com/openpreserve/jpylyzer.git +git clone -b master https://github.com/openpreserve/jpylyzer.git cd jpylyzer dpkg-buildpackage -tc lintian ../jpylyzer_*.deb diff -Nru jpylyzer-1.17.0/vagrant/README.md jpylyzer-1.18.0/vagrant/README.md --- jpylyzer-1.17.0/vagrant/README.md 2016-01-05 16:26:23.000000000 +0000 +++ jpylyzer-1.18.0/vagrant/README.md 2017-08-23 11:42:58.000000000 +0000 @@ -4,7 +4,7 @@ **Note:** the instructions below always result in a jpylyzer build that is based on the source code in the *remote* Github repo at , *not* on the local code! -##Step 1: install virtualisation software +## Step 1: install virtualisation software * Install [VirtualBox](https://www.virtualbox.org/) * Install VirtualBox Extension Pack (download link [here](https://www.virtualbox.org/)) diff -Nru jpylyzer-1.17.0/vagrant/trusty32/bootstrap.sh jpylyzer-1.18.0/vagrant/trusty32/bootstrap.sh --- jpylyzer-1.17.0/vagrant/trusty32/bootstrap.sh 2016-01-05 16:26:23.000000000 +0000 +++ jpylyzer-1.18.0/vagrant/trusty32/bootstrap.sh 2017-08-23 11:42:58.000000000 +0000 @@ -7,4 +7,5 @@ sudo apt-get install -y python-pip sudo apt-get install -y python-dev sudo pip install --use-mirrors pyinstaller +sudo pip install --use-mirrors six diff -Nru jpylyzer-1.17.0/vagrant/trusty32/buildjpylyzer.sh jpylyzer-1.18.0/vagrant/trusty32/buildjpylyzer.sh --- jpylyzer-1.17.0/vagrant/trusty32/buildjpylyzer.sh 2016-01-05 16:26:23.000000000 +0000 +++ jpylyzer-1.18.0/vagrant/trusty32/buildjpylyzer.sh 2017-08-23 11:42:58.000000000 +0000 @@ -1,7 +1,7 @@ #!/bin/bash ## Fetch and build jpylyzer -git clone https://github.com/openpreserve/jpylyzer.git +git clone -b master https://github.com/openpreserve/jpylyzer.git cd jpylyzer dpkg-buildpackage -tc lintian ../jpylyzer_*.deb diff -Nru jpylyzer-1.17.0/vagrant/trusty64/bootstrap.sh jpylyzer-1.18.0/vagrant/trusty64/bootstrap.sh --- jpylyzer-1.17.0/vagrant/trusty64/bootstrap.sh 2016-01-05 16:26:23.000000000 +0000 +++ jpylyzer-1.18.0/vagrant/trusty64/bootstrap.sh 2017-08-23 11:42:58.000000000 +0000 @@ -7,4 +7,5 @@ sudo apt-get install -y python-pip sudo apt-get install -y python-dev sudo pip install --use-mirrors pyinstaller +sudo pip install --use-mirrors six diff -Nru jpylyzer-1.17.0/vagrant/trusty64/buildjpylyzer.sh jpylyzer-1.18.0/vagrant/trusty64/buildjpylyzer.sh --- jpylyzer-1.17.0/vagrant/trusty64/buildjpylyzer.sh 2016-01-05 16:26:23.000000000 +0000 +++ jpylyzer-1.18.0/vagrant/trusty64/buildjpylyzer.sh 2017-08-23 11:42:58.000000000 +0000 @@ -1,7 +1,7 @@ #!/bin/bash ## Fetch and build jpylyzer -git clone https://github.com/openpreserve/jpylyzer.git +git clone -b master https://github.com/openpreserve/jpylyzer.git cd jpylyzer dpkg-buildpackage -tc lintian ../jpylyzer_*.deb diff -Nru jpylyzer-1.17.0/zipdir.py jpylyzer-1.18.0/zipdir.py --- jpylyzer-1.17.0/zipdir.py 2016-01-05 16:26:23.000000000 +0000 +++ jpylyzer-1.18.0/zipdir.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,67 +0,0 @@ -#!/usr/bin/env python - -# Modified from: -# http://stackoverflow.com/questions/1855095/how-to-create-a-zip-archive-of-a-directory-in-python?answertab=active#tab-top -# -# and: -# http://pymotw.com/2/zipfile/ - -import os -import zipfile -import argparse - -try: - import zlib - compression = zipfile.ZIP_DEFLATED -except: - compression = zipfile.ZIP_STORED - - -def zipdir(path, zip): - - nameBase = os.path.basename(path) - - for root, dirs, files in os.walk(path): - for file in files: - - baseNameRoot = os.path.basename(root) - - if baseNameRoot == nameBase: - archName = file - else: - archName = os.path.basename(root) + "//" + file - - print archName - - zip.write( - os.path.join(root, file), archName, compress_type=compression) - - -def parseCommandLine(): - # Create parser - parser = argparse.ArgumentParser( - description="zip all files in directory tree") - - # Add arguments - parser.add_argument('dirIn', action="store", help="input directory") - parser.add_argument('fileOut', action="store", help="output file") - - # Parse arguments - args = parser.parse_args() - - return(args) - - -def main(): - # Get input from command line - args = parseCommandLine() - dirIn = args.dirIn - fileOut = os.path.abspath(args.fileOut) - - zip = zipfile.ZipFile(fileOut, 'w') - zipdir(dirIn, zip) - - zip.close() - -if __name__ == "__main__": - main()