diff -Nru snapcraft-2.27.1+17.04/CONTRIBUTING.md snapcraft-2.28+17.04/CONTRIBUTING.md --- snapcraft-2.27.1+17.04/CONTRIBUTING.md 2016-11-02 08:25:17.000000000 +0000 +++ snapcraft-2.28+17.04/CONTRIBUTING.md 2017-03-22 12:31:58.000000000 +0000 @@ -9,8 +9,11 @@ give us permission to use your contributions. 1. Make sure a [Snapcraft bug][2] is filed for the bug you're about to fix, or - feature you're about to add. If it's a feature, mark the status as - "Wishlist." + feature you're about to add. This is not just paperwork required in order to + land something: this is where you state what's happening, and why it's a + shortcoming. Doing this before you start working on a fix also gives the + Snapcraft team a chance to give you feedback and advice, which saves time + for everyone! 2. We use a forking, feature-based workflow. @@ -20,18 +23,38 @@ changes are complete (for information about running the tests, see the [HACKING][3] document). -3. Squash commits into one, well-formatted commit. Mention the bug being - resolved in the commit message on a line all by itself like `LP: #`. - - If you really feel like there should be more than one commit in your branch, - then you're probably trying to introduce more than one feature and you should - make another branch for it. +3. Squash commits into one, well-formatted commit. If you really feel like there + should be more than one commit in your branch, then you're probably trying to + introduce more than one feature and you should make another branch for + it. + + This is important: your commit diff says what changed, but only the commit + message can say why the change was necessary. In an effort to take good care + of our `git log`, we try to follow this template for commit messages: + + + ``` + : lower-case summary of changes + + More detailed explanatory text, if necessary. Wrap it to 72 characters. + Think of this like an email, where you have a subject line and a body. + Make sure you mention the bug being fixed on a line all by itself at the + end, like so: + + LP: # + ``` + + Try to keep the summary to around 50 characters, and use the imperative mood. + A good rule of thumb is that, if you extract the `` from + the summary, it should be able to complete the following sentence: + + ``` + If applied, this commit will . + ``` 4. Submit a pull request to get changes from your branch into master. Mention - which bug is being resolved. - - If you want to get the change into 1.x as well, make a note of it on the pull - request and it can be cherry-picked after the merge. + which bug is being resolved in the description of the pull request (bonus + points if it's a hyperlink to the bug itself). [1]: http://www.ubuntu.com/legal/contributors/ [2]: https://bugs.launchpad.net/snapcraft diff -Nru snapcraft-2.27.1+17.04/debian/changelog snapcraft-2.28+17.04/debian/changelog --- snapcraft-2.27.1+17.04/debian/changelog 2017-02-17 20:07:16.000000000 +0000 +++ snapcraft-2.28+17.04/debian/changelog 2017-03-23 19:11:14.000000000 +0000 @@ -1,4 +1,84 @@ -snapcraft (2.27.1+17.04) zesty; urgency=medium +snapcraft (2.28+17.04) zesty; urgency=medium + + [ Sergio Schvezov ] + * python plugin: use stage headers if applicable. (#1156) + * docs: use correct target to generate docs. (#1159) + * packaging: snapcraft as a snap (#1158) + * tour: make it work when its a snap. (#1217) + * cleanbuild: don't copy cache into container. (#1216) + * kernel plugin: fix modprobe output parsing. (#1208) + * kernel plugin: use default per arch targets. (#1209) + * python plugin: support pbr/setup.cfg projects. (#1202) + * cleanbuild: packaging independent detection. (#1199) + * tests: remove repo.Ubuntu patch for plugins. (#1194) + * store: enable delta uploads by default. (#1196) + * repo: refactor into a package. (#1192) + * core: resolve ld link first. (#1189) + * store: enable retries for store calls. (#1184) + * New upstream release 2.28 (LP: #1675391) + + [ Joe Talbott ] + * store: set User-Agent header in store requests. (#1188) + * store: Add track support to commands. (#1161) + * state: asset tracking - store versions of stage-packages. (#1142) + * store: remove 'Series' from channel map output (#1151) + * repo: support versioned stage-packages. (#1157) + * parser: use the project loader code to find the origin's snapcraft.yaml (#1141) + * cli: rename `history` to 'list-revisions' with `revisions` alias. (#1160) + * repo: add version support for build-packages (#1185) + * project: use a more likely to be found global build-package (#1218) + + [ Paolo Pisati ] + * demos: add the minimal config changes to boot a dragonboard410c (#1147) + * kernel plugin: if no dtb target is set when the arch is arm.* build them all. (#1148) + * kernel plugin: rework MAKEFLAGS from the environment (#1150) + + [ Kyle Fazzari ] + * contribution guide: add commit message template (#1153) + * demos: make ROS demos support exiting after success. (#1201) + * catkin plugin: support building with an underlay. (#1140) + * demos: add ROS content sharing demo. (#1186) + * repo: fixup with python, not sed (#1181) + * repo: ignore symlinks to libc. (#1174) + * sources: update documentation for source-subdir (#1177) + + [ Olivier Tilloy ] + * project: expose parallel_build_count to scriptlets. (#1154) + + [ Leo Arias ] + * docs: build and push the API docs to github pages. (#1126) + * tests: pass the autopkgtest secret to the container (#1162) + * tests: update the ftp source for integration test (#1169) + * ci: install wget in the container that triggers the beta tests. (#1167) + * demos: add a message to exit the mosquitto subscriber. (#1173) + * demos: add the mount-observe plug to be able to run godd. (#1172) + * tests: support bzr branches for external tests. (#1128) + * docs: update the directory where the API pages are generated (#1163) + * demos: remove the tomcat demo snap. (#1176) + * tests: make the kernel unit tests architecture independent. (#1178) + * tests: support snap directory in external tests (#1180) + * tests: run the master tests against the staging server (#1164) + * tests: take into account the new current link. (#1187) + * ci: run the CLA check in a docker container. (#1191) + * tests: add manual tests for the kernel snaps (#1198) + * ci: allow to run individual autopkgtest suites (#1200) + * tests: expect failures for the tests that can't be run in arm64. (#1145) + + [ Jonathan Cave ] + * plainbox-provider plugin: run validate (#1095) + + [ Michael Hudson-Doyle ] + * core: fix symlink resolution in get_core_dynamic_linker. (#1170) + + [ pachulo ] + * sources: add optional "source-checksum" property (#980) + + [ Colin Watson ] + * godeps plugin: add git to build-packages. (#1179) + + -- Sergio Schvezov Thu, 23 Mar 2017 16:11:14 -0300 + +snapcraft (2.27.1) xenial; urgency=medium [ Kyle Fazzari ] * catkin plugin: produce build-ready staging area. (#1130) diff -Nru snapcraft-2.27.1+17.04/debian/tests/control snapcraft-2.28+17.04/debian/tests/control --- snapcraft-2.27.1+17.04/debian/tests/control 2017-01-28 06:12:26.000000000 +0000 +++ snapcraft-2.28+17.04/debian/tests/control 2017-03-22 12:31:58.000000000 +0000 @@ -13,6 +13,7 @@ python-flake8, python3-fixtures, python3-pexpect, + python3-pyftpdlib, python3-testscenarios Tests: snapstests diff -Nru snapcraft-2.27.1+17.04/demos/96boards-kernel/snap/snapcraft.yaml snapcraft-2.28+17.04/demos/96boards-kernel/snap/snapcraft.yaml --- snapcraft-2.27.1+17.04/demos/96boards-kernel/snap/snapcraft.yaml 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/demos/96boards-kernel/snap/snapcraft.yaml 2017-03-23 01:42:23.000000000 +0000 @@ -14,7 +14,14 @@ kconfigs: - CONFIG_LOCALVERSION="-96boards" - CONFIG_DEBUG_INFO=n + - CONFIG_EXT4_FS=y + - CONFIG_BLK_DEV_LOOP=y + - CONFIG_BLK_DEV_LOOP_MIN_COUNT=256 - CONFIG_SQUASHFS=m + - CONFIG_MMC_SDHCI=y + - CONFIG_MMC_SDHCI_PLTFM=y + - CONFIG_MMC_SDHCI_MSM=y + - CONFIG_PINCTRL_MSM8916=y kernel-initrd-modules: - squashfs kernel-initrd-firmware: @@ -29,4 +36,4 @@ 410c-firmware: plugin: tar-content source: firmware.tar # from developer.qualcomm.com v1.2 - destination: lib/firmware + destination: firmware diff -Nru snapcraft-2.27.1+17.04/demos/godd/snap/snapcraft.yaml snapcraft-2.28+17.04/demos/godd/snap/snapcraft.yaml --- snapcraft-2.27.1+17.04/demos/godd/snap/snapcraft.yaml 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/demos/godd/snap/snapcraft.yaml 2017-03-22 12:31:58.000000000 +0000 @@ -9,6 +9,7 @@ apps: godd: command: bin/godd + plugs: [mount-observe] parts: godd: diff -Nru snapcraft-2.27.1+17.04/demos/mosquitto/launchers/subscribe.py snapcraft-2.28+17.04/demos/mosquitto/launchers/subscribe.py --- snapcraft-2.27.1+17.04/demos/mosquitto/launchers/subscribe.py 2016-11-02 08:25:17.000000000 +0000 +++ snapcraft-2.28+17.04/demos/mosquitto/launchers/subscribe.py 2017-03-22 12:31:58.000000000 +0000 @@ -24,9 +24,10 @@ # Ignore the unused arguments. del unused1, unused2 _log(message.topic + ' ' + str(message.payload)) - # XXX Exit on first message simplifyies the tests a lot, so this - # subscriber can get only one message. --elopio - 2016-05-02 - sys.exit(0) + if message.payload == b'exit': + # XXX Exit on response to a received message simplifyies the tests + # --elopio - 2017-03-04 + sys.exit(0) def _log(message): diff -Nru snapcraft-2.27.1+17.04/demos/ros/src/listener/scripts/listener_node snapcraft-2.28+17.04/demos/ros/src/listener/scripts/listener_node --- snapcraft-2.27.1+17.04/demos/ros/src/listener/scripts/listener_node 2016-11-02 08:25:17.000000000 +0000 +++ snapcraft-2.28+17.04/demos/ros/src/listener/scripts/listener_node 2017-03-22 12:31:58.000000000 +0000 @@ -3,9 +3,15 @@ import rospy from std_msgs.msg import String + def callback(data): rospy.loginfo('I heard %s', data.data) + if rospy.get_param('~exit-after-receive', False): + rospy.signal_shutdown( + 'Requested to exit after message received. Exiting now.') + + def listener(): rospy.init_node('listener') diff -Nru snapcraft-2.27.1+17.04/demos/ros/src/listener/talk_and_listen.launch snapcraft-2.28+17.04/demos/ros/src/listener/talk_and_listen.launch --- snapcraft-2.27.1+17.04/demos/ros/src/listener/talk_and_listen.launch 2016-11-02 08:25:17.000000000 +0000 +++ snapcraft-2.28+17.04/demos/ros/src/listener/talk_and_listen.launch 2017-03-22 12:31:58.000000000 +0000 @@ -1,9 +1,13 @@ + + + required = "true" output = "screen" /> + required = "true" output = "screen"> + diff -Nru snapcraft-2.27.1+17.04/demos/shared-ros/README.md snapcraft-2.28+17.04/demos/shared-ros/README.md --- snapcraft-2.27.1+17.04/demos/shared-ros/README.md 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/demos/shared-ros/README.md 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,20 @@ +Shared ROS Demo +=============== + +This demo is actually made up of two snaps: + + - **ros-base**: A minimal base ROS system. This includes `roscore` as well as + typical utilities like `rosrun` and `roslaunch`. + - **ros-app**: A more streamlined ROS snap that doesn't include the components + included in ros-base, but contains a talker/listener system. It requires + the staging area from ros-base to be tarred up and used as a part during + build-time, and requires ros-base to be installed and sharing its content + during run-time. + + +## Build procedure + +1. Build ros-base snap. +2. Tar ros-base staging area: `tar czf ros-base.tar.bz2 stage/` +3. Copy that tarball into ros-app (as required by its `ros-base` part). +4. Build ros-app. diff -Nru snapcraft-2.27.1+17.04/demos/shared-ros/ros-app/bin/run-system snapcraft-2.28+17.04/demos/shared-ros/ros-app/bin/run-system --- snapcraft-2.27.1+17.04/demos/shared-ros/ros-app/bin/run-system 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/demos/shared-ros/ros-app/bin/run-system 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,29 @@ +#!/bin/bash + +case $SNAP_ARCH in +amd64) + TRIPLET="x86_64-linux-gnu" + ;; +armhf) + TRIPLET="arm-linux-gnueabihf" + ;; +arm64) + TRIPLET="aarch64-linux-gnu" + ;; +*) + TRIPLET="$SNAP_ARCH-linux-gnu" + ;; +esac + +export ROS_BASE=$SNAP/ros-base + +# Add ros-base to the PYTHONPATH +export PYTHONPATH=$PYTHONPATH:$ROS_BASE/usr/lib/python2.7/dist-packages + +# Add ros-base to LD_LIBRARY_PATH +export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$ROS_BASE/lib +export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$ROS_BASE/lib/$TRIPLET +export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$ROS_BASE/usr/lib +export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$ROS_BASE/usr/lib/$TRIPLET + +roslaunch listener talk_and_listen.launch "$@" diff -Nru snapcraft-2.27.1+17.04/demos/shared-ros/ros-app/snap/snapcraft.yaml snapcraft-2.28+17.04/demos/shared-ros/ros-app/snap/snapcraft.yaml --- snapcraft-2.27.1+17.04/demos/shared-ros/ros-app/snap/snapcraft.yaml 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/demos/shared-ros/ros-app/snap/snapcraft.yaml 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,51 @@ +name: ros-app +version: '1.0' +grade: stable +confinement: strict +summary: ROS App Snap +description: Contains talker/listener ROS packages and a .launch file. + +plugs: + # Mount the content shared from ros-base into $SNAP/ros-base + ros-base: + content: ros-base-v1 + interface: content + target: /ros-base + +apps: + launch-project: + command: run-system + plugs: [network, network-bind, ros-base] + +parts: + # The `source` here is the tarred staging area of the ros-base snap. + ros-base: + plugin: dump + source: ros-base.tar.bz2 + + # This is only used for building-- filter it out of the final snap. + prime: [-*] + + ros-app: + plugin: catkin + rosdistro: kinetic + include-roscore: false + underlay: + # Build-time location of the underlay + build-path: $SNAPCRAFT_STAGE/opt/ros/kinetic + + # Run-time location of the underlay + run-path: $SNAP/ros-base/opt/ros/kinetic + catkin-packages: + - talker + - listener + after: [ros-base] + + run-system: + plugin: dump + stage: [bin/run-system] + prime: [bin/run-system] + + mountpoint: + plugin: nil + install: mkdir $SNAPCRAFT_PART_INSTALL/ros-base diff -Nru snapcraft-2.27.1+17.04/demos/shared-ros/ros-app/src/CMakeLists.txt snapcraft-2.28+17.04/demos/shared-ros/ros-app/src/CMakeLists.txt --- snapcraft-2.27.1+17.04/demos/shared-ros/ros-app/src/CMakeLists.txt 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/demos/shared-ros/ros-app/src/CMakeLists.txt 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,47 @@ +# toplevel CMakeLists.txt for a catkin workspace +# catkin/cmake/toplevel.cmake + +cmake_minimum_required(VERSION 2.8.3) + +# optionally provide a cmake file in the workspace to override arbitrary stuff +include(workspace.cmake OPTIONAL) + +set(CATKIN_TOPLEVEL TRUE) + +# include catkin directly or via find_package() +if(EXISTS "${CMAKE_SOURCE_DIR}/catkin/cmake/all.cmake" AND EXISTS "${CMAKE_SOURCE_DIR}/catkin/CMakeLists.txt") + set(catkin_EXTRAS_DIR "${CMAKE_SOURCE_DIR}/catkin/cmake") + # include all.cmake without add_subdirectory to let it operate in same scope + include(catkin/cmake/all.cmake NO_POLICY_SCOPE) + add_subdirectory(catkin) + +else() + # use either CMAKE_PREFIX_PATH explicitly passed to CMake as a command line argument + # or CMAKE_PREFIX_PATH from the environment + if(NOT DEFINED CMAKE_PREFIX_PATH) + if(NOT "$ENV{CMAKE_PREFIX_PATH}" STREQUAL "") + string(REPLACE ":" ";" CMAKE_PREFIX_PATH $ENV{CMAKE_PREFIX_PATH}) + endif() + endif() + + # list of catkin workspaces + set(catkin_search_path "") + foreach(path ${CMAKE_PREFIX_PATH}) + if(EXISTS "${path}/.CATKIN_WORKSPACE") + list(FIND catkin_search_path ${path} _index) + if(_index EQUAL -1) + list(APPEND catkin_search_path ${path}) + endif() + endif() + endforeach() + + # search for catkin in all workspaces + set(CATKIN_TOPLEVEL_FIND_PACKAGE TRUE) + find_package(catkin REQUIRED + NO_POLICY_SCOPE + PATHS ${catkin_search_path} + NO_DEFAULT_PATH NO_CMAKE_FIND_ROOT_PATH) + unset(CATKIN_TOPLEVEL_FIND_PACKAGE) +endif() + +catkin_workspace() diff -Nru snapcraft-2.27.1+17.04/demos/shared-ros/ros-app/src/listener/CMakeLists.txt snapcraft-2.28+17.04/demos/shared-ros/ros-app/src/listener/CMakeLists.txt --- snapcraft-2.27.1+17.04/demos/shared-ros/ros-app/src/listener/CMakeLists.txt 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/demos/shared-ros/ros-app/src/listener/CMakeLists.txt 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,19 @@ +cmake_minimum_required(VERSION 2.8.3) +project(listener) + +find_package(catkin REQUIRED COMPONENTS + rospy + std_msgs +) + +catkin_package() + +install(PROGRAMS + scripts/listener_node + DESTINATION ${CATKIN_PACKAGE_BIN_DESTINATION} +) + +install(FILES + talk_and_listen.launch + DESTINATION ${CATKIN_PACKAGE_SHARE_DESTINATION} +) diff -Nru snapcraft-2.27.1+17.04/demos/shared-ros/ros-app/src/listener/package.xml snapcraft-2.28+17.04/demos/shared-ros/ros-app/src/listener/package.xml --- snapcraft-2.27.1+17.04/demos/shared-ros/ros-app/src/listener/package.xml 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/demos/shared-ros/ros-app/src/listener/package.xml 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,13 @@ + + + listener + 0.0.0 + The listener package + me + TODO + catkin + rospy + std_msgs + rospy + std_msgs + diff -Nru snapcraft-2.27.1+17.04/demos/shared-ros/ros-app/src/listener/scripts/listener_node snapcraft-2.28+17.04/demos/shared-ros/ros-app/src/listener/scripts/listener_node --- snapcraft-2.27.1+17.04/demos/shared-ros/ros-app/src/listener/scripts/listener_node 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/demos/shared-ros/ros-app/src/listener/scripts/listener_node 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,23 @@ +#!/usr/bin/env python + +import rospy +from std_msgs.msg import String + + +def callback(data): + rospy.loginfo('I heard %s', data.data) + + if rospy.get_param('~exit-after-receive', False): + rospy.signal_shutdown( + 'Requested to exit after message received. Exiting now.') + + +def listener(): + rospy.init_node('listener') + + rospy.Subscriber('babble', String, callback) + + rospy.spin() + +if __name__ == '__main__': + listener() diff -Nru snapcraft-2.27.1+17.04/demos/shared-ros/ros-app/src/listener/talk_and_listen.launch snapcraft-2.28+17.04/demos/shared-ros/ros-app/src/listener/talk_and_listen.launch --- snapcraft-2.27.1+17.04/demos/shared-ros/ros-app/src/listener/talk_and_listen.launch 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/demos/shared-ros/ros-app/src/listener/talk_and_listen.launch 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,13 @@ + + + + + + + + + + diff -Nru snapcraft-2.27.1+17.04/demos/shared-ros/ros-app/src/talker/CMakeLists.txt snapcraft-2.28+17.04/demos/shared-ros/ros-app/src/talker/CMakeLists.txt --- snapcraft-2.27.1+17.04/demos/shared-ros/ros-app/src/talker/CMakeLists.txt 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/demos/shared-ros/ros-app/src/talker/CMakeLists.txt 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,21 @@ +cmake_minimum_required(VERSION 2.8.3) +project(talker) + +find_package(catkin REQUIRED COMPONENTS + roscpp + std_msgs +) + +catkin_package() + +include_directories(${catkin_INCLUDE_DIRS}) + +add_executable(talker_node src/talker_node.cpp) + +target_link_libraries(talker_node ${catkin_LIBRARIES}) + +install(TARGETS talker_node + ARCHIVE DESTINATION ${CATKIN_PACKAGE_LIB_DESTINATION} + LIBRARY DESTINATION ${CATKIN_PACKAGE_LIB_DESTINATION} + RUNTIME DESTINATION ${CATKIN_PACKAGE_BIN_DESTINATION} +) diff -Nru snapcraft-2.27.1+17.04/demos/shared-ros/ros-app/src/talker/package.xml snapcraft-2.28+17.04/demos/shared-ros/ros-app/src/talker/package.xml --- snapcraft-2.27.1+17.04/demos/shared-ros/ros-app/src/talker/package.xml 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/demos/shared-ros/ros-app/src/talker/package.xml 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,13 @@ + + + talker + 0.0.0 + The talker package + me + TODO + catkin + roscpp + std_msgs + roscpp + std_msgs + diff -Nru snapcraft-2.27.1+17.04/demos/shared-ros/ros-app/src/talker/src/talker_node.cpp snapcraft-2.28+17.04/demos/shared-ros/ros-app/src/talker/src/talker_node.cpp --- snapcraft-2.27.1+17.04/demos/shared-ros/ros-app/src/talker/src/talker_node.cpp 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/demos/shared-ros/ros-app/src/talker/src/talker_node.cpp 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,35 @@ +#include + +#include +#include + +int main(int argc, char **argv) +{ + ros::init(argc, argv, "talker"); + + ros::NodeHandle nodeHandle; + + ros::Publisher publisher = nodeHandle.advertise("chatter", 1); + + ros::Rate loopRate(10); + + int count = 0; + while (ros::ok()) + { + std_msgs::String message; + + std::stringstream stream; + stream << "Hello world " << count++; + message.data = stream.str(); + + ROS_INFO("%s", message.data.c_str()); + + publisher.publish(message); + + ros::spinOnce(); + + loopRate.sleep(); + } + + return 0; +} diff -Nru snapcraft-2.27.1+17.04/demos/shared-ros/ros-base/snap/snapcraft.yaml snapcraft-2.28+17.04/demos/shared-ros/ros-base/snap/snapcraft.yaml --- snapcraft-2.27.1+17.04/demos/shared-ros/ros-base/snap/snapcraft.yaml 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/demos/shared-ros/ros-base/snap/snapcraft.yaml 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,22 @@ +name: ros-base +version: '1.0' +grade: stable +confinement: strict +summary: ROS Base Snap +description: Contains roscore and basic ROS utilities. + +slots: + # Consumers will need to access the PYTHONPATH as well as various libs + # contained in this snap, so share the entire $SNAP, not just the ROS + # workspace. + ros-base: + content: ros-base-v1 + interface: content + read: [/] + +parts: + ros-base: + plugin: catkin + rosdistro: kinetic + include-roscore: true + catkin-packages: [] diff -Nru snapcraft-2.27.1+17.04/demos/tomcat-maven-webapp/Makefile snapcraft-2.28+17.04/demos/tomcat-maven-webapp/Makefile --- snapcraft-2.27.1+17.04/demos/tomcat-maven-webapp/Makefile 2016-11-02 08:25:17.000000000 +0000 +++ snapcraft-2.28+17.04/demos/tomcat-maven-webapp/Makefile 1970-01-01 00:00:00.000000000 +0000 @@ -1,6 +0,0 @@ -all: - -install: - cp -a tomcat-conf/ $(DESTDIR)/ - install -D -m755 wrapper $(DESTDIR)/bin/wrapper - diff -Nru snapcraft-2.27.1+17.04/demos/tomcat-maven-webapp/snap/snapcraft.yaml snapcraft-2.28+17.04/demos/tomcat-maven-webapp/snap/snapcraft.yaml --- snapcraft-2.27.1+17.04/demos/tomcat-maven-webapp/snap/snapcraft.yaml 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/demos/tomcat-maven-webapp/snap/snapcraft.yaml 1970-01-01 00:00:00.000000000 +0000 @@ -1,28 +0,0 @@ -name: tomcat-webapp-demo -version: 1.0 -summary: Demo of Tomcat-hosted Webapp -description: This is a demo snap of a Tomcat-hosted webapp produced by snapcraft with maven. -confinement: strict - -apps: - tomcat: - command: bin/wrapper - daemon: simple - plugs: [network-bind] - environment: - # installation pathes are based of CATALINA_HOME - CATALINA_HOME: $SNAP - # writable pathes are based of CATALINA_BASE - CATALINA_BASE: $SNAP_DATA - -parts: - webapp: - plugin: maven - source: https://github.com/lool/snappy-mvn-demo.git - source-type: git - tomcat: - plugin: dump - source: https://archive.apache.org/dist/tomcat/tomcat-8/v8.0.29/bin/apache-tomcat-8.0.29.tar.gz - local-files: - plugin: make - source: . diff -Nru snapcraft-2.27.1+17.04/demos/tomcat-maven-webapp/tomcat-conf/catalina.policy snapcraft-2.28+17.04/demos/tomcat-maven-webapp/tomcat-conf/catalina.policy --- snapcraft-2.27.1+17.04/demos/tomcat-maven-webapp/tomcat-conf/catalina.policy 2016-11-02 08:25:17.000000000 +0000 +++ snapcraft-2.28+17.04/demos/tomcat-maven-webapp/tomcat-conf/catalina.policy 1970-01-01 00:00:00.000000000 +0000 @@ -1,250 +0,0 @@ -// Licensed to the Apache Software Foundation (ASF) under one or more -// contributor license agreements. See the NOTICE file distributed with -// this work for additional information regarding copyright ownership. -// The ASF licenses this file to You under the Apache License, Version 2.0 -// (the "License"); you may not use this file except in compliance with -// the License. You may obtain a copy of the License at -// -// http://www.apache.org/licenses/LICENSE-2.0 -// -// Unless required by applicable law or agreed to in writing, software -// distributed under the License is distributed on an "AS IS" BASIS, -// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -// See the License for the specific language governing permissions and -// limitations under the License. - -// ============================================================================ -// catalina.policy - Security Policy Permissions for Tomcat -// -// This file contains a default set of security policies to be enforced (by the -// JVM) when Catalina is executed with the "-security" option. In addition -// to the permissions granted here, the following additional permissions are -// granted to each web application: -// -// * Read access to the web application's document root directory -// * Read, write and delete access to the web application's working directory -// ============================================================================ - - -// ========== SYSTEM CODE PERMISSIONS ========================================= - - -// These permissions apply to javac -grant codeBase "file:${java.home}/lib/-" { - permission java.security.AllPermission; -}; - -// These permissions apply to all shared system extensions -grant codeBase "file:${java.home}/jre/lib/ext/-" { - permission java.security.AllPermission; -}; - -// These permissions apply to javac when ${java.home] points at $JAVA_HOME/jre -grant codeBase "file:${java.home}/../lib/-" { - permission java.security.AllPermission; -}; - -// These permissions apply to all shared system extensions when -// ${java.home} points at $JAVA_HOME/jre -grant codeBase "file:${java.home}/lib/ext/-" { - permission java.security.AllPermission; -}; - - -// ========== CATALINA CODE PERMISSIONS ======================================= - - -// These permissions apply to the daemon code -grant codeBase "file:${catalina.home}/bin/commons-daemon.jar" { - permission java.security.AllPermission; -}; - -// These permissions apply to the logging API -// Note: If tomcat-juli.jar is in ${catalina.base} and not in ${catalina.home}, -// update this section accordingly. -// grant codeBase "file:${catalina.base}/bin/tomcat-juli.jar" {..} -grant codeBase "file:${catalina.home}/bin/tomcat-juli.jar" { - permission java.io.FilePermission - "${java.home}${file.separator}lib${file.separator}logging.properties", "read"; - - permission java.io.FilePermission - "${catalina.base}${file.separator}conf${file.separator}logging.properties", "read"; - permission java.io.FilePermission - "${catalina.base}${file.separator}logs", "read, write"; - permission java.io.FilePermission - "${catalina.base}${file.separator}logs${file.separator}*", "read, write"; - - permission java.lang.RuntimePermission "shutdownHooks"; - permission java.lang.RuntimePermission "getClassLoader"; - permission java.lang.RuntimePermission "setContextClassLoader"; - - permission java.lang.management.ManagementPermission "monitor"; - - permission java.util.logging.LoggingPermission "control"; - - permission java.util.PropertyPermission "java.util.logging.config.class", "read"; - permission java.util.PropertyPermission "java.util.logging.config.file", "read"; - permission java.util.PropertyPermission "org.apache.juli.AsyncLoggerPollInterval", "read"; - permission java.util.PropertyPermission "org.apache.juli.AsyncMaxRecordCount", "read"; - permission java.util.PropertyPermission "org.apache.juli.AsyncOverflowDropType", "read"; - permission java.util.PropertyPermission "org.apache.juli.ClassLoaderLogManager.debug", "read"; - permission java.util.PropertyPermission "catalina.base", "read"; - - // Note: To enable per context logging configuration, permit read access to - // the appropriate file. Be sure that the logging configuration is - // secure before enabling such access. - // E.g. for the examples web application (uncomment and unwrap - // the following to be on a single line): - // permission java.io.FilePermission "${catalina.base}${file.separator} - // webapps${file.separator}examples${file.separator}WEB-INF - // ${file.separator}classes${file.separator}logging.properties", "read"; -}; - -// These permissions apply to the server startup code -grant codeBase "file:${catalina.home}/bin/bootstrap.jar" { - permission java.security.AllPermission; -}; - -// These permissions apply to the servlet API classes -// and those that are shared across all class loaders -// located in the "lib" directory -grant codeBase "file:${catalina.home}/lib/-" { - permission java.security.AllPermission; -}; - - -// If using a per instance lib directory, i.e. ${catalina.base}/lib, -// then the following permission will need to be uncommented -// grant codeBase "file:${catalina.base}/lib/-" { -// permission java.security.AllPermission; -// }; - - -// ========== WEB APPLICATION PERMISSIONS ===================================== - - -// These permissions are granted by default to all web applications -// In addition, a web application will be given a read FilePermission -// for all files and directories in its document root. -grant { - // Required for JNDI lookup of named JDBC DataSource's and - // javamail named MimePart DataSource used to send mail - permission java.util.PropertyPermission "java.home", "read"; - permission java.util.PropertyPermission "java.naming.*", "read"; - permission java.util.PropertyPermission "javax.sql.*", "read"; - - // OS Specific properties to allow read access - permission java.util.PropertyPermission "os.name", "read"; - permission java.util.PropertyPermission "os.version", "read"; - permission java.util.PropertyPermission "os.arch", "read"; - permission java.util.PropertyPermission "file.separator", "read"; - permission java.util.PropertyPermission "path.separator", "read"; - permission java.util.PropertyPermission "line.separator", "read"; - - // JVM properties to allow read access - permission java.util.PropertyPermission "java.version", "read"; - permission java.util.PropertyPermission "java.vendor", "read"; - permission java.util.PropertyPermission "java.vendor.url", "read"; - permission java.util.PropertyPermission "java.class.version", "read"; - permission java.util.PropertyPermission "java.specification.version", "read"; - permission java.util.PropertyPermission "java.specification.vendor", "read"; - permission java.util.PropertyPermission "java.specification.name", "read"; - - permission java.util.PropertyPermission "java.vm.specification.version", "read"; - permission java.util.PropertyPermission "java.vm.specification.vendor", "read"; - permission java.util.PropertyPermission "java.vm.specification.name", "read"; - permission java.util.PropertyPermission "java.vm.version", "read"; - permission java.util.PropertyPermission "java.vm.vendor", "read"; - permission java.util.PropertyPermission "java.vm.name", "read"; - - // Required for OpenJMX - permission java.lang.RuntimePermission "getAttribute"; - - // Allow read of JAXP compliant XML parser debug - permission java.util.PropertyPermission "jaxp.debug", "read"; - - // All JSPs need to be able to read this package - permission java.lang.RuntimePermission "accessClassInPackage.org.apache.tomcat"; - - // Precompiled JSPs need access to these packages. - permission java.lang.RuntimePermission "accessClassInPackage.org.apache.jasper.el"; - permission java.lang.RuntimePermission "accessClassInPackage.org.apache.jasper.runtime"; - permission java.lang.RuntimePermission - "accessClassInPackage.org.apache.jasper.runtime.*"; - - // Precompiled JSPs need access to these system properties. - permission java.util.PropertyPermission - "org.apache.jasper.runtime.BodyContentImpl.LIMIT_BUFFER", "read"; - permission java.util.PropertyPermission - "org.apache.el.parser.COERCE_TO_ZERO", "read"; - - // The cookie code needs these. - permission java.util.PropertyPermission - "org.apache.catalina.STRICT_SERVLET_COMPLIANCE", "read"; - permission java.util.PropertyPermission - "org.apache.tomcat.util.http.ServerCookie.STRICT_NAMING", "read"; - permission java.util.PropertyPermission - "org.apache.tomcat.util.http.ServerCookie.FWD_SLASH_IS_SEPARATOR", "read"; - - // Applications using Comet need to be able to access this package - permission java.lang.RuntimePermission "accessClassInPackage.org.apache.catalina.comet"; - - // Applications using WebSocket need to be able to access these packages - permission java.lang.RuntimePermission "accessClassInPackage.org.apache.tomcat.websocket"; - permission java.lang.RuntimePermission "accessClassInPackage.org.apache.tomcat.websocket.server"; -}; - - -// The Manager application needs access to the following packages to support the -// session display functionality. These settings support the following -// configurations: -// - default CATALINA_HOME == CATALINA_BASE -// - CATALINA_HOME != CATALINA_BASE, per instance Manager in CATALINA_BASE -// - CATALINA_HOME != CATALINA_BASE, shared Manager in CATALINA_HOME -grant codeBase "file:${catalina.base}/webapps/manager/-" { - permission java.lang.RuntimePermission "accessClassInPackage.org.apache.catalina"; - permission java.lang.RuntimePermission "accessClassInPackage.org.apache.catalina.ha.session"; - permission java.lang.RuntimePermission "accessClassInPackage.org.apache.catalina.manager"; - permission java.lang.RuntimePermission "accessClassInPackage.org.apache.catalina.manager.util"; - permission java.lang.RuntimePermission "accessClassInPackage.org.apache.catalina.util"; -}; -grant codeBase "file:${catalina.home}/webapps/manager/-" { - permission java.lang.RuntimePermission "accessClassInPackage.org.apache.catalina"; - permission java.lang.RuntimePermission "accessClassInPackage.org.apache.catalina.ha.session"; - permission java.lang.RuntimePermission "accessClassInPackage.org.apache.catalina.manager"; - permission java.lang.RuntimePermission "accessClassInPackage.org.apache.catalina.manager.util"; - permission java.lang.RuntimePermission "accessClassInPackage.org.apache.catalina.util"; -}; - -// You can assign additional permissions to particular web applications by -// adding additional "grant" entries here, based on the code base for that -// application, /WEB-INF/classes/, or /WEB-INF/lib/ jar files. -// -// Different permissions can be granted to JSP pages, classes loaded from -// the /WEB-INF/classes/ directory, all jar files in the /WEB-INF/lib/ -// directory, or even to individual jar files in the /WEB-INF/lib/ directory. -// -// For instance, assume that the standard "examples" application -// included a JDBC driver that needed to establish a network connection to the -// corresponding database and used the scrape taglib to get the weather from -// the NOAA web server. You might create a "grant" entries like this: -// -// The permissions granted to the context root directory apply to JSP pages. -// grant codeBase "file:${catalina.base}/webapps/examples/-" { -// permission java.net.SocketPermission "dbhost.mycompany.com:5432", "connect"; -// permission java.net.SocketPermission "*.noaa.gov:80", "connect"; -// }; -// -// The permissions granted to the context WEB-INF/classes directory -// grant codeBase "file:${catalina.base}/webapps/examples/WEB-INF/classes/-" { -// }; -// -// The permission granted to your JDBC driver -// grant codeBase "jar:file:${catalina.base}/webapps/examples/WEB-INF/lib/driver.jar!/-" { -// permission java.net.SocketPermission "dbhost.mycompany.com:5432", "connect"; -// }; -// The permission granted to the scrape taglib -// grant codeBase "jar:file:${catalina.base}/webapps/examples/WEB-INF/lib/scrape.jar!/-" { -// permission java.net.SocketPermission "*.noaa.gov:80", "connect"; -// }; - diff -Nru snapcraft-2.27.1+17.04/demos/tomcat-maven-webapp/tomcat-conf/catalina.properties snapcraft-2.28+17.04/demos/tomcat-maven-webapp/tomcat-conf/catalina.properties --- snapcraft-2.27.1+17.04/demos/tomcat-maven-webapp/tomcat-conf/catalina.properties 2016-11-02 08:25:17.000000000 +0000 +++ snapcraft-2.28+17.04/demos/tomcat-maven-webapp/tomcat-conf/catalina.properties 1970-01-01 00:00:00.000000000 +0000 @@ -1,145 +0,0 @@ -# Licensed to the Apache Software Foundation (ASF) under one or more -# contributor license agreements. See the NOTICE file distributed with -# this work for additional information regarding copyright ownership. -# The ASF licenses this file to You under the Apache License, Version 2.0 -# (the "License"); you may not use this file except in compliance with -# the License. You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -# -# List of comma-separated packages that start with or equal this string -# will cause a security exception to be thrown when -# passed to checkPackageAccess unless the -# corresponding RuntimePermission ("accessClassInPackage."+package) has -# been granted. -package.access=sun.,org.apache.catalina.,org.apache.coyote.,org.apache.jasper.,org.apache.tomcat. -# -# List of comma-separated packages that start with or equal this string -# will cause a security exception to be thrown when -# passed to checkPackageDefinition unless the -# corresponding RuntimePermission ("defineClassInPackage."+package) has -# been granted. -# -# by default, no packages are restricted for definition, and none of -# the class loaders supplied with the JDK call checkPackageDefinition. -# -package.definition=sun.,java.,org.apache.catalina.,org.apache.coyote.,\ -org.apache.jasper.,org.apache.naming.,org.apache.tomcat. - -# -# -# List of comma-separated paths defining the contents of the "common" -# classloader. Prefixes should be used to define what is the repository type. -# Path may be relative to the CATALINA_HOME or CATALINA_BASE path or absolute. -# If left as blank,the JVM system loader will be used as Catalina's "common" -# loader. -# Examples: -# "foo": Add this folder as a class repository -# "foo/*.jar": Add all the JARs of the specified folder as class -# repositories -# "foo/bar.jar": Add bar.jar as a class repository -# -# Note: Values are enclosed in double quotes ("...") in case either the -# ${catalina.base} path or the ${catalina.home} path contains a comma. -# Because double quotes are used for quoting, the double quote character -# may not appear in a path. -common.loader="${catalina.base}/lib","${catalina.base}/lib/*.jar","${catalina.home}/lib","${catalina.home}/lib/*.jar" - -# -# List of comma-separated paths defining the contents of the "server" -# classloader. Prefixes should be used to define what is the repository type. -# Path may be relative to the CATALINA_HOME or CATALINA_BASE path or absolute. -# If left as blank, the "common" loader will be used as Catalina's "server" -# loader. -# Examples: -# "foo": Add this folder as a class repository -# "foo/*.jar": Add all the JARs of the specified folder as class -# repositories -# "foo/bar.jar": Add bar.jar as a class repository -# -# Note: Values may be enclosed in double quotes ("...") in case either the -# ${catalina.base} path or the ${catalina.home} path contains a comma. -# Because double quotes are used for quoting, the double quote character -# may not appear in a path. -server.loader= - -# -# List of comma-separated paths defining the contents of the "shared" -# classloader. Prefixes should be used to define what is the repository type. -# Path may be relative to the CATALINA_BASE path or absolute. If left as blank, -# the "common" loader will be used as Catalina's "shared" loader. -# Examples: -# "foo": Add this folder as a class repository -# "foo/*.jar": Add all the JARs of the specified folder as class -# repositories -# "foo/bar.jar": Add bar.jar as a class repository -# Please note that for single jars, e.g. bar.jar, you need the URL form -# starting with file:. -# -# Note: Values may be enclosed in double quotes ("...") in case either the -# ${catalina.base} path or the ${catalina.home} path contains a comma. -# Because double quotes are used for quoting, the double quote character -# may not appear in a path. -shared.loader= - -# Default list of JAR files that should not be scanned using the JarScanner -# functionality. This is typically used to scan JARs for configuration -# information. JARs that do not contain such information may be excluded from -# the scan to speed up the scanning process. This is the default list. JARs on -# this list are excluded from all scans. The list must be a comma separated list -# of JAR file names. -# The list of JARs to skip may be over-ridden at a Context level for individual -# scan types by configuring a JarScanner with a nested JarScanFilter. -# The JARs listed below include: -# - Tomcat Bootstrap JARs -# - Tomcat API JARs -# - Catalina JARs -# - Jasper JARs -# - Tomcat JARs -# - Common non-Tomcat JARs -# - Test JARs (JUnit, Cobertura and dependencies) -tomcat.util.scan.StandardJarScanFilter.jarsToSkip=\ -bootstrap.jar,commons-daemon.jar,tomcat-juli.jar,\ -annotations-api.jar,el-api.jar,jsp-api.jar,servlet-api.jar,websocket-api.jar,\ -catalina.jar,catalina-ant.jar,catalina-ha.jar,catalina-storeconfig.jar,\ -catalina-tribes.jar,\ -jasper.jar,jasper-el.jar,ecj-*.jar,\ -tomcat-api.jar,tomcat-util.jar,tomcat-util-scan.jar,tomcat-coyote.jar,\ -tomcat-dbcp.jar,tomcat-jni.jar,tomcat-websocket.jar,\ -tomcat-i18n-en.jar,tomcat-i18n-es.jar,tomcat-i18n-fr.jar,tomcat-i18n-ja.jar,\ -tomcat-juli-adapters.jar,catalina-jmx-remote.jar,catalina-ws.jar,\ -tomcat-jdbc.jar,\ -tools.jar,\ -commons-beanutils*.jar,commons-codec*.jar,commons-collections*.jar,\ -commons-dbcp*.jar,commons-digester*.jar,commons-fileupload*.jar,\ -commons-httpclient*.jar,commons-io*.jar,commons-lang*.jar,commons-logging*.jar,\ -commons-math*.jar,commons-pool*.jar,\ -jstl.jar,taglibs-standard-spec-*.jar,\ -geronimo-spec-jaxrpc*.jar,wsdl4j*.jar,\ -ant.jar,ant-junit*.jar,aspectj*.jar,jmx.jar,h2*.jar,hibernate*.jar,httpclient*.jar,\ -jmx-tools.jar,jta*.jar,log4j*.jar,mail*.jar,slf4j*.jar,\ -xercesImpl.jar,xmlParserAPIs.jar,xml-apis.jar,\ -junit.jar,junit-*.jar,ant-launcher.jar,\ -cobertura-*.jar,asm-*.jar,dom4j-*.jar,icu4j-*.jar,jaxen-*.jar,jdom-*.jar,\ -jetty-*.jar,oro-*.jar,servlet-api-*.jar,tagsoup-*.jar,xmlParserAPIs-*.jar,\ -xom-*.jar - -# Default list of JAR files that should be scanned that overrides the default -# jarsToSkip list above. This is typically used to include a specific JAR that -# has been excluded by a broad file name pattern in the jarsToSkip list. -# The list of JARs to scan may be over-ridden at a Context level for individual -# scan types by configuring a JarScanner with a nested JarScanFilter. -tomcat.util.scan.StandardJarScanFilter.jarsToScan=log4j-core*.jar,log4j-taglib*.jar - -# String cache configuration. -tomcat.util.buf.StringCache.byte.enabled=true -#tomcat.util.buf.StringCache.char.enabled=true -#tomcat.util.buf.StringCache.trainThreshold=500000 -#tomcat.util.buf.StringCache.cacheSize=5000 diff -Nru snapcraft-2.27.1+17.04/demos/tomcat-maven-webapp/tomcat-conf/context.xml snapcraft-2.28+17.04/demos/tomcat-maven-webapp/tomcat-conf/context.xml --- snapcraft-2.27.1+17.04/demos/tomcat-maven-webapp/tomcat-conf/context.xml 2016-11-02 08:25:17.000000000 +0000 +++ snapcraft-2.28+17.04/demos/tomcat-maven-webapp/tomcat-conf/context.xml 1970-01-01 00:00:00.000000000 +0000 @@ -1,36 +0,0 @@ - - - - - - - - WEB-INF/web.xml - ${catalina.base}/conf/web.xml - - - - - - - diff -Nru snapcraft-2.27.1+17.04/demos/tomcat-maven-webapp/tomcat-conf/logging.properties snapcraft-2.28+17.04/demos/tomcat-maven-webapp/tomcat-conf/logging.properties --- snapcraft-2.27.1+17.04/demos/tomcat-maven-webapp/tomcat-conf/logging.properties 2016-11-02 08:25:17.000000000 +0000 +++ snapcraft-2.28+17.04/demos/tomcat-maven-webapp/tomcat-conf/logging.properties 1970-01-01 00:00:00.000000000 +0000 @@ -1,64 +0,0 @@ -# Licensed to the Apache Software Foundation (ASF) under one or more -# contributor license agreements. See the NOTICE file distributed with -# this work for additional information regarding copyright ownership. -# The ASF licenses this file to You under the Apache License, Version 2.0 -# (the "License"); you may not use this file except in compliance with -# the License. You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -handlers = 1catalina.org.apache.juli.AsyncFileHandler, 2localhost.org.apache.juli.AsyncFileHandler, 3manager.org.apache.juli.AsyncFileHandler, 4host-manager.org.apache.juli.AsyncFileHandler, java.util.logging.ConsoleHandler - -.handlers = 1catalina.org.apache.juli.AsyncFileHandler, java.util.logging.ConsoleHandler - -############################################################ -# Handler specific properties. -# Describes specific configuration info for Handlers. -############################################################ - -1catalina.org.apache.juli.AsyncFileHandler.level = FINE -1catalina.org.apache.juli.AsyncFileHandler.directory = ${catalina.base}/logs -1catalina.org.apache.juli.AsyncFileHandler.prefix = catalina. - -2localhost.org.apache.juli.AsyncFileHandler.level = FINE -2localhost.org.apache.juli.AsyncFileHandler.directory = ${catalina.base}/logs -2localhost.org.apache.juli.AsyncFileHandler.prefix = localhost. - -3manager.org.apache.juli.AsyncFileHandler.level = FINE -3manager.org.apache.juli.AsyncFileHandler.directory = ${catalina.base}/logs -3manager.org.apache.juli.AsyncFileHandler.prefix = manager. - -4host-manager.org.apache.juli.AsyncFileHandler.level = FINE -4host-manager.org.apache.juli.AsyncFileHandler.directory = ${catalina.base}/logs -4host-manager.org.apache.juli.AsyncFileHandler.prefix = host-manager. - -java.util.logging.ConsoleHandler.level = FINE -java.util.logging.ConsoleHandler.formatter = org.apache.juli.OneLineFormatter - - -############################################################ -# Facility specific properties. -# Provides extra control for each logger. -############################################################ - -org.apache.catalina.core.ContainerBase.[Catalina].[localhost].level = INFO -org.apache.catalina.core.ContainerBase.[Catalina].[localhost].handlers = 2localhost.org.apache.juli.AsyncFileHandler - -org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/manager].level = INFO -org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/manager].handlers = 3manager.org.apache.juli.AsyncFileHandler - -org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/host-manager].level = INFO -org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/host-manager].handlers = 4host-manager.org.apache.juli.AsyncFileHandler - -# For example, set the org.apache.catalina.util.LifecycleBase logger to log -# each component that extends LifecycleBase changing state: -#org.apache.catalina.util.LifecycleBase.level = FINE - -# To see debug messages in TldLocationsCache, uncomment the following line: -#org.apache.jasper.compiler.TldLocationsCache.level = FINE diff -Nru snapcraft-2.27.1+17.04/demos/tomcat-maven-webapp/tomcat-conf/server.xml snapcraft-2.28+17.04/demos/tomcat-maven-webapp/tomcat-conf/server.xml --- snapcraft-2.27.1+17.04/demos/tomcat-maven-webapp/tomcat-conf/server.xml 2016-11-02 08:25:17.000000000 +0000 +++ snapcraft-2.28+17.04/demos/tomcat-maven-webapp/tomcat-conf/server.xml 1970-01-01 00:00:00.000000000 +0000 @@ -1,142 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff -Nru snapcraft-2.27.1+17.04/demos/tomcat-maven-webapp/tomcat-conf/tomcat-users.xml snapcraft-2.28+17.04/demos/tomcat-maven-webapp/tomcat-conf/tomcat-users.xml --- snapcraft-2.27.1+17.04/demos/tomcat-maven-webapp/tomcat-conf/tomcat-users.xml 2016-11-02 08:25:17.000000000 +0000 +++ snapcraft-2.28+17.04/demos/tomcat-maven-webapp/tomcat-conf/tomcat-users.xml 1970-01-01 00:00:00.000000000 +0000 @@ -1,39 +0,0 @@ - - - - - - - diff -Nru snapcraft-2.27.1+17.04/demos/tomcat-maven-webapp/tomcat-conf/tomcat-users.xsd snapcraft-2.28+17.04/demos/tomcat-maven-webapp/tomcat-conf/tomcat-users.xsd --- snapcraft-2.27.1+17.04/demos/tomcat-maven-webapp/tomcat-conf/tomcat-users.xsd 2016-11-02 08:25:17.000000000 +0000 +++ snapcraft-2.28+17.04/demos/tomcat-maven-webapp/tomcat-conf/tomcat-users.xsd 1970-01-01 00:00:00.000000000 +0000 @@ -1,43 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - \ No newline at end of file diff -Nru snapcraft-2.27.1+17.04/demos/tomcat-maven-webapp/tomcat-conf/web.xml snapcraft-2.28+17.04/demos/tomcat-maven-webapp/tomcat-conf/web.xml --- snapcraft-2.27.1+17.04/demos/tomcat-maven-webapp/tomcat-conf/web.xml 2016-11-02 08:25:17.000000000 +0000 +++ snapcraft-2.28+17.04/demos/tomcat-maven-webapp/tomcat-conf/web.xml 1970-01-01 00:00:00.000000000 +0000 @@ -1,4664 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - default - org.apache.catalina.servlets.DefaultServlet - - debug - 0 - - - listings - false - - 1 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - jsp - org.apache.jasper.servlet.JspServlet - - fork - false - - - xpoweredBy - false - - 3 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - default - / - - - - - jsp - *.jsp - *.jspx - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 30 - - - - - - - - - - - - - 123 - application/vnd.lotus-1-2-3 - - - 3dml - text/vnd.in3d.3dml - - - 3ds - image/x-3ds - - - 3g2 - video/3gpp2 - - - 3gp - video/3gpp - - - 7z - application/x-7z-compressed - - - aab - application/x-authorware-bin - - - aac - audio/x-aac - - - aam - application/x-authorware-map - - - aas - application/x-authorware-seg - - - abs - audio/x-mpeg - - - abw - application/x-abiword - - - ac - application/pkix-attr-cert - - - acc - application/vnd.americandynamics.acc - - - ace - application/x-ace-compressed - - - acu - application/vnd.acucobol - - - acutc - application/vnd.acucorp - - - adp - audio/adpcm - - - aep - application/vnd.audiograph - - - afm - application/x-font-type1 - - - afp - application/vnd.ibm.modcap - - - ahead - application/vnd.ahead.space - - - ai - application/postscript - - - aif - audio/x-aiff - - - aifc - audio/x-aiff - - - aiff - audio/x-aiff - - - aim - application/x-aim - - - air - application/vnd.adobe.air-application-installer-package+zip - - - ait - application/vnd.dvb.ait - - - ami - application/vnd.amiga.ami - - - anx - application/annodex - - - apk - application/vnd.android.package-archive - - - appcache - text/cache-manifest - - - application - application/x-ms-application - - - apr - application/vnd.lotus-approach - - - arc - application/x-freearc - - - art - image/x-jg - - - asc - application/pgp-signature - - - asf - video/x-ms-asf - - - asm - text/x-asm - - - aso - application/vnd.accpac.simply.aso - - - asx - video/x-ms-asf - - - atc - application/vnd.acucorp - - - atom - application/atom+xml - - - atomcat - application/atomcat+xml - - - atomsvc - application/atomsvc+xml - - - atx - application/vnd.antix.game-component - - - au - audio/basic - - - avi - video/x-msvideo - - - avx - video/x-rad-screenplay - - - aw - application/applixware - - - axa - audio/annodex - - - axv - video/annodex - - - azf - application/vnd.airzip.filesecure.azf - - - azs - application/vnd.airzip.filesecure.azs - - - azw - application/vnd.amazon.ebook - - - bat - application/x-msdownload - - - bcpio - application/x-bcpio - - - bdf - application/x-font-bdf - - - bdm - application/vnd.syncml.dm+wbxml - - - bed - application/vnd.realvnc.bed - - - bh2 - application/vnd.fujitsu.oasysprs - - - bin - application/octet-stream - - - blb - application/x-blorb - - - blorb - application/x-blorb - - - bmi - application/vnd.bmi - - - bmp - image/bmp - - - body - text/html - - - book - application/vnd.framemaker - - - box - application/vnd.previewsystems.box - - - boz - application/x-bzip2 - - - bpk - application/octet-stream - - - btif - image/prs.btif - - - bz - application/x-bzip - - - bz2 - application/x-bzip2 - - - c - text/x-c - - - c11amc - application/vnd.cluetrust.cartomobile-config - - - c11amz - application/vnd.cluetrust.cartomobile-config-pkg - - - c4d - application/vnd.clonk.c4group - - - c4f - application/vnd.clonk.c4group - - - c4g - application/vnd.clonk.c4group - - - c4p - application/vnd.clonk.c4group - - - c4u - application/vnd.clonk.c4group - - - cab - application/vnd.ms-cab-compressed - - - caf - audio/x-caf - - - cap - application/vnd.tcpdump.pcap - - - car - application/vnd.curl.car - - - cat - application/vnd.ms-pki.seccat - - - cb7 - application/x-cbr - - - cba - application/x-cbr - - - cbr - application/x-cbr - - - cbt - application/x-cbr - - - cbz - application/x-cbr - - - cc - text/x-c - - - cct - application/x-director - - - ccxml - application/ccxml+xml - - - cdbcmsg - application/vnd.contact.cmsg - - - cdf - application/x-cdf - - - cdkey - application/vnd.mediastation.cdkey - - - cdmia - application/cdmi-capability - - - cdmic - application/cdmi-container - - - cdmid - application/cdmi-domain - - - cdmio - application/cdmi-object - - - cdmiq - application/cdmi-queue - - - cdx - chemical/x-cdx - - - cdxml - application/vnd.chemdraw+xml - - - cdy - application/vnd.cinderella - - - cer - application/pkix-cert - - - cfs - application/x-cfs-compressed - - - cgm - image/cgm - - - chat - application/x-chat - - - chm - application/vnd.ms-htmlhelp - - - chrt - application/vnd.kde.kchart - - - cif - chemical/x-cif - - - cii - application/vnd.anser-web-certificate-issue-initiation - - - cil - application/vnd.ms-artgalry - - - cla - application/vnd.claymore - - - class - application/java - - - clkk - application/vnd.crick.clicker.keyboard - - - clkp - application/vnd.crick.clicker.palette - - - clkt - application/vnd.crick.clicker.template - - - clkw - application/vnd.crick.clicker.wordbank - - - clkx - application/vnd.crick.clicker - - - clp - application/x-msclip - - - cmc - application/vnd.cosmocaller - - - cmdf - chemical/x-cmdf - - - cml - chemical/x-cml - - - cmp - application/vnd.yellowriver-custom-menu - - - cmx - image/x-cmx - - - cod - application/vnd.rim.cod - - - com - application/x-msdownload - - - conf - text/plain - - - cpio - application/x-cpio - - - cpp - text/x-c - - - cpt - application/mac-compactpro - - - crd - application/x-mscardfile - - - crl - application/pkix-crl - - - crt - application/x-x509-ca-cert - - - cryptonote - application/vnd.rig.cryptonote - - - csh - application/x-csh - - - csml - chemical/x-csml - - - csp - application/vnd.commonspace - - - css - text/css - - - cst - application/x-director - - - csv - text/csv - - - cu - application/cu-seeme - - - curl - text/vnd.curl - - - cww - application/prs.cww - - - cxt - application/x-director - - - cxx - text/x-c - - - dae - model/vnd.collada+xml - - - daf - application/vnd.mobius.daf - - - dart - application/vnd.dart - - - dataless - application/vnd.fdsn.seed - - - davmount - application/davmount+xml - - - dbk - application/docbook+xml - - - dcr - application/x-director - - - dcurl - text/vnd.curl.dcurl - - - dd2 - application/vnd.oma.dd2+xml - - - ddd - application/vnd.fujixerox.ddd - - - deb - application/x-debian-package - - - def - text/plain - - - deploy - application/octet-stream - - - der - application/x-x509-ca-cert - - - dfac - application/vnd.dreamfactory - - - dgc - application/x-dgc-compressed - - - dib - image/bmp - - - dic - text/x-c - - - dir - application/x-director - - - dis - application/vnd.mobius.dis - - - dist - application/octet-stream - - - distz - application/octet-stream - - - djv - image/vnd.djvu - - - djvu - image/vnd.djvu - - - dll - application/x-msdownload - - - dmg - application/x-apple-diskimage - - - dmp - application/vnd.tcpdump.pcap - - - dms - application/octet-stream - - - dna - application/vnd.dna - - - doc - application/msword - - - docm - application/vnd.ms-word.document.macroenabled.12 - - - docx - application/vnd.openxmlformats-officedocument.wordprocessingml.document - - - dot - application/msword - - - dotm - application/vnd.ms-word.template.macroenabled.12 - - - dotx - application/vnd.openxmlformats-officedocument.wordprocessingml.template - - - dp - application/vnd.osgi.dp - - - dpg - application/vnd.dpgraph - - - dra - audio/vnd.dra - - - dsc - text/prs.lines.tag - - - dssc - application/dssc+der - - - dtb - application/x-dtbook+xml - - - dtd - application/xml-dtd - - - dts - audio/vnd.dts - - - dtshd - audio/vnd.dts.hd - - - dump - application/octet-stream - - - dv - video/x-dv - - - dvb - video/vnd.dvb.file - - - dvi - application/x-dvi - - - dwf - model/vnd.dwf - - - dwg - image/vnd.dwg - - - dxf - image/vnd.dxf - - - dxp - application/vnd.spotfire.dxp - - - dxr - application/x-director - - - ecelp4800 - audio/vnd.nuera.ecelp4800 - - - ecelp7470 - audio/vnd.nuera.ecelp7470 - - - ecelp9600 - audio/vnd.nuera.ecelp9600 - - - ecma - application/ecmascript - - - edm - application/vnd.novadigm.edm - - - edx - application/vnd.novadigm.edx - - - efif - application/vnd.picsel - - - ei6 - application/vnd.pg.osasli - - - elc - application/octet-stream - - - emf - application/x-msmetafile - - - eml - message/rfc822 - - - emma - application/emma+xml - - - emz - application/x-msmetafile - - - eol - audio/vnd.digital-winds - - - eot - application/vnd.ms-fontobject - - - eps - application/postscript - - - epub - application/epub+zip - - - es3 - application/vnd.eszigno3+xml - - - esa - application/vnd.osgi.subsystem - - - esf - application/vnd.epson.esf - - - et3 - application/vnd.eszigno3+xml - - - etx - text/x-setext - - - eva - application/x-eva - - - evy - application/x-envoy - - - exe - application/octet-stream - - - exi - application/exi - - - ext - application/vnd.novadigm.ext - - - ez - application/andrew-inset - - - ez2 - application/vnd.ezpix-album - - - ez3 - application/vnd.ezpix-package - - - f - text/x-fortran - - - f4v - video/x-f4v - - - f77 - text/x-fortran - - - f90 - text/x-fortran - - - fbs - image/vnd.fastbidsheet - - - fcdt - application/vnd.adobe.formscentral.fcdt - - - fcs - application/vnd.isac.fcs - - - fdf - application/vnd.fdf - - - fe_launch - application/vnd.denovo.fcselayout-link - - - fg5 - application/vnd.fujitsu.oasysgp - - - fgd - application/x-director - - - fh - image/x-freehand - - - fh4 - image/x-freehand - - - fh5 - image/x-freehand - - - fh7 - image/x-freehand - - - fhc - image/x-freehand - - - fig - application/x-xfig - - - flac - audio/flac - - - fli - video/x-fli - - - flo - application/vnd.micrografx.flo - - - flv - video/x-flv - - - flw - application/vnd.kde.kivio - - - flx - text/vnd.fmi.flexstor - - - fly - text/vnd.fly - - - fm - application/vnd.framemaker - - - fnc - application/vnd.frogans.fnc - - - for - text/x-fortran - - - fpx - image/vnd.fpx - - - frame - application/vnd.framemaker - - - fsc - application/vnd.fsc.weblaunch - - - fst - image/vnd.fst - - - ftc - application/vnd.fluxtime.clip - - - fti - application/vnd.anser-web-funds-transfer-initiation - - - fvt - video/vnd.fvt - - - fxp - application/vnd.adobe.fxp - - - fxpl - application/vnd.adobe.fxp - - - fzs - application/vnd.fuzzysheet - - - g2w - application/vnd.geoplan - - - g3 - image/g3fax - - - g3w - application/vnd.geospace - - - gac - application/vnd.groove-account - - - gam - application/x-tads - - - gbr - application/rpki-ghostbusters - - - gca - application/x-gca-compressed - - - gdl - model/vnd.gdl - - - geo - application/vnd.dynageo - - - gex - application/vnd.geometry-explorer - - - ggb - application/vnd.geogebra.file - - - ggt - application/vnd.geogebra.tool - - - ghf - application/vnd.groove-help - - - gif - image/gif - - - gim - application/vnd.groove-identity-message - - - gml - application/gml+xml - - - gmx - application/vnd.gmx - - - gnumeric - application/x-gnumeric - - - gph - application/vnd.flographit - - - gpx - application/gpx+xml - - - gqf - application/vnd.grafeq - - - gqs - application/vnd.grafeq - - - gram - application/srgs - - - gramps - application/x-gramps-xml - - - gre - application/vnd.geometry-explorer - - - grv - application/vnd.groove-injector - - - grxml - application/srgs+xml - - - gsf - application/x-font-ghostscript - - - gtar - application/x-gtar - - - gtm - application/vnd.groove-tool-message - - - gtw - model/vnd.gtw - - - gv - text/vnd.graphviz - - - gxf - application/gxf - - - gxt - application/vnd.geonext - - - gz - application/x-gzip - - - h - text/x-c - - - h261 - video/h261 - - - h263 - video/h263 - - - h264 - video/h264 - - - hal - application/vnd.hal+xml - - - hbci - application/vnd.hbci - - - hdf - application/x-hdf - - - hh - text/x-c - - - hlp - application/winhlp - - - hpgl - application/vnd.hp-hpgl - - - hpid - application/vnd.hp-hpid - - - hps - application/vnd.hp-hps - - - hqx - application/mac-binhex40 - - - htc - text/x-component - - - htke - application/vnd.kenameaapp - - - htm - text/html - - - html - text/html - - - hvd - application/vnd.yamaha.hv-dic - - - hvp - application/vnd.yamaha.hv-voice - - - hvs - application/vnd.yamaha.hv-script - - - i2g - application/vnd.intergeo - - - icc - application/vnd.iccprofile - - - ice - x-conference/x-cooltalk - - - icm - application/vnd.iccprofile - - - ico - image/x-icon - - - ics - text/calendar - - - ief - image/ief - - - ifb - text/calendar - - - ifm - application/vnd.shana.informed.formdata - - - iges - model/iges - - - igl - application/vnd.igloader - - - igm - application/vnd.insors.igm - - - igs - model/iges - - - igx - application/vnd.micrografx.igx - - - iif - application/vnd.shana.informed.interchange - - - imp - application/vnd.accpac.simply.imp - - - ims - application/vnd.ms-ims - - - in - text/plain - - - ink - application/inkml+xml - - - inkml - application/inkml+xml - - - install - application/x-install-instructions - - - iota - application/vnd.astraea-software.iota - - - ipfix - application/ipfix - - - ipk - application/vnd.shana.informed.package - - - irm - application/vnd.ibm.rights-management - - - irp - application/vnd.irepository.package+xml - - - iso - application/x-iso9660-image - - - itp - application/vnd.shana.informed.formtemplate - - - ivp - application/vnd.immervision-ivp - - - ivu - application/vnd.immervision-ivu - - - jad - text/vnd.sun.j2me.app-descriptor - - - jam - application/vnd.jam - - - jar - application/java-archive - - - java - text/x-java-source - - - jisp - application/vnd.jisp - - - jlt - application/vnd.hp-jlyt - - - jnlp - application/x-java-jnlp-file - - - joda - application/vnd.joost.joda-archive - - - jpe - image/jpeg - - - jpeg - image/jpeg - - - jpg - image/jpeg - - - jpgm - video/jpm - - - jpgv - video/jpeg - - - jpm - video/jpm - - - js - application/javascript - - - jsf - text/plain - - - json - application/json - - - jsonml - application/jsonml+json - - - jspf - text/plain - - - kar - audio/midi - - - karbon - application/vnd.kde.karbon - - - kfo - application/vnd.kde.kformula - - - kia - application/vnd.kidspiration - - - kml - application/vnd.google-earth.kml+xml - - - kmz - application/vnd.google-earth.kmz - - - kne - application/vnd.kinar - - - knp - application/vnd.kinar - - - kon - application/vnd.kde.kontour - - - kpr - application/vnd.kde.kpresenter - - - kpt - application/vnd.kde.kpresenter - - - kpxx - application/vnd.ds-keypoint - - - ksp - application/vnd.kde.kspread - - - ktr - application/vnd.kahootz - - - ktx - image/ktx - - - ktz - application/vnd.kahootz - - - kwd - application/vnd.kde.kword - - - kwt - application/vnd.kde.kword - - - lasxml - application/vnd.las.las+xml - - - latex - application/x-latex - - - lbd - application/vnd.llamagraphics.life-balance.desktop - - - lbe - application/vnd.llamagraphics.life-balance.exchange+xml - - - les - application/vnd.hhe.lesson-player - - - lha - application/x-lzh-compressed - - - link66 - application/vnd.route66.link66+xml - - - list - text/plain - - - list3820 - application/vnd.ibm.modcap - - - listafp - application/vnd.ibm.modcap - - - lnk - application/x-ms-shortcut - - - log - text/plain - - - lostxml - application/lost+xml - - - lrf - application/octet-stream - - - lrm - application/vnd.ms-lrm - - - ltf - application/vnd.frogans.ltf - - - lvp - audio/vnd.lucent.voice - - - lwp - application/vnd.lotus-wordpro - - - lzh - application/x-lzh-compressed - - - m13 - application/x-msmediaview - - - m14 - application/x-msmediaview - - - m1v - video/mpeg - - - m21 - application/mp21 - - - m2a - audio/mpeg - - - m2v - video/mpeg - - - m3a - audio/mpeg - - - m3u - audio/x-mpegurl - - - m3u8 - application/vnd.apple.mpegurl - - - m4a - audio/mp4 - - - m4b - audio/mp4 - - - m4r - audio/mp4 - - - m4u - video/vnd.mpegurl - - - m4v - video/mp4 - - - ma - application/mathematica - - - mac - image/x-macpaint - - - mads - application/mads+xml - - - mag - application/vnd.ecowin.chart - - - maker - application/vnd.framemaker - - - man - text/troff - - - mar - application/octet-stream - - - mathml - application/mathml+xml - - - mb - application/mathematica - - - mbk - application/vnd.mobius.mbk - - - mbox - application/mbox - - - mc1 - application/vnd.medcalcdata - - - mcd - application/vnd.mcd - - - mcurl - text/vnd.curl.mcurl - - - mdb - application/x-msaccess - - - mdi - image/vnd.ms-modi - - - me - text/troff - - - mesh - model/mesh - - - meta4 - application/metalink4+xml - - - metalink - application/metalink+xml - - - mets - application/mets+xml - - - mfm - application/vnd.mfmp - - - mft - application/rpki-manifest - - - mgp - application/vnd.osgeo.mapguide.package - - - mgz - application/vnd.proteus.magazine - - - mid - audio/midi - - - midi - audio/midi - - - mie - application/x-mie - - - mif - application/x-mif - - - mime - message/rfc822 - - - mj2 - video/mj2 - - - mjp2 - video/mj2 - - - mk3d - video/x-matroska - - - mka - audio/x-matroska - - - mks - video/x-matroska - - - mkv - video/x-matroska - - - mlp - application/vnd.dolby.mlp - - - mmd - application/vnd.chipnuts.karaoke-mmd - - - mmf - application/vnd.smaf - - - mmr - image/vnd.fujixerox.edmics-mmr - - - mng - video/x-mng - - - mny - application/x-msmoney - - - mobi - application/x-mobipocket-ebook - - - mods - application/mods+xml - - - mov - video/quicktime - - - movie - video/x-sgi-movie - - - mp1 - audio/mpeg - - - mp2 - audio/mpeg - - - mp21 - application/mp21 - - - mp2a - audio/mpeg - - - mp3 - audio/mpeg - - - mp4 - video/mp4 - - - mp4a - audio/mp4 - - - mp4s - application/mp4 - - - mp4v - video/mp4 - - - mpa - audio/mpeg - - - mpc - application/vnd.mophun.certificate - - - mpe - video/mpeg - - - mpeg - video/mpeg - - - mpega - audio/x-mpeg - - - mpg - video/mpeg - - - mpg4 - video/mp4 - - - mpga - audio/mpeg - - - mpkg - application/vnd.apple.installer+xml - - - mpm - application/vnd.blueice.multipass - - - mpn - application/vnd.mophun.application - - - mpp - application/vnd.ms-project - - - mpt - application/vnd.ms-project - - - mpv2 - video/mpeg2 - - - mpy - application/vnd.ibm.minipay - - - mqy - application/vnd.mobius.mqy - - - mrc - application/marc - - - mrcx - application/marcxml+xml - - - ms - text/troff - - - mscml - application/mediaservercontrol+xml - - - mseed - application/vnd.fdsn.mseed - - - mseq - application/vnd.mseq - - - msf - application/vnd.epson.msf - - - msh - model/mesh - - - msi - application/x-msdownload - - - msl - application/vnd.mobius.msl - - - msty - application/vnd.muvee.style - - - mts - model/vnd.mts - - - mus - application/vnd.musician - - - musicxml - application/vnd.recordare.musicxml+xml - - - mvb - application/x-msmediaview - - - mwf - application/vnd.mfer - - - mxf - application/mxf - - - mxl - application/vnd.recordare.musicxml - - - mxml - application/xv+xml - - - mxs - application/vnd.triscape.mxs - - - mxu - video/vnd.mpegurl - - - n-gage - application/vnd.nokia.n-gage.symbian.install - - - n3 - text/n3 - - - nb - application/mathematica - - - nbp - application/vnd.wolfram.player - - - nc - application/x-netcdf - - - ncx - application/x-dtbncx+xml - - - nfo - text/x-nfo - - - ngdat - application/vnd.nokia.n-gage.data - - - nitf - application/vnd.nitf - - - nlu - application/vnd.neurolanguage.nlu - - - nml - application/vnd.enliven - - - nnd - application/vnd.noblenet-directory - - - nns - application/vnd.noblenet-sealer - - - nnw - application/vnd.noblenet-web - - - npx - image/vnd.net-fpx - - - nsc - application/x-conference - - - nsf - application/vnd.lotus-notes - - - ntf - application/vnd.nitf - - - nzb - application/x-nzb - - - oa2 - application/vnd.fujitsu.oasys2 - - - oa3 - application/vnd.fujitsu.oasys3 - - - oas - application/vnd.fujitsu.oasys - - - obd - application/x-msbinder - - - obj - application/x-tgif - - - oda - application/oda - - - - odb - application/vnd.oasis.opendocument.database - - - - odc - application/vnd.oasis.opendocument.chart - - - - odf - application/vnd.oasis.opendocument.formula - - - odft - application/vnd.oasis.opendocument.formula-template - - - - odg - application/vnd.oasis.opendocument.graphics - - - - odi - application/vnd.oasis.opendocument.image - - - - odm - application/vnd.oasis.opendocument.text-master - - - - odp - application/vnd.oasis.opendocument.presentation - - - - ods - application/vnd.oasis.opendocument.spreadsheet - - - - odt - application/vnd.oasis.opendocument.text - - - oga - audio/ogg - - - ogg - audio/ogg - - - ogv - video/ogg - - - - ogx - application/ogg - - - omdoc - application/omdoc+xml - - - onepkg - application/onenote - - - onetmp - application/onenote - - - onetoc - application/onenote - - - onetoc2 - application/onenote - - - opf - application/oebps-package+xml - - - opml - text/x-opml - - - oprc - application/vnd.palm - - - org - application/vnd.lotus-organizer - - - osf - application/vnd.yamaha.openscoreformat - - - osfpvg - application/vnd.yamaha.openscoreformat.osfpvg+xml - - - otc - application/vnd.oasis.opendocument.chart-template - - - otf - application/x-font-otf - - - - otg - application/vnd.oasis.opendocument.graphics-template - - - - oth - application/vnd.oasis.opendocument.text-web - - - oti - application/vnd.oasis.opendocument.image-template - - - - otp - application/vnd.oasis.opendocument.presentation-template - - - - ots - application/vnd.oasis.opendocument.spreadsheet-template - - - - ott - application/vnd.oasis.opendocument.text-template - - - oxps - application/oxps - - - oxt - application/vnd.openofficeorg.extension - - - p - text/x-pascal - - - p10 - application/pkcs10 - - - p12 - application/x-pkcs12 - - - p7b - application/x-pkcs7-certificates - - - p7c - application/pkcs7-mime - - - p7m - application/pkcs7-mime - - - p7r - application/x-pkcs7-certreqresp - - - p7s - application/pkcs7-signature - - - p8 - application/pkcs8 - - - pas - text/x-pascal - - - paw - application/vnd.pawaafile - - - pbd - application/vnd.powerbuilder6 - - - pbm - image/x-portable-bitmap - - - pcap - application/vnd.tcpdump.pcap - - - pcf - application/x-font-pcf - - - pcl - application/vnd.hp-pcl - - - pclxl - application/vnd.hp-pclxl - - - pct - image/pict - - - pcurl - application/vnd.curl.pcurl - - - pcx - image/x-pcx - - - pdb - application/vnd.palm - - - pdf - application/pdf - - - pfa - application/x-font-type1 - - - pfb - application/x-font-type1 - - - pfm - application/x-font-type1 - - - pfr - application/font-tdpfr - - - pfx - application/x-pkcs12 - - - pgm - image/x-portable-graymap - - - pgn - application/x-chess-pgn - - - pgp - application/pgp-encrypted - - - pic - image/pict - - - pict - image/pict - - - pkg - application/octet-stream - - - pki - application/pkixcmp - - - pkipath - application/pkix-pkipath - - - plb - application/vnd.3gpp.pic-bw-large - - - plc - application/vnd.mobius.plc - - - plf - application/vnd.pocketlearn - - - pls - audio/x-scpls - - - pml - application/vnd.ctc-posml - - - png - image/png - - - pnm - image/x-portable-anymap - - - pnt - image/x-macpaint - - - portpkg - application/vnd.macports.portpkg - - - pot - application/vnd.ms-powerpoint - - - potm - application/vnd.ms-powerpoint.template.macroenabled.12 - - - potx - application/vnd.openxmlformats-officedocument.presentationml.template - - - ppam - application/vnd.ms-powerpoint.addin.macroenabled.12 - - - ppd - application/vnd.cups-ppd - - - ppm - image/x-portable-pixmap - - - pps - application/vnd.ms-powerpoint - - - ppsm - application/vnd.ms-powerpoint.slideshow.macroenabled.12 - - - ppsx - application/vnd.openxmlformats-officedocument.presentationml.slideshow - - - ppt - application/vnd.ms-powerpoint - - - pptm - application/vnd.ms-powerpoint.presentation.macroenabled.12 - - - pptx - application/vnd.openxmlformats-officedocument.presentationml.presentation - - - pqa - application/vnd.palm - - - prc - application/x-mobipocket-ebook - - - pre - application/vnd.lotus-freelance - - - prf - application/pics-rules - - - ps - application/postscript - - - psb - application/vnd.3gpp.pic-bw-small - - - psd - image/vnd.adobe.photoshop - - - psf - application/x-font-linux-psf - - - pskcxml - application/pskc+xml - - - ptid - application/vnd.pvi.ptid1 - - - pub - application/x-mspublisher - - - pvb - application/vnd.3gpp.pic-bw-var - - - pwn - application/vnd.3m.post-it-notes - - - pya - audio/vnd.ms-playready.media.pya - - - pyv - video/vnd.ms-playready.media.pyv - - - qam - application/vnd.epson.quickanime - - - qbo - application/vnd.intu.qbo - - - qfx - application/vnd.intu.qfx - - - qps - application/vnd.publishare-delta-tree - - - qt - video/quicktime - - - qti - image/x-quicktime - - - qtif - image/x-quicktime - - - qwd - application/vnd.quark.quarkxpress - - - qwt - application/vnd.quark.quarkxpress - - - qxb - application/vnd.quark.quarkxpress - - - qxd - application/vnd.quark.quarkxpress - - - qxl - application/vnd.quark.quarkxpress - - - qxt - application/vnd.quark.quarkxpress - - - ra - audio/x-pn-realaudio - - - ram - audio/x-pn-realaudio - - - rar - application/x-rar-compressed - - - ras - image/x-cmu-raster - - - rcprofile - application/vnd.ipunplugged.rcprofile - - - rdf - application/rdf+xml - - - rdz - application/vnd.data-vision.rdz - - - rep - application/vnd.businessobjects - - - res - application/x-dtbresource+xml - - - rgb - image/x-rgb - - - rif - application/reginfo+xml - - - rip - audio/vnd.rip - - - ris - application/x-research-info-systems - - - rl - application/resource-lists+xml - - - rlc - image/vnd.fujixerox.edmics-rlc - - - rld - application/resource-lists-diff+xml - - - rm - application/vnd.rn-realmedia - - - rmi - audio/midi - - - rmp - audio/x-pn-realaudio-plugin - - - rms - application/vnd.jcp.javame.midlet-rms - - - rmvb - application/vnd.rn-realmedia-vbr - - - rnc - application/relax-ng-compact-syntax - - - roa - application/rpki-roa - - - roff - text/troff - - - rp9 - application/vnd.cloanto.rp9 - - - rpss - application/vnd.nokia.radio-presets - - - rpst - application/vnd.nokia.radio-preset - - - rq - application/sparql-query - - - rs - application/rls-services+xml - - - rsd - application/rsd+xml - - - rss - application/rss+xml - - - rtf - application/rtf - - - rtx - text/richtext - - - s - text/x-asm - - - s3m - audio/s3m - - - saf - application/vnd.yamaha.smaf-audio - - - sbml - application/sbml+xml - - - sc - application/vnd.ibm.secure-container - - - scd - application/x-msschedule - - - scm - application/vnd.lotus-screencam - - - scq - application/scvp-cv-request - - - scs - application/scvp-cv-response - - - scurl - text/vnd.curl.scurl - - - sda - application/vnd.stardivision.draw - - - sdc - application/vnd.stardivision.calc - - - sdd - application/vnd.stardivision.impress - - - sdkd - application/vnd.solent.sdkm+xml - - - sdkm - application/vnd.solent.sdkm+xml - - - sdp - application/sdp - - - sdw - application/vnd.stardivision.writer - - - see - application/vnd.seemail - - - seed - application/vnd.fdsn.seed - - - sema - application/vnd.sema - - - semd - application/vnd.semd - - - semf - application/vnd.semf - - - ser - application/java-serialized-object - - - setpay - application/set-payment-initiation - - - setreg - application/set-registration-initiation - - - sfd-hdstx - application/vnd.hydrostatix.sof-data - - - sfs - application/vnd.spotfire.sfs - - - sfv - text/x-sfv - - - sgi - image/sgi - - - sgl - application/vnd.stardivision.writer-global - - - sgm - text/sgml - - - sgml - text/sgml - - - sh - application/x-sh - - - shar - application/x-shar - - - shf - application/shf+xml - - - - sid - image/x-mrsid-image - - - sig - application/pgp-signature - - - sil - audio/silk - - - silo - model/mesh - - - sis - application/vnd.symbian.install - - - sisx - application/vnd.symbian.install - - - sit - application/x-stuffit - - - sitx - application/x-stuffitx - - - skd - application/vnd.koan - - - skm - application/vnd.koan - - - skp - application/vnd.koan - - - skt - application/vnd.koan - - - sldm - application/vnd.ms-powerpoint.slide.macroenabled.12 - - - sldx - application/vnd.openxmlformats-officedocument.presentationml.slide - - - slt - application/vnd.epson.salt - - - sm - application/vnd.stepmania.stepchart - - - smf - application/vnd.stardivision.math - - - smi - application/smil+xml - - - smil - application/smil+xml - - - smv - video/x-smv - - - smzip - application/vnd.stepmania.package - - - snd - audio/basic - - - snf - application/x-font-snf - - - so - application/octet-stream - - - spc - application/x-pkcs7-certificates - - - spf - application/vnd.yamaha.smaf-phrase - - - spl - application/x-futuresplash - - - spot - text/vnd.in3d.spot - - - spp - application/scvp-vp-response - - - spq - application/scvp-vp-request - - - spx - audio/ogg - - - sql - application/x-sql - - - src - application/x-wais-source - - - srt - application/x-subrip - - - sru - application/sru+xml - - - srx - application/sparql-results+xml - - - ssdl - application/ssdl+xml - - - sse - application/vnd.kodak-descriptor - - - ssf - application/vnd.epson.ssf - - - ssml - application/ssml+xml - - - st - application/vnd.sailingtracker.track - - - stc - application/vnd.sun.xml.calc.template - - - std - application/vnd.sun.xml.draw.template - - - stf - application/vnd.wt.stf - - - sti - application/vnd.sun.xml.impress.template - - - stk - application/hyperstudio - - - stl - application/vnd.ms-pki.stl - - - str - application/vnd.pg.format - - - stw - application/vnd.sun.xml.writer.template - - - sub - text/vnd.dvb.subtitle - - - sus - application/vnd.sus-calendar - - - susp - application/vnd.sus-calendar - - - sv4cpio - application/x-sv4cpio - - - sv4crc - application/x-sv4crc - - - svc - application/vnd.dvb.service - - - svd - application/vnd.svd - - - svg - image/svg+xml - - - svgz - image/svg+xml - - - swa - application/x-director - - - swf - application/x-shockwave-flash - - - swi - application/vnd.aristanetworks.swi - - - sxc - application/vnd.sun.xml.calc - - - sxd - application/vnd.sun.xml.draw - - - sxg - application/vnd.sun.xml.writer.global - - - sxi - application/vnd.sun.xml.impress - - - sxm - application/vnd.sun.xml.math - - - sxw - application/vnd.sun.xml.writer - - - t - text/troff - - - t3 - application/x-t3vm-image - - - taglet - application/vnd.mynfc - - - tao - application/vnd.tao.intent-module-archive - - - tar - application/x-tar - - - tcap - application/vnd.3gpp2.tcap - - - tcl - application/x-tcl - - - teacher - application/vnd.smart.teacher - - - tei - application/tei+xml - - - teicorpus - application/tei+xml - - - tex - application/x-tex - - - texi - application/x-texinfo - - - texinfo - application/x-texinfo - - - text - text/plain - - - tfi - application/thraud+xml - - - tfm - application/x-tex-tfm - - - tga - image/x-tga - - - thmx - application/vnd.ms-officetheme - - - tif - image/tiff - - - tiff - image/tiff - - - tmo - application/vnd.tmobile-livetv - - - torrent - application/x-bittorrent - - - tpl - application/vnd.groove-tool-template - - - tpt - application/vnd.trid.tpt - - - tr - text/troff - - - tra - application/vnd.trueapp - - - trm - application/x-msterminal - - - tsd - application/timestamped-data - - - tsv - text/tab-separated-values - - - ttc - application/x-font-ttf - - - ttf - application/x-font-ttf - - - ttl - text/turtle - - - twd - application/vnd.simtech-mindmapper - - - twds - application/vnd.simtech-mindmapper - - - txd - application/vnd.genomatix.tuxedo - - - txf - application/vnd.mobius.txf - - - txt - text/plain - - - u32 - application/x-authorware-bin - - - udeb - application/x-debian-package - - - ufd - application/vnd.ufdl - - - ufdl - application/vnd.ufdl - - - ulw - audio/basic - - - ulx - application/x-glulx - - - umj - application/vnd.umajin - - - unityweb - application/vnd.unity - - - uoml - application/vnd.uoml+xml - - - uri - text/uri-list - - - uris - text/uri-list - - - urls - text/uri-list - - - ustar - application/x-ustar - - - utz - application/vnd.uiq.theme - - - uu - text/x-uuencode - - - uva - audio/vnd.dece.audio - - - uvd - application/vnd.dece.data - - - uvf - application/vnd.dece.data - - - uvg - image/vnd.dece.graphic - - - uvh - video/vnd.dece.hd - - - uvi - image/vnd.dece.graphic - - - uvm - video/vnd.dece.mobile - - - uvp - video/vnd.dece.pd - - - uvs - video/vnd.dece.sd - - - uvt - application/vnd.dece.ttml+xml - - - uvu - video/vnd.uvvu.mp4 - - - uvv - video/vnd.dece.video - - - uvva - audio/vnd.dece.audio - - - uvvd - application/vnd.dece.data - - - uvvf - application/vnd.dece.data - - - uvvg - image/vnd.dece.graphic - - - uvvh - video/vnd.dece.hd - - - uvvi - image/vnd.dece.graphic - - - uvvm - video/vnd.dece.mobile - - - uvvp - video/vnd.dece.pd - - - uvvs - video/vnd.dece.sd - - - uvvt - application/vnd.dece.ttml+xml - - - uvvu - video/vnd.uvvu.mp4 - - - uvvv - video/vnd.dece.video - - - uvvx - application/vnd.dece.unspecified - - - uvvz - application/vnd.dece.zip - - - uvx - application/vnd.dece.unspecified - - - uvz - application/vnd.dece.zip - - - vcard - text/vcard - - - vcd - application/x-cdlink - - - vcf - text/x-vcard - - - vcg - application/vnd.groove-vcard - - - vcs - text/x-vcalendar - - - vcx - application/vnd.vcx - - - vis - application/vnd.visionary - - - viv - video/vnd.vivo - - - vob - video/x-ms-vob - - - vor - application/vnd.stardivision.writer - - - vox - application/x-authorware-bin - - - vrml - model/vrml - - - vsd - application/vnd.visio - - - vsf - application/vnd.vsf - - - vss - application/vnd.visio - - - vst - application/vnd.visio - - - vsw - application/vnd.visio - - - vtu - model/vnd.vtu - - - vxml - application/voicexml+xml - - - w3d - application/x-director - - - wad - application/x-doom - - - wav - audio/x-wav - - - wax - audio/x-ms-wax - - - - wbmp - image/vnd.wap.wbmp - - - wbs - application/vnd.criticaltools.wbs+xml - - - wbxml - application/vnd.wap.wbxml - - - wcm - application/vnd.ms-works - - - wdb - application/vnd.ms-works - - - wdp - image/vnd.ms-photo - - - weba - audio/webm - - - webm - video/webm - - - webp - image/webp - - - wg - application/vnd.pmi.widget - - - wgt - application/widget - - - wks - application/vnd.ms-works - - - wm - video/x-ms-wm - - - wma - audio/x-ms-wma - - - wmd - application/x-ms-wmd - - - wmf - application/x-msmetafile - - - - wml - text/vnd.wap.wml - - - - wmlc - application/vnd.wap.wmlc - - - - wmls - text/vnd.wap.wmlscript - - - - wmlsc - application/vnd.wap.wmlscriptc - - - wmv - video/x-ms-wmv - - - wmx - video/x-ms-wmx - - - wmz - application/x-msmetafile - - - woff - application/x-font-woff - - - wpd - application/vnd.wordperfect - - - wpl - application/vnd.ms-wpl - - - wps - application/vnd.ms-works - - - wqd - application/vnd.wqd - - - wri - application/x-mswrite - - - wrl - model/vrml - - - wsdl - application/wsdl+xml - - - wspolicy - application/wspolicy+xml - - - wtb - application/vnd.webturbo - - - wvx - video/x-ms-wvx - - - x32 - application/x-authorware-bin - - - x3d - model/x3d+xml - - - x3db - model/x3d+binary - - - x3dbz - model/x3d+binary - - - x3dv - model/x3d+vrml - - - x3dvz - model/x3d+vrml - - - x3dz - model/x3d+xml - - - xaml - application/xaml+xml - - - xap - application/x-silverlight-app - - - xar - application/vnd.xara - - - xbap - application/x-ms-xbap - - - xbd - application/vnd.fujixerox.docuworks.binder - - - xbm - image/x-xbitmap - - - xdf - application/xcap-diff+xml - - - xdm - application/vnd.syncml.dm+xml - - - xdp - application/vnd.adobe.xdp+xml - - - xdssc - application/dssc+xml - - - xdw - application/vnd.fujixerox.docuworks - - - xenc - application/xenc+xml - - - xer - application/patch-ops-error+xml - - - xfdf - application/vnd.adobe.xfdf - - - xfdl - application/vnd.xfdl - - - xht - application/xhtml+xml - - - xhtml - application/xhtml+xml - - - xhvml - application/xv+xml - - - xif - image/vnd.xiff - - - xla - application/vnd.ms-excel - - - xlam - application/vnd.ms-excel.addin.macroenabled.12 - - - xlc - application/vnd.ms-excel - - - xlf - application/x-xliff+xml - - - xlm - application/vnd.ms-excel - - - xls - application/vnd.ms-excel - - - xlsb - application/vnd.ms-excel.sheet.binary.macroenabled.12 - - - xlsm - application/vnd.ms-excel.sheet.macroenabled.12 - - - xlsx - application/vnd.openxmlformats-officedocument.spreadsheetml.sheet - - - xlt - application/vnd.ms-excel - - - xltm - application/vnd.ms-excel.template.macroenabled.12 - - - xltx - application/vnd.openxmlformats-officedocument.spreadsheetml.template - - - xlw - application/vnd.ms-excel - - - xm - audio/xm - - - xml - application/xml - - - xo - application/vnd.olpc-sugar - - - xop - application/xop+xml - - - xpi - application/x-xpinstall - - - xpl - application/xproc+xml - - - xpm - image/x-xpixmap - - - xpr - application/vnd.is-xpr - - - xps - application/vnd.ms-xpsdocument - - - xpw - application/vnd.intercon.formnet - - - xpx - application/vnd.intercon.formnet - - - xsl - application/xml - - - xslt - application/xslt+xml - - - xsm - application/vnd.syncml+xml - - - xspf - application/xspf+xml - - - xul - application/vnd.mozilla.xul+xml - - - xvm - application/xv+xml - - - xvml - application/xv+xml - - - xwd - image/x-xwindowdump - - - xyz - chemical/x-xyz - - - xz - application/x-xz - - - yang - application/yang - - - yin - application/yin+xml - - - z - application/x-compress - - - Z - application/x-compress - - - z1 - application/x-zmachine - - - z2 - application/x-zmachine - - - z3 - application/x-zmachine - - - z4 - application/x-zmachine - - - z5 - application/x-zmachine - - - z6 - application/x-zmachine - - - z7 - application/x-zmachine - - - z8 - application/x-zmachine - - - zaz - application/vnd.zzazz.deck+xml - - - zip - application/zip - - - zir - application/vnd.zul - - - zirz - application/vnd.zul - - - zmm - application/vnd.handheld-entertainment+xml - - - - - - - - - - - - - - - - - - index.html - index.htm - index.jsp - - - diff -Nru snapcraft-2.27.1+17.04/demos/tomcat-maven-webapp/wrapper snapcraft-2.28+17.04/demos/tomcat-maven-webapp/wrapper --- snapcraft-2.27.1+17.04/demos/tomcat-maven-webapp/wrapper 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/demos/tomcat-maven-webapp/wrapper 1970-01-01 00:00:00.000000000 +0000 @@ -1,17 +0,0 @@ -#!/bin/sh - -set -e -set -x - -# create runtime data -mkdir -p "$CATALINA_BASE/logs" -mkdir -p "$CATALINA_BASE/temp" -if ! [ -d $CATALINA_BASE/conf ]; then - cp -rd $CATALINA_HOME/tomcat-conf $CATALINA_BASE/conf -fi -if ! [ -d $CATALINA_BASE/webapps ]; then - cp -rd $CATALINA_HOME/webapps $CATALINA_BASE/ - cp $CATALINA_HOME/war/*.war $CATALINA_BASE/webapps/ -fi - -$CATALINA_HOME/bin/catalina.sh run diff -Nru snapcraft-2.27.1+17.04/external_snaps_tests/__main__.py snapcraft-2.28+17.04/external_snaps_tests/__main__.py --- snapcraft-2.27.1+17.04/external_snaps_tests/__main__.py 2016-12-15 17:13:16.000000000 +0000 +++ snapcraft-2.28+17.04/external_snaps_tests/__main__.py 2017-03-22 12:31:58.000000000 +0000 @@ -1,6 +1,6 @@ # -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*- # -# Copyright (C) 2016 Canonical Ltd +# Copyright (C) 2016-2017 Canonical Ltd # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License version 3 as @@ -53,12 +53,18 @@ if _is_git(repo): if shutil.which('git'): path = _git_clone(repo, repo_branch) - _build_snaps(path, cleanbuild, keep_dir) else: sys.exit('Please install git.') + elif _is_bzr(repo): + if shutil.which('bzr'): + path = _bzr_branch(repo) + else: + sys.exit('Please install bzr.') else: sys.exit('Unsupported repository.') + _build_snaps(path, cleanbuild, keep_dir) + def _is_git(repo): return (repo.startswith('https://github.com/') or @@ -68,7 +74,7 @@ def _git_clone(url, repo_branch=None): temp_dir = tempfile.mkdtemp(prefix='snapcraft-') - command = ['git', 'clone', url, temp_dir] + command = ['git', 'clone', '--progress', url, temp_dir] print(' '.join(command)) subprocess.check_call(command) if repo_branch: @@ -76,11 +82,26 @@ return temp_dir +def _is_bzr(repo): + return repo.startswith('lp:') + + +def _bzr_branch(url): + temp_dir = tempfile.mkdtemp(prefix='snapcraft-') + repo_dir = os.path.join(temp_dir, 'repo') + command = ['bzr', 'branch', '-v', url, repo_dir] + print(' '.join(command)) + subprocess.check_call(command) + return repo_dir + + def _build_snaps(path, cleanbuild=False, keep_dir=False): try: - for dirpath, _, filenames in os.walk(path): - if 'snapcraft.yaml' in filenames or '.snapcraft.yaml' in filenames: + for dirpath, dirnames, filenames in os.walk(path, topdown=True): + if _is_snapcraft_dir(dirpath, dirnames, filenames): _build_snap(dirpath, cleanbuild, keep_dir) + # Do not recurse in any directory. + del dirnames[:] except subprocess.CalledProcessError as e: sys.exit(e.returncode) finally: @@ -92,6 +113,13 @@ shutil.rmtree(path) +def _is_snapcraft_dir(dirpath, dirnames, filenames): + return (('snap' in dirnames and + 'snapcraft.yaml' in os.listdir(os.path.join(dirpath, 'snap'))) or + 'snapcraft.yaml' in filenames or + '.snapcraft.yaml' in filenames) + + def _build_snap(path, cleanbuild=False, keep_dir=False): snapcraft = os.path.abspath(os.path.join('bin', 'snapcraft')) print('Updating the parts cache...') diff -Nru snapcraft-2.27.1+17.04/integration_tests/both_parts_wiki snapcraft-2.28+17.04/integration_tests/both_parts_wiki --- snapcraft-2.27.1+17.04/integration_tests/both_parts_wiki 2016-11-02 08:25:17.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/both_parts_wiki 1970-01-01 00:00:00.000000000 +0000 @@ -1,5 +0,0 @@ ---- -description: a test part -maintainer: John Doe -parts: [testpart-both] -origin: lp:~joetalbott/+junk/testpart-both diff -Nru snapcraft-2.27.1+17.04/integration_tests/hidden_parts_wiki snapcraft-2.28+17.04/integration_tests/hidden_parts_wiki --- snapcraft-2.27.1+17.04/integration_tests/hidden_parts_wiki 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/hidden_parts_wiki 1970-01-01 00:00:00.000000000 +0000 @@ -1,5 +0,0 @@ ---- -description: a test part -maintainer: John Doe -parts: [hidden-testpart] -origin: lp:~joetalbott/+junk/testpart diff -Nru snapcraft-2.27.1+17.04/integration_tests/__init__.py snapcraft-2.28+17.04/integration_tests/__init__.py --- snapcraft-2.27.1+17.04/integration_tests/__init__.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/__init__.py 2017-03-23 12:02:09.000000000 +0000 @@ -16,6 +16,7 @@ import fileinput import os +import re import subprocess import time import uuid @@ -27,7 +28,7 @@ from unittest import mock import testtools from testtools import content -from testtools.matchers import Contains +from testtools.matchers import MatchesRegex from snapcraft.tests import fixture_setup @@ -93,7 +94,9 @@ self.stage_dir = 'stage' self.prime_dir = 'prime' - def run_snapcraft(self, command, project_dir=None, debug=True): + def run_snapcraft( + self, command, project_dir=None, debug=True, + pre_func=lambda: None): if project_dir: self.copy_project_to_cwd(project_dir) @@ -103,6 +106,7 @@ if debug: snapcraft_command.append('-d') try: + pre_func() snapcraft_output = subprocess.check_output( snapcraft_command + command, stderr=subprocess.STDOUT, universal_newlines=True) @@ -153,7 +157,7 @@ # Because cwd already exists, shutil.copytree would raise # FileExistsError. Use the lesser known distutils.dir_util.copy_tree dir_util.copy_tree( - os.path.join(self.snaps_dir, project_dir), os.getcwd(), + os.path.join(self.snaps_dir, project_dir), self.path, preserve_symlinks=True) def get_output_ignoring_non_zero_exit(self, binary, cwd=None): @@ -194,9 +198,9 @@ def logout(self): output = self.run_snapcraft('logout') - expected = ('Clearing credentials for Ubuntu One SSO.\n' - 'Credentials cleared.\n') - self.assertThat(output, Contains(expected)) + expected = (r'.*Clearing credentials for Ubuntu One SSO.\n.*\n' + r'Credentials cleared.\n.*') + self.assertThat(output, MatchesRegex(expected, flags=re.DOTALL)) def register(self, snap_name, private=False, wait=True): command = ['register', snap_name] diff -Nru snapcraft-2.27.1+17.04/integration_tests/missing_parts_wiki snapcraft-2.28+17.04/integration_tests/missing_parts_wiki --- snapcraft-2.27.1+17.04/integration_tests/missing_parts_wiki 2016-11-02 08:25:17.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/missing_parts_wiki 1970-01-01 00:00:00.000000000 +0000 @@ -1,5 +0,0 @@ ---- -description: a test part -maintainer: John Doe -parts: [testpart-missing] -origin: lp:~joetalbott/+junk/testpart-missing diff -Nru snapcraft-2.27.1+17.04/integration_tests/origin_options_wiki snapcraft-2.28+17.04/integration_tests/origin_options_wiki --- snapcraft-2.27.1+17.04/integration_tests/origin_options_wiki 2017-01-28 06:12:26.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/origin_options_wiki 1970-01-01 00:00:00.000000000 +0000 @@ -1,8 +0,0 @@ ---- -description: Wiki with origin modifiers -maintainer: Marco Trevisan -parts: [desktop-glib-only] -origin: https://github.com/3v1n0/snapcraft-desktop-helpers.git -origin-type: git -origin-branch: snap-launcher-arch -origin-commit: 91cc92f0a0c diff -Nru snapcraft-2.27.1+17.04/integration_tests/snaps/asset-tracking/snapcraft.yaml snapcraft-2.28+17.04/integration_tests/snaps/asset-tracking/snapcraft.yaml --- snapcraft-2.27.1+17.04/integration_tests/snaps/asset-tracking/snapcraft.yaml 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/snaps/asset-tracking/snapcraft.yaml 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,11 @@ +name: asset-tracking +version: 0.1 +summary: Summary of the most simple snap +description: Description of the most simple snap +architectures: [all] +confinement: strict + +parts: + asset-tracking: + plugin: nil + stage-packages: ['hello=2.10-1'] diff -Nru snapcraft-2.27.1+17.04/integration_tests/snaps/build-package-version/snap/snapcraft.yaml snapcraft-2.28+17.04/integration_tests/snaps/build-package-version/snap/snapcraft.yaml --- snapcraft-2.27.1+17.04/integration_tests/snaps/build-package-version/snap/snapcraft.yaml 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/snaps/build-package-version/snap/snapcraft.yaml 2017-03-23 18:31:04.000000000 +0000 @@ -0,0 +1,13 @@ +name: build-package-version +version: '0.1' +summary: install a specific version of a build-package +description: | + Install a specific version of a build-package. + +grade: stable +confinement: strict + +parts: + hello: + plugin: nil + build-packages: ['hello=2.10-1'] diff -Nru snapcraft-2.27.1+17.04/integration_tests/snaps/build-package-version-bad/snap/snapcraft.yaml snapcraft-2.28+17.04/integration_tests/snaps/build-package-version-bad/snap/snapcraft.yaml --- snapcraft-2.27.1+17.04/integration_tests/snaps/build-package-version-bad/snap/snapcraft.yaml 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/snaps/build-package-version-bad/snap/snapcraft.yaml 2017-03-23 12:02:13.000000000 +0000 @@ -0,0 +1,13 @@ +name: build-package-version +version: '0.1' +summary: install a specific, but non-existent, version of a build-package +description: | + Install a specific, but non-existent version of a build-package. + +grade: stable +confinement: strict + +parts: + hello: + plugin: nil + build-packages: ['hello=x.y-z'] diff -Nru snapcraft-2.27.1+17.04/integration_tests/snaps/build-package-version-global/snap/snapcraft.yaml snapcraft-2.28+17.04/integration_tests/snaps/build-package-version-global/snap/snapcraft.yaml --- snapcraft-2.27.1+17.04/integration_tests/snaps/build-package-version-global/snap/snapcraft.yaml 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/snaps/build-package-version-global/snap/snapcraft.yaml 2017-03-23 18:31:04.000000000 +0000 @@ -0,0 +1,13 @@ +name: build-package-version +version: '0.1' +summary: install a specific version of a build-package +description: | + Install a specific version of a global build-package. + +grade: stable +confinement: strict +build-packages: ['hello=2.10-1'] + +parts: + hello: + plugin: nil Binary files /tmp/tmpgQQeeV/kVJBirsbFX/snapcraft-2.27.1+17.04/integration_tests/snaps/checksum-algorithms/checksum.tar.gz and /tmp/tmpgQQeeV/SnLglWZAN7/snapcraft-2.28+17.04/integration_tests/snaps/checksum-algorithms/checksum.tar.gz differ diff -Nru snapcraft-2.27.1+17.04/integration_tests/snaps/checksum-algorithms/snapcraft.yaml snapcraft-2.28+17.04/integration_tests/snaps/checksum-algorithms/snapcraft.yaml --- snapcraft-2.27.1+17.04/integration_tests/snaps/checksum-algorithms/snapcraft.yaml 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/snaps/checksum-algorithms/snapcraft.yaml 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,44 @@ +name: valid-checksums +version: 1.0 +summary: Valid source-checksum's +description: | + Valid source checksums that snapcraft should accept. +confinement: strict + +parts: + checksum-md5: + plugin: dump + source: checksum.tar.gz + source-checksum: md5/d9210476aac5f367b14e513bdefdee08 + checksum-sha1: + plugin: dump + source: checksum.tar.gz + source-checksum: sha1/8d5a102a7e7e99f8962a4de2c348a96b7d54200b + checksum-sha224: + plugin: dump + source: checksum.tar.gz + source-checksum: sha224/de2fb61252548af3c87c4aab17e82601691d19e37fd3d29ea6288e56 + checksum-sha256: + plugin: dump + source: checksum.tar.gz + source-checksum: sha256/85e9d25042f92fa0a53c19d303f67036b9aab81e4c63d85319cd5577ca0d5081 + checksum-sha384: + plugin: dump + source: checksum.tar.gz + source-checksum: sha384/4d3110ea9d1683dde8624432bd4ce8afc00766f5de01077aa8d51f313e16dee18196e5f1a5d38325942408793b4b7e78 + checksum-sha512: + plugin: dump + source: checksum.tar.gz + source-checksum: sha512/b40dd8099c3dd299b889d1981fb1e87dd058134cf38146615648753dffb7bead6eac0e9bacb53c553873406eae702632e432e70383b2a345ddc5768f6c7007e4 + checksum-sha3-256: + plugin: dump + source: checksum.tar.gz + source-checksum: sha3_256/7e0777196597af4d8d7a8a561bca3212953ff63a3de59aba6ec4580abaea8719 + checksum-sha3-384: + plugin: dump + source: checksum.tar.gz + source-checksum: sha3_384/f61220214092d345a9908c45d76f0031060e28b6cd808b3ea7b289d9b852eaa0842d3433615448dd09c92b09f1da343a + checksum-sha3-512: + plugin: dump + source: checksum.tar.gz + source-checksum: sha3_512/7c49d8eb5a9738474debccef5491cd34db6b61383a887387065010677c8ce9bb14479e556c561269398c86bf20f24afdd5d6f7d04009ac05d38b605c0cfc2681 Binary files /tmp/tmpgQQeeV/kVJBirsbFX/snapcraft-2.27.1+17.04/integration_tests/snaps/checksum-algorithms-invalid/checksum.tar.gz and /tmp/tmpgQQeeV/SnLglWZAN7/snapcraft-2.28+17.04/integration_tests/snaps/checksum-algorithms-invalid/checksum.tar.gz differ diff -Nru snapcraft-2.27.1+17.04/integration_tests/snaps/checksum-algorithms-invalid/snapcraft.yaml snapcraft-2.28+17.04/integration_tests/snaps/checksum-algorithms-invalid/snapcraft.yaml --- snapcraft-2.27.1+17.04/integration_tests/snaps/checksum-algorithms-invalid/snapcraft.yaml 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/snaps/checksum-algorithms-invalid/snapcraft.yaml 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,45 @@ +name: invalid-checksums +version: 1.0 +summary: Invalid source-checksum's +description: | + Invalid source checksums that snapcraft should + check and then raise an exception. +confinement: strict + +parts: + checksum-md5: + plugin: dump + source: checksum.tar.gz + source-checksum: md5/d9210476aac5f367b14e513bdefdee09 + checksum-sha1: + plugin: dump + source: checksum.tar.gz + source-checksum: sha1/8d5a102a7e7e99f8962a4de2c348a96b7d54200c + checksum-sha224: + plugin: dump + source: checksum.tar.gz + source-checksum: sha224/de2fb61252548af3c87c4aab17e82601691d19e37fd3d29ea6288e57 + checksum-sha256: + plugin: dump + source: checksum.tar.gz + source-checksum: sha256/85e9d25042f92fa0a53c19d303f67036b9aab81e4c63d85319cd5577ca0d5082 + checksum-sha384: + plugin: dump + source: checksum.tar.gz + source-checksum: sha384/4d3110ea9d1683dde8624432bd4ce8afc00766f5de01077aa8d51f313e16dee18196e5f1a5d38325942408793b4b7e79 + checksum-sha512: + plugin: dump + source: checksum.tar.gz + source-checksum: sha512/b40dd8099c3dd299b889d1981fb1e87dd058134cf38146615648753dffb7bead6eac0e9bacb53c553873406eae702632e432e70383b2a345ddc5768f6c7007e5 + checksum-sha3-256: + plugin: dump + source: checksum.tar.gz + source-checksum: sha3_256/7e0777196597af4d8d7a8a561bca3212953ff63a3de59aba6ec4580abaea871a + checksum-sha3-384: + plugin: dump + source: checksum.tar.gz + source-checksum: sha3_384/f61220214092d345a9908c45d76f0031060e28b6cd808b3ea7b289d9b852eaa0842d3433615448dd09c92b09f1da343b + checksum-sha3-512: + plugin: dump + source: checksum.tar.gz + source-checksum: sha3_512/7c49d8eb5a9738474debccef5491cd34db6b61383a887387065010677c8ce9bb14479e556c561269398c86bf20f24afdd5d6f7d04009ac05d38b605c0cfc2682 Binary files /tmp/tmpgQQeeV/kVJBirsbFX/snapcraft-2.27.1+17.04/integration_tests/snaps/deb-with-checksum/small_0.1-1.deb and /tmp/tmpgQQeeV/SnLglWZAN7/snapcraft-2.28+17.04/integration_tests/snaps/deb-with-checksum/small_0.1-1.deb differ diff -Nru snapcraft-2.27.1+17.04/integration_tests/snaps/deb-with-checksum/snapcraft.yaml snapcraft-2.28+17.04/integration_tests/snaps/deb-with-checksum/snapcraft.yaml --- snapcraft-2.27.1+17.04/integration_tests/snaps/deb-with-checksum/snapcraft.yaml 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/snaps/deb-with-checksum/snapcraft.yaml 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,14 @@ +name: deb-with-checksum +version: '0.1' +summary: test checksum with deb's sources +description: | + This integration tests verifies that deb files can be used with checksums. + +grade: devel +confinement: devmode + +parts: + deb-source-checksum: + plugin: dump + source: small_0.1-1.deb + source-checksum: sha256/b439909266aac7384e15d041b90e98d1e7fc774e5e1ac32801859882e7012182 diff -Nru snapcraft-2.27.1+17.04/integration_tests/snaps/ftp-source/snapcraft.yaml snapcraft-2.28+17.04/integration_tests/snaps/ftp-source/snapcraft.yaml --- snapcraft-2.27.1+17.04/integration_tests/snaps/ftp-source/snapcraft.yaml 2017-01-28 06:12:26.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/snaps/ftp-source/snapcraft.yaml 2017-03-22 12:31:58.000000000 +0000 @@ -1,5 +1,5 @@ name: ftp-source -version: 0.2.3 +version: 0.1 summary: Test downloading files from ftp source description: | Make sure sources can be fetched from FTP servers. @@ -9,6 +9,6 @@ confinement: devmode parts: - libwnck: + ftp-part: plugin: dump - source: ftp://ftp.kernel.org/pub/software/admin/A3Com/A3Com-0.2.3.tar.gz + source: ftp://localhost:2121/test.tar.gz diff -Nru snapcraft-2.27.1+17.04/integration_tests/snaps/plainbox-provider/snapcraft.yaml snapcraft-2.28+17.04/integration_tests/snaps/plainbox-provider/snapcraft.yaml --- snapcraft-2.27.1+17.04/integration_tests/snaps/plainbox-provider/snapcraft.yaml 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/snaps/plainbox-provider/snapcraft.yaml 2017-03-22 12:31:58.000000000 +0000 @@ -7,17 +7,38 @@ confinement: strict parts: - plainbox-local: - plugin: python3 - python-packages: - - plainbox - - requests-oauthlib + plainbox-dev: + plugin: python build-packages: - libxml2-dev - libxslt1-dev - zlib1g-dev - build-essential + python-packages: + - requests-oauthlib + - XlsxWriter + - Jinja2 + - guacamole + - padme + source: git://git.launchpad.net/plainbox + source-type: git + checkbox-support-dev: + plugin: python + source: git://git.launchpad.net/checkbox-support + source-type: git + checkbox-ng-dev: + plugin: python + source: git://git.launchpad.net/checkbox-ng + source-type: git + stage: + - bin/checkbox* + - lib/*/*/checkbox_ng*/* + - lib/*/*/checkbox_ng*/*/* + snap: + - bin/checkbox* + - lib/*/*/checkbox_ng*/* + - lib/*/*/checkbox_ng*/*/* simple-plainbox-provider: plugin: plainbox-provider source: ./2016.com.example:simple - after: [plainbox-local] + after: [plainbox-dev, checkbox-support-dev, checkbox-ng-dev] diff -Nru snapcraft-2.27.1+17.04/integration_tests/snaps/plainbox-provider-invalid/2016.com.example:simple/manage.py snapcraft-2.28+17.04/integration_tests/snaps/plainbox-provider-invalid/2016.com.example:simple/manage.py --- snapcraft-2.27.1+17.04/integration_tests/snaps/plainbox-provider-invalid/2016.com.example:simple/manage.py 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/snaps/plainbox-provider-invalid/2016.com.example:simple/manage.py 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,11 @@ +#!/usr/bin/env python3 +from plainbox.provider_manager import setup, N_ + + +setup( + name='plainbox-provider-simple', + namespace='2016.com.example', + version="1.0", + description=N_("A really simple Plainbox provider"), + gettext_domain="2016_com_example_simple", +) diff -Nru snapcraft-2.27.1+17.04/integration_tests/snaps/plainbox-provider-invalid/2016.com.example:simple/units/simple.pxu snapcraft-2.28+17.04/integration_tests/snaps/plainbox-provider-invalid/2016.com.example:simple/units/simple.pxu --- snapcraft-2.27.1+17.04/integration_tests/snaps/plainbox-provider-invalid/2016.com.example:simple/units/simple.pxu 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/snaps/plainbox-provider-invalid/2016.com.example:simple/units/simple.pxu 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,15 @@ + +unit: category +id: simple +_name: Simple + +unit: job +id: missing-command-job +category_id: simple +_summary: A job defintion that is missing a command field +_description: + This test is intentionally defined with a missing command field. This will + cause the validate step to fail during build. +estimated_duration: 0.01 +command: true +flags: preserve-locale diff -Nru snapcraft-2.27.1+17.04/integration_tests/snaps/plainbox-provider-invalid/snapcraft.yaml snapcraft-2.28+17.04/integration_tests/snaps/plainbox-provider-invalid/snapcraft.yaml --- snapcraft-2.27.1+17.04/integration_tests/snaps/plainbox-provider-invalid/snapcraft.yaml 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/snaps/plainbox-provider-invalid/snapcraft.yaml 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,23 @@ +name: test-package +version: 0.1 +summary: Create a snap of a plainbox provider +description: | + Create a snap of a very simple plainbox that could + then be used by a checkbox application +confinement: strict + +parts: + plainbox-local: + plugin: python3 + python-packages: + - plainbox + - requests-oauthlib + build-packages: + - libxml2-dev + - libxslt1-dev + - zlib1g-dev + - build-essential + simple-plainbox-provider: + plugin: plainbox-provider + source: ./2016.com.example:simple + after: [plainbox-local] diff -Nru snapcraft-2.27.1+17.04/integration_tests/snaps/plainbox-provider-with-deps/2017.com.example:child/manage.py snapcraft-2.28+17.04/integration_tests/snaps/plainbox-provider-with-deps/2017.com.example:child/manage.py --- snapcraft-2.27.1+17.04/integration_tests/snaps/plainbox-provider-with-deps/2017.com.example:child/manage.py 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/snaps/plainbox-provider-with-deps/2017.com.example:child/manage.py 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,11 @@ +#!/usr/bin/env python3 +from plainbox.provider_manager import setup, N_ + + +setup( + name='plainbox-provider-child', + namespace='2017.com.example.child', + version="1.0", + description=N_("A child Plainbox provider"), + gettext_domain="2017_com_example_child", +) diff -Nru snapcraft-2.27.1+17.04/integration_tests/snaps/plainbox-provider-with-deps/2017.com.example:child/units/child.pxu snapcraft-2.28+17.04/integration_tests/snaps/plainbox-provider-with-deps/2017.com.example:child/units/child.pxu --- snapcraft-2.27.1+17.04/integration_tests/snaps/plainbox-provider-with-deps/2017.com.example:child/units/child.pxu 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/snaps/plainbox-provider-with-deps/2017.com.example:child/units/child.pxu 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,20 @@ + +unit: category +id: child +_name: Child + +unit: template +template-resource: 2017.com.example.parent::myresource +template-filter: myresource.type == "DISK" +template-unit: job +id: echo-name-{type} +category_id: child +_summary: A test that prints the name of devices of the type DISK +_description: + This test has a dependency on the resource job from the parent provider. The + validate build step if the parent can't be found in the stage directory. +plugin: shell +estimated_duration: 0.01 +command: + echo "disk: {name}" +flags: preserve-locale diff -Nru snapcraft-2.27.1+17.04/integration_tests/snaps/plainbox-provider-with-deps/2017.com.example:parent/manage.py snapcraft-2.28+17.04/integration_tests/snaps/plainbox-provider-with-deps/2017.com.example:parent/manage.py --- snapcraft-2.27.1+17.04/integration_tests/snaps/plainbox-provider-with-deps/2017.com.example:parent/manage.py 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/snaps/plainbox-provider-with-deps/2017.com.example:parent/manage.py 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,11 @@ +#!/usr/bin/env python3 +from plainbox.provider_manager import setup, N_ + + +setup( + name='plainbox-provider-parent', + namespace='2017.com.example.parent', + version="1.0", + description=N_("A parent Plainbox provider"), + gettext_domain="2017_com_example_parent", +) diff -Nru snapcraft-2.27.1+17.04/integration_tests/snaps/plainbox-provider-with-deps/2017.com.example:parent/units/parent.pxu snapcraft-2.28+17.04/integration_tests/snaps/plainbox-provider-with-deps/2017.com.example:parent/units/parent.pxu --- snapcraft-2.27.1+17.04/integration_tests/snaps/plainbox-provider-with-deps/2017.com.example:parent/units/parent.pxu 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/snaps/plainbox-provider-with-deps/2017.com.example:parent/units/parent.pxu 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,20 @@ + +unit: category +id: parent +_name: Parent + +unit: job +id: myresource +category_id: parent +_summary: An example resource job +_description: + Create two resource objects +plugin: resource +estimated_duration: 0.01 +command: + echo "name: resource1" + echo "type: DISK" + echo "" + echo "name: resource2" + echo "type: AUDIO" +flags: preserve-locale diff -Nru snapcraft-2.27.1+17.04/integration_tests/snaps/plainbox-provider-with-deps/snapcraft.yaml snapcraft-2.28+17.04/integration_tests/snaps/plainbox-provider-with-deps/snapcraft.yaml --- snapcraft-2.27.1+17.04/integration_tests/snaps/plainbox-provider-with-deps/snapcraft.yaml 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/snaps/plainbox-provider-with-deps/snapcraft.yaml 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,27 @@ +name: test-package +version: 0.1 +summary: Create a snap of a plainbox provider +description: | + Create a snap that has two providers with an import of a parent test to the + child. This exercises the call of manage.py validate +confinement: strict + +parts: + plainbox-local: + plugin: python3 + python-packages: + - plainbox + - requests-oauthlib + build-packages: + - libxml2-dev + - libxslt1-dev + - zlib1g-dev + - build-essential + parent-plainbox-provider: + plugin: plainbox-provider + source: ./2017.com.example:parent + after: [plainbox-local] + child-plainbox-provider: + plugin: plainbox-provider + source: ./2017.com.example:child + after: [parent-plainbox-provider] diff -Nru snapcraft-2.27.1+17.04/integration_tests/snaps/python-pbr/python2/python2_test_package/main.py snapcraft-2.28+17.04/integration_tests/snaps/python-pbr/python2/python2_test_package/main.py --- snapcraft-2.27.1+17.04/integration_tests/snaps/python-pbr/python2/python2_test_package/main.py 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/snaps/python-pbr/python2/python2_test_package/main.py 2017-03-22 15:10:01.000000000 +0000 @@ -0,0 +1,9 @@ +#!/usr/bin/env python + + +def main(): + print("hello world!") + + +if __name__ == '__main__': + main() diff -Nru snapcraft-2.27.1+17.04/integration_tests/snaps/python-pbr/python2/setup.cfg snapcraft-2.28+17.04/integration_tests/snaps/python-pbr/python2/setup.cfg --- snapcraft-2.27.1+17.04/integration_tests/snaps/python-pbr/python2/setup.cfg 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/snaps/python-pbr/python2/setup.cfg 2017-03-22 15:10:01.000000000 +0000 @@ -0,0 +1,12 @@ +[metadata] +name = python2_test_package +summary = setup.cfg test +author = snapcraft + +[files] +packages= + python2_test_package + +[entry_points] +console_scripts = + python2_test = python2_test_package.main:main diff -Nru snapcraft-2.27.1+17.04/integration_tests/snaps/python-pbr/python2/setup.py snapcraft-2.28+17.04/integration_tests/snaps/python-pbr/python2/setup.py --- snapcraft-2.27.1+17.04/integration_tests/snaps/python-pbr/python2/setup.py 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/snaps/python-pbr/python2/setup.py 2017-03-22 15:10:01.000000000 +0000 @@ -0,0 +1,5 @@ +from setuptools import setup + +setup( + setup_requires=['pbr>=2.0.0'], + pbr=True) diff -Nru snapcraft-2.27.1+17.04/integration_tests/snaps/python-pbr/python3/python3_test_package/main.py snapcraft-2.28+17.04/integration_tests/snaps/python-pbr/python3/python3_test_package/main.py --- snapcraft-2.27.1+17.04/integration_tests/snaps/python-pbr/python3/python3_test_package/main.py 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/snaps/python-pbr/python3/python3_test_package/main.py 2017-03-22 15:10:01.000000000 +0000 @@ -0,0 +1,9 @@ +#!/usr/bin/env python3 + + +def main(): + print("hello world!") + + +if __name__ == '__main__': + main() diff -Nru snapcraft-2.27.1+17.04/integration_tests/snaps/python-pbr/python3/setup.cfg snapcraft-2.28+17.04/integration_tests/snaps/python-pbr/python3/setup.cfg --- snapcraft-2.27.1+17.04/integration_tests/snaps/python-pbr/python3/setup.cfg 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/snaps/python-pbr/python3/setup.cfg 2017-03-22 15:10:01.000000000 +0000 @@ -0,0 +1,12 @@ +[metadata] +name = python3_test_package +summary = setup.cfg test +author = snapcraft + +[files] +packages= + python3_test_package + +[entry_points] +console_scripts = + python3_test = python3_test_package.main:main diff -Nru snapcraft-2.27.1+17.04/integration_tests/snaps/python-pbr/python3/setup.py snapcraft-2.28+17.04/integration_tests/snaps/python-pbr/python3/setup.py --- snapcraft-2.27.1+17.04/integration_tests/snaps/python-pbr/python3/setup.py 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/snaps/python-pbr/python3/setup.py 2017-03-22 15:10:01.000000000 +0000 @@ -0,0 +1,5 @@ +from setuptools import setup + +setup( + setup_requires=['pbr>=2.0.0'], + pbr=True) diff -Nru snapcraft-2.27.1+17.04/integration_tests/snaps/python-pbr/snapcraft.yaml snapcraft-2.28+17.04/integration_tests/snaps/python-pbr/snapcraft.yaml --- snapcraft-2.27.1+17.04/integration_tests/snaps/python-pbr/snapcraft.yaml 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/snaps/python-pbr/snapcraft.yaml 2017-03-22 15:10:01.000000000 +0000 @@ -0,0 +1,26 @@ +name: python-setup-cfg +version: '1.0' +summary: Verify that setup.cfg is taken into account. +description: | + "Verify that entry-point/console-scripts in setup.cfg are taken + into account when using pbr. + This is a regression test for LP: #1670852" +confinement: strict +grade: devel + +parts: + python2-test: + source: python2/ + plugin: python + python-version: python2 + python-packages: + - pbr==2.0.0 + stage: + - -bin/pbr + + python3-test: + source: python3/ + plugin: python + python-version: python3 + python-packages: + - pbr==2.0.0 Binary files /tmp/tmpgQQeeV/kVJBirsbFX/snapcraft-2.27.1+17.04/integration_tests/snaps/rpm-with-checksum/small-0.1-1.noarch.rpm and /tmp/tmpgQQeeV/SnLglWZAN7/snapcraft-2.28+17.04/integration_tests/snaps/rpm-with-checksum/small-0.1-1.noarch.rpm differ diff -Nru snapcraft-2.27.1+17.04/integration_tests/snaps/rpm-with-checksum/snapcraft.yaml snapcraft-2.28+17.04/integration_tests/snaps/rpm-with-checksum/snapcraft.yaml --- snapcraft-2.27.1+17.04/integration_tests/snaps/rpm-with-checksum/snapcraft.yaml 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/snaps/rpm-with-checksum/snapcraft.yaml 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,14 @@ +name: rpm-with-checksum +version: '0.1' +summary: test checksum with RPMs sources +description: | + This integration test verifies that RPM files can be with checksums. + +grade: devel +confinement: devmode + +parts: + rpm-source-checksum: + plugin: dump + source: small-0.1-1.noarch.rpm + source-checksum: sha256/130cd966bc867e0583398f5983bfde3de5be6f06a0e2e7a29f248251eed2620c diff -Nru snapcraft-2.27.1+17.04/integration_tests/snaps/scriptlet-build/Makefile snapcraft-2.28+17.04/integration_tests/snaps/scriptlet-build/Makefile --- snapcraft-2.27.1+17.04/integration_tests/snaps/scriptlet-build/Makefile 2017-01-28 06:12:26.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/snaps/scriptlet-build/Makefile 2017-03-22 12:31:58.000000000 +0000 @@ -2,6 +2,7 @@ build: touch build-build + touch jobs-$(SNAPCRAFT_PARALLEL_BUILD_COUNT) move-to-dest: touch $(DESTDIR)/build-install diff -Nru snapcraft-2.27.1+17.04/integration_tests/snaps/stage-package-version/snap/snapcraft.yaml snapcraft-2.28+17.04/integration_tests/snaps/stage-package-version/snap/snapcraft.yaml --- snapcraft-2.27.1+17.04/integration_tests/snaps/stage-package-version/snap/snapcraft.yaml 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/snaps/stage-package-version/snap/snapcraft.yaml 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,13 @@ +name: stage-package-version +version: '0.1' +summary: install a specific, but non-existent version of a stage-package +description: | + Install a specific, but non-existent version of a stage-package + +grade: stable +confinement: strict + +parts: + hello: + plugin: nil + stage-packages: ['hello=x.y-z'] diff -Nru snapcraft-2.27.1+17.04/integration_tests/snaps/staging_links_to_libc/snapcraft.yaml snapcraft-2.28+17.04/integration_tests/snaps/staging_links_to_libc/snapcraft.yaml --- snapcraft-2.27.1+17.04/integration_tests/snaps/staging_links_to_libc/snapcraft.yaml 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/snaps/staging_links_to_libc/snapcraft.yaml 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,18 @@ +name: staged-libc-links +version: '0.1' +summary: Attempt to stage libc6-dev twice +description: | + Once by pulling it in via stage-packages, and the other time via + a tarred staging area. They should not conflict. + +grade: devel +confinement: devmode + +parts: + from-package: + plugin: nil + stage-packages: [libc6-dev] + + from-tar: + plugin: dump + source: stage.tar diff -Nru snapcraft-2.27.1+17.04/integration_tests/snaps/use_libc_dl/snap/snapcraft.yaml snapcraft-2.28+17.04/integration_tests/snaps/use_libc_dl/snap/snapcraft.yaml --- snapcraft-2.27.1+17.04/integration_tests/snaps/use_libc_dl/snap/snapcraft.yaml 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/snaps/use_libc_dl/snap/snapcraft.yaml 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,21 @@ +name: libc-dl +version: '1.0' +summary: Use libdl from libc +description: | + This snap links to libdl from libc. It requires the shared library, and will + fail to build with the static library. As such, this makes a good regression + test for LP: #1665089. + +grade: devel +confinement: strict + +apps: + hello: + command: hello + +parts: + my-part: + plugin: autotools + source: src/ + stage-packages: [libc6-dev] + stage: [-usr/] diff -Nru snapcraft-2.27.1+17.04/integration_tests/snaps/use_libc_dl/src/configure.ac snapcraft-2.28+17.04/integration_tests/snaps/use_libc_dl/src/configure.ac --- snapcraft-2.27.1+17.04/integration_tests/snaps/use_libc_dl/src/configure.ac 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/snaps/use_libc_dl/src/configure.ac 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,8 @@ +AC_INIT([package], [version]) +AM_INIT_AUTOMAKE([foreign subdir-objects]) +LT_INIT +AC_CONFIG_SRCDIR([configure.ac]) +AC_CONFIG_HEADERS([config.h]) +AC_PROG_CXX +AC_CONFIG_FILES([Makefile]) +AC_OUTPUT diff -Nru snapcraft-2.27.1+17.04/integration_tests/snaps/use_libc_dl/src/hello.cpp snapcraft-2.28+17.04/integration_tests/snaps/use_libc_dl/src/hello.cpp --- snapcraft-2.27.1+17.04/integration_tests/snaps/use_libc_dl/src/hello.cpp 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/snaps/use_libc_dl/src/hello.cpp 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,5 @@ +#include + +extern "C" void hello() { + std::cout << "Hello World!" << std::endl; +} diff -Nru snapcraft-2.27.1+17.04/integration_tests/snaps/use_libc_dl/src/main.cpp snapcraft-2.28+17.04/integration_tests/snaps/use_libc_dl/src/main.cpp --- snapcraft-2.27.1+17.04/integration_tests/snaps/use_libc_dl/src/main.cpp 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/snaps/use_libc_dl/src/main.cpp 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,49 @@ +#include +#include +#include + +int main() { + // Determine library path + std::string snap(std::getenv("SNAP")); + if (snap.empty()) + { + std::cerr << "The $SNAP environment variable is not defined" << std::endl; + return 1; + } + + std::string library_path(snap + "/lib/libhello.so"); + + // Open the library + std::cout << "Opening lib" << std::endl; + void *handle = dlopen(library_path.c_str(), RTLD_LAZY); + + if (!handle) + { + std::cerr << "Cannot open lib: " << dlerror() << std::endl; + return 1; + } + + typedef void (*VoidFunctionPtr)(); + + // Loading "hello" symbol + std::cout << "Loading symbol 'hello'" << std::endl; + VoidFunctionPtr hello = reinterpret_cast(dlsym(handle, "hello")); + const char *error = dlerror(); + if (error) + { + std::cerr << "Cannot load symbol 'hello' from lib: " << error + << std::endl; + dlclose(handle); + return 1; + } + + // Now actually call the hello function out of the dlopened library + std::cout << "Calling hello" << std::endl; + hello(); + + // Finally, clean up + std::cout << "Closing lib" << std::endl; + dlclose(handle); + + return 0; +} diff -Nru snapcraft-2.27.1+17.04/integration_tests/snaps/use_libc_dl/src/Makefile.am snapcraft-2.28+17.04/integration_tests/snaps/use_libc_dl/src/Makefile.am --- snapcraft-2.27.1+17.04/integration_tests/snaps/use_libc_dl/src/Makefile.am 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/snaps/use_libc_dl/src/Makefile.am 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,6 @@ +lib_LTLIBRARIES = libhello.la +libhello_la_SOURCES = hello.cpp + +bin_PROGRAMS = hello +hello_SOURCES = main.cpp +hello_LDADD = -ldl diff -Nru snapcraft-2.27.1+17.04/integration_tests/snaps/zip/snapcraft.yaml snapcraft-2.28+17.04/integration_tests/snaps/zip/snapcraft.yaml --- snapcraft-2.27.1+17.04/integration_tests/snaps/zip/snapcraft.yaml 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/snaps/zip/snapcraft.yaml 2017-03-22 12:31:58.000000000 +0000 @@ -6,7 +6,9 @@ parts: file1: - plugin: copy + plugin: dump source: simple.zip - files: - "*": . + zip-source-checksum: + plugin: dump + source: simple.zip + source-checksum: sha256/035ae7da4bd0ff39960466353e0810f51d17193a13e8b75e767391820aed484c diff -Nru snapcraft-2.27.1+17.04/integration_tests/test_build_package_version.py snapcraft-2.28+17.04/integration_tests/test_build_package_version.py --- snapcraft-2.27.1+17.04/integration_tests/test_build_package_version.py 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/test_build_package_version.py 2017-03-23 18:31:04.000000000 +0000 @@ -0,0 +1,51 @@ +# -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*- +# +# Copyright (C) 2017 Canonical Ltd +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License version 3 as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . + +import subprocess + +import apt + +import integration_tests + + +class BuildPackageVersionTestCase(integration_tests.TestCase): + + def test_build_package_gets_version(self): + self.run_snapcraft('pull', 'build-package-version') + pkg = 'hello' + expected_version = '2.10-1' + with apt.Cache() as apt_cache: + installed_version = apt_cache[pkg].candidate.version + self.assertEqual(expected_version, + installed_version) + + def test_global_build_package_gets_version(self): + self.run_snapcraft('pull', 'build-package-version-global') + pkg = 'hello' + expected_version = '2.10-1' + with apt.Cache() as apt_cache: + installed_version = apt_cache[pkg].candidate.version + self.assertEqual(expected_version, + installed_version) + + def test_build_package_bad_version(self): + error = self.assertRaises( + subprocess.CalledProcessError, + self.run_snapcraft, 'pull', 'build-package-version-bad') + self.assertIn( + "Version 'x.y-z' for 'hello' was not found", + str(error.output) + ) diff -Nru snapcraft-2.27.1+17.04/integration_tests/test_checksum_algorithms.py snapcraft-2.28+17.04/integration_tests/test_checksum_algorithms.py --- snapcraft-2.27.1+17.04/integration_tests/test_checksum_algorithms.py 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/test_checksum_algorithms.py 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,60 @@ +# -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*- +# +# Copyright (C) 2017 Canonical Ltd +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License version 3 as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . + +import subprocess + +import integration_tests +import testscenarios + + +class ChecksumAlgorithmsTestCase(testscenarios.WithScenarios, + integration_tests.TestCase): + + scenarios = [ + (project_dir, + {'project_dir': project_dir}) for project_dir in [ + 'checksum-algorithms', + 'deb-with-checksum', + 'rpm-with-checksum'] + ] + + def test_checksum_algorithms(self): + self.run_snapcraft('pull', self.project_dir) + + +class InvalidChecksumsTestCase(testscenarios.WithScenarios, + integration_tests.TestCase): + + scenarios = [ + (part, + {'part': part}) for part in [ + 'checksum-md5', + 'checksum-sha1', + 'checksum-sha224', + 'checksum-sha256', + 'checksum-sha384', + 'checksum-sha512', + 'checksum-sha3-284', + 'checksum-sha3-256', + 'checksum-sha3-512'] + ] + + def test_checksum_invalid(self): + project_dir = 'checksum-algorithms-invalid' + self.assertRaises(subprocess.CalledProcessError, + self.run_snapcraft, + ['pull', self.part], + project_dir) diff -Nru snapcraft-2.27.1+17.04/integration_tests/test_dump_plugin.py snapcraft-2.28+17.04/integration_tests/test_dump_plugin.py --- snapcraft-2.27.1+17.04/integration_tests/test_dump_plugin.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/test_dump_plugin.py 2017-03-22 12:31:58.000000000 +0000 @@ -16,15 +16,50 @@ import os import fixtures +import tarfile +import threading + +from pyftpdlib import ( + authorizers, + handlers, + servers +) from testtools.matchers import ( DirExists, + FileContains, FileExists ) import integration_tests +class FtpServerRunning(fixtures.Fixture): + + def __init__(self, directory): + super().__init__() + self.directory = directory + + def setUp(self): + super().setUp() + self._start_ftp_server() + + def _start_ftp_server(self): + authorizer = authorizers.DummyAuthorizer() + authorizer.add_anonymous(self.directory) + handler = handlers.FTPHandler + handler.authorizer = authorizer + self.server = servers.FTPServer(('', 2121), handler) + + server_thread = threading.Thread(target=self.server.serve_forever) + server_thread.start() + self.addCleanup(self._stop_ftp_server, server_thread) + + def _stop_ftp_server(self, thread): + self.server.close_all() + thread.join() + + class DumpPluginTestCase(integration_tests.TestCase): def test_stage_dump_plugin(self): @@ -65,10 +100,18 @@ def test_download_file_from_ftp_source(self): """Download a file from a FTP source, LP: #1602323""" - - # This is needed since autopkgtest doesn't properly set it - if not os.getenv('ftp_proxy', None): - self.useFixture(fixtures.EnvironmentVariable( - 'ftp_proxy', os.getenv('http_proxy', ''))) + ftp_dir = os.path.join(self.path, 'ftp') + os.mkdir(ftp_dir) + ftp_server = FtpServerRunning(ftp_dir) + self.useFixture(ftp_server) + + test_file_path = os.path.join(self.path, 'test') + with open(test_file_path, 'w') as test_file: + test_file.write('Hello ftp') + with tarfile.open(os.path.join(ftp_dir, 'test.tar.gz'), 'w:gz') as tar: + tar.add(test_file_path) self.run_snapcraft('pull', 'ftp-source') + self.assertThat( + os.path.join(self.parts_dir, 'ftp-part', 'src', 'test'), + FileContains('Hello ftp')) diff -Nru snapcraft-2.27.1+17.04/integration_tests/test_multiarch_stage_packages.py snapcraft-2.28+17.04/integration_tests/test_multiarch_stage_packages.py --- snapcraft-2.27.1+17.04/integration_tests/test_multiarch_stage_packages.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/test_multiarch_stage_packages.py 2017-03-22 12:31:58.000000000 +0000 @@ -31,6 +31,6 @@ self.assertThat( exception.output, Contains( "Error downloading stage packages for part 'my-part': The " - "Ubuntu package 'hello:fake-arch' was not found.\n" + "package 'hello:fake-arch' was not found.\n" 'You may need to add support for this architecture with ' "'dpkg --add-architecture fake-arch'")) diff -Nru snapcraft-2.27.1+17.04/integration_tests/test_parser.py snapcraft-2.28+17.04/integration_tests/test_parser.py --- snapcraft-2.27.1+17.04/integration_tests/test_parser.py 2017-01-28 06:12:26.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/test_parser.py 2017-03-22 12:31:58.000000000 +0000 @@ -59,24 +59,157 @@ class TestParserWikis(testscenarios.WithScenarios, ParserTestCase): """Test bin/snapcraft-parser""" + def setUp(self): + temp_cwd_fixture = fixture_setup.TempCWD() + self.useFixture(temp_cwd_fixture) + super().setUp() + + # Since we're running in a temporary directory use + # the original source tree version of snapcraft-parser + if not os.getenv('SNAPCRAFT_FROM_INSTALLED', False): + self.snapcraft_parser_command = os.path.join( + os.path.dirname(__file__), '..', 'bin', 'snapcraft-parser') + + def _read_file(self, filename): + content = '' + with open(filename) as fp: + content = fp.read() + + return content + + def _setup_wiki_file(self, filename, origin, commit=None): + content = self._read_file(filename) + content = content.replace('$ORIGIN', origin) + if commit: + content = content.replace('$COMMIT', commit) + with open('wiki', 'w') as fp: + fp.write(content) + + def _setup_origin(self, snapcraft_files, repo_dir, base_dir): + os.makedirs(repo_dir) + previous_dir = os.getcwd() + os.chdir(repo_dir) + + subprocess.check_call(['git', 'init', '.'], + stdout=subprocess.DEVNULL, + stderr=subprocess.DEVNULL) + subprocess.check_output( + ['git', 'config', 'user.name', 'Test User']) + subprocess.check_output( + ['git', 'config', 'user.email', '. import os +import subprocess import testscenarios from testtools.matchers import FileExists @@ -39,3 +40,16 @@ self.stage_dir, 'providers', 'simple-plainbox-provider', 'plainbox-provider-simple.provider'), FileExists()) + + def test_snap_provider_with_deps(self): + project_dir = 'plainbox-provider-with-deps' + self.run_snapcraft('prime', project_dir) + # No assertion required as the project will fail to complete to the + # prime step if validation fails + + def test_invalid_provider(self): + project_dir = 'plainbox-provider-invalid' + # The validate command should fail during the builid step of the + # provider in this project + self.assertRaises(subprocess.CalledProcessError, self.run_snapcraft, + 'prime', project_dir) diff -Nru snapcraft-2.27.1+17.04/integration_tests/test_pull_properties.py snapcraft-2.28+17.04/integration_tests/test_pull_properties.py --- snapcraft-2.27.1+17.04/integration_tests/test_pull_properties.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/test_pull_properties.py 2017-03-22 12:31:58.000000000 +0000 @@ -45,6 +45,24 @@ # Verify that the contents of the dependencies made it in as well. self.assertTrue('foo' in state.properties) + self.assertTrue(len(state.assets['stage-packages']) > 0) self.assertTrue('stage-packages' in state.properties) self.assertEqual('bar', state.properties['foo']) self.assertEqual(['curl'], state.properties['stage-packages']) + + +class AssetTrackingTestCase(integration_tests.TestCase): + + def test_pull(self): + project_dir = 'asset-tracking' + self.run_snapcraft('pull', project_dir) + + state_file = os.path.join( + self.parts_dir, 'asset-tracking', 'state', 'pull') + self.assertThat(state_file, FileExists()) + with open(state_file) as f: + state = yaml.load(f) + + # Verify that the correct version of 'hello' is installed + self.assertTrue(len(state.assets['stage-packages']) > 0) + self.assertIn('hello=2.10-1', state.assets['stage-packages']) diff -Nru snapcraft-2.27.1+17.04/integration_tests/test_python_plugin.py snapcraft-2.28+17.04/integration_tests/test_python_plugin.py --- snapcraft-2.27.1+17.04/integration_tests/test_python_plugin.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/test_python_plugin.py 2017-03-22 15:10:01.000000000 +0000 @@ -16,9 +16,12 @@ import os from glob import glob +from subprocess import check_output +from textwrap import dedent from testtools.matchers import ( DirExists, + FileContains, FileExists ) @@ -84,6 +87,50 @@ self.assertEqual('#!/usr/bin/env python3', python3_shebang) self.assertEqual('#!/usr/bin/env python3', python_shebang) + def test_pbr_console_scripts(self): + """Verify that LP: #1670852 doesn't come back.""" + # pbr needs git repos to work + def pre_func(): + for python_dir in ['python2', 'python3']: + d = os.path.join(self.path, python_dir) + check_output(['git', 'init'], cwd=d) + check_output( + ['git', 'config', 'user.name', 'Test User'], cwd=d) + check_output( + ['git', 'config', 'user.email', '. import os +import subprocess from testtools.matchers import FileExists, Not +import snapcraft import integration_tests class RustPluginTestCase(integration_tests.TestCase): + def run_snapcraft(self, command, project_dir=None, debug=True): + try: + failed = True + super().run_snapcraft(command, project_dir, debug) + failed = False + except subprocess.CalledProcessError: + if snapcraft.ProjectOptions().deb_arch == 'arm64': + # https://github.com/rust-lang/rustup.sh/issues/82 + self.expectFailure( + 'The rustup script does not support arm64.', + self.assertFalse, failed) + else: + raise + def test_stage_rust_plugin(self): self.run_snapcraft('stage', 'rust-hello') diff -Nru snapcraft-2.27.1+17.04/integration_tests/test_scriptlets.py snapcraft-2.28+17.04/integration_tests/test_scriptlets.py --- snapcraft-2.27.1+17.04/integration_tests/test_scriptlets.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/test_scriptlets.py 2017-03-22 12:31:58.000000000 +0000 @@ -14,6 +14,7 @@ # You should have received a copy of the GNU General Public License # along with this program. If not, see . +import multiprocessing import os import integration_tests @@ -40,6 +41,9 @@ touch_file_path = os.path.join(builddir, 'build-build') self.assertThat(touch_file_path, FileExists()) + jobs_file_name = 'jobs-{}'.format(multiprocessing.cpu_count()) + touch_file_path = os.path.join(builddir, jobs_file_name) + self.assertThat(touch_file_path, FileExists()) touch_file_path = os.path.join(installdir, 'build-install') self.assertThat(touch_file_path, FileExists()) diff -Nru snapcraft-2.27.1+17.04/integration_tests/test_stage_package_version.py snapcraft-2.28+17.04/integration_tests/test_stage_package_version.py --- snapcraft-2.27.1+17.04/integration_tests/test_stage_package_version.py 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/test_stage_package_version.py 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,32 @@ +# -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*- +# +# Copyright (C) 2017 Canonical Ltd +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License version 3 as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . + +import subprocess + +import integration_tests + + +class StagePackageVersionTestCase(integration_tests.TestCase): + + def test_stage_package_gets_version(self): + error = self.assertRaises( + subprocess.CalledProcessError, + self.run_snapcraft, 'pull', 'stage-package-version') + self.assertIn( + "Error downloading stage packages for part 'hello': " + "The package 'hello=x.y-z' was not found.", + str(error.output) + ) diff -Nru snapcraft-2.27.1+17.04/integration_tests/test_stage.py snapcraft-2.28+17.04/integration_tests/test_stage.py --- snapcraft-2.27.1+17.04/integration_tests/test_stage.py 2017-02-17 19:54:46.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/test_stage.py 2017-03-22 12:31:58.000000000 +0000 @@ -70,3 +70,28 @@ self.run_snapcraft(['stage'], project_dir) self.assertThat(os.path.join(self.stage_dir, 'bin', 'hello-classic'), FileExists()) + + def test_staging_libc_links(self): + project_dir = 'staging_links_to_libc' + + # First, stage libc6-dev via stage-packages + self.run_snapcraft(['stage', 'from-package'], project_dir) + + # Now tar up the staging area + subprocess.check_call(['tar', 'cf', 'stage.tar', 'stage/']) + + # Now attempt to stage the tarred staging area once again. This should + # not conflict. + try: + self.run_snapcraft(['stage', 'from-tar'], project_dir) + except subprocess.CalledProcessError as e: + if 'have the following file paths in common' in e.output: + self.fail('Parts unexpectedly conflicted') + else: + raise + + def symlinks_to_libc_should_build(self): + """Regression test for LP: #1665089""" + + # This will fail to build if the libc symlinks are missing + self.run_snapcraft('stage', 'use_libc_dl') diff -Nru snapcraft-2.27.1+17.04/integration_tests/test_store_history.py snapcraft-2.28+17.04/integration_tests/test_store_history.py --- snapcraft-2.27.1+17.04/integration_tests/test_store_history.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/test_store_history.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,109 +0,0 @@ -# -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*- -# -# Copyright (C) 2016-2017 Canonical Ltd -# -# This program is free software: you can redistribute it and/or modify -# it under the terms of the GNU General Public License version 3 as -# published by the Free Software Foundation. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with this program. If not, see . - -import os -import re -import subprocess -import unittest -import uuid - -from testtools.matchers import Contains, FileExists, MatchesRegex - -import integration_tests - - -class HistoryTestCase(integration_tests.StoreTestCase): - - def test_history_without_login(self): - error = self.assertRaises( - subprocess.CalledProcessError, - self.run_snapcraft, ['history', 'test-snap']) - self.assertIn('No valid credentials found. Have you run "snapcraft ' - 'login"?', str(error.output)) - - def test_history_with_login_wrong_snap(self): - self.addCleanup(self.logout) - self.login() - - error = self.assertRaises( - subprocess.CalledProcessError, - self.run_snapcraft, ['history', 'mysnap']) - self.assertIn( - "Snap 'mysnap' was not found in '16' series.", str(error.output)) - - def test_history_with_login_bad_snap_with_series(self): - self.addCleanup(self.logout) - self.login() - - error = self.assertRaises( - subprocess.CalledProcessError, - self.run_snapcraft, ['history', 'mysnap', '--series=16']) - self.assertIn( - "Snap 'mysnap' was not found in '16' series.", str(error.output)) - - def test_history_with_login_bad_snap_with_arch(self): - self.addCleanup(self.logout) - self.login() - - error = self.assertRaises( - subprocess.CalledProcessError, - self.run_snapcraft, ['history', 'mysnap', '--arch=i386']) - self.assertIn( - "Snap 'mysnap' for 'i386' was not found in '16' series.", - str(error.output)) - - @unittest.skipUnless( - os.getenv('TEST_STORE', 'fake') == 'fake', 'Skip fake store.') - def test_history_fake_store(self): - self.addCleanup(self.logout) - self.login() - - output = self.run_snapcraft(['history', 'basic']) - expected = '\n'.join(( - 'Rev. Uploaded Arch Version Channels', - '2 2016-09-27T19:23:40Z i386 2.0.1 -', - '1 2016-09-27T18:38:43Z amd64 2.0.2 stable*, edge' - )) - self.assertThat(output, Contains(expected)) - - @unittest.skipUnless( - os.getenv('TEST_STORE', 'fake') == 'staging', 'Skip staging store.') - def test_history_staging_store(self): - self.addCleanup(self.logout) - self.login() - - # Build a random snap, register, push and release it. - self.copy_project_to_cwd('basic') - unique_id = uuid.uuid4().int - name = 'u1test-{}'.format(unique_id) - version = '1.0' - self.update_name_and_version(name, version) - self.run_snapcraft('snap') - snap_path = '{}_{}_{}.snap'.format(name, version, 'all') - self.assertThat(snap_path, FileExists()) - self.register(name) - self.assertEqual(0, self.push(snap_path, release='candidate,beta')) - - output = self.run_snapcraft(['history', name]) - - datetime_re = '\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}Z' - expected = '\n'.join(( - '.*', - 'Rev. Uploaded Arch Version Channels', - '1 {datetime_re} Arch: All 1 candidate\*, beta\*' - '.*', - )).format(datetime_re=datetime_re) - self.assertThat(output, MatchesRegex(expected, flags=re.DOTALL)) diff -Nru snapcraft-2.27.1+17.04/integration_tests/test_store_revisions.py snapcraft-2.28+17.04/integration_tests/test_store_revisions.py --- snapcraft-2.27.1+17.04/integration_tests/test_store_revisions.py 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/test_store_revisions.py 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,141 @@ +# -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*- +# +# Copyright (C) 2016-2017 Canonical Ltd +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License version 3 as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . + +import os +import re +import subprocess +import unittest +import uuid + +from testtools.matchers import Contains, FileExists, MatchesRegex + +import integration_tests + + +class RevisionsTestCase(integration_tests.StoreTestCase): + + def test_revisions_without_login(self): + error = self.assertRaises( + subprocess.CalledProcessError, + self.run_snapcraft, ['revisions', 'test-snap']) + self.assertIn('No valid credentials found. Have you run "snapcraft ' + 'login"?', str(error.output)) + + def test_revisions_with_login_wrong_snap(self): + self.addCleanup(self.logout) + self.login() + + error = self.assertRaises( + subprocess.CalledProcessError, + self.run_snapcraft, ['revisions', 'mysnap']) + self.assertIn( + "Snap 'mysnap' was not found in '16' series.", str(error.output)) + + def test_revisions_with_login_bad_snap_with_series(self): + self.addCleanup(self.logout) + self.login() + + error = self.assertRaises( + subprocess.CalledProcessError, + self.run_snapcraft, ['revisions', 'mysnap', '--series=16']) + self.assertIn( + "Snap 'mysnap' was not found in '16' series.", str(error.output)) + + def test_revisions_with_login_bad_snap_with_arch(self): + self.addCleanup(self.logout) + self.login() + + error = self.assertRaises( + subprocess.CalledProcessError, + self.run_snapcraft, ['revisions', 'mysnap', '--arch=i386']) + self.assertIn( + "Snap 'mysnap' for 'i386' was not found in '16' series.", + str(error.output)) + + @unittest.skipUnless( + os.getenv('TEST_STORE', 'fake') == 'fake', 'Skip fake store.') + def test_revisions_fake_store(self): + self.addCleanup(self.logout) + self.login() + + output = self.run_snapcraft(['revisions', 'basic']) + expected = '\n'.join(( + 'Rev. Uploaded Arch Version Channels', + '2 2016-09-27T19:23:40Z i386 2.0.1 -', + '1 2016-09-27T18:38:43Z amd64 2.0.2 stable*, edge' + )) + self.assertThat(output, Contains(expected)) + + @unittest.skipUnless( + os.getenv('TEST_STORE', 'fake') == 'fake', 'Skip fake store.') + def test_list_revisions_fake_store(self): + self.addCleanup(self.logout) + self.login() + + output = self.run_snapcraft(['list-revisions', 'basic']) + expected = '\n'.join(( + 'Rev. Uploaded Arch Version Channels', + '2 2016-09-27T19:23:40Z i386 2.0.1 -', + '1 2016-09-27T18:38:43Z amd64 2.0.2 stable*, edge' + )) + self.assertThat(output, Contains(expected)) + + @unittest.skipUnless( + os.getenv('TEST_STORE', 'fake') == 'fake', 'Skip fake store.') + def test_history_fake_store(self): + self.addCleanup(self.logout) + self.login() + + output = self.run_snapcraft(['history', 'basic']) + expected = '\n'.join(( + 'Rev. Uploaded Arch Version Channels', + '2 2016-09-27T19:23:40Z i386 2.0.1 -', + '1 2016-09-27T18:38:43Z amd64 2.0.2 stable*, edge' + )) + self.assertThat(output, Contains(expected)) + + self.assertThat(output, Contains( + "DEPRECATED: The 'history' command has " + "been replaced by 'list-revisions'.")) + + @unittest.skipUnless( + os.getenv('TEST_STORE', 'fake') == 'staging', 'Skip staging store.') + def test_revisions_staging_store(self): + self.addCleanup(self.logout) + self.login() + + # Build a random snap, register, push and release it. + self.copy_project_to_cwd('basic') + unique_id = uuid.uuid4().int + name = 'u1test-{}'.format(unique_id) + version = '1.0' + self.update_name_and_version(name, version) + self.run_snapcraft('snap') + snap_path = '{}_{}_{}.snap'.format(name, version, 'all') + self.assertThat(snap_path, FileExists()) + self.register(name) + self.assertEqual(0, self.push(snap_path, release='candidate,beta')) + + output = self.run_snapcraft(['revisions', name]) + + datetime_re = '\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}Z' + expected = '\n'.join(( + '.*', + 'Rev. Uploaded Arch Version Channels', + '1 {datetime_re} Arch: All 1 candidate\*, beta\*' + '.*', + )).format(datetime_re=datetime_re) + self.assertThat(output, MatchesRegex(expected, flags=re.DOTALL)) diff -Nru snapcraft-2.27.1+17.04/integration_tests/test_store_status.py snapcraft-2.28+17.04/integration_tests/test_store_status.py --- snapcraft-2.27.1+17.04/integration_tests/test_store_status.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/test_store_status.py 2017-03-22 12:31:58.000000000 +0000 @@ -72,13 +72,13 @@ output = self.run_snapcraft(['status', 'basic']) expected = '\n'.join(( - 'Arch Channel Version Revision', - 'amd64 stable 1.0-amd64 2', - ' beta 1.1-amd64 4', - ' edge ^ ^', - 'i386 stable - -', - ' beta - -', - ' edge 1.0-i386 3')) + 'Track Arch Channel Version Revision', + 'latest amd64 stable 1.0-amd64 2', + ' beta 1.1-amd64 4', + ' edge ^ ^', + ' i386 stable - -', + ' beta - -', + ' edge 1.0-i386 3')) self.assertThat(output, Contains(expected)) @unittest.skipUnless( diff -Nru snapcraft-2.27.1+17.04/integration_tests/wiki/both_parts_wiki snapcraft-2.28+17.04/integration_tests/wiki/both_parts_wiki --- snapcraft-2.27.1+17.04/integration_tests/wiki/both_parts_wiki 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/wiki/both_parts_wiki 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,6 @@ +--- +description: a test part +maintainer: John Doe +parts: [testpart-both] +origin: $ORIGIN +origin-type: git diff -Nru snapcraft-2.27.1+17.04/integration_tests/wiki/hidden_parts_wiki snapcraft-2.28+17.04/integration_tests/wiki/hidden_parts_wiki --- snapcraft-2.27.1+17.04/integration_tests/wiki/hidden_parts_wiki 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/wiki/hidden_parts_wiki 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,6 @@ +--- +description: a test part +maintainer: John Doe +parts: [testpart] +origin: $ORIGIN +origin-type: git diff -Nru snapcraft-2.27.1+17.04/integration_tests/wiki/hidden_snapcraft_yaml snapcraft-2.28+17.04/integration_tests/wiki/hidden_snapcraft_yaml --- snapcraft-2.27.1+17.04/integration_tests/wiki/hidden_snapcraft_yaml 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/wiki/hidden_snapcraft_yaml 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,5 @@ +parts: + testpart: + source: . + plugin: nil + diff -Nru snapcraft-2.27.1+17.04/integration_tests/wiki/missing_parts_wiki snapcraft-2.28+17.04/integration_tests/wiki/missing_parts_wiki --- snapcraft-2.27.1+17.04/integration_tests/wiki/missing_parts_wiki 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/wiki/missing_parts_wiki 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,6 @@ +--- +description: a test part +maintainer: John Doe +parts: [testpart-missing] +origin: $ORIGIN +origin-type: git diff -Nru snapcraft-2.27.1+17.04/integration_tests/wiki/origin_options_snapcraft_yaml snapcraft-2.28+17.04/integration_tests/wiki/origin_options_snapcraft_yaml --- snapcraft-2.27.1+17.04/integration_tests/wiki/origin_options_snapcraft_yaml 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/wiki/origin_options_snapcraft_yaml 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,5 @@ +parts: + testpart: + source: . + plugin: nil + diff -Nru snapcraft-2.27.1+17.04/integration_tests/wiki/origin_options_wiki snapcraft-2.28+17.04/integration_tests/wiki/origin_options_wiki --- snapcraft-2.27.1+17.04/integration_tests/wiki/origin_options_wiki 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/wiki/origin_options_wiki 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,7 @@ +--- +description: Wiki with origin modifiers +maintainer: Marco Trevisan +parts: [testpart] +origin: $ORIGIN +origin-branch: feature-branch +origin-type: git diff -Nru snapcraft-2.27.1+17.04/integration_tests/wiki/snap.snapcraft.yaml snapcraft-2.28+17.04/integration_tests/wiki/snap.snapcraft.yaml --- snapcraft-2.27.1+17.04/integration_tests/wiki/snap.snapcraft.yaml 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/wiki/snap.snapcraft.yaml 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,6 @@ +--- +description: A test part with snap/snapcraft.yaml +maintainer: John Doe +parts: [testpart] +origin: $ORIGIN +origin-type: git diff -Nru snapcraft-2.27.1+17.04/integration_tests/wiki/wrong_origin_options_snapcraft_yaml snapcraft-2.28+17.04/integration_tests/wiki/wrong_origin_options_snapcraft_yaml --- snapcraft-2.27.1+17.04/integration_tests/wiki/wrong_origin_options_snapcraft_yaml 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/wiki/wrong_origin_options_snapcraft_yaml 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,5 @@ +parts: + testpart: + source: . + plugin: nil + diff -Nru snapcraft-2.27.1+17.04/integration_tests/wiki/wrong_origin_options_wiki snapcraft-2.28+17.04/integration_tests/wiki/wrong_origin_options_wiki --- snapcraft-2.27.1+17.04/integration_tests/wiki/wrong_origin_options_wiki 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/wiki/wrong_origin_options_wiki 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,8 @@ +--- +description: Wiki with invalid origin modifiers +maintainer: Marco Trevisan +parts: [desktop-glib-only] +origin: $ORIGIN +origin-type: git +origin-branch: really-I-don-t-have-this-branch +origin-commit: ThisIsNotACommitID diff -Nru snapcraft-2.27.1+17.04/integration_tests/wrong_origin_options_wiki snapcraft-2.28+17.04/integration_tests/wrong_origin_options_wiki --- snapcraft-2.27.1+17.04/integration_tests/wrong_origin_options_wiki 2017-01-28 06:12:26.000000000 +0000 +++ snapcraft-2.28+17.04/integration_tests/wrong_origin_options_wiki 1970-01-01 00:00:00.000000000 +0000 @@ -1,8 +0,0 @@ ---- -description: Wiki with invalid origin modifiers -maintainer: Marco Trevisan -parts: [desktop-glib-only] -origin: https://github.com/3v1n0/snapcraft-desktop-helpers.git -origin-type: git -origin-branch: really-I-don-t-have-this-branch -origin-commit: ThisIsNotACommitID diff -Nru snapcraft-2.27.1+17.04/manual-tests.md snapcraft-2.28+17.04/manual-tests.md --- snapcraft-2.27.1+17.04/manual-tests.md 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/manual-tests.md 2017-03-22 12:31:58.000000000 +0000 @@ -50,12 +50,107 @@ 3. Run `snapcraft cleanbuild --remote ` where `` is the name you gave the remote on step 1. +# Test the PC kernel. -# Test that the Catkin plugin doesn't pass args to setup.sh +1. Get the PC kernel source: -This is a regression test for bug #1660852. + $ git clone -b pc https://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux-snap/+git/xenial + $ cd kernel -1. Build and install the `demos/ros` demo. -2. Run `ros-example.launch-project --help`. -3. Verify that you actually get the help info for `roslaunch`, not a barf of - errors. +2. Run `sudo snapcraft`. +3. Create a file called `pc-model.json` with the following contents: + + { + "type": "model", + "authority-id": "$account_id", + "brand-id": "$account_id", + "series": "16", + "model": "pc", + "architecture": "amd64", + "gadget": "pc", + "kernel": "$kernel_snap_path", + "timestamp": "$date" + } + +4. Replace `$account_id` with the value from https://myapps.developer.ubuntu.com/dev/account/ +5. Replace `$kernel_snap_path` with the path to the snap you just created. +6. Replace `$date` with the output of the command `date -Iseconds --utc`. +7. If you haven't created a key, run the following command, replacing + `$key_name` with a name for your key: + + $ snap create-key $key_name + $ snapcraft register-key + +8. Sign the model: + + $ cat pc-model.json | snap sign -k $key_name > pc.model + +10. Install ubuntu-image: + + $ sudo apt install ubuntu-image + +11. Create the image: + + $ sudo ubuntu-image --image-size 3G -O ubuntu-core-16 pc.model --extra-snaps $kernel_snap_path + +12. Start the image in kvm: + + $ kvm -smp 2 -m 1500 -netdev user,id=mynet0,hostfwd=tcp::8022-:22,hostfwd=tcp::8090-:80 -device virtio-net-pci,netdev=mynet0 -drive file=ubuntu-core-16/pc.img,format=raw + + * Check that the user can be created. + * Check that it's possible to ssh into the vm. + * Check that it's possible to install a snap. + + +# Test the dragonboard 410c kernel. + +1. Download https://developer.qualcomm.com/download/db410c/linux-board-support-package-v1.2.zip +2. Extract it and copy the file `firmware.tar` to the directory `demos/96boards-kernel`. +3. Run `snapcraft --target-arch arm64` in the `demos/96boards-kernel` directory. +4. Create a file called `dragonboard-model.json` with the following contents: + + { + "type": "model", + "authority-id": "$account_id", + "brand-id": "$account_id", + "series": "16", + "model": "dragonboard", + "architecture": "arm64", + "gadget": "dragonboard", + "kernel": "$kernel_snap_path", + "timestamp": "$date" + } + +5. Replace `$account_id` with the value from https://myapps.developer.ubuntu.com/dev/account/ +6. Replace `$kernel_snap_path` with the path to the snap you just created. +7. Replace `$date` with the output of the command `date -Iseconds --utc`. +8. If you haven't created a key, run the following command, replacing + `$key_name` with a name for your key: + + $ snap create-key $key_name + $ snapcraft register-key + +9. Sign the model: + + $ cat dragonboard-model.json | snap sign -k $key_name > dragonboard.model + +10. Install ubuntu-image: + + $ sudo apt install ubuntu-image + +11. Create the image: + + $ sudo ubuntu-image -O ubuntu-core-16 dragonboard.model --extra-snaps $kernel_snap_path + +12. Insert an sdcard into the host PC. +13. Umount the sdcard partitions. +14. Flash the image, replacing sdX with the path to the sdcard: + + $ sudo dd if=ubuntu-core-16/dragonboard.img of=/dev/sdX bs=32M + $ sync + +15. Insert the sdcard into the dragonboard, and turn it on. + + * Check that the user can be created. + * Check that it's possible to ssh into the board. + * Check that it's possible to install a snap. diff -Nru snapcraft-2.27.1+17.04/requirements-devel.txt snapcraft-2.28+17.04/requirements-devel.txt --- snapcraft-2.27.1+17.04/requirements-devel.txt 2017-01-28 06:12:26.000000000 +0000 +++ snapcraft-2.28+17.04/requirements-devel.txt 2017-03-22 12:31:58.000000000 +0000 @@ -5,3 +5,4 @@ requests_unixsocket==0.1.5 testscenarios==0.5.0 pexpect==4.2.0 +pyftpdlib==1.4.0 diff -Nru snapcraft-2.27.1+17.04/requirements.txt snapcraft-2.28+17.04/requirements.txt --- snapcraft-2.27.1+17.04/requirements.txt 2017-02-17 13:45:14.000000000 +0000 +++ snapcraft-2.28+17.04/requirements.txt 2017-03-22 12:31:58.000000000 +0000 @@ -14,7 +14,7 @@ petname==1.12 pymacaroons==0.9.2 pymacaroons-pynacl==0.9.3 -pysha3==1.0b1 +pysha3==1.0b1; python_version < '3.6' simplejson==3.8.2 tabulate==0.7.5 python-debian==0.1.28 diff -Nru snapcraft-2.27.1+17.04/runtests.sh snapcraft-2.28+17.04/runtests.sh --- snapcraft-2.27.1+17.04/runtests.sh 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/runtests.sh 2017-03-22 12:31:58.000000000 +0000 @@ -48,7 +48,7 @@ python3 -m coverage && coverage="true" run_static_tests(){ - SRC_PATHS="bin snapcraft integration_tests snaps_tests" + SRC_PATHS="bin snapcraft integration_tests snaps_tests external_snaps_tests" python3 -m flake8 --max-complexity=10 $SRC_PATHS } diff -Nru snapcraft-2.27.1+17.04/schema/snapcraft.yaml snapcraft-2.28+17.04/schema/snapcraft.yaml --- snapcraft-2.27.1+17.04/schema/snapcraft.yaml 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/schema/snapcraft.yaml 2017-03-22 12:31:58.000000000 +0000 @@ -219,6 +219,9 @@ source: type: string default: '.' + source-checksum: + type: string + default: '' source-branch: type: string default: '' diff -Nru snapcraft-2.27.1+17.04/setup.py snapcraft-2.28+17.04/setup.py --- snapcraft-2.27.1+17.04/setup.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/setup.py 2017-03-22 12:31:58.000000000 +0000 @@ -44,11 +44,12 @@ 'snapcraft.internal.deltas', 'snapcraft.internal.pluginhandler', 'snapcraft.internal.pluginhandler.stage_package_grammar', + 'snapcraft.internal.repo', 'snapcraft.internal.sources', 'snapcraft.internal.states', 'snapcraft.plugins', 'snapcraft.storeapi'], - package_data={'snapcraft.internal': ['manifest.txt']}, + package_data={'snapcraft.internal.repo': ['manifest.txt']}, scripts=['bin/snapcraft', 'bin/snapcraft-parser'], data_files=[ ('share/snapcraft/schema', diff -Nru snapcraft-2.27.1+17.04/snap/snapcraft.yaml snapcraft-2.28+17.04/snap/snapcraft.yaml --- snapcraft-2.27.1+17.04/snap/snapcraft.yaml 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/snap/snapcraft.yaml 2017-03-23 18:31:04.000000000 +0000 @@ -0,0 +1,115 @@ +name: snapcraft +version: '2.28' +summary: easily create snaps +description: | + Snapcraft aims to make upstream developers' lives easier and as such is not + a single toolset, but instead is a collection of tools that enable the + natural workflow of an upstream to be extended with a simple release step + into Snappy. + +grade: devel +confinement: classic + +apps: + snapcraft: + command: bin/snapcraft + +parts: + snapcraft: + source: . + plugin: python + requirements: requirements.txt + build-packages: + - build-essential + - libarchive13 + - libffi-dev + - libsodium-dev + - liblzma-dev + stage-packages: + - libarchive13 + - libffi6 + - libsodium18 + - squashfs-tools + - xdelta3 + prime: + - '*' + - '**/*.pyc' + after: [python, apt] + tour: + source: tour + plugin: dump + organize: + "*": tour/ + python: + source: https://www.python.org/ftp/python/3.6.0/Python-3.6.0.tar.xz + plugin: autotools + configflags: [--prefix=/usr] + build-packages: [libssl-dev] + prime: + - -usr/include + apt: + source: https://github.com/Debian/apt + source-type: git + source-tag: 1.2.19 + source-depth: 1 + plugin: autotools + prepare: | + make startup + build: | + mkdir build + cd build + ../configure + make + install: | + cd build + install -d $SNAPCRAFT_PART_INSTALL/apt + cp -r bin/methods/* $SNAPCRAFT_PART_INSTALL/apt/ + cp -r bin/methods/* $SNAPCRAFT_PART_INSTALL/apt/ + install bin/apt-key $SNAPCRAFT_PART_INSTALL/apt/ + install bin/apt-mark $SNAPCRAFT_PART_INSTALL/apt/ + install bin/apt-internal-solver $SNAPCRAFT_PART_INSTALL/apt/ + install bin/apt-helper $SNAPCRAFT_PART_INSTALL/apt/ + install -d $SNAPCRAFT_PART_INSTALL/usr/lib + install bin/libapt-inst.so.2.0.0 $SNAPCRAFT_PART_INSTALL/usr/lib/ + install bin/libapt-pkg.so $SNAPCRAFT_PART_INSTALL/usr/lib/ + install bin/libapt-pkg-5.0-0.symver $SNAPCRAFT_PART_INSTALL/usr/lib/ + install bin/libapt-private.so $SNAPCRAFT_PART_INSTALL/usr/lib/ + install bin/libapt-private-0.0-0.symver $SNAPCRAFT_PART_INSTALL/usr/lib/ + install bin/libapt-private.so.0.0.0 $SNAPCRAFT_PART_INSTALL/usr/lib/ + install bin/libapt-inst.so.2.0 $SNAPCRAFT_PART_INSTALL/usr/lib/ + install bin/libapt-inst.so $SNAPCRAFT_PART_INSTALL/usr/lib/ + install bin/libapt-pkg.so.5.0 $SNAPCRAFT_PART_INSTALL/usr/lib/ + install bin/libapt-inst-2.0-0.symver $SNAPCRAFT_PART_INSTALL/usr/lib/ + install bin/libapt-pkg.so.5.0.0 $SNAPCRAFT_PART_INSTALL/usr/lib/ + install bin/libapt-private.so.0.0 $SNAPCRAFT_PART_INSTALL/usr/lib/ + install -d $SNAPCRAFT_PART_INSTALL/usr/include + cp -r include/* $SNAPCRAFT_PART_INSTALL/usr/include/ + prime: + - -usr/include + build-packages: + - gettext + - libbz2-dev + - libcurl4-gnutls-dev + - libdb-dev + - liblz4-dev + - liblzma-dev + - zlib1g-dev + after: [python] + gpg: + source: git://git.gnupg.org/gnupg.git + source-tag: gnupg-1.4.20 + source-depth: 1 + plugin: autotools + configflags: + - --enable-minimal + - --disable-makeinfo + - --disable-ldap + - --disable-finger + - --disable-nls + prepare: | + # This is fragile but we use a fixed tag + sed -i.bak -e 's/\(^ *g10 keyserver\) po doc ${checks}$/\1 ${checks}/' Makefile.am + build-packages: + - texinfo + prime: + - bin/gpgv diff -Nru snapcraft-2.27.1+17.04/snapcraft/file_utils.py snapcraft-2.28+17.04/snapcraft/file_utils.py --- snapcraft-2.27.1+17.04/snapcraft/file_utils.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/file_utils.py 2017-03-22 12:31:58.000000000 +0000 @@ -38,6 +38,7 @@ def replace_in_file(directory, file_pattern, search_pattern, replacement): """Searches and replaces patterns that match a file pattern. + :param str directory: The directory to look for files. :param str file_pattern: The file pattern to match inside directory. :param search_pattern: A re.compile'd pattern to search for within @@ -49,8 +50,37 @@ for root, directories, files in os.walk(directory): for file_name in files: if file_pattern.match(file_name): - _search_and_replace_contents(os.path.join(root, file_name), - search_pattern, replacement) + file_path = os.path.join(root, file_name) + # Don't bother trying to rewrite a symlink. It's either invalid + # or the linked file will be rewritten on its own. + if not os.path.islink(file_path): + search_and_replace_contents( + file_path, search_pattern, replacement) + + +def search_and_replace_contents(file_path, search_pattern, replacement): + """Search file and replace any occurrence of pattern with replacement. + + :param str file_path: Path of file to be searched. + :param re.RegexObject search_pattern: Pattern for which to search. + :param str replacement: The string to replace pattern. + """ + try: + with open(file_path, 'r+') as f: + try: + original = f.read() + except UnicodeDecodeError: + # This was probably a binary file. Skip it. + return + + replaced = search_pattern.sub(replacement, original) + if replaced != original: + f.seek(0) + f.truncate() + f.write(replaced) + except PermissionError as e: + logger.warning('Unable to open {path} for writing: {error}'.format( + path=file_path, error=e)) def link_or_copy(source, destination, follow_symlinks=False): @@ -157,30 +187,6 @@ shutil.copystat(source, destination, follow_symlinks=follow_symlinks) -def _search_and_replace_contents(file_path, search_pattern, replacement): - # Don't bother trying to rewrite a symlink. It's either invalid or the - # linked file will be rewritten on its own. - if os.path.islink(file_path): - return - - try: - with open(file_path, 'r+') as f: - try: - original = f.read() - except UnicodeDecodeError: - # This was probably a binary file. Skip it. - return - - replaced = search_pattern.sub(replacement, original) - if replaced != original: - f.seek(0) - f.truncate() - f.write(replaced) - except PermissionError as e: - logger.warning('Unable to open {path} for writing: {error}'.format( - path=file_path, error=e)) - - def executable_exists(path): """Return True if 'path' exists and is readable and executable.""" return os.path.exists(path) and os.access(path, os.R_OK | os.X_OK) diff -Nru snapcraft-2.27.1+17.04/snapcraft/__init__.py snapcraft-2.28+17.04/snapcraft/__init__.py --- snapcraft-2.27.1+17.04/snapcraft/__init__.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/__init__.py 2017-03-22 12:31:58.000000000 +0000 @@ -136,9 +136,9 @@ parts is available to every snap author - just say 'after' and list the parts you want that others have already defined. - - build-packages: [deb, deb, deb...] + - build-packages: [pkg, pkg, pkg...] - A list of Ubuntu packages to install on the build host before building + A list of packages to install on the build host before building the part. The files from these packages typically will not go into the final snap unless they contain libraries that are direct dependencies of binaries within the snap (in which case they'll be discovered via `ldd`), @@ -146,7 +146,7 @@ - stage-packages: YAML list - A set of Ubuntu packages to be downloaded and unpacked to join the part + A set of packages to be downloaded and unpacked to join the part before it's built. Note that these packages are not installed on the host. Like the rest of the part, all files from these packages will make it into the final snap unless filtered out via the `snap` keyword. @@ -364,7 +364,7 @@ create_key, close, download, - history, + revisions, gated, list_keys, list_registered, diff -Nru snapcraft-2.27.1+17.04/snapcraft/internal/common.py snapcraft-2.28+17.04/snapcraft/internal/common.py --- snapcraft-2.27.1+17.04/snapcraft/internal/common.py 2017-02-17 13:45:14.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/internal/common.py 2017-03-23 18:31:04.000000000 +0000 @@ -112,6 +112,9 @@ def get_schemadir(): + snap = os.environ.get('SNAP') + if snap: + return os.path.join(snap, 'share', 'snapcraft', 'schema') return _schemadir @@ -145,6 +148,9 @@ def get_tourdir(): + snap = os.environ.get('SNAP') + if snap: + return os.path.join(snap, 'tour') return _tourdir diff -Nru snapcraft-2.27.1+17.04/snapcraft/internal/deprecations.py snapcraft-2.28+17.04/snapcraft/internal/deprecations.py --- snapcraft-2.27.1+17.04/snapcraft/internal/deprecations.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/internal/deprecations.py 2017-03-22 12:31:58.000000000 +0000 @@ -28,6 +28,7 @@ 'dn1': "The 'snap' keyword has been replaced by 'prime'.", 'dn2': "Custom plugins should now be placed in 'snap/plugins'.", 'dn3': "Assets in 'setup/gui' should now be placed in 'snap/gui'.", + 'dn4': "The 'history' command has been replaced by 'list-revisions'.", } _DEPRECATION_URL_FMT = 'http://snapcraft.io/docs/deprecation-notices/{id}' diff -Nru snapcraft-2.27.1+17.04/snapcraft/internal/libraries.py snapcraft-2.28+17.04/snapcraft/internal/libraries.py --- snapcraft-2.27.1+17.04/snapcraft/internal/libraries.py 2017-02-17 13:45:14.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/internal/libraries.py 2017-03-22 12:31:58.000000000 +0000 @@ -71,7 +71,7 @@ lib_path = os.path.join(common.get_librariesdir(), release) if not os.path.exists(lib_path): - logger.warning('No libraries to exclude from this release') + logger.debug('No libraries to exclude from this release') # Always exclude libc.so.6 return frozenset(['libc.so.6']) @@ -86,7 +86,7 @@ This may include libraries contained within the project. """ - logger.debug('Getting dependencies for {!r}'.format(elf)) + logger.debug('Getting dependencies for {!r}'.format(str(elf))) ldd_out = '' try: ldd_out = common.run_output(['ldd', elf]).split('\n') diff -Nru snapcraft-2.27.1+17.04/snapcraft/internal/lifecycle.py snapcraft-2.28+17.04/snapcraft/internal/lifecycle.py --- snapcraft-2.27.1+17.04/snapcraft/internal/lifecycle.py 2017-02-17 19:54:35.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/internal/lifecycle.py 2017-03-22 12:31:58.000000000 +0000 @@ -108,7 +108,7 @@ :returns: A dict with the snap name, version, type and architectures. """ config = snapcraft.internal.load_config(project_options) - repo.install_build_packages(config.build_tools) + repo.Repo.install_build_packages(config.build_tools) if (os.environ.get('SNAPCRAFT_SETUP_CORE') and config.data['confinement'] == 'classic'): @@ -325,14 +325,6 @@ def cleanbuild(project_options, remote=''): - if not repo.is_package_installed('lxd'): - raise EnvironmentError( - 'The lxd package is not installed, in order to use `cleanbuild` ' - 'you must install lxd onto your system. Refer to the ' - '"Ubuntu Desktop and Ubuntu Server" section on ' - 'https://linuxcontainers.org/lxd/getting-started-cli/' - '#ubuntu-desktop-and-ubuntu-server to enable a proper setup.') - config = snapcraft.internal.load_config(project_options) tar_filename = '{}_{}_source.tar.bz2'.format( config.data['name'], config.data['version']) diff -Nru snapcraft-2.27.1+17.04/snapcraft/internal/lxd.py snapcraft-2.28+17.04/snapcraft/internal/lxd.py --- snapcraft-2.27.1+17.04/snapcraft/internal/lxd.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/internal/lxd.py 2017-03-23 18:31:04.000000000 +0000 @@ -1,7 +1,7 @@ #!/usr/bin/python3 # -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*- # -# Copyright (C) 2016 Canonical Ltd +# Copyright (C) 2016-2017 Canonical Ltd # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License version 3 as @@ -17,15 +17,14 @@ import logging import os +import sys from contextlib import contextmanager -from subprocess import check_call, CalledProcessError +from subprocess import check_call, check_output, CalledProcessError from time import sleep import petname from snapcraft.internal.errors import SnapcraftEnvironmentError -from snapcraft.internal import cache - logger = logging.getLogger(__name__) @@ -43,25 +42,11 @@ self._tar_filename = tar_filename self._project_options = project_options container_name = 'snapcraft-{}'.format(petname.Generate(3, '-')) - if remote: - self._verify_remote(remote) - self._container_name = '{}:{}'.format(remote, container_name) - else: - self._container_name = container_name - - def _verify_remote(self, remote): - # There is no easy way to grep the results from `lxc remote list` - # so we try and execute a simple operation against the remote. - try: - check_call(['lxc', 'list', '{}:'.format(remote)]) - except CalledProcessError as e: - raise SnapcraftEnvironmentError( - 'There are either no permissions or the remote {!r} ' - 'does not exist.\n' - 'Verify the existing remotes by running `lxc remote list`\n' - 'To setup a new remote, follow the instructions on\n' - 'https://linuxcontainers.org/lxd/getting-started-cli/' - '#multiple-hosts'.format(remote)) from e + + if not remote: + remote = _get_default_remote() + _verify_remote(remote) + self._container_name = '{}:{}'.format(remote, container_name) def _push_file(self, src, dst): check_call(['lxc', 'file', 'push', @@ -114,20 +99,6 @@ self._push_file(self._tar_filename, dst) self._container_run(['tar', 'xvf', dst]) - logger.info('Copying snapcraft cache into container') - snapcraft_cache = cache.SnapcraftCache() - # Later versions have a `push --recursive`. For now we have to do it - # the hard way and walk the cache ourselves. - for root, dirs, files in os.walk(snapcraft_cache.cache_root): - destination_root = os.path.join( - '/root', '.cache', 'snapcraft', os.path.relpath( - root, snapcraft_cache.cache_root)) - self._container_run(['mkdir', '-p', destination_root]) - for file_path in files: - source = os.path.join(root, file_path) - destination = os.path.join(destination_root, file_path) - self._push_file(source, destination) - def _pull_snap(self): src = os.path.join('/root', self._snap_output) self._pull_file(src, self._snap_output) @@ -147,3 +118,47 @@ if retry_count == 0: raise e logger.info('Network connection established') + + +def _get_default_remote(): + """Query and return the default lxd remote. + + Use the lxc command to query for the default lxd remote. In most + cases this will return the local remote. + + :returns: default lxd remote. + :rtype: string. + :raises snapcraft.internal.errors.SnapcraftEnvironmentError: + raised if the lxc call fails. + """ + try: + default_remote = check_output(['lxc', 'remote', 'get-default']) + except CalledProcessError: + raise SnapcraftEnvironmentError( + 'You must have LXD installed in order to use cleanbuild. ' + 'However, it is either not installed or not configured ' + 'properly.\n' + 'Refer to the documentation at ' + 'https://linuxcontainers.org/lxd/getting-started-cli.') + return default_remote.decode(sys.getfilesystemencoding()).strip() + + +def _verify_remote(remote): + """Verify that the lxd remote exists. + + :param str remote: the lxd remote to verify. + :raises snapcraft.internal.errors.SnapcraftEnvironmentError: + raised if the lxc call listing the remote fails. + """ + # There is no easy way to grep the results from `lxc remote list` + # so we try and execute a simple operation against the remote. + try: + check_output(['lxc', 'list', '{}:'.format(remote)]) + except CalledProcessError as e: + raise SnapcraftEnvironmentError( + 'There are either no permissions or the remote {!r} ' + 'does not exist.\n' + 'Verify the existing remotes by running `lxc remote list`\n' + 'To setup a new remote, follow the instructions at\n' + 'https://linuxcontainers.org/lxd/getting-started-cli/' + '#multiple-hosts'.format(remote)) from e diff -Nru snapcraft-2.27.1+17.04/snapcraft/internal/manifest.txt snapcraft-2.28+17.04/snapcraft/internal/manifest.txt --- snapcraft-2.27.1+17.04/snapcraft/internal/manifest.txt 2016-11-02 08:25:17.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/internal/manifest.txt 1970-01-01 00:00:00.000000000 +0000 @@ -1,110 +0,0 @@ -adduser -apt -apt-utils -base-files -base-passwd -bash -bsdutils -coreutils -dash -debconf -debconf-i18n -debianutils -diffutils -dmsetup -dpkg -e2fslibs -e2fsprogs -file -findutils -gcc-4.9-base -gcc-5-base -gnupg -gpgv -grep -gzip -hostname -init -initscripts -insserv -libacl1 -libapparmor1 -libapt -libapt-inst1.5 -libapt-pkg4.12 -libattr1 -libaudit-common -libaudit1 -libblkid1 -libbz2-1.0 -libc-bin -libc6 -libcap2 -libcap2-bin -libcomerr2 -libcryptsetup4 -libdb5.3 -libdebconfclient0 -libdevmapper1.02.1 -libgcc1 -libgcrypt20 -libgpg-error0 -libgpm2 -libkmod2 -liblocale-gettext-perl -liblzma5 -libmagic1 -libmount1 -libncurses5 -libncursesw5 -libpam-modules -libpam-modules-bin -libpam-runtime -libpam0g -libpcre3 -libprocps3 -libreadline6 -libselinux1 -libsemanage-common -libsemanage1 -libsepol1 -libslang2 -libsmartcols1 -libss2 -libstdc++6 -libsystemd0 -libtext-charwidth-perl -libtext-iconv-perl -libtext-wrapi18n-perl -libtinfo5 -libudev1 -libusb-0.1-4 -libustr-1.0-1 -libuuid1 -locales -login -lsb-base -makedev -manpages -manpages-dev -mawk -mount -multiarch-support -ncurses-base -ncurses-bin -passwd -perl-base -procps -readline-common -sed -sensible-utils -systemd -systemd-sysv -sysv-rc -sysvinit-utils -tar -tzdata -ubuntu-keyring -udev -util-linux -zlib1g diff -Nru snapcraft-2.27.1+17.04/snapcraft/internal/parser.py snapcraft-2.28+17.04/snapcraft/internal/parser.py --- snapcraft-2.27.1+17.04/snapcraft/internal/parser.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/internal/parser.py 2017-03-22 12:31:58.000000000 +0000 @@ -51,9 +51,12 @@ from collections import OrderedDict import snapcraft -from snapcraft.internal import log, repo, sources -from snapcraft.internal.errors import SnapcraftError, InvalidWikiEntryError -from snapcraft.internal.project_loader import replace_attr +from snapcraft.internal import log, project_loader, repo, sources +from snapcraft.internal.errors import ( + SnapcraftError, + InvalidWikiEntryError, + SnapcraftEnvironmentError, +) class BadSnapcraftYAMLError(Exception): @@ -96,23 +99,8 @@ def _get_origin_data(origin_dir): origin_data = {} - snapcraft_yaml_file = os.path.join(origin_dir, 'snapcraft.yaml') - hidden_snapcraft_yaml_file = os.path.join(origin_dir, '.snapcraft.yaml') - - # read either 'snapcraft.yaml' or '.snapcraft.yaml' but not both - if not os.path.exists(snapcraft_yaml_file) and not os.path.exists( - hidden_snapcraft_yaml_file): - raise MissingSnapcraftYAMLError() - - if os.path.exists(snapcraft_yaml_file): - if os.path.exists(hidden_snapcraft_yaml_file): - raise BadSnapcraftYAMLError( - 'Origin has both "snapcraft.yaml" and ".snapcraft.yaml"') - else: - yaml_file = snapcraft_yaml_file - elif os.path.exists(hidden_snapcraft_yaml_file): - yaml_file = hidden_snapcraft_yaml_file + yaml_file = project_loader.get_snapcraft_yaml(base_dir=origin_dir) try: with open(yaml_file) as fp: origin_data = yaml.load(fp) @@ -155,7 +143,7 @@ ('$SNAPCRAFT_PROJECT_VERSION', origin_version), ] - source_part = replace_attr(source_part, replacements) + source_part = project_loader.replace_attr(source_part, replacements) if source_part: source_part = _update_source(source_part, origin) @@ -282,7 +270,8 @@ _process_wiki_entry( entry, master_parts_list, missing_parts, pending_validation_entries) - except SnapcraftError as e: + except (SnapcraftError, project_loader.SnapcraftYamlFileError, + SnapcraftEnvironmentError) as e: logger.warning(e) wiki_errors += 1 diff -Nru snapcraft-2.27.1+17.04/snapcraft/internal/parts.py snapcraft-2.28+17.04/snapcraft/internal/parts.py --- snapcraft-2.27.1+17.04/snapcraft/internal/parts.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/internal/parts.py 2017-03-22 12:31:58.000000000 +0000 @@ -1,6 +1,6 @@ # -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*- # -# Copyright (C) 2016 Canonical Ltd +# Copyright (C) 2016-2017 Canonical Ltd # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License version 3 as @@ -276,7 +276,8 @@ self.build_tools += part.code.build_packages if part.source_handler and part.source_handler.command: self.build_tools.append( - repo.get_packages_for_source_type(part.source_handler.command)) + repo.Repo.get_packages_for_source_type( + part.source_handler.command)) self.all_parts.append(part) return part @@ -308,6 +309,8 @@ self._project_options.arch_triplet, core_dynamic_linker=core_dynamic_linker) env.append('SNAPCRAFT_PART_INSTALL={}'.format(part.installdir)) + env.append('SNAPCRAFT_PARALLEL_BUILD_COUNT={}'.format( + self._project_options.parallel_build_count)) else: env += part.env(stagedir) env += project_loader._runtime_env( diff -Nru snapcraft-2.27.1+17.04/snapcraft/internal/pluginhandler/__init__.py snapcraft-2.28+17.04/snapcraft/internal/pluginhandler/__init__.py --- snapcraft-2.27.1+17.04/snapcraft/internal/pluginhandler/__init__.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/internal/pluginhandler/__init__.py 2017-03-22 12:31:58.000000000 +0000 @@ -76,6 +76,7 @@ self._name = part_name self._part_properties = _expand_part_properties( part_properties, part_schema) + self.stage_packages = [] # Some legacy parts can have a '/' in them to separate the main project # part with the subparts. This is rather unfortunate as it affects the @@ -179,6 +180,7 @@ source_handler = handler_class( properties['source'], self.sourcedir, + source_checksum=properties['source-checksum'], source_branch=properties['source-branch'], source_tag=properties['source-tag'], source_depth=properties['source-depth'], @@ -305,8 +307,8 @@ def _fetch_stage_packages(self): try: - self._stage_package_handler.fetch() - except repo.PackageNotFoundError as e: + self.stage_packages = self._stage_package_handler.fetch() + except repo.errors.PackageNotFoundError as e: raise RuntimeError("Error downloading stage packages for part " "{!r}: {}".format(self.name, e.message)) @@ -332,8 +334,8 @@ pull_properties = self.code.get_pull_properties() self.mark_done('pull', states.PullState( - pull_properties, self._part_properties, - self._project_options)) + pull_properties, part_properties=self._part_properties, + project=self._project_options, stage_packages=self.stage_packages)) def clean_pull(self, hint=''): if self.is_clean('pull'): diff -Nru snapcraft-2.27.1+17.04/snapcraft/internal/pluginhandler/stage_package_grammar/_on.py snapcraft-2.28+17.04/snapcraft/internal/pluginhandler/stage_package_grammar/_on.py --- snapcraft-2.27.1+17.04/snapcraft/internal/pluginhandler/stage_package_grammar/_on.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/internal/pluginhandler/stage_package_grammar/_on.py 2017-03-22 12:31:58.000000000 +0000 @@ -33,7 +33,7 @@ >>> import tempfile >>> from snapcraft import repo, ProjectOptions >>> with tempfile.TemporaryDirectory() as cache_dir: - ... repo_instance = repo.Ubuntu(cache_dir) + ... repo_instance = repo.Repo(cache_dir) ... options = ProjectOptions(target_deb_arch='i386') ... clause = OnStatement(on='on amd64', body=['foo'], ... project_options=options, @@ -51,9 +51,9 @@ :param project_options: Instance of ProjectOptions to use to process clause. :type project_options: snapcraft.ProjectOptions - :param repo_instance: repo.Ubuntu instance used for checking package + :param repo_instance: repo.Repo instance used for checking package validity. - :type repo_instance: repo.Ubuntu + :type repo_instance: repo.Repo """ self.selectors = _extract_on_clause_selectors(on) diff -Nru snapcraft-2.27.1+17.04/snapcraft/internal/pluginhandler/stage_package_grammar/_processor.py snapcraft-2.28+17.04/snapcraft/internal/pluginhandler/stage_package_grammar/_processor.py --- snapcraft-2.27.1+17.04/snapcraft/internal/pluginhandler/stage_package_grammar/_processor.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/internal/pluginhandler/stage_package_grammar/_processor.py 2017-03-22 12:31:58.000000000 +0000 @@ -31,9 +31,9 @@ :param project_options: Instance of ProjectOptions to use to determine stage packages. :type project_options: snapcraft.ProjectOptions - :param repo_instance: repo.Ubuntu instance used for checking package + :param repo_instance: repo.Repo instance used for checking package validity. - :type repo_instance: repo.Ubuntu + :type repo_instance: repo.Repo :return: Packages to stage :rtype: set diff -Nru snapcraft-2.27.1+17.04/snapcraft/internal/pluginhandler/stage_package_grammar/_try.py snapcraft-2.28+17.04/snapcraft/internal/pluginhandler/stage_package_grammar/_try.py --- snapcraft-2.27.1+17.04/snapcraft/internal/pluginhandler/stage_package_grammar/_try.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/internal/pluginhandler/stage_package_grammar/_try.py 2017-03-22 12:31:58.000000000 +0000 @@ -24,7 +24,7 @@ >>> import tempfile >>> from snapcraft import repo, ProjectOptions >>> with tempfile.TemporaryDirectory() as cache_dir: - ... repo_instance = repo.Ubuntu(cache_dir) + ... repo_instance = repo.Repo(cache_dir) ... options = ProjectOptions(target_deb_arch='i386') ... clause = TryStatement(body=['invalid'], project_options=options, ... repo_instance=repo_instance) @@ -40,9 +40,9 @@ :param project_options: Instance of ProjectOptions to use to process clause. :type project_options: snapcraft.ProjectOptions - :param repo_instance: repo.Ubuntu instance used for checking package + :param repo_instance: repo.Repo instance used for checking package validity. - :type repo_instance: repo.Ubuntu + :type repo_instance: repo.Repo """ self._body = body @@ -100,14 +100,14 @@ """Ensure that all packages are valid. :param packages: Iterable container of package names. - :param repo_instance: repo.Ubuntu instance to use for validity check. - :type repo_instance: repo.Ubuntu + :param repo_instance: repo.Repo instance to use for validity check. + :type repo_instance: repo.Repo For example: >>> import tempfile >>> from snapcraft import repo, ProjectOptions >>> with tempfile.TemporaryDirectory() as cache_dir: - ... ubuntu = repo.Ubuntu(cache_dir) + ... ubuntu = repo.Repo(cache_dir) ... _all_packages_valid(['valid'], ubuntu) ... _all_packages_valid(['valid', 'invalid'], ubuntu) True diff -Nru snapcraft-2.27.1+17.04/snapcraft/internal/pluginhandler/_stage_package_handler.py snapcraft-2.28+17.04/snapcraft/internal/pluginhandler/_stage_package_handler.py --- snapcraft-2.27.1+17.04/snapcraft/internal/pluginhandler/_stage_package_handler.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/internal/pluginhandler/_stage_package_handler.py 2017-03-22 12:31:58.000000000 +0000 @@ -32,7 +32,7 @@ ... cache_dir = os.path.join(tmp, 'cache') ... unpack_dir = os.path.join(tmp, 'unpack') ... handler = StagePackageHandler(['foo'], cache_dir) - ... handler.fetch() + ... pkg_list = handler.fetch() ... handler.unpack(unpack_dir) """ @@ -54,23 +54,23 @@ self._sources = sources self._project_options = project_options self.__stage_packages = None - self.__ubuntu = None + self.__repo = None @property - def _ubuntu(self): - if not self.__ubuntu: - self.__ubuntu = repo.Ubuntu( + def _repo(self): + if not self.__repo: + self.__repo = repo.Repo( self._cache_dir, sources=self._sources, project_options=self._project_options) - return self.__ubuntu + return self.__repo @property def _stage_packages(self): # Comparing to None here since after calculation it may be an empty set if self.__stage_packages is None: self.__stage_packages = stage_package_grammar.process_grammar( - self._grammar, self._project_options, self._ubuntu) + self._grammar, self._project_options, self._repo) return self.__stage_packages @@ -81,10 +81,13 @@ the cache. """ + pkg_list = [] if self._stage_packages: logger.debug('Fetching stage-packages {!r}'.format( self._stage_packages)) - self._ubuntu.get(self._stage_packages) + pkg_list = self._repo.get(self._stage_packages) + + return pkg_list def unpack(self, unpack_dir): """Unpack fetched stage packages into directory. @@ -96,4 +99,4 @@ if self._stage_packages: logger.debug('Unpacking stage-packages to {!r}'.format( unpack_dir)) - self._ubuntu.unpack(unpack_dir) + self._repo.unpack(unpack_dir) diff -Nru snapcraft-2.27.1+17.04/snapcraft/internal/project_loader.py snapcraft-2.28+17.04/snapcraft/internal/project_loader.py --- snapcraft-2.27.1+17.04/snapcraft/internal/project_loader.py 2017-02-17 19:54:35.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/internal/project_loader.py 2017-03-22 12:31:58.000000000 +0000 @@ -329,12 +329,13 @@ snap_rpaths = common.get_library_paths(snap_path, arch_triplet, existing_only=False) + # snap_rpaths before core_rpaths to prefer libraries from the snap. rpaths = formatting_utils.combine_paths( - core_rpaths + snap_rpaths, prepend='', separator=':') + snap_rpaths + core_rpaths, prepend='', separator=':') env.append('LDFLAGS="$LDFLAGS ' # Building tools to continue the build becomes problematic # with nodefaultlib. - '-Wl,-z,nodefaultlib ' + # '-Wl,-z,nodefaultlib ' '-Wl,--dynamic-linker={0} ' '-Wl,-rpath,{1}"'.format(core_dynamic_linker, rpaths)) @@ -360,13 +361,16 @@ return env -def get_snapcraft_yaml(): +def get_snapcraft_yaml(base_dir=None): possible_yamls = [ os.path.join('snap', 'snapcraft.yaml'), 'snapcraft.yaml', '.snapcraft.yaml', ] + if base_dir: + possible_yamls = [os.path.join(base_dir, x) for x in possible_yamls] + snapcraft_yamls = [y for y in possible_yamls if os.path.exists(y)] if not snapcraft_yamls: diff -Nru snapcraft-2.27.1+17.04/snapcraft/internal/repo/_base.py snapcraft-2.28+17.04/snapcraft/internal/repo/_base.py --- snapcraft-2.27.1+17.04/snapcraft/internal/repo/_base.py 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/internal/repo/_base.py 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,267 @@ +# -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*- +# +# Copyright (C) 2017 Canonical Ltd +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License version 3 as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . + +import contextlib +import fileinput +import itertools +import logging +import os +import re +import shutil +import stat + +from snapcraft import file_utils + +_BIN_PATHS = ( + 'bin', + 'sbin', + 'usr/bin', + 'usr/sbin', +) + + +logger = logging.getLogger(__name__) + + +class BaseRepo: + """Base implementation for a platform specific repo handler. + + Generally any new repo handling system would inherit from this and + implement: + + - get + - unpack + - get_package_libraries + - get_packages_for_source_type + - install_build_packages + - is_package_installed + + At the end of the `unpack` method `normalize` needs to be called to + adapt the artifacts downloaded to be generic enough for building a snap.""" + + @classmethod + def get_package_libraries(cls, package_name): + """Return a list of libraries in package_name. + + Given the contents of package_name, return the subset of what are + considered libraries from those contents, be it static or shared. + + :param str package: package name to get library contents from. + :returns: a list of libraries that package_name provides. This includes + directories. + :rtype: set. + """ + raise NotImplemented() + + @classmethod + def get_packages_for_source_type(cls, source_type): + """Return a list of packages required to to work with source_type. + + source_type can be any of the following: + + - bzr + - deb + - rpm + - git + - hg + - mercurial + - subversion + - svn + - tar + - zip + + :param str source_type: a VCS source type to handle. + :returns: a list of packages that need to be installed on the host. + :rtype: list of strings. + """ + raise NotImplementedError() + + @classmethod + def install_build_packages(cls, package_names): + """Install packages on the host required to build. + + :param package_names: a list of package names to install. + :type package_names: a list of strings. + :raises snapcraft.repo.errors.BuildPackageNotFoundError: + if one of the package_names cannot be installed. + """ + raise NotImplementedError() + + @classmethod + def is_package_installed(cls, package_name): + """Return a bool indicating if package_name is installed. + + :param str package_name: the package name to query. + :returns: True if package_name is installed if not False. + :rtype: boolean + """ + raise NotImplementedError() + + def __init__(self, rootdir, *args, **kwargs): + """Initialize a repository handler. + + :param str rootdir: the root directory to work on. This may well be to + create repo handling specific caches or download + artifacts. + """ + self.rootdir = rootdir + + def get(self, package_names): + """Get the packages. + + This method needs to be implemented by the inheriting class. It + is called all along the core of snapcraft to obtain the + package_names and store them in some repo defined location to + later unpack. + + :param str package_names: list of packages to obtain. + :returns: list of packages obtained, versioned if possible. + :rtype: list of strings. + :raises snapcraft.repo.errors.PackageNotFoundError: + when a package in package_names is not found. + """ + raise NotImplemented() + + def unpack(self, unpackdir): + """Unpack obtained packages into unpackdir. + + This method needs to be implemented by the inheriting class. It + is called all along the core of snapcraft to unpack the previously + obtained package_names in get into unpackdir. + + After the unpack logic is executed, normalize should be called. + + :param str unpackdir: target directory to unpack packages to. + """ + raise NotImplemented() + + def normalize(self, unpackdir): + """Normalize artifacts in unpackdir. + + Repo specific packages are generally created to live in a specific + distro. What normalize does is scan through the unpacked artifacts + and slightly modifies them to work better with snapcraft projects + when building and to also work within a snap's environment. + + :param str unpackdir: directory where files where unpacked. + """ + self._fix_artifacts(unpackdir) + self._fix_xml_tools(unpackdir) + self._fix_shebangs(unpackdir) + + def _fix_artifacts(self, unpackdir): + """Perform various modifications to unpacked artifacts. + + Sometimes distro packages will contain absolute symlinks (e.g. if the + relative path would go all the way to root, they just do absolute). We + can't have that, so instead clean those absolute symlinks. + + Some unpacked items will also contain suid binaries which we do not + want in the resulting snap. + """ + for root, dirs, files in os.walk(unpackdir): + # Symlinks to directories will be in dirs, while symlinks to + # non-directories will be in files. + for entry in itertools.chain(files, dirs): + path = os.path.join(root, entry) + if os.path.islink(path) and os.path.isabs(os.readlink(path)): + self._fix_symlink(path, unpackdir, root) + elif os.path.exists(path): + _fix_filemode(path) + + if path.endswith('.pc') and not os.path.islink(path): + fix_pkg_config(unpackdir, path) + + def _fix_xml_tools(self, unpackdir): + xml2_config_path = os.path.join( + unpackdir, 'usr', 'bin', 'xml2-config') + with contextlib.suppress(FileNotFoundError): + file_utils.search_and_replace_contents( + xml2_config_path, re.compile(r'prefix=/usr'), + 'prefix={}/usr'.format(unpackdir)) + + xslt_config_path = os.path.join( + unpackdir, 'usr', 'bin', 'xslt-config') + with contextlib.suppress(FileNotFoundError): + file_utils.search_and_replace_contents( + xslt_config_path, re.compile(r'prefix=/usr'), + 'prefix={}/usr'.format(unpackdir)) + + def _fix_symlink(self, path, unpackdir, root): + host_target = os.readlink(path) + if host_target in self.get_package_libraries('libc6'): + logger.debug( + "Not fixing symlink {!r}: it's pointing to libc".format( + host_target)) + return + + target = os.path.join(unpackdir, os.readlink(path)[1:]) + if (not os.path.exists(target) and not + _try_copy_local(path, target)): + return + os.remove(path) + os.symlink(os.path.relpath(target, root), path) + + def _fix_shebangs(self, unpackdir): + """Changes hard coded shebangs for files in _BIN_PATHS to use env.""" + paths = [p for p in _BIN_PATHS + if os.path.exists(os.path.join(unpackdir, p))] + for p in [os.path.join(unpackdir, p) for p in paths]: + file_utils.replace_in_file(p, re.compile(r''), + re.compile(r'#!.*python\n'), + r'#!/usr/bin/env python\n') + + +def _try_copy_local(path, target): + real_path = os.path.realpath(path) + if os.path.exists(real_path): + logger.warning( + 'Copying needed target link from the system {}'.format(real_path)) + os.makedirs(os.path.dirname(target), exist_ok=True) + shutil.copyfile(os.readlink(path), target) + return True + else: + logger.warning( + '{} will be a dangling symlink'.format(path)) + return False + + +def fix_pkg_config(root, pkg_config_file, prefix_trim=None): + """Opens a pkg_config_file and prefixes the prefix with root.""" + pattern_trim = None + if prefix_trim: + pattern_trim = re.compile( + '^prefix={}(?P.*)'.format(prefix_trim)) + pattern = re.compile('^prefix=(?P.*)') + + with fileinput.input(pkg_config_file, inplace=True) as input_file: + for line in input_file: + match = pattern.search(line) + if prefix_trim: + match_trim = pattern_trim.search(line) + if prefix_trim and match_trim: + print('prefix={}{}'.format(root, match_trim.group('prefix'))) + elif match: + print('prefix={}{}'.format(root, match.group('prefix'))) + else: + print(line, end='') + + +def _fix_filemode(path): + mode = stat.S_IMODE(os.stat(path, follow_symlinks=False).st_mode) + if mode & 0o4000 or mode & 0o2000: + logger.warning('Removing suid/guid from {}'.format(path)) + os.chmod(path, mode & 0o1777) diff -Nru snapcraft-2.27.1+17.04/snapcraft/internal/repo/_deb.py snapcraft-2.28+17.04/snapcraft/internal/repo/_deb.py --- snapcraft-2.27.1+17.04/snapcraft/internal/repo/_deb.py 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/internal/repo/_deb.py 2017-03-23 12:02:13.000000000 +0000 @@ -0,0 +1,449 @@ +# -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*- +# +# Copyright (C) 2015-2016 Canonical Ltd +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License version 3 as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . + +import contextlib +import glob +import hashlib +import logging +import os +import platform +import shutil +import stat +import string +import subprocess +import sys +import urllib +import urllib.request + +import apt +from xml.etree import ElementTree + +import snapcraft +from snapcraft import file_utils +from snapcraft.internal import cache +from snapcraft.internal.indicators import is_dumb_terminal +from ._base import BaseRepo +from . import errors + + +logger = logging.getLogger(__name__) + +_DEFAULT_SOURCES = \ + '''deb http://${prefix}.ubuntu.com/${suffix}/ ${release} main restricted +deb http://${prefix}.ubuntu.com/${suffix}/ ${release}-updates main restricted +deb http://${prefix}.ubuntu.com/${suffix}/ ${release} universe +deb http://${prefix}.ubuntu.com/${suffix}/ ${release}-updates universe +deb http://${prefix}.ubuntu.com/${suffix}/ ${release} multiverse +deb http://${prefix}.ubuntu.com/${suffix}/ ${release}-updates multiverse +deb http://${security}.ubuntu.com/${suffix} ${release}-security main restricted +deb http://${security}.ubuntu.com/${suffix} ${release}-security universe +deb http://${security}.ubuntu.com/${suffix} ${release}-security multiverse +''' +_GEOIP_SERVER = "http://geoip.ubuntu.com/lookup" +_library_list = dict() + + +class _AptCache: + + def __init__(self, deb_arch, *, sources_list=None, use_geoip=False): + self._deb_arch = deb_arch + self._sources_list = sources_list + self._use_geoip = use_geoip + + def _setup_apt(self, cache_dir): + # Do not install recommends + apt.apt_pkg.config.set('Apt::Install-Recommends', 'False') + + # Methods and solvers dir for when in the SNAP + if os.getenv('SNAP'): + snap_dir = os.getenv('SNAP') + apt_dir = os.path.join(snap_dir, 'apt') + apt.apt_pkg.config.set('Dir', apt_dir) + # yes apt is broken like that we need to append os.path.sep + apt.apt_pkg.config.set('Dir::Bin::methods', + apt_dir + os.path.sep) + apt.apt_pkg.config.set('Dir::Bin::solvers::', + apt_dir + os.path.sep) + apt_key_path = os.path.join(apt_dir, 'apt-key') + apt.apt_pkg.config.set('Dir::Bin::apt-key', apt_key_path) + gpgv_path = os.path.join(snap_dir, 'bin', 'gpgv') + apt.apt_pkg.config.set('Apt::Key::gpgvcommand', gpgv_path) + apt.apt_pkg.config.set('Dir::Etc::Trusted', + '/etc/apt/trusted.gpg') + apt.apt_pkg.config.set('Dir::Etc::TrustedParts', + '/etc/apt/trusted.gpg.d/') + + # Make sure we always use the system GPG configuration, even with + # apt.Cache(rootdir). + for key in 'Dir::Etc::Trusted', 'Dir::Etc::TrustedParts': + apt.apt_pkg.config.set(key, apt.apt_pkg.config.find_file(key)) + + # Clear up apt's Post-Invoke-Success as we are not running + # on the system. + apt.apt_pkg.config.clear('APT::Update::Post-Invoke-Success') + + self.progress = apt.progress.text.AcquireProgress() + if is_dumb_terminal(): + # Make output more suitable for logging. + self.progress.pulse = lambda owner: True + self.progress._width = 0 + + sources_list_file = os.path.join( + cache_dir, 'etc', 'apt', 'sources.list') + + os.makedirs(os.path.dirname(sources_list_file), exist_ok=True) + with open(sources_list_file, 'w') as f: + f.write(self._collected_sources_list()) + + # dpkg also needs to be in the rootdir in order to support multiarch + # (apt calls dpkg --print-foreign-architectures). + dpkg_path = shutil.which('dpkg') + if dpkg_path: + # Symlink it into place + destination = os.path.join(cache_dir, dpkg_path[1:]) + if not os.path.exists(destination): + os.makedirs(os.path.dirname(destination), exist_ok=True) + os.symlink(dpkg_path, destination) + else: + logger.warning( + "Cannot find 'dpkg' command needed to support multiarch") + + apt_cache = apt.Cache(rootdir=cache_dir, memonly=True) + apt_cache.update(fetch_progress=self.progress, + sources_list=sources_list_file) + + return apt_cache + + @contextlib.contextmanager + def archive(self, cache_dir): + try: + apt_cache = self._setup_apt(cache_dir) + apt_cache.open() + + try: + yield apt_cache + finally: + apt_cache.close() + except Exception as e: + logger.debug('Exception occured: {!r}'.format(e)) + raise e + + def sources_digest(self): + return hashlib.sha384(self._collected_sources_list().encode( + sys.getfilesystemencoding())).hexdigest() + + def _collected_sources_list(self): + if self._use_geoip or self._sources_list: + release = platform.linux_distribution()[2] + return _format_sources_list( + self._sources_list, deb_arch=self._deb_arch, + use_geoip=self._use_geoip, release=release) + + return _get_local_sources_list() + + +class Ubuntu(BaseRepo): + + @classmethod + def get_package_libraries(cls, package_name): + global _library_list + if package_name not in _library_list: + output = subprocess.check_output( + ['dpkg', '-L', package_name]).decode( + sys.getfilesystemencoding()).strip().split() + _library_list[package_name] = {i for i in output if 'lib' in i} + + return _library_list[package_name].copy() + + @classmethod + def get_packages_for_source_type(cls, source_type): + if source_type == 'bzr': + packages = 'bzr' + elif source_type == 'git': + packages = 'git' + elif source_type == 'tar': + packages = 'tar' + elif source_type == 'hg' or source_type == 'mercurial': + packages = 'mercurial' + elif source_type == 'subversion' or source_type == 'svn': + packages = 'subversion' + else: + packages = [] + + return packages + + @classmethod + def install_build_packages(cls, package_names): + unique_packages = set(package_names) + new_packages = [] + with apt.Cache() as apt_cache: + for pkg in unique_packages: + try: + pkg_name, version = _get_pkg_name_parts(pkg) + installed_version = apt_cache[pkg_name].installed + if not installed_version: + new_packages.append(pkg) + elif version and installed_version != version: + new_packages.append(pkg) + except KeyError as e: + raise errors.BuildPackageNotFoundError(e) from e + if new_packages: + new_packages.sort() + logger.info( + 'Installing build dependencies: %s', ' '.join(new_packages)) + env = os.environ.copy() + env.update({ + 'DEBIAN_FRONTEND': 'noninteractive', + 'DEBCONF_NONINTERACTIVE_SEEN': 'true', + }) + + apt_command = ['sudo', 'apt-get', + '--no-install-recommends', '-y'] + if not is_dumb_terminal(): + apt_command.extend(['-o', 'Dpkg::Progress-Fancy=1']) + apt_command.append('install') + + subprocess.check_call(apt_command + new_packages, env=env) + + try: + subprocess.check_call(['sudo', 'apt-mark', 'auto'] + + new_packages, env=env) + except subprocess.CalledProcessError as e: + logger.warning( + 'Impossible to mark packages as auto-installed: {}' + .format(e)) + + @classmethod + def is_package_installed(cls, package_name): + with apt.Cache() as apt_cache: + return apt_cache[package_name].installed + + def __init__(self, rootdir, sources=None, project_options=None): + super().__init__(rootdir) + self._downloaddir = os.path.join(rootdir, 'download') + os.makedirs(self._downloaddir, exist_ok=True) + + if not project_options: + project_options = snapcraft.ProjectOptions() + + self._apt = _AptCache( + project_options.deb_arch, sources_list=sources, + use_geoip=project_options.use_geoip) + + self._cache = cache.AptStagePackageCache( + sources_digest=self._apt.sources_digest()) + + def is_valid(self, package_name): + with self._apt.archive(self._cache.base_dir) as apt_cache: + return package_name in apt_cache + + def get(self, package_names): + with self._apt.archive(self._cache.base_dir) as apt_cache: + self._mark_install(apt_cache, package_names) + self._filter_base_packages(apt_cache, package_names) + return self._get(apt_cache) + + def _mark_install(self, apt_cache, package_names): + for name in package_names: + logger.debug('Marking {!r} (and its dependencies) to be ' + 'fetched'.format(name)) + name_arch, version = _get_pkg_name_parts(name) + try: + if version: + _set_pkg_version(apt_cache[name_arch], version) + apt_cache[name_arch].mark_install() + except KeyError: + raise errors.PackageNotFoundError(name) + + def _filter_base_packages(self, apt_cache, package_names): + manifest_dep_names = self._manifest_dep_names(apt_cache) + + skipped_essential = [] + skipped_blacklisted = [] + + # unmark some base packages here + # note that this will break the consistency check inside apt_cache + # (apt_cache.broken_count will be > 0) + # but that is ok as it was consistent before we excluded + # these base package + for pkg in apt_cache: + # those should be already on each system, it also prevents + # diving into downloading libc6 + if (pkg.candidate.priority in 'essential' and + pkg.name not in package_names): + skipped_essential.append(pkg.name) + pkg.mark_keep() + continue + if (pkg.name in manifest_dep_names and + pkg.name not in package_names): + skipped_blacklisted.append(pkg.name) + pkg.mark_keep() + continue + + if skipped_essential: + logger.debug('Skipping priority essential packages: ' + '{!r}'.format(skipped_essential)) + if skipped_blacklisted: + logger.debug('Skipping blacklisted from manifest packages: ' + '{!r}'.format(skipped_blacklisted)) + + def _get(self, apt_cache): + # Ideally we'd use apt.Cache().fetch_archives() here, but it seems to + # mangle some package names on disk such that we can't match it up to + # the archive later. We could get around this a few different ways: + # + # 1. Store each stage package in the cache named by a hash instead of + # its name from the archive. + # 2. Download packages in a different manner. + # + # In the end, (2) was chosen for minimal overhead and a simpler cache + # implementation. So we're using fetch_binary() here instead. + # Downloading each package individually has the drawback of witholding + # any clue of how long the whole pulling process will take, but that's + # something we'll have to live with. + pkg_list = [] + for package in apt_cache.get_changes(): + pkg_list.append(str(package.candidate)) + source = package.candidate.fetch_binary( + self._cache.packages_dir, progress=self._apt.progress) + destination = os.path.join( + self._downloaddir, os.path.basename(source)) + with contextlib.suppress(FileNotFoundError): + os.remove(destination) + file_utils.link_or_copy(source, destination) + + return pkg_list + + def unpack(self, unpackdir): + pkgs_abs_path = glob.glob(os.path.join(self._downloaddir, '*.deb')) + for pkg in pkgs_abs_path: + # TODO needs elegance and error control + try: + subprocess.check_call( + ['dpkg-deb', '--extract', pkg, unpackdir]) + except subprocess.CalledProcessError: + raise errors.UnpackError(pkg) + self.normalize(unpackdir) + + def _manifest_dep_names(self, apt_cache): + manifest_dep_names = set() + + with open(os.path.abspath(os.path.join(__file__, '..', + 'manifest.txt'))) as f: + for line in f: + pkg = line.strip() + if pkg in apt_cache: + manifest_dep_names.add(pkg) + + return manifest_dep_names + + +def _get_local_sources_list(): + sources_list = glob.glob('/etc/apt/sources.list.d/*.list') + sources_list.append('/etc/apt/sources.list') + + sources = '' + for source in sources_list: + with open(source) as f: + sources += f.read() + + return sources + + +def _get_geoip_country_code_prefix(): + try: + with urllib.request.urlopen(_GEOIP_SERVER) as f: + xml_data = f.read() + et = ElementTree.fromstring(xml_data) + cc = et.find("CountryCode") + if cc is None: + return "" + return cc.text.lower() + except (ElementTree.ParseError, urllib.error.URLError): + pass + return '' + + +def _format_sources_list(sources_list, *, + deb_arch, use_geoip=False, release='xenial'): + if not sources_list: + sources_list = _DEFAULT_SOURCES + + if deb_arch in ('amd64', 'i386'): + if use_geoip: + geoip_prefix = _get_geoip_country_code_prefix() + prefix = '{}.archive'.format(geoip_prefix) + else: + prefix = 'archive' + suffix = 'ubuntu' + security = 'security' + else: + prefix = 'ports' + suffix = 'ubuntu-ports' + security = 'ports' + + return string.Template(sources_list).substitute({ + 'prefix': prefix, + 'release': release, + 'suffix': suffix, + 'security': security, + }) + + +def _fix_filemode(path): + mode = stat.S_IMODE(os.stat(path, follow_symlinks=False).st_mode) + if mode & 0o4000 or mode & 0o2000: + logger.warning('Removing suid/guid from {}'.format(path)) + os.chmod(path, mode & 0o1777) + + +def _try_copy_local(path, target): + real_path = os.path.realpath(path) + if os.path.exists(real_path): + logger.warning( + 'Copying needed target link from the system {}'.format(real_path)) + os.makedirs(os.path.dirname(target), exist_ok=True) + shutil.copyfile(os.readlink(path), target) + return True + else: + logger.warning( + '{} will be a dangling symlink'.format(path)) + return False + + +def check_for_command(command): + if not shutil.which(command): + raise errors.MissingCommandError([command]) + + +def _get_pkg_name_parts(pkg_name): + """Break package name into base parts""" + + name = pkg_name + version = None + with contextlib.suppress(ValueError): + name, version = pkg_name.split('=') + + return name, version + + +def _set_pkg_version(pkg, version): + """Set cadidate version to a specific version if available""" + if version in pkg.versions: + version = pkg.versions.get(version) + pkg.candidate = version + else: + raise errors.PackageNotFoundError('{}={}'.format(pkg.name, version)) diff -Nru snapcraft-2.27.1+17.04/snapcraft/internal/repo/errors.py snapcraft-2.28+17.04/snapcraft/internal/repo/errors.py --- snapcraft-2.27.1+17.04/snapcraft/internal/repo/errors.py 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/internal/repo/errors.py 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,56 @@ +# -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*- +# +# Copyright (C) 2015-2017 Canonical Ltd +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License version 3 as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . + +from ._platform import _is_deb_based + + +class BuildPackageNotFoundError(Exception): + + def __init__(self, package_name): + self.package_name = package_name + + def __str__(self): + return ('Could not find a required package in ' + '\'build-packages\': {}'.format(str(self.package_name))) + + +class PackageNotFoundError(Exception): + + @property + def message(self): + message = 'The package {!r} was not found.'.format( + self.package_name) + # If the package was multiarch, try to help. + if _is_deb_based() and ':' in self.package_name: + (name, arch) = self.package_name.split(':', 2) + if arch: + message += ( + '\nYou may need to add support for this architecture with ' + "'dpkg --add-architecture {}'.".format(arch)) + return message + + def __init__(self, package_name): + self.package_name = package_name + + +class UnpackError(Exception): + + @property + def message(self): + return 'Error while provisioning "{}"'.format(self.package_name) + + def __init__(self, package_name): + self.package_name = package_name diff -Nru snapcraft-2.27.1+17.04/snapcraft/internal/repo/__init__.py snapcraft-2.28+17.04/snapcraft/internal/repo/__init__.py --- snapcraft-2.27.1+17.04/snapcraft/internal/repo/__init__.py 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/internal/repo/__init__.py 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,31 @@ +# -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*- +# +# Copyright (C) 2017 Canonical Ltd +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License version 3 as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +import shutil + +from snapcraft.internal.errors import MissingCommandError +from . import errors # noqa +from ._base import BaseRepo # noqa +from ._base import fix_pkg_config # noqa +from ._platform import _get_repo_for_platform +# Imported for backwards compatibility with plugins +from ._deb import Ubuntu # noqa + +Repo = _get_repo_for_platform() + + +def check_for_command(command): + if not shutil.which(command): + raise MissingCommandError([command]) diff -Nru snapcraft-2.27.1+17.04/snapcraft/internal/repo/manifest.txt snapcraft-2.28+17.04/snapcraft/internal/repo/manifest.txt --- snapcraft-2.27.1+17.04/snapcraft/internal/repo/manifest.txt 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/internal/repo/manifest.txt 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,110 @@ +adduser +apt +apt-utils +base-files +base-passwd +bash +bsdutils +coreutils +dash +debconf +debconf-i18n +debianutils +diffutils +dmsetup +dpkg +e2fslibs +e2fsprogs +file +findutils +gcc-4.9-base +gcc-5-base +gnupg +gpgv +grep +gzip +hostname +init +initscripts +insserv +libacl1 +libapparmor1 +libapt +libapt-inst1.5 +libapt-pkg4.12 +libattr1 +libaudit-common +libaudit1 +libblkid1 +libbz2-1.0 +libc-bin +libc6 +libcap2 +libcap2-bin +libcomerr2 +libcryptsetup4 +libdb5.3 +libdebconfclient0 +libdevmapper1.02.1 +libgcc1 +libgcrypt20 +libgpg-error0 +libgpm2 +libkmod2 +liblocale-gettext-perl +liblzma5 +libmagic1 +libmount1 +libncurses5 +libncursesw5 +libpam-modules +libpam-modules-bin +libpam-runtime +libpam0g +libpcre3 +libprocps3 +libreadline6 +libselinux1 +libsemanage-common +libsemanage1 +libsepol1 +libslang2 +libsmartcols1 +libss2 +libstdc++6 +libsystemd0 +libtext-charwidth-perl +libtext-iconv-perl +libtext-wrapi18n-perl +libtinfo5 +libudev1 +libusb-0.1-4 +libustr-1.0-1 +libuuid1 +locales +login +lsb-base +makedev +manpages +manpages-dev +mawk +mount +multiarch-support +ncurses-base +ncurses-bin +passwd +perl-base +procps +readline-common +sed +sensible-utils +systemd +systemd-sysv +sysv-rc +sysvinit-utils +tar +tzdata +ubuntu-keyring +udev +util-linux +zlib1g diff -Nru snapcraft-2.27.1+17.04/snapcraft/internal/repo/_platform.py snapcraft-2.28+17.04/snapcraft/internal/repo/_platform.py --- snapcraft-2.27.1+17.04/snapcraft/internal/repo/_platform.py 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/internal/repo/_platform.py 2017-03-23 18:31:04.000000000 +0000 @@ -0,0 +1,37 @@ +# -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*- +# +# Copyright (C) 2017 Canonical Ltd +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License version 3 as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . + +import platform + +from ._deb import Ubuntu + +_DEB_BASED_PLATFORM = [ + 'Ubuntu', + 'Debian', + 'debian', +] + + +def _is_deb_based(): + return platform.linux_distribution()[0] in _DEB_BASED_PLATFORM + + +def _get_repo_for_platform(): + if _is_deb_based(): + return Ubuntu + else: + raise RuntimeError( + 'snapcraft is not supported on this operating system') diff -Nru snapcraft-2.27.1+17.04/snapcraft/internal/repo.py snapcraft-2.28+17.04/snapcraft/internal/repo.py --- snapcraft-2.27.1+17.04/snapcraft/internal/repo.py 2017-02-17 19:54:46.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/internal/repo.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,529 +0,0 @@ -# -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*- -# -# Copyright (C) 2015-2016 Canonical Ltd -# -# This program is free software: you can redistribute it and/or modify -# it under the terms of the GNU General Public License version 3 as -# published by the Free Software Foundation. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with this program. If not, see . - -import contextlib -import fileinput -import glob -import hashlib -import itertools -import logging -import os -import platform -import re -import shutil -import stat -import string -import subprocess -import sys -import urllib -import urllib.request - -import apt -from xml.etree import ElementTree - -import snapcraft -from snapcraft import file_utils -from snapcraft.internal import ( - cache, - common, -) -from snapcraft.internal.errors import MissingCommandError -from snapcraft.internal.indicators import is_dumb_terminal - - -_BIN_PATHS = ( - 'bin', - 'sbin', - 'usr/bin', - 'usr/sbin', -) - -logger = logging.getLogger(__name__) - -_DEFAULT_SOURCES = \ - '''deb http://${prefix}.ubuntu.com/${suffix}/ ${release} main restricted -deb http://${prefix}.ubuntu.com/${suffix}/ ${release}-updates main restricted -deb http://${prefix}.ubuntu.com/${suffix}/ ${release} universe -deb http://${prefix}.ubuntu.com/${suffix}/ ${release}-updates universe -deb http://${prefix}.ubuntu.com/${suffix}/ ${release} multiverse -deb http://${prefix}.ubuntu.com/${suffix}/ ${release}-updates multiverse -deb http://${security}.ubuntu.com/${suffix} ${release}-security main restricted -deb http://${security}.ubuntu.com/${suffix} ${release}-security universe -deb http://${security}.ubuntu.com/${suffix} ${release}-security multiverse -''' -_GEOIP_SERVER = "http://geoip.ubuntu.com/lookup" - - -def is_package_installed(package): - """Return True if a package is installed on the system. - - :param str package: the deb package to query for. - :returns: True if the package is installed, False if not. - """ - with apt.Cache() as apt_cache: - return apt_cache[package].installed - - -def install_build_packages(packages): - unique_packages = set(packages) - new_packages = [] - with apt.Cache() as apt_cache: - for pkg in unique_packages: - try: - if not apt_cache[pkg].installed: - new_packages.append(pkg) - except KeyError as e: - raise EnvironmentError( - 'Could not find a required package in ' - '\'build-packages\': {}'.format(str(e))) - if new_packages: - new_packages.sort() - logger.info( - 'Installing build dependencies: %s', ' '.join(new_packages)) - env = os.environ.copy() - env.update({ - 'DEBIAN_FRONTEND': 'noninteractive', - 'DEBCONF_NONINTERACTIVE_SEEN': 'true', - }) - - apt_command = ['sudo', 'apt-get', - '--no-install-recommends', '-y'] - if not is_dumb_terminal(): - apt_command.extend(['-o', 'Dpkg::Progress-Fancy=1']) - apt_command.append('install') - - subprocess.check_call(apt_command + new_packages, env=env) - - try: - subprocess.check_call(['sudo', 'apt-mark', 'auto'] + - new_packages, env=env) - except subprocess.CalledProcessError as e: - logger.warning( - 'Impossible to mark packages as auto-installed: {}' - .format(e)) - - -def get_packages_for_source_type(source_type): - """Return a list with required packages to handle the source_type. - - :param source: the snapcraft source type - """ - if source_type == 'bzr': - packages = 'bzr' - elif source_type == 'git': - packages = 'git' - elif source_type == 'tar': - packages = 'tar' - elif source_type == 'hg' or source_type == 'mercurial': - packages = 'mercurial' - elif source_type == 'subversion' or source_type == 'svn': - packages = 'subversion' - else: - packages = [] - - return packages - - -class PackageNotFoundError(Exception): - - @property - def message(self): - message = 'The Ubuntu package {!r} was not found.'.format( - self.package_name) - # If the package was multiarch, try to help. - if ':' in self.package_name: - (name, arch) = self.package_name.split(':', 2) - if arch: - message += ( - '\nYou may need to add support for this architecture with ' - "'dpkg --add-architecture {}'.".format(arch)) - return message - - def __init__(self, package_name): - self.package_name = package_name - - -class UnpackError(Exception): - - @property - def message(self): - return 'Error while provisioning "{}"'.format(self.package_name) - - def __init__(self, package_name): - self.package_name = package_name - - -class _AptCache: - - def __init__(self, deb_arch, *, sources_list=None, use_geoip=False): - self._deb_arch = deb_arch - self._sources_list = sources_list - self._use_geoip = use_geoip - - def _setup_apt(self, cache_dir): - # Do not install recommends - apt.apt_pkg.config.set('Apt::Install-Recommends', 'False') - - # Make sure we always use the system GPG configuration, even with - # apt.Cache(rootdir). - for key in 'Dir::Etc::Trusted', 'Dir::Etc::TrustedParts': - apt.apt_pkg.config.set(key, apt.apt_pkg.config.find_file(key)) - - # Clear up apt's Post-Invoke-Success as we are not running - # on the system. - apt.apt_pkg.config.clear('APT::Update::Post-Invoke-Success') - - self.progress = apt.progress.text.AcquireProgress() - if is_dumb_terminal(): - # Make output more suitable for logging. - self.progress.pulse = lambda owner: True - self.progress._width = 0 - - sources_list_file = os.path.join( - cache_dir, 'etc', 'apt', 'sources.list') - - os.makedirs(os.path.dirname(sources_list_file), exist_ok=True) - with open(sources_list_file, 'w') as f: - f.write(self._collected_sources_list()) - - # dpkg also needs to be in the rootdir in order to support multiarch - # (apt calls dpkg --print-foreign-architectures). - dpkg_path = shutil.which('dpkg') - if dpkg_path: - # Symlink it into place - destination = os.path.join(cache_dir, dpkg_path[1:]) - if not os.path.exists(destination): - os.makedirs(os.path.dirname(destination), exist_ok=True) - os.symlink(dpkg_path, destination) - else: - logger.warning( - "Cannot find 'dpkg' command needed to support multiarch") - - apt_cache = apt.Cache(rootdir=cache_dir, memonly=True) - apt_cache.update(fetch_progress=self.progress, - sources_list=sources_list_file) - - return apt_cache - - @contextlib.contextmanager - def archive(self, cache_dir): - try: - apt_cache = self._setup_apt(cache_dir) - apt_cache.open() - - try: - yield apt_cache - finally: - apt_cache.close() - except Exception as e: - logger.debug('Exception occured: {!r}'.format(e)) - raise e - - def sources_digest(self): - return hashlib.sha384(self._collected_sources_list().encode( - sys.getfilesystemencoding())).hexdigest() - - def _collected_sources_list(self): - if self._use_geoip or self._sources_list: - release = platform.linux_distribution()[2] - return _format_sources_list( - self._sources_list, deb_arch=self._deb_arch, - use_geoip=self._use_geoip, release=release) - - return _get_local_sources_list() - - -class Ubuntu: - - def __init__(self, rootdir, recommends=False, sources=None, - project_options=None): - self._downloaddir = os.path.join(rootdir, 'download') - self._rootdir = rootdir - os.makedirs(self._downloaddir, exist_ok=True) - - if not project_options: - project_options = snapcraft.ProjectOptions() - - self._apt = _AptCache( - project_options.deb_arch, sources_list=sources, - use_geoip=project_options.use_geoip) - - self._cache = cache.AptStagePackageCache( - sources_digest=self._apt.sources_digest()) - - def is_valid(self, package_name): - with self._apt.archive(self._cache.base_dir) as apt_cache: - return package_name in apt_cache - - def get(self, package_names): - with self._apt.archive(self._cache.base_dir) as apt_cache: - self._get(apt_cache, package_names) - - def _get(self, apt_cache, package_names): - manifest_dep_names = self._manifest_dep_names(apt_cache) - - for name in package_names: - try: - logger.debug( - 'Marking {!r} (and its dependencies) to be fetched'.format( - name)) - apt_cache[name].mark_install() - except KeyError: - raise PackageNotFoundError(name) - - skipped_essential = [] - skipped_blacklisted = [] - - # unmark some base packages here - # note that this will break the consistency check inside apt_cache - # (apt_cache.broken_count will be > 0) - # but that is ok as it was consistent before we excluded - # these base package - for pkg in apt_cache: - # those should be already on each system, it also prevents - # diving into downloading libc6 - if (pkg.candidate.priority in 'essential' and - pkg.name not in package_names): - skipped_essential.append(pkg.name) - pkg.mark_keep() - continue - if (pkg.name in manifest_dep_names and - pkg.name not in package_names): - skipped_blacklisted.append(pkg.name) - pkg.mark_keep() - continue - - if skipped_essential: - logger.debug('Skipping priority essential packages: ' - '{!r}'.format(skipped_essential)) - if skipped_blacklisted: - logger.debug('Skipping blacklisted from manifest packages: ' - '{!r}'.format(skipped_blacklisted)) - - # Ideally we'd use apt.Cache().fetch_archives() here, but it seems to - # mangle some package names on disk such that we can't match it up to - # the archive later. We could get around this a few different ways: - # - # 1. Store each stage package in the cache named by a hash instead of - # its name from the archive. - # 2. Download packages in a different manner. - # - # In the end, (2) was chosen for minimal overhead and a simpler cache - # implementation. So we're using fetch_binary() here instead. - # Downloading each package individually has the drawback of witholding - # any clue of how long the whole pulling process will take, but that's - # something we'll have to live with. - for package in apt_cache.get_changes(): - source = package.candidate.fetch_binary( - self._cache.packages_dir, progress=self._apt.progress) - destination = os.path.join( - self._downloaddir, os.path.basename(source)) - with contextlib.suppress(FileNotFoundError): - os.remove(destination) - file_utils.link_or_copy(source, destination) - - def unpack(self, rootdir): - pkgs_abs_path = glob.glob(os.path.join(self._downloaddir, '*.deb')) - for pkg in pkgs_abs_path: - # TODO needs elegance and error control - try: - subprocess.check_call(['dpkg-deb', '--extract', pkg, rootdir]) - except subprocess.CalledProcessError: - raise UnpackError(pkg) - - _fix_artifacts(rootdir) - _fix_xml_tools(rootdir) - _fix_shebangs(rootdir) - - def _manifest_dep_names(self, apt_cache): - manifest_dep_names = set() - - with open(os.path.abspath(os.path.join(__file__, '..', - 'manifest.txt'))) as f: - for line in f: - pkg = line.strip() - if pkg in apt_cache: - manifest_dep_names.add(pkg) - - return manifest_dep_names - - -def _get_local_sources_list(): - sources_list = glob.glob('/etc/apt/sources.list.d/*.list') - sources_list.append('/etc/apt/sources.list') - - sources = '' - for source in sources_list: - with open(source) as f: - sources += f.read() - - return sources - - -def _get_geoip_country_code_prefix(): - try: - with urllib.request.urlopen(_GEOIP_SERVER) as f: - xml_data = f.read() - et = ElementTree.fromstring(xml_data) - cc = et.find("CountryCode") - if cc is None: - return "" - return cc.text.lower() - except (ElementTree.ParseError, urllib.error.URLError): - pass - return '' - - -def _format_sources_list(sources_list, *, - deb_arch, use_geoip=False, release='xenial'): - if not sources_list: - sources_list = _DEFAULT_SOURCES - - if deb_arch in ('amd64', 'i386'): - if use_geoip: - geoip_prefix = _get_geoip_country_code_prefix() - prefix = '{}.archive'.format(geoip_prefix) - else: - prefix = 'archive' - suffix = 'ubuntu' - security = 'security' - else: - prefix = 'ports' - suffix = 'ubuntu-ports' - security = 'ports' - - return string.Template(sources_list).substitute({ - 'prefix': prefix, - 'release': release, - 'suffix': suffix, - 'security': security, - }) - - -def fix_pkg_config(root, pkg_config_file, prefix_trim=None): - """Opens a pkg_config_file and prefixes the prefix with root.""" - pattern_trim = None - if prefix_trim: - pattern_trim = re.compile( - '^prefix={}(?P.*)'.format(prefix_trim)) - pattern = re.compile('^prefix=(?P.*)') - - with fileinput.input(pkg_config_file, inplace=True) as input_file: - for line in input_file: - match = pattern.search(line) - if prefix_trim: - match_trim = pattern_trim.search(line) - if prefix_trim and match_trim: - print('prefix={}{}'.format(root, match_trim.group('prefix'))) - elif match: - print('prefix={}{}'.format(root, match.group('prefix'))) - else: - print(line, end='') - - -def _fix_artifacts(debdir): - ''' - Sometimes debs will contain absolute symlinks (e.g. if the relative - path would go all the way to root, they just do absolute). We can't - have that, so instead clean those absolute symlinks. - - Some unpacked items will also contain suid binaries which we do not want in - the resulting snap. - ''' - for root, dirs, files in os.walk(debdir): - # Symlinks to directories will be in dirs, while symlinks to - # non-directories will be in files. - for entry in itertools.chain(files, dirs): - path = os.path.join(root, entry) - if os.path.islink(path) and os.path.isabs(os.readlink(path)): - _fix_symlink(path, debdir, root) - elif os.path.exists(path): - _fix_filemode(path) - - if path.endswith('.pc') and not os.path.islink(path): - fix_pkg_config(debdir, path) - - -def _fix_xml_tools(root): - xml2_config_path = os.path.join(root, 'usr', 'bin', 'xml2-config') - if os.path.isfile(xml2_config_path): - common.run( - ['sed', '-i', '-e', 's|prefix=/usr|prefix={}/usr|'. - format(root), xml2_config_path]) - - xslt_config_path = os.path.join(root, 'usr', 'bin', 'xslt-config') - if os.path.isfile(xslt_config_path): - common.run( - ['sed', '-i', '-e', 's|prefix=/usr|prefix={}/usr|'. - format(root), xslt_config_path]) - - -def _fix_symlink(path, debdir, root): - target = os.path.join(debdir, os.readlink(path)[1:]) - if _skip_link(os.readlink(path)): - logger.debug('Skipping {}'.format(target)) - return - if not os.path.exists(target) and not _try_copy_local(path, target): - return - os.remove(path) - os.symlink(os.path.relpath(target, root), path) - - -def _fix_filemode(path): - mode = stat.S_IMODE(os.stat(path, follow_symlinks=False).st_mode) - if mode & 0o4000 or mode & 0o2000: - logger.warning('Removing suid/guid from {}'.format(path)) - os.chmod(path, mode & 0o1777) - - -def _fix_shebangs(path): - """Changes hard coded shebangs for files in _BIN_PATHS to use env.""" - paths = [p for p in _BIN_PATHS if os.path.exists(os.path.join(path, p))] - for p in [os.path.join(path, p) for p in paths]: - file_utils.replace_in_file(p, re.compile(r''), - re.compile(r'#!.*python\n'), - r'#!/usr/bin/env python\n') - - -_skip_list = None - - -def _skip_link(target): - global _skip_list - if not _skip_list: - output = common.run_output(['dpkg', '-L', 'libc6']).split() - _skip_list = [i for i in output if 'lib' in i] - - return target in _skip_list - - -def _try_copy_local(path, target): - real_path = os.path.realpath(path) - if os.path.exists(real_path): - logger.warning( - 'Copying needed target link from the system {}'.format(real_path)) - os.makedirs(os.path.dirname(target), exist_ok=True) - shutil.copyfile(os.readlink(path), target) - return True - else: - logger.warning( - '{} will be a dangling symlink'.format(path)) - return False - - -def check_for_command(command): - if not shutil.which(command): - raise MissingCommandError([command]) diff -Nru snapcraft-2.27.1+17.04/snapcraft/internal/sources/_base.py snapcraft-2.28+17.04/snapcraft/internal/sources/_base.py --- snapcraft-2.27.1+17.04/snapcraft/internal/sources/_base.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/internal/sources/_base.py 2017-03-22 12:31:58.000000000 +0000 @@ -1,6 +1,6 @@ # -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*- # -# Copyright (C) 2015-2016 Canonical Ltd +# Copyright (C) 2015-2017 Canonical Ltd # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License version 3 as @@ -29,13 +29,14 @@ def __init__(self, source, source_dir, source_tag=None, source_commit=None, source_branch=None, source_depth=None, - command=None): + source_checksum=None, command=None): self.source = source self.source_dir = source_dir self.source_tag = source_tag self.source_commit = source_commit self.source_branch = source_branch self.source_depth = source_depth + self.source_checksum = source_checksum self.command = command diff -Nru snapcraft-2.27.1+17.04/snapcraft/internal/sources/_bazaar.py snapcraft-2.28+17.04/snapcraft/internal/sources/_bazaar.py --- snapcraft-2.27.1+17.04/snapcraft/internal/sources/_bazaar.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/internal/sources/_bazaar.py 2017-03-22 12:31:58.000000000 +0000 @@ -1,6 +1,6 @@ # -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*- # -# Copyright (C) 2015-2016 Canonical Ltd +# Copyright (C) 2015-2017 Canonical Ltd # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License version 3 as @@ -24,9 +24,9 @@ class Bazaar(Base): def __init__(self, source, source_dir, source_tag=None, source_commit=None, - source_branch=None, source_depth=None): + source_branch=None, source_depth=None, source_checksum=None): super().__init__(source, source_dir, source_tag, source_commit, - source_branch, source_depth, 'bzr') + source_branch, source_depth, source_checksum, 'bzr') if source_branch: raise errors.IncompatibleOptionsError( 'can\'t specify a source-branch for a bzr source') @@ -37,6 +37,9 @@ raise errors.IncompatibleOptionsError( 'can\'t specify both source-tag and source-commit for ' 'a bzr source') + if source_checksum: + raise errors.IncompatibleOptionsError( + "can't specify a source-checksum for a bzr source") def pull(self): tag_opts = [] diff -Nru snapcraft-2.27.1+17.04/snapcraft/internal/sources/_deb.py snapcraft-2.28+17.04/snapcraft/internal/sources/_deb.py --- snapcraft-2.27.1+17.04/snapcraft/internal/sources/_deb.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/internal/sources/_deb.py 2017-03-22 12:31:58.000000000 +0000 @@ -1,6 +1,6 @@ # -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*- # -# Copyright (C) 2015-2016 Canonical Ltd +# Copyright (C) 2015-2017 Canonical Ltd # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License version 3 as @@ -21,14 +21,15 @@ from . import errors from ._base import FileBase +from snapcraft.internal import sources class Deb(FileBase): def __init__(self, source, source_dir, source_tag=None, source_commit=None, - source_branch=None, source_depth=None): + source_branch=None, source_depth=None, source_checksum=None): super().__init__(source, source_dir, source_tag, source_commit, - source_branch, source_depth) + source_branch, source_depth, source_checksum) if source_tag: raise errors.IncompatibleOptionsError( 'can\'t specify a source-tag for a deb source') @@ -42,6 +43,9 @@ def provision(self, dst, clean_target=True, keep_deb=False): deb_file = os.path.join(self.source_dir, os.path.basename(self.source)) + if self.source_checksum: + sources.verify_checksum(self.source_checksum, deb_file) + if clean_target: tmp_deb = tempfile.NamedTemporaryFile().name shutil.move(deb_file, tmp_deb) diff -Nru snapcraft-2.27.1+17.04/snapcraft/internal/sources/errors.py snapcraft-2.28+17.04/snapcraft/internal/sources/errors.py --- snapcraft-2.27.1+17.04/snapcraft/internal/sources/errors.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/internal/sources/errors.py 2017-03-22 12:31:58.000000000 +0000 @@ -1,6 +1,6 @@ # -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*- # -# Copyright (C) 2015-2016 Canonical Ltd +# Copyright (C) 2015-2017 Canonical Ltd # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License version 3 as @@ -23,3 +23,12 @@ def __init__(self, message): super().__init__(message=message) + + +class DigestDoesNotMatchError(errors.SnapcraftError): + + fmt = 'Expected the digest for source to be {expected}, ' + 'but it was {calculated}' + + def __init__(self, expected, calculated): + super().__init__(expected=expected, calculated=calculated) diff -Nru snapcraft-2.27.1+17.04/snapcraft/internal/sources/_git.py snapcraft-2.28+17.04/snapcraft/internal/sources/_git.py --- snapcraft-2.27.1+17.04/snapcraft/internal/sources/_git.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/internal/sources/_git.py 2017-03-22 12:31:58.000000000 +0000 @@ -1,6 +1,6 @@ # -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*- # -# Copyright (C) 2015-2016 Canonical Ltd +# Copyright (C) 2015-2017 Canonical Ltd # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License version 3 as @@ -24,9 +24,10 @@ class Git(Base): def __init__(self, source, source_dir, source_tag=None, source_commit=None, - source_branch=None, source_depth=None, silent=False): + source_branch=None, source_depth=None, silent=False, + source_checksum=None): super().__init__(source, source_dir, source_tag, source_commit, - source_branch, source_depth, 'git') + source_branch, source_depth, source_checksum, 'git') if source_tag and source_branch: raise errors.IncompatibleOptionsError( 'can\'t specify both source-tag and source-branch for ' @@ -39,6 +40,9 @@ raise errors.IncompatibleOptionsError( 'can\'t specify both source-branch and source-commit for ' 'a git source') + if source_checksum: + raise errors.IncompatibleOptionsError( + "can't specify a source-checksum for a git source") self.kwargs = {} if silent: self.kwargs['stdout'] = subprocess.DEVNULL diff -Nru snapcraft-2.27.1+17.04/snapcraft/internal/sources/__init__.py snapcraft-2.28+17.04/snapcraft/internal/sources/__init__.py --- snapcraft-2.27.1+17.04/snapcraft/internal/sources/__init__.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/internal/sources/__init__.py 2017-03-22 12:31:58.000000000 +0000 @@ -1,6 +1,6 @@ # -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*- # -# Copyright (C) 2015-2016 Canonical Ltd +# Copyright (C) 2015-2017 Canonical Ltd # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License version 3 as @@ -33,6 +33,13 @@ control system or compression algorithm. The source-type key can tell snapcraft exactly how to treat that content. + - source-checksum: / + + Snapcraft will use the digest specified to verify the integrity of the + source. The source-type needs to be a file (tar, zip, deb or rpm) and + the algorithm either md5, sha1, sha224, sha256, sha384, sha512, sha3_256, + sha3_384 or sha3_512. + - source-depth: By default clones or branches with full history, specifying a depth @@ -55,9 +62,8 @@ - source-subdir: path - Snapcraft will checkout the repository or unpack the archive referred to - by the 'source' keyword into parts//src/ but it will only - copy the specified subdirectory into parts//build/ + When building, Snapcraft will set the working directory to be this + subdirectory within the source. Note that plugins might well define their own semantics for the 'source' keywords, because they handle specific build systems, and many languages @@ -72,6 +78,8 @@ import os import os.path import re +import hashlib +import sys from snapcraft.internal import common from ._bazaar import Bazaar # noqa @@ -84,6 +92,12 @@ from ._subversion import Subversion # noqa from ._tar import Tar # noqa from ._zip import Zip # noqa +from . import errors + +# In python >= 3.6 sha3 support is upstreamed in hashlib +if sys.version_info < (3, 6): + import sha3 # noqa + logging.getLogger('urllib3').setLevel(logging.CRITICAL) @@ -91,6 +105,7 @@ __SOURCE_DEFAULTS = { 'source': '.', 'source-commit': None, + 'source-checksum': None, 'source-depth': None, 'source-tag': None, 'source-type': None, @@ -113,6 +128,7 @@ source_type = getattr(options, 'source_type', None) source_attributes = dict( source_depth=getattr(options, 'source_depth', None), + source_checksum=getattr(options, 'source_checksum', None), source_tag=getattr(options, 'source_tag', None), source_commit=getattr(options, 'source_commit', None), source_branch=getattr(options, 'source_branch', None), @@ -170,3 +186,21 @@ raise ValueError('local source ({}) is not a directory'.format(source)) return source_type + + +def verify_checksum(source_checksum, checkfile): + try: + algorithm, digest = source_checksum.split('/', 1) + + except ValueError: + raise ValueError('invalid checksum format: {!r}' + .format(source_checksum)) + + with open(checkfile, 'rb') as f: + # This will raise an AttributeError if algorithm is unsupported + hashlib_algorithm = getattr(hashlib, algorithm) + calculated_digest = hashlib_algorithm(f.read()) + + calculated_digest = calculated_digest.hexdigest() + if digest != calculated_digest: + raise errors.DigestDoesNotMatchError(digest, calculated_digest) diff -Nru snapcraft-2.27.1+17.04/snapcraft/internal/sources/_mercurial.py snapcraft-2.28+17.04/snapcraft/internal/sources/_mercurial.py --- snapcraft-2.27.1+17.04/snapcraft/internal/sources/_mercurial.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/internal/sources/_mercurial.py 2017-03-22 12:31:58.000000000 +0000 @@ -1,6 +1,6 @@ # -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*- # -# Copyright (C) 2015-2016 Canonical Ltd +# Copyright (C) 2015-2017 Canonical Ltd # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License version 3 as @@ -24,9 +24,9 @@ class Mercurial(Base): def __init__(self, source, source_dir, source_tag=None, source_commit=None, - source_branch=None, source_depth=None): + source_branch=None, source_depth=None, source_checksum=None): super().__init__(source, source_dir, source_tag, source_commit, - source_branch, source_depth, 'hg') + source_branch, source_depth, source_checksum, 'hg') if source_tag and source_branch: raise errors.IncompatibleOptionsError( 'can\'t specify both source-tag and source-branch for a ' @@ -42,6 +42,9 @@ if source_depth: raise errors.IncompatibleOptionsError( 'can\'t specify source-depth for a mercurial source') + if source_checksum: + raise errors.IncompatibleOptionsError( + "can't specify a source-checksum for a mercurial source") def pull(self): if os.path.exists(os.path.join(self.source_dir, '.hg')): diff -Nru snapcraft-2.27.1+17.04/snapcraft/internal/sources/_rpm.py snapcraft-2.28+17.04/snapcraft/internal/sources/_rpm.py --- snapcraft-2.27.1+17.04/snapcraft/internal/sources/_rpm.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/internal/sources/_rpm.py 2017-03-22 12:31:58.000000000 +0000 @@ -1,6 +1,6 @@ # -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*- # -# Copyright (C) 2016 Neal Gompa +# Copyright (C) 2016-2017 Neal Gompa # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License version 3 as @@ -21,14 +21,15 @@ from . import errors from ._base import FileBase +from snapcraft.internal import sources class Rpm(FileBase): def __init__(self, source, source_dir, source_tag=None, source_commit=None, - source_branch=None, source_depth=None): + source_branch=None, source_depth=None, source_checksum=None): super().__init__(source, source_dir, source_tag, source_commit, - source_branch, source_depth) + source_branch, source_depth, source_checksum) if source_tag: raise errors.IncompatibleOptionsError( 'can\'t specify a source-tag for a rpm source') @@ -42,6 +43,9 @@ def provision(self, dst, clean_target=True, keep_rpm=False): rpm_file = os.path.join(self.source_dir, os.path.basename(self.source)) + if self.source_checksum: + sources.verify_checksum(self.source_checksum, rpm_file) + if clean_target: tmp_rpm = tempfile.NamedTemporaryFile().name shutil.move(rpm_file, tmp_rpm) diff -Nru snapcraft-2.27.1+17.04/snapcraft/internal/sources/_subversion.py snapcraft-2.28+17.04/snapcraft/internal/sources/_subversion.py --- snapcraft-2.27.1+17.04/snapcraft/internal/sources/_subversion.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/internal/sources/_subversion.py 2017-03-22 12:31:58.000000000 +0000 @@ -1,6 +1,6 @@ # -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*- # -# Copyright (C) 2015-2016 Canonical Ltd +# Copyright (C) 2015-2017 Canonical Ltd # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License version 3 as @@ -24,9 +24,9 @@ class Subversion(Base): def __init__(self, source, source_dir, source_tag=None, source_commit=None, - source_branch=None, source_depth=None): + source_branch=None, source_depth=None, source_checksum=None): super().__init__(source, source_dir, source_tag, source_commit, - source_branch, source_depth, 'svn') + source_branch, source_depth, source_checksum, 'svn') if source_tag: if source_branch: raise errors.IncompatibleOptionsError( @@ -41,6 +41,9 @@ if source_depth: raise errors.IncompatibleOptionsError( 'can\'t specify source-depth for a Subversion source') + if source_checksum: + raise errors.IncompatibleOptionsError( + "can't specify a source-checksum for a Subversion source") def pull(self): opts = [] diff -Nru snapcraft-2.27.1+17.04/snapcraft/internal/sources/_tar.py snapcraft-2.28+17.04/snapcraft/internal/sources/_tar.py --- snapcraft-2.27.1+17.04/snapcraft/internal/sources/_tar.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/internal/sources/_tar.py 2017-03-22 12:31:58.000000000 +0000 @@ -1,6 +1,6 @@ # -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*- # -# Copyright (C) 2015-2016 Canonical Ltd +# Copyright (C) 2015-2017 Canonical Ltd # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License version 3 as @@ -22,14 +22,15 @@ from . import errors from ._base import FileBase +from snapcraft.internal import sources class Tar(FileBase): def __init__(self, source, source_dir, source_tag=None, source_commit=None, - source_branch=None, source_depth=None): + source_branch=None, source_depth=None, source_checksum=None): super().__init__(source, source_dir, source_tag, source_commit, - source_branch, source_depth) + source_branch, source_depth, source_checksum) if source_tag: raise errors.IncompatibleOptionsError( 'can\'t specify a source-tag for a tar source') @@ -47,6 +48,9 @@ # TODO add unit tests. tarball = os.path.join(self.source_dir, os.path.basename(self.source)) + if self.source_checksum: + sources.verify_checksum(self.source_checksum, tarball) + if clean_target: tmp_tarball = tempfile.NamedTemporaryFile().name shutil.move(tarball, tmp_tarball) diff -Nru snapcraft-2.27.1+17.04/snapcraft/internal/sources/_zip.py snapcraft-2.28+17.04/snapcraft/internal/sources/_zip.py --- snapcraft-2.27.1+17.04/snapcraft/internal/sources/_zip.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/internal/sources/_zip.py 2017-03-22 12:31:58.000000000 +0000 @@ -1,6 +1,6 @@ # -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*- # -# Copyright (C) 2015-2016 Canonical Ltd +# Copyright (C) 2015-2017 Canonical Ltd # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License version 3 as @@ -21,14 +21,15 @@ from . import errors from ._base import FileBase +from snapcraft.internal import sources class Zip(FileBase): def __init__(self, source, source_dir, source_tag=None, source_commit=None, - source_branch=None, source_depth=None): + source_branch=None, source_depth=None, source_checksum=None): super().__init__(source, source_dir, source_tag, source_commit, - source_branch, source_depth) + source_branch, source_depth, source_checksum) if source_tag: raise errors.IncompatibleOptionsError( 'can\'t specify a source-tag for a zip source') @@ -42,6 +43,9 @@ def provision(self, dst, clean_target=True, keep_zip=False): zip = os.path.join(self.source_dir, os.path.basename(self.source)) + if self.source_checksum: + sources.verify_checksum(self.source_checksum, zip) + if clean_target: tmp_zip = tempfile.NamedTemporaryFile().name shutil.move(zip, tmp_zip) diff -Nru snapcraft-2.27.1+17.04/snapcraft/internal/states/_pull_state.py snapcraft-2.28+17.04/snapcraft/internal/states/_pull_state.py --- snapcraft-2.27.1+17.04/snapcraft/internal/states/_pull_state.py 2017-01-28 06:12:26.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/internal/states/_pull_state.py 2017-03-22 12:31:58.000000000 +0000 @@ -43,11 +43,15 @@ class PullState(State): yaml_tag = u'!PullState' - def __init__(self, property_names, part_properties=None, project=None): + def __init__(self, property_names, part_properties=None, project=None, + stage_packages=None): # Save this off before calling super() since we'll need it # FIXME: for 3.x the name `schema_properties` is leaking # implementation details from a higher layer. self.schema_properties = property_names + self.assets = { + 'stage-packages': stage_packages, + } super().__init__(part_properties, project) diff -Nru snapcraft-2.27.1+17.04/snapcraft/main.py snapcraft-2.28+17.04/snapcraft/main.py --- snapcraft-2.27.1+17.04/snapcraft/main.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/main.py 2017-03-22 12:31:58.000000000 +0000 @@ -43,6 +43,8 @@ snapcraft [options] push [--release ] snapcraft [options] release snapcraft [options] status [--series=] [--arch=] + snapcraft [options] list-revisions [--series=] [--arch=] + snapcraft [options] revisions [--series=] [--arch=] snapcraft [options] history [--series=] [--arch=] snapcraft [options] close ... snapcraft [options] list-plugins @@ -97,31 +99,33 @@ --series Snap series [default: {DEFAULT_SERIES}]. The available commands are: - help Obtain help for a certain plugin or topic - init Initialize a snapcraft project. - list-plugins List the available plugins that handle different types of part. - plugins Alias for list-plugins. - login Authenticate session against Ubuntu One SSO. - logout Clear session credentials. + help Obtain help for a certain plugin or topic + init Initialize a snapcraft project. + list-plugins List the available plugins that handle different types of part. + plugins Alias for list-plugins. + login Authenticate session against Ubuntu One SSO. + logout Clear session credentials. list-registered List snap names registered or shared with you. - registered Alias for list-registered. - list-keys List keys available for signing snaps. - keys Alias for list-keys. - create-key Create a key pair for signing snaps. - register-key Register a key for signing snaps. - register Register the package name in the store. - tour Setup the snapcraft examples tour in the specified directory, - or ./snapcraft-tour/. - sign-build Sign a built snap file and assert it using the developer's key. - push Pushes and optionally releases a snap to the Ubuntu Store. - upload DEPRECATED Upload a snap to the Ubuntu Store. The push command - supersedes this command. - release Release a revision of a snap to a specific channel. - status Show the current status of a snap per channel and architecture. - history List all revisions of a snap. - close Close one or more channels of a snap. - enable-ci EXPERIMENTAL enable continuous-integration systems to build and - release snaps to the Ubuntu Store. + registered Alias for list-registered. + list-keys List keys available for signing snaps. + keys Alias for list-keys. + create-key Create a key pair for signing snaps. + register-key Register a key for signing snaps. + register Register the package name in the store. + tour Setup the snapcraft examples tour in the specified directory, + or ./snapcraft-tour/. + sign-build Sign a built snap file and assert it using the developer's key. + push Pushes and optionally releases a snap to the Ubuntu Store. + upload DEPRECATED Upload a snap to the Ubuntu Store. The push command + supersedes this command. + release Release a revision of a snap to a specific channel. + status Show the current status of a snap per channel and architecture. + list-revisions List all revisions of a snap. + revisions Alias for list-revisions. + history Deprecated alias for list-revisions. + close Close one or more channels of a snap. + enable-ci EXPERIMENTAL enable continuous-integration systems to build and + release snaps to the Ubuntu Store. The available lifecycle commands are: clean Remove content - cleans downloads, builds or install artifacts. @@ -161,7 +165,12 @@ import snapcraft from snapcraft.integrations import enable_ci -from snapcraft.internal import lifecycle, log, parts +from snapcraft.internal import ( + deprecations, + lifecycle, + log, + parts, +) from snapcraft.internal.common import ( format_output_in_columns, get_terminal_width, @@ -318,7 +327,8 @@ commands = ( 'list-registered', 'registered', 'list-keys', 'keys', 'create-key', 'register-key', 'register', 'sign-build', 'upload', 'release', - 'push', 'validate', 'gated', 'history', 'status', 'close') + 'push', 'validate', 'gated', 'history', 'revisions', + 'list-revisions', 'status', 'close') return any(args.get(command) for command in commands) @@ -358,8 +368,10 @@ elif args['status']: snapcraft.status( args[''], args['--series'], args['--arch']) - elif args['history']: - snapcraft.history( + elif args['revisions'] or args['list-revisions'] or args['history']: + if args['history']: + deprecations.handle_deprecation_notice('dn4') + snapcraft.revisions( args[''], args['--series'], args['--arch']) elif args['close']: snapcraft.close(args[''], args['']) diff -Nru snapcraft-2.27.1+17.04/snapcraft/_options.py snapcraft-2.28+17.04/snapcraft/_options.py --- snapcraft-2.27.1+17.04/snapcraft/_options.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/_options.py 2017-03-22 12:31:58.000000000 +0000 @@ -21,6 +21,7 @@ from snapcraft.internal import common from snapcraft.internal.deprecations import handle_deprecation_notice +from snapcraft.internal.errors import SnapcraftEnvironmentError logger = logging.getLogger(__name__) @@ -193,18 +194,33 @@ However if core is not installed None will be returned. """ core_path = common.get_core_path() - core_dynamic_linker = self.__machine_info.get('core-dynamic-linker', - 'lib/ld-linux.so.2') - - try: - dynamic_linker_resolved_path = os.readlink( - os.path.join(core_path, core_dynamic_linker)) - dynamic_linker_path = os.path.join( - core_path, dynamic_linker_resolved_path.lstrip('/')) - except FileNotFoundError: - dynamic_linker_path = None - - return dynamic_linker_path + dynamic_linker_path = os.path.join( + core_path, + self.__machine_info.get('core-dynamic-linker', + 'lib/ld-linux.so.2')) + + # We can't use os.path.realpath because any absolute symlinks + # have to be interpreted relative to core_path, not the real + # root. + seen_paths = set() + while True: + if dynamic_linker_path in seen_paths: + raise SnapcraftEnvironmentError( + "found symlink loop resolving dynamic linker path") + + seen_paths.add(dynamic_linker_path) + if not os.path.lexists(dynamic_linker_path): + return None + if not os.path.islink(dynamic_linker_path): + return dynamic_linker_path + + link_contents = os.readlink(dynamic_linker_path) + if os.path.isabs(link_contents): + dynamic_linker_path = os.path.join( + core_path, link_contents.lstrip('/')) + else: + dynamic_linker_path = os.path.join( + os.path.dirname(dynamic_linker_path), link_contents) def _set_machine(self, target_deb_arch): self.__platform_arch = _get_platform_architecture() diff -Nru snapcraft-2.27.1+17.04/snapcraft/plugins/catkin.py snapcraft-2.28+17.04/snapcraft/plugins/catkin.py --- snapcraft-2.27.1+17.04/snapcraft/plugins/catkin.py 2017-02-17 13:45:14.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/plugins/catkin.py 2017-03-22 12:31:58.000000000 +0000 @@ -34,6 +34,19 @@ - include-roscore: (boolean) Whether or not to include roscore with the part. Defaults to true. + - underlay: + (object) + Used to inform Snapcraft that this snap isn't standalone, and is actually + overlaying a workspace from another snap via content sharing. Made up of + two properties: + - build-path: + (string) + Build-time path to existing workspace to underlay the one being built, + for example '$SNAPCRAFT_STAGE/opt/ros/kinetic'. + - run-path: + (string) + Run-time path of the underlay workspace (e.g. a subdirectory of the + content interface's 'target' attribute.) """ import contextlib @@ -53,6 +66,7 @@ formatting_utils, repo, ) +from snapcraft.internal import errors logger = logging.getLogger(__name__) @@ -96,6 +110,19 @@ 'default': 'true', } + schema['properties']['underlay'] = { + 'type': 'object', + 'properties': { + 'build-path': { + 'type': 'string', + }, + 'run-path': { + 'type': 'string', + } + }, + 'required': ['build-path', 'run-path'], + } + schema['required'].append('catkin-packages') return schema @@ -105,7 +132,7 @@ # Inform Snapcraft of the properties associated with pulling. If these # change in the YAML Snapcraft will consider the pull step dirty. return ['rosdistro', 'catkin-packages', 'source-space', - 'include-roscore'] + 'include-roscore', 'underlay'] @property def PLUGIN_STAGE_SOURCES(self): @@ -121,10 +148,16 @@ super().__init__(name, options, project) self.build_packages.extend(['libc6-dev', 'make']) + # roslib is the base requiremet to actually create a workspace with + # setup.sh and the necessary hooks. + self.stage_packages.append( + 'ros-{}-roslib'.format(self.options.rosdistro)) + # Get a unique set of packages self.catkin_packages = set(options.catkin_packages) self._rosdep_path = os.path.join(self.partdir, 'rosdep') self._compilers_path = os.path.join(self.partdir, 'compilers') + self._catkin_path = os.path.join(self.partdir, 'catkin') # The path created via the `source` key (or a combination of `source` # and `source-subdir` keys) needs to point to a valid Catkin workspace @@ -172,12 +205,6 @@ # temporarily work around that bug by forcing the locale to # C.UTF-8. 'LC_ALL=C.UTF-8', - - # This environment variable points to where the setup.sh and - # _setup_util.py files are located. This is required at both build- - # and run-time. - '_CATKIN_SETUP_DIR={}'.format(os.path.join( - root, 'opt', 'ros', self.options.rosdistro)), ] # There's a chicken and egg problem here, everything run get's an @@ -196,37 +223,17 @@ # make sure it's in the PATH before it's run. env.append('PATH=$PATH:{}/usr/bin'.format(root)) - # We need to source ROS's setup.sh at this point. However, it accepts - # arguments (thus will parse $@), and we really don't want it to, since - # $@ in this context will be meant for the app being launched - # (LP: #1660852). So we'll backup all args, source the setup.sh, then - # restore all args for the wrapper's `exec` line. - script = textwrap.dedent(''' - if [ -e {0} ]; then - # Shell quote arbitrary string by replacing every occurrence of ' - # with '\\'', then put ' at the beginning and end of the string. - # Prepare yourself, fun regex ahead. - quote() - {{ - for i; do - printf %s\\\\n "$i" | sed "s/\'/\'\\\\\\\\\'\'/g;1s/^/\'/;\$s/\$/\' \\\\\\\\/" - done - echo " " - }} - - BACKUP_ARGS=$(quote "$@") - set -- - . {0} - eval "set -- $BACKUP_ARGS" - fi - '''.format(os.path.join( # noqa - root, 'opt', 'ros', self.options.rosdistro, 'setup.sh'))) + if self.options.underlay: + script = '. {}'.format(os.path.join( + self.rosdir, 'snapcraft-setup.sh')) + else: + script = self._source_setup_sh(root, None) # Each of these lines is prepended with an `export` when the # environment is actually generated. In order to inject real shell code - # we have to hack it in by appending it on the end of an item in the - # environment. FIXME: There should be a better way to do this. - env[-1] = env[-1] + script + # we have to hack it in by appending it on the end of an item already + # in the environment. FIXME: There should be a better way to do this. + env[-1] = env[-1] + '\n\n' + script return env @@ -243,11 +250,39 @@ super().pull() # Make sure the package path exists before continuing - if not os.path.exists(self._ros_package_path): + if self.catkin_packages and not os.path.exists(self._ros_package_path): raise FileNotFoundError( 'Unable to find package path: "{}"'.format( self._ros_package_path)) + # Validate the underlay. Note that this validation can't happen in + # __init__ as the underlay will probably only be valid once a + # dependency has been staged. + catkin = None + underlay_build_path = None + if self.options.underlay: + underlay_build_path = self.options.underlay['build-path'] + if underlay_build_path: + if not os.path.isdir(underlay_build_path): + raise errors.SnapcraftEnvironmentError( + 'Requested underlay ({!r}) does not point to a valid ' + 'directory'.format(underlay_build_path)) + + if not os.path.isfile(os.path.join(underlay_build_path, + 'setup.sh')): + raise errors.SnapcraftEnvironmentError( + 'Requested underlay ({!r}) does not contain a ' + 'setup.sh'.format(underlay_build_path)) + + # Use catkin_find to discover dependencies already in the underlay + catkin = _Catkin( + self.options.rosdistro, underlay_build_path, self._catkin_path, + self.PLUGIN_STAGE_SOURCES, self.project) + catkin.setup() + + self._generate_snapcraft_setup_sh( + self.installdir, underlay_build_path) + # Pull our own compilers so we use ones that match up with the version # of ROS we're using. compilers = _Compilers( @@ -260,9 +295,12 @@ self.project) rosdep.setup() + self._setup_dependencies(rosdep, catkin) + + def _setup_dependencies(self, rosdep, catkin): # Parse the Catkin packages to pull out their system dependencies - system_dependencies = _find_system_dependencies(self.catkin_packages, - rosdep) + system_dependencies = _find_system_dependencies( + self.catkin_packages, rosdep, catkin) # If the package requires roscore, resolve it into a system dependency # as well. @@ -280,15 +318,14 @@ os.makedirs(ubuntudir, exist_ok=True) logger.info('Preparing to fetch package dependencies...') - ubuntu = repo.Ubuntu( - ubuntudir, self.project, - sources=self.PLUGIN_STAGE_SOURCES, - project_options=self.project) + ubuntu = repo.Ubuntu(ubuntudir, + sources=self.PLUGIN_STAGE_SOURCES, + project_options=self.project) logger.info('Fetching package dependencies...') try: ubuntu.get(system_dependencies) - except repo.PackageNotFoundError as e: + except repo.errors.PackageNotFoundError as e: raise RuntimeError( 'Failed to fetch system dependencies: {}'.format( e.message)) @@ -307,6 +344,64 @@ with contextlib.suppress(FileNotFoundError): shutil.rmtree(self._compilers_path) + # Remove the catkin path, if any + with contextlib.suppress(FileNotFoundError): + shutil.rmtree(self._catkin_path) + + def _source_setup_sh(self, root, underlay_path): + rosdir = os.path.join(root, 'opt', 'ros', self.options.rosdistro) + if underlay_path: + source_script = textwrap.dedent(''' + if [ -f {underlay_setup} ]; then + _CATKIN_SETUP_DIR={underlay} . {underlay_setup} + if [ -f {rosdir_setup} ]; then + set -- --extend + _CATKIN_SETUP_DIR={rosdir} . {rosdir_setup} + fi + fi + ''').format( + underlay=underlay_path, + underlay_setup=os.path.join(underlay_path, 'setup.sh'), + rosdir=rosdir, + rosdir_setup=os.path.join(rosdir, 'setup.sh')) + else: + source_script = textwrap.dedent(''' + if [ -f {rosdir_setup} ]; then + _CATKIN_SETUP_DIR={rosdir} . {rosdir_setup} + fi + ''').format( + rosdir=rosdir, + rosdir_setup=os.path.join(rosdir, 'setup.sh')) + + # We need to source ROS's setup.sh at this point. However, it accepts + # arguments (thus will parse $@), and we really don't want it to, since + # $@ in this context will be meant for the app being launched + # (LP: #1660852). So we'll backup all args, source the setup.sh, then + # restore all args for the wrapper's `exec` line. + return textwrap.dedent(''' + # Shell quote arbitrary string by replacing every occurrence of ' + # with '\\'', then put ' at the beginning and end of the string. + # Prepare yourself, fun regex ahead. + quote() + {{ + for i; do + printf %s\\\\n "$i" | sed "s/\'/\'\\\\\\\\\'\'/g;1s/^/\'/;\$s/\$/\' \\\\\\\\/" + done + echo " " + }} + + BACKUP_ARGS=$(quote "$@") + set -- + {} + eval "set -- $BACKUP_ARGS" + ''').format(source_script) # noqa + + def _generate_snapcraft_setup_sh(self, root, underlay_path): + script = self._source_setup_sh(root, underlay_path) + os.makedirs(self.rosdir, exist_ok=True) + with open(os.path.join(self.rosdir, 'snapcraft-setup.sh'), 'w') as f: + f.write(script) + @property def rosdir(self): return os.path.join(self.installdir, 'opt', 'ros', @@ -314,7 +409,7 @@ def _run_in_bash(self, commandlist, cwd=None, env=None): with tempfile.NamedTemporaryFile(mode='w') as f: - f.write('set -ex\n') + f.write('set -e\n') f.write('exec {}\n'.format(' '.join(commandlist))) f.flush() @@ -408,6 +503,10 @@ f.truncate() f.write(replaced) + if self.options.underlay: + underlay_run_path = self.options.underlay['run-path'] + self._generate_snapcraft_setup_sh('$SNAP', underlay_run_path) + def _use_in_snap_python(self): # Fix all shebangs to use the in-snap python. file_utils.replace_in_file(self.rosdir, re.compile(r''), @@ -472,8 +571,20 @@ # that instead. self._run_in_bash(catkincmd, env=compilers.environment) + def snap_fileset(self): + """Filter useless files out of the snap. + + - opt/ros//.rosinstall points to the part installdir, and + isn't useful from the snap anyway. + """ + + fileset = super().snap_fileset() + fileset.append('-{}'.format( + os.path.join('opt', 'ros', self.options.rosdistro, '.rosinstall'))) + return fileset + -def _find_system_dependencies(catkin_packages, rosdep): +def _find_system_dependencies(catkin_packages, rosdep, catkin): """Find system dependencies for a given set of Catkin packages.""" system_dependencies = {} @@ -490,13 +601,29 @@ dependency in system_dependencies): continue + if catkin: + # Before trying to resolve this dependency into a system + # dependency, see if it's already in the underlay. + try: + catkin.find(dependency) + except CatkinPackageNotFoundError: + # No package by that name is available + pass + else: + # Package was found-- don't pull anything extra to satisfy + # this dependency. + logger.debug( + 'Satisfied dependency {!r} in underlay'.format( + dependency)) + continue + # In this situation, the package depends on something that we # weren't instructed to build. It's probably a system dependency, # but the developer could have also forgotten to tell us to build # it. try: these_dependencies = rosdep.resolve_dependency(dependency) - except SystemDependencyNotFound: + except SystemDependencyNotFoundError: raise RuntimeError( "Package {!r} isn't a valid system dependency. " "Did you forget to add it to catkin-packages? If " @@ -511,8 +638,18 @@ for item in sublist) -class SystemDependencyNotFound(Exception): - pass +class SystemDependencyNotFoundError(errors.SnapcraftError): + fmt = '{system_dependency!r} does not resolve to a system dependency' + + def __init__(self, system_dependency): + super().__init__(system_dependency=system_dependency) + + +class CatkinPackageNotFoundError(errors.SnapcraftError): + fmt = 'Unable to find Catkin package {package_name!r}' + + def __init__(self, package_name): + super().__init__(package_name=package_name) class _Rosdep: @@ -593,9 +730,7 @@ 'ubuntu:{}'.format( _ROS_RELEASE_MAP[self._ros_distro])]) except subprocess.CalledProcessError: - raise SystemDependencyNotFound( - '{!r} does not resolve to a system dependency'.format( - dependency_name)) + raise SystemDependencyNotFoundError(dependency_name) # Everything that isn't a package name is prepended with the pound # sign, so we'll ignore everything with that. @@ -712,6 +847,62 @@ paths, prepend='-L', separator=' ') +class _Catkin: + def __init__(self, ros_distro, workspace, catkin_path, ubuntu_sources, + project): + self._ros_distro = ros_distro + self._workspace = workspace + self._catkin_path = catkin_path + self._ubuntu_sources = ubuntu_sources + self._project = project + self._catkin_install_path = os.path.join(self._catkin_path, 'install') + + def setup(self): + os.makedirs(self._catkin_install_path, exist_ok=True) + + # With the introduction of an underlay, we no longer know where Catkin + # is. Let's just fetch/unpack our own, and use it. + logger.info('Preparing to fetch catkin...') + ubuntu = repo.Ubuntu(self._catkin_path, sources=self._ubuntu_sources, + project_options=self._project) + logger.info('Fetching catkin...') + ubuntu.get(['ros-{}-catkin'.format(self._ros_distro)]) + + logger.info('Installing catkin...') + ubuntu.unpack(self._catkin_install_path) + + def find(self, package_name): + try: + return self._run(['--first-only', package_name]).strip() + except subprocess.CalledProcessError: + raise CatkinPackageNotFoundError(package_name) + + def _run(self, arguments): + with tempfile.NamedTemporaryFile(mode='w+') as f: + lines = ['export PYTHONPATH={}'.format(os.path.join( + self._catkin_install_path, 'usr', 'lib', 'python2.7', + 'dist-packages'))] + + ros_path = os.path.join( + self._catkin_install_path, 'opt', 'ros', self._ros_distro) + bin_paths = ( + os.path.join(ros_path, 'bin'), + os.path.join(self._catkin_install_path, 'usr', 'bin')) + lines.append('export {}'.format( + formatting_utils.format_path_variable( + 'PATH', bin_paths, prepend='', separator=':'))) + + lines.append('export _CATKIN_SETUP_DIR={}'.format(self._workspace)) + lines.append('source {}'.format(os.path.join( + self._workspace, 'setup.sh'))) + lines.append('exec "$@"') + f.write('\n'.join(lines)) + f.flush() + return subprocess.check_output( + ['/bin/bash', f.name, 'catkin_find'] + arguments, + stderr=subprocess.STDOUT).decode('utf8').strip() + + def _get_highest_version_path(path): paths = sorted(glob.glob(os.path.join(path, '*'))) if not paths: diff -Nru snapcraft-2.27.1+17.04/snapcraft/plugins/copy.py snapcraft-2.28+17.04/snapcraft/plugins/copy.py --- snapcraft-2.27.1+17.04/snapcraft/plugins/copy.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/plugins/copy.py 2017-03-22 12:31:58.000000000 +0000 @@ -40,7 +40,6 @@ import snapcraft - logger = logging.getLogger(__name__) @@ -115,7 +114,9 @@ destination_dirname = os.path.dirname(destination) normalized = os.path.normpath(os.path.join(destination_dirname, link)) if os.path.isabs(link) or not normalized.startswith(boundary): - follow_symlinks = True + # Only follow symlinks that are NOT pointing at libc (LP: #1658774) + if link not in snapcraft.repo.Repo.get_package_libraries('libc6'): + follow_symlinks = True snapcraft.common.link_or_copy(source, destination, follow_symlinks=follow_symlinks) diff -Nru snapcraft-2.27.1+17.04/snapcraft/plugins/dump.py snapcraft-2.28+17.04/snapcraft/plugins/dump.py --- snapcraft-2.27.1+17.04/snapcraft/plugins/dump.py 2016-11-02 08:25:17.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/plugins/dump.py 2017-03-22 12:31:58.000000000 +0000 @@ -56,7 +56,9 @@ destination_dirname = os.path.dirname(destination) normalized = os.path.normpath(os.path.join(destination_dirname, link)) if os.path.isabs(link) or not normalized.startswith(boundary): - follow_symlinks = True + # Only follow symlinks that are NOT pointing at libc (LP: #1658774) + if link not in snapcraft.repo.Repo.get_package_libraries('libc6'): + follow_symlinks = True try: snapcraft.file_utils.link_or_copy(source, destination, diff -Nru snapcraft-2.27.1+17.04/snapcraft/plugins/godeps.py snapcraft-2.28+17.04/snapcraft/plugins/godeps.py --- snapcraft-2.27.1+17.04/snapcraft/plugins/godeps.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/plugins/godeps.py 2017-03-22 12:31:58.000000000 +0000 @@ -98,7 +98,10 @@ def __init__(self, name, options, project): super().__init__(name, options, project) - self.build_packages.append('golang-go') + self.build_packages.extend([ + 'golang-go', + 'git', + ]) self._gopath = os.path.join(self.partdir, 'go') self._gopath_src = os.path.join(self._gopath, 'src') self._gopath_bin = os.path.join(self._gopath, 'bin') diff -Nru snapcraft-2.27.1+17.04/snapcraft/plugins/kbuild.py snapcraft-2.28+17.04/snapcraft/plugins/kbuild.py --- snapcraft-2.27.1+17.04/snapcraft/plugins/kbuild.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/plugins/kbuild.py 2017-03-22 20:15:48.000000000 +0000 @@ -59,6 +59,7 @@ import os import shutil import subprocess +import re from snapcraft import BasePlugin @@ -149,6 +150,13 @@ subprocess.check_call(cmd, shell=True, cwd=self.builddir) def do_build(self): + # Linux's kernel Makefile gets confused if it is invoked with the + # environment setup by another Linux's Makefile: + # linux/package/Makefile -> snapcraft -> linux/Makefile + # fix the problem removing the offending make option (-I...) + if 'MAKEFLAGS' in os.environ: + makeflags = re.sub('-I[\S]*', '', os.environ['MAKEFLAGS']) + os.environ['MAKEFLAGS'] = makeflags # build the software self.run(self.make_cmd + self.make_targets) diff -Nru snapcraft-2.27.1+17.04/snapcraft/plugins/kernel.py snapcraft-2.28+17.04/snapcraft/plugins/kernel.py --- snapcraft-2.27.1+17.04/snapcraft/plugins/kernel.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/plugins/kernel.py 2017-03-23 12:02:13.000000000 +0000 @@ -23,7 +23,7 @@ The following kernel specific options are provided by this plugin: - kernel-image-target: - (yaml object or string; default: bzImage) + (yaml object, string or null for default target) the default target is bzImage and can be set to any specific target. For more complex cases where one would want to use @@ -74,6 +74,13 @@ 'gz': 'gzip', } +default_kernel_image_target = { + 'amd64': 'bzImage', + 'i386': 'bzImage', + 'armhf': 'zImage', + 'arm64': 'Image.gz', +} + class KernelPlugin(kbuild.KBuildPlugin): @@ -86,7 +93,7 @@ {'type': 'string'}, {'type': 'object'}, ], - 'default': 'bzImage', + 'default': '', } schema['properties']['kernel-with-firmware'] = { @@ -166,19 +173,28 @@ self._set_kernel_targets() def _set_kernel_targets(self): - if isinstance(self.options.kernel_image_target, str): + if not self.options.kernel_image_target: + self.kernel_image_target = \ + default_kernel_image_target[self.project.deb_arch] + elif isinstance(self.options.kernel_image_target, str): self.kernel_image_target = self.options.kernel_image_target elif self.project.deb_arch in self.options.kernel_image_target: self.kernel_image_target = \ self.options.kernel_image_target[self.project.deb_arch] self.make_targets = [self.kernel_image_target, 'modules'] + self.make_install_targets = [ + 'modules_install', 'INSTALL_MOD_PATH={}'.format(self.installdir)] self.dtbs = ['{}.dtb'.format(i) for i in self.options.kernel_device_trees] if self.dtbs: self.make_targets.extend(self.dtbs) - self.make_install_targets = [ - 'modules_install', 'INSTALL_MOD_PATH={}'.format(self.installdir)] + elif (self.project.kernel_arch == "arm" or + self.project.kernel_arch == "arm64"): + self.make_targets.append('dtbs') + self.make_install_targets.extend([ + 'dtbs_install', + 'INSTALL_DTBS_PATH={}/dtbs'.format(self.installdir)]) self.make_install_targets.extend(self._get_fw_install_targets()) def _get_fw_install_targets(self): @@ -238,6 +254,7 @@ return initrd_unpacked_path def _make_initrd(self): + logger.info('Generating driver initrd for kernel release: {}'.format( self.kernel_release)) @@ -250,9 +267,9 @@ '-S', self.kernel_release, module]) modprobe_outs.extend(modprobe_out.split(os.linesep)) + modprobe_outs = [_ for _ in modprobe_outs if _] modules_path = os.path.join('lib', 'modules', self.kernel_release) - for src in set(modprobe_outs): - src = src.split()[-1:][0] + for src in set(_.split()[1] for _ in modprobe_outs): dst = os.path.join(initrd_unpacked_path, os.path.relpath(src, self.installdir)) os.makedirs(os.path.dirname(dst), exist_ok=True) diff -Nru snapcraft-2.27.1+17.04/snapcraft/plugins/plainbox_provider.py snapcraft-2.28+17.04/snapcraft/plugins/plainbox_provider.py --- snapcraft-2.27.1+17.04/snapcraft/plugins/plainbox_provider.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/plugins/plainbox_provider.py 2017-03-22 12:31:58.000000000 +0000 @@ -1,6 +1,6 @@ # -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*- # -# Copyright (C) 2016 Canonical Ltd +# Copyright (C) 2016-2017 Canonical Ltd # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License version 3 as @@ -29,6 +29,7 @@ 'sources' topic for the latter. """ +import os import re import snapcraft @@ -44,6 +45,13 @@ def build(self): super().build() + env = os.environ.copy() + provider_stage_dir = os.path.join(self.project.stage_dir, 'providers') + if os.path.exists(provider_stage_dir): + provider_dirs = [os.path.join(provider_stage_dir, provider) + for provider in os.listdir(provider_stage_dir)] + env['PROVIDERPATH'] = ':'.join(provider_dirs) + self.run(['python3', 'manage.py', 'validate'], env=env) self.run(['python3', 'manage.py', 'build']) self.run(['python3', 'manage.py', 'i18n']) self.run([ diff -Nru snapcraft-2.27.1+17.04/snapcraft/plugins/python.py snapcraft-2.28+17.04/snapcraft/plugins/python.py --- snapcraft-2.27.1+17.04/snapcraft/plugins/python.py 2017-02-17 13:45:14.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/plugins/python.py 2017-03-22 15:10:01.000000000 +0000 @@ -57,7 +57,7 @@ import stat import subprocess import tempfile -from contextlib import contextmanager +from contextlib import contextmanager, suppress from glob import glob from shutil import which from textwrap import dedent @@ -193,25 +193,36 @@ else: return unstaged_python + def _get_python_headers(self): + base_match = os.path.join('usr', 'include', '{}*'.format( + self.options.python_version)) + unstaged_python = glob(os.path.join(os.path.sep, base_match)) + staged_python = glob(os.path.join(self.project.stage_dir, base_match)) + + if staged_python: + return staged_python[0] + elif unstaged_python: + return unstaged_python[0] + else: + return '' + def _install_pip(self, download): - env = os.environ.copy() - env['PYTHONUSERBASE'] = self.installdir + env = self._get_build_env() # since we are using an independent env we need to export this too # TODO: figure out if we can move back to common.run env['SNAPCRAFT_STAGE'] = self.project.stage_dir env['SNAPCRAFT_PART_INSTALL'] = self.installdir - - args = ['pip', 'setuptools', 'wheel'] - - pip_command = [self._get_python_command(), '-m', 'pip'] - - # If python_command it is not from stage we don't have pip, which means + # If python_command is not from stage we don't have pip, which means # we are going to need to resort to the pip installed on the system # that came from build-packages. This shouldn't be a problem as # stage-packages and build-packages should match. if not self._get_python_command().startswith(self.project.stage_dir): env['PYTHONHOME'] = '/usr' + args = ['pip', 'setuptools', 'wheel'] + + pip_command = [self._get_python_command(), '-m', 'pip'] + pip = _Pip(exec_func=subprocess.check_call, runnable=pip_command, package_dir=self._python_package_dir, env=env, @@ -232,12 +243,14 @@ env['PATH'] = '{}:{}'.format( os.path.join(self.installdir, 'usr', 'bin'), os.path.expandvars('$PATH')) - headers = glob(os.path.join( - os.path.sep, 'usr', 'include', '{}*'.format( - self.options.python_version))) + + headers = self._get_python_headers() if headers: - env['CPPFLAGS'] = '-I{} {}'.format( - headers[0], env.get('CPPFLAGS', '')) + current_cppflags = env.get('CPPFLAGS', '') + env['CPPFLAGS'] = '-I{}'.format(headers) + if current_cppflags: + env['CPPFLAGS'] = '{} {}'.format( + env['CPPFLAGS'], current_cppflags) return env @@ -263,6 +276,17 @@ else: return [] + def _setup_tools_install(self, setup_file): + command = [ + self._get_python_command(), + os.path.basename(setup_file), '--no-user-cfg', 'install', + '--single-version-externally-managed', + '--user', '--record', 'install.txt'] + cwd = os.path.dirname(setup_file) + self.run( + command, env=self._get_build_env(), + cwd=cwd) + def _run_pip(self, setup, download=False): self._install_pip(download) @@ -299,6 +323,14 @@ # stage-packages need_install = [k for k in wheel_names if k not in installed] pip.install(need_install + ['--no-deps', '--upgrade']) + if os.path.exists(setup): + # pbr and others don't work using `pip install .` + # LP: #1670852 + # There is also a chance that this setup.py is distutils based + # in which case we will rely on the `pip install .` ran before + # this. + with suppress(subprocess.CalledProcessError): + self._setup_tools_install(setup) def _fix_permissions(self): for root, dirs, files in os.walk(self.installdir): diff -Nru snapcraft-2.27.1+17.04/snapcraft/storeapi/_agent.py snapcraft-2.28+17.04/snapcraft/storeapi/_agent.py --- snapcraft-2.27.1+17.04/snapcraft/storeapi/_agent.py 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/storeapi/_agent.py 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,42 @@ +# -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*- +# +# Copyright (C) 2017 Canonical Ltd +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License version 3 as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . + +import os +import platform + +import snapcraft + + +def _is_ci_env(): + env_prefixes = ['TRAVIS', 'AUTOPKGTEST_TMP'] + matches = [] + + for prefix in env_prefixes: + matches += [ + var for var in os.environ.keys() if var.startswith(prefix)] + + return len(matches) > 0 + + +def get_user_agent(): + arch = snapcraft.ProjectOptions().deb_arch + testing = '(testing) ' if _is_ci_env() else '' + return 'snapcraft/{} {}{} ({})'.format( + snapcraft.__version__, + testing, + '/'.join(platform.dist()[0:2]), # i.e. Ubuntu/16.04 + arch, + ) diff -Nru snapcraft-2.27.1+17.04/snapcraft/storeapi/errors.py snapcraft-2.28+17.04/snapcraft/storeapi/errors.py --- snapcraft-2.27.1+17.04/snapcraft/storeapi/errors.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/storeapi/errors.py 2017-03-23 12:02:09.000000000 +0000 @@ -317,10 +317,10 @@ super().__init__(error=error) -class StoreSnapHistoryError(StoreError): +class StoreSnapRevisionsError(StoreError): fmt = ( - 'Error fetching history of snap id {snap_id!r} for {arch!r} ' + 'Error fetching revisions of snap id {snap_id!r} for {arch!r} ' 'in {series!r} series: {error}.') def __init__(self, response, snap_id, series, arch): @@ -342,7 +342,7 @@ pass -class StoreSnapStatusError(StoreSnapHistoryError): +class StoreSnapStatusError(StoreSnapRevisionsError): fmt = ( 'Error fetching status of snap id {snap_id!r} for {arch!r} ' diff -Nru snapcraft-2.27.1+17.04/snapcraft/storeapi/__init__.py snapcraft-2.28+17.04/snapcraft/storeapi/__init__.py --- snapcraft-2.27.1+17.04/snapcraft/storeapi/__init__.py 2017-02-17 19:54:35.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/storeapi/__init__.py 2017-03-23 12:02:09.000000000 +0000 @@ -1,7 +1,7 @@ # -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*- # -# Copyright (C) 2016 Canonical Ltd +# Copyright (C) 2016-2017 Canonical Ltd # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License version 3 as @@ -33,12 +33,14 @@ ) import pymacaroons import requests +from requests.adapters import HTTPAdapter from simplejson.scanner import JSONDecodeError import snapcraft from snapcraft import config from snapcraft.internal.indicators import download_requests_stream from snapcraft.storeapi import ( + _agent, _upload, constants, errors, @@ -91,8 +93,12 @@ self.conf = conf self.root_url = root_url self.session = requests.Session() + # Setup max retries for all store URLs and the CDN + self.session.mount('http://', HTTPAdapter(max_retries=5)) + self.session.mount('https://', HTTPAdapter(max_retries=5)) + self._snapcraft_headers = { - 'X-SNAPCRAFT-VERSION': snapcraft.__version__ + 'User-Agent': _agent.get_user_agent(), } def request(self, method, url, params=None, headers=None, **kwargs): @@ -212,7 +218,7 @@ return self._refresh_if_necessary( self.sca.snap_release, snap_name, revision, channels) - def get_snap_history(self, snap_name, series=None, arch=None): + def get_snap_revisions(self, snap_name, series=None, arch=None): if series is None: series = constants.DEFAULT_SERIES @@ -223,7 +229,7 @@ raise errors.SnapNotFoundError(snap_name, series=series, arch=arch) response = self._refresh_if_necessary( - self.sca.snap_history, snap_id, series, arch) + self.sca.snap_revisions, snap_id, series, arch) if not response: raise errors.SnapNotFoundError(snap_name, series=series, arch=arch) @@ -624,7 +630,7 @@ if not response.ok: raise errors.StoreSnapBuildError(response) - def snap_history(self, snap_id, series, arch): + def snap_revisions(self, snap_id, series, arch): qs = {} if series: qs['series'] = series @@ -640,7 +646,8 @@ 'Content-Type': 'application/json', 'Accept': 'application/json'}) if not response.ok: - raise errors.StoreSnapHistoryError(response, snap_id, series, arch) + raise errors.StoreSnapRevisionsError( + response, snap_id, series, arch) response_json = response.json() @@ -651,8 +658,8 @@ if series: qs['series'] = series if arch: - qs['arch'] = arch - url = 'snaps/' + snap_id + '/status' + qs['architecture'] = arch + url = 'snaps/' + snap_id + '/state' if qs: url += '?' + urllib.parse.urlencode(qs) auth = _macaroon_auth(self.conf) @@ -682,7 +689,7 @@ try: results = response.json() - return results['closed_channels'], results['channel_maps'] + return results['closed_channels'], results['channel_map_tree'] except (JSONDecodeError, KeyError): logger.debug( 'Invalid response from the server on channel closing:\n' diff -Nru snapcraft-2.27.1+17.04/snapcraft/_store.py snapcraft-2.28+17.04/snapcraft/_store.py --- snapcraft-2.27.1+17.04/snapcraft/_store.py 2017-02-17 19:54:46.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/_store.py 2017-03-22 12:31:58.000000000 +0000 @@ -237,7 +237,7 @@ def list_keys(): - if not repo.is_package_installed('snapd'): + if not repo.Repo.is_package_installed('snapd'): raise EnvironmentError( 'The snapd package is not installed. In order to use `list-keys`, ' 'you must run `apt install snapd`.') @@ -259,7 +259,7 @@ def create_key(name): - if not repo.is_package_installed('snapd'): + if not repo.Repo.is_package_installed('snapd'): raise EnvironmentError( 'The snapd package is not installed. In order to use ' '`create-key`, you must run `apt install snapd`.') @@ -303,7 +303,7 @@ def register_key(name): - if not repo.is_package_installed('snapd'): + if not repo.Repo.is_package_installed('snapd'): raise EnvironmentError( 'The snapd package is not installed. In order to use ' '`register-key`, you must run `apt install snapd`.') @@ -349,7 +349,7 @@ def sign_build(snap_filename, key_name=None, local=False): - if not repo.is_package_installed('snapd'): + if not repo.Repo.is_package_installed('snapd'): raise EnvironmentError( 'The snapd package is not installed. In order to use ' '`sign-build`, you must run `apt install snapd`.') @@ -411,8 +411,7 @@ def push(snap_filename, release_channels=None): """Push a snap_filename to the store. - If the DELTA_UPLOADS_EXPERIMENTAL environment variable is set - and a cached snap is available, a delta will be generated from + If a cached snap is available, a delta will be generated from the cached snap to the new target snap and uploaded instead. In the case of a delta processing or upload failure, push will fall back to uploading the full snap. @@ -438,8 +437,7 @@ sha3_384_available = hasattr(hashlib, 'sha3_384') - if (os.environ.get('DELTA_UPLOADS_EXPERIMENTAL') and - sha3_384_available and source_snap): + if sha3_384_available and source_snap: try: result = _push_delta(snap_name, snap_filename, source_snap) except StoreDeltaApplicationError as e: @@ -461,10 +459,9 @@ logger.info('Revision {!r} of {!r} created.'.format( result['revision'], snap_name)) - if os.environ.get('DELTA_UPLOADS_EXPERIMENTAL'): - snap_cache.cache(snap_filename=snap_filename) - snap_cache.prune(deb_arch=arch, - keep_hash=calculate_sha3_384(snap_filename)) + snap_cache.cache(snap_filename=snap_filename) + snap_cache.prune(deb_arch=arch, + keep_hash=calculate_sha3_384(snap_filename)) else: logger.info('Pushing {!r}'.format(snap_name)) @@ -558,14 +555,11 @@ with _requires_login(): channels = store.release(snap_name, revision, release_channels) channel_map_tree = channels.get('channel_map_tree', {}) - for track, track_data in channel_map_tree.items(): - for series, series_data in track_data.items(): - for arch, channel_map in series_data.items(): - # This does not look good in green so we print instead - tabulated_channels = _tabulated_channel_map_tree( - track, series, arch, channel_map) - print(tabulated_channels) + # This does not look good in green so we print instead + tabulated_channels = _tabulated_channel_map_tree( + channel_map_tree) + print(tabulated_channels) if 'opened_channels' in channels: logger.info( @@ -573,28 +567,46 @@ channels['opened_channels'])) -def _tabulated_channel_map_tree(track, series, arch, channel_map): +def _tabulated_channel_map_tree(channel_map_tree): + """Tabulate channel map (LTS Channel channel-maps)""" - def _format_tree(channel_map, track, series): + def _format_tree(channel_maps, track, series): + arches = [] + + for arch, channel_map in sorted(channel_maps.items()): + arches += [ + (printable_arch, ) + + _get_text_for_channel(channel) + for (printable_arch, channel) in zip( + [arch] + [''] * len(channel_map), + channel_map + ) + ] + return [ - (printable_arch, printable_track, printable_series) + - _get_text_for_channel(channel) - for (printable_arch, printable_track, - printable_series, channel) in zip( - [track] + [''] * len(channel_map), - [arch] + [''] * len(channel_map), - [series] + [''] * len(channel_map), - channel_map + (printable_arch,) + printable_track + for (printable_arch, printable_track) in zip( + [track] + [''] * len(arches), + arches, ) ] - parsed_channels = [ - channel - for channel in _format_tree(channel_map, track, series) - ] + data = [] + for track, track_data in sorted(channel_map_tree.items()): + channel_maps = {} + for series, series_data in track_data.items(): + for arch, channel_map in series_data.items(): + channel_maps[arch] = channel_map + parsed_channels = [ + channel + for channel in _format_tree(channel_maps, track, series) + ] + data += parsed_channels + + headers = ['Track', 'Arch', 'Channel', 'Version', 'Revision'] return tabulate( - parsed_channels, numalign='left', - headers=['Track', 'Arch', 'Series', 'Channel', 'Version', 'Revision'], + data, numalign='left', + headers=headers, tablefmt='plain' ) @@ -634,9 +646,9 @@ 'Make sure the logged in account has upload permissions on ' '\'{}\' in series \'{}\'.'.format(snap_name, snap_series)) - closed_channels, status = store.close_channels(snap_id, channel_names) + closed_channels, c_m_tree = store.close_channels(snap_id, channel_names) - tabulated_status = _tabulated_status(status) + tabulated_status = _tabulated_channel_map_tree(c_m_tree) print(tabulated_status) print() @@ -677,7 +689,9 @@ with _requires_login(): status = store.get_snap_status(snap_name, series, arch) - tabulated_status = _tabulated_status(status) + channel_map_tree = status.get('channel_map_tree', {}) + # This does not look good in green so we print instead + tabulated_status = _tabulated_channel_map_tree(channel_map_tree) print(tabulated_status) @@ -687,17 +701,17 @@ for channel in channels) or '-' -def history(snap_name, series, arch): +def revisions(snap_name, series, arch): store = storeapi.StoreClient() with _requires_login(): - history = store.get_snap_history(snap_name, series, arch) + revisions = store.get_snap_revisions(snap_name, series, arch) parsed_revisions = [ (rev['revision'], rev['timestamp'], rev['arch'], rev['version'], _get_text_for_current_channels( rev['channels'], rev['current_channels'])) - for rev in history] + for rev in revisions] tabulated_revisions = tabulate( parsed_revisions, numalign='left', headers=['Rev.', 'Uploaded', 'Arch', 'Version', 'Channels'], diff -Nru snapcraft-2.27.1+17.04/snapcraft/tests/commands/test_cleanbuild.py snapcraft-2.28+17.04/snapcraft/tests/commands/test_cleanbuild.py --- snapcraft-2.27.1+17.04/snapcraft/tests/commands/test_cleanbuild.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/tests/commands/test_cleanbuild.py 2017-03-23 18:31:04.000000000 +0000 @@ -1,6 +1,6 @@ # -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*- # -# Copyright (C) 2016 Canonical Ltd +# Copyright (C) 2016-2017 Canonical Ltd # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License version 3 as @@ -17,13 +17,14 @@ import logging import os import tarfile -from unittest import mock from testtools.matchers import Contains +from unittest import mock import fixtures from snapcraft.main import main from snapcraft import tests +from snapcraft.tests.test_lxd import check_output_side_effect class CleanBuildCommandTestCase(tests.TestCase): @@ -41,16 +42,26 @@ plugin: nil """ + def setUp(self): + super().setUp() + patcher = mock.patch('snapcraft.internal.lxd.check_call') + self.check_call_mock = patcher.start() + self.addCleanup(patcher.stop) + + patcher = mock.patch('snapcraft.internal.lxd.check_output') + self.check_output_mock = patcher.start() + self.check_output_mock.side_effect = check_output_side_effect() + self.addCleanup(patcher.stop) + + patcher = mock.patch('snapcraft.internal.lxd.sleep', lambda _: None) + patcher.start() + self.addCleanup(patcher.stop) + def make_snapcraft_yaml(self, n=1): super().make_snapcraft_yaml(self.yaml_template) self.state_dir = os.path.join(self.parts_dir, 'part1', 'state') - @mock.patch('snapcraft.internal.lxd.sleep', lambda _: None) - @mock.patch('snapcraft.internal.lxd.check_call') - @mock.patch('snapcraft.internal.repo.is_package_installed') - def test_cleanbuild(self, mock_installed, mock_call): - mock_installed.return_value = True - + def test_cleanbuild(self): fake_logger = fixtures.FakeLogger(level=logging.INFO) self.useFixture(fake_logger) @@ -82,7 +93,6 @@ self.assertIn( 'Setting up container with project assets\n' - 'Copying snapcraft cache into container\n' 'Waiting for a network connection...\n' 'Network connection established\n' 'Retrieved snap-test_1.0_amd64.snap\n', @@ -106,12 +116,15 @@ Contains(os.path.join('.', 'snap', 'snapcraft.yaml')), 'snap/snapcraft unexpectedly excluded from tarball') - @mock.patch('snapcraft.internal.repo.is_package_installed') - def test_no_lxd(self, mock_installed): + def test_no_lxd(self): fake_logger = fixtures.FakeLogger(level=logging.ERROR) self.useFixture(fake_logger) - mock_installed.return_value = False + self.make_snapcraft_yaml() + + self.check_output_mock.side_effect = check_output_side_effect( + fail_on_default=True) + raised = self.assertRaises( SystemExit, main, ['cleanbuild']) @@ -120,8 +133,8 @@ self.assertEqual(1, raised.code) self.assertEqual( fake_logger.output, - 'The lxd package is not installed, in order to use `cleanbuild` ' - 'you must install lxd onto your system. Refer to the ' - '"Ubuntu Desktop and Ubuntu Server" section on ' - 'https://linuxcontainers.org/lxd/getting-started-cli/' - '#ubuntu-desktop-and-ubuntu-server to enable a proper setup.\n') + 'You must have LXD installed in order to use cleanbuild. ' + 'However, it is either not installed or not configured ' + 'properly.\n' + 'Refer to the documentation at ' + 'https://linuxcontainers.org/lxd/getting-started-cli.\n') diff -Nru snapcraft-2.27.1+17.04/snapcraft/tests/commands/test_close.py snapcraft-2.28+17.04/snapcraft/tests/commands/test_close.py --- snapcraft-2.27.1+17.04/snapcraft/tests/commands/test_close.py 2017-01-28 06:12:26.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/tests/commands/test_close.py 2017-03-22 12:31:58.000000000 +0000 @@ -62,27 +62,31 @@ } } closed_channels = ['beta'] - channel_maps = { - 'amd64': [ - {'channel': 'stable', 'info': 'none'}, - {'channel': 'candidate', 'info': 'none'}, - {'channel': 'beta', 'info': 'specific', - 'version': '1.1', 'revision': 42}, - {'channel': 'edge', 'info': 'tracking'} - ], + channel_map_tree = { + 'latest': { + '16': { + 'amd64': [ + {'channel': 'stable', 'info': 'none'}, + {'channel': 'candidate', 'info': 'none'}, + {'channel': 'beta', 'info': 'specific', + 'version': '1.1', 'revision': 42}, + {'channel': 'edge', 'info': 'tracking'} + ], + } + } } mock_close_channels.side_effect = [ - (closed_channels, channel_maps), + (closed_channels, channel_map_tree), ] main(['close', 'basic', 'beta']) self.assertEqual([ - 'Arch Channel Version Revision', - 'amd64 stable - -', - ' candidate - -', - ' beta 1.1 42', - ' edge ^ ^', + 'Track Arch Channel Version Revision', + 'latest amd64 stable - -', + ' candidate - -', + ' beta 1.1 42', + ' edge ^ ^', '', '\x1b[0;32mThe beta channel is now closed.\x1b[0m' ], self.fake_terminal.getvalue().splitlines()) @@ -97,27 +101,31 @@ } } closed_channels = ['beta', 'edge'] - channel_maps = { - 'amd64': [ - {'channel': 'stable', 'info': 'none'}, - {'channel': 'candidate', 'info': 'specific', - 'version': '1.1', 'revision': 42}, - {'channel': 'beta', 'info': 'tracking'}, - {'channel': 'edge', 'info': 'tracking'} - ], + channel_map_tree = { + 'latest': { + '16': { + 'amd64': [ + {'channel': 'stable', 'info': 'none'}, + {'channel': 'candidate', 'info': 'specific', + 'version': '1.1', 'revision': 42}, + {'channel': 'beta', 'info': 'tracking'}, + {'channel': 'edge', 'info': 'tracking'} + ], + } + } } mock_close_channels.side_effect = [ - (closed_channels, channel_maps), + (closed_channels, channel_map_tree), ] main(['close', 'basic', 'beta', 'edge']) self.assertEqual([ - 'Arch Channel Version Revision', - 'amd64 stable - -', - ' candidate 1.1 42', - ' beta ^ ^', - ' edge ^ ^', + 'Track Arch Channel Version Revision', + 'latest amd64 stable - -', + ' candidate 1.1 42', + ' beta ^ ^', + ' edge ^ ^', '', '\x1b[0;32mThe beta and edge channels are now closed.\x1b[0m' ], self.fake_terminal.getvalue().splitlines()) @@ -132,38 +140,43 @@ } } closed_channels = ['beta'] - channel_maps = { - 'amd64': [ - {'channel': 'stable', 'info': 'none'}, - {'channel': 'candidate', 'info': 'none'}, - {'channel': 'beta', 'info': 'specific', - 'version': '1.1', 'revision': 42}, - {'channel': 'edge', 'info': 'tracking'} - ], - 'armhf': [ - {'channel': 'stable', 'info': 'none'}, - {'channel': 'beta', 'info': 'specific', - 'version': '1.2', 'revision': 24}, - {'channel': 'beta', 'info': 'tracking'}, - {'channel': 'edge', 'info': 'tracking'} - ], + channel_map_tree = { + 'latest': { + '16': { + 'amd64': [ + {'channel': 'stable', 'info': 'none'}, + {'channel': 'candidate', 'info': 'none'}, + {'channel': 'beta', 'info': 'specific', + 'version': '1.1', 'revision': 42}, + {'channel': 'edge', 'info': 'tracking'} + ], + 'armhf': [ + {'channel': 'stable', 'info': 'none'}, + {'channel': 'beta', 'info': 'specific', + 'version': '1.2', 'revision': 24}, + {'channel': 'beta', 'info': 'tracking'}, + {'channel': 'edge', 'info': 'tracking'} + ], + + } + } } mock_close_channels.side_effect = [ - (closed_channels, channel_maps), + (closed_channels, channel_map_tree), ] main(['close', 'basic', 'beta']) self.assertEqual([ - 'Arch Channel Version Revision', - 'amd64 stable - -', - ' candidate - -', - ' beta 1.1 42', - ' edge ^ ^', - 'armhf stable - -', - ' beta 1.2 24', - ' beta ^ ^', - ' edge ^ ^', + 'Track Arch Channel Version Revision', + 'latest amd64 stable - -', + ' candidate - -', + ' beta 1.1 42', + ' edge ^ ^', + ' armhf stable - -', + ' beta 1.2 24', + ' beta ^ ^', + ' edge ^ ^', '', '\x1b[0;32mThe beta channel is now closed.\x1b[0m' ], self.fake_terminal.getvalue().splitlines()) diff -Nru snapcraft-2.27.1+17.04/snapcraft/tests/commands/test_create_key.py snapcraft-2.28+17.04/snapcraft/tests/commands/test_create_key.py --- snapcraft-2.27.1+17.04/snapcraft/tests/commands/test_create_key.py 2017-01-28 06:12:26.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/tests/commands/test_create_key.py 2017-03-22 12:31:58.000000000 +0000 @@ -39,7 +39,7 @@ @mock.patch('subprocess.check_call') @mock.patch('subprocess.check_output') - @mock.patch('snapcraft.internal.repo.is_package_installed') + @mock.patch('snapcraft.internal.repo.Repo.is_package_installed') def test_create_key_snapd_not_installed(self, mock_installed, mock_check_output, mock_check_call): @@ -58,7 +58,7 @@ @mock.patch('subprocess.check_call') @mock.patch('subprocess.check_output') - @mock.patch('snapcraft.internal.repo.is_package_installed') + @mock.patch('snapcraft.internal.repo.Repo.is_package_installed') def test_create_key_already_exists(self, mock_installed, mock_check_output, mock_check_call): mock_installed.return_value = True @@ -76,7 +76,7 @@ @mock.patch('subprocess.check_call') @mock.patch.object(storeapi.SCAClient, 'get_account_information') @mock.patch('subprocess.check_output') - @mock.patch('snapcraft.internal.repo.is_package_installed') + @mock.patch('snapcraft.internal.repo.Repo.is_package_installed') def test_create_key_already_registered(self, mock_installed, mock_check_output, mock_get_account_information, @@ -107,7 +107,7 @@ @mock.patch('subprocess.check_call') @mock.patch.object(storeapi.SCAClient, 'get_account_information') @mock.patch('subprocess.check_output') - @mock.patch('snapcraft.internal.repo.is_package_installed') + @mock.patch('snapcraft.internal.repo.Repo.is_package_installed') def test_create_key_successfully(self, mock_installed, mock_check_output, mock_get_account_information, mock_check_call): @@ -133,7 +133,7 @@ @mock.patch('subprocess.check_call') @mock.patch.object(storeapi.SCAClient, 'get_account_information') @mock.patch('subprocess.check_output') - @mock.patch('snapcraft.internal.repo.is_package_installed') + @mock.patch('snapcraft.internal.repo.Repo.is_package_installed') def test_create_key_without_login(self, mock_installed, mock_check_output, mock_get_account_information, mock_check_call): diff -Nru snapcraft-2.27.1+17.04/snapcraft/tests/commands/test_history.py snapcraft-2.28+17.04/snapcraft/tests/commands/test_history.py --- snapcraft-2.27.1+17.04/snapcraft/tests/commands/test_history.py 2017-01-28 06:12:26.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/tests/commands/test_history.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,191 +0,0 @@ -# -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*- -# -# Copyright (C) 2016 Canonical Ltd -# -# This program is free software: you can redistribute it and/or modify -# it under the terms of the GNU General Public License version 3 as -# published by the Free Software Foundation. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with this program. If not, see . - -import logging -from unittest import mock - -import docopt -import fixtures - -from snapcraft import ( - storeapi, - tests -) -from snapcraft.main import main -from snapcraft.tests import fixture_setup - - -class HistoryCommandTestCase(tests.TestCase): - - def setUp(self): - super().setUp() - self.fake_logger = fixtures.FakeLogger(level=logging.INFO) - self.useFixture(self.fake_logger) - self.expected = [{ - 'series': ['16'], - 'channels': [], - 'version': '2.0.1', - 'timestamp': '2016-09-27T19:23:40Z', - 'current_channels': ['beta', 'edge'], - 'arch': 'i386', - 'revision': 2 - }, { - 'series': ['16'], - 'channels': ['stable', 'edge'], - 'version': '2.0.2', - 'timestamp': '2016-09-27T18:38:43Z', - 'current_channels': ['stable', 'candidate', 'beta'], - 'arch': 'amd64', - 'revision': 1, - }] - - def test_history_without_snap_raises_exception(self): - raised = self.assertRaises( - docopt.DocoptExit, - main, ['history']) - - self.assertIn('Usage:', str(raised)) - - def test_history_with_no_permissions(self): - self.assertRaises( - SystemExit, - main, ['history', 'snap-test']) - - self.assertIn( - 'No valid credentials found. Have you run "snapcraft login"?', - self.fake_logger.output) - - @mock.patch.object(storeapi.StoreClient, 'get_account_information') - def test_history_with_3rd_party_snap(self, mock_account_api): - mock_account_api.return_value = {} - - self.assertRaises( - SystemExit, - main, ['history', 'snap-test']) - - self.assertIn( - "Snap 'snap-test' was not found in '16' series.", - self.fake_logger.output) - - @mock.patch.object(storeapi.StoreClient, 'get_account_information') - def test_history_with_3rd_party_snap_by_arch(self, mock_account_api): - mock_account_api.return_value = {} - - self.assertRaises( - SystemExit, - main, ['history', 'snap-test', '--arch=arm64']) - - self.assertIn( - "Snap 'snap-test' for 'arm64' was not found in '16' series.", - self.fake_logger.output) - - @mock.patch.object(storeapi.StoreClient, 'get_account_information') - def test_history_with_3rd_party_snap_by_series(self, mock_account_api): - mock_account_api.return_value = {} - - self.assertRaises( - SystemExit, - main, ['history', 'snap-test', '--series=15']) - - self.assertIn( - "Snap 'snap-test' was not found in '15' series.", - self.fake_logger.output) - - @mock.patch.object(storeapi.SCAClient, 'snap_history') - @mock.patch.object(storeapi.StoreClient, 'get_account_information') - def test_history_by_unknown_arch(self, mock_account_api, mock_history): - mock_history.return_value = {} - - self.assertRaises( - SystemExit, - main, ['history', 'snap-test', '--arch=some-arch']) - - self.assertIn( - "Snap 'snap-test' for 'some-arch' was not found in '16' series.", - self.fake_logger.output) - - @mock.patch.object(storeapi.SCAClient, 'snap_history') - @mock.patch.object(storeapi.StoreClient, 'get_account_information') - def test_history_by_unknown_series(self, mock_account_api, mock_history): - mock_history.return_value = {} - - self.assertRaises( - SystemExit, - main, ['history', 'snap-test', '--series=some-series']) - - self.assertIn( - "Snap 'snap-test' was not found in 'some-series' series.", - self.fake_logger.output) - - @mock.patch.object(storeapi.StoreClient, 'get_snap_history') - @mock.patch.object(storeapi.StoreClient, 'get_account_information') - def test_history(self, mock_account_api, mock_history): - fake_terminal = fixture_setup.FakeTerminal() - self.useFixture(fake_terminal) - - mock_history.return_value = self.expected - - main(['history', 'snap-test']) - - mock_history.assert_called_once_with('snap-test', '16', None) - - terminal_output = fake_terminal.getvalue() - expected_output = [ - 'Rev. Uploaded Arch Version Channels', - '2 2016-09-27T19:23:40Z i386 2.0.1 -', - '1 2016-09-27T18:38:43Z amd64 2.0.2 stable*, edge' - ] - self.assertEqual(expected_output, terminal_output.splitlines()) - - @mock.patch.object(storeapi.StoreClient, 'get_snap_history') - @mock.patch.object(storeapi.StoreClient, 'get_account_information') - def test_history_by_arch(self, mock_account_api, mock_history): - fake_terminal = fixture_setup.FakeTerminal() - self.useFixture(fake_terminal) - - mock_history.return_value = [ - rev for rev in self.expected if rev['arch'] == 'amd64'] - - main(['history', 'snap-test', '--arch=amd64']) - - mock_history.assert_called_once_with('snap-test', '16', 'amd64') - - terminal_output = fake_terminal.getvalue() - expected_output = [ - 'Rev. Uploaded Arch Version Channels', - '1 2016-09-27T18:38:43Z amd64 2.0.2 stable*, edge' - ] - self.assertEqual(expected_output, terminal_output.splitlines()) - - @mock.patch.object(storeapi.StoreClient, 'get_snap_history') - @mock.patch.object(storeapi.StoreClient, 'get_account_information') - def test_history_by_series(self, mock_account_api, mock_history): - fake_terminal = fixture_setup.FakeTerminal() - self.useFixture(fake_terminal) - - mock_history.return_value = self.expected - - main(['history', 'snap-test', '--series=16']) - - mock_history.assert_called_once_with('snap-test', '16', None) - - terminal_output = fake_terminal.getvalue() - expected_output = [ - 'Rev. Uploaded Arch Version Channels', - '2 2016-09-27T19:23:40Z i386 2.0.1 -', - '1 2016-09-27T18:38:43Z amd64 2.0.2 stable*, edge' - ] - self.assertEqual(expected_output, terminal_output.splitlines()) diff -Nru snapcraft-2.27.1+17.04/snapcraft/tests/commands/test_list_keys.py snapcraft-2.28+17.04/snapcraft/tests/commands/test_list_keys.py --- snapcraft-2.27.1+17.04/snapcraft/tests/commands/test_list_keys.py 2017-01-28 06:12:26.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/tests/commands/test_list_keys.py 2017-03-22 12:31:58.000000000 +0000 @@ -46,7 +46,7 @@ self.useFixture(self.fake_terminal) @mock.patch('subprocess.check_output') - @mock.patch('snapcraft.internal.repo.is_package_installed') + @mock.patch('snapcraft.internal.repo.Repo.is_package_installed') def test_list_keys_snapd_not_installed(self, mock_installed, mock_check_output): mock_installed.return_value = False @@ -62,7 +62,7 @@ 'The snapd package is not installed.', self.fake_logger.output) @mock.patch('subprocess.check_output') - @mock.patch('snapcraft.internal.repo.is_package_installed') + @mock.patch('snapcraft.internal.repo.Repo.is_package_installed') def test_list_keys_without_login(self, mock_installed, mock_check_output): mock_installed.return_value = True mock_check_output.side_effect = mock_snap_output @@ -79,7 +79,7 @@ @mock.patch.object(storeapi.SCAClient, 'get_account_information') @mock.patch('subprocess.check_output') - @mock.patch('snapcraft.internal.repo.is_package_installed') + @mock.patch('snapcraft.internal.repo.Repo.is_package_installed') def test_list_keys_successfully(self, mock_installed, mock_check_output, mock_get_account_information): mock_installed.return_value = True diff -Nru snapcraft-2.27.1+17.04/snapcraft/tests/commands/test_list_revisions.py snapcraft-2.28+17.04/snapcraft/tests/commands/test_list_revisions.py --- snapcraft-2.27.1+17.04/snapcraft/tests/commands/test_list_revisions.py 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/tests/commands/test_list_revisions.py 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,220 @@ +# -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*- +# +# Copyright (C) 2016 Canonical Ltd +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License version 3 as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . + +import logging +from unittest import mock + +import docopt +import fixtures +from testtools.matchers import Contains + +from snapcraft import ( + storeapi, + tests +) +from snapcraft.main import main +from snapcraft.tests import fixture_setup + + +class RevisionsCommandBaseTestCase(tests.TestCase): + + def setUp(self): + super().setUp() + self.fake_logger = fixtures.FakeLogger(level=logging.INFO) + self.useFixture(self.fake_logger) + self.expected = [{ + 'series': ['16'], + 'channels': [], + 'version': '2.0.1', + 'timestamp': '2016-09-27T19:23:40Z', + 'current_channels': ['beta', 'edge'], + 'arch': 'i386', + 'revision': 2 + }, { + 'series': ['16'], + 'channels': ['stable', 'edge'], + 'version': '2.0.2', + 'timestamp': '2016-09-27T18:38:43Z', + 'current_channels': ['stable', 'candidate', 'beta'], + 'arch': 'amd64', + 'revision': 1, + }] + + +class RevisionsCommandTestCase(RevisionsCommandBaseTestCase): + + scenarios = [ + ('list-revisions', dict(command='list-revisions')), + ('revisions', dict(command='revisions')), + ] + + def test_revisions_without_snap_raises_exception(self): + raised = self.assertRaises( + docopt.DocoptExit, + main, [self.command]) + + self.assertIn('Usage:', str(raised)) + + def test_revisions_with_no_permissions(self): + self.assertRaises( + SystemExit, + main, [self.command, 'snap-test']) + + self.assertIn( + 'No valid credentials found. Have you run "snapcraft login"?', + self.fake_logger.output) + + @mock.patch.object(storeapi.StoreClient, 'get_account_information') + def test_revisions_with_3rd_party_snap(self, mock_account_api): + mock_account_api.return_value = {} + + self.assertRaises( + SystemExit, + main, [self.command, 'snap-test']) + + self.assertIn( + "Snap 'snap-test' was not found in '16' series.", + self.fake_logger.output) + + @mock.patch.object(storeapi.StoreClient, 'get_account_information') + def test_revisions_with_3rd_party_snap_by_arch(self, mock_account_api): + mock_account_api.return_value = {} + + self.assertRaises( + SystemExit, + main, [self.command, 'snap-test', '--arch=arm64']) + + self.assertIn( + "Snap 'snap-test' for 'arm64' was not found in '16' series.", + self.fake_logger.output) + + @mock.patch.object(storeapi.StoreClient, 'get_account_information') + def test_revisions_with_3rd_party_snap_by_series(self, mock_account_api): + mock_account_api.return_value = {} + + self.assertRaises( + SystemExit, + main, [self.command, 'snap-test', '--series=15']) + + self.assertIn( + "Snap 'snap-test' was not found in '15' series.", + self.fake_logger.output) + + @mock.patch.object(storeapi.SCAClient, 'snap_revisions') + @mock.patch.object(storeapi.StoreClient, 'get_account_information') + def test_revisions_by_unknown_arch(self, mock_account_api, mock_revisions): + mock_revisions.return_value = {} + + self.assertRaises( + SystemExit, + main, [self.command, 'snap-test', '--arch=some-arch']) + + self.assertIn( + "Snap 'snap-test' for 'some-arch' was not found in '16' series.", + self.fake_logger.output) + + @mock.patch.object(storeapi.SCAClient, 'snap_revisions') + @mock.patch.object(storeapi.StoreClient, 'get_account_information') + def test_revisions_by_unknown_series(self, + mock_account_api, mock_revisions): + mock_revisions.return_value = {} + + self.assertRaises( + SystemExit, + main, [self.command, 'snap-test', '--series=some-series']) + + self.assertIn( + "Snap 'snap-test' was not found in 'some-series' series.", + self.fake_logger.output) + + @mock.patch.object(storeapi.StoreClient, 'get_snap_revisions') + @mock.patch.object(storeapi.StoreClient, 'get_account_information') + def test_revisions(self, mock_account_api, mock_revisions): + fake_terminal = fixture_setup.FakeTerminal() + self.useFixture(fake_terminal) + + mock_revisions.return_value = self.expected + + main([self.command, 'snap-test']) + + mock_revisions.assert_called_once_with('snap-test', '16', None) + + terminal_output = fake_terminal.getvalue() + expected_output = [ + 'Rev. Uploaded Arch Version Channels', + '2 2016-09-27T19:23:40Z i386 2.0.1 -', + '1 2016-09-27T18:38:43Z amd64 2.0.2 stable*, edge' + ] + self.assertEqual(expected_output, terminal_output.splitlines()) + + @mock.patch.object(storeapi.StoreClient, 'get_snap_revisions') + @mock.patch.object(storeapi.StoreClient, 'get_account_information') + def test_revisions_by_arch(self, mock_account_api, mock_revisions): + fake_terminal = fixture_setup.FakeTerminal() + self.useFixture(fake_terminal) + + mock_revisions.return_value = [ + rev for rev in self.expected if rev['arch'] == 'amd64'] + + main([self.command, 'snap-test', '--arch=amd64']) + + mock_revisions.assert_called_once_with('snap-test', '16', 'amd64') + + terminal_output = fake_terminal.getvalue() + expected_output = [ + 'Rev. Uploaded Arch Version Channels', + '1 2016-09-27T18:38:43Z amd64 2.0.2 stable*, edge' + ] + self.assertEqual(expected_output, terminal_output.splitlines()) + + @mock.patch.object(storeapi.StoreClient, 'get_snap_revisions') + @mock.patch.object(storeapi.StoreClient, 'get_account_information') + def test_revisions_by_series(self, mock_account_api, mock_revisions): + fake_terminal = fixture_setup.FakeTerminal() + self.useFixture(fake_terminal) + + mock_revisions.return_value = self.expected + + main([self.command, 'snap-test', '--series=16']) + + mock_revisions.assert_called_once_with('snap-test', '16', None) + + terminal_output = fake_terminal.getvalue() + expected_output = [ + 'Rev. Uploaded Arch Version Channels', + '2 2016-09-27T19:23:40Z i386 2.0.1 -', + '1 2016-09-27T18:38:43Z amd64 2.0.2 stable*, edge' + ] + self.assertEqual(expected_output, terminal_output.splitlines()) + + +class DeprecatedHistoryCommandTestCase(RevisionsCommandBaseTestCase): + + @mock.patch.object(storeapi.StoreClient, 'get_snap_revisions') + @mock.patch.object(storeapi.StoreClient, 'get_account_information') + def test_history_with_deprecation_message(self, mock_account_api, + mock_revisions): + fake_terminal = fixture_setup.FakeTerminal() + self.useFixture(fake_terminal) + + mock_revisions.return_value = self.expected + + main(['history', 'snap-test', '--series=16']) + + self.assertThat( + fake_terminal.getvalue(), Contains( + "DEPRECATED: The 'history' command has been replaced by " + "'list-revisions'.")) diff -Nru snapcraft-2.27.1+17.04/snapcraft/tests/commands/test_pull.py snapcraft-2.28+17.04/snapcraft/tests/commands/test_pull.py --- snapcraft-2.27.1+17.04/snapcraft/tests/commands/test_pull.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/tests/commands/test_pull.py 2017-03-22 12:31:58.000000000 +0000 @@ -105,36 +105,38 @@ self.assertFalse(os.path.exists(parts[i]['state_dir']), 'Expected for only to be a state file for pull1') - @mock.patch('snapcraft.repo.Ubuntu.get') - @mock.patch('snapcraft.repo.Ubuntu.unpack') - def test_pull_stage_packages_without_geoip(self, mock_get, mock_unpack): + @mock.patch('snapcraft.repo.Repo.get') + @mock.patch('snapcraft.repo.Repo.unpack') + def test_pull_stage_packages_without_geoip(self, mock_unpack, mock_get): yaml_part = """ pull{:d}: plugin: nil stage-packages: ['mir']""" self.make_snapcraft_yaml(n=3, yaml_part=yaml_part) + mock_get.return_value = '[mir=0.0]' project_options = mock.Mock(spec=snapcraft.ProjectOptions) project_options = main(['pull', 'pull1']) self.assertFalse(project_options.use_geoip) - @mock.patch('snapcraft.repo.Ubuntu.get') - @mock.patch('snapcraft.repo.Ubuntu.unpack') - def test_pull_stage_packages_with_geoip(self, mock_get, mock_unpack): + @mock.patch('snapcraft.repo.Repo.get') + @mock.patch('snapcraft.repo.Repo.unpack') + def test_pull_stage_packages_with_geoip(self, mock_unpack, mock_get): yaml_part = """ pull{:d}: plugin: nil stage-packages: ['mir']""" self.make_snapcraft_yaml(n=3, yaml_part=yaml_part) + mock_get.return_value = '[mir=0.0]' project_options = main(['pull', 'pull1', '--enable-geoip']) self.assertTrue(project_options.use_geoip) - @mock.patch('snapcraft.repo.Ubuntu.get') - @mock.patch('snapcraft.repo.Ubuntu.unpack') + @mock.patch('snapcraft.repo.Repo.get') + @mock.patch('snapcraft.repo.Repo.unpack') def test_pull_multiarch_stage_package(self, mock_unpack, mock_get): yaml_part = """ pull{:d}: plugin: nil @@ -142,6 +144,8 @@ self.make_snapcraft_yaml(n=3, yaml_part=yaml_part) - main(['pull', 'pull1']) + mock_get.return_value = '[mir=0.0]' + + main(['--debug', 'pull', 'pull1']) mock_get.assert_called_once_with({'mir:arch'}) diff -Nru snapcraft-2.27.1+17.04/snapcraft/tests/commands/test_push.py snapcraft-2.28+17.04/snapcraft/tests/commands/test_push.py --- snapcraft-2.27.1+17.04/snapcraft/tests/commands/test_push.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/tests/commands/test_push.py 2017-03-22 12:31:58.000000000 +0000 @@ -336,7 +336,7 @@ 'url': '/fake/url', 'revision': self.new_snap_revision, } - patcher = mock.patch.object(storeapi.StoreClient, 'get_snap_history') + patcher = mock.patch.object(storeapi.StoreClient, 'get_snap_revisions') mock_release = patcher.start() mock_release.return_value = [self.latest_snap_revision] self.addCleanup(patcher.stop) @@ -350,8 +350,6 @@ def test_push_revision_cached_with_experimental_deltas(self): self.useFixture(fixture_setup.FakeTerminal()) - if self.enable_deltas: - self.useFixture(fixture_setup.DeltaUploads()) # Create a snap main(['init']) @@ -373,15 +371,10 @@ cached_snap = os.path.join( snap_cache, file_utils.calculate_sha3_384(snap_file)) - if self.enable_deltas: - self.assertThat(cached_snap, FileExists()) - else: - self.assertThat(cached_snap, Not(FileExists())) + self.assertThat(cached_snap, FileExists()) def test_push_revision_uses_available_delta(self): self.useFixture(fixture_setup.FakeTerminal()) - if self.enable_deltas: - self.useFixture(fixture_setup.DeltaUploads()) # Create a source snap main(['init']) @@ -401,14 +394,10 @@ main(['push', new_snap_file]) _, kwargs = self.mock_upload.call_args - if self.enable_deltas: - self.assertEqual(kwargs.get('delta_format'), 'xdelta3') - else: - self.assertIsNone(kwargs.get('delta_format')) + self.assertEqual(kwargs.get('delta_format'), 'xdelta3') def test_push_with_upload_failure_falls_back(self): self.useFixture(fixture_setup.FakeTerminal()) - self.useFixture(fixture_setup.DeltaUploads()) # Create a source snap to delta from main(['init']) @@ -466,7 +455,6 @@ def test_push_revision_prune_snap_cache(self): self.useFixture(fixture_setup.FakeTerminal()) - self.useFixture(fixture_setup.DeltaUploads()) snap_revision = 9 @@ -474,7 +462,7 @@ patcher.start() self.addCleanup(patcher.stop) - patcher = mock.patch.object(storeapi.StoreClient, 'get_snap_history') + patcher = mock.patch.object(storeapi.StoreClient, 'get_snap_revisions') mock_release = patcher.start() self.addCleanup(patcher.stop) mock_release.return_value = [snap_revision] diff -Nru snapcraft-2.27.1+17.04/snapcraft/tests/commands/test_register_key.py snapcraft-2.28+17.04/snapcraft/tests/commands/test_register_key.py --- snapcraft-2.27.1+17.04/snapcraft/tests/commands/test_register_key.py 2017-01-28 06:12:26.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/tests/commands/test_register_key.py 2017-03-22 12:31:58.000000000 +0000 @@ -43,7 +43,7 @@ self.useFixture(self.fake_terminal) @mock.patch('subprocess.check_output') - @mock.patch('snapcraft.internal.repo.is_package_installed') + @mock.patch('snapcraft.internal.repo.Repo.is_package_installed') def test_register_key_snapd_not_installed(self, mock_installed, mock_check_output): mock_installed.return_value = False @@ -64,7 +64,7 @@ @mock.patch('subprocess.check_output') @mock.patch('getpass.getpass') @mock.patch('builtins.input') - @mock.patch('snapcraft.internal.repo.is_package_installed') + @mock.patch('snapcraft.internal.repo.Repo.is_package_installed') def test_register_key_successfully(self, mock_installed, mock_input, mock_getpass, mock_check_output, mock_login, @@ -102,7 +102,7 @@ @mock.patch('subprocess.check_output') @mock.patch('builtins.input') - @mock.patch('snapcraft.internal.repo.is_package_installed') + @mock.patch('snapcraft.internal.repo.Repo.is_package_installed') def test_register_key_no_keys(self, mock_installed, mock_input, mock_check_output): mock_installed.return_value = True @@ -118,7 +118,7 @@ @mock.patch('subprocess.check_output') @mock.patch('builtins.input') - @mock.patch('snapcraft.internal.repo.is_package_installed') + @mock.patch('snapcraft.internal.repo.Repo.is_package_installed') def test_register_key_no_keys_null(self, mock_installed, mock_input, mock_check_output): # Some versions of snapd serialise an empty list as "null" rather @@ -136,7 +136,7 @@ @mock.patch('subprocess.check_output') @mock.patch('builtins.input') - @mock.patch('snapcraft.internal.repo.is_package_installed') + @mock.patch('snapcraft.internal.repo.Repo.is_package_installed') def test_register_key_no_keys_with_name(self, mock_installed, mock_input, mock_check_output): mock_installed.return_value = True @@ -158,7 +158,7 @@ @mock.patch('subprocess.check_output') @mock.patch('getpass.getpass') @mock.patch('builtins.input') - @mock.patch('snapcraft.internal.repo.is_package_installed') + @mock.patch('snapcraft.internal.repo.Repo.is_package_installed') def test_register_key_login_failed(self, mock_installed, mock_input, mock_getpass, mock_check_output, mock_login, @@ -187,7 +187,7 @@ @mock.patch('subprocess.check_output') @mock.patch('getpass.getpass') @mock.patch('builtins.input') - @mock.patch('snapcraft.internal.repo.is_package_installed') + @mock.patch('snapcraft.internal.repo.Repo.is_package_installed') def test_register_key_account_info_failed(self, mock_installed, mock_input, mock_getpass, mock_check_output, mock_login, @@ -220,7 +220,7 @@ @mock.patch('subprocess.check_output') @mock.patch('getpass.getpass') @mock.patch('builtins.input') - @mock.patch('snapcraft.internal.repo.is_package_installed') + @mock.patch('snapcraft.internal.repo.Repo.is_package_installed') def test_register_key_failed(self, mock_installed, mock_input, mock_getpass, mock_check_output, mock_login, mock_get_account_information, @@ -250,7 +250,7 @@ @mock.patch('subprocess.check_output') @mock.patch('getpass.getpass') @mock.patch('builtins.input') - @mock.patch('snapcraft.internal.repo.is_package_installed') + @mock.patch('snapcraft.internal.repo.Repo.is_package_installed') def test_register_key_select_key(self, mock_installed, mock_input, mock_getpass, mock_check_output, mock_login, mock_get_account_information, diff -Nru snapcraft-2.27.1+17.04/snapcraft/tests/commands/test_release.py snapcraft-2.28+17.04/snapcraft/tests/commands/test_release.py --- snapcraft-2.27.1+17.04/snapcraft/tests/commands/test_release.py 2017-02-17 19:54:46.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/tests/commands/test_release.py 2017-03-22 12:31:58.000000000 +0000 @@ -72,11 +72,11 @@ mock_release.assert_called_once_with('nil-snap', '19', ['beta']) self.assertEqual([ - 'Track Arch Series Channel Version Revision', - 'latest amd64 16 stable - -', - ' candidate - -', - ' beta 0 19', - ' edge ^ ^', + 'Track Arch Channel Version Revision', + 'latest amd64 stable - -', + ' candidate - -', + ' beta 0 19', + ' edge ^ ^', "\x1b[0;32mThe 'beta' channel is now open.\x1b[0m", ], fake_terminal.getvalue().splitlines()) @@ -110,11 +110,11 @@ mock_release.assert_called_once_with('nil-snap', '19', ['2.1/beta']) self.assertEqual([ - 'Track Arch Series Channel Version Revision', - '2.1 amd64 16 stable - -', - ' candidate - -', - ' beta 0 19', - ' edge ^ ^', + 'Track Arch Channel Version Revision', + '2.1 amd64 stable - -', + ' candidate - -', + ' beta 0 19', + ' edge ^ ^', "\x1b[0;32mThe '2.1/beta' channel is now open.\x1b[0m", ], fake_terminal.getvalue().splitlines()) @@ -148,11 +148,11 @@ mock_release.assert_called_once_with('nil-snap', '19', ['beta']) self.assertEqual([ - 'Track Arch Series Channel Version Revision', - 'latest amd64 16 stable - -', - ' candidate - -', - ' beta 0 19', - ' edge ^ ^', + 'Track Arch Channel Version Revision', + 'latest amd64 stable - -', + ' candidate - -', + ' beta 0 19', + ' edge ^ ^', "\x1b[0;32mThe 'stable', 'beta' and 'edge' channels " "are now open.\x1b[0m", ], fake_terminal.getvalue().splitlines()) diff -Nru snapcraft-2.27.1+17.04/snapcraft/tests/commands/test_sign_build.py snapcraft-2.28+17.04/snapcraft/tests/commands/test_sign_build.py --- snapcraft-2.27.1+17.04/snapcraft/tests/commands/test_sign_build.py 2017-01-28 06:12:26.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/tests/commands/test_sign_build.py 2017-03-22 12:31:58.000000000 +0000 @@ -60,7 +60,7 @@ self.useFixture(self.snap_test) @mock.patch('subprocess.check_output') - @mock.patch('snapcraft.internal.repo.is_package_installed') + @mock.patch('snapcraft.internal.repo.Repo.is_package_installed') def test_sign_build_snapd_not_installed(self, mock_installed, mock_check_output): mock_installed.return_value = False @@ -76,7 +76,7 @@ 'The snapd package is not installed.', self.fake_logger.output) @mock.patch('subprocess.check_output') - @mock.patch('snapcraft.internal.repo.is_package_installed') + @mock.patch('snapcraft.internal.repo.Repo.is_package_installed') def test_sign_build_nonexisting_snap(self, mock_installed, mock_check_output): mock_installed.return_value = True @@ -94,7 +94,7 @@ @mock.patch.object(storeapi.SCAClient, 'get_account_information') @mock.patch('subprocess.check_output') @mock.patch('snapcraft._store._get_data_from_snap_file') - @mock.patch('snapcraft.internal.repo.is_package_installed') + @mock.patch('snapcraft.internal.repo.Repo.is_package_installed') def test_sign_build_missing_account_info( self, mock_installed, mock_get_snap_data, mock_check_output, mock_get_account_info): @@ -124,7 +124,7 @@ @mock.patch.object(storeapi.SCAClient, 'get_account_information') @mock.patch('subprocess.check_output') @mock.patch('snapcraft._store._get_data_from_snap_file') - @mock.patch('snapcraft.internal.repo.is_package_installed') + @mock.patch('snapcraft.internal.repo.Repo.is_package_installed') def test_sign_build_no_usable_keys( self, mock_installed, mock_get_snap_data, mock_check_output, mock_get_account_info): @@ -162,7 +162,7 @@ @mock.patch.object(storeapi.SCAClient, 'get_account_information') @mock.patch('subprocess.check_output') @mock.patch('snapcraft._store._get_data_from_snap_file') - @mock.patch('snapcraft.internal.repo.is_package_installed') + @mock.patch('snapcraft.internal.repo.Repo.is_package_installed') def test_sign_build_no_usable_named_key( self, mock_installed, mock_get_snap_data, mock_check_output, mock_get_account_info): @@ -200,7 +200,7 @@ @mock.patch.object(storeapi.SCAClient, 'get_account_information') @mock.patch('subprocess.check_output') @mock.patch('snapcraft._store._get_data_from_snap_file') - @mock.patch('snapcraft.internal.repo.is_package_installed') + @mock.patch('snapcraft.internal.repo.Repo.is_package_installed') def test_sign_build_unregistered_key( self, mock_installed, mock_get_snap_data, mock_check_output, mock_get_account_info): @@ -238,7 +238,7 @@ @mock.patch.object(storeapi.SCAClient, 'get_account_information') @mock.patch('subprocess.check_output') @mock.patch('snapcraft._store._get_data_from_snap_file') - @mock.patch('snapcraft.internal.repo.is_package_installed') + @mock.patch('snapcraft.internal.repo.Repo.is_package_installed') def test_sign_build_snapd_failure( self, mock_installed, mock_get_snap_data, mock_check_output, mock_get_account_info): @@ -277,7 +277,7 @@ @mock.patch.object(storeapi.SCAClient, 'get_account_information') @mock.patch('subprocess.check_output') @mock.patch('snapcraft._store._get_data_from_snap_file') - @mock.patch('snapcraft.internal.repo.is_package_installed') + @mock.patch('snapcraft.internal.repo.Repo.is_package_installed') def test_sign_build_locally_successfully( self, mock_installed, mock_get_snap_data, mock_check_output, mock_get_account_info): @@ -314,7 +314,7 @@ @mock.patch.object(storeapi.SCAClient, 'get_account_information') @mock.patch('subprocess.check_output') @mock.patch('snapcraft._store._get_data_from_snap_file') - @mock.patch('snapcraft.internal.repo.is_package_installed') + @mock.patch('snapcraft.internal.repo.Repo.is_package_installed') def test_sign_build_missing_grade( self, mock_installed, mock_get_snap_data, mock_check_output, mock_get_account_info): @@ -350,7 +350,7 @@ @mock.patch.object(storeapi.SCAClient, 'get_account_information') @mock.patch('subprocess.check_output') @mock.patch('snapcraft._store._get_data_from_snap_file') - @mock.patch('snapcraft.internal.repo.is_package_installed') + @mock.patch('snapcraft.internal.repo.Repo.is_package_installed') def test_sign_build_push_successfully( self, mock_installed, mock_get_snap_data, mock_check_output, mock_get_account_info, mock_push_snap_build): @@ -390,7 +390,7 @@ @mock.patch.object(storeapi.SCAClient, 'push_snap_build') @mock.patch.object(storeapi.SCAClient, 'get_account_information') @mock.patch('snapcraft._store._get_data_from_snap_file') - @mock.patch('snapcraft.internal.repo.is_package_installed') + @mock.patch('snapcraft.internal.repo.Repo.is_package_installed') def test_sign_build_push_existing( self, mock_installed, mock_get_snap_data, mock_get_account_info, mock_push_snap_build): diff -Nru snapcraft-2.27.1+17.04/snapcraft/tests/commands/test_status.py snapcraft-2.28+17.04/snapcraft/tests/commands/test_status.py --- snapcraft-2.27.1+17.04/snapcraft/tests/commands/test_status.py 2017-01-28 06:12:26.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/tests/commands/test_status.py 2017-03-22 12:31:58.000000000 +0000 @@ -155,7 +155,13 @@ fake_terminal = fixture_setup.FakeTerminal() self.useFixture(fake_terminal) - mock_status.return_value = self.expected + mock_status.return_value = { + 'channel_map_tree': { + 'latest': { + '16': self.expected + } + } + } main(['status', 'snap-test']) @@ -163,13 +169,13 @@ terminal_output = fake_terminal.getvalue() expected_output = [ - 'Arch Channel Version Revision', - 'amd64 stable 1.0-amd64 2', - ' beta 1.1-amd64 4', - ' edge ^ ^', - 'i386 stable - -', - ' beta - -', - ' edge 1.0-i386 3'] + 'Track Arch Channel Version Revision', + 'latest amd64 stable 1.0-amd64 2', + ' beta 1.1-amd64 4', + ' edge ^ ^', + ' i386 stable - -', + ' beta - -', + ' edge 1.0-i386 3'] self.assertEqual(expected_output, terminal_output.splitlines()) @mock.patch.object(storeapi.StoreClient, 'get_snap_status') @@ -177,7 +183,15 @@ def test_status_by_arch(self, mock_account_api, mock_status): fake_terminal = fixture_setup.FakeTerminal() self.useFixture(fake_terminal) - mock_status.return_value = {'i386': self.expected['i386']} + mock_status.return_value = { + 'channel_map_tree': { + 'latest': { + '16': { + 'i386': self.expected['i386'] + } + } + } + } main(['status', 'snap-test', '--arch=i386']) @@ -185,10 +199,10 @@ terminal_output = fake_terminal.getvalue() expected_output = [ - 'Arch Channel Version Revision', - 'i386 stable - -', - ' beta - -', - ' edge 1.0-i386 3'] + 'Track Arch Channel Version Revision', + 'latest i386 stable - -', + ' beta - -', + ' edge 1.0-i386 3'] self.assertEqual(expected_output, terminal_output.splitlines()) @mock.patch.object(storeapi.StoreClient, 'get_snap_status') @@ -196,7 +210,13 @@ def test_status_by_series(self, mock_account_api, mock_status): fake_terminal = fixture_setup.FakeTerminal() self.useFixture(fake_terminal) - mock_status.return_value = self.expected + mock_status.return_value = { + 'channel_map_tree': { + 'latest': { + '16': self.expected + } + } + } main(['status', 'snap-test', '--series=16']) @@ -204,11 +224,11 @@ terminal_output = fake_terminal.getvalue() expected_output = [ - 'Arch Channel Version Revision', - 'amd64 stable 1.0-amd64 2', - ' beta 1.1-amd64 4', - ' edge ^ ^', - 'i386 stable - -', - ' beta - -', - ' edge 1.0-i386 3'] + 'Track Arch Channel Version Revision', + 'latest amd64 stable 1.0-amd64 2', + ' beta 1.1-amd64 4', + ' edge ^ ^', + ' i386 stable - -', + ' beta - -', + ' edge 1.0-i386 3'] self.assertEqual(expected_output, terminal_output.splitlines()) diff -Nru snapcraft-2.27.1+17.04/snapcraft/tests/fake_servers.py snapcraft-2.28+17.04/snapcraft/tests/fake_servers.py --- snapcraft-2.27.1+17.04/snapcraft/tests/fake_servers.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/tests/fake_servers.py 2017-03-22 12:31:58.000000000 +0000 @@ -488,14 +488,19 @@ self.send_header('Content-Type', 'application/json') response = { 'closed_channels': channels, - 'channel_maps': { - 'amd64': [ - {'channel': 'stable', 'info': 'none'}, - {'channel': 'candidate', 'info': 'none'}, - {'channel': 'beta', 'info': 'specific', - 'version': '1.1', 'revision': 42}, - {'channel': 'edge', 'info': 'tracking'} - ] + 'channel_map_tree': { + 'latest': { + '16': { + 'amd64': [ + {'channel': 'stable', 'info': 'none'}, + {'channel': 'candidate', 'info': 'none'}, + {'channel': 'beta', 'info': 'specific', + 'version': '1.1', 'revision': 42}, + {'channel': 'edge', 'info': 'tracking'} + ] + + } + } }, } content = json.dumps(response).encode() @@ -821,8 +826,8 @@ self._handle_validation_request('no') elif parsed_path.path.startswith(snap_path): if parsed_path.path.endswith('/history'): - self._handle_snap_history() - elif parsed_path.path.endswith('/status'): + self._handle_snap_revisions() + elif parsed_path.path.endswith('/state'): self._handle_snap_status() else: logger.error( @@ -942,7 +947,7 @@ 'snaps': {'16': snaps}, }).encode()) - def _handle_snap_history(self): + def _handle_snap_revisions(self): logger.debug('Handling account request') self.send_response(200) self.send_header('Content-Type', 'application/json') @@ -980,48 +985,64 @@ self.send_header('Content-Type', 'application/json') self.end_headers() channel_map = { - 'i386': [ - { - 'info': 'none', - 'channel': 'stable' - }, - { - 'info': 'none', - 'channel': 'beta' - }, - { - 'info': 'specific', - 'version': '1.0-i386', - 'channel': 'edge', - 'revision': 3 - }, - ], - 'amd64': [ - { - 'info': 'specific', - 'version': '1.0-amd64', - 'channel': 'stable', - 'revision': 2 - }, - { - 'info': 'specific', - 'version': '1.1-amd64', - 'channel': 'beta', - 'revision': 4 - }, - { - 'info': 'tracking', - 'channel': 'edge' - }, - ], + 'channel_map_tree': { + 'latest': { + '16': { + 'i386': [ + { + 'info': 'none', + 'channel': 'stable' + }, + { + 'info': 'none', + 'channel': 'beta' + }, + { + 'info': 'specific', + 'version': '1.0-i386', + 'channel': 'edge', + 'revision': 3 + }, + ], + 'amd64': [ + { + 'info': 'specific', + 'version': '1.0-amd64', + 'channel': 'stable', + 'revision': 2 + }, + { + 'info': 'specific', + 'version': '1.1-amd64', + 'channel': 'beta', + 'revision': 4 + }, + { + 'info': 'tracking', + 'channel': 'edge' + }, + ], + + } + } + } } parsed_qs = urllib.parse.parse_qs( urllib.parse.urlparse(self.path).query) - if 'arch' in parsed_qs: - arch = parsed_qs['arch'][0] - if arch in channel_map: - output = {arch: channel_map[arch]} + if 'architecture' in parsed_qs: + arch = parsed_qs['architecture'][0] + series = channel_map['channel_map_tree']['latest']['16'] + if arch in series: + output = { + 'channel_map_tree': { + 'latest': { + '16': { + arch: series[arch] + } + } + } + } else: output = {} else: @@ -1105,6 +1126,10 @@ logger.debug( 'Handling details request for package {}, with headers {}'.format( package, self.headers)) + if 'User-Agent' not in self.headers: + self.send_response(500) + self.end_headers() + return response = self._get_details_response(package) if response is None: self.send_response(404) @@ -1151,6 +1176,10 @@ def _handle_download_request(self, snap): logger.debug('Handling download request for snap {}'.format(snap)) + if 'User-Agent' not in self.headers: + self.send_repsonse(500) + self.end_headers() + return self.send_response(200) self.send_header('Content-Type', 'application/octet-stream') self.end_headers() diff -Nru snapcraft-2.27.1+17.04/snapcraft/tests/fixture_setup.py snapcraft-2.28+17.04/snapcraft/tests/fixture_setup.py --- snapcraft-2.27.1+17.04/snapcraft/tests/fixture_setup.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/tests/fixture_setup.py 2017-03-23 12:02:09.000000000 +0000 @@ -203,6 +203,8 @@ def setUp(self): super().setUp() + # In case the variable was not set or it was empty. + self.useFixture(fixtures.EnvironmentVariable('TEST_STORE', 'fake')) self.needs_refresh = False @@ -325,7 +327,7 @@ def setUp(self): super().setUp() - test_store = os.getenv('TEST_STORE', 'fake') + test_store = os.getenv('TEST_STORE') or 'fake' if test_store == 'fake': self.useFixture(FakeStore()) self.register_delay = 0 @@ -353,14 +355,6 @@ self.user_password = os.getenv('TEST_USER_PASSWORD') -class DeltaUploads(fixtures.Fixture): - """Enable the Delta Uploads Experimental flag.""" - def setUp(self): - super().setUp() - self.useFixture(fixtures.EnvironmentVariable( - 'DELTA_UPLOADS_EXPERIMENTAL', 'True')) - - class FakePlugin(fixtures.Fixture): '''Dynamically generate a new module containing the provided plugin''' diff -Nru snapcraft-2.27.1+17.04/snapcraft/tests/__init__.py snapcraft-2.28+17.04/snapcraft/tests/__init__.py --- snapcraft-2.27.1+17.04/snapcraft/tests/__init__.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/tests/__init__.py 2017-03-22 12:31:58.000000000 +0000 @@ -1,6 +1,6 @@ # -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*- # -# Copyright (C) 2015-2016 Canonical Ltd +# Copyright (C) 2015-2017 Canonical Ltd # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License version 3 as @@ -41,7 +41,8 @@ def __init__(self, source=None, source_type=None, source_branch=None, source_tag=None, source_subdir=None, source_depth=None, - source_commit=None, disable_parallel=False): + source_commit=None, source_checksum=None, + disable_parallel=False): self.source = source self.source_type = source_type self.source_depth = source_depth diff -Nru snapcraft-2.27.1+17.04/snapcraft/tests/pluginhandler/stage_package_grammar/__init__.py snapcraft-2.28+17.04/snapcraft/tests/pluginhandler/stage_package_grammar/__init__.py --- snapcraft-2.27.1+17.04/snapcraft/tests/pluginhandler/stage_package_grammar/__init__.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/tests/pluginhandler/stage_package_grammar/__init__.py 2017-03-22 12:31:58.000000000 +0000 @@ -24,13 +24,13 @@ def setUp(self): super().setUp() - patcher = mock.patch('snapcraft.repo.Ubuntu') - self.ubuntu_mock = patcher.start() + patcher = mock.patch('snapcraft.repo.Repo') + self.repo_mock = patcher.start() self.addCleanup(patcher.stop) - self.get_mock = self.ubuntu_mock.return_value.get - self.unpack_mock = self.ubuntu_mock.return_value.unpack - self.is_valid_mock = self.ubuntu_mock.return_value.is_valid + self.get_mock = self.repo_mock.return_value.get + self.unpack_mock = self.repo_mock.return_value.unpack + self.is_valid_mock = self.repo_mock.return_value.is_valid def _is_valid(package_name): return 'invalid' not in package_name diff -Nru snapcraft-2.27.1+17.04/snapcraft/tests/pluginhandler/stage_package_grammar/test_on_statement.py snapcraft-2.28+17.04/snapcraft/tests/pluginhandler/stage_package_grammar/test_on_statement.py --- snapcraft-2.27.1+17.04/snapcraft/tests/pluginhandler/stage_package_grammar/test_on_statement.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/tests/pluginhandler/stage_package_grammar/test_on_statement.py 2017-03-22 12:31:58.000000000 +0000 @@ -27,7 +27,7 @@ def load_tests(loader, tests, ignore): - patcher = mock.patch('snapcraft.repo.Ubuntu') + patcher = mock.patch('snapcraft.repo.Repo') def _setup(test): patcher.start() @@ -165,7 +165,7 @@ options = snapcraft.ProjectOptions(target_deb_arch=self.target_arch) statement = on.OnStatement( on=self.on, body=self.body, project_options=options, - repo_instance=snapcraft.repo.Ubuntu()) + repo_instance=snapcraft.repo.Repo()) for else_body in self.else_bodies: statement.add_else(else_body) @@ -231,7 +231,7 @@ target_deb_arch=self.target_arch) statement = on.OnStatement( on=self.on, body=self.body, project_options=options, - repo_instance=snapcraft.repo.Ubuntu()) + repo_instance=snapcraft.repo.Repo()) for else_body in self.else_bodies: statement.add_else(else_body) @@ -246,7 +246,7 @@ target_deb_arch='amd64') statement = on.OnStatement( on='on i386', body=['foo'], project_options=options, - repo_instance=snapcraft.repo.Ubuntu()) + repo_instance=snapcraft.repo.Repo()) statement.add_else(None) diff -Nru snapcraft-2.27.1+17.04/snapcraft/tests/pluginhandler/stage_package_grammar/test_processor.py snapcraft-2.28+17.04/snapcraft/tests/pluginhandler/stage_package_grammar/test_processor.py --- snapcraft-2.27.1+17.04/snapcraft/tests/pluginhandler/stage_package_grammar/test_processor.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/tests/pluginhandler/stage_package_grammar/test_processor.py 2017-03-22 12:31:58.000000000 +0000 @@ -47,7 +47,7 @@ "amd64,i386' statements. These should be merged."): grammar.process_grammar( self.grammar, snapcraft.ProjectOptions(), - snapcraft.repo.Ubuntu()) + snapcraft.repo.Repo()) class BasicGrammarTestCase(GrammarTestCase): @@ -198,7 +198,7 @@ options = snapcraft.ProjectOptions(target_deb_arch=self.target_arch) self.assertThat( grammar.process_grammar( - self.grammar, options, snapcraft.repo.Ubuntu()), + self.grammar, options, snapcraft.repo.Repo()), Equals(self.expected_packages)) @@ -234,4 +234,4 @@ self.expected_exception): grammar.process_grammar( self.grammar, snapcraft.ProjectOptions(), - snapcraft.repo.Ubuntu()) + snapcraft.repo.Repo()) diff -Nru snapcraft-2.27.1+17.04/snapcraft/tests/pluginhandler/stage_package_grammar/test_try_statement.py snapcraft-2.28+17.04/snapcraft/tests/pluginhandler/stage_package_grammar/test_try_statement.py --- snapcraft-2.27.1+17.04/snapcraft/tests/pluginhandler/stage_package_grammar/test_try_statement.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/tests/pluginhandler/stage_package_grammar/test_try_statement.py 2017-03-22 12:31:58.000000000 +0000 @@ -25,15 +25,15 @@ def load_tests(loader, tests, ignore): - patcher = mock.patch('snapcraft.repo.Ubuntu') + patcher = mock.patch('snapcraft.repo.Repo') def _setup(test): - ubuntu_mock = patcher.start() + repo_mock = patcher.start() def _is_valid(package_name): return 'invalid' not in package_name - ubuntu_mock.return_value.is_valid.side_effect = _is_valid + repo_mock.return_value.is_valid.side_effect = _is_valid def _teardown(test): patcher.stop() @@ -132,7 +132,7 @@ def test_try_statement_grammar(self): statement = _try.TryStatement( body=self.body, project_options=snapcraft.ProjectOptions(), - repo_instance=snapcraft.repo.Ubuntu()) + repo_instance=snapcraft.repo.Repo()) for else_body in self.else_bodies: statement.add_else(else_body) diff -Nru snapcraft-2.27.1+17.04/snapcraft/tests/pluginhandler/test_pluginhandler.py snapcraft-2.28+17.04/snapcraft/tests/pluginhandler/test_pluginhandler.py --- snapcraft-2.27.1+17.04/snapcraft/tests/pluginhandler/test_pluginhandler.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/tests/pluginhandler/test_pluginhandler.py 2017-03-22 12:31:58.000000000 +0000 @@ -953,8 +953,8 @@ os.path.exists(handler._step_state_file(later_step)), 'Expected later step states to be cleared') - @patch('snapcraft.internal.repo.Ubuntu') - def test_pull_state(self, ubuntu_mock): + @patch('snapcraft.internal.repo.Repo') + def test_pull_state(self, repo_mock): self.assertEqual(None, self.handler.last_step()) self.handler.pull() @@ -2251,9 +2251,10 @@ def setUp(self): super().setUp() - patcher = patch.object(snapcraft.internal.repo.Ubuntu, 'get') + patcher = patch.object(snapcraft.internal.repo.Repo, 'get') setup_apt_mock = patcher.start() - setup_apt_mock.side_effect = repo.PackageNotFoundError('non-existing') + setup_apt_mock.side_effect = repo.errors.PackageNotFoundError( + 'non-existing') self.addCleanup(patcher.stop) def test_missing_stage_package_displays_nice_error(self): @@ -2267,7 +2268,7 @@ self.assertEqual( str(raised), "Error downloading stage packages for part 'stage-test': " - "The Ubuntu package 'non-existing' was not found.") + "The package 'non-existing' was not found.") class FindDependenciesTestCase(tests.TestCase): diff -Nru snapcraft-2.27.1+17.04/snapcraft/tests/pluginhandler/test_stage_package_handler.py snapcraft-2.28+17.04/snapcraft/tests/pluginhandler/test_stage_package_handler.py --- snapcraft-2.27.1+17.04/snapcraft/tests/pluginhandler/test_stage_package_handler.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/tests/pluginhandler/test_stage_package_handler.py 2017-03-22 12:31:58.000000000 +0000 @@ -32,7 +32,7 @@ def load_tests(loader, tests, ignore): - patcher = mock.patch('snapcraft.repo.Ubuntu') + patcher = mock.patch('snapcraft.repo.Repo') def _setup(test): patcher.start() @@ -58,13 +58,13 @@ self.process_grammar_mock.side_effect = process_grammar self.addCleanup(patcher.stop) - patcher = mock.patch('snapcraft.repo.Ubuntu') - self.ubuntu_mock = patcher.start() + patcher = mock.patch('snapcraft.repo.Repo') + self.repo_mock = patcher.start() self.addCleanup(patcher.stop) - self.get_mock = self.ubuntu_mock.return_value.get - self.unpack_mock = self.ubuntu_mock.return_value.unpack - self.is_valid_mock = self.ubuntu_mock.return_value.is_valid + self.get_mock = self.repo_mock.return_value.get + self.unpack_mock = self.repo_mock.return_value.unpack + self.is_valid_mock = self.repo_mock.return_value.is_valid self.cache_dir = os.path.join(os.getcwd(), 'cache') self.unpack_dir = os.path.join(os.getcwd(), 'unpack') diff -Nru snapcraft-2.27.1+17.04/snapcraft/tests/plugins/test_ant.py snapcraft-2.28+17.04/snapcraft/tests/plugins/test_ant.py --- snapcraft-2.27.1+17.04/snapcraft/tests/plugins/test_ant.py 2017-01-28 06:12:26.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/tests/plugins/test_ant.py 2017-03-22 12:31:58.000000000 +0000 @@ -36,10 +36,6 @@ self.project_options = snapcraft.ProjectOptions() - patcher = mock.patch('snapcraft.repo.Ubuntu') - self.ubuntu_mock = patcher.start() - self.addCleanup(patcher.stop) - def test_schema(self): schema = ant.AntPlugin.schema() diff -Nru snapcraft-2.27.1+17.04/snapcraft/tests/plugins/test_autotools.py snapcraft-2.28+17.04/snapcraft/tests/plugins/test_autotools.py --- snapcraft-2.27.1+17.04/snapcraft/tests/plugins/test_autotools.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/tests/plugins/test_autotools.py 2017-03-22 12:31:58.000000000 +0000 @@ -44,10 +44,6 @@ self.options = Options() self.project_options = snapcraft.ProjectOptions() - patcher = mock.patch('snapcraft.repo.Ubuntu') - self.ubuntu_mock = patcher.start() - self.addCleanup(patcher.stop) - def test_schema(self): schema = autotools.AutotoolsPlugin.schema() diff -Nru snapcraft-2.27.1+17.04/snapcraft/tests/plugins/test_catkin.py snapcraft-2.28+17.04/snapcraft/tests/plugins/test_catkin.py --- snapcraft-2.27.1+17.04/snapcraft/tests/plugins/test_catkin.py 2017-02-17 13:45:14.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/tests/plugins/test_catkin.py 2017-03-22 12:31:58.000000000 +0000 @@ -25,7 +25,8 @@ from testtools.matchers import ( Contains, Equals, - HasLength + HasLength, + MatchesRegex, ) import snapcraft @@ -34,6 +35,7 @@ repo, tests, ) +from snapcraft.internal import errors class _CompareContainers(): @@ -55,7 +57,7 @@ return True -class CatkinPluginTestCase(tests.TestCase): +class CatkinPluginBaseTestCase(tests.TestCase): def setUp(self): super().setUp() @@ -66,6 +68,7 @@ source_space = 'src' source_subdir = None include_roscore = False + underlay = None self.properties = props() self.project_options = snapcraft.ProjectOptions() @@ -83,6 +86,10 @@ self.rosdep_mock = patcher.start() self.addCleanup(patcher.stop) + patcher = mock.patch('snapcraft.plugins.catkin._Catkin') + self.catkin_mock = patcher.start() + self.addCleanup(patcher.stop) + def verify_rosdep_setup(self, rosdistro, package_path, rosdep_path, sources): self.rosdep_mock.assert_has_calls([ @@ -90,6 +97,9 @@ self.project_options), mock.call().setup()]) + +class CatkinPluginTestCase(CatkinPluginBaseTestCase): + def test_schema(self): schema = catkin.CatkinPlugin.schema() @@ -204,6 +214,40 @@ 'Expected "include-roscore" "default" to be "true", ' 'but it was "{}"'.format(include_roscore_default)) + # Check underlay property + self.assertThat(properties, Contains('underlay'), + 'Expected "underlay" to be included in properties') + + underlay = properties['underlay'] + self.assertThat(underlay, Contains('type'), + 'Expected "type" to be included in "underlay"') + self.assertThat(underlay, Contains('properties'), + 'Expected "properties" to be included in "underlay"') + self.assertThat(underlay, Contains('required'), + 'Expected "required" to be included in "underlay"') + + underlay_type = underlay['type'] + self.assertThat(underlay_type, Equals('object'), + 'Expected "underlay" "type" to be "object", but it ' + 'was "{}"'.format(underlay_type)) + + underlay_required = underlay['required'] + self.assertThat(underlay_required, HasLength(2)) + self.assertThat(underlay_required, Contains('build-path')) + self.assertThat(underlay_required, Contains('run-path')) + + underlay_properties = underlay['properties'] + self.assertThat(underlay_properties, Contains('build-path')) + self.assertThat(underlay_properties, Contains('run-path')) + + underlay_build_path = underlay_properties['build-path'] + self.assertThat(underlay_build_path, Contains('type')) + self.assertThat(underlay_build_path['type'], Equals('string')) + + underlay_run_path = underlay_properties['run-path'] + self.assertThat(underlay_run_path, Contains('type')) + self.assertThat(underlay_run_path['type'], Equals('string')) + # Check required self.assertTrue('catkin-packages' in schema['required'], 'Expected "catkin-packages" to be included in ' @@ -211,7 +255,8 @@ def test_get_pull_properties(self): expected_pull_properties = ['rosdistro', 'catkin-packages', - 'source-space', 'include-roscore'] + 'source-space', 'include-roscore', + 'underlay'] resulting_pull_properties = catkin.CatkinPlugin.get_pull_properties() self.assertThat(resulting_pull_properties, @@ -250,63 +295,6 @@ self.project_options) self.assertTrue('xenial' in plugin.PLUGIN_STAGE_SOURCES) - def test_pull_debian_dependencies(self): - plugin = catkin.CatkinPlugin('test-part', self.properties, - self.project_options) - os.makedirs(os.path.join(plugin.sourcedir, 'src')) - - self.dependencies_mock.return_value = {'foo', 'bar', 'baz'} - - plugin.pull() - - self.verify_rosdep_setup( - self.properties.rosdistro, - os.path.join(plugin.sourcedir, 'src'), - os.path.join(plugin.partdir, 'rosdep'), - plugin.PLUGIN_STAGE_SOURCES) - - # Verify that dependencies were found as expected. TODO: Would really - # like to use ANY here instead of verifying explicit arguments, but - # Python issue #25195 won't let me. - self.assertEqual(1, self.dependencies_mock.call_count) - self.assertEqual({'my_package'}, - self.dependencies_mock.call_args[0][0]) - - # Verify that the dependencies were installed - self.ubuntu_mock.return_value.get.assert_called_with( - _CompareContainers(self, ['foo', 'bar', 'baz'])) - self.ubuntu_mock.return_value.unpack.assert_called_with( - plugin.installdir) - - def test_pull_local_dependencies(self): - self.properties.catkin_packages.append('package_2') - - plugin = catkin.CatkinPlugin('test-part', self.properties, - self.project_options) - os.makedirs(os.path.join(plugin.sourcedir, 'src')) - - # No system dependencies (only local) - self.dependencies_mock.return_value = set() - - plugin.pull() - - self.verify_rosdep_setup( - self.properties.rosdistro, - os.path.join(plugin.sourcedir, 'src'), - os.path.join(plugin.partdir, 'rosdep'), - plugin.PLUGIN_STAGE_SOURCES) - - # Verify that dependencies were found as expected. TODO: Would really - # like to use ANY here instead of verifying explicit arguments, but - # Python issue #25195 won't let me. - self.assertEqual(1, self.dependencies_mock.call_count) - self.assertEqual({'my_package', 'package_2'}, - self.dependencies_mock.call_args[0][0]) - - # Verify that no .deb packages were installed - self.assertTrue(mock.call().unpack(plugin.installdir) not in - self.ubuntu_mock.mock_calls) - @mock.patch('snapcraft.plugins.catkin._Compilers') def test_pull_invalid_dependency(self, compilers_mock): plugin = catkin.CatkinPlugin('test-part', self.properties, @@ -316,17 +304,17 @@ self.dependencies_mock.return_value = ['foo'] mock_instance = self.ubuntu_mock.return_value - mock_instance.get.side_effect = repo.PackageNotFoundError('foo') + mock_instance.get.side_effect = repo.errors.PackageNotFoundError('foo') raised = self.assertRaises( RuntimeError, plugin.pull) self.assertEqual(str(raised), - 'Failed to fetch system dependencies: The Ubuntu ' + 'Failed to fetch system dependencies: The ' "package 'foo' was not found.") - def test_pull_with_roscore(self): + def test_pull_unable_to_resolve_roscore(self): self.properties.include_roscore = True plugin = catkin.CatkinPlugin('test-part', self.properties, self.project_options) @@ -335,28 +323,19 @@ # No system dependencies self.dependencies_mock.return_value = set() - def resolve(package_name): - if package_name == 'ros_core': - return ['ros-core-dependency'] - - self.rosdep_mock.return_value.resolve_dependency = resolve - - plugin.pull() + self.rosdep_mock.return_value.resolve_dependency.return_value = None - self.verify_rosdep_setup( - self.properties.rosdistro, - os.path.join(plugin.sourcedir, 'src'), - os.path.join(plugin.partdir, 'rosdep'), - plugin.PLUGIN_STAGE_SOURCES) + raised = self.assertRaises(RuntimeError, plugin.pull) - # Verify that roscore was installed - self.ubuntu_mock.return_value.get.assert_called_with( - {'ros-core-dependency'}) - self.ubuntu_mock.return_value.unpack.assert_called_with( - plugin.installdir) + self.assertEqual(str(raised), + 'Unable to determine system dependency for roscore') - def test_pull_unable_to_resolve_roscore(self): - self.properties.include_roscore = True + @mock.patch.object(catkin.CatkinPlugin, '_generate_snapcraft_setup_sh') + def test_pull_invalid_underlay(self, generate_setup_mock): + self.properties.underlay = { + 'build-path': 'test-build-path', + 'run-path': 'test-run-path' + } plugin = catkin.CatkinPlugin('test-part', self.properties, self.project_options) os.makedirs(os.path.join(plugin.sourcedir, 'src')) @@ -364,12 +343,13 @@ # No system dependencies self.dependencies_mock.return_value = set() - self.rosdep_mock.return_value.resolve_dependency.return_value = None - - raised = self.assertRaises(RuntimeError, plugin.pull) + raised = self.assertRaises( + errors.SnapcraftEnvironmentError, plugin.pull) - self.assertEqual(str(raised), - 'Unable to determine system dependency for roscore') + self.assertThat( + str(raised), + MatchesRegex( + '.*Requested underlay.*does not point to a valid directory')) def test_clean_pull(self): plugin = catkin.CatkinPlugin('test-part', self.properties, @@ -555,7 +535,7 @@ 'expected': '#!/usr/bin/env python', }, { - 'path': os.path.join(plugin.rosdir, 'bin/catkin_find'), + 'path': os.path.join(plugin.rosdir, 'bin/catkin'), 'contents': '#!/foo/baz/python', 'expected': '#!/usr/bin/env python', }, @@ -575,7 +555,7 @@ for file_info in files: with open(os.path.join(plugin.rosdir, file_info['path']), 'r') as f: - self.assertEqual(f.read(), file_info['expected']) + self.assertThat(f.read(), Equals(file_info['expected'])) @mock.patch.object(catkin.CatkinPlugin, 'run') @mock.patch.object(catkin.CatkinPlugin, 'run_output', return_value='foo') @@ -617,8 +597,155 @@ 'The absolute path to python was not replaced as ' 'expected') + @mock.patch.object(catkin.CatkinPlugin, '_source_setup_sh', + return_value='test-source-setup') + @mock.patch.object(catkin.CatkinPlugin, 'run_output', return_value='bar') + def test_run_environment(self, run_mock, source_setup_sh_mock): + plugin = catkin.CatkinPlugin('test-part', self.properties, + self.project_options) + + python_path = os.path.join( + plugin.installdir, 'usr', 'lib', 'python2.7', 'dist-packages') + os.makedirs(python_path) + + # Joining and re-splitting to get hacked script in there as well + environment = '\n'.join(plugin.env(plugin.installdir)).split('\n') + + self.assertThat(environment, Contains( + 'PYTHONPATH={}:$PYTHONPATH'.format(python_path))) + + self.assertThat(environment, Contains( + 'ROS_MASTER_URI=http://localhost:11311')) + + self.assertThat(environment, Contains('ROS_HOME=$SNAP_USER_DATA/ros')) + + self.assertThat(environment, Contains('LC_ALL=C.UTF-8')) + + source_setup_sh_mock.assert_called_with(plugin.installdir, None) + self.assertThat(environment, Contains('test-source-setup')) + + @mock.patch.object(catkin.CatkinPlugin, 'run_output', return_value='bar') + def test_run_environment_with_underlay(self, run_mock): + self.properties.underlay = { + 'build-path': 'test-build-path', + 'run-path': 'test-run-path', + } + plugin = catkin.CatkinPlugin('test-part', self.properties, + self.project_options) + + python_path = os.path.join( + plugin.installdir, 'usr', 'lib', 'python2.7', 'dist-packages') + os.makedirs(python_path) + + # Joining and re-splitting to get hacked script in there as well + environment = '\n'.join(plugin.env(plugin.installdir)).split('\n') + + self.assertThat(environment, Contains( + 'PYTHONPATH={}:$PYTHONPATH'.format(python_path))) + + self.assertThat(environment, Contains( + 'ROS_MASTER_URI=http://localhost:11311')) + + self.assertThat(environment, Contains('ROS_HOME=$SNAP_USER_DATA/ros')) + + self.assertThat(environment, Contains('LC_ALL=C.UTF-8')) + + self.assertThat(environment, Contains('. {}'.format( + os.path.join(plugin.rosdir, 'snapcraft-setup.sh')))) + + @mock.patch.object(catkin.CatkinPlugin, '_source_setup_sh', + return_value='test-source-setup') + def test_generate_snapcraft_sh(self, source_setup_sh_mock): + plugin = catkin.CatkinPlugin('test-part', self.properties, + self.project_options) + + plugin._generate_snapcraft_setup_sh('test-root', None) + source_setup_sh_mock.assert_called_with('test-root', None) + with open(os.path.join(plugin.rosdir, 'snapcraft-setup.sh'), 'r') as f: + self.assertThat(f.read(), Equals('test-source-setup')) + + def test_source_setup_sh(self): + plugin = catkin.CatkinPlugin('test-part', self.properties, + self.project_options) + + # Make sure $@ is zeroed, then setup.sh sourced, then $@ is restored + rosdir = os.path.join( + 'test-root', 'opt', 'ros', self.properties.rosdistro) + setup_path = os.path.join(rosdir, 'setup.sh') + lines_of_interest = [ + 'set --', + 'if [ -f {} ]; then'.format(setup_path), + '_CATKIN_SETUP_DIR={} . {}'.format(rosdir, setup_path), + 'fi', + 'eval "set -- $BACKUP_ARGS"', + ] + actual_lines = [] + + for line in plugin._source_setup_sh('test-root', None).split('\n'): + line = line.strip() + if line in lines_of_interest: + actual_lines.append(line) + + self.assertThat( + actual_lines, Equals(lines_of_interest), + "Expected ROS's setup.sh to be sourced after args were zeroed, " + "followed by the args being restored.") + + def test_generate_snapcraft_sh_with_underlay(self): + self.properties.underlay = { + 'build-path': 'test-build-underlay', + 'run-path': 'test-run-underlay' + } + plugin = catkin.CatkinPlugin('test-part', self.properties, + self.project_options) + + # Make sure $@ is zeroed, then setup.sh sourced, then $@ is restored + underlay_setup_path = os.path.join( + 'test-underlay', 'setup.sh') + rosdir = os.path.join( + 'test-root', 'opt', 'ros', self.properties.rosdistro) + setup_path = os.path.join(rosdir, 'setup.sh') + lines_of_interest = [ + 'set --', + 'if [ -f {} ]; then'.format(underlay_setup_path), + '_CATKIN_SETUP_DIR={} . {}'.format( + 'test-underlay', underlay_setup_path), + 'if [ -f {} ]; then'.format(setup_path), + 'set -- --extend', + '_CATKIN_SETUP_DIR={} . {}'.format(rosdir, setup_path), + 'fi', + 'fi', + 'eval "set -- $BACKUP_ARGS"', + ] + actual_lines = [] + + plugin._generate_snapcraft_setup_sh('test-root', 'test-underlay') + with open(os.path.join(plugin.rosdir, 'snapcraft-setup.sh'), 'r') as f: + for line in f: + line = line.strip() + if line in lines_of_interest: + actual_lines.append(line) + + self.assertThat( + actual_lines, Equals(lines_of_interest), + "Expected ROS's setup.sh to be sourced after args were zeroed, " + "followed by the args being restored.") + + @mock.patch.object(catkin.CatkinPlugin, 'run_output', return_value='bar') + def test_run_environment_no_python(self, run_mock): + plugin = catkin.CatkinPlugin('test-part', self.properties, + self.project_options) + + python_path = os.path.join( + plugin.installdir, 'usr', 'lib', 'python2.7', 'dist-packages') + + environment = plugin.env(plugin.installdir) + + self.assertFalse( + 'PYTHONPATH={}'.format(python_path) in environment, environment) + @mock.patch.object(catkin.CatkinPlugin, '_use_in_snap_python') - def test_prepare_build(self, use_in_snap_python_mock): + def test_prepare_build(self, use_python_mock): plugin = catkin.CatkinPlugin('test-part', self.properties, self.project_options) os.makedirs(os.path.join(plugin.rosdir, 'test')) @@ -661,25 +788,186 @@ plugin._prepare_build() - self.assertTrue(use_in_snap_python_mock.called) + self.assertTrue(use_python_mock.called) for file_info in files: path = os.path.join(plugin.rosdir, file_info['path']) with open(path, 'r') as f: self.assertThat(f.read(), Equals(file_info['expected'])) - @mock.patch.object(catkin.CatkinPlugin, '_use_in_snap_python') - def test_finish_build_cmake_paths(self, use_in_snap_python_mock): + +class PullTestCase(CatkinPluginBaseTestCase): + + scenarios = [ + ('no underlay', { + 'underlay': None, + 'expected_underlay_path': None, + }), + ('underlay', { + 'underlay': { + 'build-path': 'test-build-path', + 'run-path': 'test-run-path', + }, + 'expected_underlay_path': 'test-build-path' + }) + ] + + def setUp(self): + super().setUp() + + # Make the underlay a valid workspace + if self.underlay: + os.makedirs(self.underlay['build-path']) + open(os.path.join( + self.underlay['build-path'], 'setup.sh'), 'w').close() + self.properties.underlay = self.underlay + + @mock.patch.object(catkin.CatkinPlugin, '_generate_snapcraft_setup_sh') + def test_pull_debian_dependencies(self, generate_setup_mock): plugin = catkin.CatkinPlugin('test-part', self.properties, self.project_options) - os.makedirs(os.path.join(plugin.rosdir, 'test')) + os.makedirs(os.path.join(plugin.sourcedir, 'src')) + + self.dependencies_mock.return_value = {'foo', 'bar', 'baz'} + + plugin.pull() + + self.verify_rosdep_setup( + self.properties.rosdistro, + os.path.join(plugin.sourcedir, 'src'), + os.path.join(plugin.partdir, 'rosdep'), + plugin.PLUGIN_STAGE_SOURCES) + + # This shouldn't be called unless there's an underlay + if self.properties.underlay: + generate_setup_mock.assert_called_once_with( + plugin.installdir, self.expected_underlay_path) + else: + generate_setup_mock.assert_not_called() + + # Verify that dependencies were found as expected. TODO: Would really + # like to use ANY here instead of verifying explicit arguments, but + # Python issue #25195 won't let me. + self.assertEqual(1, self.dependencies_mock.call_count) + self.assertEqual({'my_package'}, + self.dependencies_mock.call_args[0][0]) + + # Verify that the dependencies were installed + self.ubuntu_mock.return_value.get.assert_called_with( + _CompareContainers(self, ['foo', 'bar', 'baz'])) + self.ubuntu_mock.return_value.unpack.assert_called_with( + plugin.installdir) + + @mock.patch.object(catkin.CatkinPlugin, '_generate_snapcraft_setup_sh') + def test_pull_local_dependencies(self, generate_setup_mock): + self.properties.catkin_packages.append('package_2') + + plugin = catkin.CatkinPlugin('test-part', self.properties, + self.project_options) + os.makedirs(os.path.join(plugin.sourcedir, 'src')) + + # No system dependencies (only local) + self.dependencies_mock.return_value = set() + + plugin.pull() + + self.verify_rosdep_setup( + self.properties.rosdistro, + os.path.join(plugin.sourcedir, 'src'), + os.path.join(plugin.partdir, 'rosdep'), + plugin.PLUGIN_STAGE_SOURCES) + + # This shouldn't be called unless there's an underlay + if self.properties.underlay: + generate_setup_mock.assert_called_once_with( + plugin.installdir, self.expected_underlay_path) + else: + generate_setup_mock.assert_not_called() + + # Verify that dependencies were found as expected. TODO: Would really + # like to use ANY here instead of verifying explicit arguments, but + # Python issue #25195 won't let me. + self.assertEqual(1, self.dependencies_mock.call_count) + self.assertEqual({'my_package', 'package_2'}, + self.dependencies_mock.call_args[0][0]) + + # Verify that no .deb packages were installed + self.assertTrue(mock.call().unpack(plugin.installdir) not in + self.ubuntu_mock.mock_calls) + + @mock.patch.object(catkin.CatkinPlugin, '_generate_snapcraft_setup_sh') + def test_pull_with_roscore(self, generate_setup_mock): + self.properties.include_roscore = True + plugin = catkin.CatkinPlugin('test-part', self.properties, + self.project_options) + os.makedirs(os.path.join(plugin.sourcedir, 'src')) + + # No system dependencies + self.dependencies_mock.return_value = set() + + def resolve(package_name): + if package_name == 'ros_core': + return ['ros-core-dependency'] + + self.rosdep_mock.return_value.resolve_dependency = resolve + + plugin.pull() + + self.verify_rosdep_setup( + self.properties.rosdistro, + os.path.join(plugin.sourcedir, 'src'), + os.path.join(plugin.partdir, 'rosdep'), + plugin.PLUGIN_STAGE_SOURCES) + + # This shouldn't be called unless there's an underlay + if self.properties.underlay: + generate_setup_mock.assert_called_once_with( + plugin.installdir, self.expected_underlay_path) + else: + generate_setup_mock.assert_not_called() + + # Verify that roscore was installed + self.ubuntu_mock.return_value.get.assert_called_with( + {'ros-core-dependency'}) + self.ubuntu_mock.return_value.unpack.assert_called_with( + plugin.installdir) + + +class FinishBuildTestCase(CatkinPluginBaseTestCase): + + scenarios = [ + ('no underlay', { + 'underlay': None, + 'expected_underlay_path': None, + }), + ('underlay', { + 'underlay': { + 'build-path': 'test-build-path', + 'run-path': 'test-run-path', + }, + 'expected_underlay_path': 'test-run-path' + }) + ] + + def setUp(self): + super().setUp() + + self.properties.underlay = self.underlay + self.plugin = catkin.CatkinPlugin( + 'test-part', self.properties, self.project_options) + + @mock.patch.object(catkin.CatkinPlugin, '_generate_snapcraft_setup_sh') + @mock.patch.object(catkin.CatkinPlugin, '_use_in_snap_python') + def test_finish_build_cmake_paths(self, use_python_mock, + generate_setup_mock): + os.makedirs(os.path.join(self.plugin.rosdir, 'test')) # Place a few .cmake files with incorrect paths, and some files that # shouldn't be changed. files = [ { 'path': 'fooConfig.cmake', - 'contents': '"{}/usr/lib/foo"'.format(plugin.installdir), + 'contents': '"{}/usr/lib/foo"'.format(self.plugin.installdir), 'expected': '"$ENV{SNAPCRAFT_STAGE}/usr/lib/foo"', }, { @@ -690,7 +978,7 @@ { 'path': 'test/bazConfig.cmake', 'contents': '"{0}/test/baz;{0}/usr/lib/baz"'.format( - plugin.installdir), + self.plugin.installdir), 'expected': '"$ENV{SNAPCRAFT_STAGE}/test/baz;' '$ENV{SNAPCRAFT_STAGE}/usr/lib/baz"', }, @@ -707,37 +995,50 @@ ] for file_info in files: - path = os.path.join(plugin.rosdir, file_info['path']) + path = os.path.join(self.plugin.rosdir, file_info['path']) with open(path, 'w') as f: f.write(file_info['contents']) - plugin._finish_build() + self.plugin._finish_build() + + self.assertTrue(use_python_mock.called) - self.assertTrue(use_in_snap_python_mock.called) + # This shouldn't be called unless there's an underlay + if self.properties.underlay: + generate_setup_mock.assert_called_once_with( + '$SNAP', self.expected_underlay_path) + else: + generate_setup_mock.assert_not_called() for file_info in files: - path = os.path.join(plugin.rosdir, file_info['path']) + path = os.path.join(self.plugin.rosdir, file_info['path']) with open(path, 'r') as f: self.assertThat(f.read(), Equals(file_info['expected'])) + @mock.patch.object(catkin.CatkinPlugin, '_generate_snapcraft_setup_sh') @mock.patch.object(catkin.CatkinPlugin, 'run') @mock.patch.object(catkin.CatkinPlugin, 'run_output', return_value='foo') @mock.patch.object(catkin.CatkinPlugin, '_use_in_snap_python') - def test_finish_build_cmake_prefix_path(self, use_in_snap_python_mock, - run_output_mock, run_mock): - plugin = catkin.CatkinPlugin('test-part', self.properties, - self.project_options) - - setup_file = os.path.join(plugin.rosdir, '_setup_util.py') + def test_finish_build_cmake_prefix_path(self, use_python_mock, + run_output_mock, run_mock, + generate_setup_mock): + setup_file = os.path.join(self.plugin.rosdir, '_setup_util.py') os.makedirs(os.path.dirname(setup_file)) with open(setup_file, 'w') as f: f.write("CMAKE_PREFIX_PATH = '{0}/{1};{0}\n".format( - plugin.rosdir, plugin.options.rosdistro)) + self.plugin.rosdir, self.plugin.options.rosdistro)) + + self.plugin._finish_build() - plugin._finish_build() + self.assertTrue(use_python_mock.called) - self.assertTrue(use_in_snap_python_mock.called) + # This shouldn't be called unless there's an underlay + if self.properties.underlay: + generate_setup_mock.assert_called_once_with( + '$SNAP', self.expected_underlay_path) + else: + generate_setup_mock.assert_not_called() expected = 'CMAKE_PREFIX_PATH = []\n' @@ -747,119 +1048,79 @@ 'The absolute path to python or the CMAKE_PREFIX_PATH ' 'was not replaced as expected') - @mock.patch.object(catkin.CatkinPlugin, 'run_output', return_value='bar') - def test_run_environment(self, run_mock): - plugin = catkin.CatkinPlugin('test-part', self.properties, - self.project_options) - - python_path = os.path.join( - plugin.installdir, 'usr', 'lib', 'python2.7', 'dist-packages') - os.makedirs(python_path) - - environment = plugin.env(plugin.installdir) - - self.assertTrue( - 'PYTHONPATH={}:$PYTHONPATH'.format(python_path) in - environment, environment) - - self.assertTrue('ROS_MASTER_URI=http://localhost:11311' in environment) - - self.assertTrue('ROS_HOME=$SNAP_USER_DATA/ros' in environment) - - self.assertTrue('LC_ALL=C.UTF-8' in environment) - - self.assertTrue('_CATKIN_SETUP_DIR={}'.format(os.path.join( - plugin.installdir, 'opt', 'ros', self.properties.rosdistro)) - in environment) - - # Make sure $@ is zeroed, then setup.sh sourced, then $@ is restored - lines_of_interest = [ - 'set --', - '. {}'.format(os.path.join( - plugin.installdir, 'opt', 'ros', self.properties.rosdistro, - 'setup.sh')), - 'eval "set -- $BACKUP_ARGS"', - ] - actual_lines = [] - - # Joining and re-splitting to get hacked script in there as well - for line in '\n'.join(environment).split('\n'): - line = line.strip() - if line in lines_of_interest: - actual_lines.append(line) - - self.assertThat( - actual_lines, Equals(lines_of_interest), - "Expected ROS's setup.sh to be sourced after args were zeroed, " - "followed by the args being restored.") - - @mock.patch.object(catkin.CatkinPlugin, 'run_output', return_value='bar') - def test_run_environment_no_python(self, run_mock): - plugin = catkin.CatkinPlugin('test-part', self.properties, - self.project_options) - - python_path = os.path.join( - plugin.installdir, 'usr', 'lib', 'python2.7', 'dist-packages') - - environment = plugin.env(plugin.installdir) - - self.assertFalse( - 'PYTHONPATH={}'.format(python_path) in environment, environment) - class FindSystemDependenciesTestCase(tests.TestCase): def setUp(self): super().setUp() - def test_find_system_dependencies_system_only(self): - rosdep_mock = mock.MagicMock() - rosdep_mock.get_dependencies.return_value = ['bar'] - rosdep_mock.resolve_dependency.return_value = ['baz'] + self.rosdep_mock = mock.MagicMock() + self.rosdep_mock.get_dependencies.return_value = ['bar'] - self.assertEqual({'baz'}, catkin._find_system_dependencies( - {'foo'}, rosdep_mock)) + self.catkin_mock = mock.MagicMock() + exception = catkin.CatkinPackageNotFoundError('foo') + self.catkin_mock.find.side_effect = exception - rosdep_mock.get_dependencies.assert_called_once_with('foo') - rosdep_mock.resolve_dependency.assert_called_once_with('bar') + def test_find_system_dependencies_system_only(self): + self.rosdep_mock.resolve_dependency.return_value = ['baz'] - def test_find_system_dependencies_local_only(self): - rosdep_mock = mock.MagicMock() - rosdep_mock.get_dependencies.return_value = ['bar'] + self.assertThat(catkin._find_system_dependencies( + {'foo'}, self.rosdep_mock, self.catkin_mock), Equals({'baz'})) - self.assertEqual(set(), catkin._find_system_dependencies( - {'foo', 'bar'}, rosdep_mock)) + self.rosdep_mock.get_dependencies.assert_called_once_with('foo') + self.rosdep_mock.resolve_dependency.assert_called_once_with('bar') + self.catkin_mock.find.assert_called_once_with('bar') - rosdep_mock.get_dependencies.assert_has_calls([mock.call('foo'), - mock.call('bar')], - any_order=True) - rosdep_mock.resolve_dependency.assert_not_called() + def test_find_system_dependencies_local_only(self): + self.assertThat(catkin._find_system_dependencies( + {'foo', 'bar'}, self.rosdep_mock, self.catkin_mock), + HasLength(0)) + + self.rosdep_mock.get_dependencies.assert_has_calls( + [mock.call('foo'), mock.call('bar')], any_order=True) + self.rosdep_mock.resolve_dependency.assert_not_called() + self.catkin_mock.find.assert_not_called() + + def test_find_system_dependencies_satisfied_in_stage(self): + self.catkin_mock.find.side_effect = None + self.catkin_mock.find.return_value = 'baz' + + self.assertThat(catkin._find_system_dependencies( + {'foo'}, self.rosdep_mock, self.catkin_mock), HasLength(0)) + + self.rosdep_mock.get_dependencies.assert_called_once_with('foo') + self.catkin_mock.find.assert_called_once_with('bar') + self.rosdep_mock.resolve_dependency.assert_not_called() def test_find_system_dependencies_mixed(self): - rosdep_mock = mock.MagicMock() - rosdep_mock.get_dependencies.return_value = ['bar', 'baz'] - rosdep_mock.resolve_dependency.return_value = ['qux'] - - self.assertEqual({'qux'}, catkin._find_system_dependencies( - {'foo', 'bar'}, rosdep_mock)) - - rosdep_mock.get_dependencies.assert_has_calls([mock.call('foo'), - mock.call('bar')], - any_order=True) - rosdep_mock.resolve_dependency.assert_called_once_with('baz') + self.rosdep_mock.get_dependencies.return_value = ['bar', 'baz', 'qux'] + self.rosdep_mock.resolve_dependency.return_value = ['quux'] - def test_find_system_dependencies_missing_local_dependency(self): - rosdep_mock = mock.MagicMock() + def _fake_find(package_name): + if package_name == 'qux': + return package_name + raise catkin.CatkinPackageNotFoundError(package_name) + + self.catkin_mock.find.side_effect = _fake_find + self.assertThat(catkin._find_system_dependencies( + {'foo', 'bar'}, self.rosdep_mock, self.catkin_mock), + Equals({'quux'})) + + self.rosdep_mock.get_dependencies.assert_has_calls( + [mock.call('foo'), mock.call('bar')], any_order=True) + self.rosdep_mock.resolve_dependency.assert_called_once_with('baz') + self.catkin_mock.find.assert_has_calls( + [mock.call('baz'), mock.call('qux')], any_order=True) + def test_find_system_dependencies_missing_local_dependency(self): # Setup a dependency on a non-existing package, and it doesn't resolve # to a system dependency.' - rosdep_mock.get_dependencies.return_value = ['bar'] - exception = catkin.SystemDependencyNotFound('foo') - rosdep_mock.resolve_dependency.side_effect = exception + exception = catkin.SystemDependencyNotFoundError('foo') + self.rosdep_mock.resolve_dependency.side_effect = exception raised = self.assertRaises( RuntimeError, catkin._find_system_dependencies, - {'foo'}, rosdep_mock) + {'foo'}, self.rosdep_mock, self.catkin_mock) self.assertEqual(raised.args[0], "Package 'bar' isn't a valid system dependency. Did " @@ -983,7 +1244,7 @@ 1, 'foo') raised = self.assertRaises( - catkin.SystemDependencyNotFound, + catkin.SystemDependencyNotFoundError, self.rosdep.resolve_dependency, 'bar') self.assertEqual(str(raised), @@ -1126,3 +1387,73 @@ def test_ldflags(self): self.assertThat(self.compilers.ldflags, Equals('')) + + +class CatkinFindTestCase(tests.TestCase): + def setUp(self): + super().setUp() + + self.project = snapcraft.ProjectOptions() + self.catkin = catkin._Catkin( + 'kinetic', 'workspace_path', 'catkin_path', 'sources', + self.project) + + patcher = mock.patch('snapcraft.repo.Ubuntu') + self.ubuntu_mock = patcher.start() + self.addCleanup(patcher.stop) + + patcher = mock.patch('subprocess.check_output') + self.check_output_mock = patcher.start() + self.addCleanup(patcher.stop) + + def test_setup(self): + # Return something other than a Mock to ease later assertions + self.check_output_mock.return_value = b'' + + self.catkin.setup() + + # Verify that only rospack was installed (no other .debs) + self.assertThat(self.ubuntu_mock.call_count, Equals(1)) + self.assertThat( + self.ubuntu_mock.return_value.get.call_count, Equals(1)) + self.assertThat( + self.ubuntu_mock.return_value.unpack.call_count, Equals(1)) + self.ubuntu_mock.assert_has_calls([ + mock.call(self.catkin._catkin_path, sources='sources', + project_options=self.project), + mock.call().get(['ros-kinetic-catkin']), + mock.call().unpack(self.catkin._catkin_install_path)]) + + def test_setup_can_run_multiple_times(self): + self.catkin.setup() + + # Make sure running setup() again doesn't have problems with the old + # environment. An exception will be raised if setup() can't be called + # twice. + self.catkin.setup() + + def test_find(self): + self.check_output_mock.return_value = b'bar' + + self.assertThat(self.catkin.find('foo'), Equals('bar')) + + self.assertTrue(self.check_output_mock.called) + positional_args = self.check_output_mock.call_args[0][0] + self.assertThat( + ' '.join(positional_args), + Contains('catkin_find --first-only foo')) + + def test_find_non_existing_package(self): + self.check_output_mock.side_effect = subprocess.CalledProcessError( + 1, 'foo') + + with testtools.ExpectedException( + catkin.CatkinPackageNotFoundError, + "Unable to find Catkin package 'foo'"): + self.catkin.find('foo') + + self.assertTrue(self.check_output_mock.called) + positional_args = self.check_output_mock.call_args[0][0] + self.assertThat( + ' '.join(positional_args), + Contains('catkin_find --first-only foo')) diff -Nru snapcraft-2.27.1+17.04/snapcraft/tests/plugins/test_copy.py snapcraft-2.28+17.04/snapcraft/tests/plugins/test_copy.py --- snapcraft-2.27.1+17.04/snapcraft/tests/plugins/test_copy.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/tests/plugins/test_copy.py 2017-03-22 12:31:58.000000000 +0000 @@ -26,10 +26,10 @@ CopyPlugin, _recursively_link ) -from snapcraft.tests import TestCase +from snapcraft import tests -class TestCopyPlugin(TestCase): +class TestCopyPlugin(tests.TestCase): def setUp(self): super().setUp() @@ -253,7 +253,6 @@ for symlink in symlinks: os.symlink(symlink['source'], symlink['link_name']) - c.pull() c.build() self.assertTrue(os.path.isdir(destination), @@ -326,6 +325,28 @@ with open(destination, 'r') as f: self.assertEqual(f.read(), symlink['expected_contents']) + def test_copy_symlinks_to_libc(self): + self.mock_options.files = {'*': '.'} + + c = CopyPlugin('copy', self.mock_options, self.project_options) + + # These directories are created by the pluginhandler + os.makedirs(c.builddir) + + # Even though this symlink is absolute, since it's to libc the copy + # plugin shouldn't try to follow it or modify it. + libc_libs = snapcraft.repo.Repo.get_package_libraries('libc6') + + # We don't care which lib we're testing with, as long as it's a .so. + libc_library_path = [lib for lib in libc_libs if '.so' in lib][0] + os.symlink(libc_library_path, os.path.join(c.builddir, 'libc-link')) + + c.build() + + self.assertThat( + os.path.join(c.installdir, 'libc-link'), + tests.LinkExists(libc_library_path)) + def test_copy_enable_cross_compilation(self): c = CopyPlugin('copy', self.mock_options, self.project_options) c.enable_cross_compilation() @@ -342,7 +363,7 @@ jsonschema.validate({'files': {'foo': ''}}, CopyPlugin.schema()) -class TestRecursivelyLink(TestCase): +class TestRecursivelyLink(tests.TestCase): def setUp(self): super().setUp() diff -Nru snapcraft-2.27.1+17.04/snapcraft/tests/plugins/test_dump.py snapcraft-2.28+17.04/snapcraft/tests/plugins/test_dump.py --- snapcraft-2.27.1+17.04/snapcraft/tests/plugins/test_dump.py 2017-01-28 06:12:26.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/tests/plugins/test_dump.py 2017-03-22 12:31:58.000000000 +0000 @@ -18,10 +18,10 @@ import snapcraft from snapcraft.plugins.dump import DumpPlugin -from snapcraft.tests import TestCase +from snapcraft import tests -class DumpPluginTestCase(TestCase): +class DumpPluginTestCase(tests.TestCase): def setUp(self): super().setUp() @@ -59,7 +59,6 @@ def test_dump_symlinks(self): plugin = DumpPlugin('dump', self.options, self.project_options) - os.makedirs(plugin.builddir) os.makedirs(os.path.join(plugin.builddir, 'subdir')) with open(os.path.join(plugin.builddir, 'file'), 'w') as f: f.write('foo') @@ -165,6 +164,25 @@ with open(destination, 'r') as f: self.assertEqual(f.read(), symlink['expected_contents']) + def test_dump_symlinks_to_libc(self): + plugin = DumpPlugin('dump', self.options, self.project_options) + os.makedirs(plugin.builddir) + + # Even though this symlink is absolute, since it's to libc the copy + # plugin shouldn't try to follow it or modify it. + libc_libs = snapcraft.repo.Repo.get_package_libraries('libc6') + + # We don't care which lib we're testing with, as long as it's a .so. + libc_library_path = [lib for lib in libc_libs if '.so' in lib][0] + os.symlink( + libc_library_path, os.path.join(plugin.builddir, 'libc-link')) + + plugin.build() + + self.assertThat( + os.path.join(plugin.installdir, 'libc-link'), + tests.LinkExists(libc_library_path)) + def test_dump_broken_symlink(self): self.options.source = 'src' plugin = DumpPlugin('dump', self.options, self.project_options) diff -Nru snapcraft-2.27.1+17.04/snapcraft/tests/plugins/test_gradle.py snapcraft-2.28+17.04/snapcraft/tests/plugins/test_gradle.py --- snapcraft-2.27.1+17.04/snapcraft/tests/plugins/test_gradle.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/tests/plugins/test_gradle.py 2017-03-22 12:31:58.000000000 +0000 @@ -37,10 +37,6 @@ self.project_options = snapcraft.ProjectOptions() - patcher = mock.patch('snapcraft.repo.Ubuntu') - self.ubuntu_mock = patcher.start() - self.addCleanup(patcher.stop) - # unset http and https proxies. self.useFixture(fixtures.EnvironmentVariable('http_proxy', None)) self.useFixture(fixtures.EnvironmentVariable('https_proxy', None)) diff -Nru snapcraft-2.27.1+17.04/snapcraft/tests/plugins/test_kbuild.py snapcraft-2.28+17.04/snapcraft/tests/plugins/test_kbuild.py --- snapcraft-2.27.1+17.04/snapcraft/tests/plugins/test_kbuild.py 2017-01-28 06:12:26.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/tests/plugins/test_kbuild.py 2017-03-22 12:31:58.000000000 +0000 @@ -40,10 +40,6 @@ self.options = Options() self.project_options = snapcraft.ProjectOptions() - patcher = mock.patch('snapcraft.repo.Ubuntu') - self.ubuntu_mock = patcher.start() - self.addCleanup(patcher.stop) - def test_schema(self): schema = kbuild.KBuildPlugin.schema() diff -Nru snapcraft-2.27.1+17.04/snapcraft/tests/plugins/test_kernel.py snapcraft-2.28+17.04/snapcraft/tests/plugins/test_kernel.py --- snapcraft-2.27.1+17.04/snapcraft/tests/plugins/test_kernel.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/tests/plugins/test_kernel.py 2017-03-23 12:02:13.000000000 +0000 @@ -20,7 +20,7 @@ from unittest import mock import fixtures -from testtools.matchers import HasLength +from testtools.matchers import Equals, HasLength import snapcraft from snapcraft import ( @@ -104,7 +104,7 @@ properties['kernel-image-target']['oneOf'], [{'type': 'string'}, {'type': 'object'}]) self.assertEqual( - properties['kernel-image-target']['default'], 'bzImage') + properties['kernel-image-target']['default'], '') self.assertEqual( properties['kernel-with-firmware']['type'], 'boolean') @@ -345,6 +345,9 @@ self.run_output_mock.assert_has_calls([ mock.call(modprobe_cmd + ['vfat'])]) + @mock.patch.object( + snapcraft._options.ProjectOptions, + 'kernel_arch', new='not_arm') def test_build_with_kconfigfile(self): self.options.kconfigfile = 'config' with open(self.options.kconfigfile, 'w') as f: @@ -382,6 +385,9 @@ self.assertEqual(config_contents, 'ACCEPT=y\n') self._assert_common_assets(plugin.installdir) + @mock.patch.object( + snapcraft._options.ProjectOptions, + 'kernel_arch', new='not_arm') def test_build_verbose_with_kconfigfile(self): fake_logger = fixtures.FakeLogger(level=logging.DEBUG) self.useFixture(fake_logger) @@ -438,6 +444,9 @@ self.assertEqual(config_contents, 'ACCEPT=y\n') self._assert_common_assets(plugin.installdir) + @mock.patch.object( + snapcraft._options.ProjectOptions, + 'kernel_arch', new='not_arm') def test_build_with_kconfigfile_and_kconfigs(self): self.options.kconfigfile = 'config' self.options.kconfigs = [ @@ -488,6 +497,9 @@ self.assertEqual(config_contents, expected_config) self._assert_common_assets(plugin.installdir) + @mock.patch.object( + snapcraft._options.ProjectOptions, + 'kernel_arch', new='not_arm') def test_build_with_defconfig_and_kconfigs(self): self.options.kdefconfig = ['defconfig'] self.options.kconfigs = [ @@ -545,6 +557,9 @@ self.assertEqual(config_contents, expected_config) self._assert_common_assets(plugin.installdir) + @mock.patch.object( + snapcraft._options.ProjectOptions, + 'kernel_arch', new='not_arm') def test_build_with_two_defconfigs(self): self.options.kdefconfig = ['defconfig', 'defconfig2'] @@ -585,6 +600,9 @@ self.assertTrue(os.path.exists(config_file)) self._assert_common_assets(plugin.installdir) + @mock.patch.object( + snapcraft._options.ProjectOptions, + 'kernel_arch', new='not_arm') def test_build_with_kconfigfile_and_dtbs(self): self.options.kconfigfile = 'config' with open(self.options.kconfigfile, 'w') as f: @@ -643,6 +661,9 @@ self.assertEqual( "No match for dtb 'fake-dtb.dtb' was found", str(raised)) + @mock.patch.object( + snapcraft._options.ProjectOptions, + 'kernel_arch', new='not_arm') def test_build_with_kconfigfile_and_modules(self): self.options.kconfigfile = 'config' with open(self.options.kconfigfile, 'w') as f: @@ -664,13 +685,17 @@ self.check_call_mock.side_effect = fake_unpack - def fake_modules(*args, **kwargs): - module_path = os.path.join( - plugin.installdir, 'lib', 'modules', '4.4.2', 'some-module.ko') - open(module_path, 'w').close() - return module_path + def fake_output(*args, **kwargs): + if args[0][:3] == ['modprobe', '-n', '--show-depends']: + module_path = os.path.join( + plugin.installdir, 'lib', 'modules', '4.4.2', + 'some-module.ko') + open(module_path, 'w').close() + return ('insmod {} enable_fbdev=1\n'.format(module_path)) + else: + raise Exception(args[0]) - self.run_output_mock.side_effect = fake_modules + self.run_output_mock.side_effect = fake_output plugin.build() @@ -705,6 +730,9 @@ self.assertEqual(config_contents, 'ACCEPT=y\n') self._assert_common_assets(plugin.installdir) + @mock.patch.object( + snapcraft._options.ProjectOptions, + 'kernel_arch', new='not_arm') def test_build_with_kconfigfile_and_firmware(self): self.options.kconfigfile = 'config' with open(self.options.kconfigfile, 'w') as f: @@ -756,6 +784,9 @@ self.assertTrue(os.path.exists(os.path.join( plugin.installdir, 'firmware', 'fake-fw-dir'))) + @mock.patch.object( + snapcraft._options.ProjectOptions, + 'kernel_arch', new='not_arm') def test_build_with_kconfigfile_and_no_firmware(self): self.options.kconfigfile = 'config' with open(self.options.kconfigfile, 'w') as f: @@ -830,7 +861,6 @@ def test_build_with_missing_system_map_fails(self): self.options.kconfigfile = 'config' - with open(self.options.kconfigfile, 'w') as f: f.write('ACCEPT=y\n') @@ -863,7 +893,7 @@ plugin = kernel.KernelPlugin('test-part', self.options, project_options) - self.assertEqual(plugin.make_targets, ['Image', 'modules']) + self.assertEqual(plugin.make_targets, ['Image', 'modules', 'dtbs']) def test_kernel_image_target_as_string(self): self.options.kernel_image_target = 'Image' @@ -871,7 +901,7 @@ plugin = kernel.KernelPlugin('test-part', self.options, project_options) - self.assertEqual(plugin.make_targets, ['Image', 'modules']) + self.assertEqual(plugin.make_targets, ['Image', 'modules', 'dtbs']) def test_kernel_image_target_non_existent(self): class Options: @@ -888,7 +918,7 @@ plugin = kernel.KernelPlugin('test-part', self.options, project_options) - self.assertEqual(plugin.make_targets, ['bzImage', 'modules']) + self.assertEqual(plugin.make_targets, ['bzImage', 'modules', 'dtbs']) @mock.patch.object(storeapi.StoreClient, 'download') def test_pull(self, download_mock): @@ -899,3 +929,36 @@ download_mock.assert_called_once_with( 'ubuntu-core', 'edge', plugin.os_snap, self.project_options.deb_arch, '') + + +class KernelPluginDefaulTargetsTestCase(tests.TestCase): + + scenarios = [ + ('amd64', {'deb_arch': 'amd64', 'expected': 'bzImage'}), + ('i386', {'deb_arch': 'i386', 'expected': 'bzImage'}), + ('arm64', {'deb_arch': 'arm64', 'expected': 'Image.gz'}), + ('armhf', {'deb_arch': 'armhf', 'expected': 'zImage'}), + ] + + def setUp(self): + super().setUp() + + class Options: + build_parameters = [] + kconfigfile = None + kdefconfig = [] + kconfigs = [] + kernel_image_target = '' + kernel_with_firmware = True + kernel_initrd_modules = [] + kernel_initrd_firmware = [] + kernel_device_trees = [] + kernel_initrd_compression = 'gz' + + self.options = Options() + + def test_default(self): + project = snapcraft.ProjectOptions(target_deb_arch=self.deb_arch) + plugin = kernel.KernelPlugin('test-part', self.options, project) + + self.assertThat(plugin.kernel_image_target, Equals(self.expected)) diff -Nru snapcraft-2.27.1+17.04/snapcraft/tests/plugins/test_make.py snapcraft-2.28+17.04/snapcraft/tests/plugins/test_make.py --- snapcraft-2.27.1+17.04/snapcraft/tests/plugins/test_make.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/tests/plugins/test_make.py 2017-03-22 12:31:58.000000000 +0000 @@ -39,10 +39,6 @@ self.options = Options() self.project_options = snapcraft.ProjectOptions() - patcher = mock.patch('snapcraft.repo.Ubuntu') - self.ubuntu_mock = patcher.start() - self.addCleanup(patcher.stop) - def test_schema(self): schema = make.MakePlugin.schema() diff -Nru snapcraft-2.27.1+17.04/snapcraft/tests/plugins/test_maven.py snapcraft-2.28+17.04/snapcraft/tests/plugins/test_maven.py --- snapcraft-2.27.1+17.04/snapcraft/tests/plugins/test_maven.py 2017-01-28 06:12:26.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/tests/plugins/test_maven.py 2017-03-22 12:31:58.000000000 +0000 @@ -39,10 +39,6 @@ self.options = Options() self.project_options = snapcraft.ProjectOptions() - patcher = mock.patch('snapcraft.repo.Ubuntu') - self.ubuntu_mock = patcher.start() - self.addCleanup(patcher.stop) - @staticmethod def _canonicalize_settings(settings): with io.StringIO(settings) as f: diff -Nru snapcraft-2.27.1+17.04/snapcraft/tests/plugins/test_plainbox_provider.py snapcraft-2.28+17.04/snapcraft/tests/plugins/test_plainbox_provider.py --- snapcraft-2.27.1+17.04/snapcraft/tests/plugins/test_plainbox_provider.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/tests/plugins/test_plainbox_provider.py 2017-03-22 12:31:58.000000000 +0000 @@ -1,6 +1,6 @@ # -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*- # -# Copyright (C) 2016 Canonical Ltd +# Copyright (C) 2016-2017 Canonical Ltd # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License version 3 as @@ -78,6 +78,68 @@ plugin.build() calls = [ + mock.call(['python3', 'manage.py', 'validate'], + env=os.environ.copy()), + mock.call(['python3', 'manage.py', 'build']), + mock.call(['python3', 'manage.py', 'i18n']), + mock.call(['python3', 'manage.py', 'install', + '--layout=relocatable', + '--prefix=/providers/test-part', + '--root={}'.format(plugin.installdir)]), + ] + self.mock_run.assert_has_calls(calls) + + for file_info in files: + with open(os.path.join(plugin.installdir, + file_info['path']), 'r') as f: + self.assertEqual(f.read(), file_info['expected']) + + def test_build_with_proivder_stage_dir(self): + self.useFixture(tests.fixture_setup.CleanEnvironment()) + + plugin = plainbox_provider.PlainboxProviderPlugin( + 'test-part', self.options, self.project_options) + + os.makedirs(plugin.sourcedir) + provider_path = os.path.join(self.project_options.stage_dir, + 'providers', 'test-provider') + os.makedirs(provider_path) + + # Place a few files with bad shebangs, and some files that shouldn't be + # changed. + files = [ + { + 'path': os.path.join(plugin.installdir, 'baz'), + 'contents': '#!/foo/bar/baz/python3', + 'expected': '#!/usr/bin/env python3', + }, + { + 'path': os.path.join(plugin.installdir, 'bin', 'foobar'), + 'contents': '#!/foo/baz/python3.5', + 'expected': '#!/usr/bin/env python3.5', + }, + { + 'path': os.path.join(plugin.installdir, 'foo'), + 'contents': 'foo', + 'expected': 'foo', + }, + { + 'path': os.path.join(plugin.installdir, 'bar'), + 'contents': 'bar\n#!/usr/bin/python3', + 'expected': 'bar\n#!/usr/bin/python3', + } + ] + + for file_info in files: + os.makedirs(os.path.dirname(file_info['path']), exist_ok=True) + with open(file_info['path'], 'w') as f: + f.write(file_info['contents']) + + plugin.build() + + calls = [ + mock.call(['python3', 'manage.py', 'validate'], + env={'PROVIDERPATH': provider_path}), mock.call(['python3', 'manage.py', 'build']), mock.call(['python3', 'manage.py', 'i18n']), mock.call(['python3', 'manage.py', 'install', diff -Nru snapcraft-2.27.1+17.04/snapcraft/tests/plugins/test_python.py snapcraft-2.28+17.04/snapcraft/tests/plugins/test_python.py --- snapcraft-2.27.1+17.04/snapcraft/tests/plugins/test_python.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/tests/plugins/test_python.py 2017-03-22 15:10:01.000000000 +0000 @@ -19,6 +19,7 @@ from glob import glob from unittest import mock +import fixtures from testtools.matchers import FileContains, HasLength import snapcraft @@ -237,6 +238,9 @@ def build_side_effect(): open(os.path.join(plugin.builddir, 'setup.py'), 'w').close() + os.mkdir(os.path.join(plugin.builddir, 'dist')) + open(os.path.join( + plugin.builddir, 'dist', 'package.tar'), 'w').close() mock_base_build.side_effect = build_side_effect @@ -407,3 +411,75 @@ site_path = glob(os.path.join( plugin.installdir, 'usr', 'lib', 'python*', 'sitecustomize.py'))[0] self.assertThat(site_path, FileContains(expected_sitecustomize)) + + @mock.patch.object(python.PythonPlugin, 'run_output') + @mock.patch.object(python.PythonPlugin, 'run') + def test_use_staged_python(self, run_mock, run_output_mock): + self.useFixture(fixture_setup.CleanEnvironment()) + + plugin = python.PythonPlugin('test-part', self.options, + self.project_options) + + setup_directories(plugin, self.options.python_version) + # Create the necessary hints to detect a staged python + staged_python_bin = os.path.join( + plugin.project.stage_dir, 'usr', 'bin', 'python3') + os.makedirs(os.path.dirname(staged_python_bin)) + open(staged_python_bin, 'w').close() + staged_python_include = os.path.join( + plugin.project.stage_dir, 'usr', 'include', 'python3.7') + os.makedirs(staged_python_include) + plugin.pull() + + pip_command = [ + os.path.join(plugin.project.stage_dir, 'usr', 'bin', 'python3'), + '-m', 'pip', 'download', '--disable-pip-version-check', + '--dest', os.path.join(plugin.partdir, 'packages'), '.' + ] + cwd = plugin.sourcedir + env = { + 'PYTHONUSERBASE': plugin.installdir, + 'PYTHONHOME': os.path.join(plugin.project.stage_dir, 'usr'), + 'PATH': '{}/usr/bin:$PATH'.format(plugin.installdir), + 'CPPFLAGS': '-I{}'.format(os.path.join( + plugin.project.stage_dir, 'usr', 'include', 'python3.7')) + } + run_mock.assert_called_once_with(pip_command, cwd=cwd, env=env) + + @mock.patch.object(python.PythonPlugin, 'run_output') + @mock.patch.object(python.PythonPlugin, 'run') + def test_use_staged_python_extra_cppflags(self, run_mock, run_output_mock): + self.useFixture(fixture_setup.CleanEnvironment()) + # Add some extra CPPFLAGS into the environment + self.useFixture(fixtures.EnvironmentVariable( + 'CPPFLAGS', '-I/opt/include')) + + plugin = python.PythonPlugin('test-part', self.options, + self.project_options) + + setup_directories(plugin, self.options.python_version) + # Create the necessary hints to detect a staged python + staged_python_bin = os.path.join( + plugin.project.stage_dir, 'usr', 'bin', 'python3') + os.makedirs(os.path.dirname(staged_python_bin)) + open(staged_python_bin, 'w').close() + staged_python_include = os.path.join( + plugin.project.stage_dir, 'usr', 'include', 'python3.7') + os.makedirs(staged_python_include) + + plugin.pull() + + pip_command = [ + os.path.join(plugin.project.stage_dir, 'usr', 'bin', 'python3'), + '-m', 'pip', 'download', '--disable-pip-version-check', + '--dest', os.path.join(plugin.partdir, 'packages'), '.' + ] + cwd = plugin.sourcedir + env = { + 'PYTHONUSERBASE': plugin.installdir, + 'PYTHONHOME': os.path.join(plugin.project.stage_dir, 'usr'), + 'PATH': '{}/usr/bin:$PATH'.format(plugin.installdir), + 'CPPFLAGS': '-I{} -I/opt/include'.format(os.path.join( + plugin.project.stage_dir, 'usr', 'include', 'python3.7')) + } + run_mock.assert_called_once_with(pip_command, cwd=cwd, env=env) diff -Nru snapcraft-2.27.1+17.04/snapcraft/tests/plugins/test_scons.py snapcraft-2.28+17.04/snapcraft/tests/plugins/test_scons.py --- snapcraft-2.27.1+17.04/snapcraft/tests/plugins/test_scons.py 2017-01-28 06:12:26.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/tests/plugins/test_scons.py 2017-03-22 12:31:58.000000000 +0000 @@ -37,10 +37,6 @@ self.options = Options() self.project_options = snapcraft.ProjectOptions() - patcher = mock.patch('snapcraft.repo.Ubuntu') - self.ubuntu_mock = patcher.start() - self.addCleanup(patcher.stop) - def test_schema(self): """Test validity of the Scons Plugin schema""" schema = scons.SconsPlugin.schema() diff -Nru snapcraft-2.27.1+17.04/snapcraft/tests/plugins/test_waf.py snapcraft-2.28+17.04/snapcraft/tests/plugins/test_waf.py --- snapcraft-2.27.1+17.04/snapcraft/tests/plugins/test_waf.py 2017-01-28 06:12:26.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/tests/plugins/test_waf.py 2017-03-22 12:31:58.000000000 +0000 @@ -37,10 +37,6 @@ self.options = Options() self.project_options = snapcraft.ProjectOptions() - patcher = mock.patch('snapcraft.repo.Ubuntu') - self.ubuntu_mock = patcher.start() - self.addCleanup(patcher.stop) - def test_schema(self): """Test validity of the Waf Plugin schema""" schema = waf.WafPlugin.schema() diff -Nru snapcraft-2.27.1+17.04/snapcraft/tests/repo/__init__.py snapcraft-2.28+17.04/snapcraft/tests/repo/__init__.py --- snapcraft-2.27.1+17.04/snapcraft/tests/repo/__init__.py 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/tests/repo/__init__.py 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,33 @@ +# -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*- +# +# Copyright (C) 2017 Canonical Ltd +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License version 3 as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . + +import logging +import tempfile + +import fixtures + +from snapcraft import tests + + +class RepoBaseTestCase(tests.TestCase): + + def setUp(self): + super().setUp() + fake_logger = fixtures.FakeLogger(level=logging.ERROR) + self.useFixture(fake_logger) + tempdirObj = tempfile.TemporaryDirectory() + self.addCleanup(tempdirObj.cleanup) + self.tempdir = tempdirObj.name diff -Nru snapcraft-2.27.1+17.04/snapcraft/tests/repo/test_base.py snapcraft-2.28+17.04/snapcraft/tests/repo/test_base.py --- snapcraft-2.27.1+17.04/snapcraft/tests/repo/test_base.py 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/tests/repo/test_base.py 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,270 @@ +# -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*- +# +# Copyright (C) 2017 Canonical Ltd +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License version 3 as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . + +import os +import stat +from textwrap import dedent + +from testtools.matchers import FileContains + +from snapcraft.internal import errors +from snapcraft.internal.repo import check_for_command +from snapcraft.internal.repo import BaseRepo +from snapcraft import tests +from . import RepoBaseTestCase + + +class CommandCheckTestCase(tests.TestCase): + + def test_check_for_command_not_installed(self): + self.assertRaises( + errors.MissingCommandError, + check_for_command, + 'missing-command') + + def test_check_for_command_installed(self): + check_for_command('sh') + + +class FixXmlToolsTestCase(RepoBaseTestCase): + + scenarios = [ + ('xml2-config only should fix', { + 'files': [ + { + 'path': os.path.join('root', 'usr', 'bin', 'xml2-config'), + 'content': 'prefix=/usr/foo', + 'expected': 'prefix=root/usr/foo', + }, + ] + }), + ('xml2-config only should not fix', { + 'files': [ + { + 'path': os.path.join('root', 'usr', 'bin', 'xml2-config'), + 'content': 'prefix=/foo', + 'expected': 'prefix=/foo', + }, + ] + }), + ('xslt-config only should fix', { + 'files': [ + { + 'path': os.path.join('root', 'usr', 'bin', 'xslt-config'), + 'content': 'prefix=/usr/foo', + 'expected': 'prefix=root/usr/foo', + }, + ] + }), + ('xslt-config only should not fix', { + 'files': [ + { + 'path': os.path.join('root', 'usr', 'bin', 'xslt-config'), + 'content': 'prefix=/foo', + 'expected': 'prefix=/foo', + }, + ] + }), + ('xml2-config and xslt-config', { + 'files': [ + { + 'path': os.path.join('root', 'usr', 'bin', 'xml2-config'), + 'content': 'prefix=/usr/foo', + 'expected': 'prefix=root/usr/foo', + }, + { + 'path': os.path.join('root', 'usr', 'bin', 'xslt-config'), + 'content': 'prefix=/usr/foo', + 'expected': 'prefix=root/usr/foo', + }, + ] + }), + ('xml2-config and xslt-config should not fix', { + 'files': [ + { + 'path': os.path.join('root', 'usr', 'bin', 'xml2-config'), + 'content': 'prefix=/foo', + 'expected': 'prefix=/foo', + }, + { + 'path': os.path.join('root', 'usr', 'bin', 'xslt-config'), + 'content': 'prefix=/foo', + 'expected': 'prefix=/foo', + }, + ] + }), + ] + + def test_fix_xml_tools(self): + for test_file in self.files: + path = test_file['path'] + os.makedirs(os.path.dirname(path), exist_ok=True) + with open(path, 'w') as f: + f.write(test_file['content']) + + BaseRepo('root').normalize('root') + + for test_file in self.files: + self.assertThat( + test_file['path'], FileContains(test_file['expected'])) + + +class FixShebangTestCase(RepoBaseTestCase): + + scenarios = [ + ('python bin dir', { + 'file_path': os.path.join('root', 'bin', 'a'), + 'content': '#!/usr/bin/python\nimport this', + 'expected': '#!/usr/bin/env python\nimport this', + }), + ('python3 bin dir', { + 'file_path': os.path.join('root', 'bin', 'd'), + 'content': '#!/usr/bin/python3\nraise Exception()', + 'expected': '#!/usr/bin/python3\nraise Exception()', + }), + ('sbin dir', { + 'file_path': os.path.join('root', 'sbin', 'b'), + 'content': '#!/usr/bin/python\nimport this', + 'expected': '#!/usr/bin/env python\nimport this', + }), + ('usr/bin dir', { + 'file_path': os.path.join('root', 'usr', 'bin', 'c'), + 'content': '#!/usr/bin/python\nimport this', + 'expected': '#!/usr/bin/env python\nimport this', + }), + ('usr/sbin dir', { + 'file_path': os.path.join('root', 'usr', 'sbin', 'd'), + 'content': '#!/usr/bin/python\nimport this', + 'expected': '#!/usr/bin/env python\nimport this', + }), + ('opt/bin dir', { + 'file_path': os.path.join('root', 'opt', 'bin', 'e'), + 'content': '#!/usr/bin/python\nraise Exception()', + 'expected': '#!/usr/bin/python\nraise Exception()', + }), + ] + + def test_fix_shebang(self): + os.makedirs(os.path.dirname(self.file_path), exist_ok=True) + with open(self.file_path, 'w') as fd: + fd.write(self.content) + + BaseRepo('root').normalize('root') + + with open(self.file_path, 'r') as fd: + self.assertEqual(fd.read(), self.expected) + + +class FixPkgConfigTestCase(RepoBaseTestCase): + + def test_fix_pkg_config(self): + pc_file = os.path.join(self.tempdir, 'granite.pc') + + with open(pc_file, 'w') as f: + f.write(dedent("""\ + prefix=/usr + exec_prefix=${prefix} + libdir=${prefix}/lib + includedir=${prefix}/include + + Name: granite + Description: elementary\'s Application Framework + Version: 0.4 + Libs: -L${libdir} -lgranite + Cflags: -I${includedir}/granite + Requires: cairo gee-0.8 glib-2.0 gio-unix-2.0 gobject-2.0 + """)) + + BaseRepo(self.tempdir).normalize(self.tempdir) + + expected_pc_file_content = dedent("""\ + prefix={}/usr + exec_prefix=${{prefix}} + libdir=${{prefix}}/lib + includedir=${{prefix}}/include + + Name: granite + Description: elementary's Application Framework + Version: 0.4 + Libs: -L${{libdir}} -lgranite + Cflags: -I${{includedir}}/granite + Requires: cairo gee-0.8 glib-2.0 gio-unix-2.0 gobject-2.0 + """.format(self.tempdir)) + + self.assertThat(pc_file, FileContains(expected_pc_file_content)) + + +class FixSymlinksTestCase(RepoBaseTestCase): + + scenarios = [ + ('rel-to-a', { + 'src': 'a', + 'dst': 'rel-to-a', + }), + ('abs-to-a', { + 'src': '/a', + 'dst': 'abs-to-a', + }), + ('abs-to-b', { + 'src': '/b', + 'dst': 'abs-to-b', + }), + ('rel-to-1', { + 'src': '1', + 'dst': 'rel-to-1', + }), + ('abs-to-1', { + 'src': '/1', + 'dst': 'abs-to-1' + }), + ] + + def setUp(self): + super().setUp() + os.makedirs('a') + open('1', mode='w').close() + + def test_fix_symlinks(self): + os.symlink(self.src, self.dst) + + BaseRepo(self.tempdir).normalize(self.tempdir) + + self.assertEqual(os.readlink(self.dst), self.src) + + +class FixSUIDTestCase(RepoBaseTestCase): + + scenarios = [ + ('suid_file', dict( + key='suid_file', test_mod=0o4765, expected_mod=0o0765)), + ('guid_file', dict( + key='guid_file', test_mod=0o2777, expected_mod=0o0777)), + ('suid_guid_file', dict( + key='suid_guid_file', test_mod=0o6744, expected_mod=0o0744)), + ('suid_guid_sticky_file', dict( + key='suid_guid_sticky_file', + test_mod=0o7744, expected_mod=0o1744)), + ] + + def test_fix_suid(self): + file = os.path.join(self.tempdir, self.key) + open(file, mode='w').close() + os.chmod(file, self.test_mod) + + BaseRepo(self.tempdir).normalize(self.tempdir) + + self.assertEqual( + stat.S_IMODE(os.stat(file).st_mode), self.expected_mod) diff -Nru snapcraft-2.27.1+17.04/snapcraft/tests/repo/test_deb.py snapcraft-2.28+17.04/snapcraft/tests/repo/test_deb.py --- snapcraft-2.27.1+17.04/snapcraft/tests/repo/test_deb.py 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/tests/repo/test_deb.py 2017-03-23 12:02:13.000000000 +0000 @@ -0,0 +1,304 @@ +# -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*- +# +# Copyright (C) 2015 Canonical Ltd +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License version 3 as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . + +import os +from subprocess import CalledProcessError +from unittest.mock import ANY, call, patch, MagicMock + +from testtools.matchers import ( + Contains, + FileExists, +) + +import snapcraft +from snapcraft.internal.repo import _deb +from snapcraft.internal.repo import errors +from snapcraft import tests +from . import RepoBaseTestCase + + +class UbuntuTestCase(RepoBaseTestCase): + + def setUp(self): + super().setUp() + patcher = patch('snapcraft.repo._deb.apt.Cache') + self.mock_cache = patcher.start() + self.addCleanup(patcher.stop) + + def _fetch_binary(download_dir, **kwargs): + path = os.path.join(download_dir, 'fake-package.deb') + open(path, 'w').close() + return path + + self.mock_package = MagicMock() + self.mock_package.candidate.fetch_binary.side_effect = _fetch_binary + self.mock_cache.return_value.get_changes.return_value = [ + self.mock_package] + + def test_get_pkg_name_parts_name_only(self): + name, version = _deb._get_pkg_name_parts('hello') + self.assertEqual('hello', name) + self.assertEqual(None, version) + + def test_get_pkg_name_parts_all(self): + name, version = _deb._get_pkg_name_parts('hello:i386=2.10-1') + self.assertEqual('hello:i386', name) + self.assertEqual('2.10-1', version) + + def test_get_pkg_name_parts_no_arch(self): + name, version = _deb._get_pkg_name_parts('hello=2.10-1') + self.assertEqual('hello', name) + self.assertEqual('2.10-1', version) + + @patch('snapcraft.internal.repo._deb.apt.apt_pkg') + def test_get_package(self, mock_apt_pkg): + project_options = snapcraft.ProjectOptions( + use_geoip=False) + ubuntu = _deb.Ubuntu(self.tempdir, project_options=project_options) + ubuntu.get(['fake-package']) + + mock_apt_pkg.assert_has_calls([ + call.config.set('Apt::Install-Recommends', 'False'), + call.config.find_file('Dir::Etc::Trusted'), + call.config.set('Dir::Etc::Trusted', ANY), + call.config.find_file('Dir::Etc::TrustedParts'), + call.config.set('Dir::Etc::TrustedParts', ANY), + call.config.clear('APT::Update::Post-Invoke-Success'), + ]) + + self.mock_cache.assert_has_calls([ + call(memonly=True, rootdir=ANY), + call().update(fetch_progress=ANY, sources_list=ANY), + call().open(), + ]) + + # __getitem__ is tricky + self.assertThat( + self.mock_cache.return_value.__getitem__.call_args_list, + Contains(call('fake-package'))) + + self.mock_package.assert_has_calls([ + call.candidate.fetch_binary(ANY, progress=ANY) + ]) + + # Verify that the package was actually fetched and copied into the + # requested location. + self.assertThat( + os.path.join(self.tempdir, 'download', 'fake-package.deb'), + FileExists()) + + @patch('snapcraft.repo._deb.apt.apt_pkg') + def test_get_multiarch_package(self, mock_apt_pkg): + project_options = snapcraft.ProjectOptions( + use_geoip=False) + ubuntu = _deb.Ubuntu(self.tempdir, project_options=project_options) + ubuntu.get(['fake-package:arch']) + + mock_apt_pkg.assert_has_calls([ + call.config.set('Apt::Install-Recommends', 'False'), + call.config.find_file('Dir::Etc::Trusted'), + call.config.set('Dir::Etc::Trusted', ANY), + call.config.find_file('Dir::Etc::TrustedParts'), + call.config.set('Dir::Etc::TrustedParts', ANY), + call.config.clear('APT::Update::Post-Invoke-Success'), + ]) + self.mock_cache.assert_has_calls([ + call(memonly=True, rootdir=ANY), + call().update(fetch_progress=ANY, sources_list=ANY), + call().open(), + ]) + + # __getitem__ is tricky + self.assertThat( + self.mock_cache.return_value.__getitem__.call_args_list, + Contains(call('fake-package:arch'))) + + self.mock_package.assert_has_calls([ + call.candidate.fetch_binary(ANY, progress=ANY) + ]) + + # Verify that the package was actually fetched and copied into the + # requested location. + self.assertThat( + os.path.join(self.tempdir, 'download', 'fake-package.deb'), + FileExists()) + + @patch('snapcraft.repo._deb._get_geoip_country_code_prefix') + def test_sources_is_none_uses_default(self, mock_cc): + mock_cc.return_value = 'ar' + + self.maxDiff = None + sources_list = _deb._format_sources_list( + '', use_geoip=True, deb_arch='amd64') + + expected_sources_list = \ + '''deb http://ar.archive.ubuntu.com/ubuntu/ xenial main restricted +deb http://ar.archive.ubuntu.com/ubuntu/ xenial-updates main restricted +deb http://ar.archive.ubuntu.com/ubuntu/ xenial universe +deb http://ar.archive.ubuntu.com/ubuntu/ xenial-updates universe +deb http://ar.archive.ubuntu.com/ubuntu/ xenial multiverse +deb http://ar.archive.ubuntu.com/ubuntu/ xenial-updates multiverse +deb http://security.ubuntu.com/ubuntu xenial-security main restricted +deb http://security.ubuntu.com/ubuntu xenial-security universe +deb http://security.ubuntu.com/ubuntu xenial-security multiverse +''' + self.assertEqual(sources_list, expected_sources_list) + + def test_no_geoip_uses_default_archive(self): + sources_list = _deb._format_sources_list( + _deb._DEFAULT_SOURCES, deb_arch='amd64', use_geoip=False) + + expected_sources_list = \ + '''deb http://archive.ubuntu.com/ubuntu/ xenial main restricted +deb http://archive.ubuntu.com/ubuntu/ xenial-updates main restricted +deb http://archive.ubuntu.com/ubuntu/ xenial universe +deb http://archive.ubuntu.com/ubuntu/ xenial-updates universe +deb http://archive.ubuntu.com/ubuntu/ xenial multiverse +deb http://archive.ubuntu.com/ubuntu/ xenial-updates multiverse +deb http://security.ubuntu.com/ubuntu xenial-security main restricted +deb http://security.ubuntu.com/ubuntu xenial-security universe +deb http://security.ubuntu.com/ubuntu xenial-security multiverse +''' + + self.assertEqual(sources_list, expected_sources_list) + + @patch('snapcraft.internal.repo._deb._get_geoip_country_code_prefix') + def test_sources_amd64_vivid(self, mock_cc): + self.maxDiff = None + mock_cc.return_value = 'ar' + + sources_list = _deb._format_sources_list( + _deb._DEFAULT_SOURCES, deb_arch='amd64', + use_geoip=True, release='vivid') + + expected_sources_list = \ + '''deb http://ar.archive.ubuntu.com/ubuntu/ vivid main restricted +deb http://ar.archive.ubuntu.com/ubuntu/ vivid-updates main restricted +deb http://ar.archive.ubuntu.com/ubuntu/ vivid universe +deb http://ar.archive.ubuntu.com/ubuntu/ vivid-updates universe +deb http://ar.archive.ubuntu.com/ubuntu/ vivid multiverse +deb http://ar.archive.ubuntu.com/ubuntu/ vivid-updates multiverse +deb http://security.ubuntu.com/ubuntu vivid-security main restricted +deb http://security.ubuntu.com/ubuntu vivid-security universe +deb http://security.ubuntu.com/ubuntu vivid-security multiverse +''' + self.assertEqual(sources_list, expected_sources_list) + + @patch('snapcraft.repo._deb._get_geoip_country_code_prefix') + def test_sources_armhf_trusty(self, mock_cc): + sources_list = _deb._format_sources_list( + _deb._DEFAULT_SOURCES, deb_arch='armhf', release='trusty') + + expected_sources_list = \ + '''deb http://ports.ubuntu.com/ubuntu-ports/ trusty main restricted +deb http://ports.ubuntu.com/ubuntu-ports/ trusty-updates main restricted +deb http://ports.ubuntu.com/ubuntu-ports/ trusty universe +deb http://ports.ubuntu.com/ubuntu-ports/ trusty-updates universe +deb http://ports.ubuntu.com/ubuntu-ports/ trusty multiverse +deb http://ports.ubuntu.com/ubuntu-ports/ trusty-updates multiverse +deb http://ports.ubuntu.com/ubuntu-ports trusty-security main restricted +deb http://ports.ubuntu.com/ubuntu-ports trusty-security universe +deb http://ports.ubuntu.com/ubuntu-ports trusty-security multiverse +''' + self.assertEqual(sources_list, expected_sources_list) + self.assertFalse(mock_cc.called) + + +class BuildPackagesTestCase(tests.TestCase): + + test_packages = {'package-not-installed': MagicMock(installed=False), + 'package-installed': MagicMock(installed=True), + 'another-uninstalled': MagicMock(installed=False), + 'another-installed': MagicMock(installed=True), + 'repeated-package': MagicMock(installed=False), + 'repeated-package': MagicMock(installed=False), + 'versioned-package=0.2': MagicMock(installed=False), + 'versioned-package': MagicMock(installed=True, + version='0.1')} + + def get_installable_packages(self, pkgs): + return [p for p in pkgs if not pkgs[p].installed] + + @patch('os.environ') + @patch('snapcraft.repo._deb.apt') + def install_test_packages(self, test_pkgs, mock_apt, mock_env): + mock_env.copy.return_value = {} + mock_apt_cache = mock_apt.Cache.return_value + mock_apt_cache_with = mock_apt_cache.__enter__.return_value + mock_apt_cache_with.__getitem__.side_effect = lambda p: test_pkgs[p] + + _deb.Ubuntu.install_build_packages(test_pkgs.keys()) + + @patch('snapcraft.repo._deb.is_dumb_terminal') + @patch('subprocess.check_call') + def test_install_build_package( + self, mock_check_call, mock_is_dumb_terminal): + mock_is_dumb_terminal.return_value = False + self.install_test_packages(self.test_packages) + + installable = self.get_installable_packages(self.test_packages) + mock_check_call.assert_has_calls([ + call('sudo apt-get --no-install-recommends -y ' + '-o Dpkg::Progress-Fancy=1 install'.split() + + sorted(set(installable)), + env={'DEBIAN_FRONTEND': 'noninteractive', + 'DEBCONF_NONINTERACTIVE_SEEN': 'true'}) + ]) + + @patch('snapcraft.repo._deb.is_dumb_terminal') + @patch('subprocess.check_call') + def test_install_buid_package_in_dumb_terminal( + self, mock_check_call, mock_is_dumb_terminal): + mock_is_dumb_terminal.return_value = True + self.install_test_packages(self.test_packages) + + installable = self.get_installable_packages(self.test_packages) + mock_check_call.assert_has_calls([ + call('sudo apt-get --no-install-recommends -y install'.split() + + sorted(set(installable)), + env={'DEBIAN_FRONTEND': 'noninteractive', + 'DEBCONF_NONINTERACTIVE_SEEN': 'true'}) + ]) + + @patch('subprocess.check_call') + def test_install_buid_package_marks_auto_installed(self, mock_check_call): + self.install_test_packages(self.test_packages) + + installable = self.get_installable_packages(self.test_packages) + mock_check_call.assert_has_calls([ + call('sudo apt-mark auto'.split() + + sorted(set(installable)), + env={'DEBIAN_FRONTEND': 'noninteractive', + 'DEBCONF_NONINTERACTIVE_SEEN': 'true'}) + ]) + + @patch('subprocess.check_call') + def test_mark_installed_auto_error_is_not_fatal(self, mock_check_call): + error = CalledProcessError(101, 'bad-cmd') + mock_check_call.side_effect = \ + lambda c, env: error if 'apt-mark' in c else None + self.install_test_packages(self.test_packages) + + def test_invalid_package_requested(self): + raised = self.assertRaises( + errors.BuildPackageNotFoundError, + _deb.Ubuntu.install_build_packages, + ['package-does-not-exist']) + + self.assertEqual( + "Could not find a required package in 'build-packages': " + '"The cache has no package named \'package-does-not-exist\'"', + str(raised)) diff -Nru snapcraft-2.27.1+17.04/snapcraft/tests/sources/test_bazaar.py snapcraft-2.28+17.04/snapcraft/tests/sources/test_bazaar.py --- snapcraft-2.27.1+17.04/snapcraft/tests/sources/test_bazaar.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/tests/sources/test_bazaar.py 2017-03-22 12:31:58.000000000 +0000 @@ -1,6 +1,6 @@ # -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*- # -# Copyright (C) 2015-2016 Canonical Ltd +# Copyright (C) 2015-2017 Canonical Ltd # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License version 3 as @@ -100,3 +100,14 @@ 'can\'t specify both source-tag and source-commit for ' 'a bzr source') self.assertEqual(raised.message, expected_message) + + def test_source_checksum_raises_exception(self): + raised = self.assertRaises( + sources.errors.IncompatibleOptionsError, + sources.Bazaar, + 'lp://mysource', 'source_dir', + source_checksum="md5/d9210476aac5f367b14e513bdefdee08") + + expected_message = ( + "can't specify a source-checksum for a bzr source") + self.assertEqual(raised.message, expected_message) diff -Nru snapcraft-2.27.1+17.04/snapcraft/tests/sources/test_checksum.py snapcraft-2.28+17.04/snapcraft/tests/sources/test_checksum.py --- snapcraft-2.27.1+17.04/snapcraft/tests/sources/test_checksum.py 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/tests/sources/test_checksum.py 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,82 @@ +# -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*- +# +# Copyright (C) 2017 Canonical Ltd +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License version 3 as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . + +from snapcraft import tests +from snapcraft.internal import sources + +import os +import hashlib +import zipfile +import sys + +if sys.version_info < (3, 6): + import sha3 # noqa + + +class TestChecksum(tests.TestCase): + + def setUp(self): + super().setUp() + + def test_invalid_checksum(self): + # Create a file for testing + os.makedirs(os.path.join('src')) + dummy_file = os.path.join('src', 'test') + open(dummy_file, 'w').close() + + self.assertRaises(AttributeError, sources.verify_checksum, '456/abcde', + dummy_file) + + def test_correct_checksum(self): + # Create zip file for testing + os.makedirs(os.path.join('src')) + file_to_zip = os.path.join('src', 'test.txt') + open(file_to_zip, 'w').close() + zip_file = zipfile.ZipFile(os.path.join('src', 'test.zip'), 'w') + zip_file.write(file_to_zip) + zip_file.close() + + calculated_checksum = hashlib.new('md5', + open(os.path.join('src', 'test.zip'), + 'rb').read()) + calculated_checksum = calculated_checksum.hexdigest() + + sources.verify_checksum('md5/' + calculated_checksum, + 'src/test.zip') + + def test_incorrect_checksum(self): + # Create zip file for testing + os.makedirs(os.path.join('src')) + file_to_zip = os.path.join('src', 'test.txt') + open(file_to_zip, 'w').close() + zip_file = zipfile.ZipFile(os.path.join('src', 'test.zip'), 'w') + zip_file.write(file_to_zip) + zip_file.close() + + incorrect_checksum = 'fe049cfba688aa1af88bc78191d7f904' + + calculated_checksum = hashlib.new('md5', + open(os.path.join('src', 'test.zip'), + 'rb').read()) + calculated_checksum = calculated_checksum.hexdigest() + + raised = self.assertRaises(sources.errors.DigestDoesNotMatchError, + sources.verify_checksum, + 'md5/' + incorrect_checksum, + 'src/test.zip') + + self.assertEqual(raised.expected, incorrect_checksum) + self.assertEqual(raised.calculated, calculated_checksum) diff -Nru snapcraft-2.27.1+17.04/snapcraft/tests/sources/test_git.py snapcraft-2.28+17.04/snapcraft/tests/sources/test_git.py --- snapcraft-2.27.1+17.04/snapcraft/tests/sources/test_git.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/tests/sources/test_git.py 2017-03-22 12:31:58.000000000 +0000 @@ -1,6 +1,6 @@ # -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*- # -# Copyright (C) 2015-2016 Canonical Ltd +# Copyright (C) 2015-2017 Canonical Ltd # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License version 3 as @@ -175,6 +175,17 @@ 'a git source' self.assertEqual(raised.message, expected_message) + def test_source_checksum_raises_exception(self): + raised = self.assertRaises( + sources.errors.IncompatibleOptionsError, + sources.Git, + 'git://mysource', 'source_dir', + source_checksum="md5/d9210476aac5f367b14e513bdefdee08") + + expected_message = ( + "can't specify a source-checksum for a git source") + self.assertEqual(raised.message, expected_message) + class TestGitConflicts(tests.TestCase): """Test that git pull errors don't kill the parser""" diff -Nru snapcraft-2.27.1+17.04/snapcraft/tests/sources/test_mercurial.py snapcraft-2.28+17.04/snapcraft/tests/sources/test_mercurial.py --- snapcraft-2.27.1+17.04/snapcraft/tests/sources/test_mercurial.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/tests/sources/test_mercurial.py 2017-03-22 12:31:58.000000000 +0000 @@ -1,6 +1,6 @@ # -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*- # -# Copyright (C) 2015-2016 Canonical Ltd +# Copyright (C) 2015-2017 Canonical Ltd # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License version 3 as @@ -139,3 +139,14 @@ expected_message = ( 'can\'t specify source-depth for a mercurial source') self.assertEqual(raised.message, expected_message) + + def test_source_checksum_raises_exception(self): + raised = self.assertRaises( + sources.errors.IncompatibleOptionsError, + sources.Mercurial, + 'hg://mysource', 'source_dir', + source_checksum="md5/d9210476aac5f367b14e513bdefdee08") + + expected_message = ( + "can't specify a source-checksum for a mercurial source") + self.assertEqual(raised.message, expected_message) diff -Nru snapcraft-2.27.1+17.04/snapcraft/tests/sources/test_subversion.py snapcraft-2.28+17.04/snapcraft/tests/sources/test_subversion.py --- snapcraft-2.27.1+17.04/snapcraft/tests/sources/test_subversion.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/tests/sources/test_subversion.py 2017-03-22 12:31:58.000000000 +0000 @@ -1,6 +1,6 @@ # -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*- # -# Copyright (C) 2015-2016 Canonical Ltd +# Copyright (C) 2015-2017 Canonical Ltd # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License version 3 as @@ -97,3 +97,14 @@ expected_message = ( 'can\'t specify source-depth for a Subversion source') self.assertEqual(raised.message, expected_message) + + def test_source_checksum_raises_exception(self): + raised = self.assertRaises( + sources.errors.IncompatibleOptionsError, + sources.Subversion, + 'svn://mysource', 'source_dir', + source_checksum="md5/d9210476aac5f367b14e513bdefdee08") + + expected_message = ( + "can't specify a source-checksum for a Subversion source") + self.assertEqual(raised.message, expected_message) diff -Nru snapcraft-2.27.1+17.04/snapcraft/tests/store/test_agent.py snapcraft-2.28+17.04/snapcraft/tests/store/test_agent.py --- snapcraft-2.27.1+17.04/snapcraft/tests/store/test_agent.py 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/tests/store/test_agent.py 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,66 @@ +# -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*- +# +# Copyright (C) 2017 Canonical Ltd +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License version 3 as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . + +import os +import platform + +import fixtures +from snapcraft import ( + ProjectOptions, + storeapi, + tests, + __version__ as snapcraft_version, +) + + +class UserAgentTestCase(tests.TestCase): + + def test_user_agent(self): + arch = ProjectOptions().deb_arch + expected_pre = 'snapcraft/{} '.format(snapcraft_version) + expected_post = ' {} ({})'.format( + '/'.join(platform.dist()[0:2]), # i.e. Ubuntu/16.04 + arch, + ) + actual = storeapi._agent.get_user_agent() + self.assertTrue(actual.startswith(expected_pre)) + self.assertTrue(actual.endswith(expected_post)) + + def test_in_travis_ci_env(self): + self.useFixture(fixtures.EnvironmentVariable( + 'TRAVIS_TESTING', '1')) + + self.assertTrue(storeapi._agent._is_ci_env()) + + def test_in_autopkgtest_ci_env(self): + self.useFixture(fixtures.EnvironmentVariable( + 'AUTOPKGTEST_TMP', '1')) + + self.assertTrue(storeapi._agent._is_ci_env()) + + def test_not_in_ci_env(self): + # unset any known testing environment vars + testing_vars = ['TRAVIS', 'AUTHPKGTEST_TMP'] + vars_to_unset = [] + for env_var in os.environ: + for test_var in testing_vars: + if env_var.startswith(test_var): + vars_to_unset.append(env_var) + + for var in vars_to_unset: + self.useFixture(fixtures.EnvironmentVariable(var, None)) + + self.assertFalse(storeapi._agent._is_ci_env()) diff -Nru snapcraft-2.27.1+17.04/snapcraft/tests/store/test_store_client.py snapcraft-2.28+17.04/snapcraft/tests/store/test_store_client.py --- snapcraft-2.27.1+17.04/snapcraft/tests/store/test_store_client.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/tests/store/test_store_client.py 2017-03-23 12:02:09.000000000 +0000 @@ -29,7 +29,9 @@ tests, ProjectOptions, ) -from snapcraft.storeapi import errors +from snapcraft.storeapi import ( + errors, +) from snapcraft.tests import fixture_setup @@ -888,19 +890,23 @@ def test_close_successfully(self): # Successfully closing a channels returns 'closed_channels' - # and 'channel_maps' from the Store. + # and 'channel_map_tree' from the Store. self.client.login('dummy', 'test correct password') - closed_channels, channel_maps = self.client.close_channels( + closed_channels, channel_map_tree = self.client.close_channels( 'snap-id', ['beta']) self.assertEqual(['beta'], closed_channels) self.assertEqual({ - 'amd64': [ - {'channel': 'stable', 'info': 'none'}, - {'channel': 'candidate', 'info': 'none'}, - {'channel': 'beta', 'info': 'specific', - 'revision': 42, 'version': '1.1'}, - {'channel': 'edge', 'info': 'tracking'}] - }, channel_maps) + 'latest': { + '16': { + 'amd64': [ + {'channel': 'stable', 'info': 'none'}, + {'channel': 'candidate', 'info': 'none'}, + {'channel': 'beta', 'info': 'specific', + 'revision': 42, 'version': '1.1'}, + {'channel': 'edge', 'info': 'tracking'}] + } + } + }, channel_map_tree) class MacaroonsTestCase(tests.TestCase): @@ -923,7 +929,7 @@ storeapi._macaroon_auth, conf) -class GetSnapHistoryTestCase(tests.TestCase): +class GetSnapRevisionsTestCase(tests.TestCase): def setUp(self): super().setUp() @@ -947,62 +953,64 @@ 'revision': 1, }] - def test_get_snap_history_without_login_raises_exception(self): + def test_get_snap_revisions_without_login_raises_exception(self): self.assertRaises( errors.InvalidCredentialsError, - self.client.get_snap_history, 'basic') + self.client.get_snap_revisions, 'basic') - def test_get_snap_history_successfully(self): + def test_get_snap_revisions_successfully(self): self.client.login('dummy', 'test correct password') - self.assertEqual(self.expected, self.client.get_snap_history('basic')) + self.assertEqual(self.expected, + self.client.get_snap_revisions('basic')) - def test_get_snap_history_filter_by_series(self): + def test_get_snap_revisions_filter_by_series(self): self.client.login('dummy', 'test correct password') self.assertEqual( self.expected, - self.client.get_snap_history('basic', series='16')) + self.client.get_snap_revisions('basic', series='16')) - def test_get_snap_history_filter_by_arch(self): + def test_get_snap_revisions_filter_by_arch(self): self.client.login('dummy', 'test correct password') self.assertEqual( [rev for rev in self.expected if rev['arch'] == 'amd64'], - self.client.get_snap_history('basic', arch='amd64')) + self.client.get_snap_revisions('basic', arch='amd64')) - def test_get_snap_history_filter_by_series_and_filter(self): + def test_get_snap_revisions_filter_by_series_and_filter(self): self.client.login('dummy', 'test correct password') self.assertEqual( [rev for rev in self.expected if '16' in rev['series'] and rev['arch'] == 'amd64'], - self.client.get_snap_history( + self.client.get_snap_revisions( 'basic', series='16', arch='amd64')) - def test_get_snap_history_filter_by_unknown_series(self): + def test_get_snap_revisions_filter_by_unknown_series(self): self.client.login('dummy', 'test correct password') e = self.assertRaises( storeapi.errors.SnapNotFoundError, - self.client.get_snap_history, 'basic', series='12') + self.client.get_snap_revisions, 'basic', series='12') self.assertEqual( "Snap 'basic' was not found in '12' series.", str(e)) - def test_get_snap_history_filter_by_unknown_arch(self): + def test_get_snap_revisions_filter_by_unknown_arch(self): self.client.login('dummy', 'test correct password') e = self.assertRaises( storeapi.errors.SnapNotFoundError, - self.client.get_snap_history, 'basic', arch='somearch') + self.client.get_snap_revisions, 'basic', arch='somearch') self.assertEqual( "Snap 'basic' for 'somearch' was not found in '16' series.", str(e)) - def test_get_snap_history_refreshes_macaroon(self): + def test_get_snap_revisions_refreshes_macaroon(self): self.client.login('dummy', 'test correct password') self.fake_store.needs_refresh = True - self.assertEqual(self.expected, self.client.get_snap_history('basic')) + self.assertEqual(self.expected, + self.client.get_snap_revisions('basic')) self.assertFalse(self.fake_store.needs_refresh) @mock.patch.object(storeapi.StoreClient, 'get_account_information') @mock.patch.object(storeapi.SCAClient, 'get') - def test_get_snap_history_server_error( + def test_get_snap_revisions_server_error( self, mock_sca_get, mock_account_info): mock_account_info.return_value = { 'snaps': { @@ -1015,10 +1023,10 @@ self.client.login('dummy', 'test correct password') e = self.assertRaises( - storeapi.errors.StoreSnapHistoryError, - self.client.get_snap_history, 'basic') + storeapi.errors.StoreSnapRevisionsError, + self.client.get_snap_revisions, 'basic') self.assertEqual( - "Error fetching history of snap id 'my_snap_id' for 'any arch' " + "Error fetching revisions of snap id 'my_snap_id' for 'any arch' " "in '16' series: 500 Server error.", str(e)) @@ -1030,40 +1038,46 @@ self.fake_store = self.useFixture(fixture_setup.FakeStore()) self.client = storeapi.StoreClient() self.expected = { - 'i386': [ - { - 'info': 'none', - 'channel': 'stable' - }, - { - 'info': 'none', - 'channel': 'beta' - }, - { - 'info': 'specific', - 'version': '1.0-i386', - 'channel': 'edge', - 'revision': 3 - }, - ], - 'amd64': [ - { - 'info': 'specific', - 'version': '1.0-amd64', - 'channel': 'stable', - 'revision': 2 - }, - { - 'info': 'specific', - 'version': '1.1-amd64', - 'channel': 'beta', - 'revision': 4 - }, - { - 'info': 'tracking', - 'channel': 'edge' - }, - ], + 'channel_map_tree': { + 'latest': { + '16': { + 'i386': [ + { + 'info': 'none', + 'channel': 'stable' + }, + { + 'info': 'none', + 'channel': 'beta' + }, + { + 'info': 'specific', + 'version': '1.0-i386', + 'channel': 'edge', + 'revision': 3 + }, + ], + 'amd64': [ + { + 'info': 'specific', + 'version': '1.0-amd64', + 'channel': 'stable', + 'revision': 2 + }, + { + 'info': 'specific', + 'version': '1.1-amd64', + 'channel': 'beta', + 'revision': 4 + }, + { + 'info': 'tracking', + 'channel': 'edge' + }, + ], + } + } + } } def test_get_snap_status_without_login_raises_exception(self): @@ -1083,14 +1097,28 @@ def test_get_snap_status_filter_by_arch(self): self.client.login('dummy', 'test correct password') + exp_arch = self.expected['channel_map_tree']['latest']['16']['amd64'] self.assertEqual( - {'amd64': self.expected['amd64']}, + {'channel_map_tree': { + 'latest': { + '16': { + 'amd64': exp_arch + } + } + }}, self.client.get_snap_status('basic', arch='amd64')) def test_get_snap_status_filter_by_series_and_filter(self): self.client.login('dummy', 'test correct password') + exp_arch = self.expected['channel_map_tree']['latest']['16']['amd64'] self.assertEqual( - {'amd64': self.expected['amd64']}, + {'channel_map_tree': { + 'latest': { + '16': { + 'amd64': exp_arch + } + } + }}, self.client.get_snap_status( 'basic', series='16', arch='amd64')) diff -Nru snapcraft-2.27.1+17.04/snapcraft/tests/test_lifecycle.py snapcraft-2.28+17.04/snapcraft/tests/test_lifecycle.py --- snapcraft-2.27.1+17.04/snapcraft/tests/test_lifecycle.py 2017-02-17 19:54:35.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/tests/test_lifecycle.py 2017-03-22 12:31:58.000000000 +0000 @@ -583,7 +583,7 @@ str(raised)) @mock.patch.object(snapcraft.BasePlugin, 'enable_cross_compilation') - @mock.patch('snapcraft.repo.install_build_packages') + @mock.patch('snapcraft.repo.Repo.install_build_packages') def test_pull_is_dirty_if_target_arch_changes( self, mock_install_build_packages, mock_enable_cross_compilation): self.make_snapcraft_yaml("""parts: diff -Nru snapcraft-2.27.1+17.04/snapcraft/tests/test_lxd.py snapcraft-2.28+17.04/snapcraft/tests/test_lxd.py --- snapcraft-2.27.1+17.04/snapcraft/tests/test_lxd.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/tests/test_lxd.py 2017-03-23 18:31:04.000000000 +0000 @@ -1,6 +1,6 @@ # -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*- # -# Copyright (C) 2016 Canonical Ltd +# Copyright (C) 2016-2017 Canonical Ltd # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License version 3 as @@ -14,7 +14,6 @@ # You should have received a copy of the GNU General Public License # along with this program. If not, see . -import os import logging from subprocess import CalledProcessError from unittest.mock import ( @@ -27,122 +26,90 @@ from snapcraft import tests from snapcraft import ProjectOptions -from snapcraft.internal import ( - cache, - lxd, -) +from snapcraft.internal import lxd -class LXDTestCase(tests.TestCase): +def check_output_side_effect(fail_on_remote=False, fail_on_default=False): + def call_effect(*args, **kwargs): + if args[0] == ['lxc', 'remote', 'get-default']: + if fail_on_default: + raise CalledProcessError(returncode=255, cmd=args[0]) + else: + return 'local'.encode('utf-8') + elif args[0] == ['lxc', 'list', 'my-remote:'] and fail_on_remote: + raise CalledProcessError(returncode=255, cmd=args[0]) + else: + return ''.encode('utf-8') + return call_effect - @patch('snapcraft.internal.lxd.check_call') - @patch('petname.Generate') - def test_cleanbuild(self, mock_pet, mock_call): - fake_logger = fixtures.FakeLogger(level=logging.INFO) - self.useFixture(fake_logger) - mock_pet.return_value = 'my-pet' +class LXDTestCase(tests.TestCase): - project_options = ProjectOptions() - lxd.Cleanbuilder('snap.snap', 'project.tar', project_options).execute() - expected_arch = project_options.deb_arch + def setUp(self): + super().setUp() - self.assertEqual( - 'Setting up container with project assets\n' - 'Copying snapcraft cache into container\n' - 'Waiting for a network connection...\n' - 'Network connection established\n' - 'Retrieved snap.snap\n', - fake_logger.output) + patcher = patch('snapcraft.internal.lxd.check_call') + self.check_call_mock = patcher.start() + self.addCleanup(patcher.stop) + + patcher = patch('snapcraft.internal.lxd.check_output') + self.check_output_mock = patcher.start() + self.check_output_mock.side_effect = check_output_side_effect() + self.addCleanup(patcher.stop) + + patcher = patch('snapcraft.internal.lxd.sleep', lambda _: None) + patcher.start() + self.addCleanup(patcher.stop) - mock_call.assert_has_calls([ - call(['lxc', 'launch', '-e', - 'ubuntu:xenial/{}'.format(expected_arch), - 'snapcraft-my-pet']), - call(['lxc', 'config', 'set', 'snapcraft-my-pet', - 'environment.SNAPCRAFT_SETUP_CORE', '1']), - call(['lxc', 'file', 'push', 'project.tar', - 'snapcraft-my-pet//root/project.tar']), - call(['lxc', 'exec', 'snapcraft-my-pet', '--', - 'tar', 'xvf', '/root/project.tar']), - call(['lxc', 'exec', 'snapcraft-my-pet', '--', - 'python3', '-c', - 'import urllib.request; ' - 'urllib.request.urlopen(' - '"http://start.ubuntu.com/connectivity-check.html", ' - 'timeout=5)']), - call(['lxc', 'exec', 'snapcraft-my-pet', '--', - 'apt-get', 'update']), - call(['lxc', 'exec', 'snapcraft-my-pet', '--', - 'apt-get', 'install', 'snapcraft', '-y']), - call(['lxc', 'exec', 'snapcraft-my-pet', '--', - 'snapcraft', 'snap', '--output', 'snap.snap']), - call(['lxc', 'file', 'pull', - 'snapcraft-my-pet//root/snap.snap', - 'snap.snap']), - call(['lxc', 'stop', '-f', 'snapcraft-my-pet']), - ]) - - @patch('snapcraft.internal.lxd.check_call') @patch('petname.Generate') - def test_cleanbuild_copies_cache(self, mock_pet, mock_call): + def test_cleanbuild(self, mock_pet): fake_logger = fixtures.FakeLogger(level=logging.INFO) self.useFixture(fake_logger) mock_pet.return_value = 'my-pet' - cache_dir = cache.SnapcraftCache().cache_root - os.makedirs(cache_dir) - open(os.path.join(cache_dir, 'foo'), 'w').close() - project_options = ProjectOptions() lxd.Cleanbuilder('snap.snap', 'project.tar', project_options).execute() expected_arch = project_options.deb_arch self.assertEqual( 'Setting up container with project assets\n' - 'Copying snapcraft cache into container\n' 'Waiting for a network connection...\n' 'Network connection established\n' 'Retrieved snap.snap\n', fake_logger.output) - mock_call.assert_has_calls([ + self.check_call_mock.assert_has_calls([ call(['lxc', 'launch', '-e', 'ubuntu:xenial/{}'.format(expected_arch), - 'snapcraft-my-pet']), - call(['lxc', 'config', 'set', 'snapcraft-my-pet', + 'local:snapcraft-my-pet']), + call(['lxc', 'config', 'set', 'local:snapcraft-my-pet', 'environment.SNAPCRAFT_SETUP_CORE', '1']), call(['lxc', 'file', 'push', 'project.tar', - 'snapcraft-my-pet//root/project.tar']), - call(['lxc', 'exec', 'snapcraft-my-pet', '--', + 'local:snapcraft-my-pet//root/project.tar']), + call(['lxc', 'exec', 'local:snapcraft-my-pet', '--', 'tar', 'xvf', '/root/project.tar']), - call(['lxc', 'exec', 'snapcraft-my-pet', '--', - 'mkdir', '-p', '/root/.cache/snapcraft/.']), - call(['lxc', 'file', 'push', os.path.join(cache_dir, 'foo'), - 'snapcraft-my-pet//root/.cache/snapcraft/./foo']), - call(['lxc', 'exec', 'snapcraft-my-pet', '--', + call(['lxc', 'exec', 'local:snapcraft-my-pet', '--', 'python3', '-c', 'import urllib.request; ' 'urllib.request.urlopen(' '"http://start.ubuntu.com/connectivity-check.html", ' 'timeout=5)']), - call(['lxc', 'exec', 'snapcraft-my-pet', '--', + call(['lxc', 'exec', 'local:snapcraft-my-pet', '--', 'apt-get', 'update']), - call(['lxc', 'exec', 'snapcraft-my-pet', '--', + call(['lxc', 'exec', 'local:snapcraft-my-pet', '--', 'apt-get', 'install', 'snapcraft', '-y']), - call(['lxc', 'exec', 'snapcraft-my-pet', '--', + call(['lxc', 'exec', 'local:snapcraft-my-pet', '--', 'snapcraft', 'snap', '--output', 'snap.snap']), call(['lxc', 'file', 'pull', - 'snapcraft-my-pet//root/snap.snap', + 'local:snapcraft-my-pet//root/snap.snap', 'snap.snap']), - call(['lxc', 'stop', '-f', 'snapcraft-my-pet']), + call(['lxc', 'stop', '-f', 'local:snapcraft-my-pet']), ]) - @patch('snapcraft.internal.lxd.check_call') @patch('snapcraft.internal.lxd.sleep') - def test_wait_for_network_loops(self, mock_sleep, mock_call): - mock_call.side_effect = CalledProcessError(-1, ['my-cmd']) + def test_wait_for_network_loops(self, mock_sleep): + self.check_call_mock.side_effect = CalledProcessError(-1, ['my-cmd']) cb = lxd.Cleanbuilder('snap.snap', 'project.tar', 'amd64') @@ -154,10 +121,9 @@ str(raised), "Command '['my-cmd']' returned non-zero exit status -1") - @patch('snapcraft.internal.lxd.check_call') @patch('snapcraft.internal.lxd.Cleanbuilder._container_run') @patch('snapcraft.internal.lxd.sleep') - def test_failed_build_with_debug(self, mock_sleep, mock_run, mock_call): + def test_failed_build_with_debug(self, mock_sleep, mock_run): call_list = [] def run_effect(*args, **kwargs): @@ -172,10 +138,9 @@ self.assertIn(['bash', '-i'], call_list) - @patch('snapcraft.internal.lxd.check_call') @patch('snapcraft.internal.lxd.Cleanbuilder._container_run') @patch('snapcraft.internal.lxd.sleep') - def test_failed_build_without_debug(self, mock_sleep, mock_run, mock_call): + def test_failed_build_without_debug(self, mock_sleep, mock_run): call_list = [] def run_effect(*args, **kwargs): @@ -194,9 +159,8 @@ self.assertNotIn(['bash', '-i'], call_list) - @patch('snapcraft.internal.lxd.check_call') @patch('petname.Generate') - def test_cleanbuild_with_remote(self, mock_pet, mock_call): + def test_cleanbuild_with_remote(self, mock_pet): mock_pet.return_value = 'my-pet' project_options = ProjectOptions() @@ -204,7 +168,7 @@ remote='my-remote').execute() expected_arch = project_options.deb_arch - mock_call.assert_has_calls([ + self.check_call_mock.assert_has_calls([ call(['lxc', 'launch', '-e', 'ubuntu:xenial/{}'.format(expected_arch), 'my-remote:snapcraft-my-pet']), @@ -232,18 +196,28 @@ call(['lxc', 'stop', '-f', 'my-remote:snapcraft-my-pet']), ]) - @patch('snapcraft.internal.lxd.check_call') @patch('snapcraft.internal.lxd.Cleanbuilder._container_run') @patch('snapcraft.internal.lxd.sleep') - def test_remote_does_not_exist(self, mock_sleep, mock_run, mock_call): - call_list = [] + def test_lxc_check_fails(self, mock_sleep, mock_run): + self.check_output_mock.side_effect = check_output_side_effect( + fail_on_default=True) - def call_effect(*args, **kwargs): - call_list.append(args[0]) - if args[0] == ['lxc', 'list', 'my-remote:']: - raise CalledProcessError(returncode=255, cmd=args[0]) + project_options = ProjectOptions(debug=False) + with ExpectedException( + lxd.SnapcraftEnvironmentError, + 'You must have LXD installed in order to use cleanbuild. ' + 'However, it is either not installed or not configured ' + 'properly.\n' + 'Refer to the documentation at ' + 'https://linuxcontainers.org/lxd/getting-started-cli.'): + lxd.Cleanbuilder('snap.snap', 'project.tar', + project_options) - mock_call.side_effect = call_effect + @patch('snapcraft.internal.lxd.Cleanbuilder._container_run') + @patch('snapcraft.internal.lxd.sleep') + def test_remote_does_not_exist(self, mock_sleep, mock_run): + self.check_output_mock.side_effect = check_output_side_effect( + fail_on_remote=True) project_options = ProjectOptions(debug=False) with ExpectedException(lxd.SnapcraftEnvironmentError, diff -Nru snapcraft-2.27.1+17.04/snapcraft/tests/test_parser.py snapcraft-2.28+17.04/snapcraft/tests/test_parser.py --- snapcraft-2.27.1+17.04/snapcraft/tests/test_parser.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/tests/test_parser.py 2017-03-22 12:31:58.000000000 +0000 @@ -25,12 +25,14 @@ from collections import OrderedDict import snapcraft # noqa, initialize yaml -from snapcraft.internal.errors import MissingCommandError +from snapcraft.internal.errors import ( + MissingCommandError, + SnapcraftEnvironmentError +) from snapcraft.internal import parser from snapcraft.internal.parser import ( _get_origin_data, _encode_origin, - BadSnapcraftYAMLError, PARTS_FILE, main, ) @@ -829,7 +831,7 @@ self.assertEqual(0, _get_part_list_count()) self.assertTrue( - 'Invalid wiki entry' + '1 wiki errors found' in fake_logger.output, 'Missing invalid wiki entry info in output') def test_missing_snapcraft_yaml_without_debug(self): @@ -1143,7 +1145,8 @@ 'snapcraft.yaml'), 'w') as fp: fp.write("") - self.assertRaises(BadSnapcraftYAMLError, _get_origin_data, + self.assertRaises(SnapcraftEnvironmentError, + _get_origin_data, self.tempdir_path) def test__get_origin_data_hidden_only(self): diff -Nru snapcraft-2.27.1+17.04/snapcraft/tests/test_project_loader.py snapcraft-2.28+17.04/snapcraft/tests/test_project_loader.py --- snapcraft-2.27.1+17.04/snapcraft/tests/test_project_loader.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/tests/test_project_loader.py 2017-03-22 12:31:58.000000000 +0000 @@ -1554,17 +1554,17 @@ config = project_loader.Config() environment = config.stage_env() self.assertIn( - 'LDFLAGS="$LDFLAGS -Wl,-z,nodefaultlib ' + 'LDFLAGS="$LDFLAGS ' '-Wl,--dynamic-linker={core_dynamic_linker} ' '-Wl,-rpath,' - '/snap/core/current/lib:' - '/snap/core/current/usr/lib:' - '/snap/core/current/lib/{arch_triplet}:' - '/snap/core/current/usr/lib/{arch_triplet}:' '/snap/test/current/lib:' '/snap/test/current/usr/lib:' '/snap/test/current/lib/{arch_triplet}:' - '/snap/test/current/usr/lib/{arch_triplet}"'.format( + '/snap/test/current/usr/lib/{arch_triplet}:' + '/snap/core/current/lib:' + '/snap/core/current/usr/lib:' + '/snap/core/current/lib/{arch_triplet}:' + '/snap/core/current/usr/lib/{arch_triplet}"'.format( core_dynamic_linker=dynamic_linker, arch_triplet=self.arch_triplet), environment) @@ -1735,6 +1735,27 @@ stage_dir=self.stage_dir, arch_triplet=self.arch_triplet)) + @unittest.mock.patch('snapcraft.ProjectOptions') + def test_parts_build_env_contains_parallel_build_count(self, pomock): + type(snapcraft.ProjectOptions.return_value).parallel_build_count = \ + unittest.mock.PropertyMock(return_value='fortytwo') + self.make_snapcraft_yaml("""name: test +version: "1" +summary: test +description: test +confinement: strict +grade: stable + +parts: + part1: + plugin: nil +""") + config = project_loader.Config() + part1 = [part for part in + config.parts.all_parts if part.name == 'part1'][0] + env = config.parts.build_env_for_part(part1) + self.assertIn('SNAPCRAFT_PARALLEL_BUILD_COUNT=fortytwo', env) + class ValidationBaseTestCase(tests.TestCase): diff -Nru snapcraft-2.27.1+17.04/snapcraft/tests/test_repo.py snapcraft-2.28+17.04/snapcraft/tests/test_repo.py --- snapcraft-2.27.1+17.04/snapcraft/tests/test_repo.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snapcraft/tests/test_repo.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,433 +0,0 @@ -# -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*- -# -# Copyright (C) 2015 Canonical Ltd -# -# This program is free software: you can redistribute it and/or modify -# it under the terms of the GNU General Public License version 3 as -# published by the Free Software Foundation. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with this program. If not, see . - -import fixtures -import logging -import os -import stat -import tempfile -from unittest.mock import ANY, call, patch, MagicMock -from testtools.matchers import ( - Contains, - FileExists, -) - -import snapcraft -from snapcraft import repo -from snapcraft import tests -from snapcraft.internal import errors - - -class RepoBaseTestCase(tests.TestCase): - - def setUp(self): - super().setUp() - fake_logger = fixtures.FakeLogger(level=logging.ERROR) - self.useFixture(fake_logger) - tempdirObj = tempfile.TemporaryDirectory() - self.addCleanup(tempdirObj.cleanup) - self.tempdir = tempdirObj.name - - -class UbuntuTestCase(RepoBaseTestCase): - - def setUp(self): - super().setUp() - patcher = patch('snapcraft.repo.apt.Cache') - self.mock_cache = patcher.start() - self.addCleanup(patcher.stop) - - def _fetch_binary(download_dir, **kwargs): - path = os.path.join(download_dir, 'fake-package.deb') - open(path, 'w').close() - return path - - self.mock_package = MagicMock() - self.mock_package.candidate.fetch_binary.side_effect = _fetch_binary - self.mock_cache.return_value.get_changes.return_value = [ - self.mock_package] - - @patch('snapcraft.repo.apt.apt_pkg') - def test_get_package(self, mock_apt_pkg): - project_options = snapcraft.ProjectOptions( - use_geoip=False) - ubuntu = repo.Ubuntu(self.tempdir, project_options=project_options) - ubuntu.get(['fake-package']) - - mock_apt_pkg.assert_has_calls([ - call.config.set('Apt::Install-Recommends', 'False'), - call.config.find_file('Dir::Etc::Trusted'), - call.config.set('Dir::Etc::Trusted', ANY), - call.config.find_file('Dir::Etc::TrustedParts'), - call.config.set('Dir::Etc::TrustedParts', ANY), - call.config.clear('APT::Update::Post-Invoke-Success'), - ]) - - self.mock_cache.assert_has_calls([ - call(memonly=True, rootdir=ANY), - call().update(fetch_progress=ANY, sources_list=ANY), - call().open(), - ]) - - # __getitem__ is tricky - self.assertThat( - self.mock_cache.return_value.__getitem__.call_args_list, - Contains(call('fake-package'))) - - self.mock_package.assert_has_calls([ - call.candidate.fetch_binary(ANY, progress=ANY) - ]) - - # Verify that the package was actually fetched and copied into the - # requested location. - self.assertThat( - os.path.join(self.tempdir, 'download', 'fake-package.deb'), - FileExists()) - - @patch('snapcraft.repo.apt.apt_pkg') - def test_get_multiarch_package(self, mock_apt_pkg): - project_options = snapcraft.ProjectOptions( - use_geoip=False) - ubuntu = repo.Ubuntu(self.tempdir, project_options=project_options) - ubuntu.get(['fake-package:arch']) - - mock_apt_pkg.assert_has_calls([ - call.config.set('Apt::Install-Recommends', 'False'), - call.config.find_file('Dir::Etc::Trusted'), - call.config.set('Dir::Etc::Trusted', ANY), - call.config.find_file('Dir::Etc::TrustedParts'), - call.config.set('Dir::Etc::TrustedParts', ANY), - call.config.clear('APT::Update::Post-Invoke-Success'), - ]) - self.mock_cache.assert_has_calls([ - call(memonly=True, rootdir=ANY), - call().update(fetch_progress=ANY, sources_list=ANY), - call().open(), - ]) - - # __getitem__ is tricky - self.assertThat( - self.mock_cache.return_value.__getitem__.call_args_list, - Contains(call('fake-package:arch'))) - - self.mock_package.assert_has_calls([ - call.candidate.fetch_binary(ANY, progress=ANY) - ]) - - # Verify that the package was actually fetched and copied into the - # requested location. - self.assertThat( - os.path.join(self.tempdir, 'download', 'fake-package.deb'), - FileExists()) - - @patch('snapcraft.repo._get_geoip_country_code_prefix') - def test_sources_is_none_uses_default(self, mock_cc): - mock_cc.return_value = 'ar' - - self.maxDiff = None - sources_list = repo._format_sources_list( - '', use_geoip=True, deb_arch='amd64') - - expected_sources_list = \ - '''deb http://ar.archive.ubuntu.com/ubuntu/ xenial main restricted -deb http://ar.archive.ubuntu.com/ubuntu/ xenial-updates main restricted -deb http://ar.archive.ubuntu.com/ubuntu/ xenial universe -deb http://ar.archive.ubuntu.com/ubuntu/ xenial-updates universe -deb http://ar.archive.ubuntu.com/ubuntu/ xenial multiverse -deb http://ar.archive.ubuntu.com/ubuntu/ xenial-updates multiverse -deb http://security.ubuntu.com/ubuntu xenial-security main restricted -deb http://security.ubuntu.com/ubuntu xenial-security universe -deb http://security.ubuntu.com/ubuntu xenial-security multiverse -''' - self.assertEqual(sources_list, expected_sources_list) - - def test_no_geoip_uses_default_archive(self): - sources_list = repo._format_sources_list( - repo._DEFAULT_SOURCES, deb_arch='amd64', use_geoip=False) - - expected_sources_list = \ - '''deb http://archive.ubuntu.com/ubuntu/ xenial main restricted -deb http://archive.ubuntu.com/ubuntu/ xenial-updates main restricted -deb http://archive.ubuntu.com/ubuntu/ xenial universe -deb http://archive.ubuntu.com/ubuntu/ xenial-updates universe -deb http://archive.ubuntu.com/ubuntu/ xenial multiverse -deb http://archive.ubuntu.com/ubuntu/ xenial-updates multiverse -deb http://security.ubuntu.com/ubuntu xenial-security main restricted -deb http://security.ubuntu.com/ubuntu xenial-security universe -deb http://security.ubuntu.com/ubuntu xenial-security multiverse -''' - - self.assertEqual(sources_list, expected_sources_list) - - @patch('snapcraft.repo._get_geoip_country_code_prefix') - def test_sources_amd64_vivid(self, mock_cc): - self.maxDiff = None - mock_cc.return_value = 'ar' - - sources_list = repo._format_sources_list( - repo._DEFAULT_SOURCES, deb_arch='amd64', - use_geoip=True, release='vivid') - - expected_sources_list = \ - '''deb http://ar.archive.ubuntu.com/ubuntu/ vivid main restricted -deb http://ar.archive.ubuntu.com/ubuntu/ vivid-updates main restricted -deb http://ar.archive.ubuntu.com/ubuntu/ vivid universe -deb http://ar.archive.ubuntu.com/ubuntu/ vivid-updates universe -deb http://ar.archive.ubuntu.com/ubuntu/ vivid multiverse -deb http://ar.archive.ubuntu.com/ubuntu/ vivid-updates multiverse -deb http://security.ubuntu.com/ubuntu vivid-security main restricted -deb http://security.ubuntu.com/ubuntu vivid-security universe -deb http://security.ubuntu.com/ubuntu vivid-security multiverse -''' - self.assertEqual(sources_list, expected_sources_list) - - @patch('snapcraft.repo._get_geoip_country_code_prefix') - def test_sources_armhf_trusty(self, mock_cc): - sources_list = repo._format_sources_list( - repo._DEFAULT_SOURCES, deb_arch='armhf', release='trusty') - - expected_sources_list = \ - '''deb http://ports.ubuntu.com/ubuntu-ports/ trusty main restricted -deb http://ports.ubuntu.com/ubuntu-ports/ trusty-updates main restricted -deb http://ports.ubuntu.com/ubuntu-ports/ trusty universe -deb http://ports.ubuntu.com/ubuntu-ports/ trusty-updates universe -deb http://ports.ubuntu.com/ubuntu-ports/ trusty multiverse -deb http://ports.ubuntu.com/ubuntu-ports/ trusty-updates multiverse -deb http://ports.ubuntu.com/ubuntu-ports trusty-security main restricted -deb http://ports.ubuntu.com/ubuntu-ports trusty-security universe -deb http://ports.ubuntu.com/ubuntu-ports trusty-security multiverse -''' - self.assertEqual(sources_list, expected_sources_list) - self.assertFalse(mock_cc.called) - - def test_fix_symlinks(self): - os.makedirs(self.tempdir + '/a') - open(self.tempdir + '/1', mode='w').close() - - os.symlink('a', self.tempdir + '/rel-to-a') - os.symlink('/a', self.tempdir + '/abs-to-a') - os.symlink('/b', self.tempdir + '/abs-to-b') - os.symlink('1', self.tempdir + '/rel-to-1') - os.symlink('/1', self.tempdir + '/abs-to-1') - - repo._fix_artifacts(debdir=self.tempdir) - - self.assertEqual(os.readlink(self.tempdir + '/rel-to-a'), 'a') - self.assertEqual(os.readlink(self.tempdir + '/abs-to-a'), 'a') - self.assertEqual(os.readlink(self.tempdir + '/abs-to-b'), '/b') - self.assertEqual(os.readlink(self.tempdir + '/rel-to-1'), '1') - self.assertEqual(os.readlink(self.tempdir + '/abs-to-1'), '1') - - def test_fix_pkg_config(self): - pc_file = os.path.join(self.tempdir, 'granite.pc') - - with open(pc_file, 'w') as f: - f.write('prefix=/usr\n') - f.write('exec_prefix=${prefix}\n') - f.write('libdir=${prefix}/lib\n') - f.write('includedir=${prefix}/include\n') - f.write('\n') - f.write('Name: granite\n') - f.write('Description: elementary\'s Application Framework\n') - f.write('Version: 0.4\n') - f.write('Libs: -L${libdir} -lgranite\n') - f.write('Cflags: -I${includedir}/granite\n') - f.write('Requires: cairo gee-0.8 glib-2.0 gio-unix-2.0 ' - 'gobject-2.0\n') - repo._fix_artifacts(debdir=self.tempdir) - - with open(pc_file) as f: - pc_file_content = f.read() - expected_pc_file_content = """prefix={}/usr -exec_prefix=${{prefix}} -libdir=${{prefix}}/lib -includedir=${{prefix}}/include - -Name: granite -Description: elementary's Application Framework -Version: 0.4 -Libs: -L${{libdir}} -lgranite -Cflags: -I${{includedir}}/granite -Requires: cairo gee-0.8 glib-2.0 gio-unix-2.0 gobject-2.0 -""".format(self.tempdir) - - self.assertEqual(pc_file_content, expected_pc_file_content) - - -class FixSUIDTestCase(RepoBaseTestCase): - - scenarios = [ - ('suid_file', dict( - key='suid_file', test_mod=0o4765, expected_mod=0o0765)), - ('guid_file', dict( - key='guid_file', test_mod=0o2777, expected_mod=0o0777)), - ('suid_guid_file', dict( - key='suid_guid_file', test_mod=0o6744, expected_mod=0o0744)), - ('suid_guid_sticky_file', dict( - key='suid_guid_sticky_file', - test_mod=0o7744, expected_mod=0o1744)), - ] - - def test_fix_suid(self): - file = os.path.join(self.tempdir, self.key) - open(file, mode='w').close() - os.chmod(file, self.test_mod) - - repo._fix_artifacts(debdir=self.tempdir) - self.assertEqual( - stat.S_IMODE(os.stat(file).st_mode), self.expected_mod) - - -class FixShebangTestCase(RepoBaseTestCase): - - scenarios = [ - ('python bin dir', { - 'file_path': os.path.join('root', 'bin', 'a'), - 'content': '#!/usr/bin/python\nimport this', - 'expected': '#!/usr/bin/env python\nimport this', - }), - ('python3 bin dir', { - 'file_path': os.path.join('root', 'bin', 'd'), - 'content': '#!/usr/bin/python3\nraise Exception()', - 'expected': '#!/usr/bin/python3\nraise Exception()', - }), - ('sbin dir', { - 'file_path': os.path.join('root', 'sbin', 'b'), - 'content': '#!/usr/bin/python\nimport this', - 'expected': '#!/usr/bin/env python\nimport this', - }), - ('usr/bin dir', { - 'file_path': os.path.join('root', 'usr', 'bin', 'c'), - 'content': '#!/usr/bin/python\nimport this', - 'expected': '#!/usr/bin/env python\nimport this', - }), - ('usr/sbin dir', { - 'file_path': os.path.join('root', 'usr', 'sbin', 'd'), - 'content': '#!/usr/bin/python\nimport this', - 'expected': '#!/usr/bin/env python\nimport this', - }), - ('opt/bin dir', { - 'file_path': os.path.join('root', 'opt', 'bin', 'e'), - 'content': '#!/usr/bin/python\nraise Exception()', - 'expected': '#!/usr/bin/python\nraise Exception()', - }), - ] - - def test_fix_shebang(self): - os.makedirs(os.path.dirname(self.file_path), exist_ok=True) - with open(self.file_path, 'w') as fd: - fd.write(self.content) - - repo._fix_shebangs('root') - - with open(self.file_path, 'r') as fd: - self.assertEqual(fd.read(), self.expected) - - -class BuildPackagesTestCase(tests.TestCase): - - test_packages = {'package-not-installed': MagicMock(installed=False), - 'package-installed': MagicMock(installed=True), - 'another-uninstalled': MagicMock(installed=False), - 'another-installed': MagicMock(installed=True), - 'repeated-package': MagicMock(installed=False), - 'repeated-package': MagicMock(installed=False)} - - def get_installable_packages(self, pkgs): - return [p for p in pkgs if not pkgs[p].installed] - - @patch('os.environ') - @patch('snapcraft.repo.apt') - def install_test_packages(self, test_pkgs, mock_apt, mock_env): - mock_env.copy.return_value = {} - mock_apt_cache = mock_apt.Cache.return_value - mock_apt_cache_with = mock_apt_cache.__enter__.return_value - mock_apt_cache_with.__getitem__.side_effect = lambda p: test_pkgs[p] - - repo.install_build_packages(test_pkgs.keys()) - - @patch('snapcraft.repo.is_dumb_terminal') - @patch('subprocess.check_call') - def test_install_buid_package( - self, mock_check_call, mock_is_dumb_terminal): - mock_is_dumb_terminal.return_value = False - self.install_test_packages(self.test_packages) - - installable = self.get_installable_packages(self.test_packages) - mock_check_call.assert_has_calls([ - call('sudo apt-get --no-install-recommends -y ' - '-o Dpkg::Progress-Fancy=1 install'.split() + - sorted(set(installable)), - env={'DEBIAN_FRONTEND': 'noninteractive', - 'DEBCONF_NONINTERACTIVE_SEEN': 'true'}) - ]) - - @patch('snapcraft.repo.is_dumb_terminal') - @patch('subprocess.check_call') - def test_install_buid_package_in_dumb_terminal( - self, mock_check_call, mock_is_dumb_terminal): - mock_is_dumb_terminal.return_value = True - self.install_test_packages(self.test_packages) - - installable = self.get_installable_packages(self.test_packages) - mock_check_call.assert_has_calls([ - call('sudo apt-get --no-install-recommends -y install'.split() + - sorted(set(installable)), - env={'DEBIAN_FRONTEND': 'noninteractive', - 'DEBCONF_NONINTERACTIVE_SEEN': 'true'}) - ]) - - @patch('subprocess.check_call') - def test_install_buid_package_marks_auto_installed(self, mock_check_call): - self.install_test_packages(self.test_packages) - - installable = self.get_installable_packages(self.test_packages) - mock_check_call.assert_has_calls([ - call('sudo apt-mark auto'.split() + - sorted(set(installable)), - env={'DEBIAN_FRONTEND': 'noninteractive', - 'DEBCONF_NONINTERACTIVE_SEEN': 'true'}) - ]) - - @patch('subprocess.check_call') - def test_mark_installed_auto_error_is_not_fatal(self, mock_check_call): - error = snapcraft.repo.subprocess.CalledProcessError(101, 'bad-cmd') - mock_check_call.side_effect = \ - lambda c, env: error if 'apt-mark' in c else None - self.install_test_packages(self.test_packages) - - def test_invalid_package_requested(self): - raised = self.assertRaises( - EnvironmentError, - repo.install_build_packages, - ['package-does-not-exist']) - - self.assertEqual( - "Could not find a required package in 'build-packages': " - '"The cache has no package named \'package-does-not-exist\'"', - str(raised)) - - -class CommandCheckTestCase(tests.TestCase): - - def test_check_for_command_not_installed(self): - self.assertRaises( - errors.MissingCommandError, - repo.check_for_command, - 'missing-command') - - def test_check_for_command_installed(self): - repo.check_for_command('sh') diff -Nru snapcraft-2.27.1+17.04/snaps_tests/demos_tests/test_busybox.py snapcraft-2.28+17.04/snaps_tests/demos_tests/test_busybox.py --- snapcraft-2.27.1+17.04/snaps_tests/demos_tests/test_busybox.py 2016-11-02 08:25:17.000000000 +0000 +++ snapcraft-2.28+17.04/snaps_tests/demos_tests/test_busybox.py 2017-03-22 12:31:58.000000000 +0000 @@ -34,7 +34,7 @@ '') self.addCleanup( self.run_command_in_snappy_testbed, - ['rm', '~/snap/busybox/*/busybox.test']) + ['rm', '~/snap/busybox/x*/busybox.test']) self.assert_command_in_snappy_testbed( ['/snap/bin/busybox.ls', '~/snap/busybox/x*/'], 'busybox.test\n') diff -Nru snapcraft-2.27.1+17.04/snaps_tests/demos_tests/test_godd.py snapcraft-2.28+17.04/snaps_tests/demos_tests/test_godd.py --- snapcraft-2.27.1+17.04/snaps_tests/demos_tests/test_godd.py 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/snaps_tests/demos_tests/test_godd.py 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,35 @@ +# -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*- +# +# Copyright (C) 2017 Canonical Ltd +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License version 3 as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . + +import snaps_tests + + +class GoddTestCase(snaps_tests.SnapsTestCase): + + snap_content_dir = 'godd' + + def test_godd(self): + # Build snap will raise an exception in case of error. + snap_path = self.build_snap(self.snap_content_dir) + # Install snap will raise an exception in case of error. + self.install_snap(snap_path, 'godd', '1.0') + + self.run_command_in_snappy_testbed( + 'sudo snap connect godd:mount-observe') + self.run_command_in_snappy_testbed('mkdir -p ~/snap/godd/common') + self.run_command_in_snappy_testbed('touch ~/snap/godd/common/test') + self.run_command_in_snappy_testbed( + '/snap/bin/godd ~/snap/godd/common/test /dev/null') diff -Nru snapcraft-2.27.1+17.04/snaps_tests/demos_tests/test_mosquitto.py snapcraft-2.28+17.04/snaps_tests/demos_tests/test_mosquitto.py --- snapcraft-2.27.1+17.04/snaps_tests/demos_tests/test_mosquitto.py 2016-11-02 08:25:17.000000000 +0000 +++ snapcraft-2.28+17.04/snaps_tests/demos_tests/test_mosquitto.py 2017-03-22 12:31:58.000000000 +0000 @@ -1,6 +1,6 @@ # -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*- # -# Copyright (C) 2016 Canonical Ltd +# Copyright (C) 2016-2017 Canonical Ltd # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License version 3 as @@ -38,7 +38,11 @@ ['/snap/bin/mosquitto.publish', 'test-mosquitto-topic', 'test-message'], '') self.assert_command_in_snappy_testbed( - ['cat', '/home/ubuntu/snap/mosquitto/*/' + ['/snap/bin/mosquitto.publish', 'test-mosquitto-topic', + 'exit'], '') + self.assert_command_in_snappy_testbed( + ['cat', '~/snap/mosquitto/x*/' 'mosquitto.subscriber.log'], 'MQTT subscriber connected.\n' - "test-mosquitto-topic b'test-message'\n") + "test-mosquitto-topic b'test-message'\n" + "test-mosquitto-topic b'exit'\n") diff -Nru snapcraft-2.27.1+17.04/snaps_tests/demos_tests/test_ros.py snapcraft-2.28+17.04/snaps_tests/demos_tests/test_ros.py --- snapcraft-2.27.1+17.04/snaps_tests/demos_tests/test_ros.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snaps_tests/demos_tests/test_ros.py 2017-03-22 20:15:48.000000000 +0000 @@ -14,13 +14,15 @@ # You should have received a copy of the GNU General Public License # along with this program. If not, see . -import snaps_tests - import os +import re import subprocess from platform import linux_distribution from unittest import skipUnless +import snapcraft +import snaps_tests + class ROSTestCase(snaps_tests.SnapsTestCase): @@ -29,7 +31,19 @@ @skipUnless(linux_distribution()[2] == 'xenial', 'This test fails on yakkety LP: #1614476') def test_ros(self): - snap_path = self.build_snap(self.snap_content_dir, timeout=1800) + try: + failed = True + snap_path = self.build_snap(self.snap_content_dir, timeout=1800) + failed = False + except snaps_tests.CommandError: + if snapcraft.ProjectOptions().deb_arch == 'arm64': + # https://bugs.launchpad.net/snapcraft/+bug/1662915 + self.expectFailure( + 'There are no arm64 Indigo packages in the ROS archive', + self.assertFalse, failed) + else: + raise + self.install_snap(snap_path, 'ros-example', '1.0') # check that the hardcoded /usr/bin/python in rosversion # is changed to using /usr/bin/env python @@ -38,3 +52,17 @@ "sed -n '/env/p;1q' prime/usr/bin/rosversion", cwd=os.path.join(self.path, self.snap_content_dir), shell=True) self.assertEqual(output, expected) + + # Regression test for LP: #1660852. Make sure --help actually gets + # passed to rosaunch instead of being eaten by setup.sh. + self.assert_command_in_snappy_testbed_with_regex([ + '/snap/bin/ros-example.launch-project', '--help'], + r'.*Usage: roslaunch.*') + + # Run the ROS system. By default this will never exit, but the demo + # supports an `exit-after-receive` parameter that, if true, will cause + # the system to shutdown after the listener has successfully received + # a message. + self.assert_command_in_snappy_testbed_with_regex([ + '/snap/bin/ros-example.launch-project', + 'exit-after-receive:=true'], r'.*I heard Hello world.*', re.DOTALL) diff -Nru snapcraft-2.27.1+17.04/snaps_tests/demos_tests/test_shared_ros.py snapcraft-2.28+17.04/snaps_tests/demos_tests/test_shared_ros.py --- snapcraft-2.27.1+17.04/snaps_tests/demos_tests/test_shared_ros.py 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/snaps_tests/demos_tests/test_shared_ros.py 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,60 @@ +# -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*- +# +# Copyright (C) 2015-2017 Canonical Ltd +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License version 3 as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . + +import snaps_tests + +import os +import re +import subprocess +from platform import linux_distribution +from unittest import skipUnless + + +class SharedROSTestCase(snaps_tests.SnapsTestCase): + + snap_content_dir = 'shared-ros' + + @skipUnless(linux_distribution()[2] == 'xenial', + 'This test fails on yakkety LP: #1614476') + def test_shared_ros(self): + ros_base_path = os.path.join(self.snap_content_dir, 'ros-base') + ros_app_path = os.path.join(self.snap_content_dir, 'ros-app') + + base_snap_path = self.build_snap(ros_base_path, timeout=1800) + + # Now tar up its staging area to be used to build ros-app + subprocess.check_call([ + 'tar', 'czf', os.path.join(ros_app_path, 'ros-base.tar.bz2'), '-C', + os.path.dirname(base_snap_path), 'stage'], cwd=self.src_dir) + + # Now build ros-app + app_snap_path = self.build_snap(ros_app_path, timeout=1800) + + # Install both snaps + self.install_snap(base_snap_path, 'ros-base', '1.0') + self.install_snap(app_snap_path, 'ros-app', '1.0') + + # Connect the content sharing interface + self.run_command_in_snappy_testbed( + 'sudo snap connect ros-app:ros-base ros-base:ros-base') + + # Run the ROS system. By default this will never exit, but the demo + # supports an `exit-after-receive` parameter that, if true, will cause + # the system to shutdown after the listener has successfully received + # a message. + self.assert_command_in_snappy_testbed_with_regex([ + '/snap/bin/ros-app.launch-project', + 'exit-after-receive:=true'], r'.*I heard Hello world.*', re.DOTALL) diff -Nru snapcraft-2.27.1+17.04/snaps_tests/demos_tests/tests.py snapcraft-2.28+17.04/snaps_tests/demos_tests/tests.py --- snapcraft-2.27.1+17.04/snaps_tests/demos_tests/tests.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snaps_tests/demos_tests/tests.py 2017-03-22 12:31:58.000000000 +0000 @@ -23,11 +23,6 @@ testscenarios.WithScenarios, snaps_tests.SnapsTestCase): scenarios = [ - ('godd', { - 'snap_content_dir': 'godd', - 'name': 'godd', - 'version': '1.0', - }), ('py2-project', { 'snap_content_dir': 'py2-project', 'name': 'spongeshaker', diff -Nru snapcraft-2.27.1+17.04/snaps_tests/demos_tests/test_tomcat_maven_webapp.py snapcraft-2.28+17.04/snaps_tests/demos_tests/test_tomcat_maven_webapp.py --- snapcraft-2.27.1+17.04/snaps_tests/demos_tests/test_tomcat_maven_webapp.py 2017-02-14 13:26:39.000000000 +0000 +++ snapcraft-2.28+17.04/snaps_tests/demos_tests/test_tomcat_maven_webapp.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,33 +0,0 @@ -# -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*- -# -# Copyright (C) 2015-2017 Canonical Ltd -# -# This program is free software: you can redistribute it and/or modify -# it under the terms of the GNU General Public License version 3 as -# published by the Free Software Foundation. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with this program. If not, see . - -import snapcraft - -import snaps_tests - - -class TomcatMavenWebappTestCase(snaps_tests.SnapsTestCase): - - snap_content_dir = 'tomcat-maven-webapp' - - def test_tomcat_maven_webapp(self): - if snapcraft.ProjectOptions().deb_arch == 'armhf': - # https://bugs.launchpad.net/snapcraft/+bug/1647405 - self.skipTest('The maven plugin does not support armhf') - snap_path = self.build_snap(self.snap_content_dir) - snap_name = 'tomcat-webapp-demo' - self.install_snap(snap_path, snap_name, '1.0') - self.assert_service_running(snap_name, 'tomcat') diff -Nru snapcraft-2.27.1+17.04/tools/api/conf.py snapcraft-2.28+17.04/tools/api/conf.py --- snapcraft-2.27.1+17.04/tools/api/conf.py 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/tools/api/conf.py 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,159 @@ +#!/usr/bin/env python3 +# -*- coding: utf-8 -*- +# +# snapcraft documentation build configuration file, created by +# sphinx-quickstart on Tue Feb 7 23:18:24 2017. +# +# This file is execfile()d with the current directory set to its +# containing dir. +# +# Note that not all possible configuration values are present in this +# autogenerated file. +# +# All configuration values have a default; values that are commented out +# serve to show the default. + +# If extensions (or modules to document with autodoc) are in another directory, +# add these directories to sys.path here. If the directory is relative to the +# documentation root, use os.path.abspath to make it absolute, like shown here. +# +import os +import sys +sys.path.insert(0, os.path.abspath('../../')) + + +# -- General configuration ------------------------------------------------ + +# If your documentation needs a minimal Sphinx version, state it here. +# +# needs_sphinx = '1.0' + +# Add any Sphinx extension module names here, as strings. They can be +# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom +# ones. +extensions = ['sphinx.ext.autodoc', + 'sphinx.ext.doctest', + 'sphinx.ext.todo', + 'sphinx.ext.coverage', + 'sphinx.ext.ifconfig', + 'sphinx.ext.viewcode'] + +# Add any paths that contain templates here, relative to this directory. +templates_path = ['_templates'] + +# The suffix(es) of source filenames. +# You can specify multiple suffix as a list of string: +# +# source_suffix = ['.rst', '.md'] +source_suffix = '.rst' + +# The master toctree document. +master_doc = 'index' + +# General information about the project. +project = 'snapcraft' +copyright = '2017, Canonical' +author = 'Canonical' + +# The version info for the project you're documenting, acts as replacement for +# |version| and |release|, also used in various other places throughout the +# built documents. +# +# The short X.Y version. +version = '2.26' +# The full version, including alpha/beta/rc tags. +release = '2.26' + +# The language for content autogenerated by Sphinx. Refer to documentation +# for a list of supported languages. +# +# This is also used if you do content translation via gettext catalogs. +# Usually you set "language" from the command line for these cases. +language = None + +# List of patterns, relative to source directory, that match files and +# directories to ignore when looking for source files. +# This patterns also effect to html_static_path and html_extra_path +exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store'] + +# The name of the Pygments (syntax highlighting) style to use. +pygments_style = 'sphinx' + +# If true, `todo` and `todoList` produce output, else they produce nothing. +todo_include_todos = True + + +# -- Options for HTML output ---------------------------------------------- + +# The theme to use for HTML and HTML Help pages. See the documentation for +# a list of builtin themes. +# +html_theme = 'alabaster' + +# Theme options are theme-specific and customize the look and feel of a theme +# further. For a list of options available for each theme, see the +# documentation. +# +# html_theme_options = {} + +# Add any paths that contain custom static files (such as style sheets) here, +# relative to this directory. They are copied after the builtin static files, +# so a file named "default.css" will overwrite the builtin "default.css". +html_static_path = ['_static'] + + +# -- Options for HTMLHelp output ------------------------------------------ + +# Output file base name for HTML help builder. +htmlhelp_basename = 'snapcraftdoc' + + +# -- Options for LaTeX output --------------------------------------------- + +latex_elements = { + # The paper size ('letterpaper' or 'a4paper'). + # + # 'papersize': 'letterpaper', + + # The font size ('10pt', '11pt' or '12pt'). + # + # 'pointsize': '10pt', + + # Additional stuff for the LaTeX preamble. + # + # 'preamble': '', + + # Latex figure (float) alignment + # + # 'figure_align': 'htbp', +} + +# Grouping the document tree into LaTeX files. List of tuples +# (source start file, target name, title, +# author, documentclass [howto, manual, or own class]). +latex_documents = [ + (master_doc, 'snapcraft.tex', 'snapcraft Documentation', + 'Canonical', 'manual'), +] + + +# -- Options for manual page output --------------------------------------- + +# One entry per manual page. List of tuples +# (source start file, name, description, authors, manual section). +man_pages = [ + (master_doc, 'snapcraft', 'snapcraft Documentation', + [author], 1) +] + + +# -- Options for Texinfo output ------------------------------------------- + +# Grouping the document tree into Texinfo files. List of tuples +# (source start file, target name, title, author, +# dir menu entry, description, category) +texinfo_documents = [ + (master_doc, 'snapcraft', 'snapcraft Documentation', + author, 'snapcraft', 'One line description of project.', + 'Miscellaneous'), +] diff -Nru snapcraft-2.27.1+17.04/tools/api/index.rst snapcraft-2.28+17.04/tools/api/index.rst --- snapcraft-2.27.1+17.04/tools/api/index.rst 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/tools/api/index.rst 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,8 @@ +Contents: + +.. toctree:: + :maxdepth: 3 + + modules + +Indices and tables diff -Nru snapcraft-2.27.1+17.04/tools/api/Makefile snapcraft-2.28+17.04/tools/api/Makefile --- snapcraft-2.27.1+17.04/tools/api/Makefile 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/tools/api/Makefile 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,20 @@ +# Minimal makefile for Sphinx documentation +# + +# You can set these variables from the command line. +SPHINXOPTS = +SPHINXBUILD = sphinx-build +SPHINXPROJ = snapcraft +SOURCEDIR = . +BUILDDIR = _build + +# Put it first so that "make" without argument is like "make help". +help: + @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) + +.PHONY: help Makefile + +# Catch-all target: route all unknown targets to Sphinx using the new +# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS). +%: Makefile + @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) \ No newline at end of file diff -Nru snapcraft-2.27.1+17.04/tools/gen_api_docs.sh snapcraft-2.28+17.04/tools/gen_api_docs.sh --- snapcraft-2.27.1+17.04/tools/gen_api_docs.sh 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/tools/gen_api_docs.sh 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,6 @@ +#!/bin/bash + +set -ev + +sphinx-apidoc -o tools/api snapcraft snapcraft/tests/* +make -C tools/api html diff -Nru snapcraft-2.27.1+17.04/tools/push_api_gh_pages.sh snapcraft-2.28+17.04/tools/push_api_gh_pages.sh --- snapcraft-2.27.1+17.04/tools/push_api_gh_pages.sh 1970-01-01 00:00:00.000000000 +0000 +++ snapcraft-2.28+17.04/tools/push_api_gh_pages.sh 2017-03-22 12:31:58.000000000 +0000 @@ -0,0 +1,20 @@ +#!/bin/bash + +set -ev + +GH_REPO="@github.com/snapcore/snapcraft.git" + +FULL_REPO="https://$GH_TOKEN$GH_REPO" + +cd tools/api/_build/html + +git init +git config user.name "snappy-m-o" +git config user.email "travis" + +# Do not use Jeykyll +touch .nojekyll + +git add . +git commit -m "Deployed to github pages." +git push --force --quiet $FULL_REPO master:gh-pages diff -Nru snapcraft-2.27.1+17.04/tools/retry_autopkgtest.sh snapcraft-2.28+17.04/tools/retry_autopkgtest.sh --- snapcraft-2.27.1+17.04/tools/retry_autopkgtest.sh 2016-11-17 12:41:28.000000000 +0000 +++ snapcraft-2.28+17.04/tools/retry_autopkgtest.sh 2017-03-22 12:31:58.000000000 +0000 @@ -3,22 +3,25 @@ # Retry the autopkgtest run in a pull request. # Arguments: # pr: The identifier of the pull request to test. -# [release[:architecture] ...]: A list of the names of the Ubuntu releases and -# architectures to test. By default, it will launch the tests in -# xenial:amd64. If only the release name is passed as an argument, amd64 -# will be used as the architecture. +# [release[:architecture[:test]] ...]: A list of the names of the Ubuntu releases and +# architectures and tests to run. By default, it will launch all the tests in +# `xenial:amd64`. If only the release name is passed as an argument, `amd64` +# will be used as the architecture. The possible autopkgtests are `integrationtests` +# and `snapstests`. # # Environment variables: # SNAPCRAFT_AUTOPKGTEST_SECRET: The secret to authenticate the test execution. # # Examples: -# Run the tests for pull request #123 in xenial amd64: +# Run all the tests for pull request #123 in xenial amd64: # ./tools/retry_autopkgtest.sh 123 -# Run the tests for pull request #123 in xenial armhf: +# Run all the tests for pull request #123 in xenial armhf: # ./tools/retry_autopkgtest.sh 123 xenial:armhf -# Run the tests for pull request #123 in xenial arm64, yakkety armhf and +# Run all the tests for pull request #123 in xenial arm64, yakkety armhf and # zesty amd64: # ./tools/retry_autopkgtest.sh 123 xenial:arm64 yakkety:armhf zesty +# Run the integration tests for pull request #123 in xenial amd64: +# ./tools/retry_autopkgtest.sh 123 xenial:amd64:integrationtests if [ -z "${SNAPCRAFT_AUTOPKGTEST_SECRET}" ]; then echo 'Set the secret to the environment variable SNAPCRAFT_AUTOPKGTEST_SECRET.' @@ -26,14 +29,14 @@ fi if [ "$#" -lt 1 ]; then - echo "Usage: "$0" [release[:architecture] ...]" + echo "Usage: "$0" [release[:architecture[:test]] ...]" exit 1 fi pr="$1" shift -testbeds=( "$@" ) -[ ${#testbeds[@]} -eq 0 ] && testbeds='xenial:amd64' +tests=( "$@" ) +[ ${#tests[@]} -eq 0 ] && tests='xenial:amd64' temp_dir="$(mktemp -d)" trap "rm -rf ${temp_dir}" EXIT @@ -45,9 +48,10 @@ # Save the secret to a file. echo "${SNAPCRAFT_AUTOPKGTEST_SECRET}" > "${temp_dir}/sec.txt" -for testbed in "${testbeds[@]}"; do - IFS=':' read -r release architecture <<< "$testbed" +for testrun in "${tests[@]}"; do + IFS=':' read -r release architecture testsuite <<< "$testrun" [ -z "$architecture" ] && architecture='amd64' + [ -n "$testsuite" ] && testname="&testname=${testsuite}" echo "Launching tests for the ${release} release in the ${architecture} architecture..." - "${temp_dir}/retry-github-test" "https://api.github.com/repos/snapcore/snapcraft/pulls/${pr}" "https://autopkgtest.ubuntu.com/request.cgi?release=${release}&arch=${architecture}&package=snapcraft&ppa=snappy-dev%2Fsnapcraft-daily" "${temp_dir}/sec.txt" + "${temp_dir}/retry-github-test" "https://api.github.com/repos/snapcore/snapcraft/pulls/${pr}" "https://autopkgtest.ubuntu.com/request.cgi?release=${release}&arch=${architecture}&package=snapcraft${testname}&ppa=snappy-dev%2Fsnapcraft-daily" "${temp_dir}/sec.txt" done diff -Nru snapcraft-2.27.1+17.04/.travis.yml snapcraft-2.28+17.04/.travis.yml --- snapcraft-2.27.1+17.04/.travis.yml 2017-02-17 19:54:46.000000000 +0000 +++ snapcraft-2.28+17.04/.travis.yml 2017-03-22 12:31:58.000000000 +0000 @@ -9,20 +9,13 @@ system_site_packages: true env: - global: - # Encrypted test user password. Generated with the travis gem: - # travis encrypt -r ubuntu-core/snapcraft TEST_USER_PASSWORD=$password - - secure: "gqtqTji8cie0Q2O+sRhE4MbTXGI0qTq8yPgRGFd9XlT/lB/EttFQKu72qycr/jyrvt809wjWM13QVqa5/71SoJd+Xzrmr1/leevx9Z/Wnv+IYkRAuGHW7iIDQb7MhQvpq3tw8hbGJzGxw03cUmjKJ89AAlGbwaURMat47lPsRXus8R7pl9S6r5owhBbmrQNaP9io0oPQDOAUf4pmJma1FTHAjXg0EdUwdXFUWToj15c7UJtB/MQNNTfjlwGA+/sPDqgthUEAzXmvUfXAZWnjQFZmq4ebvBIJEOQEPdLCXWGYdN2DAL7zp7WthrwFfgFFZb579rOBh0ETIMebUgBLoVSiPcn/bfzdYHcYKGf7lTJpoug5QENl+kZcuVyK7GUjf8O9tamhkYeMtUOy5Ubrcnv+Lfy9NsDPhKY05n+7tzzUVB1dePTrMHPuRZLl4OKku1AUN/S3A2xMrLO8vsWPVxcfxeb+4Y5ikYiHHpOozJHHDdPmj5raRIf3IH87W2PX0nJhg+gEgNHV1v3HBoyeqOPl4hl6/Fb9sCS/JAbbfcixkC54MHHI+opNSgZRvY0RORGHmuhHRGvfMxnwHmeOD51oV+SRGJS6A7qUq6GlBIy3/YlAY3LqqkWrHwm4EYttd4yM1FZ5s9pVnoSSJkgQ5vYK7A8a9AgZJlCcZnbzXtA=" matrix: - - TEST_SUITE=static DEPENDENCIES="apt install -y python3-pip && python3 -m pip install -r requirements-devel.txt" - - TEST_SUITE=unit DEPENDENCIES="apt install -y git libnacl-dev libsodium-dev libffi-dev libapt-pkg-dev libarchive-dev python3-pip squashfs-tools xdelta3 && python3 -m pip install -r requirements-devel.txt -r requirements.txt && python3 -m pip uninstall -y coverage && apt install -y python3-coverage" - - TEST_SUITE=integration TEST_STORE=fake DEPENDENCIES="apt install -y bzr curl git libnacl-dev libsodium-dev libffi-dev libapt-pkg-dev libarchive-dev mercurial python3-pip subversion squashfs-tools sudo snapd xdelta3 && python3 -m pip install -r requirements-devel.txt -r requirements.txt" - - TEST_SUITE=beta DEPENDENCIES="apt install -y python3-git python3-github" - -matrix: - include: - - env: CHECK_CLA=1 DEPENDENCIES="sudo apt install -y python-launchpadlib" - python: "2.7" + - DOCKER=1 CHECK_CLA=1 DEPENDENCIES="apt install -y git python-launchpadlib" + - DOCKER=1 TEST_SUITE=static DEPENDENCIES="apt install -y python3-pip && python3 -m pip install -r requirements-devel.txt" + - DOCKER=1 TEST_SUITE=unit DEPENDENCIES="apt install -y git libnacl-dev libsodium-dev libffi-dev libapt-pkg-dev libarchive-dev python3-pip squashfs-tools xdelta3 && python3 -m pip install -r requirements-devel.txt -r requirements.txt && python3 -m pip uninstall -y coverage && apt install -y python3-coverage" + - DOCKER=1 TEST_SUITE=integration DEPENDENCIES="apt install -y bzr curl git libnacl-dev libsodium-dev libffi-dev libapt-pkg-dev libarchive-dev mercurial python3-pip subversion squashfs-tools sudo snapd xdelta3 && python3 -m pip install -r requirements-devel.txt -r requirements.txt" + - DOCKER=1 GEN_DOCS=1 DEPENDENCIES="apt install libnacl-dev libsodium-dev libffi-dev libapt-pkg-dev libarchive-dev git make python3-pip python3-sphinx -y && python3 -m pip install -r requirements-devel.txt -r requirements.txt" + - DOCKER=1 TEST_SUITE=beta DEPENDENCIES="apt install -y python3-git python3-github wget" addons: apt: @@ -30,22 +23,28 @@ - python3-coverage install: - - if [ ! -z $CHECK_CLA ] && [ "$TRAVIS_PULL_REQUEST" != "false" ]; then $DEPENDENCIES; fi - - if [ ! -z $TEST_SUITE ]; then docker run --name builder -e TEST_USER_EMAIL=$TEST_USER_EMAIL - -e TEST_USER_PASSWORD=$TEST_USER_PASSWORD -e TEST_STORE=$TEST_STORE - -e TEST_SNAP_WITH_TRACKS=$TEST_SNAP_WITH_TRACKS + - if [ ! -z $DOCKER ]; then docker run --name builder + -e TRAVIS_COMMIT_RANGE=$TRAVIS_COMMIT_RANGE + -e TEST_USER_EMAIL=$TEST_USER_EMAIL -e TEST_USER_PASSWORD=$TEST_USER_PASSWORD + -e TEST_STORE=$TEST_STORE -e TEST_SNAP_WITH_TRACKS=$TEST_SNAP_WITH_TRACKS -e GITHUB_TEST_USER_NAME=$GITHUB_TEST_USER_NAME -e GITHUB_TEST_PASSWORD=$GITHUB_TEST_PASSWORD + -e SNAPCRAFT_AUTOPKGTEST_SECRET=$SNAPCRAFT_AUTOPKGTEST_SECRET -e GH_TOKEN=$GH_TOKEN -e LC_ALL=en_US.UTF-8 -v $PWD:$PWD -w $PWD -td ubuntu:xenial; fi - - if [ ! -z $TEST_SUITE ]; then docker exec -i builder locale-gen en_US.UTF-8; fi - - if [ ! -z $TEST_SUITE ]; then docker exec -i builder sed -i s/archive.ubuntu.com/us.archive.ubuntu.com/g /etc/apt/sources.list; fi - - if [ ! -z $TEST_SUITE ]; then docker exec -i builder apt update; fi - - if [ ! -z $TEST_SUITE ]; then docker exec -i builder sh -c "$DEPENDENCIES"; fi + - if [ ! -z $DOCKER ]; then docker exec -i builder locale-gen en_US.UTF-8; fi + - if [ ! -z $DOCKER ]; then docker exec -i builder sed -i s/archive.ubuntu.com/us.archive.ubuntu.com/g /etc/apt/sources.list; fi + - if [ ! -z $DOCKER ]; then docker exec -i builder apt update; fi + - if [ ! -z $DOCKER ]; then docker exec -i builder sh -c "$DEPENDENCIES"; fi script: - - if [ ! -z $CHECK_CLA ] && [ "$TRAVIS_PULL_REQUEST" != "false" ] && [ "$TRAVIS_EVENT_TYPE" != 'cron' ]; then ./tools/cla_check_in_travis.sh; fi - - if [ ! -z $TEST_SUITE ] && [ "$TEST_SUITE" != "beta" ]; then docker exec -i builder ./runtests.sh $TEST_SUITE; fi - - if [ $TRAVIS_EVENT_TYPE = "cron" ] && [ "$TEST_SUITE" == "beta" ]; then docker exec -i builder ./tools/make_beta_pr.py; fi + # CLA check, only in pull requests. + - if [ ! -z $CHECK_CLA ] && [ "$TRAVIS_PULL_REQUEST" != "false" ] && [ "$TRAVIS_EVENT_TYPE" != 'cron' ]; then docker exec -i builder ./tools/cla_check_in_travis.sh; fi + # Test suites from runtests.sh, in pull requests and landings to master. + - if [ ! -z $TEST_SUITE ] && [ "$TEST_SUITE" != "beta" ] && [ "$TRAVIS_EVENT_TYPE" != "cron" ]; then docker exec -i builder ./runtests.sh $TEST_SUITE; fi + # Trigger beta tests, only in the daily cron. + - if [ "$TRAVIS_EVENT_TYPE" = "cron" ] && [ "$TEST_SUITE" == "beta" ]; then docker exec -i builder ./tools/make_beta_pr.py; fi + # Generate docs, only in pushes to master. + - if [ ! -z $GEN_DOCS ] && [ "$TRAVIS_EVENT_TYPE" = "push" ] && [ "$TRAVIS_BRANCH" = "master" ]; then docker exec -i builder ./tools/gen_api_docs.sh && ./tools/push_api_gh_pages.sh; fi after_success: - python3 -m coverage xml