diff -Nru mitmproxy-5.1.1/CHANGELOG mitmproxy-6.0.2/CHANGELOG --- mitmproxy-5.1.1/CHANGELOG 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/CHANGELOG 1970-01-01 00:00:00.000000000 +0000 @@ -1,1036 +0,0 @@ -13 April 2020: mitmproxy 5.1.1 - * Fixed Docker images not starting due to missing shell - -13 April 2020: mitmproxy 5.1 - - ** Major Changes ** - * Initial Support for TLS 1.3 - - ** Full Changelog ** - * Reduce leaf certificate validity to one year due to upcoming browser changes (@mhils) - * Rename mitmweb's web_iface option to web_host for consistency (@oxr463) - * Sending a SIGTERM now exits mitmproxy without prompt, SIGINT still asks (@ThinkChaos) - * Don't force host header on outgoing requests (@mhils) - * Additional documentation and examples for WebSockets (@Kriechi) - * Gracefully handle hyphens in domain names (@matosconsulting) - * Fix header replacement count (@naivekun) - * Emit serverconnect event only after a connection has been established (@Prinzhorn) - * Fix ValueError in table mode of server replay flow (@ylmrx) - * HTTP/2: send all stream reset types to other connection (@rohfle) - * HTTP/2: fix WINDOW_UPDATE swallowed on closed streams (@Kriechi) - * Fix wrong behavior of --allow-hosts options (@BlownSnail) - * Additional and updated documentation for examples, WebSockets, Getting Started (@Kriechi) - -27 December 2019: mitmproxy 5.0.1 - - * Fixed precompiled Linux binaries to not crash in table mode - * Display webp images in mitmweb (@cixtor) - -16 December 2019: mitmproxy 5.0 - - ** Major Changes ** - * Added new Table UI (@Jessonsotoventura) - * Added EKU extension to certificates. This fixes support for macOS Catalina (@vin01) - - ** Security Fixes ** - * Fixed command injection vulnerabilities when exporting flows as curl/httpie commands (@cript0nauta) - * Do not echo unsanitized user input in HTTP error responses (@fimad) - - ** Full Changelog ** - * Moved to Github CI for Continuous Integration, dropping support for old Linux and macOS releases. (#3728) - * Vastly improved command parsing, in particular for setting flow filters (@typoon) - * Added a new flow export for raw responses (@mckeimic) - * URLs are now edited in an external editor (@Jessonsotoventura) - * mitmproxy now has a command history (@typoon) - * Added terminal like keyboard shortcuts for the command bar (ctrl+w, ctrl+a, ctrl+f, ...) (@typoon) - * Fixed issue with improper handling of non-ascii characters in URLs (@rjt-gupta) - * Filtering can now use unicode characters (@rjt-gupta) - * Fixed issue with user keybindings not being able to override default keybindings - * Improved installation instructions - * Added support for IPV6-only environments (@sethb157) - * Fixed bug with server replay (@rjt-gupta) - * Fixed issue with duplicate error responses (@ccssrryy) - * Users can now set a specific external editor using $MITMPROXY_EDITOR (@rjt-gupta) - * Config file can now be called `config.yml` or `config.yaml` (@ylmrx) - * Fixed crash on `view.focus.[next|prev]` (@ylmrx) - * Updated documentation to help using mitmproxy certificate on Android (@jannst) - * Added support to parse IPv6 entries from `pfctl` on MacOS. (@tomlabaude) - * Fixed instructions on how to build the documentation (@jannst) - * Added a new `--allow-hosts` option (@pierlon) - * Added support for zstd content-encoding (@tsaaristo) - * Fixed issue where the replay server would corrupt the Date header (@tonyb486) - * Improve speed for WebSocket interception (@MathieuBordere) - * Fixed issue with parsing JPEG files. (@lusceu) - * Improve example code style (@BoboTiG) - * Fixed issue converting void responses to HAR (@worldmind) - * Color coded http status codes in mitmweb (@arun-94) - * Added organization to generated certificates (@Abcdefghijklmnopqrstuvwxyzxyz) - * Errors are now displayed on sys.stderr (@JessicaFavin) - * Fixed issue with replay timestamps (@rjt-gupta) - * Fixed copying in mitmweb on macOS (@XZzYassin) - -31 July 2018: mitmproxy 4.0.4 - - * Security: Protect mitmweb against DNS rebinding. (CVE-2018-14505, @atx) - * Reduce certificate lifetime to two years to be conformant with - the current CA/Browser Forum Baseline Requirements. (@muffl0n) - (https://cabforum.org/2017/03/17/ballot-193-825-day-certificate-lifetimes/) - * Update cryptography to version 2.3. - -15 June 2018: mitmproxy 4.0.3 - - * Add support for IPv6 transparent mode on Windows (#3174) - * Add Docker images for ARMv7 - Raspberry Pi (#3190) - * Major overhaul of our release workflow - you probably won't notice it, but for us it's a big thing! - * Fix the Python version detection on Python 3.5, we now show a more intuitive error message (#3188) - * Fix application shutdown on Windows (#3172) - * Fix IPv6 scope suffixes in block addon (#3164) - * Fix options update when added (#3157) - * Fix "Edit Flow" button in mitmweb (#3136) - -15 June 2018: mitmproxy 4.0.2 - * Skipped! - - -17 May 2018: mitmproxy 4.0.1 - - ** Bugfixes ** - * The previous release had a packaging issue, so we bumped it to v4.0.1 and re-released it. - * This contains no actual bugfixes or new features. - -17 May 2018: mitmproxy 4.0 - - ** Features ** - * mitmproxy now requires Python 3.6! - * Moved the core to asyncio - which gives us a very significant performance boost! - * Reduce memory consumption by using `SO_KEEPALIVE` (#3076) - * Export request as httpie command (#3031) - * Configure mitmproxy console keybindings with the keys.yaml file. See docs for more. - - ** Breaking Changes ** - * The --conf command-line flag is now --confdir, and specifies the mitmproxy configuration - directory, instead of the options yaml file (which is at `config.yaml` under the configuration directory). - * `allow_remote` got replaced by `block_global` and `block_private` (#3100) - * No more custom events (#3093) - * The `cadir` option has been renamed to `confdir` - * We no longer magically capture print statements in addons and translate - them to logs. Please use `ctx.log.info` explicitly. - - ** Bugfixes ** - * Correctly block connections from remote clients with IPv4-mapped IPv6 client addresses (#3099) - * Expand `~` in paths during the `cut` command (#3078) - * Remove socket listen backlog constraint - * Improve handling of user script exceptions (#3050, #2837) - * Ignore signal errors on windows - * Fix traceback for commands with un-terminated escape characters (#2810) - * Fix request replay when proxy is bound to local interface (#2647) - * Fix traceback when running scripts on a flow twice (#2838) - * Fix traceback when killing intercepted flow (#2879) - * And lots of typos, docs improvements, revamped examples, and general fixes! - -05 April 2018: mitmproxy 3.0.4 - - * Fix an issue that caused mitmproxy to not retry HTTP requests on timeout. - - * Various other fixes (@kira0204, @fenilgandhi, @tran-tien-dat, @smonami, - @luzpaz, @fristonio, @kajojify, @Oliver-Fish, @hcbarry, @jplochocki, @MikeShi42, - @ghillu, @emilstahl) - -25 February 2018: mitmproxy 3.0.3 - - * Fix an issue that caused mitmproxy to lose keyboard control after spawning an external editor. - -23 February 2018: mitmproxy 3.0.1 - - * Fix a quote-related issue affecting the mitmproxy console command prompt. - -22 February 2018: mitmproxy 3.0 - - ** Major Changes ** - - * Commands: A consistent, typed mechanism that allows addons to expose actions - to users. - - * Options: A typed settings store for use by mitmproxy and addons. - - * Shift most of mitmproxy's own functionality into addons. - - * Major improvements to mitmproxy console, including an almost complete - rewrite of the user interface, integration of commands, key bindings, and - multi-pane layouts. - - * Major Improvements to mitmproxy’s web interface, mitmweb. (Matthew Shao, - Google Summer of Code 2017) - - * Major Improvements to mitmproxy’s content views and protocol layers (Ujjwal - Verma, Google Summer of Code 2017) - - * Faster JavaScript and CSS beautifiers. (Ujjwal Verma) - - - ** Minor Changes ** - - * Vastly improved JavaScript test coverage (Matthew Shao) - - * Options editor for mitmweb (Matthew Shao) - - * Static web-based flow viewer (Matthew Shao) - - * Request streaming for HTTP/1.x and HTTP/2 (Ujjwal Verma) - - * Implement more robust content views using Kaitai Struct (Ujjwal Verma) - - * Protobuf decoding now works without protoc being installed on the host - system (Ujjwal Verma) - - * PNG, GIF, and JPEG can now be parsed without Pillow, which simplifies - mitmproxy installation and moves parsing from unsafe C to pure Python (Ujjwal Verma) - - * Add parser for ICO files (Ujjwal Verma) - - * Migrate WebSockets implementation to wsproto. This reduces code size and - adds WebSocket compression support. (Ujjwal Verma) - - * Add “split view” to split mitmproxy’s UI into two separate panes. - - * Add key binding viewer and editor - - * Add a command to spawn a preconfigured Chrome browser instance from - mitmproxy - - * Fully support mitmproxy under the Windows Subsystem for Linux (WSL), work - around display errors - - * Add XSS scanner addon (@ddworken) - - * Add ability to toggle interception (@mattweidner) - - * Numerous documentation improvements (@pauloromeira, @rst0git, @rgerganov, - @fulldecent, @zhigang1992, @F1ashhimself, @vinaydargar, @jonathanrfisher1, - @BasThomas, @LuD1161, @ayamamori, @TomTasche) - - * Add filters for websocket flows (@s4chin) - - * Make it possible to create a response to CONNECT requests in http_connect - (@mengbiping) - - * Redirect stdout in scripts to ctx.log.warn (@nikofil) - - * Fix a crash when clearing the event log (@krsoninikhil) - - * Store the generated certificate for each flow (@dlenski) - - * Add --keep-host-header to retain the host header in reverse proxy mode - (@krsoninikhil) - - * Fix setting palette options (@JordanLoehr) - - * Fix a crash with brotli encoding (@whackashoe) - - * Provide certificate installation instructions on mitm.it (@ritiek) - - * Fix a bug where we did not properly fall back to IPv4 when IPv6 is unavailable (@titeuf87) - - * Fix transparent mode on IPv6-enabled macOS systems (@Ga-ryo) - - * Fix handling of HTTP messages with multiple Content-Length headers (@surajt97) - - * Fix IPv6 authority form parsing in CONNECT requests (@r1b) - - * Fix event log display in mitmweb (@syahn) - - * Remove private key from PKCS12 file in ~/.mitmproxy (@ograff). - - * Add LDAP as a proxy authentication backend (@charlesdhdt) - - * Use mypy to check the whole codebase (@iharsh234) - - * Fix a crash when duplicating flows (@iharsh234) - - * Fix testsuite when the path contains a “.” (@felixonmars) - - * Store proxy authentication with flows (@lymanZerga11) - - * Match ~d and ~u filters against pretty_host (@dequis) - - * Update WBXML content view (@davidpshaw) - - * Handle HEAD requests for mitm.it to support Chrome in transparent mode on - iOS (@tomlabaude) - - * Update dns spoofing example to use --keep-host-header (@krsoninikhil) - - * Call error handler on HTTPException (@tarnacious) - - * Make it possible to remove TLS from upstream HTTP connections - - * Update to pyOpenSSL 17.5, cryptography 2.1.4, and OpenSSL 1.1.0g - - * Make it possible to retroactively increase log verbosity. - - * Make logging from addons thread-safe - - * Tolerate imports in user scripts that match hook names (`from mitmproxy - import log`) - - * Update mitmweb to React 16, which brings performance improvements - - * Fix a bug where reverting duplicated flows crashes mitmproxy - - * Fix a bug where successive requests are sent to the wrong host after a - request has been redirected. - - * Fix a bug that binds outgoing connections to the wrong interface - - * Fix a bug where custom certificates are ignored in reverse proxy mode - - * Fix import of flows that have been created with mitmproxy 0.17 - - * Fix formatting of (IPv6) IP addresses in a number of places - - * Fix replay for HTTP/2 flows - - * Decouple mitmproxy version and flow file format version - - * Fix a bug where “mitmdump -nr” does not exit automatically - - * Fix a crash when exporting flows to curl - - * Fix formatting of sticky cookies - - * Improve script reloading reliability by polling the filesystem instead of using watchdog - - * Fix a crash when refreshing Set-Cookie headers - - * Add connection indicator to mitmweb to alert users when the proxy server stops running - - * Add support for certificates with cyrillic domains - - * Simplify output of mitmproxy --version - - * Add Request.make to simplify request creation in scripts - - * Pathoc: Include a host header on CONNECT requests - - * Remove HTML outline contentview (#2572) - - * Remove Python and Locust export (#2465) - - * Remove emojis from tox.ini because flake8 cannot parse that. :( - - - -28 April 2017: mitmproxy 2.0.2 - - * Fix mitmweb's Content-Security-Policy to work with Chrome 58+ - - * HTTP/2: actually use header normalization from hyper-h2 - - -15 March 2017: mitmproxy 2.0.1 - - * bump cryptography dependency - - * bump pyparsing dependency - - * HTTP/2: use header normalization from hyper-h2 - - -21 February 2017: mitmproxy 2.0 - - * HTTP/2 is now enabled by default. - - * Image ContentView: Parse images with Kaitai Struct (kaitai.io) instead of Pillow. - This simplifies installation, reduces binary size, and allows parsing in pure Python. - - * Web: Add missing flow filters. - - * Add transparent proxy support for OpenBSD. - - * Check the mitmproxy CA for expiration and warn the user to regenerate it if necessary. - - * Testing: Tremendous improvements, enforced 100% coverage for large parts of the - codebase, increased overall coverage. - - * Enforce individual coverage: one source file -> one test file with 100% coverage. - - * A myriad of other small improvements throughout the project. - - * Numerous bugfixes. - - -26 December 2016: mitmproxy 1.0 - - * All mitmproxy tools are now Python 3 only! We plan to support Python 3.5 and higher. - - * Web-Based User Interface: Mitmproxy now officially has a web-based user interface - called mitmweb. We consider it stable for all features currently exposed - in the UI, but it still misses a lot of mitmproxy’s options. - - * Windows Compatibility: With mitmweb, mitmproxy is now usable on Windows. - We are also introducing an installer (kindly sponsored by BitRock) that - simplifies setup. - - * Configuration: The config file format is now a single YAML file. In most cases, - converting to the new format should be trivial - please see the docs for - more information. - - * Console: Significant UI improvements - including sorting of flows by - size, type and url, status bar improvements, much faster indentation for - HTTP views, and more. - - * HTTP/2: Significant improvements, but is temporarily disabled by default - due to wide-spread protocol implementation errors on some large website - - * WebSocket: The protocol implementation is now mature, and is enabled by - default. Complete UI support is coming in the next release. Hooks for - message interception and manipulation are available. - - * A myriad of other small improvements throughout the project. - - -16 October 2016: mitmproxy 0.18 - - * Python 3 Compatibility for mitmproxy and pathod (Shadab Zafar, GSoC 2016) - - * Major improvements to mitmweb (Clemens Brunner & Jason Hao, GSoC 2016) - - * Internal Core Refactor: Separation of most features into isolated Addons - - * Initial Support for WebSockets - - * Improved HTTP/2 Support - - * Reverse Proxy Mode now automatically adjusts host headers and TLS Server Name Indication - - * Improved HAR export - - * Improved export functionality for curl, python code, raw http etc. - - * Flow URLs are now truncated in the console for better visibility - - * New filters for TCP, HTTP and marked flows. - - * Mitmproxy now handles comma-separated Cookie headers - - * Merge mitmproxy and pathod documentation - - * Mitmdump now sanitizes its console output to not include control characters - - * Improved message body handling for HTTP messages: - .raw_content provides the message body as seen on the wire - .content provides the decompressed body (e.g. un-gzipped) - .text provides the body decompressed and decoded body - - * New HTTP Message getters/setters for cookies and form contents. - - * Add ability to view only marked flows in mitmproxy - - * Improved Script Reloader (Always use polling, watch for whole directory) - - * Use tox for testing - - * Unicode support for tnetstrings - - * Add dumpfile converters for mitmproxy versions 0.11 and 0.12 - - * Numerous bugfixes - - -9 April 2016: mitmproxy 0.17 - - * Simplify repository and release structure. mitmproxy now comes as a single package, including netlib and pathod. - - * Rename the Python package from libmproxy to mitmproxy. - - * New option to add server certs to client chain (CVE-2016-2402, John Kozyrakis) - - * Enable HTTP/2 by default (Thomas Kriechbaumer) - - * Improved HAR extractor (Shadab Zafar) - - * Add icon for OSX and Windows binaries - - * Add content view for query parameters (Will Coster) - - * Initial work on Python 3 compatibility - - * locust.io export (Zohar Lorberbaum) - - * Fix XSS vulnerability in HTTP errors (Will Coster) - - * Numerous bugfixes and minor improvements - - -15 February 2016: mitmproxy 0.16 - - * Completely revised HTTP2 implementation based on hyper-h2 (Thomas Kriechbaumer) - - * Export flows as cURL command, Python code or raw HTTP (Shadab Zafar) - - * Fixed compatibility with the Android Emulator (Will Coster) - - * Script Reloader: Inline scripts are reloaded automatically if modified (Matthew Shao) - - * Inline script hooks for TCP mode (Michael J. Bazzinotti) - - * Add default ciphers to support iOS9 App Transport Security (Jorge Villacorta) - - * Basic Authentication for mitmweb (Guillem Anguera) - - * Exempt connections from interception based on TLS Server Name Indication (David Weinstein) - - * Provide Python Wheels for faster installation - - * Numerous bugfixes and minor improvements - - -4 December 2015: mitmproxy 0.15 - - * Support for loading and converting older dumpfile formats (0.13 and up) - - * Content views for inline script (@chrisczub) - - * Better handling of empty header values (Benjamin Lee/@bltb) - - * Fix a gnarly memory leak in mitmdump - - * A number of bugfixes and small improvements - - -6 November 2015: mitmproxy 0.14 - - * Statistics: 399 commits, 13 contributors, 79 closed issues, 37 closed - PRs, 103 days - - * Docs: Greatly updated docs now hosted on ReadTheDocs! - http://docs.mitmproxy.org - - * Docs: Fixed Typos, updated URLs etc. (Nick Badger, Ben Lerner, Choongwoo - Han, onlywade, Jurriaan Bremer) - - * mitmdump: Colorized TTY output - - * mitmdump: Use mitmproxy's content views for human-readable output (Chris - Czub) - - * mitmproxy and mitmdump: Support for displaying UTF8 contents - - * mitmproxy: add command line switch to disable mouse interaction (Timothy - Elliott) - - * mitmproxy: bug fixes (Choongwoo Han, sethp-jive, FreeArtMan) - - * mitmweb: bug fixes (Colin Bendell) - - * libmproxy: Add ability to fall back to TCP passthrough for non-HTTP - connections. - - * libmproxy: Avoid double-connect in case of TLS Server Name Indication. - This yields a massive speedup for TLS handshakes. - - * libmproxy: Prevent unnecessary upstream connections (macmantrl) - - * Inline Scripts: New API for HTTP Headers: - http://docs.mitmproxy.org/en/latest/dev/models.html#netlib.http.Headers - - * Inline Scripts: Properly handle exceptions in `done` hook - - * Inline Scripts: Allow relative imports, provide `__file__` - - * Examples: Add probabilistic TLS passthrough as an inline script - - * netlib: Refactored HTTP protocol handling code - - * netlib: ALPN support - - * netlib: fixed a bug in the optional certificate verification. - - * netlib: Initial Python 3.5 support (this is the first prerequisite for - 3.x support in mitmproxy) - - -24 July 2015: mitmproxy 0.13 - - * Upstream certificate validation. See the --verify-upstream-cert, - --upstream-trusted-confdir and --upstream-trusted-ca parameters. Thanks to - Kyle Morton (github.com/kyle-m) for his work on this. - - * Add HTTP transparent proxy mode. This uses the host headers from HTTP - traffic (rather than SNI and IP address information from the OS) to - implement perform transparent proxying. Thanks to github.com/ijiro123 for - this feature. - - * Add ~src and ~dst REGEX filters, allowing matching on source and - destination addresses in the form of : - - * mitmproxy console: change g/G keyboard shortcuts to match less. Thanks to - Jose Luis Honorato (github.com/jlhonora). - - * mitmproxy console: Flow marking and unmarking. Marked flows are not - deleted when the flow list is cleared. Thanks to Jake Drahos - (github.com/drahosj). - - * mitmproxy console: add marking of flows - - * Remove the certforward feature. It was added to allow exploitation of - #gotofail, which is no longer a common vulnerability. Permitting this - hugely increased the complexity of packaging and distributing mitmproxy. - - - - -3 June 2015: mitmproxy 0.12.1 - - * mitmproxy console: mouse interaction - scroll in the flow list, click on - flow to view, click to switch between tabs. - - * Update our crypto defaults: SHA256, 2048 bit RSA, 4096 bit DH parameters. - - * BUGFIX: crash under some circumstances when copying to clipboard. - - * BUGFIX: occasional crash when deleting flows. - - -18 May 2015: mitmproxy 0.12 - - * mitmproxy console: Significant revamp of the UI. The major changes are - listed below, and in addition almost every aspect of the UI has - been tweaked, and performance has improved significantly. - - * mitmproxy console: A new options screen has been created ("o" shortcut), - and many options that were previously manipulated directly via a - keybinding have been moved there. - - * mitmproxy console: Big improvement in palettes. This includes improvements - to all colour schemes. Palettes now set the terminal background colour by - default, and a new --palette-transparent option has been added to disable - this. - - * mitmproxy console: g/G shortcuts throughout mitmproxy console to jump - to the beginning/end of the current view. - - * mitmproxy console: switch palettes on the fly from the options screen. - - * mitmproxy console: A cookie editor has been added for mitmproxy console - at long last. - - * mitmproxy console: Various components of requests and responses can be - copied to the clipboard from mitmproxy - thanks to @marceloglezer. - - * Support for creating new requests from scratch in mitmproxy console (@marceloglezer). - - * SSLKEYLOGFILE environment variable to specify a logging location for TLS - master keys. This can be used with tools like Wireshark to allow TLS - decoding. - - * Server facing SSL cipher suite specification (thanks to Jim Shaver). - - * Official support for transparent proxying on FreeBSD - thanks to Mike C - (http://github.com/mike-pt). - - * Many other small bugfixes and improvemenets throughout the project. - - -29 Dec 2014: mitmproxy 0.11.2: - - * Configuration files - mitmproxy.conf, mitmdump.conf, common.conf in the - .mitmproxy directory. - * Better handling of servers that reject connections that are not SNI. - * Many other small bugfixes and improvements. - - -15 November 2014: mitmproxy 0.11.1: - - * Bug fixes: connection leaks some crashes - - -7 November 2014: mitmproxy 0.11: - - * Performance improvements for mitmproxy console - - * SOCKS5 proxy mode allows mitmproxy to act as a SOCKS5 proxy server - - * Data streaming for response bodies exceeding a threshold - (bradpeabody@gmail.com) - - * Ignore hosts or IP addresses, forwarding both HTTP and HTTPS traffic - untouched - - * Finer-grained control of traffic replay, including options to ignore - contents or parameters when matching flows (marcelo.glezer@gmail.com) - - * Pass arguments to inline scripts - - * Configurable size limit on HTTP request and response bodies - - * Per-domain specification of interception certificates and keys (see - --cert option) - - * Certificate forwarding, relaying upstream SSL certificates verbatim (see - --cert-forward) - - * Search and highlighting for HTTP request and response bodies in - mitmproxy console (pedro@worcel.com) - - * Transparent proxy support on Windows - - * Improved error messages and logging - - * Support for FreeBSD in transparent mode, using pf (zbrdge@gmail.com) - - * Content view mode for WBXML (davidshaw835@air-watch.com) - - * Better documentation, with a new section on proxy modes - - * Generic TCP proxy mode - - * Countless bugfixes and other small improvements - - * pathod: Hugely improved SSL support, including dynamic generation of certificates - using the mitproxy cacert - -7 November 2014: pathod 0.11: - - * Hugely improved SSL support, including dynamic generation of certificates - using the mitproxy cacert - - * pathoc -S dumps information on the remote SSL certificate chain - - * Big improvements to fuzzing, including random spec selection and memoization to avoid repeating randomly generated patterns - - * Reflected patterns, allowing you to embed a pathod server response specification in a pathoc request, resolving both on client side. This makes fuzzing proxies and other intermediate systems much better. - - -28 January 2014: mitmproxy 0.10: - - * Support for multiple scripts and multiple script arguments - - * Easy certificate install through the in-proxy web app, which is now - enabled by default - - * Forward proxy mode, that forwards proxy requests to an upstream HTTP server - - * Reverse proxy now works with SSL - - * Search within a request/response using the "/" and "n" shortcut keys - - * A view that beatifies CSS files if cssutils is available - - * Bug fix, documentation improvements, and more. - - -25 August 2013: mitmproxy 0.9.2: - - * Improvements to the mitmproxywrapper.py helper script for OSX. - - * Don't take minor version into account when checking for serialized file - compatibility. - - * Fix a bug causing resource exhaustion under some circumstances for SSL - connections. - - * Revamp the way we store interception certificates. We used to store these - on disk, they're now in-memory. This fixes a race condition related to - cert handling, and improves compatibility with Windows, where the rules - governing permitted file names are weird, resulting in errors for some - valid IDNA-encoded names. - - * Display transfer rates for responses in the flow list. - - * Many other small bugfixes and improvements. - - -25 August 2013: pathod 0.9.2: - - * Adapt to interface changes in netlib - - -16 June 2013: mitmproxy 0.9.1: - - * Use "correct" case for Content-Type headers added by mitmproxy. - - * Make UTF environment detection more robust. - - * Improved MIME-type detection for viewers. - - * Always read files in binary mode (Windows compatibility fix). - - * Some developer documentation. - - -15 May 2013: mitmproxy 0.9: - - * Upstream certs mode is now the default. - - * Add a WSGI container that lets you host in-proxy web applications. - - * Full transparent proxy support for Linux and OSX. - - * Introduce netlib, a common codebase for mitmproxy and pathod - (http://github.com/cortesi/netlib). - - * Full support for SNI. - - * Color palettes for mitmproxy, tailored for light and dark terminal - backgrounds. - - * Stream flows to file as responses arrive with the "W" shortcut in - mitmproxy. - - * Extend the filter language, including ~d domain match operator, ~a to - match asset flows (js, images, css). - - * Follow mode in mitmproxy ("F" shortcut) to "tail" flows as they arrive. - - * --dummy-certs option to specify and preserve the dummy certificate - directory. - - * Server replay from the current captured buffer. - - * Huge improvements in content views. We now have viewers for AMF, HTML, - JSON, Javascript, images, XML, URL-encoded forms, as well as hexadecimal - and raw views. - - * Add Set Headers, analogous to replacement hooks. Defines headers that are set - on flows, based on a matching pattern. - - * A graphical editor for path components in mitmproxy. - - * A small set of standard user-agent strings, which can be used easily in - the header editor. - - * Proxy authentication to limit access to mitmproxy - - * pathod: Proxy mode. You can now configure clients to use pathod as an - HTTP/S proxy. - - * pathoc: Proxy support, including using CONNECT to tunnel directly to - targets. - - * pathoc: client certificate support. - - * pathod: API improvements, bugfixes. - - -15 May 2013: pathod 0.9 (version synced with mitmproxy): - - * Pathod proxy mode. You can now configure clients to use pathod as an - HTTP/S proxy. - - * Pathoc proxy support, including using CONNECT to tunnel directly to - targets. - - * Pathoc client certificate support. - - * API improvements, bugfixes. - - -16 November 2012: pathod 0.3: - - A release focusing on shoring up our fuzzing capabilities, especially with - pathoc. - - * pathoc -q and -r options, output full request and response text. - - * pathod -q and -r options, add full request and response text to pathod's - log buffer. - - * pathoc and pathod -x option, makes -q and -r options log in hex dump - format. - - * pathoc -C option, specify response codes to ignore. - - * pathoc -T option, instructs pathoc to ignore timeouts. - - * pathoc -o option, a one-shot mode that exits after the first non-ignored - response. - - * pathoc and pathod -e option, which explains the resulting message by - expanding random and generated portions, and logging a reproducible - specification. - - * Streamline the specification language. HTTP response message is now - specified using the "r" mnemonic. - - * Add a "u" mnemonic for specifying User-Agent strings. Add a set of - standard user-agent strings accessible through shortcuts. - - * Major internal refactoring and cleanup. - - * Many bugfixes. - - -22 August 2012: pathod 0.2: - - * Add pathoc, a pathological HTTP client. - - * Add libpathod.test, a truss for using pathod in unit tests. - - * Add an injection operator to the specification language. - - * Allow Python escape sequences in value literals. - - * Allow execution of requests and responses from file, using the new + operator. - - * Add daemonization to Pathod, and make it more robust for public-facing use. - - * Let pathod pick an arbitrary open port if -p 0 is specified. - - * Move from Tornado to netlib, the network library written for mitmproxy. - - * Move the web application to Flask. - - * Massively expand the documentation. - - -5 April 2012: mitmproxy 0.8: - - * Detailed tutorial for Android interception. Some features that land in - this release have finally made reliable Android interception possible. - - * Upstream-cert mode, which uses information from the upstream server to - generate interception certificates. - - * Replacement patterns that let you easily do global replacements in flows - matching filter patterns. Can be specified on the command-line, or edited - interactively. - - * Much more sophisticated and usable pretty printing of request bodies. - Support for auto-indentation of Javascript, inspection of image EXIF - data, and more. - - * Details view for flows, showing connection and SSL cert information (X - keyboard shortcut). - - * Server certificates are now stored and serialized in saved traffic for - later analysis. This means that the 0.8 serialization format is NOT - compatible with 0.7. - - * Many other improvements, including bugfixes, and expanded scripting API, - and more sophisticated certificate handling. - - -20 February 2012: mitmproxy 0.7: - - * New built-in key/value editor. This lets you interactively edit URL query - strings, headers and URL-encoded form data. - - * Extend script API to allow duplication and replay of flows. - - * API for easy manipulation of URL-encoded forms and query strings. - - * Add "D" shortcut in mitmproxy to duplicate a flow. - - * Reverse proxy mode. In this mode mitmproxy acts as an HTTP server, - forwarding all traffic to a specified upstream server. - - * UI improvements - use unicode characters to make GUI more compact, - improve spacing and layout throughout. - - * Add support for filtering by HTTP method. - - * Add the ability to specify an HTTP body size limit. - - * Move to typed netstrings for serialization format - this makes 0.7 - backwards-incompatible with serialized data from 0.6! - - * Significant improvements in speed and responsiveness of UI. - - * Many minor bugfixes and improvements. - - -7 August 2011: mitmproxy 0.6: - - * New scripting API that allows much more flexible and fine-grained - rewriting of traffic. See the docs for more info. - - * Support for gzip and deflate content encodings. A new "z" - keybinding in mitmproxy to let us quickly encode and decode content, plus - automatic decoding for the "pretty" view mode. - - * An event log, viewable with the "v" shortcut in mitmproxy, and the - "-e" command-line flag in mitmdump. - - * Huge performance improvements: mitmproxy interface, loading - large numbers of flows from file. - - * A new "replace" convenience method for all flow objects, that does a - universal regex-based string replacement. - - * Header management has been rewritten to maintain both case and order. - - * Improved stability for SSL interception. - - * Default expiry time on generated SSL certs has been dropped to avoid an - OpenSSL overflow bug that caused certificates to expire in the distant - past on some systems. - - * A "pretty" view mode for JSON and form submission data. - - * Expanded documentation and examples. - - * Countless other small improvements and bugfixes. - - -27 June 2011: mitmproxy 0.5: - - * An -n option to start the tools without binding to a proxy port. - - * Allow scripts, hooks, sticky cookies etc. to run on flows loaded from - save files. - - * Regularize command-line options for mitmproxy and mitmdump. - - * Add an "SSL exception" to mitmproxy's license to remove possible - distribution issues. - - * Add a --cert-wait-time option to make mitmproxy pause after a new SSL - certificate is generated. This can pave over small discrepancies in - system time between the client and server. - - * Handle viewing big request and response bodies more elegantly. Only - render the first 100k of large documents, and try to avoid running the - XML indenter on non-XML data. - - * BUGFIX: Make the "revert" keyboard shortcut in mitmproxy work after a - flow has been replayed. - - * BUGFIX: Repair a problem that sometimes caused SSL connections to consume - 100% of CPU. - - -30 March 2011: mitmproxy 0.4 - - * Full serialization of HTTP conversations - - * Client and server replay - - * On-the-fly generation of dummy SSL certificates - - * mitmdump has "grown up" into a powerful tcpdump-like tool for HTTP/S - - * Dozens of improvements to the mitmproxy console interface - - * Python scripting hooks for programmatic modification of traffic - - -1 March 2010: mitmproxy 0.2 - - * Big speed and responsiveness improvements, thanks to Thomas Roth - - * Support urwid 0.9.9 - - * Terminal beeping based on filter expressions - - * Filter expressions for terminal beeps, limits, interceptions and sticky - cookies can now be passed on the command line. - - * Save requests and responses to file - - * Split off non-interactive dump functionality into a new tool called - mitmdump - - * "A" will now accept all intercepted connections - - * Lots of bugfixes diff -Nru mitmproxy-5.1.1/CHANGELOG.rst mitmproxy-6.0.2/CHANGELOG.rst --- mitmproxy-5.1.1/CHANGELOG.rst 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/CHANGELOG.rst 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,866 @@ +Release History +############### + +15 December 2020: mitmproxy 6.0.2 +================================= + +* Fix reading of saved flows in mitmweb. + +13 December 2020: mitmproxy 6.0.1 +================================= + +* Fix flow serialization in mitmweb. + +13 December 2020: mitmproxy 6.0 +=============================== + +* Mitmproxy now requires Python 3.8 or above. +* Deprecation of pathod and pathoc tools and modules. Future releases will not contain them! (@Kriechi) +* SSLKEYLOGFILE now supports TLS 1.3 secrets (@mhils) +* Fix query parameters in asgiapp addon (@jpstotz) +* Fix command history failing on file I/O errors (@Kriechi) +* Add example addon to suppress unwanted error messages sent by mitmproxy. (@anneborcherding) +* Updated imports and styles for web scanner helper addons. (@anneborcherding) +* Inform when underscore-formatted options are used in client arg. (@jrblixt) +* ASGIApp now ignores loaded HTTP flows from somewhere. (@linw1995) +* Binaries are now built with Python 3.9 (@mhils) +* Fixed the web UI showing blank page on clicking details tab when server address is missing (@samhita-sopho) +* Tests: Replace asynctest with stdlib mock (@felixonmars) +* MapLocal now keeps its configuration when other options are set. (@mhils) +* Host headers with non-standard ports are now properly updated in reverse proxy mode. (@mhils) +* Fix missing host header when replaying HTTP/2 flows (@Granitosaurus) + +01 November 2020: mitmproxy 5.3 +=============================== + +Full Changelog +-------------- + +* Support for Python 3.9 (@mhils) +* Add MsgPack content viewer (@tasn) +* Use `@charset` to decode CSS files if available (@prinzhorn) +* Fix links to anticache docs in mitmweb and use HTTPS for links to documentation (@rugk) +* Updated typing for WebsocketMessage.content (@prinzhorn) +* Add option `console_strip_trailing_newlines`, and no longer strip trailing newlines by default (@capt8bit) +* Prevent transparent mode from connecting to itself in the basic cases (@prinzhorn) +* Display HTTP trailers in mitmweb (@sanlengjingvv) +* Revamp onboarding app (@mhils) +* Add ASGI support for embedded apps (@mhils) +* Updated raw exports to not remove headers (@wchasekelley) +* Fix file unlinking before external viewer finishes loading (@wchasekelley) +* Add --cert-passphrase command line argument (@mirosyn) +* Add interactive tutorials to the documentation (@mplattner) +* Support `deflateRaw` for `Content-Encoding`'s (@kjoconnor) +* Fix broken requests without body on HTTP/2 (@Kriechi) +* Add support for sending (but not parsing) HTTP Trailers to the HTTP/1.1 protocol (@bburky) +* Add support to echo http trailers in dumper addon (@shiv6146) +* Fix OpenSSL requiring different CN for root and leaf certificates (@mhils) +* ... and various other fixes, documentation improvements, dependency version bumps, etc. + +18 July 2020: mitmproxy 5.2 +=========================== + +* Add Filter message to mitmdump (@sarthak212) +* Display TCP flows at flow list (@Jessonsotoventura, @nikitastupin, @mhils) +* Colorize JSON Contentview (@sarthak212) +* Fix console crash when entering regex escape character in half-open string (@sarthak212) +* Integrate contentviews to TCP flow details (@nikitastupin) +* Added add-ons that enhance the performance of web application scanners (@anneborcherding) +* Increase WebSocket message timestamp precision (@JustAnotherArchivist) +* Fix HTTP reason value on HTTP/2 reponses (@rbdixon) +* mitmweb: support wslview to open a web browser (@G-Rath) +* Fix dev version detection with parent git repo (@JustAnotherArchivist) +* Restructure examples and supported addons (@mhils) +* Certificate generation: mark SAN as critical if no CN is set (@mhils) +* Simplify Replacements with new ModifyBody addon (@mplattner) +* Rename SetHeaders addon to ModifyHeaders (@mplattner) +* mitmweb: "New -> File" menu option has been renamed to "Clear All" (@yogeshojha) +* Add new MapRemote addon to rewrite URLs of requests (@mplattner) +* Add support for HTTP Trailers to the HTTP/2 protocol (@sanlengjingvv and @Kriechi) +* Fix certificate runtime error during expire cleanup (@gorogoroumaru) +* Fixed the DNS Rebind Protection for secure support of IPv6 addresses (@tunnelpr0) +* WebSockets: match the HTTP-WebSocket flow for the ~websocket filter (@Kriechi) +* Fix deadlock caused by the "replay.client.stop" command (@gorogoroumaru) +* Add new MapLocal addon to serve local files instead of remote resources (@mplattner and @mhils) +* Add minimal TCP interception and modification (@nikitastupin) +* Add new CheckSSLPinning addon to check SSL-Pinning on client (@su-vikas) +* Add a JSON dump script: write data into a file or send to an endpoint as JSON (@emedvedev) +* Fix console output formatting (@sarthak212) +* Add example for proxy authentication using selenium (@anneborcherding and @weichweich) + +13 April 2020: mitmproxy 5.1.1 +============================== + +* Fixed Docker images not starting due to missing shell + +13 April 2020: mitmproxy 5.1 +============================ + +Major Changes +------------- + +* Initial Support for TLS 1.3 + +Full Changelog +-------------- + +* Reduce leaf certificate validity to one year due to upcoming browser changes (@mhils) +* Rename mitmweb's `web_iface` option to `web_host` for consistency (@oxr463) +* Sending a SIGTERM now exits mitmproxy without prompt, SIGINT still asks (@ThinkChaos) +* Don't force host header on outgoing requests (@mhils) +* Additional documentation and examples for WebSockets (@Kriechi) +* Gracefully handle hyphens in domain names (@matosconsulting) +* Fix header replacement count (@naivekun) +* Emit serverconnect event only after a connection has been established (@Prinzhorn) +* Fix ValueError in table mode of server replay flow (@ylmrx) +* HTTP/2: send all stream reset types to other connection (@rohfle) +* HTTP/2: fix WINDOW_UPDATE swallowed on closed streams (@Kriechi) +* Fix wrong behavior of --allow-hosts options (@BlownSnail) +* Additional and updated documentation for examples, WebSockets, Getting Started (@Kriechi) + +27 December 2019: mitmproxy 5.0.1 +================================= + +* Fixed precompiled Linux binaries to not crash in table mode +* Display webp images in mitmweb (@cixtor) + +16 December 2019: mitmproxy 5.0 +=============================== + +Major Changes +------------- + +* Added new Table UI (@Jessonsotoventura) +* Added EKU extension to certificates. This fixes support for macOS Catalina (@vin01) + +Security Fixes +-------------- + +* Fixed command injection vulnerabilities when exporting flows as curl/httpie commands (@cript0nauta) +* Do not echo unsanitized user input in HTTP error responses (@fimad) + +Full Changelog +-------------- + +* Moved to Github CI for Continuous Integration, dropping support for old Linux and macOS releases. (#3728) +* Vastly improved command parsing, in particular for setting flow filters (@typoon) +* Added a new flow export for raw responses (@mckeimic) +* URLs are now edited in an external editor (@Jessonsotoventura) +* mitmproxy now has a command history (@typoon) +* Added terminal like keyboard shortcuts for the command bar (ctrl+w, ctrl+a, ctrl+f, ...) (@typoon) +* Fixed issue with improper handling of non-ascii characters in URLs (@rjt-gupta) +* Filtering can now use unicode characters (@rjt-gupta) +* Fixed issue with user keybindings not being able to override default keybindings +* Improved installation instructions +* Added support for IPV6-only environments (@sethb157) +* Fixed bug with server replay (@rjt-gupta) +* Fixed issue with duplicate error responses (@ccssrryy) +* Users can now set a specific external editor using $MITMPROXY_EDITOR (@rjt-gupta) +* Config file can now be called `config.yml` or `config.yaml` (@ylmrx) +* Fixed crash on `view.focus.[next|prev]` (@ylmrx) +* Updated documentation to help using mitmproxy certificate on Android (@jannst) +* Added support to parse IPv6 entries from `pfctl` on MacOS. (@tomlabaude) +* Fixed instructions on how to build the documentation (@jannst) +* Added a new `--allow-hosts` option (@pierlon) +* Added support for zstd content-encoding (@tsaaristo) +* Fixed issue where the replay server would corrupt the Date header (@tonyb486) +* Improve speed for WebSocket interception (@MathieuBordere) +* Fixed issue with parsing JPEG files. (@lusceu) +* Improve example code style (@BoboTiG) +* Fixed issue converting void responses to HAR (@worldmind) +* Color coded http status codes in mitmweb (@arun-94) +* Added organization to generated certificates (@Abcdefghijklmnopqrstuvwxyzxyz) +* Errors are now displayed on sys.stderr (@JessicaFavin) +* Fixed issue with replay timestamps (@rjt-gupta) +* Fixed copying in mitmweb on macOS (@XZzYassin) + +31 July 2018: mitmproxy 4.0.4 +============================= + +* Security: Protect mitmweb against DNS rebinding. (CVE-2018-14505, @atx) +* Reduce certificate lifetime to two years to be conformant with + the current CA/Browser Forum Baseline Requirements. (@muffl0n) + (https://cabforum.org/2017/03/17/ballot-193-825-day-certificate-lifetimes/) +* Update cryptography to version 2.3. + +15 June 2018: mitmproxy 4.0.3 +============================= + +* Add support for IPv6 transparent mode on Windows (#3174) +* Add Docker images for ARMv7 - Raspberry Pi (#3190) +* Major overhaul of our release workflow - you probably won't notice it, but for us it's a big thing! +* Fix the Python version detection on Python 3.5, we now show a more intuitive error message (#3188) +* Fix application shutdown on Windows (#3172) +* Fix IPv6 scope suffixes in block addon (#3164) +* Fix options update when added (#3157) +* Fix "Edit Flow" button in mitmweb (#3136) + +15 June 2018: mitmproxy 4.0.2 +============================= + +* Skipped! + +17 May 2018: mitmproxy 4.0.1 +============================ + +Bugfixes +----------- + +* The previous release had a packaging issue, so we bumped it to v4.0.1 and re-released it. +* This contains no actual bugfixes or new features. + +17 May 2018: mitmproxy 4.0 +========================== + +Features +-------- + +* mitmproxy now requires Python 3.6! +* Moved the core to asyncio - which gives us a very significant performance boost! +* Reduce memory consumption by using `SO_KEEPALIVE` (#3076) +* Export request as httpie command (#3031) +* Configure mitmproxy console keybindings with the keys.yaml file. See docs for more. + +Breaking Changes +---------------- + +* The --conf command-line flag is now --confdir, and specifies the mitmproxy configuration + directory, instead of the options yaml file (which is at `config.yaml` under the configuration directory). +* `allow_remote` got replaced by `block_global` and `block_private` (#3100) +* No more custom events (#3093) +* The `cadir` option has been renamed to `confdir` +* We no longer magically capture print statements in addons and translate + them to logs. Please use `ctx.log.info` explicitly. + +Bugfixes +-------- + +* Correctly block connections from remote clients with IPv4-mapped IPv6 client addresses (#3099) +* Expand `~` in paths during the `cut` command (#3078) +* Remove socket listen backlog constraint +* Improve handling of user script exceptions (#3050, #2837) +* Ignore signal errors on windows +* Fix traceback for commands with un-terminated escape characters (#2810) +* Fix request replay when proxy is bound to local interface (#2647) +* Fix traceback when running scripts on a flow twice (#2838) +* Fix traceback when killing intercepted flow (#2879) +* And lots of typos, docs improvements, revamped examples, and general fixes! + +05 April 2018: mitmproxy 3.0.4 +============================== + +* Fix an issue that caused mitmproxy to not retry HTTP requests on timeout. +* Various other fixes (@kira0204, @fenilgandhi, @tran-tien-dat, @smonami, + @luzpaz, @fristonio, @kajojify, @Oliver-Fish, @hcbarry, @jplochocki, @MikeShi42, + @ghillu, @emilstahl) + +25 February 2018: mitmproxy 3.0.3 +================================= + +* Fix an issue that caused mitmproxy to lose keyboard control after spawning an external editor. + +23 February 2018: mitmproxy 3.0.1 +================================= + +* Fix a quote-related issue affecting the mitmproxy console command prompt. + +22 February 2018: mitmproxy 3.0 +=============================== + +Major Changes +------------- + +* Commands: A consistent, typed mechanism that allows addons to expose actions + to users. +* Options: A typed settings store for use by mitmproxy and addons. +* Shift most of mitmproxy's own functionality into addons. +* Major improvements to mitmproxy console, including an almost complete + rewrite of the user interface, integration of commands, key bindings, and + multi-pane layouts. +* Major Improvements to mitmproxy’s web interface, mitmweb. (Matthew Shao, + Google Summer of Code 2017) +* Major Improvements to mitmproxy’s content views and protocol layers (Ujjwal + Verma, Google Summer of Code 2017) +* Faster JavaScript and CSS beautifiers. (Ujjwal Verma) + +Minor Changes +------------- + +* Vastly improved JavaScript test coverage (Matthew Shao) +* Options editor for mitmweb (Matthew Shao) +* Static web-based flow viewer (Matthew Shao) +* Request streaming for HTTP/1.x and HTTP/2 (Ujjwal Verma) +* Implement more robust content views using Kaitai Struct (Ujjwal Verma) +* Protobuf decoding now works without protoc being installed on the host + system (Ujjwal Verma) +* PNG, GIF, and JPEG can now be parsed without Pillow, which simplifies + mitmproxy installation and moves parsing from unsafe C to pure Python (Ujjwal Verma) +* Add parser for ICO files (Ujjwal Verma) +* Migrate WebSockets implementation to wsproto. This reduces code size and + adds WebSocket compression support. (Ujjwal Verma) +* Add “split view” to split mitmproxy’s UI into two separate panes. +* Add key binding viewer and editor +* Add a command to spawn a preconfigured Chrome browser instance from + mitmproxy +* Fully support mitmproxy under the Windows Subsystem for Linux (WSL), work + around display errors +* Add XSS scanner addon (@ddworken) +* Add ability to toggle interception (@mattweidner) +* Numerous documentation improvements (@pauloromeira, @rst0git, @rgerganov, + @fulldecent, @zhigang1992, @F1ashhimself, @vinaydargar, @jonathanrfisher1, + @BasThomas, @LuD1161, @ayamamori, @TomTasche) +* Add filters for websocket flows (@s4chin) +* Make it possible to create a response to CONNECT requests in http_connect + (@mengbiping) +* Redirect stdout in scripts to ctx.log.warn (@nikofil) +* Fix a crash when clearing the event log (@krsoninikhil) +* Store the generated certificate for each flow (@dlenski) +* Add --keep-host-header to retain the host header in reverse proxy mode + (@krsoninikhil) +* Fix setting palette options (@JordanLoehr) +* Fix a crash with brotli encoding (@whackashoe) +* Provide certificate installation instructions on mitm.it (@ritiek) +* Fix a bug where we did not properly fall back to IPv4 when IPv6 is unavailable (@titeuf87) +* Fix transparent mode on IPv6-enabled macOS systems (@Ga-ryo) +* Fix handling of HTTP messages with multiple Content-Length headers (@surajt97) +* Fix IPv6 authority form parsing in CONNECT requests (@r1b) +* Fix event log display in mitmweb (@syahn) +* Remove private key from PKCS12 file in ~/.mitmproxy (@ograff). +* Add LDAP as a proxy authentication backend (@charlesdhdt) +* Use mypy to check the whole codebase (@iharsh234) +* Fix a crash when duplicating flows (@iharsh234) +* Fix testsuite when the path contains a “.” (@felixonmars) +* Store proxy authentication with flows (@lymanZerga11) +* Match ~d and ~u filters against pretty_host (@dequis) +* Update WBXML content view (@davidpshaw) +* Handle HEAD requests for mitm.it to support Chrome in transparent mode on + iOS (@tomlabaude) +* Update dns spoofing example to use --keep-host-header (@krsoninikhil) +* Call error handler on HTTPException (@tarnacious) +* Make it possible to remove TLS from upstream HTTP connections +* Update to pyOpenSSL 17.5, cryptography 2.1.4, and OpenSSL 1.1.0g +* Make it possible to retroactively increase log verbosity. +* Make logging from addons thread-safe +* Tolerate imports in user scripts that match hook names + (`from mitmproxy import log`) +* Update mitmweb to React 16, which brings performance improvements +* Fix a bug where reverting duplicated flows crashes mitmproxy +* Fix a bug where successive requests are sent to the wrong host after a + request has been redirected. +* Fix a bug that binds outgoing connections to the wrong interface +* Fix a bug where custom certificates are ignored in reverse proxy mode +* Fix import of flows that have been created with mitmproxy 0.17 +* Fix formatting of (IPv6) IP addresses in a number of places +* Fix replay for HTTP/2 flows +* Decouple mitmproxy version and flow file format version +* Fix a bug where “mitmdump -nr” does not exit automatically +* Fix a crash when exporting flows to curl +* Fix formatting of sticky cookies +* Improve script reloading reliability by polling the filesystem instead of using watchdog +* Fix a crash when refreshing Set-Cookie headers +* Add connection indicator to mitmweb to alert users when the proxy server stops running +* Add support for certificates with cyrillic domains +* Simplify output of mitmproxy --version +* Add Request.make to simplify request creation in scripts +* Pathoc: Include a host header on CONNECT requests +* Remove HTML outline contentview (#2572) +* Remove Python and Locust export (#2465) +* Remove emojis from tox.ini because flake8 cannot parse that. :( + +28 April 2017: mitmproxy 2.0.2 +============================== + +* Fix mitmweb's Content-Security-Policy to work with Chrome 58+ +* HTTP/2: actually use header normalization from hyper-h2 + +15 March 2017: mitmproxy 2.0.1 +============================== + +* bump cryptography dependency +* bump pyparsing dependency +* HTTP/2: use header normalization from hyper-h2 + +21 February 2017: mitmproxy 2.0 +=============================== + +* HTTP/2 is now enabled by default. +* Image ContentView: Parse images with Kaitai Struct (kaitai.io) instead of Pillow. + This simplifies installation, reduces binary size, and allows parsing in pure Python. +* Web: Add missing flow filters. +* Add transparent proxy support for OpenBSD. +* Check the mitmproxy CA for expiration and warn the user to regenerate it if necessary. +* Testing: Tremendous improvements, enforced 100% coverage for large parts of the + codebase, increased overall coverage. +* Enforce individual coverage: one source file -> one test file with 100% coverage. +* A myriad of other small improvements throughout the project. +* Numerous bugfixes. + +26 December 2016: mitmproxy 1.0 +=============================== + +* All mitmproxy tools are now Python 3 only! We plan to support Python 3.5 and higher. +* Web-Based User Interface: Mitmproxy now officially has a web-based user interface + called mitmweb. We consider it stable for all features currently exposed + in the UI, but it still misses a lot of mitmproxy’s options. +* Windows Compatibility: With mitmweb, mitmproxy is now usable on Windows. + We are also introducing an installer (kindly sponsored by BitRock) that + simplifies setup. +* Configuration: The config file format is now a single YAML file. In most cases, + converting to the new format should be trivial - please see the docs for + more information. +* Console: Significant UI improvements - including sorting of flows by + size, type and url, status bar improvements, much faster indentation for + HTTP views, and more. +* HTTP/2: Significant improvements, but is temporarily disabled by default + due to wide-spread protocol implementation errors on some large website +* WebSocket: The protocol implementation is now mature, and is enabled by + default. Complete UI support is coming in the next release. Hooks for + message interception and manipulation are available. +* A myriad of other small improvements throughout the project. + +16 October 2016: mitmproxy 0.18 +=============================== + +* Python 3 Compatibility for mitmproxy and pathod (Shadab Zafar, GSoC 2016) +* Major improvements to mitmweb (Clemens Brunner & Jason Hao, GSoC 2016) +* Internal Core Refactor: Separation of most features into isolated Addons +* Initial Support for WebSockets +* Improved HTTP/2 Support +* Reverse Proxy Mode now automatically adjusts host headers and TLS Server Name Indication +* Improved HAR export +* Improved export functionality for curl, python code, raw http etc. +* Flow URLs are now truncated in the console for better visibility +* New filters for TCP, HTTP and marked flows. +* Mitmproxy now handles comma-separated Cookie headers +* Merge mitmproxy and pathod documentation +* Mitmdump now sanitizes its console output to not include control characters +* Improved message body handling for HTTP messages: + `.raw_content` provides the message body as seen on the wire + `.content` provides the decompressed body (e.g. un-gzipped) + `.text` provides the body decompressed and decoded body +* New HTTP Message getters/setters for cookies and form contents. +* Add ability to view only marked flows in mitmproxy +* Improved Script Reloader (Always use polling, watch for whole directory) +* Use tox for testing +* Unicode support for tnetstrings +* Add dumpfile converters for mitmproxy versions 0.11 and 0.12 +* Numerous bugfixes + +9 April 2016: mitmproxy 0.17 +============================ + +* Simplify repository and release structure. mitmproxy now comes as a single package, including netlib and pathod. +* Rename the Python package from libmproxy to mitmproxy. +* New option to add server certs to client chain (CVE-2016-2402, John Kozyrakis) +* Enable HTTP/2 by default (Thomas Kriechbaumer) +* Improved HAR extractor (Shadab Zafar) +* Add icon for OSX and Windows binaries +* Add content view for query parameters (Will Coster) +* Initial work on Python 3 compatibility +* locust.io export (Zohar Lorberbaum) +* Fix XSS vulnerability in HTTP errors (Will Coster) +* Numerous bugfixes and minor improvements + + +15 February 2016: mitmproxy 0.16 +================================ + +* Completely revised HTTP2 implementation based on hyper-h2 (Thomas Kriechbaumer) +* Export flows as cURL command, Python code or raw HTTP (Shadab Zafar) +* Fixed compatibility with the Android Emulator (Will Coster) +* Script Reloader: Inline scripts are reloaded automatically if modified (Matthew Shao) +* Inline script hooks for TCP mode (Michael J. Bazzinotti) +* Add default ciphers to support iOS9 App Transport Security (Jorge Villacorta) +* Basic Authentication for mitmweb (Guillem Anguera) +* Exempt connections from interception based on TLS Server Name Indication (David Weinstein) +* Provide Python Wheels for faster installation +* Numerous bugfixes and minor improvements + +4 December 2015: mitmproxy 0.15 +=============================== + +* Support for loading and converting older dumpfile formats (0.13 and up) +* Content views for inline script (@chrisczub) +* Better handling of empty header values (Benjamin Lee/@bltb) +* Fix a gnarly memory leak in mitmdump +* A number of bugfixes and small improvements + +6 November 2015: mitmproxy 0.14 +=============================== + +* Statistics: 399 commits, 13 contributors, 79 closed issues, 37 closed + PRs, 103 days +* Docs: Greatly updated docs now hosted on ReadTheDocs! + http://docs.mitmproxy.org +* Docs: Fixed Typos, updated URLs etc. (Nick Badger, Ben Lerner, Choongwoo + Han, onlywade, Jurriaan Bremer) +* mitmdump: Colorized TTY output +* mitmdump: Use mitmproxy's content views for human-readable output (Chris + Czub) +* mitmproxy and mitmdump: Support for displaying UTF8 contents +* mitmproxy: add command line switch to disable mouse interaction (Timothy + Elliott) +* mitmproxy: bug fixes (Choongwoo Han, sethp-jive, FreeArtMan) +* mitmweb: bug fixes (Colin Bendell) +* libmproxy: Add ability to fall back to TCP passthrough for non-HTTP + connections. +* libmproxy: Avoid double-connect in case of TLS Server Name Indication. + This yields a massive speedup for TLS handshakes. +* libmproxy: Prevent unnecessary upstream connections (macmantrl) +* Inline Scripts: New API for HTTP Headers: + http://docs.mitmproxy.org/en/latest/dev/models.html#netlib.http.Headers +* Inline Scripts: Properly handle exceptions in `done` hook +* Inline Scripts: Allow relative imports, provide `__file__` +* Examples: Add probabilistic TLS passthrough as an inline script +* netlib: Refactored HTTP protocol handling code +* netlib: ALPN support +* netlib: fixed a bug in the optional certificate verification. +* netlib: Initial Python 3.5 support (this is the first prerequisite for + 3.x support in mitmproxy) + +24 July 2015: mitmproxy 0.13 +============================ + +* Upstream certificate validation. See the --verify-upstream-cert, + --upstream-trusted-confdir and --upstream-trusted-ca parameters. Thanks to + Kyle Morton (github.com/kyle-m) for his work on this. +* Add HTTP transparent proxy mode. This uses the host headers from HTTP + traffic (rather than SNI and IP address information from the OS) to + implement perform transparent proxying. Thanks to github.com/ijiro123 for + this feature. +* Add ~src and ~dst REGEX filters, allowing matching on source and + destination addresses in the form of : +* mitmproxy console: change g/G keyboard shortcuts to match less. Thanks to + Jose Luis Honorato (github.com/jlhonora). +* mitmproxy console: Flow marking and unmarking. Marked flows are not + deleted when the flow list is cleared. Thanks to Jake Drahos + (github.com/drahosj). +* mitmproxy console: add marking of flows +* Remove the certforward feature. It was added to allow exploitation of + #gotofail, which is no longer a common vulnerability. Permitting this + hugely increased the complexity of packaging and distributing mitmproxy. + +3 June 2015: mitmproxy 0.12.1 +============================= + +* mitmproxy console: mouse interaction - scroll in the flow list, click on + flow to view, click to switch between tabs. +* Update our crypto defaults: SHA256, 2048 bit RSA, 4096 bit DH parameters. +* BUGFIX: crash under some circumstances when copying to clipboard. +* BUGFIX: occasional crash when deleting flows. + +18 May 2015: mitmproxy 0.12 +=========================== + +* mitmproxy console: Significant revamp of the UI. The major changes are + listed below, and in addition almost every aspect of the UI has + been tweaked, and performance has improved significantly. +* mitmproxy console: A new options screen has been created ("o" shortcut), + and many options that were previously manipulated directly via a + keybinding have been moved there. +* mitmproxy console: Big improvement in palettes. This includes improvements + to all colour schemes. Palettes now set the terminal background colour by + default, and a new --palette-transparent option has been added to disable + this. +* mitmproxy console: g/G shortcuts throughout mitmproxy console to jump + to the beginning/end of the current view. +* mitmproxy console: switch palettes on the fly from the options screen. +* mitmproxy console: A cookie editor has been added for mitmproxy console + at long last. +* mitmproxy console: Various components of requests and responses can be + copied to the clipboard from mitmproxy - thanks to @marceloglezer. +* Support for creating new requests from scratch in mitmproxy console (@marceloglezer). +* SSLKEYLOGFILE environment variable to specify a logging location for TLS + master keys. This can be used with tools like Wireshark to allow TLS + decoding. +* Server facing SSL cipher suite specification (thanks to Jim Shaver). +* Official support for transparent proxying on FreeBSD - thanks to Mike C + (http://github.com/mike-pt). +* Many other small bugfixes and improvemenets throughout the project. + +29 Dec 2014: mitmproxy 0.11.2 +============================= + +* Configuration files - mitmproxy.conf, mitmdump.conf, common.conf in the + .mitmproxy directory. +* Better handling of servers that reject connections that are not SNI. +* Many other small bugfixes and improvements. + +15 November 2014: mitmproxy 0.11.1 +================================== + +* Bug fixes: connection leaks some crashes + +7 November 2014: mitmproxy 0.11 +=============================== + +* Performance improvements for mitmproxy console +* SOCKS5 proxy mode allows mitmproxy to act as a SOCKS5 proxy server +* Data streaming for response bodies exceeding a threshold + (bradpeabody@gmail.com) +* Ignore hosts or IP addresses, forwarding both HTTP and HTTPS traffic + untouched +* Finer-grained control of traffic replay, including options to ignore + contents or parameters when matching flows (marcelo.glezer@gmail.com) +* Pass arguments to inline scripts +* Configurable size limit on HTTP request and response bodies +* Per-domain specification of interception certificates and keys (see + --cert option) +* Certificate forwarding, relaying upstream SSL certificates verbatim (see + --cert-forward) +* Search and highlighting for HTTP request and response bodies in + mitmproxy console (pedro@worcel.com) +* Transparent proxy support on Windows +* Improved error messages and logging +* Support for FreeBSD in transparent mode, using pf (zbrdge@gmail.com) +* Content view mode for WBXML (davidshaw835@air-watch.com) +* Better documentation, with a new section on proxy modes +* Generic TCP proxy mode +* Countless bugfixes and other small improvements +* pathod: Hugely improved SSL support, including dynamic generation of certificates + using the mitproxy cacert + +7 November 2014: pathod 0.11 +============================ + +* Hugely improved SSL support, including dynamic generation of certificates + using the mitproxy cacert +* pathoc -S dumps information on the remote SSL certificate chain +* Big improvements to fuzzing, including random spec selection and memoization to avoid repeating randomly generated patterns +* Reflected patterns, allowing you to embed a pathod server response specification in a pathoc request, resolving both on client side. This makes fuzzing proxies and other intermediate systems much better. + + +28 January 2014: mitmproxy 0.10 +=============================== + +* Support for multiple scripts and multiple script arguments +* Easy certificate install through the in-proxy web app, which is now + enabled by default +* Forward proxy mode, that forwards proxy requests to an upstream HTTP server +* Reverse proxy now works with SSL +* Search within a request/response using the "/" and "n" shortcut keys +* A view that beatifies CSS files if cssutils is available +* Bug fix, documentation improvements, and more. + +25 August 2013: mitmproxy 0.9.2 +=============================== + +* Improvements to the mitmproxywrapper.py helper script for OSX. +* Don't take minor version into account when checking for serialized file + compatibility. +* Fix a bug causing resource exhaustion under some circumstances for SSL + connections. +* Revamp the way we store interception certificates. We used to store these + on disk, they're now in-memory. This fixes a race condition related to + cert handling, and improves compatibility with Windows, where the rules + governing permitted file names are weird, resulting in errors for some + valid IDNA-encoded names. +* Display transfer rates for responses in the flow list. +* Many other small bugfixes and improvements. + +25 August 2013: pathod 0.9.2 +============================ + +* Adapt to interface changes in netlib + +16 June 2013: mitmproxy 0.9.1 +============================= + +* Use "correct" case for Content-Type headers added by mitmproxy. +* Make UTF environment detection more robust. +* Improved MIME-type detection for viewers. +* Always read files in binary mode (Windows compatibility fix). +* Some developer documentation. + +15 May 2013: mitmproxy 0.9 +========================== + +* Upstream certs mode is now the default. +* Add a WSGI container that lets you host in-proxy web applications. +* Full transparent proxy support for Linux and OSX. +* Introduce netlib, a common codebase for mitmproxy and pathod + (http://github.com/cortesi/netlib). +* Full support for SNI. +* Color palettes for mitmproxy, tailored for light and dark terminal + backgrounds. +* Stream flows to file as responses arrive with the "W" shortcut in + mitmproxy. +* Extend the filter language, including ~d domain match operator, ~a to + match asset flows (js, images, css). +* Follow mode in mitmproxy ("F" shortcut) to "tail" flows as they arrive. +* --dummy-certs option to specify and preserve the dummy certificate + directory. +* Server replay from the current captured buffer. +* Huge improvements in content views. We now have viewers for AMF, HTML, + JSON, Javascript, images, XML, URL-encoded forms, as well as hexadecimal + and raw views. +* Add Set Headers, analogous to replacement hooks. Defines headers that are set + on flows, based on a matching pattern. +* A graphical editor for path components in mitmproxy. +* A small set of standard user-agent strings, which can be used easily in + the header editor. +* Proxy authentication to limit access to mitmproxy +* pathod: Proxy mode. You can now configure clients to use pathod as an + HTTP/S proxy. +* pathoc: Proxy support, including using CONNECT to tunnel directly to + targets. +* pathoc: client certificate support. +* pathod: API improvements, bugfixes. + +15 May 2013: pathod 0.9 (version synced with mitmproxy) +======================================================= + +* Pathod proxy mode. You can now configure clients to use pathod as an + HTTP/S proxy. +* Pathoc proxy support, including using CONNECT to tunnel directly to + targets. +* Pathoc client certificate support. +* API improvements, bugfixes. + + +16 November 2012: pathod 0.3 +============================ + +A release focusing on shoring up our fuzzing capabilities, especially with +pathoc. + +* pathoc -q and -r options, output full request and response text. +* pathod -q and -r options, add full request and response text to pathod's + log buffer. +* pathoc and pathod -x option, makes -q and -r options log in hex dump + format. +* pathoc -C option, specify response codes to ignore. +* pathoc -T option, instructs pathoc to ignore timeouts. +* pathoc -o option, a one-shot mode that exits after the first non-ignored + response. +* pathoc and pathod -e option, which explains the resulting message by + expanding random and generated portions, and logging a reproducible + specification. +* Streamline the specification language. HTTP response message is now + specified using the "r" mnemonic. +* Add a "u" mnemonic for specifying User-Agent strings. Add a set of + standard user-agent strings accessible through shortcuts. +* Major internal refactoring and cleanup. +* Many bugfixes. + +22 August 2012: pathod 0.2 +========================== + +* Add pathoc, a pathological HTTP client. +* Add libpathod.test, a truss for using pathod in unit tests. +* Add an injection operator to the specification language. +* Allow Python escape sequences in value literals. +* Allow execution of requests and responses from file, using the new + operator. +* Add daemonization to Pathod, and make it more robust for public-facing use. +* Let pathod pick an arbitrary open port if -p 0 is specified. +* Move from Tornado to netlib, the network library written for mitmproxy. +* Move the web application to Flask. +* Massively expand the documentation. + +5 April 2012: mitmproxy 0.8 +=========================== + +* Detailed tutorial for Android interception. Some features that land in + this release have finally made reliable Android interception possible. +* Upstream-cert mode, which uses information from the upstream server to + generate interception certificates. +* Replacement patterns that let you easily do global replacements in flows + matching filter patterns. Can be specified on the command-line, or edited + interactively. +* Much more sophisticated and usable pretty printing of request bodies. + Support for auto-indentation of Javascript, inspection of image EXIF + data, and more. +* Details view for flows, showing connection and SSL cert information (X + keyboard shortcut). +* Server certificates are now stored and serialized in saved traffic for + later analysis. This means that the 0.8 serialization format is NOT + compatible with 0.7. +* Many other improvements, including bugfixes, and expanded scripting API, + and more sophisticated certificate handling. + +20 February 2012: mitmproxy 0.7 +=============================== + +* New built-in key/value editor. This lets you interactively edit URL query + strings, headers and URL-encoded form data. +* Extend script API to allow duplication and replay of flows. +* API for easy manipulation of URL-encoded forms and query strings. +* Add "D" shortcut in mitmproxy to duplicate a flow. +* Reverse proxy mode. In this mode mitmproxy acts as an HTTP server, + forwarding all traffic to a specified upstream server. +* UI improvements - use unicode characters to make GUI more compact, + improve spacing and layout throughout. +* Add support for filtering by HTTP method. +* Add the ability to specify an HTTP body size limit. +* Move to typed netstrings for serialization format - this makes 0.7 + backwards-incompatible with serialized data from 0.6! + +* Significant improvements in speed and responsiveness of UI. +* Many minor bugfixes and improvements. + +7 August 2011: mitmproxy 0.6 +============================ + +* New scripting API that allows much more flexible and fine-grained + rewriting of traffic. See the docs for more info. +* Support for gzip and deflate content encodings. A new "z" + keybinding in mitmproxy to let us quickly encode and decode content, plus + automatic decoding for the "pretty" view mode. +* An event log, viewable with the "v" shortcut in mitmproxy, and the + "-e" command-line flag in mitmdump. +* Huge performance improvements: mitmproxy interface, loading + large numbers of flows from file. +* A new "replace" convenience method for all flow objects, that does a + universal regex-based string replacement. +* Header management has been rewritten to maintain both case and order. +* Improved stability for SSL interception. +* Default expiry time on generated SSL certs has been dropped to avoid an + OpenSSL overflow bug that caused certificates to expire in the distant + past on some systems. +* A "pretty" view mode for JSON and form submission data. +* Expanded documentation and examples. +* Countless other small improvements and bugfixes. + +27 June 2011: mitmproxy 0.5 +=========================== + +* An -n option to start the tools without binding to a proxy port. +* Allow scripts, hooks, sticky cookies etc. to run on flows loaded from + save files. +* Regularize command-line options for mitmproxy and mitmdump. +* Add an "SSL exception" to mitmproxy's license to remove possible + distribution issues. +* Add a --cert-wait-time option to make mitmproxy pause after a new SSL + certificate is generated. This can pave over small discrepancies in + system time between the client and server. +* Handle viewing big request and response bodies more elegantly. Only + render the first 100k of large documents, and try to avoid running the + XML indenter on non-XML data. +* BUGFIX: Make the "revert" keyboard shortcut in mitmproxy work after a + flow has been replayed. +* BUGFIX: Repair a problem that sometimes caused SSL connections to consume + 100% of CPU. + +30 March 2011: mitmproxy 0.4 +============================ + +* Full serialization of HTTP conversations +* Client and server replay +* On-the-fly generation of dummy SSL certificates +* mitmdump has "grown up" into a powerful tcpdump-like tool for HTTP/S +* Dozens of improvements to the mitmproxy console interface +* Python scripting hooks for programmatic modification of traffic + +01 March 2010: mitmproxy 0.2 +============================ + +* Big speed and responsiveness improvements, thanks to Thomas Roth +* Support urwid 0.9.9 +* Terminal beeping based on filter expressions +* Filter expressions for terminal beeps, limits, interceptions and sticky + cookies can now be passed on the command line. +* Save requests and responses to file +* Split off non-interactive dump functionality into a new tool called + mitmdump +* "A" will now accept all intercepted connections +* Lots of bugfixes diff -Nru mitmproxy-5.1.1/codecov.yml mitmproxy-6.0.2/codecov.yml --- mitmproxy-5.1.1/codecov.yml 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/codecov.yml 2020-12-15 16:41:27.000000000 +0000 @@ -1 +1,18 @@ comment: off +coverage: + status: + project: + default: false # disable the default status that measures entire project + python: + target: auto + threshold: 0.1% + paths: + - "!web/" + web: + target: auto + threshold: 0.1% + paths: + - "web/" +codecov: + notify: + after_n_builds: 7 diff -Nru mitmproxy-5.1.1/debian/changelog mitmproxy-6.0.2/debian/changelog --- mitmproxy-5.1.1/debian/changelog 2020-06-29 09:21:51.000000000 +0000 +++ mitmproxy-6.0.2/debian/changelog 2021-01-06 14:55:14.000000000 +0000 @@ -1,3 +1,34 @@ +mitmproxy (6.0.2-1) unstable; urgency=medium + + * New upstream release. + * Refresh patches and downgrade dependencies. + * Ignore test failures. + + -- Andrej Shadura Wed, 06 Jan 2021 15:55:14 +0100 + +mitmproxy (5.3.0-3) unstable; urgency=high + + * Use "not" with -k, not "-" for pytest 6 compatibility + (Closes: #977071). + + -- Andrej Shadura Mon, 04 Jan 2021 09:53:30 +0100 + +mitmproxy (5.3.0-2) unstable; urgency=medium + + * Pre-wrap dependencies of the binary package. + * Fix the dependencies of the binary package (Closes: #977926, #977927) + * Drop an explicit dependency on h11, wsproto depends on it already. + + -- Andrej Shadura Wed, 30 Dec 2020 09:51:56 +0100 + +mitmproxy (5.3.0-1) unstable; urgency=medium + + * New upstream release (Closes: #976035, #975178). + - Refresh patches + - Bump build dependencies. + + -- Andrej Shadura Mon, 30 Nov 2020 12:10:21 +0100 + mitmproxy (5.1.1-2) unstable; urgency=medium * Team upload. diff -Nru mitmproxy-5.1.1/debian/clean mitmproxy-6.0.2/debian/clean --- mitmproxy-5.1.1/debian/clean 2020-06-29 09:21:51.000000000 +0000 +++ mitmproxy-6.0.2/debian/clean 2021-01-06 14:55:14.000000000 +0000 @@ -1,2 +1,3 @@ debian/*.1 mitmproxy.egg-info/ +docs/src/assets/asciinema-* diff -Nru mitmproxy-5.1.1/debian/control mitmproxy-6.0.2/debian/control --- mitmproxy-5.1.1/debian/control 2020-06-29 09:21:51.000000000 +0000 +++ mitmproxy-6.0.2/debian/control 2021-01-06 14:55:14.000000000 +0000 @@ -1,24 +1,26 @@ Source: mitmproxy Section: net Priority: optional -Maintainer: Python Applications Packaging Team +Maintainer: Debian Python Team Uploaders: Sebastien Delafond , Andrej Shadura Build-Depends: dh-python, python3, python3-setuptools, pandoc, + python3-asgiref (>= 3.2.10), python3-blinker (>= 1.4), python3-brotli (>= 1.0), python3-certifi (>= 2019.9.11), - python3-cryptography (>= 2.8), + python3-click (>= 7.0), + python3-cryptography (>= 3.2), python3-coverage , python3-flask (>= 1.1.1), - python3-h11 (>= 0.7.0), - python3-h2 (>= 3.2.0), + python3-h2 (>= 4.0), python3-hypothesis (>= 5.8) , - python3-hyperframe (>= 5.1.0), + python3-hyperframe (>= 6.0), python3-kaitaistruct (>= 0.7), - python3-ldap3 (>= 2.6.1), + python3-ldap3 (>= 2.8), + python3-msgpack (>= 1.0.0), python3-openssl (>= 19.1.0), python3-protobuf (>= 3.6.0), python3-passlib (>= 1.6.5), @@ -47,9 +49,17 @@ Package: mitmproxy Architecture: all -Pre-Depends: dpkg (>= 1.17.14) -Depends: ${misc:Depends}, ${python3:Depends}, fonts-font-awesome (>= 4.2.0~dfsg), python3-h11, python3-pkg-resources -Conflicts: python-netlib +Pre-Depends: + dpkg (>= 1.17.14), +Depends: + fonts-font-awesome (>= 4.2.0~dfsg), + python3-h2 (>= 4.0), + python3-hyperframe (>= 6.0), + python3-pkg-resources, + ${misc:Depends}, + ${python3:Depends}, +Conflicts: + python-netlib, Description: SSL-capable man-in-the-middle HTTP proxy mitmproxy is an SSL-capable man-in-the-middle HTTP proxy. It provides a console interface that allows traffic flows to be inspected and diff -Nru mitmproxy-5.1.1/debian/patches/0001-Do-no-run-full-coverage-tests.patch mitmproxy-6.0.2/debian/patches/0001-Do-no-run-full-coverage-tests.patch --- mitmproxy-5.1.1/debian/patches/0001-Do-no-run-full-coverage-tests.patch 2020-06-29 09:21:51.000000000 +0000 +++ mitmproxy-6.0.2/debian/patches/0001-Do-no-run-full-coverage-tests.patch 2021-01-06 14:55:14.000000000 +0000 @@ -3,15 +3,15 @@ Subject: Do no run full coverage tests --- - setup.cfg | 82 -------------------------------------------------------- + setup.cfg | 87 -------------------------------------------------------- test/conftest.py | 2 -- - 2 files changed, 84 deletions(-) + 2 files changed, 89 deletions(-) diff --git a/setup.cfg b/setup.cfg -index d0dcc2d..80b6878 100644 +index 5006359..a2323e8 100644 --- a/setup.cfg +++ b/setup.cfg -@@ -9,18 +9,6 @@ addons = file,open,basestring,xrange,unicode,long,cmp +@@ -9,20 +9,6 @@ addons = file,open,basestring,xrange,unicode,long,cmp testpaths = test addopts = --capture=no --color=yes @@ -23,14 +23,16 @@ -show_missing = True -exclude_lines = - pragma: no cover -- raise NotImplementedError() +- raise NotImplementedError +- raise AssertionError - if typing.TYPE_CHECKING: - if TYPE_CHECKING: +- @overload - [mypy] ignore_missing_imports = True -@@ -32,73 +20,3 @@ ignore_errors = True +@@ -34,76 +20,3 @@ ignore_errors = True [mypy-test.*] ignore_errors = True @@ -57,13 +59,16 @@ -[tool:individual_coverage] -exclude = - mitmproxy/addons/onboardingapp/app.py +- mitmproxy/addons/session.py - mitmproxy/addons/termlog.py - mitmproxy/contentviews/base.py - mitmproxy/controller.py - mitmproxy/ctx.py - mitmproxy/exceptions.py - mitmproxy/flow.py +- mitmproxy/io/db.py - mitmproxy/io/io.py +- mitmproxy/io/protobuf.py - mitmproxy/io/tnetstring.py - mitmproxy/log.py - mitmproxy/master.py @@ -105,10 +110,10 @@ - pathod/test.py - release/hooks diff --git a/test/conftest.py b/test/conftest.py -index 7c7dec4..da401b4 100644 +index 5ce4b6f..463bd22 100644 --- a/test/conftest.py +++ b/test/conftest.py -@@ -5,8 +5,6 @@ from mitmproxy.utils import data +@@ -5,8 +5,6 @@ from mitmproxy.utils import data, compat import pytest diff -Nru mitmproxy-5.1.1/debian/patches/0002-Remove-upper-bound-versions-dependencies.patch mitmproxy-6.0.2/debian/patches/0002-Remove-upper-bound-versions-dependencies.patch --- mitmproxy-5.1.1/debian/patches/0002-Remove-upper-bound-versions-dependencies.patch 2020-06-29 09:21:51.000000000 +0000 +++ mitmproxy-6.0.2/debian/patches/0002-Remove-upper-bound-versions-dependencies.patch 2021-01-06 14:55:14.000000000 +0000 @@ -1,51 +1,55 @@ From: =?utf-8?q?S=C3=A9bastien_Delafond?= Date: Sun, 5 Aug 2018 13:57:01 +0200 -Subject: Remove upper-bound versions dependencies +Subject: Remove upper-bound versions dependencies and downgrade dependencies --- setup.py | 68 ++++++++++++++++++++++++++++++++-------------------------------- 1 file changed, 34 insertions(+), 34 deletions(-) diff --git a/setup.py b/setup.py -index 07e8286..7c7a1ea 100644 +index 485020a..9ac4e78 100644 --- a/setup.py +++ b/setup.py -@@ -62,49 +62,49 @@ setup( +@@ -67,46 +67,46 @@ setup( # https://packaging.python.org/en/latest/requirements/#install-requires # It is not considered best practice to use install_requires to pin dependencies to specific versions. install_requires=[ +- "asgiref>=3.2.10,<3.4", - "blinker>=1.4, <1.5", - "Brotli>=1.0,<1.1", ++ "asgiref>=3.2.10", + "blinker>=1.4", + "Brotli>=1.0", "certifi>=2019.9.11", # no semver here - this should always be on the last release! - "click>=7.0,<8", -- "cryptography>=2.9,<3.0", +- "cryptography>=3.3,<3.4", - "flask>=1.1.1,<1.2", -- "h2>=3.2.0,<4", -- "hyperframe>=5.1.0,<6", -- "kaitaistruct>=0.7,<0.9", -- "ldap3>=2.6.1,<2.8", +- "h2>=4.0,<5", +- "hyperframe>=6.0,<7", +- "kaitaistruct>=0.7,<0.10", +- "ldap3>=2.8,<2.9", +- "msgpack>=1.0.0, <1.1.0", - "passlib>=1.6.5, <1.8", -- "protobuf>=3.6.0, <3.12", +- "protobuf>=3.14,<3.15", - "pyasn1>=0.3.1,<0.5", -- "pyOpenSSL>=19.1.0,<19.2", +- "pyOpenSSL>=20.0,<20.1", - "pyparsing>=2.4.2,<2.5", - "pyperclip>=1.6.0,<1.9", - "ruamel.yaml>=0.16,<0.17", -- "sortedcontainers>=2.1.0,<2.2", +- "sortedcontainers>=2.3,<2.4", - "tornado>=4.3,<7", -- "urwid>=2.1.0,<2.2", -- "wsproto>=0.14,<0.16", +- "urwid>=2.1.1,<2.2", +- "wsproto>=1.0,<1.1", - "publicsuffix2>=2.20190812,<3", -- "zstandard>=0.11,<0.14", +- "zstandard>=0.11,<0.15", + "click>=7.0", -+ "cryptography>=2.9", ++ "cryptography>=3.2", + "flask>=1.1.1", -+ "h2>=3.2.0", -+ "hyperframe>=5.1.0", ++ "h2>=4.0", ++ "hyperframe>=6.0", + "kaitaistruct>=0.7", -+ "ldap3>=2.6.1", ++ "ldap3>=2.8", ++ "msgpack>=1.0.0", + "passlib>=1.6.5", + "protobuf>=3.6.0", + "pyasn1>=0.3.1", @@ -53,9 +57,9 @@ + "pyparsing>=2.4.2", + "pyperclip>=1.6.0", + "ruamel.yaml>=0.16", -+ "sortedcontainers>=2.1.0", ++ "sortedcontainers>=2.1", + "tornado>=4.3", -+ "urwid>=2.1.0", ++ "urwid>=2.1.1", + "wsproto>=0.14", + "publicsuffix2>=2.20190812", + "zstandard>=0.11", @@ -66,31 +70,24 @@ + "pydivert>=2.0.3", ], 'dev': [ - "asynctest>=0.12.0", -- "Flask>=1.0,<1.2", -- "hypothesis>=5.8,<5.9", +- "hypothesis>=5.8,<6", - "parver>=0.1,<2.0", -- "pytest-asyncio>=0.10.0,<0.11", +- "pytest-asyncio>=0.10.0,<0.14,!=0.14", - "pytest-cov>=2.7.1,<3", - "pytest-timeout>=1.3.3,<2", -- "pytest-xdist>=1.29,<2", -- "pytest>=5.1.3,<6", +- "pytest-xdist>=2.1.0,<3", +- "pytest>=6.1.0,<7", - "requests>=2.9.1,<3", -- "tox>=3.5,<3.15", -+ "Flask>=1.0", +- "tox>=3.5,<4", + "hypothesis>=5.8", + "parver>=0.1", -+ "pytest-asyncio>=0.10.0", ++ "pytest-asyncio>=0.10.0,!=0.14", + "pytest-cov>=2.7.1", + "pytest-timeout>=1.3.3", -+ "pytest-xdist>=1.29", -+ "pytest>=5.1.3", ++ "pytest-xdist>=2.1.0", ++ "pytest>=6.1.0", + "requests>=2.9.1", + "tox>=3.5", - ], - 'examples': [ -- "beautifulsoup4>=4.4.1,<4.9" -+ "beautifulsoup4>=4.4.1" ] } ) diff -Nru mitmproxy-5.1.1/debian/patches/0003-Remove-test_xss_scanner.py.patch mitmproxy-6.0.2/debian/patches/0003-Remove-test_xss_scanner.py.patch --- mitmproxy-5.1.1/debian/patches/0003-Remove-test_xss_scanner.py.patch 2020-06-29 09:21:51.000000000 +0000 +++ mitmproxy-6.0.2/debian/patches/0003-Remove-test_xss_scanner.py.patch 1970-01-01 00:00:00.000000000 +0000 @@ -1,406 +0,0 @@ -From: =?utf-8?q?S=C3=A9bastien_Delafond?= -Date: Sun, 5 Aug 2018 13:57:59 +0200 -Subject: Remove test_xss_scanner.py - ---- - test/examples/test_xss_scanner.py | 391 -------------------------------------- - 1 file changed, 391 deletions(-) - delete mode 100644 test/examples/test_xss_scanner.py - -diff --git a/test/examples/test_xss_scanner.py b/test/examples/test_xss_scanner.py -deleted file mode 100644 -index 25237c4..0000000 ---- a/test/examples/test_xss_scanner.py -+++ /dev/null -@@ -1,391 +0,0 @@ --import pytest --import requests --from examples.complex import xss_scanner as xss --from mitmproxy.test import tflow, tutils -- -- --class TestXSSScanner(): -- def test_get_XSS_info(self): -- # First type of exploit: -- # Exploitable: -- xss_info = xss.get_XSS_data(b"" % -- xss.FULL_PAYLOAD, -- "https://example.com", -- "End of URL") -- expected_xss_info = xss.XSSData('https://example.com', -- "End of URL", -- '" % -- xss.FULL_PAYLOAD.replace(b"'", b"%27").replace(b'"', b"%22"), -- "https://example.com", -- "End of URL") -- expected_xss_info = xss.XSSData("https://example.com", -- "End of URL", -- '" % -- xss.FULL_PAYLOAD.replace(b"'", b"%27").replace(b'"', b"%22").replace(b"/", b"%2F"), -- "https://example.com", -- "End of URL") -- assert xss_info is None -- # Second type of exploit: -- # Exploitable: -- xss_info = xss.get_XSS_data(b"" % -- xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b">", b"%3E").replace(b"\"", b"%22"), -- "https://example.com", -- "End of URL") -- expected_xss_info = xss.XSSData("https://example.com", -- "End of URL", -- "';alert(0);g='", -- xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b">", b"%3E") -- .replace(b"\"", b"%22").decode('utf-8')) -- assert xss_info == expected_xss_info -- # Non-Exploitable: -- xss_info = xss.get_XSS_data(b"" % -- xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b"\"", b"%22").replace(b"'", b"%22"), -- "https://example.com", -- "End of URL") -- assert xss_info is None -- # Third type of exploit: -- # Exploitable: -- xss_info = xss.get_XSS_data(b"" % -- xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b">", b"%3E").replace(b"'", b"%27"), -- "https://example.com", -- "End of URL") -- expected_xss_info = xss.XSSData("https://example.com", -- "End of URL", -- '";alert(0);g="', -- xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b">", b"%3E") -- .replace(b"'", b"%27").decode('utf-8')) -- assert xss_info == expected_xss_info -- # Non-Exploitable: -- xss_info = xss.get_XSS_data(b"" % -- xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b"'", b"%27").replace(b"\"", b"%22"), -- "https://example.com", -- "End of URL") -- assert xss_info is None -- # Fourth type of exploit: Test -- # Exploitable: -- xss_info = xss.get_XSS_data(b"Test" % -- xss.FULL_PAYLOAD, -- "https://example.com", -- "End of URL") -- expected_xss_info = xss.XSSData("https://example.com", -- "End of URL", -- "'>", -- xss.FULL_PAYLOAD.decode('utf-8')) -- assert xss_info == expected_xss_info -- # Non-Exploitable: -- xss_info = xss.get_XSS_data(b"Test" % -- xss.FULL_PAYLOAD.replace(b"'", b"%27"), -- "https://example.com", -- "End of URL") -- assert xss_info is None -- # Fifth type of exploit: Test -- # Exploitable: -- xss_info = xss.get_XSS_data(b"Test" % -- xss.FULL_PAYLOAD.replace(b"'", b"%27"), -- "https://example.com", -- "End of URL") -- expected_xss_info = xss.XSSData("https://example.com", -- "End of URL", -- "\">", -- xss.FULL_PAYLOAD.replace(b"'", b"%27").decode('utf-8')) -- assert xss_info == expected_xss_info -- # Non-Exploitable: -- xss_info = xss.get_XSS_data(b"Test" % -- xss.FULL_PAYLOAD.replace(b"'", b"%27").replace(b"\"", b"%22"), -- "https://example.com", -- "End of URL") -- assert xss_info is None -- # Sixth type of exploit: Test -- # Exploitable: -- xss_info = xss.get_XSS_data(b"Test" % -- xss.FULL_PAYLOAD, -- "https://example.com", -- "End of URL") -- expected_xss_info = xss.XSSData("https://example.com", -- "End of URL", -- ">", -- xss.FULL_PAYLOAD.decode('utf-8')) -- assert xss_info == expected_xss_info -- # Non-Exploitable -- xss_info = xss.get_XSS_data(b"Test" % -- xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b">", b"%3E") -- .replace(b"=", b"%3D"), -- "https://example.com", -- "End of URL") -- assert xss_info is None -- # Seventh type of exploit: PAYLOAD -- # Exploitable: -- xss_info = xss.get_XSS_data(b"%s" % -- xss.FULL_PAYLOAD, -- "https://example.com", -- "End of URL") -- expected_xss_info = xss.XSSData("https://example.com", -- "End of URL", -- "", -- xss.FULL_PAYLOAD.decode('utf-8')) -- assert xss_info == expected_xss_info -- # Non-Exploitable -- xss_info = xss.get_XSS_data(b"%s" % -- xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b">", b"%3E").replace(b"/", b"%2F"), -- "https://example.com", -- "End of URL") -- assert xss_info is None -- # Eighth type of exploit: Test -- # Exploitable: -- xss_info = xss.get_XSS_data(b"Test" % -- xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b">", b"%3E"), -- "https://example.com", -- "End of URL") -- expected_xss_info = xss.XSSData("https://example.com", -- "End of URL", -- "Javascript:alert(0)", -- xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b">", b"%3E").decode('utf-8')) -- assert xss_info == expected_xss_info -- # Non-Exploitable: -- xss_info = xss.get_XSS_data(b"Test" % -- xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b">", b"%3E") -- .replace(b"=", b"%3D"), -- "https://example.com", -- "End of URL") -- assert xss_info is None -- # Ninth type of exploit: Test -- # Exploitable: -- xss_info = xss.get_XSS_data(b"Test" % -- xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b">", b"%3E"), -- "https://example.com", -- "End of URL") -- expected_xss_info = xss.XSSData("https://example.com", -- "End of URL", -- '" onmouseover="alert(0)" t="', -- xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b">", b"%3E").decode('utf-8')) -- assert xss_info == expected_xss_info -- # Non-Exploitable: -- xss_info = xss.get_XSS_data(b"Test" % -- xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b">", b"%3E") -- .replace(b'"', b"%22"), -- "https://example.com", -- "End of URL") -- assert xss_info is None -- # Tenth type of exploit: Test -- # Exploitable: -- xss_info = xss.get_XSS_data(b"Test" % -- xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b">", b"%3E"), -- "https://example.com", -- "End of URL") -- expected_xss_info = xss.XSSData("https://example.com", -- "End of URL", -- "' onmouseover='alert(0)' t='", -- xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b">", b"%3E").decode('utf-8')) -- assert xss_info == expected_xss_info -- # Non-Exploitable: -- xss_info = xss.get_XSS_data(b"Test" % -- xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b">", b"%3E") -- .replace(b"'", b"%22"), -- "https://example.com", -- "End of URL") -- assert xss_info is None -- # Eleventh type of exploit: Test -- # Exploitable: -- xss_info = xss.get_XSS_data(b"Test" % -- xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b">", b"%3E"), -- "https://example.com", -- "End of URL") -- expected_xss_info = xss.XSSData("https://example.com", -- "End of URL", -- " onmouseover=alert(0) t=", -- xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b">", b"%3E").decode('utf-8')) -- assert xss_info == expected_xss_info -- # Non-Exploitable: -- xss_info = xss.get_XSS_data(b"Test" % -- xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b">", b"%3E") -- .replace(b"=", b"%3D"), -- "https://example.com", -- "End of URL") -- assert xss_info is None -- -- def test_get_SQLi_data(self): -- sqli_data = xss.get_SQLi_data("SQL syntax MySQL", -- "", -- "https://example.com", -- "End of URL") -- expected_sqli_data = xss.SQLiData("https://example.com", -- "End of URL", -- "SQL syntax.*MySQL", -- "MySQL") -- assert sqli_data == expected_sqli_data -- sqli_data = xss.get_SQLi_data("SQL syntax MySQL", -- "SQL syntax MySQL", -- "https://example.com", -- "End of URL") -- assert sqli_data is None -- -- def test_inside_quote(self): -- assert not xss.inside_quote("'", b"no", 0, b"no") -- assert xss.inside_quote("'", b"yes", 0, b"'yes'") -- assert xss.inside_quote("'", b"yes", 1, b"'yes'otherJunk'yes'more") -- assert not xss.inside_quote("'", b"longStringNotInIt", 1, b"short") -- -- def test_paths_to_text(self): -- text = xss.paths_to_text("""

STRING

-- -- """, "STRING") -- expected_text = ["/html/head/h1", "/html/script"] -- assert text == expected_text -- assert xss.paths_to_text("""""", "STRING") == [] -- -- def mocked_requests_vuln(*args, headers=None, cookies=None): -- class MockResponse: -- def __init__(self, html, headers=None, cookies=None): -- self.text = html -- return MockResponse("%s" % xss.FULL_PAYLOAD) -- -- def mocked_requests_invuln(*args, headers=None, cookies=None): -- class MockResponse: -- def __init__(self, html, headers=None, cookies=None): -- self.text = html -- return MockResponse("") -- -- def test_test_end_of_url_injection(self, get_request_vuln): -- xss_info = xss.test_end_of_URL_injection("", "https://example.com/index.html", {})[0] -- expected_xss_info = xss.XSSData('https://example.com/index.html/1029zxcs\'d"aoso[sb]po(pc)se;sl/bsl\\eq=3847asd', -- 'End of URL', -- '', -- '1029zxcs\\\'d"aoso[sb]po(pc)se;sl/bsl\\\\eq=3847asd') -- sqli_info = xss.test_end_of_URL_injection("", "https://example.com/", {})[1] -- assert xss_info == expected_xss_info -- assert sqli_info is None -- -- def test_test_referer_injection(self, get_request_vuln): -- xss_info = xss.test_referer_injection("", "https://example.com/", {})[0] -- expected_xss_info = xss.XSSData('https://example.com/', -- 'Referer', -- '', -- '1029zxcs\\\'d"aoso[sb]po(pc)se;sl/bsl\\\\eq=3847asd') -- sqli_info = xss.test_referer_injection("", "https://example.com/", {})[1] -- assert xss_info == expected_xss_info -- assert sqli_info is None -- -- def test_test_user_agent_injection(self, get_request_vuln): -- xss_info = xss.test_user_agent_injection("", "https://example.com/", {})[0] -- expected_xss_info = xss.XSSData('https://example.com/', -- 'User Agent', -- '', -- '1029zxcs\\\'d"aoso[sb]po(pc)se;sl/bsl\\\\eq=3847asd') -- sqli_info = xss.test_user_agent_injection("", "https://example.com/", {})[1] -- assert xss_info == expected_xss_info -- assert sqli_info is None -- -- def test_test_query_injection(self, get_request_vuln): -- -- xss_info = xss.test_query_injection("", "https://example.com/vuln.php?cmd=ls", {})[0] -- expected_xss_info = xss.XSSData('https://example.com/vuln.php?cmd=1029zxcs\'d"aoso[sb]po(pc)se;sl/bsl\\eq=3847asd', -- 'Query', -- '', -- '1029zxcs\\\'d"aoso[sb]po(pc)se;sl/bsl\\\\eq=3847asd') -- sqli_info = xss.test_query_injection("", "https://example.com/vuln.php?cmd=ls", {})[1] -- assert xss_info == expected_xss_info -- assert sqli_info is None -- -- @pytest.fixture(scope='function') -- def logger(self, monkeypatch): -- class Logger(): -- def __init__(self): -- self.args = [] -- -- def info(self, str): -- self.args.append(str) -- -- def error(self, str): -- self.args.append(str) -- -- logger = Logger() -- monkeypatch.setattr("mitmproxy.ctx.log", logger) -- yield logger -- -- @pytest.fixture(scope='function') -- def get_request_vuln(self, monkeypatch): -- monkeypatch.setattr(requests, 'get', self.mocked_requests_vuln) -- -- @pytest.fixture(scope='function') -- def get_request_invuln(self, monkeypatch): -- monkeypatch.setattr(requests, 'get', self.mocked_requests_invuln) -- -- @pytest.fixture(scope='function') -- def mock_gethostbyname(self, monkeypatch): -- def gethostbyname(domain): -- claimed_domains = ["google.com"] -- if domain not in claimed_domains: -- from socket import gaierror -- raise gaierror("[Errno -2] Name or service not known") -- else: -- return '216.58.221.46' -- -- monkeypatch.setattr("socket.gethostbyname", gethostbyname) -- -- def test_find_unclaimed_URLs(self, logger, mock_gethostbyname): -- xss.find_unclaimed_URLs("", -- "https://example.com") -- assert logger.args == [] -- xss.find_unclaimed_URLs("", -- "https://example.com") -- assert logger.args[0] == 'XSS found in https://example.com due to unclaimed URL "http://unclaimedDomainName.com".' -- xss.find_unclaimed_URLs("", -- "https://example.com") -- assert logger.args[1] == 'XSS found in https://example.com due to unclaimed URL "http://unclaimedDomainName.com".' -- xss.find_unclaimed_URLs("", -- "https://example.com") -- assert logger.args[2] == 'XSS found in https://example.com due to unclaimed URL "http://unclaimedDomainName.com".' -- -- def test_log_XSS_data(self, logger): -- xss.log_XSS_data(None) -- assert logger.args == [] -- # self, url: str, injection_point: str, exploit: str, line: str -- xss.log_XSS_data(xss.XSSData('https://example.com', -- 'Location', -- 'String', -- 'Line of HTML')) -- assert logger.args[0] == '===== XSS Found ====' -- assert logger.args[1] == 'XSS URL: https://example.com' -- assert logger.args[2] == 'Injection Point: Location' -- assert logger.args[3] == 'Suggested Exploit: String' -- assert logger.args[4] == 'Line: Line of HTML' -- -- def test_log_SQLi_data(self, logger): -- xss.log_SQLi_data(None) -- assert logger.args == [] -- xss.log_SQLi_data(xss.SQLiData('https://example.com', -- 'Location', -- 'Oracle.*Driver', -- 'Oracle')) -- assert logger.args[0] == '===== SQLi Found =====' -- assert logger.args[1] == 'SQLi URL: https://example.com' -- assert logger.args[2] == 'Injection Point: Location' -- assert logger.args[3] == 'Regex used: Oracle.*Driver' -- -- def test_get_cookies(self): -- mocked_req = tutils.treq() -- mocked_req.cookies = [("cookieName2", "cookieValue2")] -- mocked_flow = tflow.tflow(req=mocked_req) -- # It only uses the request cookies -- assert xss.get_cookies(mocked_flow) == {"cookieName2": "cookieValue2"} -- -- def test_response(self, get_request_invuln, logger): -- mocked_flow = tflow.tflow( -- req=tutils.treq(path=b"index.html?q=1"), -- resp=tutils.tresp(content=b'') -- ) -- xss.response(mocked_flow) -- assert logger.args == [] -- -- def test_data_equals(self): -- xssData = xss.XSSData("a", "b", "c", "d") -- sqliData = xss.SQLiData("a", "b", "c", "d") -- assert xssData == xssData -- assert sqliData == sqliData diff -Nru mitmproxy-5.1.1/debian/patches/0005-Remove-test_readfile.py.patch mitmproxy-6.0.2/debian/patches/0005-Remove-test_readfile.py.patch --- mitmproxy-5.1.1/debian/patches/0005-Remove-test_readfile.py.patch 2020-06-29 09:21:51.000000000 +0000 +++ mitmproxy-6.0.2/debian/patches/0005-Remove-test_readfile.py.patch 2021-01-06 14:55:14.000000000 +0000 @@ -9,7 +9,7 @@ diff --git a/test/mitmproxy/addons/test_readfile.py b/test/mitmproxy/addons/test_readfile.py deleted file mode 100644 -index 94e18cd..0000000 +index 0383beb..0000000 --- a/test/mitmproxy/addons/test_readfile.py +++ /dev/null @@ -1,122 +0,0 @@ @@ -17,7 +17,7 @@ -import io - -import pytest --import asynctest +-from unittest import mock - -import mitmproxy.io -from mitmproxy import exceptions @@ -69,17 +69,17 @@ - - tf = tmpdir.join("tfile") - -- with asynctest.patch('mitmproxy.master.Master.load_flow') as mck: +- with mock.patch('mitmproxy.master.Master.load_flow') as mck: - tf.write(data.getvalue()) - tctx.configure( - rf, - rfile = str(tf), - readfile_filter = ".*" - ) -- assert not mck.awaited +- mck.assert_not_awaited() - rf.running() - await asyncio.sleep(0) -- assert mck.awaited +- mck.assert_awaited() - - tf.write(corrupt_data.getvalue()) - tctx.configure(rf, rfile=str(tf)) @@ -108,16 +108,16 @@ - - -class TestReadFileStdin: -- @asynctest.patch('sys.stdin') +- @mock.patch('sys.stdin') - @pytest.mark.asyncio - async def test_stdin(self, stdin, data, corrupt_data): - rf = readfile.ReadFileStdin() - with taddons.context(rf): -- with asynctest.patch('mitmproxy.master.Master.load_flow') as mck: +- with mock.patch('mitmproxy.master.Master.load_flow') as mck: - stdin.buffer = data -- assert not mck.awaited +- mck.assert_not_awaited() - await rf.load_flows(stdin.buffer) -- assert mck.awaited +- mck.assert_awaited() - - stdin.buffer = corrupt_data - with pytest.raises(exceptions.FlowReadException): @@ -128,10 +128,10 @@ - rf = readfile.ReadFileStdin() - with taddons.context(rf) as tctx: - tf = tmpdir.join("tfile") -- with asynctest.patch('mitmproxy.master.Master.load_flow') as mck: +- with mock.patch('mitmproxy.master.Master.load_flow') as mck: - tf.write(data.getvalue()) - tctx.configure(rf, rfile=str(tf)) -- assert not mck.awaited +- mck.assert_not_awaited() - rf.running() - await asyncio.sleep(0) -- assert mck.awaited +- mck.assert_awaited() diff -Nru mitmproxy-5.1.1/debian/patches/0006-Delete-asciinema-for-which-we-only-have-minified-ver.patch mitmproxy-6.0.2/debian/patches/0006-Delete-asciinema-for-which-we-only-have-minified-ver.patch --- mitmproxy-5.1.1/debian/patches/0006-Delete-asciinema-for-which-we-only-have-minified-ver.patch 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/debian/patches/0006-Delete-asciinema-for-which-we-only-have-minified-ver.patch 2021-01-06 14:55:14.000000000 +0000 @@ -0,0 +1,30 @@ +From: Andrej Shadura +Date: Mon, 30 Nov 2020 18:59:50 +0100 +Subject: Delete asciinema for which we only have minified version + +--- + docs/src/themes/mitmproxydocs/layouts/partials/header.html | 11 ----------- + 1 file changed, 11 deletions(-) + +diff --git a/docs/src/themes/mitmproxydocs/layouts/partials/header.html b/docs/src/themes/mitmproxydocs/layouts/partials/header.html +index a665f6a..8210b56 100644 +--- a/docs/src/themes/mitmproxydocs/layouts/partials/header.html ++++ b/docs/src/themes/mitmproxydocs/layouts/partials/header.html +@@ -17,17 +17,6 @@ + {{ $style := resources.Get "style.scss" | toCSS | minify }} + + +- {{ if .Params.has_asciinema }} +- {{- $styles := resources.Get "asciinema-player.css" | minify | fingerprint }} +- +- +- {{- $styles := resources.Get "asciinema-player.js" | minify | fingerprint }} +- +- +- {{- $styles := resources.Get "asciinema-tutorial.js" | minify | fingerprint }} +- +- {{ end }} +- + {{ range .AlternativeOutputFormats -}} + {{ printf `` .Rel .MediaType.Type .Permalink $.Site.Title | safeHTML }} + {{ end -}} diff -Nru mitmproxy-5.1.1/debian/patches/0006-Patch-out-zstd.patch mitmproxy-6.0.2/debian/patches/0006-Patch-out-zstd.patch --- mitmproxy-5.1.1/debian/patches/0006-Patch-out-zstd.patch 2020-06-29 09:21:51.000000000 +0000 +++ mitmproxy-6.0.2/debian/patches/0006-Patch-out-zstd.patch 2021-01-06 14:55:14.000000000 +0000 @@ -8,7 +8,7 @@ 2 files changed, 5 insertions(+), 6 deletions(-) diff --git a/mitmproxy/net/http/encoding.py b/mitmproxy/net/http/encoding.py -index 16d399c..07fd037 100644 +index 9ed2163..0f4cf3e 100644 --- a/mitmproxy/net/http/encoding.py +++ b/mitmproxy/net/http/encoding.py @@ -9,7 +9,7 @@ from io import BytesIO @@ -18,38 +18,38 @@ -import zstandard as zstd +#import zstandard as zstd - from typing import Union, Optional, AnyStr # noqa + from typing import Union, Optional, AnyStr, overload # noqa -@@ -53,7 +53,7 @@ def decode( +@@ -67,7 +67,7 @@ def decode( decoded = custom_decode[encoding](encoded) except KeyError: - decoded = codecs.decode(encoded, encoding, errors) + decoded = codecs.decode(encoded, encoding, errors) # type: ignore - if encoding in ("gzip", "deflate", "br", "zstd"): + if encoding in ("gzip", "deflate", "br"): _cache = CachedDecode(encoded, encoding, errors, decoded) return decoded except TypeError: -@@ -94,7 +94,7 @@ def encode(decoded: Optional[str], encoding: str, errors: str='strict') -> Optio +@@ -123,7 +123,7 @@ def encode(decoded: Union[None, str, bytes], encoding, errors='strict') -> Union encoded = custom_encode[encoding](decoded) except KeyError: - encoded = codecs.encode(decoded, encoding, errors) + encoded = codecs.encode(decoded, encoding, errors) # type: ignore - if encoding in ("gzip", "deflate", "br", "zstd"): + if encoding in ("gzip", "deflate", "br"): _cache = CachedDecode(encoded, encoding, errors, decoded) return encoded except TypeError: -@@ -188,7 +188,7 @@ custom_decode = { - "gzip": decode_gzip, +@@ -218,7 +218,7 @@ custom_decode = { "deflate": decode_deflate, + "deflateRaw": decode_deflate, "br": decode_brotli, - "zstd": decode_zstd, + #"zstd": decode_zstd, } custom_encode = { "none": identity, -@@ -196,7 +196,7 @@ custom_encode = { - "gzip": encode_gzip, +@@ -227,7 +227,7 @@ custom_encode = { "deflate": encode_deflate, + "deflateRaw": encode_deflate, "br": encode_brotli, - "zstd": encode_zstd, + #"zstd": encode_zstd, @@ -57,11 +57,11 @@ __all__ = ["encode", "decode"] diff --git a/setup.py b/setup.py -index 7c7a1ea..d766bd4 100644 +index 9ac4e78..c097eee 100644 --- a/setup.py +++ b/setup.py -@@ -84,7 +84,6 @@ setup( - "urwid>=2.1.0", +@@ -91,7 +91,6 @@ setup( + "urwid>=2.1.1", "wsproto>=0.14", "publicsuffix2>=2.20190812", - "zstandard>=0.11", diff -Nru mitmproxy-5.1.1/debian/patches/0007-Revert-use-OpenSSL-s-hostname-validation.patch mitmproxy-6.0.2/debian/patches/0007-Revert-use-OpenSSL-s-hostname-validation.patch --- mitmproxy-5.1.1/debian/patches/0007-Revert-use-OpenSSL-s-hostname-validation.patch 2020-06-29 09:21:51.000000000 +0000 +++ mitmproxy-6.0.2/debian/patches/0007-Revert-use-OpenSSL-s-hostname-validation.patch 1970-01-01 00:00:00.000000000 +0000 @@ -1,1000 +0,0 @@ -From: Andrej Shadura -Date: Fri, 19 Jun 2020 10:25:06 +0200 -Subject: Revert "use OpenSSL's hostname validation" - -This reverts commit e768f5ba83dd92d380f3e03da5da6c68edb3f45b. ---- - mitmproxy/net/tls.py | 51 +++++++------ - setup.py | 2 +- - test/mitmproxy/data/servercert/9da13359.0 | 42 +++++------ - test/mitmproxy/data/servercert/self-signed.pem | 87 +++++++++++---------- - test/mitmproxy/data/servercert/trusted-leaf.pem | 85 ++++++++++----------- - test/mitmproxy/data/servercert/trusted-root.pem | 88 +++++++++++----------- - .../net/data/verificationcerts/9da13359.0 | 42 +++++------ - .../net/data/verificationcerts/generate.py | 54 ++++++------- - .../net/data/verificationcerts/self-signed.crt | 41 +++++----- - .../net/data/verificationcerts/self-signed.key | 54 ++++++------- - .../net/data/verificationcerts/trusted-leaf.crt | 39 +++++----- - .../net/data/verificationcerts/trusted-leaf.key | 54 ++++++------- - .../net/data/verificationcerts/trusted-root.crt | 42 +++++------ - .../net/data/verificationcerts/trusted-root.key | 54 ++++++------- - .../net/data/verificationcerts/trusted-root.srl | 2 +- - test/mitmproxy/net/test_tcp.py | 20 ++--- - 16 files changed, 372 insertions(+), 385 deletions(-) - -diff --git a/mitmproxy/net/tls.py b/mitmproxy/net/tls.py -index d8e943d..36c01cd 100644 ---- a/mitmproxy/net/tls.py -+++ b/mitmproxy/net/tls.py -@@ -7,13 +7,14 @@ import os - import struct - import threading - import typing -+from ssl import match_hostname, CertificateError - - import certifi - from OpenSSL import SSL - from kaitaistruct import KaitaiStream - --import mitmproxy.options --from mitmproxy import certs, exceptions -+import mitmproxy.options # noqa -+from mitmproxy import exceptions, certs - from mitmproxy.contrib.kaitaistruct import tls_client_hello - from mitmproxy.net import check - -@@ -253,11 +254,33 @@ def create_client_context( - depth: int, - is_cert_verified: bool - ) -> bool: -- if is_cert_verified and depth == 0 and not sni: -- conn.cert_error = exceptions.InvalidCertificateException( -- f"Certificate verification error for {address}: Cannot validate hostname, SNI missing." -- ) -- is_cert_verified = False -+ if is_cert_verified and depth == 0: -+ # Verify hostname of leaf certificate. -+ cert = certs.Cert(x509) -+ try: -+ crt: typing.Dict[str, typing.Any] = dict( -+ subjectAltName=[("DNS", x.decode("ascii", "strict")) for x in cert.altnames] -+ ) -+ if cert.cn: -+ crt["subject"] = [[["commonName", cert.cn.decode("ascii", "strict")]]] -+ if sni: -+ # SNI hostnames allow support of IDN by using ASCII-Compatible Encoding -+ # Conversion algorithm is in RFC 3490 which is implemented by idna codec -+ # https://docs.python.org/3/library/codecs.html#text-encodings -+ # https://tools.ietf.org/html/rfc6066#section-3 -+ # https://tools.ietf.org/html/rfc4985#section-3 -+ hostname = sni.encode("idna").decode("ascii") -+ else: -+ hostname = "no-hostname" -+ match_hostname(crt, hostname) -+ except (ValueError, CertificateError) as e: -+ conn.cert_error = exceptions.InvalidCertificateException( -+ "Certificate verification error for {}: {}".format( -+ sni or repr(address), -+ str(e) -+ ) -+ ) -+ is_cert_verified = False - elif is_cert_verified: - pass - else: -@@ -279,20 +302,6 @@ def create_client_context( - **sslctx_kwargs, - ) - -- if sni: -- # Manually enable hostname verification on the context object. -- # https://wiki.openssl.org/index.php/Hostname_validation -- param = SSL._lib.SSL_CTX_get0_param(context._context) -- # Matching on the CN is disabled in both Chrome and Firefox, so we disable it, too. -- # https://www.chromestatus.com/feature/4981025180483584 -- SSL._lib.X509_VERIFY_PARAM_set_hostflags( -- param, -- SSL._lib.X509_CHECK_FLAG_NO_PARTIAL_WILDCARDS | SSL._lib.X509_CHECK_FLAG_NEVER_CHECK_SUBJECT -- ) -- SSL._openssl_assert( -- SSL._lib.X509_VERIFY_PARAM_set1_host(param, sni.encode("idna"), 0) == 1 -- ) -- - # Client Certs - if cert: - try: -diff --git a/setup.py b/setup.py -index 7c7a1ea..2620d9f 100644 ---- a/setup.py -+++ b/setup.py -@@ -66,7 +66,7 @@ setup( - "Brotli>=1.0", - "certifi>=2019.9.11", # no semver here - this should always be on the last release! - "click>=7.0", -- "cryptography>=2.9", -+ "cryptography>=2.8", - "flask>=1.1.1", - "h2>=3.2.0", - "hyperframe>=5.1.0", -diff --git a/test/mitmproxy/data/servercert/9da13359.0 b/test/mitmproxy/data/servercert/9da13359.0 -index 88ed414..5868a30 100644 ---- a/test/mitmproxy/data/servercert/9da13359.0 -+++ b/test/mitmproxy/data/servercert/9da13359.0 -@@ -1,21 +1,21 @@ -------BEGIN CERTIFICATE----- --MIIDazCCAlOgAwIBAgIUTE2oFYATbkkytGOt0+CIAOPpaeEwDQYJKoZIhvcNAQEL --BQAwRTELMAkGA1UEBhMCQVUxEzARBgNVBAgMClNvbWUtU3RhdGUxITAfBgNVBAoM --GEludGVybmV0IFdpZGdpdHMgUHR5IEx0ZDAeFw0xOTExMjMwMDA1MDlaFw0zOTEx --MTgwMDA1MDlaMEUxCzAJBgNVBAYTAkFVMRMwEQYDVQQIDApTb21lLVN0YXRlMSEw --HwYDVQQKDBhJbnRlcm5ldCBXaWRnaXRzIFB0eSBMdGQwggEiMA0GCSqGSIb3DQEB --AQUAA4IBDwAwggEKAoIBAQDZ7fqv4iC6738OvOC/ONFmhOVmMNe2SrGtk+7KIv/M --d7MJJKpIygwc99EfL3m6WFjvSQ8CABO9GnLIeoB6RzX5X904dPQzIekMkE27J4D5 --wzYWAmYsu9md//9rjadaA01zHvxtpc30AsToOrRa2Rv7Lwotvcop2bBCXXZMdgvD --BpCuCOl3cQ346LZBmED41q77P7SgnI98gCPuyyMVAFTZE4NnFUloVERTcMMo5Vwv --tXFT41B6FOqtyhZFY89W1SeNKlm+n8zu+/aUF5c9usxwvQzj79pzBFYS9xMZzbxl --BmU91GbLFNU32O+wp0jX6c1MfluGvRjcHQpK+gHRRJdZAgMBAAGjUzBRMB0GA1Ud --DgQWBBSTmtfFP/zQDiCbte3AG3vchjSekjAfBgNVHSMEGDAWgBSTmtfFP/zQDiCb --te3AG3vchjSekjAPBgNVHRMBAf8EBTADAQH/MA0GCSqGSIb3DQEBCwUAA4IBAQB6 --CnGmyJKVdD41Qs8QbyZ5VvrAkSq7DkL4CAIDy0pClOYjx51R16g4/dXPd8nuDjVB --GbRuO/ZHlKeO+RAVbV4s74KeBmQ4uyuZMvuiJGCDtTgu+fbgp1l41vj21bkDnz/X --0aVrwC4clh6O1/AFCY7cYygp22QJ0zGpIb9d0ly6QDMEocHJKasuQ8/0GPvv6wkY --hwNub2ScsVFHPrFzzufZblBk3Ks0YZw+IsCMTY7QtGb1TZhGeX9xAydqNMnmIxNj --XOE2FKx4ISQNFwf/U1LEAruO7PjbbsWx5FHr4oYV9TCLoERZFofwcGGM9o/cPc4P --CH8fGvFYoU30PG+xX0Pc -------END CERTIFICATE----- -+-----BEGIN CERTIFICATE----- -+MIIDXTCCAkWgAwIBAgIJALzkvKyFAwWYMA0GCSqGSIb3DQEBCwUAMEUxCzAJBgNV -+BAYTAkFVMRMwEQYDVQQIDApTb21lLVN0YXRlMSEwHwYDVQQKDBhJbnRlcm5ldCBX -+aWRnaXRzIFB0eSBMdGQwHhcNMTgwOTA3MDgyMjUxWhcNMzgwOTAyMDgyMjUxWjBF -+MQswCQYDVQQGEwJBVTETMBEGA1UECAwKU29tZS1TdGF0ZTEhMB8GA1UECgwYSW50 -+ZXJuZXQgV2lkZ2l0cyBQdHkgTHRkMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIB -+CgKCAQEAkvT4Y1ML8Gg4x5aFVygIW022tJsEyfuW4HsEvIarAGpFtUNkw0dOoAxv -+Gv71I0KAWOXxtc9DUjtq7ZXJcG+dBiheTYJ40lrYhHvUt7J37nrUF5v5trQzoE9I -+6WGtzTV8C8RI6F+M6GtwxgwRBgqkuXiK5ExPXAjGMMJZWtiEXHxGTNB6F4zy884m -+VjVtQi8jy+c5g21Awp3z5HrQLM210zwvgi7Rygk6/UM0AnmST4o32SqFSd+0MFUJ -+f3pH3xczfKmhU/TBoVEWRB1YzwixsJrzDOB8wOGnNKCsl45hYUJZZ8epVkyrvFZQ -+iMkwIqqBJbkU6H7fZBl68TJ8ascUCQIDAQABo1AwTjAdBgNVHQ4EFgQUkurgHlw1 -+xMP2wrsrGPTk0ofxCyowHwYDVR0jBBgwFoAUkurgHlw1xMP2wrsrGPTk0ofxCyow -+DAYDVR0TBAUwAwEB/zANBgkqhkiG9w0BAQsFAAOCAQEAcdExVlSvH6aVExNiQO3k -+cMamj+78woDn9x563vwzaGP24KvOXk1B/IJp5kqu3ZsXS0I0Mz6xwXHAXeuxaj06 -+cKgEpHKKgClLblXo2zWqo/3V1UFFpOVP/NhI3r21b+fPrS46rP0mw75haQCph8/8 -+buQr0OeAYbElliY/ji+cJiCJB8A/D13fUMV/NUUfPW/UE6497jOmz+6PtZNAoOFx -+evrmDcbCzbJxacyLJX04rsrt6DO09jb/+5lFm5Aqr6ySKasrmheIGEisl4o9Zbuy -+5PvYgbOEmFgPATIiWGpBO/rqwDdsmgyYFl+YfFoW0akXUVhDb2e5iRDx6Rs0fmN/ -+NA== -+-----END CERTIFICATE----- -diff --git a/test/mitmproxy/data/servercert/self-signed.pem b/test/mitmproxy/data/servercert/self-signed.pem -index 55145c5..d35284b 100644 ---- a/test/mitmproxy/data/servercert/self-signed.pem -+++ b/test/mitmproxy/data/servercert/self-signed.pem -@@ -1,49 +1,46 @@ - -----BEGIN CERTIFICATE----- --MIIDjTCCAnWgAwIBAgIUb0mIVHKB+bu2PGObggokU3Re7xMwDQYJKoZIhvcNAQEL --BQAwRTELMAkGA1UEBhMCQVUxEzARBgNVBAgMClNvbWUtU3RhdGUxITAfBgNVBAoM --GEludGVybmV0IFdpZGdpdHMgUHR5IEx0ZDAeFw0xOTExMjMwMDA1MDlaFw0zOTEx --MTgwMDA1MDlaMEUxCzAJBgNVBAYTAkFVMRMwEQYDVQQIDApTb21lLVN0YXRlMSEw --HwYDVQQKDBhJbnRlcm5ldCBXaWRnaXRzIFB0eSBMdGQwggEiMA0GCSqGSIb3DQEB --AQUAA4IBDwAwggEKAoIBAQDF6eqyZD2z83buUF4T+6AY0Zoe925a2AHOhGtHJMLo --9AD7FF1Xi1iksEvxbOI6mreHtvYKzUpfNsA3DkFdSO91HMSkdvWcDcExpW62sNK9 --gQQrpcCx7DogOrFiGSIHI1LIy1y6YEJma3G71SgGbw7g1QF64dTX9+BzVQJsloT2 --H3ZxTi8Fb6APJq6d/Tp67GTM8U82vM+FjLKzfH7RMSEPvSyvWSibw3AZP6owFaxz --DJtXpR7evOZbiZxqXmGOBl5OQPu9GdDA3Fyi9Drp7xa234loqd1a8PYyL8qWV/2G --HM3IJOzG8Y5PUJhL8CkDbx4LJ9LfzeSuBnQPUf2ZNalZAgMBAAGjdTBzMB0GA1Ud --DgQWBBQbkS0mAGPD4XjOv9GKzGMyR9ix3TAfBgNVHSMEGDAWgBQbkS0mAGPD4XjO --v9GKzGMyR9ix3TAPBgNVHRMBAf8EBTADAQH/MCAGA1UdEQQZMBeCFWV4YW1wbGUu --bWl0bXByb3h5Lm9yZzANBgkqhkiG9w0BAQsFAAOCAQEAqZ64xpu3qrTlCc555OoP --GgobUFP3qv07d0/r48cOyYdAdlEHhvmDPlqWTB9e4ZYtWZMlocY9DpCywzKTa7F7 --Ad8BwS0a/No3wVdl1UEkIGYxuD//jbd77Mrpf5URvQco85o/bicn+H0GAOchYt1P --jP1VShqsRv6WiTs5kn1/JwVoafddl1jBlMDmCqDv4loAZJYHzie0CqdjjSeorFfE --8FG8OLwmEnmIW6VnanRH8coH9MBbZ+dRtCavS+Q8s0R77dJM1sCp5/4yKcr5D/PD --+dQN9f+iugLxDdBQjiRyadWX9/l4n/h8ezabZ4cNsiWbRXrp5VS0nGNmAK3XvPAu --ow== -+MIIDEzCCAfugAwIBAgIJAKzH8k6aKTP6MA0GCSqGSIb3DQEBCwUAMCAxHjAcBgNV -+BAMMFWV4YW1wbGUubWl0bXByb3h5Lm9yZzAeFw0xODA5MDcwODIyNTNaFw0zODA5 -+MDIwODIyNTNaMCAxHjAcBgNVBAMMFWV4YW1wbGUubWl0bXByb3h5Lm9yZzCCASIw -+DQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMOjJMp2o5eLQEmYJqMZzLBi61h9 -+fsCVMvS8hgrH1Cg5q/RaLBLrZ8nILKmFZBapMUEFkUwQLB864tdTMaX7p+jNv3sM -+5LWEIYkTIbu6qV7QerKdubS1hpdFtQGRM1Q+C7H86FzF02DSKzNSmQc4fNed/lQM -+qo/jOm1xx4TZFR4j58BrmmoOfNP44IyrwXsPyXbMsukKixVEB3vQ2oyGDAyG6dYi -+VvM8PVL5yhX3BJ0D1Ky6hgGHJeirm0Cd8qqdSC/SWNdu1bGzg/xyUX5XFaHlIi7Y -+5YhD7ZDLvC76MeCWkfo4DaSB0CWmtG4l1TtHM2JqP8qf2l2LsABKs0q/a+UCAwEA -+AaNQME4wHQYDVR0OBBYEFIc9YAXgnGRhPTEcN/j+k/dxMdKqMB8GA1UdIwQYMBaA -+FIc9YAXgnGRhPTEcN/j+k/dxMdKqMAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcNAQEL -+BQADggEBAD9qKci3Pr4/2WGx+sv8gOpKchC9eF2dXc5hA3xbDw7T6oRLUBAY8Pty -+JF7DHMovT+w7FPRYT8rSc190fbSwVRHAnEaqAzaxteImCp/qYgdBHOz39eG4c93W -+YrYvA1VdUDPcUnisEVWguDsKJGFg+G6pw+8Wkf/hCrJJkriTFogGvzg6ptdQatvE -+dpSkionfbuZKz+7lny6sCBGoMRIFBd22MHJsSQOyTb06Lwc5dpdF9c5vysPRzShJ -+5kkgIjGpTmWp+Ud8BAMQH8EDhJMkJ7iw1+07UQ9MUmXCp9Xgim6x1ri2/yoz9HeO -+83VCkD9YWufrzOrsXpo04rMYtoKo+lw= - -----END CERTIFICATE----- - -----BEGIN RSA PRIVATE KEY----- --MIIEpAIBAAKCAQEAxenqsmQ9s/N27lBeE/ugGNGaHvduWtgBzoRrRyTC6PQA+xRd --V4tYpLBL8WziOpq3h7b2Cs1KXzbANw5BXUjvdRzEpHb1nA3BMaVutrDSvYEEK6XA --sew6IDqxYhkiByNSyMtcumBCZmtxu9UoBm8O4NUBeuHU1/fgc1UCbJaE9h92cU4v --BW+gDyaunf06euxkzPFPNrzPhYyys3x+0TEhD70sr1kom8NwGT+qMBWscwybV6Ue --3rzmW4mcal5hjgZeTkD7vRnQwNxcovQ66e8Wtt+JaKndWvD2Mi/Kllf9hhzNyCTs --xvGOT1CYS/ApA28eCyfS383krgZ0D1H9mTWpWQIDAQABAoIBAGl4H8+TZeJ5E18q --ywfhJ08ym+x2tYOJ62SP4s+WApy8M62aC6g0pTeWj9IH0YOjobycPwBAqKqW9dYh --Lao1zQ5fF1gB4R+ZoOQBIkAPeS7uCzfrbAYlOlCklpUNibm+FEbXQQI9fAUyqviL --Pno3QvmD6fb/VDsHaMBthA40JIU4C3yUUcSooKKUC3XKmbRDg5/stKbVdi6co5AQ --OMNFj4smts3rXtk5tKMBep4AgKKdClf4IuhpWEqZDxN6e+hZA7g1VOmOIE+hKEEO --ODBkjHNgHIFbsr8GTlZ+GSkHBxMNVdHxntJTU5a2jPiTNTTnhqu2hcVdkyDwLb9x --TGUCdyUCgYEA/ZPwI2KaT4Lj8b+6DCV4v3nZF8nLCtt1LiXxJWexdnr+3q/ZfNjN --5sKVl9rIynz2zoJDVr1MI8yjO6OmV7GBW2wgJsSFwg0vnV/bs75KqPZWZXqy/mrp --HnXd82XUweyyOpmBg4cCjmcnMTrdvCHvfgx7JafnUDqeXNwriWeC5Y8CgYEAx83d --qHgAT2rlnCQYRmCOGb5fuyQ+pSM4p8h1AjueLrTNiWcqX57ct9eVWjymD+AgevFZ --hZH9m6TI4nCb/5ukcvjQb4g1kyY3l7rZiiJ3kf/c22kVsVUagJGeiW6Ypnox4r1o --oFXEwAw1qSGIqOIjYVB2DqvjUl3+UjWCl9SOnpcCgYEA0TSdWUQ/VTwCvW9VmjHM --FgT8I5Ebj+CRI7qv4hFTqxE8dxKTl1nzPd/ptTgOkmhY4vU7gzN3vs1VGp4gXZcX --xwpE2Fcol3lzgB4Wz4s+Y3mgu+ZoCFjB7ZyGugmYZ0nVnV0KKi5X4I6gGhCb4VwK --D29SpjWJNHq4LpqC3MDmkGcCgYAxnKOKXmmtTpy+3ZONfhIqwEOjA0fu10UNHFA5 --grYvYMOcd5pk7dxeZdB2/JI7ZOqLvHv/F5YCXLNozo9ds7bsuW2AFDFBXX72VPYJ --P6+y9/ZOINS7GKeg/wd/lo+e3r6eT2u4TDOzgBSe722wiZ5BXqpB0Fp8rEwm+5R2 --wNe89wKBgQC59SOa4EW3gfjLRaH6hEPcMEzjr0Dt5ilBgntYbI+DsCePwEX28hAK --fd7A6XoCqjnwLpAmh0cmNRsYjtPCzg6TXUdrhfnR/3r7DHzvsxaC06BfkPZJ0+Fu --rP3el/oKSPaDsZKjW6RA5oqlIF82o45MNl2oCG82LgVtu25e3HbJSA== -+MIIEpgIBAAKCAQEAw6Mkynajl4tASZgmoxnMsGLrWH1+wJUy9LyGCsfUKDmr9Fos -+EutnycgsqYVkFqkxQQWRTBAsHzri11Mxpfun6M2/ewzktYQhiRMhu7qpXtB6sp25 -+tLWGl0W1AZEzVD4LsfzoXMXTYNIrM1KZBzh8153+VAyqj+M6bXHHhNkVHiPnwGua -+ag580/jgjKvBew/Jdsyy6QqLFUQHe9DajIYMDIbp1iJW8zw9UvnKFfcEnQPUrLqG -+AYcl6KubQJ3yqp1IL9JY127VsbOD/HJRflcVoeUiLtjliEPtkMu8Lvox4JaR+jgN -+pIHQJaa0biXVO0czYmo/yp/aXYuwAEqzSr9r5QIDAQABAoIBAQC1RpgymlffdgJt -+rvQuMRu/XQlhh3dJj3YV3BIAL0VguH+i/WLVbRdQm5D2y0kAzml7LGODrYCUt4W1 -+q7rXaCYfy3Xf2QSbRQGl9/pL7xw9ZMQseYW38nPx+39LInYDWzKPDB9qx0uj7Vpm -+ReTSEf9r81PUIaBxj0V2X/VWHag5scBjXoflQLxyV6i1UvTuWyhYvX1Bbaj02MqV -+tGNMrjbj25Wx2za53VDonzNA6RMZzkWGMfzmkkGM6kjGzLEsveys+bYCt7Fs7slR -+4oby0bIUmN7iqLhqlEhS4weWW4iHlq17X7CZeQAE1XeVZBz1N4G8FLjND2eyqb2N -+RAcQqp3BAoGBAOiU3WTu/kSccteDRxR1gVRPqEgfoLDwyAb7ORVUWX4Ii/z/soMw -+xZ2MlYPLnp3Fyu/hKhJPC1LzkD4CGHCTJJ1NnUudtDxl2Zh1FZYGmv1hi1TID/cm -+G0+3XhlJgztS41+AzxTNMulV1yieT2HIRIoRpdSx1UIA72l42YqjrwUNAoGBANdV -+/Ib+3hAfFtSMMI1qZQXvlKEoDRbUOCYuBVkTK8oQJQH6MLDokHZ8sXBAqi9383b1 -+XmhQBJZ//yMy0AqFa2QBlkK0Gizzhh7BLSjIT2LREf66B2cWzhgdhbSp6Nuk+3DK -+NfibxsFAPpW05HqtfxhbjrLfoE8VvTuMGQ8AaXw5AoGBAISo7IL2wrdV2TdN3Mwx -+ndv+N4kz6Q8jt6QrxUqCOy1lKJvdKPAlcIJFvr5W9RkeyXr7nmilB1uAK4UC4vfL -+JfZHX/HSeQx+N5f7KJ3TFLJz4eow1tJsvOVCPP0FbkH3LFO7/+HojSKEYN39NmAa -+v+VU3Zas/GvSZrxtPwASDvE9AoGBAJOBbluW6MzITx5H7dZhRFR9miWOxvCVbOUS -+b01mKX/f8UnadVIp7RONNQr88NdVZqxdRk9USOBDS6Vz4DjkzfySbbjBoJCcPIqC -+r4mZNXAuYRJJolqGr6SrTHTGUyFqcWcAzVnAc7TbakOoxz4V7NLlnOmA8FJcROUu -+gdfZ42hZAoGBANyL2IQ+L92iYVvqsz1zPBlvemevx8zP6GmlzvTcVexyDTIiLg6W -+BVil5zRDPJdDiFfBK18Qg1mJoE4SjLTg+yGww9ef37Zb9kZypy6pM6AbRWILZ1Gv -+7UsWUzk6rgcQpDdpJCUEt+AD3LQJTxxuoIhZePvC2GLkzsjZA7ZyB5+S - -----END RSA PRIVATE KEY----- -diff --git a/test/mitmproxy/data/servercert/trusted-leaf.pem b/test/mitmproxy/data/servercert/trusted-leaf.pem -index 29bd38f..a2c2531 100644 ---- a/test/mitmproxy/data/servercert/trusted-leaf.pem -+++ b/test/mitmproxy/data/servercert/trusted-leaf.pem -@@ -1,48 +1,45 @@ - -----BEGIN CERTIFICATE----- --MIIDajCCAlKgAwIBAgIJAKPoP2hXKVv4MA0GCSqGSIb3DQEBCwUAMEUxCzAJBgNV --BAYTAkFVMRMwEQYDVQQIDApTb21lLVN0YXRlMSEwHwYDVQQKDBhJbnRlcm5ldCBX --aWRnaXRzIFB0eSBMdGQwHhcNMTkxMTIzMDAwNTA5WhcNMzkxMTE4MDAwNTA5WjBF --MQswCQYDVQQGEwJBVTETMBEGA1UECAwKU29tZS1TdGF0ZTEhMB8GA1UECgwYSW50 --ZXJuZXQgV2lkZ2l0cyBQdHkgTHRkMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIB --CgKCAQEA4BZUGKkf52583Kps2SinCzRtGFaGIqtP1YfjaN1H21c08hEERRWf0GqW --PIbZz0NRyqOVM0O+4A9cxKrdqDpALBNXdgB62Ven1NNE31ZjsbFgwbORPTiOQo+6 --Xsu+2KHQdqAGOXSqOdNSoZV/HqAKa41iweg6lUf3yrO/hpfnlXTFbuM/oYFARrr9 --YWY7qH5BEvcZBf4Sg3cWGjdbzYg8S6hnFBBkHPDMbS3xra7H8uru8aG9L70aiq0C --3WsGf1/Qctz3cccB6BdnbiXji0QlP0miT+b52hEzDvNjdXvoObgbOZsUUcLrbz8q --LqaQPj6TA2PYpKcxEdJLDcbN++tglQIDAQABo10wWzAfBgNVHSMEGDAWgBSTmtfF --P/zQDiCbte3AG3vchjSekjAJBgNVHRMEAjAAMAsGA1UdDwQEAwIFoDAgBgNVHREE --GTAXghVleGFtcGxlLm1pdG1wcm94eS5vcmcwDQYJKoZIhvcNAQELBQADggEBAMoK --8+KTYF10dbQ8Hl3Fq4Iux7eutAPuKUXUz9dnCfDRhZukdl+gahRoNvkyMBdXOvvx --R9Z2+Guo8NOYKgT1mJtS/c8IxTkjZj9doujJOVD2wTowSfoaT9z+/EJa+6Fp9+xd --YVsO3E/2Vxai8PCNx8JTXr2axcnBDvpHPRXF21hOI8N94SPAcmLTZsdsTELjrGGa --/BA0y+pCEwW6cY9mMHVAAvRoMoqfocBVI7nrYBaQfFoKuwxscxO679eEv+lbSjul --z/VNdWfqrDhFFSzwRSVchapQ9q1EeTzv++wZRwI5bT0Ib6DFTJB3J5+9RihlFYfU --GI17CI0D//DsFic7QJ0= -+MIIC4TCCAckCCQCj6D9oVylb9zANBgkqhkiG9w0BAQsFADBFMQswCQYDVQQGEwJB -+VTETMBEGA1UECAwKU29tZS1TdGF0ZTEhMB8GA1UECgwYSW50ZXJuZXQgV2lkZ2l0 -+cyBQdHkgTHRkMB4XDTE4MDkwNzA4MjI1MloXDTM4MDkwMjA4MjI1MlowIDEeMBwG -+A1UEAwwVZXhhbXBsZS5taXRtcHJveHkub3JnMIIBIjANBgkqhkiG9w0BAQEFAAOC -+AQ8AMIIBCgKCAQEAqKzVdsRKgthv6V/dk3Tncy4ymbACs383nGutjulExvroNOCw -+b0y0e7unNGbtXxFQqSvA7eGaT1yRNfoMbXGSS+sn8A3gB6/s2A0Sw7KeSDdoaqEq -+F/LzRBed1YkxSyy0GXuTd7HXNIoFn/eF1tqxgViWdfyFD85qY4yJ+luofdm7IcPM -+ENPzV4nKzDh2PdJpQrEokWz2jM0zefC3IYnFHXY5bA3MnhE03/P0VxeEYkBdmEAt -+O1U2Bkw9SKCLy9zF13ks6/dDZ9LjMtRKI83gQS5z3S3bA45YxFuyeLWgVsJ2NYTa -+j9/8c4xwOjg9TpkCvcmZiPUYGddPHWoKqAAhBwIDAQABMA0GCSqGSIb3DQEBCwUA -+A4IBAQAf8cjxunN4Y7NUD2Z/SNOJ/s0uWJtTPV6m4FxSwwD0wfbsyirPchmattLc -+BabrQkeMMm8gMOrORfanXQwvLZvX0aDf96EgLSfHv8Iqeol5Byrgkn7UORXl20Jt -+8UNRURUZYtWxn08P8dlhxQUncPF/UxCesC8x0cihqv+YTB3TX1sni9mOqPCYY8yH -+E8kCW4zTJ0J9OQUHq9qdYQM/PGVm99+DWBItUeZAva8Rqj1FN3f9j1eWB+EjfYu7 -+ztsTInpNWP4tIh6vIFtuaGr077cJawTe6YVyNxVqquI9+2fpSPkt7tCTIhbQ4AmM -+DeHzn+KjfKN8ooWqmcfmUZWaADe0 - -----END CERTIFICATE----- - -----BEGIN RSA PRIVATE KEY----- --MIIEowIBAAKCAQEA4BZUGKkf52583Kps2SinCzRtGFaGIqtP1YfjaN1H21c08hEE --RRWf0GqWPIbZz0NRyqOVM0O+4A9cxKrdqDpALBNXdgB62Ven1NNE31ZjsbFgwbOR --PTiOQo+6Xsu+2KHQdqAGOXSqOdNSoZV/HqAKa41iweg6lUf3yrO/hpfnlXTFbuM/ --oYFARrr9YWY7qH5BEvcZBf4Sg3cWGjdbzYg8S6hnFBBkHPDMbS3xra7H8uru8aG9 --L70aiq0C3WsGf1/Qctz3cccB6BdnbiXji0QlP0miT+b52hEzDvNjdXvoObgbOZsU --UcLrbz8qLqaQPj6TA2PYpKcxEdJLDcbN++tglQIDAQABAoIBAG5V8DB4TdI5T9ej --Ppcqch2NQc5DBCbb7SI5l5qRogj5BoPOJykQ/bC0WqcQyvxHrGU3aIZma/yM8+OO --Mjfb/q71ExJyKAsOIwAiyn2hXtMmgHq/vNrFFx7lACIe9ihafHd8UbRGom54g+41 --2vKsYJUWd7L8cqQAXJz9JmfSMeAfRBVVLyOpENSVT8gM/LgPokmZbg0wRJTA5oa3 --10f4Ax1HqXO3cQgKKnrvIU72gB7MQ9Y2G+P2eAmiAlzGr41N5sv7uFz0JrFHhqtC --qc7QrGVVNtxanu0s/LWgfQ+LrKldTysX16j/j6J0tg7Pfuq9cMFPklU98dv2LF5J --jMB8eQECgYEA84+Yy/IGVjUPVb3HjmCmWUoxt4Aqop/12/4+1cBNS6h58iLJpcWc --FVh8m3UbuE7shimEg0fZ1rPPkqepKlDw8/sKXDni/khBASOAh2v1RhvOxNGjxDfn --ZvcK+BG99kU8ZWQHpKmn8OGfYswL2Fvo67L9XFg/wDnk1QpJx7E1GVUCgYEA64gg --G6qnsah1NKSW64BPVKi/Gby83l4oa1amjHqq/Unq6iuGB4/QhK4BzW0aOGI7RMTE --SPh5NF5ZhfT3LWD/EnaMm6QxMTSsL9TDIg36WTVZTkbODFOJczg19EdTs1pa6OXW --0NM6psLu6Nf/QmHJzyYxSZ83uqPOM305xgooKkECgYEAnA0XMySglr9sUd1EbJ7U --NkVpUU8XAhdHKWrey4lofN83MsLDPCk+dha5z8jat94pgVQ8iPiSRBP1HNu7cVdm --6ouf+bNFEvMsYxRiF2I+RmsuscA4E1JWOwxxxLtpYM6/gZ7znrbs2VNWEbD2retF --cy69UltgjUMKsMzktMN/Z/kCgYAbzV285lAVMIVlSWhnNCYpICIur5C7zvGGehv+ --yRwV+fu42JphmiBLCR89WHuX3ECSxYdF9c6Y1+pJXbkvqhtx2nyOgrsry8PngX3n --Ly82CI4aJ1F7MwEukJwN0b2XljrU8wyAae6qcKgy5AxFkbV4tlFrF1hEt8FHYqjH --L7u+AQKBgBnN2vjOVagtlZjJadg5ueR7uKnfk7/TGMtZjNf89C1UZH3TqWnnj6p6 --rLLe4tnA3EebFtGeVWlmnw5k166HRBEd7KuebOCq3hkNlf71aHwhDJ0bIdDwo1g+ --FTpG2vqkqpuEb+2FzDjZT5EdPnj8tUGthREDXpXqsdjFb6+2Enjx -+MIIEogIBAAKCAQEAqKzVdsRKgthv6V/dk3Tncy4ymbACs383nGutjulExvroNOCw -+b0y0e7unNGbtXxFQqSvA7eGaT1yRNfoMbXGSS+sn8A3gB6/s2A0Sw7KeSDdoaqEq -+F/LzRBed1YkxSyy0GXuTd7HXNIoFn/eF1tqxgViWdfyFD85qY4yJ+luofdm7IcPM -+ENPzV4nKzDh2PdJpQrEokWz2jM0zefC3IYnFHXY5bA3MnhE03/P0VxeEYkBdmEAt -+O1U2Bkw9SKCLy9zF13ks6/dDZ9LjMtRKI83gQS5z3S3bA45YxFuyeLWgVsJ2NYTa -+j9/8c4xwOjg9TpkCvcmZiPUYGddPHWoKqAAhBwIDAQABAoIBABsUC/zSDEgvKOAl -+RLP8a3+hJfxoNjbMsIfK/YTYy/LJqud6PrjPbpYCjRgrgeXmKLXP0VwfAJ/G84Tf -+zIjxV5Qaf0HZaGKzimkwyBdkoGZlhry/fLt1hDolNHBoYuJ3nb4NiaIIiczkb3y7 -+xt+0IhTqvNTaIh5ke83ZbPklJ8p0HAzw1q6+iRFZiZKH1iVRwJyIyK654wpNQ93W -+SqUKa3uAbqw+Bx9fzyEunANFwoBcZka9oSR9bTlhGB8HPHZFVYKgZvE4n9WOclKW -+E75pGG6vFYZkxBdqcjFNlPKKZRisDuey28teiHXThh1MvYxRdaMq4oxOE1J+n17k -+F2gQolECgYEA01j1FlExu45+U36n3tCCyS50dTtf0Qpi71c1s5DZyT+AAB2ZSGXm -+VBtKgVRNg/iWfHn5b/zHF30OtgIzcsrU66cWMwIXPUQigXh8Cteve7VMs03hce1w -+wsFwLoyvdWEam32YAymqRgN3H6JQim82IJJ3YlWrgEytBnvLkADCihkCgYEAzE/g -+8aoxDZJUwbvaZjLwuydmvc+aAwanVgqvtkca4x99oPhNQna6O0jXXAtJXMA3SHp5 -+QYMDKh98BqCXyfXd+1Semc9pgAPz7l4j09WG7Zdap3xinTOkUsmTAz/2T967HIsP -+6qP7RUiwjmbUk8ZGcKsNjoxzPA4JURYimKB5qB8CgYBTjlnnJtaYpi8/Z1WK+7iZ -+PSqBpqWtCYQvx7TNdzkDHX3Hjewp+U9kdR2xn9i9kiw8riR1p+Q2XxTP1HLusU4Y -+lIhsRilV6XgS48V2q+sO55CZWvMEjbEE7mEhpjFAINHaI39T0Mcmwvv3n75j3K/z -+lLRqRiB1qtrFM3A5UHOZEQKBgBPQm2xUqTU7v+SaJ3BJ+HbuN1SpUbKBbrE1kB0J -+gF4Oq8x0yGltwloFkn1mytKoAbSRzDjCUAhBzXGHGbGImuLJLiiUqRK1T28Kyka9 -+KrzYNP6RXa8JVyKAUjW6elT8sQDvq7eB99icWCM3bd53GFXNAR+WF4b3hYfLscdD -+qQjZAoGAB3ah068Qscb2Ef3+eufa+EvOfMDrNNvlEoZXRlhviTg3NEwjyqbxaCIy -+6Xg+rWvgJm9UE2RBOcCNeghkEabmL1+8DvmDiV1lt9oJtqULBirvalp2H3+9yiTk -+j5dnVcRF6cYvNFmwTr2WZFBGgq96d/Zmbx3o3MIqSBc8I6pLNzo= - -----END RSA PRIVATE KEY----- -diff --git a/test/mitmproxy/data/servercert/trusted-root.pem b/test/mitmproxy/data/servercert/trusted-root.pem -index 2248d9f..a53cb89 100644 ---- a/test/mitmproxy/data/servercert/trusted-root.pem -+++ b/test/mitmproxy/data/servercert/trusted-root.pem -@@ -1,48 +1,48 @@ - -----BEGIN CERTIFICATE----- --MIIDazCCAlOgAwIBAgIUTE2oFYATbkkytGOt0+CIAOPpaeEwDQYJKoZIhvcNAQEL --BQAwRTELMAkGA1UEBhMCQVUxEzARBgNVBAgMClNvbWUtU3RhdGUxITAfBgNVBAoM --GEludGVybmV0IFdpZGdpdHMgUHR5IEx0ZDAeFw0xOTExMjMwMDA1MDlaFw0zOTEx --MTgwMDA1MDlaMEUxCzAJBgNVBAYTAkFVMRMwEQYDVQQIDApTb21lLVN0YXRlMSEw --HwYDVQQKDBhJbnRlcm5ldCBXaWRnaXRzIFB0eSBMdGQwggEiMA0GCSqGSIb3DQEB --AQUAA4IBDwAwggEKAoIBAQDZ7fqv4iC6738OvOC/ONFmhOVmMNe2SrGtk+7KIv/M --d7MJJKpIygwc99EfL3m6WFjvSQ8CABO9GnLIeoB6RzX5X904dPQzIekMkE27J4D5 --wzYWAmYsu9md//9rjadaA01zHvxtpc30AsToOrRa2Rv7Lwotvcop2bBCXXZMdgvD --BpCuCOl3cQ346LZBmED41q77P7SgnI98gCPuyyMVAFTZE4NnFUloVERTcMMo5Vwv --tXFT41B6FOqtyhZFY89W1SeNKlm+n8zu+/aUF5c9usxwvQzj79pzBFYS9xMZzbxl --BmU91GbLFNU32O+wp0jX6c1MfluGvRjcHQpK+gHRRJdZAgMBAAGjUzBRMB0GA1Ud --DgQWBBSTmtfFP/zQDiCbte3AG3vchjSekjAfBgNVHSMEGDAWgBSTmtfFP/zQDiCb --te3AG3vchjSekjAPBgNVHRMBAf8EBTADAQH/MA0GCSqGSIb3DQEBCwUAA4IBAQB6 --CnGmyJKVdD41Qs8QbyZ5VvrAkSq7DkL4CAIDy0pClOYjx51R16g4/dXPd8nuDjVB --GbRuO/ZHlKeO+RAVbV4s74KeBmQ4uyuZMvuiJGCDtTgu+fbgp1l41vj21bkDnz/X --0aVrwC4clh6O1/AFCY7cYygp22QJ0zGpIb9d0ly6QDMEocHJKasuQ8/0GPvv6wkY --hwNub2ScsVFHPrFzzufZblBk3Ks0YZw+IsCMTY7QtGb1TZhGeX9xAydqNMnmIxNj --XOE2FKx4ISQNFwf/U1LEAruO7PjbbsWx5FHr4oYV9TCLoERZFofwcGGM9o/cPc4P --CH8fGvFYoU30PG+xX0Pc -+MIIDXTCCAkWgAwIBAgIJALzkvKyFAwWYMA0GCSqGSIb3DQEBCwUAMEUxCzAJBgNV -+BAYTAkFVMRMwEQYDVQQIDApTb21lLVN0YXRlMSEwHwYDVQQKDBhJbnRlcm5ldCBX -+aWRnaXRzIFB0eSBMdGQwHhcNMTgwOTA3MDgyMjUxWhcNMzgwOTAyMDgyMjUxWjBF -+MQswCQYDVQQGEwJBVTETMBEGA1UECAwKU29tZS1TdGF0ZTEhMB8GA1UECgwYSW50 -+ZXJuZXQgV2lkZ2l0cyBQdHkgTHRkMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIB -+CgKCAQEAkvT4Y1ML8Gg4x5aFVygIW022tJsEyfuW4HsEvIarAGpFtUNkw0dOoAxv -+Gv71I0KAWOXxtc9DUjtq7ZXJcG+dBiheTYJ40lrYhHvUt7J37nrUF5v5trQzoE9I -+6WGtzTV8C8RI6F+M6GtwxgwRBgqkuXiK5ExPXAjGMMJZWtiEXHxGTNB6F4zy884m -+VjVtQi8jy+c5g21Awp3z5HrQLM210zwvgi7Rygk6/UM0AnmST4o32SqFSd+0MFUJ -+f3pH3xczfKmhU/TBoVEWRB1YzwixsJrzDOB8wOGnNKCsl45hYUJZZ8epVkyrvFZQ -+iMkwIqqBJbkU6H7fZBl68TJ8ascUCQIDAQABo1AwTjAdBgNVHQ4EFgQUkurgHlw1 -+xMP2wrsrGPTk0ofxCyowHwYDVR0jBBgwFoAUkurgHlw1xMP2wrsrGPTk0ofxCyow -+DAYDVR0TBAUwAwEB/zANBgkqhkiG9w0BAQsFAAOCAQEAcdExVlSvH6aVExNiQO3k -+cMamj+78woDn9x563vwzaGP24KvOXk1B/IJp5kqu3ZsXS0I0Mz6xwXHAXeuxaj06 -+cKgEpHKKgClLblXo2zWqo/3V1UFFpOVP/NhI3r21b+fPrS46rP0mw75haQCph8/8 -+buQr0OeAYbElliY/ji+cJiCJB8A/D13fUMV/NUUfPW/UE6497jOmz+6PtZNAoOFx -+evrmDcbCzbJxacyLJX04rsrt6DO09jb/+5lFm5Aqr6ySKasrmheIGEisl4o9Zbuy -+5PvYgbOEmFgPATIiWGpBO/rqwDdsmgyYFl+YfFoW0akXUVhDb2e5iRDx6Rs0fmN/ -+NA== - -----END CERTIFICATE----- - -----BEGIN RSA PRIVATE KEY----- --MIIEpAIBAAKCAQEA2e36r+Iguu9/DrzgvzjRZoTlZjDXtkqxrZPuyiL/zHezCSSq --SMoMHPfRHy95ulhY70kPAgATvRpyyHqAekc1+V/dOHT0MyHpDJBNuyeA+cM2FgJm --LLvZnf//a42nWgNNcx78baXN9ALE6Dq0Wtkb+y8KLb3KKdmwQl12THYLwwaQrgjp --d3EN+Oi2QZhA+Nau+z+0oJyPfIAj7ssjFQBU2RODZxVJaFREU3DDKOVcL7VxU+NQ --ehTqrcoWRWPPVtUnjSpZvp/M7vv2lBeXPbrMcL0M4+/acwRWEvcTGc28ZQZlPdRm --yxTVN9jvsKdI1+nNTH5bhr0Y3B0KSvoB0USXWQIDAQABAoIBAQCWWYbgHSPzlBOW --eVyc0Hg3QGx7aisIStP2Kt9NeYP87oAISNFqUmq0+Yu+9iQHGbiRrVe7S45SopKa --GVnWApcMKsUWlCl9tWFxF4VpH0HuDm2cFZ+kMR1b0ifHbf0NLsYaLEB+7Sr/s4Fh --rk6LdsnFK5jcIdn9sX/W6WAaND69Ft45sMXuAKPeqwySm7t77nlIEp4lJy/F/rvS --UUV77UVL/5GqC1HOP1kXXEJDxHtNrDPh0fF49Qo/6JPRmLIOia0yKUS5+8lKHh6V --J7nWTucyvviKoYvdJHpi9w2+DTI7mT33Go1Ss4QgHihzcURTtfjsn7yOelS6z8XM --YOxURwoZAoGBAPZpod0Zpjm9trf547reFPLfAJUiKDj1wYXnlpSFAzOf7pSXDrfW --L5dc3c5x2DiBXUlBu03ZWo7wVOKv7d9ByTrvXEqT1/KTqmHoI6Mzy0rn9wU0FmL4 --ASPcpPMnNStOSmRfe9HITDVvgLYo2wQEu1MXf5rGs5HjTRpEGVdWRkRPAoGBAOJo --pQ9uRtaLlCF3KXoRr8HCBKI64IWDki7pbbLTglOBbmptQLfE2GPc/FPBsQwR3KBX --m18A2pc7CRUKzsDavb7SyXTqtP3NF/AguTjGXIv/45QD6A96SStaYR0ZXSA3/uhZ --rAaSSXxaI3mDTc32pXoXJDp7K+CPc5w00I8u1/fXAoGAe1UPoPyPiGL+K0M1yngR --gCZBwmMgQrIutHjfk2Kn4ZTw8wpQYY8grt/aXNP6Zv3I1TvDJgneG6EKu5NWueHR --eGAJj4JEGbPzGaH5BFyOKeXEa6RQeCStXWe4X8OGBzDeZzKrZKqeCjjO8V2tkWtU --3xfp1GwTwLdGBhmDnYUfEl0CgYEA1vzVF6T4gQtDGtADQ5V91je8nKvZvQ4lhoRD --lVZAX7j8tvSNSrMRYypZM9Mtoi9n1524vGqcJpR5WFDN6NUM7iFMCMhCGupgO7Vn --DCFXidzvJgLbna7Zwd/tbWtDQa/KTqmvrwHD49/X5a+n9tapZRiKXznMfUzaU87W --589sZjsCgYBQvtZpNQRqTDNOSuIJGsloMNp1HG2Keoj/nBQ9idkw29rUqhae7ZAa --Csk91ix+AMR8nzQSkpHWOk2+RjdVAQ7m87bnQM6mmAJTcipikDfFxNqOWt7JkM2h --Ydk36LHBIBVLOeqMXrhCHvai3h6efUz7wjGhNxRJ2OGrzWFfLgVsqA== -+MIIEpAIBAAKCAQEAkvT4Y1ML8Gg4x5aFVygIW022tJsEyfuW4HsEvIarAGpFtUNk -+w0dOoAxvGv71I0KAWOXxtc9DUjtq7ZXJcG+dBiheTYJ40lrYhHvUt7J37nrUF5v5 -+trQzoE9I6WGtzTV8C8RI6F+M6GtwxgwRBgqkuXiK5ExPXAjGMMJZWtiEXHxGTNB6 -+F4zy884mVjVtQi8jy+c5g21Awp3z5HrQLM210zwvgi7Rygk6/UM0AnmST4o32SqF -+Sd+0MFUJf3pH3xczfKmhU/TBoVEWRB1YzwixsJrzDOB8wOGnNKCsl45hYUJZZ8ep -+VkyrvFZQiMkwIqqBJbkU6H7fZBl68TJ8ascUCQIDAQABAoIBAC/1mopvs9nFaaJZ -+UTLccb26YwIWBT4VyWuBOk58dJoyFIXPdLb2MoaxCCF7S20yasiYYoW/Gm1fzsmy -+tIbpJgm4au5Iwj2EQF0cPJOmvtUpaMY7tQcXUDHlLhpcMmhiKBV+/Xw4krfXOHqp -+vXSHTLLq0Akpjkyu4F9RTfAD8U5tEbpPsCGcsSJEHxPgqDexITDwB/yuhvrKKUwY -+t8WQBWO5M8D6Z1HGTFovIa86eX4hUKKbNB8sE7yi1wGxbOloIOQESOcqiisP5GGN -+d6r5k9jBwZXlyh7GR+GNILF+n6ctdOFr6MQQEKvDzjh/IVYADen19909Ed7Wn0gR -+C0Ec2gECgYEAwruIv33GWxGFvQmAUtPaHyhkFOiIpTrmZGeaJe7uLvAF86wK9v35 -+wN2fH69JczO1iEWuqDLZ4nEorx6UvjSFBGYTXT1dAnULJ1KHvkk6JjbXq1srTY42 -+U2h33XfJiNgrXlj6v7tMhKD/nBsfyw8v4aHxxUkJl2HomPSv7C7EvhECgYEAwTFp -+sUwDXVeputWwBvDUXHgGaVks28QHXvYH7Q0WsbFjb6lZVH/FxLXvGs7aaZk8WuHQ -+JJcXmEkTs1QDMMWoOlZw9WJv5Zlopq31oYHp8dt2EuO/PQcYmXYEG7JIHJhx8mfL -+f3Y4ix/hnvnITTg7bRNpcxwGEqyg16kalP8PXnkCgYEAgml5cVTYLFEV0b21NMMw -+RsGUFPSN3qoNdZx0fYb/+GtCcSf8x+DbDDDfyiZn+EDfB/4ys+4qQR4rcuv2DVO6 -+6XE68qyPx39/EryQr/z2dnUwBlAuNehRtZY3ABii3YR3tt28P/89hW0VAgSgTCtF -+k8QS2F7Lj5hAX38u+etwUyECgYBWjf7eckHnpgjjLi3JTki2jQfCVzOj2nW689ul -+NwH95o24T1U4aG6ArUpM5nQwb3j89sK8Qf1OOx9abr9nMIcoa+X76nhbk5mxY6rz -+CzN3Km4CFItvmihJSPiaOAva0+npQtuHZb37hvMcuKgnAJSPT+0kp1+JKlJ9jMPe -+EVAfcQKBgQCDxRNXxn4ILyH3kx8lrch7kD5fp/7KifDjAFFJ0DK2e2xsxLUToh2I -+PdMuzCUv4LL8kRsQ/+mUJY4YlOV9OKVAZbI/gPw9NnzBUBugz+wy0OG/nyS5k7G5 -+MIzZm8yx3RieTNhwmw25NDLyGApHGQYsQ7DM/daA/wwdjFsyncxHSg== - -----END RSA PRIVATE KEY----- -diff --git a/test/mitmproxy/net/data/verificationcerts/9da13359.0 b/test/mitmproxy/net/data/verificationcerts/9da13359.0 -index 88ed414..5868a30 100644 ---- a/test/mitmproxy/net/data/verificationcerts/9da13359.0 -+++ b/test/mitmproxy/net/data/verificationcerts/9da13359.0 -@@ -1,21 +1,21 @@ -------BEGIN CERTIFICATE----- --MIIDazCCAlOgAwIBAgIUTE2oFYATbkkytGOt0+CIAOPpaeEwDQYJKoZIhvcNAQEL --BQAwRTELMAkGA1UEBhMCQVUxEzARBgNVBAgMClNvbWUtU3RhdGUxITAfBgNVBAoM --GEludGVybmV0IFdpZGdpdHMgUHR5IEx0ZDAeFw0xOTExMjMwMDA1MDlaFw0zOTEx --MTgwMDA1MDlaMEUxCzAJBgNVBAYTAkFVMRMwEQYDVQQIDApTb21lLVN0YXRlMSEw --HwYDVQQKDBhJbnRlcm5ldCBXaWRnaXRzIFB0eSBMdGQwggEiMA0GCSqGSIb3DQEB --AQUAA4IBDwAwggEKAoIBAQDZ7fqv4iC6738OvOC/ONFmhOVmMNe2SrGtk+7KIv/M --d7MJJKpIygwc99EfL3m6WFjvSQ8CABO9GnLIeoB6RzX5X904dPQzIekMkE27J4D5 --wzYWAmYsu9md//9rjadaA01zHvxtpc30AsToOrRa2Rv7Lwotvcop2bBCXXZMdgvD --BpCuCOl3cQ346LZBmED41q77P7SgnI98gCPuyyMVAFTZE4NnFUloVERTcMMo5Vwv --tXFT41B6FOqtyhZFY89W1SeNKlm+n8zu+/aUF5c9usxwvQzj79pzBFYS9xMZzbxl --BmU91GbLFNU32O+wp0jX6c1MfluGvRjcHQpK+gHRRJdZAgMBAAGjUzBRMB0GA1Ud --DgQWBBSTmtfFP/zQDiCbte3AG3vchjSekjAfBgNVHSMEGDAWgBSTmtfFP/zQDiCb --te3AG3vchjSekjAPBgNVHRMBAf8EBTADAQH/MA0GCSqGSIb3DQEBCwUAA4IBAQB6 --CnGmyJKVdD41Qs8QbyZ5VvrAkSq7DkL4CAIDy0pClOYjx51R16g4/dXPd8nuDjVB --GbRuO/ZHlKeO+RAVbV4s74KeBmQ4uyuZMvuiJGCDtTgu+fbgp1l41vj21bkDnz/X --0aVrwC4clh6O1/AFCY7cYygp22QJ0zGpIb9d0ly6QDMEocHJKasuQ8/0GPvv6wkY --hwNub2ScsVFHPrFzzufZblBk3Ks0YZw+IsCMTY7QtGb1TZhGeX9xAydqNMnmIxNj --XOE2FKx4ISQNFwf/U1LEAruO7PjbbsWx5FHr4oYV9TCLoERZFofwcGGM9o/cPc4P --CH8fGvFYoU30PG+xX0Pc -------END CERTIFICATE----- -+-----BEGIN CERTIFICATE----- -+MIIDXTCCAkWgAwIBAgIJALzkvKyFAwWYMA0GCSqGSIb3DQEBCwUAMEUxCzAJBgNV -+BAYTAkFVMRMwEQYDVQQIDApTb21lLVN0YXRlMSEwHwYDVQQKDBhJbnRlcm5ldCBX -+aWRnaXRzIFB0eSBMdGQwHhcNMTgwOTA3MDgyMjUxWhcNMzgwOTAyMDgyMjUxWjBF -+MQswCQYDVQQGEwJBVTETMBEGA1UECAwKU29tZS1TdGF0ZTEhMB8GA1UECgwYSW50 -+ZXJuZXQgV2lkZ2l0cyBQdHkgTHRkMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIB -+CgKCAQEAkvT4Y1ML8Gg4x5aFVygIW022tJsEyfuW4HsEvIarAGpFtUNkw0dOoAxv -+Gv71I0KAWOXxtc9DUjtq7ZXJcG+dBiheTYJ40lrYhHvUt7J37nrUF5v5trQzoE9I -+6WGtzTV8C8RI6F+M6GtwxgwRBgqkuXiK5ExPXAjGMMJZWtiEXHxGTNB6F4zy884m -+VjVtQi8jy+c5g21Awp3z5HrQLM210zwvgi7Rygk6/UM0AnmST4o32SqFSd+0MFUJ -+f3pH3xczfKmhU/TBoVEWRB1YzwixsJrzDOB8wOGnNKCsl45hYUJZZ8epVkyrvFZQ -+iMkwIqqBJbkU6H7fZBl68TJ8ascUCQIDAQABo1AwTjAdBgNVHQ4EFgQUkurgHlw1 -+xMP2wrsrGPTk0ofxCyowHwYDVR0jBBgwFoAUkurgHlw1xMP2wrsrGPTk0ofxCyow -+DAYDVR0TBAUwAwEB/zANBgkqhkiG9w0BAQsFAAOCAQEAcdExVlSvH6aVExNiQO3k -+cMamj+78woDn9x563vwzaGP24KvOXk1B/IJp5kqu3ZsXS0I0Mz6xwXHAXeuxaj06 -+cKgEpHKKgClLblXo2zWqo/3V1UFFpOVP/NhI3r21b+fPrS46rP0mw75haQCph8/8 -+buQr0OeAYbElliY/ji+cJiCJB8A/D13fUMV/NUUfPW/UE6497jOmz+6PtZNAoOFx -+evrmDcbCzbJxacyLJX04rsrt6DO09jb/+5lFm5Aqr6ySKasrmheIGEisl4o9Zbuy -+5PvYgbOEmFgPATIiWGpBO/rqwDdsmgyYFl+YfFoW0akXUVhDb2e5iRDx6Rs0fmN/ -+NA== -+-----END CERTIFICATE----- -diff --git a/test/mitmproxy/net/data/verificationcerts/generate.py b/test/mitmproxy/net/data/verificationcerts/generate.py -index 1e09138..8439c9e 100644 ---- a/test/mitmproxy/net/data/verificationcerts/generate.py -+++ b/test/mitmproxy/net/data/verificationcerts/generate.py -@@ -5,10 +5,10 @@ import subprocess - import shlex - import os - import shutil --import textwrap -+ - - ROOT_CA = "trusted-root" --SUBJECT = "example.mitmproxy.org" -+SUBJECT = "/CN=example.mitmproxy.org/" - - - def do(args): -@@ -18,39 +18,29 @@ def do(args): - return output - - --def genrsa(cert: str): -- do(f"openssl genrsa -out {cert}.key 2048") -+def genrsa(cert): -+ do("openssl genrsa -out {cert}.key 2048".format(cert=cert)) - - --def sign(cert: str, subject: str): -- with open(f"openssl-{cert}.conf", "w") as f: -- f.write(textwrap.dedent(f""" -- authorityKeyIdentifier=keyid,issuer -- basicConstraints=CA:FALSE -- keyUsage = digitalSignature, keyEncipherment -- subjectAltName = {subject} -- """)) -- do(f"openssl x509 -req -in {cert}.csr " -- f"-CA {ROOT_CA}.crt " -- f"-CAkey {ROOT_CA}.key " -- f"-CAcreateserial " -- f"-days 7300 " -- f"-sha256 " -- f"-extfile \"openssl-{cert}.conf\" " -- f"-out {cert}.crt" -+def sign(cert): -+ do("openssl x509 -req -in {cert}.csr " -+ "-CA {root_ca}.crt " -+ "-CAkey {root_ca}.key " -+ "-CAcreateserial " -+ "-days 7300 " -+ "-out {cert}.crt".format(root_ca=ROOT_CA, cert=cert) - ) -- os.remove(f"openssl-{cert}.conf") - - --def mkcert(cert, subject): -+def mkcert(cert, args): - genrsa(cert) -- do(f"openssl req -new -nodes -batch " -- f"-key {cert}.key " -- f"-addext \"subjectAltName = {subject}\" " -- f"-out {cert}.csr" -+ do("openssl req -new -nodes -batch " -+ "-key {cert}.key " -+ "{args} " -+ "-out {cert}.csr".format(cert=cert, args=args) - ) -- sign(cert, subject) -- os.remove(f"{cert}.csr") -+ sign(cert) -+ os.remove("{cert}.csr".format(cert=cert)) - - - # create trusted root CA -@@ -64,13 +54,13 @@ h = do("openssl x509 -hash -noout -in trusted-root.crt").decode("ascii").strip() - shutil.copyfile("trusted-root.crt", "{}.0".format(h)) - - # create trusted leaf cert. --mkcert("trusted-leaf", f'DNS:{SUBJECT}') -+mkcert("trusted-leaf", "-subj {}".format(SUBJECT)) - - # create self-signed cert - genrsa("self-signed") - do("openssl req -x509 -new -nodes -batch " - "-key self-signed.key " -- f'-addext "subjectAltName = DNS:{SUBJECT}" ' -+ "-subj {} " - "-days 7300 " -- "-out self-signed.crt" -- ) -\ No newline at end of file -+ "-out self-signed.crt".format(SUBJECT) -+ ) -diff --git a/test/mitmproxy/net/data/verificationcerts/self-signed.crt b/test/mitmproxy/net/data/verificationcerts/self-signed.crt -index a02d9f3..6e234b8 100644 ---- a/test/mitmproxy/net/data/verificationcerts/self-signed.crt -+++ b/test/mitmproxy/net/data/verificationcerts/self-signed.crt -@@ -1,22 +1,19 @@ -------BEGIN CERTIFICATE----- --MIIDjTCCAnWgAwIBAgIUb0mIVHKB+bu2PGObggokU3Re7xMwDQYJKoZIhvcNAQEL --BQAwRTELMAkGA1UEBhMCQVUxEzARBgNVBAgMClNvbWUtU3RhdGUxITAfBgNVBAoM --GEludGVybmV0IFdpZGdpdHMgUHR5IEx0ZDAeFw0xOTExMjMwMDA1MDlaFw0zOTEx --MTgwMDA1MDlaMEUxCzAJBgNVBAYTAkFVMRMwEQYDVQQIDApTb21lLVN0YXRlMSEw --HwYDVQQKDBhJbnRlcm5ldCBXaWRnaXRzIFB0eSBMdGQwggEiMA0GCSqGSIb3DQEB --AQUAA4IBDwAwggEKAoIBAQDF6eqyZD2z83buUF4T+6AY0Zoe925a2AHOhGtHJMLo --9AD7FF1Xi1iksEvxbOI6mreHtvYKzUpfNsA3DkFdSO91HMSkdvWcDcExpW62sNK9 --gQQrpcCx7DogOrFiGSIHI1LIy1y6YEJma3G71SgGbw7g1QF64dTX9+BzVQJsloT2 --H3ZxTi8Fb6APJq6d/Tp67GTM8U82vM+FjLKzfH7RMSEPvSyvWSibw3AZP6owFaxz --DJtXpR7evOZbiZxqXmGOBl5OQPu9GdDA3Fyi9Drp7xa234loqd1a8PYyL8qWV/2G --HM3IJOzG8Y5PUJhL8CkDbx4LJ9LfzeSuBnQPUf2ZNalZAgMBAAGjdTBzMB0GA1Ud --DgQWBBQbkS0mAGPD4XjOv9GKzGMyR9ix3TAfBgNVHSMEGDAWgBQbkS0mAGPD4XjO --v9GKzGMyR9ix3TAPBgNVHRMBAf8EBTADAQH/MCAGA1UdEQQZMBeCFWV4YW1wbGUu --bWl0bXByb3h5Lm9yZzANBgkqhkiG9w0BAQsFAAOCAQEAqZ64xpu3qrTlCc555OoP --GgobUFP3qv07d0/r48cOyYdAdlEHhvmDPlqWTB9e4ZYtWZMlocY9DpCywzKTa7F7 --Ad8BwS0a/No3wVdl1UEkIGYxuD//jbd77Mrpf5URvQco85o/bicn+H0GAOchYt1P --jP1VShqsRv6WiTs5kn1/JwVoafddl1jBlMDmCqDv4loAZJYHzie0CqdjjSeorFfE --8FG8OLwmEnmIW6VnanRH8coH9MBbZ+dRtCavS+Q8s0R77dJM1sCp5/4yKcr5D/PD --+dQN9f+iugLxDdBQjiRyadWX9/l4n/h8ezabZ4cNsiWbRXrp5VS0nGNmAK3XvPAu --ow== -------END CERTIFICATE----- -+-----BEGIN CERTIFICATE----- -+MIIDEzCCAfugAwIBAgIJAKzH8k6aKTP6MA0GCSqGSIb3DQEBCwUAMCAxHjAcBgNV -+BAMMFWV4YW1wbGUubWl0bXByb3h5Lm9yZzAeFw0xODA5MDcwODIyNTNaFw0zODA5 -+MDIwODIyNTNaMCAxHjAcBgNVBAMMFWV4YW1wbGUubWl0bXByb3h5Lm9yZzCCASIw -+DQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMOjJMp2o5eLQEmYJqMZzLBi61h9 -+fsCVMvS8hgrH1Cg5q/RaLBLrZ8nILKmFZBapMUEFkUwQLB864tdTMaX7p+jNv3sM -+5LWEIYkTIbu6qV7QerKdubS1hpdFtQGRM1Q+C7H86FzF02DSKzNSmQc4fNed/lQM -+qo/jOm1xx4TZFR4j58BrmmoOfNP44IyrwXsPyXbMsukKixVEB3vQ2oyGDAyG6dYi -+VvM8PVL5yhX3BJ0D1Ky6hgGHJeirm0Cd8qqdSC/SWNdu1bGzg/xyUX5XFaHlIi7Y -+5YhD7ZDLvC76MeCWkfo4DaSB0CWmtG4l1TtHM2JqP8qf2l2LsABKs0q/a+UCAwEA -+AaNQME4wHQYDVR0OBBYEFIc9YAXgnGRhPTEcN/j+k/dxMdKqMB8GA1UdIwQYMBaA -+FIc9YAXgnGRhPTEcN/j+k/dxMdKqMAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcNAQEL -+BQADggEBAD9qKci3Pr4/2WGx+sv8gOpKchC9eF2dXc5hA3xbDw7T6oRLUBAY8Pty -+JF7DHMovT+w7FPRYT8rSc190fbSwVRHAnEaqAzaxteImCp/qYgdBHOz39eG4c93W -+YrYvA1VdUDPcUnisEVWguDsKJGFg+G6pw+8Wkf/hCrJJkriTFogGvzg6ptdQatvE -+dpSkionfbuZKz+7lny6sCBGoMRIFBd22MHJsSQOyTb06Lwc5dpdF9c5vysPRzShJ -+5kkgIjGpTmWp+Ud8BAMQH8EDhJMkJ7iw1+07UQ9MUmXCp9Xgim6x1ri2/yoz9HeO -+83VCkD9YWufrzOrsXpo04rMYtoKo+lw= -+-----END CERTIFICATE----- -diff --git a/test/mitmproxy/net/data/verificationcerts/self-signed.key b/test/mitmproxy/net/data/verificationcerts/self-signed.key -index b6da998..3546c3f 100644 ---- a/test/mitmproxy/net/data/verificationcerts/self-signed.key -+++ b/test/mitmproxy/net/data/verificationcerts/self-signed.key -@@ -1,27 +1,27 @@ -------BEGIN RSA PRIVATE KEY----- --MIIEpAIBAAKCAQEAxenqsmQ9s/N27lBeE/ugGNGaHvduWtgBzoRrRyTC6PQA+xRd --V4tYpLBL8WziOpq3h7b2Cs1KXzbANw5BXUjvdRzEpHb1nA3BMaVutrDSvYEEK6XA --sew6IDqxYhkiByNSyMtcumBCZmtxu9UoBm8O4NUBeuHU1/fgc1UCbJaE9h92cU4v --BW+gDyaunf06euxkzPFPNrzPhYyys3x+0TEhD70sr1kom8NwGT+qMBWscwybV6Ue --3rzmW4mcal5hjgZeTkD7vRnQwNxcovQ66e8Wtt+JaKndWvD2Mi/Kllf9hhzNyCTs --xvGOT1CYS/ApA28eCyfS383krgZ0D1H9mTWpWQIDAQABAoIBAGl4H8+TZeJ5E18q --ywfhJ08ym+x2tYOJ62SP4s+WApy8M62aC6g0pTeWj9IH0YOjobycPwBAqKqW9dYh --Lao1zQ5fF1gB4R+ZoOQBIkAPeS7uCzfrbAYlOlCklpUNibm+FEbXQQI9fAUyqviL --Pno3QvmD6fb/VDsHaMBthA40JIU4C3yUUcSooKKUC3XKmbRDg5/stKbVdi6co5AQ --OMNFj4smts3rXtk5tKMBep4AgKKdClf4IuhpWEqZDxN6e+hZA7g1VOmOIE+hKEEO --ODBkjHNgHIFbsr8GTlZ+GSkHBxMNVdHxntJTU5a2jPiTNTTnhqu2hcVdkyDwLb9x --TGUCdyUCgYEA/ZPwI2KaT4Lj8b+6DCV4v3nZF8nLCtt1LiXxJWexdnr+3q/ZfNjN --5sKVl9rIynz2zoJDVr1MI8yjO6OmV7GBW2wgJsSFwg0vnV/bs75KqPZWZXqy/mrp --HnXd82XUweyyOpmBg4cCjmcnMTrdvCHvfgx7JafnUDqeXNwriWeC5Y8CgYEAx83d --qHgAT2rlnCQYRmCOGb5fuyQ+pSM4p8h1AjueLrTNiWcqX57ct9eVWjymD+AgevFZ --hZH9m6TI4nCb/5ukcvjQb4g1kyY3l7rZiiJ3kf/c22kVsVUagJGeiW6Ypnox4r1o --oFXEwAw1qSGIqOIjYVB2DqvjUl3+UjWCl9SOnpcCgYEA0TSdWUQ/VTwCvW9VmjHM --FgT8I5Ebj+CRI7qv4hFTqxE8dxKTl1nzPd/ptTgOkmhY4vU7gzN3vs1VGp4gXZcX --xwpE2Fcol3lzgB4Wz4s+Y3mgu+ZoCFjB7ZyGugmYZ0nVnV0KKi5X4I6gGhCb4VwK --D29SpjWJNHq4LpqC3MDmkGcCgYAxnKOKXmmtTpy+3ZONfhIqwEOjA0fu10UNHFA5 --grYvYMOcd5pk7dxeZdB2/JI7ZOqLvHv/F5YCXLNozo9ds7bsuW2AFDFBXX72VPYJ --P6+y9/ZOINS7GKeg/wd/lo+e3r6eT2u4TDOzgBSe722wiZ5BXqpB0Fp8rEwm+5R2 --wNe89wKBgQC59SOa4EW3gfjLRaH6hEPcMEzjr0Dt5ilBgntYbI+DsCePwEX28hAK --fd7A6XoCqjnwLpAmh0cmNRsYjtPCzg6TXUdrhfnR/3r7DHzvsxaC06BfkPZJ0+Fu --rP3el/oKSPaDsZKjW6RA5oqlIF82o45MNl2oCG82LgVtu25e3HbJSA== -------END RSA PRIVATE KEY----- -+-----BEGIN RSA PRIVATE KEY----- -+MIIEpgIBAAKCAQEAw6Mkynajl4tASZgmoxnMsGLrWH1+wJUy9LyGCsfUKDmr9Fos -+EutnycgsqYVkFqkxQQWRTBAsHzri11Mxpfun6M2/ewzktYQhiRMhu7qpXtB6sp25 -+tLWGl0W1AZEzVD4LsfzoXMXTYNIrM1KZBzh8153+VAyqj+M6bXHHhNkVHiPnwGua -+ag580/jgjKvBew/Jdsyy6QqLFUQHe9DajIYMDIbp1iJW8zw9UvnKFfcEnQPUrLqG -+AYcl6KubQJ3yqp1IL9JY127VsbOD/HJRflcVoeUiLtjliEPtkMu8Lvox4JaR+jgN -+pIHQJaa0biXVO0czYmo/yp/aXYuwAEqzSr9r5QIDAQABAoIBAQC1RpgymlffdgJt -+rvQuMRu/XQlhh3dJj3YV3BIAL0VguH+i/WLVbRdQm5D2y0kAzml7LGODrYCUt4W1 -+q7rXaCYfy3Xf2QSbRQGl9/pL7xw9ZMQseYW38nPx+39LInYDWzKPDB9qx0uj7Vpm -+ReTSEf9r81PUIaBxj0V2X/VWHag5scBjXoflQLxyV6i1UvTuWyhYvX1Bbaj02MqV -+tGNMrjbj25Wx2za53VDonzNA6RMZzkWGMfzmkkGM6kjGzLEsveys+bYCt7Fs7slR -+4oby0bIUmN7iqLhqlEhS4weWW4iHlq17X7CZeQAE1XeVZBz1N4G8FLjND2eyqb2N -+RAcQqp3BAoGBAOiU3WTu/kSccteDRxR1gVRPqEgfoLDwyAb7ORVUWX4Ii/z/soMw -+xZ2MlYPLnp3Fyu/hKhJPC1LzkD4CGHCTJJ1NnUudtDxl2Zh1FZYGmv1hi1TID/cm -+G0+3XhlJgztS41+AzxTNMulV1yieT2HIRIoRpdSx1UIA72l42YqjrwUNAoGBANdV -+/Ib+3hAfFtSMMI1qZQXvlKEoDRbUOCYuBVkTK8oQJQH6MLDokHZ8sXBAqi9383b1 -+XmhQBJZ//yMy0AqFa2QBlkK0Gizzhh7BLSjIT2LREf66B2cWzhgdhbSp6Nuk+3DK -+NfibxsFAPpW05HqtfxhbjrLfoE8VvTuMGQ8AaXw5AoGBAISo7IL2wrdV2TdN3Mwx -+ndv+N4kz6Q8jt6QrxUqCOy1lKJvdKPAlcIJFvr5W9RkeyXr7nmilB1uAK4UC4vfL -+JfZHX/HSeQx+N5f7KJ3TFLJz4eow1tJsvOVCPP0FbkH3LFO7/+HojSKEYN39NmAa -+v+VU3Zas/GvSZrxtPwASDvE9AoGBAJOBbluW6MzITx5H7dZhRFR9miWOxvCVbOUS -+b01mKX/f8UnadVIp7RONNQr88NdVZqxdRk9USOBDS6Vz4DjkzfySbbjBoJCcPIqC -+r4mZNXAuYRJJolqGr6SrTHTGUyFqcWcAzVnAc7TbakOoxz4V7NLlnOmA8FJcROUu -+gdfZ42hZAoGBANyL2IQ+L92iYVvqsz1zPBlvemevx8zP6GmlzvTcVexyDTIiLg6W -+BVil5zRDPJdDiFfBK18Qg1mJoE4SjLTg+yGww9ef37Zb9kZypy6pM6AbRWILZ1Gv -+7UsWUzk6rgcQpDdpJCUEt+AD3LQJTxxuoIhZePvC2GLkzsjZA7ZyB5+S -+-----END RSA PRIVATE KEY----- -diff --git a/test/mitmproxy/net/data/verificationcerts/trusted-leaf.crt b/test/mitmproxy/net/data/verificationcerts/trusted-leaf.crt -index d71cbd0..3ccae70 100644 ---- a/test/mitmproxy/net/data/verificationcerts/trusted-leaf.crt -+++ b/test/mitmproxy/net/data/verificationcerts/trusted-leaf.crt -@@ -1,21 +1,18 @@ -------BEGIN CERTIFICATE----- --MIIDajCCAlKgAwIBAgIJAKPoP2hXKVv4MA0GCSqGSIb3DQEBCwUAMEUxCzAJBgNV --BAYTAkFVMRMwEQYDVQQIDApTb21lLVN0YXRlMSEwHwYDVQQKDBhJbnRlcm5ldCBX --aWRnaXRzIFB0eSBMdGQwHhcNMTkxMTIzMDAwNTA5WhcNMzkxMTE4MDAwNTA5WjBF --MQswCQYDVQQGEwJBVTETMBEGA1UECAwKU29tZS1TdGF0ZTEhMB8GA1UECgwYSW50 --ZXJuZXQgV2lkZ2l0cyBQdHkgTHRkMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIB --CgKCAQEA4BZUGKkf52583Kps2SinCzRtGFaGIqtP1YfjaN1H21c08hEERRWf0GqW --PIbZz0NRyqOVM0O+4A9cxKrdqDpALBNXdgB62Ven1NNE31ZjsbFgwbORPTiOQo+6 --Xsu+2KHQdqAGOXSqOdNSoZV/HqAKa41iweg6lUf3yrO/hpfnlXTFbuM/oYFARrr9 --YWY7qH5BEvcZBf4Sg3cWGjdbzYg8S6hnFBBkHPDMbS3xra7H8uru8aG9L70aiq0C --3WsGf1/Qctz3cccB6BdnbiXji0QlP0miT+b52hEzDvNjdXvoObgbOZsUUcLrbz8q --LqaQPj6TA2PYpKcxEdJLDcbN++tglQIDAQABo10wWzAfBgNVHSMEGDAWgBSTmtfF --P/zQDiCbte3AG3vchjSekjAJBgNVHRMEAjAAMAsGA1UdDwQEAwIFoDAgBgNVHREE --GTAXghVleGFtcGxlLm1pdG1wcm94eS5vcmcwDQYJKoZIhvcNAQELBQADggEBAMoK --8+KTYF10dbQ8Hl3Fq4Iux7eutAPuKUXUz9dnCfDRhZukdl+gahRoNvkyMBdXOvvx --R9Z2+Guo8NOYKgT1mJtS/c8IxTkjZj9doujJOVD2wTowSfoaT9z+/EJa+6Fp9+xd --YVsO3E/2Vxai8PCNx8JTXr2axcnBDvpHPRXF21hOI8N94SPAcmLTZsdsTELjrGGa --/BA0y+pCEwW6cY9mMHVAAvRoMoqfocBVI7nrYBaQfFoKuwxscxO679eEv+lbSjul --z/VNdWfqrDhFFSzwRSVchapQ9q1EeTzv++wZRwI5bT0Ib6DFTJB3J5+9RihlFYfU --GI17CI0D//DsFic7QJ0= -------END CERTIFICATE----- -+-----BEGIN CERTIFICATE----- -+MIIC4TCCAckCCQCj6D9oVylb9zANBgkqhkiG9w0BAQsFADBFMQswCQYDVQQGEwJB -+VTETMBEGA1UECAwKU29tZS1TdGF0ZTEhMB8GA1UECgwYSW50ZXJuZXQgV2lkZ2l0 -+cyBQdHkgTHRkMB4XDTE4MDkwNzA4MjI1MloXDTM4MDkwMjA4MjI1MlowIDEeMBwG -+A1UEAwwVZXhhbXBsZS5taXRtcHJveHkub3JnMIIBIjANBgkqhkiG9w0BAQEFAAOC -+AQ8AMIIBCgKCAQEAqKzVdsRKgthv6V/dk3Tncy4ymbACs383nGutjulExvroNOCw -+b0y0e7unNGbtXxFQqSvA7eGaT1yRNfoMbXGSS+sn8A3gB6/s2A0Sw7KeSDdoaqEq -+F/LzRBed1YkxSyy0GXuTd7HXNIoFn/eF1tqxgViWdfyFD85qY4yJ+luofdm7IcPM -+ENPzV4nKzDh2PdJpQrEokWz2jM0zefC3IYnFHXY5bA3MnhE03/P0VxeEYkBdmEAt -+O1U2Bkw9SKCLy9zF13ks6/dDZ9LjMtRKI83gQS5z3S3bA45YxFuyeLWgVsJ2NYTa -+j9/8c4xwOjg9TpkCvcmZiPUYGddPHWoKqAAhBwIDAQABMA0GCSqGSIb3DQEBCwUA -+A4IBAQAf8cjxunN4Y7NUD2Z/SNOJ/s0uWJtTPV6m4FxSwwD0wfbsyirPchmattLc -+BabrQkeMMm8gMOrORfanXQwvLZvX0aDf96EgLSfHv8Iqeol5Byrgkn7UORXl20Jt -+8UNRURUZYtWxn08P8dlhxQUncPF/UxCesC8x0cihqv+YTB3TX1sni9mOqPCYY8yH -+E8kCW4zTJ0J9OQUHq9qdYQM/PGVm99+DWBItUeZAva8Rqj1FN3f9j1eWB+EjfYu7 -+ztsTInpNWP4tIh6vIFtuaGr077cJawTe6YVyNxVqquI9+2fpSPkt7tCTIhbQ4AmM -+DeHzn+KjfKN8ooWqmcfmUZWaADe0 -+-----END CERTIFICATE----- -diff --git a/test/mitmproxy/net/data/verificationcerts/trusted-leaf.key b/test/mitmproxy/net/data/verificationcerts/trusted-leaf.key -index e62ee61..7db3186 100644 ---- a/test/mitmproxy/net/data/verificationcerts/trusted-leaf.key -+++ b/test/mitmproxy/net/data/verificationcerts/trusted-leaf.key -@@ -1,27 +1,27 @@ -------BEGIN RSA PRIVATE KEY----- --MIIEowIBAAKCAQEA4BZUGKkf52583Kps2SinCzRtGFaGIqtP1YfjaN1H21c08hEE --RRWf0GqWPIbZz0NRyqOVM0O+4A9cxKrdqDpALBNXdgB62Ven1NNE31ZjsbFgwbOR --PTiOQo+6Xsu+2KHQdqAGOXSqOdNSoZV/HqAKa41iweg6lUf3yrO/hpfnlXTFbuM/ --oYFARrr9YWY7qH5BEvcZBf4Sg3cWGjdbzYg8S6hnFBBkHPDMbS3xra7H8uru8aG9 --L70aiq0C3WsGf1/Qctz3cccB6BdnbiXji0QlP0miT+b52hEzDvNjdXvoObgbOZsU --UcLrbz8qLqaQPj6TA2PYpKcxEdJLDcbN++tglQIDAQABAoIBAG5V8DB4TdI5T9ej --Ppcqch2NQc5DBCbb7SI5l5qRogj5BoPOJykQ/bC0WqcQyvxHrGU3aIZma/yM8+OO --Mjfb/q71ExJyKAsOIwAiyn2hXtMmgHq/vNrFFx7lACIe9ihafHd8UbRGom54g+41 --2vKsYJUWd7L8cqQAXJz9JmfSMeAfRBVVLyOpENSVT8gM/LgPokmZbg0wRJTA5oa3 --10f4Ax1HqXO3cQgKKnrvIU72gB7MQ9Y2G+P2eAmiAlzGr41N5sv7uFz0JrFHhqtC --qc7QrGVVNtxanu0s/LWgfQ+LrKldTysX16j/j6J0tg7Pfuq9cMFPklU98dv2LF5J --jMB8eQECgYEA84+Yy/IGVjUPVb3HjmCmWUoxt4Aqop/12/4+1cBNS6h58iLJpcWc --FVh8m3UbuE7shimEg0fZ1rPPkqepKlDw8/sKXDni/khBASOAh2v1RhvOxNGjxDfn --ZvcK+BG99kU8ZWQHpKmn8OGfYswL2Fvo67L9XFg/wDnk1QpJx7E1GVUCgYEA64gg --G6qnsah1NKSW64BPVKi/Gby83l4oa1amjHqq/Unq6iuGB4/QhK4BzW0aOGI7RMTE --SPh5NF5ZhfT3LWD/EnaMm6QxMTSsL9TDIg36WTVZTkbODFOJczg19EdTs1pa6OXW --0NM6psLu6Nf/QmHJzyYxSZ83uqPOM305xgooKkECgYEAnA0XMySglr9sUd1EbJ7U --NkVpUU8XAhdHKWrey4lofN83MsLDPCk+dha5z8jat94pgVQ8iPiSRBP1HNu7cVdm --6ouf+bNFEvMsYxRiF2I+RmsuscA4E1JWOwxxxLtpYM6/gZ7znrbs2VNWEbD2retF --cy69UltgjUMKsMzktMN/Z/kCgYAbzV285lAVMIVlSWhnNCYpICIur5C7zvGGehv+ --yRwV+fu42JphmiBLCR89WHuX3ECSxYdF9c6Y1+pJXbkvqhtx2nyOgrsry8PngX3n --Ly82CI4aJ1F7MwEukJwN0b2XljrU8wyAae6qcKgy5AxFkbV4tlFrF1hEt8FHYqjH --L7u+AQKBgBnN2vjOVagtlZjJadg5ueR7uKnfk7/TGMtZjNf89C1UZH3TqWnnj6p6 --rLLe4tnA3EebFtGeVWlmnw5k166HRBEd7KuebOCq3hkNlf71aHwhDJ0bIdDwo1g+ --FTpG2vqkqpuEb+2FzDjZT5EdPnj8tUGthREDXpXqsdjFb6+2Enjx -------END RSA PRIVATE KEY----- -+-----BEGIN RSA PRIVATE KEY----- -+MIIEogIBAAKCAQEAqKzVdsRKgthv6V/dk3Tncy4ymbACs383nGutjulExvroNOCw -+b0y0e7unNGbtXxFQqSvA7eGaT1yRNfoMbXGSS+sn8A3gB6/s2A0Sw7KeSDdoaqEq -+F/LzRBed1YkxSyy0GXuTd7HXNIoFn/eF1tqxgViWdfyFD85qY4yJ+luofdm7IcPM -+ENPzV4nKzDh2PdJpQrEokWz2jM0zefC3IYnFHXY5bA3MnhE03/P0VxeEYkBdmEAt -+O1U2Bkw9SKCLy9zF13ks6/dDZ9LjMtRKI83gQS5z3S3bA45YxFuyeLWgVsJ2NYTa -+j9/8c4xwOjg9TpkCvcmZiPUYGddPHWoKqAAhBwIDAQABAoIBABsUC/zSDEgvKOAl -+RLP8a3+hJfxoNjbMsIfK/YTYy/LJqud6PrjPbpYCjRgrgeXmKLXP0VwfAJ/G84Tf -+zIjxV5Qaf0HZaGKzimkwyBdkoGZlhry/fLt1hDolNHBoYuJ3nb4NiaIIiczkb3y7 -+xt+0IhTqvNTaIh5ke83ZbPklJ8p0HAzw1q6+iRFZiZKH1iVRwJyIyK654wpNQ93W -+SqUKa3uAbqw+Bx9fzyEunANFwoBcZka9oSR9bTlhGB8HPHZFVYKgZvE4n9WOclKW -+E75pGG6vFYZkxBdqcjFNlPKKZRisDuey28teiHXThh1MvYxRdaMq4oxOE1J+n17k -+F2gQolECgYEA01j1FlExu45+U36n3tCCyS50dTtf0Qpi71c1s5DZyT+AAB2ZSGXm -+VBtKgVRNg/iWfHn5b/zHF30OtgIzcsrU66cWMwIXPUQigXh8Cteve7VMs03hce1w -+wsFwLoyvdWEam32YAymqRgN3H6JQim82IJJ3YlWrgEytBnvLkADCihkCgYEAzE/g -+8aoxDZJUwbvaZjLwuydmvc+aAwanVgqvtkca4x99oPhNQna6O0jXXAtJXMA3SHp5 -+QYMDKh98BqCXyfXd+1Semc9pgAPz7l4j09WG7Zdap3xinTOkUsmTAz/2T967HIsP -+6qP7RUiwjmbUk8ZGcKsNjoxzPA4JURYimKB5qB8CgYBTjlnnJtaYpi8/Z1WK+7iZ -+PSqBpqWtCYQvx7TNdzkDHX3Hjewp+U9kdR2xn9i9kiw8riR1p+Q2XxTP1HLusU4Y -+lIhsRilV6XgS48V2q+sO55CZWvMEjbEE7mEhpjFAINHaI39T0Mcmwvv3n75j3K/z -+lLRqRiB1qtrFM3A5UHOZEQKBgBPQm2xUqTU7v+SaJ3BJ+HbuN1SpUbKBbrE1kB0J -+gF4Oq8x0yGltwloFkn1mytKoAbSRzDjCUAhBzXGHGbGImuLJLiiUqRK1T28Kyka9 -+KrzYNP6RXa8JVyKAUjW6elT8sQDvq7eB99icWCM3bd53GFXNAR+WF4b3hYfLscdD -+qQjZAoGAB3ah068Qscb2Ef3+eufa+EvOfMDrNNvlEoZXRlhviTg3NEwjyqbxaCIy -+6Xg+rWvgJm9UE2RBOcCNeghkEabmL1+8DvmDiV1lt9oJtqULBirvalp2H3+9yiTk -+j5dnVcRF6cYvNFmwTr2WZFBGgq96d/Zmbx3o3MIqSBc8I6pLNzo= -+-----END RSA PRIVATE KEY----- -diff --git a/test/mitmproxy/net/data/verificationcerts/trusted-root.crt b/test/mitmproxy/net/data/verificationcerts/trusted-root.crt -index 88ed414..5868a30 100644 ---- a/test/mitmproxy/net/data/verificationcerts/trusted-root.crt -+++ b/test/mitmproxy/net/data/verificationcerts/trusted-root.crt -@@ -1,21 +1,21 @@ -------BEGIN CERTIFICATE----- --MIIDazCCAlOgAwIBAgIUTE2oFYATbkkytGOt0+CIAOPpaeEwDQYJKoZIhvcNAQEL --BQAwRTELMAkGA1UEBhMCQVUxEzARBgNVBAgMClNvbWUtU3RhdGUxITAfBgNVBAoM --GEludGVybmV0IFdpZGdpdHMgUHR5IEx0ZDAeFw0xOTExMjMwMDA1MDlaFw0zOTEx --MTgwMDA1MDlaMEUxCzAJBgNVBAYTAkFVMRMwEQYDVQQIDApTb21lLVN0YXRlMSEw --HwYDVQQKDBhJbnRlcm5ldCBXaWRnaXRzIFB0eSBMdGQwggEiMA0GCSqGSIb3DQEB --AQUAA4IBDwAwggEKAoIBAQDZ7fqv4iC6738OvOC/ONFmhOVmMNe2SrGtk+7KIv/M --d7MJJKpIygwc99EfL3m6WFjvSQ8CABO9GnLIeoB6RzX5X904dPQzIekMkE27J4D5 --wzYWAmYsu9md//9rjadaA01zHvxtpc30AsToOrRa2Rv7Lwotvcop2bBCXXZMdgvD --BpCuCOl3cQ346LZBmED41q77P7SgnI98gCPuyyMVAFTZE4NnFUloVERTcMMo5Vwv --tXFT41B6FOqtyhZFY89W1SeNKlm+n8zu+/aUF5c9usxwvQzj79pzBFYS9xMZzbxl --BmU91GbLFNU32O+wp0jX6c1MfluGvRjcHQpK+gHRRJdZAgMBAAGjUzBRMB0GA1Ud --DgQWBBSTmtfFP/zQDiCbte3AG3vchjSekjAfBgNVHSMEGDAWgBSTmtfFP/zQDiCb --te3AG3vchjSekjAPBgNVHRMBAf8EBTADAQH/MA0GCSqGSIb3DQEBCwUAA4IBAQB6 --CnGmyJKVdD41Qs8QbyZ5VvrAkSq7DkL4CAIDy0pClOYjx51R16g4/dXPd8nuDjVB --GbRuO/ZHlKeO+RAVbV4s74KeBmQ4uyuZMvuiJGCDtTgu+fbgp1l41vj21bkDnz/X --0aVrwC4clh6O1/AFCY7cYygp22QJ0zGpIb9d0ly6QDMEocHJKasuQ8/0GPvv6wkY --hwNub2ScsVFHPrFzzufZblBk3Ks0YZw+IsCMTY7QtGb1TZhGeX9xAydqNMnmIxNj --XOE2FKx4ISQNFwf/U1LEAruO7PjbbsWx5FHr4oYV9TCLoERZFofwcGGM9o/cPc4P --CH8fGvFYoU30PG+xX0Pc -------END CERTIFICATE----- -+-----BEGIN CERTIFICATE----- -+MIIDXTCCAkWgAwIBAgIJALzkvKyFAwWYMA0GCSqGSIb3DQEBCwUAMEUxCzAJBgNV -+BAYTAkFVMRMwEQYDVQQIDApTb21lLVN0YXRlMSEwHwYDVQQKDBhJbnRlcm5ldCBX -+aWRnaXRzIFB0eSBMdGQwHhcNMTgwOTA3MDgyMjUxWhcNMzgwOTAyMDgyMjUxWjBF -+MQswCQYDVQQGEwJBVTETMBEGA1UECAwKU29tZS1TdGF0ZTEhMB8GA1UECgwYSW50 -+ZXJuZXQgV2lkZ2l0cyBQdHkgTHRkMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIB -+CgKCAQEAkvT4Y1ML8Gg4x5aFVygIW022tJsEyfuW4HsEvIarAGpFtUNkw0dOoAxv -+Gv71I0KAWOXxtc9DUjtq7ZXJcG+dBiheTYJ40lrYhHvUt7J37nrUF5v5trQzoE9I -+6WGtzTV8C8RI6F+M6GtwxgwRBgqkuXiK5ExPXAjGMMJZWtiEXHxGTNB6F4zy884m -+VjVtQi8jy+c5g21Awp3z5HrQLM210zwvgi7Rygk6/UM0AnmST4o32SqFSd+0MFUJ -+f3pH3xczfKmhU/TBoVEWRB1YzwixsJrzDOB8wOGnNKCsl45hYUJZZ8epVkyrvFZQ -+iMkwIqqBJbkU6H7fZBl68TJ8ascUCQIDAQABo1AwTjAdBgNVHQ4EFgQUkurgHlw1 -+xMP2wrsrGPTk0ofxCyowHwYDVR0jBBgwFoAUkurgHlw1xMP2wrsrGPTk0ofxCyow -+DAYDVR0TBAUwAwEB/zANBgkqhkiG9w0BAQsFAAOCAQEAcdExVlSvH6aVExNiQO3k -+cMamj+78woDn9x563vwzaGP24KvOXk1B/IJp5kqu3ZsXS0I0Mz6xwXHAXeuxaj06 -+cKgEpHKKgClLblXo2zWqo/3V1UFFpOVP/NhI3r21b+fPrS46rP0mw75haQCph8/8 -+buQr0OeAYbElliY/ji+cJiCJB8A/D13fUMV/NUUfPW/UE6497jOmz+6PtZNAoOFx -+evrmDcbCzbJxacyLJX04rsrt6DO09jb/+5lFm5Aqr6ySKasrmheIGEisl4o9Zbuy -+5PvYgbOEmFgPATIiWGpBO/rqwDdsmgyYFl+YfFoW0akXUVhDb2e5iRDx6Rs0fmN/ -+NA== -+-----END CERTIFICATE----- -diff --git a/test/mitmproxy/net/data/verificationcerts/trusted-root.key b/test/mitmproxy/net/data/verificationcerts/trusted-root.key -index 11a4d5a..c690751 100644 ---- a/test/mitmproxy/net/data/verificationcerts/trusted-root.key -+++ b/test/mitmproxy/net/data/verificationcerts/trusted-root.key -@@ -1,27 +1,27 @@ -------BEGIN RSA PRIVATE KEY----- --MIIEpAIBAAKCAQEA2e36r+Iguu9/DrzgvzjRZoTlZjDXtkqxrZPuyiL/zHezCSSq --SMoMHPfRHy95ulhY70kPAgATvRpyyHqAekc1+V/dOHT0MyHpDJBNuyeA+cM2FgJm --LLvZnf//a42nWgNNcx78baXN9ALE6Dq0Wtkb+y8KLb3KKdmwQl12THYLwwaQrgjp --d3EN+Oi2QZhA+Nau+z+0oJyPfIAj7ssjFQBU2RODZxVJaFREU3DDKOVcL7VxU+NQ --ehTqrcoWRWPPVtUnjSpZvp/M7vv2lBeXPbrMcL0M4+/acwRWEvcTGc28ZQZlPdRm --yxTVN9jvsKdI1+nNTH5bhr0Y3B0KSvoB0USXWQIDAQABAoIBAQCWWYbgHSPzlBOW --eVyc0Hg3QGx7aisIStP2Kt9NeYP87oAISNFqUmq0+Yu+9iQHGbiRrVe7S45SopKa --GVnWApcMKsUWlCl9tWFxF4VpH0HuDm2cFZ+kMR1b0ifHbf0NLsYaLEB+7Sr/s4Fh --rk6LdsnFK5jcIdn9sX/W6WAaND69Ft45sMXuAKPeqwySm7t77nlIEp4lJy/F/rvS --UUV77UVL/5GqC1HOP1kXXEJDxHtNrDPh0fF49Qo/6JPRmLIOia0yKUS5+8lKHh6V --J7nWTucyvviKoYvdJHpi9w2+DTI7mT33Go1Ss4QgHihzcURTtfjsn7yOelS6z8XM --YOxURwoZAoGBAPZpod0Zpjm9trf547reFPLfAJUiKDj1wYXnlpSFAzOf7pSXDrfW --L5dc3c5x2DiBXUlBu03ZWo7wVOKv7d9ByTrvXEqT1/KTqmHoI6Mzy0rn9wU0FmL4 --ASPcpPMnNStOSmRfe9HITDVvgLYo2wQEu1MXf5rGs5HjTRpEGVdWRkRPAoGBAOJo --pQ9uRtaLlCF3KXoRr8HCBKI64IWDki7pbbLTglOBbmptQLfE2GPc/FPBsQwR3KBX --m18A2pc7CRUKzsDavb7SyXTqtP3NF/AguTjGXIv/45QD6A96SStaYR0ZXSA3/uhZ --rAaSSXxaI3mDTc32pXoXJDp7K+CPc5w00I8u1/fXAoGAe1UPoPyPiGL+K0M1yngR --gCZBwmMgQrIutHjfk2Kn4ZTw8wpQYY8grt/aXNP6Zv3I1TvDJgneG6EKu5NWueHR --eGAJj4JEGbPzGaH5BFyOKeXEa6RQeCStXWe4X8OGBzDeZzKrZKqeCjjO8V2tkWtU --3xfp1GwTwLdGBhmDnYUfEl0CgYEA1vzVF6T4gQtDGtADQ5V91je8nKvZvQ4lhoRD --lVZAX7j8tvSNSrMRYypZM9Mtoi9n1524vGqcJpR5WFDN6NUM7iFMCMhCGupgO7Vn --DCFXidzvJgLbna7Zwd/tbWtDQa/KTqmvrwHD49/X5a+n9tapZRiKXznMfUzaU87W --589sZjsCgYBQvtZpNQRqTDNOSuIJGsloMNp1HG2Keoj/nBQ9idkw29rUqhae7ZAa --Csk91ix+AMR8nzQSkpHWOk2+RjdVAQ7m87bnQM6mmAJTcipikDfFxNqOWt7JkM2h --Ydk36LHBIBVLOeqMXrhCHvai3h6efUz7wjGhNxRJ2OGrzWFfLgVsqA== -------END RSA PRIVATE KEY----- -+-----BEGIN RSA PRIVATE KEY----- -+MIIEpAIBAAKCAQEAkvT4Y1ML8Gg4x5aFVygIW022tJsEyfuW4HsEvIarAGpFtUNk -+w0dOoAxvGv71I0KAWOXxtc9DUjtq7ZXJcG+dBiheTYJ40lrYhHvUt7J37nrUF5v5 -+trQzoE9I6WGtzTV8C8RI6F+M6GtwxgwRBgqkuXiK5ExPXAjGMMJZWtiEXHxGTNB6 -+F4zy884mVjVtQi8jy+c5g21Awp3z5HrQLM210zwvgi7Rygk6/UM0AnmST4o32SqF -+Sd+0MFUJf3pH3xczfKmhU/TBoVEWRB1YzwixsJrzDOB8wOGnNKCsl45hYUJZZ8ep -+VkyrvFZQiMkwIqqBJbkU6H7fZBl68TJ8ascUCQIDAQABAoIBAC/1mopvs9nFaaJZ -+UTLccb26YwIWBT4VyWuBOk58dJoyFIXPdLb2MoaxCCF7S20yasiYYoW/Gm1fzsmy -+tIbpJgm4au5Iwj2EQF0cPJOmvtUpaMY7tQcXUDHlLhpcMmhiKBV+/Xw4krfXOHqp -+vXSHTLLq0Akpjkyu4F9RTfAD8U5tEbpPsCGcsSJEHxPgqDexITDwB/yuhvrKKUwY -+t8WQBWO5M8D6Z1HGTFovIa86eX4hUKKbNB8sE7yi1wGxbOloIOQESOcqiisP5GGN -+d6r5k9jBwZXlyh7GR+GNILF+n6ctdOFr6MQQEKvDzjh/IVYADen19909Ed7Wn0gR -+C0Ec2gECgYEAwruIv33GWxGFvQmAUtPaHyhkFOiIpTrmZGeaJe7uLvAF86wK9v35 -+wN2fH69JczO1iEWuqDLZ4nEorx6UvjSFBGYTXT1dAnULJ1KHvkk6JjbXq1srTY42 -+U2h33XfJiNgrXlj6v7tMhKD/nBsfyw8v4aHxxUkJl2HomPSv7C7EvhECgYEAwTFp -+sUwDXVeputWwBvDUXHgGaVks28QHXvYH7Q0WsbFjb6lZVH/FxLXvGs7aaZk8WuHQ -+JJcXmEkTs1QDMMWoOlZw9WJv5Zlopq31oYHp8dt2EuO/PQcYmXYEG7JIHJhx8mfL -+f3Y4ix/hnvnITTg7bRNpcxwGEqyg16kalP8PXnkCgYEAgml5cVTYLFEV0b21NMMw -+RsGUFPSN3qoNdZx0fYb/+GtCcSf8x+DbDDDfyiZn+EDfB/4ys+4qQR4rcuv2DVO6 -+6XE68qyPx39/EryQr/z2dnUwBlAuNehRtZY3ABii3YR3tt28P/89hW0VAgSgTCtF -+k8QS2F7Lj5hAX38u+etwUyECgYBWjf7eckHnpgjjLi3JTki2jQfCVzOj2nW689ul -+NwH95o24T1U4aG6ArUpM5nQwb3j89sK8Qf1OOx9abr9nMIcoa+X76nhbk5mxY6rz -+CzN3Km4CFItvmihJSPiaOAva0+npQtuHZb37hvMcuKgnAJSPT+0kp1+JKlJ9jMPe -+EVAfcQKBgQCDxRNXxn4ILyH3kx8lrch7kD5fp/7KifDjAFFJ0DK2e2xsxLUToh2I -+PdMuzCUv4LL8kRsQ/+mUJY4YlOV9OKVAZbI/gPw9NnzBUBugz+wy0OG/nyS5k7G5 -+MIzZm8yx3RieTNhwmw25NDLyGApHGQYsQ7DM/daA/wwdjFsyncxHSg== -+-----END RSA PRIVATE KEY----- -diff --git a/test/mitmproxy/net/data/verificationcerts/trusted-root.srl b/test/mitmproxy/net/data/verificationcerts/trusted-root.srl -index ee465f9..cf60e2a 100644 ---- a/test/mitmproxy/net/data/verificationcerts/trusted-root.srl -+++ b/test/mitmproxy/net/data/verificationcerts/trusted-root.srl -@@ -1 +1 @@ --A3E83F6857295BF8 -+A3E83F6857295BF7 -diff --git a/test/mitmproxy/net/test_tcp.py b/test/mitmproxy/net/test_tcp.py -index ba9b7eb..f204b8b 100644 ---- a/test/mitmproxy/net/test_tcp.py -+++ b/test/mitmproxy/net/test_tcp.py -@@ -219,7 +219,7 @@ class TestInvalidTrustFile(tservers.ServerTestBase): - c.convert_to_tls( - sni="example.mitmproxy.org", - verify=SSL.VERIFY_PEER, -- ca_pemfile=cdata.path("data/verificationcerts/generate.py") -+ ca_pemfile=tdata.path("mitmproxy/net/data/verificationcerts/generate.py") - ) - - -@@ -265,7 +265,7 @@ class TestSSLUpstreamCertVerificationWBadServerCert(tservers.ServerTestBase): - c.convert_to_tls( - sni="example.mitmproxy.org", - verify=SSL.VERIFY_PEER, -- ca_pemfile=cdata.path("data/verificationcerts/trusted-root.crt") -+ ca_pemfile=tdata.path("mitmproxy/net/data/verificationcerts/trusted-root.crt") - ) - - assert c.ssl_verification_error -@@ -289,7 +289,7 @@ class TestSSLUpstreamCertVerificationWBadHostname(tservers.ServerTestBase): - with pytest.raises(exceptions.TlsException): - c.convert_to_tls( - verify=SSL.VERIFY_PEER, -- ca_pemfile=cdata.path("data/verificationcerts/trusted-root.crt") -+ ca_pemfile=tdata.path("mitmproxy/net/data/verificationcerts/trusted-root.crt") - ) - - def test_mode_none_should_pass_without_sni(self, tdata): -@@ -297,10 +297,10 @@ class TestSSLUpstreamCertVerificationWBadHostname(tservers.ServerTestBase): - with c.connect(): - c.convert_to_tls( - verify=SSL.VERIFY_NONE, -- ca_path=cdata.path("data/verificationcerts/") -+ ca_path=tdata.path("mitmproxy/net/data/verificationcerts/") - ) - -- assert "Cannot validate hostname, SNI missing." in str(c.ssl_verification_error) -+ assert "'no-hostname' doesn't match" in str(c.ssl_verification_error) - - def test_should_fail(self, tdata): - c = tcp.TCPClient(("127.0.0.1", self.port)) -@@ -309,7 +309,7 @@ class TestSSLUpstreamCertVerificationWBadHostname(tservers.ServerTestBase): - c.convert_to_tls( - sni="mitmproxy.org", - verify=SSL.VERIFY_PEER, -- ca_pemfile=cdata.path("data/verificationcerts/trusted-root.crt") -+ ca_pemfile=tdata.path("mitmproxy/net/data/verificationcerts/trusted-root.crt") - ) - assert c.ssl_verification_error - -@@ -328,7 +328,7 @@ class TestSSLUpstreamCertVerificationWValidCertChain(tservers.ServerTestBase): - c.convert_to_tls( - sni="example.mitmproxy.org", - verify=SSL.VERIFY_PEER, -- ca_pemfile=cdata.path("data/verificationcerts/trusted-root.crt") -+ ca_pemfile=tdata.path("mitmproxy/net/data/verificationcerts/trusted-root.crt") - ) - - assert c.ssl_verification_error is None -@@ -344,7 +344,7 @@ class TestSSLUpstreamCertVerificationWValidCertChain(tservers.ServerTestBase): - c.convert_to_tls( - sni="example.mitmproxy.org", - verify=SSL.VERIFY_PEER, -- ca_path=cdata.path("data/verificationcerts/") -+ ca_path=tdata.path("mitmproxy/net/data/verificationcerts/") - ) - - assert c.ssl_verification_error is None -@@ -376,14 +376,14 @@ class TestSSLClientCert(tservers.ServerTestBase): - c = tcp.TCPClient(("127.0.0.1", self.port)) - with c.connect(): - c.convert_to_tls( -- cert=cdata.path("data/clientcert/client.pem")) -+ cert=tdata.path("mitmproxy/net/data/clientcert/client.pem")) - assert c.rfile.readline().strip() == b"1" - - def test_clientcert_err(self, tdata): - c = tcp.TCPClient(("127.0.0.1", self.port)) - with c.connect(): - with pytest.raises(exceptions.TlsException): -- c.convert_to_tls(cert=cdata.path("data/clientcert/make")) -+ c.convert_to_tls(cert=tdata.path("mitmproxy/net/data/clientcert/make")) - - - class TestSNI(tservers.ServerTestBase): diff -Nru mitmproxy-5.1.1/debian/patches/0007-revert-use-OpenSSLs-keylog-callback.patch mitmproxy-6.0.2/debian/patches/0007-revert-use-OpenSSLs-keylog-callback.patch --- mitmproxy-5.1.1/debian/patches/0007-revert-use-OpenSSLs-keylog-callback.patch 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/debian/patches/0007-revert-use-OpenSSLs-keylog-callback.patch 2021-01-06 14:55:14.000000000 +0000 @@ -0,0 +1,80 @@ +Revert use OpenSSL's keylog callback for SSLKEYLOGFILE, refs #3994 + +Original From: Maximilian Hils +Original Date: Thu, 19 Nov 2020 14:39:04 +0100 + +--- + mitmproxy/net/tls.py | 43 ++++++++++------------------------ + test/mitmproxy/net/test_tls.py | 2 +- + 2 files changed, 13 insertions(+), 32 deletions(-) + +--- a/mitmproxy/net/tls.py ++++ b/mitmproxy/net/tls.py +@@ -86,17 +86,36 @@ class MasterSecretLogger: + # required for functools.wraps, which pyOpenSSL uses. + __name__ = "MasterSecretLogger" + +- def __call__(self, connection, keymaterial): +- with self.lock: +- if not self.f: +- d = os.path.dirname(self.filename) +- if not os.path.isdir(d): +- os.makedirs(d) +- self.f = open(self.filename, "ab") +- self.f.write(b"\n") +- self.f.write(keymaterial) +- self.f.write(b"\n") +- self.f.flush() ++ def __call__(self, connection, where, ret): ++ done_now = ( ++ where == SSL.SSL_CB_HANDSHAKE_DONE and ret == 1 ++ ) ++ # this is a horrendous workaround for https://github.com/mitmproxy/mitmproxy/pull/3692#issuecomment-608454530: ++ # OpenSSL 1.1.1f decided to not make connection.master_key() fail in the SSL_CB_HANDSHAKE_DONE callback. ++ # To support various OpenSSL versions and still log master secrets, we now mark connections where this has ++ # happened and then try again on the next event. This is ugly and shouldn't be done, but eventually we ++ # replace this with context.set_keylog_callback anyways. ++ done_previously_but_not_logged_yet = ( ++ hasattr(connection, "_still_needs_masterkey") ++ ) ++ if done_now or done_previously_but_not_logged_yet: ++ with self.lock: ++ if not self.f: ++ d = os.path.dirname(self.filename) ++ if not os.path.isdir(d): ++ os.makedirs(d) ++ self.f = open(self.filename, "ab") ++ self.f.write(b"\r\n") ++ try: ++ client_random = binascii.hexlify(connection.client_random()) ++ masterkey = binascii.hexlify(connection.master_key()) ++ except (AssertionError, SSL.Error): # careful: exception type changes between pyOpenSSL versions ++ connection._still_needs_masterkey = True ++ else: ++ self.f.write(b"CLIENT_RANDOM %s %s\r\n" % (client_random, masterkey)) ++ self.f.flush() ++ if hasattr(connection, "_still_needs_masterkey"): ++ delattr(connection, "_still_needs_masterkey") + + def close(self): + with self.lock: +@@ -184,7 +203,7 @@ def _create_ssl_context( + + # SSLKEYLOGFILE + if log_master_secret: +- context.set_keylog_callback(log_master_secret) ++ context.set_info_callback(log_master_secret) + + if alpn_protos is not None: + # advertise application layer protocols +--- a/test/mitmproxy/net/test_tls.py ++++ b/test/mitmproxy/net/test_tls.py +@@ -43,7 +43,7 @@ class TestMasterSecretLogger(tservers.Se + + tls.log_master_secret.close() + with open(logfile, "rb") as f: +- assert f.read().count(b"SERVER_HANDSHAKE_TRAFFIC_SECRET") >= 2 ++ assert f.read().count(b"CLIENT_RANDOM") >= 2 + + tls.log_master_secret = _logfun + diff -Nru mitmproxy-5.1.1/debian/patches/series mitmproxy-6.0.2/debian/patches/series --- mitmproxy-5.1.1/debian/patches/series 2020-06-29 09:21:51.000000000 +0000 +++ mitmproxy-6.0.2/debian/patches/series 2021-01-06 14:55:14.000000000 +0000 @@ -1,7 +1,7 @@ 0001-Do-no-run-full-coverage-tests.patch 0002-Remove-upper-bound-versions-dependencies.patch -0003-Remove-test_xss_scanner.py.patch 0004-Remove-test_cibuild.py.patch 0005-Remove-test_readfile.py.patch 0006-Patch-out-zstd.patch -0007-Revert-use-OpenSSL-s-hostname-validation.patch +0006-Delete-asciinema-for-which-we-only-have-minified-ver.patch +0007-revert-use-OpenSSLs-keylog-callback.patch diff -Nru mitmproxy-5.1.1/debian/rules mitmproxy-6.0.2/debian/rules --- mitmproxy-5.1.1/debian/rules 2020-06-29 09:21:51.000000000 +0000 +++ mitmproxy-6.0.2/debian/rules 2021-01-06 14:55:14.000000000 +0000 @@ -1,13 +1,16 @@ #!/usr/bin/make -f export PYBUILD_NAME=mitmproxy -export PYBUILD_TEST_ARGS="-k-TestHARDump -k-TestScripts -k-test_mitmweb -k-test_mitmdump -k-TestDaemonSSL -k-test_tcp -k-TestHTTPS -k-TestMasterSecretLogger" +export PYBUILD_TEST_ARGS=-k"not TestHARDump" -k"not TestScripts" -k"not test_mitmweb" -k"not test_mitmdump" -k"not TestDaemonSSL" -k"not test_tcp" -k"not TestHTTPS" -k"not TestMasterSecretLogger" -k"not test_get_version" export LC_ALL=C.UTF-8 %: dh $@ --with python3 --buildsystem=pybuild +override_dh_auto_test: + -dh_auto_test + override_dh_python3: dh_python3 rm -fr debian/mitmproxy/usr/lib/python3/dist-packages/mitmproxy/addons/onboardingapp/static/fontawesome/ diff -Nru mitmproxy-5.1.1/dev.ps1 mitmproxy-6.0.2/dev.ps1 --- mitmproxy-5.1.1/dev.ps1 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/dev.ps1 2020-12-15 16:41:27.000000000 +0000 @@ -1,10 +1,5 @@ $ErrorActionPreference = "Stop" -$pyver = python --version -if($pyver -notmatch "3\.[6-9]") { - Write-Warning "Unexpected Python version, expected Python 3.6 or above: $pyver" -} - python -m venv .\venv --copies & .\venv\Scripts\activate.ps1 diff -Nru mitmproxy-5.1.1/docs/build.sh mitmproxy-6.0.2/docs/build.sh --- mitmproxy-5.1.1/docs/build.sh 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/build.sh 2020-12-15 16:41:27.000000000 +0000 @@ -8,16 +8,12 @@ SCRIPTPATH="$( cd "$(dirname "$0")" >/dev/null 2>&1 ; pwd -P )" pushd ${SCRIPTPATH} -for script in scripts/* ; do +for script in scripts/*.py ; do output="${script##*/}" output="src/generated/${output%.*}.html" echo "Generating output for ${script} into ${output} ..." "${script}" > "${output}" done -output="src/content/addons-examples.md" -echo "Generating examples content page into ${output} ..." -./render_examples.py > "${output}" - cd src hugo diff -Nru mitmproxy-5.1.1/docs/ci.sh mitmproxy-6.0.2/docs/ci.sh --- mitmproxy-5.1.1/docs/ci.sh 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/ci.sh 2020-12-15 16:41:27.000000000 +0000 @@ -2,7 +2,6 @@ set -o errexit set -o pipefail -set -o nounset # set -o xtrace # This script gets run from CI to render and upload docs for the master branch. diff -Nru mitmproxy-5.1.1/docs/.gitignore mitmproxy-6.0.2/docs/.gitignore --- mitmproxy-5.1.1/docs/.gitignore 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/.gitignore 2020-12-15 16:41:27.000000000 +0000 @@ -3,4 +3,3 @@ node_modules/ public/ src/resources/_gen/ -src/content/addons-examples.md diff -Nru mitmproxy-5.1.1/docs/modd.conf mitmproxy-6.0.2/docs/modd.conf --- mitmproxy-5.1.1/docs/modd.conf 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/modd.conf 2020-12-15 16:41:27.000000000 +0000 @@ -1,3 +1,7 @@ +scripts/*.py { + prep: ./build.sh +} + { daemon: cd src; hugo server -D } diff -Nru mitmproxy-5.1.1/docs/render_examples.py mitmproxy-6.0.2/docs/render_examples.py --- mitmproxy-5.1.1/docs/render_examples.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/render_examples.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,50 +0,0 @@ -#!/usr/bin/env python3 - -import os -import textwrap -from pathlib import Path - -print(""" ---- -title: "Examples" -menu: - addons: - weight: 6 ---- - -# Examples of Addons and Scripts - -The most recent set of examples is also available [on our GitHub project](https://github.com/mitmproxy/mitmproxy/tree/master/examples). - -""") - -base = os.path.dirname(os.path.realpath(__file__)) -examples_path = os.path.join(base, 'src/examples/') -pathlist = Path(examples_path).glob('**/*.py') - -examples = [os.path.relpath(str(p), examples_path) for p in sorted(pathlist)] -examples = [p for p in examples if not os.path.basename(p) == '__init__.py'] -examples = [p for p in examples if not os.path.basename(p).startswith('test_')] - -current_dir = None -current_level = 2 -for ex in examples: - if os.path.dirname(ex) != current_dir: - current_dir = os.path.dirname(ex) - sanitized = current_dir.replace('/', '').replace('.', '') - print(" * [Examples: {}]({{{{< relref \"addons-examples#{}\">}}}})".format(current_dir, sanitized)) - - sanitized = ex.replace('/', '').replace('.', '') - print(" * [{}]({{{{< relref \"addons-examples#example-{}\">}}}})".format(os.path.basename(ex), sanitized)) - -current_dir = None -current_level = 2 -for ex in examples: - if os.path.dirname(ex) != current_dir: - current_dir = os.path.dirname(ex) - print("#" * current_level, current_dir) - - print(textwrap.dedent(""" - {} Example: {} - {{{{< example src="{}" lang="py" >}}}} - """.format("#" * (current_level + 1), ex, "examples/" + ex))) diff -Nru mitmproxy-5.1.1/docs/scripts/clirecording/clidirector.py mitmproxy-6.0.2/docs/scripts/clirecording/clidirector.py --- mitmproxy-5.1.1/docs/scripts/clirecording/clidirector.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/scripts/clirecording/clidirector.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,160 @@ +import json +import libtmux +import random +import subprocess +import threading +import time +import typing + + +class InstructionSpec(typing.NamedTuple): + instruction: str + time_from: float + time_to: float + + +class CliDirector: + def __init__(self): + self.record_start = None + self.pause_between_keys = 0.2 + self.instructions: typing.List[InstructionSpec] = [] + + def start(self, filename: str, width: int = 0, height: int = 0) -> libtmux.Session: + self.start_session(width, height) + self.start_recording(filename) + return self.tmux_session + + def start_session(self, width: int = 0, height: int = 0) -> libtmux.Session: + self.tmux_server = libtmux.Server() + self.tmux_session = self.tmux_server.new_session(session_name="asciinema_recorder", kill_session=True) + self.tmux_pane = self.tmux_session.attached_window.attached_pane + self.tmux_version = self.tmux_pane.display_message("#{version}", True) + if width and height: + self.resize_window(width, height) + self.pause(3) + return self.tmux_session + + def start_recording(self, filename: str) -> None: + self.asciinema_proc = subprocess.Popen([ + "asciinema", "rec", "-y", "--overwrite", "-c", "tmux attach -t asciinema_recorder", filename]) + self.pause(1.5) + self.record_start = time.time() + + def resize_window(self, width: int, height: int) -> None: + subprocess.Popen(["resize", "-s", str(height), str(width)], stdout=subprocess.PIPE, stderr=subprocess.PIPE) + + def end(self) -> None: + self.end_recording() + self.end_session() + + def end_recording(self) -> None: + self.asciinema_proc.terminate() + self.asciinema_proc.wait(timeout=5) + self.record_start = None + self.instructions = [] + + def end_session(self) -> None: + self.tmux_session.kill_session() + + def press_key(self, keys: str, count=1, pause: typing.Optional[float] = None, target = None) -> None: + if pause is None: + pause = self.pause_between_keys + if target is None: + target = self.tmux_pane + for i in range(count): + if keys == " ": + keys = "Space" + target.send_keys(cmd=keys, enter=False, suppress_history=False) + + # inspired by https://github.com/dmotz/TuringType + real_pause = random.uniform(0, pause) + 0.4 * pause + if keys == "Space": + real_pause += 1.5 * pause + elif keys == ".": + real_pause += pause + elif random.random() > 0.75: + real_pause += pause + elif random.random() > 0.95: + real_pause += 2 * pause + self.pause(real_pause) + + def type(self, keys: str, pause: typing.Optional[float] = None, target = None) -> None: + if pause is None: + pause = self.pause_between_keys + if target is None: + target = self.tmux_pane + target.select_pane() + for key in keys: + self.press_key(key, pause=pause, target=target) + + def exec(self, keys: str, target = None) -> None: + if target is None: + target = self.tmux_pane + self.type(keys, target=target) + self.pause(1.25) + self.press_key("Enter", target=target) + self.pause(0.5) + + def focus_pane(self, pane: libtmux.Pane, set_active_pane: bool = True) -> None: + pane.select_pane() + if set_active_pane: + self.tmux_pane = pane + + def pause(self, seconds: float) -> None: + time.sleep(seconds) + + def run_external(self, command: str) -> None: + subprocess.run(command, shell=True) + + def message(self, msg: str, duration: typing.Optional[int] = None, add_instruction: bool = True, instruction_html: str = "") -> None: + if duration is None: + duration = len(msg) * 0.08 # seconds + self.tmux_session.set_option("display-time", int(duration * 1000)) # milliseconds + self.tmux_pane.display_message(" " + msg) + + if add_instruction or instruction_html: + if not instruction_html: + instruction_html = msg + self.instruction(instruction=instruction_html, duration=duration) + self.pause(duration + 0.5) + + def popup(self, content: str, duration: int = 4) -> None: + # todo: check if installed tmux version supports display-popup + + # tmux's display-popup is blocking, so we close it in a separate thread + t = threading.Thread(target=self.close_popup, args=[duration]) + t.start() + + lines = content.splitlines() + self.tmux_pane.cmd("display-popup", "", *lines) + t.join() + + def close_popup(self, duration: float = 0) -> None: + self.pause(duration) + self.tmux_pane.cmd("display-popup", "-C") + + def instruction(self, instruction: str, duration: float = 3, time_from: typing.Optional[float] = None) -> None: + if time_from is None: + time_from = self.current_time + + self.instructions.append(InstructionSpec( + instruction = str(len(self.instructions) + 1) + ". " + instruction, + time_from = round(time_from, 1), + time_to = round(time_from + duration, 1) + )) + + def save_instructions(self, output_path: str) -> None: + instr_as_dicts = [] + for instr in self.instructions: + instr_as_dicts.append(instr._asdict()) + with open(output_path, 'w', encoding='utf-8') as f: + json.dump(instr_as_dicts, f, ensure_ascii=False, indent=4) + + @property + def current_time(self) -> float: + now = time.time() + return round(now - self.record_start, 1) + + @property + def current_pane(self) -> libtmux.Pane: + return self.tmux_pane diff -Nru mitmproxy-5.1.1/docs/scripts/clirecording/docker/tmux.conf mitmproxy-6.0.2/docs/scripts/clirecording/docker/tmux.conf --- mitmproxy-5.1.1/docs/scripts/clirecording/docker/tmux.conf 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/scripts/clirecording/docker/tmux.conf 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,21 @@ +set -g default-terminal "screen-256color" + +set-option -g status-position top + +set -g status-style "bg=#000000,fg=#ffffff" +set -g message-style "bg=#3273dc,fg=#ffffff" + +set -g status-justify left +set -g status-left "" +set -g status-right "" + +setw -g window-status-current-format "" + + +# pane options +setw -g pane-base-index 1 +setw -g pane-border-format " Terminal Window #P --------------------------------------------------------------------------------------------------------" +setw -g pane-border-status top +setw -g pane-border-lines simple +setw -g pane-border-style "fg=#cccccc" +setw -g pane-active-border-style "fg=#ffffff" diff -Nru mitmproxy-5.1.1/docs/scripts/clirecording/Dockerfile mitmproxy-6.0.2/docs/scripts/clirecording/Dockerfile --- mitmproxy-5.1.1/docs/scripts/clirecording/Dockerfile 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/scripts/clirecording/Dockerfile 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,45 @@ +# todo: use a more lightweight base, e.g., Alpine Linux +FROM ubuntu:18.04 + +ENV LANG en_US.UTF-8 +ENV LANGUAGE en_US.UTF-8 +ENV TERM screen-256color + +# install mitmproxy, asciinema, and dependencies +RUN apt-get update && apt-get install -y \ + asciinema \ + autoconf \ + automake \ + autotools-dev \ + bison \ + curl \ + git \ + libevent-dev \ + libtool \ + locales \ + m4 \ + make \ + ncurses-dev \ + pkg-config \ + python3-pip \ + python3 \ + wget \ + xterm \ + && locale-gen --purge "en_US.UTF-8" \ + && update-locale "LANG=en_US.UTF-8" \ + && pip3 install libtmux curl requests mitmproxy + +# install latest tmux (to support popups) +RUN git clone https://github.com/tmux/tmux.git \ + && cd tmux \ + && sh autogen.sh \ + && ./configure && make && make install + +WORKDIR /root/clidirector + +COPY ./docker/tmux.conf ../.tmux.conf +COPY clidirector.py screenplays.py record.py ./ + +RUN echo 'PS1="[tutorial@mitmproxy] $ "' >> /root/.bashrc + +ENTRYPOINT [ "./record.py" ] diff -Nru mitmproxy-5.1.1/docs/scripts/clirecording/generate_recordings.sh mitmproxy-6.0.2/docs/scripts/clirecording/generate_recordings.sh --- mitmproxy-5.1.1/docs/scripts/clirecording/generate_recordings.sh 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/scripts/clirecording/generate_recordings.sh 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,6 @@ +#!/usr/bin/env bash + +docker build --pull --rm -t mitmproxy-clirecorder:latest . +docker run -i -t --rm \ + -v "$(pwd)"/../../src/static/recordings:/root/clidirector/recordings \ + mitmproxy-clirecorder:latest diff -Nru mitmproxy-5.1.1/docs/scripts/clirecording/record.py mitmproxy-6.0.2/docs/scripts/clirecording/record.py --- mitmproxy-5.1.1/docs/scripts/clirecording/record.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/scripts/clirecording/record.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,12 @@ +#!/usr/bin/env python3 + +from clidirector import CliDirector +import screenplays + + +if __name__ == '__main__': + director = CliDirector() + screenplays.record_user_interface(director) + screenplays.record_intercept_requests(director) + screenplays.record_modify_requests(director) + screenplays.record_replay_requests(director) diff -Nru mitmproxy-5.1.1/docs/scripts/clirecording/screenplays.py mitmproxy-6.0.2/docs/scripts/clirecording/screenplays.py --- mitmproxy-5.1.1/docs/scripts/clirecording/screenplays.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/scripts/clirecording/screenplays.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,278 @@ +#!/usr/bin/env python3 + +from clidirector import CliDirector + + +def record_user_interface(d: CliDirector): + tmux = d.start_session(width=120, height=36) + window = tmux.attached_window + + d.start_recording("recordings/mitmproxy_user_interface.cast") + d.message("Welcome to the mitmproxy tutorial. In this lesson we cover the user interface.") + d.pause(1) + d.exec("mitmproxy") + d.pause(3) + + d.message("This is the default view of mitmproxy.") + d.message("mitmproxy adds rows to the view as new requests come in.") + d.message("Let’s generate some requests using `curl` in a separate terminal.") + + pane_top = d.current_pane + pane_bottom = window.split_window(attach=True) + pane_bottom.resize_pane(height=12) + + d.focus_pane(pane_bottom) + d.pause(2) + + d.type("curl") + d.message("Use curl’s `--proxy` option to configure mitmproxy as a proxy.") + d.type(" --proxy http://127.0.0.1:8080") + + d.message("We use the text-based weather service `wttr.in`.") + d.exec(" \"http://wttr.in/Dunedin?0\"") + + d.pause(2) + d.press_key("Up") + d.press_key("Left", count=3) + d.press_key("BSpace", count=7) + d.exec("Innsbruck") + + d.pause(2) + d.exec("exit", target=pane_bottom) + + d.focus_pane(pane_top) + + d.message("You see the requests to `wttr.in` in the list of flows.") + + d.message("mitmproxy is controlled using keyboard shortcuts.") + d.message("Use your arrow keys `↑` and `↓` to change the focused flow (`>>`).") + d.press_key("Down", pause=0.5) + d.press_key("Up", pause=0.5) + d.press_key("Down", pause=0.5) + d.press_key("Up", pause=0.5) + + d.message("The focused flow (`>>`) is used as a target for various commands.") + + d.message("One such command shows the flow details, it is bound to `ENTER`.") + + d.message("Press `ENTER` to view the details of the focused flow.") + d.press_key("Enter") + + d.message("The flow details view has 3 panes: request, response, and detail.") + d.message("Use your arrow keys `←` and `→` to switch between panes.") + d.press_key("Right", count=2, pause=2.5) + d.press_key("Left", count=2, pause=1) + + d.message("Press `q` to exit the current view.",) + d.type("q") + + d.message("Press `?` to get a list of all available keyboard shortcuts.") + d.type("?") + d.pause(2) + d.press_key("Down", count=20, pause=0.25) + + d.message("Tip: Remember the `?` shortcut. It works in every view.") + d.message("Press `q` to exit the current view.") + d.type("q") + + d.message("Each shortcut is internally bound to a command.") + d.message("You can also execute commands directly (without using shortcuts).") + d.message("Press `:` to open the command prompt at the bottom.") + d.type(":") + + d.message("Enter `console.view.flow @focus`.") + d.type("console.view.flow @focus") + + d.message("The command `console.view.flow` opens the details view for a flow.") + + d.message("The argument `@focus` defines the target flow.") + + d.message("Press `ENTER` to execute the command.") + d.press_key("Enter") + + d.message("Commands unleash the full power of mitmproxy, i.e., to configure interceptions.") + + d.message("You now know basics of mitmproxy’s UI and how to control it.") + d.pause(1) + + d.message("In the next lesson you will learn to intercept flows.") + d.save_instructions("recordings/mitmproxy_user_interface_instructions.json") + d.end() + + +def record_intercept_requests(d: CliDirector): + tmux = d.start_session(width=120, height=36) + window = tmux.attached_window + + d.start_recording("recordings/mitmproxy_intercept_requests.cast") + d.message("Welcome to the mitmproxy tutorial. In this lesson we cover the interception of requests.") + d.pause(1) + d.exec("mitmproxy") + d.pause(3) + + d.message("We first need to configure mitmproxy to intercept requests.") + + d.message("Press `i` to prepopulate mitmproxy’s command prompt with `set intercept ''`.") + d.type("i") + d.pause(2) + + d.message("We use the flow filter expression `~u ` to only intercept specific URLs.") + d.message("Additionally, we use the filter `~q` to only intercept requests, but not responses.") + d.message("We combine both flow filters using `&`.") + + d.message("Enter `~u /Dunedin & ~q` between the quotes of the `set intercept` command and press `ENTER`.") + d.exec("~u /Dunedin & ~q") + d.message("The bottom bar shows that the interception has been configured.") + + d.message("Let’s generate a request using `curl` in a separate terminal.") + + pane_top = d.current_pane + pane_bottom = window.split_window(attach=True) + pane_bottom.resize_pane(height=12) + + d.focus_pane(pane_bottom) + d.pause(2) + + d.exec("curl --proxy http://127.0.0.1:8080 \"http://wttr.in/Dunedin?0\"") + d.pause(2) + + d.focus_pane(pane_top) + + d.message("You see a new line in in the list of flows.") + d.message("The new flow is displayed in red to indicate that it has been intercepted.") + d.message("Put the focus (`>>`) on the intercepted flow. This is already the case in our example.") + d.message("Press `a` to resume this flow without making any changes.") + d.type("a") + d.pause(2) + + d.focus_pane(pane_bottom) + + d.message("Submit another request and focus its flow.") + d.press_key("Up") + d.press_key("Enter") + d.pause(2) + + d.focus_pane(pane_top) + d.press_key("Down") + d.pause(1) + + d.message("Press `X` to kill this flow, i.e., discard it without forwarding it to its final destination `wttr.in`.") + d.type("X") + d.pause(3) + + d.message("In the next lesson you will learn to modify intercepted flows.") + d.save_instructions("recordings/mitmproxy_intercept_requests_instructions.json") + d.end() + + +def record_modify_requests(d: CliDirector): + tmux = d.start_session(width=120, height=36) + window = tmux.attached_window + + d.start_recording("recordings/mitmproxy_modify_requests.cast") + d.message("Welcome to the mitmproxy tutorial. In this lesson we cover the modification of intercepted requests.") + d.pause(1) + d.exec("mitmproxy") + d.pause(3) + + d.message("We configure and use the same interception rule as in the last tutorial.") + d.message("Press `i` to prepopulate mitmproxy’s command prompt, enter the flow filter `~u /Dunedin & ~q`, and press `ENTER`.") + d.type("i") + d.pause(2) + d.exec("~u /Dunedin & ~q") + + d.message("Let’s generate a request using `curl` in a separate terminal.") + + pane_top = d.current_pane + pane_bottom = window.split_window(attach=True) + pane_bottom.resize_pane(height=12) + + d.focus_pane(pane_bottom) + d.pause(2) + + d.exec("curl --proxy http://127.0.0.1:8080 \"http://wttr.in/Dunedin?0\"") + d.pause(2) + + d.focus_pane(pane_top) + + d.message("We now want to modify the intercepted request.") + d.message("Put the focus (`>>`) on the intercepted flow. This is already the case in our example.") + + d.message("Press `ENTER` to open the details view for the intercepted flow.") + d.press_key("Enter") + + d.message("Press `e` to edit the intercepted flow.") + d.type("e") + + d.message("mitmproxy asks which part to modify.") + + d.message("Select `path` by using your arrow keys and press `ENTER`.") + d.press_key("Down", count=3, pause=0.5) + d.pause(1) + d.press_key("Enter") + + d.message("mitmproxy shows all path components line by line, in our example its just `Dunedin`.") + d.message("Press `ENTER` to modify the selected path component.") + d.press_key("Down", pause=2) + d.press_key("Enter") + + d.message("Replace `Dunedin` with `Innsbruck`.") + d.press_key("BSpace", count=7, pause=0.5) + d.type("Innsbruck", pause=0.5) + + d.message("Press `ESC` to confirm your change.") + d.press_key("Escape") + + d.message("Press `q` to go back to the flow details view.") + d.type("q") + + d.message("Press `a` to resume the intercepted flow.") + d.type("a") + d.pause(2) + + d.message("You see that the request URL was modified and `wttr.in` replied with the weather report for `Innsbruck`.") + + d.message("In the next lesson you will learn to replay flows.") + d.save_instructions("recordings/mitmproxy_modify_requests_instructions.json") + d.end() + + +def record_replay_requests(d: CliDirector): + tmux = d.start_session(width=120, height=36) + window = tmux.attached_window + + d.start_recording("recordings/mitmproxy_replay_requests.cast") + d.message("Welcome to the mitmproxy tutorial. In this lesson we cover replaying requests.") + d.pause(1) + d.exec("mitmproxy") + d.pause(3) + + d.message("Let’s generate a request that we can replay. We use `curl` in a separate terminal.") + + pane_top = d.current_pane + pane_bottom = window.split_window(attach=True) + pane_bottom.resize_pane(height=12) + + d.focus_pane(pane_bottom) + d.pause(2) + + d.exec("curl --proxy http://127.0.0.1:8080 \"http://wttr.in/Dunedin?0\"") + d.pause(2) + + d.focus_pane(pane_top) + + d.message("We now want to replay the this request.") + d.message("Put the focus (`>>`) on the request that should be replayed. This is already the case in our example.") + d.message("Press `r` to replay the request.") + d.type("r") + + d.message("Note that no new rows are added for replayed flows, but the existing row is updated.") + d.message("Every time you press `r`, mitmproxy sends this request to the server again and updates the flow.") + d.press_key("r", count=4, pause=1) + + d.message("You can also modify a flow before replaying it.") + d.message("It works as shown in the previous lesson, by pressing `e`.") + + d.message("Congratulations! You have completed all lessons of the mitmproxy tutorial.") + d.save_instructions("recordings/mitmproxy_replay_requests_instructions.json") + d.end() diff -Nru mitmproxy-5.1.1/docs/scripts/examples.py mitmproxy-6.0.2/docs/scripts/examples.py --- mitmproxy-5.1.1/docs/scripts/examples.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/scripts/examples.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,48 @@ +#!/usr/bin/env python3 + +import re +from pathlib import Path + +here = Path(__file__).absolute().parent +example_dir = here / ".." / "src" / "examples" / "addons" +examples = example_dir.glob('*.py') + +overview = [] +listings = [] + +for example in examples: + code = example.read_text() + slug = str(example.with_suffix("").relative_to(example_dir)) + slug = re.sub(r"[^a-zA-Z]", "-", slug) + match = re.search(r''' + ^ + (?:[#][^\n]*\n)? # there might be a shebang + """ + \s* + (.+?) + \s* + (?:\n\n|""") # stop on empty line or end of comment + ''', code, re.VERBOSE) + if match: + comment = " — " + match.group(1) + else: + comment = "" + overview.append( + f" * [{example.name}](#{slug}){comment}" + ) + listings.append(f""" +

Example: {example.name}

+ +```python +{code} +``` +""") +print("\n".join(overview)) +print(""" +### Community Examples + +Additional examples contributed by the mitmproxy community can be found +[on GitHub](https://github.com/mitmproxy/mitmproxy/tree/master/examples/contrib). + +""") +print("\n".join(listings)) diff -Nru mitmproxy-5.1.1/docs/src/assets/asciinema-player.css mitmproxy-6.0.2/docs/src/assets/asciinema-player.css --- mitmproxy-5.1.1/docs/src/assets/asciinema-player.css 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/assets/asciinema-player.css 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,2563 @@ +.asciinema-player-wrapper { + position: relative; + text-align: center; + outline: none; +} +.asciinema-player-wrapper .title-bar { + display: none; + top: -78px; + transition: top 0.15s linear; + position: absolute; + left: 0; + right: 0; + box-sizing: content-box; + font-size: 20px; + line-height: 1em; + padding: 15px; + font-family: sans-serif; + color: white; + background-color: rgba(0, 0, 0, 0.8); +} +.asciinema-player-wrapper .title-bar img { + vertical-align: middle; + height: 48px; + margin-right: 16px; +} +.asciinema-player-wrapper .title-bar a { + color: white; + text-decoration: underline; +} +.asciinema-player-wrapper .title-bar a:hover { + text-decoration: none; +} +.asciinema-player-wrapper:fullscreen { + background-color: #000; + width: 100%; + height: 100%; + display: -webkit-flex; + display: -ms-flexbox; + display: flex; + -webkit-justify-content: center; + justify-content: center; + -webkit-align-items: center; + align-items: center; +} +.asciinema-player-wrapper:fullscreen .asciinema-player { + position: static; +} +.asciinema-player-wrapper:fullscreen .title-bar { + display: initial; +} +.asciinema-player-wrapper:fullscreen.hud .title-bar { + top: 0; +} +.asciinema-player-wrapper:-webkit-full-screen { + background-color: #000; + width: 100%; + height: 100%; + display: -webkit-flex; + display: -ms-flexbox; + display: flex; + -webkit-justify-content: center; + justify-content: center; + -webkit-align-items: center; + align-items: center; +} +.asciinema-player-wrapper:-webkit-full-screen .asciinema-player { + position: static; +} +.asciinema-player-wrapper:-webkit-full-screen .title-bar { + display: initial; +} +.asciinema-player-wrapper:-webkit-full-screen.hud .title-bar { + top: 0; +} +.asciinema-player-wrapper:-moz-full-screen { + background-color: #000; + width: 100%; + height: 100%; + display: -webkit-flex; + display: -ms-flexbox; + display: flex; + -webkit-justify-content: center; + justify-content: center; + -webkit-align-items: center; + align-items: center; +} +.asciinema-player-wrapper:-moz-full-screen .asciinema-player { + position: static; +} +.asciinema-player-wrapper:-moz-full-screen .title-bar { + display: initial; +} +.asciinema-player-wrapper:-moz-full-screen.hud .title-bar { + top: 0; +} +.asciinema-player-wrapper:-ms-fullscreen { + background-color: #000; + width: 100%; + height: 100%; + display: -webkit-flex; + display: -ms-flexbox; + display: flex; + -webkit-justify-content: center; + justify-content: center; + -webkit-align-items: center; + align-items: center; +} +.asciinema-player-wrapper:-ms-fullscreen .asciinema-player { + position: static; +} +.asciinema-player-wrapper:-ms-fullscreen .title-bar { + display: initial; +} +.asciinema-player-wrapper:-ms-fullscreen.hud .title-bar { + top: 0; +} +.asciinema-player-wrapper .asciinema-player { + text-align: left; + display: inline-block; + padding: 0px; + position: relative; + box-sizing: content-box; + -moz-box-sizing: content-box; + -webkit-box-sizing: content-box; + overflow: hidden; + max-width: 100%; +} +.asciinema-terminal { + box-sizing: content-box; + -moz-box-sizing: content-box; + -webkit-box-sizing: content-box; + overflow: hidden; + padding: 0; + margin: 0px; + display: block; + white-space: pre; + border: 0; + word-wrap: normal; + word-break: normal; + border-radius: 0; + border-style: solid; + cursor: text; + border-width: 0.5em; + font-family: Consolas, Menlo, 'Bitstream Vera Sans Mono', monospace, 'Powerline Symbols'; + line-height: 1.3333333333em; +} +.asciinema-terminal .line { + letter-spacing: normal; + overflow: hidden; + height: 1.3333333333em; +} +.asciinema-terminal .line span { + padding: 0; + display: inline-block; + height: 1.3333333333em; +} +.asciinema-terminal .line { + display: block; + width: 200%; +} +.asciinema-terminal .bright { + font-weight: bold; +} +.asciinema-terminal .underline { + text-decoration: underline; +} +.asciinema-terminal .italic { + font-style: italic; +} +.asciinema-terminal.font-small { + font-size: 12px; +} +.asciinema-terminal.font-medium { + font-size: 18px; +} +.asciinema-terminal.font-big { + font-size: 24px; +} +.asciinema-player .control-bar { + width: 100%; + height: 32px; + background: rgba(0, 0, 0, 0.8); + /* no gradient fallback */ + background: -moz-linear-gradient(top, rgba(0, 0, 0, 0.5) 0%, #000000 25%, #000000 100%); + /* FF3.6-15 */ + background: -webkit-linear-gradient(top, rgba(0, 0, 0, 0.5) 0%, #000000 25%, #000000 100%); + /* Chrome10-25,Safari5.1-6 */ + background: linear-gradient(to bottom, rgba(0, 0, 0, 0.5) 0%, #000000 25%, #000000 100%); + /* W3C, IE10+, FF16+, Chrome26+, Opera12+, Safari7+ */ + color: #bbbbbb; + box-sizing: content-box; + line-height: 1; + position: absolute; + bottom: -35px; + left: 0; + transition: bottom 0.15s linear; +} +.asciinema-player .control-bar * { + box-sizing: inherit; + font-size: 0; +} +.asciinema-player .control-bar svg.icon path { + fill: #bbbbbb; +} +.asciinema-player .control-bar .playback-button { + display: block; + float: left; + cursor: pointer; + height: 12px; + width: 12px; + padding: 10px; +} +.asciinema-player .control-bar .playback-button svg { + height: 12px; + width: 12px; +} +.asciinema-player .control-bar .timer { + display: block; + float: left; + width: 50px; + height: 100%; + text-align: center; + font-family: Helvetica, Arial, sans-serif; + font-size: 11px; + font-weight: bold; + line-height: 32px; + cursor: default; +} +.asciinema-player .control-bar .timer span { + display: inline-block; + font-size: inherit; +} +.asciinema-player .control-bar .timer .time-remaining { + display: none; +} +.asciinema-player .control-bar .timer:hover .time-elapsed { + display: none; +} +.asciinema-player .control-bar .timer:hover .time-remaining { + display: inline; +} +.asciinema-player .control-bar .progressbar { + display: block; + overflow: hidden; + height: 100%; + padding: 0 10px; +} +.asciinema-player .control-bar .progressbar .bar { + display: block; + cursor: pointer; + height: 100%; + padding-top: 15px; + font-size: 0; +} +.asciinema-player .control-bar .progressbar .bar .gutter { + display: block; + height: 3px; + background-color: #333; +} +.asciinema-player .control-bar .progressbar .bar .gutter span { + display: inline-block; + height: 100%; + background-color: #bbbbbb; + border-radius: 3px; +} +.asciinema-player .control-bar.live .progressbar .bar { + cursor: default; +} +.asciinema-player .control-bar .fullscreen-button { + display: block; + float: right; + width: 14px; + height: 14px; + padding: 9px; + cursor: pointer; +} +.asciinema-player .control-bar .fullscreen-button svg { + width: 14px; + height: 14px; +} +.asciinema-player .control-bar .fullscreen-button svg:first-child { + display: inline; +} +.asciinema-player .control-bar .fullscreen-button svg:last-child { + display: none; +} +.asciinema-player-wrapper.hud .control-bar { + bottom: 0px; +} +.asciinema-player-wrapper:fullscreen .fullscreen-button svg:first-child { + display: none; +} +.asciinema-player-wrapper:fullscreen .fullscreen-button svg:last-child { + display: inline; +} +.asciinema-player-wrapper:-webkit-full-screen .fullscreen-button svg:first-child { + display: none; +} +.asciinema-player-wrapper:-webkit-full-screen .fullscreen-button svg:last-child { + display: inline; +} +.asciinema-player-wrapper:-moz-full-screen .fullscreen-button svg:first-child { + display: none; +} +.asciinema-player-wrapper:-moz-full-screen .fullscreen-button svg:last-child { + display: inline; +} +.asciinema-player-wrapper:-ms-fullscreen .fullscreen-button svg:first-child { + display: none; +} +.asciinema-player-wrapper:-ms-fullscreen .fullscreen-button svg:last-child { + display: inline; +} +.asciinema-player .loading { + z-index: 10; + background-repeat: no-repeat; + background-position: center; + position: absolute; + top: 0; + left: 0; + right: 0; + bottom: 32px; + background-color: rgba(0, 0, 0, 0.5); +} +.asciinema-player .start-prompt { + z-index: 10; + background-repeat: no-repeat; + background-position: center; + position: absolute; + top: 0; + left: 0; + right: 0; + bottom: 32px; + z-index: 20; + cursor: pointer; +} +.asciinema-player .start-prompt .play-button { + font-size: 0px; +} +.asciinema-player .start-prompt .play-button { + position: absolute; + left: 0; + top: 0; + right: 0; + bottom: 0; + text-align: center; + color: white; + display: table; + width: 100%; + height: 100%; +} +.asciinema-player .start-prompt .play-button div { + vertical-align: middle; + display: table-cell; +} +.asciinema-player .start-prompt .play-button div span { + width: 96px; + height: 96px; + display: inline-block; +} +@-webkit-keyframes expand { + 0% { + -webkit-transform: scale(0); + } + 50% { + -webkit-transform: scale(1); + } + 100% { + z-index: 1; + } +} +@-moz-keyframes expand { + 0% { + -moz-transform: scale(0); + } + 50% { + -moz-transform: scale(1); + } + 100% { + z-index: 1; + } +} +@-o-keyframes expand { + 0% { + -o-transform: scale(0); + } + 50% { + -o-transform: scale(1); + } + 100% { + z-index: 1; + } +} +@keyframes expand { + 0% { + transform: scale(0); + } + 50% { + transform: scale(1); + } + 100% { + z-index: 1; + } +} +.loader { + position: absolute; + left: 50%; + top: 50%; + margin: -20px 0 0 -20px; + background-color: white; + border-radius: 50%; + box-shadow: 0 0 0 6.66667px #141414; + width: 40px; + height: 40px; +} +.loader:before, +.loader:after { + content: ""; + position: absolute; + left: 50%; + top: 50%; + display: block; + margin: -21px 0 0 -21px; + border-radius: 50%; + z-index: 2; + width: 42px; + height: 42px; +} +.loader:before { + background-color: #141414; + -webkit-animation: expand 1.6s linear infinite both; + -moz-animation: expand 1.6s linear infinite both; + animation: expand 1.6s linear infinite both; +} +.loader:after { + background-color: white; + -webkit-animation: expand 1.6s linear 0.8s infinite both; + -moz-animation: expand 1.6s linear 0.8s infinite both; + animation: expand 1.6s linear 0.8s infinite both; +} +.asciinema-terminal .fg-16 { + color: #000000; +} +.asciinema-terminal .bg-16 { + background-color: #000000; +} +.asciinema-terminal .fg-17 { + color: #00005f; +} +.asciinema-terminal .bg-17 { + background-color: #00005f; +} +.asciinema-terminal .fg-18 { + color: #000087; +} +.asciinema-terminal .bg-18 { + background-color: #000087; +} +.asciinema-terminal .fg-19 { + color: #0000af; +} +.asciinema-terminal .bg-19 { + background-color: #0000af; +} +.asciinema-terminal .fg-20 { + color: #0000d7; +} +.asciinema-terminal .bg-20 { + background-color: #0000d7; +} +.asciinema-terminal .fg-21 { + color: #0000ff; +} +.asciinema-terminal .bg-21 { + background-color: #0000ff; +} +.asciinema-terminal .fg-22 { + color: #005f00; +} +.asciinema-terminal .bg-22 { + background-color: #005f00; +} +.asciinema-terminal .fg-23 { + color: #005f5f; +} +.asciinema-terminal .bg-23 { + background-color: #005f5f; +} +.asciinema-terminal .fg-24 { + color: #005f87; +} +.asciinema-terminal .bg-24 { + background-color: #005f87; +} +.asciinema-terminal .fg-25 { + color: #005faf; +} +.asciinema-terminal .bg-25 { + background-color: #005faf; +} +.asciinema-terminal .fg-26 { + color: #005fd7; +} +.asciinema-terminal .bg-26 { + background-color: #005fd7; +} +.asciinema-terminal .fg-27 { + color: #005fff; +} +.asciinema-terminal .bg-27 { + background-color: #005fff; +} +.asciinema-terminal .fg-28 { + color: #008700; +} +.asciinema-terminal .bg-28 { + background-color: #008700; +} +.asciinema-terminal .fg-29 { + color: #00875f; +} +.asciinema-terminal .bg-29 { + background-color: #00875f; +} +.asciinema-terminal .fg-30 { + color: #008787; +} +.asciinema-terminal .bg-30 { + background-color: #008787; +} +.asciinema-terminal .fg-31 { + color: #0087af; +} +.asciinema-terminal .bg-31 { + background-color: #0087af; +} +.asciinema-terminal .fg-32 { + color: #0087d7; +} +.asciinema-terminal .bg-32 { + background-color: #0087d7; +} +.asciinema-terminal .fg-33 { + color: #0087ff; +} +.asciinema-terminal .bg-33 { + background-color: #0087ff; +} +.asciinema-terminal .fg-34 { + color: #00af00; +} +.asciinema-terminal .bg-34 { + background-color: #00af00; +} +.asciinema-terminal .fg-35 { + color: #00af5f; +} +.asciinema-terminal .bg-35 { + background-color: #00af5f; +} +.asciinema-terminal .fg-36 { + color: #00af87; +} +.asciinema-terminal .bg-36 { + background-color: #00af87; +} +.asciinema-terminal .fg-37 { + color: #00afaf; +} +.asciinema-terminal .bg-37 { + background-color: #00afaf; +} +.asciinema-terminal .fg-38 { + color: #00afd7; +} +.asciinema-terminal .bg-38 { + background-color: #00afd7; +} +.asciinema-terminal .fg-39 { + color: #00afff; +} +.asciinema-terminal .bg-39 { + background-color: #00afff; +} +.asciinema-terminal .fg-40 { + color: #00d700; +} +.asciinema-terminal .bg-40 { + background-color: #00d700; +} +.asciinema-terminal .fg-41 { + color: #00d75f; +} +.asciinema-terminal .bg-41 { + background-color: #00d75f; +} +.asciinema-terminal .fg-42 { + color: #00d787; +} +.asciinema-terminal .bg-42 { + background-color: #00d787; +} +.asciinema-terminal .fg-43 { + color: #00d7af; +} +.asciinema-terminal .bg-43 { + background-color: #00d7af; +} +.asciinema-terminal .fg-44 { + color: #00d7d7; +} +.asciinema-terminal .bg-44 { + background-color: #00d7d7; +} +.asciinema-terminal .fg-45 { + color: #00d7ff; +} +.asciinema-terminal .bg-45 { + background-color: #00d7ff; +} +.asciinema-terminal .fg-46 { + color: #00ff00; +} +.asciinema-terminal .bg-46 { + background-color: #00ff00; +} +.asciinema-terminal .fg-47 { + color: #00ff5f; +} +.asciinema-terminal .bg-47 { + background-color: #00ff5f; +} +.asciinema-terminal .fg-48 { + color: #00ff87; +} +.asciinema-terminal .bg-48 { + background-color: #00ff87; +} +.asciinema-terminal .fg-49 { + color: #00ffaf; +} +.asciinema-terminal .bg-49 { + background-color: #00ffaf; +} +.asciinema-terminal .fg-50 { + color: #00ffd7; +} +.asciinema-terminal .bg-50 { + background-color: #00ffd7; +} +.asciinema-terminal .fg-51 { + color: #00ffff; +} +.asciinema-terminal .bg-51 { + background-color: #00ffff; +} +.asciinema-terminal .fg-52 { + color: #5f0000; +} +.asciinema-terminal .bg-52 { + background-color: #5f0000; +} +.asciinema-terminal .fg-53 { + color: #5f005f; +} +.asciinema-terminal .bg-53 { + background-color: #5f005f; +} +.asciinema-terminal .fg-54 { + color: #5f0087; +} +.asciinema-terminal .bg-54 { + background-color: #5f0087; +} +.asciinema-terminal .fg-55 { + color: #5f00af; +} +.asciinema-terminal .bg-55 { + background-color: #5f00af; +} +.asciinema-terminal .fg-56 { + color: #5f00d7; +} +.asciinema-terminal .bg-56 { + background-color: #5f00d7; +} +.asciinema-terminal .fg-57 { + color: #5f00ff; +} +.asciinema-terminal .bg-57 { + background-color: #5f00ff; +} +.asciinema-terminal .fg-58 { + color: #5f5f00; +} +.asciinema-terminal .bg-58 { + background-color: #5f5f00; +} +.asciinema-terminal .fg-59 { + color: #5f5f5f; +} +.asciinema-terminal .bg-59 { + background-color: #5f5f5f; +} +.asciinema-terminal .fg-60 { + color: #5f5f87; +} +.asciinema-terminal .bg-60 { + background-color: #5f5f87; +} +.asciinema-terminal .fg-61 { + color: #5f5faf; +} +.asciinema-terminal .bg-61 { + background-color: #5f5faf; +} +.asciinema-terminal .fg-62 { + color: #5f5fd7; +} +.asciinema-terminal .bg-62 { + background-color: #5f5fd7; +} +.asciinema-terminal .fg-63 { + color: #5f5fff; +} +.asciinema-terminal .bg-63 { + background-color: #5f5fff; +} +.asciinema-terminal .fg-64 { + color: #5f8700; +} +.asciinema-terminal .bg-64 { + background-color: #5f8700; +} +.asciinema-terminal .fg-65 { + color: #5f875f; +} +.asciinema-terminal .bg-65 { + background-color: #5f875f; +} +.asciinema-terminal .fg-66 { + color: #5f8787; +} +.asciinema-terminal .bg-66 { + background-color: #5f8787; +} +.asciinema-terminal .fg-67 { + color: #5f87af; +} +.asciinema-terminal .bg-67 { + background-color: #5f87af; +} +.asciinema-terminal .fg-68 { + color: #5f87d7; +} +.asciinema-terminal .bg-68 { + background-color: #5f87d7; +} +.asciinema-terminal .fg-69 { + color: #5f87ff; +} +.asciinema-terminal .bg-69 { + background-color: #5f87ff; +} +.asciinema-terminal .fg-70 { + color: #5faf00; +} +.asciinema-terminal .bg-70 { + background-color: #5faf00; +} +.asciinema-terminal .fg-71 { + color: #5faf5f; +} +.asciinema-terminal .bg-71 { + background-color: #5faf5f; +} +.asciinema-terminal .fg-72 { + color: #5faf87; +} +.asciinema-terminal .bg-72 { + background-color: #5faf87; +} +.asciinema-terminal .fg-73 { + color: #5fafaf; +} +.asciinema-terminal .bg-73 { + background-color: #5fafaf; +} +.asciinema-terminal .fg-74 { + color: #5fafd7; +} +.asciinema-terminal .bg-74 { + background-color: #5fafd7; +} +.asciinema-terminal .fg-75 { + color: #5fafff; +} +.asciinema-terminal .bg-75 { + background-color: #5fafff; +} +.asciinema-terminal .fg-76 { + color: #5fd700; +} +.asciinema-terminal .bg-76 { + background-color: #5fd700; +} +.asciinema-terminal .fg-77 { + color: #5fd75f; +} +.asciinema-terminal .bg-77 { + background-color: #5fd75f; +} +.asciinema-terminal .fg-78 { + color: #5fd787; +} +.asciinema-terminal .bg-78 { + background-color: #5fd787; +} +.asciinema-terminal .fg-79 { + color: #5fd7af; +} +.asciinema-terminal .bg-79 { + background-color: #5fd7af; +} +.asciinema-terminal .fg-80 { + color: #5fd7d7; +} +.asciinema-terminal .bg-80 { + background-color: #5fd7d7; +} +.asciinema-terminal .fg-81 { + color: #5fd7ff; +} +.asciinema-terminal .bg-81 { + background-color: #5fd7ff; +} +.asciinema-terminal .fg-82 { + color: #5fff00; +} +.asciinema-terminal .bg-82 { + background-color: #5fff00; +} +.asciinema-terminal .fg-83 { + color: #5fff5f; +} +.asciinema-terminal .bg-83 { + background-color: #5fff5f; +} +.asciinema-terminal .fg-84 { + color: #5fff87; +} +.asciinema-terminal .bg-84 { + background-color: #5fff87; +} +.asciinema-terminal .fg-85 { + color: #5fffaf; +} +.asciinema-terminal .bg-85 { + background-color: #5fffaf; +} +.asciinema-terminal .fg-86 { + color: #5fffd7; +} +.asciinema-terminal .bg-86 { + background-color: #5fffd7; +} +.asciinema-terminal .fg-87 { + color: #5fffff; +} +.asciinema-terminal .bg-87 { + background-color: #5fffff; +} +.asciinema-terminal .fg-88 { + color: #870000; +} +.asciinema-terminal .bg-88 { + background-color: #870000; +} +.asciinema-terminal .fg-89 { + color: #87005f; +} +.asciinema-terminal .bg-89 { + background-color: #87005f; +} +.asciinema-terminal .fg-90 { + color: #870087; +} +.asciinema-terminal .bg-90 { + background-color: #870087; +} +.asciinema-terminal .fg-91 { + color: #8700af; +} +.asciinema-terminal .bg-91 { + background-color: #8700af; +} +.asciinema-terminal .fg-92 { + color: #8700d7; +} +.asciinema-terminal .bg-92 { + background-color: #8700d7; +} +.asciinema-terminal .fg-93 { + color: #8700ff; +} +.asciinema-terminal .bg-93 { + background-color: #8700ff; +} +.asciinema-terminal .fg-94 { + color: #875f00; +} +.asciinema-terminal .bg-94 { + background-color: #875f00; +} +.asciinema-terminal .fg-95 { + color: #875f5f; +} +.asciinema-terminal .bg-95 { + background-color: #875f5f; +} +.asciinema-terminal .fg-96 { + color: #875f87; +} +.asciinema-terminal .bg-96 { + background-color: #875f87; +} +.asciinema-terminal .fg-97 { + color: #875faf; +} +.asciinema-terminal .bg-97 { + background-color: #875faf; +} +.asciinema-terminal .fg-98 { + color: #875fd7; +} +.asciinema-terminal .bg-98 { + background-color: #875fd7; +} +.asciinema-terminal .fg-99 { + color: #875fff; +} +.asciinema-terminal .bg-99 { + background-color: #875fff; +} +.asciinema-terminal .fg-100 { + color: #878700; +} +.asciinema-terminal .bg-100 { + background-color: #878700; +} +.asciinema-terminal .fg-101 { + color: #87875f; +} +.asciinema-terminal .bg-101 { + background-color: #87875f; +} +.asciinema-terminal .fg-102 { + color: #878787; +} +.asciinema-terminal .bg-102 { + background-color: #878787; +} +.asciinema-terminal .fg-103 { + color: #8787af; +} +.asciinema-terminal .bg-103 { + background-color: #8787af; +} +.asciinema-terminal .fg-104 { + color: #8787d7; +} +.asciinema-terminal .bg-104 { + background-color: #8787d7; +} +.asciinema-terminal .fg-105 { + color: #8787ff; +} +.asciinema-terminal .bg-105 { + background-color: #8787ff; +} +.asciinema-terminal .fg-106 { + color: #87af00; +} +.asciinema-terminal .bg-106 { + background-color: #87af00; +} +.asciinema-terminal .fg-107 { + color: #87af5f; +} +.asciinema-terminal .bg-107 { + background-color: #87af5f; +} +.asciinema-terminal .fg-108 { + color: #87af87; +} +.asciinema-terminal .bg-108 { + background-color: #87af87; +} +.asciinema-terminal .fg-109 { + color: #87afaf; +} +.asciinema-terminal .bg-109 { + background-color: #87afaf; +} +.asciinema-terminal .fg-110 { + color: #87afd7; +} +.asciinema-terminal .bg-110 { + background-color: #87afd7; +} +.asciinema-terminal .fg-111 { + color: #87afff; +} +.asciinema-terminal .bg-111 { + background-color: #87afff; +} +.asciinema-terminal .fg-112 { + color: #87d700; +} +.asciinema-terminal .bg-112 { + background-color: #87d700; +} +.asciinema-terminal .fg-113 { + color: #87d75f; +} +.asciinema-terminal .bg-113 { + background-color: #87d75f; +} +.asciinema-terminal .fg-114 { + color: #87d787; +} +.asciinema-terminal .bg-114 { + background-color: #87d787; +} +.asciinema-terminal .fg-115 { + color: #87d7af; +} +.asciinema-terminal .bg-115 { + background-color: #87d7af; +} +.asciinema-terminal .fg-116 { + color: #87d7d7; +} +.asciinema-terminal .bg-116 { + background-color: #87d7d7; +} +.asciinema-terminal .fg-117 { + color: #87d7ff; +} +.asciinema-terminal .bg-117 { + background-color: #87d7ff; +} +.asciinema-terminal .fg-118 { + color: #87ff00; +} +.asciinema-terminal .bg-118 { + background-color: #87ff00; +} +.asciinema-terminal .fg-119 { + color: #87ff5f; +} +.asciinema-terminal .bg-119 { + background-color: #87ff5f; +} +.asciinema-terminal .fg-120 { + color: #87ff87; +} +.asciinema-terminal .bg-120 { + background-color: #87ff87; +} +.asciinema-terminal .fg-121 { + color: #87ffaf; +} +.asciinema-terminal .bg-121 { + background-color: #87ffaf; +} +.asciinema-terminal .fg-122 { + color: #87ffd7; +} +.asciinema-terminal .bg-122 { + background-color: #87ffd7; +} +.asciinema-terminal .fg-123 { + color: #87ffff; +} +.asciinema-terminal .bg-123 { + background-color: #87ffff; +} +.asciinema-terminal .fg-124 { + color: #af0000; +} +.asciinema-terminal .bg-124 { + background-color: #af0000; +} +.asciinema-terminal .fg-125 { + color: #af005f; +} +.asciinema-terminal .bg-125 { + background-color: #af005f; +} +.asciinema-terminal .fg-126 { + color: #af0087; +} +.asciinema-terminal .bg-126 { + background-color: #af0087; +} +.asciinema-terminal .fg-127 { + color: #af00af; +} +.asciinema-terminal .bg-127 { + background-color: #af00af; +} +.asciinema-terminal .fg-128 { + color: #af00d7; +} +.asciinema-terminal .bg-128 { + background-color: #af00d7; +} +.asciinema-terminal .fg-129 { + color: #af00ff; +} +.asciinema-terminal .bg-129 { + background-color: #af00ff; +} +.asciinema-terminal .fg-130 { + color: #af5f00; +} +.asciinema-terminal .bg-130 { + background-color: #af5f00; +} +.asciinema-terminal .fg-131 { + color: #af5f5f; +} +.asciinema-terminal .bg-131 { + background-color: #af5f5f; +} +.asciinema-terminal .fg-132 { + color: #af5f87; +} +.asciinema-terminal .bg-132 { + background-color: #af5f87; +} +.asciinema-terminal .fg-133 { + color: #af5faf; +} +.asciinema-terminal .bg-133 { + background-color: #af5faf; +} +.asciinema-terminal .fg-134 { + color: #af5fd7; +} +.asciinema-terminal .bg-134 { + background-color: #af5fd7; +} +.asciinema-terminal .fg-135 { + color: #af5fff; +} +.asciinema-terminal .bg-135 { + background-color: #af5fff; +} +.asciinema-terminal .fg-136 { + color: #af8700; +} +.asciinema-terminal .bg-136 { + background-color: #af8700; +} +.asciinema-terminal .fg-137 { + color: #af875f; +} +.asciinema-terminal .bg-137 { + background-color: #af875f; +} +.asciinema-terminal .fg-138 { + color: #af8787; +} +.asciinema-terminal .bg-138 { + background-color: #af8787; +} +.asciinema-terminal .fg-139 { + color: #af87af; +} +.asciinema-terminal .bg-139 { + background-color: #af87af; +} +.asciinema-terminal .fg-140 { + color: #af87d7; +} +.asciinema-terminal .bg-140 { + background-color: #af87d7; +} +.asciinema-terminal .fg-141 { + color: #af87ff; +} +.asciinema-terminal .bg-141 { + background-color: #af87ff; +} +.asciinema-terminal .fg-142 { + color: #afaf00; +} +.asciinema-terminal .bg-142 { + background-color: #afaf00; +} +.asciinema-terminal .fg-143 { + color: #afaf5f; +} +.asciinema-terminal .bg-143 { + background-color: #afaf5f; +} +.asciinema-terminal .fg-144 { + color: #afaf87; +} +.asciinema-terminal .bg-144 { + background-color: #afaf87; +} +.asciinema-terminal .fg-145 { + color: #afafaf; +} +.asciinema-terminal .bg-145 { + background-color: #afafaf; +} +.asciinema-terminal .fg-146 { + color: #afafd7; +} +.asciinema-terminal .bg-146 { + background-color: #afafd7; +} +.asciinema-terminal .fg-147 { + color: #afafff; +} +.asciinema-terminal .bg-147 { + background-color: #afafff; +} +.asciinema-terminal .fg-148 { + color: #afd700; +} +.asciinema-terminal .bg-148 { + background-color: #afd700; +} +.asciinema-terminal .fg-149 { + color: #afd75f; +} +.asciinema-terminal .bg-149 { + background-color: #afd75f; +} +.asciinema-terminal .fg-150 { + color: #afd787; +} +.asciinema-terminal .bg-150 { + background-color: #afd787; +} +.asciinema-terminal .fg-151 { + color: #afd7af; +} +.asciinema-terminal .bg-151 { + background-color: #afd7af; +} +.asciinema-terminal .fg-152 { + color: #afd7d7; +} +.asciinema-terminal .bg-152 { + background-color: #afd7d7; +} +.asciinema-terminal .fg-153 { + color: #afd7ff; +} +.asciinema-terminal .bg-153 { + background-color: #afd7ff; +} +.asciinema-terminal .fg-154 { + color: #afff00; +} +.asciinema-terminal .bg-154 { + background-color: #afff00; +} +.asciinema-terminal .fg-155 { + color: #afff5f; +} +.asciinema-terminal .bg-155 { + background-color: #afff5f; +} +.asciinema-terminal .fg-156 { + color: #afff87; +} +.asciinema-terminal .bg-156 { + background-color: #afff87; +} +.asciinema-terminal .fg-157 { + color: #afffaf; +} +.asciinema-terminal .bg-157 { + background-color: #afffaf; +} +.asciinema-terminal .fg-158 { + color: #afffd7; +} +.asciinema-terminal .bg-158 { + background-color: #afffd7; +} +.asciinema-terminal .fg-159 { + color: #afffff; +} +.asciinema-terminal .bg-159 { + background-color: #afffff; +} +.asciinema-terminal .fg-160 { + color: #d70000; +} +.asciinema-terminal .bg-160 { + background-color: #d70000; +} +.asciinema-terminal .fg-161 { + color: #d7005f; +} +.asciinema-terminal .bg-161 { + background-color: #d7005f; +} +.asciinema-terminal .fg-162 { + color: #d70087; +} +.asciinema-terminal .bg-162 { + background-color: #d70087; +} +.asciinema-terminal .fg-163 { + color: #d700af; +} +.asciinema-terminal .bg-163 { + background-color: #d700af; +} +.asciinema-terminal .fg-164 { + color: #d700d7; +} +.asciinema-terminal .bg-164 { + background-color: #d700d7; +} +.asciinema-terminal .fg-165 { + color: #d700ff; +} +.asciinema-terminal .bg-165 { + background-color: #d700ff; +} +.asciinema-terminal .fg-166 { + color: #d75f00; +} +.asciinema-terminal .bg-166 { + background-color: #d75f00; +} +.asciinema-terminal .fg-167 { + color: #d75f5f; +} +.asciinema-terminal .bg-167 { + background-color: #d75f5f; +} +.asciinema-terminal .fg-168 { + color: #d75f87; +} +.asciinema-terminal .bg-168 { + background-color: #d75f87; +} +.asciinema-terminal .fg-169 { + color: #d75faf; +} +.asciinema-terminal .bg-169 { + background-color: #d75faf; +} +.asciinema-terminal .fg-170 { + color: #d75fd7; +} +.asciinema-terminal .bg-170 { + background-color: #d75fd7; +} +.asciinema-terminal .fg-171 { + color: #d75fff; +} +.asciinema-terminal .bg-171 { + background-color: #d75fff; +} +.asciinema-terminal .fg-172 { + color: #d78700; +} +.asciinema-terminal .bg-172 { + background-color: #d78700; +} +.asciinema-terminal .fg-173 { + color: #d7875f; +} +.asciinema-terminal .bg-173 { + background-color: #d7875f; +} +.asciinema-terminal .fg-174 { + color: #d78787; +} +.asciinema-terminal .bg-174 { + background-color: #d78787; +} +.asciinema-terminal .fg-175 { + color: #d787af; +} +.asciinema-terminal .bg-175 { + background-color: #d787af; +} +.asciinema-terminal .fg-176 { + color: #d787d7; +} +.asciinema-terminal .bg-176 { + background-color: #d787d7; +} +.asciinema-terminal .fg-177 { + color: #d787ff; +} +.asciinema-terminal .bg-177 { + background-color: #d787ff; +} +.asciinema-terminal .fg-178 { + color: #d7af00; +} +.asciinema-terminal .bg-178 { + background-color: #d7af00; +} +.asciinema-terminal .fg-179 { + color: #d7af5f; +} +.asciinema-terminal .bg-179 { + background-color: #d7af5f; +} +.asciinema-terminal .fg-180 { + color: #d7af87; +} +.asciinema-terminal .bg-180 { + background-color: #d7af87; +} +.asciinema-terminal .fg-181 { + color: #d7afaf; +} +.asciinema-terminal .bg-181 { + background-color: #d7afaf; +} +.asciinema-terminal .fg-182 { + color: #d7afd7; +} +.asciinema-terminal .bg-182 { + background-color: #d7afd7; +} +.asciinema-terminal .fg-183 { + color: #d7afff; +} +.asciinema-terminal .bg-183 { + background-color: #d7afff; +} +.asciinema-terminal .fg-184 { + color: #d7d700; +} +.asciinema-terminal .bg-184 { + background-color: #d7d700; +} +.asciinema-terminal .fg-185 { + color: #d7d75f; +} +.asciinema-terminal .bg-185 { + background-color: #d7d75f; +} +.asciinema-terminal .fg-186 { + color: #d7d787; +} +.asciinema-terminal .bg-186 { + background-color: #d7d787; +} +.asciinema-terminal .fg-187 { + color: #d7d7af; +} +.asciinema-terminal .bg-187 { + background-color: #d7d7af; +} +.asciinema-terminal .fg-188 { + color: #d7d7d7; +} +.asciinema-terminal .bg-188 { + background-color: #d7d7d7; +} +.asciinema-terminal .fg-189 { + color: #d7d7ff; +} +.asciinema-terminal .bg-189 { + background-color: #d7d7ff; +} +.asciinema-terminal .fg-190 { + color: #d7ff00; +} +.asciinema-terminal .bg-190 { + background-color: #d7ff00; +} +.asciinema-terminal .fg-191 { + color: #d7ff5f; +} +.asciinema-terminal .bg-191 { + background-color: #d7ff5f; +} +.asciinema-terminal .fg-192 { + color: #d7ff87; +} +.asciinema-terminal .bg-192 { + background-color: #d7ff87; +} +.asciinema-terminal .fg-193 { + color: #d7ffaf; +} +.asciinema-terminal .bg-193 { + background-color: #d7ffaf; +} +.asciinema-terminal .fg-194 { + color: #d7ffd7; +} +.asciinema-terminal .bg-194 { + background-color: #d7ffd7; +} +.asciinema-terminal .fg-195 { + color: #d7ffff; +} +.asciinema-terminal .bg-195 { + background-color: #d7ffff; +} +.asciinema-terminal .fg-196 { + color: #ff0000; +} +.asciinema-terminal .bg-196 { + background-color: #ff0000; +} +.asciinema-terminal .fg-197 { + color: #ff005f; +} +.asciinema-terminal .bg-197 { + background-color: #ff005f; +} +.asciinema-terminal .fg-198 { + color: #ff0087; +} +.asciinema-terminal .bg-198 { + background-color: #ff0087; +} +.asciinema-terminal .fg-199 { + color: #ff00af; +} +.asciinema-terminal .bg-199 { + background-color: #ff00af; +} +.asciinema-terminal .fg-200 { + color: #ff00d7; +} +.asciinema-terminal .bg-200 { + background-color: #ff00d7; +} +.asciinema-terminal .fg-201 { + color: #ff00ff; +} +.asciinema-terminal .bg-201 { + background-color: #ff00ff; +} +.asciinema-terminal .fg-202 { + color: #ff5f00; +} +.asciinema-terminal .bg-202 { + background-color: #ff5f00; +} +.asciinema-terminal .fg-203 { + color: #ff5f5f; +} +.asciinema-terminal .bg-203 { + background-color: #ff5f5f; +} +.asciinema-terminal .fg-204 { + color: #ff5f87; +} +.asciinema-terminal .bg-204 { + background-color: #ff5f87; +} +.asciinema-terminal .fg-205 { + color: #ff5faf; +} +.asciinema-terminal .bg-205 { + background-color: #ff5faf; +} +.asciinema-terminal .fg-206 { + color: #ff5fd7; +} +.asciinema-terminal .bg-206 { + background-color: #ff5fd7; +} +.asciinema-terminal .fg-207 { + color: #ff5fff; +} +.asciinema-terminal .bg-207 { + background-color: #ff5fff; +} +.asciinema-terminal .fg-208 { + color: #ff8700; +} +.asciinema-terminal .bg-208 { + background-color: #ff8700; +} +.asciinema-terminal .fg-209 { + color: #ff875f; +} +.asciinema-terminal .bg-209 { + background-color: #ff875f; +} +.asciinema-terminal .fg-210 { + color: #ff8787; +} +.asciinema-terminal .bg-210 { + background-color: #ff8787; +} +.asciinema-terminal .fg-211 { + color: #ff87af; +} +.asciinema-terminal .bg-211 { + background-color: #ff87af; +} +.asciinema-terminal .fg-212 { + color: #ff87d7; +} +.asciinema-terminal .bg-212 { + background-color: #ff87d7; +} +.asciinema-terminal .fg-213 { + color: #ff87ff; +} +.asciinema-terminal .bg-213 { + background-color: #ff87ff; +} +.asciinema-terminal .fg-214 { + color: #ffaf00; +} +.asciinema-terminal .bg-214 { + background-color: #ffaf00; +} +.asciinema-terminal .fg-215 { + color: #ffaf5f; +} +.asciinema-terminal .bg-215 { + background-color: #ffaf5f; +} +.asciinema-terminal .fg-216 { + color: #ffaf87; +} +.asciinema-terminal .bg-216 { + background-color: #ffaf87; +} +.asciinema-terminal .fg-217 { + color: #ffafaf; +} +.asciinema-terminal .bg-217 { + background-color: #ffafaf; +} +.asciinema-terminal .fg-218 { + color: #ffafd7; +} +.asciinema-terminal .bg-218 { + background-color: #ffafd7; +} +.asciinema-terminal .fg-219 { + color: #ffafff; +} +.asciinema-terminal .bg-219 { + background-color: #ffafff; +} +.asciinema-terminal .fg-220 { + color: #ffd700; +} +.asciinema-terminal .bg-220 { + background-color: #ffd700; +} +.asciinema-terminal .fg-221 { + color: #ffd75f; +} +.asciinema-terminal .bg-221 { + background-color: #ffd75f; +} +.asciinema-terminal .fg-222 { + color: #ffd787; +} +.asciinema-terminal .bg-222 { + background-color: #ffd787; +} +.asciinema-terminal .fg-223 { + color: #ffd7af; +} +.asciinema-terminal .bg-223 { + background-color: #ffd7af; +} +.asciinema-terminal .fg-224 { + color: #ffd7d7; +} +.asciinema-terminal .bg-224 { + background-color: #ffd7d7; +} +.asciinema-terminal .fg-225 { + color: #ffd7ff; +} +.asciinema-terminal .bg-225 { + background-color: #ffd7ff; +} +.asciinema-terminal .fg-226 { + color: #ffff00; +} +.asciinema-terminal .bg-226 { + background-color: #ffff00; +} +.asciinema-terminal .fg-227 { + color: #ffff5f; +} +.asciinema-terminal .bg-227 { + background-color: #ffff5f; +} +.asciinema-terminal .fg-228 { + color: #ffff87; +} +.asciinema-terminal .bg-228 { + background-color: #ffff87; +} +.asciinema-terminal .fg-229 { + color: #ffffaf; +} +.asciinema-terminal .bg-229 { + background-color: #ffffaf; +} +.asciinema-terminal .fg-230 { + color: #ffffd7; +} +.asciinema-terminal .bg-230 { + background-color: #ffffd7; +} +.asciinema-terminal .fg-231 { + color: #ffffff; +} +.asciinema-terminal .bg-231 { + background-color: #ffffff; +} +.asciinema-terminal .fg-232 { + color: #080808; +} +.asciinema-terminal .bg-232 { + background-color: #080808; +} +.asciinema-terminal .fg-233 { + color: #121212; +} +.asciinema-terminal .bg-233 { + background-color: #121212; +} +.asciinema-terminal .fg-234 { + color: #1c1c1c; +} +.asciinema-terminal .bg-234 { + background-color: #1c1c1c; +} +.asciinema-terminal .fg-235 { + color: #262626; +} +.asciinema-terminal .bg-235 { + background-color: #262626; +} +.asciinema-terminal .fg-236 { + color: #303030; +} +.asciinema-terminal .bg-236 { + background-color: #303030; +} +.asciinema-terminal .fg-237 { + color: #3a3a3a; +} +.asciinema-terminal .bg-237 { + background-color: #3a3a3a; +} +.asciinema-terminal .fg-238 { + color: #444444; +} +.asciinema-terminal .bg-238 { + background-color: #444444; +} +.asciinema-terminal .fg-239 { + color: #4e4e4e; +} +.asciinema-terminal .bg-239 { + background-color: #4e4e4e; +} +.asciinema-terminal .fg-240 { + color: #585858; +} +.asciinema-terminal .bg-240 { + background-color: #585858; +} +.asciinema-terminal .fg-241 { + color: #626262; +} +.asciinema-terminal .bg-241 { + background-color: #626262; +} +.asciinema-terminal .fg-242 { + color: #6c6c6c; +} +.asciinema-terminal .bg-242 { + background-color: #6c6c6c; +} +.asciinema-terminal .fg-243 { + color: #767676; +} +.asciinema-terminal .bg-243 { + background-color: #767676; +} +.asciinema-terminal .fg-244 { + color: #808080; +} +.asciinema-terminal .bg-244 { + background-color: #808080; +} +.asciinema-terminal .fg-245 { + color: #8a8a8a; +} +.asciinema-terminal .bg-245 { + background-color: #8a8a8a; +} +.asciinema-terminal .fg-246 { + color: #949494; +} +.asciinema-terminal .bg-246 { + background-color: #949494; +} +.asciinema-terminal .fg-247 { + color: #9e9e9e; +} +.asciinema-terminal .bg-247 { + background-color: #9e9e9e; +} +.asciinema-terminal .fg-248 { + color: #a8a8a8; +} +.asciinema-terminal .bg-248 { + background-color: #a8a8a8; +} +.asciinema-terminal .fg-249 { + color: #b2b2b2; +} +.asciinema-terminal .bg-249 { + background-color: #b2b2b2; +} +.asciinema-terminal .fg-250 { + color: #bcbcbc; +} +.asciinema-terminal .bg-250 { + background-color: #bcbcbc; +} +.asciinema-terminal .fg-251 { + color: #c6c6c6; +} +.asciinema-terminal .bg-251 { + background-color: #c6c6c6; +} +.asciinema-terminal .fg-252 { + color: #d0d0d0; +} +.asciinema-terminal .bg-252 { + background-color: #d0d0d0; +} +.asciinema-terminal .fg-253 { + color: #dadada; +} +.asciinema-terminal .bg-253 { + background-color: #dadada; +} +.asciinema-terminal .fg-254 { + color: #e4e4e4; +} +.asciinema-terminal .bg-254 { + background-color: #e4e4e4; +} +.asciinema-terminal .fg-255 { + color: #eeeeee; +} +.asciinema-terminal .bg-255 { + background-color: #eeeeee; +} +.asciinema-theme-asciinema .asciinema-terminal { + color: #cccccc; + background-color: #121314; + border-color: #121314; +} +.asciinema-theme-asciinema .fg-bg { + color: #121314; +} +.asciinema-theme-asciinema .bg-fg { + background-color: #cccccc; +} +.asciinema-theme-asciinema .fg-0 { + color: #000000; +} +.asciinema-theme-asciinema .bg-0 { + background-color: #000000; +} +.asciinema-theme-asciinema .fg-1 { + color: #dd3c69; +} +.asciinema-theme-asciinema .bg-1 { + background-color: #dd3c69; +} +.asciinema-theme-asciinema .fg-2 { + color: #4ebf22; +} +.asciinema-theme-asciinema .bg-2 { + background-color: #4ebf22; +} +.asciinema-theme-asciinema .fg-3 { + color: #ddaf3c; +} +.asciinema-theme-asciinema .bg-3 { + background-color: #ddaf3c; +} +.asciinema-theme-asciinema .fg-4 { + color: #26b0d7; +} +.asciinema-theme-asciinema .bg-4 { + background-color: #26b0d7; +} +.asciinema-theme-asciinema .fg-5 { + color: #b954e1; +} +.asciinema-theme-asciinema .bg-5 { + background-color: #b954e1; +} +.asciinema-theme-asciinema .fg-6 { + color: #54e1b9; +} +.asciinema-theme-asciinema .bg-6 { + background-color: #54e1b9; +} +.asciinema-theme-asciinema .fg-7 { + color: #d9d9d9; +} +.asciinema-theme-asciinema .bg-7 { + background-color: #d9d9d9; +} +.asciinema-theme-asciinema .fg-8 { + color: #4d4d4d; +} +.asciinema-theme-asciinema .bg-8 { + background-color: #4d4d4d; +} +.asciinema-theme-asciinema .fg-9 { + color: #dd3c69; +} +.asciinema-theme-asciinema .bg-9 { + background-color: #dd3c69; +} +.asciinema-theme-asciinema .fg-10 { + color: #4ebf22; +} +.asciinema-theme-asciinema .bg-10 { + background-color: #4ebf22; +} +.asciinema-theme-asciinema .fg-11 { + color: #ddaf3c; +} +.asciinema-theme-asciinema .bg-11 { + background-color: #ddaf3c; +} +.asciinema-theme-asciinema .fg-12 { + color: #26b0d7; +} +.asciinema-theme-asciinema .bg-12 { + background-color: #26b0d7; +} +.asciinema-theme-asciinema .fg-13 { + color: #b954e1; +} +.asciinema-theme-asciinema .bg-13 { + background-color: #b954e1; +} +.asciinema-theme-asciinema .fg-14 { + color: #54e1b9; +} +.asciinema-theme-asciinema .bg-14 { + background-color: #54e1b9; +} +.asciinema-theme-asciinema .fg-15 { + color: #ffffff; +} +.asciinema-theme-asciinema .bg-15 { + background-color: #ffffff; +} +.asciinema-theme-asciinema .fg-8, +.asciinema-theme-asciinema .fg-9, +.asciinema-theme-asciinema .fg-10, +.asciinema-theme-asciinema .fg-11, +.asciinema-theme-asciinema .fg-12, +.asciinema-theme-asciinema .fg-13, +.asciinema-theme-asciinema .fg-14, +.asciinema-theme-asciinema .fg-15 { + font-weight: bold; +} +.asciinema-theme-tango .asciinema-terminal { + color: #cccccc; + background-color: #121314; + border-color: #121314; +} +.asciinema-theme-tango .fg-bg { + color: #121314; +} +.asciinema-theme-tango .bg-fg { + background-color: #cccccc; +} +.asciinema-theme-tango .fg-0 { + color: #000000; +} +.asciinema-theme-tango .bg-0 { + background-color: #000000; +} +.asciinema-theme-tango .fg-1 { + color: #cc0000; +} +.asciinema-theme-tango .bg-1 { + background-color: #cc0000; +} +.asciinema-theme-tango .fg-2 { + color: #4e9a06; +} +.asciinema-theme-tango .bg-2 { + background-color: #4e9a06; +} +.asciinema-theme-tango .fg-3 { + color: #c4a000; +} +.asciinema-theme-tango .bg-3 { + background-color: #c4a000; +} +.asciinema-theme-tango .fg-4 { + color: #3465a4; +} +.asciinema-theme-tango .bg-4 { + background-color: #3465a4; +} +.asciinema-theme-tango .fg-5 { + color: #75507b; +} +.asciinema-theme-tango .bg-5 { + background-color: #75507b; +} +.asciinema-theme-tango .fg-6 { + color: #06989a; +} +.asciinema-theme-tango .bg-6 { + background-color: #06989a; +} +.asciinema-theme-tango .fg-7 { + color: #d3d7cf; +} +.asciinema-theme-tango .bg-7 { + background-color: #d3d7cf; +} +.asciinema-theme-tango .fg-8 { + color: #555753; +} +.asciinema-theme-tango .bg-8 { + background-color: #555753; +} +.asciinema-theme-tango .fg-9 { + color: #ef2929; +} +.asciinema-theme-tango .bg-9 { + background-color: #ef2929; +} +.asciinema-theme-tango .fg-10 { + color: #8ae234; +} +.asciinema-theme-tango .bg-10 { + background-color: #8ae234; +} +.asciinema-theme-tango .fg-11 { + color: #fce94f; +} +.asciinema-theme-tango .bg-11 { + background-color: #fce94f; +} +.asciinema-theme-tango .fg-12 { + color: #729fcf; +} +.asciinema-theme-tango .bg-12 { + background-color: #729fcf; +} +.asciinema-theme-tango .fg-13 { + color: #ad7fa8; +} +.asciinema-theme-tango .bg-13 { + background-color: #ad7fa8; +} +.asciinema-theme-tango .fg-14 { + color: #34e2e2; +} +.asciinema-theme-tango .bg-14 { + background-color: #34e2e2; +} +.asciinema-theme-tango .fg-15 { + color: #eeeeec; +} +.asciinema-theme-tango .bg-15 { + background-color: #eeeeec; +} +.asciinema-theme-tango .fg-8, +.asciinema-theme-tango .fg-9, +.asciinema-theme-tango .fg-10, +.asciinema-theme-tango .fg-11, +.asciinema-theme-tango .fg-12, +.asciinema-theme-tango .fg-13, +.asciinema-theme-tango .fg-14, +.asciinema-theme-tango .fg-15 { + font-weight: bold; +} +.asciinema-theme-solarized-dark .asciinema-terminal { + color: #839496; + background-color: #002b36; + border-color: #002b36; +} +.asciinema-theme-solarized-dark .fg-bg { + color: #002b36; +} +.asciinema-theme-solarized-dark .bg-fg { + background-color: #839496; +} +.asciinema-theme-solarized-dark .fg-0 { + color: #073642; +} +.asciinema-theme-solarized-dark .bg-0 { + background-color: #073642; +} +.asciinema-theme-solarized-dark .fg-1 { + color: #dc322f; +} +.asciinema-theme-solarized-dark .bg-1 { + background-color: #dc322f; +} +.asciinema-theme-solarized-dark .fg-2 { + color: #859900; +} +.asciinema-theme-solarized-dark .bg-2 { + background-color: #859900; +} +.asciinema-theme-solarized-dark .fg-3 { + color: #b58900; +} +.asciinema-theme-solarized-dark .bg-3 { + background-color: #b58900; +} +.asciinema-theme-solarized-dark .fg-4 { + color: #268bd2; +} +.asciinema-theme-solarized-dark .bg-4 { + background-color: #268bd2; +} +.asciinema-theme-solarized-dark .fg-5 { + color: #d33682; +} +.asciinema-theme-solarized-dark .bg-5 { + background-color: #d33682; +} +.asciinema-theme-solarized-dark .fg-6 { + color: #2aa198; +} +.asciinema-theme-solarized-dark .bg-6 { + background-color: #2aa198; +} +.asciinema-theme-solarized-dark .fg-7 { + color: #eee8d5; +} +.asciinema-theme-solarized-dark .bg-7 { + background-color: #eee8d5; +} +.asciinema-theme-solarized-dark .fg-8 { + color: #002b36; +} +.asciinema-theme-solarized-dark .bg-8 { + background-color: #002b36; +} +.asciinema-theme-solarized-dark .fg-9 { + color: #cb4b16; +} +.asciinema-theme-solarized-dark .bg-9 { + background-color: #cb4b16; +} +.asciinema-theme-solarized-dark .fg-10 { + color: #586e75; +} +.asciinema-theme-solarized-dark .bg-10 { + background-color: #586e75; +} +.asciinema-theme-solarized-dark .fg-11 { + color: #657b83; +} +.asciinema-theme-solarized-dark .bg-11 { + background-color: #657b83; +} +.asciinema-theme-solarized-dark .fg-12 { + color: #839496; +} +.asciinema-theme-solarized-dark .bg-12 { + background-color: #839496; +} +.asciinema-theme-solarized-dark .fg-13 { + color: #6c71c4; +} +.asciinema-theme-solarized-dark .bg-13 { + background-color: #6c71c4; +} +.asciinema-theme-solarized-dark .fg-14 { + color: #93a1a1; +} +.asciinema-theme-solarized-dark .bg-14 { + background-color: #93a1a1; +} +.asciinema-theme-solarized-dark .fg-15 { + color: #fdf6e3; +} +.asciinema-theme-solarized-dark .bg-15 { + background-color: #fdf6e3; +} +.asciinema-theme-solarized-light .asciinema-terminal { + color: #657b83; + background-color: #fdf6e3; + border-color: #fdf6e3; +} +.asciinema-theme-solarized-light .fg-bg { + color: #fdf6e3; +} +.asciinema-theme-solarized-light .bg-fg { + background-color: #657b83; +} +.asciinema-theme-solarized-light .fg-0 { + color: #073642; +} +.asciinema-theme-solarized-light .bg-0 { + background-color: #073642; +} +.asciinema-theme-solarized-light .fg-1 { + color: #dc322f; +} +.asciinema-theme-solarized-light .bg-1 { + background-color: #dc322f; +} +.asciinema-theme-solarized-light .fg-2 { + color: #859900; +} +.asciinema-theme-solarized-light .bg-2 { + background-color: #859900; +} +.asciinema-theme-solarized-light .fg-3 { + color: #b58900; +} +.asciinema-theme-solarized-light .bg-3 { + background-color: #b58900; +} +.asciinema-theme-solarized-light .fg-4 { + color: #268bd2; +} +.asciinema-theme-solarized-light .bg-4 { + background-color: #268bd2; +} +.asciinema-theme-solarized-light .fg-5 { + color: #d33682; +} +.asciinema-theme-solarized-light .bg-5 { + background-color: #d33682; +} +.asciinema-theme-solarized-light .fg-6 { + color: #2aa198; +} +.asciinema-theme-solarized-light .bg-6 { + background-color: #2aa198; +} +.asciinema-theme-solarized-light .fg-7 { + color: #eee8d5; +} +.asciinema-theme-solarized-light .bg-7 { + background-color: #eee8d5; +} +.asciinema-theme-solarized-light .fg-8 { + color: #002b36; +} +.asciinema-theme-solarized-light .bg-8 { + background-color: #002b36; +} +.asciinema-theme-solarized-light .fg-9 { + color: #cb4b16; +} +.asciinema-theme-solarized-light .bg-9 { + background-color: #cb4b16; +} +.asciinema-theme-solarized-light .fg-10 { + color: #586e75; +} +.asciinema-theme-solarized-light .bg-10 { + background-color: #586e75; +} +.asciinema-theme-solarized-light .fg-11 { + color: #657c83; +} +.asciinema-theme-solarized-light .bg-11 { + background-color: #657c83; +} +.asciinema-theme-solarized-light .fg-12 { + color: #839496; +} +.asciinema-theme-solarized-light .bg-12 { + background-color: #839496; +} +.asciinema-theme-solarized-light .fg-13 { + color: #6c71c4; +} +.asciinema-theme-solarized-light .bg-13 { + background-color: #6c71c4; +} +.asciinema-theme-solarized-light .fg-14 { + color: #93a1a1; +} +.asciinema-theme-solarized-light .bg-14 { + background-color: #93a1a1; +} +.asciinema-theme-solarized-light .fg-15 { + color: #fdf6e3; +} +.asciinema-theme-solarized-light .bg-15 { + background-color: #fdf6e3; +} +.asciinema-theme-seti .asciinema-terminal { + color: #cacecd; + background-color: #111213; + border-color: #111213; +} +.asciinema-theme-seti .fg-bg { + color: #111213; +} +.asciinema-theme-seti .bg-fg { + background-color: #cacecd; +} +.asciinema-theme-seti .fg-0 { + color: #323232; +} +.asciinema-theme-seti .bg-0 { + background-color: #323232; +} +.asciinema-theme-seti .fg-1 { + color: #c22832; +} +.asciinema-theme-seti .bg-1 { + background-color: #c22832; +} +.asciinema-theme-seti .fg-2 { + color: #8ec43d; +} +.asciinema-theme-seti .bg-2 { + background-color: #8ec43d; +} +.asciinema-theme-seti .fg-3 { + color: #e0c64f; +} +.asciinema-theme-seti .bg-3 { + background-color: #e0c64f; +} +.asciinema-theme-seti .fg-4 { + color: #43a5d5; +} +.asciinema-theme-seti .bg-4 { + background-color: #43a5d5; +} +.asciinema-theme-seti .fg-5 { + color: #8b57b5; +} +.asciinema-theme-seti .bg-5 { + background-color: #8b57b5; +} +.asciinema-theme-seti .fg-6 { + color: #8ec43d; +} +.asciinema-theme-seti .bg-6 { + background-color: #8ec43d; +} +.asciinema-theme-seti .fg-7 { + color: #eeeeee; +} +.asciinema-theme-seti .bg-7 { + background-color: #eeeeee; +} +.asciinema-theme-seti .fg-8 { + color: #323232; +} +.asciinema-theme-seti .bg-8 { + background-color: #323232; +} +.asciinema-theme-seti .fg-9 { + color: #c22832; +} +.asciinema-theme-seti .bg-9 { + background-color: #c22832; +} +.asciinema-theme-seti .fg-10 { + color: #8ec43d; +} +.asciinema-theme-seti .bg-10 { + background-color: #8ec43d; +} +.asciinema-theme-seti .fg-11 { + color: #e0c64f; +} +.asciinema-theme-seti .bg-11 { + background-color: #e0c64f; +} +.asciinema-theme-seti .fg-12 { + color: #43a5d5; +} +.asciinema-theme-seti .bg-12 { + background-color: #43a5d5; +} +.asciinema-theme-seti .fg-13 { + color: #8b57b5; +} +.asciinema-theme-seti .bg-13 { + background-color: #8b57b5; +} +.asciinema-theme-seti .fg-14 { + color: #8ec43d; +} +.asciinema-theme-seti .bg-14 { + background-color: #8ec43d; +} +.asciinema-theme-seti .fg-15 { + color: #ffffff; +} +.asciinema-theme-seti .bg-15 { + background-color: #ffffff; +} +.asciinema-theme-seti .fg-8, +.asciinema-theme-seti .fg-9, +.asciinema-theme-seti .fg-10, +.asciinema-theme-seti .fg-11, +.asciinema-theme-seti .fg-12, +.asciinema-theme-seti .fg-13, +.asciinema-theme-seti .fg-14, +.asciinema-theme-seti .fg-15 { + font-weight: bold; +} +/* Based on Monokai from base16 collection - https://github.com/chriskempson/base16 */ +.asciinema-theme-monokai .asciinema-terminal { + color: #f8f8f2; + background-color: #272822; + border-color: #272822; +} +.asciinema-theme-monokai .fg-bg { + color: #272822; +} +.asciinema-theme-monokai .bg-fg { + background-color: #f8f8f2; +} +.asciinema-theme-monokai .fg-0 { + color: #272822; +} +.asciinema-theme-monokai .bg-0 { + background-color: #272822; +} +.asciinema-theme-monokai .fg-1 { + color: #f92672; +} +.asciinema-theme-monokai .bg-1 { + background-color: #f92672; +} +.asciinema-theme-monokai .fg-2 { + color: #a6e22e; +} +.asciinema-theme-monokai .bg-2 { + background-color: #a6e22e; +} +.asciinema-theme-monokai .fg-3 { + color: #f4bf75; +} +.asciinema-theme-monokai .bg-3 { + background-color: #f4bf75; +} +.asciinema-theme-monokai .fg-4 { + color: #66d9ef; +} +.asciinema-theme-monokai .bg-4 { + background-color: #66d9ef; +} +.asciinema-theme-monokai .fg-5 { + color: #ae81ff; +} +.asciinema-theme-monokai .bg-5 { + background-color: #ae81ff; +} +.asciinema-theme-monokai .fg-6 { + color: #a1efe4; +} +.asciinema-theme-monokai .bg-6 { + background-color: #a1efe4; +} +.asciinema-theme-monokai .fg-7 { + color: #f8f8f2; +} +.asciinema-theme-monokai .bg-7 { + background-color: #f8f8f2; +} +.asciinema-theme-monokai .fg-8 { + color: #75715e; +} +.asciinema-theme-monokai .bg-8 { + background-color: #75715e; +} +.asciinema-theme-monokai .fg-9 { + color: #f92672; +} +.asciinema-theme-monokai .bg-9 { + background-color: #f92672; +} +.asciinema-theme-monokai .fg-10 { + color: #a6e22e; +} +.asciinema-theme-monokai .bg-10 { + background-color: #a6e22e; +} +.asciinema-theme-monokai .fg-11 { + color: #f4bf75; +} +.asciinema-theme-monokai .bg-11 { + background-color: #f4bf75; +} +.asciinema-theme-monokai .fg-12 { + color: #66d9ef; +} +.asciinema-theme-monokai .bg-12 { + background-color: #66d9ef; +} +.asciinema-theme-monokai .fg-13 { + color: #ae81ff; +} +.asciinema-theme-monokai .bg-13 { + background-color: #ae81ff; +} +.asciinema-theme-monokai .fg-14 { + color: #a1efe4; +} +.asciinema-theme-monokai .bg-14 { + background-color: #a1efe4; +} +.asciinema-theme-monokai .fg-15 { + color: #f9f8f5; +} +.asciinema-theme-monokai .bg-15 { + background-color: #f9f8f5; +} +.asciinema-theme-monokai .fg-8, +.asciinema-theme-monokai .fg-9, +.asciinema-theme-monokai .fg-10, +.asciinema-theme-monokai .fg-11, +.asciinema-theme-monokai .fg-12, +.asciinema-theme-monokai .fg-13, +.asciinema-theme-monokai .fg-14, +.asciinema-theme-monokai .fg-15 { + font-weight: bold; +} diff -Nru mitmproxy-5.1.1/docs/src/assets/asciinema-player.js mitmproxy-6.0.2/docs/src/assets/asciinema-player.js --- mitmproxy-5.1.1/docs/src/assets/asciinema-player.js 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/assets/asciinema-player.js 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,1213 @@ +/** + * asciinema-player v2.6.1 + * + * Copyright 2011-2018, Marcin Kulik + * + */ + +// CustomEvent polyfill from MDN (https://developer.mozilla.org/en-US/docs/Web/API/CustomEvent/CustomEvent) + +(function () { + if (typeof window.CustomEvent === "function") return false; + + function CustomEvent ( event, params ) { + params = params || { bubbles: false, cancelable: false, detail: undefined }; + var evt = document.createEvent( 'CustomEvent'); + evt.initCustomEvent(event, params.bubbles, params.cancelable, params.detail); + return evt; + } + + CustomEvent.prototype = window.Event.prototype; + + window.CustomEvent = CustomEvent; +})(); + +/** + * @license + * Copyright (c) 2014 The Polymer Project Authors. All rights reserved. + * This code may only be used under the BSD style license found at http://polymer.github.io/LICENSE.txt + * The complete set of authors may be found at http://polymer.github.io/AUTHORS.txt + * The complete set of contributors may be found at http://polymer.github.io/CONTRIBUTORS.txt + * Code distributed by Google as part of the polymer project is also + * subject to an additional IP rights grant found at http://polymer.github.io/PATENTS.txt + */ +// @version 0.7.22 +"undefined"==typeof WeakMap&&!function(){var e=Object.defineProperty,t=Date.now()%1e9,n=function(){this.name="__st"+(1e9*Math.random()>>>0)+(t++ +"__")};n.prototype={set:function(t,n){var o=t[this.name];return o&&o[0]===t?o[1]=n:e(t,this.name,{value:[t,n],writable:!0}),this},get:function(e){var t;return(t=e[this.name])&&t[0]===e?t[1]:void 0},"delete":function(e){var t=e[this.name];return t&&t[0]===e?(t[0]=t[1]=void 0,!0):!1},has:function(e){var t=e[this.name];return t?t[0]===e:!1}},window.WeakMap=n}(),function(e){function t(e){E.push(e),b||(b=!0,w(o))}function n(e){return window.ShadowDOMPolyfill&&window.ShadowDOMPolyfill.wrapIfNeeded(e)||e}function o(){b=!1;var e=E;E=[],e.sort(function(e,t){return e.uid_-t.uid_});var t=!1;e.forEach(function(e){var n=e.takeRecords();r(e),n.length&&(e.callback_(n,e),t=!0)}),t&&o()}function r(e){e.nodes_.forEach(function(t){var n=v.get(t);n&&n.forEach(function(t){t.observer===e&&t.removeTransientObservers()})})}function i(e,t){for(var n=e;n;n=n.parentNode){var o=v.get(n);if(o)for(var r=0;r0){var r=n[o-1],i=p(r,e);if(i)return void(n[o-1]=i)}else t(this.observer);n[o]=e},addListeners:function(){this.addListeners_(this.target)},addListeners_:function(e){var t=this.options;t.attributes&&e.addEventListener("DOMAttrModified",this,!0),t.characterData&&e.addEventListener("DOMCharacterDataModified",this,!0),t.childList&&e.addEventListener("DOMNodeInserted",this,!0),(t.childList||t.subtree)&&e.addEventListener("DOMNodeRemoved",this,!0)},removeListeners:function(){this.removeListeners_(this.target)},removeListeners_:function(e){var t=this.options;t.attributes&&e.removeEventListener("DOMAttrModified",this,!0),t.characterData&&e.removeEventListener("DOMCharacterDataModified",this,!0),t.childList&&e.removeEventListener("DOMNodeInserted",this,!0),(t.childList||t.subtree)&&e.removeEventListener("DOMNodeRemoved",this,!0)},addTransientObserver:function(e){if(e!==this.target){this.addListeners_(e),this.transientObservedNodes.push(e);var t=v.get(e);t||v.set(e,t=[]),t.push(this)}},removeTransientObservers:function(){var e=this.transientObservedNodes;this.transientObservedNodes=[],e.forEach(function(e){this.removeListeners_(e);for(var t=v.get(e),n=0;n=0)){n.push(e);for(var o,r=e.querySelectorAll("link[rel="+a+"]"),d=0,s=r.length;s>d&&(o=r[d]);d++)o["import"]&&i(o["import"],t,n);t(e)}}var a=window.HTMLImports?window.HTMLImports.IMPORT_LINK_TYPE:"none";e.forDocumentTree=r,e.forSubtree=t}),window.CustomElements.addModule(function(e){function t(e,t){return n(e,t)||o(e,t)}function n(t,n){return e.upgrade(t,n)?!0:void(n&&a(t))}function o(e,t){b(e,function(e){return n(e,t)?!0:void 0})}function r(e){N.push(e),y||(y=!0,setTimeout(i))}function i(){y=!1;for(var e,t=N,n=0,o=t.length;o>n&&(e=t[n]);n++)e();N=[]}function a(e){_?r(function(){d(e)}):d(e)}function d(e){e.__upgraded__&&!e.__attached&&(e.__attached=!0,e.attachedCallback&&e.attachedCallback())}function s(e){u(e),b(e,function(e){u(e)})}function u(e){_?r(function(){c(e)}):c(e)}function c(e){e.__upgraded__&&e.__attached&&(e.__attached=!1,e.detachedCallback&&e.detachedCallback())}function l(e){for(var t=e,n=window.wrap(document);t;){if(t==n)return!0;t=t.parentNode||t.nodeType===Node.DOCUMENT_FRAGMENT_NODE&&t.host}}function f(e){if(e.shadowRoot&&!e.shadowRoot.__watched){g.dom&&console.log("watching shadow-root for: ",e.localName);for(var t=e.shadowRoot;t;)w(t),t=t.olderShadowRoot}}function p(e,n){if(g.dom){var o=n[0];if(o&&"childList"===o.type&&o.addedNodes&&o.addedNodes){for(var r=o.addedNodes[0];r&&r!==document&&!r.host;)r=r.parentNode;var i=r&&(r.URL||r._URL||r.host&&r.host.localName)||"";i=i.split("/?").shift().split("/").pop()}console.group("mutations (%d) [%s]",n.length,i||"")}var a=l(e);n.forEach(function(e){"childList"===e.type&&(M(e.addedNodes,function(e){e.localName&&t(e,a)}),M(e.removedNodes,function(e){e.localName&&s(e)}))}),g.dom&&console.groupEnd()}function m(e){for(e=window.wrap(e),e||(e=window.wrap(document));e.parentNode;)e=e.parentNode;var t=e.__observer;t&&(p(e,t.takeRecords()),i())}function w(e){if(!e.__observer){var t=new MutationObserver(p.bind(this,e));t.observe(e,{childList:!0,subtree:!0}),e.__observer=t}}function v(e){e=window.wrap(e),g.dom&&console.group("upgradeDocument: ",e.baseURI.split("/").pop());var n=e===window.wrap(document);t(e,n),w(e),g.dom&&console.groupEnd()}function h(e){E(e,v)}var g=e.flags,b=e.forSubtree,E=e.forDocumentTree,_=window.MutationObserver._isPolyfilled&&g["throttle-attached"];e.hasPolyfillMutations=_,e.hasThrottledAttached=_;var y=!1,N=[],M=Array.prototype.forEach.call.bind(Array.prototype.forEach),O=Element.prototype.createShadowRoot;O&&(Element.prototype.createShadowRoot=function(){var e=O.call(this);return window.CustomElements.watchShadow(this),e}),e.watchShadow=f,e.upgradeDocumentTree=h,e.upgradeDocument=v,e.upgradeSubtree=o,e.upgradeAll=t,e.attached=a,e.takeRecords=m}),window.CustomElements.addModule(function(e){function t(t,o){if("template"===t.localName&&window.HTMLTemplateElement&&HTMLTemplateElement.decorate&&HTMLTemplateElement.decorate(t),!t.__upgraded__&&t.nodeType===Node.ELEMENT_NODE){var r=t.getAttribute("is"),i=e.getRegisteredDefinition(t.localName)||e.getRegisteredDefinition(r);if(i&&(r&&i.tag==t.localName||!r&&!i["extends"]))return n(t,i,o)}}function n(t,n,r){return a.upgrade&&console.group("upgrade:",t.localName),n.is&&t.setAttribute("is",n.is),o(t,n),t.__upgraded__=!0,i(t),r&&e.attached(t),e.upgradeSubtree(t,r),a.upgrade&&console.groupEnd(),t}function o(e,t){Object.__proto__?e.__proto__=t.prototype:(r(e,t.prototype,t["native"]),e.__proto__=t.prototype)}function r(e,t,n){for(var o={},r=t;r!==n&&r!==HTMLElement.prototype;){for(var i,a=Object.getOwnPropertyNames(r),d=0;i=a[d];d++)o[i]||(Object.defineProperty(e,i,Object.getOwnPropertyDescriptor(r,i)),o[i]=1);r=Object.getPrototypeOf(r)}}function i(e){e.createdCallback&&e.createdCallback()}var a=e.flags;e.upgrade=t,e.upgradeWithDefinition=n,e.implementPrototype=o}),window.CustomElements.addModule(function(e){function t(t,o){var s=o||{};if(!t)throw new Error("document.registerElement: first argument `name` must not be empty");if(t.indexOf("-")<0)throw new Error("document.registerElement: first argument ('name') must contain a dash ('-'). Argument provided was '"+String(t)+"'.");if(r(t))throw new Error("Failed to execute 'registerElement' on 'Document': Registration failed for type '"+String(t)+"'. The type name is invalid.");if(u(t))throw new Error("DuplicateDefinitionError: a type with name '"+String(t)+"' is already registered");return s.prototype||(s.prototype=Object.create(HTMLElement.prototype)),s.__name=t.toLowerCase(),s["extends"]&&(s["extends"]=s["extends"].toLowerCase()),s.lifecycle=s.lifecycle||{},s.ancestry=i(s["extends"]),a(s),d(s),n(s.prototype),c(s.__name,s),s.ctor=l(s),s.ctor.prototype=s.prototype,s.prototype.constructor=s.ctor,e.ready&&v(document),s.ctor}function n(e){if(!e.setAttribute._polyfilled){var t=e.setAttribute;e.setAttribute=function(e,n){o.call(this,e,n,t)};var n=e.removeAttribute;e.removeAttribute=function(e){o.call(this,e,null,n)},e.setAttribute._polyfilled=!0}}function o(e,t,n){e=e.toLowerCase();var o=this.getAttribute(e);n.apply(this,arguments);var r=this.getAttribute(e);this.attributeChangedCallback&&r!==o&&this.attributeChangedCallback(e,o,r)}function r(e){for(var t=0;t<_.length;t++)if(e===_[t])return!0}function i(e){var t=u(e);return t?i(t["extends"]).concat([t]):[]}function a(e){for(var t,n=e["extends"],o=0;t=e.ancestry[o];o++)n=t.is&&t.tag;e.tag=n||e.__name,n&&(e.is=e.__name)}function d(e){if(!Object.__proto__){var t=HTMLElement.prototype;if(e.is){var n=document.createElement(e.tag);t=Object.getPrototypeOf(n)}for(var o,r=e.prototype,i=!1;r;)r==t&&(i=!0),o=Object.getPrototypeOf(r),o&&(r.__proto__=o),r=o;i||console.warn(e.tag+" prototype not found in prototype chain for "+e.is),e["native"]=t}}function s(e){return g(M(e.tag),e)}function u(e){return e?y[e.toLowerCase()]:void 0}function c(e,t){y[e]=t}function l(e){return function(){return s(e)}}function f(e,t,n){return e===N?p(t,n):O(e,t)}function p(e,t){e&&(e=e.toLowerCase()),t&&(t=t.toLowerCase());var n=u(t||e);if(n){if(e==n.tag&&t==n.is)return new n.ctor;if(!t&&!n.is)return new n.ctor}var o;return t?(o=p(e),o.setAttribute("is",t),o):(o=M(e),e.indexOf("-")>=0&&b(o,HTMLElement),o)}function m(e,t){var n=e[t];e[t]=function(){var e=n.apply(this,arguments);return h(e),e}}var w,v=(e.isIE,e.upgradeDocumentTree),h=e.upgradeAll,g=e.upgradeWithDefinition,b=e.implementPrototype,E=e.useNative,_=["annotation-xml","color-profile","font-face","font-face-src","font-face-uri","font-face-format","font-face-name","missing-glyph"],y={},N="http://www.w3.org/1999/xhtml",M=document.createElement.bind(document),O=document.createElementNS.bind(document);w=Object.__proto__||E?function(e,t){return e instanceof t}:function(e,t){if(e instanceof t)return!0;for(var n=e;n;){if(n===t.prototype)return!0;n=n.__proto__}return!1},m(Node.prototype,"cloneNode"),m(document,"importNode"),document.registerElement=t,document.createElement=p,document.createElementNS=f,e.registry=y,e["instanceof"]=w,e.reservedTagList=_,e.getRegisteredDefinition=u,document.register=document.registerElement}),function(e){function t(){i(window.wrap(document)),window.CustomElements.ready=!0;var e=window.requestAnimationFrame||function(e){setTimeout(e,16)};e(function(){setTimeout(function(){window.CustomElements.readyTime=Date.now(),window.HTMLImports&&(window.CustomElements.elapsed=window.CustomElements.readyTime-window.HTMLImports.readyTime),document.dispatchEvent(new CustomEvent("WebComponentsReady",{bubbles:!0}))})})}var n=e.useNative,o=e.initializeModules;e.isIE;if(n){var r=function(){};e.watchShadow=r,e.upgrade=r,e.upgradeAll=r,e.upgradeDocumentTree=r,e.upgradeSubtree=r,e.takeRecords=r,e["instanceof"]=function(e,t){return e instanceof t}}else o();var i=e.upgradeDocumentTree,a=e.upgradeDocument;if(window.wrap||(window.ShadowDOMPolyfill?(window.wrap=window.ShadowDOMPolyfill.wrapIfNeeded,window.unwrap=window.ShadowDOMPolyfill.unwrapIfNeeded):window.wrap=window.unwrap=function(e){return e}),window.HTMLImports&&(window.HTMLImports.__importsParsingHook=function(e){e["import"]&&a(wrap(e["import"]))}),"complete"===document.readyState||e.flags.eager)t();else if("interactive"!==document.readyState||window.attachEvent||window.HTMLImports&&!window.HTMLImports.ready){var d=window.HTMLImports&&!window.HTMLImports.ready?"HTMLImportsLoaded":"DOMContentLoaded";window.addEventListener(d,t)}else t()}(window.CustomElements); +if(typeof Math.imul == "undefined" || (Math.imul(0xffffffff,5) == 0)) { + Math.imul = function (a, b) { + var ah = (a >>> 16) & 0xffff; + var al = a & 0xffff; + var bh = (b >>> 16) & 0xffff; + var bl = b & 0xffff; + // the shift by 0 fixes the sign on the high part + // the final |0 converts the unsigned value into a signed value + return ((al * bl) + (((ah * bl + al * bh) << 16) >>> 0)|0); + } +} + +/** + * React v15.5.4 + * + * Copyright 2013-present, Facebook, Inc. + * All rights reserved. + * + * This source code is licensed under the BSD-style license found in the + * LICENSE file in the root directory of this source tree. An additional grant + * of patent rights can be found in the PATENTS file in the same directory. + * + */ +!function(t){if("object"==typeof exports&&"undefined"!=typeof module)module.exports=t();else if("function"==typeof define&&define.amd)define([],t);else{var e;e="undefined"!=typeof window?window:"undefined"!=typeof global?global:"undefined"!=typeof self?self:this,e.React=t()}}(function(){return function t(e,n,r){function o(u,a){if(!n[u]){if(!e[u]){var s="function"==typeof require&&require;if(!a&&s)return s(u,!0);if(i)return i(u,!0);var c=new Error("Cannot find module '"+u+"'");throw c.code="MODULE_NOT_FOUND",c}var l=n[u]={exports:{}};e[u][0].call(l.exports,function(t){var n=e[u][1][t];return o(n||t)},l,l.exports,t,e,n,r)}return n[u].exports}for(var i="function"==typeof require&&require,u=0;u1){for(var y=Array(d),h=0;h1){for(var m=Array(v),b=0;b8&&C<=11),x=32,w=String.fromCharCode(x),T={beforeInput:{phasedRegistrationNames:{bubbled:"onBeforeInput",captured:"onBeforeInputCapture"},dependencies:["topCompositionEnd","topKeyPress","topTextInput","topPaste"]},compositionEnd:{phasedRegistrationNames:{bubbled:"onCompositionEnd",captured:"onCompositionEndCapture"},dependencies:["topBlur","topCompositionEnd","topKeyDown","topKeyPress","topKeyUp","topMouseDown"]},compositionStart:{phasedRegistrationNames:{bubbled:"onCompositionStart",captured:"onCompositionStartCapture"},dependencies:["topBlur","topCompositionStart","topKeyDown","topKeyPress","topKeyUp","topMouseDown"]},compositionUpdate:{phasedRegistrationNames:{bubbled:"onCompositionUpdate",captured:"onCompositionUpdateCapture"},dependencies:["topBlur","topCompositionUpdate","topKeyDown","topKeyPress","topKeyUp","topMouseDown"]}},k=!1,P=null,S={eventTypes:T,extractEvents:function(e,t,n,r){return[u(e,t,n,r),p(e,t,n,r)]}};t.exports=S},{123:123,19:19,20:20,78:78,82:82}],4:[function(e,t,n){"use strict";function r(e,t){return e+t.charAt(0).toUpperCase()+t.substring(1)}var o={animationIterationCount:!0,borderImageOutset:!0,borderImageSlice:!0,borderImageWidth:!0,boxFlex:!0,boxFlexGroup:!0,boxOrdinalGroup:!0,columnCount:!0,flex:!0,flexGrow:!0,flexPositive:!0,flexShrink:!0,flexNegative:!0,flexOrder:!0,gridRow:!0,gridColumn:!0,fontWeight:!0,lineClamp:!0,lineHeight:!0,opacity:!0,order:!0,orphans:!0,tabSize:!0,widows:!0,zIndex:!0,zoom:!0,fillOpacity:!0,floodOpacity:!0,stopOpacity:!0,strokeDasharray:!0,strokeDashoffset:!0,strokeMiterlimit:!0,strokeOpacity:!0,strokeWidth:!0},i=["Webkit","ms","Moz","O"];Object.keys(o).forEach(function(e){i.forEach(function(t){o[r(t,e)]=o[e]})});var a={background:{backgroundAttachment:!0,backgroundColor:!0,backgroundImage:!0,backgroundPositionX:!0,backgroundPositionY:!0,backgroundRepeat:!0},backgroundPosition:{backgroundPositionX:!0,backgroundPositionY:!0},border:{borderWidth:!0,borderStyle:!0,borderColor:!0},borderBottom:{borderBottomWidth:!0,borderBottomStyle:!0,borderBottomColor:!0},borderLeft:{borderLeftWidth:!0,borderLeftStyle:!0,borderLeftColor:!0},borderRight:{borderRightWidth:!0,borderRightStyle:!0,borderRightColor:!0},borderTop:{borderTopWidth:!0,borderTopStyle:!0,borderTopColor:!0},font:{fontStyle:!0,fontVariant:!0,fontWeight:!0,fontSize:!0,lineHeight:!0,fontFamily:!0},outline:{outlineWidth:!0,outlineStyle:!0,outlineColor:!0}},s={isUnitlessNumber:o,shorthandPropertyExpansions:a};t.exports=s},{}],5:[function(e,t,n){"use strict";var r=e(4),o=e(123),i=(e(58),e(125),e(94)),a=e(136),s=e(140),u=(e(142),s(function(e){return a(e)})),l=!1,c="cssFloat";if(o.canUseDOM){var p=document.createElement("div").style;try{p.font=""}catch(e){l=!0}void 0===document.documentElement.style.cssFloat&&(c="styleFloat")}var d={createMarkupForStyles:function(e,t){var n="";for(var r in e)if(e.hasOwnProperty(r)){var o=e[r];null!=o&&(n+=u(r)+":",n+=i(r,o,t)+";")}return n||null},setValueForStyles:function(e,t,n){var o=e.style;for(var a in t)if(t.hasOwnProperty(a)){var s=i(a,t[a],n);if("float"!==a&&"cssFloat"!==a||(a=c),s)o[a]=s;else{var u=l&&r.shorthandPropertyExpansions[a];if(u)for(var p in u)o[p]="";else o[a]=""}}}};t.exports=d},{123:123,125:125,136:136,140:140,142:142,4:4,58:58,94:94}],6:[function(e,t,n){"use strict";function r(e,t){if(!(e instanceof t))throw new TypeError("Cannot call a class as a function")}var o=e(112),i=e(24),a=(e(137),function(){function e(t){r(this,e),this._callbacks=null,this._contexts=null,this._arg=t}return e.prototype.enqueue=function(e,t){this._callbacks=this._callbacks||[],this._callbacks.push(e),this._contexts=this._contexts||[],this._contexts.push(t)},e.prototype.notifyAll=function(){var e=this._callbacks,t=this._contexts,n=this._arg;if(e&&t){e.length!==t.length&&o("24"),this._callbacks=null,this._contexts=null;for(var r=0;r8));var A=!1;b.canUseDOM&&(A=k("input")&&(!document.documentMode||document.documentMode>11));var D={get:function(){return O.get.call(this)},set:function(e){I=""+e,O.set.call(this,e)}},L={eventTypes:S,extractEvents:function(e,t,n,o){var i,a,s=t?E.getNodeFromInstance(t):window;if(r(s)?R?i=u:a=l:P(s)?A?i=f:(i=m,a=h):v(s)&&(i=g),i){var c=i(e,t);if(c){var p=w.getPooled(S.change,c,n,o);return p.type="change",C.accumulateTwoPhaseDispatches(p),p}}a&&a(e,s,t),"topBlur"===e&&y(t,s)}};t.exports=L},{102:102,109:109,110:110,123:123,16:16,19:19,33:33,71:71,80:80}],8:[function(e,t,n){"use strict";function r(e,t){return Array.isArray(t)&&(t=t[1]),t?t.nextSibling:e.firstChild}function o(e,t,n){c.insertTreeBefore(e,t,n)}function i(e,t,n){Array.isArray(t)?s(e,t[0],t[1],n):m(e,t,n)}function a(e,t){if(Array.isArray(t)){var n=t[1];t=t[0],u(e,t,n),e.removeChild(n)}e.removeChild(t)}function s(e,t,n,r){for(var o=t;;){var i=o.nextSibling;if(m(e,o,r),o===n)break;o=i}}function u(e,t,n){for(;;){var r=t.nextSibling;if(r===n)break;e.removeChild(r)}}function l(e,t,n){var r=e.parentNode,o=e.nextSibling;o===t?n&&m(r,document.createTextNode(n),o):n?(h(o,n),u(r,o,t)):u(r,e,t)}var c=e(9),p=e(13),d=(e(33),e(58),e(93)),f=e(114),h=e(115),m=d(function(e,t,n){e.insertBefore(t,n)}),v=p.dangerouslyReplaceNodeWithMarkup,g={dangerouslyReplaceNodeWithMarkup:v,replaceDelimitedText:l,processUpdates:function(e,t){for(var n=0;n-1||a("96",e),!l.plugins[n]){t.extractEvents||a("97",e),l.plugins[n]=t;var r=t.eventTypes;for(var i in r)o(r[i],t,i)||a("98",i,e)}}}function o(e,t,n){l.eventNameDispatchConfigs.hasOwnProperty(n)&&a("99",n),l.eventNameDispatchConfigs[n]=e;var r=e.phasedRegistrationNames;if(r){for(var o in r)if(r.hasOwnProperty(o)){var s=r[o];i(s,t,n)}return!0}return!!e.registrationName&&(i(e.registrationName,t,n),!0)}function i(e,t,n){l.registrationNameModules[e]&&a("100",e),l.registrationNameModules[e]=t,l.registrationNameDependencies[e]=t.eventTypes[n].dependencies}var a=e(112),s=(e(137),null),u={},l={plugins:[],eventNameDispatchConfigs:{},registrationNameModules:{},registrationNameDependencies:{},possibleRegistrationNames:null,injectEventPluginOrder:function(e){s&&a("101"),s=Array.prototype.slice.call(e),r()},injectEventPluginsByName:function(e){var t=!1;for(var n in e)if(e.hasOwnProperty(n)){var o=e[n];u.hasOwnProperty(n)&&u[n]===o||(u[n]&&a("102",n),u[n]=o,t=!0)}t&&r()},getPluginModuleForEvent:function(e){var t=e.dispatchConfig;if(t.registrationName)return l.registrationNameModules[t.registrationName]||null;if(void 0!==t.phasedRegistrationNames){var n=t.phasedRegistrationNames;for(var r in n)if(n.hasOwnProperty(r)){var o=l.registrationNameModules[n[r]];if(o)return o}}return null},_resetEventPlugins:function(){s=null;for(var e in u)u.hasOwnProperty(e)&&delete u[e];l.plugins.length=0;var t=l.eventNameDispatchConfigs;for(var n in t)t.hasOwnProperty(n)&&delete t[n];var r=l.registrationNameModules;for(var o in r)r.hasOwnProperty(o)&&delete r[o]}};t.exports=l},{112:112,137:137}],18:[function(e,t,n){"use strict";function r(e){return"topMouseUp"===e||"topTouchEnd"===e||"topTouchCancel"===e}function o(e){return"topMouseMove"===e||"topTouchMove"===e}function i(e){return"topMouseDown"===e||"topTouchStart"===e}function a(e,t,n,r){var o=e.type||"unknown-event";e.currentTarget=g.getNodeFromInstance(r),t?m.invokeGuardedCallbackWithCatch(o,n,e):m.invokeGuardedCallback(o,n,e),e.currentTarget=null}function s(e,t){var n=e._dispatchListeners,r=e._dispatchInstances;if(Array.isArray(n))for(var o=0;o1?1-t:void 0;return this._fallbackText=o.slice(e,s),this._fallbackText}}),i.addPoolingTo(r),t.exports=r},{106:106,143:143,24:24}],21:[function(e,t,n){"use strict";var r=e(11),o=r.injection.MUST_USE_PROPERTY,i=r.injection.HAS_BOOLEAN_VALUE,a=r.injection.HAS_NUMERIC_VALUE,s=r.injection.HAS_POSITIVE_NUMERIC_VALUE,u=r.injection.HAS_OVERLOADED_BOOLEAN_VALUE,l={isCustomAttribute:RegExp.prototype.test.bind(new RegExp("^(data|aria)-["+r.ATTRIBUTE_NAME_CHAR+"]*$")),Properties:{accept:0,acceptCharset:0,accessKey:0,action:0,allowFullScreen:i,allowTransparency:0,alt:0,as:0,async:i,autoComplete:0,autoPlay:i,capture:i,cellPadding:0,cellSpacing:0,charSet:0,challenge:0,checked:o|i,cite:0,classID:0,className:0,cols:s,colSpan:0,content:0,contentEditable:0,contextMenu:0,controls:i,coords:0,crossOrigin:0,data:0,dateTime:0,default:i,defer:i,dir:0,disabled:i,download:u,draggable:0,encType:0,form:0,formAction:0,formEncType:0,formMethod:0,formNoValidate:i,formTarget:0,frameBorder:0,headers:0,height:0,hidden:i,high:0,href:0,hrefLang:0,htmlFor:0,httpEquiv:0,icon:0,id:0,inputMode:0,integrity:0,is:0,keyParams:0,keyType:0,kind:0,label:0,lang:0,list:0,loop:i,low:0,manifest:0,marginHeight:0,marginWidth:0,max:0,maxLength:0,media:0,mediaGroup:0,method:0,min:0,minLength:0,multiple:o|i,muted:o|i,name:0,nonce:0,noValidate:i,open:i,optimum:0,pattern:0,placeholder:0,playsInline:i,poster:0,preload:0,profile:0,radioGroup:0,readOnly:i,referrerPolicy:0,rel:0,required:i,reversed:i,role:0,rows:s,rowSpan:a,sandbox:0,scope:0,scoped:i,scrolling:0,seamless:i,selected:o|i,shape:0,size:s,sizes:0,span:s,spellCheck:0,src:0,srcDoc:0,srcLang:0,srcSet:0,start:a,step:0,style:0,summary:0,tabIndex:0,target:0,title:0,type:0,useMap:0,value:0,width:0,wmode:0,wrap:0,about:0,datatype:0,inlist:0,prefix:0,property:0,resource:0,typeof:0,vocab:0,autoCapitalize:0,autoCorrect:0,autoSave:0,color:0,itemProp:0,itemScope:i,itemType:0,itemID:0,itemRef:0,results:0,security:0,unselectable:0},DOMAttributeNames:{acceptCharset:"accept-charset",className:"class",htmlFor:"for",httpEquiv:"http-equiv"},DOMPropertyNames:{},DOMMutationMethods:{value:function(e,t){if(null==t)return e.removeAttribute("value");"number"!==e.type||!1===e.hasAttribute("value")?e.setAttribute("value",""+t):e.validity&&!e.validity.badInput&&e.ownerDocument.activeElement!==e&&e.setAttribute("value",""+t)}}};t.exports=l},{11:11}],22:[function(e,t,n){"use strict";function r(e){var t={"=":"=0",":":"=2"};return"$"+(""+e).replace(/[=:]/g,function(e){return t[e]})}function o(e){var t={"=0":"=","=2":":"};return(""+("."===e[0]&&"$"===e[1]?e.substring(2):e.substring(1))).replace(/(=0|=2)/g,function(e){return t[e]})}var i={escape:r,unescape:o};t.exports=i},{}],23:[function(e,t,n){"use strict";function r(e){null!=e.checkedLink&&null!=e.valueLink&&s("87")}function o(e){r(e),(null!=e.value||null!=e.onChange)&&s("88")}function i(e){r(e),(null!=e.checked||null!=e.onChange)&&s("89")}function a(e){if(e){var t=e.getName();if(t)return" Check the render method of `"+t+"`."}return""}var s=e(112),u=e(64),l=e(145),c=e(120),p=l(c.isValidElement),d=(e(137),e(142),{button:!0,checkbox:!0,image:!0,hidden:!0,radio:!0,reset:!0,submit:!0}),f={value:function(e,t,n){return!e[t]||d[e.type]||e.onChange||e.readOnly||e.disabled?null:new Error("You provided a `value` prop to a form field without an `onChange` handler. This will render a read-only field. If the field should be mutable use `defaultValue`. Otherwise, set either `onChange` or `readOnly`.")},checked:function(e,t,n){return!e[t]||e.onChange||e.readOnly||e.disabled?null:new Error("You provided a `checked` prop to a form field without an `onChange` handler. This will render a read-only field. If the field should be mutable use `defaultChecked`. Otherwise, set either `onChange` or `readOnly`.")},onChange:p.func},h={},m={checkPropTypes:function(e,t,n){for(var r in f){if(f.hasOwnProperty(r))var o=f[r](t,r,e,"prop",null,u);o instanceof Error&&!(o.message in h)&&(h[o.message]=!0,a(n))}},getValue:function(e){return e.valueLink?(o(e),e.valueLink.value):e.value},getChecked:function(e){return e.checkedLink?(i(e),e.checkedLink.value):e.checked},executeOnChange:function(e,t){return e.valueLink?(o(e),e.valueLink.requestChange(t.target.value)):e.checkedLink?(i(e),e.checkedLink.requestChange(t.target.checked)):e.onChange?e.onChange.call(void 0,t):void 0}};t.exports=m},{112:112,120:120,137:137,142:142,145:145,64:64}],24:[function(e,t,n){"use strict";var r=e(112),o=(e(137),function(e){var t=this;if(t.instancePool.length){var n=t.instancePool.pop();return t.call(n,e),n}return new t(e)}),i=function(e,t){var n=this;if(n.instancePool.length){var r=n.instancePool.pop();return n.call(r,e,t),r}return new n(e,t)},a=function(e,t,n){var r=this;if(r.instancePool.length){var o=r.instancePool.pop();return r.call(o,e,t,n),o}return new r(e,t,n)},s=function(e,t,n,r){var o=this;if(o.instancePool.length){var i=o.instancePool.pop();return o.call(i,e,t,n,r),i}return new o(e,t,n,r)},u=function(e){var t=this;e instanceof t||r("25"),e.destructor(),t.instancePool.length=0||null!=t.is}function h(e){var t=e.type;d(t),this._currentElement=e,this._tag=t.toLowerCase(),this._namespaceURI=null,this._renderedChildren=null,this._previousStyle=null,this._previousStyleCopy=null,this._hostNode=null,this._hostParent=null,this._rootNodeID=0,this._domID=0,this._hostContainerInfo=null,this._wrapperState=null,this._topLevelWrapper=null,this._flags=0}var m=e(112),v=e(143),g=e(2),y=e(5),_=e(9),C=e(10),b=e(11),E=e(12),x=e(16),w=e(17),T=e(25),k=e(32),P=e(33),S=e(38),N=e(39),M=e(40),I=e(43),O=(e(58),e(61)),R=e(68),A=(e(129),e(95)),D=(e(137),e(109),e(141),e(118),e(142),k),L=x.deleteListener,U=P.getNodeFromInstance,F=T.listenTo,j=w.registrationNameModules,V={string:!0,number:!0},B="__html",W={children:null,dangerouslySetInnerHTML:null,suppressContentEditableWarning:null},H=11,q={topAbort:"abort",topCanPlay:"canplay",topCanPlayThrough:"canplaythrough",topDurationChange:"durationchange",topEmptied:"emptied",topEncrypted:"encrypted",topEnded:"ended",topError:"error",topLoadedData:"loadeddata",topLoadedMetadata:"loadedmetadata",topLoadStart:"loadstart",topPause:"pause",topPlay:"play",topPlaying:"playing",topProgress:"progress",topRateChange:"ratechange",topSeeked:"seeked",topSeeking:"seeking",topStalled:"stalled",topSuspend:"suspend",topTimeUpdate:"timeupdate",topVolumeChange:"volumechange",topWaiting:"waiting"},K={area:!0,base:!0,br:!0,col:!0,embed:!0,hr:!0,img:!0,input:!0,keygen:!0,link:!0,meta:!0,param:!0,source:!0,track:!0,wbr:!0},z={listing:!0,pre:!0,textarea:!0},Y=v({menuitem:!0},K),X=/^[a-zA-Z][a-zA-Z:_\.\-\d]*$/,Q={},G={}.hasOwnProperty,$=1;h.displayName="ReactDOMComponent",h.Mixin={mountComponent:function(e,t,n,r){this._rootNodeID=$++,this._domID=n._idCounter++,this._hostParent=t,this._hostContainerInfo=n;var i=this._currentElement.props;switch(this._tag){case"audio":case"form":case"iframe":case"img":case"link":case"object":case"source":case"video":this._wrapperState={listeners:null},e.getReactMountReady().enqueue(c,this);break;case"input":S.mountWrapper(this,i,t),i=S.getHostProps(this,i),e.getReactMountReady().enqueue(c,this);break;case"option":N.mountWrapper(this,i,t),i=N.getHostProps(this,i);break;case"select":M.mountWrapper(this,i,t),i=M.getHostProps(this,i),e.getReactMountReady().enqueue(c,this);break;case"textarea":I.mountWrapper(this,i,t),i=I.getHostProps(this,i),e.getReactMountReady().enqueue(c,this)}o(this,i);var a,p;null!=t?(a=t._namespaceURI,p=t._tag):n._tag&&(a=n._namespaceURI,p=n._tag),(null==a||a===C.svg&&"foreignobject"===p)&&(a=C.html),a===C.html&&("svg"===this._tag?a=C.svg:"math"===this._tag&&(a=C.mathml)),this._namespaceURI=a;var d;if(e.useCreateElement){var f,h=n._ownerDocument;if(a===C.html)if("script"===this._tag){var m=h.createElement("div"),v=this._currentElement.type;m.innerHTML="<"+v+">",f=m.removeChild(m.firstChild)}else f=i.is?h.createElement(this._currentElement.type,i.is):h.createElement(this._currentElement.type);else f=h.createElementNS(a,this._currentElement.type);P.precacheNode(this,f),this._flags|=D.hasCachedChildNodes,this._hostParent||E.setAttributeForRoot(f),this._updateDOMProperties(null,i,e);var y=_(f);this._createInitialChildren(e,i,r,y),d=y}else{var b=this._createOpenTagMarkupAndPutListeners(e,i),x=this._createContentMarkup(e,i,r);d=!x&&K[this._tag]?b+"/>":b+">"+x+""}switch(this._tag){case"input":e.getReactMountReady().enqueue(s,this),i.autoFocus&&e.getReactMountReady().enqueue(g.focusDOMComponent,this);break;case"textarea":e.getReactMountReady().enqueue(u,this),i.autoFocus&&e.getReactMountReady().enqueue(g.focusDOMComponent,this);break;case"select":case"button":i.autoFocus&&e.getReactMountReady().enqueue(g.focusDOMComponent,this);break;case"option":e.getReactMountReady().enqueue(l,this)}return d},_createOpenTagMarkupAndPutListeners:function(e,t){var n="<"+this._currentElement.type;for(var r in t)if(t.hasOwnProperty(r)){var o=t[r];if(null!=o)if(j.hasOwnProperty(r))o&&i(this,r,o,e);else{"style"===r&&(o&&(o=this._previousStyleCopy=v({},t.style)),o=y.createMarkupForStyles(o,this));var a=null;null!=this._tag&&f(this._tag,t)?W.hasOwnProperty(r)||(a=E.createMarkupForCustomAttribute(r,o)):a=E.createMarkupForProperty(r,o),a&&(n+=" "+a)}}return e.renderToStaticMarkup?n:(this._hostParent||(n+=" "+E.createMarkupForRoot()),n+=" "+E.createMarkupForID(this._domID))},_createContentMarkup:function(e,t,n){var r="",o=t.dangerouslySetInnerHTML;if(null!=o)null!=o.__html&&(r=o.__html);else{var i=V[typeof t.children]?t.children:null,a=null!=i?null:t.children;if(null!=i)r=A(i);else if(null!=a){var s=this.mountChildren(a,e,n);r=s.join("")}}return z[this._tag]&&"\n"===r.charAt(0)?"\n"+r:r},_createInitialChildren:function(e,t,n,r){var o=t.dangerouslySetInnerHTML;if(null!=o)null!=o.__html&&_.queueHTML(r,o.__html);else{var i=V[typeof t.children]?t.children:null,a=null!=i?null:t.children;if(null!=i)""!==i&&_.queueText(r,i);else if(null!=a)for(var s=this.mountChildren(a,e,n),u=0;u"},receiveComponent:function(){},getHostNode:function(){return i.getNodeFromInstance(this)},unmountComponent:function(){i.uncacheNode(this)}}),t.exports=a},{143:143,33:33,9:9}],36:[function(e,t,n){"use strict";var r={useCreateElement:!0,useFiber:!1};t.exports=r},{}],37:[function(e,t,n){"use strict";var r=e(8),o=e(33),i={dangerouslyProcessChildrenUpdates:function(e,t){var n=o.getNodeFromInstance(e);r.processUpdates(n,t)}};t.exports=i},{33:33,8:8}],38:[function(e,t,n){"use strict";function r(){this._rootNodeID&&d.updateWrapper(this)}function o(e){return"checkbox"===e.type||"radio"===e.type?null!=e.checked:null!=e.value}function i(e){var t=this._currentElement.props,n=l.executeOnChange(t,e);p.asap(r,this);var o=t.name;if("radio"===t.type&&null!=o){for(var i=c.getNodeFromInstance(this),s=i;s.parentNode;)s=s.parentNode;for(var u=s.querySelectorAll("input[name="+JSON.stringify(""+o)+'][type="radio"]'),d=0;dt.end?(n=t.end,r=t.start):(n=t.start,r=t.end),o.moveToElementText(e),o.moveStart("character",n),o.setEndPoint("EndToStart",o),o.moveEnd("character",r-n),o.select()}function s(e,t){if(window.getSelection){var n=window.getSelection(),r=e[c()].length,o=Math.min(t.start,r),i=void 0===t.end?o:Math.min(t.end,r);if(!n.extend&&o>i){var a=i;i=o,o=a}var s=l(e,o),u=l(e,i);if(s&&u){var p=document.createRange();p.setStart(s.node,s.offset),n.removeAllRanges(),o>i?(n.addRange(p),n.extend(u.node,u.offset)):(p.setEnd(u.node,u.offset),n.addRange(p))}}}var u=e(123),l=e(105),c=e(106),p=u.canUseDOM&&"selection"in document&&!("getSelection"in window),d={getOffsets:p?o:i,setOffsets:p?a:s};t.exports=d},{105:105,106:106,123:123}],42:[function(e,t,n){"use strict";var r=e(112),o=e(143),i=e(8),a=e(9),s=e(33),u=e(95),l=(e(137),e(118),function(e){this._currentElement=e,this._stringText=""+e, +this._hostNode=null,this._hostParent=null,this._domID=0,this._mountIndex=0,this._closingComment=null,this._commentNodes=null});o(l.prototype,{mountComponent:function(e,t,n,r){var o=n._idCounter++,i=" react-text: "+o+" ";if(this._domID=o,this._hostParent=t,e.useCreateElement){var l=n._ownerDocument,c=l.createComment(i),p=l.createComment(" /react-text "),d=a(l.createDocumentFragment());return a.queueChild(d,a(c)),this._stringText&&a.queueChild(d,a(l.createTextNode(this._stringText))),a.queueChild(d,a(p)),s.precacheNode(this,c),this._closingComment=p,d}var f=u(this._stringText);return e.renderToStaticMarkup?f:""+f+""},receiveComponent:function(e,t){if(e!==this._currentElement){this._currentElement=e;var n=""+e;if(n!==this._stringText){this._stringText=n;var r=this.getHostNode();i.replaceDelimitedText(r[0],r[1],n)}}},getHostNode:function(){var e=this._commentNodes;if(e)return e;if(!this._closingComment)for(var t=s.getNodeFromInstance(this),n=t.nextSibling;;){if(null==n&&r("67",this._domID),8===n.nodeType&&" /react-text "===n.nodeValue){this._closingComment=n;break}n=n.nextSibling}return e=[this._hostNode,this._closingComment],this._commentNodes=e,e},unmountComponent:function(){this._closingComment=null,this._commentNodes=null,s.uncacheNode(this)}}),t.exports=l},{112:112,118:118,137:137,143:143,33:33,8:8,9:9,95:95}],43:[function(e,t,n){"use strict";function r(){this._rootNodeID&&c.updateWrapper(this)}function o(e){var t=this._currentElement.props,n=s.executeOnChange(t,e);return l.asap(r,this),n}var i=e(112),a=e(143),s=e(23),u=e(33),l=e(71),c=(e(137),e(142),{getHostProps:function(e,t){return null!=t.dangerouslySetInnerHTML&&i("91"),a({},t,{value:void 0,defaultValue:void 0,children:""+e._wrapperState.initialValue,onChange:e._wrapperState.onChange})},mountWrapper:function(e,t){var n=s.getValue(t),r=n;if(null==n){var a=t.defaultValue,u=t.children;null!=u&&(null!=a&&i("92"),Array.isArray(u)&&(u.length<=1||i("93"),u=u[0]),a=""+u),null==a&&(a=""),r=a}e._wrapperState={initialValue:""+r,listeners:null,onChange:o.bind(e)}},updateWrapper:function(e){var t=e._currentElement.props,n=u.getNodeFromInstance(e),r=s.getValue(t);if(null!=r){var o=""+r;o!==n.value&&(n.value=o),null==t.defaultValue&&(n.defaultValue=o)}null!=t.defaultValue&&(n.defaultValue=t.defaultValue)},postMountWrapper:function(e){var t=u.getNodeFromInstance(e),n=t.textContent;n===e._wrapperState.initialValue&&(t.value=n)}});t.exports=c},{112:112,137:137,142:142,143:143,23:23,33:33,71:71}],44:[function(e,t,n){"use strict";function r(e,t){"_hostNode"in e||u("33"),"_hostNode"in t||u("33");for(var n=0,r=e;r;r=r._hostParent)n++;for(var o=0,i=t;i;i=i._hostParent)o++;for(;n-o>0;)e=e._hostParent,n--;for(;o-n>0;)t=t._hostParent,o--;for(var a=n;a--;){if(e===t)return e;e=e._hostParent,t=t._hostParent}return null}function o(e,t){"_hostNode"in e||u("35"),"_hostNode"in t||u("35");for(;t;){if(t===e)return!0;t=t._hostParent}return!1}function i(e){return"_hostNode"in e||u("36"),e._hostParent}function a(e,t,n){for(var r=[];e;)r.push(e),e=e._hostParent;var o;for(o=r.length;o-- >0;)t(r[o],"captured",n);for(o=0;o0;)n(u[l],"captured",i)}var u=e(112);e(137);t.exports={isAncestor:o,getLowestCommonAncestor:r,getParentInstance:i,traverseTwoPhase:a,traverseEnterLeave:s}},{112:112,137:137}],45:[function(e,t,n){"use strict";var r=e(120),o=e(30),i=o;r.addons&&(r.__SECRET_INJECTED_REACT_DOM_DO_NOT_USE_OR_YOU_WILL_BE_FIRED=i),t.exports=i},{120:120,30:30}],46:[function(e,t,n){"use strict";function r(){this.reinitializeTransaction()}var o=e(143),i=e(71),a=e(89),s=e(129),u={initialize:s,close:function(){d.isBatchingUpdates=!1}},l={initialize:s,close:i.flushBatchedUpdates.bind(i)},c=[l,u];o(r.prototype,a,{getTransactionWrappers:function(){return c}});var p=new r,d={isBatchingUpdates:!1,batchedUpdates:function(e,t,n,r,o,i){var a=d.isBatchingUpdates;return d.isBatchingUpdates=!0,a?e(t,n,r,o,i):p.perform(e,null,t,n,r,o,i)}};t.exports=d},{129:129,143:143,71:71,89:89}],47:[function(e,t,n){"use strict";function r(){x||(x=!0,y.EventEmitter.injectReactEventListener(g),y.EventPluginHub.injectEventPluginOrder(s),y.EventPluginUtils.injectComponentTree(d),y.EventPluginUtils.injectTreeTraversal(h),y.EventPluginHub.injectEventPluginsByName({SimpleEventPlugin:E,EnterLeaveEventPlugin:u,ChangeEventPlugin:a,SelectEventPlugin:b,BeforeInputEventPlugin:i}),y.HostComponent.injectGenericComponentClass(p),y.HostComponent.injectTextComponentClass(m),y.DOMProperty.injectDOMPropertyConfig(o),y.DOMProperty.injectDOMPropertyConfig(l),y.DOMProperty.injectDOMPropertyConfig(C),y.EmptyComponent.injectEmptyComponentFactory(function(e){return new f(e)}),y.Updates.injectReconcileTransaction(_),y.Updates.injectBatchingStrategy(v),y.Component.injectEnvironment(c))}var o=e(1),i=e(3),a=e(7),s=e(14),u=e(15),l=e(21),c=e(27),p=e(31),d=e(33),f=e(35),h=e(44),m=e(42),v=e(46),g=e(52),y=e(55),_=e(65),C=e(73),b=e(74),E=e(75),x=!1;t.exports={inject:r}},{1:1,14:14,15:15,21:21,27:27,3:3,31:31,33:33,35:35,42:42,44:44,46:46,52:52,55:55,65:65,7:7,73:73,74:74,75:75}],48:[function(e,t,n){"use strict";var r="function"==typeof Symbol&&Symbol.for&&Symbol.for("react.element")||60103;t.exports=r},{}],49:[function(e,t,n){"use strict";var r,o={injectEmptyComponentFactory:function(e){r=e}},i={create:function(e){return r(e)}};i.injection=o,t.exports=i},{}],50:[function(e,t,n){"use strict";function r(e,t,n){try{t(n)}catch(e){null===o&&(o=e)}}var o=null,i={invokeGuardedCallback:r,invokeGuardedCallbackWithCatch:r,rethrowCaughtError:function(){if(o){var e=o;throw o=null,e}}};t.exports=i},{}],51:[function(e,t,n){"use strict";function r(e){o.enqueueEvents(e),o.processEventQueue(!1)}var o=e(16),i={handleTopLevel:function(e,t,n,i){r(o.extractEvents(e,t,n,i))}};t.exports=i},{16:16}],52:[function(e,t,n){"use strict";function r(e){for(;e._hostParent;)e=e._hostParent;var t=p.getNodeFromInstance(e),n=t.parentNode;return p.getClosestInstanceFromNode(n)}function o(e,t){this.topLevelType=e,this.nativeEvent=t,this.ancestors=[]}function i(e){var t=f(e.nativeEvent),n=p.getClosestInstanceFromNode(t),o=n;do{e.ancestors.push(o),o=o&&r(o)}while(o);for(var i=0;i/," "+i.CHECKSUM_ATTR_NAME+'="'+t+'"$&')},canReuseMarkup:function(e,t){var n=t.getAttribute(i.CHECKSUM_ATTR_NAME);return n=n&&parseInt(n,10),r(e)===n}};t.exports=i},{92:92}],60:[function(e,t,n){"use strict";function r(e,t){for(var n=Math.min(e.length,t.length),r=0;r.":"function"==typeof t?" Instead of passing a class like Foo, pass React.createElement(Foo) or .":null!=t&&void 0!==t.props?" This may be caused by unintentionally loading two independent copies of React.":"");var a,s=v.createElement(F,{child:t});if(e){var u=E.get(e);a=u._processChildContext(u._context)}else a=P;var c=d(n);if(c){var p=c._currentElement,h=p.props.child;if(M(h,t)){var m=c._renderedComponent.getPublicInstance(),g=r&&function(){r.call(m)};return j._updateRootComponent(c,s,a,n,g),m}j.unmountComponentAtNode(n)}var y=o(n),_=y&&!!i(y),C=l(n),b=_&&!c&&!C,x=j._renderNewRootComponent(s,n,b,a)._renderedComponent.getPublicInstance();return r&&r.call(x),x},render:function(e,t,n){return j._renderSubtreeIntoContainer(null,e,t,n)},unmountComponentAtNode:function(e){c(e)||f("40");var t=d(e);return t?(delete L[t._instance.rootID],k.batchedUpdates(u,t,e,!1),!0):(l(e),1===e.nodeType&&e.hasAttribute(O),!1)},_mountImageIntoNode:function(e,t,n,i,a){if(c(t)||f("41"),i){var s=o(t);if(x.canReuseMarkup(e,s))return void y.precacheNode(n,s);var u=s.getAttribute(x.CHECKSUM_ATTR_NAME);s.removeAttribute(x.CHECKSUM_ATTR_NAME);var l=s.outerHTML;s.setAttribute(x.CHECKSUM_ATTR_NAME,u);var p=e,d=r(p,l),m=" (client) "+p.substring(d-20,d+20)+"\n (server) "+l.substring(d-20,d+20);t.nodeType===A&&f("42",m)}if(t.nodeType===A&&f("43"),a.useCreateElement){for(;t.lastChild;)t.removeChild(t.lastChild);h.insertTreeBefore(t,e,null)}else N(t,e),y.precacheNode(n,t.firstChild)}};t.exports=j},{108:108,11:11,112:112,114:114,116:116,119:119,120:120,130:130,137:137,142:142,25:25,33:33,34:34,36:36,53:53,57:57,58:58,59:59,66:66,70:70,71:71,9:9}],61:[function(e,t,n){"use strict";function r(e,t,n){return{type:"INSERT_MARKUP",content:e,fromIndex:null,fromNode:null,toIndex:n,afterNode:t}}function o(e,t,n){return{type:"MOVE_EXISTING",content:null,fromIndex:e._mountIndex,fromNode:d.getHostNode(e),toIndex:n,afterNode:t}}function i(e,t){return{type:"REMOVE_NODE",content:null,fromIndex:e._mountIndex,fromNode:t,toIndex:null,afterNode:null}}function a(e){return{type:"SET_MARKUP",content:e,fromIndex:null,fromNode:null,toIndex:null,afterNode:null}}function s(e){return{type:"TEXT_CONTENT",content:e,fromIndex:null,fromNode:null,toIndex:null,afterNode:null}}function u(e,t){return t&&(e=e||[],e.push(t)),e}function l(e,t){p.processChildrenUpdates(e,t)}var c=e(112),p=e(28),d=(e(57),e(58),e(119),e(66)),f=e(26),h=(e(129),e(97)),m=(e(137),{Mixin:{_reconcilerInstantiateChildren:function(e,t,n){return f.instantiateChildren(e,t,n)},_reconcilerUpdateChildren:function(e,t,n,r,o,i){var a;return a=h(t,0),f.updateChildren(e,a,n,r,o,this,this._hostContainerInfo,i,0),a},mountChildren:function(e,t,n){var r=this._reconcilerInstantiateChildren(e,t,n);this._renderedChildren=r;var o=[],i=0;for(var a in r)if(r.hasOwnProperty(a)){var s=r[a],u=d.mountComponent(s,t,this,this._hostContainerInfo,n,0);s._mountIndex=i++,o.push(u)}return o},updateTextContent:function(e){var t=this._renderedChildren;f.unmountChildren(t,!1);for(var n in t)t.hasOwnProperty(n)&&c("118");l(this,[s(e)])},updateMarkup:function(e){var t=this._renderedChildren;f.unmountChildren(t,!1);for(var n in t)t.hasOwnProperty(n)&&c("118");l(this,[a(e)])},updateChildren:function(e,t,n){this._updateChildren(e,t,n)},_updateChildren:function(e,t,n){var r=this._renderedChildren,o={},i=[],a=this._reconcilerUpdateChildren(r,e,i,o,t,n);if(a||r){var s,c=null,p=0,f=0,h=0,m=null;for(s in a)if(a.hasOwnProperty(s)){var v=r&&r[s],g=a[s];v===g?(c=u(c,this.moveChild(v,m,p,f)),f=Math.max(v._mountIndex,f),v._mountIndex=p):(v&&(f=Math.max(v._mountIndex,f)),c=u(c,this._mountChildAtIndex(g,i[h],m,p,t,n)),h++),p++,m=d.getHostNode(g)}for(s in o)o.hasOwnProperty(s)&&(c=u(c,this._unmountChild(r[s],o[s])));c&&l(this,c),this._renderedChildren=a}},unmountChildren:function(e){var t=this._renderedChildren;f.unmountChildren(t,e),this._renderedChildren=null},moveChild:function(e,t,n,r){if(e._mountIndex0&&r.length<20?n+" (keys: "+r.join(", ")+")":n}function i(e,t){var n=s.get(e);return n||null}var a=e(112),s=(e(119),e(57)),u=(e(58),e(71)),l=(e(137),e(142),{isMounted:function(e){var t=s.get(e);return!!t&&!!t._renderedComponent},enqueueCallback:function(e,t,n){l.validateCallback(t,n);var o=i(e);if(!o)return null;o._pendingCallbacks?o._pendingCallbacks.push(t):o._pendingCallbacks=[t],r(o)},enqueueCallbackInternal:function(e,t){e._pendingCallbacks?e._pendingCallbacks.push(t):e._pendingCallbacks=[t],r(e)},enqueueForceUpdate:function(e){var t=i(e,"forceUpdate");t&&(t._pendingForceUpdate=!0,r(t))},enqueueReplaceState:function(e,t,n){var o=i(e,"replaceState");o&&(o._pendingStateQueue=[t],o._pendingReplaceState=!0,void 0!==n&&null!==n&&(l.validateCallback(n,"replaceState"),o._pendingCallbacks?o._pendingCallbacks.push(n):o._pendingCallbacks=[n]),r(o))},enqueueSetState:function(e,t){var n=i(e,"setState");n&&((n._pendingStateQueue||(n._pendingStateQueue=[])).push(t),r(n))},enqueueElementInternal:function(e,t,n){e._pendingElement=t,e._context=n,r(e)},validateCallback:function(e,t){e&&"function"!=typeof e&&a("122",t,o(e))}});t.exports=l},{112:112,119:119,137:137,142:142,57:57,58:58,71:71}],71:[function(e,t,n){"use strict";function r(){P.ReactReconcileTransaction&&b||c("123")}function o(){this.reinitializeTransaction(),this.dirtyComponentsLength=null,this.callbackQueue=d.getPooled(),this.reconcileTransaction=P.ReactReconcileTransaction.getPooled(!0)}function i(e,t,n,o,i,a){return r(),b.batchedUpdates(e,t,n,o,i,a)}function a(e,t){return e._mountOrder-t._mountOrder}function s(e){var t=e.dirtyComponentsLength;t!==g.length&&c("124",t,g.length),g.sort(a),y++;for(var n=0;n]/;t.exports=o},{}],96:[function(e,t,n){"use strict";function r(e){if(null==e)return null;if(1===e.nodeType)return e;var t=a.get(e);if(t)return t=s(t),t?i.getNodeFromInstance(t):null;"function"==typeof e.render?o("44"):o("45",Object.keys(e))}var o=e(112),i=(e(119),e(33)),a=e(57),s=e(103);e(137),e(142);t.exports=r},{103:103,112:112,119:119,137:137,142:142,33:33,57:57}],97:[function(e,t,n){(function(n){"use strict";function r(e,t,n,r){if(e&&"object"==typeof e){var o=e;void 0===o[n]&&null!=t&&(o[n]=t)}}function o(e,t){if(null==e)return e;var n={};return i(e,r,n),n}var i=(e(22),e(117));e(142);void 0!==n&&n.env,t.exports=o}).call(this,void 0)},{117:117,142:142,22:22}],98:[function(e,t,n){"use strict";function r(e,t,n){Array.isArray(e)?e.forEach(t,n):e&&t.call(n,e)}t.exports=r},{}],99:[function(e,t,n){"use strict";function r(e){var t,n=e.keyCode;return"charCode"in e?0===(t=e.charCode)&&13===n&&(t=13):t=n,t>=32||13===t?t:0}t.exports=r},{}],100:[function(e,t,n){"use strict";function r(e){if(e.key){var t=i[e.key]||e.key;if("Unidentified"!==t)return t}if("keypress"===e.type){var n=o(e);return 13===n?"Enter":String.fromCharCode(n)}return"keydown"===e.type||"keyup"===e.type?a[e.keyCode]||"Unidentified":""}var o=e(99),i={Esc:"Escape",Spacebar:" ",Left:"ArrowLeft",Up:"ArrowUp",Right:"ArrowRight",Down:"ArrowDown",Del:"Delete",Win:"OS",Menu:"ContextMenu",Apps:"ContextMenu",Scroll:"ScrollLock",MozPrintableKey:"Unidentified"},a={8:"Backspace",9:"Tab",12:"Clear",13:"Enter",16:"Shift",17:"Control",18:"Alt",19:"Pause",20:"CapsLock",27:"Escape",32:" ",33:"PageUp",34:"PageDown",35:"End",36:"Home",37:"ArrowLeft",38:"ArrowUp",39:"ArrowRight",40:"ArrowDown",45:"Insert",46:"Delete",112:"F1",113:"F2",114:"F3",115:"F4",116:"F5",117:"F6",118:"F7",119:"F8",120:"F9",121:"F10",122:"F11",123:"F12",144:"NumLock",145:"ScrollLock",224:"Meta"};t.exports=r},{99:99}],101:[function(e,t,n){"use strict";function r(e){var t=this,n=t.nativeEvent;if(n.getModifierState)return n.getModifierState(e);var r=i[e];return!!r&&!!n[r]}function o(e){return r}var i={Alt:"altKey",Control:"ctrlKey",Meta:"metaKey",Shift:"shiftKey"};t.exports=o},{}],102:[function(e,t,n){"use strict";function r(e){var t=e.target||e.srcElement||window;return t.correspondingUseElement&&(t=t.correspondingUseElement),3===t.nodeType?t.parentNode:t}t.exports=r},{}],103:[function(e,t,n){"use strict";function r(e){for(var t;(t=e._renderedNodeType)===o.COMPOSITE;)e=e._renderedComponent;return t===o.HOST?e._renderedComponent:t===o.EMPTY?null:void 0}var o=e(62);t.exports=r},{62:62}],104:[function(e,t,n){"use strict";function r(e){var t=e&&(o&&e[o]||e[i]);if("function"==typeof t)return t}var o="function"==typeof Symbol&&Symbol.iterator,i="@@iterator";t.exports=r},{}],105:[function(e,t,n){"use strict";function r(e){for(;e&&e.firstChild;)e=e.firstChild;return e}function o(e){for(;e;){if(e.nextSibling)return e.nextSibling;e=e.parentNode}}function i(e,t){for(var n=r(e),i=0,a=0;n;){if(3===n.nodeType){if(a=i+n.textContent.length,i<=t&&a>=t)return{node:n,offset:t-i};i=a}n=r(o(n))}}t.exports=i},{}],106:[function(e,t,n){"use strict";function r(){return!i&&o.canUseDOM&&(i="textContent"in document.documentElement?"textContent":"innerText"),i}var o=e(123),i=null;t.exports=r},{123:123}],107:[function(e,t,n){"use strict";function r(e,t){var n={};return n[e.toLowerCase()]=t.toLowerCase(),n["Webkit"+e]="webkit"+t,n["Moz"+e]="moz"+t,n["ms"+e]="MS"+t,n["O"+e]="o"+t.toLowerCase(),n}function o(e){if(s[e])return s[e];if(!a[e])return e;var t=a[e];for(var n in t)if(t.hasOwnProperty(n)&&n in u)return s[e]=t[n];return""}var i=e(123),a={animationend:r("Animation","AnimationEnd"),animationiteration:r("Animation","AnimationIteration"),animationstart:r("Animation","AnimationStart"),transitionend:r("Transition","TransitionEnd")},s={},u={};i.canUseDOM&&(u=document.createElement("div").style,"AnimationEvent"in window||(delete a.animationend.animation,delete a.animationiteration.animation,delete a.animationstart.animation),"TransitionEvent"in window||delete a.transitionend.transition),t.exports=o},{123:123}],108:[function(e,t,n){"use strict";function r(e){if(e){var t=e.getName();if(t)return" Check the render method of `"+t+"`."}return""}function o(e){return"function"==typeof e&&void 0!==e.prototype&&"function"==typeof e.prototype.mountComponent&&"function"==typeof e.prototype.receiveComponent}function i(e,t){var n;if(null===e||!1===e)n=l.create(i);else if("object"==typeof e){var s=e,u=s.type;if("function"!=typeof u&&"string"!=typeof u){var d="";d+=r(s._owner),a("130",null==u?u:typeof u,d)}"string"==typeof s.type?n=c.createInternalComponent(s):o(s.type)?(n=new s.type(s),n.getHostNode||(n.getHostNode=n.getNativeNode)):n=new p(s)}else"string"==typeof e||"number"==typeof e?n=c.createInstanceForText(e):a("131",typeof e);return n._mountIndex=0,n._mountImage=null,n}var a=e(112),s=e(143),u=e(29),l=e(49),c=e(54),p=(e(121),e(137),e(142),function(e){this.construct(e)});s(p.prototype,u,{_instantiateReactComponent:i}),t.exports=i},{112:112,121:121,137:137,142:142,143:143,29:29,49:49,54:54}],109:[function(e,t,n){"use strict";function r(e,t){if(!i.canUseDOM||t&&!("addEventListener"in document))return!1;var n="on"+e,r=n in document;if(!r){var a=document.createElement("div");a.setAttribute(n,"return;"),r="function"==typeof a[n]}return!r&&o&&"wheel"===e&&(r=document.implementation.hasFeature("Events.wheel","3.0")),r}var o,i=e(123);i.canUseDOM&&(o=document.implementation&&document.implementation.hasFeature&&!0!==document.implementation.hasFeature("","")),t.exports=r},{123:123}],110:[function(e,t,n){"use strict";function r(e){var t=e&&e.nodeName&&e.nodeName.toLowerCase();return"input"===t?!!o[e.type]:"textarea"===t}var o={color:!0,date:!0,datetime:!0,"datetime-local":!0,email:!0,month:!0,number:!0,password:!0,range:!0,search:!0,tel:!0,text:!0,time:!0,url:!0,week:!0};t.exports=r},{}],111:[function(e,t,n){"use strict";function r(e){return'"'+o(e)+'"'}var o=e(95);t.exports=r},{95:95}],112:[function(e,t,n){"use strict";function r(e){for(var t=arguments.length-1,n="Minified React error #"+e+"; visit http://facebook.github.io/react/docs/error-decoder.html?invariant="+e,r=0;r]/,u=e(93),l=u(function(e,t){if(e.namespaceURI!==i.svg||"innerHTML"in e)e.innerHTML=t;else{r=r||document.createElement("div"),r.innerHTML=""+t+"";for(var n=r.firstChild;n.firstChild;)e.appendChild(n.firstChild)}});if(o.canUseDOM){var c=document.createElement("div");c.innerHTML=" ",""===c.innerHTML&&(l=function(e,t){if(e.parentNode&&e.parentNode.replaceChild(e,e),a.test(t)||"<"===t[0]&&s.test(t)){e.innerHTML=String.fromCharCode(65279)+t;var n=e.firstChild;1===n.data.length?e.removeChild(n):n.deleteData(0,1)}else e.innerHTML=t}),c=null}t.exports=l},{10:10,123:123,93:93}],115:[function(e,t,n){"use strict";var r=e(123),o=e(95),i=e(114),a=function(e,t){if(t){var n=e.firstChild;if(n&&n===e.lastChild&&3===n.nodeType)return void(n.nodeValue=t)}e.textContent=t};r.canUseDOM&&("textContent"in document.documentElement||(a=function(e,t){if(3===e.nodeType)return void(e.nodeValue=t);i(e,o(t))})),t.exports=a},{114:114,123:123,95:95}],116:[function(e,t,n){"use strict";function r(e,t){var n=null===e||!1===e,r=null===t||!1===t;if(n||r)return n===r;var o=typeof e,i=typeof t;return"string"===o||"number"===o?"string"===i||"number"===i:"object"===i&&e.type===t.type&&e.key===t.key}t.exports=r},{}],117:[function(e,t,n){"use strict";function r(e,t){return e&&"object"==typeof e&&null!=e.key?l.escape(e.key):t.toString(36)}function o(e,t,n,i){var d=typeof e;if("undefined"!==d&&"boolean"!==d||(e=null),null===e||"string"===d||"number"===d||"object"===d&&e.$$typeof===s)return n(i,e,""===t?c+r(e,0):t),1;var f,h,m=0,v=""===t?c:t+p;if(Array.isArray(e))for(var g=0;g":"<"+e+">",s[e]=!a.firstChild),s[e]?d[e]:null}var o=e(123),i=e(137),a=o.canUseDOM?document.createElement("div"):null,s={},u=[1,'"],l=[1,"","
"],c=[3,"","
"],p=[1,'',""],d={"*":[1,"?
","
"],area:[1,"",""],col:[2,"","
"],legend:[1,"
","
"],param:[1,"",""],tr:[2,"","
"],optgroup:u,option:u,caption:l,colgroup:l,tbody:l,tfoot:l,thead:l,td:c,th:c};["circle","clipPath","defs","ellipse","g","image","line","linearGradient","mask","path","pattern","polygon","polyline","radialGradient","rect","stop","text","tspan"].forEach(function(e){d[e]=p,s[e]=!0}),t.exports=r},{123:123,137:137}],134:[function(e,t,n){"use strict";function r(e){return e.Window&&e instanceof e.Window?{x:e.pageXOffset||e.document.documentElement.scrollLeft,y:e.pageYOffset||e.document.documentElement.scrollTop}:{x:e.scrollLeft,y:e.scrollTop}}t.exports=r},{}],135:[function(e,t,n){"use strict";function r(e){return e.replace(o,"-$1").toLowerCase()}var o=/([A-Z])/g;t.exports=r},{}],136:[function(e,t,n){"use strict";function r(e){return o(e).replace(i,"-ms-")}var o=e(135),i=/^ms-/;t.exports=r},{135:135}],137:[function(e,t,n){"use strict";function r(e,t,n,r,i,a,s,u){if(o(t),!e){var l;if(void 0===t)l=new Error("Minified exception occurred; use the non-minified dev environment for the full error message and additional helpful warnings.");else{var c=[n,r,i,a,s,u],p=0;l=new Error(t.replace(/%s/g,function(){return c[p++]})),l.name="Invariant Violation"}throw l.framesToPop=1,l}}var o=function(e){};t.exports=r},{}],138:[function(e,t,n){"use strict";function r(e){var t=e?e.ownerDocument||e:document,n=t.defaultView||window;return!(!e||!("function"==typeof n.Node?e instanceof n.Node:"object"==typeof e&&"number"==typeof e.nodeType&&"string"==typeof e.nodeName))}t.exports=r},{}],139:[function(e,t,n){"use strict";function r(e){return o(e)&&3==e.nodeType}var o=e(138);t.exports=r},{138:138}],140:[function(e,t,n){"use strict";function r(e){var t={};return function(n){return t.hasOwnProperty(n)||(t[n]=e.call(this,n)),t[n]}}t.exports=r},{}],141:[function(e,t,n){"use strict";function r(e,t){return e===t?0!==e||0!==t||1/e==1/t:e!==e&&t!==t}function o(e,t){if(r(e,t))return!0;if("object"!=typeof e||null===e||"object"!=typeof t||null===t)return!1;var n=Object.keys(e),o=Object.keys(t);if(n.length!==o.length)return!1;for(var a=0;a 0x10FFFF || // not a valid Unicode code point + floor(codePoint) != codePoint // not an integer + ) { + throw RangeError('Invalid code point: ' + codePoint); + } + if (codePoint <= 0xFFFF) { // BMP code point + codeUnits.push(codePoint); + } else { // Astral code point; split in surrogate halves + // http://mathiasbynens.be/notes/javascript-encoding#surrogate-formulae + codePoint -= 0x10000; + highSurrogate = (codePoint >> 10) + 0xD800; + lowSurrogate = (codePoint % 0x400) + 0xDC00; + codeUnits.push(highSurrogate, lowSurrogate); + } + if (index + 1 == length || codeUnits.length > MAX_SIZE) { + result += stringFromCharCode.apply(null, codeUnits); + codeUnits.length = 0; + } + } + return result; + }; + if (defineProperty) { + defineProperty(String, 'fromCodePoint', { + 'value': fromCodePoint, + 'configurable': true, + 'writable': true + }); + } else { + String.fromCodePoint = fromCodePoint; + } + }()); +} + +/*! http://mths.be/codepointat v0.1.0 by @mathias */ +if (!String.prototype.codePointAt) { + (function() { + 'use strict'; // needed to support `apply`/`call` with `undefined`/`null` + var codePointAt = function(position) { + if (this == null) { + throw TypeError(); + } + var string = String(this); + var size = string.length; + // `ToInteger` + var index = position ? Number(position) : 0; + if (index != index) { // better `isNaN` + index = 0; + } + // Account for out-of-bounds indices: + if (index < 0 || index >= size) { + return undefined; + } + // Get the first code unit + var first = string.charCodeAt(index); + var second; + if ( // check if it’s the start of a surrogate pair + first >= 0xD800 && first <= 0xDBFF && // high surrogate + size > index + 1 // there is a next code unit + ) { + second = string.charCodeAt(index + 1); + if (second >= 0xDC00 && second <= 0xDFFF) { // low surrogate + // http://mathiasbynens.be/notes/javascript-encoding#surrogate-formulae + return (first - 0xD800) * 0x400 + second - 0xDC00 + 0x10000; + } + } + return first; + }; + if (Object.defineProperty) { + Object.defineProperty(String.prototype, 'codePointAt', { + 'value': codePointAt, + 'configurable': true, + 'writable': true + }); + } else { + String.prototype.codePointAt = codePointAt; + } + }()); +} + +function registerAsciinemaPlayerElement() { + var AsciinemaPlayerProto = Object.create(HTMLElement.prototype); + + function merge() { + var merged = {}; + for (var i=0; i>>0),ma=0;function na(a,b,c){return a.call.apply(a.bind,arguments)} +function oa(a,b,c){if(!a)throw Error();if(2b?1:0};var ua=Array.prototype.indexOf?function(a,b,c){return Array.prototype.indexOf.call(a,b,c)}:function(a,b,c){c=null==c?0:0>c?Math.max(0,a.length+c):c;if(ca(a))return ca(b)&&1==b.length?a.indexOf(b,c):-1;for(;cb?null:ca(a)?a.charAt(b):a[b]}function ya(a,b){var c=ua(a,b),d;(d=0<=c)&&Array.prototype.splice.call(a,c,1);return d}function za(a,b){a.sort(b||Aa)}function Ca(a,b){for(var c=Array(a.length),d=0;db?1:a2*this.Fc&&Na(this),!0):!1};function Na(a){if(a.Fc!=a.ib.length){for(var b=0,c=0;ba){var b=Ra[a];if(b)return b}b=new Qa([a|0],0>a?-1:0);-128<=a&&128>a&&(Ra[a]=b);return b}function Ta(a){if(isNaN(a)||!isFinite(a))return Ua;if(0>a)return Ta(-a).kb();for(var b=[],c=1,d=0;a>=c;d++)b[d]=a/c|0,c*=Va;return new Qa(b,0)}var Va=4294967296,Ua=Sa(0),Wa=Sa(1),Xa=Sa(16777216);g=Qa.prototype; +g.Of=function(){return 0a||36>>0).toString(a);c=e;if(c.hc())return f+d;for(;6>f.length;)f="0"+f;d=""+f+d}};function Ya(a,b){return 0>b?0:bthis.compare(Xa)};g.Ve=function(a){return 0>=this.compare(a)};g.compare=function(a){a=this.ze(a);return a.Eb()?-1:a.hc()?0:1};g.kb=function(){return this.Hf().add(Wa)}; +g.add=function(a){for(var b=Math.max(this.Ma.length,a.Ma.length),c=[],d=0,e=0;e<=b;e++){var f=d+(Ya(this,e)&65535)+(Ya(a,e)&65535),h=(f>>>16)+(Ya(this,e)>>>16)+(Ya(a,e)>>>16);d=h>>>16;f&=65535;h&=65535;c[e]=h<<16|f}return new Qa(c,c[c.length-1]&-2147483648?-1:0)};g.ze=function(a){return this.add(a.kb())}; +g.multiply=function(a){if(this.hc()||a.hc())return Ua;if(this.Eb())return a.Eb()?this.kb().multiply(a.kb()):this.kb().multiply(a).kb();if(a.Eb())return this.multiply(a.kb()).kb();if(this.Ue()&&a.Ue())return Ta(this.vd()*a.vd());for(var b=this.Ma.length+a.Ma.length,c=[],d=0;d<2*b;d++)c[d]=0;for(d=0;d>>16,h=Ya(this,d)&65535,k=Ya(a,e)>>>16,l=Ya(a,e)&65535;c[2*d+2*e]+=h*l;ab(c,2*d+2*e);c[2*d+2*e+1]+=f*l;ab(c,2*d+2*e+1);c[2*d+2*e+1]+= +h*k;ab(c,2*d+2*e+1);c[2*d+2*e+2]+=f*k;ab(c,2*d+2*e+2)}for(d=0;d>>16,a[b]&=65535,b++} +function Za(a,b){if(b.hc())throw Error("division by zero");if(a.hc())return Ua;if(a.Eb())return b.Eb()?Za(a.kb(),b.kb()):Za(a.kb(),b).kb();if(b.Eb())return Za(a,b.kb()).kb();if(30=f?1:Math.pow(2,f-48);h=Ta(e);for(var k=h.multiply(b);k.Eb()||k.xf(d);)e-=f,h=Ta(e),k=h.multiply(b);h.hc()&&(h=Wa);c=c.add(h);d=d.ze(k)}return c}g.Hf=function(){for(var a=this.Ma.length,b=[],c=0;c>5;a%=32;for(var c=this.Ma.length+b+(0>>32-a:Ya(this,e-b);return new Qa(d,this.Lc)}; +g.ad=function(a){var b=a>>5;a%=32;for(var c=this.Ma.length-b,d=[],e=0;e>>a|Ya(this,e+b+1)<<32-a:Ya(this,e+b);return new Qa(d,this.Lc)};function cb(a,b){null!=a&&this.append.apply(this,arguments)}g=cb.prototype;g.xc="";g.set=function(a){this.xc=""+a};g.append=function(a,b,c){this.xc+=String(a);if(null!=b)for(var d=1;d>>16&65535)*d+c*(b>>>16&65535)<<16>>>0)|0};function hd(a){a=gd(a|0,-862048943);return gd(a<<15|a>>>-15,461845907)} +function id(a,b){var c=(a|0)^(b|0);return gd(c<<13|c>>>-13,5)+-430675100|0}function jd(a,b){var c=(a|0)^b;c=gd(c^c>>>16,-2048144789);c=gd(c^c>>>13,-1028477387);return c^c>>>16}function kd(a){a:{var b=1;for(var c=0;;)if(b>2)}function qd(a){return a instanceof rd} +function sd(a,b){if(a.Zb===b.Zb)return 0;var c=wb(a.fb);if(t(c?b.fb:c))return-1;if(t(a.fb)){if(wb(b.fb))return 1;c=Aa(a.fb,b.fb);return 0===c?Aa(a.name,b.name):c}return Aa(a.name,b.name)}function rd(a,b,c,d,e){this.fb=a;this.name=b;this.Zb=c;this.Oc=d;this.hb=e;this.m=2154168321;this.J=4096}g=rd.prototype;g.toString=function(){return this.Zb};g.equiv=function(a){return this.K(null,a)};g.K=function(a,b){return b instanceof rd?this.Zb===b.Zb:!1}; +g.call=function(){var a=null;a=function(a,c,d){switch(arguments.length){case 2:return D.c(c,this);case 3:return D.l(c,this,d)}throw Error("Invalid arity: "+(arguments.length-1));};a.c=function(a,c){return D.c(c,this)};a.l=function(a,c,d){return D.l(c,this,d)};return a}();g.apply=function(a,b){return this.call.apply(this,[this].concat(Gb(b)))};g.h=function(a){return D.c(a,this)};g.c=function(a,b){return D.l(a,this,b)};g.P=function(){return this.hb}; +g.T=function(a,b){return new rd(this.fb,this.name,this.Zb,this.Oc,b)};g.U=function(){var a=this.Oc;return null!=a?a:this.Oc=a=pd(kd(this.name),nd(this.fb))};g.hd=function(){return this.name};g.jd=function(){return this.fb};g.R=function(a,b){return Jc(b,this.Zb)};var td=function td(a){switch(arguments.length){case 1:return td.h(arguments[0]);case 2:return td.c(arguments[0],arguments[1]);default:throw Error(["Invalid arity: ",v.h(arguments.length)].join(""));}}; +td.h=function(a){if(a instanceof rd)return a;var b=a.indexOf("/");return 1>b?td.c(null,a):td.c(a.substring(0,b),a.substring(b+1,a.length))};td.c=function(a,b){var c=null!=a?[v.h(a),"/",v.h(b)].join(""):b;return new rd(a,b,c,null,null)};td.L=2;function ud(a){return null!=a?a.J&131072||q===a.Tf?!0:a.J?!1:Ab(cd,a):Ab(cd,a)} +function E(a){if(null==a)return null;if(null!=a&&(a.m&8388608||q===a.Pe))return a.S(null);if(vb(a)||"string"===typeof a)return 0===a.length?null:new Jb(a,0,null);if(Ab(Bc,a))return Cc(a);throw Error([v.h(a)," is not ISeqable"].join(""));}function y(a){if(null==a)return null;if(null!=a&&(a.m&64||q===a.G))return a.Ia(null);a=E(a);return null==a?null:Wb(a)}function vd(a){return null!=a?null!=a&&(a.m&64||q===a.G)?a.bb(null):(a=E(a))?Yb(a):wd:wd} +function z(a){return null==a?null:null!=a&&(a.m&128||q===a.Id)?a.Ka(null):E(vd(a))}var G=function G(a){switch(arguments.length){case 1:return G.h(arguments[0]);case 2:return G.c(arguments[0],arguments[1]);default:for(var c=[],d=arguments.length,e=0;;)if(e=d)return-1;!(0c&&(c+=d,c=0>c?0:c);for(;;)if(cc?d+c:c;for(;;)if(0<=c){if(G.c(Vd(a,c),b))return c;--c}else return-1}function Yd(a,b){this.o=a;this.i=b} +Yd.prototype.ja=function(){return this.ia?0:a};g.Rc=function(){var a=this.W(null);return 0d)c=1;else if(0===c)c=0;else a:for(d=0;;){var e=Ke(Vd(a,d),Vd(b,d));if(0===e&&d+1>1&1431655765;a=(a&858993459)+(a>>2&858993459);return 16843009*(a+(a>>4)&252645135)>>24} +var v=function v(a){switch(arguments.length){case 0:return v.B();case 1:return v.h(arguments[0]);default:for(var c=[],d=arguments.length,e=0;;)if(ed:e))c[d]=a.next(),d+=1;else return qf(new nf(c,0,d),Rf.h?Rf.h(a):Rf.call(null,a))}else return null},null,null)};function Sf(a,b,c,d,e,f){this.buffer=a;this.ub=b;this.pe=c;this.Rb=d;this.ye=e;this.Gf=f} +Sf.prototype.step=function(){if(this.ub!==Nf)return!0;for(;;)if(this.ub===Nf)if(this.buffer.Td()){if(this.pe)return!1;if(this.ye.ja()){if(this.Gf)var a=P(this.Rb,ae(null,this.ye.next()));else a=this.ye.next(),a=this.Rb.c?this.Rb.c(null,a):this.Rb.call(null,null,a);Hd(a)&&(this.Rb.h?this.Rb.h(null):this.Rb.call(null,null),this.pe=!0)}else this.Rb.h?this.Rb.h(null):this.Rb.call(null,null),this.pe=!0}else this.ub=this.buffer.remove();else return!0};Sf.prototype.ja=function(){return this.step()}; +Sf.prototype.next=function(){if(this.ja()){var a=this.ub;this.ub=Nf;return a}throw Error("No such element");};Sf.prototype.remove=function(){return Error("Unsupported operation")};Sf.prototype[Fb]=function(){return yd(this)}; +function Tf(a,b){var c=new Sf(Qf,Nf,!1,null,b,!1);c.Rb=function(){var b=function(a){return function(){function b(b,c){a.buffer=a.buffer.add(c);return b}var c=null;c=function(a,c){switch(arguments.length){case 0:return null;case 1:return a;case 2:return b.call(this,a,c)}throw Error("Invalid arity: "+(arguments.length-1));};c.B=function(){return null};c.h=function(a){return a};c.c=b;return c}()}(c);return a.h?a.h(b):a.call(null,b)}();return c} +function Uf(a,b){var c=Kf(b);c=Tf(a,c);c=Rf(c);return t(c)?c:wd}function Vf(a,b){for(;;){if(null==E(b))return!0;var c=y(b);c=a.h?a.h(c):a.call(null,c);if(t(c)){c=a;var d=z(b);a=c;b=d}else return!1}}function Wf(a,b){for(;;)if(E(b)){var c=y(b);c=a.h?a.h(c):a.call(null,c);if(t(c))return c;c=a;var d=z(b);a=c;b=d}else return null}function Xf(a){if(Ge(a))return 0===(a&1);throw Error(["Argument must be an integer: ",v.h(a)].join(""));} +function Yf(a){return function(){function b(b,c){return wb(a.c?a.c(b,c):a.call(null,b,c))}function c(b){return wb(a.h?a.h(b):a.call(null,b))}function d(){return wb(a.B?a.B():a.call(null))}var e=null,f=function(){function b(a,b,d){var e=null;if(2a?0:a-1>>>5<<5}function Jg(a,b,c){for(;;){if(0===b)return c;var d=Gg(a);d.o[0]=c;c=d;b-=5}} +var Kg=function Kg(a,b,c,d){var f=Hg(c),h=a.F-1>>>b&31;5===b?f.o[h]=d:(c=c.o[h],null!=c?(b-=5,a=Kg.M?Kg.M(a,b,c,d):Kg.call(null,a,b,c,d)):a=Jg(null,b-5,d),f.o[h]=a);return f};function Lg(a,b){throw Error(["No item ",v.h(a)," in vector of length ",v.h(b)].join(""));}function Mg(a,b){if(b>=Ig(a))return a.fa;for(var c=a.root,d=a.shift;;)if(0>>d&31];d=e}else return c.o} +var Ng=function Ng(a,b,c,d,e){var h=Hg(c);if(0===b)h.o[d&31]=e;else{var k=d>>>b&31;b-=5;c=c.o[k];a=Ng.Z?Ng.Z(a,b,c,d,e):Ng.call(null,a,b,c,d,e);h.o[k]=a}return h},Og=function Og(a,b,c){var e=a.F-2>>>b&31;if(5=this.F)a=new Jb(this.fa,0,null);else{a:{a=this.root;for(var b=this.shift;;)if(0this.F-Ig(this)){for(var c=this.fa.length,d=Array(c+1),e=0;;)if(e>>5>1<b)return new R(null,b,5,T,a,null);for(var c=32,d=(new R(null,32,5,T,a.slice(0,32),null)).Pc(null);;)if(cb||this.end<=this.start+b?Lg(b,this.end-this.start):A.c(this.Ja,this.start+b)};g.ka=function(a,b,c){return 0>b||this.end<=this.start+b?c:A.l(this.Ja,this.start+b,c)}; +g.dc=function(a,b,c){a=this.start+b;if(0>b||this.end+1<=a)throw Error(["Index ",v.h(b)," out of bounds [0,",v.h(this.W(null)),"]"].join(""));b=this.meta;c=K.l(this.Ja,a,c);var d=this.end;a+=1;return Zg(b,c,this.start,d>a?d:a,null)};g.ba=function(){return null!=this.Ja&&q===this.Ja.fe?Qg(this.Ja,this.start,this.end):new Jf(Hf,this)};g.P=function(){return this.meta};g.W=function(){return this.end-this.start};g.Ac=function(){return A.c(this.Ja,this.end-1)}; +g.Bc=function(){if(this.start===this.end)throw Error("Can't pop empty vector");return Zg(this.meta,this.Ja,this.start,this.end-1,null)};g.Rc=function(){return this.start!==this.end?new Zd(this,this.end-this.start-1,null):null};g.U=function(){var a=this.w;return null!=a?a:this.w=a=Ad(this)};g.K=function(a,b){return $d(this,b)};g.oa=function(){return tc(he,this.meta)};g.Fa=function(a,b){return null!=this.Ja&&q===this.Ja.fe?Rg(this.Ja,b,this.start,this.end):Kd(this,b)}; +g.Ga=function(a,b,c){return null!=this.Ja&&q===this.Ja.fe?Sg(this.Ja,b,c,this.start,this.end):Ld(this,b,c)};g.O=function(a,b,c){if("number"===typeof b)return this.dc(null,b,c);throw Error("Subvec's key for assoc must be a number.");};g.S=function(){var a=this;return function(b){return function e(d){return d===a.end?null:ae(A.c(a.Ja,d),new kf(null,function(){return function(){return e(d+1)}}(b),null,null))}}(this)(a.start)};g.T=function(a,b){return Zg(b,this.Ja,this.start,this.end,this.w)}; +g.X=function(a,b){return Zg(this.meta,qc(this.Ja,this.end,b),this.start,this.end+1,null)};g.call=function(){var a=null;a=function(a,c,d){switch(arguments.length){case 2:return this.$(null,c);case 3:return this.ka(null,c,d)}throw Error("Invalid arity: "+(arguments.length-1));};a.c=function(a,c){return this.$(null,c)};a.l=function(a,c,d){return this.ka(null,c,d)};return a}();g.apply=function(a,b){return this.call.apply(this,[this].concat(Gb(b)))};g.h=function(a){return this.$(null,a)}; +g.c=function(a,b){return this.ka(null,a,b)};Yg.prototype[Fb]=function(){return yd(this)};function Zg(a,b,c,d,e){for(;;)if(b instanceof Yg)c=b.start+c,d=b.start+d,b=b.Ja;else{if(!ze(b))throw Error("v must satisfy IVector");var f=H(b);if(0>c||0>d||c>f||d>f)throw Error("Index out of bounds");return new Yg(a,b,c,d,e)}}function $g(a,b){return a===b.la?b:new Fg(a,Gb(b.o))} +var ah=function ah(a,b,c,d){c=$g(a.root.la,c);var f=a.F-1>>>b&31;if(5===b)a=d;else{var h=c.o[f];null!=h?(b-=5,a=ah.M?ah.M(a,b,h,d):ah.call(null,a,b,h,d)):a=Jg(a.root.la,b-5,d)}c.o[f]=a;return c};function Tg(a,b,c,d){this.F=a;this.shift=b;this.root=c;this.fa=d;this.J=88;this.m=275}g=Tg.prototype; +g.Dc=function(a,b){if(this.root.la){if(32>this.F-Ig(this))this.fa[this.F&31]=b;else{var c=new Fg(this.root.la,this.fa),d=[null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null];d[0]=b;this.fa=d;if(this.F>>>5>1<>>d&31,m=k(d-5,f.o[p]);f.o[p]=m}return f}}(a)(a.shift,a.root)}();a.root=d}return a}if(b===a.F)return a.Dc(null,c);throw Error(["Index ",v.h(b)," out of bounds for TransientVector of length",v.h(a.F)].join(""));}throw Error("assoc! after persistent!");} +g.W=function(){if(this.root.la)return this.F;throw Error("count after persistent!");};g.$=function(a,b){if(this.root.la)return(0<=b&&b=c)return new r(this.meta,this.F-1,d,null);G.c(b,this.o[e])||(d[f]=this.o[e],d[f+1]=this.o[e+1],f+=2);e+=2}}else return this}; +g.O=function(a,b,c){a=ih(this.o,b);if(-1===a){if(this.Fb?4:2*(b+1));Be(this.o,0,c,0,2*b);return new xh(a,this.na,c)};g.qd=function(){return yh(this.o,0,null)};g.Jc=function(a,b){return vh(this.o,a,b)};g.sc=function(a,b,c,d){var e=1<<(b>>>a&31);if(0===(this.na&e))return d;var f=$e(this.na&e-1);e=this.o[2*f];f=this.o[2*f+1];return null==e?f.sc(a+5,b,c,d):rh(c,e)?f:d}; +g.Kb=function(a,b,c,d,e,f){var h=1<<(c>>>b&31),k=$e(this.na&h-1);if(0===(this.na&h)){var l=$e(this.na);if(2*l>>b&31]=zh.Kb(a,b+5,c,d,e,f);for(e=d=0;;)if(32>d)0!== +(this.na>>>d&1)&&(k[d]=null!=this.o[e]?zh.Kb(a,b+5,od(this.o[e]),this.o[e],this.o[e+1],f):this.o[e+1],e+=2),d+=1;else break;return new Ah(a,l+1,k)}b=Array(2*(l+4));Be(this.o,0,b,0,2*k);b[2*k]=d;b[2*k+1]=e;Be(this.o,2*k,b,2*(k+1),2*(l-k));f.H=!0;a=this.Gc(a);a.o=b;a.na|=h;return a}l=this.o[2*k];h=this.o[2*k+1];if(null==l)return l=h.Kb(a,b+5,c,d,e,f),l===h?this:uh(this,a,2*k+1,l);if(rh(d,l))return e===h?this:uh(this,a,2*k+1,e);f.H=!0;f=b+5;b=od(l);if(b===c)e=new Bh(null,b,2,[l,h,d,e]);else{var p=new qh; +e=zh.Kb(a,f,b,l,h,p).Kb(a,f,c,d,e,p)}d=2*k;k=2*k+1;a=this.Gc(a);a.o[d]=null;a.o[k]=e;return a}; +g.Jb=function(a,b,c,d,e){var f=1<<(b>>>a&31),h=$e(this.na&f-1);if(0===(this.na&f)){var k=$e(this.na);if(16<=k){h=[null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null];h[b>>>a&31]=zh.Jb(a+5,b,c,d,e);for(d=c=0;;)if(32>c)0!==(this.na>>>c&1)&&(h[c]=null!=this.o[d]?zh.Jb(a+5,od(this.o[d]),this.o[d],this.o[d+1],e):this.o[d+1],d+=2),c+=1;else break;return new Ah(null,k+1,h)}a=Array(2*(k+1));Be(this.o, +0,a,0,2*h);a[2*h]=c;a[2*h+1]=d;Be(this.o,2*h,a,2*(h+1),2*(k-h));e.H=!0;return new xh(null,this.na|f,a)}var l=this.o[2*h];f=this.o[2*h+1];if(null==l)return k=f.Jb(a+5,b,c,d,e),k===f?this:new xh(null,this.na,sh(this.o,2*h+1,k));if(rh(c,l))return d===f?this:new xh(null,this.na,sh(this.o,2*h+1,d));e.H=!0;e=this.na;k=this.o;a+=5;var p=od(l);if(p===b)c=new Bh(null,p,2,[l,f,c,d]);else{var m=new qh;c=zh.Jb(a,p,l,f,m).Jb(a,b,c,d,m)}a=2*h;h=2*h+1;d=Gb(k);d[a]=null;d[h]=c;return new xh(null,e,d)}; +g.rd=function(a,b,c){var d=1<<(b>>>a&31);if(0===(this.na&d))return this;var e=$e(this.na&d-1),f=this.o[2*e],h=this.o[2*e+1];return null==f?(a=h.rd(a+5,b,c),a===h?this:null!=a?new xh(null,this.na,sh(this.o,2*e+1,a)):this.na===d?null:new xh(null,this.na^d,th(this.o,e))):rh(c,f)?new xh(null,this.na^d,th(this.o,e)):this};g.ba=function(){return new wh(this.o,0,null,null)};var zh=new xh(null,0,[]);function Ch(a,b,c){this.o=a;this.i=b;this.Lb=c} +Ch.prototype.ja=function(){for(var a=this.o.length;;){if(null!=this.Lb&&this.Lb.ja())return!0;if(this.i>>a&31];return null!=e?e.sc(a+5,b,c,d):d};g.Kb=function(a,b,c,d,e,f){var h=c>>>b&31,k=this.o[h];if(null==k)return a=uh(this,a,h,zh.Kb(a,b+5,c,d,e,f)),a.F+=1,a;b=k.Kb(a,b+5,c,d,e,f);return b===k?this:uh(this,a,h,b)}; +g.Jb=function(a,b,c,d,e){var f=b>>>a&31,h=this.o[f];if(null==h)return new Ah(null,this.F+1,sh(this.o,f,zh.Jb(a+5,b,c,d,e)));a=h.Jb(a+5,b,c,d,e);return a===h?this:new Ah(null,this.F,sh(this.o,f,a))}; +g.rd=function(a,b,c){var d=b>>>a&31,e=this.o[d];if(null!=e){a=e.rd(a+5,b,c);if(a===e)d=this;else if(null==a)if(8>=this.F)a:{e=this.o;a=e.length;b=Array(2*(this.F-1));c=0;for(var f=1,h=0;;)if(ca?d:rh(c,this.o[a])?this.o[a+1]:d}; +g.Kb=function(a,b,c,d,e,f){if(c===this.ec){b=Eh(this.o,this.F,d);if(-1===b){if(this.o.length>2*this.F)return b=2*this.F,c=2*this.F+1,a=this.Gc(a),a.o[b]=d,a.o[c]=e,f.H=!0,a.F+=1,a;c=this.o.length;b=Array(c+2);Be(this.o,0,b,0,c);b[c]=d;b[c+1]=e;f.H=!0;d=this.F+1;a===this.la?(this.o=b,this.F=d,a=this):a=new Bh(this.la,this.ec,d,b);return a}return this.o[b+1]===e?this:uh(this,a,b+1,e)}return(new xh(a,1<<(this.ec>>>b&31),[null,this,null,null])).Kb(a,b,c,d,e,f)}; +g.Jb=function(a,b,c,d,e){return b===this.ec?(a=Eh(this.o,this.F,c),-1===a?(a=2*this.F,b=Array(a+2),Be(this.o,0,b,0,a),b[a]=c,b[a+1]=d,e.H=!0,new Bh(null,this.ec,this.F+1,b)):G.c(this.o[a+1],d)?this:new Bh(null,this.ec,this.F,sh(this.o,a+1,d))):(new xh(null,1<<(this.ec>>>a&31),[null,this])).Jb(a,b,c,d,e)};g.rd=function(a,b,c){a=Eh(this.o,this.F,c);return-1===a?this:1===this.F?null:new Bh(null,this.ec,this.F-1,th(this.o,Ze(a)))};g.ba=function(){return new wh(this.o,0,null,null)}; +function Fh(a,b,c,d,e){this.meta=a;this.Mb=b;this.i=c;this.s=d;this.w=e;this.m=32374988;this.J=0}g=Fh.prototype;g.toString=function(){return fd(this)};g.equiv=function(a){return this.K(null,a)};g.indexOf=function(){var a=null;a=function(a,c){switch(arguments.length){case 1:return Ud(this,a,0);case 2:return Ud(this,a,c)}throw Error("Invalid arity: "+(arguments.length-1));};a.h=function(a){return Ud(this,a,0)};a.c=function(a,c){return Ud(this,a,c)};return a}(); +g.lastIndexOf=function(){function a(a){return Xd(this,a,H(this))}var b=null;b=function(b,d){switch(arguments.length){case 1:return a.call(this,b);case 2:return Xd(this,b,d)}throw Error("Invalid arity: "+(arguments.length-1));};b.h=a;b.c=function(a,b){return Xd(this,a,b)};return b}();g.P=function(){return this.meta};g.Ka=function(){return null==this.s?yh(this.Mb,this.i+2,null):yh(this.Mb,this.i,z(this.s))};g.U=function(){var a=this.w;return null!=a?a:this.w=a=Ad(this)}; +g.K=function(a,b){return $d(this,b)};g.oa=function(){return tc(wd,this.meta)};g.Fa=function(a,b){return ce(b,this)};g.Ga=function(a,b,c){return de(b,c,this)};g.Ia=function(){return null==this.s?new R(null,2,5,T,[this.Mb[this.i],this.Mb[this.i+1]],null):y(this.s)};g.bb=function(){var a=null==this.s?yh(this.Mb,this.i+2,null):yh(this.Mb,this.i,z(this.s));return null!=a?a:wd};g.S=function(){return this};g.T=function(a,b){return new Fh(b,this.Mb,this.i,this.s,this.w)};g.X=function(a,b){return ae(b,this)}; +Fh.prototype[Fb]=function(){return yd(this)};function yh(a,b,c){if(null==c)for(c=a.length;;)if(bthis.F?H(z(this))+1:this.F};g.U=function(){var a=this.w;return null!=a?a:this.w=a=Ad(this)};g.K=function(a,b){return $d(this,b)};g.oa=function(){return tc(wd,this.meta)};g.Fa=function(a,b){return ce(b,this)};g.Ga=function(a,b,c){return de(b,c,this)};g.Ia=function(){var a=this.stack;return null==a?null:nc(a)};g.bb=function(){var a=y(this.stack);a=Mh(this.vc?a.right:a.left,z(this.stack),this.vc);return null!=a?new Nh(null,a,this.vc,this.F-1,null):wd};g.S=function(){return this}; +g.T=function(a,b){return new Nh(b,this.stack,this.vc,this.F,this.w)};g.X=function(a,b){return ae(b,this)};Nh.prototype[Fb]=function(){return yd(this)};function Oh(a,b,c){return new Nh(null,Mh(a,null,b),b,c,null)} +function Ph(a,b,c,d){return c instanceof Qh?c.left instanceof Qh?new Qh(c.key,c.H,c.left.bc(),new Rh(a,b,c.right,d,null),null):c.right instanceof Qh?new Qh(c.right.key,c.right.H,new Rh(c.key,c.H,c.left,c.right.left,null),new Rh(a,b,c.right.right,d,null),null):new Rh(a,b,c,d,null):new Rh(a,b,c,d,null)} +function Sh(a,b,c,d){return d instanceof Qh?d.right instanceof Qh?new Qh(d.key,d.H,new Rh(a,b,c,d.left,null),d.right.bc(),null):d.left instanceof Qh?new Qh(d.left.key,d.left.H,new Rh(a,b,c,d.left.left,null),new Rh(d.key,d.H,d.left.right,d.right,null),null):new Rh(a,b,c,d,null):new Rh(a,b,c,d,null)} +function Th(a,b,c,d){if(c instanceof Qh)return new Qh(a,b,c.bc(),d,null);if(d instanceof Rh)return Sh(a,b,c,d.ud());if(d instanceof Qh&&d.left instanceof Rh)return new Qh(d.left.key,d.left.H,new Rh(a,b,c,d.left.left,null),Sh(d.key,d.H,d.left.right,d.right.ud()),null);throw Error("red-black tree invariant violation");} +function Uh(a,b,c,d){if(d instanceof Qh)return new Qh(a,b,c,d.bc(),null);if(c instanceof Rh)return Ph(a,b,c.ud(),d);if(c instanceof Qh&&c.right instanceof Rh)return new Qh(c.right.key,c.right.H,Ph(c.key,c.H,c.left.ud(),c.right.left),new Rh(a,b,c.right.right,d,null),null);throw Error("red-black tree invariant violation");} +var Vh=function Vh(a,b,c){var e=null!=a.left?function(){var e=a.left;return Vh.l?Vh.l(e,b,c):Vh.call(null,e,b,c)}():c;if(Hd(e))return e;var f=function(){var c=a.key,f=a.H;return b.l?b.l(e,c,f):b.call(null,e,c,f)}();if(Hd(f))return f;if(null!=a.right){var h=a.right;return Vh.l?Vh.l(h,b,f):Vh.call(null,h,b,f)}return f};function Rh(a,b,c,d,e){this.key=a;this.H=b;this.left=c;this.right=d;this.w=e;this.m=32402207;this.J=0}g=Rh.prototype; +g.lastIndexOf=function(){function a(a){return Xd(this,a,H(this))}var b=null;b=function(b,d){switch(arguments.length){case 1:return a.call(this,b);case 2:return Xd(this,b,d)}throw Error("Invalid arity: "+(arguments.length-1));};b.h=a;b.c=function(a,b){return Xd(this,a,b)};return b}(); +g.indexOf=function(){var a=null;a=function(a,c){switch(arguments.length){case 1:return Ud(this,a,0);case 2:return Ud(this,a,c)}throw Error("Invalid arity: "+(arguments.length-1));};a.h=function(a){return Ud(this,a,0)};a.c=function(a,c){return Ud(this,a,c)};return a}();g.Ee=function(a){return a.He(this)};g.ud=function(){return new Qh(this.key,this.H,this.left,this.right,null)};g.bc=function(){return this};g.De=function(a){return a.Ge(this)};g.replace=function(a,b,c,d){return new Rh(a,b,c,d,null)}; +g.Ge=function(a){return new Rh(a.key,a.H,this,a.right,null)};g.He=function(a){return new Rh(a.key,a.H,a.left,this,null)};g.Jc=function(a,b){return Vh(this,a,b)};g.V=function(a,b){return this.ka(null,b,null)};g.I=function(a,b,c){return this.ka(null,b,c)};g.$=function(a,b){if(0===b)return this.key;if(1===b)return this.H;throw Error("Index out of bounds");};g.ka=function(a,b,c){return 0===b?this.key:1===b?this.H:c};g.dc=function(a,b,c){return(new R(null,2,5,T,[this.key,this.H],null)).dc(null,b,c)}; +g.P=function(){return null};g.W=function(){return 2};g.fd=function(){return this.key};g.gd=function(){return this.H};g.Ac=function(){return this.H};g.Bc=function(){return new R(null,1,5,T,[this.key],null)};g.U=function(){var a=this.w;return null!=a?a:this.w=a=Ad(this)};g.K=function(a,b){return $d(this,b)};g.oa=function(){return he};g.Fa=function(a,b){return Kd(this,b)};g.Ga=function(a,b,c){return Ld(this,b,c)};g.O=function(a,b,c){return K.l(new R(null,2,5,T,[this.key,this.H],null),b,c)}; +g.yc=function(a,b){return 0===b||1===b};g.S=function(){var a=this.key;return Tb(Tb(wd,this.H),a)};g.T=function(a,b){return tc(new R(null,2,5,T,[this.key,this.H],null),b)};g.X=function(a,b){return new R(null,3,5,T,[this.key,this.H,b],null)}; +g.call=function(){var a=null;a=function(a,c,d){switch(arguments.length){case 2:return this.$(null,c);case 3:return this.ka(null,c,d)}throw Error("Invalid arity: "+(arguments.length-1));};a.c=function(a,c){return this.$(null,c)};a.l=function(a,c,d){return this.ka(null,c,d)};return a}();g.apply=function(a,b){return this.call.apply(this,[this].concat(Gb(b)))};g.h=function(a){return this.$(null,a)};g.c=function(a,b){return this.ka(null,a,b)};Rh.prototype[Fb]=function(){return yd(this)}; +function Qh(a,b,c,d,e){this.key=a;this.H=b;this.left=c;this.right=d;this.w=e;this.m=32402207;this.J=0}g=Qh.prototype;g.lastIndexOf=function(){function a(a){return Xd(this,a,H(this))}var b=null;b=function(b,d){switch(arguments.length){case 1:return a.call(this,b);case 2:return Xd(this,b,d)}throw Error("Invalid arity: "+(arguments.length-1));};b.h=a;b.c=function(a,b){return Xd(this,a,b)};return b}(); +g.indexOf=function(){var a=null;a=function(a,c){switch(arguments.length){case 1:return Ud(this,a,0);case 2:return Ud(this,a,c)}throw Error("Invalid arity: "+(arguments.length-1));};a.h=function(a){return Ud(this,a,0)};a.c=function(a,c){return Ud(this,a,c)};return a}();g.Ee=function(a){return new Qh(this.key,this.H,this.left,a,null)};g.ud=function(){throw Error("red-black tree invariant violation");};g.bc=function(){return new Rh(this.key,this.H,this.left,this.right,null)}; +g.De=function(a){return new Qh(this.key,this.H,a,this.right,null)};g.replace=function(a,b,c,d){return new Qh(a,b,c,d,null)};g.Ge=function(a){return this.left instanceof Qh?new Qh(this.key,this.H,this.left.bc(),new Rh(a.key,a.H,this.right,a.right,null),null):this.right instanceof Qh?new Qh(this.right.key,this.right.H,new Rh(this.key,this.H,this.left,this.right.left,null),new Rh(a.key,a.H,this.right.right,a.right,null),null):new Rh(a.key,a.H,this,a.right,null)}; +g.He=function(a){return this.right instanceof Qh?new Qh(this.key,this.H,new Rh(a.key,a.H,a.left,this.left,null),this.right.bc(),null):this.left instanceof Qh?new Qh(this.left.key,this.left.H,new Rh(a.key,a.H,a.left,this.left.left,null),new Rh(this.key,this.H,this.left.right,this.right,null),null):new Rh(a.key,a.H,a.left,this,null)};g.Jc=function(a,b){return Vh(this,a,b)};g.V=function(a,b){return this.ka(null,b,null)};g.I=function(a,b,c){return this.ka(null,b,c)}; +g.$=function(a,b){if(0===b)return this.key;if(1===b)return this.H;throw Error("Index out of bounds");};g.ka=function(a,b,c){return 0===b?this.key:1===b?this.H:c};g.dc=function(a,b,c){return(new R(null,2,5,T,[this.key,this.H],null)).dc(null,b,c)};g.P=function(){return null};g.W=function(){return 2};g.fd=function(){return this.key};g.gd=function(){return this.H};g.Ac=function(){return this.H};g.Bc=function(){return new R(null,1,5,T,[this.key],null)}; +g.U=function(){var a=this.w;return null!=a?a:this.w=a=Ad(this)};g.K=function(a,b){return $d(this,b)};g.oa=function(){return he};g.Fa=function(a,b){return Kd(this,b)};g.Ga=function(a,b,c){return Ld(this,b,c)};g.O=function(a,b,c){return K.l(new R(null,2,5,T,[this.key,this.H],null),b,c)};g.yc=function(a,b){return 0===b||1===b};g.S=function(){var a=this.key;return Tb(Tb(wd,this.H),a)};g.T=function(a,b){return tc(new R(null,2,5,T,[this.key,this.H],null),b)}; +g.X=function(a,b){return new R(null,3,5,T,[this.key,this.H,b],null)};g.call=function(){var a=null;a=function(a,c,d){switch(arguments.length){case 2:return this.$(null,c);case 3:return this.ka(null,c,d)}throw Error("Invalid arity: "+(arguments.length-1));};a.c=function(a,c){return this.$(null,c)};a.l=function(a,c,d){return this.ka(null,c,d)};return a}();g.apply=function(a,b){return this.call.apply(this,[this].concat(Gb(b)))};g.h=function(a){return this.$(null,a)}; +g.c=function(a,b){return this.ka(null,a,b)};Qh.prototype[Fb]=function(){return yd(this)}; +var Wh=function Wh(a,b,c,d,e){if(null==b)return new Qh(c,d,null,null,null);var h=function(){var d=b.key;return a.c?a.c(c,d):a.call(null,c,d)}();if(0===h)return e[0]=b,null;if(0>h)return h=function(){var h=b.left;return Wh.Z?Wh.Z(a,h,c,d,e):Wh.call(null,a,h,c,d,e)}(),null!=h?b.De(h):null;h=function(){var h=b.right;return Wh.Z?Wh.Z(a,h,c,d,e):Wh.call(null,a,h,c,d,e)}();return null!=h?b.Ee(h):null},Xh=function Xh(a,b){if(null==a)return b;if(null==b)return a;if(a instanceof Qh){if(b instanceof Qh){var d= +function(){var d=a.right,f=b.left;return Xh.c?Xh.c(d,f):Xh.call(null,d,f)}();return d instanceof Qh?new Qh(d.key,d.H,new Qh(a.key,a.H,a.left,d.left,null),new Qh(b.key,b.H,d.right,b.right,null),null):new Qh(a.key,a.H,a.left,new Qh(b.key,b.H,d,b.right,null),null)}return new Qh(a.key,a.H,a.left,function(){var d=a.right;return Xh.c?Xh.c(d,b):Xh.call(null,d,b)}(),null)}if(b instanceof Qh)return new Qh(b.key,b.H,function(){var d=b.left;return Xh.c?Xh.c(a,d):Xh.call(null,a,d)}(),b.right,null);d=function(){var d= +a.right,f=b.left;return Xh.c?Xh.c(d,f):Xh.call(null,d,f)}();return d instanceof Qh?new Qh(d.key,d.H,new Rh(a.key,a.H,a.left,d.left,null),new Rh(b.key,b.H,d.right,b.right,null),null):Th(a.key,a.H,a.left,new Rh(b.key,b.H,d,b.right,null))},Yh=function Yh(a,b,c,d){if(null!=b){var f=function(){var d=b.key;return a.c?a.c(c,d):a.call(null,c,d)}();if(0===f)return d[0]=b,Xh(b.left,b.right);if(0>f)return f=function(){var f=b.left;return Yh.M?Yh.M(a,f,c,d):Yh.call(null,a,f,c,d)}(),null!=f||null!=d[0]?b.left instanceof +Rh?Th(b.key,b.H,f,b.right):new Qh(b.key,b.H,f,b.right,null):null;f=function(){var f=b.right;return Yh.M?Yh.M(a,f,c,d):Yh.call(null,a,f,c,d)}();return null!=f||null!=d[0]?b.right instanceof Rh?Uh(b.key,b.H,b.left,f):new Qh(b.key,b.H,b.left,f,null):null}return null},Zh=function Zh(a,b,c,d){var f=b.key,h=a.c?a.c(c,f):a.call(null,c,f);return 0===h?b.replace(f,d,b.left,b.right):0>h?b.replace(f,b.H,function(){var f=b.left;return Zh.M?Zh.M(a,f,c,d):Zh.call(null,a,f,c,d)}(),b.right):b.replace(f,b.H,b.left, +function(){var f=b.right;return Zh.M?Zh.M(a,f,c,d):Zh.call(null,a,f,c,d)}())};function $h(a,b,c,d,e){this.Bb=a;this.mc=b;this.F=c;this.meta=d;this.w=e;this.m=418776847;this.J=8192}g=$h.prototype;g.forEach=function(a){for(var b=E(this),c=null,d=0,e=0;;)if(ed?c.left:c.right}else return null}g.has=function(a){return He(this,a)};g.V=function(a,b){return this.I(null,b,null)}; +g.I=function(a,b,c){a=ai(this,b);return null!=a?a.H:c};g.Qc=function(a,b,c){return null!=this.mc?Jd(Vh(this.mc,b,c)):c};g.P=function(){return this.meta};g.W=function(){return this.F};g.Rc=function(){return 0(a.h?a.h(c):a.call(null,c))?b:c};Ai.A=function(a,b,c,d){return Mb(function(b,c){return Ai.l(a,b,c)},Ai.l(a,b,c),d)};Ai.N=function(a){var b=y(a),c=z(a);a=y(c);var d=z(c);c=y(d);d=z(d);return Ai.A(b,a,c,d)};Ai.L=3;function Bi(a,b){return new kf(null,function(){var c=E(b);if(c){var d=y(c);d=a.h?a.h(d):a.call(null,d);c=t(d)?ae(y(c),Bi(a,vd(c))):null}else c=null;return c},null,null)}function Di(a,b,c){this.i=a;this.end=b;this.step=c} +Di.prototype.ja=function(){return 0this.end};Di.prototype.next=function(){var a=this.i;this.i+=this.step;return a};function Ei(a,b,c,d,e){this.meta=a;this.start=b;this.end=c;this.step=d;this.w=e;this.m=32375006;this.J=139264}g=Ei.prototype;g.toString=function(){return fd(this)};g.equiv=function(a){return this.K(null,a)}; +g.indexOf=function(){var a=null;a=function(a,c){switch(arguments.length){case 1:return Ud(this,a,0);case 2:return Ud(this,a,c)}throw Error("Invalid arity: "+(arguments.length-1));};a.h=function(a){return Ud(this,a,0)};a.c=function(a,c){return Ud(this,a,c)};return a}(); +g.lastIndexOf=function(){function a(a){return Xd(this,a,H(this))}var b=null;b=function(b,d){switch(arguments.length){case 1:return a.call(this,b);case 2:return Xd(this,b,d)}throw Error("Invalid arity: "+(arguments.length-1));};b.h=a;b.c=function(a,b){return Xd(this,a,b)};return b}();g.$=function(a,b){if(0<=b&&bthis.end&&0===this.step)return this.start;throw Error("Index out of bounds");}; +g.ka=function(a,b,c){return 0<=b&&bthis.end&&0===this.step?this.start:c};g.ba=function(){return new Di(this.start,this.end,this.step)};g.P=function(){return this.meta};g.Ka=function(){return 0this.end?new Ei(this.meta,this.start+this.step,this.end,this.step,null):null}; +g.W=function(){return wb(this.S(null))?0:Math.ceil((this.end-this.start)/this.step)};g.U=function(){var a=this.w;return null!=a?a:this.w=a=Ad(this)};g.K=function(a,b){return $d(this,b)};g.oa=function(){return tc(wd,this.meta)};g.Fa=function(a,b){return Kd(this,b)};g.Ga=function(a,b,c){for(a=this.start;;)if(0this.end){c=b.c?b.c(c,a):b.call(null,c,a);if(Hd(c))return B(c);a+=this.step}else return c};g.Ia=function(){return null==this.S(null)?null:this.start}; +g.bb=function(){return null!=this.S(null)?new Ei(this.meta,this.start+this.step,this.end,this.step,null):wd};g.S=function(){return 0this.step?this.start>this.end?this:null:this.start===this.end?null:this};g.T=function(a,b){return new Ei(b,this.start,this.end,this.step,this.w)};g.X=function(a,b){return ae(b,this)};Ei.prototype[Fb]=function(){return yd(this)};function Fi(a,b,c){return new Ei(null,a,b,c,null)} +function Gi(a,b){return new R(null,2,5,T,[Bi(a,b),ng(a,b)],null)} +function Hi(a){var b=y;return function(){function c(c,d,e){return new R(null,2,5,T,[b.l?b.l(c,d,e):b.call(null,c,d,e),a.l?a.l(c,d,e):a.call(null,c,d,e)],null)}function d(c,d){return new R(null,2,5,T,[b.c?b.c(c,d):b.call(null,c,d),a.c?a.c(c,d):a.call(null,c,d)],null)}function e(c){return new R(null,2,5,T,[b.h?b.h(c):b.call(null,c),a.h?a.h(c):a.call(null,c)],null)}function f(){return new R(null,2,5,T,[b.B?b.B():b.call(null),a.B?a.B():a.call(null)],null)}var h=null,k=function(){function c(a,b,c,e){var f= +null;if(3lb)return Jc(a,"#");Jc(a,c);if(0===tb.h(f))E(h)&&Jc(a,function(){var a=Ki.h(f);return t(a)?a:"..."}());else{if(E(h)){var l=y(h);b.l?b.l(l,a,f):b.call(null,l,a,f)}for(var p=z(h),m=tb.h(f)-1;;)if(!p||null!=m&&0===m){E(p)&&0===m&&(Jc(a,d),Jc(a,function(){var a=Ki.h(f);return t(a)?a:"..."}()));break}else{Jc(a,d);var u=y(p);c=a;h=f;b.l?b.l(u,c,h):b.call(null,u,c,h);var w=z(p);c=m-1;p=w;m=c}}return Jc(a,e)}finally{lb=k}} +function Li(a,b){for(var c=E(b),d=null,e=0,f=0;;)if(fH(a)?a.toUpperCase():[v.h(a.substring(0,1).toUpperCase()),v.h(a.substring(1))].join("")} +function Qo(a){if("string"===typeof a)return a;a=jf(a);var b=Fo(a,/-/),c=E(b);b=y(c);c=z(c);return t(Oo.h?Oo.h(b):Oo.call(null,b))?a:Kb(v,b,ig.c(Po,c))}function Ro(a){var b=function(){var b=function(){var b=me(a);return b?(b=a.displayName,t(b)?b:a.name):b}();if(t(b))return b;b=function(){var b=null!=a?a.J&4096||q===a.Oe?!0:!1:!1;return b?jf(a):b}();if(t(b))return b;b=qe(a);return xe(b)?Tk.h(b):null}();return Do(""+v.h(b),"$",".")}var So=!1;if("undefined"===typeof To)var To=0;function Uo(a){return setTimeout(a,16)}var Vo="undefined"===typeof window||null==window.document?Uo:function(){var a=window,b=a.requestAnimationFrame;if(t(b))return b;b=a.webkitRequestAnimationFrame;if(t(b))return b;b=a.mozRequestAnimationFrame;if(t(b))return b;a=a.msRequestAnimationFrame;return t(a)?a:Uo}();function Wo(a,b){return a.cljsMountOrder-b.cljsMountOrder}if("undefined"===typeof Xo)var Xo=function(){return null};function Yo(a){this.Yd=a} +function Zo(a,b){var c=a[b];if(null==c)return null;a[b]=null;for(var d=c.length,e=0;;)if(e=d&&a.push(gq(c));return a}}(e),[b,c],a))}};if("undefined"===typeof jq)var jq=null;function kq(){if(null!=jq)return jq;if("undefined"!==typeof ReactDOM)return jq=ReactDOM;if("undefined"!==typeof require){var a=jq=require("react-dom");if(t(a))return a;throw Error("require('react-dom') failed");}throw Error("js/ReactDOM is missing");}if("undefined"===typeof lq)var lq=dg.h(Ef); +function mq(a,b,c){var d=So;So=!0;try{return kq().render(a.B?a.B():a.call(null),b,function(){return function(){var d=So;So=!1;try{return gg.M(lq,K,b,new R(null,2,5,T,[a,b],null)),Zo(bp,"afterRender"),null!=c?c.B?c.B():c.call(null):null}finally{So=d}}}(d))}finally{So=d}}function nq(a,b){return mq(a,b,null)}function oq(a,b,c){qp();return mq(function(){return gq(me(a)?a.B?a.B():a.call(null):a)},b,c)}Wp=function(a){return kq().findDOMNode(a)};function pq(a){switch(arguments.length){case 2:return oq(arguments[0],arguments[1],null);case 3:return oq(arguments[0],arguments[1],arguments[2]);default:throw Error(["Invalid arity: ",v.h(arguments.length)].join(""));}}function qq(a,b){return oq(a,b,null)} +da("reagent.core.force_update_all",function(){qp();qp();for(var a=E(mh(B(lq))),b=null,c=0,d=0;;)if(d=Number(c)?a:a=-1Number(a)?"-":0<=b.indexOf("+")?"+":0<=b.indexOf(" ")?" ":"";0<=Number(a)&&(d=f+d);if(isNaN(c)||d.length>=Number(c))return d;d=isNaN(e)?Math.abs(Number(a)).toString():Math.abs(Number(a)).toFixed(e);a=Number(c)-d.length-f.length;0<=b.indexOf("-",0)?d=f+d+sa(" ",a):(b=0<=b.indexOf("0",0)?"0":" ",d=f+sa(b,a)+d);return d};yq.fc.d=function(a,b,c,d,e,f,h,k){return yq.fc.f(parseInt(a,10),b,c,d,0,f,h,k)}; +yq.fc.i=yq.fc.d;yq.fc.u=yq.fc.d;function zq(a){var b=be([Vk,null]);return wg.c(t(a)?a:Ef,function(){return function e(a){return new kf(null,function(){for(var b=a;;)if(b=E(b)){if(Ae(b)){var d=Wc(b),k=H(d),l=of(k);a:for(var p=0;;)if(p=H(h)&&Vf(function(){return function(a){return!(a instanceof Xq)}}(b,c,d,e,f,h),h)))throw Error(Bq("%s is not a valid sequence schema; %s%s%s",be([a,"a valid sequence schema consists of zero or more `one` elements, ","followed by zero or more `optional` elements, followed by an optional ", +"schema that will match the remaining elements."])));return new R(null,2,5,T,[O.c(c,f),y(h)],null)} +R.prototype.xb=function(){var a=this,b=Zq(a),c=J(b,0,null),d=J(b,1,null);return Wg(O.c(function(){return function(a,b,c,d){return function m(e){return new kf(null,function(){return function(){for(;;){var a=E(e);if(a){if(Ae(a)){var b=Wc(a),c=H(b),d=of(c);return function(){for(var a=0;;)if(ac?f:c;return $r(a,ea?0:a}():function(){var a=e-b;return f>a?f:a}())} +function gs(a,b){var c=null!=a&&(a.m&64||q===a.G)?P(U,a):a,d=D.c(c,pl);d=null!=d&&(d.m&64||q===d.G)?P(U,d):d;var e=D.c(d,Aj),f=D.c(c,Yj),h=D.c(c,no);return $r(c,e>f?function(){var a=h-1,c=e+b;return a=a}}(l,p,a,c,c,d,e,f,h,k),h),l,p);return Zr(c,d)} +function it(a,b){var c=null!=a&&(a.m&64||q===a.G)?P(U,a):a,d=D.c(c,pl),e=null!=d&&(d.m&64||q===d.G)?P(U,d):d,f=D.c(e,zn),h=D.c(c,tk),k=D.c(c,fl),l=b-1;d=J(cf(Bi(function(a,b,c,d,e,f,h){return function(a){return h>a}}(l,a,c,c,d,e,f,h,k),h)),l,0);return Zr(c,d)}function jt(a){return K.l(a,im,Ve)}function kt(a){return K.l(a,im,Hr)}function lt(a,b,c){return K.l(a,b,c)}function mt(a,b,c){return Wg(O.A(jg(b,a),new R(null,1,5,T,[c],null),be([jg(H(a)-b-1,kg(b,a))])))} +function nt(a,b){var c=null!=a&&(a.m&64||q===a.G)?P(U,a):a,d=D.c(c,pl),e=null!=d&&(d.m&64||q===d.G)?P(U,d):d;d=D.c(e,zn);e=D.c(e,Aj);var f=D.c(c,fl);D.c(c,no);var h=D.c(c,Oj),k=D.c(c,Rj),l=D.c(c,$l),p=D.c(c,im);p=95b?p.h?p.h(b):p.call(null,b):b;h=tr(p,h);return G.c(f,d+1)?t(k)?K.l(Yr(zg(c,new R(null,3,5,T,[il,e,d],null),h),d+1),vk,!0):zg(c,new R(null,3,5,T,[il,e,d],null),h):Yr(Ag.Z(c,new R(null,2,5,T,[il,e],null),t(l)?mt:lt,d,h),d+1)} +function ot(a,b){var c=null!=a&&(a.m&64||q===a.G)?P(U,a):a,d=D.c(c,Rj),e=D.c(c,vk);t(t(d)?e:d)&&(c=null!=c&&(c.m&64||q===c.G)?P(U,c):c,d=D.c(c,pl),d=null!=d&&(d.m&64||q===d.G)?P(U,d):d,d=D.c(d,Aj),e=D.c(c,no),c=Yr(c,0),c=G.c(e,d+1)?Tr.h(c):$r(c,d+1));return c=nt(c,b)}function pt(a){a=null!=a&&(a.m&64||q===a.G)?P(U,a):a;var b=D.c(a,fl),c=D.c(a,no);return K.l(a,il,Wg(qg(c,Wg(qg(b,new R(null,2,5,T,[69,Ef],null))))))} +function qt(a){a=null!=a&&(a.m&64||q===a.G)?P(U,a):a;var b=D.c(a,pl);b=null!=b&&(b.m&64||q===b.G)?P(U,b):b;b=D.c(b,Aj);var c=D.c(a,fl),d=D.c(a,Oj);return zg(a,new R(null,2,5,T,[il,b],null),gr.c(c,d))}function rt(a,b,c){return Wg(O.c(jg(b,a),qg(H(a)-b,vr(c))))}function st(a,b,c){return Wg(O.c(qg(b+1,vr(c)),kg(b+1,a)))} +function tt(a){a=null!=a&&(a.m&64||q===a.G)?P(U,a):a;var b=D.c(a,pl),c=null!=b&&(b.m&64||q===b.G)?P(U,b):b;b=D.c(c,zn);c=D.c(c,Aj);var d=D.c(a,fl),e=D.c(a,Oj);--d;return Ag.Z(a,new R(null,2,5,T,[il,c],null),rt,b=k?Zr(c,k-1):c,m=Mb(D,p,new R(null,2,5,T,[pl,zn],null));return Ag.l(p,new R(null,2,5,T,[il,h],null),function(a,b,c,d,e,f,h,k,m,l,p,Q){return function(a){return Wg(O.A(jg(b,a),kg(b+c,a),be([qg(c,vr(Q))])))}}(p,m,function(){var a=k-m;return b=a}}(c,b)(b)}()))return Gu(a,b+64);throw Jt;}catch(h){if(h instanceof Error){var d=h;if(d===Jt)try{if(55===b)return Bg(a,V,ms);throw Jt;}catch(k){if(k instanceof Error){var e=k;if(e===Jt)try{if(56===b)return Bg(a,V,ns);throw Jt;}catch(l){if(l instanceof Error){var f=l;if(f===Jt)try{if(99===b)return du(a); +throw Jt;}catch(p){if(p instanceof Error){d=p;if(d===Jt)throw Jt;throw d;}throw p;}else throw f;}else throw l;}else throw e;}else throw k;}else throw d;}else throw h;}else throw Jt;}catch(h){if(h instanceof Error)if(d=h,d===Jt)try{if(35===c)try{if(56===b)return Bg(a,V,pt);throw Jt;}catch(k){if(k instanceof Error){e=k;if(e===Jt)throw Jt;throw e;}throw k;}else throw Jt;}catch(k){if(k instanceof Error)if(e=k,e===Jt)try{if(40===c)try{if(48===b)return Zt(a);throw Jt;}catch(l){if(l instanceof Error){f= +l;if(f===Jt)return $t(a);throw f;}throw l;}else throw Jt;}catch(l){if(l instanceof Error){f=l;if(f===Jt)return a;throw f;}throw l;}else throw e;else throw k;}else throw d;else throw h;}},function(a){return a},function(a){return a},Gu,function(a,b){return Cg(a,V,ot,b)},function(a,b){var c=function(){switch(b){case 64:return eu;case 65:return fu;case 66:return gu;case 67:return hu;case 68:return iu;case 69:return ju;case 70:return ku;case 71:return lu;case 72:return mu;case 73:return nu;case 74:return ou; +case 75:return pu;case 76:return su;case 77:return tu;case 80:return uu;case 83:return qu;case 84:return ru;case 87:return vu;case 88:return wu;case 90:return xu;case 96:return lu;case 97:return hu;case 100:return Du;case 101:return fu;case 102:return mu;case 103:return yu;case 104:return zu;case 108:return Au;case 109:return Cu;case 112:return Eu;case 114:return Fu;default:return null}}();return t(c)?c.h?c.h(a):c.call(null,a):a},function(a){return a},function(a,b){return K.l(a,kk,ge.c(kk.h(a),b))}, +function(a){return a},function(a,b){return K.l(a,rk,ge.c(rk.h(a),b))},function(a){return a},function(a){return a},function(a){return K.A(a,rk,he,be([kk,he]))}]);function Iu(a,b){for(var c=a,d=Tl.h(c),e=b;;){var f=y(e);if(t(f)){var h=160<=f?65:f;h=D.c(d.h?d.h(xq):d.call(null,xq),h);d=J(h,0,null);h=J(h,1,null);a:for(;;)if(E(h)){var k=y(h);k=Hu.h?Hu.h(k):Hu.call(null,k);c=k.c?k.c(c,f):k.call(null,c,f);h=z(h)}else break a;e=vd(e)}else return K.l(c,Tl,d)}} +function Ju(a,b){var c=xg(function(a){return a.codePointAt(0)},b);return Iu(a,c)} +function Ku(a,b){try{if(ze(b)&&3===H(b)){var c=Vd(b,0),d=Vd(b,1),e=Vd(b,2);return[v.h(a+8),";2;",v.h(c),";",v.h(d),";",v.h(e)].join("")}throw Jt;}catch(k){if(k instanceof Error){var f=k;if(f===Jt)try{if(t(function(){return function(){return function(a){return 8>a}}(f)(b)}()))return""+v.h(a+b);throw Jt;}catch(l){if(l instanceof Error){var h=l;if(h===Jt)try{if(t(function(){return function(){return function(a){return 16>a}}(h,f)(b)}()))return""+v.h(a+52+b);throw Jt;}catch(p){if(p instanceof Error){c= +p;if(c===Jt)return[v.h(a+8),";5;",v.h(b)].join("");throw c;}throw p;}else throw h;}else throw l;}else throw f;}else throw k;}}ag.c(Ku,30);ag.c(Ku,40);var Lu=function Lu(a){if(null!=a&&null!=a.yd)return a.yd(a);var c=Lu[n(null==a?null:a)];if(null!=c)return c.h?c.h(a):c.call(null,a);c=Lu._;if(null!=c)return c.h?c.h(a):c.call(null,a);throw Cb("Screen.lines",a);},Mu=function Mu(a){if(null!=a&&null!=a.xd)return a.xd(a);var c=Mu[n(null==a?null:a)];if(null!=c)return c.h?c.h(a):c.call(null,a);c=Mu._;if(null!=c)return c.h?c.h(a):c.call(null,a);throw Cb("Screen.cursor",a);};function Nu(a,b){var c=0parseFloat(Iv)){Hv=String(Kv);break a}}Hv=Iv}var gb={}; +function Lv(a){return fb(a,function(){for(var b=0,c=ra(String(Hv)).split("."),d=ra(String(a)).split("."),e=Math.max(c.length,d.length),f=0;0==b&&f=a.keyCode)a.keyCode=-1}catch(b){}};var Uv="closure_listenable_"+(1E6*Math.random()|0),Vv=0;function Wv(a,b,c,d,e){this.listener=a;this.Xd=null;this.src=b;this.type=c;this.capture=!!d;this.Ub=e;this.key=++Vv;this.$c=this.Fd=!1}function Xv(a){a.$c=!0;a.listener=null;a.Xd=null;a.src=null;a.Ub=null};function Yv(a){this.src=a;this.rb={};this.wd=0}Yv.prototype.add=function(a,b,c,d,e){var f=a.toString();a=this.rb[f];a||(a=this.rb[f]=[],this.wd++);var h=Zv(a,b,d,e);-1e.keyCode||void 0!=e.returnValue)){a:{var f=!1;if(0==e.keyCode)try{e.keyCode=-1;break a}catch(l){f=!0}if(f||void 0==e.returnValue)e.returnValue=!0}e=[];for(f=c.currentTarget;f;f=f.parentNode)e.push(f);f=a.type;for(var h=e.length-1;!c.Kc&&0<=h;h--){c.currentTarget=e[h];var k=nw(e[h],f,!0,c);d=d&&k}for(h=0;!c.Kc&& +h>>0);function fw(a){if(ha(a))return a;a[pw]||(a[pw]=function(b){return a.handleEvent(b)});return a[pw]};function qw(){wv.call(this);this.Ib=new Yv(this);this.ff=this;this.ve=null}qa(qw,wv);qw.prototype[Uv]=!0;g=qw.prototype;g.addEventListener=function(a,b,c,d){dw(this,a,b,c,d)};g.removeEventListener=function(a,b,c,d){lw(this,a,b,c,d)}; +g.dispatchEvent=function(a){var b,c=this.ve;if(c)for(b=[];c;c=c.ve)b.push(c);c=this.ff;var d=a.type||a;if(ca(a))a=new Sv(a,c);else if(a instanceof Sv)a.target=a.target||c;else{var e=a;a=new Sv(d,c);Ia(a,e)}e=!0;if(b)for(var f=b.length-1;!a.Kc&&0<=f;f--){var h=a.currentTarget=b[f];e=rw(h,d,!0,a)&&e}a.Kc||(h=a.currentTarget=c,e=rw(h,d,!0,a)&&e,a.Kc||(e=rw(h,d,!1,a)&&e));if(b)for(f=0;!a.Kc&&fthis.head?(Yw(this.o,this.fa,a,0,this.o.length-this.fa),Yw(this.o,0,a,this.o.length-this.fa,this.head),this.fa=0,this.head=this.length,this.o=a):this.fa===this.head?(this.head=this.fa=0,this.o=a):null};function ax(a,b){for(var c=a.length,d=0;;)if(da)){a+=1;continue}break}hx=!1;return 0c)return a;a:for(;;){var e=cMath.random()&&15>d)d+=1;else break a;if(d>this.level){for(var e=this.level+1;;)if(e<=d+1)c[e]=this.header,e+=1;else break;this.level=d}for(d=Ex(a,b,Array(d));;)return 0<=this.level?(c=c[0].forward,d.forward[0]=c[0],c[0]=d):null}; +Gx.prototype.remove=function(a){var b=Array(15),c=Fx(this.header,a,this.level,b);c=0===c.forward.length?null:c.forward[0];if(null!=c&&c.key===a){for(a=0;;)if(a<=this.level){var d=b[a].forward;c===(ad)return c===b.header?null:c;var e;a:for(e=c;;){e=d=a)break a}null!=e?(--d,c=e):--d}}Gx.prototype.S=function(){return function(a){return function d(c){return new kf(null,function(){return function(){return null==c?null:ae(new R(null,2,5,T,[c.key,c.H],null),d(c.forward[0]))}}(a),null,null)}}(this)(this.header.forward[0])}; +Gx.prototype.R=function(a,b,c){return Y(b,function(){return function(a){return Y(b,Qi,""," ","",c,a)}}(this),"{",", ","}",c,this)};var Ix=new Gx(Ex(null,null,0),0);function Jx(a){var b=(new Date).valueOf()+a,c=Hx(b),d=t(t(c)?c.keya:b)?a+8:a,[v.h(c),v.h(a)].join("")):null} +function Vy(a){var b=J(a,0,null),c=J(a,1,null);a=J(a,2,null);return["rgb(",v.h(b),",",v.h(c),",",v.h(a),")"].join("")} +var Wy=hj(function(a){a=null!=a&&(a.m&64||q===a.G)?P(U,a):a;var b=D.c(a,Nk),c=D.c(a,pl);a=K.l(a,Nk,t(c)?wb(b):b);var d=null!=a&&(a.m&64||q===a.G)?P(U,a):a,e=D.c(d,Ok),f=D.c(d,Tn);b=D.c(d,Kj);var h=D.c(d,dk);c=D.c(d,Vl);var k=D.c(d,Nk),l=D.c(d,Yn);d=D.c(d,pl);var p=t(k)?t(e)?e:"fg":f;e=Uy(t(k)?t(f)?f:"bg":e,b,"fg-");h=Uy(p,h,"bg-");c=vg(ub,new R(null,6,5,T,[e,h,t(b)?"bright":null,t(l)?"italic":null,t(c)?"underline":null,t(d)?"cursor":null],null));if(E(c))a:for(b=new cb,c=E(c);;)if(null!=c)b.append(""+ +v.h(y(c))),c=z(c),null!=c&&b.append(" ");else{b=b.toString();break a}else b=null;l=null!=a&&(a.m&64||q===a.G)?P(U,a):a;a=D.c(l,Ok);c=D.c(l,Tn);h=D.c(l,Nk);l=t(h)?c:a;a=t(h)?a:c;a=hi.A(be([t(ze.h?ze.h(l):ze.call(null,l))?new r(null,1,[ik,Vy(l)],null):null,t(ze.h?ze.h(a):ze.call(null,a))?new r(null,1,[al,Vy(a)],null):null]));return hi.A(be([t(b)?new r(null,1,[vn,b],null):null,t(a)?new r(null,1,[fm,a],null):null]))}); +function Xy(a,b){var c=J(a,0,null),d=J(a,1,null);d=Bg(d,pl,function(){return function(a){return t(a)?B(b):a}}(a,c,d));return new R(null,3,5,T,[ro,Wy.h?Wy.h(d):Wy.call(null,d),c],null)}function Yy(a,b){var c=J(a,0,null),d=J(a,1,null),e=jg(b,c);e=E(e)?new R(null,2,5,T,[Eo(e),d],null):null;var f=K.l(d,pl,!0);f=new R(null,2,5,T,[Vd(c,b),f],null);c=kg(b+1,c);d=E(c)?new R(null,2,5,T,[Eo(c),d],null):null;return vg(ub,new R(null,3,5,T,[e,f,d],null))} +function Zy(a,b){for(var c=he,d=a,e=b;;)if(E(d)){var f=y(d),h=J(f,0,null);J(f,1,null);h=H(h);if(h<=e)c=ge.c(c,f),d=vd(d),e-=h;else return O.A(c,Yy(f,e),be([vd(d)]))}else return c}function $y(a,b,c){a=t(B(b))?Zy(B(a),B(b)):B(a);return new R(null,2,5,T,[Lm,Ii(bg(function(){return function(a,b){return pe(new R(null,3,5,T,[Xy,b,c],null),new r(null,1,[mk,a],null))}}(a),a))],null)}var qA=new ti(null,new r(null,3,["small",null,"medium",null,"big",null],null),null); +function rA(a,b,c,d,e){var f=yp(function(){var a=B(c);return t(qA.h?qA.h(a):qA.call(null,a))?["font-",v.h(a)].join(""):null}),h=yp(function(){return function(){var d=B(a),e=B(b),f=B(c);f=t(qA.h?qA.h(f):qA.call(null,f))?null:new r(null,1,[wk,f],null);return hi.A(be([new r(null,2,[fl,[v.h(d),"ch"].join(""),no,[v.h(1.3333333333*e),"em"].join("")],null),f]))}}(f)),k=yp(function(){return function(){return Lu(B(d))}}(f,h)),l=yp(function(a,c,d){return function(){return xg(function(a,b,c){return function(d){return yp(function(a, +b,c){return function(){return D.c(B(c),d)}}(a,b,c))}}(a,c,d),Fi(0,B(b),1))}}(f,h,k)),p=yp(function(){return function(){return Mu(B(d))}}(f,h,k,l)),m=yp(function(a,b,c,d,e){return function(){return zn.h(B(e))}}(f,h,k,l,p)),u=yp(function(a,b,c,d,e){return function(){return Aj.h(B(e))}}(f,h,k,l,p,m)),w=yp(function(a,b,c,d,e){return function(){return On.h(B(e))}}(f,h,k,l,p,m,u));return function(a,b,c,d,f,h,k,l){return function(){return new R(null,3,5,T,[Gm,new r(null,2,[vn,B(a),fm,B(b)],null),bg(function(a, +b,c,d,f,h,k,l){return function(m,p){var u=yp(function(a,b,c,d,e,f,h,k){return function(){var a=B(k);return t(a)?(a=G.c(m,B(h)))?B(f):a:a}}(a,b,c,d,f,h,k,l));return pe(new R(null,4,5,T,[$y,p,u,e],null),new r(null,1,[mk,m],null))}}(a,b,c,d,f,h,k,l),B(d))],null)}}(f,h,k,l,p,m,u,w)} +function sA(){return new R(null,2,5,T,[Ym,new r(null,4,[Mn,"1.1",Fl,"0 0 866.0254037844387 866.0254037844387",vn,"icon",mo,new r(null,1,[An,'\x3cdefs\x3e \x3cmask id\x3d"small-triangle-mask"\x3e \x3crect width\x3d"100%" height\x3d"100%" fill\x3d"white"/\x3e \x3cpolygon points\x3d"508.01270189221935 433.01270189221935, 208.0127018922194 259.8076211353316, 208.01270189221927 606.217782649107" fill\x3d"black"\x3e\x3c/polygon\x3e \x3c/mask\x3e \x3c/defs\x3e \x3cpolygon points\x3d"808.0127018922194 433.01270189221935, 58.01270189221947 -1.1368683772161603e-13, 58.01270189221913 866.0254037844386" mask\x3d"url(#small-triangle-mask)" fill\x3d"white"\x3e\x3c/polygon\x3e \x3cpolyline points\x3d"481.2177826491071 333.0127018922194, 134.80762113533166 533.0127018922194" stroke\x3d"white" stroke-width\x3d"90"\x3e\x3c/polyline\x3e'],null)], +null)],null)}function tA(){return new R(null,3,5,T,[Ym,new r(null,3,[Mn,"1.1",Fl,"0 0 12 12",vn,"icon"],null),new R(null,2,5,T,[Fj,new r(null,1,[pn,"M1,0 L11,6 L1,12 Z"],null)],null)],null)}function uA(){return new R(null,4,5,T,[Ym,new r(null,3,[Mn,"1.1",Fl,"0 0 12 12",vn,"icon"],null),new R(null,2,5,T,[Fj,new r(null,1,[pn,"M1,0 L4,0 L4,12 L1,12 Z"],null)],null),new R(null,2,5,T,[Fj,new r(null,1,[pn,"M8,0 L11,0 L11,12 L8,12 Z"],null)],null)],null)} +function vA(){return new R(null,4,5,T,[Ym,new r(null,3,[Mn,"1.1",Fl,"0 0 12 12",vn,"icon"],null),new R(null,2,5,T,[Fj,new r(null,1,[pn,"M12,0 L7,0 L9,2 L7,4 L8,5 L10,3 L12,5 Z"],null)],null),new R(null,2,5,T,[Fj,new r(null,1,[pn,"M0,12 L0,7 L2,9 L4,7 L5,8 L3,10 L5,12 Z"],null)],null)],null)} +function wA(){return new R(null,4,5,T,[Ym,new r(null,3,[Mn,"1.1",Fl,"0 0 12 12",vn,"icon"],null),new R(null,2,5,T,[Fj,new r(null,1,[pn,"M7,5 L7,0 L9,2 L11,0 L12,1 L10,3 L12,5 Z"],null)],null),new R(null,2,5,T,[Fj,new r(null,1,[pn,"M5,7 L0,7 L2,9 L0,11 L1,12 L3,10 L5,12 Z"],null)],null)],null)}function xA(a,b){return function(b){return function(){return new R(null,3,5,T,[cl,new r(null,1,[Sl,b],null),new R(null,1,5,T,[t(B(a))?uA:tA],null)],null)}}(Ty(b,new fy(null,null,null)))} +function yA(a){return 10>a?["0",v.h(a)].join(""):a}function zA(a){var b=Math.floor((a%60+60)%60);return[v.h(yA(Math.floor(a/60))),":",v.h(yA(b))].join("")}function AA(a,b){var c=T,d=new R(null,2,5,T,[Yk,zA(B(a))],null),e=T;var f=B(a);var h=B(b);f=["-",v.h(zA(h-f))].join("");return new R(null,3,5,c,[Ml,d,new R(null,2,5,e,[co,f],null)],null)} +function BA(){function a(a){a.preventDefault();return Ry(a.currentTarget.parentNode.parentNode.parentNode)}return function(){return new R(null,4,5,T,[un,new r(null,1,[Sl,a],null),new R(null,1,5,T,[vA],null),new R(null,1,5,T,[wA],null)],null)}} +function CA(a,b){var c=Sy(b,function(a){var b=a.currentTarget.offsetWidth,c=a.currentTarget.getBoundingClientRect();return cy(Nu(a.clientX-c.left,b)/b)}),d=yp(function(){return function(){return[v.h(100*B(a)),"%"].join("")}}(c));return function(a,b){return function(){return new R(null,2,5,T,[Vj,new R(null,3,5,T,[Bl,new r(null,1,[Ql,a],null),new R(null,2,5,T,[Cj,new R(null,2,5,T,[ro,new r(null,1,[fm,new r(null,1,[fl,B(b)],null)],null)],null)],null)],null)],null)}}(c,d)} +function DA(a,b,c,d){return function(e){return function(){return new R(null,5,5,T,[Kk,new R(null,3,5,T,[xA,a,d],null),new R(null,3,5,T,[AA,b,c],null),new R(null,1,5,T,[BA],null),new R(null,3,5,T,[CA,e,d],null)],null)}}(yp(function(){return B(b)/B(c)}))} +function EA(a){return function(a){return function(){return new R(null,3,5,T,[ol,new r(null,1,[Sl,a],null),new R(null,2,5,T,[Xk,new R(null,2,5,T,[km,new R(null,2,5,T,[ro,new R(null,1,5,T,[sA],null)],null)],null)],null)],null)}}(Ty(a,new fy(null,null,null)))}function FA(){return new R(null,2,5,T,[Ek,new R(null,1,5,T,[xn],null)],null)}function GA(a){return Wf(function(b){return a[b]},new R(null,4,5,T,["altKey","shiftKey","metaKey","ctrlKey"],null))} +function HA(a){var b=t(GA(a))?null:function(){switch(a.key){case " ":return new fy(null,null,null);case "f":return bm;case "0":return cy(0);case "1":return cy(.1);case "2":return cy(.2);case "3":return cy(.3);case "4":return cy(.4);case "5":return cy(.5);case "6":return cy(.6);case "7":return cy(.7);case "8":return cy(.8);case "9":return cy(.9);default:return null}}();if(t(b))return b;switch(a.key){case "\x3e":return new ey(null,null,null);case "\x3c":return new dy(null,null,null);default:return null}} +function IA(a){if(t(GA(a)))return null;switch(a.which){case 37:return new ay(null,null,null);case 39:return new $x(null,null,null);default:return null}}function JA(a){var b=HA(a);return t(b)?(a.preventDefault(),G.c(b,bm)?(Ry(a.currentTarget),null):b):null}function KA(a){var b=IA(a);return t(b)?(a.preventDefault(),b):null} +function LA(a,b,c,d){a=t(a)?['"',v.h(a),'"'].join(""):"untitled";return new R(null,4,5,T,[dl,t(d)?new R(null,2,5,T,[jo,new r(null,1,[zl,d],null)],null):null,a,t(b)?new R(null,3,5,T,[ro," by ",t(c)?new R(null,3,5,T,[lo,new r(null,1,[ho,c],null),b],null):b],null):null],null)} +function MA(a){var b=Mx(1,ig.h(iy)),c=Kx(1);lx(function(c){return function(){var d=function(){return function(a){return function(){function b(b){for(;;){a:try{for(;;){var c=a(b);if(!N(c,Z)){var d=c;break a}}}catch(x){if(x instanceof Object)b[5]=x,Cx(b),d=Z;else throw x;}if(!N(d,Z))return d}}function c(){var a=[null,null,null,null,null,null,null,null,null,null,null,null];a[0]=d;a[1]=1;return a}var d=null;d=function(a){switch(arguments.length){case 0:return c.call(this);case 1:return b.call(this,a)}throw Error("Invalid arity: "+ +(arguments.length-1));};d.B=c;d.h=b;return d}()}(function(){return function(c){var d=c[1];if(7===d)return c[7]=c[2],Ax(c,12,b,!1);if(1===d)return c[2]=null,c[1]=2,Z;if(4===d)return c[8]=c[2],Ax(c,5,b,!0);if(6===d)return d=Jx(3E3),Ux(c,8,new R(null,2,5,T,[a,d],null));if(3===d)return Bx(c,c[2]);if(12===d)return c[9]=c[2],c[2]=null,c[1]=2,Z;if(2===d)return zx(c,4,a);if(11===d)return c[2]=c[2],c[1]=7,Z;if(9===d)return c[2]=null,c[1]=6,Z;if(5===d)return c[10]=c[2],c[2]=null,c[1]=6,Z;if(10===d)return c[2]= +null,c[1]=11,Z;if(8===d){var e=c[2];d=J(e,0,null);e=J(e,1,null);e=G.c(e,a);c[11]=d;c[1]=e?9:10;return Z}return null}}(c),c)}(),f=function(){var a=d.B?d.B():d.call(null);a[6]=c;return a}();return yx(f)}}(c));return b} +function NA(a,b){var c=dg.h(b),d=Kx(1);lx(function(b,c){return function(){var d=function(){return function(a){return function(){function b(b){for(;;){a:try{for(;;){var c=a(b);if(!N(c,Z)){var d=c;break a}}}catch(F){if(F instanceof Object)b[5]=F,Cx(b),d=Z;else throw F;}if(!N(d,Z))return d}}function c(){var a=[null,null,null,null,null,null,null,null,null,null,null,null,null];a[0]=d;a[1]=1;return a}var d=null;d=function(a){switch(arguments.length){case 0:return c.call(this);case 1:return b.call(this, +a)}throw Error("Invalid arity: "+(arguments.length-1));};d.B=c;d.h=b;return d}()}(function(b,c){return function(d){var e=d[1];if(7===e){var f=d[7],h=wb(null==f);d[8]=d[2];d[1]=h?8:9;return Z}if(20===e)return f=d[7],d[1]=t(q===f.Fe)?23:24,Z;if(27===e)return d[2]=!1,d[1]=28,Z;if(1===e)return d[2]=null,d[1]=2,Z;if(24===e)return f=d[7],d[1]=t(!f.Tc)?26:27,Z;if(4===e){f=d[7];var k=d[9];h=d[2];var l=J(h,0,null),m=J(h,1,null);d[10]=m;d[7]=l;d[9]=h;d[1]=t(null==l)?5:6;return Z}return 15===e?(d[2]=!1,d[1]= +16,Z):21===e?(f=d[7],h=Ab(Yx,f),d[2]=h,d[1]=22,Z):31===e?(d[11]=d[2],d[2]=null,d[1]=2,Z):13===e?(d[2]=d[2],d[1]=10,Z):22===e?(d[1]=t(d[2])?29:30,Z):29===e?(f=d[7],h=B(a),h=Zx(f,h),h=gg.l(c,wo,h),d[2]=h,d[1]=31,Z):6===e?(d[2]=null,d[1]=7,Z):28===e?(d[2]=d[2],d[1]=25,Z):25===e?(d[2]=d[2],d[1]=22,Z):17===e?(m=d[10],f=d[7],k=d[9],h=gg.c(a,function(){return function(a,b){return function(a){return Xx(b,a)}}(k,f,m,m,f,k,e,b,c)}()),d[2]=h,d[1]=19,Z):3===e?Bx(d,d[2]):12===e?(f=d[7],d[1]=t(!f.Tc)?14:15,Z): +2===e?(h=B(c),h=E(h),Ux(d,4,h)):23===e?(d[2]=!0,d[1]=25,Z):19===e?(f=d[7],h=wb(null==f),d[12]=d[2],d[1]=h?20:21,Z):11===e?(d[2]=!0,d[1]=13,Z):9===e?(f=d[7],h=Ab(Wx,f),d[2]=h,d[1]=10,Z):5===e?(m=d[10],h=gg.l(c,re,m),d[2]=h,d[1]=7,Z):14===e?(f=d[7],h=Ab(Wx,f),d[2]=h,d[1]=16,Z):26===e?(f=d[7],h=Ab(Yx,f),d[2]=h,d[1]=28,Z):16===e?(d[2]=d[2],d[1]=13,Z):30===e?(d[2]=null,d[1]=31,Z):10===e?(d[1]=t(d[2])?17:18,Z):18===e?(d[2]=null,d[1]=19,Z):8===e?(f=d[7],d[1]=t(q===f.sb)?11:12,Z):null}}(b,c),b,c)}(),e=function(){var a= +d.B?d.B():d.call(null);a[6]=b;return a}();return yx(e)}}(d,c));return d} +function OA(a,b,c){c=Ty(c,!0);var d=Sy(b,JA),e=Sy(b,KA),f=yp(function(){return function(){return Hm.h(B(a))}}(c,d,e)),h=yp(function(){return function(){return el.h(B(a))}}(c,d,e,f)),k=yp(function(a,b,c,d,e){return function(){var a=B(d);return t(a)?a:B(e)}}(c,d,e,f,h)),l=yp(function(b,c,d,e,f,h){return function(){var b=Gk.h(B(a));b=t(b)?b:wb(B(h));return t(b)?"hud":null}}(c,d,e,f,h,k)),p=yp(function(){return function(){return["asciinema-theme-",v.h(gm.h(B(a)))].join("")}}(c,d,e,f,h,k,l)),m=yp(function(){return function(){var b= +fl.h(B(a));return t(b)?b:80}}(c,d,e,f,h,k,l,p)),u=yp(function(){return function(){var b=no.h(B(a));return t(b)?b:24}}(c,d,e,f,h,k,l,p,m)),w=yp(function(){return function(){return wk.h(B(a))}}(c,d,e,f,h,k,l,p,m,u)),x=yp(function(){return function(){return V.h(B(a))}}(c,d,e,f,h,k,l,p,m,u,w)),C=yp(function(){return function(){return ml.h(B(a))}}(c,d,e,f,h,k,l,p,m,u,w,x)),F=yp(function(){return function(){return jn.h(B(a))}}(c,d,e,f,h,k,l,p,m,u,w,x,C)),I=yp(function(){return function(){return Uj.h(B(a))}}(c, +d,e,f,h,k,l,p,m,u,w,x,C,F)),M=yp(function(){return function(){return wl.h(B(a))}}(c,d,e,f,h,k,l,p,m,u,w,x,C,F,I)),S=B(a),X=null!=S&&(S.m&64||q===S.G)?P(U,S):S,Ga=D.c(X,ki),db=D.c(X,li),Q=D.c(X,mi),xb=D.c(X,ni);return function(a,c,d,e,f,h,k,l,m,p,u,w,x,C,F,I,M,S,Q,X,Ga,db){return function(){return new R(null,3,5,T,[Cn,new r(null,5,[Jj,-1,Zj,c,Rn,d,Vm,a,vn,B(k)],null),new R(null,7,5,T,[Sm,new r(null,1,[vn,B(l)],null),new R(null,6,5,T,[rA,m,p,u,w,x],null),new R(null,5,5,T,[DA,C,F,I,b],null),t(t(Q)?Q: +X)?new R(null,5,5,T,[LA,Q,X,Ga,db],null):null,t(B(h))?null:new R(null,2,5,T,[EA,b],null),t(B(e))?new R(null,1,5,T,[FA],null):null],null)],null)}}(c,d,e,f,h,k,l,p,m,u,w,x,C,F,I,M,S,X,Ga,db,Q,xb)} +function PA(a){var b=Kx(null),c=Kx(new dx(bx(1),1));return function(b,c){return function(){return Pp(new r(null,4,[ln,"asciinema-player",Dm,function(b,c){return function(){return OA(a,b,c)}}(b,c),$k,function(b,c){return function(){var d=ty(Gl.h(B(a))),e=MA(c);Tx(e,b);return NA(a,Je([b,d]))}}(b,c),Wm,function(){return function(){return uy(Gl.h(B(a)))}}(b,c)],null))}}(b,c)};function QA(a,b){var c=null!=b&&(b.m&64||q===b.G)?P(U,b):b,d=D.c(c,Ak),e=D.c(c,Gl);d=a.h?a.h(d):a.call(null,d);zy(e,d);return K.l(c,Ak,d)}$x.prototype.sb=q;$x.prototype.qb=function(a,b){var c=null!=b&&(b.m&64||q===b.G)?P(U,b):b,d=D.c(c,Uj),e=D.c(c,wl),f=D.c(c,Gl);t(e)&&yy(f,Nu(d+5,e));return c};ay.prototype.sb=q;ay.prototype.qb=function(a,b){var c=null!=b&&(b.m&64||q===b.G)?P(U,b):b,d=D.c(c,Uj),e=D.c(c,wl),f=D.c(c,Gl);t(e)&&yy(f,Nu(d+-5,e));return c};by.prototype.sb=q; +by.prototype.qb=function(a,b){var c=null!=b&&(b.m&64||q===b.G)?P(U,b):b,d=D.c(c,wl),e=D.c(c,Gl);t(d)&&(d*=nn.h(this),yy(e,d));return c};dy.prototype.sb=q;dy.prototype.qb=function(a,b){return QA(function(){return function(a){return a/2}}(this),b)};ey.prototype.sb=q;ey.prototype.qb=function(a,b){return QA(function(){return function(a){return 2*a}}(this),b)};fy.prototype.sb=q;fy.prototype.qb=function(a,b){xy(Gl.h(b));return b};gy.prototype.sb=q;gy.prototype.qb=function(a,b){return K.l(b,ml,so.h(this))}; +hy.prototype.sb=q;hy.prototype.qb=function(a,b){return K.l(b,Gk,so.h(this))};jy.prototype.sb=q;jy.prototype.qb=function(a,b){var c=null!=a&&(a.m&64||q===a.G)?P(U,a):a;D.c(c,fl);D.c(c,no);D.c(c,wl);c=null!=b&&(b.m&64||q===b.G)?P(U,b):b;var d=D.c(c,fl),e=D.c(c,no),f=null!=this&&(this.m&64||q===this.G)?P(U,this):this,h=D.c(f,fl),k=D.c(f,no);f=D.c(f,wl);return K.A(c,fl,t(d)?d:h,be([no,t(e)?e:k,wl,f]))};ky.prototype.sb=q;ky.prototype.qb=function(a,b){return K.l(b,Hm,Hm.h(this))};oy.prototype.sb=q; +oy.prototype.qb=function(a,b){var c=null!=b&&(b.m&64||q===b.G)?P(U,b):b,d=D.c(c,oi);t(d)&&(ap(bp),d.B?d.B():d.call(null));return c};ry.prototype.sb=q;ry.prototype.qb=function(a,b){return K.l(b,Uj,Zk.h(this))};function RA(){return ig.l(function(a,b){return new R(null,2,5,T,[a,new gy(b,null,null,null)],null)},rg(function(a){return a+.5},.5),og(new R(null,2,5,T,[!1,!0],null)))}function SA(a){var b=Dy(RA());return K.l(K.l(a,ml,!0),Ol,b)} +function TA(a){a=null!=a&&(a.m&64||q===a.G)?P(U,a):a;var b=D.c(a,Ol);Tw(b);return K.l(K.l(a,ml,!0),Ol,null)}function UA(a){a=null!=a&&(a.m&64||q===a.G)?P(U,a):a;a=D.c(a,Ol);return t(a)?Je([a]):vi}my.prototype.sb=q; +my.prototype.qb=function(a,b){var c=null!=a&&(a.m&64||q===a.G)?P(U,a):a;D.c(c,jn);var d=null!=b&&(b.m&64||q===b.G)?P(U,b):b,e=D.c(d,jn);c=D.c(d,pi);var f=D.c(d,qi),h=null!=this&&(this.m&64||q===this.G)?P(U,this):this;h=D.c(h,jn);if(G.c(e,h))return d;d=K.A(d,jn,h,be([el,!0]));if(t(h))return t(c)&&(c.B?c.B():c.call(null)),SA(d);t(f)&&(f.B?f.B():f.call(null));return TA(d)};my.prototype.Fe=q;my.prototype.de=function(a,b){return UA(b)};py.prototype.sb=q; +py.prototype.qb=function(a,b){var c=K.l(b,V,V.h(this));c=null!=c&&(c.m&64||q===c.G)?P(U,c):c;var d=D.c(c,Ol);return t(d)?SA(TA(c)):c};py.prototype.Fe=q;py.prototype.de=function(a,b){return UA(b)};function VA(a){return t(a)?(a=ig.c(parseFloat,Fo(""+v.h(a),/:/)),a=ig.l(Ye,cf(a),rg(function(){return function(a){return 60*a}}(a),1)),P(Xe,a)):null} +function WA(a,b,c){t(a)?"string"===typeof a?t(0===a.indexOf("data:application/json;base64,"))?(b=a.substring(29).replace(RegExp("\\s","g"),""),b=JSON.parse(atob(b)),b=fj(b),b=new r(null,1,[V,new r(null,1,[il,b],null)],null)):t(0===a.indexOf("data:text/plain,"))?(a=a.substring(16),b=Ju(Ot(t(b)?b:80,t(c)?c:24),a),b=new r(null,1,[V,b],null)):b=t(0===a.indexOf("npt:"))?new r(null,1,[Zk,VA(a.substring(4))],null):null:b=new r(null,1,[V,new r(null,1,[il,a],null)],null):b=null;return b} +var XA=new r(null,2,[pl,new r(null,1,[On,!1],null),il,he],null); +function YA(a,b){var c=null!=b&&(b.m&64||q===b.G)?P(U,b):b,d=D.c(c,no),e=D.l(c,wk,"small"),f=D.l(c,Ak,1),h=D.c(c,Hk),k=D.c(c,fl),l=D.c(c,rl),p=D.l(c,cm,!1),m=D.l(c,gm,"asciinema"),u=D.c(c,qm),w=D.c(c,Bm),x=D.l(c,vm,!1),C=D.l(c,Em,!1),F=function(){var a=VA(h);return t(a)?a:0}();w=WA(w,k,d);var I=null!=w&&(w.m&64||q===w.G)?P(U,w):w;w=D.c(I,V);I=D.c(I,Zk);var M=t(I)?I:wb(w)&&0 currTime) { + instructions[i].classList.add("is-active"); + } + else { + instructions[i].classList.remove("is-active"); + } + } +} + +document.addEventListener("DOMContentLoaded", function() { + var asciinemaPlayer = document.getElementById('asciinema-player'); + asciinemaPlayer.instructions = asciinemaPlayer.parentElement.getElementsByClassName("panel-block"); + asciinemaPlayer.addEventListener("play", playEvent); + asciinemaPlayer.addEventListener('pause', pauseEvent); + + for(let i = 0; i < asciinemaPlayer.instructions.length; i++) { + var instruction = asciinemaPlayer.instructions[i]; + instruction.addEventListener("click", function(e){ + asciinemaPlayer.currentTime = this.getAttribute("data-from"); + asciinemaPlayer.play(); + e.preventDefault(); + return false; + }); + } +}); diff -Nru mitmproxy-5.1.1/docs/src/assets/style.scss mitmproxy-6.0.2/docs/src/assets/style.scss --- mitmproxy-5.1.1/docs/src/assets/style.scss 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/assets/style.scss 2020-12-15 16:41:27.000000000 +0000 @@ -5,6 +5,9 @@ $warning-invert: #FFFFFF; $family-sans-serif: BlinkMacSystemFont, -apple-system, "Segoe UI", "Roboto", "Oxygen", "Ubuntu", "Cantarell", "Fira Sans", "Droid Sans", "Helvetica Neue", "Helvetica", "Arial", sans-serif, 'Font Awesome 5 Free', 'Font Awesome 5 Brands' !default; +$panel-heading-size: 1em; +$panel-heading-weight: 600; + /*!* bulma.io v0.8.0 | MIT License | github.com/jgthms/bulma */ @import "./bulma/utilities/_all"; @@ -92,3 +95,28 @@ figure.has-border img { box-shadow: 0 0 20px 0 rgba(0, 0, 0, 0.25); } + +.asciicast-wrapper { + margin: 2rem 0; + + asciinema-player { + display: block; + margin-bottom: 1rem; + } + + // reset bulma pre styles + pre.asciinema-terminal { + padding: 0; + overflow-x: hidden; + -webkit-overflow-scrolling: auto; + } + + .panel-block { + justify-content: space-between; + } + + .panel-block.is-active .tag { + background-color: $link; + color: $white; + } +} diff -Nru mitmproxy-5.1.1/docs/src/config.toml mitmproxy-6.0.2/docs/src/config.toml --- mitmproxy-5.1.1/docs/src/config.toml 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/config.toml 2020-12-15 16:41:27.000000000 +0000 @@ -4,7 +4,10 @@ theme = "mitmproxydocs" publishDir = "../public" RelativeURLs = true -googleAnalytics = "UA-4150636-13" +pygmentsCodefences = true [indexes] - tag = "tags" +tag = "tags" + +[markup.goldmark.renderer] +unsafe = true diff -Nru mitmproxy-5.1.1/docs/src/content/addons-commands.md mitmproxy-6.0.2/docs/src/content/addons-commands.md --- mitmproxy-5.1.1/docs/src/content/addons-commands.md 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/content/addons-commands.md 2020-12-15 16:41:27.000000000 +0000 @@ -23,16 +23,16 @@ To see this example in action, start mitmproxy console with the addon loaded: -{{< highlight bash >}} +```bash > mitmproxy -s ./examples/addons/commands-simple.py -{{< /highlight >}} +``` Now, make sure the event log is showing, and then execute the command at the prompt (started by typing ":"): -{{< highlight none>}} +``` :myaddon.inc -{{< /highlight >}} +``` Notice that tab completion works - our addon command has complete parity with builtin commands. There are a few things to note about this example: @@ -67,28 +67,28 @@ Start by loading the addon into mitmproxy and sending some traffic through so we have flows to work with: -{{< highlight bash >}} +```bash > mitmproxy -s ./examples/addons/commands-flows.py -{{< /highlight >}} +``` We can now invoke our toy command in various ways. Let's begin by running it just on the currently focused flow: -{{< highlight none >}} +``` :myaddon.addheader @focus -{{< /highlight >}} +``` We can also invoke it on all flows: -{{< highlight none >}} +``` :myaddon.addheader @all -{{< /highlight >}} +``` Or only flows from **google.com**: -{{< highlight none >}} +``` :myaddon.addheader ~d google.com -{{< /highlight >}} +``` What's more, we can trivially bind these commands to keyboard shortcuts within mitmproxy if we plan to use them frequently. Flow selectors combined with @@ -107,9 +107,9 @@ and writes it to a path which is specified as the second argument to the command. Try invoking it like this: -{{< highlight none >}} +``` :myaddon.histogram @all /tmp/xxx -{{< /highlight >}} +``` Notice that mitmproxy provides tab completion both for the flow specification and the path. diff -Nru mitmproxy-5.1.1/docs/src/content/addons-events.md mitmproxy-6.0.2/docs/src/content/addons-events.md --- mitmproxy-5.1.1/docs/src/content/addons-events.md 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/content/addons-events.md 2020-12-15 16:41:27.000000000 +0000 @@ -13,7 +13,7 @@ change traffic on the fly. For instance, here is an addon that adds a response header with a count of the number of responses seen: -{{< example src="examples/addons/addheader.py" lang="py" >}} +{{< example src="examples/addons/http-add-header.py" lang="py" >}} ## Supported Events diff -Nru mitmproxy-5.1.1/docs/src/content/addons-examples.md mitmproxy-6.0.2/docs/src/content/addons-examples.md --- mitmproxy-5.1.1/docs/src/content/addons-examples.md 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/content/addons-examples.md 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,11 @@ + +--- +title: "Example Addons" +menu: + addons: + weight: 6 +--- + +# Example Addons + +{{< readfile file="/generated/examples.html" markdown="true" >}} diff -Nru mitmproxy-5.1.1/docs/src/content/addons-options.md mitmproxy-6.0.2/docs/src/content/addons-options.md --- mitmproxy-5.1.1/docs/src/content/addons-options.md 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/content/addons-options.md 2020-12-15 16:41:27.000000000 +0000 @@ -28,15 +28,15 @@ single `addheader` option with type `bool`. Let's try this out by running the script in mitmproxy console: -{{< highlight bash >}} +```bash > mitmproxy -s ./examples/addons/options-simple.py -{{< /highlight >}} +``` You can now use CURL to make a request through the proxy like this: -{{< highlight bash >}} +```bash > env http_proxy=http://localhost:8080 curl -I http://google.com -{{< /highlight >}} +``` If you run this request immediately, you'll notice that no count header is added. This is because our default value for the option was `false`. Press `O` @@ -45,22 +45,22 @@ and false. Set the value to `true`, and you should see a result something like this: -{{< highlight bash >}} +```bash > env http_proxy=http://localhost:8080 curl -I http://google.com HTTP/1.1 301 Moved Permanently Location: http://www.google.com/ Content-Length: 219 count: 1 -{{< /highlight >}} +``` When this addon is loaded, the `addheader` setting is available in the persistent [YAML configuration file]({{< relref "concepts-options" >}}). You can also over-ride the value directly from the command-line for any of the tools using the `--set` flag: -{{< highlight bash >}} +```bash mitmproxy -s ./examples/addons/options-simple.py --set addheader=true -{{< /highlight >}} +``` ## Handling configuration updates @@ -85,11 +85,11 @@ the option is changed. If we try to load the script with an incorrect value, we now see an error: -{{< highlight none >}} +``` > mitmdump -s ./examples/addons/options-configure.py --set addheader=1000 Loading script: ./examples/addons/options-configure.py /Users/cortesi/mitmproxy/mitmproxy/venv/bin/mitmdump: addheader must be <= 100 -{{< /highlight >}} +``` ## Supported Types diff -Nru mitmproxy-5.1.1/docs/src/content/addons-overview.md mitmproxy-6.0.2/docs/src/content/addons-overview.md --- mitmproxy-5.1.1/docs/src/content/addons-overview.md 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/content/addons-overview.md 2020-12-15 16:41:27.000000000 +0000 @@ -34,9 +34,9 @@ Here, for example, is a command that shows the API documentation for the mitmproxy's HTTP flow classes: -{{< highlight bash >}} +```bash pydoc mitmproxy.http -{{< /highlight >}} +``` You will be referring to the mitmproxy API documentation frequently, so keep **pydoc** or an equivalent handy. @@ -54,9 +54,9 @@ it into your mitmproxy tool of choice. We'll use mitmpdump in these examples, but the flag is identical for all tools: -{{< highlight bash >}} +```bash > mitmdump -s ./anatomy.py -{{< /highlight >}} +``` Here are a few things to note about the code above: diff -Nru mitmproxy-5.1.1/docs/src/content/addons-scripting.md mitmproxy-6.0.2/docs/src/content/addons-scripting.md --- mitmproxy-5.1.1/docs/src/content/addons-scripting.md 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/content/addons-scripting.md 2020-12-15 16:41:27.000000000 +0000 @@ -13,13 +13,13 @@ handler functions in the module scope. For instance, here is a complete script that adds a header to every request. -{{< example src="examples/addons/scripting-headers.py" lang="py" >}} +{{< example src="examples/addons/scripting-minimal-example.py" lang="py" >}} Here's another example that intercepts requests to a particular URL and sends an arbitrary response instead: -{{< example src="examples/simple/send_reply_from_proxy.py" lang="py" >}} +{{< example src="examples/addons/http-reply-from-proxy.py" lang="py" >}} All events around the HTTP protocol [can be found here]({{< relref "addons-events#http-events">}}). @@ -31,7 +31,7 @@ The WebSocket protocol initially looks like a regular HTTP request, before the client and server agree to upgrade the connection to WebSocket. All scripting events for initial HTTP handshake, and also the dedicated WebSocket events [can be found here]({{< relref "addons-events#websocket-events">}}). -{{< example src="examples/simple/websocket_messages.py" lang="py" >}} +{{< example src="examples/addons/websocket-simple.py" lang="py" >}} For WebSocket-related objects please look at the [websocket][] module to find all attributes that you can use when scripting. @@ -43,7 +43,7 @@ All events around the TCP protocol [can be found here]({{< relref "addons-events#tcp-events">}}). -{{< example src="examples/complex/tcp_message.py" lang="py" >}} +{{< example src="examples/addons/tcp-simple.py" lang="py" >}} For WebSocket-related objects please look at the [tcp][] module to find all attributes that you can use when scripting. diff -Nru mitmproxy-5.1.1/docs/src/content/concepts-certificates.md mitmproxy-6.0.2/docs/src/content/concepts-certificates.md --- mitmproxy-5.1.1/docs/src/content/concepts-certificates.md 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/content/concepts-certificates.md 2020-12-15 16:41:27.000000000 +0000 @@ -24,9 +24,6 @@ Click on the relevant icon, follow the setup instructions for the platform you're on and you are good to go. -Note: If you are using an iOS device, you should be using the Safari browser -so that it opens the proper prompts for installing the certificate. - ## Installing the mitmproxy CA certificate manually Sometimes using the quick install app is not an option - Java or the iOS @@ -35,26 +32,27 @@ documentation for some common platforms. The mitmproxy CA cert is located in `~/.mitmproxy` after it has been generated at the first start of mitmproxy. -- [IOS](http://jasdev.me/intercepting-ios-traffic) +- curl on the command line: + `curl --proxy 127.0.0.1:8080 --cacert ~/.mitmproxy/mitmproxy-ca-cert.pem https://example.com/` +- wget on the command line: + `wget -e https_proxy=127.0.0.1:8080 --ca-certificate ~/.mitmproxy/mitmproxy-ca-cert.pem https://example.com/` +- [macOS](https://support.apple.com/guide/keychain-access/add-certificates-to-a-keychain-kyca2431/mac) +- [Ubuntu/Debian]( https://askubuntu.com/questions/73287/how-do-i-install-a-root-certificate/94861#94861) +- [Mozilla Firefox](https://wiki.mozilla.org/MozillaRootCertificate#Mozilla_Firefox) +- [Chrome on Linux](https://stackoverflow.com/a/15076602/198996) +- [iOS](http://jasdev.me/intercepting-ios-traffic) On recent iOS versions you also need to enable full trust for the mitmproxy root certificate: 1. Go to Settings > General > About > Certificate Trust Settings. 2. Under "Enable full trust for root certificates", turn on trust for the mitmproxy certificate. - [iOS Simulator](https://github.com/ADVTOOLS/ADVTrustStore#how-to-use-advtruststore) -- [Java](https://docs.oracle.com/cd/E19906-01/820-4916/geygn/index.html) +- [Java](https://docs.oracle.com/cd/E19906-01/820-4916/geygn/index.html): + `sudo keytool -importcert -alias mitmproxy -storepass changeit -keystore $JAVA_HOME/lib/security/cacerts -trustcacerts -file ~/.mitmproxy/mitmproxy-ca-cert.pem` - [Android/Android Simulator](http://wiki.cacert.org/FAQ/ImportRootCert#Android_Phones_.26_Tablets) - [Windows](https://web.archive.org/web/20160612045445/http://windows.microsoft.com/en-ca/windows/import-export-certificates-private-keys#1TC=windows-7) -- [Windows (automated)](https://technet.microsoft.com/en-us/library/cc732443.aspx) - -{{< highlight bash >}} -certutil -addstore root mitmproxy-ca-cert.cer -{{< / highlight >}} - -- [Mac OS X](https://support.apple.com/kb/PH20129) -- [Ubuntu/Debian]( https://askubuntu.com/questions/73287/how-do-i-install-a-root-certificate/94861#94861) -- [Mozilla Firefox](https://wiki.mozilla.org/MozillaRootCertificate#Mozilla_Firefox) -- [Chrome on Linux](https://stackoverflow.com/a/15076602/198996) +- [Windows (automated)](https://technet.microsoft.com/en-us/library/cc732443.aspx): + `certutil -addstore root mitmproxy-ca-cert.cer` ## The mitmproxy certificate authority @@ -65,7 +63,7 @@ CA out of the box, you will see an SSL certificate warning every time you visit a new SSL domain through mitmproxy. When you are testing a single site through a browser, just accepting the bogus SSL cert manually is not too much trouble, but -there are a many circumstances where you will want to configure your testing +there are many circumstances where you will want to configure your testing system or browser to trust the mitmproxy CA as a signing root authority. For security reasons, the mitmproxy CA is generated uniquely on the first start and is not shared between mitmproxy installations on different devices. @@ -117,26 +115,26 @@ For example, you can generate a certificate in this format using these instructions: -{{< highlight bash >}} +```bash openssl genrsa -out cert.key 2048 # (Specify the mitm domain as Common Name, e.g. \*.google.com) openssl req -new -x509 -key cert.key -out cert.crt -cat cert.key cert.crt \> cert.pem -{{< / highlight >}} +cat cert.key cert.crt > cert.pem +``` Now, you can run mitmproxy with the generated certificate: **For all domain names** -{{< highlight bash >}} +```bash mitmproxy --cert *=cert.pem -{{< / highlight >}} +``` **For specific domain names** -{{< highlight bash >}} +```bash mitmproxy --cert *.example.com=cert.pem -{{< / highlight >}} +``` **Note:** `*.example.com` is for all the subdomains. You can also use `www.example.com` for a particular subdomain. diff -Nru mitmproxy-5.1.1/docs/src/content/concepts-commands.md mitmproxy-6.0.2/docs/src/content/concepts-commands.md --- mitmproxy-5.1.1/docs/src/content/concepts-commands.md 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/content/concepts-commands.md 2020-12-15 16:41:27.000000000 +0000 @@ -22,14 +22,14 @@ browser (by default accessible with the `C` key binding). -# Working with flows +# Working with Flows Many of mitmproxy's commands take flows as arguments. For instance, the signature for the client replay commands looks like this: -{{< highlight none >}} +``` replay.client [flow] -{{< /highlight >}} +``` That means that it expects a sequence of one or more flows. This is where [flow @@ -40,23 +40,38 @@ Fire up mitmproxy console, and intercept some traffic so we have flows to work with. Now type the following command: -{{< highlight none >}} +``` :replay.client @focus -{{< /highlight >}} +``` Make sure you try using tab completion for the command name and the flow specification. The `@focus` specifiers expands to the currently focused flow, so you should see this flow replay. However, replay can take any number of flows. Try the following command: -{{< highlight none >}} +``` :replay.client @all -{{< /highlight >}} +``` Now you should see all flows replay one by one. We have the full power of the mitmproxy filter language at our disposal here, so we could also, for example, just replay flows for a specific domain: -{{< highlight none >}} +``` :replay.client "~d google.com" -{{< /highlight >}} +``` + +# Custom Key Bindings + +Mitmproxy's key bindings can be customized to your needs in the +`~/.mitmproxy/keys.yaml` file. This file consists of a sequence of maps, with +the following keys: + +* `key` (**mandatory**): The key to bind. +* `cmd` (**mandatory**): The command to execute when the key is pressed. +* `context`: A list of contexts in which the key should be bound. By default this is **global** (i.e. the key is bound everywhere). Valid contexts are `chooser`, `commands`, `dataviewer`, `eventlog`, `flowlist`, `flowview`, `global`, `grideditor`, `help`, `keybindings`, `options`. +* `help`: A help string for the binding which will be shown in the key binding browser. + +#### Example + +{{< example src="examples/keys.yaml" lang="yaml" >}} diff -Nru mitmproxy-5.1.1/docs/src/content/concepts-howmitmproxyworks.md mitmproxy-6.0.2/docs/src/content/concepts-howmitmproxyworks.md --- mitmproxy-5.1.1/docs/src/content/concepts-howmitmproxyworks.md 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/content/concepts-howmitmproxyworks.md 2020-12-15 16:41:27.000000000 +0000 @@ -25,9 +25,9 @@ client connects directly to the proxy, and makes a request that looks like this: -{{< highlight http >}} +```http GET http://example.com/index.html HTTP/1.1 -{{< / highlight >}} +``` This is a proxy GET request - an extended form of the vanilla HTTP GET request that includes a schema and host specification, and it includes @@ -47,9 +47,9 @@ different. The client connects to the proxy and makes a request that looks like this: -{{< highlight http >}} +```http CONNECT example.com:443 HTTP/1.1 -{{< / highlight >}} +``` A conventional proxy can neither view nor manipulate a TLS-encrypted data stream, so a CONNECT request simply asks the proxy to open a pipe @@ -91,9 +91,9 @@ example, both of these values are "example.com". But what if the client had initiated the connection as follows: -{{< highlight http >}} +```http CONNECT 10.1.1.1:443 HTTP/1.1 -{{< / highlight >}} +``` Using the IP address is perfectly legitimate because it gives us enough information to initiate the pipe, even though it doesn't reveal the @@ -182,9 +182,9 @@ client has initiated the connection, it makes a vanilla HTTP request, which might look something like this: -{{< highlight http >}} +```http GET /index.html HTTP/1.1 -{{< / highlight >}} +``` Note that this request differs from the explicit proxy variation, in that it omits the scheme and hostname. How, then, do we know which diff -Nru mitmproxy-5.1.1/docs/src/content/concepts-modes.md mitmproxy-6.0.2/docs/src/content/concepts-modes.md --- mitmproxy-5.1.1/docs/src/content/concepts-modes.md 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/content/concepts-modes.md 2020-12-15 16:41:27.000000000 +0000 @@ -24,7 +24,7 @@ 1. Start mitmproxy. 2. Configure your client to use mitmproxy by explicitly setting an HTTP - proxy. + proxy. By default, mitmproxy listens on port 8080. 3. Quick Check: You should already be able to visit an unencrypted HTTP site through the proxy. 4. Open the magic domain **mitm.it** and install the certificate for your diff -Nru mitmproxy-5.1.1/docs/src/content/concepts-protocols.md mitmproxy-6.0.2/docs/src/content/concepts-protocols.md --- mitmproxy-5.1.1/docs/src/content/concepts-protocols.md 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/content/concepts-protocols.md 2020-12-15 16:41:27.000000000 +0000 @@ -16,7 +16,7 @@ HTTP/1.0 and HTTP/1.1 support in mitmproxy is based on our custom HTTP stack, which takes care of all semantics and on-the-wire parsing/serialization tasks. -mitmproxy currently does not support HTTP trailers - but if you want to send +mitmproxy currently does not support parsing HTTP trailers - but if you want to send us a PR, we promise to take look! ## HTTP/2 @@ -29,9 +29,6 @@ API. mitmproxy supports the majority of HTTP/2 feature and tries to transparently pass-through as much information as possible. -mitmproxy currently does not support HTTP/2 trailers - but if you want to send -us a PR, we promise to take look! - mitmproxy currently does not support HTTP/2 Cleartext (h2c) since none of the major browser vendors have implemented it. @@ -55,8 +52,7 @@ [RFC7692: Compression Extensions for WebSocket](http://tools.ietf.org/html/rfc7692) -WebSocket support in mitmproxy is based on [wsproto] -(https://github.com/python-hyper/wsproto) project. It fully encapsulates +WebSocket support in mitmproxy is based on [wsproto](https://github.com/python-hyper/wsproto) project. It fully encapsulates WebSocket frames/messages/connections and provides an easy-to-use event-based API. diff -Nru mitmproxy-5.1.1/docs/src/content/howto-ignoredomains.md mitmproxy-6.0.2/docs/src/content/howto-ignoredomains.md --- mitmproxy-5.1.1/docs/src/content/howto-ignoredomains.md 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/content/howto-ignoredomains.md 2020-12-15 16:41:27.000000000 +0000 @@ -60,7 +60,7 @@ becomes \\.) and use this as your ignore pattern: -{{< highlight none >}} +``` >>> mitmdump -v 127.0.0.1:50588: clientconnect 127.0.0.1:50588: request @@ -70,11 +70,11 @@ -> example.com:443 ^C >>> mitmproxy --ignore-hosts ^example\.com:443$ -{{< /highlight >}} +``` Here are some other examples for ignore patterns: -{{< highlight none >}} +``` # Exempt traffic from the iOS App Store (the regex is lax, but usually just works): --ignore-hosts apple.com:443 # "Correct" version without false-positives: @@ -87,17 +87,20 @@ --ignore-hosts 17\.178\.96\.59:443 # IP address range: --ignore-hosts 17\.178\.\d+\.\d+:443 -{{< / highlight >}} +``` -This option can also be used to whitelist some domains through negative lookahead expressions. However, ignore patterns are always matched against the IP address of the target before being matched against its domain name. Thus, the pattern must allow any IP addresses using an expression like `^(?![0-9\.]+:)` in order for domains whitelisting to work. Here are examples of such patterns: +This option can also be used to only allow some specific domains through negative lookahead expressions. However, ignore +patterns are always matched against the IP address of the target before being matched against its domain name. Thus, the +pattern must allow any IP addresses using an expression like `^(?![0-9\.]+:)` in order for this to work. +Here are examples of such patterns: -{{< highlight none >}} +``` # Ignore everything but example.com and mitmproxy.org (not subdomains): --ignore-hosts '^(?![0-9\.]+:)(?!example\.com:)(?!mitmproxy\.org:)' # Ignore everything but example.com and its subdomains: --ignore-hosts '^(?![0-9\.]+:)(?!([^\.:]+\.)*example\.com:)' -{{< / highlight >}} +``` **Footnotes** diff -Nru mitmproxy-5.1.1/docs/src/content/howto-install-system-trusted-ca-android.md mitmproxy-6.0.2/docs/src/content/howto-install-system-trusted-ca-android.md --- mitmproxy-5.1.1/docs/src/content/howto-install-system-trusted-ca-android.md 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/content/howto-install-system-trusted-ca-android.md 2020-12-15 16:41:27.000000000 +0000 @@ -9,7 +9,7 @@ [Since Android 7, apps ignore user certificates](https://android-developers.googleblog.com/2016/07/changes-to-trusted-certificate.html), unless they are configured to use them. As most applications do not explicitly opt in to use user certificates, we need to place our mitmproxy CA certificate in the system certificate store, -in order to avid having to patch each application, which we want to monitor. +in order to avoid having to patch each application, which we want to monitor. Please note, that apps can decide to ignore the system certificate store and maintain their own CA certificates. In this case you have to patch the application. @@ -23,22 +23,22 @@ ## 2. Rename certificate Enter your certificate folder -{{< highlight bash >}} +```bash cd ~/.mitmproxy/ -{{< / highlight >}} +``` - CA Certificates in Android are stored by the name of their hash, with a '0' as extension - Now generate the hash of your certificate -{{< highlight bash >}} +```bash openssl x509 -inform PEM -subject_hash_old -in mitmproxy-ca-cert.cer | head -1 -{{< / highlight >}} +``` Lets assume, the output is `c8450d0d` We can now copy `mitmproxy-ca-cert.cer` to `c8450d0d.0` and our system certificate is ready to use -{{< highlight bash >}} +```bash cp mitmproxy-ca-cert.cer c8450d0d.0 -{{< / highlight >}} +``` ## 3. Insert certificate into system certificate store @@ -50,37 +50,37 @@ - Keep in mind, that the **emulator will load a clean system image when starting without `-writable-system` option**. - This means you always have to start the emulator with `-writable-system` option in order to use your certificate -{{< highlight bash >}} +```bash emulator -avd -writable-system -{{< / highlight >}} +``` - Restart adb as root -{{< highlight bash >}} +```bash adb root -{{< / highlight >}} +``` - Get write access to `/system` on the device - In earlier versions (API LEVEL < 28) of Android you have to use `adb shell "mount -o rw,remount /system"` -{{< highlight bash >}} +```bash adb shell "mount -o rw,remount /" -{{< / highlight >}} +``` - Push your certificate to the system certificate store and set file permissions -{{< highlight bash >}} +```bash adb push c8450d0d.0 /system/etc/security/cacerts adb shell "chmod 664 /system/etc/security/cacerts/c8450d0d.0" -{{< / highlight >}} +``` ## 4. Reboot device and enjoy decrypted TLS traffic - Reboot your device. - You CA certificate should now be system trusted -{{< highlight bash >}} +```bash adb reboot -{{< / highlight >}} +``` -**Remember**: You **always** have to start the emulator using the `-writable-system` option in order to use your certificate \ No newline at end of file +**Remember**: You **always** have to start the emulator using the `-writable-system` option in order to use your certificate diff -Nru mitmproxy-5.1.1/docs/src/content/howto-kubernetes.md mitmproxy-6.0.2/docs/src/content/howto-kubernetes.md --- mitmproxy-5.1.1/docs/src/content/howto-kubernetes.md 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/content/howto-kubernetes.md 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,13 @@ +--- +title: "Kubernetes Services" +menu: + howto: + weight: 1 +--- + +# Kubernetes Services + +The [github.com/soluble-ai/kubetap](https://github.com/soluble-ai/kubetap) project +provides a kubectl plugin for easily deploying mitmproxy to proxy Kubernetes Services. + +For usage and documentation, please refer to the [kubetap project site](https://soluble-ai.github.io/kubetap/). diff -Nru mitmproxy-5.1.1/docs/src/content/howto-transparent.md mitmproxy-6.0.2/docs/src/content/howto-transparent.md --- mitmproxy-5.1.1/docs/src/content/howto-transparent.md 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/content/howto-transparent.md 2020-12-15 16:41:27.000000000 +0000 @@ -34,10 +34,10 @@ ### 1. Enable IP forwarding. -{{< highlight bash >}} +```bash sysctl -w net.ipv4.ip_forward=1 sysctl -w net.ipv6.conf.all.forwarding=1 -{{< / highlight >}} +``` This makes sure that your machine forwards packets instead of rejecting them. @@ -46,9 +46,9 @@ ### 2. Disable ICMP redirects. -{{< highlight bash >}} +```bash sysctl -w net.ipv4.conf.all.send_redirects=0 -{{< / highlight >}} +``` If your test device is on the same physical network, your machine shouldn't inform the device that there's a shorter route available by skipping the proxy. @@ -60,12 +60,12 @@ Details will differ according to your setup, but the ruleset should look something like this: -{{< highlight bash >}} +```bash iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 80 -j REDIRECT --to-port 8080 iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 443 -j REDIRECT --to-port 8080 ip6tables -t nat -A PREROUTING -i eth0 -p tcp --dport 80 -j REDIRECT --to-port 8080 ip6tables -t nat -A PREROUTING -i eth0 -p tcp --dport 443 -j REDIRECT --to-port 8080 -{{< / highlight >}} +``` If you want to persist this across reboots, you can use the `iptables-persistent` package (see [here](http://www.microhowto.info/howto/make_the_configuration_of_iptables_persistent_on_debian.html)). @@ -74,9 +74,9 @@ You probably want a command like this: -{{< highlight bash >}} +```bash mitmproxy --mode transparent --showhost -{{< / highlight >}} +``` The `--mode transparent` option turns on transparent mode, and the `--showhost` argument tells mitmproxy to use the value of the Host header for URL display. @@ -92,24 +92,24 @@ Create a user to run the mitmproxy -{{< highlight bash >}} +```bash sudo useradd --create-home mitmproxyuser -sudo -u mitmproxyuser bash -c 'cd ~ && pip install --user mitmproxy' -{{< / highlight >}} +sudo -u mitmproxyuser -H bash -c 'cd ~ && pip install --user mitmproxy' +``` Then, configure the iptables rules to redirect all traffic from our local machine to mitmproxy. **Note**, as soon as you run these, you won't be able to perform successful network calls *until* you start mitmproxy. If you run into issues, `iptables -t nat -F` is a heavy handed way to flush (clear) *all* the rules from the iptables `nat` table (which includes any other rules you had configured). -{{< highlight bash >}} +```bash iptables -t nat -A OUTPUT -p tcp -m owner ! --uid-owner mitmproxyuser --dport 80 -j REDIRECT --to-port 8080 iptables -t nat -A OUTPUT -p tcp -m owner ! --uid-owner mitmproxyuser --dport 443 -j REDIRECT --to-port 8080 ip6tables -t nat -A OUTPUT -p tcp -m owner ! --uid-owner mitmproxyuser --dport 80 -j REDIRECT --to-port 8080 ip6tables -t nat -A OUTPUT -p tcp -m owner ! --uid-owner mitmproxyuser --dport 443 -j REDIRECT --to-port 8080 -{{< / highlight >}} +``` This will redirect the packets from all users other than `mitmproxyuser` on the machine to mitmproxy. To avoid circularity, run mitmproxy as the user `mitmproxyuser`. Hence step **4** should look like: -{{< highlight bash >}} -sudo -u mitmproxyuser bash -c '$HOME/.local/bin/mitmproxy --mode transparent --showhost --set block_global=false' -{{< / highlight >}} +```bash +sudo -u mitmproxyuser -H bash -c '$HOME/.local/bin/mitmproxy --mode transparent --showhost --set block_global=false' +``` @@ -117,16 +117,16 @@ ### 1. Enable IP forwarding. -{{< highlight bash >}} +```bash sudo sysctl -w net.inet.ip.forwarding=1 -{{< / highlight >}} +``` ### 2. Place the following two lines in **/etc/pf.conf**. -{{< highlight none >}} +``` mitm_if = "re2" pass in quick proto tcp from $mitm_if to port { 80, 443 } divert-to 127.0.0.1 port 8080 -{{< / highlight >}} +``` These rules tell pf to divert all traffic from `$mitm_if` destined for port 80 or 443 to the local mitmproxy instance running on port 8080. You should replace @@ -134,23 +134,23 @@ ### 3. Configure pf with the rules. -{{< highlight bash >}} +```bash doas pfctl -f /etc/pf.conf -{{< / highlight >}} +``` ### 4. And now enable it. -{{< highlight bash >}} +```bash doas pfctl -e -{{< / highlight >}} +``` ### 5. Fire up mitmproxy. You probably want a command like this: -{{< highlight bash >}} +```bash mitmproxy --mode transparent --listen-host 127.0.0.1 --showhost -{{< / highlight >}} +``` The `--mode transparent` option turns on transparent mode, and the `--showhost` argument tells mitmproxy to use the value of the Host header for URL display. @@ -184,16 +184,16 @@ ### 1. Enable IP forwarding. -{{< highlight bash >}} +```bash sudo sysctl -w net.inet.ip.forwarding=1 -{{< / highlight >}} +``` ### 2. Place the following line in a file called, say, **pf.conf**. -{{< highlight none >}} +``` rdr pass on en0 inet proto tcp to any port {80, 443} -> 127.0.0.1 port 8080 -{{< / highlight >}} +``` This rule tells pf to redirect all traffic destined for port 80 or 443 to the local mitmproxy instance running on port 8080. You should replace @@ -201,24 +201,24 @@ ### 3. Configure pf with the rules. -{{< highlight bash >}} +```bash sudo pfctl -f pf.conf -{{< / highlight >}} +``` ### 4. And now enable it. -{{< highlight bash >}} +```bash sudo pfctl -e -{{< / highlight >}} +``` ### 5. Configure sudoers to allow mitmproxy to access pfctl. Edit the file **/etc/sudoers** on your system as root. Add the following line to the end of the file: -{{< highlight none >}} +``` ALL ALL=NOPASSWD: /sbin/pfctl -s state -{{< / highlight >}} +``` Note that this allows any user on the system to run the command `/sbin/pfctl -s state` as root without a password. This only allows inspection of the state @@ -229,9 +229,9 @@ You probably want a command like this: -{{< highlight bash >}} +```bash mitmproxy --mode transparent --showhost -{{< / highlight >}} +``` The `--mode transparent` flag turns on transparent mode, and the `--showhost` argument tells mitmproxy to use the value of the Host header for URL display. @@ -256,7 +256,7 @@ Follow steps **1, 2** as above, but in step **2** change the contents of the file **pf.conf** to -{{< highlight none >}} +``` #The ports to redirect to proxy redir_ports = "{http, https}" @@ -274,13 +274,13 @@ rdr pass proto tcp from any to any port $redir_ports -> $tproxy pass out route-to (lo0 127.0.0.1) proto tcp from any to any port $redir_ports user { != $tproxy_user } -{{< / highlight >}} +``` Follow steps **3-5** above. This will redirect the packets from all users other than `nobody` on the machine to mitmproxy. To avoid circularity, run mitmproxy as the user `nobody`. Hence step **6** should look like: -{{< highlight bash >}} +```bash sudo -u nobody mitmproxy --mode transparent --showhost -{{< / highlight >}} +``` ## "Full" transparent mode on Linux @@ -289,7 +289,7 @@ be used to use the client's IP address for server-side connections. The following config is required for this mode to work: -{{< highlight bash >}} +```bash CLIENT_NET=192.168.1.0/24 TABLE_ID=100 MARK=1 @@ -303,15 +303,15 @@ ip rule add fwmark $MARK lookup $TABLE_ID ip route add local $CLIENT_NET dev lo table $TABLE_ID -{{< / highlight >}} +``` This mode does require root privileges though. There's a wrapper in the examples directory called 'mitmproxy_shim.c', which will enable you to use this mode with dropped privileges. It can be used as follows: -{{< highlight bash >}} +```bash gcc examples/complex/full_transparency_shim.c -o mitmproxy_shim -lcap sudo chown root:root mitmproxy_shim sudo chmod u+s mitmproxy_shim ./mitmproxy_shim $(which mitmproxy) --mode transparent --set spoof-source-address -{{< / highlight >}} +``` diff -Nru mitmproxy-5.1.1/docs/src/content/howto-transparent-vms.md mitmproxy-6.0.2/docs/src/content/howto-transparent-vms.md --- mitmproxy-5.1.1/docs/src/content/howto-transparent-vms.md 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/content/howto-transparent-vms.md 2020-12-15 16:41:27.000000000 +0000 @@ -16,9 +16,9 @@ First, we have to find out under which name Ubuntu has mapped our network interfaces. You can find this information with: -{{< highlight bash >}} +```bash ip link -{{< / highlight >}} +``` Usually with Ubuntu and Virtualbox, **eth0** or **enp0s3** (Ubuntu 15.10 and newer) is connected to the internet and **eth1** or **enp0s8** (Ubuntu 15.10 and newer) is connected to the internal network that will be proxified and configured to use a static ip (192.168.3.1). If the names differ, use the ones you got from the *ip link* command. @@ -46,27 +46,27 @@ **/etc/NetworkManager/NetworkManager.conf** and if on Ubuntu 16.04 or newer running: -{{< highlight bash >}} +```bash sudo systemctl restart NetworkManager -{{< / highlight >}} +``` If on Ubuntu 12.04 or 14.04 running: -{{< highlight bash >}} +```bash sudo restart network-manager -{{< / highlight >}} +``` afterwards. Now, dnsmasq can be be installed and configured: -{{< highlight bash >}} +```bash sudo apt-get install dnsmasq -{{< / highlight >}} +``` Replace **/etc/dnsmasq.conf** with the following configuration: -{{< highlight none >}} +``` # Listen for DNS requests on the internal network interface=eth1 bind-interfaces @@ -75,21 +75,21 @@ # Broadcast gateway and dns server information dhcp-option=option:router,192.168.3.1 dhcp-option=option:dns-server,192.168.3.1 -{{< / highlight >}} +``` Apply changes: If on Ubuntu 16.04 or newer: -{{< highlight bash >}} +```bash sudo systemctl restart dnsmasq -{{< / highlight >}} +``` If on Ubuntu 12.04 or 14.04: -{{< highlight bash >}} +```bash sudo service dnsmasq restart -{{< / highlight >}} +``` Your **proxied machine** in the internal virtual network should now receive an IP address via DHCP: @@ -101,19 +101,19 @@ To redirect traffic to mitmproxy, we need to enable IP forwarding and add two iptables rules: -{{< highlight bash >}} +```bash sudo sysctl -w net.ipv4.ip_forward=1 sudo iptables -t nat -A PREROUTING -i eth1 -p tcp --dport 80 -j REDIRECT --to-port 8080 sudo iptables -t nat -A PREROUTING -i eth1 -p tcp --dport 443 -j REDIRECT --to-port 8080 -{{< / highlight >}} +``` ## 4. Run mitmproxy Finally, we can run mitmproxy in transparent mode with -{{< highlight bash >}} +```bash mitmproxy --mode transparent -{{< / highlight >}} +``` The proxied machine cannot to leak any data outside of HTTP or DNS requests. If required, you can now [install the mitmproxy certificates on the proxied diff -Nru mitmproxy-5.1.1/docs/src/content/howto-wireshark-tls.md mitmproxy-6.0.2/docs/src/content/howto-wireshark-tls.md --- mitmproxy-5.1.1/docs/src/content/howto-wireshark-tls.md 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/content/howto-wireshark-tls.md 2020-12-15 16:41:27.000000000 +0000 @@ -13,13 +13,13 @@ Key logging is enabled by setting the environment variable `SSLKEYLOGFILE` so that it points to a writable text file: -{{< highlight bash >}} +```bash SSLKEYLOGFILE="$PWD/.mitmproxy/sslkeylogfile.txt" mitmproxy -{{< / highlight >}} +``` You can also `export` this environment variable to make it persistent for all applications started from your current shell session. You can specify the key file path in Wireshark via `Edit -> Preferences -> -Protocols -> SSL -> (Pre)-Master-Secret log filename`. If your SSLKEYLOGFILE +Protocols -> TLS -> (Pre)-Master-Secret log filename`. If your SSLKEYLOGFILE does not exist yet, just create an empty text file, so you can select it in Wireshark (or run mitmproxy to create and collect master secrets). diff -Nru mitmproxy-5.1.1/docs/src/content/_index.md mitmproxy-6.0.2/docs/src/content/_index.md --- mitmproxy-5.1.1/docs/src/content/_index.md 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/content/_index.md 2020-12-15 16:41:27.000000000 +0000 @@ -8,20 +8,7 @@ # Introduction -The mitmproxy project's tools are a set of front-ends that expose common -underlying functionality. - -**mitmproxy** is an interactive, SSL/TLS-capable intercepting proxy with a console interface for HTTP/1, HTTP/2, and WebSockets. - -**mitmdump** is the command-line version of mitmproxy. Think tcpdump for HTTP. - -**mitmweb** is a web-based interface for mitmproxy. - -Documentation, tutorials and distribution packages can be found on the -[mitmproxy website](https://mitmproxy.org). - -Development information and our source code can be found in our -[GitHub repository](https://github.com/mitmproxy/mitmproxy). +mitmproxy is a set of tools that provide an interactive, SSL/TLS-capable intercepting proxy for HTTP/1, HTTP/2, and WebSockets. ## Features @@ -34,4 +21,114 @@ - Transparent proxy mode on macOS and Linux - Make scripted changes to HTTP traffic using Python - SSL/TLS certificates for interception are generated on the fly -- And much, much more... +- And [much, much more...]({{< relref "overview-features">}}) + + +## 3 Powerful Core Tools + +The mitmproxy project's tools are a set of front-ends that expose common +underlying functionality. When we talk about "mitmproxy" we usually refer to any of the three tools - they +are just different front-ends to the same core proxy. + +**mitmproxy** is an interactive, SSL/TLS-capable intercepting proxy with a console interface for HTTP/1, HTTP/2, and WebSockets. + +**mitmweb** is a web-based interface for mitmproxy. + +**mitmdump** is the command-line version of mitmproxy. Think tcpdump for HTTP. + +Distribution packages can be found on the [mitmproxy website](https://mitmproxy.org). +Development information and our source code can be found in our +[GitHub repository](https://github.com/mitmproxy/mitmproxy). + +### mitmproxy + +{{< figure src="/screenshots/mitmproxy.png" >}} + +**mitmproxy** is a console tool that allows interactive examination and +modification of HTTP traffic. It differs from mitmdump in that all flows are +kept in memory, which means that it's intended for taking and manipulating +small-ish samples. Use the `?` shortcut key to view, context-sensitive +documentation from any **mitmproxy** screen. + +--- + +### mitmweb + +{{< figure src="/screenshots/mitmweb.png" >}} + +**mitmweb** is mitmproxy's web-based user interface that allows +interactive examination and modification of HTTP traffic. Like +mitmproxy, it differs from mitmdump in that all flows are kept in +memory, which means that it's intended for taking and manipulating +small-ish samples. + +{{% note %}} +Mitmweb is currently in beta. We consider it stable for all features +currently exposed in the UI, but it still misses a lot of mitmproxy's +features. +{{% /note %}} + +--- + +### mitmdump + +**mitmdump** is the command-line companion to mitmproxy. It provides +tcpdump-like functionality to let you view, record, and programmatically +transform HTTP traffic. See the `--help` flag output for complete +documentation. + + +#### Example: Saving traffic + +```bash +mitmdump -w outfile +``` + +Start up mitmdump in proxy mode, and write all traffic to **outfile**. + +#### Filtering saved traffic + +```bash +mitmdump -nr infile -w outfile "~m post" +``` + +Start mitmdump without binding to the proxy port (`-n`), read all flows +from infile, apply the specified filter expression (only match POSTs), +and write to outfile. + +#### Client replay + +```bash +mitmdump -nC outfile +``` + +Start mitmdump without binding to the proxy port (`-n`), then replay all +requests from outfile (`-C filename`). Flags combine in the obvious way, +so you can replay requests from one file, and write the resulting flows +to another: + +```bash +mitmdump -nC srcfile -w dstfile +``` + +See the [client-side replay]({{< relref "overview-features#client-side-replay" +>}}) section for more information. + +#### Running a script + +```bash +mitmdump -s examples/simple/add_header.py +``` + +This runs the **add_header.py** example script, which simply adds a new +header to all responses. + +#### Scripted data transformation + +```bash +mitmdump -ns examples/simple/add_header.py -r srcfile -w dstfile +``` + +This command loads flows from **srcfile**, transforms it according to +the specified script, then writes it back to **dstfile**. + diff -Nru mitmproxy-5.1.1/docs/src/content/mitmproxytutorial-interceptrequests.md mitmproxy-6.0.2/docs/src/content/mitmproxytutorial-interceptrequests.md --- mitmproxy-5.1.1/docs/src/content/mitmproxytutorial-interceptrequests.md 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/content/mitmproxytutorial-interceptrequests.md 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,22 @@ +--- +title: "Intercept Requests" +menu: + mitmproxytutorial: + weight: 2 +has_asciinema: true +--- + +# Intercept Requests + +A powerful feature of mitmproxy is the interception of requests. +An intercepted request is paused so that the user can modify (or discard) the request before sending it to the server. +mitmproxy's `set intercept` command configures interceptions. +The command is bound to shortcut `i` by default. + +Intercepting *all* requests is usually not desired as it constantly interrupts your browsing. +Thus, mitmproxy expects a [flow filter expression]({{< relref "concepts-filters" >}}) as the first argument to `set intercept` to selectively intercept requests. +In the tutorial below we use the flow filter `~u ` that filters flows by matching the regular expressing on the URL of the request. + +{{% asciicast file="mitmproxy_intercept_requests" poster="0:3" instructions=true %}} + +In the next lesson, you will learn to modify intercepted flows before sending them to the server. diff -Nru mitmproxy-5.1.1/docs/src/content/mitmproxytutorial-modifyrequests.md mitmproxy-6.0.2/docs/src/content/mitmproxytutorial-modifyrequests.md --- mitmproxy-5.1.1/docs/src/content/mitmproxytutorial-modifyrequests.md 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/content/mitmproxytutorial-modifyrequests.md 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,17 @@ +--- +title: "Modify Requests" +menu: + mitmproxytutorial: + weight: 3 +has_asciinema: true +--- + +# Modify Requests + +In the previous step we resumed intercepted requests without changes. +The full power of interceptions comes to play when we modify an intercepted request before forwarding it to its destination. +You can continue with the window and the already configured interception rule from the previous step. + +{{% asciicast file="mitmproxy_modify_requests" poster="0:3" instructions=true %}} + +In the next lesson, you will learn to replay previous flows. diff -Nru mitmproxy-5.1.1/docs/src/content/mitmproxytutorial-replayrequests.md mitmproxy-6.0.2/docs/src/content/mitmproxytutorial-replayrequests.md --- mitmproxy-5.1.1/docs/src/content/mitmproxytutorial-replayrequests.md 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/content/mitmproxytutorial-replayrequests.md 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,22 @@ +--- +title: "Replay Requests" +menu: + mitmproxytutorial: + weight: 4 +has_asciinema: true +--- + +# Replay Requests + +Another powerful feature of mitmproxy is replaying previous flows. +Two types of replays are supported: + +* **Client-side Replay:** mitmproxy replays previous client requests, i.e., sends the same request to the server again. +* **Server-side Replay:** mitmproxy replays server responses for requests that match an earlier recorded request. + +In this tutorial we focus on the more common use case of client-side replays. +See the docs for more info on [server-side replay]({{< relref "overview-features#server-side-replay" >}}). + +{{% asciicast file="mitmproxy_replay_requests" poster="0:3" instructions=true %}} + +You are almost done with this tutorial. In the last step you find more mitmproxy-related resources to discover. diff -Nru mitmproxy-5.1.1/docs/src/content/mitmproxytutorial-userinterface.md mitmproxy-6.0.2/docs/src/content/mitmproxytutorial-userinterface.md --- mitmproxy-5.1.1/docs/src/content/mitmproxytutorial-userinterface.md 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/content/mitmproxytutorial-userinterface.md 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,19 @@ +--- +title: "User Interface" +menu: + mitmproxytutorial: + weight: 1 +has_asciinema: true +--- + +# User Interface + +First of all, we need to become familiar with mitmproxy's user interface. +Open the terminal window in which you started mitmproxy. +You are in the default view of mitmproxy, which shows a list of flows. +You should see your browser's HTTP requests to load this tutorial. +mitmproxy adds rows to the view as new requests come in. + +{{% asciicast file="mitmproxy_user_interface" poster="0:3" instructions=true %}} + +In the next lesson, you will learn to intercept requests before sending them to the server. diff -Nru mitmproxy-5.1.1/docs/src/content/mitmproxytutorial-whatsnext.md mitmproxy-6.0.2/docs/src/content/mitmproxytutorial-whatsnext.md --- mitmproxy-5.1.1/docs/src/content/mitmproxytutorial-whatsnext.md 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/content/mitmproxytutorial-whatsnext.md 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,22 @@ +--- +title: "What's Next" +menu: + mitmproxytutorial: + weight: 5 +--- + +# What's Next + +Congratulations! You have successfully completed the mitmproxy tutorial. 🎉 + +In this tutorial we have used mitmproxy to inspect requests initiated by curl. +You probably also want to inspect web traffic from your browser or some other tool. +To do so, you need to [configure mitmproxy as your client's proxy]({{< relref "overview-getting-started#configure-your-browser-or-device" >}}). + +This tutorial only covered the most fundamental mitmproxy features. We recommend strongly to skim through the following +section as well, which describes most core concepts in more detail. + +# Feedback? + +We hope this tutorial was worthwhile and helped you getting up to speed with mitmproxy. +Is there anything you feel is missing? Or anything that is not clear? Please let us know in our dedicated issue on GitHub. diff -Nru mitmproxy-5.1.1/docs/src/content/overview-features.md mitmproxy-6.0.2/docs/src/content/overview-features.md --- mitmproxy-5.1.1/docs/src/content/overview-features.md 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/content/overview-features.md 2020-12-15 16:41:27.000000000 +0000 @@ -1,20 +1,21 @@ --- title: "Features" -menu: "overview" menu: - overview: + concepts: weight: 4 --- -# Mitmproxy Core Features +# Features - [Anticache](#anticache) - [Client-side replay](#client-side-replay) +- [Map Local](#map-local) +- [Map Remote](#map-remote) +- [Modify Body](#modify-body) +- [Modify Headers](#modify-headers) - [Proxy Authentication](#proxy-authentication) -- [Replacements](#replacements) - [Server-side replay](#server-side-replay) -- [Set Headers](#set-headers) - [Sticky Auth](#sticky-auth) - [Sticky Cookies](#sticky-cookies) - [Streaming](#streaming) @@ -41,34 +42,149 @@ You may want to use client-side replay in conjunction with the `anticache` option, to make sure the server responds with complete data. -## Proxy Authentication -Asks the user for authentication before they are permitted to use the proxy. -Authentication headers are stripped from the flows, so they are not passed to -upstream servers. For now, only HTTP Basic authentication is supported. The -proxy auth options are not compatible with the transparent, socks or reverse -proxy mode. +## Map Local + +The `map_local` option lets you specify an arbitrary number of patterns that +define redirections of HTTP requests to local files or directories. +The local file is fetched instead of the original resource +and transparently returned to the client. + +`map_local` patterns look like this: + +``` +|url-regex|local-path +|flow-filter|url-regex|local-path +``` + +* **local-path** is the file or directory that should be served to the client. + +* **url-regex** is a regular expression applied on the request URL. It must match for a redirect to take place. + +* **flow-filter** is an optional mitmproxy [filter expression]({{< relref "concepts-filters">}}) +that additionally constrains which requests will be redirected. + +### Examples + +Pattern | Description +------- | ----------- +`|example.com/main.js|~/main-local.js` | Replace `example.com/main.js` with `~/main-local.js`. +`|example.com/static|~/static` | Replace `example.com/static/foo/bar.css` with `~/static/foo/bar.css`. +`|example.com/static/foo|~/static` | Replace `example.com/static/foo/bar.css` with `~/static/bar.css`. +`|~m GET|example.com/static|~/static` | Replace `example.com/static/foo/bar.css` with `~/static/foo/bar.css` (but only for GET requests). + +### Details + +If *local-path* is a file, this file will always be served. File changes will be reflected immediately, there is no caching. + +If *local-path* is a directory, *url-regex* is used to split the request URL in two parts and part on the right is appended to *local-path*, excluding the query string. +However, if *url-regex* contains a regex capturing group, this behavior changes and the first capturing group is appended instead (and query strings are not stripped). +Special characters are mapped to `_`. If the file cannot be found, `/index.html` is appended and we try again. Directory traversal outside of the originally specified directory is not possible. + +To illustrate this, consider the following example which maps all requests for `example.org/css*` to the local directory `~/static-css`. + +
+                  ┌── url regex ──┬─ local path ─┐
+map_local option: |example.com/css|~/static-css
+                            │
+                            │    URL is split here
+                            ▼            ▼
+HTTP Request URL: https://example.com/css/print/main.css?timestamp=123
+                                               │                ▼
+                                               ▼              query string is ignored
+Served File:      Preferred: ~/static-css/print/main.css
+                   Fallback: ~/static-css/print/main.css/index.html
+                  Otherwise: 404 response without content
+
+ +If the file depends on the query string, we can use regex capturing groups. In this example, all `GET` requests for +`example.org/index.php?page=` are mapped to `~/static-dir/`: + +
+                    flow
+                  ┌filter┬─────────── url regex ───────────┬─ local path ─┐
+map_local option: |~m GET|example.com/index.php\\?page=(.+)|~/static-dir
+                            │                           │
+                            │                           │ regex group = suffix
+                            ▼                           ▼
+HTTP Request URL: https://example.com/index.php?page=aboutus
+                                                        │
+                                                        ▼
+Served File:                 Preferred: ~/static-dir/aboutus
+                              Fallback: ~/static-dir/aboutus/index.html
+                             Otherwise: 404 response without content
+
+ + + + +## Map Remote + +The `map_remote` option lets you specify an arbitrary number of patterns that +define replacements within HTTP request URLs before they are sent to a server. +The substituted URL is fetched instead of the original resource +and the corresponding HTTP response is returned transparently to the client. +Note that if the original destination uses HTTP2, the substituted destination +needs to support HTTP2 as well, otherwise the substituted request may fail. +As a workaround you can start mitmproxy with the `--no-http2` flag to disable HTTP2. +`map_remote` patterns look like this: + +``` +|flow-filter|url-regex|replacement +|url-regex|replacement +``` + +* **flow-filter** is an optional mitmproxy [filter expression]({{< relref "concepts-filters">}}) +that defines which requests the `map_remote` option applies to. + +* **url-regex** is a valid Python regular expression that defines what gets replaced in the URLs of requests. +* **replacement** is a string literal that is substituted in. -## Replacements +The _separator_ is arbitrary, and is defined by the first character. -The `replacements` option lets you specify an arbitrary number of patterns that -define text replacements within flows. A replacement pattern looks like this: +### Examples + +Map all requests ending with `.jpg` to `https://placedog.net/640/480?random`. +Note that this might fail if the original HTTP request destination uses HTTP2 but the replaced +destination does not support HTTP2. + +``` +|.*\.jpg$|https://placedog.net/640/480?random +``` + +Re-route all GET requests from `example.org` to `mitmproxy.org` (using `|` as the separator): + +``` +|~m GET|//example.org/|//mitmproxy.org/ +``` + + +## Modify Body + +The `modify_body` option lets you specify an arbitrary number of patterns that +define replacements within bodies of flows. `modify_body` patterns look like this: + +``` +/flow-filter/body-regex/replacement +/flow-filter/body-regex/@file-path +/body-regex/replacement +/body-regex/@file-path +``` -{{< highlight none >}} -/patt/regex/replacement -{{< / highlight >}} - -Here, **patt** is a mitmproxy filter expression that defines which flows a -replacement applies to, **regex** is a valid Python regular expression that -defines what gets replaced, and **replacement** is a string literal that is -substituted in. The separator is arbitrary, and defined by the first character. -If the replacement string literal starts with `@`, it is treated as a file path -from which the replacement is read. +* **flow-filter** is an optional mitmproxy [filter expression]({{< relref "concepts-filters">}}) +that defines which flows a replacement applies to. -Replace hooks fire when either a client request or a server response is +* **body-regex** is a valid Python regular expression that defines what gets replaced. + +* **replacement** is a string literal that is substituted in. If the replacement string +literal starts with `@` as in `@file-path`, it is treated as a **file path** from which the replacement is read. + +The _separator_ is arbitrary, and is defined by the first character. + +Modify hooks fire when either a client request or a server response is received. Only the matching flow component is affected: so, for example, -if a replace hook is triggered on server response, the replacement is +if a modify hook is triggered on server response, the replacement is only run on the Response object leaving the Request intact. You control whether the hook triggers on the request, response or both using the filter pattern. If you need finer-grained control than this, it's simple @@ -76,17 +192,89 @@ ### Examples -Replace `foo` with `bar` in requests: +Replace `foo` with `bar` in bodies of requests: -{{< highlight none >}} -:~q:foo:bar -{{< / highlight >}} +``` +/~q/foo/bar +``` Replace `foo` with the data read from `~/xss-exploit`: -{{< highlight bash >}} -mitmdump --replacements :~q:foo:@~/xss-exploit -{{< / highlight >}} +```bash +mitmdump --modify-body :~q:foo:@~/xss-exploit +``` + + +## Modify Headers + +The `modify_headers` option lets you specify a set of headers to be modified. +New headers can be added, and existing headers can be overwritten or removed. +`modify_headers` patterns look like this: + +``` +/flow-filter/name/value +/flow-filter/name/@file-path +/name/value +/name/@file-path +``` + +* **flow-filter** is an optional mitmproxy [filter expression]({{< relref "concepts-filters">}}) +that defines which flows to modify headers on. + +* **name** is the header name to be set, replaced or removed. + +* **value** is the header value to be set or replaced. An empty **value** removes existing +headers with **name**. If the value string literal starts with `@` as in +`@file-path`, it is treated as a **file path** from which the replacement is read. + +The _separator_ is arbitrary, and is defined by the first character. + +Existing headers are overwritten by default. This can be changed using a filter expression. + +Modify hooks fire when either a client request or a server response is +received. Only the matching flow component is affected: so, for example, +if a modify hook is triggered on server response, the replacement is +only run on the Response object leaving the Request intact. You control +whether the hook triggers on the request, response or both using the +filter pattern. If you need finer-grained control than this, it's simple +to create a script using the replacement API on Flow components. + +### Examples + +Set the `Host` header to `example.org` for all requests (existing `Host` +headers are replaced): + +``` +/~q/Host/example.org +``` + +Set the `Host` header to `example.org` for all requests that do not have an +existing `Host` header: + +``` +/~q & !~h Host:/Host/example.org +``` + +Set the `User-Agent` header to the data read from `~/useragent.txt` for all requests +(existing `User-Agent` headers are replaced): + +``` +/~q/Host/@~/useragent.txt +``` + +Remove existing `Host` headers from all requests: + +``` +/~q/Host/ +``` + +## Proxy Authentication + +Asks the user for authentication before they are permitted to use the proxy. +Authentication headers are stripped from the flows, so they are not passed to +upstream servers. For now, only HTTP Basic authentication is supported. The +proxy auth options are not compatible with the transparent, socks or reverse +proxy mode. ## Server-side replay @@ -131,19 +319,6 @@ Otherwise, the unmatched requests is forwarded to the upstream server. If forwarding is not desired, you can use the --kill (-k) switch to prevent that. -## Set Headers - -The `setheaders` option lets you specify a set of headers to be added to -requests or responses, based on a filter pattern. A `setheaders` expression -looks like this: - -{{< highlight none >}} -/patt/name/value -{{< / highlight >}} - -Here, **patt** is a mitmproxy filter expression that defines which flows to set -headers on, and **name** and **value** are the header name and the value to set -respectively. ## Sticky auth @@ -154,6 +329,7 @@ data-role="program">mitmproxy doesn't (yet) support replay of HTTP Digest authentication. + ## Sticky cookies When the `stickycookie` option is set, **mitmproxy** will add the cookie most @@ -171,6 +347,7 @@ authentication process once, and simply replay it on startup every time you need to interact with the secured resources. + ## Streaming By default, mitmproxy will read an entire request/response, perform any @@ -189,8 +366,7 @@ streamed. Requests/Responses that should be tagged for streaming by setting their ``.stream`` attribute to ``True``: -{{< example src="examples/complex/stream.py" lang="py" >}} - +{{< example src="examples/addons/http-stream-simple.py" lang="py" >}} ### Websockets @@ -201,6 +377,7 @@ server. In contrast to HTTP streaming, where the body is not stored, the message payload will still be stored in the WebSocket flow. + ## Upstream Certificates When mitmproxy receives a connection destined for an SSL-protected service, it diff -Nru mitmproxy-5.1.1/docs/src/content/overview-getting-started.md mitmproxy-6.0.2/docs/src/content/overview-getting-started.md --- mitmproxy-5.1.1/docs/src/content/overview-getting-started.md 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/content/overview-getting-started.md 2020-12-15 16:41:27.000000000 +0000 @@ -8,21 +8,22 @@ # Getting Started -You have already [installed]({{< relref "overview-installation">}}) mitmproxy on +We assume you have already [installed]({{< relref "overview-installation">}}) mitmproxy on your machine. -# Launch the tool you need -You can start any of our three tools from the command line / terminal: +## Launch the tool you need - * [mitmproxy]({{< relref "tools-mitmproxy">}}) -> gives you an interactive TUI - * [mitmdump]({{< relref "tools-mitmdump">}}) -> gives you a plain and simple terminal output - * [mitmweb]({{< relref "tools-mitmweb">}}) -> gives you a browser-based GUI +You can start any of our three tools from the command line / terminal. -When we talk about "mitmproxy" we usually refer to any of the three tools - they -are just different front-ends to the same core proxy. + * **mitmproxy** gives you an interactive TUI + * **mitmweb** gives you a browser-based GUI + * **mitmdump** gives you a plain and simple terminal output -# Configure your browser or device +In case you use the console-based version of mitmproxy, we highly recommend you to take the [tutorial]({{< relref "mitmproxytutorial-userinterface" >}}) to get started. + + +## Configure your browser or device For the basic setup as [regular proxy]({{< relref "concepts-modes#regular-proxy">}}), you need to configure your browser or device @@ -34,16 +35,22 @@ You can check that your web traffic is going through mitmproxy by browsing to http://mitm.it - it should present you with a [simple page]({{< relref -"concepts-certificates/#quick-setup">}}) to install the mitmproxy Certificate +"concepts-certificates#quick-setup">}}) to install the mitmproxy Certificate Authority - which is also the next steps. Follow the instructions for your OS / system and install the CA (and make sure to enable it, some system require multiple steps!). -# Verifying everything works + +## Verifying everything works At this point your running mitmproxy instance should already show the first HTTP flows from your client. You can test that all TLS-encrypted web traffic is working as expected by browsing to https://mitmproxy.org - it should show up as new flow and you can inspect it. -Done. + +## Resources + +* [**StackOverflow**](https://stackoverflow.com/questions/tagged/mitmproxy): If you want to ask usage questions, please do so on StackOverflow. +* [**GitHub**](https://github.com/mitmproxy/): If you want to contribute to mitmproxy or submit a bug report, please do so on GitHub. +* [**Slack**](https://mitmproxy.slack.com): If you want to get in touch with the developers or other users, please use our Slack channel. diff -Nru mitmproxy-5.1.1/docs/src/content/overview-installation.md mitmproxy-6.0.2/docs/src/content/overview-installation.md --- mitmproxy-5.1.1/docs/src/content/overview-installation.md 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/content/overview-installation.md 2020-12-15 16:41:27.000000000 +0000 @@ -15,9 +15,9 @@ The recommended way to install mitmproxy on macOS is to use [Homebrew](https://brew.sh/): -{{< highlight bash >}} +```bash brew install mitmproxy -{{< / highlight >}} +``` Alternatively, you can download standalone binaries on [mitmproxy.org](https://mitmproxy.org/). @@ -46,16 +46,16 @@ menu. Both executables are added to your PATH and can be invoked from the command line. -# Advanced Installation +## Advanced Installation -## Development Setup +### Development Setup If you would like to install mitmproxy directly from source code or the GitHub master branch, please see the our [README](https://github.com/mitmproxy/mitmproxy#installation) on GitHub. -## Installation from the Python Package Index (PyPI) +### Installation from the Python Package Index (PyPI) If your mitmproxy addons require the installation of additional Python packages, you can install mitmproxy from [PyPI](https://pypi.org/project/mitmproxy/). @@ -72,12 +72,12 @@ To install additional Python packages, run `pipx inject mitmproxy `. -## Docker Images +### Docker Images You can use the official mitmproxy images from [DockerHub](https://hub.docker.com/r/mitmproxy/mitmproxy/). -## Security Considerations for Binary Packages +### Security Considerations for Binary Packages Our pre-compiled binary packages and Docker images include a self-contained Python 3 environment, a recent version of OpenSSL that support ALPN and HTTP/2, diff -Nru mitmproxy-5.1.1/docs/src/content/tools-mitmdump.md mitmproxy-6.0.2/docs/src/content/tools-mitmdump.md --- mitmproxy-5.1.1/docs/src/content/tools-mitmdump.md 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/content/tools-mitmdump.md 1970-01-01 00:00:00.000000000 +0000 @@ -1,70 +0,0 @@ ---- -title: "mitmdump" -menu: "tools" -menu: - tools: - weight: 2 ---- - -## mitmdump - -**mitmdump** is the command-line companion to mitmproxy. It provides -tcpdump-like functionality to let you view, record, and programmatically -transform HTTP traffic. See the `--help` flag output for complete -documentation. - - -### Example: Saving traffic - -{{< highlight bash >}} -mitmdump -w outfile -{{< / highlight >}} - -Start up mitmdump in proxy mode, and write all traffic to **outfile**. - -### Filtering saved traffic - -{{< highlight bash >}} -mitmdump -nr infile -w outfile "~m post" -{{< / highlight >}} - -Start mitmdump without binding to the proxy port (`-n`), read all flows -from infile, apply the specified filter expression (only match POSTs), -and write to outfile. - -### Client replay - -{{< highlight bash >}} -mitmdump -nC outfile -{{< / highlight >}} - -Start mitmdump without binding to the proxy port (`-n`), then replay all -requests from outfile (`-C filename`). Flags combine in the obvious way, -so you can replay requests from one file, and write the resulting flows -to another: - -{{< highlight bash >}} -mitmdump -nC srcfile -w dstfile -{{< / highlight >}} - -See the [client-side replay]({{< relref "overview-features#client-side-replay" ->}}) section for more information. - -### Running a script - -{{< highlight bash >}} -mitmdump -s examples/simple/add_header.py -{{< / highlight >}} - -This runs the **add_header.py** example script, which simply adds a new -header to all responses. - -### Scripted data transformation - -{{< highlight bash >}} -mitmdump -ns examples/simple/add_header.py -r srcfile -w dstfile -{{< / highlight >}} - -This command loads flows from **srcfile**, transforms it according to -the specified script, then writes it back to **dstfile**. - diff -Nru mitmproxy-5.1.1/docs/src/content/tools-mitmproxy.md mitmproxy-6.0.2/docs/src/content/tools-mitmproxy.md --- mitmproxy-5.1.1/docs/src/content/tools-mitmproxy.md 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/content/tools-mitmproxy.md 1970-01-01 00:00:00.000000000 +0000 @@ -1,36 +0,0 @@ ---- -title: "mitmproxy" -menu: "tools" -menu: - tools: - weight: 1 ---- - -## mitmproxy - -{{< figure src="/screenshots/mitmproxy.png" >}} - -**mitmproxy** is a console tool that allows interactive examination and -modification of HTTP traffic. It differs from mitmdump in that all flows are -kept in memory, which means that it's intended for taking and manipulating -small-ish samples. Use the `?` shortcut key to view, context-sensitive -documentation from any **mitmproxy** screen. - -### Key binding configuration - -Mitmproxy's key bindings can be customized through in the -`~/.mitmproxy/keys.yaml` file. This file consists of a sequence of maps, with -the following keys: - -* `key` (**mandatory**): The key to bind. -* `cmd` (**mandatory**): The command to execute when the key is pressed. -* `context`: A list of contexts in which the key should be bound. By default this is **global** (i.e. the key is bound everywhere). Valid contexts are `chooser`, `commands`, `dataviewer`, `eventlog`, `flowlist`, `flowview`, `global`, `grideditor`, `help`, `keybindings`, `options`. -* `help`: A help string for the binding which will be shown in the key binding browser. - -#### Example - -{{< example src="examples/keys.yaml" lang="yaml" >}} - - - - diff -Nru mitmproxy-5.1.1/docs/src/content/tools-mitmweb.md mitmproxy-6.0.2/docs/src/content/tools-mitmweb.md --- mitmproxy-5.1.1/docs/src/content/tools-mitmweb.md 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/content/tools-mitmweb.md 1970-01-01 00:00:00.000000000 +0000 @@ -1,23 +0,0 @@ ---- -title: "mitmweb" -menu: "tools" -menu: - tools: - weight: 3 ---- - -## mitmweb - -{{< figure src="/screenshots/mitmweb.png" >}} - -**mitmweb** is mitmproxy's web-based user interface that allows -interactive examination and modification of HTTP traffic. Like -mitmproxy, it differs from mitmdump in that all flows are kept in -memory, which means that it's intended for taking and manipulating -small-ish samples. - -{{% note %}} -Mitmweb is currently in beta. We consider it stable for all features -currently exposed in the UI, but it still misses a lot of mitmproxy's -features. -{{% /note %}} diff -Nru mitmproxy-5.1.1/docs/src/content/tute-clientreplay.md mitmproxy-6.0.2/docs/src/content/tute-clientreplay.md --- mitmproxy-5.1.1/docs/src/content/tute-clientreplay.md 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/content/tute-clientreplay.md 2020-12-15 16:41:27.000000000 +0000 @@ -25,9 +25,9 @@ ## 1. Run mitmdump to record our HTTP conversation to a file. -{{< highlight bash >}} +```bash mitmdump -w wireless-login -{{< / highlight >}} +``` ## 2. Point your browser at the mitmdump instance. @@ -41,9 +41,9 @@ And that's it\! You now have a serialised version of the login process in the file wireless-login, and you can replay it at any time like this: -{{< highlight bash >}} +```bash mitmdump -C wireless-login -{{< / highlight >}} +``` ## Embellishments @@ -58,9 +58,9 @@ really needed and I somehow feel compelled to trim them anyway. So, we fire up the mitmproxy console tool on our serialised conversation, like so: -{{< highlight bash >}} +```bash mitmproxy -r wireless-login -{{< / highlight >}} +``` We can now go through and manually delete (using the d keyboard shortcut) everything we want to trim. When diff -Nru mitmproxy-5.1.1/docs/src/content/tute-highscores.md mitmproxy-6.0.2/docs/src/content/tute-highscores.md --- mitmproxy-5.1.1/docs/src/content/tute-highscores.md 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/content/tute-highscores.md 2020-12-15 16:41:27.000000000 +0000 @@ -33,13 +33,13 @@ this tantalising URL: -{{< highlight none >}} +``` https://service.gc.apple.com/WebObjects/GKGameStatsService.woa/wa/submitScore -{{< / highlight >}} +``` The contents of the submission are particularly interesting: -{{< highlight xml >}} +```xml scores @@ -57,7 +57,7 @@ -{{< / highlight >}} +``` This is a [property list](https://en.wikipedia.org/wiki/Property_list), containing an identifier for the game, a score (55, in this case), and a @@ -75,7 +75,7 @@ variable) will now fire up. Lets bump the score up to something a bit more ambitious: -{{< highlight xml >}} +```xml scores @@ -93,7 +93,7 @@ -{{< / highlight >}} +``` Save the file and exit your editor. diff -Nru mitmproxy-5.1.1/docs/src/examples/addons/addheader.py mitmproxy-6.0.2/docs/src/examples/addons/addheader.py --- mitmproxy-5.1.1/docs/src/examples/addons/addheader.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/addons/addheader.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,13 +0,0 @@ - -class AddHeader: - def __init__(self): - self.num = 0 - - def response(self, flow): - self.num = self.num + 1 - flow.response.headers["count"] = str(self.num) - - -addons = [ - AddHeader() -] diff -Nru mitmproxy-5.1.1/docs/src/examples/addons/anatomy.py mitmproxy-6.0.2/docs/src/examples/addons/anatomy.py --- mitmproxy-5.1.1/docs/src/examples/addons/anatomy.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/addons/anatomy.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,3 +1,8 @@ +""" +Basic skeleton of a mitmproxy addon. + +Run as follows: mitmproxy -s anatomy.py +""" from mitmproxy import ctx diff -Nru mitmproxy-5.1.1/docs/src/examples/addons/commands-flows.py mitmproxy-6.0.2/docs/src/examples/addons/commands-flows.py --- mitmproxy-5.1.1/docs/src/examples/addons/commands-flows.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/addons/commands-flows.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,3 +1,4 @@ +"""Handle flows as command arguments.""" import typing from mitmproxy import command @@ -6,9 +7,6 @@ class MyAddon: - def __init__(self): - self.num = 0 - @command.command("myaddon.addheader") def addheader(self, flows: typing.Sequence[flow.Flow]) -> None: for f in flows: diff -Nru mitmproxy-5.1.1/docs/src/examples/addons/commands-paths.py mitmproxy-6.0.2/docs/src/examples/addons/commands-paths.py --- mitmproxy-5.1.1/docs/src/examples/addons/commands-paths.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/addons/commands-paths.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,3 +1,4 @@ +"""Handle file paths as command arguments.""" import typing from mitmproxy import command @@ -7,9 +8,6 @@ class MyAddon: - def __init__(self): - self.num = 0 - @command.command("myaddon.histogram") def histogram( self, @@ -22,7 +20,7 @@ with open(path, "w+") as fp: for cnt, dom in sorted([(v, k) for (k, v) in totals.items()]): - fp.write("%s: %s\n" % (cnt, dom)) + fp.write(f"{cnt}: {dom}\n") ctx.log.alert("done") diff -Nru mitmproxy-5.1.1/docs/src/examples/addons/commands-simple.py mitmproxy-6.0.2/docs/src/examples/addons/commands-simple.py --- mitmproxy-5.1.1/docs/src/examples/addons/commands-simple.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/addons/commands-simple.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,3 +1,4 @@ +"""Add a custom command to mitmproxy's command prompt.""" from mitmproxy import command from mitmproxy import ctx @@ -9,7 +10,7 @@ @command.command("myaddon.inc") def inc(self) -> None: self.num += 1 - ctx.log.info("num = %s" % self.num) + ctx.log.info(f"num = {self.num}") addons = [ diff -Nru mitmproxy-5.1.1/docs/src/examples/addons/contentview.py mitmproxy-6.0.2/docs/src/examples/addons/contentview.py --- mitmproxy-5.1.1/docs/src/examples/addons/contentview.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/addons/contentview.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,27 @@ +""" +Add a custom message body pretty-printer for use inside mitmproxy. + +This example shows how one can add a custom contentview to mitmproxy, +which is used to pretty-print HTTP bodies for example. +The content view API is explained in the mitmproxy.contentviews module. +""" +from mitmproxy import contentviews + + +class ViewSwapCase(contentviews.View): + name = "swapcase" + content_types = ["text/plain"] + + def __call__(self, data, **metadata) -> contentviews.TViewResult: + return "case-swapped text", contentviews.format_text(data.swapcase()) + + +view = ViewSwapCase() + + +def load(l): + contentviews.add(view) + + +def done(): + contentviews.remove(view) diff -Nru mitmproxy-5.1.1/docs/src/examples/addons/duplicate-modify-replay.py mitmproxy-6.0.2/docs/src/examples/addons/duplicate-modify-replay.py --- mitmproxy-5.1.1/docs/src/examples/addons/duplicate-modify-replay.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/addons/duplicate-modify-replay.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,15 @@ +"""Take incoming HTTP requests and replay them with modified parameters.""" +from mitmproxy import ctx + + +def request(flow): + # Avoid an infinite loop by not replaying already replayed requests + if flow.is_replay == "request": + return + flow = flow.copy() + # Only interactive tools have a view. If we have one, add a duplicate entry + # for our flow. + if "view" in ctx.master.addons: + ctx.master.commands.call("view.flows.add", [flow]) + flow.request.path = "/changed" + ctx.master.commands.call("replay.client", [flow]) diff -Nru mitmproxy-5.1.1/docs/src/examples/addons/events-http-specific.py mitmproxy-6.0.2/docs/src/examples/addons/events-http-specific.py --- mitmproxy-5.1.1/docs/src/examples/addons/events-http-specific.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/addons/events-http-specific.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,8 +1,8 @@ +"""HTTP-specific events.""" import mitmproxy.http class Events: - # HTTP lifecycle def http_connect(self, flow: mitmproxy.http.HTTPFlow): """ An HTTP CONNECT request was received. Setting a non 2xx response on diff -Nru mitmproxy-5.1.1/docs/src/examples/addons/events.py mitmproxy-6.0.2/docs/src/examples/addons/events.py --- mitmproxy-5.1.1/docs/src/examples/addons/events.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/addons/events.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,11 +1,9 @@ +"""Generic event hooks.""" import typing import mitmproxy.addonmanager import mitmproxy.connections -import mitmproxy.http import mitmproxy.log -import mitmproxy.tcp -import mitmproxy.websocket import mitmproxy.proxy.protocol diff -Nru mitmproxy-5.1.1/docs/src/examples/addons/events-tcp-specific.py mitmproxy-6.0.2/docs/src/examples/addons/events-tcp-specific.py --- mitmproxy-5.1.1/docs/src/examples/addons/events-tcp-specific.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/addons/events-tcp-specific.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,8 +1,8 @@ +"""TCP-specific events.""" import mitmproxy.tcp class Events: - # TCP lifecycle def tcp_start(self, flow: mitmproxy.tcp.TCPFlow): """ A TCP connection has started. diff -Nru mitmproxy-5.1.1/docs/src/examples/addons/events-websocket-specific.py mitmproxy-6.0.2/docs/src/examples/addons/events-websocket-specific.py --- mitmproxy-5.1.1/docs/src/examples/addons/events-websocket-specific.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/addons/events-websocket-specific.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,9 +1,10 @@ +"""WebSocket-specific events.""" import mitmproxy.http import mitmproxy.websocket class Events: - # Websocket lifecycle + # WebSocket lifecycle def websocket_handshake(self, flow: mitmproxy.http.HTTPFlow): """ Called when a client wants to establish a WebSocket connection. The @@ -14,7 +15,7 @@ def websocket_start(self, flow: mitmproxy.websocket.WebSocketFlow): """ - A websocket connection has commenced. + A WebSocket connection has commenced. """ def websocket_message(self, flow: mitmproxy.websocket.WebSocketFlow): @@ -27,10 +28,10 @@ def websocket_error(self, flow: mitmproxy.websocket.WebSocketFlow): """ - A websocket connection has had an error. + A WebSocket connection has had an error. """ def websocket_end(self, flow: mitmproxy.websocket.WebSocketFlow): """ - A websocket connection has ended. + A WebSocket connection has ended. """ diff -Nru mitmproxy-5.1.1/docs/src/examples/addons/filter-flows.py mitmproxy-6.0.2/docs/src/examples/addons/filter-flows.py --- mitmproxy-5.1.1/docs/src/examples/addons/filter-flows.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/addons/filter-flows.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,26 @@ +""" +Use mitmproxy's filter pattern in scripts. +""" +from mitmproxy import flowfilter +from mitmproxy import ctx, http + + +class Filter: + def __init__(self): + self.filter: flowfilter.TFilter = None + + def configure(self, updated): + self.filter = flowfilter.parse(ctx.options.flowfilter) + + def load(self, l): + l.add_option( + "flowfilter", str, "", "Check that flow matches filter." + ) + + def response(self, flow: http.HTTPFlow) -> None: + if flowfilter.match(self.filter, flow): + ctx.log.info("Flow matches filter:") + ctx.log.info(flow) + + +addons = [Filter()] diff -Nru mitmproxy-5.1.1/docs/src/examples/addons/http-add-header.py mitmproxy-6.0.2/docs/src/examples/addons/http-add-header.py --- mitmproxy-5.1.1/docs/src/examples/addons/http-add-header.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/addons/http-add-header.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,15 @@ +"""Add an HTTP header to each response.""" + + +class AddHeader: + def __init__(self): + self.num = 0 + + def response(self, flow): + self.num = self.num + 1 + flow.response.headers["count"] = str(self.num) + + +addons = [ + AddHeader() +] diff -Nru mitmproxy-5.1.1/docs/src/examples/addons/http-modify-form.py mitmproxy-6.0.2/docs/src/examples/addons/http-modify-form.py --- mitmproxy-5.1.1/docs/src/examples/addons/http-modify-form.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/addons/http-modify-form.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,14 @@ +"""Modify an HTTP form submission.""" +from mitmproxy import http + + +def request(flow: http.HTTPFlow) -> None: + if flow.request.urlencoded_form: + # If there's already a form, one can just add items to the dict: + flow.request.urlencoded_form["mitmproxy"] = "rocks" + else: + # One can also just pass new form data. + # This sets the proper content type and overrides the body. + flow.request.urlencoded_form = [ + ("foo", "bar") + ] diff -Nru mitmproxy-5.1.1/docs/src/examples/addons/http-modify-query-string.py mitmproxy-6.0.2/docs/src/examples/addons/http-modify-query-string.py --- mitmproxy-5.1.1/docs/src/examples/addons/http-modify-query-string.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/addons/http-modify-query-string.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,6 @@ +"""Modify HTTP query parameters.""" +from mitmproxy import http + + +def request(flow: http.HTTPFlow) -> None: + flow.request.query["mitmproxy"] = "rocks" diff -Nru mitmproxy-5.1.1/docs/src/examples/addons/http-redirect-requests.py mitmproxy-6.0.2/docs/src/examples/addons/http-redirect-requests.py --- mitmproxy-5.1.1/docs/src/examples/addons/http-redirect-requests.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/addons/http-redirect-requests.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,10 @@ +"""Redirect HTTP requests to another server.""" +from mitmproxy import http + + +def request(flow: http.HTTPFlow) -> None: + # pretty_host takes the "Host" header of the request into account, + # which is useful in transparent mode where we usually only have the IP + # otherwise. + if flow.request.pretty_host == "example.org": + flow.request.host = "mitmproxy.org" diff -Nru mitmproxy-5.1.1/docs/src/examples/addons/http-reply-from-proxy.py mitmproxy-6.0.2/docs/src/examples/addons/http-reply-from-proxy.py --- mitmproxy-5.1.1/docs/src/examples/addons/http-reply-from-proxy.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/addons/http-reply-from-proxy.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,11 @@ +"""Send a reply from the proxy without sending any data to the remote server.""" +from mitmproxy import http + + +def request(flow: http.HTTPFlow) -> None: + if flow.request.pretty_url == "http://example.com/path": + flow.response = http.HTTPResponse.make( + 200, # (optional) status code + b"Hello World", # (optional) content + {"Content-Type": "text/html"} # (optional) headers + ) diff -Nru mitmproxy-5.1.1/docs/src/examples/addons/http-stream-modify.py mitmproxy-6.0.2/docs/src/examples/addons/http-stream-modify.py --- mitmproxy-5.1.1/docs/src/examples/addons/http-stream-modify.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/addons/http-stream-modify.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,21 @@ +""" +Modify a streamed response. + +Generally speaking, we recommend *not* to stream messages you need to modify. +Modifying streamed responses is tricky and brittle: + - If the transfer encoding isn't chunked, you cannot simply change the content length. + - If you want to replace all occurrences of "foobar", make sure to catch the cases + where one chunk ends with [...]foo" and the next starts with "bar[...]. +""" + + +def modify(chunks): + """ + chunks is a generator that can be used to iterate over all chunks. + """ + for chunk in chunks: + yield chunk.replace("foo", "bar") + + +def responseheaders(flow): + flow.response.stream = modify diff -Nru mitmproxy-5.1.1/docs/src/examples/addons/http-stream-simple.py mitmproxy-6.0.2/docs/src/examples/addons/http-stream-simple.py --- mitmproxy-5.1.1/docs/src/examples/addons/http-stream-simple.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/addons/http-stream-simple.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,14 @@ +""" +Select which responses should be streamed. + +Enable response streaming for all HTTP flows. +This is equivalent to passing `--set stream_large_bodies=1` to mitmproxy. +""" + + +def responseheaders(flow): + """ + Enables streaming for all responses. + This is equivalent to passing `--set stream_large_bodies=1` to mitmproxy. + """ + flow.response.stream = True diff -Nru mitmproxy-5.1.1/docs/src/examples/addons/http-trailers.py mitmproxy-6.0.2/docs/src/examples/addons/http-trailers.py --- mitmproxy-5.1.1/docs/src/examples/addons/http-trailers.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/addons/http-trailers.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,54 @@ +""" +This script simply prints all received HTTP Trailers. + +HTTP requests and responses can container trailing headers which are sent after +the body is fully transmitted. Such trailers need to be announced in the initial +headers by name, so the receiving endpoint can wait and read them after the +body. +""" + +from mitmproxy import http +from mitmproxy.net.http import Headers + + +def request(flow: http.HTTPFlow): + if flow.request.trailers: + print("HTTP Trailers detected! Request contains:", flow.request.trailers) + + if flow.request.path == "/inject_trailers": + if flow.request.is_http10: + # HTTP/1.0 doesn't support trailers + return + elif flow.request.is_http11: + if not flow.request.content: + # Avoid sending a body on GET requests or a 0 byte chunked body with trailers. + # Otherwise some servers return 400 Bad Request. + return + # HTTP 1.1 requires transfer-encoding: chunked to send trailers + flow.request.headers["transfer-encoding"] = "chunked" + # HTTP 2+ supports trailers on all requests/responses + + flow.request.headers["trailer"] = "x-my-injected-trailer-header" + flow.request.trailers = Headers([ + (b"x-my-injected-trailer-header", b"foobar") + ]) + print("Injected a new request trailer...", flow.request.headers["trailer"]) + + +def response(flow: http.HTTPFlow): + if flow.response.trailers: + print("HTTP Trailers detected! Response contains:", flow.response.trailers) + + if flow.request.path == "/inject_trailers": + if flow.request.is_http10: + return + elif flow.request.is_http11: + if not flow.response.content: + return + flow.response.headers["transfer-encoding"] = "chunked" + + flow.response.headers["trailer"] = "x-my-injected-trailer-header" + flow.response.trailers = Headers([ + (b"x-my-injected-trailer-header", b"foobar") + ]) + print("Injected a new response trailer...", flow.response.headers["trailer"]) diff -Nru mitmproxy-5.1.1/docs/src/examples/addons/internet_in_mirror.py mitmproxy-6.0.2/docs/src/examples/addons/internet_in_mirror.py --- mitmproxy-5.1.1/docs/src/examples/addons/internet_in_mirror.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/addons/internet_in_mirror.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,11 @@ +""" +Mirror all web pages. + +Useful if you are living down under. +""" +from mitmproxy import http + + +def response(flow: http.HTTPFlow) -> None: + reflector = b"" + flow.response.content = flow.response.content.replace(b"", reflector) diff -Nru mitmproxy-5.1.1/docs/src/examples/addons/io-read-saved-flows.py mitmproxy-6.0.2/docs/src/examples/addons/io-read-saved-flows.py --- mitmproxy-5.1.1/docs/src/examples/addons/io-read-saved-flows.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/addons/io-read-saved-flows.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,20 @@ +#!/usr/bin/env python +""" +Read a mitmproxy dump file. +""" +from mitmproxy import io +from mitmproxy.exceptions import FlowReadException +import pprint +import sys + +with open(sys.argv[1], "rb") as logfile: + freader = io.FlowReader(logfile) + pp = pprint.PrettyPrinter(indent=4) + try: + for f in freader.stream(): + print(f) + print(f.request.host) + pp.pprint(f.get_state()) + print("") + except FlowReadException as e: + print(f"Flow file corrupted: {e}") diff -Nru mitmproxy-5.1.1/docs/src/examples/addons/io-write-flow-file.py mitmproxy-6.0.2/docs/src/examples/addons/io-write-flow-file.py --- mitmproxy-5.1.1/docs/src/examples/addons/io-write-flow-file.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/addons/io-write-flow-file.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,29 @@ +""" +Generate a mitmproxy dump file. + +This script demonstrates how to generate a mitmproxy dump file, +as it would also be generated by passing `-w` to mitmproxy. +In contrast to `-w`, this gives you full control over which +flows should be saved and also allows you to rotate files or log +to multiple files in parallel. +""" +import random +import sys +from mitmproxy import io, http +import typing # noqa + + +class Writer: + def __init__(self, path: str) -> None: + self.f: typing.IO[bytes] = open(path, "wb") + self.w = io.FlowWriter(self.f) + + def response(self, flow: http.HTTPFlow) -> None: + if random.choice([True, False]): + self.w.add(flow) + + def done(self): + self.f.close() + + +addons = [Writer(sys.argv[1])] diff -Nru mitmproxy-5.1.1/docs/src/examples/addons/log-events.py mitmproxy-6.0.2/docs/src/examples/addons/log-events.py --- mitmproxy-5.1.1/docs/src/examples/addons/log-events.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/addons/log-events.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,8 @@ +"""Post messages to mitmproxy's event log.""" +from mitmproxy import ctx + + +def load(l): + ctx.log.info("This is some informative text.") + ctx.log.warn("This is a warning.") + ctx.log.error("This is an error.") diff -Nru mitmproxy-5.1.1/docs/src/examples/addons/nonblocking.py mitmproxy-6.0.2/docs/src/examples/addons/nonblocking.py --- mitmproxy-5.1.1/docs/src/examples/addons/nonblocking.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/addons/nonblocking.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,17 @@ +""" +Make events hooks non-blocking. + +When event hooks are decorated with @concurrent, they will be run in their own thread, freeing the main event loop. +Please note that this generally opens the door to race conditions and decreases performance if not required. +""" +import time + +from mitmproxy.script import concurrent + + +@concurrent # Remove this and see what happens +def request(flow): + # This is ugly in mitmproxy's UI, but you don't want to use mitmproxy.ctx.log from a different thread. + print(f"handle request: {flow.request.host}{flow.request.path}") + time.sleep(5) + print(f"start request: {flow.request.host}{flow.request.path}") diff -Nru mitmproxy-5.1.1/docs/src/examples/addons/options-configure.py mitmproxy-6.0.2/docs/src/examples/addons/options-configure.py --- mitmproxy-5.1.1/docs/src/examples/addons/options-configure.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/addons/options-configure.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,3 +1,4 @@ +"""React to configuration changes.""" import typing from mitmproxy import ctx diff -Nru mitmproxy-5.1.1/docs/src/examples/addons/options-simple.py mitmproxy-6.0.2/docs/src/examples/addons/options-simple.py --- mitmproxy-5.1.1/docs/src/examples/addons/options-simple.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/addons/options-simple.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,3 +1,10 @@ +""" +Add a new mitmproxy option. + +Usage: + + mitmproxy -s options-simple.py --set addheader true +""" from mitmproxy import ctx diff -Nru mitmproxy-5.1.1/docs/src/examples/addons/scripting-headers.py mitmproxy-6.0.2/docs/src/examples/addons/scripting-headers.py --- mitmproxy-5.1.1/docs/src/examples/addons/scripting-headers.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/addons/scripting-headers.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,3 +0,0 @@ - -def request(flow): - flow.request.headers["myheader"] = "value" diff -Nru mitmproxy-5.1.1/docs/src/examples/addons/scripting-minimal-example.py mitmproxy-6.0.2/docs/src/examples/addons/scripting-minimal-example.py --- mitmproxy-5.1.1/docs/src/examples/addons/scripting-minimal-example.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/addons/scripting-minimal-example.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,2 @@ +def request(flow): + flow.request.headers["myheader"] = "value" diff -Nru mitmproxy-5.1.1/docs/src/examples/addons/shutdown.py mitmproxy-6.0.2/docs/src/examples/addons/shutdown.py --- mitmproxy-5.1.1/docs/src/examples/addons/shutdown.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/addons/shutdown.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,18 @@ +""" +A simple way of shutting down the mitmproxy instance to stop everything. + +Usage: + + mitmproxy -s shutdown.py + + and then send a HTTP request to trigger the shutdown: + curl --proxy localhost:8080 http://example.com/path +""" +from mitmproxy import ctx, http + + +def request(flow: http.HTTPFlow) -> None: + # a random condition to make this example a bit more interactive + if flow.request.pretty_url == "http://example.com/path": + ctx.log.info("Shutting down everything...") + ctx.master.shutdown() diff -Nru mitmproxy-5.1.1/docs/src/examples/addons/tcp-simple.py mitmproxy-6.0.2/docs/src/examples/addons/tcp-simple.py --- mitmproxy-5.1.1/docs/src/examples/addons/tcp-simple.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/addons/tcp-simple.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,24 @@ +""" +Process individual messages from a TCP connection. + +This script replaces full occurences of "foo" with "bar" and prints various details for each message. +Please note that TCP is stream-based and *not* message-based. mitmproxy splits stream contents into "messages" +as they are received by socket.recv(). This is pretty arbitrary and should not be relied on. +However, it is sometimes good enough as a quick hack. + +Example Invocation: + + mitmdump --rawtcp --tcp-hosts ".*" -s examples/tcp-simple.py +""" +from mitmproxy.utils import strutils +from mitmproxy import ctx +from mitmproxy import tcp + + +def tcp_message(flow: tcp.TCPFlow): + message = flow.messages[-1] + message.content = message.content.replace(b"foo", b"bar") + + ctx.log.info( + f"tcp_message[from_client={message.from_client}), content={strutils.bytes_to_escaped_str(message.content)}]" + ) diff -Nru mitmproxy-5.1.1/docs/src/examples/addons/websocket-inject-message.py mitmproxy-6.0.2/docs/src/examples/addons/websocket-inject-message.py --- mitmproxy-5.1.1/docs/src/examples/addons/websocket-inject-message.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/addons/websocket-inject-message.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,24 @@ +""" +Inject a WebSocket message into a running connection. + +This example shows how to inject a WebSocket message to the client. +Every new WebSocket connection will trigger a new asyncio task that +periodically injects a new message to the client. +""" +import asyncio +import mitmproxy.websocket + + +class InjectWebSocketMessage: + async def inject(self, flow: mitmproxy.websocket.WebSocketFlow): + i = 0 + while not flow.ended and not flow.error: + await asyncio.sleep(5) + flow.inject_message(flow.client_conn, f'This is the #{i} injected message!') + i += 1 + + def websocket_start(self, flow): + asyncio.get_event_loop().create_task(self.inject(flow)) + + +addons = [InjectWebSocketMessage()] diff -Nru mitmproxy-5.1.1/docs/src/examples/addons/websocket-simple.py mitmproxy-6.0.2/docs/src/examples/addons/websocket-simple.py --- mitmproxy-5.1.1/docs/src/examples/addons/websocket-simple.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/addons/websocket-simple.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,21 @@ +"""Process individual messages from a WebSocket connection.""" +import re +from mitmproxy import ctx + + +def websocket_message(flow): + # get the latest message + message = flow.messages[-1] + + # was the message sent from the client or server? + if message.from_client: + ctx.log.info(f"Client sent a message: {message.content}") + else: + ctx.log.info(f"Server sent a message: {message.content}") + + # manipulate the message content + message.content = re.sub(r'^Hello', 'HAPPY', message.content) + + if 'FOOBAR' in message.content: + # kill the message and not send it to the other endpoint + message.kill() diff -Nru mitmproxy-5.1.1/docs/src/examples/addons/wsgi-flask-app.py mitmproxy-6.0.2/docs/src/examples/addons/wsgi-flask-app.py --- mitmproxy-5.1.1/docs/src/examples/addons/wsgi-flask-app.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/addons/wsgi-flask-app.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,27 @@ +""" +Host a WSGI app in mitmproxy. + +This example shows how to graft a WSGI app onto mitmproxy. In this +instance, we're using the Flask framework (http://flask.pocoo.org/) to expose +a single simplest-possible page. +""" +from flask import Flask +from mitmproxy.addons import asgiapp + +app = Flask("proxapp") + + +@app.route('/') +def hello_world() -> str: + return 'Hello World!' + + +addons = [ + # Host app at the magic domain "example.com" on port 80. Requests to this + # domain and port combination will now be routed to the WSGI app instance. + asgiapp.WSGIApp(app, "example.com", 80) + # SSL works too, but the magic domain needs to be resolvable from the mitmproxy machine due to mitmproxy's design. + # mitmproxy will connect to said domain and use serve its certificate (unless --no-upstream-cert is set) + # but won't send any data. + # mitmproxy.ctx.master.apps.add(app, "example.com", 443) +] diff -Nru mitmproxy-5.1.1/docs/src/examples/complex/block_dns_over_https.py mitmproxy-6.0.2/docs/src/examples/complex/block_dns_over_https.py --- mitmproxy-5.1.1/docs/src/examples/complex/block_dns_over_https.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/complex/block_dns_over_https.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,186 +0,0 @@ -""" -This module is for blocking DNS over HTTPS requests. - -It loads a blocklist of IPs and hostnames that are known to serve DNS over HTTPS requests. -It also uses headers, query params, and paths to detect DoH (and block it) -""" -from typing import List - -from mitmproxy import ctx - -# known DoH providers' hostnames and IP addresses to block -default_blocklist: dict = { - "hostnames": [ - "dns.adguard.com", "dns-family.adguard.com", "dns.google", "cloudflare-dns.com", - "mozilla.cloudflare-dns.com", "security.cloudflare-dns.com", "family.cloudflare-dns.com", - "dns.quad9.net", "dns9.quad9.net", "dns10.quad9.net", "dns11.quad9.net", "doh.opendns.com", - "doh.familyshield.opendns.com", "doh.cleanbrowsing.org", "doh.xfinity.com", "dohdot.coxlab.net", - "odvr.nic.cz", "doh.dnslify.com", "dns.nextdns.io", "dns.dnsoverhttps.net", "doh.crypto.sx", - "doh.powerdns.org", "doh-fi.blahdns.com", "doh-jp.blahdns.com", "doh-de.blahdns.com", - "doh.ffmuc.net", "dns.dns-over-https.com", "doh.securedns.eu", "dns.rubyfish.cn", - "dns.containerpi.com", "dns.containerpi.com", "dns.containerpi.com", "doh-2.seby.io", - "doh.seby.io", "commons.host", "doh.dnswarden.com", "doh.dnswarden.com", "doh.dnswarden.com", - "dns-nyc.aaflalo.me", "dns.aaflalo.me", "doh.applied-privacy.net", "doh.captnemo.in", - "doh.tiar.app", "doh.tiarap.org", "doh.dns.sb", "rdns.faelix.net", "doh.li", "doh.armadillodns.net", - "jp.tiar.app", "jp.tiarap.org", "doh.42l.fr", "dns.hostux.net", "dns.hostux.net", "dns.aa.net.uk", - "adblock.mydns.network", "ibksturm.synology.me", "jcdns.fun", "ibuki.cgnat.net", "dns.twnic.tw", - "example.doh.blockerdns.com", "dns.digitale-gesellschaft.ch", "doh.libredns.gr", - "doh.centraleu.pi-dns.com", "doh.northeu.pi-dns.com", "doh.westus.pi-dns.com", - "doh.eastus.pi-dns.com", "dns.flatuslifir.is", "private.canadianshield.cira.ca", - "protected.canadianshield.cira.ca", "family.canadianshield.cira.ca", "dns.google.com", - "dns.google.com" - ], - "ips": [ - "104.16.248.249", "104.16.248.249", "104.16.249.249", "104.16.249.249", "104.18.2.55", - "104.18.26.128", "104.18.27.128", "104.18.3.55", "104.18.44.204", "104.18.44.204", - "104.18.45.204", "104.18.45.204", "104.182.57.196", "104.236.178.232", "104.24.122.53", - "104.24.123.53", "104.28.0.106", "104.28.1.106", "104.31.90.138", "104.31.91.138", - "115.159.131.230", "116.202.176.26", "116.203.115.192", "136.144.215.158", "139.59.48.222", - "139.99.222.72", "146.112.41.2", "146.112.41.3", "146.185.167.43", "149.112.112.10", - "149.112.112.11", "149.112.112.112", "149.112.112.9", "149.112.121.10", "149.112.121.20", - "149.112.121.30", "149.112.122.10", "149.112.122.20", "149.112.122.30", "159.69.198.101", - "168.235.81.167", "172.104.93.80", "172.65.3.223", "174.138.29.175", "174.68.248.77", - "176.103.130.130", "176.103.130.131", "176.103.130.132", "176.103.130.134", "176.56.236.175", - "178.62.214.105", "185.134.196.54", "185.134.197.54", "185.213.26.187", "185.216.27.142", - "185.228.168.10", "185.228.168.168", "185.235.81.1", "185.26.126.37", "185.26.126.37", - "185.43.135.1", "185.95.218.42", "185.95.218.43", "195.30.94.28", "2001:148f:fffe::1", - "2001:19f0:7001:3259:5400:2ff:fe71:bc9", "2001:19f0:7001:5554:5400:2ff:fe57:3077", - "2001:19f0:7001:5554:5400:2ff:fe57:3077", "2001:19f0:7001:5554:5400:2ff:fe57:3077", - "2001:4860:4860::8844", "2001:4860:4860::8888", - "2001:4b98:dc2:43:216:3eff:fe86:1d28", "2001:558:fe21:6b:96:113:151:149", - "2001:608:a01::3", "2001:678:888:69:c45d:2738:c3f2:1878", "2001:8b0::2022", "2001:8b0::2023", - "2001:c50:ffff:1:101:101:101:101", "210.17.9.228", "217.169.20.22", "217.169.20.23", - "2400:6180:0:d0::5f73:4001", "2400:8902::f03c:91ff:feda:c514", "2604:180:f3::42", - "2604:a880:1:20::51:f001", "2606:4700::6810:f8f9", "2606:4700::6810:f9f9", "2606:4700::6812:1a80", - "2606:4700::6812:1b80", "2606:4700::6812:237", "2606:4700::6812:337", "2606:4700:3033::6812:2ccc", - "2606:4700:3033::6812:2dcc", "2606:4700:3033::6818:7b35", "2606:4700:3034::681c:16a", - "2606:4700:3035::6818:7a35", "2606:4700:3035::681f:5a8a", "2606:4700:3036::681c:6a", - "2606:4700:3036::681f:5b8a", "2606:4700:60:0:a71e:6467:cef8:2a56", "2620:10a:80bb::10", - "2620:10a:80bb::20", "2620:10a:80bb::30" "2620:10a:80bc::10", "2620:10a:80bc::20", - "2620:10a:80bc::30", "2620:119:fc::2", "2620:119:fc::3", "2620:fe::10", "2620:fe::11", - "2620:fe::9", "2620:fe::fe:10", "2620:fe::fe:11", "2620:fe::fe:9", "2620:fe::fe", - "2a00:5a60::ad1:ff", "2a00:5a60::ad2:ff", "2a00:5a60::bad1:ff", "2a00:5a60::bad2:ff", - "2a00:d880:5:bf0::7c93", "2a01:4f8:1c0c:8233::1", "2a01:4f8:1c1c:6b4b::1", "2a01:4f8:c2c:52bf::1", - "2a01:4f9:c010:43ce::1", "2a01:4f9:c01f:4::abcd", "2a01:7c8:d002:1ef:5054:ff:fe40:3703", - "2a01:9e00::54", "2a01:9e00::55", "2a01:9e01::54", "2a01:9e01::55", - "2a02:1205:34d5:5070:b26e:bfff:fe1d:e19b", "2a03:4000:38:53c::2", - "2a03:b0c0:0:1010::e9a:3001", "2a04:bdc7:100:70::abcd", "2a05:fc84::42", "2a05:fc84::43", - "2a07:a8c0::", "2a0d:4d00:81::1", "2a0d:5600:33:3::abcd", "35.198.2.76", "35.231.247.227", - "45.32.55.94", "45.67.219.208", "45.76.113.31", "45.77.180.10", "45.90.28.0", - "46.101.66.244", "46.227.200.54", "46.227.200.55", "46.239.223.80", "8.8.4.4", - "8.8.8.8", "83.77.85.7", "88.198.91.187", "9.9.9.10", "9.9.9.11", "9.9.9.9", - "94.130.106.88", "95.216.181.228", "95.216.212.177", "96.113.151.148", - ] -} - -# additional hostnames to block -additional_doh_names: List[str] = [ - 'dns.google.com' -] - -# additional IPs to block -additional_doh_ips: List[str] = [ - -] - -doh_hostnames, doh_ips = default_blocklist['hostnames'], default_blocklist['ips'] - -# convert to sets for faster lookups -doh_hostnames = set(doh_hostnames) -doh_ips = set(doh_ips) - - -def _has_dns_message_content_type(flow): - """ - Check if HTTP request has a DNS-looking 'Content-Type' header - - :param flow: mitmproxy flow - :return: True if 'Content-Type' header is DNS-looking, False otherwise - """ - doh_content_types = ['application/dns-message'] - if 'Content-Type' in flow.request.headers: - if flow.request.headers['Content-Type'] in doh_content_types: - return True - return False - - -def _request_has_dns_query_string(flow): - """ - Check if the query string of a request contains the parameter 'dns' - - :param flow: mitmproxy flow - :return: True is 'dns' is a parameter in the query string, False otherwise - """ - return 'dns' in flow.request.query - - -def _request_is_dns_json(flow): - """ - Check if the request looks like DoH with JSON. - - The only known implementations of DoH with JSON are Cloudflare and Google. - - For more info, see: - - https://developers.cloudflare.com/1.1.1.1/dns-over-https/json-format/ - - https://developers.google.com/speed/public-dns/docs/doh/json - - :param flow: mitmproxy flow - :return: True is request looks like DNS JSON, False otherwise - """ - # Header 'Accept: application/dns-json' is required in Cloudflare's DoH JSON API - # or they return a 400 HTTP response code - if 'Accept' in flow.request.headers: - if flow.request.headers['Accept'] == 'application/dns-json': - return True - # Google's DoH JSON API is https://dns.google/resolve - path = flow.request.path.split('?')[0] - if flow.request.host == 'dns.google' and path == '/resolve': - return True - return False - - -def _request_has_doh_looking_path(flow): - """ - Check if the path looks like it's DoH. - Most common one is '/dns-query', likely because that's what's in the RFC - - :param flow: mitmproxy flow - :return: True if path looks like it's DoH, otherwise False - """ - doh_paths = [ - '/dns-query', # used in example in RFC 8484 (see https://tools.ietf.org/html/rfc8484#section-4.1.1) - ] - path = flow.request.path.split('?')[0] - return path in doh_paths - - -def _requested_hostname_is_in_doh_blacklist(flow): - """ - Check if server hostname is in our DoH provider blacklist. - - The current blacklist is taken from https://github.com/curl/curl/wiki/DNS-over-HTTPS. - - :param flow: mitmproxy flow - :return: True if server's hostname is in DoH blacklist, otherwise False - """ - hostname = flow.request.host - ip = flow.server_conn.address - return hostname in doh_hostnames or hostname in doh_ips or ip in doh_ips - - -doh_request_detection_checks = [ - _has_dns_message_content_type, - _request_has_dns_query_string, - _request_is_dns_json, - _requested_hostname_is_in_doh_blacklist, - _request_has_doh_looking_path -] - - -def request(flow): - for check in doh_request_detection_checks: - is_doh = check(flow) - if is_doh: - ctx.log.warn("[DoH Detection] DNS over HTTPS request detected via method \"%s\"" % check.__name__) - flow.kill() - break diff -Nru mitmproxy-5.1.1/docs/src/examples/complex/change_upstream_proxy.py mitmproxy-6.0.2/docs/src/examples/complex/change_upstream_proxy.py --- mitmproxy-5.1.1/docs/src/examples/complex/change_upstream_proxy.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/complex/change_upstream_proxy.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,27 +0,0 @@ -from mitmproxy import http -import typing - -# This scripts demonstrates how mitmproxy can switch to a second/different upstream proxy -# in upstream proxy mode. -# -# Usage: mitmdump -U http://default-upstream-proxy.local:8080/ -s change_upstream_proxy.py -# -# If you want to change the target server, you should modify flow.request.host and flow.request.port - - -def proxy_address(flow: http.HTTPFlow) -> typing.Tuple[str, int]: - # Poor man's loadbalancing: route every second domain through the alternative proxy. - if hash(flow.request.host) % 2 == 1: - return ("localhost", 8082) - else: - return ("localhost", 8081) - - -def request(flow: http.HTTPFlow) -> None: - if flow.request.method == "CONNECT": - # If the decision is done by domain, one could also modify the server address here. - # We do it after CONNECT here to have the request data available as well. - return - address = proxy_address(flow) - if flow.live: - flow.live.change_upstream_proxy_server(address) # type: ignore diff -Nru mitmproxy-5.1.1/docs/src/examples/complex/dns_spoofing.py mitmproxy-6.0.2/docs/src/examples/complex/dns_spoofing.py --- mitmproxy-5.1.1/docs/src/examples/complex/dns_spoofing.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/complex/dns_spoofing.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,57 +0,0 @@ -""" -This script makes it possible to use mitmproxy in scenarios where IP spoofing -has been used to redirect connections to mitmproxy. The way this works is that -we rely on either the TLS Server Name Indication (SNI) or the Host header of the -HTTP request. Of course, this is not foolproof - if an HTTPS connection comes -without SNI, we don't know the actual target and cannot construct a certificate -that looks valid. Similarly, if there's no Host header or a spoofed Host header, -we're out of luck as well. Using transparent mode is the better option most of -the time. - -Usage: - mitmproxy - -p 443 - -s dns_spoofing.py - # Used as the target location if neither SNI nor host header are present. - --mode reverse:http://example.com/ - # To avoid auto rewriting of host header by the reverse proxy target. - --set keep_host_header - mitmdump - -p 80 - --mode reverse:http://localhost:443/ - - (Setting up a single proxy instance and using iptables to redirect to it - works as well) -""" -import re - -# This regex extracts splits the host header into host and port. -# Handles the edge case of IPv6 addresses containing colons. -# https://bugzilla.mozilla.org/show_bug.cgi?id=45891 -parse_host_header = re.compile(r"^(?P[^:]+|\[.+\])(?::(?P\d+))?$") - - -class Rerouter: - def request(self, flow): - if flow.client_conn.tls_established: - flow.request.scheme = "https" - sni = flow.client_conn.connection.get_servername() - port = 443 - else: - flow.request.scheme = "http" - sni = None - port = 80 - - host_header = flow.request.host_header - m = parse_host_header.match(host_header) - if m: - host_header = m.group("host").strip("[]") - if m.group("port"): - port = int(m.group("port")) - - flow.request.host_header = host_header - flow.request.host = sni or host_header - flow.request.port = port - - -addons = [Rerouter()] diff -Nru mitmproxy-5.1.1/docs/src/examples/complex/dup_and_replay.py mitmproxy-6.0.2/docs/src/examples/complex/dup_and_replay.py --- mitmproxy-5.1.1/docs/src/examples/complex/dup_and_replay.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/complex/dup_and_replay.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,14 +0,0 @@ -from mitmproxy import ctx - - -def request(flow): - # Avoid an infinite loop by not replaying already replayed requests - if flow.request.is_replay: - return - flow = flow.copy() - # Only interactive tools have a view. If we have one, add a duplicate entry - # for our flow. - if "view" in ctx.master.addons: - ctx.master.commands.call("view.flows.add", [flow]) - flow.request.path = "/changed" - ctx.master.commands.call("replay.client", [flow]) diff -Nru mitmproxy-5.1.1/docs/src/examples/complex/full_transparency_shim.c mitmproxy-6.0.2/docs/src/examples/complex/full_transparency_shim.c --- mitmproxy-5.1.1/docs/src/examples/complex/full_transparency_shim.c 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/complex/full_transparency_shim.c 1970-01-01 00:00:00.000000000 +0000 @@ -1,87 +0,0 @@ -#define _GNU_SOURCE -#include -#include -#include -#include -#include -#include -#include - -/* This setuid wrapper can be used to run mitmproxy in full transparency mode, as a normal user. - * It will set the required capabilities (CAP_NET_RAW), drop privileges, and will then run argv[1] - * with the same capabilities. - * - * It can be compiled as follows: - * gcc examples/mitmproxy_shim.c -o mitmproxy_shim -lcap -*/ - -int set_caps(cap_t cap_struct, cap_value_t *cap_list, size_t bufsize) { - int cap_count = bufsize / sizeof(cap_list[0]); - - if (cap_set_flag(cap_struct, CAP_PERMITTED, cap_count, cap_list, CAP_SET) || - cap_set_flag(cap_struct, CAP_EFFECTIVE, cap_count, cap_list, CAP_SET) || - cap_set_flag(cap_struct, CAP_INHERITABLE, cap_count, cap_list, CAP_SET)) { - if (cap_count < 2) { - fprintf(stderr, "Cannot manipulate capability data structure as user: %s.\n", strerror(errno)); - } else { - fprintf(stderr, "Cannot manipulate capability data structure as root: %s.\n", strerror(errno)); - } - return -1; - } - - if (cap_count < 2) { - if (prctl(PR_CAP_AMBIENT, PR_CAP_AMBIENT_RAISE, CAP_NET_RAW, 0, 0)) { - fprintf(stderr, "Failed to add CAP_NET_RAW to the ambient set: %s.\n", strerror(errno)); - return -2; - } - } - - if (cap_set_proc(cap_struct)) { - if (cap_count < 2) { - fprintf(stderr, "Cannot set capabilities as user: %s.\n", strerror(errno)); - } else { - fprintf(stderr, "Cannot set capabilities as root: %s.\n", strerror(errno)); - } - return -3; - } - - if (cap_count > 1) { - if (prctl(PR_SET_KEEPCAPS, 1L)) { - fprintf(stderr, "Cannot keep capabilities after dropping privileges: %s.\n", strerror(errno)); - return -4; - } - if (cap_clear(cap_struct)) { - fprintf(stderr, "Cannot clear capability data structure: %s.\n", strerror(errno)); - return -5; - } - } -} - -int main(int argc, char **argv, char **envp) { - cap_t cap_struct = cap_init(); - cap_value_t root_caps[2] = { CAP_NET_RAW, CAP_SETUID }; - cap_value_t user_caps[1] = { CAP_NET_RAW }; - uid_t user = getuid(); - int res; - - if (setresuid(0, 0, 0)) { - fprintf(stderr, "Cannot switch to root: %s.\n", strerror(errno)); - return 1; - } - - if (res = set_caps(cap_struct, root_caps, sizeof(root_caps))) - return res; - - if (setresuid(user, user, user)) { - fprintf(stderr, "Cannot drop root privileges: %s.\n", strerror(errno)); - return 2; - } - - if (res = set_caps(cap_struct, user_caps, sizeof(user_caps))) - return res; - - if (execve(argv[1], argv + 1, envp)) { - fprintf(stderr, "Failed to execute %s: %s\n", argv[1], strerror(errno)); - return 3; - } -} diff -Nru mitmproxy-5.1.1/docs/src/examples/complex/har_dump.py mitmproxy-6.0.2/docs/src/examples/complex/har_dump.py --- mitmproxy-5.1.1/docs/src/examples/complex/har_dump.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/complex/har_dump.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,222 +0,0 @@ -""" -This inline script can be used to dump flows as HAR files. - -example cmdline invocation: -mitmdump -s ./har_dump.py --set hardump=./dump.har - -filename endwith '.zhar' will be compressed: -mitmdump -s ./har_dump.py --set hardump=./dump.zhar -""" - - -import json -import base64 -import zlib -import os -import typing # noqa - -from datetime import datetime -from datetime import timezone - -import mitmproxy - -from mitmproxy import connections # noqa -from mitmproxy import version -from mitmproxy import ctx -from mitmproxy.utils import strutils -from mitmproxy.net.http import cookies - -HAR: typing.Dict = {} - -# A list of server seen till now is maintained so we can avoid -# using 'connect' time for entries that use an existing connection. -SERVERS_SEEN: typing.Set[connections.ServerConnection] = set() - - -def load(l): - l.add_option( - "hardump", str, "", "HAR dump path.", - ) - - -def configure(updated): - HAR.update({ - "log": { - "version": "1.2", - "creator": { - "name": "mitmproxy har_dump", - "version": "0.1", - "comment": "mitmproxy version %s" % version.MITMPROXY - }, - "entries": [] - } - }) - - -def response(flow): - """ - Called when a server response has been received. - """ - - # -1 indicates that these values do not apply to current request - ssl_time = -1 - connect_time = -1 - - if flow.server_conn and flow.server_conn not in SERVERS_SEEN: - connect_time = (flow.server_conn.timestamp_tcp_setup - - flow.server_conn.timestamp_start) - - if flow.server_conn.timestamp_tls_setup is not None: - ssl_time = (flow.server_conn.timestamp_tls_setup - - flow.server_conn.timestamp_tcp_setup) - - SERVERS_SEEN.add(flow.server_conn) - - # Calculate raw timings from timestamps. DNS timings can not be calculated - # for lack of a way to measure it. The same goes for HAR blocked. - # mitmproxy will open a server connection as soon as it receives the host - # and port from the client connection. So, the time spent waiting is actually - # spent waiting between request.timestamp_end and response.timestamp_start - # thus it correlates to HAR wait instead. - timings_raw = { - 'send': flow.request.timestamp_end - flow.request.timestamp_start, - 'receive': flow.response.timestamp_end - flow.response.timestamp_start, - 'wait': flow.response.timestamp_start - flow.request.timestamp_end, - 'connect': connect_time, - 'ssl': ssl_time, - } - - # HAR timings are integers in ms, so we re-encode the raw timings to that format. - timings = { - k: int(1000 * v) if v != -1 else -1 - for k, v in timings_raw.items() - } - - # full_time is the sum of all timings. - # Timings set to -1 will be ignored as per spec. - full_time = sum(v for v in timings.values() if v > -1) - - started_date_time = datetime.fromtimestamp(flow.request.timestamp_start, timezone.utc).isoformat() - - # Response body size and encoding - response_body_size = len(flow.response.raw_content) if flow.response.raw_content else 0 - response_body_decoded_size = len(flow.response.content) if flow.response.content else 0 - response_body_compression = response_body_decoded_size - response_body_size - - entry = { - "startedDateTime": started_date_time, - "time": full_time, - "request": { - "method": flow.request.method, - "url": flow.request.url, - "httpVersion": flow.request.http_version, - "cookies": format_request_cookies(flow.request.cookies.fields), - "headers": name_value(flow.request.headers), - "queryString": name_value(flow.request.query or {}), - "headersSize": len(str(flow.request.headers)), - "bodySize": len(flow.request.content), - }, - "response": { - "status": flow.response.status_code, - "statusText": flow.response.reason, - "httpVersion": flow.response.http_version, - "cookies": format_response_cookies(flow.response.cookies.fields), - "headers": name_value(flow.response.headers), - "content": { - "size": response_body_size, - "compression": response_body_compression, - "mimeType": flow.response.headers.get('Content-Type', '') - }, - "redirectURL": flow.response.headers.get('Location', ''), - "headersSize": len(str(flow.response.headers)), - "bodySize": response_body_size, - }, - "cache": {}, - "timings": timings, - } - - # Store binary data as base64 - if strutils.is_mostly_bin(flow.response.content): - entry["response"]["content"]["text"] = base64.b64encode(flow.response.content).decode() - entry["response"]["content"]["encoding"] = "base64" - else: - entry["response"]["content"]["text"] = flow.response.get_text(strict=False) - - if flow.request.method in ["POST", "PUT", "PATCH"]: - params = [ - {"name": a, "value": b} - for a, b in flow.request.urlencoded_form.items(multi=True) - ] - entry["request"]["postData"] = { - "mimeType": flow.request.headers.get("Content-Type", ""), - "text": flow.request.get_text(strict=False), - "params": params - } - - if flow.server_conn.connected(): - entry["serverIPAddress"] = str(flow.server_conn.ip_address[0]) - - HAR["log"]["entries"].append(entry) - - -def done(): - """ - Called once on script shutdown, after any other events. - """ - if ctx.options.hardump: - json_dump: str = json.dumps(HAR, indent=2) - - if ctx.options.hardump == '-': - mitmproxy.ctx.log(json_dump) - else: - raw: bytes = json_dump.encode() - if ctx.options.hardump.endswith('.zhar'): - raw = zlib.compress(raw, 9) - - with open(os.path.expanduser(ctx.options.hardump), "wb") as f: - f.write(raw) - - mitmproxy.ctx.log("HAR dump finished (wrote %s bytes to file)" % len(json_dump)) - - -def format_cookies(cookie_list): - rv = [] - - for name, value, attrs in cookie_list: - cookie_har = { - "name": name, - "value": value, - } - - # HAR only needs some attributes - for key in ["path", "domain", "comment"]: - if key in attrs: - cookie_har[key] = attrs[key] - - # These keys need to be boolean! - for key in ["httpOnly", "secure"]: - cookie_har[key] = bool(key in attrs) - - # Expiration time needs to be formatted - expire_ts = cookies.get_expiration_ts(attrs) - if expire_ts is not None: - cookie_har["expires"] = datetime.fromtimestamp(expire_ts, timezone.utc).isoformat() - - rv.append(cookie_har) - - return rv - - -def format_request_cookies(fields): - return format_cookies(cookies.group_cookies(fields)) - - -def format_response_cookies(fields): - return format_cookies((c[0], c[1][0], c[1][1]) for c in fields) - - -def name_value(obj): - """ - Convert (key, value) pairs to HAR format. - """ - return [{"name": k, "value": v} for k, v in obj.items()] diff -Nru mitmproxy-5.1.1/docs/src/examples/complex/mitmproxywrapper.py mitmproxy-6.0.2/docs/src/examples/complex/mitmproxywrapper.py --- mitmproxy-5.1.1/docs/src/examples/complex/mitmproxywrapper.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/complex/mitmproxywrapper.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,165 +0,0 @@ -#!/usr/bin/env python -# -# Helper tool to enable/disable OS X proxy and wrap mitmproxy -# -# Get usage information with: -# -# mitmproxywrapper.py -h -# - -import subprocess -import re -import argparse -import contextlib -import os -import sys - - -class Wrapper: - def __init__(self, port, extra_arguments=None): - self.port = port - self.extra_arguments = extra_arguments - - def run_networksetup_command(self, *arguments): - return subprocess.check_output( - ['sudo', 'networksetup'] + list(arguments)) - - def proxy_state_for_service(self, service): - state = self.run_networksetup_command( - '-getwebproxy', - service).splitlines() - return dict([re.findall(r'([^:]+): (.*)', line)[0] for line in state]) - - def enable_proxy_for_service(self, service): - print('Enabling proxy on {}...'.format(service)) - for subcommand in ['-setwebproxy', '-setsecurewebproxy']: - self.run_networksetup_command( - subcommand, service, '127.0.0.1', str( - self.port)) - - def disable_proxy_for_service(self, service): - print('Disabling proxy on {}...'.format(service)) - for subcommand in ['-setwebproxystate', '-setsecurewebproxystate']: - self.run_networksetup_command(subcommand, service, 'Off') - - def interface_name_to_service_name_map(self): - order = self.run_networksetup_command('-listnetworkserviceorder') - mapping = re.findall( - r'\(\d+\)\s(.*)$\n\(.*Device: (.+)\)$', - order, - re.MULTILINE) - return dict([(b, a) for (a, b) in mapping]) - - def run_command_with_input(self, command, input): - popen = subprocess.Popen( - command, - stdin=subprocess.PIPE, - stdout=subprocess.PIPE) - (stdout, stderr) = popen.communicate(input) - return stdout - - def primary_interace_name(self): - scutil_script = 'get State:/Network/Global/IPv4\nd.show\n' - stdout = self.run_command_with_input('/usr/sbin/scutil', scutil_script) - interface, = re.findall(r'PrimaryInterface\s*:\s*(.+)', stdout) - return interface - - def primary_service_name(self): - return self.interface_name_to_service_name_map()[ - self.primary_interace_name()] - - def proxy_enabled_for_service(self, service): - return self.proxy_state_for_service(service)['Enabled'] == 'Yes' - - def toggle_proxy(self): - new_state = not self.proxy_enabled_for_service( - self.primary_service_name()) - for service_name in self.connected_service_names(): - if self.proxy_enabled_for_service(service_name) and not new_state: - self.disable_proxy_for_service(service_name) - elif not self.proxy_enabled_for_service(service_name) and new_state: - self.enable_proxy_for_service(service_name) - - def connected_service_names(self): - scutil_script = 'list\n' - stdout = self.run_command_with_input('/usr/sbin/scutil', scutil_script) - service_ids = re.findall(r'State:/Network/Service/(.+)/IPv4', stdout) - - service_names = [] - for service_id in service_ids: - scutil_script = 'show Setup:/Network/Service/{}\n'.format( - service_id) - stdout = self.run_command_with_input( - '/usr/sbin/scutil', - scutil_script) - service_name, = re.findall(r'UserDefinedName\s*:\s*(.+)', stdout) - service_names.append(service_name) - - return service_names - - def wrap_mitmproxy(self): - with self.wrap_proxy(): - cmd = ['mitmproxy', '-p', str(self.port)] - if self.extra_arguments: - cmd.extend(self.extra_arguments) - subprocess.check_call(cmd) - - def wrap_honeyproxy(self): - with self.wrap_proxy(): - popen = subprocess.Popen('honeyproxy.sh') - try: - popen.wait() - except KeyboardInterrupt: - popen.terminate() - - @contextlib.contextmanager - def wrap_proxy(self): - connected_service_names = self.connected_service_names() - for service_name in connected_service_names: - if not self.proxy_enabled_for_service(service_name): - self.enable_proxy_for_service(service_name) - - yield - - for service_name in connected_service_names: - if self.proxy_enabled_for_service(service_name): - self.disable_proxy_for_service(service_name) - - @classmethod - def ensure_superuser(cls): - if os.getuid() != 0: - print('Relaunching with sudo...') - os.execv('/usr/bin/sudo', ['/usr/bin/sudo'] + sys.argv) - - @classmethod - def main(cls): - parser = argparse.ArgumentParser( - description='Helper tool for OS X proxy configuration and mitmproxy.', - epilog='Any additional arguments will be passed on unchanged to mitmproxy.') - parser.add_argument( - '-t', - '--toggle', - action='store_true', - help='just toggle the proxy configuration') - # parser.add_argument('--honeyproxy', action='store_true', help='run honeyproxy instead of mitmproxy') - parser.add_argument( - '-p', - '--port', - type=int, - help='override the default port of 8080', - default=8080) - args, extra_arguments = parser.parse_known_args() - - wrapper = cls(port=args.port, extra_arguments=extra_arguments) - - if args.toggle: - wrapper.toggle_proxy() - # elif args.honeyproxy: - # wrapper.wrap_honeyproxy() - else: - wrapper.wrap_mitmproxy() - - -if __name__ == '__main__': - Wrapper.ensure_superuser() - Wrapper.main() diff -Nru mitmproxy-5.1.1/docs/src/examples/complex/nonblocking.py mitmproxy-6.0.2/docs/src/examples/complex/nonblocking.py --- mitmproxy-5.1.1/docs/src/examples/complex/nonblocking.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/complex/nonblocking.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,11 +0,0 @@ -import time - -from mitmproxy.script import concurrent - - -@concurrent # Remove this and see what happens -def request(flow): - # This is ugly in mitmproxy's UI, but you don't want to use mitmproxy.ctx.log from a different thread. - print("handle request: %s%s" % (flow.request.host, flow.request.path)) - time.sleep(5) - print("start request: %s%s" % (flow.request.host, flow.request.path)) diff -Nru mitmproxy-5.1.1/docs/src/examples/complex/README.md mitmproxy-6.0.2/docs/src/examples/complex/README.md --- mitmproxy-5.1.1/docs/src/examples/complex/README.md 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/complex/README.md 1970-01-01 00:00:00.000000000 +0000 @@ -1,19 +0,0 @@ -## Complex Examples - -| Filename | Description | -|:-------------------------|:----------------------------------------------------------------------------------------------| -| block_dns_over_https.py | Use mitmproxy to block DNS over HTTPS (DoH) queries | -| change_upstream_proxy.py | Dynamically change the upstream proxy. | -| dns_spoofing.py | Use mitmproxy in a DNS spoofing scenario. | -| dup_and_replay.py | Duplicates each request, changes it, and then replays the modified request. | -| full_transparency_shim.c | Setuid wrapper that can be used to run mitmproxy in full transparency mode, as a normal user. | -| har_dump.py | Dump flows as HAR files. | -| mitmproxywrapper.py | Bracket mitmproxy run with proxy enable/disable on OS X | -| nonblocking.py | Demonstrate parallel processing with a blocking script | -| remote_debug.py | This script enables remote debugging of the mitmproxy _UI_ with PyCharm. | -| sslstrip.py | sslstrip-like functionality implemented with mitmproxy | -| stream.py | Enable streaming for all responses. | -| stream_modify.py | Modify a streamed response body. | -| tcp_message.py | Modify a raw TCP connection | -| tls_passthrough.py | Use conditional TLS interception based on a user-defined strategy. | -| xss_scanner.py | Scan all visited webpages. | diff -Nru mitmproxy-5.1.1/docs/src/examples/complex/remote_debug.py mitmproxy-6.0.2/docs/src/examples/complex/remote_debug.py --- mitmproxy-5.1.1/docs/src/examples/complex/remote_debug.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/complex/remote_debug.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,21 +0,0 @@ -""" -This script enables remote debugging of the mitmproxy *UI* with PyCharm. -For general debugging purposes, it is easier to just debug mitmdump within PyCharm. - -Usage: - - pip install pydevd on the mitmproxy machine - - Open the Run/Debug Configuration dialog box in PyCharm, and select the - Python Remote Debug configuration type. - - Debugging works in the way that mitmproxy connects to the debug server - on startup. Specify host and port that mitmproxy can use to reach your - PyCharm instance on startup. - - Adjust this inline script accordingly. - - Start debug server in PyCharm - - Set breakpoints - - Start mitmproxy -s remote_debug.py -""" - - -def load(l): - import pydevd_pycharm - pydevd_pycharm.settrace("localhost", port=5678, stdoutToServer=True, stderrToServer=True, suspend=False) diff -Nru mitmproxy-5.1.1/docs/src/examples/complex/sslstrip.py mitmproxy-6.0.2/docs/src/examples/complex/sslstrip.py --- mitmproxy-5.1.1/docs/src/examples/complex/sslstrip.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/complex/sslstrip.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,63 +0,0 @@ -""" -This script implements an sslstrip-like attack based on mitmproxy. -https://moxie.org/software/sslstrip/ -""" -import re -import urllib.parse -import typing # noqa - -from mitmproxy import http - -# set of SSL/TLS capable hosts -secure_hosts: typing.Set[str] = set() - - -def request(flow: http.HTTPFlow) -> None: - flow.request.headers.pop('If-Modified-Since', None) - flow.request.headers.pop('Cache-Control', None) - - # do not force https redirection - flow.request.headers.pop('Upgrade-Insecure-Requests', None) - - # proxy connections to SSL-enabled hosts - if flow.request.pretty_host in secure_hosts: - flow.request.scheme = 'https' - flow.request.port = 443 - - # We need to update the request destination to whatever is specified in the host header: - # Having no TLS Server Name Indication from the client and just an IP address as request.host - # in transparent mode, TLS server name certificate validation would fail. - flow.request.host = flow.request.pretty_host - - -def response(flow: http.HTTPFlow) -> None: - assert flow.response - flow.response.headers.pop('Strict-Transport-Security', None) - flow.response.headers.pop('Public-Key-Pins', None) - - # strip links in response body - flow.response.content = flow.response.content.replace(b'https://', b'http://') - - # strip meta tag upgrade-insecure-requests in response body - csp_meta_tag_pattern = br'' - flow.response.content = re.sub(csp_meta_tag_pattern, b'', flow.response.content, flags=re.IGNORECASE) - - # strip links in 'Location' header - if flow.response.headers.get('Location', '').startswith('https://'): - location = flow.response.headers['Location'] - hostname = urllib.parse.urlparse(location).hostname - if hostname: - secure_hosts.add(hostname) - flow.response.headers['Location'] = location.replace('https://', 'http://', 1) - - # strip upgrade-insecure-requests in Content-Security-Policy header - csp_header = flow.response.headers.get('Content-Security-Policy', '') - if re.search('upgrade-insecure-requests', csp_header, flags=re.IGNORECASE): - csp = flow.response.headers['Content-Security-Policy'] - new_header = re.sub(r'upgrade-insecure-requests[;\s]*', '', csp, flags=re.IGNORECASE) - flow.response.headers['Content-Security-Policy'] = new_header - - # strip secure flag from 'Set-Cookie' headers - cookies = flow.response.headers.get_all('Set-Cookie') - cookies = [re.sub(r';\s*secure\s*', '', s) for s in cookies] - flow.response.headers.set_all('Set-Cookie', cookies) diff -Nru mitmproxy-5.1.1/docs/src/examples/complex/stream_modify.py mitmproxy-6.0.2/docs/src/examples/complex/stream_modify.py --- mitmproxy-5.1.1/docs/src/examples/complex/stream_modify.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/complex/stream_modify.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,20 +0,0 @@ -""" -This inline script modifies a streamed response. -If you do not need streaming, see the modify_response_body example. -Be aware that content replacement isn't trivial: - - If the transfer encoding isn't chunked, you cannot simply change the content length. - - If you want to replace all occurrences of "foobar", make sure to catch the cases - where one chunk ends with [...]foo" and the next starts with "bar[...]. -""" - - -def modify(chunks): - """ - chunks is a generator that can be used to iterate over all chunks. - """ - for chunk in chunks: - yield chunk.replace("foo", "bar") - - -def responseheaders(flow): - flow.response.stream = modify diff -Nru mitmproxy-5.1.1/docs/src/examples/complex/stream.py mitmproxy-6.0.2/docs/src/examples/complex/stream.py --- mitmproxy-5.1.1/docs/src/examples/complex/stream.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/complex/stream.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,6 +0,0 @@ -def responseheaders(flow): - """ - Enables streaming for all responses. - This is equivalent to passing `--set stream_large_bodies=1` to mitmproxy. - """ - flow.response.stream = True diff -Nru mitmproxy-5.1.1/docs/src/examples/complex/tcp_message.py mitmproxy-6.0.2/docs/src/examples/complex/tcp_message.py --- mitmproxy-5.1.1/docs/src/examples/complex/tcp_message.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/complex/tcp_message.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,27 +0,0 @@ -""" -tcp_message Inline Script Hook API Demonstration ------------------------------------------------- - -* modifies packets containing "foo" to "bar" -* prints various details for each packet. - -example cmdline invocation: -mitmdump --rawtcp --tcp-host ".*" -s examples/complex/tcp_message.py -""" -from mitmproxy.utils import strutils -from mitmproxy import ctx -from mitmproxy import tcp - - -def tcp_message(flow: tcp.TCPFlow): - message = flow.messages[-1] - old_content = message.content - message.content = old_content.replace(b"foo", b"bar") - - ctx.log.info( - "[tcp_message{}] from {} to {}:\n{}".format( - " (modified)" if message.content != old_content else "", - "client" if message.from_client else "server", - "server" if message.from_client else "client", - strutils.bytes_to_escaped_str(message.content)) - ) diff -Nru mitmproxy-5.1.1/docs/src/examples/complex/tls_passthrough.py mitmproxy-6.0.2/docs/src/examples/complex/tls_passthrough.py --- mitmproxy-5.1.1/docs/src/examples/complex/tls_passthrough.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/complex/tls_passthrough.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,146 +0,0 @@ -""" -This inline script allows conditional TLS Interception based -on a user-defined strategy. - -Example: - - > mitmdump -s tls_passthrough.py - - 1. curl --proxy http://localhost:8080 https://example.com --insecure - // works - we'll also see the contents in mitmproxy - - 2. curl --proxy http://localhost:8080 https://example.com --insecure - // still works - we'll also see the contents in mitmproxy - - 3. curl --proxy http://localhost:8080 https://example.com - // fails with a certificate error, which we will also see in mitmproxy - - 4. curl --proxy http://localhost:8080 https://example.com - // works again, but mitmproxy does not intercept and we do *not* see the contents - -Authors: Maximilian Hils, Matthew Tuusberg -""" -import collections -import random - -from enum import Enum - -import mitmproxy -from mitmproxy import ctx -from mitmproxy.exceptions import TlsProtocolException -from mitmproxy.proxy.protocol import TlsLayer, RawTCPLayer - - -class InterceptionResult(Enum): - success = True - failure = False - skipped = None - - -class _TlsStrategy: - """ - Abstract base class for interception strategies. - """ - - def __init__(self): - # A server_address -> interception results mapping - self.history = collections.defaultdict(lambda: collections.deque(maxlen=200)) - - def should_intercept(self, server_address): - """ - Returns: - True, if we should attempt to intercept the connection. - False, if we want to employ pass-through instead. - """ - raise NotImplementedError() - - def record_success(self, server_address): - self.history[server_address].append(InterceptionResult.success) - - def record_failure(self, server_address): - self.history[server_address].append(InterceptionResult.failure) - - def record_skipped(self, server_address): - self.history[server_address].append(InterceptionResult.skipped) - - -class ConservativeStrategy(_TlsStrategy): - """ - Conservative Interception Strategy - only intercept if there haven't been any failed attempts - in the history. - """ - - def should_intercept(self, server_address): - if InterceptionResult.failure in self.history[server_address]: - return False - return True - - -class ProbabilisticStrategy(_TlsStrategy): - """ - Fixed probability that we intercept a given connection. - """ - - def __init__(self, p): - self.p = p - super(ProbabilisticStrategy, self).__init__() - - def should_intercept(self, server_address): - return random.uniform(0, 1) < self.p - - -class TlsFeedback(TlsLayer): - """ - Monkey-patch _establish_tls_with_client to get feedback if TLS could be established - successfully on the client connection (which may fail due to cert pinning). - """ - - def _establish_tls_with_client(self): - server_address = self.server_conn.address - - try: - super(TlsFeedback, self)._establish_tls_with_client() - except TlsProtocolException as e: - tls_strategy.record_failure(server_address) - raise e - else: - tls_strategy.record_success(server_address) - - -# inline script hooks below. - -tls_strategy = None - - -def load(l): - l.add_option( - "tlsstrat", int, 0, "TLS passthrough strategy (0-100)", - ) - - -def configure(updated): - global tls_strategy - if ctx.options.tlsstrat > 0: - tls_strategy = ProbabilisticStrategy(float(ctx.options.tlsstrat) / 100.0) - else: - tls_strategy = ConservativeStrategy() - - -def next_layer(next_layer): - """ - This hook does the actual magic - if the next layer is planned to be a TLS layer, - we check if we want to enter pass-through mode instead. - """ - if isinstance(next_layer, TlsLayer) and next_layer._client_tls: - server_address = next_layer.server_conn.address - - if tls_strategy.should_intercept(server_address): - # We try to intercept. - # Monkey-Patch the layer to get feedback from the TLSLayer if interception worked. - next_layer.__class__ = TlsFeedback - else: - # We don't intercept - reply with a pass-through layer and add a "skipped" entry. - mitmproxy.ctx.log("TLS passthrough for %s" % repr(next_layer.server_conn.address), "info") - next_layer_replacement = RawTCPLayer(next_layer.ctx, ignore=True) - next_layer.reply.send(next_layer_replacement) - tls_strategy.record_skipped(server_address) diff -Nru mitmproxy-5.1.1/docs/src/examples/complex/websocket_inject_message.py mitmproxy-6.0.2/docs/src/examples/complex/websocket_inject_message.py --- mitmproxy-5.1.1/docs/src/examples/complex/websocket_inject_message.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/complex/websocket_inject_message.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,23 +0,0 @@ -""" -This example shows how to inject a WebSocket message to the client. -Every new WebSocket connection will trigger a new asyncio task that -periodically injects a new message to the client. -""" -import asyncio -import mitmproxy.websocket - - -class InjectWebSocketMessage: - - async def inject(self, flow: mitmproxy.websocket.WebSocketFlow): - i = 0 - while not flow.ended and not flow.error: - await asyncio.sleep(5) - flow.inject_message(flow.client_conn, 'This is the #{} injected message!'.format(i)) - i += 1 - - def websocket_start(self, flow): - asyncio.get_event_loop().create_task(self.inject(flow)) - - -addons = [InjectWebSocketMessage()] diff -Nru mitmproxy-5.1.1/docs/src/examples/complex/xss_scanner.py mitmproxy-6.0.2/docs/src/examples/complex/xss_scanner.py --- mitmproxy-5.1.1/docs/src/examples/complex/xss_scanner.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/complex/xss_scanner.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,419 +0,0 @@ -r""" - - __ __ _____ _____ _____ - \ \ / // ____/ ____| / ____| - \ V /| (___| (___ | (___ ___ __ _ _ __ _ __ ___ _ __ - > < \___ \\___ \ \___ \ / __/ _` | '_ \| '_ \ / _ \ '__| - / . \ ____) |___) | ____) | (_| (_| | | | | | | | __/ | - /_/ \_\_____/_____/ |_____/ \___\__,_|_| |_|_| |_|\___|_| - - -This script automatically scans all visited webpages for XSS and SQLi vulnerabilities. - -Usage: mitmproxy -s xss_scanner.py - -This script scans for vulnerabilities by injecting a fuzzing payload (see PAYLOAD below) into 4 different places -and examining the HTML to look for XSS and SQLi injection vulnerabilities. The XSS scanning functionality works by -looking to see whether it is possible to inject HTML based off of of where the payload appears in the page and what -characters are escaped. In addition, it also looks for any script tags that load javascript from unclaimed domains. -The SQLi scanning functionality works by using regular expressions to look for errors from a number of different -common databases. Since it is only looking for errors, it will not find blind SQLi vulnerabilities. - -The 4 places it injects the payload into are: -1. URLs (e.g. https://example.com/ -> https://example.com/PAYLOAD/) -2. Queries (e.g. https://example.com/index.html?a=b -> https://example.com/index.html?a=PAYLOAD) -3. Referers (e.g. The referer changes from https://example.com to PAYLOAD) -4. User Agents (e.g. The UA changes from Chrome to PAYLOAD) - -Reports from this script show up in the event log (viewable by pressing e) and formatted like: - -===== XSS Found ==== -XSS URL: http://daviddworken.com/vulnerableUA.php -Injection Point: User Agent -Suggested Exploit: -Line: 1029zxcs'd"aoso[sb]po(pc)se;sl/bsl\eq=3847asd - -""" - -from html.parser import HTMLParser -from typing import Dict, Union, Tuple, Optional, List, NamedTuple -from urllib.parse import urlparse -import re -import socket - -import requests - -from mitmproxy import http -from mitmproxy import ctx - - -# The actual payload is put between a frontWall and a backWall to make it easy -# to locate the payload with regular expressions -FRONT_WALL = b"1029zxc" -BACK_WALL = b"3847asd" -PAYLOAD = b"""s'd"aoso[sb]po(pc)se;sl/bsl\\eq=""" -FULL_PAYLOAD = FRONT_WALL + PAYLOAD + BACK_WALL - -# A XSSData is a named tuple with the following fields: -# - url -> str -# - injection_point -> str -# - exploit -> str -# - line -> str -XSSData = NamedTuple('XSSData', [('url', str), - ('injection_point', str), - ('exploit', str), - ('line', str)]) - -# A SQLiData is named tuple with the following fields: -# - url -> str -# - injection_point -> str -# - regex -> str -# - dbms -> str -SQLiData = NamedTuple('SQLiData', [('url', str), - ('injection_point', str), - ('regex', str), - ('dbms', str)]) - - -VulnData = Tuple[Optional[XSSData], Optional[SQLiData]] -Cookies = Dict[str, str] - - -def get_cookies(flow: http.HTTPFlow) -> Cookies: - """ Return a dict going from cookie names to cookie values - - Note that it includes both the cookies sent in the original request and - the cookies sent by the server """ - return {name: value for name, value in flow.request.cookies.fields} - - -def find_unclaimed_URLs(body, requestUrl): - """ Look for unclaimed URLs in script tags and log them if found""" - def getValue(attrs: List[Tuple[str, str]], attrName: str) -> Optional[str]: - for name, value in attrs: - if attrName == name: - return value - return None - - class ScriptURLExtractor(HTMLParser): - script_URLs: List[str] = [] - - def handle_starttag(self, tag, attrs): - if (tag == "script" or tag == "iframe") and "src" in [name for name, value in attrs]: - self.script_URLs.append(getValue(attrs, "src")) - if tag == "link" and getValue(attrs, "rel") == "stylesheet" and "href" in [name for name, value in attrs]: - self.script_URLs.append(getValue(attrs, "href")) - - parser = ScriptURLExtractor() - parser.feed(body) - for url in parser.script_URLs: - url_parser = urlparse(url) - domain = url_parser.netloc - try: - socket.gethostbyname(domain) - except socket.gaierror: - ctx.log.error(f"XSS found in {requestUrl} due to unclaimed URL \"{url}\".") - - -def test_end_of_URL_injection(original_body: str, request_URL: str, cookies: Cookies) -> VulnData: - """ Test the given URL for XSS via injection onto the end of the URL and - log the XSS if found """ - parsed_URL = urlparse(request_URL) - path = parsed_URL.path - if path != "" and path[-1] != "/": # ensure the path ends in a / - path += "/" - path += FULL_PAYLOAD.decode('utf-8') # the path must be a string while the payload is bytes - url = parsed_URL._replace(path=path).geturl() - body = requests.get(url, cookies=cookies).text.lower() - xss_info = get_XSS_data(body, url, "End of URL") - sqli_info = get_SQLi_data(body, original_body, url, "End of URL") - return xss_info, sqli_info - - -def test_referer_injection(original_body: str, request_URL: str, cookies: Cookies) -> VulnData: - """ Test the given URL for XSS via injection into the referer and - log the XSS if found """ - body = requests.get(request_URL, headers={'referer': FULL_PAYLOAD}, cookies=cookies).text.lower() - xss_info = get_XSS_data(body, request_URL, "Referer") - sqli_info = get_SQLi_data(body, original_body, request_URL, "Referer") - return xss_info, sqli_info - - -def test_user_agent_injection(original_body: str, request_URL: str, cookies: Cookies) -> VulnData: - """ Test the given URL for XSS via injection into the user agent and - log the XSS if found """ - body = requests.get(request_URL, headers={'User-Agent': FULL_PAYLOAD}, cookies=cookies).text.lower() - xss_info = get_XSS_data(body, request_URL, "User Agent") - sqli_info = get_SQLi_data(body, original_body, request_URL, "User Agent") - return xss_info, sqli_info - - -def test_query_injection(original_body: str, request_URL: str, cookies: Cookies): - """ Test the given URL for XSS via injection into URL queries and - log the XSS if found """ - parsed_URL = urlparse(request_URL) - query_string = parsed_URL.query - # queries is a list of parameters where each parameter is set to the payload - queries = [query.split("=")[0] + "=" + FULL_PAYLOAD.decode('utf-8') for query in query_string.split("&")] - new_query_string = "&".join(queries) - new_URL = parsed_URL._replace(query=new_query_string).geturl() - body = requests.get(new_URL, cookies=cookies).text.lower() - xss_info = get_XSS_data(body, new_URL, "Query") - sqli_info = get_SQLi_data(body, original_body, new_URL, "Query") - return xss_info, sqli_info - - -def log_XSS_data(xss_info: Optional[XSSData]) -> None: - """ Log information about the given XSS to mitmproxy """ - # If it is None, then there is no info to log - if not xss_info: - return - ctx.log.error("===== XSS Found ====") - ctx.log.error("XSS URL: %s" % xss_info.url) - ctx.log.error("Injection Point: %s" % xss_info.injection_point) - ctx.log.error("Suggested Exploit: %s" % xss_info.exploit) - ctx.log.error("Line: %s" % xss_info.line) - - -def log_SQLi_data(sqli_info: Optional[SQLiData]) -> None: - """ Log information about the given SQLi to mitmproxy """ - if not sqli_info: - return - ctx.log.error("===== SQLi Found =====") - ctx.log.error("SQLi URL: %s" % sqli_info.url) - ctx.log.error("Injection Point: %s" % sqli_info.injection_point) - ctx.log.error("Regex used: %s" % sqli_info.regex) - ctx.log.error("Suspected DBMS: %s" % sqli_info.dbms) - return - - -def get_SQLi_data(new_body: str, original_body: str, request_URL: str, injection_point: str) -> Optional[SQLiData]: - """ Return a SQLiDict if there is a SQLi otherwise return None - String String URL String -> (SQLiDict or None) """ - # Regexes taken from Damn Small SQLi Scanner: https://github.com/stamparm/DSSS/blob/master/dsss.py#L17 - DBMS_ERRORS = { - "MySQL": (r"SQL syntax.*MySQL", r"Warning.*mysql_.*", r"valid MySQL result", r"MySqlClient\."), - "PostgreSQL": (r"PostgreSQL.*ERROR", r"Warning.*\Wpg_.*", r"valid PostgreSQL result", r"Npgsql\."), - "Microsoft SQL Server": (r"Driver.* SQL[\-\_\ ]*Server", r"OLE DB.* SQL Server", r"(\W|\A)SQL Server.*Driver", - r"Warning.*mssql_.*", r"(\W|\A)SQL Server.*[0-9a-fA-F]{8}", - r"(?s)Exception.*\WSystem\.Data\.SqlClient\.", r"(?s)Exception.*\WRoadhouse\.Cms\."), - "Microsoft Access": (r"Microsoft Access Driver", r"JET Database Engine", r"Access Database Engine"), - "Oracle": (r"\bORA-[0-9][0-9][0-9][0-9]", r"Oracle error", r"Oracle.*Driver", r"Warning.*\Woci_.*", r"Warning.*\Wora_.*"), - "IBM DB2": (r"CLI Driver.*DB2", r"DB2 SQL error", r"\bdb2_\w+\("), - "SQLite": (r"SQLite/JDBCDriver", r"SQLite.Exception", r"System.Data.SQLite.SQLiteException", r"Warning.*sqlite_.*", - r"Warning.*SQLite3::", r"\[SQLITE_ERROR\]"), - "Sybase": (r"(?i)Warning.*sybase.*", r"Sybase message", r"Sybase.*Server message.*"), - } - for dbms, regexes in DBMS_ERRORS.items(): - for regex in regexes: # type: ignore - if re.search(regex, new_body, re.IGNORECASE) and not re.search(regex, original_body, re.IGNORECASE): - return SQLiData(request_URL, - injection_point, - regex, - dbms) - return None - - -# A qc is either ' or " -def inside_quote(qc: str, substring_bytes: bytes, text_index: int, body_bytes: bytes) -> bool: - """ Whether the Numberth occurrence of the first string in the second - string is inside quotes as defined by the supplied QuoteChar """ - substring = substring_bytes.decode('utf-8') - body = body_bytes.decode('utf-8') - num_substrings_found = 0 - in_quote = False - for index, char in enumerate(body): - # Whether the next chunk of len(substring) chars is the substring - next_part_is_substring = ( - (not (index + len(substring) > len(body))) and - (body[index:index + len(substring)] == substring) - ) - # Whether this char is escaped with a \ - is_not_escaped = ( - (index - 1 < 0 or index - 1 > len(body)) or - (body[index - 1] != "\\") - ) - if char == qc and is_not_escaped: - in_quote = not in_quote - if next_part_is_substring: - if num_substrings_found == text_index: - return in_quote - num_substrings_found += 1 - return False - - -def paths_to_text(html: str, string: str) -> List[str]: - """ Return list of Paths to a given str in the given HTML tree - - Note that it does a BFS """ - - def remove_last_occurence_of_sub_string(string: str, substr: str) -> str: - """ Delete the last occurrence of substr from str - String String -> String - """ - index = string.rfind(substr) - return string[:index] + string[index + len(substr):] - - class PathHTMLParser(HTMLParser): - currentPath = "" - paths: List[str] = [] - - def handle_starttag(self, tag, attrs): - self.currentPath += ("/" + tag) - - def handle_endtag(self, tag): - self.currentPath = remove_last_occurence_of_sub_string(self.currentPath, "/" + tag) - - def handle_data(self, data): - if string in data: - self.paths.append(self.currentPath) - - parser = PathHTMLParser() - parser.feed(html) - return parser.paths - - -def get_XSS_data(body: Union[str, bytes], request_URL: str, injection_point: str) -> Optional[XSSData]: - """ Return a XSSDict if there is a XSS otherwise return None """ - def in_script(text, index, body) -> bool: - """ Whether the Numberth occurrence of the first string in the second - string is inside a script tag """ - paths = paths_to_text(body.decode('utf-8'), text.decode("utf-8")) - try: - path = paths[index] - return "script" in path - except IndexError: - return False - - def in_HTML(text: bytes, index: int, body: bytes) -> bool: - """ Whether the Numberth occurrence of the first string in the second - string is inside the HTML but not inside a script tag or part of - a HTML attribute""" - # if there is a < then lxml will interpret that as a tag, so only search for the stuff before it - text = text.split(b"<")[0] - paths = paths_to_text(body.decode('utf-8'), text.decode("utf-8")) - try: - path = paths[index] - return "script" not in path - except IndexError: - return False - - def inject_javascript_handler(html: str) -> bool: - """ Whether you can inject a Javascript:alert(0) as a link """ - class injectJSHandlerHTMLParser(HTMLParser): - injectJSHandler = False - - def handle_starttag(self, tag, attrs): - for name, value in attrs: - if name == "href" and value.startswith(FRONT_WALL.decode('utf-8')): - self.injectJSHandler = True - - parser = injectJSHandlerHTMLParser() - parser.feed(html) - return parser.injectJSHandler - # Only convert the body to bytes if needed - if isinstance(body, str): - body = bytes(body, 'utf-8') - # Regex for between 24 and 72 (aka 24*3) characters encapsulated by the walls - regex = re.compile(b"""%s.{24,72}?%s""" % (FRONT_WALL, BACK_WALL)) - matches = regex.findall(body) - for index, match in enumerate(matches): - # Where the string is injected into the HTML - in_script_val = in_script(match, index, body) - in_HTML_val = in_HTML(match, index, body) - in_tag = not in_script_val and not in_HTML_val - in_single_quotes = inside_quote("'", match, index, body) - in_double_quotes = inside_quote('"', match, index, body) - # Whether you can inject: - inject_open_angle = b"aoso" in match # close angle brackets - inject_single_quotes = b"s'd" in match # single quotes - inject_double_quotes = b'd"ao' in match # double quotes - inject_slash = b"sl/bsl" in match # forward slashes - inject_semi = b"se;sl" in match # semicolons - inject_equals = b"eq=" in match # equals sign - if in_script_val and inject_slash and inject_open_angle and inject_close_angle: # e.g. - return XSSData(request_URL, - injection_point, - ' - return XSSData(request_URL, - injection_point, - "';alert(0);g='", - match.decode('utf-8')) - elif in_script_val and in_double_quotes and inject_double_quotes and inject_semi: # e.g. - return XSSData(request_URL, - injection_point, - '";alert(0);g="', - match.decode('utf-8')) - elif in_tag and in_single_quotes and inject_single_quotes and inject_open_angle and inject_close_angle and inject_slash: - # e.g. Test - return XSSData(request_URL, - injection_point, - "'>", - match.decode('utf-8')) - elif in_tag and in_double_quotes and inject_double_quotes and inject_open_angle and inject_close_angle and inject_slash: - # e.g. Test - return XSSData(request_URL, - injection_point, - '">', - match.decode('utf-8')) - elif in_tag and not in_double_quotes and not in_single_quotes and inject_open_angle and inject_close_angle and inject_slash: - # e.g. Test - return XSSData(request_URL, - injection_point, - '>', - match.decode('utf-8')) - elif inject_javascript_handler(body.decode('utf-8')): # e.g. Test - return XSSData(request_URL, - injection_point, - 'Javascript:alert(0)', - match.decode('utf-8')) - elif in_tag and in_double_quotes and inject_double_quotes and inject_equals: # e.g. Test - return XSSData(request_URL, - injection_point, - '" onmouseover="alert(0)" t="', - match.decode('utf-8')) - elif in_tag and in_single_quotes and inject_single_quotes and inject_equals: # e.g. Test - return XSSData(request_URL, - injection_point, - "' onmouseover='alert(0)' t='", - match.decode('utf-8')) - elif in_tag and not in_single_quotes and not in_double_quotes and inject_equals: # e.g. Test - return XSSData(request_URL, - injection_point, - " onmouseover=alert(0) t=", - match.decode('utf-8')) - elif in_HTML_val and not in_script_val and inject_open_angle and inject_close_angle and inject_slash: # e.g. PAYLOAD - return XSSData(request_URL, - injection_point, - '', - match.decode('utf-8')) - else: - return None - return None - - -# response is mitmproxy's entry point -def response(flow: http.HTTPFlow) -> None: - assert flow.response - cookies_dict = get_cookies(flow) - resp = flow.response.get_text(strict=False) - assert resp - # Example: http://xss.guru/unclaimedScriptTag.html - find_unclaimed_URLs(resp, flow.request.url) - results = test_end_of_URL_injection(resp, flow.request.url, cookies_dict) - log_XSS_data(results[0]) - log_SQLi_data(results[1]) - # Example: https://daviddworken.com/vulnerableReferer.php - results = test_referer_injection(resp, flow.request.url, cookies_dict) - log_XSS_data(results[0]) - log_SQLi_data(results[1]) - # Example: https://daviddworken.com/vulnerableUA.php - results = test_user_agent_injection(resp, flow.request.url, cookies_dict) - log_XSS_data(results[0]) - log_SQLi_data(results[1]) - if "?" in flow.request.url: - # Example: https://daviddworken.com/vulnerable.php?name= - results = test_query_injection(resp, flow.request.url, cookies_dict) - log_XSS_data(results[0]) - log_SQLi_data(results[1]) diff -Nru mitmproxy-5.1.1/docs/src/examples/contrib/block_dns_over_https.py mitmproxy-6.0.2/docs/src/examples/contrib/block_dns_over_https.py --- mitmproxy-5.1.1/docs/src/examples/contrib/block_dns_over_https.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/contrib/block_dns_over_https.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,186 @@ +""" +This module is for blocking DNS over HTTPS requests. + +It loads a blocklist of IPs and hostnames that are known to serve DNS over HTTPS requests. +It also uses headers, query params, and paths to detect DoH (and block it) +""" +from typing import List + +from mitmproxy import ctx + +# known DoH providers' hostnames and IP addresses to block +default_blocklist: dict = { + "hostnames": [ + "dns.adguard.com", "dns-family.adguard.com", "dns.google", "cloudflare-dns.com", + "mozilla.cloudflare-dns.com", "security.cloudflare-dns.com", "family.cloudflare-dns.com", + "dns.quad9.net", "dns9.quad9.net", "dns10.quad9.net", "dns11.quad9.net", "doh.opendns.com", + "doh.familyshield.opendns.com", "doh.cleanbrowsing.org", "doh.xfinity.com", "dohdot.coxlab.net", + "odvr.nic.cz", "doh.dnslify.com", "dns.nextdns.io", "dns.dnsoverhttps.net", "doh.crypto.sx", + "doh.powerdns.org", "doh-fi.blahdns.com", "doh-jp.blahdns.com", "doh-de.blahdns.com", + "doh.ffmuc.net", "dns.dns-over-https.com", "doh.securedns.eu", "dns.rubyfish.cn", + "dns.containerpi.com", "dns.containerpi.com", "dns.containerpi.com", "doh-2.seby.io", + "doh.seby.io", "commons.host", "doh.dnswarden.com", "doh.dnswarden.com", "doh.dnswarden.com", + "dns-nyc.aaflalo.me", "dns.aaflalo.me", "doh.applied-privacy.net", "doh.captnemo.in", + "doh.tiar.app", "doh.tiarap.org", "doh.dns.sb", "rdns.faelix.net", "doh.li", "doh.armadillodns.net", + "jp.tiar.app", "jp.tiarap.org", "doh.42l.fr", "dns.hostux.net", "dns.hostux.net", "dns.aa.net.uk", + "adblock.mydns.network", "ibksturm.synology.me", "jcdns.fun", "ibuki.cgnat.net", "dns.twnic.tw", + "example.doh.blockerdns.com", "dns.digitale-gesellschaft.ch", "doh.libredns.gr", + "doh.centraleu.pi-dns.com", "doh.northeu.pi-dns.com", "doh.westus.pi-dns.com", + "doh.eastus.pi-dns.com", "dns.flatuslifir.is", "private.canadianshield.cira.ca", + "protected.canadianshield.cira.ca", "family.canadianshield.cira.ca", "dns.google.com", + "dns.google.com" + ], + "ips": [ + "104.16.248.249", "104.16.248.249", "104.16.249.249", "104.16.249.249", "104.18.2.55", + "104.18.26.128", "104.18.27.128", "104.18.3.55", "104.18.44.204", "104.18.44.204", + "104.18.45.204", "104.18.45.204", "104.182.57.196", "104.236.178.232", "104.24.122.53", + "104.24.123.53", "104.28.0.106", "104.28.1.106", "104.31.90.138", "104.31.91.138", + "115.159.131.230", "116.202.176.26", "116.203.115.192", "136.144.215.158", "139.59.48.222", + "139.99.222.72", "146.112.41.2", "146.112.41.3", "146.185.167.43", "149.112.112.10", + "149.112.112.11", "149.112.112.112", "149.112.112.9", "149.112.121.10", "149.112.121.20", + "149.112.121.30", "149.112.122.10", "149.112.122.20", "149.112.122.30", "159.69.198.101", + "168.235.81.167", "172.104.93.80", "172.65.3.223", "174.138.29.175", "174.68.248.77", + "176.103.130.130", "176.103.130.131", "176.103.130.132", "176.103.130.134", "176.56.236.175", + "178.62.214.105", "185.134.196.54", "185.134.197.54", "185.213.26.187", "185.216.27.142", + "185.228.168.10", "185.228.168.168", "185.235.81.1", "185.26.126.37", "185.26.126.37", + "185.43.135.1", "185.95.218.42", "185.95.218.43", "195.30.94.28", "2001:148f:fffe::1", + "2001:19f0:7001:3259:5400:2ff:fe71:bc9", "2001:19f0:7001:5554:5400:2ff:fe57:3077", + "2001:19f0:7001:5554:5400:2ff:fe57:3077", "2001:19f0:7001:5554:5400:2ff:fe57:3077", + "2001:4860:4860::8844", "2001:4860:4860::8888", + "2001:4b98:dc2:43:216:3eff:fe86:1d28", "2001:558:fe21:6b:96:113:151:149", + "2001:608:a01::3", "2001:678:888:69:c45d:2738:c3f2:1878", "2001:8b0::2022", "2001:8b0::2023", + "2001:c50:ffff:1:101:101:101:101", "210.17.9.228", "217.169.20.22", "217.169.20.23", + "2400:6180:0:d0::5f73:4001", "2400:8902::f03c:91ff:feda:c514", "2604:180:f3::42", + "2604:a880:1:20::51:f001", "2606:4700::6810:f8f9", "2606:4700::6810:f9f9", "2606:4700::6812:1a80", + "2606:4700::6812:1b80", "2606:4700::6812:237", "2606:4700::6812:337", "2606:4700:3033::6812:2ccc", + "2606:4700:3033::6812:2dcc", "2606:4700:3033::6818:7b35", "2606:4700:3034::681c:16a", + "2606:4700:3035::6818:7a35", "2606:4700:3035::681f:5a8a", "2606:4700:3036::681c:6a", + "2606:4700:3036::681f:5b8a", "2606:4700:60:0:a71e:6467:cef8:2a56", "2620:10a:80bb::10", + "2620:10a:80bb::20", "2620:10a:80bb::30" "2620:10a:80bc::10", "2620:10a:80bc::20", + "2620:10a:80bc::30", "2620:119:fc::2", "2620:119:fc::3", "2620:fe::10", "2620:fe::11", + "2620:fe::9", "2620:fe::fe:10", "2620:fe::fe:11", "2620:fe::fe:9", "2620:fe::fe", + "2a00:5a60::ad1:ff", "2a00:5a60::ad2:ff", "2a00:5a60::bad1:ff", "2a00:5a60::bad2:ff", + "2a00:d880:5:bf0::7c93", "2a01:4f8:1c0c:8233::1", "2a01:4f8:1c1c:6b4b::1", "2a01:4f8:c2c:52bf::1", + "2a01:4f9:c010:43ce::1", "2a01:4f9:c01f:4::abcd", "2a01:7c8:d002:1ef:5054:ff:fe40:3703", + "2a01:9e00::54", "2a01:9e00::55", "2a01:9e01::54", "2a01:9e01::55", + "2a02:1205:34d5:5070:b26e:bfff:fe1d:e19b", "2a03:4000:38:53c::2", + "2a03:b0c0:0:1010::e9a:3001", "2a04:bdc7:100:70::abcd", "2a05:fc84::42", "2a05:fc84::43", + "2a07:a8c0::", "2a0d:4d00:81::1", "2a0d:5600:33:3::abcd", "35.198.2.76", "35.231.247.227", + "45.32.55.94", "45.67.219.208", "45.76.113.31", "45.77.180.10", "45.90.28.0", + "46.101.66.244", "46.227.200.54", "46.227.200.55", "46.239.223.80", "8.8.4.4", + "8.8.8.8", "83.77.85.7", "88.198.91.187", "9.9.9.10", "9.9.9.11", "9.9.9.9", + "94.130.106.88", "95.216.181.228", "95.216.212.177", "96.113.151.148", + ] +} + +# additional hostnames to block +additional_doh_names: List[str] = [ + 'dns.google.com' +] + +# additional IPs to block +additional_doh_ips: List[str] = [ + +] + +doh_hostnames, doh_ips = default_blocklist['hostnames'], default_blocklist['ips'] + +# convert to sets for faster lookups +doh_hostnames = set(doh_hostnames) +doh_ips = set(doh_ips) + + +def _has_dns_message_content_type(flow): + """ + Check if HTTP request has a DNS-looking 'Content-Type' header + + :param flow: mitmproxy flow + :return: True if 'Content-Type' header is DNS-looking, False otherwise + """ + doh_content_types = ['application/dns-message'] + if 'Content-Type' in flow.request.headers: + if flow.request.headers['Content-Type'] in doh_content_types: + return True + return False + + +def _request_has_dns_query_string(flow): + """ + Check if the query string of a request contains the parameter 'dns' + + :param flow: mitmproxy flow + :return: True is 'dns' is a parameter in the query string, False otherwise + """ + return 'dns' in flow.request.query + + +def _request_is_dns_json(flow): + """ + Check if the request looks like DoH with JSON. + + The only known implementations of DoH with JSON are Cloudflare and Google. + + For more info, see: + - https://developers.cloudflare.com/1.1.1.1/dns-over-https/json-format/ + - https://developers.google.com/speed/public-dns/docs/doh/json + + :param flow: mitmproxy flow + :return: True is request looks like DNS JSON, False otherwise + """ + # Header 'Accept: application/dns-json' is required in Cloudflare's DoH JSON API + # or they return a 400 HTTP response code + if 'Accept' in flow.request.headers: + if flow.request.headers['Accept'] == 'application/dns-json': + return True + # Google's DoH JSON API is https://dns.google/resolve + path = flow.request.path.split('?')[0] + if flow.request.host == 'dns.google' and path == '/resolve': + return True + return False + + +def _request_has_doh_looking_path(flow): + """ + Check if the path looks like it's DoH. + Most common one is '/dns-query', likely because that's what's in the RFC + + :param flow: mitmproxy flow + :return: True if path looks like it's DoH, otherwise False + """ + doh_paths = [ + '/dns-query', # used in example in RFC 8484 (see https://tools.ietf.org/html/rfc8484#section-4.1.1) + ] + path = flow.request.path.split('?')[0] + return path in doh_paths + + +def _requested_hostname_is_in_doh_blocklist(flow): + """ + Check if server hostname is in our DoH provider blocklist. + + The current blocklist is taken from https://github.com/curl/curl/wiki/DNS-over-HTTPS. + + :param flow: mitmproxy flow + :return: True if server's hostname is in DoH blocklist, otherwise False + """ + hostname = flow.request.host + ip = flow.server_conn.address + return hostname in doh_hostnames or hostname in doh_ips or ip in doh_ips + + +doh_request_detection_checks = [ + _has_dns_message_content_type, + _request_has_dns_query_string, + _request_is_dns_json, + _requested_hostname_is_in_doh_blocklist, + _request_has_doh_looking_path +] + + +def request(flow): + for check in doh_request_detection_checks: + is_doh = check(flow) + if is_doh: + ctx.log.warn("[DoH Detection] DNS over HTTPS request detected via method \"%s\"" % check.__name__) + flow.kill() + break diff -Nru mitmproxy-5.1.1/docs/src/examples/contrib/change_upstream_proxy.py mitmproxy-6.0.2/docs/src/examples/contrib/change_upstream_proxy.py --- mitmproxy-5.1.1/docs/src/examples/contrib/change_upstream_proxy.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/contrib/change_upstream_proxy.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,27 @@ +from mitmproxy import http +import typing + +# This scripts demonstrates how mitmproxy can switch to a second/different upstream proxy +# in upstream proxy mode. +# +# Usage: mitmdump -U http://default-upstream-proxy.local:8080/ -s change_upstream_proxy.py +# +# If you want to change the target server, you should modify flow.request.host and flow.request.port + + +def proxy_address(flow: http.HTTPFlow) -> typing.Tuple[str, int]: + # Poor man's loadbalancing: route every second domain through the alternative proxy. + if hash(flow.request.host) % 2 == 1: + return ("localhost", 8082) + else: + return ("localhost", 8081) + + +def request(flow: http.HTTPFlow) -> None: + if flow.request.method == "CONNECT": + # If the decision is done by domain, one could also modify the server address here. + # We do it after CONNECT here to have the request data available as well. + return + address = proxy_address(flow) + if flow.live: + flow.live.change_upstream_proxy_server(address) # type: ignore diff -Nru mitmproxy-5.1.1/docs/src/examples/contrib/check_ssl_pinning.py mitmproxy-6.0.2/docs/src/examples/contrib/check_ssl_pinning.py --- mitmproxy-5.1.1/docs/src/examples/contrib/check_ssl_pinning.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/contrib/check_ssl_pinning.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,84 @@ +import mitmproxy +from mitmproxy import ctx +from mitmproxy.certs import Cert +import ipaddress +import OpenSSL +import time + + +# Certificate for client connection is generated in dummy_cert() in certs.py. Monkeypatching +# the function to generate test cases for SSL Pinning. + +def monkey_dummy_cert(privkey, cacert, commonname, sans): + ss = [] + for i in sans: + try: + ipaddress.ip_address(i.decode("ascii")) + except ValueError: + # Change values in Certificate's Alt Name as well. + if ctx.options.certwrongCN: + ss.append(b"DNS:%sm" % i) + else: + ss.append(b"DNS:%s" % i) + else: + ss.append(b"IP:%s" % i) + ss = b", ".join(ss) + + cert = OpenSSL.crypto.X509() + if ctx.options.certbeginon: + # Set certificate start time somewhere in the future + cert.gmtime_adj_notBefore(3600 * 48) + else: + cert.gmtime_adj_notBefore(-3600 * 48) + + if ctx.options.certexpire: + # sets the expire date of the certificate in the past. + cert.gmtime_adj_notAfter(-3600 * 24) + else: + cert.gmtime_adj_notAfter(94608000) # = 24 * 60 * 60 * 365 * 3 + + cert.set_issuer(cacert.get_subject()) + if commonname is not None and len(commonname) < 64: + if ctx.options.certwrongCN: + # append an extra char to make certs common name different than original one. + # APpending a char in the end of the domain name. + new_cn = commonname + b'm' + cert.get_subject().CN = new_cn + + else: + cert.get_subject().CN = commonname + + cert.set_serial_number(int(time.time() * 10000)) + if ss: + cert.set_version(2) + cert.add_extensions( + [OpenSSL.crypto.X509Extension(b"subjectAltName", False, ss)]) + cert.set_pubkey(cacert.get_pubkey()) + cert.sign(privkey, "sha256") + return Cert(cert) + + +class CheckSSLPinning: + def load(self, loader): + loader.add_option( + "certbeginon", bool, False, + """ + Sets SSL Certificate's 'Begins On' time in future. + """ + ) + loader.add_option( + "certexpire", bool, False, + """ + Sets SSL Certificate's 'Expires On' time in the past. + """ + ) + + loader.add_option( + "certwrongCN", bool, False, + """ + Sets SSL Certificate's CommonName(CN) different from the domain name. + """ + ) + + def clientconnect(self, layer): + mitmproxy.certs.dummy_cert = monkey_dummy_cert diff -Nru mitmproxy-5.1.1/docs/src/examples/contrib/dns_spoofing.py mitmproxy-6.0.2/docs/src/examples/contrib/dns_spoofing.py --- mitmproxy-5.1.1/docs/src/examples/contrib/dns_spoofing.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/contrib/dns_spoofing.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,57 @@ +""" +This script makes it possible to use mitmproxy in scenarios where IP spoofing +has been used to redirect connections to mitmproxy. The way this works is that +we rely on either the TLS Server Name Indication (SNI) or the Host header of the +HTTP request. Of course, this is not foolproof - if an HTTPS connection comes +without SNI, we don't know the actual target and cannot construct a certificate +that looks valid. Similarly, if there's no Host header or a spoofed Host header, +we're out of luck as well. Using transparent mode is the better option most of +the time. + +Usage: + mitmproxy + -p 443 + -s dns_spoofing.py + # Used as the target location if neither SNI nor host header are present. + --mode reverse:http://example.com/ + # To avoid auto rewriting of host header by the reverse proxy target. + --set keep_host_header + mitmdump + -p 80 + --mode reverse:http://localhost:443/ + + (Setting up a single proxy instance and using iptables to redirect to it + works as well) +""" +import re + +# This regex extracts splits the host header into host and port. +# Handles the edge case of IPv6 addresses containing colons. +# https://bugzilla.mozilla.org/show_bug.cgi?id=45891 +parse_host_header = re.compile(r"^(?P[^:]+|\[.+\])(?::(?P\d+))?$") + + +class Rerouter: + def request(self, flow): + if flow.client_conn.tls_established: + flow.request.scheme = "https" + sni = flow.client_conn.connection.get_servername() + port = 443 + else: + flow.request.scheme = "http" + sni = None + port = 80 + + host_header = flow.request.host_header + m = parse_host_header.match(host_header) + if m: + host_header = m.group("host").strip("[]") + if m.group("port"): + port = int(m.group("port")) + + flow.request.host_header = host_header + flow.request.host = sni or host_header + flow.request.port = port + + +addons = [Rerouter()] diff -Nru mitmproxy-5.1.1/docs/src/examples/contrib/full_transparency_shim.c mitmproxy-6.0.2/docs/src/examples/contrib/full_transparency_shim.c --- mitmproxy-5.1.1/docs/src/examples/contrib/full_transparency_shim.c 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/contrib/full_transparency_shim.c 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,87 @@ +#define _GNU_SOURCE +#include +#include +#include +#include +#include +#include +#include + +/* This setuid wrapper can be used to run mitmproxy in full transparency mode, as a normal user. + * It will set the required capabilities (CAP_NET_RAW), drop privileges, and will then run argv[1] + * with the same capabilities. + * + * It can be compiled as follows: + * gcc examples/mitmproxy_shim.c -o mitmproxy_shim -lcap +*/ + +int set_caps(cap_t cap_struct, cap_value_t *cap_list, size_t bufsize) { + int cap_count = bufsize / sizeof(cap_list[0]); + + if (cap_set_flag(cap_struct, CAP_PERMITTED, cap_count, cap_list, CAP_SET) || + cap_set_flag(cap_struct, CAP_EFFECTIVE, cap_count, cap_list, CAP_SET) || + cap_set_flag(cap_struct, CAP_INHERITABLE, cap_count, cap_list, CAP_SET)) { + if (cap_count < 2) { + fprintf(stderr, "Cannot manipulate capability data structure as user: %s.\n", strerror(errno)); + } else { + fprintf(stderr, "Cannot manipulate capability data structure as root: %s.\n", strerror(errno)); + } + return -1; + } + + if (cap_count < 2) { + if (prctl(PR_CAP_AMBIENT, PR_CAP_AMBIENT_RAISE, CAP_NET_RAW, 0, 0)) { + fprintf(stderr, "Failed to add CAP_NET_RAW to the ambient set: %s.\n", strerror(errno)); + return -2; + } + } + + if (cap_set_proc(cap_struct)) { + if (cap_count < 2) { + fprintf(stderr, "Cannot set capabilities as user: %s.\n", strerror(errno)); + } else { + fprintf(stderr, "Cannot set capabilities as root: %s.\n", strerror(errno)); + } + return -3; + } + + if (cap_count > 1) { + if (prctl(PR_SET_KEEPCAPS, 1L)) { + fprintf(stderr, "Cannot keep capabilities after dropping privileges: %s.\n", strerror(errno)); + return -4; + } + if (cap_clear(cap_struct)) { + fprintf(stderr, "Cannot clear capability data structure: %s.\n", strerror(errno)); + return -5; + } + } +} + +int main(int argc, char **argv, char **envp) { + cap_t cap_struct = cap_init(); + cap_value_t root_caps[2] = { CAP_NET_RAW, CAP_SETUID }; + cap_value_t user_caps[1] = { CAP_NET_RAW }; + uid_t user = getuid(); + int res; + + if (setresuid(0, 0, 0)) { + fprintf(stderr, "Cannot switch to root: %s.\n", strerror(errno)); + return 1; + } + + if (res = set_caps(cap_struct, root_caps, sizeof(root_caps))) + return res; + + if (setresuid(user, user, user)) { + fprintf(stderr, "Cannot drop root privileges: %s.\n", strerror(errno)); + return 2; + } + + if (res = set_caps(cap_struct, user_caps, sizeof(user_caps))) + return res; + + if (execve(argv[1], argv + 1, envp)) { + fprintf(stderr, "Failed to execute %s: %s\n", argv[1], strerror(errno)); + return 3; + } +} diff -Nru mitmproxy-5.1.1/docs/src/examples/contrib/har_dump.py mitmproxy-6.0.2/docs/src/examples/contrib/har_dump.py --- mitmproxy-5.1.1/docs/src/examples/contrib/har_dump.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/contrib/har_dump.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,222 @@ +""" +This inline script can be used to dump flows as HAR files. + +example cmdline invocation: +mitmdump -s ./har_dump.py --set hardump=./dump.har + +filename endwith '.zhar' will be compressed: +mitmdump -s ./har_dump.py --set hardump=./dump.zhar +""" + + +import json +import base64 +import zlib +import os +import typing # noqa + +from datetime import datetime +from datetime import timezone + +import mitmproxy + +from mitmproxy import connections # noqa +from mitmproxy import version +from mitmproxy import ctx +from mitmproxy.utils import strutils +from mitmproxy.net.http import cookies + +HAR: typing.Dict = {} + +# A list of server seen till now is maintained so we can avoid +# using 'connect' time for entries that use an existing connection. +SERVERS_SEEN: typing.Set[connections.ServerConnection] = set() + + +def load(l): + l.add_option( + "hardump", str, "", "HAR dump path.", + ) + + +def configure(updated): + HAR.update({ + "log": { + "version": "1.2", + "creator": { + "name": "mitmproxy har_dump", + "version": "0.1", + "comment": "mitmproxy version %s" % version.MITMPROXY + }, + "entries": [] + } + }) + + +def response(flow): + """ + Called when a server response has been received. + """ + + # -1 indicates that these values do not apply to current request + ssl_time = -1 + connect_time = -1 + + if flow.server_conn and flow.server_conn not in SERVERS_SEEN: + connect_time = (flow.server_conn.timestamp_tcp_setup - + flow.server_conn.timestamp_start) + + if flow.server_conn.timestamp_tls_setup is not None: + ssl_time = (flow.server_conn.timestamp_tls_setup - + flow.server_conn.timestamp_tcp_setup) + + SERVERS_SEEN.add(flow.server_conn) + + # Calculate raw timings from timestamps. DNS timings can not be calculated + # for lack of a way to measure it. The same goes for HAR blocked. + # mitmproxy will open a server connection as soon as it receives the host + # and port from the client connection. So, the time spent waiting is actually + # spent waiting between request.timestamp_end and response.timestamp_start + # thus it correlates to HAR wait instead. + timings_raw = { + 'send': flow.request.timestamp_end - flow.request.timestamp_start, + 'receive': flow.response.timestamp_end - flow.response.timestamp_start, + 'wait': flow.response.timestamp_start - flow.request.timestamp_end, + 'connect': connect_time, + 'ssl': ssl_time, + } + + # HAR timings are integers in ms, so we re-encode the raw timings to that format. + timings = { + k: int(1000 * v) if v != -1 else -1 + for k, v in timings_raw.items() + } + + # full_time is the sum of all timings. + # Timings set to -1 will be ignored as per spec. + full_time = sum(v for v in timings.values() if v > -1) + + started_date_time = datetime.fromtimestamp(flow.request.timestamp_start, timezone.utc).isoformat() + + # Response body size and encoding + response_body_size = len(flow.response.raw_content) if flow.response.raw_content else 0 + response_body_decoded_size = len(flow.response.content) if flow.response.content else 0 + response_body_compression = response_body_decoded_size - response_body_size + + entry = { + "startedDateTime": started_date_time, + "time": full_time, + "request": { + "method": flow.request.method, + "url": flow.request.url, + "httpVersion": flow.request.http_version, + "cookies": format_request_cookies(flow.request.cookies.fields), + "headers": name_value(flow.request.headers), + "queryString": name_value(flow.request.query or {}), + "headersSize": len(str(flow.request.headers)), + "bodySize": len(flow.request.content), + }, + "response": { + "status": flow.response.status_code, + "statusText": flow.response.reason, + "httpVersion": flow.response.http_version, + "cookies": format_response_cookies(flow.response.cookies.fields), + "headers": name_value(flow.response.headers), + "content": { + "size": response_body_size, + "compression": response_body_compression, + "mimeType": flow.response.headers.get('Content-Type', '') + }, + "redirectURL": flow.response.headers.get('Location', ''), + "headersSize": len(str(flow.response.headers)), + "bodySize": response_body_size, + }, + "cache": {}, + "timings": timings, + } + + # Store binary data as base64 + if strutils.is_mostly_bin(flow.response.content): + entry["response"]["content"]["text"] = base64.b64encode(flow.response.content).decode() + entry["response"]["content"]["encoding"] = "base64" + else: + entry["response"]["content"]["text"] = flow.response.get_text(strict=False) + + if flow.request.method in ["POST", "PUT", "PATCH"]: + params = [ + {"name": a, "value": b} + for a, b in flow.request.urlencoded_form.items(multi=True) + ] + entry["request"]["postData"] = { + "mimeType": flow.request.headers.get("Content-Type", ""), + "text": flow.request.get_text(strict=False), + "params": params + } + + if flow.server_conn.connected(): + entry["serverIPAddress"] = str(flow.server_conn.ip_address[0]) + + HAR["log"]["entries"].append(entry) + + +def done(): + """ + Called once on script shutdown, after any other events. + """ + if ctx.options.hardump: + json_dump: str = json.dumps(HAR, indent=2) + + if ctx.options.hardump == '-': + mitmproxy.ctx.log(json_dump) + else: + raw: bytes = json_dump.encode() + if ctx.options.hardump.endswith('.zhar'): + raw = zlib.compress(raw, 9) + + with open(os.path.expanduser(ctx.options.hardump), "wb") as f: + f.write(raw) + + mitmproxy.ctx.log("HAR dump finished (wrote %s bytes to file)" % len(json_dump)) + + +def format_cookies(cookie_list): + rv = [] + + for name, value, attrs in cookie_list: + cookie_har = { + "name": name, + "value": value, + } + + # HAR only needs some attributes + for key in ["path", "domain", "comment"]: + if key in attrs: + cookie_har[key] = attrs[key] + + # These keys need to be boolean! + for key in ["httpOnly", "secure"]: + cookie_har[key] = bool(key in attrs) + + # Expiration time needs to be formatted + expire_ts = cookies.get_expiration_ts(attrs) + if expire_ts is not None: + cookie_har["expires"] = datetime.fromtimestamp(expire_ts, timezone.utc).isoformat() + + rv.append(cookie_har) + + return rv + + +def format_request_cookies(fields): + return format_cookies(cookies.group_cookies(fields)) + + +def format_response_cookies(fields): + return format_cookies((c[0], c[1][0], c[1][1]) for c in fields) + + +def name_value(obj): + """ + Convert (key, value) pairs to HAR format. + """ + return [{"name": k, "value": v} for k, v in obj.items()] diff -Nru mitmproxy-5.1.1/docs/src/examples/contrib/jsondump.py mitmproxy-6.0.2/docs/src/examples/contrib/jsondump.py --- mitmproxy-5.1.1/docs/src/examples/contrib/jsondump.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/contrib/jsondump.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,257 @@ +""" +This script serializes the entire traffic dump, including websocket traffic, +as JSON, and either sends it to a URL or writes to a file. The serialization +format is optimized for Elasticsearch; the script can be used to send all +captured traffic to Elasticsearch directly. + +Usage: + + mitmproxy + --mode reverse:http://example.com/ + -s examples/complex/jsondump.py + +Configuration: + + Send to a URL: + + cat > ~/.mitmproxy/config.yaml < ~/.mitmproxy/config.yaml <) and expands them to absolute links +# In practice this can be used to front an indexing spider that may not have the capability to expand relative page links. +# Usage: mitmdump -s link_expander.py or mitmproxy -s link_expander.py + +import re +from urllib.parse import urljoin + + +def response(flow): + + if "Content-Type" in flow.response.headers and flow.response.headers["Content-Type"].find("text/html") != -1: + pageUrl = flow.request.url + pageText = flow.response.text + pattern = (r"]*?\s+)?href=(?P[\"'])" + r"(?P(?!https?:\/\/|ftps?:\/\/|\/\/|#|javascript:|mailto:).*?)(?P=delimiter)") + rel_matcher = re.compile(pattern, flags=re.IGNORECASE) + rel_matches = rel_matcher.finditer(pageText) + map_dict = {} + for match_num, match in enumerate(rel_matches): + (delimiter, rel_link) = match.group("delimiter", "link") + abs_link = urljoin(pageUrl, rel_link) + map_dict["{0}{1}{0}".format(delimiter, rel_link)] = "{0}{1}{0}".format(delimiter, abs_link) + for map in map_dict.items(): + pageText = pageText.replace(*map) + # Uncomment the following to print the expansion mapping + # print("{0} -> {1}".format(*map)) + flow.response.text = pageText \ No newline at end of file diff -Nru mitmproxy-5.1.1/docs/src/examples/contrib/mitmproxywrapper.py mitmproxy-6.0.2/docs/src/examples/contrib/mitmproxywrapper.py --- mitmproxy-5.1.1/docs/src/examples/contrib/mitmproxywrapper.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/contrib/mitmproxywrapper.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,165 @@ +#!/usr/bin/env python +# +# Helper tool to enable/disable OS X proxy and wrap mitmproxy +# +# Get usage information with: +# +# mitmproxywrapper.py -h +# + +import subprocess +import re +import argparse +import contextlib +import os +import sys + + +class Wrapper: + def __init__(self, port, extra_arguments=None): + self.port = port + self.extra_arguments = extra_arguments + + def run_networksetup_command(self, *arguments): + return subprocess.check_output( + ['sudo', 'networksetup'] + list(arguments)) + + def proxy_state_for_service(self, service): + state = self.run_networksetup_command( + '-getwebproxy', + service).splitlines() + return dict([re.findall(r'([^:]+): (.*)', line)[0] for line in state]) + + def enable_proxy_for_service(self, service): + print(f'Enabling proxy on {service}...') + for subcommand in ['-setwebproxy', '-setsecurewebproxy']: + self.run_networksetup_command( + subcommand, service, '127.0.0.1', str( + self.port)) + + def disable_proxy_for_service(self, service): + print(f'Disabling proxy on {service}...') + for subcommand in ['-setwebproxystate', '-setsecurewebproxystate']: + self.run_networksetup_command(subcommand, service, 'Off') + + def interface_name_to_service_name_map(self): + order = self.run_networksetup_command('-listnetworkserviceorder') + mapping = re.findall( + r'\(\d+\)\s(.*)$\n\(.*Device: (.+)\)$', + order, + re.MULTILINE) + return {b: a for (a, b) in mapping} + + def run_command_with_input(self, command, input): + popen = subprocess.Popen( + command, + stdin=subprocess.PIPE, + stdout=subprocess.PIPE) + (stdout, stderr) = popen.communicate(input) + return stdout + + def primary_interace_name(self): + scutil_script = 'get State:/Network/Global/IPv4\nd.show\n' + stdout = self.run_command_with_input('/usr/sbin/scutil', scutil_script) + interface, = re.findall(r'PrimaryInterface\s*:\s*(.+)', stdout) + return interface + + def primary_service_name(self): + return self.interface_name_to_service_name_map()[ + self.primary_interace_name()] + + def proxy_enabled_for_service(self, service): + return self.proxy_state_for_service(service)['Enabled'] == 'Yes' + + def toggle_proxy(self): + new_state = not self.proxy_enabled_for_service( + self.primary_service_name()) + for service_name in self.connected_service_names(): + if self.proxy_enabled_for_service(service_name) and not new_state: + self.disable_proxy_for_service(service_name) + elif not self.proxy_enabled_for_service(service_name) and new_state: + self.enable_proxy_for_service(service_name) + + def connected_service_names(self): + scutil_script = 'list\n' + stdout = self.run_command_with_input('/usr/sbin/scutil', scutil_script) + service_ids = re.findall(r'State:/Network/Service/(.+)/IPv4', stdout) + + service_names = [] + for service_id in service_ids: + scutil_script = 'show Setup:/Network/Service/{}\n'.format( + service_id) + stdout = self.run_command_with_input( + '/usr/sbin/scutil', + scutil_script) + service_name, = re.findall(r'UserDefinedName\s*:\s*(.+)', stdout) + service_names.append(service_name) + + return service_names + + def wrap_mitmproxy(self): + with self.wrap_proxy(): + cmd = ['mitmproxy', '-p', str(self.port)] + if self.extra_arguments: + cmd.extend(self.extra_arguments) + subprocess.check_call(cmd) + + def wrap_honeyproxy(self): + with self.wrap_proxy(): + popen = subprocess.Popen('honeyproxy.sh') + try: + popen.wait() + except KeyboardInterrupt: + popen.terminate() + + @contextlib.contextmanager + def wrap_proxy(self): + connected_service_names = self.connected_service_names() + for service_name in connected_service_names: + if not self.proxy_enabled_for_service(service_name): + self.enable_proxy_for_service(service_name) + + yield + + for service_name in connected_service_names: + if self.proxy_enabled_for_service(service_name): + self.disable_proxy_for_service(service_name) + + @classmethod + def ensure_superuser(cls): + if os.getuid() != 0: + print('Relaunching with sudo...') + os.execv('/usr/bin/sudo', ['/usr/bin/sudo'] + sys.argv) + + @classmethod + def main(cls): + parser = argparse.ArgumentParser( + description='Helper tool for OS X proxy configuration and mitmproxy.', + epilog='Any additional arguments will be passed on unchanged to mitmproxy.') + parser.add_argument( + '-t', + '--toggle', + action='store_true', + help='just toggle the proxy configuration') + # parser.add_argument('--honeyproxy', action='store_true', help='run honeyproxy instead of mitmproxy') + parser.add_argument( + '-p', + '--port', + type=int, + help='override the default port of 8080', + default=8080) + args, extra_arguments = parser.parse_known_args() + + wrapper = cls(port=args.port, extra_arguments=extra_arguments) + + if args.toggle: + wrapper.toggle_proxy() + # elif args.honeyproxy: + # wrapper.wrap_honeyproxy() + else: + wrapper.wrap_mitmproxy() + + +if __name__ == '__main__': + Wrapper.ensure_superuser() + Wrapper.main() diff -Nru mitmproxy-5.1.1/docs/src/examples/contrib/modify_body_inject_iframe.py mitmproxy-6.0.2/docs/src/examples/contrib/modify_body_inject_iframe.py --- mitmproxy-5.1.1/docs/src/examples/contrib/modify_body_inject_iframe.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/contrib/modify_body_inject_iframe.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,26 @@ +# (this script works best with --anticache) +from bs4 import BeautifulSoup +from mitmproxy import ctx, http + + +class Injector: + def load(self, loader): + loader.add_option( + "iframe", str, "", "IFrame to inject" + ) + + def response(self, flow: http.HTTPFlow) -> None: + if ctx.options.iframe: + html = BeautifulSoup(flow.response.content, "html.parser") + if html.body: + iframe = html.new_tag( + "iframe", + src=ctx.options.iframe, + frameborder=0, + height=0, + width=0) + html.body.insert(0, iframe) + flow.response.content = str(html).encode("utf8") + + +addons = [Injector()] diff -Nru mitmproxy-5.1.1/docs/src/examples/contrib/README.md mitmproxy-6.0.2/docs/src/examples/contrib/README.md --- mitmproxy-5.1.1/docs/src/examples/contrib/README.md 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/contrib/README.md 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,4 @@ +# Community-Contributed Examples + +Examples in this directory are contributed by the mitmproxy community. +We do _not_ maintain them, but we welcome PRs that add/fix/modernize/clean up examples. \ No newline at end of file diff -Nru mitmproxy-5.1.1/docs/src/examples/contrib/remote-debug.py mitmproxy-6.0.2/docs/src/examples/contrib/remote-debug.py --- mitmproxy-5.1.1/docs/src/examples/contrib/remote-debug.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/contrib/remote-debug.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,21 @@ +""" +This script enables remote debugging of the mitmproxy console *UI* with PyCharm. +For general debugging purposes, it is easier to just debug mitmdump within PyCharm. + +Usage: + - pip install pydevd on the mitmproxy machine + - Open the Run/Debug Configuration dialog box in PyCharm, and select the + Python Remote Debug configuration type. + - Debugging works in the way that mitmproxy connects to the debug server + on startup. Specify host and port that mitmproxy can use to reach your + PyCharm instance on startup. + - Adjust this inline script accordingly. + - Start debug server in PyCharm + - Set breakpoints + - Start mitmproxy -s remote_debug.py +""" + + +def load(l): + import pydevd_pycharm + pydevd_pycharm.settrace("localhost", port=5678, stdoutToServer=True, stderrToServer=True, suspend=False) diff -Nru mitmproxy-5.1.1/docs/src/examples/contrib/sslstrip.py mitmproxy-6.0.2/docs/src/examples/contrib/sslstrip.py --- mitmproxy-5.1.1/docs/src/examples/contrib/sslstrip.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/contrib/sslstrip.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,63 @@ +""" +This script implements an sslstrip-like attack based on mitmproxy. +https://moxie.org/software/sslstrip/ +""" +import re +import urllib.parse +import typing # noqa + +from mitmproxy import http + +# set of SSL/TLS capable hosts +secure_hosts: typing.Set[str] = set() + + +def request(flow: http.HTTPFlow) -> None: + flow.request.headers.pop('If-Modified-Since', None) + flow.request.headers.pop('Cache-Control', None) + + # do not force https redirection + flow.request.headers.pop('Upgrade-Insecure-Requests', None) + + # proxy connections to SSL-enabled hosts + if flow.request.pretty_host in secure_hosts: + flow.request.scheme = 'https' + flow.request.port = 443 + + # We need to update the request destination to whatever is specified in the host header: + # Having no TLS Server Name Indication from the client and just an IP address as request.host + # in transparent mode, TLS server name certificate validation would fail. + flow.request.host = flow.request.pretty_host + + +def response(flow: http.HTTPFlow) -> None: + assert flow.response + flow.response.headers.pop('Strict-Transport-Security', None) + flow.response.headers.pop('Public-Key-Pins', None) + + # strip links in response body + flow.response.content = flow.response.content.replace(b'https://', b'http://') + + # strip meta tag upgrade-insecure-requests in response body + csp_meta_tag_pattern = br'' + flow.response.content = re.sub(csp_meta_tag_pattern, b'', flow.response.content, flags=re.IGNORECASE) + + # strip links in 'Location' header + if flow.response.headers.get('Location', '').startswith('https://'): + location = flow.response.headers['Location'] + hostname = urllib.parse.urlparse(location).hostname + if hostname: + secure_hosts.add(hostname) + flow.response.headers['Location'] = location.replace('https://', 'http://', 1) + + # strip upgrade-insecure-requests in Content-Security-Policy header + csp_header = flow.response.headers.get('Content-Security-Policy', '') + if re.search('upgrade-insecure-requests', csp_header, flags=re.IGNORECASE): + csp = flow.response.headers['Content-Security-Policy'] + new_header = re.sub(r'upgrade-insecure-requests[;\s]*', '', csp, flags=re.IGNORECASE) + flow.response.headers['Content-Security-Policy'] = new_header + + # strip secure flag from 'Set-Cookie' headers + cookies = flow.response.headers.get_all('Set-Cookie') + cookies = [re.sub(r';\s*secure\s*', '', s) for s in cookies] + flow.response.headers.set_all('Set-Cookie', cookies) diff -Nru mitmproxy-5.1.1/docs/src/examples/contrib/suppress_error_responses.py mitmproxy-6.0.2/docs/src/examples/contrib/suppress_error_responses.py --- mitmproxy-5.1.1/docs/src/examples/contrib/suppress_error_responses.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/contrib/suppress_error_responses.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,16 @@ +""" +This script suppresses the 502 Bad Gateway messages, mitmproxy sends if the server is not responsing correctly. +For example, this functionality can be helpful if mitmproxy is used in between a web scanner and a web application. +Without this script, if the web application under test crashes, mitmproxy will send 502 Bad Gateway responses. +These responses are irritating the web application scanner since they obfuscate the actual problem. +""" +from mitmproxy import http +from mitmproxy.exceptions import HttpSyntaxException + + +def error(self, flow: http.HTTPFlow): + """Kills the flow if it has an error different to HTTPSyntaxException. + Sometimes, web scanners generate malformed HTTP syntax on purpose and we do not want to kill these requests. + """ + if flow.error is not None and not isinstance(flow.error, HttpSyntaxException): + flow.kill() diff -Nru mitmproxy-5.1.1/docs/src/examples/contrib/test_har_dump.py mitmproxy-6.0.2/docs/src/examples/contrib/test_har_dump.py --- mitmproxy-5.1.1/docs/src/examples/contrib/test_har_dump.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/contrib/test_har_dump.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,84 @@ +import json + +from mitmproxy.test import tflow +from mitmproxy.test import tutils +from mitmproxy.test import taddons +from mitmproxy.net.http import cookies + + +class TestHARDump: + def flow(self, resp_content=b'message'): + times = dict( + timestamp_start=746203272, + timestamp_end=746203272, + ) + + # Create a dummy flow for testing + return tflow.tflow( + req=tutils.treq(method=b'GET', **times), + resp=tutils.tresp(content=resp_content, **times) + ) + + def test_simple(self, tmpdir, tdata): + with taddons.context() as tctx: + a = tctx.script(tdata.path("../examples/complex/har_dump.py")) + path = str(tmpdir.join("somefile")) + tctx.configure(a, hardump=path) + tctx.invoke(a, "response", self.flow()) + tctx.invoke(a, "done") + with open(path) as inp: + har = json.load(inp) + assert len(har["log"]["entries"]) == 1 + + def test_base64(self, tmpdir, tdata): + with taddons.context() as tctx: + a = tctx.script(tdata.path("../examples/complex/har_dump.py")) + path = str(tmpdir.join("somefile")) + tctx.configure(a, hardump=path) + + tctx.invoke( + a, "response", self.flow(resp_content=b"foo" + b"\xFF" * 10) + ) + tctx.invoke(a, "done") + with open(path) as inp: + har = json.load(inp) + assert har["log"]["entries"][0]["response"]["content"]["encoding"] == "base64" + + def test_format_cookies(self, tdata): + with taddons.context() as tctx: + a = tctx.script(tdata.path("../examples/complex/har_dump.py")) + + CA = cookies.CookieAttrs + + f = a.format_cookies([("n", "v", CA([("k", "v")]))])[0] + assert f['name'] == "n" + assert f['value'] == "v" + assert not f['httpOnly'] + assert not f['secure'] + + f = a.format_cookies([("n", "v", CA([("httponly", None), ("secure", None)]))])[0] + assert f['httpOnly'] + assert f['secure'] + + f = a.format_cookies([("n", "v", CA([("expires", "Mon, 24-Aug-2037 00:00:00 GMT")]))])[0] + assert f['expires'] + + def test_binary(self, tmpdir, tdata): + with taddons.context() as tctx: + a = tctx.script(tdata.path("../examples/complex/har_dump.py")) + path = str(tmpdir.join("somefile")) + tctx.configure(a, hardump=path) + + f = self.flow() + f.request.method = "POST" + f.request.headers["content-type"] = "application/x-www-form-urlencoded" + f.request.content = b"foo=bar&baz=s%c3%bc%c3%9f" + f.response.headers["random-junk"] = bytes(range(256)) + f.response.content = bytes(range(256)) + + tctx.invoke(a, "response", f) + tctx.invoke(a, "done") + + with open(path) as inp: + har = json.load(inp) + assert len(har["log"]["entries"]) == 1 diff -Nru mitmproxy-5.1.1/docs/src/examples/contrib/test_jsondump.py mitmproxy-6.0.2/docs/src/examples/contrib/test_jsondump.py --- mitmproxy-5.1.1/docs/src/examples/contrib/test_jsondump.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/contrib/test_jsondump.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,68 @@ +import json +import base64 + +from mitmproxy.test import tflow +from mitmproxy.test import tutils +from mitmproxy.test import taddons + +import requests_mock + +example_dir = tutils.test_data.push("../examples") + + +class TestJSONDump: + def echo_response(self, request, context): + self.request = {'json': request.json(), 'headers': request.headers} + return '' + + def flow(self, resp_content=b'message'): + times = dict( + timestamp_start=746203272, + timestamp_end=746203272, + ) + + # Create a dummy flow for testing + return tflow.tflow( + req=tutils.treq(method=b'GET', **times), + resp=tutils.tresp(content=resp_content, **times) + ) + + def test_simple(self, tmpdir): + with taddons.context() as tctx: + a = tctx.script(example_dir.path("complex/jsondump.py")) + path = str(tmpdir.join("jsondump.out")) + tctx.configure(a, dump_destination=path) + tctx.invoke(a, "response", self.flow()) + tctx.invoke(a, "done") + with open(path) as inp: + entry = json.loads(inp.readline()) + assert entry['response']['content'] == 'message' + + def test_contentencode(self, tmpdir): + with taddons.context() as tctx: + a = tctx.script(example_dir.path("complex/jsondump.py")) + path = str(tmpdir.join("jsondump.out")) + content = b"foo" + b"\xFF" * 10 + tctx.configure(a, dump_destination=path, dump_encodecontent=True) + + tctx.invoke( + a, "response", self.flow(resp_content=content) + ) + tctx.invoke(a, "done") + with open(path) as inp: + entry = json.loads(inp.readline()) + assert entry['response']['content'] == base64.b64encode(content).decode('utf-8') + + def test_http(self, tmpdir): + with requests_mock.Mocker() as mock: + mock.post('http://my-server', text=self.echo_response) + with taddons.context() as tctx: + a = tctx.script(example_dir.path("complex/jsondump.py")) + tctx.configure(a, dump_destination='http://my-server', + dump_username='user', dump_password='pass') + + tctx.invoke(a, "response", self.flow()) + tctx.invoke(a, "done") + + assert self.request['json']['response']['content'] == 'message' + assert self.request['headers']['Authorization'] == 'Basic dXNlcjpwYXNz' diff -Nru mitmproxy-5.1.1/docs/src/examples/contrib/test_xss_scanner.py mitmproxy-6.0.2/docs/src/examples/contrib/test_xss_scanner.py --- mitmproxy-5.1.1/docs/src/examples/contrib/test_xss_scanner.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/contrib/test_xss_scanner.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,391 @@ +import pytest +import requests +from examples.complex import xss_scanner as xss +from mitmproxy.test import tflow, tutils + + +class TestXSSScanner(): + def test_get_XSS_info(self): + # First type of exploit: + # Exploitable: + xss_info = xss.get_XSS_data(b"" % + xss.FULL_PAYLOAD, + "https://example.com", + "End of URL") + expected_xss_info = xss.XSSData('https://example.com', + "End of URL", + '" % + xss.FULL_PAYLOAD.replace(b"'", b"%27").replace(b'"', b"%22"), + "https://example.com", + "End of URL") + expected_xss_info = xss.XSSData("https://example.com", + "End of URL", + '" % + xss.FULL_PAYLOAD.replace(b"'", b"%27").replace(b'"', b"%22").replace(b"/", b"%2F"), + "https://example.com", + "End of URL") + assert xss_info is None + # Second type of exploit: + # Exploitable: + xss_info = xss.get_XSS_data(b"" % + xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b">", b"%3E").replace(b"\"", b"%22"), + "https://example.com", + "End of URL") + expected_xss_info = xss.XSSData("https://example.com", + "End of URL", + "';alert(0);g='", + xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b">", b"%3E") + .replace(b"\"", b"%22").decode('utf-8')) + assert xss_info == expected_xss_info + # Non-Exploitable: + xss_info = xss.get_XSS_data(b"" % + xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b"\"", b"%22").replace(b"'", b"%22"), + "https://example.com", + "End of URL") + assert xss_info is None + # Third type of exploit: + # Exploitable: + xss_info = xss.get_XSS_data(b"" % + xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b">", b"%3E").replace(b"'", b"%27"), + "https://example.com", + "End of URL") + expected_xss_info = xss.XSSData("https://example.com", + "End of URL", + '";alert(0);g="', + xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b">", b"%3E") + .replace(b"'", b"%27").decode('utf-8')) + assert xss_info == expected_xss_info + # Non-Exploitable: + xss_info = xss.get_XSS_data(b"" % + xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b"'", b"%27").replace(b"\"", b"%22"), + "https://example.com", + "End of URL") + assert xss_info is None + # Fourth type of exploit: Test + # Exploitable: + xss_info = xss.get_XSS_data(b"Test" % + xss.FULL_PAYLOAD, + "https://example.com", + "End of URL") + expected_xss_info = xss.XSSData("https://example.com", + "End of URL", + "'>", + xss.FULL_PAYLOAD.decode('utf-8')) + assert xss_info == expected_xss_info + # Non-Exploitable: + xss_info = xss.get_XSS_data(b"Test" % + xss.FULL_PAYLOAD.replace(b"'", b"%27"), + "https://example.com", + "End of URL") + assert xss_info is None + # Fifth type of exploit: Test + # Exploitable: + xss_info = xss.get_XSS_data(b"Test" % + xss.FULL_PAYLOAD.replace(b"'", b"%27"), + "https://example.com", + "End of URL") + expected_xss_info = xss.XSSData("https://example.com", + "End of URL", + "\">", + xss.FULL_PAYLOAD.replace(b"'", b"%27").decode('utf-8')) + assert xss_info == expected_xss_info + # Non-Exploitable: + xss_info = xss.get_XSS_data(b"Test" % + xss.FULL_PAYLOAD.replace(b"'", b"%27").replace(b"\"", b"%22"), + "https://example.com", + "End of URL") + assert xss_info is None + # Sixth type of exploit: Test + # Exploitable: + xss_info = xss.get_XSS_data(b"Test" % + xss.FULL_PAYLOAD, + "https://example.com", + "End of URL") + expected_xss_info = xss.XSSData("https://example.com", + "End of URL", + ">", + xss.FULL_PAYLOAD.decode('utf-8')) + assert xss_info == expected_xss_info + # Non-Exploitable + xss_info = xss.get_XSS_data(b"Test" % + xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b">", b"%3E") + .replace(b"=", b"%3D"), + "https://example.com", + "End of URL") + assert xss_info is None + # Seventh type of exploit: PAYLOAD + # Exploitable: + xss_info = xss.get_XSS_data(b"%s" % + xss.FULL_PAYLOAD, + "https://example.com", + "End of URL") + expected_xss_info = xss.XSSData("https://example.com", + "End of URL", + "", + xss.FULL_PAYLOAD.decode('utf-8')) + assert xss_info == expected_xss_info + # Non-Exploitable + xss_info = xss.get_XSS_data(b"%s" % + xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b">", b"%3E").replace(b"/", b"%2F"), + "https://example.com", + "End of URL") + assert xss_info is None + # Eighth type of exploit: Test + # Exploitable: + xss_info = xss.get_XSS_data(b"Test" % + xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b">", b"%3E"), + "https://example.com", + "End of URL") + expected_xss_info = xss.XSSData("https://example.com", + "End of URL", + "Javascript:alert(0)", + xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b">", b"%3E").decode('utf-8')) + assert xss_info == expected_xss_info + # Non-Exploitable: + xss_info = xss.get_XSS_data(b"Test" % + xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b">", b"%3E") + .replace(b"=", b"%3D"), + "https://example.com", + "End of URL") + assert xss_info is None + # Ninth type of exploit: Test + # Exploitable: + xss_info = xss.get_XSS_data(b"Test" % + xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b">", b"%3E"), + "https://example.com", + "End of URL") + expected_xss_info = xss.XSSData("https://example.com", + "End of URL", + '" onmouseover="alert(0)" t="', + xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b">", b"%3E").decode('utf-8')) + assert xss_info == expected_xss_info + # Non-Exploitable: + xss_info = xss.get_XSS_data(b"Test" % + xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b">", b"%3E") + .replace(b'"', b"%22"), + "https://example.com", + "End of URL") + assert xss_info is None + # Tenth type of exploit: Test + # Exploitable: + xss_info = xss.get_XSS_data(b"Test" % + xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b">", b"%3E"), + "https://example.com", + "End of URL") + expected_xss_info = xss.XSSData("https://example.com", + "End of URL", + "' onmouseover='alert(0)' t='", + xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b">", b"%3E").decode('utf-8')) + assert xss_info == expected_xss_info + # Non-Exploitable: + xss_info = xss.get_XSS_data(b"Test" % + xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b">", b"%3E") + .replace(b"'", b"%22"), + "https://example.com", + "End of URL") + assert xss_info is None + # Eleventh type of exploit: Test + # Exploitable: + xss_info = xss.get_XSS_data(b"Test" % + xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b">", b"%3E"), + "https://example.com", + "End of URL") + expected_xss_info = xss.XSSData("https://example.com", + "End of URL", + " onmouseover=alert(0) t=", + xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b">", b"%3E").decode('utf-8')) + assert xss_info == expected_xss_info + # Non-Exploitable: + xss_info = xss.get_XSS_data(b"Test" % + xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b">", b"%3E") + .replace(b"=", b"%3D"), + "https://example.com", + "End of URL") + assert xss_info is None + + def test_get_SQLi_data(self): + sqli_data = xss.get_SQLi_data("SQL syntax MySQL", + "", + "https://example.com", + "End of URL") + expected_sqli_data = xss.SQLiData("https://example.com", + "End of URL", + "SQL syntax.*MySQL", + "MySQL") + assert sqli_data == expected_sqli_data + sqli_data = xss.get_SQLi_data("SQL syntax MySQL", + "SQL syntax MySQL", + "https://example.com", + "End of URL") + assert sqli_data is None + + def test_inside_quote(self): + assert not xss.inside_quote("'", b"no", 0, b"no") + assert xss.inside_quote("'", b"yes", 0, b"'yes'") + assert xss.inside_quote("'", b"yes", 1, b"'yes'otherJunk'yes'more") + assert not xss.inside_quote("'", b"longStringNotInIt", 1, b"short") + + def test_paths_to_text(self): + text = xss.paths_to_text("""

STRING

+ + """, "STRING") + expected_text = ["/html/head/h1", "/html/script"] + assert text == expected_text + assert xss.paths_to_text("""""", "STRING") == [] + + def mocked_requests_vuln(*args, headers=None, cookies=None): + class MockResponse: + def __init__(self, html, headers=None, cookies=None): + self.text = html + return MockResponse("%s" % xss.FULL_PAYLOAD) + + def mocked_requests_invuln(*args, headers=None, cookies=None): + class MockResponse: + def __init__(self, html, headers=None, cookies=None): + self.text = html + return MockResponse("") + + def test_test_end_of_url_injection(self, get_request_vuln): + xss_info = xss.test_end_of_URL_injection("", "https://example.com/index.html", {})[0] + expected_xss_info = xss.XSSData('https://example.com/index.html/1029zxcs\'d"aoso[sb]po(pc)se;sl/bsl\\eq=3847asd', + 'End of URL', + '', + '1029zxcs\\\'d"aoso[sb]po(pc)se;sl/bsl\\\\eq=3847asd') + sqli_info = xss.test_end_of_URL_injection("", "https://example.com/", {})[1] + assert xss_info == expected_xss_info + assert sqli_info is None + + def test_test_referer_injection(self, get_request_vuln): + xss_info = xss.test_referer_injection("", "https://example.com/", {})[0] + expected_xss_info = xss.XSSData('https://example.com/', + 'Referer', + '', + '1029zxcs\\\'d"aoso[sb]po(pc)se;sl/bsl\\\\eq=3847asd') + sqli_info = xss.test_referer_injection("", "https://example.com/", {})[1] + assert xss_info == expected_xss_info + assert sqli_info is None + + def test_test_user_agent_injection(self, get_request_vuln): + xss_info = xss.test_user_agent_injection("", "https://example.com/", {})[0] + expected_xss_info = xss.XSSData('https://example.com/', + 'User Agent', + '', + '1029zxcs\\\'d"aoso[sb]po(pc)se;sl/bsl\\\\eq=3847asd') + sqli_info = xss.test_user_agent_injection("", "https://example.com/", {})[1] + assert xss_info == expected_xss_info + assert sqli_info is None + + def test_test_query_injection(self, get_request_vuln): + + xss_info = xss.test_query_injection("", "https://example.com/vuln.php?cmd=ls", {})[0] + expected_xss_info = xss.XSSData('https://example.com/vuln.php?cmd=1029zxcs\'d"aoso[sb]po(pc)se;sl/bsl\\eq=3847asd', + 'Query', + '', + '1029zxcs\\\'d"aoso[sb]po(pc)se;sl/bsl\\\\eq=3847asd') + sqli_info = xss.test_query_injection("", "https://example.com/vuln.php?cmd=ls", {})[1] + assert xss_info == expected_xss_info + assert sqli_info is None + + @pytest.fixture(scope='function') + def logger(self, monkeypatch): + class Logger(): + def __init__(self): + self.args = [] + + def info(self, str): + self.args.append(str) + + def error(self, str): + self.args.append(str) + + logger = Logger() + monkeypatch.setattr("mitmproxy.ctx.log", logger) + yield logger + + @pytest.fixture(scope='function') + def get_request_vuln(self, monkeypatch): + monkeypatch.setattr(requests, 'get', self.mocked_requests_vuln) + + @pytest.fixture(scope='function') + def get_request_invuln(self, monkeypatch): + monkeypatch.setattr(requests, 'get', self.mocked_requests_invuln) + + @pytest.fixture(scope='function') + def mock_gethostbyname(self, monkeypatch): + def gethostbyname(domain): + claimed_domains = ["google.com"] + if domain not in claimed_domains: + from socket import gaierror + raise gaierror("[Errno -2] Name or service not known") + else: + return '216.58.221.46' + + monkeypatch.setattr("socket.gethostbyname", gethostbyname) + + def test_find_unclaimed_URLs(self, logger, mock_gethostbyname): + xss.find_unclaimed_URLs("", + "https://example.com") + assert logger.args == [] + xss.find_unclaimed_URLs("", + "https://example.com") + assert logger.args[0] == 'XSS found in https://example.com due to unclaimed URL "http://unclaimedDomainName.com".' + xss.find_unclaimed_URLs("", + "https://example.com") + assert logger.args[1] == 'XSS found in https://example.com due to unclaimed URL "http://unclaimedDomainName.com".' + xss.find_unclaimed_URLs("", + "https://example.com") + assert logger.args[2] == 'XSS found in https://example.com due to unclaimed URL "http://unclaimedDomainName.com".' + + def test_log_XSS_data(self, logger): + xss.log_XSS_data(None) + assert logger.args == [] + # self, url: str, injection_point: str, exploit: str, line: str + xss.log_XSS_data(xss.XSSData('https://example.com', + 'Location', + 'String', + 'Line of HTML')) + assert logger.args[0] == '===== XSS Found ====' + assert logger.args[1] == 'XSS URL: https://example.com' + assert logger.args[2] == 'Injection Point: Location' + assert logger.args[3] == 'Suggested Exploit: String' + assert logger.args[4] == 'Line: Line of HTML' + + def test_log_SQLi_data(self, logger): + xss.log_SQLi_data(None) + assert logger.args == [] + xss.log_SQLi_data(xss.SQLiData('https://example.com', + 'Location', + 'Oracle.*Driver', + 'Oracle')) + assert logger.args[0] == '===== SQLi Found =====' + assert logger.args[1] == 'SQLi URL: https://example.com' + assert logger.args[2] == 'Injection Point: Location' + assert logger.args[3] == 'Regex used: Oracle.*Driver' + + def test_get_cookies(self): + mocked_req = tutils.treq() + mocked_req.cookies = [("cookieName2", "cookieValue2")] + mocked_flow = tflow.tflow(req=mocked_req) + # It only uses the request cookies + assert xss.get_cookies(mocked_flow) == {"cookieName2": "cookieValue2"} + + def test_response(self, get_request_invuln, logger): + mocked_flow = tflow.tflow( + req=tutils.treq(path=b"index.html?q=1"), + resp=tutils.tresp(content=b'') + ) + xss.response(mocked_flow) + assert logger.args == [] + + def test_data_equals(self): + xssData = xss.XSSData("a", "b", "c", "d") + sqliData = xss.SQLiData("a", "b", "c", "d") + assert xssData == xssData + assert sqliData == sqliData diff -Nru mitmproxy-5.1.1/docs/src/examples/contrib/tls_passthrough.py mitmproxy-6.0.2/docs/src/examples/contrib/tls_passthrough.py --- mitmproxy-5.1.1/docs/src/examples/contrib/tls_passthrough.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/contrib/tls_passthrough.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,146 @@ +""" +This inline script allows conditional TLS Interception based +on a user-defined strategy. + +Example: + + > mitmdump -s tls_passthrough.py + + 1. curl --proxy http://localhost:8080 https://example.com --insecure + // works - we'll also see the contents in mitmproxy + + 2. curl --proxy http://localhost:8080 https://example.com --insecure + // still works - we'll also see the contents in mitmproxy + + 3. curl --proxy http://localhost:8080 https://example.com + // fails with a certificate error, which we will also see in mitmproxy + + 4. curl --proxy http://localhost:8080 https://example.com + // works again, but mitmproxy does not intercept and we do *not* see the contents + +Authors: Maximilian Hils, Matthew Tuusberg +""" +import collections +import random + +from enum import Enum + +import mitmproxy +from mitmproxy import ctx +from mitmproxy.exceptions import TlsProtocolException +from mitmproxy.proxy.protocol import TlsLayer, RawTCPLayer + + +class InterceptionResult(Enum): + success = True + failure = False + skipped = None + + +class _TlsStrategy: + """ + Abstract base class for interception strategies. + """ + + def __init__(self): + # A server_address -> interception results mapping + self.history = collections.defaultdict(lambda: collections.deque(maxlen=200)) + + def should_intercept(self, server_address): + """ + Returns: + True, if we should attempt to intercept the connection. + False, if we want to employ pass-through instead. + """ + raise NotImplementedError() + + def record_success(self, server_address): + self.history[server_address].append(InterceptionResult.success) + + def record_failure(self, server_address): + self.history[server_address].append(InterceptionResult.failure) + + def record_skipped(self, server_address): + self.history[server_address].append(InterceptionResult.skipped) + + +class ConservativeStrategy(_TlsStrategy): + """ + Conservative Interception Strategy - only intercept if there haven't been any failed attempts + in the history. + """ + + def should_intercept(self, server_address): + if InterceptionResult.failure in self.history[server_address]: + return False + return True + + +class ProbabilisticStrategy(_TlsStrategy): + """ + Fixed probability that we intercept a given connection. + """ + + def __init__(self, p): + self.p = p + super().__init__() + + def should_intercept(self, server_address): + return random.uniform(0, 1) < self.p + + +class TlsFeedback(TlsLayer): + """ + Monkey-patch _establish_tls_with_client to get feedback if TLS could be established + successfully on the client connection (which may fail due to cert pinning). + """ + + def _establish_tls_with_client(self): + server_address = self.server_conn.address + + try: + super()._establish_tls_with_client() + except TlsProtocolException as e: + tls_strategy.record_failure(server_address) + raise e + else: + tls_strategy.record_success(server_address) + + +# inline script hooks below. + +tls_strategy = None + + +def load(l): + l.add_option( + "tlsstrat", int, 0, "TLS passthrough strategy (0-100)", + ) + + +def configure(updated): + global tls_strategy + if ctx.options.tlsstrat > 0: + tls_strategy = ProbabilisticStrategy(float(ctx.options.tlsstrat) / 100.0) + else: + tls_strategy = ConservativeStrategy() + + +def next_layer(next_layer): + """ + This hook does the actual magic - if the next layer is planned to be a TLS layer, + we check if we want to enter pass-through mode instead. + """ + if isinstance(next_layer, TlsLayer) and next_layer._client_tls: + server_address = next_layer.server_conn.address + + if tls_strategy.should_intercept(server_address): + # We try to intercept. + # Monkey-Patch the layer to get feedback from the TLSLayer if interception worked. + next_layer.__class__ = TlsFeedback + else: + # We don't intercept - reply with a pass-through layer and add a "skipped" entry. + mitmproxy.ctx.log("TLS passthrough for %s" % repr(next_layer.server_conn.address), "info") + next_layer_replacement = RawTCPLayer(next_layer.ctx, ignore=True) + next_layer.reply.send(next_layer_replacement) + tls_strategy.record_skipped(server_address) diff -Nru mitmproxy-5.1.1/docs/src/examples/contrib/webscanner_helper/mapping.py mitmproxy-6.0.2/docs/src/examples/contrib/webscanner_helper/mapping.py --- mitmproxy-5.1.1/docs/src/examples/contrib/webscanner_helper/mapping.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/contrib/webscanner_helper/mapping.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,144 @@ +import copy +import logging +import typing +from typing import Dict + +from bs4 import BeautifulSoup + +from mitmproxy.http import HTTPFlow +from examples.contrib.webscanner_helper.urldict import URLDict + +NO_CONTENT = object() + + +class MappingAddonConfig: + HTML_PARSER = "html.parser" + + +class MappingAddon: + """ The mapping add-on can be used in combination with web application scanners to reduce their false positives. + + Many web application scanners produce false positives caused by dynamically changing content of web applications + such as the current time or current measurements. When testing for injection vulnerabilities, web application + scanners are tricked into thinking they changed the content with the injected payload. In realty, the content of + the web application changed notwithstanding the scanner's input. When the mapping add-on is used to map the content + to a fixed value, these false positives can be avoided. + """ + + OPT_MAPPING_FILE = "mapping_file" + """File where urls and css selector to mapped content is stored. + + Elements will be replaced with the content given in this file. If the content is none it will be set to the first + seen value. + + Example: + + { + "http://10.10.10.10": { + "body": "My Text" + }, + "URL": { + "css selector": "Replace with this" + } + } + """ + + OPT_MAP_PERSISTENT = "map_persistent" + """Whether to store all new content in the configuration file.""" + + def __init__(self, filename: str, persistent: bool = False) -> None: + """ Initializes the mapping add-on + + Args: + filename: str that provides the name of the file in which the urls and css selectors to mapped content is + stored. + persistent: bool that indicates whether to store all new content in the configuration file. + + Example: + The file in which the mapping config is given should be in the following format: + { + "http://10.10.10.10": { + "body": "My Text" + }, + "": { + "": "Replace with this" + } + } + """ + self.filename = filename + self.persistent = persistent + self.logger = logging.getLogger(self.__class__.__name__) + with open(filename) as f: + self.mapping_templates = URLDict.load(f) + + def load(self, loader): + loader.add_option( + self.OPT_MAPPING_FILE, str, "", + "File where replacement configuration is stored." + ) + loader.add_option( + self.OPT_MAP_PERSISTENT, bool, False, + "Whether to store all new content in the configuration file." + ) + + def configure(self, updated): + if self.OPT_MAPPING_FILE in updated: + self.filename = updated[self.OPT_MAPPING_FILE] + with open(self.filename) as f: + self.mapping_templates = URLDict.load(f) + + if self.OPT_MAP_PERSISTENT in updated: + self.persistent = updated[self.OPT_MAP_PERSISTENT] + + def replace(self, soup: BeautifulSoup, css_sel: str, replace: BeautifulSoup) -> None: + """Replaces the content of soup that matches the css selector with the given replace content.""" + for content in soup.select(css_sel): + self.logger.debug(f"replace \"{content}\" with \"{replace}\"") + content.replace_with(copy.copy(replace)) + + def apply_template(self, soup: BeautifulSoup, template: Dict[str, typing.Union[BeautifulSoup]]) -> None: + """Applies the given mapping template to the given soup.""" + for css_sel, replace in template.items(): + mapped = soup.select(css_sel) + if not mapped: + self.logger.warning(f"Could not find \"{css_sel}\", can not freeze anything.") + else: + self.replace(soup, css_sel, BeautifulSoup(replace, features=MappingAddonConfig.HTML_PARSER)) + + def response(self, flow: HTTPFlow) -> None: + """If a response is received, check if we should replace some content. """ + try: + templates = self.mapping_templates[flow] + res = flow.response + if res is not None: + encoding = res.headers.get("content-encoding", "utf-8") + content_type = res.headers.get("content-type", "text/html") + + if "text/html" in content_type and encoding == "utf-8": + content = BeautifulSoup(res.content, MappingAddonConfig.HTML_PARSER) + for template in templates: + self.apply_template(content, template) + res.content = content.encode(encoding) + else: + self.logger.warning(f"Unsupported content type '{content_type}' or content encoding '{encoding}'") + except KeyError: + pass + + def done(self) -> None: + """Dumps all new content into the configuration file if self.persistent is set.""" + if self.persistent: + + # make sure that all items are strings and not soups. + def value_dumper(value): + store = {} + if value is None: + return "None" + try: + for css_sel, soup in value.items(): + store[css_sel] = str(soup) + except: + raise RuntimeError(value) + return store + + with open(self.filename, "w") as f: + self.mapping_templates.dump(f, value_dumper) diff -Nru mitmproxy-5.1.1/docs/src/examples/contrib/webscanner_helper/proxyauth_selenium.py mitmproxy-6.0.2/docs/src/examples/contrib/webscanner_helper/proxyauth_selenium.py --- mitmproxy-5.1.1/docs/src/examples/contrib/webscanner_helper/proxyauth_selenium.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/contrib/webscanner_helper/proxyauth_selenium.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,132 @@ +import abc +import logging +import random +import string +import time +from typing import Dict, List, cast, Any + +import mitmproxy.http +from mitmproxy import flowfilter +from mitmproxy import master +from mitmproxy.script import concurrent +from selenium import webdriver + +logger = logging.getLogger(__name__) + +cookie_key_name = { + "path": "Path", + "expires": "Expires", + "domain": "Domain", + "is_http_only": "HttpOnly", + "is_secure": "Secure" +} + + +def randomString(string_length=10): + """Generate a random string of fixed length """ + letters = string.ascii_lowercase + return ''.join(random.choice(letters) for i in range(string_length)) + + +class AuthorizationOracle(abc.ABC): + """Abstract class for an authorization oracle which decides if a given request or response is authenticated.""" + + @abc.abstractmethod + def is_unauthorized_request(self, flow: mitmproxy.http.HTTPFlow) -> bool: + pass + + @abc.abstractmethod + def is_unauthorized_response(self, flow: mitmproxy.http.HTTPFlow) -> bool: + pass + + +class SeleniumAddon: + """ This Addon can be used in combination with web application scanners in order to help them to authenticate + against a web application. + + Since the authentication is highly dependant on the web application, this add-on includes the abstract method + *login*. In order to use the add-on, a class for the web application inheriting from SeleniumAddon needs to be + created. This class needs to include the concrete selenium actions necessary to authenticate against the web + application. In addition, an authentication oracle which inherits from AuthorizationOracle should be created. + """ + + def __init__(self, fltr: str, domain: str, + auth_oracle: AuthorizationOracle): + self.filter = flowfilter.parse(fltr) + self.auth_oracle = auth_oracle + self.domain = domain + self.browser = None + self.set_cookies = False + + options = webdriver.FirefoxOptions() + options.headless = True + + profile = webdriver.FirefoxProfile() + profile.set_preference('network.proxy.type', 0) + self.browser = webdriver.Firefox(firefox_profile=profile, + options=options) + self.cookies: List[Dict[str, str]] = [] + + def _login(self, flow): + self.cookies = self.login(flow) + self.browser.get("about:blank") + self._set_request_cookies(flow) + self.set_cookies = True + + def request(self, flow: mitmproxy.http.HTTPFlow): + if flow.request.is_replay: + logger.warning("Caught replayed request: " + str(flow)) + if (not self.filter or self.filter(flow)) and self.auth_oracle.is_unauthorized_request(flow): + logger.debug("unauthorized request detected, perform login") + self._login(flow) + + # has to be concurrent because replay.client is blocking and replayed flows + # will also call response + @concurrent + def response(self, flow: mitmproxy.http.HTTPFlow): + if flow.response and (self.filter is None or self.filter(flow)): + if self.auth_oracle.is_unauthorized_response(flow): + self._login(flow) + new_flow = flow.copy() + if master and hasattr(master, 'commands'): + # cast necessary for mypy + cast(Any, master).commands.call("replay.client", [new_flow]) + count = 0 + while new_flow.response is None and count < 10: + logger.error("waiting since " + str(count) + " ...") + count = count + 1 + time.sleep(1) + if new_flow.response: + flow.response = new_flow.response + else: + logger.warning("Could not call 'replay.client' command since master was not initialized yet.") + + if self.set_cookies and flow.response: + logger.debug("set set-cookie header for response") + self._set_set_cookie_headers(flow) + self.set_cookies = False + + def done(self): + self.browser.close() + + def _set_set_cookie_headers(self, flow: mitmproxy.http.HTTPFlow): + if flow.response and self.cookies: + for cookie in self.cookies: + parts = [f"{cookie['name']}={cookie['value']}"] + for k, v in cookie_key_name.items(): + if k in cookie and isinstance(cookie[k], str): + parts.append(f"{v}={cookie[k]}") + elif k in cookie and isinstance(cookie[k], bool) and cookie[k]: + parts.append(cookie[k]) + encoded_c = "; ".join(parts) + flow.response.headers["set-cookie"] = encoded_c + + def _set_request_cookies(self, flow: mitmproxy.http.HTTPFlow): + if self.cookies: + cookies = "; ".join( + map(lambda c: f"{c['name']}={c['value']}", self.cookies)) + flow.request.headers["cookie"] = cookies + + @abc.abstractmethod + def login(self, flow: mitmproxy.http.HTTPFlow) -> List[Dict[str, str]]: + pass diff -Nru mitmproxy-5.1.1/docs/src/examples/contrib/webscanner_helper/test_mapping.py mitmproxy-6.0.2/docs/src/examples/contrib/webscanner_helper/test_mapping.py --- mitmproxy-5.1.1/docs/src/examples/contrib/webscanner_helper/test_mapping.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/contrib/webscanner_helper/test_mapping.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,165 @@ +from typing import TextIO, Callable +from unittest import mock +from unittest.mock import MagicMock + +from mitmproxy.test import tflow +from mitmproxy.test import tutils + +from examples.contrib.webscanner_helper.mapping import MappingAddon, MappingAddonConfig + + +class TestConfig: + + def test_config(self): + assert MappingAddonConfig.HTML_PARSER == "html.parser" + + +url = "http://10.10.10.10" +new_content = "My Text" +mapping_content = f'{{"{url}": {{"body": "{new_content}"}}}}' + + +class TestMappingAddon: + + def test_init(self, tmpdir): + tmpfile = tmpdir.join("tmpfile") + with open(tmpfile, "w") as tfile: + tfile.write(mapping_content) + mapping = MappingAddon(tmpfile) + assert "My Text" in str(mapping.mapping_templates._dump()) + + def test_load(self, tmpdir): + tmpfile = tmpdir.join("tmpfile") + with open(tmpfile, "w") as tfile: + tfile.write(mapping_content) + mapping = MappingAddon(tmpfile) + loader = MagicMock() + + mapping.load(loader) + assert 'mapping_file' in str(loader.add_option.call_args_list) + assert 'map_persistent' in str(loader.add_option.call_args_list) + + def test_configure(self, tmpdir): + tmpfile = tmpdir.join("tmpfile") + with open(tmpfile, "w") as tfile: + tfile.write(mapping_content) + mapping = MappingAddon(tmpfile) + new_filename = "My new filename" + updated = {str(mapping.OPT_MAPPING_FILE): new_filename, str(mapping.OPT_MAP_PERSISTENT): True} + + open_mock = mock.mock_open(read_data="{}") + with mock.patch("builtins.open", open_mock): + mapping.configure(updated) + assert new_filename in str(open_mock.mock_calls) + assert mapping.filename == new_filename + assert mapping.persistent + + def test_response_filtered(self, tmpdir): + tmpfile = tmpdir.join("tmpfile") + with open(tmpfile, "w") as tfile: + tfile.write(mapping_content) + mapping = MappingAddon(tmpfile) + f = tflow.tflow(resp=tutils.tresp()) + test_content = b"Test" + f.response.content = test_content + + mapping.response(f) + assert f.response.content == test_content + + def test_response(self, tmpdir): + tmpfile = tmpdir.join("tmpfile") + with open(tmpfile, "w") as tfile: + tfile.write(mapping_content) + mapping = MappingAddon(tmpfile) + f = tflow.tflow(resp=tutils.tresp()) + test_content = b" Test " + f.response.content = test_content + f.request.url = url + + mapping.response(f) + assert f.response.content.decode("utf-8") == new_content + + def test_response_content_type(self, tmpdir): + tmpfile = tmpdir.join("tmpfile") + with open(tmpfile, "w") as tfile: + tfile.write(mapping_content) + mapping = MappingAddon(tmpfile) + f = tflow.tflow(resp=tutils.tresp()) + test_content = b" Test " + f.response.content = test_content + f.request.url = url + f.response.headers.add("content-type", "content-type") + + mapping.response(f) + assert f.response.content == test_content + + def test_response_not_existing(self, tmpdir): + tmpfile = tmpdir.join("tmpfile") + with open(tmpfile, "w") as tfile: + tfile.write(mapping_content) + mapping = MappingAddon(tmpfile) + f = tflow.tflow(resp=tutils.tresp()) + test_content = b" Test " + f.response.content = test_content + f.request.url = url + mapping.response(f) + assert f.response.content == test_content + + def test_persistance_false(self, tmpdir): + tmpfile = tmpdir.join("tmpfile") + with open(tmpfile, "w") as tfile: + tfile.write(mapping_content) + mapping = MappingAddon(tmpfile) + + open_mock = mock.mock_open(read_data="{}") + with mock.patch("builtins.open", open_mock): + mapping.done() + assert len(open_mock.mock_calls) == 0 + + def test_persistance_true(self, tmpdir): + tmpfile = tmpdir.join("tmpfile") + with open(tmpfile, "w") as tfile: + tfile.write(mapping_content) + mapping = MappingAddon(tmpfile, persistent=True) + + open_mock = mock.mock_open(read_data="{}") + with mock.patch("builtins.open", open_mock): + mapping.done() + with open(tmpfile) as tfile: + results = tfile.read() + assert len(open_mock.mock_calls) != 0 + assert results == mapping_content + + def test_persistance_true_add_content(self, tmpdir): + tmpfile = tmpdir.join("tmpfile") + with open(tmpfile, "w") as tfile: + tfile.write(mapping_content) + mapping = MappingAddon(tmpfile, persistent=True) + + f = tflow.tflow(resp=tutils.tresp()) + test_content = b" Test " + f.response.content = test_content + f.request.url = url + + mapping.response(f) + mapping.done() + with open(tmpfile) as tfile: + results = tfile.read() + assert mapping_content in results + + def mock_dump(self, f: TextIO, value_dumper: Callable): + assert value_dumper(None) == "None" + try: + value_dumper("Test") + except RuntimeError: + assert True + else: + assert False + + def test_dump(selfself, tmpdir): + tmpfile = tmpdir.join("tmpfile") + with open(tmpfile, "w") as tfile: + tfile.write("{}") + mapping = MappingAddon(tmpfile, persistent=True) + with mock.patch('examples.complex.webscanner_helper.urldict.URLDict.dump', selfself.mock_dump): + mapping.done() diff -Nru mitmproxy-5.1.1/docs/src/examples/contrib/webscanner_helper/test_proxyauth_selenium.py mitmproxy-6.0.2/docs/src/examples/contrib/webscanner_helper/test_proxyauth_selenium.py --- mitmproxy-5.1.1/docs/src/examples/contrib/webscanner_helper/test_proxyauth_selenium.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/contrib/webscanner_helper/test_proxyauth_selenium.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,120 @@ +from unittest import mock +from unittest.mock import MagicMock + +import pytest + +from mitmproxy.test import tflow +from mitmproxy.test import tutils +from mitmproxy.http import HTTPFlow + +from examples.contrib.webscanner_helper.proxyauth_selenium import logger, randomString, AuthorizationOracle, \ + SeleniumAddon + + +class TestRandomString: + + def test_random_string(self): + res = randomString() + assert isinstance(res, str) + assert len(res) == 10 + + res_5 = randomString(5) + assert isinstance(res_5, str) + assert len(res_5) == 5 + + +class AuthenticationOracleTest(AuthorizationOracle): + def is_unauthorized_request(self, flow: HTTPFlow) -> bool: + return True + + def is_unauthorized_response(self, flow: HTTPFlow) -> bool: + return True + + +oracle = AuthenticationOracleTest() + + +@pytest.fixture(scope="module", autouse=True) +def selenium_addon(request): + addon = SeleniumAddon(fltr=r"~u http://example\.com/login\.php", domain=r"~d http://example\.com", + auth_oracle=oracle) + browser = MagicMock() + addon.browser = browser + yield addon + + def fin(): + addon.browser.close() + + request.addfinalizer(fin) + + +class TestSeleniumAddon: + + def test_request_replay(self, selenium_addon): + f = tflow.tflow(resp=tutils.tresp()) + f.request.is_replay = True + with mock.patch.object(logger, 'warning') as mock_warning: + selenium_addon.request(f) + mock_warning.assert_called() + + def test_request(self, selenium_addon): + f = tflow.tflow(resp=tutils.tresp()) + f.request.url = "http://example.com/login.php" + selenium_addon.set_cookies = False + assert not selenium_addon.set_cookies + with mock.patch.object(logger, 'debug') as mock_debug: + selenium_addon.request(f) + mock_debug.assert_called() + assert selenium_addon.set_cookies + + def test_request_filtered(self, selenium_addon): + f = tflow.tflow(resp=tutils.tresp()) + selenium_addon.set_cookies = False + assert not selenium_addon.set_cookies + selenium_addon.request(f) + assert not selenium_addon.set_cookies + + def test_request_cookies(self, selenium_addon): + f = tflow.tflow(resp=tutils.tresp()) + f.request.url = "http://example.com/login.php" + selenium_addon.set_cookies = False + assert not selenium_addon.set_cookies + with mock.patch.object(logger, 'debug') as mock_debug: + with mock.patch('examples.complex.webscanner_helper.proxyauth_selenium.SeleniumAddon.login', + return_value=[{"name": "cookie", "value": "test"}]) as mock_login: + selenium_addon.request(f) + mock_debug.assert_called() + assert selenium_addon.set_cookies + mock_login.assert_called() + + def test_request_filter_None(self, selenium_addon): + f = tflow.tflow(resp=tutils.tresp()) + fltr = selenium_addon.filter + selenium_addon.filter = None + assert not selenium_addon.filter + selenium_addon.set_cookies = False + assert not selenium_addon.set_cookies + + with mock.patch.object(logger, 'debug') as mock_debug: + selenium_addon.request(f) + mock_debug.assert_called() + selenium_addon.filter = fltr + assert selenium_addon.set_cookies + + def test_response(self, selenium_addon): + f = tflow.tflow(resp=tutils.tresp()) + f.request.url = "http://example.com/login.php" + selenium_addon.set_cookies = False + with mock.patch('examples.complex.webscanner_helper.proxyauth_selenium.SeleniumAddon.login', + return_value=[]) as mock_login: + selenium_addon.response(f) + mock_login.assert_called() + + def test_response_cookies(self, selenium_addon): + f = tflow.tflow(resp=tutils.tresp()) + f.request.url = "http://example.com/login.php" + selenium_addon.set_cookies = False + with mock.patch('examples.complex.webscanner_helper.proxyauth_selenium.SeleniumAddon.login', + return_value=[{"name": "cookie", "value": "test"}]) as mock_login: + selenium_addon.response(f) + mock_login.assert_called() diff -Nru mitmproxy-5.1.1/docs/src/examples/contrib/webscanner_helper/test_urldict.py mitmproxy-6.0.2/docs/src/examples/contrib/webscanner_helper/test_urldict.py --- mitmproxy-5.1.1/docs/src/examples/contrib/webscanner_helper/test_urldict.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/contrib/webscanner_helper/test_urldict.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,89 @@ +from mitmproxy.test import tflow, tutils +from examples.contrib.webscanner_helper.urldict import URLDict + +url = "http://10.10.10.10" +new_content_body = "New Body" +new_content_title = "New Title" +content = f'{{"body": "{new_content_body}", "title": "{new_content_title}"}}' +url_error = "i~nvalid" +input_file_content = f'{{"{url}": {content}}}' +input_file_content_error = f'{{"{url_error}": {content}}}' + + +class TestUrlDict: + + def test_urldict_empty(self): + urldict = URLDict() + dump = urldict.dumps() + assert dump == '{}' + + def test_urldict_loads(self): + urldict = URLDict.loads(input_file_content) + dump = urldict.dumps() + assert dump == input_file_content + + def test_urldict_set_error(self, tmpdir): + tmpfile = tmpdir.join("tmpfile") + with open(tmpfile, "w") as tfile: + tfile.write(input_file_content_error) + with open(tmpfile) as tfile: + try: + URLDict.load(tfile) + except ValueError: + assert True + else: + assert False + + def test_urldict_get(self, tmpdir): + tmpfile = tmpdir.join("tmpfile") + with open(tmpfile, "w") as tfile: + tfile.write(input_file_content) + with open(tmpfile) as tfile: + urldict = URLDict.load(tfile) + + f = tflow.tflow(resp=tutils.tresp()) + f.request.url = url + selection = urldict[f] + assert "body" in selection[0] + assert new_content_body in selection[0]["body"] + assert "title" in selection[0] + assert new_content_title in selection[0]["title"] + + selection_get = urldict.get(f) + assert "body" in selection_get[0] + assert new_content_body in selection_get[0]["body"] + assert "title" in selection_get[0] + assert new_content_title in selection_get[0]["title"] + + try: + urldict["body"] + except KeyError: + assert True + else: + assert False + + assert urldict.get("body", default="default") == "default" + + def test_urldict_dumps(self, tmpdir): + tmpfile = tmpdir.join("tmpfile") + with open(tmpfile, "w") as tfile: + tfile.write(input_file_content) + with open(tmpfile) as tfile: + urldict = URLDict.load(tfile) + + dump = urldict.dumps() + assert dump == input_file_content + + def test_urldict_dump(self, tmpdir): + tmpfile = tmpdir.join("tmpfile") + outfile = tmpdir.join("outfile") + with open(tmpfile, "w") as tfile: + tfile.write(input_file_content) + with open(tmpfile) as tfile: + urldict = URLDict.load(tfile) + with open(outfile, "w") as ofile: + urldict.dump(ofile) + + with open(outfile) as ofile: + output = ofile.read() + assert output == input_file_content diff -Nru mitmproxy-5.1.1/docs/src/examples/contrib/webscanner_helper/test_urlindex.py mitmproxy-6.0.2/docs/src/examples/contrib/webscanner_helper/test_urlindex.py --- mitmproxy-5.1.1/docs/src/examples/contrib/webscanner_helper/test_urlindex.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/contrib/webscanner_helper/test_urlindex.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,235 @@ +import json +from json import JSONDecodeError +from pathlib import Path +from unittest import mock +from typing import List +from unittest.mock import patch + +from mitmproxy.test import tflow +from mitmproxy.test import tutils + +from examples.contrib.webscanner_helper.urlindex import UrlIndexWriter, SetEncoder, JSONUrlIndexWriter, \ + TextUrlIndexWriter, WRITER, \ + filter_404, \ + UrlIndexAddon + + +class TestBaseClass: + + @patch.multiple(UrlIndexWriter, __abstractmethods__=set()) + def test_base_class(self, tmpdir): + tmpfile = tmpdir.join("tmpfile") + index_writer = UrlIndexWriter(tmpfile) + index_writer.load() + index_writer.add_url(tflow.tflow()) + index_writer.save() + + +class TestSetEncoder: + + def test_set_encoder_set(self): + test_set = {"foo", "bar", "42"} + result = SetEncoder.default(SetEncoder(), test_set) + assert isinstance(result, List) + assert 'foo' in result + assert 'bar' in result + assert '42' in result + + def test_set_encoder_str(self): + test_str = "test" + try: + SetEncoder.default(SetEncoder(), test_str) + except TypeError: + assert True + else: + assert False + + +class TestJSONUrlIndexWriter: + + def test_load(self, tmpdir): + tmpfile = tmpdir.join("tmpfile") + with open(tmpfile, "w") as tfile: + tfile.write( + "{\"http://example.com:80\": {\"/\": {\"GET\": [301]}}, \"http://www.example.com:80\": {\"/\": {\"GET\": [302]}}}") + writer = JSONUrlIndexWriter(filename=tmpfile) + writer.load() + assert 'http://example.com:80' in writer.host_urls + assert '/' in writer.host_urls['http://example.com:80'] + assert 'GET' in writer.host_urls['http://example.com:80']['/'] + assert 301 in writer.host_urls['http://example.com:80']['/']['GET'] + + def test_load_empty(self, tmpdir): + tmpfile = tmpdir.join("tmpfile") + with open(tmpfile, "w") as tfile: + tfile.write("{}") + writer = JSONUrlIndexWriter(filename=tmpfile) + writer.load() + assert len(writer.host_urls) == 0 + + def test_load_nonexisting(self, tmpdir): + tmpfile = tmpdir.join("tmpfile") + writer = JSONUrlIndexWriter(filename=tmpfile) + writer.load() + assert len(writer.host_urls) == 0 + + def test_add(self, tmpdir): + tmpfile = tmpdir.join("tmpfile") + writer = JSONUrlIndexWriter(filename=tmpfile) + f = tflow.tflow(resp=tutils.tresp()) + url = f"{f.request.scheme}://{f.request.host}:{f.request.port}" + writer.add_url(f) + assert url in writer.host_urls + assert f.request.path in writer.host_urls[url] + + def test_save(self, tmpdir): + tmpfile = tmpdir.join("tmpfile") + writer = JSONUrlIndexWriter(filename=tmpfile) + f = tflow.tflow(resp=tutils.tresp()) + url = f"{f.request.scheme}://{f.request.host}:{f.request.port}" + writer.add_url(f) + writer.save() + + with open(tmpfile) as results: + try: + content = json.load(results) + except JSONDecodeError: + assert False + assert url in content + + +class TestTestUrlIndexWriter: + def test_load(self, tmpdir): + tmpfile = tmpdir.join("tmpfile") + with open(tmpfile, "w") as tfile: + tfile.write( + "2020-04-22T05:41:08.679231 STATUS: 200 METHOD: GET URL:http://example.com") + writer = TextUrlIndexWriter(filename=tmpfile) + writer.load() + assert True + + def test_load_empty(self, tmpdir): + tmpfile = tmpdir.join("tmpfile") + with open(tmpfile, "w") as tfile: + tfile.write("{}") + writer = TextUrlIndexWriter(filename=tmpfile) + writer.load() + assert True + + def test_load_nonexisting(self, tmpdir): + tmpfile = tmpdir.join("tmpfile") + writer = TextUrlIndexWriter(filename=tmpfile) + writer.load() + assert True + + def test_add(self, tmpdir): + tmpfile = tmpdir.join("tmpfile") + writer = TextUrlIndexWriter(filename=tmpfile) + f = tflow.tflow(resp=tutils.tresp()) + url = f"{f.request.scheme}://{f.request.host}:{f.request.port}" + method = f.request.method + code = f.response.status_code + writer.add_url(f) + + with open(tmpfile) as results: + content = results.read() + assert url in content + assert method in content + assert str(code) in content + + def test_save(self, tmpdir): + tmpfile = tmpdir.join("tmpfile") + writer = TextUrlIndexWriter(filename=tmpfile) + f = tflow.tflow(resp=tutils.tresp()) + url = f"{f.request.scheme}://{f.request.host}:{f.request.port}" + method = f.request.method + code = f.response.status_code + writer.add_url(f) + writer.save() + + with open(tmpfile) as results: + content = results.read() + assert url in content + assert method in content + assert str(code) in content + + +class TestWriter: + def test_writer_dict(self): + assert "json" in WRITER + assert isinstance(WRITER["json"], JSONUrlIndexWriter.__class__) + assert "text" in WRITER + assert isinstance(WRITER["text"], TextUrlIndexWriter.__class__) + + +class TestFilter: + def test_filer_true(self): + f = tflow.tflow(resp=tutils.tresp()) + assert filter_404(f) + + def test_filter_false(self): + f = tflow.tflow(resp=tutils.tresp()) + f.response.status_code = 404 + assert not filter_404(f) + + +class TestUrlIndexAddon: + + def test_init(self, tmpdir): + tmpfile = tmpdir.join("tmpfile") + UrlIndexAddon(tmpfile) + + def test_init_format(self, tmpdir): + tmpfile = tmpdir.join("tmpfile") + try: + UrlIndexAddon(tmpfile, index_format="test") + except ValueError: + assert True + else: + assert False + + def test_init_filter(self, tmpdir): + tmpfile = tmpdir.join("tmpfile") + try: + UrlIndexAddon(tmpfile, index_filter="i~nvalid") + except ValueError: + assert True + else: + assert False + + def test_init_append(self, tmpdir): + tmpfile = tmpdir.join("tmpfile") + with open(tmpfile, "w") as tfile: + tfile.write("") + url_index = UrlIndexAddon(tmpfile, append=False) + f = tflow.tflow(resp=tutils.tresp()) + with mock.patch('examples.complex.webscanner_helper.urlindex.JSONUrlIndexWriter.add_url'): + url_index.response(f) + assert not Path(tmpfile).exists() + + def test_response(self, tmpdir): + tmpfile = tmpdir.join("tmpfile") + url_index = UrlIndexAddon(tmpfile) + f = tflow.tflow(resp=tutils.tresp()) + with mock.patch('examples.complex.webscanner_helper.urlindex.JSONUrlIndexWriter.add_url') as mock_add_url: + url_index.response(f) + mock_add_url.assert_called() + + def test_response_None(self, tmpdir): + tmpfile = tmpdir.join("tmpfile") + url_index = UrlIndexAddon(tmpfile) + url_index.index_filter = None + f = tflow.tflow(resp=tutils.tresp()) + try: + url_index.response(f) + except ValueError: + assert True + else: + assert False + + def test_done(self, tmpdir): + tmpfile = tmpdir.join("tmpfile") + url_index = UrlIndexAddon(tmpfile) + with mock.patch('examples.complex.webscanner_helper.urlindex.JSONUrlIndexWriter.save') as mock_save: + url_index.done() + mock_save.assert_called() diff -Nru mitmproxy-5.1.1/docs/src/examples/contrib/webscanner_helper/test_urlinjection.py mitmproxy-6.0.2/docs/src/examples/contrib/webscanner_helper/test_urlinjection.py --- mitmproxy-5.1.1/docs/src/examples/contrib/webscanner_helper/test_urlinjection.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/contrib/webscanner_helper/test_urlinjection.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,112 @@ +import json +from unittest import mock + +from mitmproxy import flowfilter +from mitmproxy.test import tflow +from mitmproxy.test import tutils + +from examples.contrib.webscanner_helper.urlinjection import InjectionGenerator, HTMLInjection, RobotsInjection, \ + SitemapInjection, \ + UrlInjectionAddon, logger + +index = json.loads( + "{\"http://example.com:80\": {\"/\": {\"GET\": [301]}}, \"http://www.example.com:80\": {\"/test\": {\"POST\": [302]}}}") + + +class TestInjectionGenerator: + + def test_inject(self): + f = tflow.tflow(resp=tutils.tresp()) + injection_generator = InjectionGenerator() + injection_generator.inject(index=index, flow=f) + assert True + + +class TestHTMLInjection: + + def test_inject_not404(self): + html_injection = HTMLInjection() + f = tflow.tflow(resp=tutils.tresp()) + + with mock.patch.object(logger, 'warning') as mock_warning: + html_injection.inject(index, f) + assert mock_warning.called + + def test_inject_insert(self): + html_injection = HTMLInjection(insert=True) + f = tflow.tflow(resp=tutils.tresp()) + assert "example.com" not in str(f.response.content) + html_injection.inject(index, f) + assert "example.com" in str(f.response.content) + + def test_inject_insert_body(self): + html_injection = HTMLInjection(insert=True) + f = tflow.tflow(resp=tutils.tresp()) + f.response.text = "" + assert "example.com" not in str(f.response.content) + html_injection.inject(index, f) + assert "example.com" in str(f.response.content) + + def test_inject_404(self): + html_injection = HTMLInjection() + f = tflow.tflow(resp=tutils.tresp()) + f.response.status_code = 404 + assert "example.com" not in str(f.response.content) + html_injection.inject(index, f) + assert "example.com" in str(f.response.content) + + +class TestRobotsInjection: + + def test_inject_not404(self): + robots_injection = RobotsInjection() + f = tflow.tflow(resp=tutils.tresp()) + + with mock.patch.object(logger, 'warning') as mock_warning: + robots_injection.inject(index, f) + assert mock_warning.called + + def test_inject_404(self): + robots_injection = RobotsInjection() + f = tflow.tflow(resp=tutils.tresp()) + f.response.status_code = 404 + assert "Allow: /test" not in str(f.response.content) + robots_injection.inject(index, f) + assert "Allow: /test" in str(f.response.content) + + +class TestSitemapInjection: + + def test_inject_not404(self): + sitemap_injection = SitemapInjection() + f = tflow.tflow(resp=tutils.tresp()) + + with mock.patch.object(logger, 'warning') as mock_warning: + sitemap_injection.inject(index, f) + assert mock_warning.called + + def test_inject_404(self): + sitemap_injection = SitemapInjection() + f = tflow.tflow(resp=tutils.tresp()) + f.response.status_code = 404 + assert "http://example.com:80/" not in str(f.response.content) + sitemap_injection.inject(index, f) + assert "http://example.com:80/" in str(f.response.content) + + +class TestUrlInjectionAddon: + + def test_init(self, tmpdir): + tmpfile = tmpdir.join("tmpfile") + with open(tmpfile, "w") as tfile: + json.dump(index, tfile) + flt = f"~u .*/site.html$" + url_injection = UrlInjectionAddon(f"~u .*/site.html$", tmpfile, HTMLInjection(insert=True)) + assert "http://example.com:80" in url_injection.url_store + fltr = flowfilter.parse(flt) + f = tflow.tflow(resp=tutils.tresp()) + f.request.url = "http://example.com/site.html" + assert fltr(f) + assert "http://example.com:80" not in str(f.response.content) + url_injection.response(f) + assert "http://example.com:80" in str(f.response.content) diff -Nru mitmproxy-5.1.1/docs/src/examples/contrib/webscanner_helper/test_watchdog.py mitmproxy-6.0.2/docs/src/examples/contrib/webscanner_helper/test_watchdog.py --- mitmproxy-5.1.1/docs/src/examples/contrib/webscanner_helper/test_watchdog.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/contrib/webscanner_helper/test_watchdog.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,84 @@ +import time +from pathlib import Path +from unittest import mock + +from mitmproxy.connections import ServerConnection +from mitmproxy.exceptions import HttpSyntaxException +from mitmproxy.test import tflow +from mitmproxy.test import tutils +import multiprocessing + +from examples.contrib.webscanner_helper.watchdog import WatchdogAddon, logger + + +class TestWatchdog: + + def test_init_file(self, tmpdir): + tmpfile = tmpdir.join("tmpfile") + with open(tmpfile, "w") as tfile: + tfile.write("") + event = multiprocessing.Event() + try: + WatchdogAddon(event, Path(tmpfile)) + except RuntimeError: + assert True + else: + assert False + + def test_init_dir(self, tmpdir): + event = multiprocessing.Event() + mydir = tmpdir.join("mydir") + assert not Path(mydir).exists() + WatchdogAddon(event, Path(mydir)) + assert Path(mydir).exists() + + def test_serverconnect(self, tmpdir): + event = multiprocessing.Event() + w = WatchdogAddon(event, Path(tmpdir), timeout=10) + with mock.patch('mitmproxy.connections.ServerConnection.settimeout') as mock_set_timeout: + w.serverconnect(ServerConnection("127.0.0.1")) + mock_set_timeout.assert_called() + + def test_serverconnect_None(self, tmpdir): + event = multiprocessing.Event() + w = WatchdogAddon(event, Path(tmpdir)) + with mock.patch('mitmproxy.connections.ServerConnection.settimeout') as mock_set_timeout: + w.serverconnect(ServerConnection("127.0.0.1")) + assert not mock_set_timeout.called + + def test_trigger(self, tmpdir): + event = multiprocessing.Event() + w = WatchdogAddon(event, Path(tmpdir)) + f = tflow.tflow(resp=tutils.tresp()) + f.error = "Test Error" + + with mock.patch.object(logger, 'error') as mock_error: + open_mock = mock.mock_open() + with mock.patch("pathlib.Path.open", open_mock, create=True): + w.error(f) + mock_error.assert_called() + open_mock.assert_called() + + def test_trigger_http_synatx(self, tmpdir): + event = multiprocessing.Event() + w = WatchdogAddon(event, Path(tmpdir)) + f = tflow.tflow(resp=tutils.tresp()) + f.error = HttpSyntaxException() + assert isinstance(f.error, HttpSyntaxException) + + with mock.patch.object(logger, 'error') as mock_error: + open_mock = mock.mock_open() + with mock.patch("pathlib.Path.open", open_mock, create=True): + w.error(f) + assert not mock_error.called + assert not open_mock.called + + def test_timeout(self, tmpdir): + event = multiprocessing.Event() + w = WatchdogAddon(event, Path(tmpdir)) + + assert w.not_in_timeout(None, None) + assert w.not_in_timeout(time.time, None) + with mock.patch('time.time', return_value=5): + assert not w.not_in_timeout(3, 20) + assert w.not_in_timeout(3, 1) diff -Nru mitmproxy-5.1.1/docs/src/examples/contrib/webscanner_helper/urldict.py mitmproxy-6.0.2/docs/src/examples/contrib/webscanner_helper/urldict.py --- mitmproxy-5.1.1/docs/src/examples/contrib/webscanner_helper/urldict.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/contrib/webscanner_helper/urldict.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,91 @@ +import itertools +import json +import typing +from collections.abc import MutableMapping +from typing import Any, Dict, Generator, List, TextIO, Callable + +from mitmproxy import flowfilter +from mitmproxy.http import HTTPFlow + + +def f_id(x): + return x + + +class URLDict(MutableMapping): + """Data structure to store information using filters as keys.""" + + def __init__(self): + self.store: Dict[flowfilter.TFilter, Any] = {} + + def __getitem__(self, key, *, count=0): + if count: + ret = itertools.islice(self.get_generator(key), 0, count) + else: + ret = list(self.get_generator(key)) + + if ret: + return ret + else: + raise KeyError + + def __setitem__(self, key: str, value): + fltr = flowfilter.parse(key) + if fltr: + self.store.__setitem__(fltr, value) + else: + raise ValueError("Not a valid filter") + + def __delitem__(self, key): + self.store.__delitem__(key) + + def __iter__(self): + return self.store.__iter__() + + def __len__(self): + return self.store.__len__() + + def get_generator(self, flow: HTTPFlow) -> Generator[Any, None, None]: + + for fltr, value in self.store.items(): + if flowfilter.match(fltr, flow): + yield value + + def get(self, flow: HTTPFlow, default=None, *, count=0) -> List[Any]: + try: + return self.__getitem__(flow, count=count) + except KeyError: + return default + + @classmethod + def _load(cls, json_obj, value_loader: Callable = f_id): + url_dict = cls() + for fltr, value in json_obj.items(): + url_dict[fltr] = value_loader(value) + return url_dict + + @classmethod + def load(cls, f: TextIO, value_loader: Callable = f_id): + json_obj = json.load(f) + return cls._load(json_obj, value_loader) + + @classmethod + def loads(cls, json_str: str, value_loader: Callable = f_id): + json_obj = json.loads(json_str) + return cls._load(json_obj, value_loader) + + def _dump(self, value_dumper: Callable = f_id) -> Dict: + dumped: Dict[typing.Union[flowfilter.TFilter, str], Any] = {} + for fltr, value in self.store.items(): + if hasattr(fltr, 'pattern'): + # cast necessary for mypy + dumped[typing.cast(Any, fltr).pattern] = value_dumper(value) + else: + dumped[str(fltr)] = value_dumper(value) + return dumped + + def dump(self, f: TextIO, value_dumper: Callable = f_id): + json.dump(self._dump(value_dumper), f) + + def dumps(self, value_dumper: Callable = f_id): + return json.dumps(self._dump(value_dumper)) diff -Nru mitmproxy-5.1.1/docs/src/examples/contrib/webscanner_helper/urlindex.py mitmproxy-6.0.2/docs/src/examples/contrib/webscanner_helper/urlindex.py --- mitmproxy-5.1.1/docs/src/examples/contrib/webscanner_helper/urlindex.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/contrib/webscanner_helper/urlindex.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,168 @@ +import abc +import datetime +import json +import logging +from pathlib import Path +from typing import Type, Dict, Union, Optional + +from mitmproxy import flowfilter +from mitmproxy.http import HTTPFlow + +logger = logging.getLogger(__name__) + + +class UrlIndexWriter(abc.ABC): + """Abstract Add-on to write seen URLs. + + For example, these URLs can be injected in a web application to improve the crawling of web application scanners. + The injection can be done using the URLInjection Add-on. + """ + + def __init__(self, filename: Path): + """Initializes the UrlIndexWriter. + + Args: + filename: Path to file to which the URL index will be written. + """ + self.filepath = filename + + @abc.abstractmethod + def load(self): + """Load existing URL index.""" + pass + + @abc.abstractmethod + def add_url(self, flow: HTTPFlow): + """Add new URL to URL index.""" + pass + + @abc.abstractmethod + def save(self): + pass + + +class SetEncoder(json.JSONEncoder): + def default(self, obj): + if isinstance(obj, set): + return list(obj) + return json.JSONEncoder.default(self, obj) + + +class JSONUrlIndexWriter(UrlIndexWriter): + """Writes seen URLs as JSON.""" + + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + self.host_urls = {} + + def load(self): + if self.filepath.exists(): + with self.filepath.open("r") as f: + self.host_urls = json.load(f) + for host in self.host_urls.keys(): + for path, methods in self.host_urls[host].items(): + for method, codes in methods.items(): + self.host_urls[host][path] = {method: set(codes)} + + def add_url(self, flow: HTTPFlow): + req = flow.request + res = flow.response + + if req is not None and res is not None: + urls = self.host_urls.setdefault(f"{req.scheme}://{req.host}:{req.port}", dict()) + methods = urls.setdefault(req.path, {}) + codes = methods.setdefault(req.method, set()) + codes.add(res.status_code) + + def save(self): + with self.filepath.open("w") as f: + json.dump(self.host_urls, f, cls=SetEncoder) + + +class TextUrlIndexWriter(UrlIndexWriter): + """Writes seen URLs as text.""" + + def load(self): + pass + + def add_url(self, flow: HTTPFlow): + res = flow.response + req = flow.request + if res is not None and req is not None: + with self.filepath.open("a+") as f: + f.write(f"{datetime.datetime.utcnow().isoformat()} STATUS: {res.status_code} METHOD: " + f"{req.method} URL:{req.url}\n") + + def save(self): + pass + + +WRITER: Dict[str, Type[UrlIndexWriter]] = { + "json": JSONUrlIndexWriter, + "text": TextUrlIndexWriter, +} + + +def filter_404(flow) -> bool: + """Filters responses with status code 404.""" + return flow.response.status_code != 404 + + +class UrlIndexAddon: + """Add-on to write seen URLs, either as JSON or as text. + + For example, these URLs can be injected in a web application to improve the crawling of web application scanners. + The injection can be done using the URLInjection Add-on. + """ + + index_filter: Optional[Union[str, flowfilter.TFilter]] + writer: UrlIndexWriter + + OPT_FILEPATH = "URLINDEX_FILEPATH" + OPT_APPEND = "URLINDEX_APPEND" + OPT_INDEX_FILTER = "URLINDEX_FILTER" + + def __init__(self, file_path: Union[str, Path], append: bool = True, + index_filter: Union[str, flowfilter.TFilter] = filter_404, index_format: str = "json"): + """ Initializes the urlindex add-on. + + Args: + file_path: Path to file to which the URL index will be written. Can either be given as str or Path. + append: Bool to decide whether to append new URLs to the given file (as opposed to overwrite the contents + of the file) + index_filer: A mitmproxy filter with which the seen URLs will be filtered before being written. Can either + be given as str or as flowfilter.TFilter + index_format: The format of the URL index, can either be "json" or "text". + """ + + if isinstance(index_filter, str): + self.index_filter = flowfilter.parse(index_filter) + if self.index_filter is None: + raise ValueError("Invalid filter expression.") + else: + self.index_filter = index_filter + + file_path = Path(file_path) + try: + self.writer = WRITER[index_format.lower()](file_path) + except KeyError: + raise ValueError(f"Format '{index_format}' is not supported.") + + if not append and file_path.exists(): + file_path.unlink() + + self.writer.load() + + def response(self, flow: HTTPFlow): + """Checks if the response should be included in the URL based on the index_filter and adds it to the URL index + if appropriate. + """ + if isinstance(self.index_filter, str) or self.index_filter is None: + raise ValueError("Invalid filter expression.") + else: + if self.index_filter(flow): + self.writer.add_url(flow) + + def done(self): + """Writes the URL index.""" + self.writer.save() diff -Nru mitmproxy-5.1.1/docs/src/examples/contrib/webscanner_helper/urlinjection.py mitmproxy-6.0.2/docs/src/examples/contrib/webscanner_helper/urlinjection.py --- mitmproxy-5.1.1/docs/src/examples/contrib/webscanner_helper/urlinjection.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/contrib/webscanner_helper/urlinjection.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,173 @@ +import abc +import html +import json +import logging + +from mitmproxy import flowfilter +from mitmproxy.http import HTTPFlow + +logger = logging.getLogger(__name__) + + +class InjectionGenerator: + """Abstract class for an generator of the injection content in order to inject the URL index.""" + ENCODING = "UTF8" + + @abc.abstractmethod + def inject(self, index, flow: HTTPFlow): + """Injects the given URL index into the given flow.""" + pass + + +class HTMLInjection(InjectionGenerator): + """Injects the URL index either by creating a new HTML page or by appending is to an existing page.""" + + def __init__(self, insert: bool = False): + """Initializes the HTMLInjection. + + Args: + insert: boolean to decide whether to insert the URL index to an existing page (True) or to create a new + page containing the URL index. + """ + self.insert = insert + + @classmethod + def _form_html(cls, url): + return f"
" + + @classmethod + def _link_html(cls, url): + return f"link to {url}" + + @classmethod + def index_html(cls, index): + link_htmls = [] + for scheme_netloc, paths in index.items(): + for path, methods in paths.items(): + url = scheme_netloc + path + if "POST" in methods: + link_htmls.append(cls._form_html(url)) + + if "GET" in methods: + link_htmls.append(cls._link_html(url)) + return "".join(link_htmls) + + @classmethod + def landing_page(cls, index): + return ( + "" + + cls.index_html(index) + + "" + ) + + def inject(self, index, flow: HTTPFlow): + if flow.response is not None: + if flow.response.status_code != 404 and not self.insert: + logger.warning( + f"URL '{flow.request.url}' didn't return 404 status, " + f"index page would overwrite valid page.") + elif self.insert: + content = (flow.response + .content + .decode(self.ENCODING, "backslashreplace")) + if "" in content: + content = content.replace("", self.index_html(index) + "") + else: + content += self.index_html(index) + flow.response.content = content.encode(self.ENCODING) + else: + flow.response.content = (self.landing_page(index) + .encode(self.ENCODING)) + + +class RobotsInjection(InjectionGenerator): + """Injects the URL index by creating a new robots.txt including the URLs.""" + + def __init__(self, directive="Allow"): + self.directive = directive + + @classmethod + def robots_txt(cls, index, directive="Allow"): + lines = ["User-agent: *"] + for scheme_netloc, paths in index.items(): + for path, methods in paths.items(): + lines.append(directive + ": " + path) + return "\n".join(lines) + + def inject(self, index, flow: HTTPFlow): + if flow.response is not None: + if flow.response.status_code != 404: + logger.warning( + f"URL '{flow.request.url}' didn't return 404 status, " + f"index page would overwrite valid page.") + else: + flow.response.content = self.robots_txt(index, + self.directive).encode( + self.ENCODING) + + +class SitemapInjection(InjectionGenerator): + """Injects the URL index by creating a new sitemap including the URLs.""" + + @classmethod + def sitemap(cls, index): + lines = [ + ""] + for scheme_netloc, paths in index.items(): + for path, methods in paths.items(): + url = scheme_netloc + path + lines.append(f"{html.escape(url)}") + lines.append("") + return "\n".join(lines) + + def inject(self, index, flow: HTTPFlow): + if flow.response is not None: + if flow.response.status_code != 404: + logger.warning( + f"URL '{flow.request.url}' didn't return 404 status, " + f"index page would overwrite valid page.") + else: + flow.response.content = self.sitemap(index).encode(self.ENCODING) + + +class UrlInjectionAddon: + """ The UrlInjection add-on can be used in combination with web application scanners to improve their crawling + performance. + + The given URls will be injected into the web application. With this, web application scanners can find pages to + crawl much easier. Depending on the Injection generator, the URLs will be injected at different places of the + web application. It is possible to create a landing page which includes the URL (HTMLInjection()), to inject the + URLs to an existing page (HTMLInjection(insert=True)), to create a robots.txt containing the URLs + (RobotsInjection()) or to create a sitemap.xml which includes the URLS (SitemapInjection()). + It is necessary that the web application scanner can find the newly created page containing the URL index. For + example, the newly created page can be set as starting point for the web application scanner. + The URL index needed for the injection can be generated by the UrlIndex Add-on. + """ + + def __init__(self, flt: str, url_index_file: str, + injection_gen: InjectionGenerator): + """Initializes the UrlIndex add-on. + + Args: + flt: mitmproxy filter to decide on which pages the URLs will be injected (str). + url_index_file: Path to the file which includes the URL index in JSON format (e.g. generated by the UrlIndexAddon), given + as str. + injection_gen: InjectionGenerator that should be used to inject the URLs into the web application. + """ + self.name = f"{self.__class__.__name__}-{injection_gen.__class__.__name__}-{self.__hash__()}" + self.flt = flowfilter.parse(flt) + self.injection_gen = injection_gen + with open(url_index_file) as f: + self.url_store = json.load(f) + + def response(self, flow: HTTPFlow): + """Checks if the response matches the filter and such should be injected. + Injects the URL index if appropriate. + """ + if flow.response is not None: + if self.flt is not None and self.flt(flow): + self.injection_gen.inject(self.url_store, flow) + flow.response.status_code = 200 + flow.response.headers["content-type"] = "text/html" + logger.debug(f"Set status code to 200 and set content to logged " + f"urls. Method: {self.injection_gen}") diff -Nru mitmproxy-5.1.1/docs/src/examples/contrib/webscanner_helper/watchdog.py mitmproxy-6.0.2/docs/src/examples/contrib/webscanner_helper/watchdog.py --- mitmproxy-5.1.1/docs/src/examples/contrib/webscanner_helper/watchdog.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/contrib/webscanner_helper/watchdog.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,71 @@ +import pathlib +import time +import typing +import logging +from datetime import datetime + +import mitmproxy.connections +import mitmproxy.http +from mitmproxy.addons.export import curl_command, raw +from mitmproxy.exceptions import HttpSyntaxException + +logger = logging.getLogger(__name__) + + +class WatchdogAddon(): + """ The Watchdog Add-on can be used in combination with web application scanners in oder to check if the device + under test responds correctls to the scanner's responses. + + The Watchdog Add-on checks if the device under test responds correctly to the scanner's responses. + If the Watchdog sees that the DUT is no longer responding correctly, an multiprocessing event is set. + This information can be used to restart the device under test if necessary. + """ + + def __init__(self, event, outdir: pathlib.Path, timeout=None): + """Initializes the Watchdog. + + Args: + event: multiprocessing.Event that will be set if the watchdog is triggered. + outdir: path to a directory in which the triggering requests will be saved (curl and raw). + timeout_conn: float that specifies the timeout for the server connection + """ + self.error_event = event + self.flow_dir = outdir + if self.flow_dir.exists() and not self.flow_dir.is_dir(): + raise RuntimeError("Watchtdog output path must be a directory.") + elif not self.flow_dir.exists(): + self.flow_dir.mkdir(parents=True) + self.last_trigger: typing.Union[None, float] = None + self.timeout: typing.Union[None, float] = timeout + + def serverconnect(self, conn: mitmproxy.connections.ServerConnection): + if self.timeout is not None: + conn.settimeout(self.timeout) + + @classmethod + def not_in_timeout(cls, last_triggered, timeout): + """Checks if current error lies not in timeout after last trigger (potential reset of connection).""" + return last_triggered is None or timeout is None or (time.time() - last_triggered > timeout) + + def error(self, flow): + """ Checks if the watchdog will be triggered. + + Only triggers watchdog for timeouts after last reset and if flow.error is set (shows that error is a server + error). Ignores HttpSyntaxException Errors since this can be triggered on purpose by web application scanner. + + Args: + flow: mitmproxy.http.flow + """ + if (self.not_in_timeout(self.last_trigger, self.timeout) + and flow.error is not None and not isinstance(flow.error, HttpSyntaxException)): + + self.last_trigger = time.time() + logger.error(f"Watchdog triggered! Cause: {flow}") + self.error_event.set() + + # save the request which might have caused the problem + if flow.request: + with (self.flow_dir / f"{datetime.utcnow().isoformat()}.curl").open("w") as f: + f.write(curl_command(flow)) + with (self.flow_dir / f"{datetime.utcnow().isoformat()}.raw").open("wb") as f: + f.write(raw(flow)) diff -Nru mitmproxy-5.1.1/docs/src/examples/contrib/xss_scanner.py mitmproxy-6.0.2/docs/src/examples/contrib/xss_scanner.py --- mitmproxy-5.1.1/docs/src/examples/contrib/xss_scanner.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/contrib/xss_scanner.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,423 @@ +r""" + + __ __ _____ _____ _____ + \ \ / // ____/ ____| / ____| + \ V /| (___| (___ | (___ ___ __ _ _ __ _ __ ___ _ __ + > < \___ \\___ \ \___ \ / __/ _` | '_ \| '_ \ / _ \ '__| + / . \ ____) |___) | ____) | (_| (_| | | | | | | | __/ | + /_/ \_\_____/_____/ |_____/ \___\__,_|_| |_|_| |_|\___|_| + + +This script automatically scans all visited webpages for XSS and SQLi vulnerabilities. + +Usage: mitmproxy -s xss_scanner.py + +This script scans for vulnerabilities by injecting a fuzzing payload (see PAYLOAD below) into 4 different places +and examining the HTML to look for XSS and SQLi injection vulnerabilities. The XSS scanning functionality works by +looking to see whether it is possible to inject HTML based off of of where the payload appears in the page and what +characters are escaped. In addition, it also looks for any script tags that load javascript from unclaimed domains. +The SQLi scanning functionality works by using regular expressions to look for errors from a number of different +common databases. Since it is only looking for errors, it will not find blind SQLi vulnerabilities. + +The 4 places it injects the payload into are: +1. URLs (e.g. https://example.com/ -> https://example.com/PAYLOAD/) +2. Queries (e.g. https://example.com/index.html?a=b -> https://example.com/index.html?a=PAYLOAD) +3. Referers (e.g. The referer changes from https://example.com to PAYLOAD) +4. User Agents (e.g. The UA changes from Chrome to PAYLOAD) + +Reports from this script show up in the event log (viewable by pressing e) and formatted like: + +===== XSS Found ==== +XSS URL: http://daviddworken.com/vulnerableUA.php +Injection Point: User Agent +Suggested Exploit: +Line: 1029zxcs'd"aoso[sb]po(pc)se;sl/bsl\eq=3847asd + +""" + +from html.parser import HTMLParser +from typing import Dict, Union, Tuple, Optional, List, NamedTuple +from urllib.parse import urlparse +import re +import socket + +import requests + +from mitmproxy import http +from mitmproxy import ctx + + +# The actual payload is put between a frontWall and a backWall to make it easy +# to locate the payload with regular expressions +FRONT_WALL = b"1029zxc" +BACK_WALL = b"3847asd" +PAYLOAD = b"""s'd"aoso[sb]po(pc)se;sl/bsl\\eq=""" +FULL_PAYLOAD = FRONT_WALL + PAYLOAD + BACK_WALL + + +# A XSSData is a named tuple with the following fields: +# - url -> str +# - injection_point -> str +# - exploit -> str +# - line -> str +class XSSData(NamedTuple): + url: str + injection_point: str + exploit: str + line: str + + +# A SQLiData is named tuple with the following fields: +# - url -> str +# - injection_point -> str +# - regex -> str +# - dbms -> str +class SQLiData(NamedTuple): + url: str + injection_point: str + regex: str + dbms: str + + +VulnData = Tuple[Optional[XSSData], Optional[SQLiData]] +Cookies = Dict[str, str] + + +def get_cookies(flow: http.HTTPFlow) -> Cookies: + """ Return a dict going from cookie names to cookie values + - Note that it includes both the cookies sent in the original request and + the cookies sent by the server """ + return {name: value for name, value in flow.request.cookies.fields} + + +def find_unclaimed_URLs(body, requestUrl): + """ Look for unclaimed URLs in script tags and log them if found""" + def getValue(attrs: List[Tuple[str, str]], attrName: str) -> Optional[str]: + for name, value in attrs: + if attrName == name: + return value + return None + + class ScriptURLExtractor(HTMLParser): + script_URLs: List[str] = [] + + def handle_starttag(self, tag, attrs): + if (tag == "script" or tag == "iframe") and "src" in [name for name, value in attrs]: + self.script_URLs.append(getValue(attrs, "src")) + if tag == "link" and getValue(attrs, "rel") == "stylesheet" and "href" in [name for name, value in attrs]: + self.script_URLs.append(getValue(attrs, "href")) + + parser = ScriptURLExtractor() + parser.feed(body) + for url in parser.script_URLs: + url_parser = urlparse(url) + domain = url_parser.netloc + try: + socket.gethostbyname(domain) + except socket.gaierror: + ctx.log.error(f"XSS found in {requestUrl} due to unclaimed URL \"{url}\".") + + +def test_end_of_URL_injection(original_body: str, request_URL: str, cookies: Cookies) -> VulnData: + """ Test the given URL for XSS via injection onto the end of the URL and + log the XSS if found """ + parsed_URL = urlparse(request_URL) + path = parsed_URL.path + if path != "" and path[-1] != "/": # ensure the path ends in a / + path += "/" + path += FULL_PAYLOAD.decode('utf-8') # the path must be a string while the payload is bytes + url = parsed_URL._replace(path=path).geturl() + body = requests.get(url, cookies=cookies).text.lower() + xss_info = get_XSS_data(body, url, "End of URL") + sqli_info = get_SQLi_data(body, original_body, url, "End of URL") + return xss_info, sqli_info + + +def test_referer_injection(original_body: str, request_URL: str, cookies: Cookies) -> VulnData: + """ Test the given URL for XSS via injection into the referer and + log the XSS if found """ + body = requests.get(request_URL, headers={'referer': FULL_PAYLOAD}, cookies=cookies).text.lower() + xss_info = get_XSS_data(body, request_URL, "Referer") + sqli_info = get_SQLi_data(body, original_body, request_URL, "Referer") + return xss_info, sqli_info + + +def test_user_agent_injection(original_body: str, request_URL: str, cookies: Cookies) -> VulnData: + """ Test the given URL for XSS via injection into the user agent and + log the XSS if found """ + body = requests.get(request_URL, headers={'User-Agent': FULL_PAYLOAD}, cookies=cookies).text.lower() + xss_info = get_XSS_data(body, request_URL, "User Agent") + sqli_info = get_SQLi_data(body, original_body, request_URL, "User Agent") + return xss_info, sqli_info + + +def test_query_injection(original_body: str, request_URL: str, cookies: Cookies): + """ Test the given URL for XSS via injection into URL queries and + log the XSS if found """ + parsed_URL = urlparse(request_URL) + query_string = parsed_URL.query + # queries is a list of parameters where each parameter is set to the payload + queries = [query.split("=")[0] + "=" + FULL_PAYLOAD.decode('utf-8') for query in query_string.split("&")] + new_query_string = "&".join(queries) + new_URL = parsed_URL._replace(query=new_query_string).geturl() + body = requests.get(new_URL, cookies=cookies).text.lower() + xss_info = get_XSS_data(body, new_URL, "Query") + sqli_info = get_SQLi_data(body, original_body, new_URL, "Query") + return xss_info, sqli_info + + +def log_XSS_data(xss_info: Optional[XSSData]) -> None: + """ Log information about the given XSS to mitmproxy """ + # If it is None, then there is no info to log + if not xss_info: + return + ctx.log.error("===== XSS Found ====") + ctx.log.error("XSS URL: %s" % xss_info.url) + ctx.log.error("Injection Point: %s" % xss_info.injection_point) + ctx.log.error("Suggested Exploit: %s" % xss_info.exploit) + ctx.log.error("Line: %s" % xss_info.line) + + +def log_SQLi_data(sqli_info: Optional[SQLiData]) -> None: + """ Log information about the given SQLi to mitmproxy """ + if not sqli_info: + return + ctx.log.error("===== SQLi Found =====") + ctx.log.error("SQLi URL: %s" % sqli_info.url) + ctx.log.error("Injection Point: %s" % sqli_info.injection_point) + ctx.log.error("Regex used: %s" % sqli_info.regex) + ctx.log.error("Suspected DBMS: %s" % sqli_info.dbms) + return + + +def get_SQLi_data(new_body: str, original_body: str, request_URL: str, injection_point: str) -> Optional[SQLiData]: + """ Return a SQLiDict if there is a SQLi otherwise return None + String String URL String -> (SQLiDict or None) """ + # Regexes taken from Damn Small SQLi Scanner: https://github.com/stamparm/DSSS/blob/master/dsss.py#L17 + DBMS_ERRORS = { + "MySQL": (r"SQL syntax.*MySQL", r"Warning.*mysql_.*", r"valid MySQL result", r"MySqlClient\."), + "PostgreSQL": (r"PostgreSQL.*ERROR", r"Warning.*\Wpg_.*", r"valid PostgreSQL result", r"Npgsql\."), + "Microsoft SQL Server": (r"Driver.* SQL[\-\_\ ]*Server", r"OLE DB.* SQL Server", r"(\W|\A)SQL Server.*Driver", + r"Warning.*mssql_.*", r"(\W|\A)SQL Server.*[0-9a-fA-F]{8}", + r"(?s)Exception.*\WSystem\.Data\.SqlClient\.", r"(?s)Exception.*\WRoadhouse\.Cms\."), + "Microsoft Access": (r"Microsoft Access Driver", r"JET Database Engine", r"Access Database Engine"), + "Oracle": (r"\bORA-[0-9][0-9][0-9][0-9]", r"Oracle error", r"Oracle.*Driver", r"Warning.*\Woci_.*", r"Warning.*\Wora_.*"), + "IBM DB2": (r"CLI Driver.*DB2", r"DB2 SQL error", r"\bdb2_\w+\("), + "SQLite": (r"SQLite/JDBCDriver", r"SQLite.Exception", r"System.Data.SQLite.SQLiteException", r"Warning.*sqlite_.*", + r"Warning.*SQLite3::", r"\[SQLITE_ERROR\]"), + "Sybase": (r"(?i)Warning.*sybase.*", r"Sybase message", r"Sybase.*Server message.*"), + } + for dbms, regexes in DBMS_ERRORS.items(): + for regex in regexes: # type: ignore + if re.search(regex, new_body, re.IGNORECASE) and not re.search(regex, original_body, re.IGNORECASE): + return SQLiData(request_URL, + injection_point, + regex, + dbms) + return None + + +# A qc is either ' or " +def inside_quote(qc: str, substring_bytes: bytes, text_index: int, body_bytes: bytes) -> bool: + """ Whether the Numberth occurrence of the first string in the second + string is inside quotes as defined by the supplied QuoteChar """ + substring = substring_bytes.decode('utf-8') + body = body_bytes.decode('utf-8') + num_substrings_found = 0 + in_quote = False + for index, char in enumerate(body): + # Whether the next chunk of len(substring) chars is the substring + next_part_is_substring = ( + (not (index + len(substring) > len(body))) and + (body[index:index + len(substring)] == substring) + ) + # Whether this char is escaped with a \ + is_not_escaped = ( + (index - 1 < 0 or index - 1 > len(body)) or + (body[index - 1] != "\\") + ) + if char == qc and is_not_escaped: + in_quote = not in_quote + if next_part_is_substring: + if num_substrings_found == text_index: + return in_quote + num_substrings_found += 1 + return False + + +def paths_to_text(html: str, string: str) -> List[str]: + """ Return list of Paths to a given str in the given HTML tree + - Note that it does a BFS """ + + def remove_last_occurence_of_sub_string(string: str, substr: str) -> str: + """ Delete the last occurrence of substr from str + String String -> String + """ + index = string.rfind(substr) + return string[:index] + string[index + len(substr):] + + class PathHTMLParser(HTMLParser): + currentPath = "" + paths: List[str] = [] + + def handle_starttag(self, tag, attrs): + self.currentPath += ("/" + tag) + + def handle_endtag(self, tag): + self.currentPath = remove_last_occurence_of_sub_string(self.currentPath, "/" + tag) + + def handle_data(self, data): + if string in data: + self.paths.append(self.currentPath) + + parser = PathHTMLParser() + parser.feed(html) + return parser.paths + + +def get_XSS_data(body: Union[str, bytes], request_URL: str, injection_point: str) -> Optional[XSSData]: + """ Return a XSSDict if there is a XSS otherwise return None """ + def in_script(text, index, body) -> bool: + """ Whether the Numberth occurrence of the first string in the second + string is inside a script tag """ + paths = paths_to_text(body.decode('utf-8'), text.decode("utf-8")) + try: + path = paths[index] + return "script" in path + except IndexError: + return False + + def in_HTML(text: bytes, index: int, body: bytes) -> bool: + """ Whether the Numberth occurrence of the first string in the second + string is inside the HTML but not inside a script tag or part of + a HTML attribute""" + # if there is a < then lxml will interpret that as a tag, so only search for the stuff before it + text = text.split(b"<")[0] + paths = paths_to_text(body.decode('utf-8'), text.decode("utf-8")) + try: + path = paths[index] + return "script" not in path + except IndexError: + return False + + def inject_javascript_handler(html: str) -> bool: + """ Whether you can inject a Javascript:alert(0) as a link """ + class injectJSHandlerHTMLParser(HTMLParser): + injectJSHandler = False + + def handle_starttag(self, tag, attrs): + for name, value in attrs: + if name == "href" and value.startswith(FRONT_WALL.decode('utf-8')): + self.injectJSHandler = True + + parser = injectJSHandlerHTMLParser() + parser.feed(html) + return parser.injectJSHandler + # Only convert the body to bytes if needed + if isinstance(body, str): + body = bytes(body, 'utf-8') + # Regex for between 24 and 72 (aka 24*3) characters encapsulated by the walls + regex = re.compile(b"""%s.{24,72}?%s""" % (FRONT_WALL, BACK_WALL)) + matches = regex.findall(body) + for index, match in enumerate(matches): + # Where the string is injected into the HTML + in_script_val = in_script(match, index, body) + in_HTML_val = in_HTML(match, index, body) + in_tag = not in_script_val and not in_HTML_val + in_single_quotes = inside_quote("'", match, index, body) + in_double_quotes = inside_quote('"', match, index, body) + # Whether you can inject: + inject_open_angle = b"aoso" in match # close angle brackets + inject_single_quotes = b"s'd" in match # single quotes + inject_double_quotes = b'd"ao' in match # double quotes + inject_slash = b"sl/bsl" in match # forward slashes + inject_semi = b"se;sl" in match # semicolons + inject_equals = b"eq=" in match # equals sign + if in_script_val and inject_slash and inject_open_angle and inject_close_angle: # e.g. + return XSSData(request_URL, + injection_point, + ' + return XSSData(request_URL, + injection_point, + "';alert(0);g='", + match.decode('utf-8')) + elif in_script_val and in_double_quotes and inject_double_quotes and inject_semi: # e.g. + return XSSData(request_URL, + injection_point, + '";alert(0);g="', + match.decode('utf-8')) + elif in_tag and in_single_quotes and inject_single_quotes and inject_open_angle and inject_close_angle and inject_slash: + # e.g. Test + return XSSData(request_URL, + injection_point, + "'>", + match.decode('utf-8')) + elif in_tag and in_double_quotes and inject_double_quotes and inject_open_angle and inject_close_angle and inject_slash: + # e.g. Test + return XSSData(request_URL, + injection_point, + '">', + match.decode('utf-8')) + elif in_tag and not in_double_quotes and not in_single_quotes and inject_open_angle and inject_close_angle and inject_slash: + # e.g. Test + return XSSData(request_URL, + injection_point, + '>', + match.decode('utf-8')) + elif inject_javascript_handler(body.decode('utf-8')): # e.g. Test + return XSSData(request_URL, + injection_point, + 'Javascript:alert(0)', + match.decode('utf-8')) + elif in_tag and in_double_quotes and inject_double_quotes and inject_equals: # e.g. Test + return XSSData(request_URL, + injection_point, + '" onmouseover="alert(0)" t="', + match.decode('utf-8')) + elif in_tag and in_single_quotes and inject_single_quotes and inject_equals: # e.g. Test + return XSSData(request_URL, + injection_point, + "' onmouseover='alert(0)' t='", + match.decode('utf-8')) + elif in_tag and not in_single_quotes and not in_double_quotes and inject_equals: # e.g. Test + return XSSData(request_URL, + injection_point, + " onmouseover=alert(0) t=", + match.decode('utf-8')) + elif in_HTML_val and not in_script_val and inject_open_angle and inject_close_angle and inject_slash: # e.g. PAYLOAD + return XSSData(request_URL, + injection_point, + '', + match.decode('utf-8')) + else: + return None + return None + + +# response is mitmproxy's entry point +def response(flow: http.HTTPFlow) -> None: + assert flow.response + cookies_dict = get_cookies(flow) + resp = flow.response.get_text(strict=False) + assert resp + # Example: http://xss.guru/unclaimedScriptTag.html + find_unclaimed_URLs(resp, flow.request.url) + results = test_end_of_URL_injection(resp, flow.request.url, cookies_dict) + log_XSS_data(results[0]) + log_SQLi_data(results[1]) + # Example: https://daviddworken.com/vulnerableReferer.php + results = test_referer_injection(resp, flow.request.url, cookies_dict) + log_XSS_data(results[0]) + log_SQLi_data(results[1]) + # Example: https://daviddworken.com/vulnerableUA.php + results = test_user_agent_injection(resp, flow.request.url, cookies_dict) + log_XSS_data(results[0]) + log_SQLi_data(results[1]) + if "?" in flow.request.url: + # Example: https://daviddworken.com/vulnerable.php?name= + results = test_query_injection(resp, flow.request.url, cookies_dict) + log_XSS_data(results[0]) + log_SQLi_data(results[1]) diff -Nru mitmproxy-5.1.1/docs/src/examples/README.md mitmproxy-6.0.2/docs/src/examples/README.md --- mitmproxy-5.1.1/docs/src/examples/README.md 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/README.md 2020-12-15 16:41:27.000000000 +0000 @@ -1,15 +1,9 @@ -# Mitmproxy Scripting API +# Mitmproxy Examples Mitmproxy has a powerful scripting API that allows you to control almost any aspect of traffic being proxied. In fact, much of mitmproxy’s own core functionality is implemented using the exact same API -exposed to scripters (see [mitmproxy/addons](../mitmproxy/addons)). + (see [mitmproxy/addons](../mitmproxy/addons)). -This directory contains some examples of the scripting API. We recommend to start with the -ones in [simple/](./simple). | :warning: | If you are browsing this on GitHub, make sure to select the git tag matching your mitmproxy version. | |------------|------------------------------------------------------------------------------------------------------| - - -Some inline scripts may require additional dependencies, which can be installed using -`pip install mitmproxy[examples]`. \ No newline at end of file diff -Nru mitmproxy-5.1.1/docs/src/examples/simple/add_header_class.py mitmproxy-6.0.2/docs/src/examples/simple/add_header_class.py --- mitmproxy-5.1.1/docs/src/examples/simple/add_header_class.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/simple/add_header_class.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,9 +0,0 @@ -from mitmproxy import http - - -class AddHeader: - def response(self, flow: http.HTTPFlow) -> None: - flow.response.headers["newheader"] = "foo" - - -addons = [AddHeader()] diff -Nru mitmproxy-5.1.1/docs/src/examples/simple/add_header.py mitmproxy-6.0.2/docs/src/examples/simple/add_header.py --- mitmproxy-5.1.1/docs/src/examples/simple/add_header.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/simple/add_header.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,5 +0,0 @@ -from mitmproxy import http - - -def response(flow: http.HTTPFlow) -> None: - flow.response.headers["newheader"] = "foo" diff -Nru mitmproxy-5.1.1/docs/src/examples/simple/custom_contentview.py mitmproxy-6.0.2/docs/src/examples/simple/custom_contentview.py --- mitmproxy-5.1.1/docs/src/examples/simple/custom_contentview.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/simple/custom_contentview.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,24 +0,0 @@ -""" -This example shows how one can add a custom contentview to mitmproxy. -The content view API is explained in the mitmproxy.contentviews module. -""" -from mitmproxy import contentviews - - -class ViewSwapCase(contentviews.View): - name = "swapcase" - content_types = ["text/plain"] - - def __call__(self, data, **metadata) -> contentviews.TViewResult: - return "case-swapped text", contentviews.format_text(data.swapcase()) - - -view = ViewSwapCase() - - -def load(l): - contentviews.add(view) - - -def done(): - contentviews.remove(view) diff -Nru mitmproxy-5.1.1/docs/src/examples/simple/custom_option.py mitmproxy-6.0.2/docs/src/examples/simple/custom_option.py --- mitmproxy-5.1.1/docs/src/examples/simple/custom_option.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/simple/custom_option.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,21 +0,0 @@ -""" -This example shows how addons can register custom options -that can be configured at startup or during execution -from the options dialog within mitmproxy. - -Example: - -$ mitmproxy --set custom=true -$ mitmproxy --set custom # shorthand for boolean options -""" -from mitmproxy import ctx - - -def load(l): - ctx.log.info("Registering option 'custom'") - l.add_option("custom", bool, False, "A custom option") - - -def configure(updated): - if "custom" in updated: - ctx.log.info("custom option value: %s" % ctx.options.custom) diff -Nru mitmproxy-5.1.1/docs/src/examples/simple/filter_flows.py mitmproxy-6.0.2/docs/src/examples/simple/filter_flows.py --- mitmproxy-5.1.1/docs/src/examples/simple/filter_flows.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/simple/filter_flows.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,26 +0,0 @@ -""" -This script demonstrates how to use mitmproxy's filter pattern in scripts. -""" -from mitmproxy import flowfilter -from mitmproxy import ctx, http - - -class Filter: - def __init__(self): - self.filter: flowfilter.TFilter = None - - def configure(self, updated): - self.filter = flowfilter.parse(ctx.options.flowfilter) - - def load(self, l): - l.add_option( - "flowfilter", str, "", "Check that flow matches filter." - ) - - def response(self, flow: http.HTTPFlow) -> None: - if flowfilter.match(self.filter, flow): - ctx.log.info("Flow matches filter:") - ctx.log.info(flow) - - -addons = [Filter()] diff -Nru mitmproxy-5.1.1/docs/src/examples/simple/internet_in_mirror.py mitmproxy-6.0.2/docs/src/examples/simple/internet_in_mirror.py --- mitmproxy-5.1.1/docs/src/examples/simple/internet_in_mirror.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/simple/internet_in_mirror.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,9 +0,0 @@ -""" -This script reflects all content passing through the proxy. -""" -from mitmproxy import http - - -def response(flow: http.HTTPFlow) -> None: - reflector = b"" - flow.response.content = flow.response.content.replace(b"", reflector) diff -Nru mitmproxy-5.1.1/docs/src/examples/simple/io_read_dumpfile.py mitmproxy-6.0.2/docs/src/examples/simple/io_read_dumpfile.py --- mitmproxy-5.1.1/docs/src/examples/simple/io_read_dumpfile.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/simple/io_read_dumpfile.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,21 +0,0 @@ -#!/usr/bin/env python -# -# Simple script showing how to read a mitmproxy dump file -# -from mitmproxy import io -from mitmproxy.exceptions import FlowReadException -import pprint -import sys - - -with open(sys.argv[1], "rb") as logfile: - freader = io.FlowReader(logfile) - pp = pprint.PrettyPrinter(indent=4) - try: - for f in freader.stream(): - print(f) - print(f.request.host) - pp.pprint(f.get_state()) - print("") - except FlowReadException as e: - print("Flow file corrupted: {}".format(e)) diff -Nru mitmproxy-5.1.1/docs/src/examples/simple/io_write_dumpfile.py mitmproxy-6.0.2/docs/src/examples/simple/io_write_dumpfile.py --- mitmproxy-5.1.1/docs/src/examples/simple/io_write_dumpfile.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/simple/io_write_dumpfile.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,27 +0,0 @@ -""" -This script demonstrates how to generate a mitmproxy dump file, -as it would also be generated by passing `-w` to mitmproxy. -In contrast to `-w`, this gives you full control over which -flows should be saved and also allows you to rotate files or log -to multiple files in parallel. -""" -import random -import sys -from mitmproxy import io, http -import typing # noqa - - -class Writer: - def __init__(self, path: str) -> None: - self.f: typing.IO[bytes] = open(path, "wb") - self.w = io.FlowWriter(self.f) - - def response(self, flow: http.HTTPFlow) -> None: - if random.choice([True, False]): - self.w.add(flow) - - def done(self): - self.f.close() - - -addons = [Writer(sys.argv[1])] diff -Nru mitmproxy-5.1.1/docs/src/examples/simple/link_expander.py mitmproxy-6.0.2/docs/src/examples/simple/link_expander.py --- mitmproxy-5.1.1/docs/src/examples/simple/link_expander.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/simple/link_expander.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,28 +0,0 @@ -# This script determines if request is an HTML webpage and if so seeks out -# relative links () and expands them to absolute links -# In practice this can be used to front an indexing spider that may not have the capability to expand relative page links. -# Usage: mitmdump -s link_expander.py or mitmproxy -s link_expander.py - -import re -from urllib.parse import urljoin - - -def response(flow): - - if "Content-Type" in flow.response.headers and flow.response.headers["Content-Type"].find("text/html") != -1: - pageUrl = flow.request.url - pageText = flow.response.text - pattern = (r"]*?\s+)?href=(?P[\"'])" - r"(?P(?!https?:\/\/|ftps?:\/\/|\/\/|#|javascript:|mailto:).*?)(?P=delimiter)") - rel_matcher = re.compile(pattern, flags=re.IGNORECASE) - rel_matches = rel_matcher.finditer(pageText) - map_dict = {} - for match_num, match in enumerate(rel_matches): - (delimiter, rel_link) = match.group("delimiter", "link") - abs_link = urljoin(pageUrl, rel_link) - map_dict["{0}{1}{0}".format(delimiter, rel_link)] = "{0}{1}{0}".format(delimiter, abs_link) - for map in map_dict.items(): - pageText = pageText.replace(*map) - # Uncomment the following to print the expansion mapping - # print("{0} -> {1}".format(*map)) - flow.response.text = pageText \ No newline at end of file diff -Nru mitmproxy-5.1.1/docs/src/examples/simple/log_events.py mitmproxy-6.0.2/docs/src/examples/simple/log_events.py --- mitmproxy-5.1.1/docs/src/examples/simple/log_events.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/simple/log_events.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,7 +0,0 @@ -from mitmproxy import ctx - - -def load(l): - ctx.log.info("This is some informative text.") - ctx.log.warn("This is a warning.") - ctx.log.error("This is an error.") diff -Nru mitmproxy-5.1.1/docs/src/examples/simple/modify_body_inject_iframe.py mitmproxy-6.0.2/docs/src/examples/simple/modify_body_inject_iframe.py --- mitmproxy-5.1.1/docs/src/examples/simple/modify_body_inject_iframe.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/simple/modify_body_inject_iframe.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,26 +0,0 @@ -# (this script works best with --anticache) -from bs4 import BeautifulSoup -from mitmproxy import ctx, http - - -class Injector: - def load(self, loader): - loader.add_option( - "iframe", str, "", "IFrame to inject" - ) - - def response(self, flow: http.HTTPFlow) -> None: - if ctx.options.iframe: - html = BeautifulSoup(flow.response.content, "html.parser") - if html.body: - iframe = html.new_tag( - "iframe", - src=ctx.options.iframe, - frameborder=0, - height=0, - width=0) - html.body.insert(0, iframe) - flow.response.content = str(html).encode("utf8") - - -addons = [Injector()] diff -Nru mitmproxy-5.1.1/docs/src/examples/simple/modify_form.py mitmproxy-6.0.2/docs/src/examples/simple/modify_form.py --- mitmproxy-5.1.1/docs/src/examples/simple/modify_form.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/simple/modify_form.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,13 +0,0 @@ -from mitmproxy import http - - -def request(flow: http.HTTPFlow) -> None: - if flow.request.urlencoded_form: - # If there's already a form, one can just add items to the dict: - flow.request.urlencoded_form["mitmproxy"] = "rocks" - else: - # One can also just pass new form data. - # This sets the proper content type and overrides the body. - flow.request.urlencoded_form = [ - ("foo", "bar") - ] diff -Nru mitmproxy-5.1.1/docs/src/examples/simple/modify_querystring.py mitmproxy-6.0.2/docs/src/examples/simple/modify_querystring.py --- mitmproxy-5.1.1/docs/src/examples/simple/modify_querystring.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/simple/modify_querystring.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,5 +0,0 @@ -from mitmproxy import http - - -def request(flow: http.HTTPFlow) -> None: - flow.request.query["mitmproxy"] = "rocks" diff -Nru mitmproxy-5.1.1/docs/src/examples/simple/README.md mitmproxy-6.0.2/docs/src/examples/simple/README.md --- mitmproxy-5.1.1/docs/src/examples/simple/README.md 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/simple/README.md 1970-01-01 00:00:00.000000000 +0000 @@ -1,19 +0,0 @@ -## Simple Examples - -| Filename | Description | -| :----------------------------- | :--------------------------------------------------------------------------- | -| add_header.py | Simple script that just adds a header to every request. | -| custom_contentview.py | Add a custom content view to the mitmproxy UI. | -| custom_option.py | Add arguments to a script. | -| filter_flows.py | This script demonstrates how to use mitmproxy's filter pattern in scripts. | -| io_read_dumpfile.py | Read a dumpfile generated by mitmproxy. | -| io_write_dumpfile.py | Only write selected flows into a mitmproxy dumpfile. | -| link_expander.py | Discover relative links in HTML traffic and replace them with absolute paths | -| log_events.py | Use mitmproxy's logging API. | -| modify_body_inject_iframe.py | Inject configurable iframe into pages. | -| modify_form.py | Modify HTTP form submissions. | -| modify_querystring.py | Modify HTTP query strings. | -| redirect_requests.py | Redirect a request to a different server. | -| send_reply_from_proxy.py | Send a HTTP response directly from the proxy. | -| internet_in_mirror.py | Turn all images upside down. | -| wsgi_flask_app.py | Embed a WSGI app into mitmproxy. | diff -Nru mitmproxy-5.1.1/docs/src/examples/simple/redirect_requests.py mitmproxy-6.0.2/docs/src/examples/simple/redirect_requests.py --- mitmproxy-5.1.1/docs/src/examples/simple/redirect_requests.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/simple/redirect_requests.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,12 +0,0 @@ -""" -This example shows two ways to redirect flows to another server. -""" -from mitmproxy import http - - -def request(flow: http.HTTPFlow) -> None: - # pretty_host takes the "Host" header of the request into account, - # which is useful in transparent mode where we usually only have the IP - # otherwise. - if flow.request.pretty_host == "example.org": - flow.request.host = "mitmproxy.org" diff -Nru mitmproxy-5.1.1/docs/src/examples/simple/send_reply_from_proxy.py mitmproxy-6.0.2/docs/src/examples/simple/send_reply_from_proxy.py --- mitmproxy-5.1.1/docs/src/examples/simple/send_reply_from_proxy.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/simple/send_reply_from_proxy.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,17 +0,0 @@ -""" -This example shows how to send a reply from the proxy immediately -without sending any data to the remote server. -""" -from mitmproxy import http - - -def request(flow: http.HTTPFlow) -> None: - # pretty_url takes the "Host" header of the request into account, which - # is useful in transparent mode where we usually only have the IP otherwise. - - if flow.request.pretty_url == "http://example.com/path": - flow.response = http.HTTPResponse.make( - 200, # (optional) status code - b"Hello World", # (optional) content - {"Content-Type": "text/html"} # (optional) headers - ) diff -Nru mitmproxy-5.1.1/docs/src/examples/simple/websocket_messages.py mitmproxy-6.0.2/docs/src/examples/simple/websocket_messages.py --- mitmproxy-5.1.1/docs/src/examples/simple/websocket_messages.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/simple/websocket_messages.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,20 +0,0 @@ -import re -from mitmproxy import ctx - - -def websocket_message(flow): - # get the latest message - message = flow.messages[-1] - - # was the message sent from the client or server? - if message.from_client: - ctx.log.info("Client sent a message: {}".format(message.content)) - else: - ctx.log.info("Server sent a message: {}".format(message.content)) - - # manipulate the message content - message.content = re.sub(r'^Hello', 'HAPPY', message.content) - - if 'FOOBAR' in message.content: - # kill the message and not send it to the other endpoint - message.kill() diff -Nru mitmproxy-5.1.1/docs/src/examples/simple/wsgi_flask_app.py mitmproxy-6.0.2/docs/src/examples/simple/wsgi_flask_app.py --- mitmproxy-5.1.1/docs/src/examples/simple/wsgi_flask_app.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/examples/simple/wsgi_flask_app.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,25 +0,0 @@ -""" -This example shows how to graft a WSGI app onto mitmproxy. In this -instance, we're using the Flask framework (http://flask.pocoo.org/) to expose -a single simplest-possible page. -""" -from flask import Flask -from mitmproxy.addons import wsgiapp - -app = Flask("proxapp") - - -@app.route('/') -def hello_world() -> str: - return 'Hello World!' - - -addons = [ - # Host app at the magic domain "proxapp.local" on port 80. Requests to this - # domain and port combination will now be routed to the WSGI app instance. - wsgiapp.WSGIApp(app, "proxapp.local", 80) - # SSL works too, but the magic domain needs to be resolvable from the mitmproxy machine due to mitmproxy's design. - # mitmproxy will connect to said domain and use serve its certificate (unless --no-upstream-cert is set) - # but won't send any data. - # mitmproxy.ctx.master.apps.add(app, "example.com", 443) -] diff -Nru mitmproxy-5.1.1/docs/src/layouts/partials/sidebar.html mitmproxy-6.0.2/docs/src/layouts/partials/sidebar.html --- mitmproxy-5.1.1/docs/src/layouts/partials/sidebar.html 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/docs/src/layouts/partials/sidebar.html 2020-12-15 16:41:27.000000000 +0000 @@ -5,11 +5,11 @@
{% block content %} {% endblock %}
- + diff -Nru mitmproxy-5.1.1/mitmproxy/addons/onboarding.py mitmproxy-6.0.2/mitmproxy/addons/onboarding.py --- mitmproxy-5.1.1/mitmproxy/addons/onboarding.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/addons/onboarding.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,4 +1,4 @@ -from mitmproxy.addons import wsgiapp +from mitmproxy.addons import asgiapp from mitmproxy.addons.onboardingapp import app from mitmproxy import ctx @@ -6,11 +6,11 @@ APP_PORT = 80 -class Onboarding(wsgiapp.WSGIApp): +class Onboarding(asgiapp.WSGIApp): name = "onboarding" def __init__(self): - super().__init__(app, None, None) + super().__init__(app, APP_HOST, APP_PORT) def load(self, loader): loader.add_option( diff -Nru mitmproxy-5.1.1/mitmproxy/addons/proxyauth.py mitmproxy-6.0.2/mitmproxy/addons/proxyauth.py --- mitmproxy-5.1.1/mitmproxy/addons/proxyauth.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/addons/proxyauth.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,18 +1,18 @@ import binascii import weakref -import ldap3 +from typing import MutableMapping from typing import Optional -from typing import MutableMapping # noqa from typing import Tuple +import ldap3 import passlib.apache import mitmproxy.net.http -from mitmproxy import connections # noqa +from mitmproxy import ctx from mitmproxy import exceptions from mitmproxy import http -from mitmproxy import ctx from mitmproxy.net.http import status_codes +from mitmproxy.utils import compat REALM = "mitmproxy" @@ -49,7 +49,7 @@ self.singleuser = None self.ldapconn = None self.ldapserver = None - self.authenticated: MutableMapping[connections.ClientConnection, Tuple[str, str]] = weakref.WeakKeyDictionary() + self.authenticated: MutableMapping[compat.Client, Tuple[str, str]] = weakref.WeakKeyDictionary() """Contains all connections that are permanently authenticated after an HTTP CONNECT""" def load(self, loader): @@ -85,12 +85,12 @@ if self.is_proxy_auth(): return http.make_error_response( status_codes.PROXY_AUTH_REQUIRED, - headers=mitmproxy.net.http.Headers(Proxy_Authenticate='Basic realm="{}"'.format(REALM)), + headers=mitmproxy.net.http.Headers(Proxy_Authenticate=f'Basic realm="{REALM}"'), ) else: return http.make_error_response( status_codes.UNAUTHORIZED, - headers=mitmproxy.net.http.Headers(WWW_Authenticate='Basic realm="{}"'.format(REALM)), + headers=mitmproxy.net.http.Headers(WWW_Authenticate=f'Basic realm="{REALM}"'), ) def check(self, f: http.HTTPFlow) -> Optional[Tuple[str, str]]: diff -Nru mitmproxy-5.1.1/mitmproxy/addons/readfile.py mitmproxy-6.0.2/mitmproxy/addons/readfile.py --- mitmproxy-5.1.1/mitmproxy/addons/readfile.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/addons/readfile.py 2020-12-15 16:41:27.000000000 +0000 @@ -48,7 +48,7 @@ continue await ctx.master.load_flow(flow) cnt += 1 - except (IOError, exceptions.FlowReadException) as e: + except (OSError, exceptions.FlowReadException) as e: if cnt: ctx.log.warn("Flow file corrupted - loaded %i flows." % cnt) else: @@ -62,8 +62,8 @@ try: with open(path, "rb") as f: return await self.load_flows(f) - except IOError as e: - ctx.log.error("Cannot load flows: {}".format(e)) + except OSError as e: + ctx.log.error(f"Cannot load flows: {e}") raise exceptions.FlowReadException(str(e)) from e async def doread(self, rfile): diff -Nru mitmproxy-5.1.1/mitmproxy/addons/replace.py mitmproxy-6.0.2/mitmproxy/addons/replace.py --- mitmproxy-5.1.1/mitmproxy/addons/replace.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/addons/replace.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,117 +0,0 @@ -import os -import re -import typing - -from mitmproxy import exceptions -from mitmproxy import flowfilter -from mitmproxy import ctx - - -def parse_hook(s): - """ - Returns a (pattern, regex, replacement) tuple. - - The general form for a replacement hook is as follows: - - /patt/regex/replacement - - The first character specifies the separator. Example: - - :~q:foo:bar - - If only two clauses are specified, the pattern is set to match - universally (i.e. ".*"). Example: - - /foo/bar/ - - Clauses are parsed from left to right. Extra separators are taken to be - part of the final clause. For instance, the replacement clause below is - "foo/bar/": - - /one/two/foo/bar/ - """ - sep, rem = s[0], s[1:] - parts = rem.split(sep, 2) - if len(parts) == 2: - patt = ".*" - a, b = parts - elif len(parts) == 3: - patt, a, b = parts - else: - raise exceptions.OptionsError( - "Invalid replacement specifier: %s" % s - ) - return patt, a, b - - -class Replace: - def __init__(self): - self.lst = [] - - def load(self, loader): - loader.add_option( - "replacements", typing.Sequence[str], [], - """ - Replacement patterns of the form "/pattern/regex/replacement", where - the separator can be any character. - """ - ) - - def configure(self, updated): - """ - .replacements is a list of tuples (fpat, rex, s): - - fpatt: a string specifying a filter pattern. - rex: a regular expression, as string. - s: the replacement string - """ - if "replacements" in updated: - lst = [] - for rep in ctx.options.replacements: - fpatt, rex, s = parse_hook(rep) - - flt = flowfilter.parse(fpatt) - if not flt: - raise exceptions.OptionsError( - "Invalid filter pattern: %s" % fpatt - ) - try: - # We should ideally escape here before trying to compile - re.compile(rex) - except re.error as e: - raise exceptions.OptionsError( - "Invalid regular expression: %s - %s" % (rex, str(e)) - ) - if s.startswith("@") and not os.path.isfile(s[1:]): - raise exceptions.OptionsError( - "Invalid file path: {}".format(s[1:]) - ) - lst.append((rex, s, flt)) - self.lst = lst - - def execute(self, f): - for rex, s, flt in self.lst: - if flt(f): - if f.response: - self.replace(f.response, rex, s) - else: - self.replace(f.request, rex, s) - - def request(self, flow): - if not flow.reply.has_message: - self.execute(flow) - - def response(self, flow): - if not flow.reply.has_message: - self.execute(flow) - - def replace(self, obj, rex, s): - if s.startswith("@"): - s = os.path.expanduser(s[1:]) - try: - with open(s, "rb") as f: - s = f.read() - except IOError: - ctx.log.warn("Could not read replacement file: %s" % s) - return - obj.replace(rex, s, flags=re.DOTALL) diff -Nru mitmproxy-5.1.1/mitmproxy/addons/save.py mitmproxy-6.0.2/mitmproxy/addons/save.py --- mitmproxy-5.1.1/mitmproxy/addons/save.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/addons/save.py 2020-12-15 16:41:27.000000000 +0000 @@ -38,7 +38,7 @@ def start_stream_to_path(self, path, flt): try: f = self.open_file(path) - except IOError as v: + except OSError as v: raise exceptions.OptionsError(str(v)) self.stream = io.FilteredFlowWriter(f, flt) self.active_flows = set() @@ -68,7 +68,7 @@ """ try: f = self.open_file(path) - except IOError as v: + except OSError as v: raise exceptions.CommandError(v) from v stream = io.FlowWriter(f) for i in flows: @@ -107,6 +107,6 @@ if self.stream: for f in self.active_flows: self.stream.add(f) - self.active_flows = set([]) + self.active_flows = set() self.stream.fo.close() self.stream = None diff -Nru mitmproxy-5.1.1/mitmproxy/addons/script.py mitmproxy-6.0.2/mitmproxy/addons/script.py --- mitmproxy-5.1.1/mitmproxy/addons/script.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/addons/script.py 2020-12-15 16:41:27.000000000 +0000 @@ -51,7 +51,7 @@ lineno = "" if hasattr(exc, "lineno"): lineno = str(exc.lineno) - log_msg = "in script {}:{} {}".format(path, lineno, exception) + log_msg = f"in script {path}:{lineno} {exception}" if tb: etype, value, tback = sys.exc_info() tback = addonmanager.cut_traceback(tback, "invoke_addon") diff -Nru mitmproxy-5.1.1/mitmproxy/addons/serverplayback.py mitmproxy-6.0.2/mitmproxy/addons/serverplayback.py --- mitmproxy-5.1.1/mitmproxy/addons/serverplayback.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/addons/serverplayback.py 2020-12-15 16:41:27.000000000 +0000 @@ -202,10 +202,10 @@ if rflow: assert rflow.response response = rflow.response.copy() - response.is_replay = True if ctx.options.server_replay_refresh: response.refresh() f.response = response + f.is_replay = "response" elif ctx.options.server_replay_kill_extra: ctx.log.warn( "server_playback: killed non-replay request {}".format( diff -Nru mitmproxy-5.1.1/mitmproxy/addons/session.py mitmproxy-6.0.2/mitmproxy/addons/session.py --- mitmproxy-5.1.1/mitmproxy/addons/session.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/addons/session.py 2020-12-15 16:41:27.000000000 +0000 @@ -17,7 +17,7 @@ from mitmproxy.utils.data import pkg_data -class KeyifyList(object): +class KeyifyList: def __init__(self, inner, key): self.inner = inner self.key = key @@ -87,7 +87,7 @@ def _create_session(self): script_path = pkg_data.path("io/sql/session_create.sql") - with open(script_path, 'r') as qry: + with open(script_path) as qry: self.con.executescript(qry.read()) self.con.commit() diff -Nru mitmproxy-5.1.1/mitmproxy/addons/setheaders.py mitmproxy-6.0.2/mitmproxy/addons/setheaders.py --- mitmproxy-5.1.1/mitmproxy/addons/setheaders.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/addons/setheaders.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,85 +0,0 @@ -import typing - -from mitmproxy import exceptions -from mitmproxy import flowfilter -from mitmproxy import ctx - - -def parse_setheader(s): - """ - Returns a (pattern, regex, replacement) tuple. - - The general form for a replacement hook is as follows: - - /patt/regex/replacement - - The first character specifies the separator. Example: - - :~q:foo:bar - - If only two clauses are specified, the pattern is set to match - universally (i.e. ".*"). Example: - - /foo/bar/ - - Clauses are parsed from left to right. Extra separators are taken to be - part of the final clause. For instance, the replacement clause below is - "foo/bar/": - - /one/two/foo/bar/ - """ - sep, rem = s[0], s[1:] - parts = rem.split(sep, 2) - if len(parts) == 2: - patt = ".*" - a, b = parts - elif len(parts) == 3: - patt, a, b = parts - else: - raise exceptions.OptionsError( - "Invalid replacement specifier: %s" % s - ) - return patt, a, b - - -class SetHeaders: - def __init__(self): - self.lst = [] - - def load(self, loader): - loader.add_option( - "setheaders", typing.Sequence[str], [], - """ - Header set pattern of the form "/pattern/header/value", where the - separator can be any character. - """ - ) - - def configure(self, updated): - if "setheaders" in updated: - self.lst = [] - for shead in ctx.options.setheaders: - fpatt, header, value = parse_setheader(shead) - - flt = flowfilter.parse(fpatt) - if not flt: - raise exceptions.OptionsError( - "Invalid setheader filter pattern %s" % fpatt - ) - self.lst.append((fpatt, header, value, flt)) - - def run(self, f, hdrs): - for _, header, value, flt in self.lst: - if flt(f): - hdrs.pop(header, None) - for _, header, value, flt in self.lst: - if flt(f): - hdrs.add(header, value) - - def request(self, flow): - if not flow.reply.has_message: - self.run(flow, flow.request.headers) - - def response(self, flow): - if not flow.reply.has_message: - self.run(flow, flow.response.headers) diff -Nru mitmproxy-5.1.1/mitmproxy/addons/termstatus.py mitmproxy-6.0.2/mitmproxy/addons/termstatus.py --- mitmproxy-5.1.1/mitmproxy/addons/termstatus.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/addons/termstatus.py 2020-12-15 16:41:27.000000000 +0000 @@ -9,7 +9,7 @@ class TermStatus: def running(self): - if ctx.options.server: + if ctx.master.server.bound: ctx.log.info( "Proxy server listening at http://{}".format( human.format_address(ctx.master.server.address) diff -Nru mitmproxy-5.1.1/mitmproxy/addons/view.py mitmproxy-6.0.2/mitmproxy/addons/view.py --- mitmproxy-5.1.1/mitmproxy/addons/view.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/addons/view.py 2020-12-15 16:41:27.000000000 +0000 @@ -21,7 +21,10 @@ from mitmproxy import connections from mitmproxy import ctx from mitmproxy import io -from mitmproxy import http # noqa +from mitmproxy import http +from mitmproxy import tcp +from mitmproxy.utils import compat, human + # The underlying sorted list implementation expects the sort key to be stable # for the lifetime of the object. However, if we sort by size, for instance, @@ -38,7 +41,7 @@ def __init__(self, view): self.view = view - def generate(self, f: http.HTTPFlow) -> typing.Any: # pragma: no cover + def generate(self, f: mitmproxy.flow.Flow) -> typing.Any: # pragma: no cover pass def refresh(self, f): @@ -68,32 +71,49 @@ class OrderRequestStart(_OrderKey): - def generate(self, f: http.HTTPFlow) -> int: - return f.request.timestamp_start or 0 + def generate(self, f: mitmproxy.flow.Flow) -> float: + return f.timestamp_start class OrderRequestMethod(_OrderKey): - def generate(self, f: http.HTTPFlow) -> str: - return f.request.method + def generate(self, f: mitmproxy.flow.Flow) -> str: + if isinstance(f, http.HTTPFlow): + return f.request.method + elif isinstance(f, tcp.TCPFlow): + return "TCP" + else: + raise NotImplementedError() class OrderRequestURL(_OrderKey): - def generate(self, f: http.HTTPFlow) -> str: - return f.request.url + def generate(self, f: mitmproxy.flow.Flow) -> str: + if isinstance(f, http.HTTPFlow): + return f.request.url + elif isinstance(f, tcp.TCPFlow): + return human.format_address(f.server_conn.address) + else: + raise NotImplementedError() class OrderKeySize(_OrderKey): - def generate(self, f: http.HTTPFlow) -> int: - s = 0 - if f.request.raw_content: - s += len(f.request.raw_content) - if f.response and f.response.raw_content: - s += len(f.response.raw_content) - return s - + def generate(self, f: mitmproxy.flow.Flow) -> int: + if isinstance(f, http.HTTPFlow): + size = 0 + if f.request.raw_content: + size += len(f.request.raw_content) + if f.response and f.response.raw_content: + size += len(f.response.raw_content) + return size + elif isinstance(f, tcp.TCPFlow): + size = 0 + for message in f.messages: + size += len(message.content) + return size + else: + raise NotImplementedError() -matchall = flowfilter.parse(".") +matchall = flowfilter.parse("~http | ~tcp") orders = [ ("t", "time"), @@ -440,8 +460,12 @@ req = http.HTTPRequest.make(method.upper(), url) except ValueError as e: raise exceptions.CommandError("Invalid URL: %s" % e) - c = connections.ClientConnection.make_dummy(("", 0)) - s = connections.ServerConnection.make_dummy((req.host, req.port)) + if compat.new_proxy_core: # pragma: no cover + c = compat.Client(("", 0), ("", 0), req.timestamp_start - 0.0001) + s = compat.Server((req.host, req.port)) + else: # pragma: no cover + c = connections.ClientConnection.make_dummy(("", 0)) + s = connections.ServerConnection.make_dummy((req.host, req.port)) f = http.HTTPFlow(c, s) f.request = req f.request.headers["Host"] = req.host @@ -459,7 +483,7 @@ # get new flows each time. It would be more efficient to just have a # .newid() method or something. self.add([i.copy()]) - except IOError as e: + except OSError as e: ctx.log.error(e.strerror) except exceptions.FlowReadException as e: ctx.log.error(str(e)) @@ -555,6 +579,18 @@ def kill(self, f): self.update([f]) + def tcp_start(self, f): + self.add([f]) + + def tcp_message(self, f): + self.update([f]) + + def tcp_error(self, f): + self.update([f]) + + def tcp_end(self, f): + self.update([f]) + def update(self, flows: typing.Sequence[mitmproxy.flow.Flow]) -> None: """ Updates a list of flows. If flow is not in the state, it's ignored. diff -Nru mitmproxy-5.1.1/mitmproxy/addons/wsgiapp.py mitmproxy-6.0.2/mitmproxy/addons/wsgiapp.py --- mitmproxy-5.1.1/mitmproxy/addons/wsgiapp.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/addons/wsgiapp.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,42 +0,0 @@ -from mitmproxy import ctx -from mitmproxy import exceptions - -from mitmproxy.net import wsgi -from mitmproxy import version - - -class WSGIApp: - """ - An addon that hosts a WSGI app within mitproxy, at a specified - hostname and port. - """ - def __init__(self, app, host, port): - self.app, self.host, self.port = app, host, port - - @property - def name(self): - return "wsgiapp:%s:%s" % (self.host, self.port) - - def serve(self, app, flow): - """ - Serves app on flow, and prevents further handling of the flow. - """ - app = wsgi.WSGIAdaptor( - app, - flow.request.pretty_host, - flow.request.port, - version.MITMPROXY - ) - err = app.serve( - flow, - flow.client_conn.wfile, - **{"mitmproxy.master": ctx.master} - ) - if err: - ctx.log.error("Error in wsgi app. %s" % err) - raise exceptions.AddonHalt() - flow.reply.kill() - - def request(self, f): - if (f.request.pretty_host, f.request.port) == (self.host, self.port): - self.serve(self.app, f) diff -Nru mitmproxy-5.1.1/mitmproxy/certs.py mitmproxy-6.0.2/mitmproxy/certs.py --- mitmproxy-5.1.1/mitmproxy/certs.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/certs.py 2020-12-15 16:41:27.000000000 +0000 @@ -106,7 +106,10 @@ cert.gmtime_adj_notBefore(-3600 * 48) cert.gmtime_adj_notAfter(DEFAULT_EXP_DUMMY_CERT) cert.set_issuer(cacert.get_subject()) - if commonname is not None and len(commonname) < 64: + is_valid_commonname = ( + commonname is not None and len(commonname) < 64 + ) + if is_valid_commonname: cert.get_subject().CN = commonname if organization is not None: cert.get_subject().O = organization @@ -114,7 +117,13 @@ if ss: cert.set_version(2) cert.add_extensions( - [OpenSSL.crypto.X509Extension(b"subjectAltName", False, ss)]) + [OpenSSL.crypto.X509Extension( + b"subjectAltName", + # RFC 5280 §4.2.1.6: subjectAltName is critical if subject is empty. + not is_valid_commonname, + ss + )] + ) cert.add_extensions([ OpenSSL.crypto.X509Extension( b"extendedKeyUsage", @@ -164,9 +173,7 @@ self.expire_queue.append(entry) if len(self.expire_queue) > self.STORE_CAP: d = self.expire_queue.pop(0) - for k, v in list(self.certs.items()): - if v == d: - del self.certs[k] + self.certs = {k: v for k, v in self.certs.items() if v != d} @staticmethod def load_dhparam(path): @@ -189,7 +196,7 @@ return dh @classmethod - def from_store(cls, path, basename, key_size): + def from_store(cls, path, basename, key_size, passphrase: typing.Optional[bytes] = None): ca_path = os.path.join(path, basename + "-ca.pem") if not os.path.exists(ca_path): key, ca = cls.create_store(path, basename, key_size) @@ -201,7 +208,8 @@ raw) key = OpenSSL.crypto.load_privatekey( OpenSSL.crypto.FILETYPE_PEM, - raw) + raw, + passphrase) dh_path = os.path.join(path, basename + "-dhparam.pem") dh = cls.load_dhparam(dh_path) return cls(key, ca, ca_path, dh) @@ -273,7 +281,7 @@ return key, ca - def add_cert_file(self, spec: str, path: str) -> None: + def add_cert_file(self, spec: str, path: str, passphrase: typing.Optional[bytes] = None) -> None: with open(path, "rb") as f: raw = f.read() cert = Cert( @@ -283,7 +291,8 @@ try: privatekey = OpenSSL.crypto.load_privatekey( OpenSSL.crypto.FILETYPE_PEM, - raw) + raw, + passphrase) except Exception: privatekey = self.default_privatekey self.add_cert( @@ -458,7 +467,7 @@ ) @property - def cn(self): + def cn(self) -> typing.Optional[bytes]: c = None for i in self.subject: if i[0] == b"CN": @@ -466,7 +475,7 @@ return c @property - def organization(self): + def organization(self) -> typing.Optional[bytes]: c = None for i in self.subject: if i[0] == b"O": @@ -474,7 +483,7 @@ return c @property - def altnames(self): + def altnames(self) -> typing.List[bytes]: """ Returns: All DNS altnames. @@ -488,9 +497,9 @@ dec = decode(ext.get_data(), asn1Spec=_GeneralNames()) except PyAsn1Error: continue - for i in dec[0]: - if i[0].hasValue(): - e = i[0].asOctets() + for x in dec[0]: + if x[0].hasValue(): + e = x[0].asOctets() altnames.append(e) return altnames diff -Nru mitmproxy-5.1.1/mitmproxy/command_lexer.py mitmproxy-6.0.2/mitmproxy/command_lexer.py --- mitmproxy-5.1.1/mitmproxy/command_lexer.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/command_lexer.py 2020-12-15 16:41:27.000000000 +0000 @@ -12,9 +12,9 @@ r''' (["']) # start quote (?: - (?!\1)[^\\] # unescaped character that is not our quote nor the begin of an escape sequence. We can't use \1 in [] - | (?:\\.) # escape sequence + | + (?!\1). # unescaped character that is not our quote nor the begin of an escape sequence. We can't use \1 in [] )* (?:\1|$) # end quote ''', diff -Nru mitmproxy-5.1.1/mitmproxy/command.py mitmproxy-6.0.2/mitmproxy/command.py --- mitmproxy-5.1.1/mitmproxy/command.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/command.py 2020-12-15 16:41:27.000000000 +0000 @@ -159,7 +159,7 @@ self.add(o.command_name, o) except exceptions.CommandError as e: self.master.log.warn( - "Could not load command %s: %s" % (o.command_name, e) + f"Could not load command {o.command_name}: {e}" ) def add(self, path: str, func: typing.Callable): diff -Nru mitmproxy-5.1.1/mitmproxy/connections.py mitmproxy-6.0.2/mitmproxy/connections.py --- mitmproxy-5.1.1/mitmproxy/connections.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/connections.py 2020-12-15 16:41:27.000000000 +0000 @@ -2,6 +2,7 @@ import time import typing import uuid +import warnings from mitmproxy import certs from mitmproxy import exceptions @@ -19,7 +20,6 @@ Attributes: address: Remote address tls_established: True if TLS is established, False otherwise - clientcert: The TLS client certificate mitmcert: The MITM'ed TLS server certificate presented to the client timestamp_start: Connection start timestamp timestamp_tls_setup: TLS established timestamp @@ -42,7 +42,6 @@ self.wfile = None self.rfile = None self.address = None - self.clientcert = None self.tls_established = None self.id = str(uuid.uuid4()) @@ -61,7 +60,7 @@ def __repr__(self): if self.tls_established: - tls = "[{}] ".format(self.tls_version) + tls = f"[{self.tls_version}] " else: tls = "" @@ -86,11 +85,19 @@ def __hash__(self): return hash(self.id) + # Sans-io attributes. + state = 0 + sockname = ("", 0) + error = None + tls = None + certificate_list = () + alpn_offers = None + cipher_list = None + _stateobject_attributes = dict( id=str, address=tuple, tls_established=bool, - clientcert=certs.Cert, mitmcert=certs.Cert, timestamp_start=float, timestamp_tls_setup=float, @@ -100,8 +107,32 @@ alpn_proto_negotiated=bytes, tls_version=str, tls_extensions=typing.List[typing.Tuple[int, bytes]], + # sans-io exclusives + state=int, + sockname=tuple, + error=str, + tls=bool, + certificate_list=typing.List[certs.Cert], + alpn_offers=typing.List[bytes], + cipher_list=typing.List[str], ) + @property + def clientcert(self) -> typing.Optional[certs.Cert]: # pragma: no cover + warnings.warn(".clientcert is deprecated, use .certificate_list instead.", PendingDeprecationWarning) + if self.certificate_list: + return self.certificate_list[0] + else: + return None + + @clientcert.setter + def clientcert(self, val): # pragma: no cover + warnings.warn(".clientcert is deprecated, use .certificate_list instead.", PendingDeprecationWarning) + if val: + self.certificate_list = [val] + else: + self.certificate_list = [] + def send(self, message): if isinstance(message, list): message = b''.join(message) @@ -119,7 +150,6 @@ return cls.from_state(dict( id=str(uuid.uuid4()), address=address, - clientcert=None, mitmcert=None, tls_established=False, timestamp_start=None, @@ -130,6 +160,13 @@ alpn_proto_negotiated=None, tls_version=None, tls_extensions=None, + state=0, + sockname=("", 0), + error=None, + tls=False, + certificate_list=[], + alpn_offers=[], + cipher_list=[], )) def convert_to_tls(self, cert, *args, **kwargs): @@ -168,7 +205,6 @@ ip_address: Resolved remote IP address. source_address: Local IP address or client's source IP address. tls_established: True if TLS is established, False otherwise - cert: The certificate presented by the remote during the TLS handshake sni: Server Name Indication sent by the proxy during the TLS handshake alpn_proto_negotiated: The negotiated application protocol tls_version: TLS version @@ -221,13 +257,22 @@ def __hash__(self): return hash(self.id) + # Sans-io attributes. + state = 0 + error = None + tls = None + certificate_list = () + alpn_offers = None + cipher_name = None + cipher_list = None + via2 = None + _stateobject_attributes = dict( id=str, address=tuple, ip_address=tuple, source_address=tuple, tls_established=bool, - cert=certs.Cert, sni=str, alpn_proto_negotiated=bytes, tls_version=str, @@ -235,8 +280,33 @@ timestamp_tcp_setup=float, timestamp_tls_setup=float, timestamp_end=float, + # sans-io exclusives + state=int, + error=str, + tls=bool, + certificate_list=typing.List[certs.Cert], + alpn_offers=typing.List[bytes], + cipher_name=str, + cipher_list=typing.List[str], + via2=None, ) + @property + def cert(self) -> typing.Optional[certs.Cert]: # pragma: no cover + warnings.warn(".cert is deprecated, use .certificate_list instead.", PendingDeprecationWarning) + if self.certificate_list: + return self.certificate_list[0] + else: + return None + + @cert.setter + def cert(self, val): # pragma: no cover + warnings.warn(".cert is deprecated, use .certificate_list instead.", PendingDeprecationWarning) + if val: + self.certificate_list = [val] + else: + self.certificate_list = [] + @classmethod def from_state(cls, state): f = cls(tuple()) @@ -249,7 +319,6 @@ id=str(uuid.uuid4()), address=address, ip_address=address, - cert=None, sni=address[0], alpn_proto_negotiated=None, tls_version=None, @@ -259,7 +328,15 @@ timestamp_tcp_setup=None, timestamp_tls_setup=None, timestamp_end=None, - via=None + via=None, + state=0, + error=None, + tls=False, + certificate_list=[], + alpn_offers=[], + cipher_name=None, + cipher_list=[], + via2=None, )) def connect(self): diff -Nru mitmproxy-5.1.1/mitmproxy/contentviews/auto.py mitmproxy-6.0.2/mitmproxy/contentviews/auto.py --- mitmproxy-5.1.1/mitmproxy/contentviews/auto.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/contentviews/auto.py 2020-12-15 16:41:27.000000000 +0000 @@ -12,7 +12,7 @@ ctype = headers.get("content-type") if data and ctype: ct = http.parse_content_type(ctype) if ctype else None - ct = "%s/%s" % (ct[0], ct[1]) + ct = "{}/{}".format(ct[0], ct[1]) if ct in contentviews.content_types_map: return contentviews.content_types_map[ct][0](data, **metadata) elif strutils.is_xml(data): diff -Nru mitmproxy-5.1.1/mitmproxy/contentviews/image/image_parser.py mitmproxy-6.0.2/mitmproxy/contentviews/image/image_parser.py --- mitmproxy-5.1.1/mitmproxy/contentviews/image/image_parser.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/contentviews/image/image_parser.py 2020-12-15 16:41:27.000000000 +0000 @@ -15,7 +15,7 @@ img = png.Png(KaitaiStream(io.BytesIO(data))) parts = [ ('Format', 'Portable network graphics'), - ('Size', "{0} x {1} px".format(img.ihdr.width, img.ihdr.height)) + ('Size', f"{img.ihdr.width} x {img.ihdr.height} px") ] for chunk in img.chunks: if chunk.type == 'gAMA': @@ -23,7 +23,7 @@ elif chunk.type == 'pHYs': aspectx = chunk.body.pixels_per_unit_x aspecty = chunk.body.pixels_per_unit_y - parts.append(('aspect', "{0} x {1}".format(aspectx, aspecty))) + parts.append(('aspect', f"{aspectx} x {aspecty}")) elif chunk.type == 'tEXt': parts.append((chunk.body.keyword, chunk.body.text)) elif chunk.type == 'iTXt': @@ -38,8 +38,8 @@ descriptor = img.logical_screen_descriptor parts = [ ('Format', 'Compuserve GIF'), - ('Version', "GIF{}".format(img.hdr.version)), - ('Size', "{} x {} px".format(descriptor.screen_width, descriptor.screen_height)), + ('Version', f"GIF{img.hdr.version}"), + ('Size', f"{descriptor.screen_width} x {descriptor.screen_height} px"), ('background', str(descriptor.bg_color_index)) ] ext_blocks = [] @@ -66,10 +66,10 @@ ] for segment in img.segments: if segment.marker._name_ == 'sof0': - parts.append(('Size', "{0} x {1} px".format(segment.data.image_width, segment.data.image_height))) + parts.append(('Size', f"{segment.data.image_width} x {segment.data.image_height} px")) if segment.marker._name_ == 'app0': - parts.append(('jfif_version', "({0}, {1})".format(segment.data.version_major, segment.data.version_minor))) - parts.append(('jfif_density', "({0}, {1})".format(segment.data.density_x, segment.data.density_y))) + parts.append(('jfif_version', f"({segment.data.version_major}, {segment.data.version_minor})")) + parts.append(('jfif_density', f"({segment.data.density_x}, {segment.data.density_y})")) parts.append(('jfif_unit', str(segment.data.density_units._value_))) if segment.marker._name_ == 'com': parts.append(('comment', str(segment.data))) diff -Nru mitmproxy-5.1.1/mitmproxy/contentviews/image/view.py mitmproxy-6.0.2/mitmproxy/contentviews/image/view.py --- mitmproxy-5.1.1/mitmproxy/contentviews/image/view.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/contentviews/image/view.py 2020-12-15 16:41:27.000000000 +0000 @@ -41,7 +41,7 @@ ("Image Format", image_type or "unknown") ] if image_type: - view_name = "{} Image".format(image_type.upper()) + view_name = f"{image_type.upper()} Image" else: view_name = "Unknown Image" return view_name, base.format_dict(multidict.MultiDict(image_metadata)) diff -Nru mitmproxy-5.1.1/mitmproxy/contentviews/__init__.py mitmproxy-6.0.2/mitmproxy/contentviews/__init__.py --- mitmproxy-5.1.1/mitmproxy/contentviews/__init__.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/contentviews/__init__.py 2020-12-15 16:41:27.000000000 +0000 @@ -9,9 +9,10 @@ Thus, the View API is very minimalistic. The only arguments are `data` and `**metadata`, where `data` is the actual content (as bytes). The contents on -metadata depend on the protocol in use. For HTTP, the message headers are -passed as the ``headers`` keyword argument. For HTTP requests, the query -parameters are passed as the ``query`` keyword argument. +metadata depend on the protocol in use. For HTTP, the message headers and +message trailers are passed as the ``headers`` and ``trailers`` keyword +argument. For HTTP requests, the query parameters are passed as the ``query`` +keyword argument. """ import traceback from typing import Dict, Optional # noqa @@ -22,7 +23,7 @@ from mitmproxy.utils import strutils from . import ( auto, raw, hex, json, xml_html, wbxml, javascript, css, - urlencoded, multipart, image, query, protobuf + urlencoded, multipart, image, query, protobuf, msgpack ) from .base import View, KEY_MAX, format_text, format_dict, TViewResult @@ -103,6 +104,7 @@ metadata["query"] = message.query if isinstance(message, http.Message): metadata["headers"] = message.headers + metadata["trailers"] = message.trailers metadata["message"] = message metadata["flow"] = flow @@ -111,7 +113,20 @@ ) if enc: - description = "{} {}".format(enc, description) + description = f"{enc} {description}" + + return description, lines, error + + +def get_tcp_content_view(viewname: str, data: bytes): + viewmode = get(viewname) + if not viewmode: + viewmode = get("auto") + + # https://github.com/mitmproxy/mitmproxy/pull/3970#issuecomment-623024447 + assert viewmode + + description, lines, error = get_content_view(viewmode, data) return description, lines, error @@ -162,6 +177,7 @@ add(image.ViewImage()) add(query.ViewQuery()) add(protobuf.ViewProtobuf()) +add(msgpack.ViewMsgPack()) __all__ = [ "View", "KEY_MAX", "format_text", "format_dict", "TViewResult", diff -Nru mitmproxy-5.1.1/mitmproxy/contentviews/json.py mitmproxy-6.0.2/mitmproxy/contentviews/json.py --- mitmproxy-5.1.1/mitmproxy/contentviews/json.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/contentviews/json.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,16 +1,39 @@ +import re import json -from typing import Optional + +import typing from mitmproxy.contentviews import base +PARSE_ERROR = object() + -def pretty_json(s: bytes) -> Optional[bytes]: +def parse_json(s: bytes) -> typing.Any: try: - p = json.loads(s.decode('utf-8')) + return json.loads(s.decode('utf-8')) except ValueError: - return None - pretty = json.dumps(p, sort_keys=True, indent=4, ensure_ascii=False) - return pretty.encode("utf8", "strict") + return PARSE_ERROR + + +def format_json(data: typing.Any) -> typing.Iterator[base.TViewLine]: + encoder = json.JSONEncoder(indent=4, sort_keys=True, ensure_ascii=False) + current_line: base.TViewLine = [] + for chunk in encoder.iterencode(data): + if "\n" in chunk: + rest_of_last_line, chunk = chunk.split("\n", maxsplit=1) + # rest_of_last_line is a delimiter such as , or [ + current_line.append(('text', rest_of_last_line)) + yield current_line + current_line = [] + if re.match(r'\s*"', chunk): + current_line.append(('json_string', chunk)) + elif re.match(r'\s*\d', chunk): + current_line.append(('json_number', chunk)) + elif re.match(r'\s*(true|null|false)', chunk): + current_line.append(('json_boolean', chunk)) + else: + current_line.append(('text', chunk)) + yield current_line class ViewJSON(base.View): @@ -22,6 +45,6 @@ ] def __call__(self, data, **metadata): - pj = pretty_json(data) - if pj: - return "JSON", base.format_text(pj) + data = parse_json(data) + if data is not PARSE_ERROR: + return "JSON", format_json(data) diff -Nru mitmproxy-5.1.1/mitmproxy/contentviews/msgpack.py mitmproxy-6.0.2/mitmproxy/contentviews/msgpack.py --- mitmproxy-5.1.1/mitmproxy/contentviews/msgpack.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/contentviews/msgpack.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,50 @@ +import typing + +import msgpack + + +from mitmproxy.contentviews import base + +PARSE_ERROR = object() + + +def parse_msgpack(s: bytes) -> typing.Any: + try: + return msgpack.unpackb(s, raw=False) + except (ValueError, msgpack.ExtraData, msgpack.FormatError, msgpack.StackError): + return PARSE_ERROR + + +def pretty(value, htchar=" ", lfchar="\n", indent=0): + nlch = lfchar + htchar * (indent + 1) + if type(value) is dict: + items = [ + nlch + repr(key) + ": " + pretty(value[key], htchar, lfchar, indent + 1) + for key in value + ] + return "{%s}" % (",".join(items) + lfchar + htchar * indent) + elif type(value) is list: + items = [ + nlch + pretty(item, htchar, lfchar, indent + 1) + for item in value + ] + return "[%s]" % (",".join(items) + lfchar + htchar * indent) + else: + return repr(value) + + +def format_msgpack(data): + return base.format_text(pretty(data)) + + +class ViewMsgPack(base.View): + name = "MsgPack" + content_types = [ + "application/msgpack", + "application/x-msgpack", + ] + + def __call__(self, data, **metadata): + data = parse_msgpack(data) + if data is not PARSE_ERROR: + return "MsgPack", format_msgpack(data) diff -Nru mitmproxy-5.1.1/mitmproxy/contentviews/multipart.py mitmproxy-6.0.2/mitmproxy/contentviews/multipart.py --- mitmproxy-5.1.1/mitmproxy/contentviews/multipart.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/contentviews/multipart.py 2020-12-15 16:41:27.000000000 +0000 @@ -10,8 +10,7 @@ @staticmethod def _format(v): yield [("highlight", "Form data:\n")] - for message in base.format_dict(multidict.MultiDict(v)): - yield message + yield from base.format_dict(multidict.MultiDict(v)) def __call__(self, data, **metadata): headers = metadata.get("headers", {}) diff -Nru mitmproxy-5.1.1/mitmproxy/controller.py mitmproxy-6.0.2/mitmproxy/controller.py --- mitmproxy-5.1.1/mitmproxy/controller.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/controller.py 2020-12-15 16:41:27.000000000 +0000 @@ -89,7 +89,7 @@ """ if self.state != "start": raise exceptions.ControlException( - "Reply is {}, but expected it to be start.".format(self.state) + f"Reply is {self.state}, but expected it to be start." ) self._state = "taken" @@ -101,7 +101,7 @@ """ if self.state != "taken": raise exceptions.ControlException( - "Reply is {}, but expected it to be taken.".format(self.state) + f"Reply is {self.state}, but expected it to be taken." ) if not self.has_message: raise exceptions.ControlException("There is no reply message.") @@ -119,7 +119,7 @@ def send(self, msg, force=False): if self.state not in {"start", "taken"}: raise exceptions.ControlException( - "Reply is {}, but expected it to be start or taken.".format(self.state) + f"Reply is {self.state}, but expected it to be start or taken." ) if self.has_message and not force: raise exceptions.ControlException("There is already a reply message.") diff -Nru mitmproxy-5.1.1/mitmproxy/coretypes/bidi.py mitmproxy-6.0.2/mitmproxy/coretypes/bidi.py --- mitmproxy-5.1.1/mitmproxy/coretypes/bidi.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/coretypes/bidi.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,5 +1,3 @@ - - class BiDi: """ diff -Nru mitmproxy-5.1.1/mitmproxy/coretypes/multidict.py mitmproxy-6.0.2/mitmproxy/coretypes/multidict.py --- mitmproxy-5.1.1/mitmproxy/coretypes/multidict.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/coretypes/multidict.py 2020-12-15 16:41:27.000000000 +0000 @@ -60,7 +60,7 @@ yield key def __len__(self): - return len(set(self._kconv(key) for key, _ in self.fields)) + return len({self._kconv(key) for key, _ in self.fields}) def __eq__(self, other): if isinstance(other, MultiDict): diff -Nru mitmproxy-5.1.1/mitmproxy/coretypes/serializable.py mitmproxy-6.0.2/mitmproxy/coretypes/serializable.py --- mitmproxy-5.1.1/mitmproxy/coretypes/serializable.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/coretypes/serializable.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,5 +1,8 @@ import abc import uuid +from typing import Type, TypeVar + +T = TypeVar('T', bound='Serializable') class Serializable(metaclass=abc.ABCMeta): @@ -9,7 +12,7 @@ @classmethod @abc.abstractmethod - def from_state(cls, state): + def from_state(cls: Type[T], state) -> T: """ Create a new object from the given state. """ @@ -29,7 +32,7 @@ """ raise NotImplementedError() - def copy(self): + def copy(self: T) -> T: state = self.get_state() if isinstance(state, dict) and "id" in state: state["id"] = str(uuid.uuid4()) diff -Nru mitmproxy-5.1.1/mitmproxy/eventsequence.py mitmproxy-6.0.2/mitmproxy/eventsequence.py --- mitmproxy-5.1.1/mitmproxy/eventsequence.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/eventsequence.py 2020-12-15 16:41:27.000000000 +0000 @@ -90,6 +90,6 @@ try: e = _iterate_map[type(f)] except KeyError as err: - raise TypeError("Unknown flow type: {}".format(f)) from err + raise TypeError(f"Unknown flow type: {f}") from err else: yield from e(f) diff -Nru mitmproxy-5.1.1/mitmproxy/flowfilter.py mitmproxy-6.0.2/mitmproxy/flowfilter.py --- mitmproxy-5.1.1/mitmproxy/flowfilter.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/flowfilter.py 2020-12-15 16:41:27.000000000 +0000 @@ -39,10 +39,8 @@ import pyparsing as pp -from mitmproxy import flow -from mitmproxy import http -from mitmproxy import tcp -from mitmproxy import websocket +from mitmproxy import flow, http, tcp, websocket +from mitmproxy.net.websocket import check_handshake def only(*types): @@ -104,11 +102,15 @@ class FWebSocket(_Action): code = "websocket" - help = "Match WebSocket flows" + help = "Match WebSocket flows (and HTTP-WebSocket handshake flows)" - @only(websocket.WebSocketFlow) + @only(http.HTTPFlow, websocket.WebSocketFlow) def __call__(self, f): - return True + m = ( + (isinstance(f, http.HTTPFlow) and f.request and check_handshake(f.request.headers)) + or isinstance(f, websocket.WebSocketFlow) + ) + return m class FTCP(_Action): diff -Nru mitmproxy-5.1.1/mitmproxy/flow.py mitmproxy-6.0.2/mitmproxy/flow.py --- mitmproxy-5.1.1/mitmproxy/flow.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/flow.py 2020-12-15 16:41:27.000000000 +0000 @@ -2,14 +2,14 @@ import typing # noqa import uuid -from mitmproxy import connections -from mitmproxy import controller, exceptions # noqa +from mitmproxy import controller +from mitmproxy import exceptions from mitmproxy import stateobject from mitmproxy import version +from mitmproxy.utils import compat class Error(stateobject.StateObject): - """ An Error. @@ -24,6 +24,8 @@ timestamp: Seconds since the epoch """ + KILLED_MESSAGE = "Connection killed." + def __init__(self, msg: str, timestamp=None) -> None: """ @type msg: str @@ -62,8 +64,8 @@ def __init__( self, type: str, - client_conn: connections.ClientConnection, - server_conn: connections.ServerConnection, + client_conn: compat.Client, + server_conn: compat.Server, live: bool=None ) -> None: self.type = type @@ -77,15 +79,17 @@ self._backup: typing.Optional[Flow] = None self.reply: typing.Optional[controller.Reply] = None self.marked: bool = False + self.is_replay: typing.Optional[str] = None self.metadata: typing.Dict[str, typing.Any] = dict() _stateobject_attributes = dict( id=str, error=Error, - client_conn=connections.ClientConnection, - server_conn=connections.ServerConnection, + client_conn=compat.Client, + server_conn=compat.Server, type=str, intercepted=bool, + is_replay=str, marked=bool, metadata=typing.Dict[str, typing.Any], ) @@ -154,7 +158,7 @@ """ Kill this request. """ - self.error = Error("Connection killed") + self.error = Error(Error.KILLED_MESSAGE) self.intercepted = False self.reply.kill(force=True) self.live = False @@ -180,3 +184,8 @@ if self.reply.state == "taken": self.reply.ack() self.reply.commit() + + @property + def timestamp_start(self) -> float: + """Start time of the flow.""" + return self.client_conn.timestamp_start diff -Nru mitmproxy-5.1.1/mitmproxy/http.py mitmproxy-6.0.2/mitmproxy/http.py --- mitmproxy-5.1.1/mitmproxy/http.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/http.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,136 +1,13 @@ import html -from typing import Optional - -from mitmproxy import connections +import time +from typing import Optional, Tuple from mitmproxy import flow from mitmproxy import version from mitmproxy.net import http +from mitmproxy.utils import compat - -class HTTPRequest(http.Request): - """ - A mitmproxy HTTP request. - """ - - # This is a very thin wrapper on top of :py:class:`mitmproxy.net.http.Request` and - # may be removed in the future. - - def __init__( - self, - first_line_format, - method, - scheme, - host, - port, - path, - http_version, - headers, - content, - timestamp_start=None, - timestamp_end=None, - is_replay=False, - ): - http.Request.__init__( - self, - first_line_format, - method, - scheme, - host, - port, - path, - http_version, - headers, - content, - timestamp_start, - timestamp_end, - ) - # Is this request replayed? - self.is_replay = is_replay - self.stream = None - - def get_state(self): - state = super().get_state() - state["is_replay"] = self.is_replay - return state - - def set_state(self, state): - state = state.copy() - self.is_replay = state.pop("is_replay") - super().set_state(state) - - @classmethod - def wrap(self, request): - """ - Wraps an existing :py:class:`mitmproxy.net.http.Request`. - """ - req = HTTPRequest( - first_line_format=request.data.first_line_format, - method=request.data.method, - scheme=request.data.scheme, - host=request.data.host, - port=request.data.port, - path=request.data.path, - http_version=request.data.http_version, - headers=request.data.headers, - content=request.data.content, - timestamp_start=request.data.timestamp_start, - timestamp_end=request.data.timestamp_end, - ) - return req - - def __hash__(self): - return id(self) - - -class HTTPResponse(http.Response): - """ - A mitmproxy HTTP response. - """ - - # This is a very thin wrapper on top of :py:class:`mitmproxy.net.http.Response` and - # may be removed in the future. - - def __init__( - self, - http_version, - status_code, - reason, - headers, - content, - timestamp_start=None, - timestamp_end=None, - is_replay=False - ): - http.Response.__init__( - self, - http_version, - status_code, - reason, - headers, - content, - timestamp_start=timestamp_start, - timestamp_end=timestamp_end, - ) - - # Is this request replayed? - self.is_replay = is_replay - self.stream = None - - @classmethod - def wrap(self, response): - """ - Wraps an existing :py:class:`mitmproxy.net.http.Response`. - """ - resp = HTTPResponse( - http_version=response.data.http_version, - status_code=response.data.status_code, - reason=response.data.reason, - headers=response.data.headers, - content=response.data.content, - timestamp_start=response.data.timestamp_start, - timestamp_end=response.data.timestamp_end, - ) - return resp +HTTPRequest = http.Request +HTTPResponse = http.Response class HTTPFlow(flow.Flow): @@ -138,16 +15,16 @@ An HTTPFlow is a collection of objects representing a single HTTP transaction. """ - request: HTTPRequest - response: Optional[HTTPResponse] = None + request: http.Request + response: Optional[http.Response] = None error: Optional[flow.Error] = None """ Note that it's possible for a Flow to have both a response and an error object. This might happen, for instance, when a response was received from the server, but there was an error sending it back to the client. """ - server_conn: connections.ServerConnection - client_conn: connections.ClientConnection + server_conn: compat.Server + client_conn: compat.Client intercepted: bool = False """ Is this flow currently being intercepted? """ mode: str @@ -160,8 +37,8 @@ _stateobject_attributes = flow.Flow._stateobject_attributes.copy() # mypy doesn't support update with kwargs _stateobject_attributes.update(dict( - request=HTTPRequest, - response=HTTPResponse, + request=http.Request, + response=http.Response, mode=str )) @@ -169,10 +46,14 @@ s = " float: + return self.request.timestamp_start + def copy(self): f = super().copy() if self.request: @@ -181,27 +62,13 @@ f.response = self.response.copy() return f - def replace(self, pattern, repl, *args, **kwargs): - """ - Replaces a regular expression pattern with repl in both request and - response of the flow. Encoded content will be decoded before - replacement, and re-encoded afterwards. - - Returns the number of replacements made. - """ - c = self.request.replace(pattern, repl, *args, **kwargs) - if self.response: - c += self.response.replace(pattern, repl, *args, **kwargs) - return c - def make_error_response( status_code: int, message: str = "", headers: Optional[http.Headers] = None, -) -> HTTPResponse: - reason = http.status_codes.RESPONSES.get(status_code, "Unknown") - body = """ +) -> http.Response: + body: bytes = """ {status_code} {reason} @@ -213,7 +80,7 @@ """.strip().format( status_code=status_code, - reason=reason, + reason=http.status_codes.RESPONSES.get(status_code, "Unknown"), message=html.escape(message), ).encode("utf8", "replace") @@ -225,34 +92,40 @@ Content_Type="text/html" ) - return HTTPResponse( - b"HTTP/1.1", - status_code, - reason, - headers, - body, - ) + return http.Response.make(status_code, body, headers) -def make_connect_request(address): - return HTTPRequest( - "authority", b"CONNECT", None, address[0], address[1], None, b"HTTP/1.1", - http.Headers(), b"" +def make_connect_request(address: Tuple[str, int]) -> http.Request: + return http.Request( + host=address[0], + port=address[1], + method=b"CONNECT", + scheme=b"", + authority=f"{address[0]}:{address[1]}".encode(), + path=b"", + http_version=b"HTTP/1.1", + headers=http.Headers(), + content=b"", + trailers=None, + timestamp_start=time.time(), + timestamp_end=time.time(), ) def make_connect_response(http_version): # Do not send any response headers as it breaks proxying non-80 ports on # Android emulators using the -http-proxy option. - return HTTPResponse( + return http.Response( http_version, 200, b"Connection established", http.Headers(), b"", + None, + time.time(), + time.time(), ) -expect_continue_response = HTTPResponse( - b"HTTP/1.1", 100, b"Continue", http.Headers(), b"" -) +def make_expect_continue_response(): + return http.Response.make(100) diff -Nru mitmproxy-5.1.1/mitmproxy/__init__.py mitmproxy-6.0.2/mitmproxy/__init__.py --- mitmproxy-5.1.1/mitmproxy/__init__.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/__init__.py 2020-12-15 16:41:27.000000000 +0000 @@ -5,4 +5,5 @@ # workaround for # https://github.com/tornadoweb/tornado/issues/2751 # https://www.tornadoweb.org/en/stable/index.html#installation + # (copied multiple times in the codebase, please remove all occurrences) asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy()) diff -Nru mitmproxy-5.1.1/mitmproxy/io/compat.py mitmproxy-6.0.2/mitmproxy/io/compat.py --- mitmproxy-5.1.1/mitmproxy/io/compat.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/io/compat.py 2020-12-15 16:41:27.000000000 +0000 @@ -172,6 +172,64 @@ return data +def convert_7_8(data): + data["version"] = 8 + data["request"]["trailers"] = None + if data["response"] is not None: + data["response"]["trailers"] = None + return data + + +def convert_8_9(data): + data["version"] = 9 + data["request"].pop("first_line_format") + data["request"]["authority"] = b"" + is_request_replay = data["request"].pop("is_replay", False) + is_response_replay = False + if data["response"] is not None: + is_response_replay = data["response"].pop("is_replay", False) + if is_request_replay: # pragma: no cover + data["is_replay"] = "request" + elif is_response_replay: # pragma: no cover + data["is_replay"] = "response" + else: + data["is_replay"] = None + return data + + +def convert_9_10(data): + data["version"] = 10 + + def conv_conn(conn): + conn["state"] = 0 + conn["error"] = None + conn["tls"] = conn["tls_established"] + alpn = conn["alpn_proto_negotiated"] + conn["alpn_offers"] = [alpn] if alpn else None + cipher = conn["cipher_name"] + conn["cipher_list"] = [cipher] if cipher else None + + def conv_cconn(conn): + conn["sockname"] = ("", 0) + cc = conn.pop("clientcert", None) + conn["certificate_list"] = [cc] if cc else [] + conv_conn(conn) + + def conv_sconn(conn): + crt = conn.pop("cert", None) + conn["certificate_list"] = [crt] if crt else [] + conn["cipher_name"] = None + conn["via2"] = None + conv_conn(conn) + + conv_cconn(data["client_conn"]) + conv_sconn(data["server_conn"]) + if data["server_conn"]["via"]: + conv_sconn(data["server_conn"]["via"]) + + return data + + def _convert_dict_keys(o: Any) -> Any: if isinstance(o, dict): return {strutils.always_str(k): _convert_dict_keys(v) for k, v in o.items()} @@ -226,6 +284,9 @@ 4: convert_4_5, 5: convert_5_6, 6: convert_6_7, + 7: convert_7_8, + 8: convert_8_9, + 9: convert_9_10, } @@ -243,8 +304,8 @@ flow_data = converters[flow_version](flow_data) else: should_upgrade = ( - isinstance(flow_version, int) - and flow_version > version.FLOW_FORMAT_VERSION + isinstance(flow_version, int) + and flow_version > version.FLOW_FORMAT_VERSION ) raise ValueError( "{} cannot read files with flow format version {}{}.".format( diff -Nru mitmproxy-5.1.1/mitmproxy/io/db.py mitmproxy-6.0.2/mitmproxy/io/db.py --- mitmproxy-5.1.1/mitmproxy/io/db.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/io/db.py 2020-12-15 16:41:27.000000000 +0000 @@ -36,5 +36,5 @@ flows = [] self._c.execute('SELECT pbuf_blob FROM FLOWS') for row in self._c.fetchall(): - flows.append((protobuf.loads(row[0]))) + flows.append(protobuf.loads(row[0])) return flows diff -Nru mitmproxy-5.1.1/mitmproxy/io/__init__.py mitmproxy-6.0.2/mitmproxy/io/__init__.py --- mitmproxy-5.1.1/mitmproxy/io/__init__.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/io/__init__.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,4 +1,3 @@ - from .io import FlowWriter, FlowReader, FilteredFlowWriter, read_flows_from_paths from .db import DBHandler diff -Nru mitmproxy-5.1.1/mitmproxy/io/io.py mitmproxy-6.0.2/mitmproxy/io/io.py --- mitmproxy-5.1.1/mitmproxy/io/io.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/io/io.py 2020-12-15 16:41:27.000000000 +0000 @@ -82,6 +82,6 @@ path = os.path.expanduser(path) with open(path, "rb") as f: flows.extend(FlowReader(f).stream()) - except IOError as e: + except OSError as e: raise exceptions.FlowReadException(e.strerror) return flows diff -Nru mitmproxy-5.1.1/mitmproxy/io/proto/http_pb2.py mitmproxy-6.0.2/mitmproxy/io/proto/http_pb2.py --- mitmproxy-5.1.1/mitmproxy/io/proto/http_pb2.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/io/proto/http_pb2.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,13 +1,10 @@ # Generated by the protocol buffer compiler. DO NOT EDIT! # source: http.proto - -import sys -_b=sys.version_info[0]<3 and (lambda x:x) or (lambda x:x.encode('latin1')) +"""Generated protocol buffer code.""" from google.protobuf import descriptor as _descriptor from google.protobuf import message as _message from google.protobuf import reflection as _reflection from google.protobuf import symbol_database as _symbol_database -from google.protobuf import descriptor_pb2 # @@protoc_insertion_point(imports) _sym_db = _symbol_database.Default() @@ -19,7 +16,9 @@ name='http.proto', package='', syntax='proto2', - serialized_pb=_b('\n\nhttp.proto\"\xf4\x01\n\x08HTTPFlow\x12\x1d\n\x07request\x18\x01 \x01(\x0b\x32\x0c.HTTPRequest\x12\x1f\n\x08response\x18\x02 \x01(\x0b\x32\r.HTTPResponse\x12\x19\n\x05\x65rror\x18\x03 \x01(\x0b\x32\n.HTTPError\x12&\n\x0b\x63lient_conn\x18\x04 \x01(\x0b\x32\x11.ClientConnection\x12&\n\x0bserver_conn\x18\x05 \x01(\x0b\x32\x11.ServerConnection\x12\x13\n\x0bintercepted\x18\x06 \x01(\x08\x12\x0e\n\x06marked\x18\x07 \x01(\x08\x12\x0c\n\x04mode\x18\x08 \x01(\t\x12\n\n\x02id\x18\t \x01(\t\"\xfa\x01\n\x0bHTTPRequest\x12\x19\n\x11\x66irst_line_format\x18\x01 \x01(\t\x12\x0e\n\x06method\x18\x02 \x01(\t\x12\x0e\n\x06scheme\x18\x03 \x01(\t\x12\x0c\n\x04host\x18\x04 \x01(\t\x12\x0c\n\x04port\x18\x05 \x01(\x05\x12\x0c\n\x04path\x18\x06 \x01(\t\x12\x14\n\x0chttp_version\x18\x07 \x01(\t\x12\x1c\n\x07headers\x18\x08 \x03(\x0b\x32\x0b.HTTPHeader\x12\x0f\n\x07\x63ontent\x18\t \x01(\x0c\x12\x17\n\x0ftimestamp_start\x18\n \x01(\x01\x12\x15\n\rtimestamp_end\x18\x0b \x01(\x01\x12\x11\n\tis_replay\x18\x0c \x01(\x08\"\xbb\x01\n\x0cHTTPResponse\x12\x14\n\x0chttp_version\x18\x01 \x01(\t\x12\x13\n\x0bstatus_code\x18\x02 \x01(\x05\x12\x0e\n\x06reason\x18\x03 \x01(\t\x12\x1c\n\x07headers\x18\x04 \x03(\x0b\x32\x0b.HTTPHeader\x12\x0f\n\x07\x63ontent\x18\x05 \x01(\x0c\x12\x17\n\x0ftimestamp_start\x18\x06 \x01(\x01\x12\x15\n\rtimestamp_end\x18\x07 \x01(\x01\x12\x11\n\tis_replay\x18\x08 \x01(\x08\"+\n\tHTTPError\x12\x0b\n\x03msg\x18\x01 \x01(\t\x12\x11\n\ttimestamp\x18\x02 \x01(\x01\")\n\nHTTPHeader\x12\x0c\n\x04name\x18\x01 \x01(\t\x12\r\n\x05value\x18\x02 \x01(\t\"%\n\x07\x41\x64\x64ress\x12\x0c\n\x04host\x18\x01 \x01(\t\x12\x0c\n\x04port\x18\x02 \x01(\x05\"\xc2\x02\n\x10\x43lientConnection\x12\n\n\x02id\x18\x01 \x01(\t\x12\x19\n\x07\x61\x64\x64ress\x18\x02 \x01(\x0b\x32\x08.Address\x12\x17\n\x0ftls_established\x18\x03 \x01(\x08\x12\x12\n\nclientcert\x18\x04 \x01(\t\x12\x10\n\x08mitmcert\x18\x05 \x01(\t\x12\x17\n\x0ftimestamp_start\x18\x06 \x01(\x01\x12\x1b\n\x13timestamp_tls_setup\x18\x07 \x01(\x01\x12\x15\n\rtimestamp_end\x18\x08 \x01(\x01\x12\x0b\n\x03sni\x18\t \x01(\t\x12\x13\n\x0b\x63ipher_name\x18\n \x01(\t\x12\x1d\n\x15\x61lpn_proto_negotiated\x18\x0b \x01(\x0c\x12\x13\n\x0btls_version\x18\x0c \x01(\t\x12%\n\x0etls_extensions\x18\r \x03(\x0b\x32\r.TLSExtension\"\xeb\x02\n\x10ServerConnection\x12\n\n\x02id\x18\x01 \x01(\t\x12\x19\n\x07\x61\x64\x64ress\x18\x02 \x01(\x0b\x32\x08.Address\x12\x1c\n\nip_address\x18\x03 \x01(\x0b\x32\x08.Address\x12 \n\x0esource_address\x18\x04 \x01(\x0b\x32\x08.Address\x12\x17\n\x0ftls_established\x18\x05 \x01(\x08\x12\x0c\n\x04\x63\x65rt\x18\x06 \x01(\t\x12\x0b\n\x03sni\x18\x07 \x01(\t\x12\x1d\n\x15\x61lpn_proto_negotiated\x18\x08 \x01(\x0c\x12\x13\n\x0btls_version\x18\t \x01(\t\x12\x17\n\x0ftimestamp_start\x18\n \x01(\x01\x12\x1b\n\x13timestamp_tcp_setup\x18\x0b \x01(\x01\x12\x1b\n\x13timestamp_tls_setup\x18\x0c \x01(\x01\x12\x15\n\rtimestamp_end\x18\r \x01(\x01\x12\x1e\n\x03via\x18\x0e \x01(\x0b\x32\x11.ServerConnection\"*\n\x0cTLSExtension\x12\x0b\n\x03int\x18\x01 \x01(\x03\x12\r\n\x05\x62ytes\x18\x02 \x01(\x0c') + serialized_options=None, + create_key=_descriptor._internal_create_key, + serialized_pb=b'\n\nhttp.proto\"\xf4\x01\n\x08HTTPFlow\x12\x1d\n\x07request\x18\x01 \x01(\x0b\x32\x0c.HTTPRequest\x12\x1f\n\x08response\x18\x02 \x01(\x0b\x32\r.HTTPResponse\x12\x19\n\x05\x65rror\x18\x03 \x01(\x0b\x32\n.HTTPError\x12&\n\x0b\x63lient_conn\x18\x04 \x01(\x0b\x32\x11.ClientConnection\x12&\n\x0bserver_conn\x18\x05 \x01(\x0b\x32\x11.ServerConnection\x12\x13\n\x0bintercepted\x18\x06 \x01(\x08\x12\x0e\n\x06marked\x18\x07 \x01(\x08\x12\x0c\n\x04mode\x18\x08 \x01(\t\x12\n\n\x02id\x18\t \x01(\t\"\xfa\x01\n\x0bHTTPRequest\x12\x19\n\x11\x66irst_line_format\x18\x01 \x01(\t\x12\x0e\n\x06method\x18\x02 \x01(\t\x12\x0e\n\x06scheme\x18\x03 \x01(\t\x12\x0c\n\x04host\x18\x04 \x01(\t\x12\x0c\n\x04port\x18\x05 \x01(\x05\x12\x0c\n\x04path\x18\x06 \x01(\t\x12\x14\n\x0chttp_version\x18\x07 \x01(\t\x12\x1c\n\x07headers\x18\x08 \x03(\x0b\x32\x0b.HTTPHeader\x12\x0f\n\x07\x63ontent\x18\t \x01(\x0c\x12\x17\n\x0ftimestamp_start\x18\n \x01(\x01\x12\x15\n\rtimestamp_end\x18\x0b \x01(\x01\x12\x11\n\tis_replay\x18\x0c \x01(\x08\"\xbb\x01\n\x0cHTTPResponse\x12\x14\n\x0chttp_version\x18\x01 \x01(\t\x12\x13\n\x0bstatus_code\x18\x02 \x01(\x05\x12\x0e\n\x06reason\x18\x03 \x01(\t\x12\x1c\n\x07headers\x18\x04 \x03(\x0b\x32\x0b.HTTPHeader\x12\x0f\n\x07\x63ontent\x18\x05 \x01(\x0c\x12\x17\n\x0ftimestamp_start\x18\x06 \x01(\x01\x12\x15\n\rtimestamp_end\x18\x07 \x01(\x01\x12\x11\n\tis_replay\x18\x08 \x01(\x08\"+\n\tHTTPError\x12\x0b\n\x03msg\x18\x01 \x01(\t\x12\x11\n\ttimestamp\x18\x02 \x01(\x01\")\n\nHTTPHeader\x12\x0c\n\x04name\x18\x01 \x01(\t\x12\r\n\x05value\x18\x02 \x01(\t\"%\n\x07\x41\x64\x64ress\x12\x0c\n\x04host\x18\x01 \x01(\t\x12\x0c\n\x04port\x18\x02 \x01(\x05\"\xc2\x02\n\x10\x43lientConnection\x12\n\n\x02id\x18\x01 \x01(\t\x12\x19\n\x07\x61\x64\x64ress\x18\x02 \x01(\x0b\x32\x08.Address\x12\x17\n\x0ftls_established\x18\x03 \x01(\x08\x12\x12\n\nclientcert\x18\x04 \x01(\t\x12\x10\n\x08mitmcert\x18\x05 \x01(\t\x12\x17\n\x0ftimestamp_start\x18\x06 \x01(\x01\x12\x1b\n\x13timestamp_tls_setup\x18\x07 \x01(\x01\x12\x15\n\rtimestamp_end\x18\x08 \x01(\x01\x12\x0b\n\x03sni\x18\t \x01(\t\x12\x13\n\x0b\x63ipher_name\x18\n \x01(\t\x12\x1d\n\x15\x61lpn_proto_negotiated\x18\x0b \x01(\x0c\x12\x13\n\x0btls_version\x18\x0c \x01(\t\x12%\n\x0etls_extensions\x18\r \x03(\x0b\x32\r.TLSExtension\"\xeb\x02\n\x10ServerConnection\x12\n\n\x02id\x18\x01 \x01(\t\x12\x19\n\x07\x61\x64\x64ress\x18\x02 \x01(\x0b\x32\x08.Address\x12\x1c\n\nip_address\x18\x03 \x01(\x0b\x32\x08.Address\x12 \n\x0esource_address\x18\x04 \x01(\x0b\x32\x08.Address\x12\x17\n\x0ftls_established\x18\x05 \x01(\x08\x12\x0c\n\x04\x63\x65rt\x18\x06 \x01(\t\x12\x0b\n\x03sni\x18\x07 \x01(\t\x12\x1d\n\x15\x61lpn_proto_negotiated\x18\x08 \x01(\x0c\x12\x13\n\x0btls_version\x18\t \x01(\t\x12\x17\n\x0ftimestamp_start\x18\n \x01(\x01\x12\x1b\n\x13timestamp_tcp_setup\x18\x0b \x01(\x01\x12\x1b\n\x13timestamp_tls_setup\x18\x0c \x01(\x01\x12\x15\n\rtimestamp_end\x18\r \x01(\x01\x12\x1e\n\x03via\x18\x0e \x01(\x0b\x32\x11.ServerConnection\"*\n\x0cTLSExtension\x12\x0b\n\x03int\x18\x01 \x01(\x03\x12\r\n\x05\x62ytes\x18\x02 \x01(\x0c' ) @@ -31,6 +30,7 @@ filename=None, file=DESCRIPTOR, containing_type=None, + create_key=_descriptor._internal_create_key, fields=[ _descriptor.FieldDescriptor( name='request', full_name='HTTPFlow.request', index=0, @@ -38,70 +38,70 @@ has_default_value=False, default_value=None, message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), _descriptor.FieldDescriptor( name='response', full_name='HTTPFlow.response', index=1, number=2, type=11, cpp_type=10, label=1, has_default_value=False, default_value=None, message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), _descriptor.FieldDescriptor( name='error', full_name='HTTPFlow.error', index=2, number=3, type=11, cpp_type=10, label=1, has_default_value=False, default_value=None, message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), _descriptor.FieldDescriptor( name='client_conn', full_name='HTTPFlow.client_conn', index=3, number=4, type=11, cpp_type=10, label=1, has_default_value=False, default_value=None, message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), _descriptor.FieldDescriptor( name='server_conn', full_name='HTTPFlow.server_conn', index=4, number=5, type=11, cpp_type=10, label=1, has_default_value=False, default_value=None, message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), _descriptor.FieldDescriptor( name='intercepted', full_name='HTTPFlow.intercepted', index=5, number=6, type=8, cpp_type=7, label=1, has_default_value=False, default_value=False, message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), _descriptor.FieldDescriptor( name='marked', full_name='HTTPFlow.marked', index=6, number=7, type=8, cpp_type=7, label=1, has_default_value=False, default_value=False, message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), _descriptor.FieldDescriptor( name='mode', full_name='HTTPFlow.mode', index=7, number=8, type=9, cpp_type=9, label=1, - has_default_value=False, default_value=_b("").decode('utf-8'), + has_default_value=False, default_value=b"".decode('utf-8'), message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), _descriptor.FieldDescriptor( name='id', full_name='HTTPFlow.id', index=8, number=9, type=9, cpp_type=9, label=1, - has_default_value=False, default_value=_b("").decode('utf-8'), + has_default_value=False, default_value=b"".decode('utf-8'), message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), ], extensions=[ ], nested_types=[], enum_types=[ ], - options=None, + serialized_options=None, is_extendable=False, syntax='proto2', extension_ranges=[], @@ -118,98 +118,99 @@ filename=None, file=DESCRIPTOR, containing_type=None, + create_key=_descriptor._internal_create_key, fields=[ _descriptor.FieldDescriptor( name='first_line_format', full_name='HTTPRequest.first_line_format', index=0, number=1, type=9, cpp_type=9, label=1, - has_default_value=False, default_value=_b("").decode('utf-8'), + has_default_value=False, default_value=b"".decode('utf-8'), message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), _descriptor.FieldDescriptor( name='method', full_name='HTTPRequest.method', index=1, number=2, type=9, cpp_type=9, label=1, - has_default_value=False, default_value=_b("").decode('utf-8'), + has_default_value=False, default_value=b"".decode('utf-8'), message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), _descriptor.FieldDescriptor( name='scheme', full_name='HTTPRequest.scheme', index=2, number=3, type=9, cpp_type=9, label=1, - has_default_value=False, default_value=_b("").decode('utf-8'), + has_default_value=False, default_value=b"".decode('utf-8'), message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), _descriptor.FieldDescriptor( name='host', full_name='HTTPRequest.host', index=3, number=4, type=9, cpp_type=9, label=1, - has_default_value=False, default_value=_b("").decode('utf-8'), + has_default_value=False, default_value=b"".decode('utf-8'), message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), _descriptor.FieldDescriptor( name='port', full_name='HTTPRequest.port', index=4, number=5, type=5, cpp_type=1, label=1, has_default_value=False, default_value=0, message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), _descriptor.FieldDescriptor( name='path', full_name='HTTPRequest.path', index=5, number=6, type=9, cpp_type=9, label=1, - has_default_value=False, default_value=_b("").decode('utf-8'), + has_default_value=False, default_value=b"".decode('utf-8'), message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), _descriptor.FieldDescriptor( name='http_version', full_name='HTTPRequest.http_version', index=6, number=7, type=9, cpp_type=9, label=1, - has_default_value=False, default_value=_b("").decode('utf-8'), + has_default_value=False, default_value=b"".decode('utf-8'), message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), _descriptor.FieldDescriptor( name='headers', full_name='HTTPRequest.headers', index=7, number=8, type=11, cpp_type=10, label=3, has_default_value=False, default_value=[], message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), _descriptor.FieldDescriptor( name='content', full_name='HTTPRequest.content', index=8, number=9, type=12, cpp_type=9, label=1, - has_default_value=False, default_value=_b(""), + has_default_value=False, default_value=b"", message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), _descriptor.FieldDescriptor( name='timestamp_start', full_name='HTTPRequest.timestamp_start', index=9, number=10, type=1, cpp_type=5, label=1, has_default_value=False, default_value=float(0), message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), _descriptor.FieldDescriptor( name='timestamp_end', full_name='HTTPRequest.timestamp_end', index=10, number=11, type=1, cpp_type=5, label=1, has_default_value=False, default_value=float(0), message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), _descriptor.FieldDescriptor( name='is_replay', full_name='HTTPRequest.is_replay', index=11, number=12, type=8, cpp_type=7, label=1, has_default_value=False, default_value=False, message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), ], extensions=[ ], nested_types=[], enum_types=[ ], - options=None, + serialized_options=None, is_extendable=False, syntax='proto2', extension_ranges=[], @@ -226,70 +227,71 @@ filename=None, file=DESCRIPTOR, containing_type=None, + create_key=_descriptor._internal_create_key, fields=[ _descriptor.FieldDescriptor( name='http_version', full_name='HTTPResponse.http_version', index=0, number=1, type=9, cpp_type=9, label=1, - has_default_value=False, default_value=_b("").decode('utf-8'), + has_default_value=False, default_value=b"".decode('utf-8'), message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), _descriptor.FieldDescriptor( name='status_code', full_name='HTTPResponse.status_code', index=1, number=2, type=5, cpp_type=1, label=1, has_default_value=False, default_value=0, message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), _descriptor.FieldDescriptor( name='reason', full_name='HTTPResponse.reason', index=2, number=3, type=9, cpp_type=9, label=1, - has_default_value=False, default_value=_b("").decode('utf-8'), + has_default_value=False, default_value=b"".decode('utf-8'), message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), _descriptor.FieldDescriptor( name='headers', full_name='HTTPResponse.headers', index=3, number=4, type=11, cpp_type=10, label=3, has_default_value=False, default_value=[], message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), _descriptor.FieldDescriptor( name='content', full_name='HTTPResponse.content', index=4, number=5, type=12, cpp_type=9, label=1, - has_default_value=False, default_value=_b(""), + has_default_value=False, default_value=b"", message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), _descriptor.FieldDescriptor( name='timestamp_start', full_name='HTTPResponse.timestamp_start', index=5, number=6, type=1, cpp_type=5, label=1, has_default_value=False, default_value=float(0), message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), _descriptor.FieldDescriptor( name='timestamp_end', full_name='HTTPResponse.timestamp_end', index=6, number=7, type=1, cpp_type=5, label=1, has_default_value=False, default_value=float(0), message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), _descriptor.FieldDescriptor( name='is_replay', full_name='HTTPResponse.is_replay', index=7, number=8, type=8, cpp_type=7, label=1, has_default_value=False, default_value=False, message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), ], extensions=[ ], nested_types=[], enum_types=[ ], - options=None, + serialized_options=None, is_extendable=False, syntax='proto2', extension_ranges=[], @@ -306,28 +308,29 @@ filename=None, file=DESCRIPTOR, containing_type=None, + create_key=_descriptor._internal_create_key, fields=[ _descriptor.FieldDescriptor( name='msg', full_name='HTTPError.msg', index=0, number=1, type=9, cpp_type=9, label=1, - has_default_value=False, default_value=_b("").decode('utf-8'), + has_default_value=False, default_value=b"".decode('utf-8'), message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), _descriptor.FieldDescriptor( name='timestamp', full_name='HTTPError.timestamp', index=1, number=2, type=1, cpp_type=5, label=1, has_default_value=False, default_value=float(0), message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), ], extensions=[ ], nested_types=[], enum_types=[ ], - options=None, + serialized_options=None, is_extendable=False, syntax='proto2', extension_ranges=[], @@ -344,28 +347,29 @@ filename=None, file=DESCRIPTOR, containing_type=None, + create_key=_descriptor._internal_create_key, fields=[ _descriptor.FieldDescriptor( name='name', full_name='HTTPHeader.name', index=0, number=1, type=9, cpp_type=9, label=1, - has_default_value=False, default_value=_b("").decode('utf-8'), + has_default_value=False, default_value=b"".decode('utf-8'), message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), _descriptor.FieldDescriptor( name='value', full_name='HTTPHeader.value', index=1, number=2, type=9, cpp_type=9, label=1, - has_default_value=False, default_value=_b("").decode('utf-8'), + has_default_value=False, default_value=b"".decode('utf-8'), message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), ], extensions=[ ], nested_types=[], enum_types=[ ], - options=None, + serialized_options=None, is_extendable=False, syntax='proto2', extension_ranges=[], @@ -382,28 +386,29 @@ filename=None, file=DESCRIPTOR, containing_type=None, + create_key=_descriptor._internal_create_key, fields=[ _descriptor.FieldDescriptor( name='host', full_name='Address.host', index=0, number=1, type=9, cpp_type=9, label=1, - has_default_value=False, default_value=_b("").decode('utf-8'), + has_default_value=False, default_value=b"".decode('utf-8'), message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), _descriptor.FieldDescriptor( name='port', full_name='Address.port', index=1, number=2, type=5, cpp_type=1, label=1, has_default_value=False, default_value=0, message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), ], extensions=[ ], nested_types=[], enum_types=[ ], - options=None, + serialized_options=None, is_extendable=False, syntax='proto2', extension_ranges=[], @@ -420,105 +425,106 @@ filename=None, file=DESCRIPTOR, containing_type=None, + create_key=_descriptor._internal_create_key, fields=[ _descriptor.FieldDescriptor( name='id', full_name='ClientConnection.id', index=0, number=1, type=9, cpp_type=9, label=1, - has_default_value=False, default_value=_b("").decode('utf-8'), + has_default_value=False, default_value=b"".decode('utf-8'), message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), _descriptor.FieldDescriptor( name='address', full_name='ClientConnection.address', index=1, number=2, type=11, cpp_type=10, label=1, has_default_value=False, default_value=None, message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), _descriptor.FieldDescriptor( name='tls_established', full_name='ClientConnection.tls_established', index=2, number=3, type=8, cpp_type=7, label=1, has_default_value=False, default_value=False, message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), _descriptor.FieldDescriptor( name='clientcert', full_name='ClientConnection.clientcert', index=3, number=4, type=9, cpp_type=9, label=1, - has_default_value=False, default_value=_b("").decode('utf-8'), + has_default_value=False, default_value=b"".decode('utf-8'), message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), _descriptor.FieldDescriptor( name='mitmcert', full_name='ClientConnection.mitmcert', index=4, number=5, type=9, cpp_type=9, label=1, - has_default_value=False, default_value=_b("").decode('utf-8'), + has_default_value=False, default_value=b"".decode('utf-8'), message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), _descriptor.FieldDescriptor( name='timestamp_start', full_name='ClientConnection.timestamp_start', index=5, number=6, type=1, cpp_type=5, label=1, has_default_value=False, default_value=float(0), message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), _descriptor.FieldDescriptor( name='timestamp_tls_setup', full_name='ClientConnection.timestamp_tls_setup', index=6, number=7, type=1, cpp_type=5, label=1, has_default_value=False, default_value=float(0), message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), _descriptor.FieldDescriptor( name='timestamp_end', full_name='ClientConnection.timestamp_end', index=7, number=8, type=1, cpp_type=5, label=1, has_default_value=False, default_value=float(0), message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), _descriptor.FieldDescriptor( name='sni', full_name='ClientConnection.sni', index=8, number=9, type=9, cpp_type=9, label=1, - has_default_value=False, default_value=_b("").decode('utf-8'), + has_default_value=False, default_value=b"".decode('utf-8'), message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), _descriptor.FieldDescriptor( name='cipher_name', full_name='ClientConnection.cipher_name', index=9, number=10, type=9, cpp_type=9, label=1, - has_default_value=False, default_value=_b("").decode('utf-8'), + has_default_value=False, default_value=b"".decode('utf-8'), message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), _descriptor.FieldDescriptor( name='alpn_proto_negotiated', full_name='ClientConnection.alpn_proto_negotiated', index=10, number=11, type=12, cpp_type=9, label=1, - has_default_value=False, default_value=_b(""), + has_default_value=False, default_value=b"", message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), _descriptor.FieldDescriptor( name='tls_version', full_name='ClientConnection.tls_version', index=11, number=12, type=9, cpp_type=9, label=1, - has_default_value=False, default_value=_b("").decode('utf-8'), + has_default_value=False, default_value=b"".decode('utf-8'), message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), _descriptor.FieldDescriptor( name='tls_extensions', full_name='ClientConnection.tls_extensions', index=12, number=13, type=11, cpp_type=10, label=3, has_default_value=False, default_value=[], message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), ], extensions=[ ], nested_types=[], enum_types=[ ], - options=None, + serialized_options=None, is_extendable=False, syntax='proto2', extension_ranges=[], @@ -535,112 +541,113 @@ filename=None, file=DESCRIPTOR, containing_type=None, + create_key=_descriptor._internal_create_key, fields=[ _descriptor.FieldDescriptor( name='id', full_name='ServerConnection.id', index=0, number=1, type=9, cpp_type=9, label=1, - has_default_value=False, default_value=_b("").decode('utf-8'), + has_default_value=False, default_value=b"".decode('utf-8'), message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), _descriptor.FieldDescriptor( name='address', full_name='ServerConnection.address', index=1, number=2, type=11, cpp_type=10, label=1, has_default_value=False, default_value=None, message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), _descriptor.FieldDescriptor( name='ip_address', full_name='ServerConnection.ip_address', index=2, number=3, type=11, cpp_type=10, label=1, has_default_value=False, default_value=None, message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), _descriptor.FieldDescriptor( name='source_address', full_name='ServerConnection.source_address', index=3, number=4, type=11, cpp_type=10, label=1, has_default_value=False, default_value=None, message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), _descriptor.FieldDescriptor( name='tls_established', full_name='ServerConnection.tls_established', index=4, number=5, type=8, cpp_type=7, label=1, has_default_value=False, default_value=False, message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), _descriptor.FieldDescriptor( name='cert', full_name='ServerConnection.cert', index=5, number=6, type=9, cpp_type=9, label=1, - has_default_value=False, default_value=_b("").decode('utf-8'), + has_default_value=False, default_value=b"".decode('utf-8'), message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), _descriptor.FieldDescriptor( name='sni', full_name='ServerConnection.sni', index=6, number=7, type=9, cpp_type=9, label=1, - has_default_value=False, default_value=_b("").decode('utf-8'), + has_default_value=False, default_value=b"".decode('utf-8'), message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), _descriptor.FieldDescriptor( name='alpn_proto_negotiated', full_name='ServerConnection.alpn_proto_negotiated', index=7, number=8, type=12, cpp_type=9, label=1, - has_default_value=False, default_value=_b(""), + has_default_value=False, default_value=b"", message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), _descriptor.FieldDescriptor( name='tls_version', full_name='ServerConnection.tls_version', index=8, number=9, type=9, cpp_type=9, label=1, - has_default_value=False, default_value=_b("").decode('utf-8'), + has_default_value=False, default_value=b"".decode('utf-8'), message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), _descriptor.FieldDescriptor( name='timestamp_start', full_name='ServerConnection.timestamp_start', index=9, number=10, type=1, cpp_type=5, label=1, has_default_value=False, default_value=float(0), message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), _descriptor.FieldDescriptor( name='timestamp_tcp_setup', full_name='ServerConnection.timestamp_tcp_setup', index=10, number=11, type=1, cpp_type=5, label=1, has_default_value=False, default_value=float(0), message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), _descriptor.FieldDescriptor( name='timestamp_tls_setup', full_name='ServerConnection.timestamp_tls_setup', index=11, number=12, type=1, cpp_type=5, label=1, has_default_value=False, default_value=float(0), message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), _descriptor.FieldDescriptor( name='timestamp_end', full_name='ServerConnection.timestamp_end', index=12, number=13, type=1, cpp_type=5, label=1, has_default_value=False, default_value=float(0), message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), _descriptor.FieldDescriptor( name='via', full_name='ServerConnection.via', index=13, number=14, type=11, cpp_type=10, label=1, has_default_value=False, default_value=None, message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), ], extensions=[ ], nested_types=[], enum_types=[ ], - options=None, + serialized_options=None, is_extendable=False, syntax='proto2', extension_ranges=[], @@ -657,6 +664,7 @@ filename=None, file=DESCRIPTOR, containing_type=None, + create_key=_descriptor._internal_create_key, fields=[ _descriptor.FieldDescriptor( name='int', full_name='TLSExtension.int', index=0, @@ -664,21 +672,21 @@ has_default_value=False, default_value=0, message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), _descriptor.FieldDescriptor( name='bytes', full_name='TLSExtension.bytes', index=1, number=2, type=12, cpp_type=9, label=1, - has_default_value=False, default_value=_b(""), + has_default_value=False, default_value=b"", message_type=None, enum_type=None, containing_type=None, is_extension=False, extension_scope=None, - options=None, file=DESCRIPTOR), + serialized_options=None, file=DESCRIPTOR, create_key=_descriptor._internal_create_key), ], extensions=[ ], nested_types=[], enum_types=[ ], - options=None, + serialized_options=None, is_extendable=False, syntax='proto2', extension_ranges=[], @@ -712,67 +720,67 @@ DESCRIPTOR.message_types_by_name['TLSExtension'] = _TLSEXTENSION _sym_db.RegisterFileDescriptor(DESCRIPTOR) -HTTPFlow = _reflection.GeneratedProtocolMessageType('HTTPFlow', (_message.Message,), dict( - DESCRIPTOR = _HTTPFLOW, - __module__ = 'http_pb2' +HTTPFlow = _reflection.GeneratedProtocolMessageType('HTTPFlow', (_message.Message,), { + 'DESCRIPTOR' : _HTTPFLOW, + '__module__' : 'http_pb2' # @@protoc_insertion_point(class_scope:HTTPFlow) - )) + }) _sym_db.RegisterMessage(HTTPFlow) -HTTPRequest = _reflection.GeneratedProtocolMessageType('HTTPRequest', (_message.Message,), dict( - DESCRIPTOR = _HTTPREQUEST, - __module__ = 'http_pb2' +HTTPRequest = _reflection.GeneratedProtocolMessageType('HTTPRequest', (_message.Message,), { + 'DESCRIPTOR' : _HTTPREQUEST, + '__module__' : 'http_pb2' # @@protoc_insertion_point(class_scope:HTTPRequest) - )) + }) _sym_db.RegisterMessage(HTTPRequest) -HTTPResponse = _reflection.GeneratedProtocolMessageType('HTTPResponse', (_message.Message,), dict( - DESCRIPTOR = _HTTPRESPONSE, - __module__ = 'http_pb2' +HTTPResponse = _reflection.GeneratedProtocolMessageType('HTTPResponse', (_message.Message,), { + 'DESCRIPTOR' : _HTTPRESPONSE, + '__module__' : 'http_pb2' # @@protoc_insertion_point(class_scope:HTTPResponse) - )) + }) _sym_db.RegisterMessage(HTTPResponse) -HTTPError = _reflection.GeneratedProtocolMessageType('HTTPError', (_message.Message,), dict( - DESCRIPTOR = _HTTPERROR, - __module__ = 'http_pb2' +HTTPError = _reflection.GeneratedProtocolMessageType('HTTPError', (_message.Message,), { + 'DESCRIPTOR' : _HTTPERROR, + '__module__' : 'http_pb2' # @@protoc_insertion_point(class_scope:HTTPError) - )) + }) _sym_db.RegisterMessage(HTTPError) -HTTPHeader = _reflection.GeneratedProtocolMessageType('HTTPHeader', (_message.Message,), dict( - DESCRIPTOR = _HTTPHEADER, - __module__ = 'http_pb2' +HTTPHeader = _reflection.GeneratedProtocolMessageType('HTTPHeader', (_message.Message,), { + 'DESCRIPTOR' : _HTTPHEADER, + '__module__' : 'http_pb2' # @@protoc_insertion_point(class_scope:HTTPHeader) - )) + }) _sym_db.RegisterMessage(HTTPHeader) -Address = _reflection.GeneratedProtocolMessageType('Address', (_message.Message,), dict( - DESCRIPTOR = _ADDRESS, - __module__ = 'http_pb2' +Address = _reflection.GeneratedProtocolMessageType('Address', (_message.Message,), { + 'DESCRIPTOR' : _ADDRESS, + '__module__' : 'http_pb2' # @@protoc_insertion_point(class_scope:Address) - )) + }) _sym_db.RegisterMessage(Address) -ClientConnection = _reflection.GeneratedProtocolMessageType('ClientConnection', (_message.Message,), dict( - DESCRIPTOR = _CLIENTCONNECTION, - __module__ = 'http_pb2' +ClientConnection = _reflection.GeneratedProtocolMessageType('ClientConnection', (_message.Message,), { + 'DESCRIPTOR' : _CLIENTCONNECTION, + '__module__' : 'http_pb2' # @@protoc_insertion_point(class_scope:ClientConnection) - )) + }) _sym_db.RegisterMessage(ClientConnection) -ServerConnection = _reflection.GeneratedProtocolMessageType('ServerConnection', (_message.Message,), dict( - DESCRIPTOR = _SERVERCONNECTION, - __module__ = 'http_pb2' +ServerConnection = _reflection.GeneratedProtocolMessageType('ServerConnection', (_message.Message,), { + 'DESCRIPTOR' : _SERVERCONNECTION, + '__module__' : 'http_pb2' # @@protoc_insertion_point(class_scope:ServerConnection) - )) + }) _sym_db.RegisterMessage(ServerConnection) -TLSExtension = _reflection.GeneratedProtocolMessageType('TLSExtension', (_message.Message,), dict( - DESCRIPTOR = _TLSEXTENSION, - __module__ = 'http_pb2' +TLSExtension = _reflection.GeneratedProtocolMessageType('TLSExtension', (_message.Message,), { + 'DESCRIPTOR' : _TLSEXTENSION, + '__module__' : 'http_pb2' # @@protoc_insertion_point(class_scope:TLSExtension) - )) + }) _sym_db.RegisterMessage(TLSExtension) diff -Nru mitmproxy-5.1.1/mitmproxy/io/protobuf.py mitmproxy-6.0.2/mitmproxy/io/protobuf.py --- mitmproxy-5.1.1/mitmproxy/io/protobuf.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/io/protobuf.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,3 +1,7 @@ +""" +Note: This module is currently unmaintained as we don't use it internally. +We're happy to accept PRs that fix bugs, but we won't provide any continuous maintenance. +""" import typing from mitmproxy import flow @@ -106,8 +110,8 @@ def _load_http_request(o: http_pb2.HTTPRequest) -> HTTPRequest: d: dict = {} - _move_attrs(o, d, ['first_line_format', 'method', 'scheme', 'host', 'port', 'path', 'http_version', 'content', - 'timestamp_start', 'timestamp_end', 'is_replay']) + _move_attrs(o, d, ['host', 'port', 'method', 'scheme', 'authority', 'path', 'http_version', 'content', + 'timestamp_start', 'timestamp_end']) if d['content'] is None: d['content'] = b"" d["headers"] = [] @@ -120,7 +124,7 @@ def _load_http_response(o: http_pb2.HTTPResponse) -> HTTPResponse: d: dict = {} _move_attrs(o, d, ['http_version', 'status_code', 'reason', - 'content', 'timestamp_start', 'timestamp_end', 'is_replay']) + 'content', 'timestamp_start', 'timestamp_end']) if d['content'] is None: d['content'] = b"" d["headers"] = [] diff -Nru mitmproxy-5.1.1/mitmproxy/io/tnetstring.py mitmproxy-6.0.2/mitmproxy/io/tnetstring.py --- mitmproxy-5.1.1/mitmproxy/io/tnetstring.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/io/tnetstring.py 2020-12-15 16:41:27.000000000 +0000 @@ -222,7 +222,7 @@ val, data = pop(data) d[key] = val # type: ignore return d - raise ValueError("unknown type tag: {}".format(data_type)) + raise ValueError(f"unknown type tag: {data_type}") def pop(data: bytes) -> typing.Tuple[TSerializable, bytes]: @@ -242,7 +242,7 @@ except IndexError: # This fires if len(data) < dlen, meaning we don't need # to further validate that data is the right length. - raise ValueError("not a tnetstring: invalid length prefix: {}".format(length)) + raise ValueError(f"not a tnetstring: invalid length prefix: {length}") # Parse the data based on the type tag. return parse(data_type, data), remain diff -Nru mitmproxy-5.1.1/mitmproxy/log.py mitmproxy-6.0.2/mitmproxy/log.py --- mitmproxy-5.1.1/mitmproxy/log.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/log.py 2020-12-15 16:41:27.000000000 +0000 @@ -12,7 +12,7 @@ return False def __repr__(self): - return "LogEntry({}, {})".format(self.msg, self.level) + return f"LogEntry({self.msg}, {self.level})" class Log: diff -Nru mitmproxy-5.1.1/mitmproxy/master.py mitmproxy-6.0.2/mitmproxy/master.py --- mitmproxy-5.1.1/mitmproxy/master.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/master.py 2020-12-15 16:41:27.000000000 +0000 @@ -90,7 +90,7 @@ if not self.should_exit.is_set(): # pragma: no cover self.shutdown() loop = asyncio.get_event_loop() - tasks = asyncio.all_tasks(loop) if sys.version_info >= (3, 7) else asyncio.Task.all_tasks(loop) + tasks = asyncio.all_tasks(loop) for p in tasks: p.cancel() loop.close() @@ -152,10 +152,15 @@ self.waiting_flows.append(f) if isinstance(f, websocket.WebSocketFlow): - hf = [hf for hf in self.waiting_flows if hf.id == f.metadata['websocket_handshake']][0] - f.handshake_flow = hf - self.waiting_flows.remove(hf) - self._change_reverse_host(f.handshake_flow) + hfs = [hf for hf in self.waiting_flows if hf.id == f.metadata['websocket_handshake']] + if hfs: + hf = hfs[0] + f.handshake_flow = hf + self.waiting_flows.remove(hf) + self._change_reverse_host(f.handshake_flow) + else: + # this will fail - but at least it will load the remaining flows + f.handshake_flow = http.HTTPFlow(None, None) f.reply = controller.DummyReply() for e, o in eventsequence.iterate(f): diff -Nru mitmproxy-5.1.1/mitmproxy/net/check.py mitmproxy-6.0.2/mitmproxy/net/check.py --- mitmproxy-5.1.1/mitmproxy/net/check.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/net/check.py 2020-12-15 16:41:27.000000000 +0000 @@ -3,28 +3,37 @@ # Allow underscore in host name # Note: This could be a DNS label, a hostname, a FQDN, or an IP +from typing import AnyStr + _label_valid = re.compile(br"[A-Z\d\-_]{1,63}$", re.IGNORECASE) -def is_valid_host(host: bytes) -> bool: +def is_valid_host(host: AnyStr) -> bool: """ Checks if the passed bytes are a valid DNS hostname or an IPv4/IPv6 address. """ + if isinstance(host, str): + try: + host_bytes = host.encode("idna") + except UnicodeError: + return False + else: + host_bytes = host try: - host.decode("idna") + host_bytes.decode("idna") except ValueError: return False # RFC1035: 255 bytes or less. - if len(host) > 255: + if len(host_bytes) > 255: return False - if host and host[-1:] == b".": - host = host[:-1] + if host_bytes and host_bytes.endswith(b"."): + host_bytes = host_bytes[:-1] # DNS hostname - if all(_label_valid.match(x) for x in host.split(b".")): + if all(_label_valid.match(x) for x in host_bytes.split(b".")): return True # IPv4/IPv6 address try: - ipaddress.ip_address(host.decode('idna')) + ipaddress.ip_address(host_bytes.decode('idna')) return True except ValueError: return False diff -Nru mitmproxy-5.1.1/mitmproxy/net/http/cookies.py mitmproxy-6.0.2/mitmproxy/net/http/cookies.py --- mitmproxy-5.1.1/mitmproxy/net/http/cookies.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/net/http/cookies.py 2020-12-15 16:41:27.000000000 +0000 @@ -201,7 +201,7 @@ if k.lower() not in specials and _has_special(v): v = ESCAPE.sub(r"\\\1", v) v = '"%s"' % v - vals.append("%s=%s" % (k, v)) + vals.append(f"{k}={v}") return sep.join(vals) diff -Nru mitmproxy-5.1.1/mitmproxy/net/http/encoding.py mitmproxy-6.0.2/mitmproxy/net/http/encoding.py --- mitmproxy-5.1.1/mitmproxy/net/http/encoding.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/net/http/encoding.py 2020-12-15 16:41:27.000000000 +0000 @@ -11,8 +11,7 @@ import brotli import zstandard as zstd -from typing import Union, Optional, AnyStr # noqa - +from typing import Union, Optional, AnyStr, overload # noqa # We have a shared single-element cache for encoding and decoding. # This is quite useful in practice, e.g. @@ -24,9 +23,24 @@ _cache = CachedDecode(None, None, None, None) +@overload +def decode(encoded: None, encoding: str, errors: str = 'strict') -> None: + ... + + +@overload +def decode(encoded: str, encoding: str, errors: str = 'strict') -> str: + ... + + +@overload +def decode(encoded: bytes, encoding: str, errors: str = 'strict') -> Union[str, bytes]: + ... + + def decode( - encoded: Optional[bytes], encoding: str, errors: str='strict' -) -> Optional[AnyStr]: + encoded: Union[None, str, bytes], encoding: str, errors: str = 'strict' +) -> Union[None, str, bytes]: """ Decode the given input object @@ -41,10 +55,10 @@ global _cache cached = ( - isinstance(encoded, bytes) and - _cache.encoded == encoded and - _cache.encoding == encoding and - _cache.errors == errors + isinstance(encoded, bytes) and + _cache.encoded == encoded and + _cache.encoding == encoding and + _cache.errors == errors ) if cached: return _cache.decoded @@ -52,7 +66,7 @@ try: decoded = custom_decode[encoding](encoded) except KeyError: - decoded = codecs.decode(encoded, encoding, errors) + decoded = codecs.decode(encoded, encoding, errors) # type: ignore if encoding in ("gzip", "deflate", "br", "zstd"): _cache = CachedDecode(encoded, encoding, errors, decoded) return decoded @@ -67,7 +81,22 @@ )) -def encode(decoded: Optional[str], encoding: str, errors: str='strict') -> Optional[AnyStr]: +@overload +def encode(decoded: None, encoding: str, errors: str = 'strict') -> None: + ... + + +@overload +def encode(decoded: str, encoding: str, errors: str = 'strict') -> Union[str, bytes]: + ... + + +@overload +def encode(decoded: bytes, encoding: str, errors: str = 'strict') -> bytes: + ... + + +def encode(decoded: Union[None, str, bytes], encoding, errors='strict') -> Union[None, str, bytes]: """ Encode the given input object @@ -82,10 +111,10 @@ global _cache cached = ( - isinstance(decoded, bytes) and - _cache.decoded == decoded and - _cache.encoding == encoding and - _cache.errors == errors + isinstance(decoded, bytes) and + _cache.decoded == decoded and + _cache.encoding == encoding and + _cache.errors == errors ) if cached: return _cache.encoded @@ -93,7 +122,7 @@ try: encoded = custom_encode[encoding](decoded) except KeyError: - encoded = codecs.encode(decoded, encoding, errors) + encoded = codecs.encode(decoded, encoding, errors) # type: ignore if encoding in ("gzip", "deflate", "br", "zstd"): _cache = CachedDecode(encoded, encoding, errors, decoded) return encoded @@ -150,7 +179,7 @@ except zstd.ZstdError: # If the zstd stream is streamed without a size header, # try decoding with a 10MiB output buffer - return zstd_ctx.decompress(content, max_output_size=10 * 2**20) + return zstd_ctx.decompress(content, max_output_size=10 * 2 ** 20) def encode_zstd(content: bytes) -> bytes: @@ -187,6 +216,7 @@ "identity": identity, "gzip": decode_gzip, "deflate": decode_deflate, + "deflateRaw": decode_deflate, "br": decode_brotli, "zstd": decode_zstd, } @@ -195,6 +225,7 @@ "identity": identity, "gzip": encode_gzip, "deflate": encode_deflate, + "deflateRaw": encode_deflate, "br": encode_brotli, "zstd": encode_zstd, } diff -Nru mitmproxy-5.1.1/mitmproxy/net/http/headers.py mitmproxy-6.0.2/mitmproxy/net/http/headers.py --- mitmproxy-5.1.1/mitmproxy/net/http/headers.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/net/http/headers.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,9 +1,10 @@ -import re - import collections +from typing import Dict, Optional, Tuple + from mitmproxy.coretypes import multidict from mitmproxy.utils import strutils + # See also: http://lucumr.pocoo.org/2013/7/2/the-updated-guide-to-unicode/ @@ -147,48 +148,8 @@ else: return super().items() - def replace(self, pattern, repl, flags=0, count=0): - """ - Replaces a regular expression pattern with repl in each "name: value" - header line. - - Returns: - The number of replacements made. - """ - if isinstance(pattern, str): - pattern = strutils.escaped_str_to_bytes(pattern) - if isinstance(repl, str): - repl = strutils.escaped_str_to_bytes(repl) - pattern = re.compile(pattern, flags) - replacements = 0 - flag_count = count > 0 - count_reached = False - fields = [] - for name, value in self.fields: - if count_reached: - fields.append((name, value)) - continue - line, n = pattern.subn(repl, name + b": " + value, count=count) - try: - name, value = line.split(b": ", 1) - except ValueError: - # We get a ValueError if the replacement removed the ": " - # There's not much we can do about this, so we just keep the header as-is. - pass - else: - replacements += n - fields.append((name, value)) - if flag_count: - count -= n - if count == 0: - count_reached = True - continue - fields.append((name, value)) - self.fields = tuple(fields) - return replacements - -def parse_content_type(c): +def parse_content_type(c: str) -> Optional[Tuple[str, str, Dict[str, str]]]: """ A simple parser for content-type values. Returns a (type, subtype, parameters) tuple, where type and subtype are strings, and parameters @@ -217,9 +178,9 @@ def assemble_content_type(type, subtype, parameters): if not parameters: - return "{}/{}".format(type, subtype) + return f"{type}/{subtype}" params = "; ".join( - "{}={}".format(k, v) + f"{k}={v}" for k, v in parameters.items() ) return "{}/{}; {}".format( diff -Nru mitmproxy-5.1.1/mitmproxy/net/http/http1/assemble.py mitmproxy-6.0.2/mitmproxy/net/http/http1/assemble.py --- mitmproxy-5.1.1/mitmproxy/net/http/http1/assemble.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/net/http/http1/assemble.py 2020-12-15 16:41:27.000000000 +0000 @@ -5,7 +5,7 @@ if request.data.content is None: raise exceptions.HttpException("Cannot assemble flow with missing content") head = assemble_request_head(request) - body = b"".join(assemble_body(request.data.headers, [request.data.content])) + body = b"".join(assemble_body(request.data.headers, [request.data.content], request.data.trailers)) return head + body @@ -19,7 +19,7 @@ if response.data.content is None: raise exceptions.HttpException("Cannot assemble flow with missing content") head = assemble_response_head(response) - body = b"".join(assemble_body(response.data.headers, [response.data.content])) + body = b"".join(assemble_body(response.data.headers, [response.data.content], response.data.trailers)) return head + body @@ -29,13 +29,18 @@ return b"%s\r\n%s\r\n" % (first_line, headers) -def assemble_body(headers, body_chunks): +def assemble_body(headers, body_chunks, trailers): if "chunked" in headers.get("transfer-encoding", "").lower(): for chunk in body_chunks: if chunk: yield b"%x\r\n%s\r\n" % (len(chunk), chunk) - yield b"0\r\n\r\n" + if trailers: + yield b"0\r\n%s\r\n" % trailers + else: + yield b"0\r\n\r\n" else: + if trailers: + raise exceptions.HttpException("Sending HTTP/1.1 trailer headers requires transfer-encoding: chunked") for chunk in body_chunks: yield chunk @@ -45,31 +50,26 @@ Args: request_data (mitmproxy.net.http.request.RequestData) """ - form = request_data.first_line_format - if form == "relative": + if request_data.method.upper() == b"CONNECT": return b"%s %s %s" % ( request_data.method, - request_data.path, + request_data.authority, request_data.http_version ) - elif form == "authority": - return b"%s %s:%d %s" % ( + elif request_data.authority: + return b"%s %s://%s%s %s" % ( request_data.method, - request_data.host, - request_data.port, + request_data.scheme, + request_data.authority, + request_data.path, request_data.http_version ) - elif form == "absolute": - return b"%s %s://%s:%d%s %s" % ( + else: + return b"%s %s %s" % ( request_data.method, - request_data.scheme, - request_data.host, - request_data.port, request_data.path, request_data.http_version ) - else: - raise RuntimeError("Invalid request form") def _assemble_request_headers(request_data): diff -Nru mitmproxy-5.1.1/mitmproxy/net/http/http1/read.py mitmproxy-6.0.2/mitmproxy/net/http/http1/read.py --- mitmproxy-5.1.1/mitmproxy/net/http/http1/read.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/net/http/http1/read.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,15 +1,13 @@ -import time -import sys import re - +import sys +import time import typing +from mitmproxy import exceptions +from mitmproxy.net.http import headers from mitmproxy.net.http import request from mitmproxy.net.http import response -from mitmproxy.net.http import headers from mitmproxy.net.http import url -from mitmproxy.net import check -from mitmproxy import exceptions def get_header_tokens(headers, key): @@ -51,7 +49,7 @@ if hasattr(rfile, "reset_timestamps"): rfile.reset_timestamps() - form, method, scheme, host, port, path, http_version = _read_request_line(rfile) + host, port, method, scheme, authority, path, http_version = _read_request_line(rfile) headers = _read_headers(rfile) if hasattr(rfile, "first_byte_timestamp"): @@ -59,7 +57,7 @@ timestamp_start = rfile.first_byte_timestamp return request.Request( - form, method, scheme, host, port, path, http_version, headers, None, timestamp_start + host, port, method, scheme, authority, path, http_version, headers, None, None, timestamp_start, None ) @@ -98,7 +96,7 @@ # more accurate timestamp_start timestamp_start = rfile.first_byte_timestamp - return response.Response(http_version, status_code, message, headers, None, timestamp_start) + return response.Response(http_version, status_code, message, headers, None, None, timestamp_start, None) def read_body(rfile, expected_size, limit=None, max_chunk_size=4096): @@ -126,8 +124,7 @@ max_chunk_size = limit if expected_size is None: - for x in _read_chunked(rfile, limit): - yield x + yield from _read_chunked(rfile, limit) elif expected_size >= 0: if limit is not None and expected_size > limit: raise exceptions.HttpException( @@ -153,7 +150,7 @@ bytes_left -= chunk_size not_done = rfile.read(1) if not_done: - raise exceptions.HttpException("HTTP body too large. Limit is {}.".format(limit)) + raise exceptions.HttpException(f"HTTP body too large. Limit is {limit}.") def connection_close(http_version, headers): @@ -170,7 +167,10 @@ elif "keep-alive" in tokens: return False - return http_version != "HTTP/1.1" and http_version != b"HTTP/1.1" + return http_version not in ( + "HTTP/1.1", b"HTTP/1.1", + "HTTP/2.0", b"HTTP/2.0", + ) def expected_http_body_size( @@ -248,45 +248,32 @@ raise exceptions.HttpReadDisconnect("Client disconnected") try: - method, path, http_version = line.split() + method, target, http_version = line.split() - if path == b"*" or path.startswith(b"/"): - form = "relative" - scheme, host, port = None, None, None + if target == b"*" or target.startswith(b"/"): + scheme, authority, path = b"", b"", target + host, port = "", 0 elif method == b"CONNECT": - form = "authority" - host, port = _parse_authority_form(path) - scheme, path = None, None + scheme, authority, path = b"", target, b"" + host, port = url.parse_authority(authority, check=True) + if not port: + raise ValueError else: - form = "absolute" - scheme, host, port, path = url.parse(path) + scheme, rest = target.split(b"://", maxsplit=1) + authority, path_ = rest.split(b"/", maxsplit=1) + path = b"/" + path_ + host, port = url.parse_authority(authority, check=True) + port = port or url.default_port(scheme) + if not port: + raise ValueError + # TODO: we can probably get rid of this check? + url.parse(target) _check_http_version(http_version) except ValueError: - raise exceptions.HttpSyntaxException("Bad HTTP request line: {}".format(line)) - - return form, method, scheme, host, port, path, http_version - - -def _parse_authority_form(hostport): - """ - Returns (host, port) if hostport is a valid authority-form host specification. - http://tools.ietf.org/html/draft-luotonen-web-proxy-tunneling-01 section 3.1 - - Raises: - ValueError, if the input is malformed - """ - try: - host, port = hostport.rsplit(b":", 1) - if host.startswith(b"[") and host.endswith(b"]"): - host = host[1:-1] - port = int(port) - if not check.is_valid_host(host) or not check.is_valid_port(port): - raise ValueError() - except ValueError: - raise exceptions.HttpSyntaxException("Invalid host specification: {}".format(hostport)) + raise exceptions.HttpSyntaxException(f"Bad HTTP request line: {line}") - return host, port + return host, port, method, scheme, authority, path, http_version def _read_response_line(rfile): @@ -306,14 +293,14 @@ _check_http_version(http_version) except ValueError: - raise exceptions.HttpSyntaxException("Bad HTTP response line: {}".format(line)) + raise exceptions.HttpSyntaxException(f"Bad HTTP response line: {line}") return http_version, status_code, message def _check_http_version(http_version): if not re.match(br"^HTTP/\d\.\d$", http_version): - raise exceptions.HttpSyntaxException("Unknown HTTP version: {}".format(http_version)) + raise exceptions.HttpSyntaxException(f"Unknown HTTP version: {http_version}") def _read_headers(rfile): @@ -369,7 +356,7 @@ try: length = int(line, 16) except ValueError: - raise exceptions.HttpSyntaxException("Invalid chunked encoding length: {}".format(line)) + raise exceptions.HttpSyntaxException(f"Invalid chunked encoding length: {line}") total += length if total > limit: raise exceptions.HttpException( diff -Nru mitmproxy-5.1.1/mitmproxy/net/http/http2/framereader.py mitmproxy-6.0.2/mitmproxy/net/http/http2/framereader.py --- mitmproxy-5.1.1/mitmproxy/net/http/http2/framereader.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/net/http/http2/framereader.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,25 +0,0 @@ -import codecs - -import hyperframe.frame -from mitmproxy import exceptions - - -def read_raw_frame(rfile): - header = rfile.safe_read(9) - length = int(codecs.encode(header[:3], 'hex_codec'), 16) - - if length == 4740180: - raise exceptions.HttpException("Length field looks more like HTTP/1.1:\n{}".format(rfile.read(-1))) - - body = rfile.safe_read(length) - return [header, body] - - -def parse_frame(header, body=None): - if body is None: - body = header[9:] - header = header[:9] - - frame, _ = hyperframe.frame.Frame.parse_frame_header(header) - frame.parse_body(memoryview(body)) - return frame diff -Nru mitmproxy-5.1.1/mitmproxy/net/http/http2/__init__.py mitmproxy-6.0.2/mitmproxy/net/http/http2/__init__.py --- mitmproxy-5.1.1/mitmproxy/net/http/http2/__init__.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/net/http/http2/__init__.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,8 +0,0 @@ -from mitmproxy.net.http.http2.framereader import read_raw_frame, parse_frame -from mitmproxy.net.http.http2.utils import parse_headers - -__all__ = [ - "read_raw_frame", - "parse_frame", - "parse_headers", -] diff -Nru mitmproxy-5.1.1/mitmproxy/net/http/http2/utils.py mitmproxy-6.0.2/mitmproxy/net/http/http2/utils.py --- mitmproxy-5.1.1/mitmproxy/net/http/http2/utils.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/net/http/http2/utils.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,37 +0,0 @@ -from mitmproxy.net.http import url - - -def parse_headers(headers): - authority = headers.get(':authority', '').encode() - method = headers.get(':method', 'GET').encode() - scheme = headers.get(':scheme', 'https').encode() - path = headers.get(':path', '/').encode() - - headers.pop(":method", None) - headers.pop(":scheme", None) - headers.pop(":path", None) - - host = None - port = None - - if method == b'CONNECT': - raise NotImplementedError("CONNECT over HTTP/2 is not implemented.") - - if path == b'*' or path.startswith(b"/"): - first_line_format = "relative" - else: - first_line_format = "absolute" - scheme, host, port, _ = url.parse(path) - - if authority: - host, _, port = authority.partition(b':') - - if not host: - host = b'localhost' - - if not port: - port = 443 if scheme == b'https' else 80 - - port = int(port) - - return first_line_format, method, scheme, host, port, path diff -Nru mitmproxy-5.1.1/mitmproxy/net/http/http2.py mitmproxy-6.0.2/mitmproxy/net/http/http2.py --- mitmproxy-5.1.1/mitmproxy/net/http/http2.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/net/http/http2.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,28 @@ +import codecs + +from hyperframe.frame import Frame + +from mitmproxy import exceptions + + +def read_frame(rfile, parse=True): + """ + Reads a full HTTP/2 frame from a file-like object. + + Returns a parsed frame and the consumed bytes. + """ + header = rfile.safe_read(9) + length = int(codecs.encode(header[:3], 'hex_codec'), 16) + + if length == 4740180: + raise exceptions.HttpException("Length field looks more like HTTP/1.1:\n{}".format(rfile.read(-1))) + + body = rfile.safe_read(length) + + if parse: + frame, _ = Frame.parse_frame_header(header) + frame.parse_body(memoryview(body)) + else: + frame = None + + return frame, b''.join([header, body]) diff -Nru mitmproxy-5.1.1/mitmproxy/net/http/message.py mitmproxy-6.0.2/mitmproxy/net/http/message.py --- mitmproxy-5.1.1/mitmproxy/net/http/message.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/net/http/message.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,48 +1,54 @@ import re -from typing import Optional # noqa +from dataclasses import dataclass, fields +from typing import Callable, Optional, Union, cast -from mitmproxy.utils import strutils -from mitmproxy.net.http import encoding from mitmproxy.coretypes import serializable -from mitmproxy.net.http import headers as mheaders +from mitmproxy.net.http import encoding +from mitmproxy.net.http.headers import Headers, assemble_content_type, parse_content_type +from mitmproxy.utils import strutils, typecheck +@dataclass class MessageData(serializable.Serializable): - headers: mheaders.Headers - content: bytes http_version: bytes + headers: Headers + content: Optional[bytes] + trailers: Optional[Headers] timestamp_start: float - timestamp_end: float + timestamp_end: Optional[float] - def __eq__(self, other): - if isinstance(other, MessageData): - return self.__dict__ == other.__dict__ - return False + # noinspection PyUnreachableCode + if __debug__: + def __post_init__(self): + for field in fields(self): + val = getattr(self, field.name) + typecheck.check_option_type(field.name, val, field.type) def set_state(self, state): for k, v in state.items(): - if k == "headers": - v = mheaders.Headers.from_state(v) + if k in ("headers", "trailers") and v is not None: + v = Headers.from_state(v) setattr(self, k, v) def get_state(self): state = vars(self).copy() state["headers"] = state["headers"].get_state() + if state["trailers"] is not None: + state["trailers"] = state["trailers"].get_state() return state @classmethod def from_state(cls, state): - state["headers"] = mheaders.Headers.from_state(state["headers"]) + state["headers"] = Headers.from_state(state["headers"]) + if state["trailers"] is not None: + state["trailers"] = Headers.from_state(state["trailers"]) return cls(**state) class Message(serializable.Serializable): - data: MessageData - - def __eq__(self, other): - if isinstance(other, Message): - return self.data == other.data - return False + @classmethod + def from_state(cls, state): + return cls(**state) def get_state(self): return self.data.get_state() @@ -50,27 +56,56 @@ def set_state(self, state): self.data.set_state(state) - @classmethod - def from_state(cls, state): - state["headers"] = mheaders.Headers.from_state(state["headers"]) - return cls(**state) + data: MessageData + stream: Union[Callable, bool] = False @property - def headers(self): + def http_version(self) -> str: """ - Message headers object + Version string, e.g. "HTTP/1.1" + """ + return self.data.http_version.decode("utf-8", "surrogateescape") + + @http_version.setter + def http_version(self, http_version: Union[str, bytes]) -> None: + self.data.http_version = strutils.always_bytes(http_version, "utf-8", "surrogateescape") - Returns: - mitmproxy.net.http.Headers + @property + def is_http10(self) -> bool: + return self.data.http_version == b"HTTP/1.0" + + @property + def is_http11(self) -> bool: + return self.data.http_version == b"HTTP/1.1" + + @property + def is_http2(self) -> bool: + return self.data.http_version == b"HTTP/2.0" + + @property + def headers(self) -> Headers: + """ + The HTTP headers. """ return self.data.headers @headers.setter - def headers(self, h): + def headers(self, h: Headers) -> None: self.data.headers = h @property - def raw_content(self) -> bytes: + def trailers(self) -> Optional[Headers]: + """ + The HTTP trailers. + """ + return self.data.trailers + + @trailers.setter + def trailers(self, h: Optional[Headers]) -> None: + self.data.trailers = h + + @property + def raw_content(self) -> Optional[bytes]: """ The raw (potentially compressed) HTTP message body as bytes. @@ -79,10 +114,10 @@ return self.data.content @raw_content.setter - def raw_content(self, content): + def raw_content(self, content: Optional[bytes]) -> None: self.data.content = content - def get_content(self, strict: bool=True) -> Optional[bytes]: + def get_content(self, strict: bool = True) -> Optional[bytes]: """ The uncompressed HTTP message body as bytes. @@ -99,7 +134,7 @@ content = encoding.decode(self.raw_content, ce) # A client may illegally specify a byte -> str encoding here (e.g. utf8) if isinstance(content, str): - raise ValueError("Invalid Content-Encoding: {}".format(ce)) + raise ValueError(f"Invalid Content-Encoding: {ce}") return content except ValueError: if strict: @@ -108,15 +143,14 @@ else: return self.raw_content - def set_content(self, value): + def set_content(self, value: Optional[bytes]) -> None: if value is None: self.raw_content = None return if not isinstance(value, bytes): raise TypeError( - "Message content must be bytes, not {}. " + f"Message content must be bytes, not {type(value).__name__}. " "Please use .text if you want to assign a str." - .format(type(value).__name__) ) ce = self.headers.get("content-encoding") try: @@ -131,45 +165,34 @@ content = property(get_content, set_content) @property - def http_version(self): - """ - Version string, e.g. "HTTP/1.1" - """ - return self.data.http_version.decode("utf-8", "surrogateescape") - - @http_version.setter - def http_version(self, http_version): - self.data.http_version = strutils.always_bytes(http_version, "utf-8", "surrogateescape") - - @property - def timestamp_start(self): + def timestamp_start(self) -> float: """ First byte timestamp """ return self.data.timestamp_start @timestamp_start.setter - def timestamp_start(self, timestamp_start): + def timestamp_start(self, timestamp_start: float) -> None: self.data.timestamp_start = timestamp_start @property - def timestamp_end(self): + def timestamp_end(self) -> Optional[float]: """ Last byte timestamp """ return self.data.timestamp_end @timestamp_end.setter - def timestamp_end(self, timestamp_end): + def timestamp_end(self, timestamp_end: Optional[float]): self.data.timestamp_end = timestamp_end def _get_content_type_charset(self) -> Optional[str]: - ct = mheaders.parse_content_type(self.headers.get("content-type", "")) + ct = parse_content_type(self.headers.get("content-type", "")) if ct: return ct[2].get("charset") return None - def _guess_encoding(self, content=b"") -> str: + def _guess_encoding(self, content: bytes = b"") -> str: enc = self._get_content_type_charset() if not enc: if "json" in self.headers.get("content-type", ""): @@ -179,6 +202,12 @@ if meta_charset: enc = meta_charset.group(1).decode("ascii", "ignore") if not enc: + if "text/css" in self.headers.get("content-type", ""): + # @charset rule must be the very first thing. + css_charset = re.match(rb"""@charset "([^"]+)";""", content) + if css_charset: + enc = css_charset.group(1).decode("ascii", "ignore") + if not enc: enc = "latin-1" # Use GB 18030 as the superset of GB2312 and GBK to fix common encoding problems on Chinese websites. if enc.lower() in ("gb2312", "gbk"): @@ -186,7 +215,7 @@ return enc - def get_text(self, strict: bool=True) -> Optional[str]: + def get_text(self, strict: bool = True) -> Optional[str]: """ The uncompressed and decoded HTTP message body as text. @@ -200,13 +229,13 @@ return None enc = self._guess_encoding(content) try: - return encoding.decode(content, enc) + return cast(str, encoding.decode(content, enc)) except ValueError: if strict: raise return content.decode("utf8", "surrogateescape") - def set_text(self, text): + def set_text(self, text: Optional[str]) -> None: if text is None: self.content = None return @@ -216,15 +245,15 @@ self.content = encoding.encode(text, enc) except ValueError: # Fall back to UTF-8 and update the content-type header. - ct = mheaders.parse_content_type(self.headers.get("content-type", "")) or ("text", "plain", {}) + ct = parse_content_type(self.headers.get("content-type", "")) or ("text", "plain", {}) ct[2]["charset"] = "utf-8" - self.headers["content-type"] = mheaders.assemble_content_type(*ct) + self.headers["content-type"] = assemble_content_type(*ct) enc = "utf8" self.content = text.encode(enc, "surrogateescape") text = property(get_text, set_text) - def decode(self, strict=True): + def decode(self, strict: bool = True) -> None: """ Decodes body based on the current Content-Encoding header, then removes the header. If there is no Content-Encoding header, no @@ -237,7 +266,7 @@ self.headers.pop("content-encoding", None) self.content = decoded - def encode(self, e): + def encode(self, e: str) -> None: """ Encodes body with the encoding e, where e is "gzip", "deflate", "identity", "br", or "zstd". Any existing content-encodings are overwritten, @@ -250,24 +279,3 @@ self.content = self.raw_content if "content-encoding" not in self.headers: raise ValueError("Invalid content encoding {}".format(repr(e))) - - def replace(self, pattern, repl, flags=0, count=0): - """ - Replaces a regular expression pattern with repl in both the headers - and the body of the message. Encoded body will be decoded - before replacement, and re-encoded afterwards. - - Returns: - The number of replacements made. - """ - if isinstance(pattern, str): - pattern = strutils.escaped_str_to_bytes(pattern) - if isinstance(repl, str): - repl = strutils.escaped_str_to_bytes(repl) - replacements = 0 - if self.content: - self.content, replacements = re.subn( - pattern, repl, self.content, flags=flags, count=count - ) - replacements += self.headers.replace(pattern, repl, flags=flags, count=count) - return replacements diff -Nru mitmproxy-5.1.1/mitmproxy/net/http/request.py mitmproxy-6.0.2/mitmproxy/net/http/request.py --- mitmproxy-5.1.1/mitmproxy/net/http/request.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/net/http/request.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,84 +1,90 @@ -import re -import urllib import time -from typing import Optional, AnyStr, Dict, Iterable, Tuple, Union +import urllib.parse +from dataclasses import dataclass +from typing import Dict, Iterable, Optional, Tuple, Union +import mitmproxy.net.http.url from mitmproxy.coretypes import multidict -from mitmproxy.utils import strutils -from mitmproxy.net.http import multipart -from mitmproxy.net.http import cookies -from mitmproxy.net.http import headers as nheaders +from mitmproxy.net.http import cookies, multipart from mitmproxy.net.http import message -import mitmproxy.net.http.url - -# This regex extracts & splits the host header into host and port. -# Handles the edge case of IPv6 addresses containing colons. -# https://bugzilla.mozilla.org/show_bug.cgi?id=45891 -host_header_re = re.compile(r"^(?P[^:]+|\[.+\])(?::(?P\d+))?$") +from mitmproxy.net.http.headers import Headers +from mitmproxy.utils.strutils import always_bytes, always_str +@dataclass class RequestData(message.MessageData): + host: str + port: int + method: bytes + scheme: bytes + authority: bytes + path: bytes + + +class Request(message.Message): + """ + An HTTP request. + """ + data: RequestData + def __init__( - self, - first_line_format, - method, - scheme, - host, - port, - path, - http_version, - headers=(), - content=None, - timestamp_start=None, - timestamp_end=None + self, + host: str, + port: int, + method: bytes, + scheme: bytes, + authority: bytes, + path: bytes, + http_version: bytes, + headers: Union[Headers, Tuple[Tuple[bytes, bytes], ...]], + content: Optional[bytes], + trailers: Union[None, Headers, Tuple[Tuple[bytes, bytes], ...]], + timestamp_start: float, + timestamp_end: Optional[float], ): + # auto-convert invalid types to retain compatibility with older code. + if isinstance(host, bytes): + host = host.decode("idna", "strict") if isinstance(method, str): method = method.encode("ascii", "strict") if isinstance(scheme, str): scheme = scheme.encode("ascii", "strict") - if isinstance(host, str): - host = host.encode("idna", "strict") + if isinstance(authority, str): + authority = authority.encode("ascii", "strict") if isinstance(path, str): path = path.encode("ascii", "strict") if isinstance(http_version, str): http_version = http_version.encode("ascii", "strict") - if not isinstance(headers, nheaders.Headers): - headers = nheaders.Headers(headers) - if isinstance(content, str): - raise ValueError("Content must be bytes, not {}".format(type(content).__name__)) - - self.first_line_format = first_line_format - self.method = method - self.scheme = scheme - self.host = host - self.port = port - self.path = path - self.http_version = http_version - self.headers = headers - self.content = content - self.timestamp_start = timestamp_start - self.timestamp_end = timestamp_end - - -class Request(message.Message): - """ - An HTTP request. - """ - data: RequestData - def __init__(self, *args, **kwargs): - super().__init__() - self.data = RequestData(*args, **kwargs) + if isinstance(content, str): + raise ValueError(f"Content must be bytes, not {type(content).__name__}") + if not isinstance(headers, Headers): + headers = Headers(headers) + if trailers is not None and not isinstance(trailers, Headers): + trailers = Headers(trailers) + + self.data = RequestData( + host=host, + port=port, + method=method, + scheme=scheme, + authority=authority, + path=path, + http_version=http_version, + headers=headers, + content=content, + trailers=trailers, + timestamp_start=timestamp_start, + timestamp_end=timestamp_end, + ) - def __repr__(self): + def __repr__(self) -> str: if self.host and self.port: - hostport = "{}:{}".format(self.host, self.port) + hostport = f"{self.host}:{self.port}" else: hostport = "" path = self.path or "" - return "Request({} {}{})".format( - self.method, hostport, path - ) + return f"Request({self.method} {hostport}{path})" @classmethod def make( @@ -86,138 +92,134 @@ method: str, url: str, content: Union[bytes, str] = "", - headers: Union[Dict[str, AnyStr], Iterable[Tuple[bytes, bytes]]] = () - ): + headers: Union[Headers, Dict[Union[str, bytes], Union[str, bytes]], Iterable[Tuple[bytes, bytes]]] = () + ) -> "Request": """ Simplified API for creating request objects. """ - req = cls( - "absolute", - method, - "", - "", - "", - "", - "HTTP/1.1", - (), - b"" - ) - - req.url = url - req.timestamp_start = time.time() - # Headers can be list or dict, we differentiate here. - if isinstance(headers, dict): - req.headers = nheaders.Headers(**headers) + if isinstance(headers, Headers): + pass + elif isinstance(headers, dict): + headers = Headers( + (always_bytes(k, "utf-8", "surrogateescape"), + always_bytes(v, "utf-8", "surrogateescape")) + for k, v in headers.items() + ) elif isinstance(headers, Iterable): - req.headers = nheaders.Headers(headers) + headers = Headers(headers) else: raise TypeError("Expected headers to be an iterable or dict, but is {}.".format( type(headers).__name__ )) + req = cls( + "", + 0, + method.encode("utf-8", "surrogateescape"), + b"", + b"", + b"", + b"HTTP/1.1", + headers, + b"", + None, + time.time(), + time.time(), + ) + + req.url = url # Assign this manually to update the content-length header. if isinstance(content, bytes): req.content = content elif isinstance(content, str): req.text = content else: - raise TypeError("Expected content to be str or bytes, but is {}.".format( - type(content).__name__ - )) + raise TypeError(f"Expected content to be str or bytes, but is {type(content).__name__}.") return req - def replace(self, pattern, repl, flags=0, count=0): - """ - Replaces a regular expression pattern with repl in the headers, the - request path and the body of the request. Encoded content will be - decoded before replacement, and re-encoded afterwards. - - Returns: - The number of replacements made. - """ - if isinstance(pattern, str): - pattern = strutils.escaped_str_to_bytes(pattern) - if isinstance(repl, str): - repl = strutils.escaped_str_to_bytes(repl) - - c = super().replace(pattern, repl, flags, count) - self.path, pc = re.subn( - pattern, repl, self.data.path, flags=flags, count=count - ) - c += pc - return c - @property - def first_line_format(self): + def first_line_format(self) -> str: """ HTTP request form as defined in `RFC7230 `_. origin-form and asterisk-form are subsumed as "relative". """ - return self.data.first_line_format - - @first_line_format.setter - def first_line_format(self, first_line_format): - self.data.first_line_format = first_line_format + if self.method == "CONNECT": + return "authority" + elif self.authority: + return "absolute" + else: + return "relative" @property - def method(self): + def method(self) -> str: """ HTTP request method, e.g. "GET". """ return self.data.method.decode("utf-8", "surrogateescape").upper() @method.setter - def method(self, method): - self.data.method = strutils.always_bytes(method, "utf-8", "surrogateescape") + def method(self, val: Union[str, bytes]) -> None: + self.data.method = always_bytes(val, "utf-8", "surrogateescape") @property - def scheme(self): + def scheme(self) -> str: """ HTTP request scheme, which should be "http" or "https". """ - if self.data.scheme is None: - return None return self.data.scheme.decode("utf-8", "surrogateescape") @scheme.setter - def scheme(self, scheme): - self.data.scheme = strutils.always_bytes(scheme, "utf-8", "surrogateescape") + def scheme(self, val: Union[str, bytes]) -> None: + self.data.scheme = always_bytes(val, "utf-8", "surrogateescape") @property - def host(self): + def authority(self) -> str: """ - Target host. This may be parsed from the raw request - (e.g. from a ``GET http://example.com/ HTTP/1.1`` request line) - or inferred from the proxy mode (e.g. an IP in transparent mode). + HTTP request authority. - Setting the host attribute also updates the host header, if present. + For HTTP/1, this is the authority portion of the request target + (in either absolute-form or authority-form) + + For HTTP/2, this is the :authority pseudo header. """ - if not self.data.host: - return self.data.host try: - return self.data.host.decode("idna") + return self.data.authority.decode("idna") except UnicodeError: - return self.data.host.decode("utf8", "surrogateescape") + return self.data.authority.decode("utf8", "surrogateescape") - @host.setter - def host(self, host): - if isinstance(host, str): + @authority.setter + def authority(self, val: Union[str, bytes]) -> None: + if isinstance(val, str): try: - # There's no non-strict mode for IDNA encoding. - # We don't want this operation to fail though, so we try - # utf8 as a last resort. - host = host.encode("idna", "strict") + val = val.encode("idna", "strict") except UnicodeError: - host = host.encode("utf8", "surrogateescape") + val = val.encode("utf8", "surrogateescape") # type: ignore + self.data.authority = val - self.data.host = host + @property + def host(self) -> str: + """ + Target host. This may be parsed from the raw request + (e.g. from a ``GET http://example.com/ HTTP/1.1`` request line) + or inferred from the proxy mode (e.g. an IP in transparent mode). + + Setting the host attribute also updates the host header and authority information, if present. + """ + return self.data.host + + @host.setter + def host(self, val: Union[str, bytes]) -> None: + self.data.host = always_str(val, "idna", "strict") # Update host header - if self.host_header is not None: - self.host_header = host + if "Host" in self.data.headers: + self.data.headers["Host"] = val + # Update authority + if self.data.authority: + self.authority = mitmproxy.net.http.url.hostport(self.scheme, self.host, self.port) @property def host_header(self) -> Optional[str]: @@ -225,111 +227,92 @@ The request's host/authority header. This property maps to either ``request.headers["Host"]`` or - ``request.headers[":authority"]``, depending on whether it's HTTP/1.x or HTTP/2.0. + ``request.authority``, depending on whether it's HTTP/1.x or HTTP/2.0. """ - if ":authority" in self.headers: - return self.headers[":authority"] - if "Host" in self.headers: - return self.headers["Host"] - return None + if self.is_http2: + return self.authority or self.data.headers.get("Host", None) + else: + return self.data.headers.get("Host", None) @host_header.setter - def host_header(self, val: Optional[str]) -> None: + def host_header(self, val: Union[None, str, bytes]) -> None: if val is None: + if self.is_http2: + self.data.authority = b"" self.headers.pop("Host", None) - self.headers.pop(":authority", None) - elif self.host_header is not None: - # Update any existing headers. - if ":authority" in self.headers: - self.headers[":authority"] = val - if "Host" in self.headers: - self.headers["Host"] = val else: - # Only add the correct new header. - if self.http_version.upper().startswith("HTTP/2"): - self.headers[":authority"] = val - else: + if self.is_http2: + self.authority = val # type: ignore + if not self.is_http2 or "Host" in self.headers: + # For h2, we only overwrite, but not create, as :authority is the h2 host header. self.headers["Host"] = val - @host_header.deleter - def host_header(self): - self.host_header = None - @property - def port(self): + def port(self) -> int: """ Target port """ return self.data.port @port.setter - def port(self, port): + def port(self, port: int) -> None: self.data.port = port @property - def path(self): + def path(self) -> str: """ HTTP request path, e.g. "/index.html". - Guaranteed to start with a slash, except for OPTIONS requests, which may just be "*". + Usually starts with a slash, except for OPTIONS requests, which may just be "*". """ - if self.data.path is None: - return None - else: - return self.data.path.decode("utf-8", "surrogateescape") + return self.data.path.decode("utf-8", "surrogateescape") @path.setter - def path(self, path): - self.data.path = strutils.always_bytes(path, "utf-8", "surrogateescape") + def path(self, val: Union[str, bytes]) -> None: + self.data.path = always_bytes(val, "utf-8", "surrogateescape") @property - def url(self): + def url(self) -> str: """ - The URL string, constructed from the request's URL components + The URL string, constructed from the request's URL components. """ if self.first_line_format == "authority": - return "%s:%d" % (self.host, self.port) + return f"{self.host}:{self.port}" return mitmproxy.net.http.url.unparse(self.scheme, self.host, self.port, self.path) @url.setter - def url(self, url): - self.scheme, self.host, self.port, self.path = mitmproxy.net.http.url.parse(url) - - def _parse_host_header(self): - """Extract the host and port from Host header""" - host = self.host_header - if not host: - return None, None - port = None - m = host_header_re.match(host) - if m: - host = m.group("host").strip("[]") - if m.group("port"): - port = int(m.group("port")) - return host, port + def url(self, val: Union[str, bytes]) -> None: + val = always_str(val, "utf-8", "surrogateescape") + self.scheme, self.host, self.port, self.path = mitmproxy.net.http.url.parse(val) @property - def pretty_host(self): + def pretty_host(self) -> str: """ - Similar to :py:attr:`host`, but using the Host headers as an additional preferred data source. + Similar to :py:attr:`host`, but using the host/:authority header as an additional (preferred) data source. This is useful in transparent mode where :py:attr:`host` is only an IP address, but may not reflect the actual destination as the Host header could be spoofed. """ - host, port = self._parse_host_header() - if not host: + authority = self.host_header + if authority: + return mitmproxy.net.http.url.parse_authority(authority, check=False)[0] + else: return self.host - if not port: - port = 443 if self.scheme == 'https' else 80 - # Prefer the original address if host header has an unexpected form - return host if port == self.port else self.host @property - def pretty_url(self): + def pretty_url(self) -> str: """ Like :py:attr:`url`, but using :py:attr:`pretty_host` instead of :py:attr:`host`. """ if self.first_line_format == "authority": - return "%s:%d" % (self.pretty_host, self.port) - return mitmproxy.net.http.url.unparse(self.scheme, self.pretty_host, self.port, self.path) + return self.authority + + host_header = self.host_header + if not host_header: + return self.url + + pretty_host, pretty_port = mitmproxy.net.http.url.parse_authority(host_header, check=False) + pretty_port = pretty_port or mitmproxy.net.http.url.default_port(self.scheme) or 443 + + return mitmproxy.net.http.url.unparse(self.scheme, pretty_host, pretty_port, self.path) def _get_query(self): query = urllib.parse.urlparse(self.url).query @@ -396,7 +379,7 @@ _, _, _, params, query, fragment = urllib.parse.urlparse(self.url) self.path = urllib.parse.urlunparse(["", "", path, params, query, fragment]) - def anticache(self): + def anticache(self) -> None: """ Modifies this request to remove headers that might produce a cached response. That is, we remove ETags and If-Modified-Since headers. @@ -408,14 +391,14 @@ for i in delheaders: self.headers.pop(i, None) - def anticomp(self): + def anticomp(self) -> None: """ Modifies this request to remove headers that will compress the resource's data. """ self.headers["accept-encoding"] = "identity" - def constrain_encoding(self): + def constrain_encoding(self) -> None: """ Limits the permissible Accept-Encoding values, based on what we can decode appropriately. diff -Nru mitmproxy-5.1.1/mitmproxy/net/http/response.py mitmproxy-6.0.2/mitmproxy/net/http/response.py --- mitmproxy-5.1.1/mitmproxy/net/http/response.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/net/http/response.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,46 +1,25 @@ import time -from email.utils import parsedate_tz, formatdate, mktime_tz -from mitmproxy.utils import human -from mitmproxy.coretypes import multidict -from mitmproxy.net.http import cookies -from mitmproxy.net.http import headers as nheaders -from mitmproxy.net.http import message -from mitmproxy.net.http import status_codes -from mitmproxy.utils import strutils -from typing import AnyStr -from typing import Dict +from dataclasses import dataclass +from email.utils import formatdate, mktime_tz, parsedate_tz +from typing import Mapping from typing import Iterable +from typing import Optional from typing import Tuple from typing import Union +from mitmproxy.coretypes import multidict +from mitmproxy.net.http import cookies, message +from mitmproxy.net.http import status_codes +from mitmproxy.net.http.headers import Headers +from mitmproxy.utils import human +from mitmproxy.utils import strutils +from mitmproxy.utils.strutils import always_bytes -class ResponseData(message.MessageData): - def __init__( - self, - http_version, - status_code, - reason=None, - headers=(), - content=None, - timestamp_start=None, - timestamp_end=None - ): - if isinstance(http_version, str): - http_version = http_version.encode("ascii", "strict") - if isinstance(reason, str): - reason = reason.encode("ascii", "strict") - if not isinstance(headers, nheaders.Headers): - headers = nheaders.Headers(headers) - if isinstance(content, str): - raise ValueError("Content must be bytes, not {}".format(type(content).__name__)) - self.http_version = http_version - self.status_code = status_code - self.reason = reason - self.headers = headers - self.content = content - self.timestamp_start = timestamp_start - self.timestamp_end = timestamp_end +@dataclass +class ResponseData(message.MessageData): + status_code: int + reason: bytes class Response(message.Message): @@ -49,87 +28,119 @@ """ data: ResponseData - def __init__(self, *args, **kwargs): - super().__init__() - self.data = ResponseData(*args, **kwargs) + def __init__( + self, + http_version: bytes, + status_code: int, + reason: bytes, + headers: Union[Headers, Tuple[Tuple[bytes, bytes], ...]], + content: Optional[bytes], + trailers: Union[None, Headers, Tuple[Tuple[bytes, bytes], ...]], + timestamp_start: float, + timestamp_end: Optional[float], + ): + # auto-convert invalid types to retain compatibility with older code. + if isinstance(http_version, str): + http_version = http_version.encode("ascii", "strict") + if isinstance(reason, str): + reason = reason.encode("ascii", "strict") + + if isinstance(content, str): + raise ValueError("Content must be bytes, not {}".format(type(content).__name__)) + if not isinstance(headers, Headers): + headers = Headers(headers) + if trailers is not None and not isinstance(trailers, Headers): + trailers = Headers(trailers) + + self.data = ResponseData( + http_version=http_version, + status_code=status_code, + reason=reason, + headers=headers, + content=content, + trailers=trailers, + timestamp_start=timestamp_start, + timestamp_end=timestamp_end, + ) - def __repr__(self): + def __repr__(self) -> str: if self.raw_content: - details = "{}, {}".format( - self.headers.get("content-type", "unknown content type"), - human.pretty_size(len(self.raw_content)) - ) + ct = self.headers.get("content-type", "unknown content type") + size = human.pretty_size(len(self.raw_content)) + details = f"{ct}, {size}" else: details = "no content" - return "Response({status_code} {reason}, {details})".format( - status_code=self.status_code, - reason=self.reason, - details=details - ) + return f"Response({self.status_code}, {details})" @classmethod def make( cls, - status_code: int=200, - content: Union[bytes, str]=b"", - headers: Union[Dict[str, AnyStr], Iterable[Tuple[bytes, bytes]]]=() - ): + status_code: int = 200, + content: Union[bytes, str] = b"", + headers: Union[Headers, Mapping[str, Union[str, bytes]], Iterable[Tuple[bytes, bytes]]] = () + ) -> "Response": """ Simplified API for creating response objects. """ - resp = cls( - b"HTTP/1.1", - status_code, - status_codes.RESPONSES.get(status_code, "").encode(), - (), - None - ) - - # Headers can be list or dict, we differentiate here. - if isinstance(headers, dict): - resp.headers = nheaders.Headers(**headers) + if isinstance(headers, Headers): + headers = headers + elif isinstance(headers, dict): + headers = Headers( + (always_bytes(k, "utf-8", "surrogateescape"), + always_bytes(v, "utf-8", "surrogateescape")) + for k, v in headers.items() + ) elif isinstance(headers, Iterable): - resp.headers = nheaders.Headers(headers) + headers = Headers(headers) else: raise TypeError("Expected headers to be an iterable or dict, but is {}.".format( type(headers).__name__ )) + resp = cls( + b"HTTP/1.1", + status_code, + status_codes.RESPONSES.get(status_code, "").encode(), + headers, + None, + None, + time.time(), + time.time(), + ) + # Assign this manually to update the content-length header. if isinstance(content, bytes): resp.content = content elif isinstance(content, str): resp.text = content else: - raise TypeError("Expected content to be str or bytes, but is {}.".format( - type(content).__name__ - )) + raise TypeError(f"Expected content to be str or bytes, but is {type(content).__name__}.") return resp @property - def status_code(self): + def status_code(self) -> int: """ HTTP Status Code, e.g. ``200``. """ return self.data.status_code @status_code.setter - def status_code(self, status_code): + def status_code(self, status_code: int) -> None: self.data.status_code = status_code @property - def reason(self): + def reason(self) -> str: """ HTTP Reason Phrase, e.g. "Not Found". - This is always :py:obj:`None` for HTTP2 requests, because HTTP2 responses do not contain a reason phrase. + HTTP/2 responses do not contain a reason phrase, an empty string will be returned instead. """ # Encoding: http://stackoverflow.com/a/16674906/934719 - return self.data.reason.decode("ISO-8859-1", "surrogateescape") + return self.data.reason.decode("ISO-8859-1") @reason.setter - def reason(self, reason): - self.data.reason = strutils.always_bytes(reason, "ISO-8859-1", "surrogateescape") + def reason(self, reason: Union[str, bytes]) -> None: + self.data.reason = strutils.always_bytes(reason, "ISO-8859-1") def _get_cookies(self): h = self.headers.get_all("set-cookie") diff -Nru mitmproxy-5.1.1/mitmproxy/net/http/status_codes.py mitmproxy-6.0.2/mitmproxy/net/http/status_codes.py --- mitmproxy-5.1.1/mitmproxy/net/http/status_codes.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/net/http/status_codes.py 2020-12-15 16:41:27.000000000 +0000 @@ -36,6 +36,7 @@ REQUESTED_RANGE_NOT_SATISFIABLE = 416 EXPECTATION_FAILED = 417 IM_A_TEAPOT = 418 +CLIENT_CLOSED_REQUEST = 499 INTERNAL_SERVER_ERROR = 500 NOT_IMPLEMENTED = 501 @@ -91,6 +92,7 @@ REQUESTED_RANGE_NOT_SATISFIABLE: "Requested Range not satisfiable", EXPECTATION_FAILED: "Expectation Failed", IM_A_TEAPOT: "I'm a teapot", + CLIENT_CLOSED_REQUEST: "Client Closed Request", # 500 INTERNAL_SERVER_ERROR: "Internal Server Error", diff -Nru mitmproxy-5.1.1/mitmproxy/net/http/url.py mitmproxy-6.0.2/mitmproxy/net/http/url.py --- mitmproxy-5.1.1/mitmproxy/net/http/url.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/net/http/url.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,8 +1,17 @@ +import re import urllib.parse +from typing import AnyStr, Optional from typing import Sequence from typing import Tuple from mitmproxy.net import check +# This regex extracts & splits the host header into host and port. +# Handles the edge case of IPv6 addresses containing colons. +# https://bugzilla.mozilla.org/show_bug.cgi?id=45891 +from mitmproxy.net.check import is_valid_host, is_valid_port +from mitmproxy.utils.strutils import always_str + +_authority_re = re.compile(r"^(?P[^:]+|\[.+\])(?::(?P\d+))?$") def parse(url): @@ -21,6 +30,8 @@ Raises: ValueError, if the URL is not properly formatted. """ + # FIXME: We shouldn't rely on urllib here. + # Size of Ascii character after encoding is 1 byte which is same as its size # But non-Ascii character's size after encoding will be more than its size def ascii_check(l): @@ -45,7 +56,7 @@ if isinstance(parsed, urllib.parse.ParseResult): parsed = parsed.encode("ascii") - port = parsed.port # Returns None if port number invalid in Py3.5. Will throw ValueError in Py3.6 + port = parsed.port if not port: port = 443 if parsed.scheme == b"https" else 80 @@ -61,7 +72,7 @@ return parsed.scheme, host, port, full_path -def unparse(scheme, host, port, path=""): +def unparse(scheme: str, host: str, port: int, path: str = "") -> str: """ Returns a URL string, constructed from the specified components. @@ -70,10 +81,11 @@ """ if path == "*": path = "" - return "%s://%s%s" % (scheme, hostport(scheme, host, port), path) + authority = hostport(scheme, host, port) + return f"{scheme}://{authority}{path}" -def encode(s: Sequence[Tuple[str, str]], similar_to: str=None) -> str: +def encode(s: Sequence[Tuple[str, str]], similar_to: str = None) -> str: """ Takes a list of (key, value) tuples and returns a urlencoded string. If similar_to is passed, the output is formatted similar to the provided urlencoded string. @@ -100,7 +112,7 @@ return urllib.parse.parse_qsl(s, keep_blank_values=True, errors='surrogateescape') -def quote(b: str, safe: str="/") -> str: +def quote(b: str, safe: str = "/") -> str: """ Returns: An ascii-encodable str. @@ -118,14 +130,59 @@ return urllib.parse.unquote(s, errors="surrogateescape") -def hostport(scheme, host, port): +def hostport(scheme: AnyStr, host: AnyStr, port: int) -> AnyStr: """ - Returns the host component, with a port specifcation if needed. + Returns the host component, with a port specification if needed. """ - if (port, scheme) in [(80, "http"), (443, "https"), (80, b"http"), (443, b"https")]: + if default_port(scheme) == port: return host else: if isinstance(host, bytes): return b"%s:%d" % (host, port) else: return "%s:%d" % (host, port) + + +def default_port(scheme: AnyStr) -> Optional[int]: + return { + "http": 80, + b"http": 80, + "https": 443, + b"https": 443, + }.get(scheme, None) + + +def parse_authority(authority: AnyStr, check: bool) -> Tuple[str, Optional[int]]: + """Extract the host and port from host header/authority information + + Raises: + ValueError, if check is True and the authority information is malformed. + """ + try: + if isinstance(authority, bytes): + authority_str = authority.decode("idna") + else: + authority_str = authority + m = _authority_re.match(authority_str) + if not m: + raise ValueError + + host = m.group("host") + if host.startswith("[") and host.endswith("]"): + host = host[1:-1] + if not is_valid_host(host): + raise ValueError + + if m.group("port"): + port = int(m.group("port")) + if not is_valid_port(port): + raise ValueError + return host, port + else: + return host, None + + except ValueError: + if check: + raise + else: + return always_str(authority, "utf-8", "surrogateescape"), None diff -Nru mitmproxy-5.1.1/mitmproxy/net/server_spec.py mitmproxy-6.0.2/mitmproxy/net/server_spec.py --- mitmproxy-5.1.1/mitmproxy/net/server_spec.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/net/server_spec.py 2020-12-15 16:41:27.000000000 +0000 @@ -39,19 +39,19 @@ """ m = server_spec_re.match(server_spec) if not m: - raise ValueError("Invalid server specification: {}".format(server_spec)) + raise ValueError(f"Invalid server specification: {server_spec}") # defaulting to https/port 443 may annoy some folks, but it's secure-by-default. scheme = m.group("scheme") or "https" if scheme not in ("http", "https"): - raise ValueError("Invalid server scheme: {}".format(scheme)) + raise ValueError(f"Invalid server scheme: {scheme}") host = m.group("host") # IPv6 brackets if host.startswith("[") and host.endswith("]"): host = host[1:-1] if not check.is_valid_host(host.encode("idna")): - raise ValueError("Invalid hostname: {}".format(host)) + raise ValueError(f"Invalid hostname: {host}") if m.group("port"): port = int(m.group("port")) @@ -61,7 +61,7 @@ "https": 443 }[scheme] if not check.is_valid_port(port): - raise ValueError("Invalid port: {}".format(port)) + raise ValueError(f"Invalid port: {port}") return ServerSpec(scheme, (host, port)) diff -Nru mitmproxy-5.1.1/mitmproxy/net/tcp.py mitmproxy-6.0.2/mitmproxy/net/tcp.py --- mitmproxy-5.1.1/mitmproxy/net/tcp.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/net/tcp.py 2020-12-15 16:41:27.000000000 +0000 @@ -20,7 +20,7 @@ socket_fileobject = socket.SocketIO # workaround for https://bugs.python.org/issue29515 -# Python 3.6 for Windows is missing a constant +# Python 3.8 for Windows is missing a constant, fixed in 3.9 IPPROTO_IPV6 = getattr(socket, "IPPROTO_IPV6", 41) @@ -80,7 +80,7 @@ if hasattr(self.o, "flush"): try: self.o.flush() - except (socket.error, IOError) as v: + except OSError as v: raise exceptions.TcpDisconnect(str(v)) def write(self, v): @@ -97,7 +97,7 @@ r = self.o.write(v) self.add_log(v[:r]) return r - except (SSL.Error, socket.error) as e: + except (SSL.Error, OSError) as e: raise exceptions.TcpDisconnect(str(e)) @@ -125,14 +125,16 @@ # underlying BIO could not satisfy the needs of SSL_read() to continue the # operation. In this case a call to SSL_get_error with the return value of # SSL_read() will yield SSL_ERROR_WANT_READ or SSL_ERROR_WANT_WRITE. - if (time.time() - start) < self.o.gettimeout(): + # 300 is OpenSSL default timeout + timeout = self.o.gettimeout() or 300 + if (time.time() - start) < timeout: time.sleep(0.1) continue else: raise exceptions.TcpTimeout() except socket.timeout: raise exceptions.TcpTimeout() - except socket.error as e: + except OSError as e: raise exceptions.TcpDisconnect(str(e)) except SSL.SysCallError as e: if e.args == (-1, 'Unexpected EOF'): @@ -176,7 +178,7 @@ raise exceptions.TcpDisconnect() else: raise exceptions.TcpReadIncomplete( - "Expected %s bytes, got %s" % (length, len(result)) + "Expected {} bytes, got {}".format(length, len(result)) ) return result @@ -195,7 +197,7 @@ if isinstance(self.o, socket_fileobject): try: return self.o._sock.recv(length, socket.MSG_PEEK) - except socket.error as e: + except OSError as e: raise exceptions.TcpException(repr(e)) elif isinstance(self.o, SSL.Connection): try: @@ -266,7 +268,7 @@ # Now we can close the other half as well. sock.shutdown(socket.SHUT_RD) - except socket.error: + except OSError: pass sock.close() @@ -440,7 +442,7 @@ sock.connect(sa) return sock - except socket.error as _: + except OSError as _: err = _ if sock is not None: sock.close() @@ -448,12 +450,12 @@ if err is not None: raise err else: - raise socket.error("getaddrinfo returns an empty list") # pragma: no cover + raise OSError("getaddrinfo returns an empty list") # pragma: no cover def connect(self): try: connection = self.create_connection() - except (socket.error, IOError) as err: + except OSError as err: raise exceptions.TcpException( 'Error connecting to "%s": %s' % (self.address[0], err) @@ -553,7 +555,7 @@ self.__shutdown_request = False if self.address[0] == 'localhost': - raise socket.error("Binding to 'localhost' is prohibited. Please use '::1' or '127.0.0.1' directly.") + raise OSError("Binding to 'localhost' is prohibited. Please use '::1' or '127.0.0.1' directly.") self.socket = None @@ -566,7 +568,7 @@ self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1) self.socket.setsockopt(IPPROTO_IPV6, socket.IPV6_V6ONLY, 0) self.socket.bind(self.address) - except socket.error: + except OSError: if self.socket: self.socket.close() self.socket = None @@ -578,7 +580,7 @@ self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1) self.socket.bind(self.address) - except socket.error: + except OSError: if self.socket: self.socket.close() self.socket = None @@ -618,7 +620,7 @@ if self.socket in r: connection, client_address = self.socket.accept() t = basethread.BaseThread( - "TCPConnectionHandler (%s: %s:%s -> %s:%s)" % ( + "TCPConnectionHandler ({}: {}:{} -> {}:{})".format( self.__class__.__name__, client_address[0], client_address[1], @@ -652,11 +654,11 @@ # none. if traceback: exc = str(traceback.format_exc()) - print(u'-' * 40, file=fp) + print('-' * 40, file=fp) print( - u"Error in processing of request from %s" % repr(client_address), file=fp) + "Error in processing of request from %s" % repr(client_address), file=fp) print(exc, file=fp) - print(u'-' * 40, file=fp) + print('-' * 40, file=fp) def handle_client_connection(self, conn, client_address): # pragma: no cover """ diff -Nru mitmproxy-5.1.1/mitmproxy/net/tls.py mitmproxy-6.0.2/mitmproxy/net/tls.py --- mitmproxy-5.1.1/mitmproxy/net/tls.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/net/tls.py 2020-12-15 16:41:27.000000000 +0000 @@ -79,43 +79,24 @@ class MasterSecretLogger: def __init__(self, filename): - self.filename = filename + self.filename = os.path.expanduser(filename) self.f = None self.lock = threading.Lock() # required for functools.wraps, which pyOpenSSL uses. __name__ = "MasterSecretLogger" - def __call__(self, connection, where, ret): - done_now = ( - where == SSL.SSL_CB_HANDSHAKE_DONE and ret == 1 - ) - # this is a horrendous workaround for https://github.com/mitmproxy/mitmproxy/pull/3692#issuecomment-608454530: - # OpenSSL 1.1.1f decided to not make connection.master_key() fail in the SSL_CB_HANDSHAKE_DONE callback. - # To support various OpenSSL versions and still log master secrets, we now mark connections where this has - # happened and then try again on the next event. This is ugly and shouldn't be done, but eventually we - # replace this with context.set_keylog_callback anyways. - done_previously_but_not_logged_yet = ( - hasattr(connection, "_still_needs_masterkey") - ) - if done_now or done_previously_but_not_logged_yet: - with self.lock: - if not self.f: - d = os.path.dirname(self.filename) - if not os.path.isdir(d): - os.makedirs(d) - self.f = open(self.filename, "ab") - self.f.write(b"\r\n") - try: - client_random = binascii.hexlify(connection.client_random()) - masterkey = binascii.hexlify(connection.master_key()) - except (AssertionError, SSL.Error): # careful: exception type changes between pyOpenSSL versions - connection._still_needs_masterkey = True - else: - self.f.write(b"CLIENT_RANDOM %s %s\r\n" % (client_random, masterkey)) - self.f.flush() - if hasattr(connection, "_still_needs_masterkey"): - delattr(connection, "_still_needs_masterkey") + def __call__(self, connection, keymaterial): + with self.lock: + if not self.f: + d = os.path.dirname(self.filename) + if not os.path.isdir(d): + os.makedirs(d) + self.f = open(self.filename, "ab") + self.f.write(b"\n") + self.f.write(keymaterial) + self.f.write(b"\n") + self.f.flush() def close(self): with self.lock: @@ -203,7 +184,7 @@ # SSLKEYLOGFILE if log_master_secret: - context.set_info_callback(log_master_secret) + context.set_keylog_callback(log_master_secret) if alpn_protos is not None: # advertise application layer protocols @@ -297,7 +278,7 @@ if cert: try: context.use_privatekey_file(cert) - context.use_certificate_file(cert) + context.use_certificate_chain_file(cert) except SSL.Error as v: raise exceptions.TlsException("SSL client certificate error: %s" % str(v)) return context @@ -321,7 +302,7 @@ request_client_cert: bool = False, chain_file=None, dhparams=None, - extra_chain_certs: typing.Iterable[certs.Cert] = None, + extra_chain_certs: typing.Optional[typing.Iterable[certs.Cert]] = None, **sslctx_kwargs ) -> SSL.Context: """ diff -Nru mitmproxy-5.1.1/mitmproxy/net/websocket.py mitmproxy-6.0.2/mitmproxy/net/websocket.py --- mitmproxy-5.1.1/mitmproxy/net/websocket.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/net/websocket.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,156 @@ +""" +Collection of WebSocket protocol utility functions (RFC6455) +Spec: https://tools.ietf.org/html/rfc6455 +""" + + +import base64 +import hashlib +import os +import struct + +from wsproto.utilities import ACCEPT_GUID +from wsproto.handshake import WEBSOCKET_VERSION +from wsproto.frame_protocol import RsvBits, Header, Frame, XorMaskerSimple, XorMaskerNull + +from mitmproxy.net import http +from mitmproxy.utils import bits, strutils + + +def read_frame(rfile, parse=True): + """ + Reads a full WebSocket frame from a file-like object. + + Returns a parsed frame header, parsed frame, and the consumed bytes. + """ + + consumed_bytes = b'' + + def consume(len): + nonlocal consumed_bytes + d = rfile.safe_read(len) + consumed_bytes += d + return d + + first_byte, second_byte = consume(2) + fin = bits.getbit(first_byte, 7) + rsv1 = bits.getbit(first_byte, 6) + rsv2 = bits.getbit(first_byte, 5) + rsv3 = bits.getbit(first_byte, 4) + opcode = first_byte & 0xF + mask_bit = bits.getbit(second_byte, 7) + length_code = second_byte & 0x7F + + # payload_len > 125 indicates you need to read more bytes + # to get the actual payload length + if length_code <= 125: + payload_len = length_code + elif length_code == 126: + payload_len, = struct.unpack("!H", consume(2)) + else: # length_code == 127: + payload_len, = struct.unpack("!Q", consume(8)) + + # masking key only present if mask bit set + if mask_bit == 1: + masking_key = consume(4) + masker = XorMaskerSimple(masking_key) + else: + masking_key = None + masker = XorMaskerNull() + + masked_payload = consume(payload_len) + + if parse: + header = Header( + fin=fin, + rsv=RsvBits(rsv1, rsv2, rsv3), + opcode=opcode, + payload_len=payload_len, + masking_key=masking_key, + ) + frame = Frame( + opcode=opcode, + payload=masker.process(masked_payload), + frame_finished=fin, + message_finished=fin + ) + else: + header = None + frame = None + + return header, frame, consumed_bytes + + +def client_handshake_headers(version=None, key=None, protocol=None, extensions=None): + """ + Create the headers for a valid HTTP upgrade request. If Key is not + specified, it is generated, and can be found in sec-websocket-key in + the returned header set. + + Returns an instance of http.Headers + """ + if version is None: + version = WEBSOCKET_VERSION + if key is None: + key = base64.b64encode(os.urandom(16)).decode('ascii') + h = http.Headers( + connection="upgrade", + upgrade="websocket", + sec_websocket_version=version, + sec_websocket_key=key, + ) + if protocol is not None: + h['sec-websocket-protocol'] = protocol + if extensions is not None: + h['sec-websocket-extensions'] = extensions + return h + + +def server_handshake_headers(client_key, protocol=None, extensions=None): + """ + The server response is a valid HTTP 101 response. + + Returns an instance of http.Headers + """ + h = http.Headers( + connection="upgrade", + upgrade="websocket", + sec_websocket_accept=create_server_nonce(client_key), + ) + if protocol is not None: + h['sec-websocket-protocol'] = protocol + if extensions is not None: + h['sec-websocket-extensions'] = extensions + return h + + +def check_handshake(headers): + return ( + "upgrade" in headers.get("connection", "").lower() and + headers.get("upgrade", "").lower() == "websocket" and + (headers.get("sec-websocket-key") is not None or headers.get("sec-websocket-accept") is not None) + ) + + +def create_server_nonce(client_nonce): + return base64.b64encode(hashlib.sha1(strutils.always_bytes(client_nonce) + ACCEPT_GUID).digest()) + + +def check_client_version(headers): + return headers.get("sec-websocket-version", "") == WEBSOCKET_VERSION + + +def get_extensions(headers): + return headers.get("sec-websocket-extensions", None) + + +def get_protocol(headers): + return headers.get("sec-websocket-protocol", None) + + +def get_client_key(headers): + return headers.get("sec-websocket-key", None) + + +def get_server_accept(headers): + return headers.get("sec-websocket-accept", None) diff -Nru mitmproxy-5.1.1/mitmproxy/net/websockets/frame.py mitmproxy-6.0.2/mitmproxy/net/websockets/frame.py --- mitmproxy-5.1.1/mitmproxy/net/websockets/frame.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/net/websockets/frame.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,274 +0,0 @@ -import os -import struct -import io - -from mitmproxy.net import tcp -from mitmproxy.utils import strutils -from mitmproxy.utils import bits -from mitmproxy.utils import human -from mitmproxy.coretypes import bidi -from .masker import Masker - - -MAX_16_BIT_INT = (1 << 16) -MAX_64_BIT_INT = (1 << 64) - -DEFAULT = object() - -# RFC 6455, Section 5.2 - Base Framing Protocol -OPCODE = bidi.BiDi( - CONTINUE=0x00, - TEXT=0x01, - BINARY=0x02, - CLOSE=0x08, - PING=0x09, - PONG=0x0a -) - -# RFC 6455, Section 7.4.1 - Defined Status Codes -CLOSE_REASON = bidi.BiDi( - NORMAL_CLOSURE=1000, - GOING_AWAY=1001, - PROTOCOL_ERROR=1002, - UNSUPPORTED_DATA=1003, - RESERVED=1004, - RESERVED_NO_STATUS=1005, - RESERVED_ABNORMAL_CLOSURE=1006, - INVALID_PAYLOAD_DATA=1007, - POLICY_VIOLATION=1008, - MESSAGE_TOO_BIG=1009, - MANDATORY_EXTENSION=1010, - INTERNAL_ERROR=1011, - RESERVED_TLS_HANDHSAKE_FAILED=1015, -) - - -class FrameHeader: - - def __init__( - self, - opcode=OPCODE.TEXT, - payload_length=0, - fin=False, - rsv1=False, - rsv2=False, - rsv3=False, - masking_key=DEFAULT, - mask=DEFAULT, - length_code=DEFAULT - ): - if not 0 <= opcode < 2 ** 4: - raise ValueError("opcode must be 0-16") - self.opcode = opcode - self.payload_length = payload_length - self.fin = fin - self.rsv1 = rsv1 - self.rsv2 = rsv2 - self.rsv3 = rsv3 - - if length_code is DEFAULT: - self.length_code = self._make_length_code(self.payload_length) - else: - self.length_code = length_code - - if (mask is DEFAULT and masking_key is DEFAULT) or mask == 0 or mask is False: - self.mask = False - self.masking_key = b"" - elif mask is DEFAULT: - self.mask = 1 - self.masking_key = masking_key - elif masking_key is DEFAULT: - self.mask = mask - self.masking_key = os.urandom(4) - else: - self.mask = mask - self.masking_key = masking_key - - if self.masking_key and len(self.masking_key) != 4: - raise ValueError("Masking key must be 4 bytes.") - - @classmethod - def _make_length_code(self, length): - """ - A WebSocket frame contains an initial length_code, and an optional - extended length code to represent the actual length if length code is - larger than 125 - """ - if length <= 125: - return length - elif length >= 126 and length <= 65535: - return 126 - else: - return 127 - - def __repr__(self): - vals = [ - "ws frame:", - OPCODE.get_name(self.opcode, hex(self.opcode)).lower() - ] - flags = [] - for i in ["fin", "rsv1", "rsv2", "rsv3", "mask"]: - if getattr(self, i): - flags.append(i) - if flags: - vals.extend([":", "|".join(flags)]) - if self.masking_key: - vals.append(":key=%s" % repr(self.masking_key)) - if self.payload_length: - vals.append(" %s" % human.pretty_size(self.payload_length)) - return "".join(vals) - - def __bytes__(self): - first_byte = bits.setbit(0, 7, self.fin) - first_byte = bits.setbit(first_byte, 6, self.rsv1) - first_byte = bits.setbit(first_byte, 5, self.rsv2) - first_byte = bits.setbit(first_byte, 4, self.rsv3) - first_byte = first_byte | self.opcode - - second_byte = bits.setbit(self.length_code, 7, self.mask) - - b = bytes([first_byte, second_byte]) - - if self.payload_length < 126: - pass - elif self.payload_length < MAX_16_BIT_INT: - # '!H' pack as 16 bit unsigned short - # add 2 byte extended payload length - b += struct.pack('!H', self.payload_length) - elif self.payload_length < MAX_64_BIT_INT: - # '!Q' = pack as 64 bit unsigned long long - # add 8 bytes extended payload length - b += struct.pack('!Q', self.payload_length) - else: - raise ValueError("Payload length exceeds 64bit integer") - - if self.masking_key: - b += self.masking_key - return b - - @classmethod - def from_file(cls, fp): - """ - read a WebSocket frame header - """ - first_byte, second_byte = fp.safe_read(2) - fin = bits.getbit(first_byte, 7) - rsv1 = bits.getbit(first_byte, 6) - rsv2 = bits.getbit(first_byte, 5) - rsv3 = bits.getbit(first_byte, 4) - opcode = first_byte & 0xF - mask_bit = bits.getbit(second_byte, 7) - length_code = second_byte & 0x7F - - # payload_length > 125 indicates you need to read more bytes - # to get the actual payload length - if length_code <= 125: - payload_length = length_code - elif length_code == 126: - payload_length, = struct.unpack("!H", fp.safe_read(2)) - else: # length_code == 127: - payload_length, = struct.unpack("!Q", fp.safe_read(8)) - - # masking key only present if mask bit set - if mask_bit == 1: - masking_key = fp.safe_read(4) - else: - masking_key = None - - return cls( - fin=fin, - rsv1=rsv1, - rsv2=rsv2, - rsv3=rsv3, - opcode=opcode, - mask=mask_bit, - length_code=length_code, - payload_length=payload_length, - masking_key=masking_key, - ) - - def __eq__(self, other): - if isinstance(other, FrameHeader): - return bytes(self) == bytes(other) - return False - - -class Frame: - """ - Represents a single WebSocket frame. - Constructor takes human readable forms of the frame components. - from_bytes() reads from a file-like object to create a new Frame. - - WebSocket frame as defined in RFC6455 - - 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 - +-+-+-+-+-------+-+-------------+-------------------------------+ - |F|R|R|R| opcode|M| Payload len | Extended payload length | - |I|S|S|S| (4) |A| (7) | (16/64) | - |N|V|V|V| |S| | (if payload len==126/127) | - | |1|2|3| |K| | | - +-+-+-+-+-------+-+-------------+ - - - - - - - - - - - - - - - + - | Extended payload length continued, if payload len == 127 | - + - - - - - - - - - - - - - - - +-------------------------------+ - | |Masking-key, if MASK set to 1 | - +-------------------------------+-------------------------------+ - | Masking-key (continued) | Payload Data | - +-------------------------------- - - - - - - - - - - - - - - - + - : Payload Data continued ... : - + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + - | Payload Data continued ... | - +---------------------------------------------------------------+ - """ - - def __init__(self, payload=b"", **kwargs): - self.payload = payload - kwargs["payload_length"] = kwargs.get("payload_length", len(payload)) - self.header = FrameHeader(**kwargs) - - @classmethod - def from_bytes(cls, bytestring): - """ - Construct a websocket frame from an in-memory bytestring - to construct a frame from a stream of bytes, use from_file() directly - """ - return cls.from_file(tcp.Reader(io.BytesIO(bytestring))) - - def __repr__(self): - ret = repr(self.header) - if self.payload: - ret = ret + "\nPayload:\n" + strutils.bytes_to_escaped_str(self.payload) - return ret - - def __bytes__(self): - """ - Serialize the frame to wire format. Returns a string. - """ - b = bytes(self.header) - if self.header.masking_key: - b += Masker(self.header.masking_key)(self.payload) - else: - b += self.payload - return b - - @classmethod - def from_file(cls, fp): - """ - read a WebSocket frame sent by a server or client - - fp is a "file like" object that could be backed by a network - stream or a disk or an in memory stream reader - """ - header = FrameHeader.from_file(fp) - payload = fp.safe_read(header.payload_length) - - if header.mask == 1 and header.masking_key: - payload = Masker(header.masking_key)(payload) - - frame = cls(payload) - frame.header = header - return frame - - def __eq__(self, other): - if isinstance(other, Frame): - return bytes(self) == bytes(other) - return False diff -Nru mitmproxy-5.1.1/mitmproxy/net/websockets/__init__.py mitmproxy-6.0.2/mitmproxy/net/websockets/__init__.py --- mitmproxy-5.1.1/mitmproxy/net/websockets/__init__.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/net/websockets/__init__.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,35 +0,0 @@ -from .frame import FrameHeader -from .frame import Frame -from .frame import OPCODE -from .frame import CLOSE_REASON -from .masker import Masker -from .utils import MAGIC -from .utils import VERSION -from .utils import client_handshake_headers -from .utils import server_handshake_headers -from .utils import check_handshake -from .utils import check_client_version -from .utils import create_server_nonce -from .utils import get_extensions -from .utils import get_protocol -from .utils import get_client_key -from .utils import get_server_accept - -__all__ = [ - "FrameHeader", - "Frame", - "OPCODE", - "CLOSE_REASON", - "Masker", - "MAGIC", - "VERSION", - "client_handshake_headers", - "server_handshake_headers", - "check_handshake", - "check_client_version", - "create_server_nonce", - "get_extensions", - "get_protocol", - "get_client_key", - "get_server_accept", -] diff -Nru mitmproxy-5.1.1/mitmproxy/net/websockets/masker.py mitmproxy-6.0.2/mitmproxy/net/websockets/masker.py --- mitmproxy-5.1.1/mitmproxy/net/websockets/masker.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/net/websockets/masker.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,29 +0,0 @@ -import sys - - -class Masker: - """ - Data sent from the server must be masked to prevent malicious clients - from sending data over the wire in predictable patterns. - - Servers do not have to mask data they send to the client. - https://tools.ietf.org/html/rfc6455#section-5.3 - """ - - def __init__(self, key): - self.key = key - self.offset = 0 - - def mask(self, offset, data): - datalen = len(data) - offset_mod = offset % 4 - data = int.from_bytes(data, sys.byteorder) - num_keys = (datalen + offset_mod + 3) // 4 - mask = int.from_bytes((self.key * num_keys)[offset_mod:datalen + - offset_mod], sys.byteorder) - return (data ^ mask).to_bytes(datalen, sys.byteorder) - - def __call__(self, data): - ret = self.mask(self.offset, data) - self.offset += len(ret) - return ret diff -Nru mitmproxy-5.1.1/mitmproxy/net/websockets/utils.py mitmproxy-6.0.2/mitmproxy/net/websockets/utils.py --- mitmproxy-5.1.1/mitmproxy/net/websockets/utils.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/net/websockets/utils.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,90 +0,0 @@ -""" -Collection of WebSocket protocol utility functions (RFC6455) -Spec: https://tools.ietf.org/html/rfc6455 -""" - - -import base64 -import hashlib -import os - -from mitmproxy.net import http -from mitmproxy.utils import strutils - -MAGIC = b'258EAFA5-E914-47DA-95CA-C5AB0DC85B11' -VERSION = "13" - - -def client_handshake_headers(version=None, key=None, protocol=None, extensions=None): - """ - Create the headers for a valid HTTP upgrade request. If Key is not - specified, it is generated, and can be found in sec-websocket-key in - the returned header set. - - Returns an instance of http.Headers - """ - if version is None: - version = VERSION - if key is None: - key = base64.b64encode(os.urandom(16)).decode('ascii') - h = http.Headers( - connection="upgrade", - upgrade="websocket", - sec_websocket_version=version, - sec_websocket_key=key, - ) - if protocol is not None: - h['sec-websocket-protocol'] = protocol - if extensions is not None: - h['sec-websocket-extensions'] = extensions - return h - - -def server_handshake_headers(client_key, protocol=None, extensions=None): - """ - The server response is a valid HTTP 101 response. - - Returns an instance of http.Headers - """ - h = http.Headers( - connection="upgrade", - upgrade="websocket", - sec_websocket_accept=create_server_nonce(client_key), - ) - if protocol is not None: - h['sec-websocket-protocol'] = protocol - if extensions is not None: - h['sec-websocket-extensions'] = extensions - return h - - -def check_handshake(headers): - return ( - "upgrade" in headers.get("connection", "").lower() and - headers.get("upgrade", "").lower() == "websocket" and - (headers.get("sec-websocket-key") is not None or headers.get("sec-websocket-accept") is not None) - ) - - -def create_server_nonce(client_nonce): - return base64.b64encode(hashlib.sha1(strutils.always_bytes(client_nonce) + MAGIC).digest()) - - -def check_client_version(headers): - return headers.get("sec-websocket-version", "") == VERSION - - -def get_extensions(headers): - return headers.get("sec-websocket-extensions", None) - - -def get_protocol(headers): - return headers.get("sec-websocket-protocol", None) - - -def get_client_key(headers): - return headers.get("sec-websocket-key", None) - - -def get_server_accept(headers): - return headers.get("sec-websocket-accept", None) diff -Nru mitmproxy-5.1.1/mitmproxy/net/wsgi.py mitmproxy-6.0.2/mitmproxy/net/wsgi.py --- mitmproxy-5.1.1/mitmproxy/net/wsgi.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/net/wsgi.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,165 +0,0 @@ -import time -import traceback -import urllib -import io - -from mitmproxy.net import http -from mitmproxy.utils import strutils - - -class ClientConn: - - def __init__(self, address): - self.address = address - - -class Flow: - - def __init__(self, address, request): - self.client_conn = ClientConn(address) - self.request = request - - -class Request: - - def __init__(self, scheme, method, path, http_version, headers, content): - self.scheme, self.method, self.path = scheme, method, path - self.headers, self.content = headers, content - self.http_version = http_version - - -def date_time_string(): - """Return the current date and time formatted for a message header.""" - WEEKS = ['Mon', 'Tue', 'Wed', 'Thu', 'Fri', 'Sat', 'Sun'] - MONTHS = [ - None, - 'Jan', 'Feb', 'Mar', 'Apr', 'May', 'Jun', - 'Jul', 'Aug', 'Sep', 'Oct', 'Nov', 'Dec' - ] - now = time.time() - year, month, day, hh, mm, ss, wd, y_, z_ = time.gmtime(now) - s = "%s, %02d %3s %4d %02d:%02d:%02d GMT" % ( - WEEKS[wd], - day, MONTHS[month], year, - hh, mm, ss - ) - return s - - -class WSGIAdaptor: - - def __init__(self, app, domain, port, sversion): - self.app, self.domain, self.port, self.sversion = app, domain, port, sversion - - def make_environ(self, flow, errsoc, **extra): - """ - Raises: - ValueError, if the content-encoding is invalid. - """ - path = strutils.always_str(flow.request.path, "latin-1") - if '?' in path: - path_info, query = strutils.always_str(path, "latin-1").split('?', 1) - else: - path_info = path - query = '' - environ = { - 'wsgi.version': (1, 0), - 'wsgi.url_scheme': strutils.always_str(flow.request.scheme, "latin-1"), - 'wsgi.input': io.BytesIO(flow.request.content or b""), - 'wsgi.errors': errsoc, - 'wsgi.multithread': True, - 'wsgi.multiprocess': False, - 'wsgi.run_once': False, - 'SERVER_SOFTWARE': self.sversion, - 'REQUEST_METHOD': strutils.always_str(flow.request.method, "latin-1"), - 'SCRIPT_NAME': '', - 'PATH_INFO': urllib.parse.unquote(path_info), - 'QUERY_STRING': query, - 'CONTENT_TYPE': strutils.always_str(flow.request.headers.get('Content-Type', ''), "latin-1"), - 'CONTENT_LENGTH': strutils.always_str(flow.request.headers.get('Content-Length', ''), "latin-1"), - 'SERVER_NAME': self.domain, - 'SERVER_PORT': str(self.port), - 'SERVER_PROTOCOL': strutils.always_str(flow.request.http_version, "latin-1"), - } - environ.update(extra) - if flow.client_conn.address: - environ["REMOTE_ADDR"] = strutils.always_str(flow.client_conn.address[0], "latin-1") - environ["REMOTE_PORT"] = flow.client_conn.address[1] - - for key, value in flow.request.headers.items(): - key = 'HTTP_' + strutils.always_str(key, "latin-1").upper().replace('-', '_') - if key not in ('HTTP_CONTENT_TYPE', 'HTTP_CONTENT_LENGTH'): - environ[key] = value - return environ - - def error_page(self, soc, headers_sent, s): - """ - Make a best-effort attempt to write an error page. If headers are - already sent, we just bung the error into the page. - """ - c = """ - -

Internal Server Error

- 
{err}"
- - """.format(err=s).strip().encode() - - if not headers_sent: - soc.write(b"HTTP/1.1 500 Internal Server Error\r\n") - soc.write(b"Content-Type: text/html\r\n") - soc.write("Content-Length: {length}\r\n".format(length=len(c)).encode()) - soc.write(b"\r\n") - soc.write(c) - - def serve(self, request, soc, **env): - state = dict( - response_started=False, - headers_sent=False, - status=None, - headers=None - ) - - def write(data): - if not state["headers_sent"]: - soc.write("HTTP/1.1 {status}\r\n".format(status=state["status"]).encode()) - headers = state["headers"] - if 'server' not in headers: - headers["Server"] = self.sversion - if 'date' not in headers: - headers["Date"] = date_time_string() - soc.write(bytes(headers)) - soc.write(b"\r\n") - state["headers_sent"] = True - if data: - soc.write(data) - soc.flush() - - def start_response(status, headers, exc_info=None): - if exc_info: - if state["headers_sent"]: - raise exc_info[1] - elif state["status"]: - raise AssertionError('Response already started') - state["status"] = status - state["headers"] = http.Headers([[strutils.always_bytes(k), strutils.always_bytes(v)] for k, v in headers]) - if exc_info: - self.error_page(soc, state["headers_sent"], traceback.format_tb(exc_info[2])) - state["headers_sent"] = True - - errs = io.BytesIO() - try: - dataiter = self.app( - self.make_environ(request, errs, **env), start_response - ) - for i in dataiter: - write(i) - if not state["headers_sent"]: - write(b"") - except Exception: - try: - s = traceback.format_exc() - errs.write(s.encode("utf-8", "replace")) - self.error_page(soc, state["headers_sent"], s) - except Exception: # pragma: no cover - pass - return errs.getvalue() diff -Nru mitmproxy-5.1.1/mitmproxy/options.py mitmproxy-6.0.2/mitmproxy/options.py --- mitmproxy-5.1.1/mitmproxy/options.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/options.py 2020-12-15 16:41:27.000000000 +0000 @@ -48,6 +48,10 @@ """ ) self.add_option( + "cert_passphrase", Optional[str], None, + "Passphrase for decrypting the private key provided in the --cert option." + ) + self.add_option( "ciphers_client", Optional[str], None, "Set supported ciphers for client connections using OpenSSL syntax." ) @@ -164,8 +168,8 @@ "tcp_hosts", Sequence[str], [], """ Generic TCP SSL proxy mode for all hosts that match the pattern. - Similar to --ignore, but SSL connections are intercepted. The - communication contents are printed to the log in verbose mode. + Similar to --ignore-hosts, but SSL connections are intercepted. + The communication contents are printed to the log in verbose mode. """ ) self.add_option( @@ -181,5 +185,11 @@ TLS key size for certificates and CA. """ ) + self.add_option( + "relax_http_form_validation", bool, False, + """ + Disable HTTP form validation. + """ + ) self.update(**kwargs) diff -Nru mitmproxy-5.1.1/mitmproxy/optmanager.py mitmproxy-6.0.2/mitmproxy/optmanager.py --- mitmproxy-5.1.1/mitmproxy/optmanager.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/optmanager.py 2020-12-15 16:41:27.000000000 +0000 @@ -26,7 +26,7 @@ def __init__( self, name: str, - typespec: type, + typespec: typing.Union[type, object], # object for Optional[x], which is not a type. default: typing.Any, help: str, choices: typing.Optional[typing.Sequence[str]] @@ -40,7 +40,7 @@ self.choices = choices def __repr__(self): - return "{value} [{type}]".format(value=self.current(), type=self.typespec) + return f"{self.current()} [{self.typespec}]" @property def default(self): @@ -101,7 +101,7 @@ def add_option( self, name: str, - typespec: type, + typespec: typing.Union[type, object], default: typing.Any, help: str, choices: typing.Optional[typing.Sequence[str]] = None @@ -517,18 +517,18 @@ for p in paths: p = os.path.expanduser(p) if os.path.exists(p) and os.path.isfile(p): - with open(p, "rt", encoding="utf8") as f: + with open(p, encoding="utf8") as f: try: txt = f.read() except UnicodeDecodeError as e: raise exceptions.OptionsError( - "Error reading %s: %s" % (p, e) + f"Error reading {p}: {e}" ) try: load(opts, txt) except exceptions.OptionsError as e: raise exceptions.OptionsError( - "Error reading %s: %s" % (p, e) + f"Error reading {p}: {e}" ) @@ -563,12 +563,12 @@ """ path = os.path.expanduser(path) if os.path.exists(path) and os.path.isfile(path): - with open(path, "rt", encoding="utf8") as f: + with open(path, encoding="utf8") as f: try: data = f.read() except UnicodeDecodeError as e: raise exceptions.OptionsError( - "Error trying to modify %s: %s" % (path, e) + f"Error trying to modify {path}: {e}" ) else: data = "" diff -Nru mitmproxy-5.1.1/mitmproxy/platform/openbsd.py mitmproxy-6.0.2/mitmproxy/platform/openbsd.py --- mitmproxy-5.1.1/mitmproxy/platform/openbsd.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/platform/openbsd.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,3 +1,2 @@ - def original_addr(csock): return csock.getsockname() diff -Nru mitmproxy-5.1.1/mitmproxy/platform/pf.py mitmproxy-6.0.2/mitmproxy/platform/pf.py --- mitmproxy-5.1.1/mitmproxy/platform/pf.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/platform/pf.py 2020-12-15 16:41:27.000000000 +0000 @@ -15,10 +15,10 @@ s = s.decode() # ALL tcp 192.168.1.13:57474 -> 23.205.82.58:443 ESTABLISHED:ESTABLISHED - specv4 = "%s:%s" % (address, port) + specv4 = f"{address}:{port}" # ALL tcp 2a01:e35:8bae:50f0:9d9b:ef0d:2de3:b733[58505] -> 2606:4700:30::681f:4ad0[443] ESTABLISHED:ESTABLISHED - specv6 = "%s[%s]" % (address, port) + specv6 = f"{address}[{port}]" for i in s.split("\n"): if "ESTABLISHED:ESTABLISHED" in i and specv4 in i: diff -Nru mitmproxy-5.1.1/mitmproxy/platform/windows.py mitmproxy-6.0.2/mitmproxy/platform/windows.py --- mitmproxy-5.1.1/mitmproxy/platform/windows.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/platform/windows.py 2020-12-15 16:41:27.000000000 +0000 @@ -68,7 +68,7 @@ if addr is None: raise RuntimeError("Cannot resolve original destination.") return tuple(addr) - except (EOFError, socket.error): + except (EOFError, OSError): self._connect() return self.original_addr(csock) @@ -91,7 +91,7 @@ except KeyError: server = None write(server, self.wfile) - except (EOFError, socket.error): + except (EOFError, OSError): pass diff -Nru mitmproxy-5.1.1/mitmproxy/proxy/config.py mitmproxy-6.0.2/mitmproxy/proxy/config.py --- mitmproxy-5.1.1/mitmproxy/proxy/config.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/proxy/config.py 2020-12-15 16:41:27.000000000 +0000 @@ -62,10 +62,12 @@ os.path.dirname(certstore_path) ) key_size = options.key_size + passphrase = options.cert_passphrase.encode("utf-8") if options.cert_passphrase else None self.certstore = certs.CertStore.from_store( certstore_path, moptions.CONF_BASENAME, - key_size + key_size, + passphrase ) for c in options.certs: @@ -79,7 +81,7 @@ "Certificate file does not exist: %s" % cert ) try: - self.certstore.add_cert_file(parts[0], cert) + self.certstore.add_cert_file(parts[0], cert, passphrase) except crypto.Error: raise exceptions.OptionsError( "Invalid certificate format: %s" % cert diff -Nru mitmproxy-5.1.1/mitmproxy/proxy/modes/transparent_proxy.py mitmproxy-6.0.2/mitmproxy/proxy/modes/transparent_proxy.py --- mitmproxy-5.1.1/mitmproxy/proxy/modes/transparent_proxy.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/proxy/modes/transparent_proxy.py 2020-12-15 16:41:27.000000000 +0000 @@ -10,7 +10,7 @@ def __call__(self): try: - self.server_conn.address = platform.original_addr(self.client_conn.connection) + self.set_server(platform.original_addr(self.client_conn.connection)) except Exception as e: raise exceptions.ProtocolException("Transparent mode failure: %s" % repr(e)) diff -Nru mitmproxy-5.1.1/mitmproxy/proxy/protocol/base.py mitmproxy-6.0.2/mitmproxy/proxy/protocol/base.py --- mitmproxy-5.1.1/mitmproxy/proxy/protocol/base.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/proxy/protocol/base.py 2020-12-15 16:41:27.000000000 +0000 @@ -107,9 +107,14 @@ """ address = self.server_conn.address if address: + forbidden_hosts = ["localhost", "127.0.0.1", "::1"] + + if self.config.options.listen_host: + forbidden_hosts.append(self.config.options.listen_host) + self_connect = ( address[1] == self.config.options.listen_port and - address[0] in ("localhost", "127.0.0.1", "::1") + address[0] in forbidden_hosts ) if self_connect: raise exceptions.ProtocolException( diff -Nru mitmproxy-5.1.1/mitmproxy/proxy/protocol/http1.py mitmproxy-6.0.2/mitmproxy/proxy/protocol/http1.py --- mitmproxy-5.1.1/mitmproxy/proxy/protocol/http1.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/proxy/protocol/http1.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,6 +1,5 @@ -from mitmproxy import http -from mitmproxy.proxy.protocol import http as httpbase from mitmproxy.net.http import http1 +from mitmproxy.proxy.protocol import http as httpbase from mitmproxy.utils import human @@ -11,9 +10,7 @@ self.mode = mode def read_request_headers(self, flow): - return http.HTTPRequest.wrap( - http1.read_request_head(self.client_conn.rfile) - ) + return http1.read_request_head(self.client_conn.rfile) def read_request_body(self, request): expected_size = http1.expected_http_body_size(request) @@ -23,23 +20,32 @@ human.parse_size(self.config.options.body_size_limit) ) + def read_request_trailers(self, request): + if "Trailer" in request.headers: + # TODO: not implemented yet + self.log("HTTP/1.1 request trailer headers are not implemented yet!", "warn") + return None + def send_request_headers(self, request): headers = http1.assemble_request_head(request) self.server_conn.wfile.write(headers) self.server_conn.wfile.flush() def send_request_body(self, request, chunks): - for chunk in http1.assemble_body(request.headers, chunks): + for chunk in http1.assemble_body(request.headers, chunks, request.trailers): self.server_conn.wfile.write(chunk) self.server_conn.wfile.flush() + def send_request_trailers(self, request): + # HTTP/1.1 request trailer headers are sent in the body + pass + def send_request(self, request): self.server_conn.wfile.write(http1.assemble_request(request)) self.server_conn.wfile.flush() def read_response_headers(self): - resp = http1.read_response_head(self.server_conn.rfile) - return http.HTTPResponse.wrap(resp) + return http1.read_response_head(self.server_conn.rfile) def read_response_body(self, request, response): expected_size = http1.expected_http_body_size(request, response) @@ -49,16 +55,27 @@ human.parse_size(self.config.options.body_size_limit) ) + def read_response_trailers(self, request, response): + # Trailers should actually be parsed unconditionally, the "Trailer" header is optional + if "Trailer" in response.headers: + # TODO: not implemented yet + self.log("HTTP/1.1 trailer headers are not implemented yet!", "warn") + return None + def send_response_headers(self, response): raw = http1.assemble_response_head(response) self.client_conn.wfile.write(raw) self.client_conn.wfile.flush() def send_response_body(self, response, chunks): - for chunk in http1.assemble_body(response.headers, chunks): + for chunk in http1.assemble_body(response.headers, chunks, response.trailers): self.client_conn.wfile.write(chunk) self.client_conn.wfile.flush() + def send_response_trailers(self, response): + # HTTP/1.1 response trailer headers are sent in the body + pass + def check_close_connection(self, flow): request_close = http1.connection_close( flow.request.http_version, diff -Nru mitmproxy-5.1.1/mitmproxy/proxy/protocol/http2.py mitmproxy-6.0.2/mitmproxy/proxy/protocol/http2.py --- mitmproxy-5.1.1/mitmproxy/proxy/protocol/http2.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/proxy/protocol/http2.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,6 +1,7 @@ import threading import time import functools +import types from typing import Dict, Callable, Any, List, Optional # noqa import h2.exceptions @@ -8,7 +9,7 @@ from h2 import events import queue -from mitmproxy import connections # noqa +from mitmproxy import connections, flow # noqa from mitmproxy import exceptions from mitmproxy import http from mitmproxy.proxy.protocol import base @@ -16,7 +17,7 @@ import mitmproxy.net.http from mitmproxy.net import tcp from mitmproxy.coretypes import basethread -from mitmproxy.net.http import http2, headers +from mitmproxy.net.http import http2, headers, url from mitmproxy.utils import human @@ -55,7 +56,7 @@ self.send_headers(stream_id, headers.fields, **kwargs) self.conn.send(self.data_to_send()) - def safe_send_body(self, raise_zombie: Callable, stream_id: int, chunks: List[bytes]): + def safe_send_body(self, raise_zombie: Callable, stream_id: int, chunks: List[bytes], end_stream=True): for chunk in chunks: position = 0 while position < len(chunk): @@ -75,10 +76,11 @@ finally: self.lock.release() position += max_outbound_frame_size - with self.lock: - raise_zombie() - self.end_stream(stream_id) - self.conn.send(self.data_to_send()) + if end_stream: + with self.lock: + raise_zombie() + self.end_stream(stream_id) + self.conn.send(self.data_to_send()) class Http2Layer(base.Layer): @@ -170,7 +172,7 @@ elif isinstance(event, events.PriorityUpdated): return self._handle_priority_updated(eid, event) elif isinstance(event, events.TrailersReceived): - raise NotImplementedError('TrailersReceived not implemented') + return self._handle_trailers(eid, event, is_server, other_conn) # fail-safe for unhandled events return True @@ -179,22 +181,21 @@ headers = mitmproxy.net.http.Headers([[k, v] for k, v in event.headers]) self.streams[eid] = Http2SingleStreamLayer(self, self.connections[self.client_conn], eid, headers) self.streams[eid].timestamp_start = time.time() - self.streams[eid].no_body = (event.stream_ended is not None) if event.priority_updated is not None: self.streams[eid].priority_exclusive = event.priority_updated.exclusive self.streams[eid].priority_depends_on = event.priority_updated.depends_on self.streams[eid].priority_weight = event.priority_updated.weight self.streams[eid].handled_priority_event = event.priority_updated self.streams[eid].start() - self.streams[eid].request_arrived.set() + self.streams[eid].request_message.arrived.set() return True def _handle_response_received(self, eid, event): headers = mitmproxy.net.http.Headers([[k, v] for k, v in event.headers]) self.streams[eid].queued_data_length = 0 self.streams[eid].timestamp_start = time.time() - self.streams[eid].response_headers = headers - self.streams[eid].response_arrived.set() + self.streams[eid].response_message.headers = headers + self.streams[eid].response_message.arrived.set() return True def _handle_data_received(self, eid, event, source_conn): @@ -205,7 +206,7 @@ event.stream_id, h2.errors.ErrorCodes.REFUSED_STREAM ) - self.log("HTTP body too large. Limit is {}.".format(bsl), "info") + self.log(f"HTTP body too large. Limit is {bsl}.", "info") else: self.streams[eid].data_queue.put(event.data) self.streams[eid].queued_data_length += len(event.data) @@ -219,7 +220,7 @@ def _handle_stream_ended(self, eid): self.streams[eid].timestamp_end = time.time() - self.streams[eid].data_finished.set() + self.streams[eid].stream_ended.set() return True def _handle_stream_reset(self, eid, event, is_server, other_conn): @@ -233,8 +234,13 @@ self.connections[other_conn].safe_reset_stream(other_stream_id, event.error_code) return True + def _handle_trailers(self, eid, event, is_server, other_conn): + trailers = mitmproxy.net.http.Headers([[k, v] for k, v in event.headers]) + self.streams[eid].trailers = trailers + return True + def _handle_remote_settings_changed(self, event, other_conn): - new_settings = dict([(key, cs.new_value) for (key, cs) in event.changed_settings.items()]) + new_settings = {key: cs.new_value for (key, cs) in event.changed_settings.items()} self.connections[other_conn].safe_update_settings(new_settings) return True @@ -277,8 +283,8 @@ self.streams[event.pushed_stream_id].pushed = True self.streams[event.pushed_stream_id].parent_stream_id = parent_eid self.streams[event.pushed_stream_id].timestamp_end = time.time() - self.streams[event.pushed_stream_id].request_arrived.set() - self.streams[event.pushed_stream_id].request_data_finished.set() + self.streams[event.pushed_stream_id].request_message.arrived.set() + self.streams[event.pushed_stream_id].request_message.stream_ended.set() self.streams[event.pushed_stream_id].start() return True @@ -354,7 +360,7 @@ with self.connections[source_conn].lock: try: - raw_frame = b''.join(http2.read_raw_frame(source_conn.rfile)) + _, consumed_bytes = http2.read_frame(source_conn.rfile) except: # read frame failed: connection closed self._kill_all_streams() @@ -364,7 +370,7 @@ self.log("HTTP/2 connection entered closed state already", "debug") return - incoming_events = self.connections[source_conn].receive_data(raw_frame) + incoming_events = self.connections[source_conn].receive_data(consumed_bytes) source_conn.send(self.connections[source_conn].data_to_send()) for event in incoming_events: @@ -392,32 +398,31 @@ class Http2SingleStreamLayer(httpbase._HttpTransmissionLayer, basethread.BaseThread): + class Message: + def __init__(self, headers=None): + self.headers: Optional[mitmproxy.net.http.Headers] = headers # headers are the first thing to be received on a new stream + self.data_queue: queue.Queue[bytes] = queue.Queue() # contains raw contents of DATA frames + self.queued_data_length = 0 # used to enforce mitmproxy's config.options.body_size_limit + self.trailers: Optional[mitmproxy.net.http.Headers] = None # trailers are received after stream_ended is set + + self.arrived = threading.Event() # indicates the HEADERS+CONTINUTATION frames have been received + self.stream_ended = threading.Event() # indicates the a frame with the END_STREAM flag has been received + def __init__(self, ctx, h2_connection, stream_id: int, request_headers: mitmproxy.net.http.Headers) -> None: super().__init__( - ctx, name="Http2SingleStreamLayer-{}".format(stream_id) + ctx, name=f"Http2SingleStreamLayer-{stream_id}" ) self.h2_connection = h2_connection self.zombie: Optional[float] = None self.client_stream_id: int = stream_id self.server_stream_id: Optional[int] = None - self.request_headers = request_headers - self.response_headers: Optional[mitmproxy.net.http.Headers] = None self.pushed = False self.timestamp_start: Optional[float] = None self.timestamp_end: Optional[float] = None - self.request_arrived = threading.Event() - self.request_data_queue: queue.Queue[bytes] = queue.Queue() - self.request_queued_data_length = 0 - self.request_data_finished = threading.Event() - - self.response_arrived = threading.Event() - self.response_data_queue: queue.Queue[bytes] = queue.Queue() - self.response_queued_data_length = 0 - self.response_data_finished = threading.Event() - - self.no_body = False + self.request_message = self.Message(request_headers) + self.response_message = self.Message() self.priority_exclusive: bool self.priority_depends_on: Optional[int] = None @@ -427,10 +432,10 @@ def kill(self): if not self.zombie: self.zombie = time.time() - self.request_data_finished.set() - self.request_arrived.set() - self.response_arrived.set() - self.response_data_finished.set() + self.request_message.stream_ended.set() + self.request_message.arrived.set() + self.response_message.arrived.set() + self.response_message.stream_ended.set() def connect(self): # pragma: no cover raise exceptions.Http2ProtocolException("HTTP2 layer should already have a connection.") @@ -448,77 +453,106 @@ @property def data_queue(self): - if self.response_arrived.is_set(): - return self.response_data_queue + if self.response_message.arrived.is_set(): + return self.response_message.data_queue else: - return self.request_data_queue + return self.request_message.data_queue @property def queued_data_length(self): - if self.response_arrived.is_set(): - return self.response_queued_data_length + if self.response_message.arrived.is_set(): + return self.response_message.queued_data_length else: - return self.request_queued_data_length + return self.request_message.queued_data_length @queued_data_length.setter def queued_data_length(self, v): - self.request_queued_data_length = v + self.request_message.queued_data_length = v @property - def data_finished(self): - if self.response_arrived.is_set(): - return self.response_data_finished + def stream_ended(self): + # This indicates that all message headers, the full message body, and all trailers have been received + # https://tools.ietf.org/html/rfc7540#section-8.1 + if self.response_message.arrived.is_set(): + return self.response_message.stream_ended else: - return self.request_data_finished + return self.request_message.stream_ended + + @property + def trailers(self): + if self.response_message.arrived.is_set(): + return self.response_message.trailers + else: + return self.request_message.trailers + + @trailers.setter + def trailers(self, v): + if self.response_message.arrived.is_set(): + self.response_message.trailers = v + else: + self.request_message.trailers = v def raise_zombie(self, pre_command=None): # pragma: no cover connection_closed = self.h2_connection.state_machine.state == h2.connection.ConnectionState.CLOSED if self.zombie is not None or connection_closed: if pre_command is not None: pre_command() - raise exceptions.Http2ZombieException("Connection or stream already dead: {}, {}".format(self.zombie, connection_closed)) + raise exceptions.Http2ZombieException(f"Connection or stream already dead: {self.zombie}, {connection_closed}") @detect_zombie_stream def read_request_headers(self, flow): - self.request_arrived.wait() + self.request_message.arrived.wait() self.raise_zombie() if self.pushed: flow.metadata['h2-pushed-stream'] = True - first_line_format, method, scheme, host, port, path = http2.parse_headers(self.request_headers) + # pseudo header must be present, see https://http2.github.io/http2-spec/#rfc.section.8.1.2.3 + authority = self.request_message.headers.pop(':authority', "") + method = self.request_message.headers.pop(':method') + scheme = self.request_message.headers.pop(':scheme') + path = self.request_message.headers.pop(':path') + + host, port = url.parse_authority(authority, check=True) + port = port or url.default_port(scheme) or 0 + return http.HTTPRequest( - first_line_format, - method, - scheme, host, port, - path, + method.encode(), + scheme.encode(), + authority.encode(), + path.encode(), b"HTTP/2.0", - self.request_headers, + self.request_message.headers, None, - timestamp_start=self.timestamp_start, - timestamp_end=self.timestamp_end, + None, + self.timestamp_start, + self.timestamp_end, ) @detect_zombie_stream def read_request_body(self, request): if not request.stream: - self.request_data_finished.wait() + self.request_message.stream_ended.wait() while True: try: - yield self.request_data_queue.get(timeout=0.1) + yield self.request_message.data_queue.get(timeout=0.1) except queue.Empty: # pragma: no cover pass - if self.request_data_finished.is_set(): + if self.request_message.stream_ended.is_set(): self.raise_zombie() - while self.request_data_queue.qsize() > 0: - yield self.request_data_queue.get() + while self.request_message.data_queue.qsize() > 0: + yield self.request_message.data_queue.get() break self.raise_zombie() @detect_zombie_stream + def read_request_trailers(self, request): + return self.request_message.trailers + + @detect_zombie_stream def send_request_headers(self, request): if self.pushed: # nothing to do here @@ -545,6 +579,8 @@ self.server_to_client_stream_ids[self.server_stream_id] = self.client_stream_id headers = request.headers.copy() + if request.authority: + headers.insert(0, ":authority", request.authority) headers.insert(0, ":path", request.path) headers.insert(0, ":method", request.method) headers.insert(0, ":scheme", request.scheme) @@ -567,7 +603,7 @@ self.raise_zombie, self.server_stream_id, headers, - end_stream=self.no_body, + end_stream=(False if request.content or request.trailers or request.stream else True), priority_exclusive=priority_exclusive, priority_depends_on=priority_depends_on, priority_weight=priority_weight, @@ -584,26 +620,32 @@ # nothing to do here return - if not self.no_body: + if isinstance(chunks, types.GeneratorType) or (chunks and chunks[0]): self.connections[self.server_conn].safe_send_body( self.raise_zombie, self.server_stream_id, - chunks + chunks, + end_stream=(request.trailers is None), ) @detect_zombie_stream - def send_request(self, message): - self.send_request_headers(message) - self.send_request_body(message, [message.content]) + def send_request_trailers(self, request): + self._send_trailers(self.server_conn, request.trailers) + + @detect_zombie_stream + def send_request(self, request): + self.send_request_headers(request) + self.send_request_body(request, [request.content]) + self.send_request_trailers(request) @detect_zombie_stream def read_response_headers(self): - self.response_arrived.wait() + self.response_message.arrived.wait() self.raise_zombie() - status_code = int(self.response_headers.get(':status', 502)) - headers = self.response_headers.copy() + status_code = int(self.response_message.headers.get(':status', 502)) + headers = self.response_message.headers.copy() headers.pop(":status", None) return http.HTTPResponse( @@ -612,6 +654,7 @@ reason=b'', headers=headers, content=None, + trailers=None, timestamp_start=self.timestamp_start, timestamp_end=self.timestamp_end, ) @@ -620,17 +663,21 @@ def read_response_body(self, request, response): while True: try: - yield self.response_data_queue.get(timeout=0.1) + yield self.response_message.data_queue.get(timeout=0.1) except queue.Empty: # pragma: no cover pass - if self.response_data_finished.is_set(): + if self.response_message.stream_ended.is_set(): self.raise_zombie() - while self.response_data_queue.qsize() > 0: - yield self.response_data_queue.get() + while self.response_message.data_queue.qsize() > 0: + yield self.response_message.data_queue.get() break self.raise_zombie() @detect_zombie_stream + def read_response_trailers(self, request, response): + return self.response_message.trailers + + @detect_zombie_stream def send_response_headers(self, response): headers = response.headers.copy() headers.insert(0, ":status", str(response.status_code)) @@ -642,15 +689,31 @@ ) @detect_zombie_stream - def send_response_body(self, _response, chunks): + def send_response_body(self, response, chunks): self.connections[self.client_conn].safe_send_body( self.raise_zombie, self.client_stream_id, - chunks + chunks, + end_stream=(response.trailers is None), ) + @detect_zombie_stream + def send_response_trailers(self, response): + self._send_trailers(self.client_conn, response.trailers) + + def _send_trailers(self, conn, trailers): + if not trailers: + return + with self.connections[conn].lock: + self.connections[conn].safe_send_headers( + self.raise_zombie, + self.client_stream_id, + trailers, + end_stream=True + ) + def __call__(self): # pragma: no cover - raise EnvironmentError('Http2SingleStreamLayer must be run as thread') + raise OSError('Http2SingleStreamLayer must be run as thread') def run(self): layer = httpbase.HttpLayer(self, self.mode) @@ -663,8 +726,8 @@ except exceptions.ProtocolException as e: # pragma: no cover self.log(repr(e), "info") except exceptions.SetServerNotAllowedException as e: # pragma: no cover - self.log("Changing the Host server for HTTP/2 connections not allowed: {}".format(e), "info") + self.log(f"Changing the Host server for HTTP/2 connections not allowed: {e}", "info") except exceptions.Kill: # pragma: no cover - self.log("Connection killed", "info") + self.log(flow.Error.KILLED_MESSAGE, "info") self.kill() diff -Nru mitmproxy-5.1.1/mitmproxy/proxy/protocol/http.py mitmproxy-6.0.2/mitmproxy/proxy/protocol/http.py --- mitmproxy-5.1.1/mitmproxy/proxy/protocol/http.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/proxy/protocol/http.py 2020-12-15 16:41:27.000000000 +0000 @@ -8,9 +8,10 @@ from mitmproxy import exceptions from mitmproxy import http from mitmproxy import flow +from mitmproxy.net.http import url from mitmproxy.proxy.protocol import base from mitmproxy.proxy.protocol.websocket import WebSocketLayer -from mitmproxy.net import websockets +from mitmproxy.net import websocket class _HttpTransmissionLayer(base.Layer): @@ -20,6 +21,9 @@ def read_request_body(self, request): raise NotImplementedError() + def read_request_trailers(self, request): + raise NotImplementedError() + def send_request(self, request): raise NotImplementedError() @@ -30,11 +34,15 @@ raise NotImplementedError() yield "this is a generator" # pragma: no cover + def read_response_trailers(self, request, response): + raise NotImplementedError() + def read_response(self, request): response = self.read_response_headers() response.data.content = b"".join( self.read_response_body(request, response) ) + response.data.trailers = self.read_response_trailers(request, response) return response def send_response(self, response): @@ -42,6 +50,7 @@ raise exceptions.HttpException("Cannot assemble flow with missing content") self.send_response_headers(response) self.send_response_body(response, [response.data.content]) + self.send_response_trailers(response) def send_response_headers(self, response): raise NotImplementedError() @@ -49,6 +58,9 @@ def send_response_body(self, response, chunks): raise NotImplementedError() + def send_response_trailers(self, response, chunks): + raise NotImplementedError() + def check_close_connection(self, f): raise NotImplementedError() @@ -90,8 +102,8 @@ def _send_connect_request(self): self.log("Sending CONNECT request", "debug", [ - "Proxy Server: {}".format(self.ctx.server_conn.address), - "Connect to: {}:{}".format(self.connect_request.host, self.connect_request.port) + f"Proxy Server: {self.ctx.server_conn.address}", + f"Connect to: {self.connect_request.host}:{self.connect_request.port}" ]) self.send_request(self.connect_request) resp = self.read_response(self.connect_request) @@ -137,22 +149,24 @@ def validate_request_form(mode, request): - if request.first_line_format == "absolute" and request.scheme != "http": + if request.first_line_format == "absolute" and request.scheme not in ("http", "https"): raise exceptions.HttpException( "Invalid request scheme: %s" % request.scheme ) allowed_request_forms = MODE_REQUEST_FORMS[mode] if request.first_line_format not in allowed_request_forms: + if request.is_http2 and mode is HTTPMode.transparent and request.first_line_format == "absolute": + return # dirty hack: h2 may have authority info. will be fixed properly with sans-io. if mode == HTTPMode.transparent: - err_message = textwrap.dedent(( + err_message = textwrap.dedent( """ Mitmproxy received an {} request even though it is not running in regular mode. This usually indicates a misconfiguration, please see the mitmproxy mode documentation for details. """ - )).strip().format("HTTP CONNECT" if request.first_line_format == "authority" else "absolute-form") + ).strip().format("HTTP CONNECT" if request.first_line_format == "authority" else "absolute-form") else: - err_message = "Invalid HTTP request form (expected: %s, got: %s)" % ( + err_message = "Invalid HTTP request form (expected: {}, got: {})".format( " or ".join(allowed_request_forms), request.first_line_format ) raise exceptions.HttpException(err_message) @@ -241,7 +255,7 @@ def _process_flow(self, f): try: try: - request = self.read_request_headers(f) + request: http.HTTPRequest = self.read_request_headers(f) except exceptions.HttpReadDisconnect: # don't throw an error for disconnects that happen # before/between requests. @@ -255,6 +269,7 @@ f.request.data.content = b"".join( self.read_request_body(f.request) ) + f.request.data.trailers = self.read_request_trailers(f.request) f.request.timestamp_end = time.time() self.channel.ask("http_connect", f) @@ -267,21 +282,26 @@ self.send_error_response(400, msg) return False - validate_request_form(self.mode, request) + if not self.config.options.relax_http_form_validation: + validate_request_form(self.mode, request) self.channel.ask("requestheaders", f) # Re-validate request form in case the user has changed something. - validate_request_form(self.mode, request) + if not self.config.options.relax_http_form_validation: + validate_request_form(self.mode, request) if request.headers.get("expect", "").lower() == "100-continue": # TODO: We may have to use send_response_headers for HTTP2 # here. - self.send_response(http.expect_continue_response) + self.send_response(http.make_expect_continue_response()) request.headers.pop("expect") if f.request.stream: f.request.data.content = None else: f.request.data.content = b"".join(self.read_request_body(request)) + + f.request.data.trailers = self.read_request_trailers(f.request) + request.timestamp_end = time.time() except exceptions.HttpException as e: # We optimistically guess there might be an HTTP client on the @@ -294,7 +314,7 @@ self.log( "request", "warn", - ["HTTP protocol error in client request: {}".format(e)] + [f"HTTP protocol error in client request: {e}"] ) return False @@ -303,11 +323,14 @@ # set first line format to relative in regular mode, # see https://github.com/mitmproxy/mitmproxy/issues/1759 if self.mode is HTTPMode.regular and request.first_line_format == "absolute": - request.first_line_format = "relative" + request.authority = "" # update host header in reverse proxy mode if self.config.options.mode.startswith("reverse:") and not self.config.options.keep_host_header: - f.request.host_header = self.config.upstream_server.address[0] + f.request.host_header = url.hostport( + self.config.upstream_server.scheme, + *self.config.upstream_server.address + ) # Determine .scheme, .host and .port attributes for inline scripts. For # absolute-form requests, they are directly given in the request. For @@ -317,15 +340,17 @@ if self.mode is HTTPMode.transparent: # Setting request.host also updates the host header, which we want # to preserve - host_header = f.request.host_header - f.request.host = self.__initial_server_address[0] - f.request.port = self.__initial_server_address[1] - f.request.host_header = host_header # set again as .host overwrites this. - f.request.scheme = "https" if self.__initial_server_tls else "http" + f.request.data.host = self.__initial_server_address[0] + f.request.data.port = self.__initial_server_address[1] + f.request.data.scheme = b"https" if self.__initial_server_tls else b"http" self.channel.ask("request", f) try: - if websockets.check_handshake(request.headers) and websockets.check_client_version(request.headers): + valid = ( + websocket.check_handshake(request.headers) and + websocket.check_client_version(request.headers) + ) + if valid: f.metadata['websocket'] = True # We only support RFC6455 with WebSocket version 13 # allow inline scripts to manipulate the client handshake @@ -340,6 +365,7 @@ def get_response(): self.send_request_headers(f.request) + if f.request.stream: chunks = self.read_request_body(f.request) if callable(f.request.stream): @@ -348,6 +374,8 @@ else: self.send_request_body(f.request, [f.request.data.content]) + self.send_request_trailers(f.request) + f.response = self.read_response_headers() try: @@ -406,6 +434,8 @@ # we now need to emulate the responseheaders hook. self.channel.ask("responseheaders", f) + f.response.data.trailers = self.read_response_trailers(f.request, f.response) + self.log("response", "debug", [repr(f.response)]) self.channel.ask("response", f) @@ -436,8 +466,8 @@ # received after e.g. a WebSocket upgrade request. # Check for WebSocket handshake is_websocket = ( - websockets.check_handshake(f.request.headers) and - websockets.check_handshake(f.response.headers) + websocket.check_handshake(f.request.headers) and + websocket.check_handshake(f.response.headers) ) if is_websocket and not self.config.options.websocket: self.log( @@ -473,7 +503,7 @@ response = http.make_error_response(code, message, headers) self.send_response(response) except (exceptions.NetlibException, h2.exceptions.H2Error, exceptions.Http2ProtocolException): - self.log("Failed to send error response to client: {}".format(message), "debug") + self.log(f"Failed to send error response to client: {message}", "debug") def change_upstream_proxy_server(self, address): # Make set_upstream_proxy_server always available, diff -Nru mitmproxy-5.1.1/mitmproxy/proxy/protocol/__init__.py mitmproxy-6.0.2/mitmproxy/proxy/protocol/__init__.py --- mitmproxy-5.1.1/mitmproxy/proxy/protocol/__init__.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/proxy/protocol/__init__.py 2020-12-15 16:41:27.000000000 +0000 @@ -15,7 +15,7 @@ - Http1Layer - HttpLayer - TLSLayer - - WebsocketLayer (or TCPLayer) + - WebSocketLayer (or TCPLayer) Every layer acts as a read-only context for its inner layers (see :py:class:`Layer`). To communicate with an outer layer, a layer can use diff -Nru mitmproxy-5.1.1/mitmproxy/proxy/protocol/rawtcp.py mitmproxy-6.0.2/mitmproxy/proxy/protocol/rawtcp.py --- mitmproxy-5.1.1/mitmproxy/proxy/protocol/rawtcp.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/proxy/protocol/rawtcp.py 2020-12-15 16:41:27.000000000 +0000 @@ -63,7 +63,7 @@ self.channel.ask("tcp_message", f) dst.sendall(tcp_message.content) - except (socket.error, exceptions.TcpException, SSL.Error) as e: + except (OSError, exceptions.TcpException, SSL.Error) as e: if not self.ignore: f.error = flow.Error("TCP connection closed unexpectedly: {}".format(repr(e))) self.channel.tell("tcp_error", f) diff -Nru mitmproxy-5.1.1/mitmproxy/proxy/protocol/tls.py mitmproxy-6.0.2/mitmproxy/proxy/protocol/tls.py --- mitmproxy-5.1.1/mitmproxy/proxy/protocol/tls.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/proxy/protocol/tls.py 2020-12-15 16:41:27.000000000 +0000 @@ -464,7 +464,7 @@ ) proto = self.alpn_for_client_connection.decode() if self.alpn_for_client_connection else '-' - self.log("ALPN selected by server: {}".format(proto), "debug") + self.log(f"ALPN selected by server: {proto}", "debug") def _find_cert(self): """ diff -Nru mitmproxy-5.1.1/mitmproxy/proxy/protocol/websocket.py mitmproxy-6.0.2/mitmproxy/proxy/protocol/websocket.py --- mitmproxy-5.1.1/mitmproxy/proxy/protocol/websocket.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/proxy/protocol/websocket.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,19 +1,15 @@ import queue -import socket from OpenSSL import SSL - import wsproto from wsproto import events, WSConnection from wsproto.connection import ConnectionType from wsproto.events import AcceptConnection, CloseConnection, Message, Ping, Request from wsproto.extensions import PerMessageDeflate -from mitmproxy import exceptions -from mitmproxy import flow +from mitmproxy import exceptions, flow from mitmproxy.proxy.protocol import base -from mitmproxy.net import tcp -from mitmproxy.net import websockets +from mitmproxy.net import tcp, websocket from mitmproxy.websocket import WebSocketFlow, WebSocketMessage from mitmproxy.utils import strutils @@ -79,6 +75,10 @@ assert isinstance(next(self.connections[self.server_conn].events()), events.AcceptConnection) def _handle_event(self, event, source_conn, other_conn, is_server): + self.log( + "WebSocket Event from {}: {}".format("server" if is_server else "client", event), + "debug" + ) if isinstance(event, events.Message): return self._handle_message(event, source_conn, other_conn, is_server) elif isinstance(event, events.Ping): @@ -199,8 +199,17 @@ other_conn = self.server_conn if conn == self.client_conn.connection else self.client_conn is_server = (source_conn == self.server_conn) - frame = websockets.Frame.from_file(source_conn.rfile) - data = self.connections[source_conn].receive_data(bytes(frame)) + header, frame, consumed_bytes = websocket.read_frame(source_conn.rfile) + self.log( + "WebSocket Frame from {}: {}, {}".format( + "server" if is_server else "client", + header, + frame, + ), + "debug" + ) + + data = self.connections[source_conn].receive_data(consumed_bytes) source_conn.send(data) if close_received: @@ -210,7 +219,7 @@ if not self._handle_event(event, source_conn, other_conn, is_server): if not close_received: close_received = True - except (socket.error, exceptions.TcpException, SSL.Error) as e: + except (OSError, exceptions.TcpException, SSL.Error) as e: s = 'server' if is_server else 'client' self.flow.error = flow.Error("WebSocket connection closed unexpectedly by {}: {}".format(s, repr(e))) self.channel.tell("websocket_error", self.flow) diff -Nru mitmproxy-5.1.1/mitmproxy/proxy/server.py mitmproxy-6.0.2/mitmproxy/proxy/server.py --- mitmproxy-5.1.1/mitmproxy/proxy/server.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/proxy/server.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,7 +1,7 @@ import sys import traceback -from mitmproxy import exceptions +from mitmproxy import exceptions, flow from mitmproxy import connections from mitmproxy import controller # noqa from mitmproxy import http @@ -120,7 +120,7 @@ root_layer = self.channel.ask("clientconnect", root_layer) root_layer() except exceptions.Kill: - self.log("Connection killed", "info") + self.log(flow.Error.KILLED_MESSAGE, "info") except exceptions.ProtocolException as e: if isinstance(e, exceptions.ClientHandshakeException): self.log( diff -Nru mitmproxy-5.1.1/mitmproxy/stateobject.py mitmproxy-6.0.2/mitmproxy/stateobject.py --- mitmproxy-5.1.1/mitmproxy/stateobject.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/stateobject.py 2020-12-15 16:41:27.000000000 +0000 @@ -46,7 +46,7 @@ else: setattr(self, attr, make_object(cls, val)) if state: - raise RuntimeWarning("Unexpected State in __setstate__: {}".format(state)) + raise RuntimeWarning(f"Unexpected State in __setstate__: {state}") def _process(typeinfo: typecheck.Type, val: typing.Any, make: bool) -> typing.Any: @@ -65,7 +65,7 @@ elif typename.startswith("typing.Tuple"): Ts = typecheck.tuple_types(typeinfo) if len(Ts) != len(val): - raise ValueError("Invalid data. Expected {}, got {}.".format(Ts, val)) + raise ValueError(f"Invalid data. Expected {Ts}, got {val}.") return tuple( _process(T, x, make) for T, x in zip(Ts, val) ) diff -Nru mitmproxy-5.1.1/mitmproxy/test/taddons.py mitmproxy-6.0.2/mitmproxy/test/taddons.py --- mitmproxy-5.1.1/mitmproxy/test/taddons.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/test/taddons.py 2020-12-15 16:41:27.000000000 +0000 @@ -28,7 +28,7 @@ def dump_log(self, outf=sys.stdout): for i in self.logs: - print("%s: %s" % (i.level, i.msg), file=outf) + print(f"{i.level}: {i.msg}", file=outf) def has_log(self, txt, level=None): for i in self.logs: diff -Nru mitmproxy-5.1.1/mitmproxy/test/tflow.py mitmproxy-6.0.2/mitmproxy/test/tflow.py --- mitmproxy-5.1.1/mitmproxy/test/tflow.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/test/tflow.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,15 +1,16 @@ import io import uuid -from mitmproxy.net import websockets from mitmproxy.test import tutils from mitmproxy import tcp from mitmproxy import websocket from mitmproxy import controller from mitmproxy import http -from mitmproxy import connections from mitmproxy import flow from mitmproxy.net import http as net_http +from mitmproxy.utils import compat + +from wsproto.frame_protocol import Opcode def ttcpflow(client_conn=True, server_conn=True, messages=True, err=None): @@ -40,25 +41,27 @@ server_conn = tserver_conn() if handshake_flow is True: req = http.HTTPRequest( - "relative", - "GET", - "http", "example.com", 80, - "/ws", - "HTTP/1.1", + b"GET", + b"http", + b"example.com", + b"/ws", + b"HTTP/1.1", headers=net_http.Headers( connection="upgrade", upgrade="websocket", sec_websocket_version="13", sec_websocket_key="1234", ), + content=b'', + trailers=None, timestamp_start=946681200, timestamp_end=946681201, - content=b'' + ) resp = http.HTTPResponse( - "HTTP/1.1", + b"HTTP/1.1", 101, reason=net_http.status_codes.RESPONSES.get(101), headers=net_http.Headers( @@ -66,9 +69,10 @@ upgrade='websocket', sec_websocket_accept=b'', ), + content=b'', + trailers=None, timestamp_start=946681202, timestamp_end=946681203, - content=b'', ) handshake_flow = http.HTTPFlow(client_conn, server_conn) handshake_flow.request = req @@ -81,9 +85,9 @@ if messages is True: messages = [ - websocket.WebSocketMessage(websockets.OPCODE.BINARY, True, b"hello binary"), - websocket.WebSocketMessage(websockets.OPCODE.TEXT, True, "hello text".encode()), - websocket.WebSocketMessage(websockets.OPCODE.TEXT, False, "it's me".encode()), + websocket.WebSocketMessage(Opcode.BINARY, True, b"hello binary"), + websocket.WebSocketMessage(Opcode.TEXT, True, b"hello text"), + websocket.WebSocketMessage(Opcode.TEXT, False, b"it's me"), ] if err is True: err = terr() @@ -114,11 +118,6 @@ if err is True: err = terr() - if req: - req = http.HTTPRequest.wrap(req) - if resp: - resp = http.HTTPResponse.wrap(resp) - f = http.HTTPFlow(client_conn, server_conn) f.request = req f.response = resp @@ -148,16 +147,12 @@ return f -def tclient_conn(): - """ - @return: mitmproxy.proxy.connection.ClientConnection - """ - c = connections.ClientConnection.from_state(dict( +def tclient_conn() -> compat.Client: + c = compat.Client.from_state(dict( id=str(uuid.uuid4()), address=("127.0.0.1", 22), - clientcert=None, mitmcert=None, - tls_established=False, + tls_established=True, timestamp_start=946681200, timestamp_tls_setup=946681201, timestamp_end=946681206, @@ -166,6 +161,13 @@ alpn_proto_negotiated=b"http/1.1", tls_version="TLSv1.2", tls_extensions=[(0x00, bytes.fromhex("000e00000b6578616d"))], + state=0, + sockname=("", 0), + error=None, + tls=False, + certificate_list=[], + alpn_offers=[], + cipher_list=[], )) c.reply = controller.DummyReply() c.rfile = io.BytesIO() @@ -173,25 +175,29 @@ return c -def tserver_conn(): - """ - @return: mitmproxy.proxy.connection.ServerConnection - """ - c = connections.ServerConnection.from_state(dict( +def tserver_conn() -> compat.Server: + c = compat.Server.from_state(dict( id=str(uuid.uuid4()), address=("address", 22), source_address=("address", 22), ip_address=("192.168.0.1", 22), - cert=None, timestamp_start=946681202, timestamp_tcp_setup=946681203, timestamp_tls_setup=946681204, timestamp_end=946681205, - tls_established=False, + tls_established=True, sni="address", alpn_proto_negotiated=None, tls_version="TLSv1.2", via=None, + state=0, + error=None, + tls=False, + certificate_list=[], + alpn_offers=[], + cipher_name=None, + cipher_list=[], + via2=None, )) c.reply = controller.DummyReply() c.rfile = io.BytesIO() diff -Nru mitmproxy-5.1.1/mitmproxy/test/tutils.py mitmproxy-6.0.2/mitmproxy/test/tutils.py --- mitmproxy-5.1.1/mitmproxy/test/tutils.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/test/tutils.py 2020-12-15 16:41:27.000000000 +0000 @@ -12,21 +12,22 @@ return tcp.Reader(fp) -def treq(**kwargs): +def treq(**kwargs) -> http.Request: """ Returns: mitmproxy.net.http.Request """ default = dict( - first_line_format="relative", + host="address", + port=22, method=b"GET", scheme=b"http", - host=b"address", - port=22, + authority=b"", path=b"/path", http_version=b"HTTP/1.1", headers=http.Headers(((b"header", b"qvalue"), (b"content-length", b"7"))), content=b"content", + trailers=None, timestamp_start=946681200, timestamp_end=946681201, ) @@ -34,7 +35,7 @@ return http.Request(**default) -def tresp(**kwargs): +def tresp(**kwargs) -> http.Response: """ Returns: mitmproxy.net.http.Response @@ -45,6 +46,7 @@ reason=b"OK", headers=http.Headers(((b"header-response", b"svalue"), (b"content-length", b"7"))), content=b"message", + trailers=None, timestamp_start=946681202, timestamp_end=946681203, ) diff -Nru mitmproxy-5.1.1/mitmproxy/tools/cmdline.py mitmproxy-6.0.2/mitmproxy/tools/cmdline.py --- mitmproxy-5.1.1/mitmproxy/tools/cmdline.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/tools/cmdline.py 2020-12-15 16:41:27.000000000 +0000 @@ -67,6 +67,7 @@ # Proxy SSL options group = parser.add_argument_group("SSL") opts.make_parser(group, "certs", metavar="SPEC") + opts.make_parser(group, "cert_passphrase", metavar="PASS") opts.make_parser(group, "ssl_insecure", short="k") opts.make_parser(group, "key_size", metavar="KEY_SIZE") @@ -81,13 +82,21 @@ opts.make_parser(group, "server_replay_nopop") opts.make_parser(group, "server_replay_refresh") - # Replacements - group = parser.add_argument_group("Replacements") - opts.make_parser(group, "replacements", metavar="PATTERN", short="R") - - # Set headers - group = parser.add_argument_group("Set Headers") - opts.make_parser(group, "setheaders", metavar="PATTERN", short="H") + # Map Remote + group = parser.add_argument_group("Map Remote") + opts.make_parser(group, "map_remote", metavar="PATTERN", short="M") + + # Map Local + group = parser.add_argument_group("Map Local") + opts.make_parser(group, "map_local", metavar="PATTERN") + + # Modify Body + group = parser.add_argument_group("Modify Body") + opts.make_parser(group, "modify_body", metavar="PATTERN", short="B") + + # Modify headers + group = parser.add_argument_group("Modify Headers") + opts.make_parser(group, "modify_headers", metavar="PATTERN", short="H") def mitmproxy(opts): diff -Nru mitmproxy-5.1.1/mitmproxy/tools/console/commander/commander.py mitmproxy-6.0.2/mitmproxy/tools/console/commander/commander.py --- mitmproxy-5.1.1/mitmproxy/tools/console/commander/commander.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/tools/console/commander/commander.py 2020-12-15 16:41:27.000000000 +0000 @@ -28,15 +28,17 @@ if o.startswith(start): self.options.append(o) self.options.sort() - self.offset = 0 + self.pos = -1 def cycle(self, forward: bool = True) -> str: if not self.options: return self.start - ret = self.options[self.offset] - delta = 1 if forward else -1 - self.offset = (self.offset + delta) % len(self.options) - return ret + if self.pos == -1: + self.pos = 0 if forward else len(self.options) - 1 + else: + delta = 1 if forward else -1 + self.pos = (self.pos + delta) % len(self.options) + return self.options[self.pos] class CompletionState(typing.NamedTuple): diff -Nru mitmproxy-5.1.1/mitmproxy/tools/console/common.py mitmproxy-6.0.2/mitmproxy/tools/console/common.py --- mitmproxy-5.1.1/mitmproxy/tools/console/common.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/tools/console/common.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,7 +1,6 @@ +import enum import platform import typing -import datetime -import time import math from functools import lru_cache from publicsuffix2 import get_sld, get_tld @@ -9,7 +8,10 @@ import urwid import urwid.util +from mitmproxy import flow +from mitmproxy.http import HTTPFlow from mitmproxy.utils import human +from mitmproxy.tcp import TCPFlow # Detect Windows Subsystem for Linux IS_WSL = "Microsoft" in platform.platform() @@ -82,7 +84,7 @@ return ret -def fcol(s, attr): +def fcol(s: str, attr: str) -> typing.Tuple[str, int, urwid.Text]: s = str(s) return ( "fixed", @@ -96,29 +98,61 @@ if urwid.util.detected_encoding: - SYMBOL_REPLAY = u"\u21ba" - SYMBOL_RETURN = u"\u2190" - SYMBOL_MARK = u"\u25cf" - SYMBOL_UP = u"\u21E7" - SYMBOL_DOWN = u"\u21E9" - SYMBOL_ELLIPSIS = u"\u2026" + SYMBOL_REPLAY = "\u21ba" + SYMBOL_RETURN = "\u2190" + SYMBOL_MARK = "\u25cf" + SYMBOL_UP = "\u21E7" + SYMBOL_DOWN = "\u21E9" + SYMBOL_ELLIPSIS = "\u2026" + SYMBOL_FROM_CLIENT = "\u21d2" + SYMBOL_TO_CLIENT = "\u21d0" else: - SYMBOL_REPLAY = u"[r]" - SYMBOL_RETURN = u"<-" - SYMBOL_MARK = "[m]" + SYMBOL_REPLAY = "[r]" + SYMBOL_RETURN = "<-" + SYMBOL_MARK = "#" SYMBOL_UP = "^" SYMBOL_DOWN = " " SYMBOL_ELLIPSIS = "~" + SYMBOL_FROM_CLIENT = "->" + SYMBOL_TO_CLIENT = "<-" +SCHEME_STYLES = { + 'http': 'scheme_http', + 'https': 'scheme_https', + 'tcp': 'scheme_tcp', +} +HTTP_REQUEST_METHOD_STYLES = { + 'GET': 'method_get', + 'POST': 'method_post', + 'DELETE': 'method_delete', + 'HEAD': 'method_head', + 'PUT': 'method_put' +} +HTTP_RESPONSE_CODE_STYLE = { + 2: "code_200", + 3: "code_300", + 4: "code_400", + 5: "code_500", +} + + +class RenderMode(enum.Enum): + TABLE = 1 + """The flow list in table format, i.e. one row per flow.""" + LIST = 2 + """The flow list in list format, i.e. potentially multiple rows per flow.""" + DETAILVIEW = 3 + """The top lines in the detail view.""" -def fixlen(s, maxlen): + +def fixlen(s: str, maxlen: int) -> str: if len(s) <= maxlen: return s.ljust(maxlen) else: return s[0:maxlen - len(SYMBOL_ELLIPSIS)] + SYMBOL_ELLIPSIS -def fixlen_r(s, maxlen): +def fixlen_r(s: str, maxlen: int) -> str: if len(s) <= maxlen: return s.rjust(maxlen) else: @@ -130,7 +164,7 @@ self.text = text self.attr = attr self.align = align - super(TruncatedText, self).__init__() + super().__init__() def pack(self, size, focus=False): return (len(self.text), 1) @@ -233,8 +267,8 @@ for i in range(len(s)): c = s[i] if ((i < i_query and c == '/') or - (i < i_query and i > i_last_slash and c == '.') or - (i == i_query)): + (i < i_query and i > i_last_slash and c == '.') or + (i == i_query)): a = 'url_punctuation' elif i > i_query: if in_val: @@ -268,294 +302,428 @@ 'https:': 'scheme_https', } return [ - (schemes.get(parts[0], "scheme_other"), len(parts[0]) - 1), - ('url_punctuation', 3), # :// - ] + colorize_host(parts[2]) + colorize_req('/' + parts[3]) + (schemes.get(parts[0], "scheme_other"), len(parts[0]) - 1), + ('url_punctuation', 3), # :// + ] + colorize_host(parts[2]) + colorize_req('/' + parts[3]) + + +def format_http_content_type(content_type: str) -> typing.Tuple[str, str]: + content_type = content_type.split(";")[0] + if content_type.endswith('/javascript'): + style = 'content_script' + elif content_type.startswith('text/'): + style = 'content_text' + elif (content_type.startswith('image/') or + content_type.startswith('video/') or + content_type.startswith('font/') or + "/x-font-" in content_type): + style = 'content_media' + elif content_type.endswith('/json') or content_type.endswith('/xml'): + style = 'content_data' + elif content_type.startswith('application/'): + style = 'content_raw' + else: + style = 'content_other' + return content_type, style + + +def format_duration(duration: float) -> typing.Tuple[str, str]: + pretty_duration = human.pretty_duration(duration) + style = 'gradient_%02d' % int(99 - 100 * min(math.log2(1 + 1000 * duration) / 12, 0.99)) + return pretty_duration, style + + +def format_size(num_bytes: int) -> typing.Tuple[str, str]: + pretty_size = human.pretty_size(num_bytes) + style = 'gradient_%02d' % int(99 - 100 * min(math.log2(1 + num_bytes) / 20, 0.99)) + return pretty_size, style + + +def format_left_indicators( + *, + focused: bool, + intercepted: bool, + timestamp: float +): + indicators: typing.List[typing.Union[str, typing.Tuple[str, str]]] = [] + if focused: + indicators.append(("focus", ">>")) + else: + indicators.append(" ") + pretty_timestamp = human.format_timestamp(timestamp)[-8:] + if intercepted: + indicators.append(("intercept", pretty_timestamp)) + else: + indicators.append(("text", pretty_timestamp)) + return "fixed", 10, urwid.Text(indicators) + + +def format_right_indicators( + *, + replay: bool, + marked: bool +): + indicators: typing.List[typing.Union[str, typing.Tuple[str, str]]] = [] + if replay: + indicators.append(("replay", SYMBOL_REPLAY)) + else: + indicators.append(" ") + if marked: + indicators.append(("mark", SYMBOL_MARK)) + else: + indicators.append(" ") + return "fixed", 2, urwid.Text(indicators) @lru_cache(maxsize=800) -def raw_format_list(f): - f = dict(f) - pile = [] +def format_http_flow_list( + *, + render_mode: RenderMode, + focused: bool, + marked: bool, + is_replay: bool, + request_method: str, + request_scheme: str, + request_host: str, + request_path: str, + request_url: str, + request_http_version: str, + request_timestamp: float, + request_is_push_promise: bool, + intercepted: bool, + response_code: typing.Optional[int], + response_reason: typing.Optional[str], + response_content_length: typing.Optional[int], + response_content_type: typing.Optional[str], + duration: typing.Optional[float], + error_message: typing.Optional[str], +) -> urwid.Widget: req = [] - if f["extended"]: + + if render_mode is RenderMode.DETAILVIEW: + req.append(fcol(human.format_timestamp(request_timestamp), "highlight")) + else: + if focused: + req.append(fcol(">>", "focus")) + else: + req.append(fcol(" ", "focus")) + + method_style = HTTP_REQUEST_METHOD_STYLES.get(request_method, "method_other") + req.append(fcol(request_method, method_style)) + + if request_is_push_promise: + req.append(fcol('PUSH_PROMISE', 'method_http2_push')) + + preamble_len = sum(x[1] for x in req) + len(req) - 1 + + if request_http_version not in ("HTTP/1.0", "HTTP/1.1"): + request_url += " " + request_http_version + if intercepted and not response_code: + url_style = "intercept" + elif response_code or error_message: + url_style = "text" + else: + url_style = "title" + + if render_mode is RenderMode.DETAILVIEW: req.append( - fcol( - human.format_timestamp(f["req_timestamp"]), - "highlight" - ) + urwid.Text([(url_style, request_url)]) ) else: - req.append(fcol(">>" if f["focus"] else " ", "focus")) + req.append(truncated_plain(request_url, url_style)) - if f["marked"]: - req.append(fcol(SYMBOL_MARK, "mark")) + req.append(format_right_indicators(replay=is_replay, marked=marked)) - if f["req_is_replay"]: - req.append(fcol(SYMBOL_REPLAY, "replay")) + resp = [ + ("fixed", preamble_len, urwid.Text("")) + ] + if response_code: + if intercepted: + style = "intercept" + else: + style = "" + + status_style = style or HTTP_RESPONSE_CODE_STYLE.get(response_code // 100, "code_other") + resp.append(fcol(SYMBOL_RETURN, status_style)) + resp.append(fcol(str(response_code), status_style)) + if response_reason and render_mode is RenderMode.DETAILVIEW: + resp.append(fcol(response_reason, status_style)) + + if response_content_type: + ct, ct_style = format_http_content_type(response_content_type) + resp.append(fcol(ct, style or ct_style)) + + if response_content_length: + size, size_style = format_size(response_content_length) + elif response_content_length == 0: + size = "[no content]" + size_style = "text" + else: + size = "[content missing]" + size_style = "text" + resp.append(fcol(size, style or size_style)) + + if duration: + dur, dur_style = format_duration(duration) + resp.append(fcol(dur, style or dur_style)) + elif error_message: + resp.append(fcol(SYMBOL_RETURN, "error")) + resp.append(urwid.Text([("error", error_message)])) - req.append(fcol(f["req_method"], "method")) + return urwid.Pile([ + urwid.Columns(req, dividechars=1), + urwid.Columns(resp, dividechars=1) + ]) - preamble = sum(i[1] for i in req) + len(req) - 1 - if f["intercepted"] and not f["acked"]: - uc = "intercept" - elif "resp_code" in f or "err_msg" in f: - uc = "text" +@lru_cache(maxsize=800) +def format_http_flow_table( + *, + render_mode: RenderMode, + focused: bool, + marked: bool, + is_replay: typing.Optional[str], + request_method: str, + request_scheme: str, + request_host: str, + request_path: str, + request_url: str, + request_http_version: str, + request_timestamp: float, + request_is_push_promise: bool, + intercepted: bool, + response_code: typing.Optional[int], + response_reason: typing.Optional[str], + response_content_length: typing.Optional[int], + response_content_type: typing.Optional[str], + duration: typing.Optional[float], + error_message: typing.Optional[str], +) -> urwid.Widget: + items = [ + format_left_indicators( + focused=focused, + intercepted=intercepted, + timestamp=request_timestamp + ) + ] + + if intercepted and not response_code: + request_style = "intercept" else: - uc = "title" + request_style = "" - url = f["req_url"] + scheme_style = request_style or SCHEME_STYLES.get(request_scheme, "scheme_other") + items.append(fcol(fixlen(request_scheme.upper(), 5), scheme_style)) - if f["cols"] and len(url) > f["cols"]: - url = url[:f["cols"]] + "…" + if request_is_push_promise: + method_style = 'method_http2_push' + else: + method_style = request_style or HTTP_REQUEST_METHOD_STYLES.get(request_method, "method_other") + items.append(fcol(fixlen(request_method, 4), method_style)) - if f["req_http_version"] not in ("HTTP/1.0", "HTTP/1.1"): - url += " " + f["req_http_version"] - req.append( - urwid.Text([(uc, url)]) - ) + items.append(('weight', 0.25, TruncatedText(request_host, colorize_host(request_host), 'right'))) + items.append(('weight', 1.0, TruncatedText(request_path, colorize_req(request_path), 'left'))) - pile.append(urwid.Columns(req, dividechars=1)) + if intercepted and response_code: + response_style = "intercept" + else: + response_style = "" - resp = [] - resp.append( - ("fixed", preamble, urwid.Text("")) - ) + if response_code: - if "resp_code" in f: - codes = { - 2: "code_200", - 3: "code_300", - 4: "code_400", - 5: "code_500", - } - ccol = codes.get(f["resp_code"] // 100, "code_other") - resp.append(fcol(SYMBOL_RETURN, ccol)) - if f["resp_is_replay"]: - resp.append(fcol(SYMBOL_REPLAY, "replay")) - resp.append(fcol(f["resp_code"], ccol)) - if f["extended"]: - resp.append(fcol(f["resp_reason"], ccol)) - if f["intercepted"] and f["resp_code"] and not f["acked"]: - rc = "intercept" - else: - rc = "text" - - if f["resp_ctype"]: - resp.append(fcol(f["resp_ctype"], rc)) - resp.append(fcol(f["resp_clen"], rc)) - pretty_duration = human.pretty_duration(f["duration"]) - resp.append(fcol(pretty_duration, rc)) + status = str(response_code) + status_style = response_style or HTTP_RESPONSE_CODE_STYLE.get(response_code // 100, "code_other") - elif f["err_msg"]: - resp.append(fcol(SYMBOL_RETURN, "error")) - resp.append( - urwid.Text([ - ( - "error", - f["err_msg"] - ) - ]) - ) - pile.append(urwid.Columns(resp, dividechars=1)) - return urwid.Pile(pile) + if response_content_length and response_content_type: + content, content_style = format_http_content_type(response_content_type) + content_style = response_style or content_style + elif response_content_length: + content = '' + content_style = 'content_none' + elif response_content_length == 0: + content = "[no content]" + content_style = 'content_none' + else: + content = "[content missing]" + content_style = 'content_none' + + elif error_message: + status = 'err' + status_style = 'error' + content = error_message + content_style = 'error' + + else: + status = '' + status_style = 'text' + content = '' + content_style = '' + + items.append(fcol(fixlen(status, 3), status_style)) + items.append(('weight', 0.15, truncated_plain(content, content_style, 'right'))) + + if response_content_length: + size, size_style = format_size(response_content_length) + items.append(fcol(fixlen_r(size, 5), response_style or size_style)) + else: + items.append(("fixed", 5, urwid.Text(""))) + + if duration: + duration_pretty, duration_style = format_duration(duration) + items.append(fcol(fixlen_r(duration_pretty, 5), response_style or duration_style)) + else: + items.append(("fixed", 5, urwid.Text(""))) + + items.append(format_right_indicators( + replay=bool(is_replay), + marked=marked + )) + return urwid.Columns(items, dividechars=1, min_width=15) @lru_cache(maxsize=800) -def raw_format_table(f): - f = dict(f) - pile = [] - req = [] +def format_tcp_flow( + *, + render_mode: RenderMode, + focused: bool, + timestamp_start: float, + marked: bool, + client_address, + server_address, + total_size: int, + duration: typing.Optional[float], + error_message: typing.Optional[str], +): + conn = f"{human.format_address(client_address)} <-> {human.format_address(server_address)}" + + items = [] + + if render_mode in (RenderMode.TABLE, RenderMode.DETAILVIEW): + items.append( + format_left_indicators(focused=focused, intercepted=False, timestamp=timestamp_start) + ) + else: + if focused: + items.append(fcol(">>", "focus")) + else: + items.append(fcol(" ", "focus")) - cursor = [' ', 'focus'] - if f['focus']: - cursor[0] = '>' - req.append(fcol(*cursor)) - - if f.get('resp_is_replay', False) or f.get('req_is_replay', False): - req.append(fcol(SYMBOL_REPLAY, 'replay')) - if f['marked']: - req.append(fcol(SYMBOL_MARK, 'mark')) - - if f["two_line"]: - req.append(TruncatedText(f["req_url"], colorize_url(f["req_url"]), 'left')) - pile.append(urwid.Columns(req, dividechars=1)) - - req = [] - req.append(fcol(' ', 'text')) - - if f["intercepted"] and not f["acked"]: - uc = "intercept" - elif "resp_code" in f or f["err_msg"] is not None: - uc = "highlight" - else: - uc = "title" - - if f["extended"]: - s = human.format_timestamp(f["req_timestamp"]) - else: - s = datetime.datetime.fromtimestamp(time.mktime(time.localtime(f["req_timestamp"]))).strftime("%H:%M:%S") - req.append(fcol(s, uc)) - - methods = { - 'GET': 'method_get', - 'POST': 'method_post', - 'DELETE': 'method_delete', - 'HEAD': 'method_head', - 'PUT': 'method_put' - } - uc = methods.get(f["req_method"], "method_other") - if f['extended']: - req.append(fcol(f["req_method"], uc)) - if f["req_promise"]: - req.append(fcol('PUSH_PROMISE', 'method_http2_push')) - else: - if f["req_promise"]: - uc = 'method_http2_push' - req.append(("fixed", 4, truncated_plain(f["req_method"], uc))) - - if f["two_line"]: - req.append(fcol(f["req_http_version"], 'text')) - else: - schemes = { - 'http': 'scheme_http', - 'https': 'scheme_https', - } - req.append(fcol(fixlen(f["req_scheme"].upper(), 5), schemes.get(f["req_scheme"], "scheme_other"))) - - req.append(('weight', 0.25, TruncatedText(f["req_host"], colorize_host(f["req_host"]), 'right'))) - req.append(('weight', 1.0, TruncatedText(f["req_path"], colorize_req(f["req_path"]), 'left'))) - - ret = (' ' * len(SYMBOL_RETURN), 'text') - status = ('', 'text') - content = ('', 'text') - size = ('', 'text') - duration = ('', 'text') - - if "resp_code" in f: - codes = { - 2: "code_200", - 3: "code_300", - 4: "code_400", - 5: "code_500", - } - ccol = codes.get(f["resp_code"] // 100, "code_other") - ret = (SYMBOL_RETURN, ccol) - status = (str(f["resp_code"]), ccol) - - if f["resp_len"] < 0: - if f["intercepted"] and f["resp_code"] and not f["acked"]: - rc = "intercept" - else: - rc = "content_none" + if render_mode is RenderMode.TABLE: + items.append(fcol("TCP ", SCHEME_STYLES["tcp"])) + else: + items.append(fcol("TCP", SCHEME_STYLES["tcp"])) + + items.append(('weight', 1.0, truncated_plain(conn, "text", 'left'))) + if error_message: + items.append(('weight', 1.0, truncated_plain(error_message, "error", 'left'))) + + if total_size: + size, size_style = format_size(total_size) + items.append(fcol(fixlen_r(size, 5), size_style)) + else: + items.append(("fixed", 5, urwid.Text(""))) + + if duration: + duration_pretty, duration_style = format_duration(duration) + items.append(fcol(fixlen_r(duration_pretty, 5), duration_style)) + else: + items.append(("fixed", 5, urwid.Text(""))) - if f["resp_len"] == -1: - contentdesc = "[content missing]" + items.append(format_right_indicators(replay=False, marked=marked)) + + return urwid.Pile([ + urwid.Columns(items, dividechars=1, min_width=15) + ]) + + +def format_flow( + f: flow.Flow, + *, + render_mode: RenderMode, + hostheader: bool = False, # pass options directly if we need more stuff from them + focused: bool = True, +) -> urwid.Widget: + """ + This functions calls the proper renderer depending on the flow type. + We also want to cache the renderer output, so we extract all attributes + relevant for display and call the render with only that. This assures that rows + are updated if the flow is changed. + """ + duration: typing.Optional[float] + error_message: typing.Optional[str] + if f.error: + error_message = f.error.msg + else: + error_message = None + + if isinstance(f, TCPFlow): + total_size = 0 + for message in f.messages: + total_size += len(message.content) + if f.messages: + duration = f.messages[-1].timestamp - f.timestamp_start + else: + duration = None + return format_tcp_flow( + render_mode=render_mode, + focused=focused, + timestamp_start=f.timestamp_start, + marked=f.marked, + client_address=f.client_conn.address, + server_address=f.server_conn.address, + total_size=total_size, + duration=duration, + error_message=error_message, + ) + elif isinstance(f, HTTPFlow): + intercepted = ( + f.intercepted and not (f.reply and f.reply.state == "committed") + ) + response_content_length: typing.Optional[int] + if f.response: + if f.response.raw_content is not None: + response_content_length = len(f.response.raw_content) + else: + response_content_length = None + response_code: typing.Optional[int] = f.response.status_code + response_reason: typing.Optional[str] = f.response.reason + response_content_type = f.response.headers.get("content-type") + if f.response.timestamp_end: + duration = max([f.response.timestamp_end - f.request.timestamp_start, 0]) else: - contentdesc = "[no content]" - content = (contentdesc, rc) + duration = None else: - if f["resp_ctype"]: - ctype = f["resp_ctype"].split(";")[0] - if ctype.endswith('/javascript'): - rc = 'content_script' - elif ctype.startswith('text/'): - rc = 'content_text' - elif (ctype.startswith('image/') or - ctype.startswith('video/') or - ctype.startswith('font/') or - "/x-font-" in ctype): - rc = 'content_media' - elif ctype.endswith('/json') or ctype.endswith('/xml'): - rc = 'content_data' - elif ctype.startswith('application/'): - rc = 'content_raw' - else: - rc = 'content_other' - content = (ctype, rc) - - rc = 'gradient_%02d' % int(99 - 100 * min(math.log2(1 + f["resp_len"]) / 20, 0.99)) + response_content_length = None + response_code = None + response_reason = None + response_content_type = None + duration = None - size_str = human.pretty_size(f["resp_len"]) - if not f['extended']: - # shorten to 5 chars max - if len(size_str) > 5: - size_str = size_str[0:4].rstrip('.') + size_str[-1:] - size = (size_str, rc) - - if f['duration'] is not None: - rc = 'gradient_%02d' % int(99 - 100 * min(math.log2(1 + 1000 * f['duration']) / 12, 0.99)) - duration = (human.pretty_duration(f['duration']), rc) - - elif f["err_msg"]: - status = ('Err', 'error') - content = f["err_msg"], 'error' - - if f["two_line"]: - req.append(fcol(*ret)) - req.append(fcol(fixlen(status[0], 3), status[1])) - req.append(('weight', 0.15, truncated_plain(content[0], content[1], 'right'))) - if f['extended']: - req.append(fcol(*size)) - else: - req.append(fcol(fixlen_r(size[0], 5), size[1])) - req.append(fcol(fixlen_r(duration[0], 5), duration[1])) - - pile.append(urwid.Columns(req, dividechars=1, min_width=15)) - - return urwid.Pile(pile) - - -def format_flow(f, focus, extended=False, hostheader=False, cols=False, layout='default'): - acked = False - if f.reply and f.reply.state == "committed": - acked = True - d = dict( - focus=focus, - extended=extended, - two_line=extended or cols < 100, - cols=cols, - intercepted=f.intercepted, - acked=acked, - req_timestamp=f.request.timestamp_start, - req_is_replay=f.request.is_replay, - req_method=f.request.method, - req_promise='h2-pushed-stream' in f.metadata, - req_url=f.request.pretty_url if hostheader else f.request.url, - req_scheme=f.request.scheme, - req_host=f.request.pretty_host if hostheader else f.request.host, - req_path=f.request.path, - req_http_version=f.request.http_version, - err_msg=f.error.msg if f.error else None, - marked=f.marked, - ) - if f.response: - if f.response.raw_content: - content_len = len(f.response.raw_content) - contentdesc = human.pretty_size(len(f.response.raw_content)) - elif f.response.raw_content is None: - content_len = -1 - contentdesc = "[content missing]" - else: - content_len = -2 - contentdesc = "[no content]" - - duration = None - if f.response.timestamp_end and f.request.timestamp_start: - duration = max([f.response.timestamp_end - f.request.timestamp_start, 0]) - - d.update(dict( - resp_code=f.response.status_code, - resp_reason=f.response.reason, - resp_is_replay=f.response.is_replay, - resp_len=content_len, - resp_ctype=f.response.headers.get("content-type"), - resp_clen=contentdesc, + if render_mode in (RenderMode.LIST, RenderMode.DETAILVIEW): + render_func = format_http_flow_list + else: + render_func = format_http_flow_table + return render_func( + render_mode=render_mode, + focused=focused, + marked=f.marked, + is_replay=f.is_replay, + request_method=f.request.method, + request_scheme=f.request.scheme, + request_host=f.request.pretty_host if hostheader else f.request.host, + request_path=f.request.path, + request_url=f.request.pretty_url if hostheader else f.request.url, + request_http_version=f.request.http_version, + request_timestamp=f.request.timestamp_start, + request_is_push_promise='h2-pushed-stream' in f.metadata, + intercepted=intercepted, + response_code=response_code, + response_reason=response_reason, + response_content_length=response_content_length, + response_content_type=response_content_type, duration=duration, - )) + error_message=error_message, + ) - if ((layout == 'default' and cols < 100) or layout == "list"): - return raw_format_list(tuple(sorted(d.items()))) else: - return raw_format_table(tuple(sorted(d.items()))) + raise NotImplementedError() diff -Nru mitmproxy-5.1.1/mitmproxy/tools/console/consoleaddons.py mitmproxy-6.0.2/mitmproxy/tools/console/consoleaddons.py --- mitmproxy-5.1.1/mitmproxy/tools/console/consoleaddons.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/tools/console/consoleaddons.py 2020-12-15 16:41:27.000000000 +0000 @@ -9,6 +9,7 @@ from mitmproxy import flow from mitmproxy import http from mitmproxy import log +from mitmproxy import tcp from mitmproxy.tools.console import keymap from mitmproxy.tools.console import overlay from mitmproxy.tools.console import signals @@ -112,20 +113,23 @@ choices=sorted(console_palettes), ) loader.add_option( - "console_palette_transparent", bool, False, + "console_palette_transparent", bool, True, "Set transparent background for palette." ) loader.add_option( "console_mouse", bool, True, "Console mouse interaction." ) - loader.add_option( "console_flowlist_layout", str, "default", "Set the flowlist layout", choices=sorted(console_flowlist_layout) ) + loader.add_option( + "console_strip_trailing_newlines", bool, False, + "Strip trailing newlines from edited request/response bodies." + ) @command.command("console.layout.options") def layout_options(self) -> typing.Sequence[str]: @@ -291,6 +295,8 @@ Prompt the user to edit a command with a (possibly empty) starting value. """ quoted = " ".join(command_lexer.quote(x) for x in command_str) + if quoted: + quoted += " " signals.status_prompt_command.send(partial=quoted) @command.command("console.command.set") @@ -334,9 +340,10 @@ @command.command("console.view.flow") def view_flow(self, flow: flow.Flow) -> None: """View a flow.""" - if hasattr(flow, "request"): - # FIME: Also set focus? + if isinstance(flow, (http.HTTPFlow, tcp.TCPFlow)): self.master.switch_view("flowview") + else: + ctx.log.warn(f"No detail view for {type(flow).__name__}.") @command.command("console.exit") def exit(self) -> None: @@ -381,22 +388,30 @@ """ Possible components for console.edit.focus. """ - return [ - "cookies", - "urlencoded form", - "multipart form", - "path", - "method", - "query", - "reason", - "request-headers", - "response-headers", - "request-body", - "response-body", - "status_code", - "set-cookies", - "url", - ] + flow = self.master.view.focus.flow + focus_options = [] + + if type(flow) == tcp.TCPFlow: + focus_options = ["tcp-message"] + elif type(flow) == http.HTTPFlow: + focus_options = [ + "cookies", + "urlencoded form", + "multipart form", + "path", + "method", + "query", + "reason", + "request-headers", + "response-headers", + "request-body", + "response-body", + "status_code", + "set-cookies", + "url", + ] + + return focus_options @command.command("console.edit.focus") @command.argument("flow_part", type=mitmproxy.types.Choice("console.edit.focus.options")) @@ -437,13 +452,14 @@ else: message = flow.response c = self.master.spawn_editor(message.get_content(strict=False) or b"") - # Fix an issue caused by some editors when editing a - # request/response body. Many editors make it hard to save a - # file without a terminating newline on the last line. When - # editing message bodies, this can cause problems. For now, I - # just strip the newlines off the end of the body when we return - # from an editor. - message.content = c.rstrip(b"\n") + # Many editors make it hard to save a file without a terminating + # newline on the last line. When editing message bodies, this can + # cause problems. We strip trailing newlines by default, but this + # behavior is configurable. + if self.master.options.console_strip_trailing_newlines: + message.content = c.rstrip(b"\n") + else: + message.content = c elif flow_part == "set-cookies": self.master.switch_view("edit_focus_setcookies") elif flow_part == "url": @@ -456,6 +472,10 @@ "console.command", ["flow.set", "@focus", flow_part] ) + elif flow_part == "tcp-message": + message = flow.messages[-1] + c = self.master.spawn_editor(message.content or b"") + message.content = c.rstrip(b"\n") def _grideditor(self): gewidget = self.master.window.current("grideditor") @@ -513,7 +533,7 @@ [strutils.always_str(x) or "" for x in row] # type: ignore ) ctx.log.alert("Saved %s rows as CSV." % (len(rows))) - except IOError as e: + except OSError as e: ctx.log.error(str(e)) @command.command("console.grideditor.editor") @@ -540,7 +560,7 @@ try: self.master.commands.call_strings( "view.settings.setval", - ["@focus", "flowview_mode_%s" % (idx,), mode] + ["@focus", f"flowview_mode_{idx}", mode] ) except exceptions.CommandError as e: ctx.log.error(str(e)) @@ -564,7 +584,7 @@ return self.master.commands.call_strings( "view.settings.getval", - ["@focus", "flowview_mode_%s" % (idx,), self.master.options.console_default_contentview] + ["@focus", f"flowview_mode_{idx}", self.master.options.console_default_contentview] ) @command.command("console.key.contexts") diff -Nru mitmproxy-5.1.1/mitmproxy/tools/console/defaultkeys.py mitmproxy-6.0.2/mitmproxy/tools/console/defaultkeys.py --- mitmproxy-5.1.1/mitmproxy/tools/console/defaultkeys.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/tools/console/defaultkeys.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,4 +1,3 @@ - def map(km): km.add(":", "console.command ", ["commonkey", "global"], "Command prompt") km.add("?", "console.view.help", ["global"], "View help") @@ -26,7 +25,7 @@ km.add("ctrl f", "console.nav.pagedown", ["global"], "Page down") km.add("ctrl b", "console.nav.pageup", ["global"], "Page up") - km.add("I", "set intercept_active toggle", ["global"], "Toggle intercept") + km.add("I", "set intercept_active toggle", ["global"], "Toggle whether the filtering via the intercept option is enabled") km.add("i", "console.command.set intercept", ["global"], "Set intercept") km.add("W", "console.command.set save_stream_file", ["global"], "Stream to file") km.add("A", "flow.resume @all", ["flowlist", "flowview"], "Resume all intercepted flows") diff -Nru mitmproxy-5.1.1/mitmproxy/tools/console/eventlog.py mitmproxy-6.0.2/mitmproxy/tools/console/eventlog.py --- mitmproxy-5.1.1/mitmproxy/tools/console/eventlog.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/tools/console/eventlog.py 2020-12-15 16:41:27.000000000 +0000 @@ -46,7 +46,7 @@ def add_event(self, event_store, entry: log.LogEntry): if log.log_tier(self.master.options.console_eventlog_verbosity) < log.log_tier(entry.level): return - txt = "%s: %s" % (entry.level, str(entry.msg)) + txt = "{}: {}".format(entry.level, str(entry.msg)) if entry.level in ("error", "warn", "alert"): e = urwid.Text((entry.level, txt)) else: diff -Nru mitmproxy-5.1.1/mitmproxy/tools/console/flowdetailview.py mitmproxy-6.0.2/mitmproxy/tools/console/flowdetailview.py --- mitmproxy-5.1.1/mitmproxy/tools/console/flowdetailview.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/tools/console/flowdetailview.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,5 +1,7 @@ +import typing import urwid +import mitmproxy.flow from mitmproxy import http from mitmproxy.tools.console import common, searchable from mitmproxy.utils import human @@ -10,16 +12,27 @@ if base is not None and getattr(base, attr): return human.format_timestamp_with_milli(getattr(base, attr)) else: - return "active" + # in mitmdump we serialize before a connection is closed. + # loading those flows at a later point shouldn't display "active". + # We also use a ndash (and not a regular dash) so that it is sorted + # after other timestamps. We may need to revisit that in the future if it turns out + # to render ugly in consoles. + return "–" -def flowdetails(state, flow: http.HTTPFlow): +def flowdetails(state, flow: mitmproxy.flow.Flow): text = [] sc = flow.server_conn cc = flow.client_conn - req = flow.request - resp = flow.response + req: typing.Optional[http.HTTPRequest] + resp: typing.Optional[http.HTTPResponse] + if isinstance(flow, http.HTTPFlow): + req = flow.request + resp = flow.response + else: + req = None + resp = None metadata = flow.metadata if metadata is not None and len(metadata) > 0: @@ -37,7 +50,7 @@ if resp: parts.append(("HTTP Version", resp.http_version)) if sc.alpn_proto_negotiated: - parts.append(("ALPN", sc.alpn_proto_negotiated)) + parts.append(("ALPN", strutils.bytes_to_escaped_str(sc.alpn_proto_negotiated))) text.extend( common.format_keyvals(parts, indent=4) @@ -100,11 +113,12 @@ if cc.tls_version: parts.append(("TLS Version", cc.tls_version)) if cc.sni: - parts.append(("Server Name Indication", cc.sni)) + parts.append(("Server Name Indication", + strutils.bytes_to_escaped_str(strutils.always_bytes(cc.sni, "idna")))) if cc.cipher_name: parts.append(("Cipher Name", cc.cipher_name)) if cc.alpn_proto_negotiated: - parts.append(("ALPN", cc.alpn_proto_negotiated)) + parts.append(("ALPN", strutils.bytes_to_escaped_str(cc.alpn_proto_negotiated))) text.extend( common.format_keyvals(parts, indent=4) @@ -126,6 +140,12 @@ maybe_timestamp(cc, "timestamp_tls_setup") ) ) + parts.append( + ( + "Client conn. closed", + maybe_timestamp(cc, "timestamp_end") + ) + ) if sc is not None and sc.timestamp_start: parts.append( @@ -147,6 +167,12 @@ maybe_timestamp(sc, "timestamp_tls_setup") ) ) + parts.append( + ( + "Server conn. closed", + maybe_timestamp(sc, "timestamp_end") + ) + ) if req is not None and req.timestamp_start: parts.append( diff -Nru mitmproxy-5.1.1/mitmproxy/tools/console/flowlist.py mitmproxy-6.0.2/mitmproxy/tools/console/flowlist.py --- mitmproxy-5.1.1/mitmproxy/tools/console/flowlist.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/tools/console/flowlist.py 2020-12-15 16:41:27.000000000 +0000 @@ -14,12 +14,17 @@ def get_text(self): cols, _ = self.master.ui.get_cols_rows() + layout = self.master.options.console_flowlist_layout + if layout == "list" or (layout == 'default' and cols < 100): + render_mode = common.RenderMode.LIST + else: + render_mode = common.RenderMode.TABLE + return common.format_flow( self.flow, - self.flow is self.master.view.focus.flow, + render_mode=render_mode, + focused=self.flow is self.master.view.focus.flow, hostheader=self.master.options.showhost, - cols=cols, - layout=self.master.options.console_flowlist_layout ) def selectable(self): @@ -27,9 +32,8 @@ def mouse_event(self, size, event, button, col, row, focus): if event == "mouse press" and button == 1: - if self.flow.request: - self.master.commands.execute("console.view.flow @focus") - return True + self.master.commands.execute("console.view.flow @focus") + return True def keypress(self, size, key): return key diff -Nru mitmproxy-5.1.1/mitmproxy/tools/console/flowview.py mitmproxy-6.0.2/mitmproxy/tools/console/flowview.py --- mitmproxy-5.1.1/mitmproxy/tools/console/flowview.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/tools/console/flowview.py 2020-12-15 16:41:27.000000000 +0000 @@ -5,9 +5,11 @@ import urwid +import mitmproxy.flow from mitmproxy import contentviews from mitmproxy import ctx from mitmproxy import http +from mitmproxy import tcp from mitmproxy.tools.console import common from mitmproxy.tools.console import layoutwidget from mitmproxy.tools.console import flowdetailview @@ -24,8 +26,8 @@ class FlowViewHeader(urwid.WidgetWrap): def __init__( - self, - master: "mitmproxy.tools.console.master.ConsoleMaster", + self, + master: "mitmproxy.tools.console.master.ConsoleMaster", ) -> None: self.master = master self.focus_changed() @@ -35,11 +37,8 @@ if self.master.view.focus.flow: self._w = common.format_flow( self.master.view.focus.flow, - False, - extended=True, + render_mode=common.RenderMode.DETAILVIEW, hostheader=self.master.options.showhost, - cols=cols, - layout=self.master.options.console_flowlist_layout ) else: self._w = urwid.Pile([]) @@ -52,45 +51,126 @@ self.show() self.last_displayed_body = None - def focus_changed(self): - if self.master.view.focus.flow: - self.tabs = [ - (self.tab_request, self.view_request), - (self.tab_response, self.view_response), - (self.tab_details, self.view_details), - ] - self.show() - else: - self.master.window.pop() - @property def view(self): return self.master.view @property - def flow(self): + def flow(self) -> mitmproxy.flow.Flow: return self.master.view.focus.flow - def tab_request(self): - if self.flow.intercepted and not self.flow.response: + def focus_changed(self): + if self.flow: + if isinstance(self.flow, http.HTTPFlow): + self.tabs = [ + (self.tab_http_request, self.view_request), + (self.tab_http_response, self.view_response), + (self.tab_details, self.view_details), + ] + elif isinstance(self.flow, tcp.TCPFlow): + self.tabs = [ + (self.tab_tcp_stream, self.view_tcp_stream), + (self.tab_details, self.view_details), + ] + self.show() + else: + self.master.window.pop() + + def tab_http_request(self): + flow = self.flow + assert isinstance(flow, http.HTTPFlow) + if self.flow.intercepted and not flow.response: return "Request intercepted" else: return "Request" - def tab_response(self): - if self.flow.intercepted and self.flow.response: + def tab_http_response(self): + flow = self.flow + assert isinstance(flow, http.HTTPFlow) + if self.flow.intercepted and flow.response: return "Response intercepted" else: return "Response" + def tab_tcp_stream(self): + return "TCP Stream" + def tab_details(self): return "Detail" def view_request(self): - return self.conn_text(self.flow.request) + flow = self.flow + assert isinstance(flow, http.HTTPFlow) + return self.conn_text(flow.request) def view_response(self): - return self.conn_text(self.flow.response) + flow = self.flow + assert isinstance(flow, http.HTTPFlow) + return self.conn_text(flow.response) + + def _contentview_status_bar(self, description: str, viewmode: str): + cols = [ + urwid.Text( + [ + ("heading", description), + ] + ), + urwid.Text( + [ + " ", + ('heading', "["), + ('heading_key', "m"), + ('heading', (":%s]" % viewmode)), + ], + align="right" + ) + ] + contentview_status_bar = urwid.AttrWrap(urwid.Columns(cols), "heading") + return contentview_status_bar + + def view_tcp_stream(self) -> urwid.Widget: + flow = self.flow + assert isinstance(flow, tcp.TCPFlow) + + if not flow.messages: + return searchable.Searchable([urwid.Text(("highlight", "No messages."))]) + + viewmode = self.master.commands.call("console.flowview.mode") + + # Merge adjacent TCP "messages". For detailed explanation of this code block see: + # https://github.com/mitmproxy/mitmproxy/pull/3970/files/469bd32582f764f9a29607efa4f5b04bd87961fb#r418670880 + from_client = None + messages = [] + for message in flow.messages: + if message.from_client is not from_client: + messages.append(message.content) + from_client = message.from_client + else: + messages[-1] += message.content + + widget_lines = [] + + from_client = flow.messages[0].from_client + for m in messages: + _, lines, _ = contentviews.get_tcp_content_view(viewmode, m) + + for line in lines: + if from_client: + line.insert(0, ("from_client", f"{common.SYMBOL_FROM_CLIENT} ")) + else: + line.insert(0, ("to_client", f"{common.SYMBOL_TO_CLIENT} ")) + + widget_lines.append(urwid.Text(line)) + + from_client = not from_client + + if flow.intercepted: + markup = widget_lines[-1].get_text()[0] + widget_lines[-1].set_text(("intercept", markup)) + + widget_lines.insert(0, self._contentview_status_bar(viewmode.capitalize(), viewmode)) + + return searchable.Searchable(widget_lines) def view_details(self): return flowdetailview.flowdetails(self.view, self.flow) @@ -229,7 +309,7 @@ def __init__(self, master): super().__init__( FlowDetails(master), - header = FlowViewHeader(master), + header=FlowViewHeader(master), ) self.master = master diff -Nru mitmproxy-5.1.1/mitmproxy/tools/console/grideditor/base.py mitmproxy-6.0.2/mitmproxy/tools/console/grideditor/base.py --- mitmproxy-5.1.1/mitmproxy/tools/console/grideditor/base.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/tools/console/grideditor/base.py 2020-12-15 16:41:27.000000000 +0000 @@ -16,7 +16,7 @@ try: with open(filename, "r" if escaped else "rb") as f: d = f.read() - except IOError as v: + except OSError as v: raise exceptions.CommandError(v) if escaped: try: @@ -155,7 +155,7 @@ def set_value(self, val, focus, focus_col, errors=None): if not errors: - errors = set([]) + errors = set() row = list(self.lst[focus][0]) row[focus_col] = val self.lst[focus] = [tuple(row), errors] @@ -171,7 +171,7 @@ self.focus = pos self.lst.insert( self.focus, - ([c.blank() for c in self.editor.columns], set([])) + ([c.blank() for c in self.editor.columns], set()) ) self.focus_col = 0 self.start_edit() diff -Nru mitmproxy-5.1.1/mitmproxy/tools/console/keymap.py mitmproxy-6.0.2/mitmproxy/tools/console/keymap.py --- mitmproxy-5.1.1/mitmproxy/tools/console/keymap.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/tools/console/keymap.py 2020-12-15 16:41:27.000000000 +0000 @@ -161,7 +161,7 @@ "ctx": lambda x: isinstance(x, list) and [isinstance(v, str) for v in x], "help": lambda x: isinstance(x, str), } -requiredKeyAttrs = set(["key", "cmd"]) +requiredKeyAttrs = {"key", "cmd"} class KeymapConfig: @@ -186,18 +186,18 @@ def load_path(self, km, p): if os.path.exists(p) and os.path.isfile(p): - with open(p, "rt", encoding="utf8") as f: + with open(p, encoding="utf8") as f: try: txt = f.read() except UnicodeDecodeError as e: raise KeyBindingError( - "Encoding error - expected UTF8: %s: %s" % (p, e) + f"Encoding error - expected UTF8: {p}: {e}" ) try: vals = self.parse(txt) except KeyBindingError as e: raise KeyBindingError( - "Error reading %s: %s" % (p, e) + f"Error reading {p}: {e}" ) from e for v in vals: user_ctxs = v.get("ctx", ["global"]) @@ -212,7 +212,7 @@ ) except ValueError as e: raise KeyBindingError( - "Error reading %s: %s" % (p, e) + f"Error reading {p}: {e}" ) from e def parse(self, text): diff -Nru mitmproxy-5.1.1/mitmproxy/tools/console/layoutwidget.py mitmproxy-6.0.2/mitmproxy/tools/console/layoutwidget.py --- mitmproxy-5.1.1/mitmproxy/tools/console/layoutwidget.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/tools/console/layoutwidget.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,5 +1,3 @@ - - class LayoutWidget: """ All top-level layout widgets and all widgets that may be set in an diff -Nru mitmproxy-5.1.1/mitmproxy/tools/console/master.py mitmproxy-6.0.2/mitmproxy/tools/console/master.py --- mitmproxy-5.1.1/mitmproxy/tools/console/master.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/tools/console/master.py 2020-12-15 16:41:27.000000000 +0000 @@ -11,6 +11,7 @@ import tempfile import typing # noqa import contextlib +import threading import urwid @@ -171,7 +172,9 @@ signals.status_message.send( message="Can't start external viewer: %s" % " ".join(c) ) - os.unlink(name) + # add a small delay before deletion so that the file is not removed before being loaded by the viewer + t = threading.Timer(1.0, os.unlink, args=[name]) + t.start() def set_palette(self, opts, updated): self.ui.register_palette( diff -Nru mitmproxy-5.1.1/mitmproxy/tools/console/palettes.py mitmproxy-6.0.2/mitmproxy/tools/console/palettes.py --- mitmproxy-5.1.1/mitmproxy/tools/console/palettes.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/tools/console/palettes.py 2020-12-15 16:41:27.000000000 +0000 @@ -22,9 +22,8 @@ 'option_selected_key', # List and Connections - 'method', 'method_get', 'method_post', 'method_delete', 'method_other', 'method_head', 'method_put', 'method_http2_push', - 'scheme_http', 'scheme_https', 'scheme_other', + 'scheme_http', 'scheme_https', 'scheme_tcp', 'scheme_other', 'url_punctuation', 'url_domain', 'url_filename', 'url_extension', 'url_query_key', 'url_query_value', 'content_none', 'content_text', 'content_script', 'content_media', 'content_data', 'content_raw', 'content_other', 'focus', @@ -35,6 +34,12 @@ # Hex view 'offset', + # JSON view + 'json_string', 'json_number', 'json_boolean', + + # TCP flow details + 'from_client', 'to_client', + # Grid Editor 'focusfield', 'focusfield_error', 'field_error', 'editfield', @@ -121,7 +126,6 @@ option_active_selected = ('light red', 'light gray'), # List and Connections - method = ('dark cyan', 'default'), method_get = ('light green', 'default'), method_post = ('brown', 'default'), method_delete = ('light red', 'default'), @@ -132,6 +136,7 @@ scheme_http = ('dark cyan', 'default'), scheme_https = ('dark green', 'default'), + scheme_tcp=('dark magenta', 'default'), scheme_other = ('dark magenta', 'default'), url_punctuation = ('light gray', 'default'), @@ -170,6 +175,15 @@ # Hex view offset = ('dark cyan', 'default'), + # JSON view + json_string = ('dark blue', 'default'), + json_number = ('light magenta', 'default'), + json_boolean = ('dark magenta', 'default'), + + # TCP flow details + from_client = ('light blue', 'default'), + to_client = ('light red', 'default'), + # Grid Editor focusfield = ('black', 'light gray'), focusfield_error = ('dark red', 'light gray'), @@ -221,7 +235,6 @@ option_active_selected = ('light red', 'light gray'), # List and Connections - method = ('dark cyan', 'default'), method_get = ('dark green', 'default'), method_post = ('brown', 'default'), method_head = ('dark cyan', 'default'), @@ -232,6 +245,7 @@ scheme_http = ('dark cyan', 'default'), scheme_https = ('light green', 'default'), + scheme_tcp=('light magenta', 'default'), scheme_other = ('light magenta', 'default'), url_punctuation = ('dark gray', 'default'), @@ -270,6 +284,15 @@ # Hex view offset = ('dark blue', 'default'), + # JSON view + json_string = ('dark blue', 'default'), + json_number = ('light magenta', 'default'), + json_boolean = ('dark magenta', 'default'), + + # TCP flow details + from_client = ('dark blue', 'default'), + to_client = ('dark red', 'default'), + # Grid Editor focusfield = ('black', 'light gray'), focusfield_error = ('dark red', 'light gray'), @@ -340,7 +363,6 @@ # List and Connections - method = ('dark cyan', 'default'), method_get = (sol_green, 'default'), method_post = (sol_orange, 'default'), method_head = (sol_cyan, 'default'), @@ -351,6 +373,7 @@ scheme_http = (sol_cyan, 'default'), scheme_https = ('light green', 'default'), + scheme_tcp=('light magenta', 'default'), scheme_other = ('light magenta', 'default'), url_punctuation = ('dark gray', 'default'), @@ -380,6 +403,15 @@ # Hex view offset = (sol_cyan, 'default'), + # JSON view + json_string = (sol_cyan, 'default'), + json_number = (sol_blue, 'default'), + json_boolean = (sol_magenta, 'default'), + + # TCP flow details + from_client = (sol_blue, 'default'), + to_client = (sol_red, 'default'), + # Grid Editor focusfield = (sol_base00, sol_base2), focusfield_error = (sol_red, sol_base2), @@ -416,7 +448,6 @@ # List and Connections focus = (sol_base1, 'default'), - method = (sol_cyan, 'default'), method_get = (sol_green, 'default'), method_post = (sol_orange, 'default'), method_delete = (sol_red, 'default'), @@ -454,6 +485,15 @@ # Hex view offset = (sol_cyan, 'default'), + # JSON view + json_string = (sol_cyan, 'default'), + json_number = (sol_blue, 'default'), + json_boolean = (sol_magenta, 'default'), + + # TCP flow details + from_client = (sol_blue, 'default'), + to_client = (sol_red, 'default'), + # Grid Editor focusfield = (sol_base0, sol_base02), focusfield_error = (sol_red, sol_base02), diff -Nru mitmproxy-5.1.1/mitmproxy/tools/console/statusbar.py mitmproxy-6.0.2/mitmproxy/tools/console/statusbar.py --- mitmproxy-5.1.1/mitmproxy/tools/console/statusbar.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/tools/console/statusbar.py 2020-12-15 16:41:27.000000000 +0000 @@ -20,7 +20,7 @@ pth = os.path.expanduser(pth) try: return self.callback(pth, *self.args) - except IOError as v: + except OSError as v: signals.status_message.send(message=v.strerror) @@ -128,7 +128,7 @@ mkup.append(",") prompt.extend(mkup) prompt.append(")? ") - self.onekey = set(i[1] for i in keys) + self.onekey = {i[1] for i in keys} self._w = urwid.Edit(prompt, "") self.prompting = PromptStub(callback, args) @@ -206,12 +206,12 @@ sreplay = self.master.commands.call("replay.server.count") creplay = self.master.commands.call("replay.client.count") - if len(self.master.options.setheaders): + if len(self.master.options.modify_headers): r.append("[") r.append(("heading_key", "H")) r.append("eaders]") - if len(self.master.options.replacements): - r.append("[%d replacements]" % len(self.master.options.replacements)) + if len(self.master.options.modify_body): + r.append("[%d body modifications]" % len(self.master.options.modify_body)) if creplay: r.append("[") r.append(("heading_key", "cplayback")) @@ -305,14 +305,14 @@ marked = "M" t = [ - ('heading', ("%s %s [%s/%s]" % (arrow, marked, offset, fc)).ljust(11)), + ('heading', (f"{arrow} {marked} [{offset}/{fc}]").ljust(11)), ] if self.master.options.server: host = self.master.options.listen_host if host == "0.0.0.0" or host == "": host = "*" - boundaddr = "[%s:%s]" % (host, self.master.options.listen_port) + boundaddr = f"[{host}:{self.master.options.listen_port}]" else: boundaddr = "" t.extend(self.get_status()) diff -Nru mitmproxy-5.1.1/mitmproxy/tools/console/tabs.py mitmproxy-6.0.2/mitmproxy/tools/console/tabs.py --- mitmproxy-5.1.1/mitmproxy/tools/console/tabs.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/tools/console/tabs.py 2020-12-15 16:41:27.000000000 +0000 @@ -50,7 +50,7 @@ headers = [] for i in range(len(self.tabs)): txt = self.tabs[i][0]() - if i == self.tab_offset: + if i == self.tab_offset % len(self.tabs): headers.append( Tab( i, @@ -70,7 +70,7 @@ ) headers = urwid.Columns(headers, dividechars=1) self._w = urwid.Frame( - body = self.tabs[self.tab_offset][1](), + body = self.tabs[self.tab_offset % len(self.tabs)][1](), header = headers ) self._w.set_focus("body") diff -Nru mitmproxy-5.1.1/mitmproxy/tools/_main.py mitmproxy-6.0.2/mitmproxy/tools/_main.py --- mitmproxy-5.1.1/mitmproxy/tools/_main.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/tools/_main.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,173 +0,0 @@ -""" -This file contains python3.6+ syntax! -Feel free to import and use whatever new package you deem necessary. -""" - -import os -import sys -import asyncio -import argparse -import signal -import typing - -from mitmproxy.tools import cmdline -from mitmproxy import exceptions, master -from mitmproxy import options -from mitmproxy import optmanager -from mitmproxy import proxy -from mitmproxy.utils import debug, arg_check - - -def assert_utf8_env(): - spec = "" - for i in ["LANG", "LC_CTYPE", "LC_ALL"]: - spec += os.environ.get(i, "").lower() - if "utf" not in spec: - print( - "Error: mitmproxy requires a UTF console environment.", - file=sys.stderr - ) - print( - "Set your LANG environment variable to something like en_US.UTF-8", - file=sys.stderr - ) - sys.exit(1) - - -def process_options(parser, opts, args): - if args.version: - print(debug.dump_system_info()) - sys.exit(0) - if args.quiet or args.options or args.commands: - # also reduce log verbosity if --options or --commands is passed, - # we don't want log messages from regular startup then. - args.termlog_verbosity = 'error' - args.flow_detail = 0 - if args.verbose: - args.termlog_verbosity = 'debug' - args.flow_detail = 2 - - adict = {} - for n in dir(args): - if n in opts: - adict[n] = getattr(args, n) - opts.merge(adict) - - return proxy.config.ProxyConfig(opts) - - -def run( - master_cls: typing.Type[master.Master], - make_parser: typing.Callable[[options.Options], argparse.ArgumentParser], - arguments: typing.Sequence[str], - extra: typing.Callable[[typing.Any], dict] = None -) -> master.Master: # pragma: no cover - """ - extra: Extra argument processing callable which returns a dict of - options. - """ - debug.register_info_dumpers() - - opts = options.Options() - master = master_cls(opts) - - parser = make_parser(opts) - - # To make migration from 2.x to 3.0 bearable. - if "-R" in sys.argv and sys.argv[sys.argv.index("-R") + 1].startswith("http"): - print("-R is used for specifying replacements.\n" - "To use mitmproxy in reverse mode please use --mode reverse:SPEC instead") - - try: - args = parser.parse_args(arguments) - except SystemExit: - arg_check.check() - sys.exit(1) - - try: - opts.set(*args.setoptions, defer=True) - optmanager.load_paths( - opts, - os.path.join(opts.confdir, "config.yaml"), - os.path.join(opts.confdir, "config.yml"), - ) - pconf = process_options(parser, opts, args) - server: typing.Any = None - if pconf.options.server: - try: - server = proxy.server.ProxyServer(pconf) - except exceptions.ServerException as v: - print(str(v), file=sys.stderr) - sys.exit(1) - else: - server = proxy.server.DummyServer(pconf) - - master.server = server - if args.options: - print(optmanager.dump_defaults(opts)) - sys.exit(0) - if args.commands: - master.commands.dump() - sys.exit(0) - if extra: - opts.update(**extra(args)) - - loop = asyncio.get_event_loop() - try: - loop.add_signal_handler(signal.SIGINT, getattr(master, "prompt_for_exit", master.shutdown)) - loop.add_signal_handler(signal.SIGTERM, master.shutdown) - except NotImplementedError: - # Not supported on Windows - pass - - # Make sure that we catch KeyboardInterrupts on Windows. - # https://stackoverflow.com/a/36925722/934719 - if os.name == "nt": - async def wakeup(): - while True: - await asyncio.sleep(0.2) - asyncio.ensure_future(wakeup()) - - master.run() - except exceptions.OptionsError as e: - print("%s: %s" % (sys.argv[0], e), file=sys.stderr) - sys.exit(1) - except (KeyboardInterrupt, RuntimeError): - pass - return master - - -def mitmproxy(args=None) -> typing.Optional[int]: # pragma: no cover - if os.name == "nt": - print("Error: mitmproxy's console interface is not supported on Windows. " - "You can run mitmdump or mitmweb instead.", file=sys.stderr) - return 1 - assert_utf8_env() - from mitmproxy.tools import console - run(console.master.ConsoleMaster, cmdline.mitmproxy, args) - return None - - -def mitmdump(args=None) -> typing.Optional[int]: # pragma: no cover - from mitmproxy.tools import dump - - def extra(args): - if args.filter_args: - v = " ".join(args.filter_args) - return dict( - save_stream_filter=v, - readfile_filter=v, - dumper_filter=v, - ) - return {} - - m = run(dump.DumpMaster, cmdline.mitmdump, args, extra) - if m and m.errorcheck.has_errored: # type: ignore - return 1 - return None - - -def mitmweb(args=None) -> typing.Optional[int]: # pragma: no cover - from mitmproxy.tools import web - run(web.master.WebMaster, cmdline.mitmweb, args) - return None diff -Nru mitmproxy-5.1.1/mitmproxy/tools/main.py mitmproxy-6.0.2/mitmproxy/tools/main.py --- mitmproxy-5.1.1/mitmproxy/tools/main.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/tools/main.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,22 +1,169 @@ -""" -This file must be kept in a python2.7 and python3.5 compatible syntax! -DO NOT use type annotations or other python3.6-only features that makes this file unparsable by older interpreters! -""" +import os +import sys +import asyncio +import argparse +import signal +import typing -from __future__ import print_function # this is here for the version check to work on Python 2. +from mitmproxy.tools import cmdline +from mitmproxy import exceptions, master +from mitmproxy import options +from mitmproxy import optmanager +from mitmproxy import proxy +from mitmproxy.utils import compat, debug, arg_check + + +def assert_utf8_env(): + spec = "" + for i in ["LANG", "LC_CTYPE", "LC_ALL"]: + spec += os.environ.get(i, "").lower() + if "utf" not in spec: + print( + "Error: mitmproxy requires a UTF console environment.", + file=sys.stderr + ) + print( + "Set your LANG environment variable to something like en_US.UTF-8", + file=sys.stderr + ) + sys.exit(1) + + +def process_options(parser, opts, args): + if args.version: + print(debug.dump_system_info()) + sys.exit(0) + if args.quiet or args.options or args.commands: + # also reduce log verbosity if --options or --commands is passed, + # we don't want log messages from regular startup then. + args.termlog_verbosity = 'error' + args.flow_detail = 0 + if args.verbose: + args.termlog_verbosity = 'debug' + args.flow_detail = 2 + + adict = {} + for n in dir(args): + if n in opts: + adict[n] = getattr(args, n) + opts.merge(adict) + + return proxy.config.ProxyConfig(opts) + + +def run( + master_cls: typing.Type[master.Master], + make_parser: typing.Callable[[options.Options], argparse.ArgumentParser], + arguments: typing.Sequence[str], + extra: typing.Callable[[typing.Any], dict] = None +) -> master.Master: # pragma: no cover + """ + extra: Extra argument processing callable which returns a dict of + options. + """ + debug.register_info_dumpers() + + opts = options.Options() + master = master_cls(opts) + + parser = make_parser(opts) + + # To make migration from 2.x to 3.0 bearable. + if "-R" in sys.argv and sys.argv[sys.argv.index("-R") + 1].startswith("http"): + print("To use mitmproxy in reverse mode please use --mode reverse:SPEC instead") + + try: + args = parser.parse_args(arguments) + except SystemExit: + arg_check.check() + sys.exit(1) + + try: + opts.set(*args.setoptions, defer=True) + optmanager.load_paths( + opts, + os.path.join(opts.confdir, "config.yaml"), + os.path.join(opts.confdir, "config.yml"), + ) + pconf = process_options(parser, opts, args) + server: typing.Any = None + if pconf.options.server and not compat.new_proxy_core: # new core initializes itself as an addon + try: + server = proxy.server.ProxyServer(pconf) + except exceptions.ServerException as v: + print(str(v), file=sys.stderr) + sys.exit(1) + else: + server = proxy.server.DummyServer(pconf) + + master.server = server + if args.options: + print(optmanager.dump_defaults(opts)) + sys.exit(0) + if args.commands: + master.commands.dump() + sys.exit(0) + if extra: + if(args.filter_args): + master.log.info(f"Only processing flows that match \"{' & '.join(args.filter_args)}\"") + opts.update(**extra(args)) + + loop = asyncio.get_event_loop() + try: + loop.add_signal_handler(signal.SIGINT, getattr(master, "prompt_for_exit", master.shutdown)) + loop.add_signal_handler(signal.SIGTERM, master.shutdown) + except NotImplementedError: + # Not supported on Windows + pass + + # Make sure that we catch KeyboardInterrupts on Windows. + # https://stackoverflow.com/a/36925722/934719 + if os.name == "nt": + async def wakeup(): + while True: + await asyncio.sleep(0.2) + asyncio.ensure_future(wakeup()) + + master.run() + except exceptions.OptionsError as e: + print("{}: {}".format(sys.argv[0], e), file=sys.stderr) + sys.exit(1) + except (KeyboardInterrupt, RuntimeError): + pass + return master + + +def mitmproxy(args=None) -> typing.Optional[int]: # pragma: no cover + if os.name == "nt": + print("Error: mitmproxy's console interface is not supported on Windows. " + "You can run mitmdump or mitmweb instead.", file=sys.stderr) + return 1 + assert_utf8_env() + from mitmproxy.tools import console + run(console.master.ConsoleMaster, cmdline.mitmproxy, args) + return None + + +def mitmdump(args=None) -> typing.Optional[int]: # pragma: no cover + from mitmproxy.tools import dump + + def extra(args): + if args.filter_args: + v = " ".join(args.filter_args) + return dict( + save_stream_filter=v, + readfile_filter=v, + dumper_filter=v, + ) + return {} + + m = run(dump.DumpMaster, cmdline.mitmdump, args, extra) + if m and m.errorcheck.has_errored: # type: ignore + return 1 + return None -import sys -if sys.version_info < (3, 6): - # This must be before any mitmproxy imports, as they already break! - # Keep all other imports below with the 'noqa' magic comment. - print("#" * 76, file=sys.stderr) - print("# mitmproxy requires Python 3.6 or higher! #", file=sys.stderr) - print("#" + " " * 74 + "#", file=sys.stderr) - print("# Please upgrade your Python intepreter or use our mitmproxy binaries from #", file=sys.stderr) - print("# https://mitmproxy.org. If your operating system does not include the #", file=sys.stderr) - print("# required Python version, you can try using pyenv or similar tools. #", file=sys.stderr) - print("#" * 76, file=sys.stderr) - sys.exit(1) -else: - from ._main import * # noqa +def mitmweb(args=None) -> typing.Optional[int]: # pragma: no cover + from mitmproxy.tools import web + run(web.master.WebMaster, cmdline.mitmweb, args) + return None diff -Nru mitmproxy-5.1.1/mitmproxy/tools/web/app.py mitmproxy-6.0.2/mitmproxy/tools/web/app.py --- mitmproxy-5.1.1/mitmproxy/tools/web/app.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/tools/web/app.py 2020-12-15 16:41:27.000000000 +0000 @@ -33,6 +33,7 @@ f = { "id": flow.id, "intercepted": flow.intercepted, + "is_replay": flow.is_replay, "client_conn": flow.client_conn.get_state(), "server_conn": flow.server_conn.get_state(), "type": flow.type, @@ -72,7 +73,7 @@ "contentHash": content_hash, "timestamp_start": flow.request.timestamp_start, "timestamp_end": flow.request.timestamp_end, - "is_replay": flow.request.is_replay, + "is_replay": flow.is_replay == "request", # TODO: remove, use flow.is_replay instead. "pretty_host": flow.request.pretty_host, } if flow.response: @@ -91,10 +92,16 @@ "contentHash": content_hash, "timestamp_start": flow.response.timestamp_start, "timestamp_end": flow.response.timestamp_end, - "is_replay": flow.response.is_replay, + "is_replay": flow.is_replay == "response", # TODO: remove, use flow.is_replay instead. } - f.get("server_conn", {}).pop("cert", None) + if flow.response.data.trailers: + f["response"]["trailers"] = tuple(flow.response.data.trailers.items(True)) + + f.get("server_conn", {}).pop("certificate_list", None) + f.get("client_conn", {}).pop("certificate_list", None) f.get("client_conn", {}).pop("mitmcert", None) + f.get("server_conn", {}).pop("alpn_offers", None) + f.get("client_conn", {}).pop("alpn_offers", None) return f @@ -120,7 +127,7 @@ if isinstance(chunk, list): chunk = tornado.escape.json_encode(chunk) self.set_header("Content-Type", "application/json; charset=UTF-8") - super(RequestHandler, self).write(chunk) + super().write(chunk) def set_default_headers(self): super().set_default_headers() @@ -301,10 +308,14 @@ request.headers.clear() for header in v: request.headers.add(*header) + elif k == "trailers": + request.trailers.clear() + for trailer in v: + request.trailers.add(*trailer) elif k == "content": request.text = v else: - raise APIError(400, "Unknown update request.{}: {}".format(k, v)) + raise APIError(400, f"Unknown update request.{k}: {v}") elif a == "response" and hasattr(flow, "response"): response = flow.response @@ -317,12 +328,16 @@ response.headers.clear() for header in v: response.headers.add(*header) + elif k == "trailers": + response.trailers.clear() + for trailer in v: + response.trailers.add(*trailer) elif k == "content": response.text = v else: - raise APIError(400, "Unknown update response.{}: {}".format(k, v)) + raise APIError(400, f"Unknown update response.{k}: {v}") else: - raise APIError(400, "Unknown update {}: {}".format(a, b)) + raise APIError(400, f"Unknown update {a}: {b}") except APIError: flow.revert() raise @@ -383,7 +398,7 @@ filename = self.flow.request.path.split("?")[0].split("/")[-1] filename = re.sub(r'[^-\w" .()]', "", filename) - cd = "attachment; filename={}".format(filename) + cd = f"attachment; filename={filename}" self.set_header("Content-Disposition", cd) self.set_header("Content-Type", "application/text") self.set_header("X-Content-Type-Options", "nosniff") @@ -437,14 +452,14 @@ def put(self): update = self.json - option_whitelist = { + allowed_options = { "intercept", "showhost", "upstream_cert", "ssl_insecure", "rawtcp", "http2", "websocket", "anticache", "anticomp", "stickycookie", "stickyauth", "stream_large_bodies" } for k in update: - if k not in option_whitelist: - raise APIError(400, "Unknown setting {}".format(k)) + if k not in allowed_options: + raise APIError(400, f"Unknown setting {k}") self.master.options.update(**update) @@ -457,7 +472,7 @@ try: self.master.options.update(**update) except Exception as err: - raise APIError(400, "{}".format(err)) + raise APIError(400, f"{err}") class SaveOptions(RequestHandler): @@ -496,7 +511,7 @@ self.add_handlers("dns-rebind-protection", [(r"/.*", DnsRebind)]) self.add_handlers( # make mitmweb accessible by IP only to prevent DNS rebinding. - r'^(localhost|[0-9.:\[\]]+)$', + r'^(localhost|[0-9.]+|\[[0-9a-fA-F:]+\])$', [ (r"/", IndexHandler), (r"/filter-help(?:\.json)?", FilterHelp), diff -Nru mitmproxy-5.1.1/mitmproxy/tools/web/master.py mitmproxy-6.0.2/mitmproxy/tools/web/master.py --- mitmproxy-5.1.1/mitmproxy/tools/web/master.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/tools/web/master.py 2020-12-15 16:41:27.000000000 +0000 @@ -106,8 +106,8 @@ iol = tornado.ioloop.IOLoop.instance() http_server = tornado.httpserver.HTTPServer(self.app) http_server.listen(self.options.web_port, self.options.web_host) - web_url = "http://{}:{}/".format(self.options.web_host, self.options.web_port) + web_url = f"http://{self.options.web_host}:{self.options.web_port}/" self.log.info( - "Web server listening at {}".format(web_url), + f"Web server listening at {web_url}", ) self.run_loop(iol.start) diff -Nru mitmproxy-5.1.1/mitmproxy/tools/web/static/app.css mitmproxy-6.0.2/mitmproxy/tools/web/static/app.css --- mitmproxy-5.1.1/mitmproxy/tools/web/static/app.css 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/tools/web/static/app.css 2020-12-15 16:41:27.000000000 +0000 @@ -1,2 +1,2 @@ -html{box-sizing:border-box}*,:after,:before{box-sizing:inherit}.resource-icon{width:32px;height:32px}.resource-icon-css{background-image:url(images/chrome-devtools/resourceCSSIcon.png)}.resource-icon-document{background-image:url(images/chrome-devtools/resourceDocumentIcon.png)}.resource-icon-js{background-image:url(images/chrome-devtools/resourceJSIcon.png)}.resource-icon-plain{background-image:url(images/chrome-devtools/resourcePlainIcon.png)}.resource-icon-executable{background-image:url(images/resourceExecutableIcon.png)}.resource-icon-flash{background-image:url(images/resourceFlashIcon.png)}.resource-icon-image{background-image:url(images/resourceImageIcon.png)}.resource-icon-java{background-image:url(images/resourceJavaIcon.png)}.resource-icon-not-modified{background-image:url(images/resourceNotModifiedIcon.png)}.resource-icon-redirect{background-image:url(images/resourceRedirectIcon.png)}#container,#mitmproxy,body,html{height:100%;margin:0;overflow:hidden}#container{display:flex;flex-direction:column;outline:0}#container>.eventlog,#container>footer,#container>header{flex:0 0 auto}.main-view{flex:1 1 auto;height:0;display:flex;flex-direction:row}.main-view.vertical{flex-direction:column}.main-view .flow-detail,.main-view .flow-table{flex:1 1 auto}.splitter{flex:0 0 1px;background-color:#aaa;position:relative}.splitter>div{position:absolute}.splitter.splitter-x{cursor:col-resize}.splitter.splitter-x>div{margin-left:-1px;width:4px;height:100%}.splitter.splitter-y{cursor:row-resize}.splitter.splitter-y>div{margin-top:-1px;height:4px;width:100%}.nav-tabs{border-bottom:solid #a6a6a6 1px}.nav-tabs a{display:inline-block;border:solid transparent 1px;text-decoration:none}.nav-tabs a.active{background-color:#fff;border-color:#a6a6a6;border-bottom-color:#fff}.nav-tabs a.special{color:#fff;background-color:#396cad;border-bottom-color:#396cad}.nav-tabs a.special:hover{background-color:#5386c6}.nav-tabs-lg a{padding:3px 14px;margin:0 2px -1px}.nav-tabs-sm a{padding:0 7px;margin:2px 2px -1px}.nav-tabs-sm a.nav-action{float:right;padding:0;margin:1px 0 0}header{padding-top:6px;background-color:#fff}header>div{display:block;margin:0;padding:0;border-bottom:solid #a6a6a6 1px;height:85px;overflow:visible}.menu-group{margin:0 3px;display:inline-block;height:85px;vertical-align:top}.menu-content{height:69px;text-align:center}.menu-content>.btn{height:69px;text-align:center;margin:0 1px;padding:12px 5px;border:none;border-radius:0}.menu-content>.btn i{font-size:20px;display:block;margin:0 auto 5px}.menu-entry{text-align:left;height:23px;line-height:1;padding:.5rem 1rem}.menu-entry label{font-size:1.2rem;font-weight:400;margin:0}.menu-entry input[type=checkbox]{margin:0 2px;vertical-align:middle}.menu-legend{height:16px;text-align:center;font-size:12px;padding:0 5px}.menu-group+.menu-group:before{margin-left:-3px;content:" ";border-left:solid 1px #e6e6e6;margin-top:10px;height:65px;position:absolute}.menu-main{margin-left:-2px;margin-right:-3px;padding:2px 5px}.filter-input{position:relative;min-height:1px;padding-left:2.5px;padding-right:2.5px;padding:2.5px}@media (min-width:768px){.filter-input{float:left;width:41.66666667%}}@media (max-width:767px){.filter-input{padding:2px 2.5px}.filter-input>.form-control,.filter-input>.input-group-addon,.filter-input>.input-group-btn>.btn{height:23.5px;padding:1px 5px;font-size:12px;line-height:1.5}}.filter-input .popover{top:27px;left:43px;display:block;max-width:none;opacity:.9}@media (max-width:767px){.filter-input .popover{top:16px;left:29px;right:2px}}.filter-input .popover .popover-content{max-height:500px;overflow-y:auto}.filter-input .popover .popover-content tr{cursor:pointer}.filter-input .popover .popover-content tr:hover{background-color:hsla(209,52%,84%,.5)!important}.connection-indicator{display:inline;padding:.2em .6em .3em;font-size:75%;font-weight:700;line-height:1;color:#fff;text-align:center;white-space:nowrap;vertical-align:baseline;border-radius:.25em;float:right;margin:5px;opacity:1;transition:all 1s linear}a.connection-indicator:focus,a.connection-indicator:hover{color:#fff;text-decoration:none;cursor:pointer}.connection-indicator:empty{display:none}.btn .connection-indicator{position:relative;top:-1px}.connection-indicator.fetching,.connection-indicator.init{background-color:#5bc0de}.connection-indicator.established{background-color:#5cb85c;opacity:0}.connection-indicator.error{background-color:#d9534f;transition:all .2s linear}.connection-indicator.offline{background-color:#f0ad4e;opacity:1}.flow-table{width:100%;overflow-y:scroll;overflow-x:hidden}.flow-table table{width:100%;table-layout:fixed}.flow-table thead{background-color:#f2f2f2;line-height:23px}.flow-table th{font-weight:400;box-shadow:0 1px 0 #a6a6a6;position:relative!important;padding-left:1px;-webkit-touch-callout:none;-webkit-user-select:none;-khtml-user-select:none;-moz-user-select:none;-ms-user-select:none;user-select:none}.flow-table th.sort-asc,.flow-table th.sort-desc{background-color:#fafafa}.flow-table th.sort-asc:after,.flow-table th.sort-desc:after{font:normal normal normal 14px/1 FontAwesome;position:absolute;right:3px;top:3px;padding:2px;background-color:rgba(250,250,250,.8)}.flow-table th.sort-asc:after{content:"\f0de"}.flow-table th.sort-desc:after{content:"\f0dd"}.flow-table tr{cursor:pointer}.flow-table tr:nth-child(even){background-color:rgba(0,0,0,.05)}.flow-table tr.selected{background-color:hsla(209,52%,84%,.5)!important}.flow-table tr.highlighted{background-color:hsla(48,100%,50%,.4)}.flow-table tr.highlighted:nth-child(even){background-color:hsla(48,100%,50%,.5)}.flow-table td{overflow:hidden;white-space:nowrap;text-overflow:ellipsis}.flow-table tr.intercepted:not(.has-response) .col-method,.flow-table tr.intercepted:not(.has-response) .col-path{color:#ff7f00}.flow-table tr.intercepted.has-response .col-size,.flow-table tr.intercepted.has-response .col-status,.flow-table tr.intercepted.has-response .col-time{color:#ff7f00}.flow-table .fa{line-height:inherit}.flow-table .fa.pull-right{margin-left:0}.flow-table .col-tls{width:10px}.flow-table .col-tls-https{background-color:rgba(0,185,0,.5)}.flow-table .col-icon{width:32px}.flow-table .col-path .fa-repeat{color:green}.flow-table .col-path .fa-pause{color:#ff7f00}.flow-table .col-path .fa-exclamation,.flow-table .col-path .fa-times{color:#8b0000}.flow-table .col-method{width:60px}.flow-table .col-status{width:50px}.flow-table .col-size{width:70px}.flow-table .col-time{width:50px}.flow-table td.col-size,.flow-table td.col-time{text-align:right}.flow-detail{width:100%;overflow:hidden;display:flex;flex-direction:column}.flow-detail nav{background-color:#f2f2f2}.flow-detail section{overflow-y:scroll}.flow-detail section>article{overflow:auto;padding:5px 12px 0}.flow-detail section>footer{box-shadow:0 0 3px gray;padding:2px;margin:0;height:23px}.flow-detail section.detail,.flow-detail section.error{overflow:auto;padding:5px 12px 0}.flow-detail .first-line{font-family:Menlo,Monaco,Consolas,"Courier New",monospace;background-color:#428bca;color:#fff;margin:0 -8px;padding:4px 8px;border-radius:5px;word-break:break-all;max-height:100px;overflow-y:auto}.flow-detail .first-line .inline-input.editable{border-color:rgba(255,255,255,.5)}.flow-detail .request-line{margin-bottom:2px}.flow-detail hr{margin:0 0 5px}.inline-input{display:inline;margin:0 -3px;padding:0 3px;border:solid transparent 1px}.inline-input.editable{border-color:#ccc}.inline-input[contenteditable]{background-color:rgba(255,255,255,.2)}.inline-input[contenteditable].has-warning{color:#ffb8b8}.view-all-content-btn{float:right;margin-bottom:12px}.flow-detail table{font-family:Menlo,Monaco,Consolas,"Courier New",monospace;width:100%;table-layout:fixed;word-break:break-all}.flow-detail table tr:not(:first-child){border-top:1px solid #f7f7f7}.flow-detail table td{vertical-align:top}.connection-table td:first-child{width:50%;padding-right:1em}.header-table td{line-height:1.3em}.header-table .header-name{width:33%}.header-table .header-colon{position:absolute;opacity:0}.header-table .inline-input{display:inline-block;width:100%;height:100%}.connection-table td,.timing-table td{overflow:hidden;text-overflow:ellipsis;white-space:nowrap}.flowview-image{text-align:center}.flowview-image img{max-width:100%;max-height:100%}.edit-flow-container{position:fixed;right:20px}.edit-flow{cursor:pointer;position:absolute;right:0;top:5px;height:40px;width:40px;border-radius:20px;z-index:10000;background-color:rgba(255,255,255,.7);border:solid 2px rgba(248,145,59,.7);text-align:center;font-size:22px;line-height:37px;transition:all .1s ease-in-out}.edit-flow:hover{background-color:rgba(239,108,0,.7);color:rgba(0,0,0,.8);border:solid 2px transparent}.eventlog{height:200px;flex:0 0 auto;display:flex;flex-direction:column}.eventlog>div{background-color:#f2f2f2;padding:0 5px;flex:0 0 auto;border-top:1px solid #aaa;cursor:row-resize}.eventlog>pre{flex:1 1 auto;margin:0;border-radius:0;overflow-x:auto;overflow-y:scroll;background-color:#fcfcfc}.eventlog .fa-close{cursor:pointer;float:right;color:grey;padding:3px 0;padding-left:10px}.eventlog .fa-close:hover{color:#000}.eventlog .btn-toggle{margin-top:-2px;margin-left:3px;padding:2px 2px;font-size:10px;line-height:10px;border-radius:2px}.eventlog .label{cursor:pointer;vertical-align:middle;display:inline-block;margin-top:-2px;margin-left:3px}footer{box-shadow:0 -1px 3px #d3d3d3;padding:0 10px 3px}footer .label{margin-right:3px}.CodeMirror{border:1px solid #ccc;height:auto!important}.CodeMirror{font-family:monospace;height:300px;color:#000;direction:ltr}.CodeMirror-lines{padding:4px 0}.CodeMirror pre{padding:0 4px}.CodeMirror-gutter-filler,.CodeMirror-scrollbar-filler{background-color:#fff}.CodeMirror-gutters{border-right:1px solid #ddd;background-color:#f7f7f7;white-space:nowrap}.CodeMirror-linenumber{padding:0 3px 0 5px;min-width:20px;text-align:right;color:#999;white-space:nowrap}.CodeMirror-guttermarker{color:#000}.CodeMirror-guttermarker-subtle{color:#999}.CodeMirror-cursor{border-left:1px solid #000;border-right:none;width:0}.CodeMirror div.CodeMirror-secondarycursor{border-left:1px solid silver}.cm-fat-cursor .CodeMirror-cursor{width:auto;border:0!important;background:#7e7}.cm-fat-cursor div.CodeMirror-cursors{z-index:1}.cm-fat-cursor-mark{background-color:rgba(20,255,20,.5);-webkit-animation:blink 1.06s steps(1) infinite;-moz-animation:blink 1.06s steps(1) infinite;animation:blink 1.06s steps(1) infinite}.cm-animate-fat-cursor{width:auto;border:0;-webkit-animation:blink 1.06s steps(1) infinite;-moz-animation:blink 1.06s steps(1) infinite;animation:blink 1.06s steps(1) infinite;background-color:#7e7}@-moz-keyframes blink{50%{background-color:transparent}}@-webkit-keyframes blink{50%{background-color:transparent}}@keyframes blink{50%{background-color:transparent}}.cm-tab{display:inline-block;text-decoration:inherit}.CodeMirror-rulers{position:absolute;left:0;right:0;top:-50px;bottom:-20px;overflow:hidden}.CodeMirror-ruler{border-left:1px solid #ccc;top:0;bottom:0;position:absolute}.cm-s-default .cm-header{color:#00f}.cm-s-default .cm-quote{color:#090}.cm-negative{color:#d44}.cm-positive{color:#292}.cm-header,.cm-strong{font-weight:700}.cm-em{font-style:italic}.cm-link{text-decoration:underline}.cm-strikethrough{text-decoration:line-through}.cm-s-default .cm-keyword{color:#708}.cm-s-default .cm-atom{color:#219}.cm-s-default .cm-number{color:#164}.cm-s-default .cm-def{color:#00f}.cm-s-default .cm-variable-2{color:#05a}.cm-s-default .cm-type,.cm-s-default .cm-variable-3{color:#085}.cm-s-default .cm-comment{color:#a50}.cm-s-default .cm-string{color:#a11}.cm-s-default .cm-string-2{color:#f50}.cm-s-default .cm-meta{color:#555}.cm-s-default .cm-qualifier{color:#555}.cm-s-default .cm-builtin{color:#30a}.cm-s-default .cm-bracket{color:#997}.cm-s-default .cm-tag{color:#170}.cm-s-default .cm-attribute{color:#00c}.cm-s-default .cm-hr{color:#999}.cm-s-default .cm-link{color:#00c}.cm-s-default .cm-error{color:red}.cm-invalidchar{color:red}.CodeMirror-composing{border-bottom:2px solid}div.CodeMirror span.CodeMirror-matchingbracket{color:#0f0}div.CodeMirror span.CodeMirror-nonmatchingbracket{color:#f22}.CodeMirror-matchingtag{background:rgba(255,150,0,.3)}.CodeMirror-activeline-background{background:#e8f2ff}.CodeMirror{position:relative;overflow:hidden;background:#fff}.CodeMirror-scroll{overflow:scroll!important;margin-bottom:-30px;margin-right:-30px;padding-bottom:30px;height:100%;outline:0;position:relative}.CodeMirror-sizer{position:relative;border-right:30px solid transparent}.CodeMirror-gutter-filler,.CodeMirror-hscrollbar,.CodeMirror-scrollbar-filler,.CodeMirror-vscrollbar{position:absolute;z-index:6;display:none}.CodeMirror-vscrollbar{right:0;top:0;overflow-x:hidden;overflow-y:scroll}.CodeMirror-hscrollbar{bottom:0;left:0;overflow-y:hidden;overflow-x:scroll}.CodeMirror-scrollbar-filler{right:0;bottom:0}.CodeMirror-gutter-filler{left:0;bottom:0}.CodeMirror-gutters{position:absolute;left:0;top:0;min-height:100%;z-index:3}.CodeMirror-gutter{white-space:normal;height:100%;display:inline-block;vertical-align:top;margin-bottom:-30px}.CodeMirror-gutter-wrapper{position:absolute;z-index:4;background:0 0!important;border:none!important}.CodeMirror-gutter-background{position:absolute;top:0;bottom:0;z-index:4}.CodeMirror-gutter-elt{position:absolute;cursor:default;z-index:4}.CodeMirror-gutter-wrapper ::selection{background-color:transparent}.CodeMirror-gutter-wrapper ::-moz-selection{background-color:transparent}.CodeMirror-lines{cursor:text;min-height:1px}.CodeMirror pre{-moz-border-radius:0;-webkit-border-radius:0;border-radius:0;border-width:0;background:0 0;font-family:inherit;font-size:inherit;margin:0;white-space:pre;word-wrap:normal;line-height:inherit;color:inherit;z-index:2;position:relative;overflow:visible;-webkit-tap-highlight-color:transparent;-webkit-font-variant-ligatures:contextual;font-variant-ligatures:contextual}.CodeMirror-wrap pre{word-wrap:break-word;white-space:pre-wrap;word-break:normal}.CodeMirror-linebackground{position:absolute;left:0;right:0;top:0;bottom:0;z-index:0}.CodeMirror-linewidget{position:relative;z-index:2;overflow:auto}.CodeMirror-rtl pre{direction:rtl}.CodeMirror-code{outline:0}.CodeMirror-gutter,.CodeMirror-gutters,.CodeMirror-linenumber,.CodeMirror-scroll,.CodeMirror-sizer{-moz-box-sizing:content-box;box-sizing:content-box}.CodeMirror-measure{position:absolute;width:100%;height:0;overflow:hidden;visibility:hidden}.CodeMirror-cursor{position:absolute;pointer-events:none}.CodeMirror-measure pre{position:static}div.CodeMirror-cursors{visibility:hidden;position:relative;z-index:3}div.CodeMirror-dragcursors{visibility:visible}.CodeMirror-focused div.CodeMirror-cursors{visibility:visible}.CodeMirror-selected{background:#d9d9d9}.CodeMirror-focused .CodeMirror-selected{background:#d7d4f0}.CodeMirror-crosshair{cursor:crosshair}.CodeMirror-line::selection,.CodeMirror-line>span::selection,.CodeMirror-line>span>span::selection{background:#d7d4f0}.CodeMirror-line::-moz-selection,.CodeMirror-line>span::-moz-selection,.CodeMirror-line>span>span::-moz-selection{background:#d7d4f0}.cm-searching{background-color:#ffa;background-color:rgba(255,255,0,.4)}.cm-force-border{padding-right:.1px}@media print{.CodeMirror div.CodeMirror-cursors{visibility:hidden}}.cm-tab-wrap-hack:after{content:''}span.CodeMirror-selectedtext{background:0 0}.contentview .header{font-weight:700}.contentview .highlight{font-weight:700}.contentview .offset{color:#00f}.contentview .codeeditor{margin-bottom:12px}.modal-visible{display:block}.modal-dialog{overflow-y:initial!important}.modal-body{max-height:calc(100vh - 20px);overflow-y:auto} +html{box-sizing:border-box}*,:after,:before{box-sizing:inherit}.resource-icon{width:32px;height:32px}.resource-icon-css{background-image:url(images/chrome-devtools/resourceCSSIcon.png)}.resource-icon-document{background-image:url(images/chrome-devtools/resourceDocumentIcon.png)}.resource-icon-js{background-image:url(images/chrome-devtools/resourceJSIcon.png)}.resource-icon-plain{background-image:url(images/chrome-devtools/resourcePlainIcon.png)}.resource-icon-executable{background-image:url(images/resourceExecutableIcon.png)}.resource-icon-flash{background-image:url(images/resourceFlashIcon.png)}.resource-icon-image{background-image:url(images/resourceImageIcon.png)}.resource-icon-java{background-image:url(images/resourceJavaIcon.png)}.resource-icon-not-modified{background-image:url(images/resourceNotModifiedIcon.png)}.resource-icon-redirect{background-image:url(images/resourceRedirectIcon.png)}#container,#mitmproxy,body,html{height:100%;margin:0;overflow:hidden}#container{display:flex;flex-direction:column;outline:0}#container>.eventlog,#container>footer,#container>header{flex:0 0 auto}.main-view{flex:1 1 auto;height:0;display:flex;flex-direction:row}.main-view.vertical{flex-direction:column}.main-view .flow-detail,.main-view .flow-table{flex:1 1 auto}.splitter{flex:0 0 1px;background-color:#aaa;position:relative}.splitter>div{position:absolute}.splitter.splitter-x{cursor:col-resize}.splitter.splitter-x>div{margin-left:-1px;width:4px;height:100%}.splitter.splitter-y{cursor:row-resize}.splitter.splitter-y>div{margin-top:-1px;height:4px;width:100%}.nav-tabs{border-bottom:solid #a6a6a6 1px}.nav-tabs a{display:inline-block;border:solid transparent 1px;text-decoration:none}.nav-tabs a.active{background-color:#fff;border-color:#a6a6a6;border-bottom-color:#fff}.nav-tabs a.special{color:#fff;background-color:#396cad;border-bottom-color:#396cad}.nav-tabs a.special:hover{background-color:#5386c6}.nav-tabs-lg a{padding:3px 14px;margin:0 2px -1px}.nav-tabs-sm a{padding:0 7px;margin:2px 2px -1px}.nav-tabs-sm a.nav-action{float:right;padding:0;margin:1px 0 0}header{padding-top:6px;background-color:#fff}header>div{display:block;margin:0;padding:0;border-bottom:solid #a6a6a6 1px;height:85px;overflow:visible}.menu-group{margin:0 3px;display:inline-block;height:85px;vertical-align:top}.menu-content{height:69px;text-align:center}.menu-content>.btn{height:69px;text-align:center;margin:0 1px;padding:12px 5px;border:none;border-radius:0}.menu-content>.btn i{font-size:20px;display:block;margin:0 auto 5px}.menu-entry{text-align:left;height:23px;line-height:1;padding:.5rem 1rem}.menu-entry label{font-size:1.2rem;font-weight:400;margin:0}.menu-entry input[type=checkbox]{margin:0 2px;vertical-align:middle}.menu-legend{height:16px;text-align:center;font-size:12px;padding:0 5px}.menu-group+.menu-group:before{margin-left:-3px;content:" ";border-left:solid 1px #e6e6e6;margin-top:10px;height:65px;position:absolute}.menu-main{margin-left:-2px;margin-right:-3px;padding:2px 5px}.filter-input{position:relative;min-height:1px;padding-left:2.5px;padding-right:2.5px;padding:2.5px}@media (min-width:768px){.filter-input{float:left;width:41.66666667%}}@media (max-width:767px){.filter-input{padding:2px 2.5px}.filter-input>.form-control,.filter-input>.input-group-addon,.filter-input>.input-group-btn>.btn{height:23.5px;padding:1px 5px;font-size:12px;line-height:1.5}}.filter-input .popover{top:27px;left:43px;display:block;max-width:none;opacity:.9}@media (max-width:767px){.filter-input .popover{top:16px;left:29px;right:2px}}.filter-input .popover .popover-content{max-height:500px;overflow-y:auto}.filter-input .popover .popover-content tr{cursor:pointer}.filter-input .popover .popover-content tr:hover{background-color:hsla(209,52%,84%,.5)!important}.connection-indicator{display:inline;padding:.2em .6em .3em;font-size:75%;font-weight:700;line-height:1;color:#fff;text-align:center;white-space:nowrap;vertical-align:baseline;border-radius:.25em;float:right;margin:5px;opacity:1;transition:all 1s linear}a.connection-indicator:focus,a.connection-indicator:hover{color:#fff;text-decoration:none;cursor:pointer}.connection-indicator:empty{display:none}.btn .connection-indicator{position:relative;top:-1px}.connection-indicator.fetching,.connection-indicator.init{background-color:#5bc0de}.connection-indicator.established{background-color:#5cb85c;opacity:0}.connection-indicator.error{background-color:#d9534f;transition:all .2s linear}.connection-indicator.offline{background-color:#f0ad4e;opacity:1}.flow-table{width:100%;overflow-y:scroll;overflow-x:hidden}.flow-table table{width:100%;table-layout:fixed}.flow-table thead{background-color:#f2f2f2;line-height:23px}.flow-table th{font-weight:400;box-shadow:0 1px 0 #a6a6a6;position:relative!important;padding-left:1px;-webkit-touch-callout:none;-webkit-user-select:none;-khtml-user-select:none;-moz-user-select:none;-ms-user-select:none;user-select:none}.flow-table th.sort-asc,.flow-table th.sort-desc{background-color:#fafafa}.flow-table th.sort-asc:after,.flow-table th.sort-desc:after{font:normal normal normal 14px/1 FontAwesome;position:absolute;right:3px;top:3px;padding:2px;background-color:rgba(250,250,250,.8)}.flow-table th.sort-asc:after{content:"\f0de"}.flow-table th.sort-desc:after{content:"\f0dd"}.flow-table tr{cursor:pointer}.flow-table tr:nth-child(even){background-color:rgba(0,0,0,.05)}.flow-table tr.selected{background-color:hsla(209,52%,84%,.5)!important}.flow-table tr.highlighted{background-color:hsla(48,100%,50%,.4)}.flow-table tr.highlighted:nth-child(even){background-color:hsla(48,100%,50%,.5)}.flow-table td{overflow:hidden;white-space:nowrap;text-overflow:ellipsis}.flow-table tr.intercepted:not(.has-response) .col-method,.flow-table tr.intercepted:not(.has-response) .col-path{color:#ff7f00}.flow-table tr.intercepted.has-response .col-size,.flow-table tr.intercepted.has-response .col-status,.flow-table tr.intercepted.has-response .col-time{color:#ff7f00}.flow-table .fa{line-height:inherit}.flow-table .fa.pull-right{margin-left:0}.flow-table .col-tls{width:10px}.flow-table .col-tls-https{background-color:rgba(0,185,0,.5)}.flow-table .col-icon{width:32px}.flow-table .col-path .fa-repeat{color:green}.flow-table .col-path .fa-pause{color:#ff7f00}.flow-table .col-path .fa-exclamation,.flow-table .col-path .fa-times{color:#8b0000}.flow-table .col-method{width:60px}.flow-table .col-status{width:50px}.flow-table .col-size{width:70px}.flow-table .col-time{width:50px}.flow-table td.col-size,.flow-table td.col-time{text-align:right}.flow-detail{width:100%;overflow:hidden;display:flex;flex-direction:column}.flow-detail nav{background-color:#f2f2f2}.flow-detail section{overflow-y:scroll}.flow-detail section>article{overflow:auto;padding:5px 12px 0}.flow-detail section>footer{box-shadow:0 0 3px gray;padding:2px;margin:0;height:23px}.flow-detail section.detail,.flow-detail section.error{overflow:auto;padding:5px 12px 0}.flow-detail .first-line{font-family:Menlo,Monaco,Consolas,"Courier New",monospace;background-color:#428bca;color:#fff;margin:0 -8px;padding:4px 8px;border-radius:5px;word-break:break-all;max-height:100px;overflow-y:auto}.flow-detail .first-line .inline-input.editable{border-color:rgba(255,255,255,.5)}.flow-detail .request-line{margin-bottom:2px}.flow-detail hr{margin:0 0 5px}.inline-input{display:inline;margin:0 -3px;padding:0 3px;border:solid transparent 1px}.inline-input.editable{border-color:#ccc}.inline-input[contenteditable]{background-color:rgba(255,255,255,.2)}.inline-input[contenteditable].has-warning{color:#ffb8b8}.view-all-content-btn{float:right;margin-bottom:12px}.flow-detail table{font-family:Menlo,Monaco,Consolas,"Courier New",monospace;width:100%;table-layout:fixed;word-break:break-all}.flow-detail table tr:not(:first-child){border-top:1px solid #f7f7f7}.flow-detail table td{vertical-align:top}.connection-table td:first-child{width:50%;padding-right:1em}.header-table td{line-height:1.3em}.header-table .header-name{width:33%}.header-table .header-colon{position:absolute;opacity:0}.header-table .inline-input{display:inline-block;width:100%;height:100%}.connection-table td,.timing-table td{overflow:hidden;text-overflow:ellipsis;white-space:nowrap}.flowview-image{text-align:center}.flowview-image img{max-width:100%;max-height:100%}.edit-flow-container{position:fixed;right:20px}.edit-flow{cursor:pointer;position:absolute;right:0;top:5px;height:40px;width:40px;border-radius:20px;z-index:10000;background-color:rgba(255,255,255,.7);border:solid 2px rgba(248,145,59,.7);text-align:center;font-size:22px;line-height:37px;transition:all .1s ease-in-out}.edit-flow:hover{background-color:rgba(239,108,0,.7);color:rgba(0,0,0,.8);border:solid 2px transparent}.eventlog{height:200px;flex:0 0 auto;display:flex;flex-direction:column}.eventlog>div{background-color:#f2f2f2;padding:0 5px;flex:0 0 auto;border-top:1px solid #aaa;cursor:row-resize}.eventlog>pre{flex:1 1 auto;margin:0;border-radius:0;overflow-x:auto;overflow-y:scroll;background-color:#fcfcfc}.eventlog .fa-close{cursor:pointer;float:right;color:grey;padding:3px 0;padding-left:10px}.eventlog .fa-close:hover{color:#000}.eventlog .btn-toggle{margin-top:-2px;margin-left:3px;padding:2px 2px;font-size:10px;line-height:10px;border-radius:2px}.eventlog .label{cursor:pointer;vertical-align:middle;display:inline-block;margin-top:-2px;margin-left:3px}footer{box-shadow:0 -1px 3px #d3d3d3;padding:0 10px 3px}footer .label{margin-right:3px}.CodeMirror{border:1px solid #ccc;height:auto!important}.CodeMirror{font-family:monospace;height:300px;color:#000;direction:ltr}.CodeMirror-lines{padding:4px 0}.CodeMirror pre{padding:0 4px}.CodeMirror-gutter-filler,.CodeMirror-scrollbar-filler{background-color:#fff}.CodeMirror-gutters{border-right:1px solid #ddd;background-color:#f7f7f7;white-space:nowrap}.CodeMirror-linenumber{padding:0 3px 0 5px;min-width:20px;text-align:right;color:#999;white-space:nowrap}.CodeMirror-guttermarker{color:#000}.CodeMirror-guttermarker-subtle{color:#999}.CodeMirror-cursor{border-left:1px solid #000;border-right:none;width:0}.CodeMirror div.CodeMirror-secondarycursor{border-left:1px solid silver}.cm-fat-cursor .CodeMirror-cursor{width:auto;border:0!important;background:#7e7}.cm-fat-cursor div.CodeMirror-cursors{z-index:1}.cm-fat-cursor-mark{background-color:rgba(20,255,20,.5);-webkit-animation:blink 1.06s steps(1) infinite;-moz-animation:blink 1.06s steps(1) infinite;animation:blink 1.06s steps(1) infinite}.cm-animate-fat-cursor{width:auto;border:0;-webkit-animation:blink 1.06s steps(1) infinite;-moz-animation:blink 1.06s steps(1) infinite;animation:blink 1.06s steps(1) infinite;background-color:#7e7}@-moz-keyframes blink{50%{background-color:transparent}}@-webkit-keyframes blink{50%{background-color:transparent}}@keyframes blink{50%{background-color:transparent}}.cm-tab{display:inline-block;text-decoration:inherit}.CodeMirror-rulers{position:absolute;left:0;right:0;top:-50px;bottom:-20px;overflow:hidden}.CodeMirror-ruler{border-left:1px solid #ccc;top:0;bottom:0;position:absolute}.cm-s-default .cm-header{color:#00f}.cm-s-default .cm-quote{color:#090}.cm-negative{color:#d44}.cm-positive{color:#292}.cm-header,.cm-strong{font-weight:700}.cm-em{font-style:italic}.cm-link{text-decoration:underline}.cm-strikethrough{text-decoration:line-through}.cm-s-default .cm-keyword{color:#708}.cm-s-default .cm-atom{color:#219}.cm-s-default .cm-number{color:#164}.cm-s-default .cm-def{color:#00f}.cm-s-default .cm-variable-2{color:#05a}.cm-s-default .cm-type,.cm-s-default .cm-variable-3{color:#085}.cm-s-default .cm-comment{color:#a50}.cm-s-default .cm-string{color:#a11}.cm-s-default .cm-string-2{color:#f50}.cm-s-default .cm-meta{color:#555}.cm-s-default .cm-qualifier{color:#555}.cm-s-default .cm-builtin{color:#30a}.cm-s-default .cm-bracket{color:#997}.cm-s-default .cm-tag{color:#170}.cm-s-default .cm-attribute{color:#00c}.cm-s-default .cm-hr{color:#999}.cm-s-default .cm-link{color:#00c}.cm-s-default .cm-error{color:red}.cm-invalidchar{color:red}.CodeMirror-composing{border-bottom:2px solid}div.CodeMirror span.CodeMirror-matchingbracket{color:#0f0}div.CodeMirror span.CodeMirror-nonmatchingbracket{color:#f22}.CodeMirror-matchingtag{background:rgba(255,150,0,.3)}.CodeMirror-activeline-background{background:#e8f2ff}.CodeMirror{position:relative;overflow:hidden;background:#fff}.CodeMirror-scroll{overflow:scroll!important;margin-bottom:-30px;margin-right:-30px;padding-bottom:30px;height:100%;outline:0;position:relative}.CodeMirror-sizer{position:relative;border-right:30px solid transparent}.CodeMirror-gutter-filler,.CodeMirror-hscrollbar,.CodeMirror-scrollbar-filler,.CodeMirror-vscrollbar{position:absolute;z-index:6;display:none}.CodeMirror-vscrollbar{right:0;top:0;overflow-x:hidden;overflow-y:scroll}.CodeMirror-hscrollbar{bottom:0;left:0;overflow-y:hidden;overflow-x:scroll}.CodeMirror-scrollbar-filler{right:0;bottom:0}.CodeMirror-gutter-filler{left:0;bottom:0}.CodeMirror-gutters{position:absolute;left:0;top:0;min-height:100%;z-index:3}.CodeMirror-gutter{white-space:normal;height:100%;display:inline-block;vertical-align:top;margin-bottom:-30px}.CodeMirror-gutter-wrapper{position:absolute;z-index:4;background:0 0!important;border:none!important}.CodeMirror-gutter-background{position:absolute;top:0;bottom:0;z-index:4}.CodeMirror-gutter-elt{position:absolute;cursor:default;z-index:4}.CodeMirror-gutter-wrapper ::selection{background-color:transparent}.CodeMirror-gutter-wrapper ::-moz-selection{background-color:transparent}.CodeMirror-lines{cursor:text;min-height:1px}.CodeMirror pre{-moz-border-radius:0;-webkit-border-radius:0;border-radius:0;border-width:0;background:0 0;font-family:inherit;font-size:inherit;margin:0;white-space:pre;word-wrap:normal;line-height:inherit;color:inherit;z-index:2;position:relative;overflow:visible;-webkit-tap-highlight-color:transparent;-webkit-font-variant-ligatures:contextual;font-variant-ligatures:contextual}.CodeMirror-wrap pre{word-wrap:break-word;white-space:pre-wrap;word-break:normal}.CodeMirror-linebackground{position:absolute;left:0;right:0;top:0;bottom:0;z-index:0}.CodeMirror-linewidget{position:relative;z-index:2;overflow:auto}.CodeMirror-rtl pre{direction:rtl}.CodeMirror-code{outline:0}.CodeMirror-gutter,.CodeMirror-gutters,.CodeMirror-linenumber,.CodeMirror-scroll,.CodeMirror-sizer{-moz-box-sizing:content-box;box-sizing:content-box}.CodeMirror-measure{position:absolute;width:100%;height:0;overflow:hidden;visibility:hidden}.CodeMirror-cursor{position:absolute;pointer-events:none}.CodeMirror-measure pre{position:static}div.CodeMirror-cursors{visibility:hidden;position:relative;z-index:3}div.CodeMirror-dragcursors{visibility:visible}.CodeMirror-focused div.CodeMirror-cursors{visibility:visible}.CodeMirror-selected{background:#d9d9d9}.CodeMirror-focused .CodeMirror-selected{background:#d7d4f0}.CodeMirror-crosshair{cursor:crosshair}.CodeMirror-line::selection,.CodeMirror-line>span::selection,.CodeMirror-line>span>span::selection{background:#d7d4f0}.CodeMirror-line::-moz-selection,.CodeMirror-line>span::-moz-selection,.CodeMirror-line>span>span::-moz-selection{background:#d7d4f0}.cm-searching{background-color:#ffa;background-color:rgba(255,255,0,.4)}.cm-force-border{padding-right:.1px}@media print{.CodeMirror div.CodeMirror-cursors{visibility:hidden}}.cm-tab-wrap-hack:after{content:''}span.CodeMirror-selectedtext{background:0 0}.contentview .header{font-weight:700}.contentview .highlight{font-weight:700}.contentview .offset{color:#00f}.contentview .codeeditor{margin-bottom:12px}.modal-visible{display:block}.modal-dialog{overflow-y:initial!important}.modal-body{max-height:calc(100vh - 200px);overflow-y:auto} /*# sourceMappingURL=app.css.map */ diff -Nru mitmproxy-5.1.1/mitmproxy/tools/web/static/app.js mitmproxy-6.0.2/mitmproxy/tools/web/static/app.js --- mitmproxy-5.1.1/mitmproxy/tools/web/static/app.js 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/tools/web/static/app.js 2020-12-15 16:41:27.000000000 +0000 @@ -47,7 +47,7 @@ "use strict";function _interopRequireWildcard(e){if(e&&e.__esModule)return e;var t={};if(null!=e)for(var o in e)Object.prototype.hasOwnProperty.call(e,o)&&(t[o]=e[o]);return t.default=e,t}function _interopRequireDefault(e){return e&&e.__esModule?e:{default:e}}function _classCallCheck(e,t){if(!(e instanceof t))throw new TypeError("Cannot call a class as a function")}function _possibleConstructorReturn(e,t){if(!e)throw new ReferenceError("this hasn't been initialised - super() hasn't been called");return!t||"object"!=typeof t&&"function"!=typeof t?e:t}function _inherits(e,t){if("function"!=typeof t&&null!==t)throw new TypeError("Super expression must either be null or a function, not "+typeof t);e.prototype=Object.create(t&&t.prototype,{constructor:{value:e,enumerable:!1,writable:!0,configurable:!0}}),t&&(Object.setPrototypeOf?Object.setPrototypeOf(e,t):e.__proto__=t)}Object.defineProperty(exports,"__esModule",{value:!0}),exports.PureFlowTable=void 0;var _createClass=function(){function e(e,t){for(var o=0;os+c&&(r.scrollTop=u-c)}}},{key:"componentWillReceiveProps",value:function(e){e.selected&&e.selected!==this.props.selected&&(this.shouldScrollIntoView=!0)}},{key:"onViewportUpdate",value:function(){var e=_reactDom2.default.findDOMNode(this),t=e.scrollTop,o=(0,_VirtualScroll.calcVScroll)({viewportTop:t,viewportHeight:e.offsetHeight,itemCount:this.props.flows.length,rowHeight:this.props.rowHeight});this.state.viewportTop===t&&(0,_shallowequal2.default)(this.state.vScroll,o)||this.setState({vScroll:o,viewportTop:t})}},{key:"render",value:function(){var e=this,t=this.state,o=t.vScroll,l=t.viewportTop,r=this.props,i=r.flows,a=r.selected,n=r.highlight,u=n?_filt2.default.parse(n):function(){return!1};return _react2.default.createElement("div",{className:"flow-table",onScroll:this.onViewportUpdate},_react2.default.createElement("table",null,_react2.default.createElement("thead",{ref:"head",style:{transform:"translateY("+l+"px)"}},_react2.default.createElement(_FlowTableHead2.default,null)),_react2.default.createElement("tbody",null,_react2.default.createElement("tr",{style:{height:o.paddingTop}}),i.slice(o.start,o.end).map(function(t){return _react2.default.createElement(_FlowRow2.default,{key:t.id,flow:t,selected:t===a,highlighted:u(t),onSelect:e.props.selectFlow})}),_react2.default.createElement("tr",{style:{height:o.paddingBottom}}))))}}]),t}();FlowTable.propTypes={selectFlow:_propTypes2.default.func.isRequired,flows:_propTypes2.default.array.isRequired,rowHeight:_propTypes2.default.number,highlight:_propTypes2.default.string,selected:_propTypes2.default.object},FlowTable.defaultProps={rowHeight:32};var PureFlowTable=exports.PureFlowTable=(0,_AutoScroll2.default)(FlowTable);exports.default=(0,_reactRedux.connect)(function(e){return{flows:e.flows.view,highlight:e.flows.highlight,selected:e.flows.byId[e.flows.selected[0]]}},{selectFlow:flowsActions.select})(PureFlowTable); },{"../ducks/flows":56,"../filt/filt":67,"./FlowTable/FlowRow":18,"./FlowTable/FlowTableHead":19,"./helpers/AutoScroll":52,"./helpers/VirtualScroll":53,"prop-types":"prop-types","react":"react","react-dom":"react-dom","react-redux":"react-redux","shallowequal":"shallowequal"}],17:[function(require,module,exports){ -"use strict";function _interopRequireDefault(e){return e&&e.__esModule?e:{default:e}}function TLSColumn(e){var t=e.flow;return _react2.default.createElement("td",{className:(0,_classnames2.default)("col-tls","https"===t.request.scheme?"col-tls-https":"col-tls-http")})}function IconColumn(e){var t=e.flow;return _react2.default.createElement("td",{className:"col-icon"},_react2.default.createElement("div",{className:(0,_classnames2.default)("resource-icon",IconColumn.getIcon(t))}))}function PathColumn(e){var t=e.flow,s=void 0;return t.error&&(s="Connection killed"===t.error.msg?_react2.default.createElement("i",{className:"fa fa-fw fa-times pull-right"}):_react2.default.createElement("i",{className:"fa fa-fw fa-exclamation pull-right"})),_react2.default.createElement("td",{className:"col-path"},t.request.is_replay&&_react2.default.createElement("i",{className:"fa fa-fw fa-repeat pull-right"}),t.intercepted&&_react2.default.createElement("i",{className:"fa fa-fw fa-pause pull-right"}),s,_utils.RequestUtils.pretty_url(t.request))}function MethodColumn(e){var t=e.flow;return _react2.default.createElement("td",{className:"col-method"},t.request.method)}function StatusColumn(e){var t=e.flow,s="darkred";return t.response&&100<=t.response.status_code&&t.response.status_code<200?s="green":t.response&&200<=t.response.status_code&&t.response.status_code<300?s="darkgreen":t.response&&300<=t.response.status_code&&t.response.status_code<400?s="lightblue":t.response&&400<=t.response.status_code&&t.response.status_code<500?s="lightred":t.response&&500<=t.response.status_code&&t.response.status_code<600&&(s="lightred"),_react2.default.createElement("td",{className:"col-status",style:{color:s}},t.response&&t.response.status_code)}function SizeColumn(e){var t=e.flow;return _react2.default.createElement("td",{className:"col-size"},(0,_utils2.formatSize)(SizeColumn.getTotalSize(t)))}function TimeColumn(e){var t=e.flow;return _react2.default.createElement("td",{className:"col-time"},t.response?(0,_utils2.formatTimeDelta)(1e3*(t.response.timestamp_end-t.request.timestamp_start)):"...")}Object.defineProperty(exports,"__esModule",{value:!0}),exports.TLSColumn=TLSColumn,exports.IconColumn=IconColumn,exports.PathColumn=PathColumn,exports.MethodColumn=MethodColumn,exports.StatusColumn=StatusColumn,exports.SizeColumn=SizeColumn,exports.TimeColumn=TimeColumn;var _react=require("react"),_react2=_interopRequireDefault(_react),_classnames=require("classnames"),_classnames2=_interopRequireDefault(_classnames),_utils=require("../../flow/utils.js"),_utils2=require("../../utils.js");TLSColumn.headerClass="col-tls",TLSColumn.headerName="",IconColumn.headerClass="col-icon",IconColumn.headerName="",IconColumn.getIcon=function(e){if(!e.response)return"resource-icon-plain";var t=_utils.ResponseUtils.getContentType(e.response)||"";return 304===e.response.status_code?"resource-icon-not-modified":300<=e.response.status_code&&e.response.status_code<400?"resource-icon-redirect":t.indexOf("image")>=0?"resource-icon-image":t.indexOf("javascript")>=0?"resource-icon-js":t.indexOf("css")>=0?"resource-icon-css":t.indexOf("html")>=0?"resource-icon-document":"resource-icon-plain"},PathColumn.headerClass="col-path",PathColumn.headerName="Path",MethodColumn.headerClass="col-method",MethodColumn.headerName="Method",StatusColumn.headerClass="col-status",StatusColumn.headerName="Status",SizeColumn.getTotalSize=function(e){var t=e.request.contentLength;return e.response&&(t+=e.response.contentLength||0),t},SizeColumn.headerClass="col-size",SizeColumn.headerName="Size",TimeColumn.headerClass="col-time",TimeColumn.headerName="Time",exports.default=[TLSColumn,IconColumn,PathColumn,MethodColumn,StatusColumn,SizeColumn,TimeColumn]; +"use strict";function _interopRequireDefault(e){return e&&e.__esModule?e:{default:e}}function TLSColumn(e){var t=e.flow;return _react2.default.createElement("td",{className:(0,_classnames2.default)("col-tls","https"===t.request.scheme?"col-tls-https":"col-tls-http")})}function IconColumn(e){var t=e.flow;return _react2.default.createElement("td",{className:"col-icon"},_react2.default.createElement("div",{className:(0,_classnames2.default)("resource-icon",IconColumn.getIcon(t))}))}function PathColumn(e){var t=e.flow,s=void 0;return t.error&&(s="Connection killed."===t.error.msg?_react2.default.createElement("i",{className:"fa fa-fw fa-times pull-right"}):_react2.default.createElement("i",{className:"fa fa-fw fa-exclamation pull-right"})),_react2.default.createElement("td",{className:"col-path"},t.request.is_replay&&_react2.default.createElement("i",{className:"fa fa-fw fa-repeat pull-right"}),t.intercepted&&_react2.default.createElement("i",{className:"fa fa-fw fa-pause pull-right"}),s,_utils.RequestUtils.pretty_url(t.request))}function MethodColumn(e){var t=e.flow;return _react2.default.createElement("td",{className:"col-method"},t.request.method)}function StatusColumn(e){var t=e.flow,s="darkred";return t.response&&100<=t.response.status_code&&t.response.status_code<200?s="green":t.response&&200<=t.response.status_code&&t.response.status_code<300?s="darkgreen":t.response&&300<=t.response.status_code&&t.response.status_code<400?s="lightblue":t.response&&400<=t.response.status_code&&t.response.status_code<500?s="lightred":t.response&&500<=t.response.status_code&&t.response.status_code<600&&(s="lightred"),_react2.default.createElement("td",{className:"col-status",style:{color:s}},t.response&&t.response.status_code)}function SizeColumn(e){var t=e.flow;return _react2.default.createElement("td",{className:"col-size"},(0,_utils2.formatSize)(SizeColumn.getTotalSize(t)))}function TimeColumn(e){var t=e.flow;return _react2.default.createElement("td",{className:"col-time"},t.response?(0,_utils2.formatTimeDelta)(1e3*(t.response.timestamp_end-t.request.timestamp_start)):"...")}Object.defineProperty(exports,"__esModule",{value:!0}),exports.TLSColumn=TLSColumn,exports.IconColumn=IconColumn,exports.PathColumn=PathColumn,exports.MethodColumn=MethodColumn,exports.StatusColumn=StatusColumn,exports.SizeColumn=SizeColumn,exports.TimeColumn=TimeColumn;var _react=require("react"),_react2=_interopRequireDefault(_react),_classnames=require("classnames"),_classnames2=_interopRequireDefault(_classnames),_utils=require("../../flow/utils.js"),_utils2=require("../../utils.js");TLSColumn.headerClass="col-tls",TLSColumn.headerName="",IconColumn.headerClass="col-icon",IconColumn.headerName="",IconColumn.getIcon=function(e){if(!e.response)return"resource-icon-plain";var t=_utils.ResponseUtils.getContentType(e.response)||"";return 304===e.response.status_code?"resource-icon-not-modified":300<=e.response.status_code&&e.response.status_code<400?"resource-icon-redirect":t.indexOf("image")>=0?"resource-icon-image":t.indexOf("javascript")>=0?"resource-icon-js":t.indexOf("css")>=0?"resource-icon-css":t.indexOf("html")>=0?"resource-icon-document":"resource-icon-plain"},PathColumn.headerClass="col-path",PathColumn.headerName="Path",MethodColumn.headerClass="col-method",MethodColumn.headerName="Method",StatusColumn.headerClass="col-status",StatusColumn.headerName="Status",SizeColumn.getTotalSize=function(e){var t=e.request.contentLength;return e.response&&(t+=e.response.contentLength||0),t},SizeColumn.headerClass="col-size",SizeColumn.headerName="Size",TimeColumn.headerClass="col-time",TimeColumn.headerName="Time",exports.default=[TLSColumn,IconColumn,PathColumn,MethodColumn,StatusColumn,SizeColumn,TimeColumn]; },{"../../flow/utils.js":68,"../../utils.js":70,"classnames":"classnames","react":"react"}],18:[function(require,module,exports){ "use strict";function _interopRequireDefault(e){return e&&e.__esModule?e:{default:e}}function FlowRow(e){var t=e.flow,r=e.selected,l=e.highlighted,s=e.onSelect,o=(0,_classnames2.default)({selected:r,highlighted:l,intercepted:t.intercepted,"has-request":t.request,"has-response":t.response});return _react2.default.createElement("tr",{className:o,onClick:function(){return s(t.id)}},_FlowColumns2.default.map(function(e){return _react2.default.createElement(e,{key:e.name,flow:t})}))}Object.defineProperty(exports,"__esModule",{value:!0});var _react=require("react"),_react2=_interopRequireDefault(_react),_propTypes=require("prop-types"),_propTypes2=_interopRequireDefault(_propTypes),_classnames=require("classnames"),_classnames2=_interopRequireDefault(_classnames),_FlowColumns=require("./FlowColumns"),_FlowColumns2=_interopRequireDefault(_FlowColumns),_utils=require("../../utils");FlowRow.propTypes={onSelect:_propTypes2.default.func.isRequired,flow:_propTypes2.default.object.isRequired,highlighted:_propTypes2.default.bool,selected:_propTypes2.default.bool},exports.default=(0,_utils.pure)(FlowRow); @@ -59,13 +59,13 @@ "use strict";function _interopRequireDefault(e){return e&&e.__esModule?e:{default:e}}function FlowView(e){var r=e.flow,a=e.tabName,t=e.selectTab,s=["request","response","error"].filter(function(e){return r[e]});s.push("details"),s.indexOf(a)<0&&(a="response"===a&&r.error?"error":"error"===a&&r.response?"response":s[0]);var l=allTabs[_lodash2.default.capitalize(a)];return _react2.default.createElement("div",{className:"flow-detail"},_react2.default.createElement(_Nav2.default,{tabs:s,active:a,onSelectTab:t}),_react2.default.createElement(l,{flow:r}))}Object.defineProperty(exports,"__esModule",{value:!0}),exports.allTabs=void 0;var _react=require("react"),_react2=_interopRequireDefault(_react),_reactRedux=require("react-redux"),_lodash=require("lodash"),_lodash2=_interopRequireDefault(_lodash),_Nav=require("./FlowView/Nav"),_Nav2=_interopRequireDefault(_Nav),_Messages=require("./FlowView/Messages"),_Details=require("./FlowView/Details"),_Details2=_interopRequireDefault(_Details),_flow=require("../ducks/ui/flow"),allTabs=exports.allTabs={Request:_Messages.Request,Response:_Messages.Response,Error:_Messages.ErrorView,Details:_Details2.default};exports.default=(0,_reactRedux.connect)(function(e){return{flow:e.flows.byId[e.flows.selected[0]],tabName:e.ui.flow.tab}},{selectTab:_flow.selectTab})(FlowView); },{"../ducks/ui/flow":60,"./FlowView/Details":21,"./FlowView/Messages":23,"./FlowView/Nav":24,"lodash":"lodash","react":"react","react-redux":"react-redux"}],21:[function(require,module,exports){ -"use strict";function _interopRequireDefault(e){return e&&e.__esModule?e:{default:e}}function TimeStamp(e){var t=e.t,a=e.deltaTo,n=e.title;return t?_react2.default.createElement("tr",null,_react2.default.createElement("td",null,n,":"),_react2.default.createElement("td",null,(0,_utils.formatTimeStamp)(t),a&&_react2.default.createElement("span",{className:"text-muted"},"(",(0,_utils.formatTimeDelta)(1e3*(t-a)),")"))):_react2.default.createElement("tr",null)}function ConnectionInfo(e){var t=e.conn;return _react2.default.createElement("table",{className:"connection-table"},_react2.default.createElement("tbody",null,_react2.default.createElement("tr",{key:"address"},_react2.default.createElement("td",null,"Address:"),_react2.default.createElement("td",null,t.address.join(":"))),t.sni&&_react2.default.createElement("tr",{key:"sni"},_react2.default.createElement("td",null,_react2.default.createElement("abbr",{title:"TLS Server Name Indication"},"TLS SNI:")),_react2.default.createElement("td",null,t.sni))))}function CertificateInfo(e){var t=e.flow;return _react2.default.createElement("div",null,t.client_conn.cert&&[_react2.default.createElement("h4",{key:"name"},"Client Certificate"),_react2.default.createElement("pre",{key:"value",style:{maxHeight:100}},t.client_conn.cert)],t.server_conn.cert&&[_react2.default.createElement("h4",{key:"name"},"Server Certificate"),_react2.default.createElement("pre",{key:"value",style:{maxHeight:100}},t.server_conn.cert)])}function Timing(e){var t=e.flow,a=t.server_conn,n=t.client_conn,r=t.request,l=t.response,c=[{title:"Server conn. initiated",t:a.timestamp_start,deltaTo:r.timestamp_start},{title:"Server conn. TCP handshake",t:a.timestamp_tcp_setup,deltaTo:r.timestamp_start},{title:"Server conn. SSL handshake",t:a.timestamp_ssl_setup,deltaTo:r.timestamp_start},{title:"Client conn. established",t:n.timestamp_start,deltaTo:r.timestamp_start},{title:"Client conn. SSL handshake",t:n.timestamp_ssl_setup,deltaTo:r.timestamp_start},{title:"First request byte",t:r.timestamp_start},{title:"Request complete",t:r.timestamp_end,deltaTo:r.timestamp_start},l&&{title:"First response byte",t:l.timestamp_start,deltaTo:r.timestamp_start},l&&{title:"Response complete",t:l.timestamp_end,deltaTo:r.timestamp_start}];return _react2.default.createElement("div",null,_react2.default.createElement("h4",null,"Timing"),_react2.default.createElement("table",{className:"timing-table"},_react2.default.createElement("tbody",null,c.filter(function(e){return e}).sort(function(e,t){return e.t-t.t}).map(function(e){return _react2.default.createElement(TimeStamp,_extends({key:e.title},e))}))))}function Details(e){var t=e.flow;return _react2.default.createElement("section",{className:"detail"},_react2.default.createElement("h4",null,"Client Connection"),_react2.default.createElement(ConnectionInfo,{conn:t.client_conn}),_react2.default.createElement("h4",null,"Server Connection"),_react2.default.createElement(ConnectionInfo,{conn:t.server_conn}),_react2.default.createElement(CertificateInfo,{flow:t}),_react2.default.createElement(Timing,{flow:t}))}Object.defineProperty(exports,"__esModule",{value:!0});var _extends=Object.assign||function(e){for(var t=1;t=0||Object.prototype.hasOwnProperty.call(e,n)&&(r[n]=e[n]);return r}function _classCallCheck(e,t){if(!(e instanceof t))throw new TypeError("Cannot call a class as a function")}function _possibleConstructorReturn(e,t){if(!e)throw new ReferenceError("this hasn't been initialised - super() hasn't been called");return!t||"object"!=typeof t&&"function"!=typeof t?e:t}function _inherits(e,t){if("function"!=typeof t&&null!==t)throw new TypeError("Super expression must either be null or a function, not "+typeof t);e.prototype=Object.create(t&&t.prototype,{constructor:{value:e,enumerable:!1,writable:!0,configurable:!0}}),t&&(Object.setPrototypeOf?Object.setPrototypeOf(e,t):e.__proto__=t)}Object.defineProperty(exports,"__esModule",{value:!0}),exports.HeaderEditor=void 0;var _extends=Object.assign||function(e){for(var t=1;t0&&(r.preventDefault(),this.refs[e-1+"-value"].focus())}},{key:"render",value:function(){var e=this,t=this.props,r=t.message,n=t.readonly;return _react2.default.createElement("table",{className:"header-table"},_react2.default.createElement("tbody",null,r.headers.map(function(t,r){return _react2.default.createElement("tr",{key:r},_react2.default.createElement("td",{className:"header-name"},_react2.default.createElement(HeaderEditor,{ref:r+"-key",content:t[0],readonly:n,onDone:function(t){return e.onChange(r,0,t)},onRemove:function(t){return e.onRemove(r,0,t)},onTab:function(t){return e.onTab(r,0,t)}}),_react2.default.createElement("span",{className:"header-colon"},":")),_react2.default.createElement("td",{className:"header-value"},_react2.default.createElement(HeaderEditor,{ref:r+"-value",content:t[1],readonly:n,onDone:function(t){return e.onChange(r,1,t)},onRemove:function(t){return e.onRemove(r,1,t)},onTab:function(t){return e.onTab(r,1,t)}})))})))}}]),t}();Headers.propTypes={onChange:_propTypes2.default.func.isRequired,message:_propTypes2.default.object.isRequired},exports.default=Headers; +"use strict";function _interopRequireDefault(e){return e&&e.__esModule?e:{default:e}}function _objectWithoutProperties(e,t){var r={};for(var n in e)t.indexOf(n)>=0||Object.prototype.hasOwnProperty.call(e,n)&&(r[n]=e[n]);return r}function _classCallCheck(e,t){if(!(e instanceof t))throw new TypeError("Cannot call a class as a function")}function _possibleConstructorReturn(e,t){if(!e)throw new ReferenceError("this hasn't been initialised - super() hasn't been called");return!t||"object"!=typeof t&&"function"!=typeof t?e:t}function _inherits(e,t){if("function"!=typeof t&&null!==t)throw new TypeError("Super expression must either be null or a function, not "+typeof t);e.prototype=Object.create(t&&t.prototype,{constructor:{value:e,enumerable:!1,writable:!0,configurable:!0}}),t&&(Object.setPrototypeOf?Object.setPrototypeOf(e,t):e.__proto__=t)}Object.defineProperty(exports,"__esModule",{value:!0}),exports.HeaderEditor=void 0;var _extends=Object.assign||function(e){for(var t=1;t0&&(r.preventDefault(),this.refs[e-1+"-value"].focus())}},{key:"render",value:function(){var e=this,t=this.props,r=t.message,n=t.readonly;return r[this.props.type]?_react2.default.createElement("table",{className:"header-table"},_react2.default.createElement("tbody",null,r[this.props.type].map(function(t,r){return _react2.default.createElement("tr",{key:r},_react2.default.createElement("td",{className:"header-name"},_react2.default.createElement(HeaderEditor,{ref:r+"-key",content:t[0],readonly:n,onDone:function(t){return e.onChange(r,0,t)},onRemove:function(t){return e.onRemove(r,0,t)},onTab:function(t){return e.onTab(r,0,t)}}),_react2.default.createElement("span",{className:"header-colon"},":")),_react2.default.createElement("td",{className:"header-value"},_react2.default.createElement(HeaderEditor,{ref:r+"-value",content:t[1],readonly:n,onDone:function(t){return e.onChange(r,1,t)},onRemove:function(t){return e.onRemove(r,1,t)},onTab:function(t){return e.onTab(r,1,t)}})))}))):_react2.default.createElement("table",{className:"header-table"},_react2.default.createElement("tbody",null))}}]),t}();Headers.propTypes={onChange:_propTypes2.default.func.isRequired,message:_propTypes2.default.object.isRequired,type:_propTypes2.default.string.isRequired},Headers.defaultProps={type:"headers"},exports.default=Headers; },{"../../utils":70,"../ValueEditor/ValueEditor":44,"prop-types":"prop-types","react":"react","react-dom":"react-dom"}],23:[function(require,module,exports){ -"use strict";function _interopRequireWildcard(e){if(e&&e.__esModule)return e;var t={};if(null!=e)for(var r in e)Object.prototype.hasOwnProperty.call(e,r)&&(t[r]=e[r]);return t.default=e,t}function _interopRequireDefault(e){return e&&e.__esModule?e:{default:e}}function _classCallCheck(e,t){if(!(e instanceof t))throw new TypeError("Cannot call a class as a function")}function _possibleConstructorReturn(e,t){if(!e)throw new ReferenceError("this hasn't been initialised - super() hasn't been called");return!t||"object"!=typeof t&&"function"!=typeof t?e:t}function _inherits(e,t){if("function"!=typeof t&&null!==t)throw new TypeError("Super expression must either be null or a function, not "+typeof t);e.prototype=Object.create(t&&t.prototype,{constructor:{value:e,enumerable:!1,writable:!0,configurable:!0}}),t&&(Object.setPrototypeOf?Object.setPrototypeOf(e,t):e.__proto__=t)}function RequestLine(e){var t=e.flow,r=e.readonly,n=e.updateFlow;return _react2.default.createElement("div",{className:"first-line request-line"},_react2.default.createElement("div",null,_react2.default.createElement(_ValueEditor2.default,{content:t.request.method,readonly:r,onDone:function(e){return n({request:{method:e}})}})," ",_react2.default.createElement(_ValidateEditor2.default,{content:_utils.RequestUtils.pretty_url(t.request),readonly:r,onDone:function(e){return n({request:_extends({path:""},(0,_utils.parseUrl)(e))})},isValid:function(e){return!!(0,_utils.parseUrl)(e).host}})," ",_react2.default.createElement(_ValidateEditor2.default,{content:t.request.http_version,readonly:r,onDone:function(e){return n({request:{http_version:e}})},isValid:_utils.isValidHttpVersion})))}function ResponseLine(e){var t=e.flow,r=e.readonly,n=e.updateFlow;return _react2.default.createElement("div",{className:"first-line response-line"},_react2.default.createElement(_ValidateEditor2.default,{content:t.response.http_version,readonly:r,onDone:function(e){return n({response:{http_version:e}})},isValid:_utils.isValidHttpVersion})," ",_react2.default.createElement(_ValidateEditor2.default,{content:t.response.status_code+"",readonly:r,onDone:function(e){return n({response:{code:parseInt(e)}})},isValid:function(e){return/^\d+$/.test(e)}})," ",_react2.default.createElement(_ValueEditor2.default,{content:t.response.reason,readonly:r,onDone:function(e){return n({response:{msg:e}})}}))}function ErrorView(e){var t=e.flow;return _react2.default.createElement("section",{className:"error"},_react2.default.createElement("div",{className:"alert alert-warning"},t.error.msg,_react2.default.createElement("div",null,_react2.default.createElement("small",null,(0,_utils2.formatTimeStamp)(t.error.timestamp)))))}Object.defineProperty(exports,"__esModule",{value:!0}),exports.Response=exports.Request=void 0;var _createClass=function(){function e(e,t){for(var r=0;r0?"/flows/"+t.flows.selected[0]+"/"+t.ui.flow.tab:"/flows",i&&(l+="?"+i);var n=window.location.pathname;"blank"===n&&(n="/"),window.location.hash.substr(1)!==l&&history.replaceState(void 0,"",n+"#"+l)}function initialize(e){updateStoreFromUrl(e),e.subscribe(function(){return updateUrlFromStore(e)})}Object.defineProperty(exports,"__esModule",{value:!0});var _slicedToArray=function(){function e(e,r){var t=[],o=!0,i=!1,l=void 0;try{for(var n,a=e[Symbol.iterator]();!(o=(n=a.next()).done)&&(t.push(n.value),!r||t.length!==r);o=!0);}catch(e){i=!0,l=e}finally{try{!o&&a.return&&a.return()}finally{if(i)throw l}}return t}return function(r,t){if(Array.isArray(r))return r;if(Symbol.iterator in Object(r))return e(r,t);throw new TypeError("Invalid attempt to destructure non-iterable instance")}}();exports.updateStoreFromUrl=updateStoreFromUrl,exports.updateUrlFromStore=updateUrlFromStore,exports.default=initialize;var _flows=require("./ducks/flows"),_flow=require("./ducks/ui/flow"),_eventLog=require("./ducks/eventLog"),Query={SEARCH:"s",HIGHLIGHT:"h",SHOW_EVENTLOG:"e"}; },{"./ducks/eventLog":55,"./ducks/flows":56,"./ducks/ui/flow":60}],70:[function(require,module,exports){ -"use strict";function _interopRequireDefault(e){return e&&e.__esModule?e:{default:e}}function _classCallCheck(e,t){if(!(e instanceof t))throw new TypeError("Cannot call a class as a function")}function _possibleConstructorReturn(e,t){if(!e)throw new ReferenceError("this hasn't been initialised - super() hasn't been called");return!t||"object"!=typeof t&&"function"!=typeof t?e:t}function _inherits(e,t){if("function"!=typeof t&&null!==t)throw new TypeError("Super expression must either be null or a function, not "+typeof t);e.prototype=Object.create(t&&t.prototype,{constructor:{value:e,enumerable:!1,writable:!0,configurable:!0}}),t&&(Object.setPrototypeOf?Object.setPrototypeOf(e,t):e.__proto__=t)}function reverseString(e){return String.fromCharCode.apply(String,_lodash2.default.map(e.split(""),function(e){return 65535-e.charCodeAt(0)}))+end}function getCookie(e){var t=document.cookie.match(new RegExp("\\b"+e+"=([^;]*)\\b"));return t?t[1]:void 0}function fetchApi(e){var t=arguments.length>1&&void 0!==arguments[1]?arguments[1]:{};return t.method&&"GET"!==t.method?-1===e.indexOf("?")?e+="?"+xsrf:e+="&"+xsrf:e+=".json",e.startsWith("/")&&(e="."+e),fetch(e,_extends({credentials:"same-origin"},t))}function getDiff(e,t){var r=_extends({},t);for(var o in e)_lodash2.default.isEqual(t[o],e[o])?r[o]=void 0:"[object Object]"===Object.prototype.toString.call(t[o])&&"[object Object]"===Object.prototype.toString.call(e[o])&&(r[o]=getDiff(e[o],t[o]));return r}Object.defineProperty(exports,"__esModule",{value:!0}),exports.pure=exports.formatTimeStamp=exports.formatTimeDelta=exports.formatSize=exports.Key=void 0;var _createClass=function(){function e(e,t){for(var r=0;re);r++);var o;return o=e%Math.pow(1024,r)==0?0:1,(e/Math.pow(1024,r)).toFixed(o)+t[r]},formatTimeDelta=exports.formatTimeDelta=function(e){for(var t=e,r=[1e3,60,60],o=0;Math.abs(t)>=r[o]&&o1&&void 0!==arguments[1]?arguments[1]:{};return t.method&&"GET"!==t.method?-1===e.indexOf("?")?e+="?"+xsrf:e+="&"+xsrf:e+=".json",e.startsWith("/")&&(e="."+e),fetch(e,_extends({credentials:"same-origin"},t))}function getDiff(e,t){var r=_extends({},t);for(var o in e)_lodash2.default.isEqual(t[o],e[o])?r[o]=void 0:"[object Object]"===Object.prototype.toString.call(t[o])&&"[object Object]"===Object.prototype.toString.call(e[o])&&(r[o]=getDiff(e[o],t[o]));return r}Object.defineProperty(exports,"__esModule",{value:!0}),exports.pure=exports.formatTimeStamp=exports.formatTimeDelta=exports.formatSize=exports.Key=void 0;var _createClass=function(){function e(e,t){for(var r=0;re);r++);var o;return o=e%Math.pow(1024,r)==0?0:1,(e/Math.pow(1024,r)).toFixed(o)+t[r]},formatTimeDelta=exports.formatTimeDelta=function(e){for(var t=e,r=[1e3,60,60],o=0;Math.abs(t)>=r[o]&&o1&&void 0!==arguments[1])||arguments[1],r=new Date(1e3*e);if(t)var o=r.getTime()-60*r.getTimezoneOffset()*1e3,n=new Date(o).toISOString();else n=r.toISOString();return n.replace("T"," ").replace("Z","")},end=String.fromCharCode(65535),xsrf="_xsrf="+getCookie("_xsrf");fetchApi.put=function(e,t,r){return fetchApi(e,_extends({method:"PUT",headers:{"Content-Type":"application/json"},body:JSON.stringify(t)},r))};var pure=exports.pure=function(e){var t,r;return r=t=function(t){function r(){return _classCallCheck(this,r),_possibleConstructorReturn(this,(r.__proto__||Object.getPrototypeOf(r)).apply(this,arguments))}return _inherits(r,_react2.default.PureComponent),_createClass(r,[{key:"render",value:function(){return e(this.props)}}]),r}(),t.displayName=e.name,r}; },{"lodash":"lodash","react":"react"}]},{},[1]) diff -Nru mitmproxy-5.1.1/mitmproxy/tools/web/webaddons.py mitmproxy-6.0.2/mitmproxy/tools/web/webaddons.py --- mitmproxy-5.1.1/mitmproxy/tools/web/webaddons.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/tools/web/webaddons.py 2020-12-15 16:41:27.000000000 +0000 @@ -24,11 +24,11 @@ def running(self): if hasattr(ctx.options, "web_open_browser") and ctx.options.web_open_browser: - web_url = "http://{}:{}/".format(ctx.options.web_host, ctx.options.web_port) + web_url = f"http://{ctx.options.web_host}:{ctx.options.web_port}/" success = open_browser(web_url) if not success: ctx.log.info( - "No web browser found. Please open a browser and point it to {}".format(web_url), + f"No web browser found. Please open a browser and point it to {web_url}", ) @@ -45,6 +45,8 @@ """ browsers = ( "windows-default", "macosx", + "wslview %s", + "x-www-browser %s", "gnome-open %s", "google-chrome", "chrome", "chromium", "chromium-browser", "firefox", "opera", "safari", ) @@ -54,6 +56,6 @@ except webbrowser.Error: pass else: - b.open(url) - return True + if b.open(url): + return True return False diff -Nru mitmproxy-5.1.1/mitmproxy/utils/arg_check.py mitmproxy-6.0.2/mitmproxy/utils/arg_check.py --- mitmproxy-5.1.1/mitmproxy/utils/arg_check.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/utils/arg_check.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,4 +1,5 @@ import sys +import re DEPRECATED = """ --confdir @@ -55,6 +56,7 @@ --insecure -c --replace +--replacements -i -f --filter @@ -97,6 +99,7 @@ "--insecure": "--ssl-insecure", "-c": "-C", "--replace": "--replacements", + "--replacements": ["--modify-body", "--modify-headers"], "-i": "--intercept", "-f": "--view-filter", "--filter": "--view-filter", @@ -115,7 +118,7 @@ for option in ("-e", "--eventlog", "--norefresh"): if option in args: - print("{} has been removed.".format(option)) + print(f"{option} has been removed.") for option in ("--nonanonymous", "--singleuser", "--htpasswd"): if option in args: @@ -129,11 +132,15 @@ for option in REPLACED.splitlines(): if option in args: + if isinstance(REPLACEMENTS.get(option), list): + new_options = REPLACEMENTS.get(option) + else: + new_options = [REPLACEMENTS.get(option)] print( "{} is deprecated.\n" "Please use `{}` instead.".format( option, - REPLACEMENTS.get(option) + "` or `".join(new_options) ) ) @@ -147,3 +154,12 @@ REPLACEMENTS.get(option, None) or option.lstrip("-").replace("-", "_") ) ) + + # Check for underscores in the options. Options always follow '--'. + for argument in args: + underscoreParam = re.search('[-]{2}((.*?_)(.*?(\s|$)))+', argument) + if underscoreParam is not None: + print("{} uses underscores, please use hyphens {}".format( + argument, + argument.replace('_', '-')) + ) diff -Nru mitmproxy-5.1.1/mitmproxy/utils/asyncio_utils.py mitmproxy-6.0.2/mitmproxy/utils/asyncio_utils.py --- mitmproxy-5.1.1/mitmproxy/utils/asyncio_utils.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/utils/asyncio_utils.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,65 @@ +import asyncio +import sys +import time +from collections.abc import Coroutine +from typing import Optional + +from mitmproxy.utils import human + + +def cancel_task(task: asyncio.Task, message: str) -> None: + """Like task.cancel(), but optionally with a message if the Python version supports it.""" + if sys.version_info >= (3, 9): + task.cancel(message) # type: ignore + else: # pragma: no cover + task.cancel() + + +def create_task( + coro: Coroutine, *, + name: str, + client: Optional[tuple] = None, + ignore_closed_loop: bool = True, +) -> Optional[asyncio.Task]: + """ + Like asyncio.create_task, but also store some debug info on the task object. + + If ignore_closed_loop is True, the task will be silently discarded if the event loop is closed. + This is currently useful during shutdown where no new tasks can be spawned. + Ideally we stop closing the event loop during shutdown and then remove this parameter. + """ + try: + t = asyncio.create_task(coro, name=name) + except RuntimeError: + if ignore_closed_loop: + coro.close() + return None + else: + raise + set_task_debug_info(t, name=name, client=client) + return t + + +def set_task_debug_info( + task: asyncio.Task, + *, + name: str, + client: Optional[tuple] = None, +) -> None: + """Set debug info for an externally-spawned task.""" + task.created = time.time() # type: ignore + task.set_name(name) + if client: + task.client = client # type: ignore + + +def task_repr(task: asyncio.Task) -> str: + """Get a task representation with debug info.""" + name = task.get_name() + age = getattr(task, "created", "") + if age: + age = f" (age: {time.time() - age:.0f}s)" + client = getattr(task, "client", "") + if client: + client = f"{human.format_address(client)}: " + return f"{client}{name}{age}" diff -Nru mitmproxy-5.1.1/mitmproxy/utils/compat.py mitmproxy-6.0.2/mitmproxy/utils/compat.py --- mitmproxy-5.1.1/mitmproxy/utils/compat.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/utils/compat.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,13 @@ +new_proxy_core = False +"""If true, use mitmproxy's new sans-io proxy core.""" + +if new_proxy_core: # pragma: no cover + from mitmproxy.proxy2 import context + + Client = context.Client # type: ignore + Server = context.Server # type: ignore +else: # pragma: no cover + from mitmproxy import connections + + Client = connections.ClientConnection # type: ignore + Server = connections.ServerConnection # type: ignore diff -Nru mitmproxy-5.1.1/mitmproxy/utils/debug.py mitmproxy-6.0.2/mitmproxy/utils/debug.py --- mitmproxy-5.1.1/mitmproxy/utils/debug.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/utils/debug.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,90 +1,110 @@ +import asyncio import gc +import linecache import os import platform import signal import sys import threading import traceback +from contextlib import redirect_stdout from OpenSSL import SSL - from mitmproxy import version +from mitmproxy.utils import asyncio_utils def dump_system_info(): mitmproxy_version = version.get_dev_version() data = [ - "Mitmproxy: {}".format(mitmproxy_version), - "Python: {}".format(platform.python_version()), + f"Mitmproxy: {mitmproxy_version}", + f"Python: {platform.python_version()}", "OpenSSL: {}".format(SSL.SSLeay_version(SSL.SSLEAY_VERSION).decode()), - "Platform: {}".format(platform.platform()), + f"Platform: {platform.platform()}", ] return "\n".join(data) def dump_info(signal=None, frame=None, file=sys.stdout, testing=False): # pragma: no cover - print("****************************************************", file=file) - print("Summary", file=file) - print("=======", file=file) - - try: - import psutil - except: - print("(psutil not installed, skipping some debug info)", file=file) - else: - p = psutil.Process() - print("num threads: ", p.num_threads(), file=file) - if hasattr(p, "num_fds"): - print("num fds: ", p.num_fds(), file=file) - print("memory: ", p.memory_info(), file=file) - - print(file=file) - print("Files", file=file) - print("=====", file=file) - for i in p.open_files(): - print(i, file=file) - - print(file=file) - print("Connections", file=file) - print("===========", file=file) - for i in p.connections(): - print(i, file=file) - - print(file=file) - print("Threads", file=file) - print("=======", file=file) - bthreads = [] - for i in threading.enumerate(): - if hasattr(i, "_threadinfo"): - bthreads.append(i) + with redirect_stdout(file): + print("****************************************************") + print("Summary") + print("=======") + + try: + import psutil + except: + print("(psutil not installed, skipping some debug info)") + else: + p = psutil.Process() + print("num threads: ", p.num_threads()) + if hasattr(p, "num_fds"): + print("num fds: ", p.num_fds()) + print("memory: ", p.memory_info()) + + print() + print("Files") + print("=====") + for i in p.open_files(): + print(i) + + print() + print("Connections") + print("===========") + for i in p.connections(): + print(i) + + print() + print("Threads") + print("=======") + bthreads = [] + for i in threading.enumerate(): + if hasattr(i, "_threadinfo"): + bthreads.append(i) + else: + print(i.name) + bthreads.sort(key=lambda x: x._thread_started) + for i in bthreads: + print(i._threadinfo()) + + print() + print("Memory") + print("=======") + gc.collect() + d = {} + for i in gc.get_objects(): + t = str(type(i)) + if "mitmproxy" in t: + d[t] = d.setdefault(t, 0) + 1 + itms = list(d.items()) + itms.sort(key=lambda x: x[1]) + for i in itms[-20:]: + print(i[1], i[0]) + + try: + asyncio.get_running_loop() + except RuntimeError: + pass else: - print(i.name, file=file) - bthreads.sort(key=lambda x: x._thread_started) - for i in bthreads: - print(i._threadinfo(), file=file) - - print(file=file) - print("Memory", file=file) - print("=======", file=file) - gc.collect() - d = {} - for i in gc.get_objects(): - t = str(type(i)) - if "mitmproxy" in t: - d[t] = d.setdefault(t, 0) + 1 - itms = list(d.items()) - itms.sort(key=lambda x: x[1]) - for i in itms[-20:]: - print(i[1], i[0], file=file) - print("****************************************************", file=file) + print() + print("Tasks") + print("=======") + for task in asyncio.all_tasks(): + f = task.get_stack(limit=1)[0] + line = linecache.getline(f.f_code.co_filename, f.f_lineno, f.f_globals).strip() + line = f"{line} # at {os.path.basename(f.f_code.co_filename)}:{f.f_lineno}" + print(f"{asyncio_utils.task_repr(task)}\n" + f" {line}") + + print("****************************************************") if not testing: sys.exit(1) def dump_stacks(signal=None, frame=None, file=sys.stdout, testing=False): - id2name = dict([(th.ident, th.name) for th in threading.enumerate()]) + id2name = {th.ident: th.name for th in threading.enumerate()} code = [] for threadId, stack in sys._current_frames().items(): code.append( diff -Nru mitmproxy-5.1.1/mitmproxy/utils/human.py mitmproxy-6.0.2/mitmproxy/utils/human.py --- mitmproxy-5.1.1/mitmproxy/utils/human.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/utils/human.py 2020-12-15 16:41:27.000000000 +0000 @@ -24,7 +24,7 @@ if x == int(x): x = int(x) return str(x) + suf - return "%s%s" % (size, SIZE_TABLE[0][0]) + return "{}{}".format(size, SIZE_TABLE[0][0]) @functools.lru_cache() @@ -75,6 +75,7 @@ return d.strftime("%Y-%m-%d %H:%M:%S.%f")[:-3] +@functools.lru_cache() def format_address(address: typing.Optional[tuple]) -> str: """ This function accepts IPv4/IPv6 tuples and diff -Nru mitmproxy-5.1.1/mitmproxy/utils/spec.py mitmproxy-6.0.2/mitmproxy/utils/spec.py --- mitmproxy-5.1.1/mitmproxy/utils/spec.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/utils/spec.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,28 @@ +import typing +from mitmproxy import flowfilter + + +def _match_all(flow) -> bool: + return True + + +def parse_spec(option: str) -> typing.Tuple[flowfilter.TFilter, str, str]: + """ + Parse strings in the following format: + + [/flow-filter]/subject/replacement + + """ + sep, rem = option[0], option[1:] + parts = rem.split(sep, 2) + if len(parts) == 2: + subject, replacement = parts + return _match_all, subject, replacement + elif len(parts) == 3: + patt, subject, replacement = parts + flow_filter = flowfilter.parse(patt) + if not flow_filter: + raise ValueError(f"Invalid filter pattern: {patt}") + return flow_filter, subject, replacement + else: + raise ValueError("Invalid number of parameters (2 or 3 are expected)") diff -Nru mitmproxy-5.1.1/mitmproxy/utils/strutils.py mitmproxy-6.0.2/mitmproxy/utils/strutils.py --- mitmproxy-5.1.1/mitmproxy/utils/strutils.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/utils/strutils.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,27 +1,47 @@ import codecs import io import re -from typing import Iterable, Optional, Union, cast +from typing import Iterable, Union, overload -def always_bytes(str_or_bytes: Union[str, bytes, None], *encode_args) -> Optional[bytes]: - if isinstance(str_or_bytes, bytes) or str_or_bytes is None: - return cast(Optional[bytes], str_or_bytes) +# https://mypy.readthedocs.io/en/stable/more_types.html#function-overloading + +@overload +def always_bytes(str_or_bytes: None, *encode_args) -> None: + ... + + +@overload +def always_bytes(str_or_bytes: Union[str, bytes], *encode_args) -> bytes: + ... + + +def always_bytes(str_or_bytes: Union[None, str, bytes], *encode_args) -> Union[None, bytes]: + if str_or_bytes is None or isinstance(str_or_bytes, bytes): + return str_or_bytes elif isinstance(str_or_bytes, str): return str_or_bytes.encode(*encode_args) else: raise TypeError("Expected str or bytes, but got {}.".format(type(str_or_bytes).__name__)) -def always_str(str_or_bytes: Union[str, bytes, None], *decode_args) -> Optional[str]: +@overload +def always_str(str_or_bytes: None, *encode_args) -> None: + ... + + +@overload +def always_str(str_or_bytes: Union[str, bytes], *encode_args) -> str: + ... + + +def always_str(str_or_bytes: Union[None, str, bytes], *decode_args) -> Union[None, str]: """ Returns, str_or_bytes unmodified, if """ - if str_or_bytes is None: - return None - if isinstance(str_or_bytes, str): - return cast(str, str_or_bytes) + if str_or_bytes is None or isinstance(str_or_bytes, str): + return str_or_bytes elif isinstance(str_or_bytes, bytes): return str_or_bytes.decode(*decode_args) else: @@ -72,7 +92,7 @@ """ if not isinstance(data, bytes): - raise ValueError("data must be bytes, but is {}".format(data.__class__.__name__)) + raise ValueError(f"data must be bytes, but is {data.__class__.__name__}") # We always insert a double-quote here so that we get a single-quoted string back # https://stackoverflow.com/questions/29019340/why-does-python-use-different-quotes-for-representing-strings-depending-on-their ret = repr(b'"' + data).lstrip("b")[2:-1] @@ -95,7 +115,7 @@ ValueError, if the escape sequence is invalid. """ if not isinstance(data, str): - raise ValueError("data must be str, but is {}".format(data.__class__.__name__)) + raise ValueError(f"data must be str, but is {data.__class__.__name__}") # This one is difficult - we use an undocumented Python API here # as per http://stackoverflow.com/a/23151714/934719 @@ -134,12 +154,12 @@ A generator of (offset, hex, str) tuples """ for i in range(0, len(s), 16): - offset = "{:0=10x}".format(i) + offset = f"{i:0=10x}" part = s[i:i + 16] - x = " ".join("{:0=2x}".format(i) for i in part) + x = " ".join(f"{i:0=2x}" for i in part) x = x.ljust(47) # 16*2 + 15 part_repr = always_str(escape_control_characters( - part.decode("ascii", "replace").replace(u"\ufffd", u"."), + part.decode("ascii", "replace").replace("\ufffd", "."), False )) yield (offset, x, part_repr) @@ -210,7 +230,7 @@ """ buf = io.StringIO() parts = split_special_areas(data, area_delimiter) - rex = re.compile(r"[{}]".format(control_characters)) + rex = re.compile(fr"[{control_characters}]") for i, x in enumerate(parts): if i % 2: x = rex.sub(_move_to_private_code_plane, x) diff -Nru mitmproxy-5.1.1/mitmproxy/utils/typecheck.py mitmproxy-6.0.2/mitmproxy/utils/typecheck.py --- mitmproxy-5.1.1/mitmproxy/utils/typecheck.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/utils/typecheck.py 2020-12-15 16:41:27.000000000 +0000 @@ -39,7 +39,7 @@ typename = str(typeinfo) - if typename.startswith("typing.Union"): + if typename.startswith("typing.Union") or typename.startswith("typing.Optional"): for T in union_types(typeinfo): try: check_option_type(name, value, T) @@ -55,7 +55,7 @@ if len(types) != len(value): raise e for i, (x, T) in enumerate(zip(value, types)): - check_option_type("{}[{}]".format(name, i), x, T) + check_option_type(f"{name}[{i}]", x, T) return elif typename.startswith("typing.Sequence"): T = sequence_type(typeinfo) @@ -71,6 +71,8 @@ elif typename.startswith("typing.Any"): return elif not isinstance(value, typeinfo): + if typeinfo is float and isinstance(value, int): + return raise e @@ -81,6 +83,8 @@ t = 'optional str' elif typespec == typing.Sequence[str]: t = 'sequence of str' + elif typespec == typing.Optional[int]: + t = 'optional int' else: raise NotImplementedError return t diff -Nru mitmproxy-5.1.1/mitmproxy/version.py mitmproxy-6.0.2/mitmproxy/version.py --- mitmproxy-5.1.1/mitmproxy/version.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/version.py 2020-12-15 16:41:27.000000000 +0000 @@ -2,13 +2,13 @@ import subprocess import sys -VERSION = "5.1.1" +VERSION = "6.0.2" PATHOD = "pathod " + VERSION MITMPROXY = "mitmproxy " + VERSION # Serialization format version. This is displayed nowhere, it just needs to be incremented by one # for each change in the file format. -FLOW_FORMAT_VERSION = 7 +FLOW_FORMAT_VERSION = 10 def get_dev_version() -> str: @@ -20,6 +20,14 @@ here = os.path.abspath(os.path.join(os.path.dirname(__file__), "..")) try: + # Check that we're in the mitmproxy repository: https://github.com/mitmproxy/mitmproxy/issues/3987 + # cb0e3287090786fad566feb67ac07b8ef361b2c3 is the first mitmproxy commit. + subprocess.run( + ['git', 'cat-file', '-e', 'cb0e3287090786fad566feb67ac07b8ef361b2c3'], + stdout=subprocess.DEVNULL, + stderr=subprocess.DEVNULL, + cwd=here, + check=True) git_describe = subprocess.check_output( ['git', 'describe', '--tags', '--long'], stderr=subprocess.STDOUT, diff -Nru mitmproxy-5.1.1/mitmproxy/websocket.py mitmproxy-6.0.2/mitmproxy/websocket.py --- mitmproxy-5.1.1/mitmproxy/websocket.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/mitmproxy/websocket.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,14 +1,15 @@ import time import queue -from typing import List, Optional +import warnings +from typing import List, Optional, Union from wsproto.frame_protocol import CloseReason from wsproto.frame_protocol import Opcode from mitmproxy import flow -from mitmproxy.net import websockets +from mitmproxy.net import websocket from mitmproxy.coretypes import serializable -from mitmproxy.utils import strutils, human +from mitmproxy.utils import strutils, human, compat class WebSocketMessage(serializable.Serializable): @@ -17,7 +18,7 @@ """ def __init__( - self, type: int, from_client: bool, content: bytes, timestamp: Optional[int]=None, killed: bool=False + self, type: int, from_client: bool, content: Union[bytes, str], timestamp: Optional[float]=None, killed: bool=False ) -> None: self.type = Opcode(type) # type: ignore """indicates either TEXT or BINARY (from wsproto.frame_protocol.Opcode).""" @@ -25,7 +26,7 @@ """True if this messages was sent by the client.""" self.content = content """A byte-string representing the content of this message.""" - self.timestamp: int = timestamp or int(time.time()) + self.timestamp: float = timestamp or time.time() """Timestamp of when this message was received or created.""" self.killed = killed """True if this messages was killed and should not be sent to the other endpoint.""" @@ -53,12 +54,18 @@ It will not be sent to the other endpoint. This has no effect in streaming mode. """ - self.killed = True + if compat.new_proxy_core: # pragma: no cover + warnings.warn("WebSocketMessage.kill is deprecated, set an empty content instead.", + PendingDeprecationWarning) + # empty str or empty bytes. + self.content = type(self.content)() + else: # pragma: no cover + self.killed = True class WebSocketFlow(flow.Flow): """ - A WebSocketFlow is a simplified representation of a Websocket connection. + A WebSocketFlow is a simplified representation of a WebSocket connection. """ def __init__(self, client_conn, server_conn, handshake_flow, live=None): @@ -85,12 +92,12 @@ self._inject_messages_server = queue.Queue(maxsize=1) if handshake_flow: - self.client_key = websockets.get_client_key(handshake_flow.request.headers) - self.client_protocol = websockets.get_protocol(handshake_flow.request.headers) - self.client_extensions = websockets.get_extensions(handshake_flow.request.headers) - self.server_accept = websockets.get_server_accept(handshake_flow.response.headers) - self.server_protocol = websockets.get_protocol(handshake_flow.response.headers) - self.server_extensions = websockets.get_extensions(handshake_flow.response.headers) + self.client_key = websocket.get_client_key(handshake_flow.request.headers) + self.client_protocol = websocket.get_protocol(handshake_flow.request.headers) + self.client_extensions = websocket.get_extensions(handshake_flow.request.headers) + self.server_accept = websocket.get_server_accept(handshake_flow.response.headers) + self.server_protocol = websocket.get_protocol(handshake_flow.response.headers) + self.server_extensions = websocket.get_extensions(handshake_flow.response.headers) else: self.client_key = '' self.client_protocol = '' diff -Nru mitmproxy-5.1.1/pathod/__init__.py mitmproxy-6.0.2/pathod/__init__.py --- mitmproxy-5.1.1/pathod/__init__.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/pathod/__init__.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,18 @@ +import os +import sys + +import warnings + +warnings.warn( + "pathod and pathoc modules are deprecated, see https://github.com/mitmproxy/mitmproxy/issues/4273", + DeprecationWarning, + stacklevel=2 +) + + +def print_tool_deprecation_message(): + print("####", file=sys.stderr) + print(f"### {os.path.basename(sys.argv[0])} is deprecated and will not be part of future mitmproxy releases!", file=sys.stderr) + print("### See https://github.com/mitmproxy/mitmproxy/issues/4273 for more information.", file=sys.stderr) + print("####", file=sys.stderr) + print("", file=sys.stderr) diff -Nru mitmproxy-5.1.1/pathod/language/actions.py mitmproxy-6.0.2/pathod/language/actions.py --- mitmproxy-5.1.1/pathod/language/actions.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/pathod/language/actions.py 2020-12-15 16:41:27.000000000 +0000 @@ -70,7 +70,7 @@ return e.setParseAction(lambda x: cls(*x)) def spec(self): - return "p%s,%s" % (self.offset, self.seconds) + return f"p{self.offset},{self.seconds}" def intermediate(self, settings): return (self.offset, "pause", self.seconds) @@ -116,7 +116,7 @@ return e.setParseAction(lambda x: cls(*x)) def spec(self): - return "i%s,%s" % (self.offset, self.value.spec()) + return f"i{self.offset},{self.value.spec()}" def intermediate(self, settings): return ( diff -Nru mitmproxy-5.1.1/pathod/language/base.py mitmproxy-6.0.2/pathod/language/base.py --- mitmproxy-5.1.1/pathod/language/base.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/pathod/language/base.py 2020-12-15 16:41:27.000000000 +0000 @@ -295,7 +295,7 @@ return e.setParseAction(lambda x: cls(*x)) def spec(self): - return "%s%s=%s" % (self.preamble, self.key.spec(), self.value.spec()) + return f"{self.preamble}{self.key.spec()}={self.value.spec()}" def freeze(self, settings): return self.__class__( @@ -369,7 +369,7 @@ s = self.value.spec() if s[1:-1].lower() in self.options: s = s[1:-1].lower() - return "%s%s" % (self.preamble, s) + return f"{self.preamble}{s}" def freeze(self, settings): return self.__class__(self.value.freeze(settings)) @@ -403,7 +403,7 @@ return [self.value] def spec(self): - return "%s%s" % (self.preamble, self.value.decode()) + return f"{self.preamble}{self.value.decode()}" def freeze(self, settings_): return self @@ -430,7 +430,7 @@ return [self.value.get_generator(settings)] def spec(self): - return "%s%s" % (self.preamble, self.value.spec()) + return f"{self.preamble}{self.value.spec()}" def freeze(self, settings): return self.__class__(self.value.freeze(settings)) @@ -454,7 +454,7 @@ # This check will fail if we know the length upfront if lenguess is not None and lenguess != self.length: raise exceptions.RenderError( - "Invalid value length: '%s' is %s bytes, should be %s." % ( + "Invalid value length: '{}' is {} bytes, should be {}.".format( self.spec(), lenguess, self.length @@ -468,7 +468,7 @@ # file inputs if l != self.length: raise exceptions.RenderError( - "Invalid value length: '%s' is %s bytes, should be %s." % ( + "Invalid value length: '{}' is {} bytes, should be {}.".format( self.spec(), l, self.length @@ -503,7 +503,7 @@ return e.setParseAction(parse) def spec(self): - return "%s%s" % ("-" if not self.value else "", self.name) + return "{}{}".format("-" if not self.value else "", self.name) class IntField(_Component): @@ -537,4 +537,4 @@ return [str(self.value)] def spec(self): - return "%s%s" % (self.preamble, self.origvalue) + return f"{self.preamble}{self.origvalue}" diff -Nru mitmproxy-5.1.1/pathod/language/exceptions.py mitmproxy-6.0.2/pathod/language/exceptions.py --- mitmproxy-5.1.1/pathod/language/exceptions.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/pathod/language/exceptions.py 2020-12-15 16:41:27.000000000 +0000 @@ -15,7 +15,7 @@ self.col = col def marked(self): - return "%s\n%s" % (self.s, " " * (self.col - 1) + "^") + return "{}\n{}".format(self.s, " " * (self.col - 1) + "^") def __str__(self): - return "%s at char %s" % (self.msg, self.col) + return f"{self.msg} at char {self.col}" diff -Nru mitmproxy-5.1.1/pathod/language/generators.py mitmproxy-6.0.2/pathod/language/generators.py --- mitmproxy-5.1.1/pathod/language/generators.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/pathod/language/generators.py 2020-12-15 16:41:27.000000000 +0000 @@ -70,7 +70,7 @@ return rand_byte(chars) def __repr__(self): - return "%s random from %s" % (self.length, self.dtype) + return f"{self.length} random from {self.dtype}" class FileGenerator: diff -Nru mitmproxy-5.1.1/pathod/language/http2.py mitmproxy-6.0.2/pathod/language/http2.py --- mitmproxy-5.1.1/pathod/language/http2.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/pathod/language/http2.py 2020-12-15 16:41:27.000000000 +0000 @@ -158,7 +158,7 @@ ) def __init__(self, tokens): - super(Response, self).__init__(tokens) + super().__init__(tokens) self.rendered_values = None self.stream_id = 2 @@ -190,11 +190,14 @@ body = body.string() resp = http.Response( - b'HTTP/2.0', - int(self.status_code.string()), - b'', - headers, - body, + http_version=b'HTTP/2.0', + status_code=int(self.status_code.string()), + reason=b'', + headers=headers, + content=body, + trailers=None, + timestamp_start=0, + timestamp_end=0 ) resp.stream_id = self.stream_id @@ -223,7 +226,7 @@ logattrs = ["method", "path"] def __init__(self, tokens): - super(Request, self).__init__(tokens) + super().__init__(tokens) self.rendered_values = None self.stream_id = 1 @@ -273,15 +276,18 @@ body = body.string() req = http.Request( - b'', + "", + 0, self.method.string(), b'http', b'', - b'', path, - (2, 0), + b"HTTP/2.0", headers, body, + None, + 0, + 0, ) req.stream_id = self.stream_id diff -Nru mitmproxy-5.1.1/pathod/language/http.py mitmproxy-6.0.2/pathod/language/http.py --- mitmproxy-5.1.1/pathod/language/http.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/pathod/language/http.py 2020-12-15 16:41:27.000000000 +0000 @@ -2,9 +2,8 @@ import pyparsing as pp -from mitmproxy.net.http import url -import mitmproxy.net.websockets -from mitmproxy.net.http import status_codes, user_agents +from mitmproxy.net import websocket +from mitmproxy.net.http import status_codes, url, user_agents from . import base, exceptions, actions, message # TODO: use mitmproxy.net.semantics.protocol assemble method, @@ -200,7 +199,7 @@ 1, StatusCode(101) ) - headers = mitmproxy.net.websockets.server_handshake_headers( + headers = websocket.server_handshake_headers( settings.websocket_key ) for i in headers.fields: @@ -312,7 +311,7 @@ 1, Method("get") ) - for i in mitmproxy.net.websockets.client_handshake_headers().fields: + for i in websocket.client_handshake_headers().fields: if not get_header(i[0], self.headers): tokens.append( Header( diff -Nru mitmproxy-5.1.1/pathod/language/message.py mitmproxy-6.0.2/pathod/language/message.py --- mitmproxy-5.1.1/pathod/language/message.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/pathod/language/message.py 2020-12-15 16:41:27.000000000 +0000 @@ -14,7 +14,7 @@ logattrs: typing.List[str] = [] def __init__(self, tokens): - track = set([]) + track = set() for i in tokens: if i.unique_name: if i.unique_name in track: @@ -133,7 +133,7 @@ ] def spec(self): - return "%s%s" % (self.preamble, self.value.spec()) + return f"{self.preamble}{self.value.spec()}" def freeze(self, settings): f = self.parsed.freeze(settings).spec() diff -Nru mitmproxy-5.1.1/pathod/language/websockets_frame.py mitmproxy-6.0.2/pathod/language/websockets_frame.py --- mitmproxy-5.1.1/pathod/language/websockets_frame.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/pathod/language/websockets_frame.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,274 @@ +import sys +import os +import struct +import io + +from wsproto.frame_protocol import Opcode + +from mitmproxy.net import tcp +from mitmproxy.utils import bits, human, strutils + + +MAX_16_BIT_INT = (1 << 16) +MAX_64_BIT_INT = (1 << 64) + +DEFAULT = object() + + +class Masker: + """ + Data sent from the server must be masked to prevent malicious clients + from sending data over the wire in predictable patterns. + + Servers do not have to mask data they send to the client. + https://tools.ietf.org/html/rfc6455#section-5.3 + """ + + def __init__(self, key): + self.key = key + self.offset = 0 + + def mask(self, offset, data): + datalen = len(data) + offset_mod = offset % 4 + data = int.from_bytes(data, sys.byteorder) + num_keys = (datalen + offset_mod + 3) // 4 + mask = int.from_bytes((self.key * num_keys)[offset_mod:datalen + + offset_mod], sys.byteorder) + return (data ^ mask).to_bytes(datalen, sys.byteorder) + + def __call__(self, data): + ret = self.mask(self.offset, data) + self.offset += len(ret) + return ret + + +class FrameHeader: + + def __init__( + self, + opcode=Opcode.TEXT, + payload_length=0, + fin=False, + rsv1=False, + rsv2=False, + rsv3=False, + masking_key=DEFAULT, + mask=DEFAULT, + length_code=DEFAULT + ): + if not 0 <= opcode < 2 ** 4: + raise ValueError("opcode must be 0-16") + self.opcode = opcode + self.payload_length = payload_length + self.fin = fin + self.rsv1 = rsv1 + self.rsv2 = rsv2 + self.rsv3 = rsv3 + + if length_code is DEFAULT: + self.length_code = self._make_length_code(self.payload_length) + else: + self.length_code = length_code + + if (mask is DEFAULT and masking_key is DEFAULT) or mask == 0 or mask is False: + self.mask = False + self.masking_key = b"" + elif mask is DEFAULT: + self.mask = 1 + self.masking_key = masking_key + elif masking_key is DEFAULT: + self.mask = mask + self.masking_key = os.urandom(4) + else: + self.mask = mask + self.masking_key = masking_key + + if self.masking_key and len(self.masking_key) != 4: + raise ValueError("Masking key must be 4 bytes.") + + @classmethod + def _make_length_code(self, length): + """ + A WebSocket frame contains an initial length_code, and an optional + extended length code to represent the actual length if length code is + larger than 125 + """ + if length <= 125: + return length + elif length >= 126 and length <= 65535: + return 126 + else: + return 127 + + def __repr__(self): + vals = [ + "ws frame:", + Opcode(self.opcode).name.lower() + ] + flags = [] + for i in ["fin", "rsv1", "rsv2", "rsv3", "mask"]: + if getattr(self, i): + flags.append(i) + if flags: + vals.extend([":", "|".join(flags)]) + if self.masking_key: + vals.append(":key=%s" % repr(self.masking_key)) + if self.payload_length: + vals.append(" %s" % human.pretty_size(self.payload_length)) + return "".join(vals) + + def __bytes__(self): + first_byte = bits.setbit(0, 7, self.fin) + first_byte = bits.setbit(first_byte, 6, self.rsv1) + first_byte = bits.setbit(first_byte, 5, self.rsv2) + first_byte = bits.setbit(first_byte, 4, self.rsv3) + first_byte = first_byte | self.opcode + + second_byte = bits.setbit(self.length_code, 7, self.mask) + + b = bytes([first_byte, second_byte]) + + if self.payload_length < 126: + pass + elif self.payload_length < MAX_16_BIT_INT: + # '!H' pack as 16 bit unsigned short + # add 2 byte extended payload length + b += struct.pack('!H', self.payload_length) + elif self.payload_length < MAX_64_BIT_INT: + # '!Q' = pack as 64 bit unsigned long long + # add 8 bytes extended payload length + b += struct.pack('!Q', self.payload_length) + else: + raise ValueError("Payload length exceeds 64bit integer") + + if self.masking_key: + b += self.masking_key + return b + + @classmethod + def from_file(cls, fp): + """ + read a WebSocket frame header + """ + first_byte, second_byte = fp.safe_read(2) + fin = bits.getbit(first_byte, 7) + rsv1 = bits.getbit(first_byte, 6) + rsv2 = bits.getbit(first_byte, 5) + rsv3 = bits.getbit(first_byte, 4) + opcode = first_byte & 0xF + mask_bit = bits.getbit(second_byte, 7) + length_code = second_byte & 0x7F + + # payload_length > 125 indicates you need to read more bytes + # to get the actual payload length + if length_code <= 125: + payload_length = length_code + elif length_code == 126: + payload_length, = struct.unpack("!H", fp.safe_read(2)) + else: # length_code == 127: + payload_length, = struct.unpack("!Q", fp.safe_read(8)) + + # masking key only present if mask bit set + if mask_bit == 1: + masking_key = fp.safe_read(4) + else: + masking_key = None + + return cls( + fin=fin, + rsv1=rsv1, + rsv2=rsv2, + rsv3=rsv3, + opcode=opcode, + mask=mask_bit, + length_code=length_code, + payload_length=payload_length, + masking_key=masking_key, + ) + + def __eq__(self, other): + if isinstance(other, FrameHeader): + return bytes(self) == bytes(other) + return False + + +class Frame: + """ + Represents a single WebSocket frame. + Constructor takes human readable forms of the frame components. + from_bytes() reads from a file-like object to create a new Frame. + + WebSocket frame as defined in RFC6455 + + 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 + +-+-+-+-+-------+-+-------------+-------------------------------+ + |F|R|R|R| opcode|M| Payload len | Extended payload length | + |I|S|S|S| (4) |A| (7) | (16/64) | + |N|V|V|V| |S| | (if payload len==126/127) | + | |1|2|3| |K| | | + +-+-+-+-+-------+-+-------------+ - - - - - - - - - - - - - - - + + | Extended payload length continued, if payload len == 127 | + + - - - - - - - - - - - - - - - +-------------------------------+ + | |Masking-key, if MASK set to 1 | + +-------------------------------+-------------------------------+ + | Masking-key (continued) | Payload Data | + +-------------------------------- - - - - - - - - - - - - - - - + + : Payload Data continued ... : + + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + | Payload Data continued ... | + +---------------------------------------------------------------+ + """ + + def __init__(self, payload=b"", **kwargs): + self.payload = payload + kwargs["payload_length"] = kwargs.get("payload_length", len(payload)) + self.header = FrameHeader(**kwargs) + + @classmethod + def from_bytes(cls, bytestring): + """ + Construct a websocket frame from an in-memory bytestring + to construct a frame from a stream of bytes, use from_file() directly + """ + return cls.from_file(tcp.Reader(io.BytesIO(bytestring))) + + def __repr__(self): + ret = repr(self.header) + if self.payload: + ret = ret + "\nPayload:\n" + strutils.bytes_to_escaped_str(self.payload) + return ret + + def __bytes__(self): + """ + Serialize the frame to wire format. Returns a string. + """ + b = bytes(self.header) + if self.header.masking_key: + b += Masker(self.header.masking_key)(self.payload) + else: + b += self.payload + return b + + @classmethod + def from_file(cls, fp): + """ + read a WebSocket frame sent by a server or client + + fp is a "file like" object that could be backed by a network + stream or a disk or an in memory stream reader + """ + header = FrameHeader.from_file(fp) + payload = fp.safe_read(header.payload_length) + + if header.mask == 1 and header.masking_key: + payload = Masker(header.masking_key)(payload) + + frame = cls(payload) + frame.header = header + return frame + + def __eq__(self, other): + if isinstance(other, Frame): + return bytes(self) == bytes(other) + return False diff -Nru mitmproxy-5.1.1/pathod/language/websockets.py mitmproxy-6.0.2/pathod/language/websockets.py --- mitmproxy-5.1.1/pathod/language/websockets.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/pathod/language/websockets.py 2020-12-15 16:41:27.000000000 +0000 @@ -4,9 +4,11 @@ import pyparsing as pp -import mitmproxy.net.websockets +from wsproto.frame_protocol import Opcode + from mitmproxy.utils import strutils -from . import base, generators, actions, message +from . import base, generators, actions, message, websockets_frame + NESTED_LEADER = b"pathod!" @@ -17,12 +19,12 @@ class OpCode(base.IntField): names: typing.Dict[str, int] = { - "continue": mitmproxy.net.websockets.OPCODE.CONTINUE, - "text": mitmproxy.net.websockets.OPCODE.TEXT, - "binary": mitmproxy.net.websockets.OPCODE.BINARY, - "close": mitmproxy.net.websockets.OPCODE.CLOSE, - "ping": mitmproxy.net.websockets.OPCODE.PING, - "pong": mitmproxy.net.websockets.OPCODE.PONG, + "continue": Opcode.CONTINUATION, + "text": Opcode.TEXT, + "binary": Opcode.BINARY, + "close": Opcode.CLOSE, + "ping": Opcode.PING, + "pong": Opcode.PONG, } max = 15 preamble = "c" @@ -217,11 +219,19 @@ v = getattr(self, i, None) if v is not None: frameparts[i] = v.value - frame = mitmproxy.net.websockets.FrameHeader(**frameparts) + + # import wsproto.frame_protocol + # wsproto.frame_protocol.Frame( + # opcode=frameparts["opcode"], + # payload=None, + # frame_finished=frameparts["fin"] + # ) + + frame = websockets_frame.FrameHeader(**frameparts) vals = [bytes(frame)] if bodygen: if frame.masking_key and not self.rawbody: - masker = mitmproxy.net.websockets.Masker(frame.masking_key) + masker = websockets_frame.Masker(frame.masking_key) vals.append( generators.TransformGenerator( bodygen, diff -Nru mitmproxy-5.1.1/pathod/log.py mitmproxy-6.0.2/pathod/log.py --- mitmproxy-5.1.1/pathod/log.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/pathod/log.py 2020-12-15 16:41:27.000000000 +0000 @@ -65,7 +65,7 @@ strutils.escape_control_characters( data .decode("ascii", "replace") - .replace(u"\ufffd", u".") + .replace("\ufffd", ".") ) ) for i in data.split("\n"): diff -Nru mitmproxy-5.1.1/pathod/pathoc_cmdline.py mitmproxy-6.0.2/pathod/pathoc_cmdline.py --- mitmproxy-5.1.1/pathod/pathoc_cmdline.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/pathod/pathoc_cmdline.py 2020-12-15 16:41:27.000000000 +0000 @@ -6,7 +6,7 @@ from mitmproxy.net import tls from mitmproxy import version from mitmproxy.net.http import user_agents -from . import pathoc, language +from . import print_tool_deprecation_message, pathoc, language def args_pathoc(argv, stdout=sys.stdout, stderr=sys.stderr): @@ -223,5 +223,6 @@ def go_pathoc(): # pragma: no cover + print_tool_deprecation_message() args = args_pathoc(sys.argv) pathoc.main(args) diff -Nru mitmproxy-5.1.1/pathod/pathoc.py mitmproxy-6.0.2/pathod/pathoc.py --- mitmproxy-5.1.1/pathod/pathoc.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/pathod/pathoc.py 2020-12-15 16:41:27.000000000 +0000 @@ -11,17 +11,13 @@ import OpenSSL.crypto import logging -from mitmproxy import certs -from mitmproxy import exceptions -from mitmproxy.net import tcp, tls -from mitmproxy.net import websockets -from mitmproxy.net import socks +from mitmproxy import certs, exceptions +from mitmproxy.net import tcp, tls, socks from mitmproxy.net import http as net_http from mitmproxy.coretypes import basethread from mitmproxy.utils import strutils -from pathod import log -from pathod import language +from pathod import language, log from pathod.protocols import http2 @@ -51,20 +47,20 @@ parts.append(" Certificate [%s]" % n) parts.append("\tSubject: ") for cn in i.get_subject().get_components(): - parts.append("\t\t%s=%s" % ( + parts.append("\t\t{}={}".format( strutils.always_str(cn[0], "utf8"), strutils.always_str(cn[1], "utf8")) ) parts.append("\tIssuer: ") for cn in i.get_issuer().get_components(): - parts.append("\t\t%s=%s" % ( + parts.append("\t\t{}={}".format( strutils.always_str(cn[0], "utf8"), strutils.always_str(cn[1], "utf8")) ) parts.extend( [ "\tVersion: %s" % i.get_version(), - "\tValidity: %s - %s" % ( + "\tValidity: {} - {}".format( strutils.always_str(i.get_notBefore(), "utf8"), strutils.always_str(i.get_notAfter(), "utf8") ), @@ -78,7 +74,7 @@ OpenSSL.crypto.TYPE_DSA: "DSA" } t = types.get(pk.type(), "Uknown") - parts.append("\tPubkey: %s bit %s" % (pk.bits(), t)) + parts.append(f"\tPubkey: {pk.bits()} bit {t}") s = certs.Cert(i) if s.altnames: parts.append("\tSANs: %s" % " ".join(strutils.always_str(n, "utf8") for n in s.altnames)) @@ -139,7 +135,7 @@ for rfile in r: with self.logger.ctx() as log: try: - frm = websockets.Frame.from_file(self.rfile) + frm = language.websockets_frame.Frame.from_file(self.rfile) except exceptions.TcpDisconnect: return self.frames_queue.put(frm) @@ -237,15 +233,18 @@ def http_connect(self, connect_to): req = net_http.Request( - first_line_format='authority', - method='CONNECT', - scheme=None, - host=connect_to[0].encode("idna"), + host=connect_to[0], port=connect_to[1], - path=None, - http_version='HTTP/1.1', - headers=[(b"Host", connect_to[0].encode("idna"))], + method=b'CONNECT', + scheme=b"", + authority=f"{connect_to[0]}:{connect_to[1]}".encode(), + path=b"", + http_version=b'HTTP/1.1', + headers=((b"Host", connect_to[0].encode("idna")),), content=b'', + trailers=None, + timestamp_start=0, + timestamp_end=0, ) self.wfile.write(net_http.http1.assemble_request(req)) self.wfile.flush() @@ -437,14 +436,18 @@ # build a dummy request to read the response # ideally this would be returned directly from language.serve dummy_req = net_http.Request( - first_line_format="relative", + host="localhost", + port=80, method=req["method"], scheme=b"http", - host=b"localhost", - port=80, + authority=b"", path=b"/", http_version=b"HTTP/1.1", + headers=(), content=b'', + trailers=None, + timestamp_start=time.time(), + timestamp_end=None, ) resp = self.protocol.read_response(self.rfile, dummy_req) @@ -460,7 +463,7 @@ raise finally: if resp: - lg("<< %s %s: %s bytes" % ( + lg("<< {} {}: {} bytes".format( resp.status_code, strutils.escape_control_characters(resp.reason) if resp.reason else "", len(resp.content) )) if resp.status_code in self.ignorecodes: diff -Nru mitmproxy-5.1.1/pathod/pathod_cmdline.py mitmproxy-6.0.2/pathod/pathod_cmdline.py --- mitmproxy-5.1.1/pathod/pathod_cmdline.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/pathod/pathod_cmdline.py 2020-12-15 16:41:27.000000000 +0000 @@ -7,7 +7,7 @@ from mitmproxy.net import tls from mitmproxy.utils import human from mitmproxy import version -from . import pathod +from . import print_tool_deprecation_message, pathod def parse_anchor_spec(s): @@ -231,5 +231,6 @@ def go_pathod(): # pragma: no cover + print_tool_deprecation_message() args = args_pathod(sys.argv) pathod.main(args) diff -Nru mitmproxy-5.1.1/pathod/pathod.py mitmproxy-6.0.2/pathod/pathod.py --- mitmproxy-5.1.1/pathod/pathod.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/pathod/pathod.py 2020-12-15 16:41:27.000000000 +0000 @@ -3,18 +3,14 @@ import os import sys import threading -from mitmproxy.net import tcp, tls -from mitmproxy import certs as mcerts -from mitmproxy.net import websockets -from mitmproxy import version import urllib -from mitmproxy import exceptions -from pathod import language -from pathod import utils -from pathod import log -from pathod import protocols import typing # noqa +from mitmproxy import certs as mcerts, exceptions, version +from mitmproxy.net import tcp, tls, websocket + +from pathod import language, utils, log, protocols + DEFAULT_CERT_DOMAIN = b"pathod.net" CONFDIR = "~/.mitmproxy" @@ -177,8 +173,8 @@ m = utils.MemBool() - valid_websocket_handshake = websockets.check_handshake(headers) - self.settings.websocket_key = websockets.get_client_key(headers) + valid_websocket_handshake = websocket.check_handshake(headers) + self.settings.websocket_key = websocket.get_client_key(headers) # If this is a websocket initiation, we respond with a proper # server response, unless over-ridden. @@ -491,7 +487,7 @@ utils.daemonize() try: - print("%s listening on %s" % ( + print("{} listening on {}".format( version.PATHOD, repr(pd.address) )) diff -Nru mitmproxy-5.1.1/pathod/protocols/http2.py mitmproxy-6.0.2/pathod/protocols/http2.py --- mitmproxy-5.1.1/pathod/protocols/http2.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/pathod/protocols/http2.py 2020-12-15 16:41:27.000000000 +0000 @@ -2,14 +2,13 @@ import time import hyperframe.frame -from hpack.hpack import Encoder, Decoder +from hpack.hpack import Decoder, Encoder -from mitmproxy.net.http import http2 import mitmproxy.net.http.headers -import mitmproxy.net.http.response import mitmproxy.net.http.request +import mitmproxy.net.http.response from mitmproxy.coretypes import bidi - +from mitmproxy.net.http import http2, url from .. import language @@ -98,20 +97,28 @@ timestamp_end = time.time() - first_line_format, method, scheme, host, port, path = http2.parse_headers(headers) - - request = mitmproxy.net.http.request.Request( - first_line_format, - method, - scheme, - host, - port, - path, - b"HTTP/2.0", - headers, - body, - timestamp_start, - timestamp_end, + # pseudo header must be present, see https://http2.github.io/http2-spec/#rfc.section.8.1.2.3 + authority = headers.pop(':authority', "") + method = headers.pop(':method', "") + scheme = headers.pop(':scheme', "") + path = headers.pop(':path', "") + + host, port = url.parse_authority(authority, check=False) + port = port or url.default_port(scheme) or 0 + + request = mitmproxy.net.http.Request( + host=host, + port=port, + method=method.encode(), + scheme=scheme.encode(), + authority=authority.encode(), + path=path.encode(), + http_version=b"HTTP/2.0", + headers=headers, + content=body, + trailers=None, + timestamp_start=timestamp_start, + timestamp_end=timestamp_end, ) request.stream_id = stream_id @@ -149,11 +156,12 @@ timestamp_end = None response = mitmproxy.net.http.response.Response( - b"HTTP/2.0", - int(headers.get(':status', 502)), - b'', - headers, - body, + http_version=b"HTTP/2.0", + status_code=int(headers.get(':status', 502)), + reason=b'', + headers=headers, + content=body, + trailers=None, timestamp_start=timestamp_start, timestamp_end=timestamp_end, ) @@ -252,7 +260,8 @@ def read_frame(self, hide=False): while True: - frm = http2.parse_frame(*http2.read_raw_frame(self.tcp_handler.rfile)) + frm, _ = http2.read_frame(self.tcp_handler.rfile) + if not hide and self.dump_frames: # pragma: no cover print("<< " + repr(frm)) diff -Nru mitmproxy-5.1.1/pathod/protocols/websockets.py mitmproxy-6.0.2/pathod/protocols/websockets.py --- mitmproxy-5.1.1/pathod/protocols/websockets.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/pathod/protocols/websockets.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,6 +1,5 @@ import time -from mitmproxy.net import websockets from pathod import language from mitmproxy import exceptions @@ -15,7 +14,7 @@ with logger.ctx() as lg: started = time.time() try: - frm = websockets.Frame.from_file(self.pathod_handler.rfile) + frm = language.websockets_frame.Frame.from_file(self.pathod_handler.rfile) except exceptions.NetlibException as e: lg("Error reading websocket frame: %s" % e) return None, None diff -Nru mitmproxy-5.1.1/pathod/test.py mitmproxy-6.0.2/pathod/test.py --- mitmproxy-5.1.1/pathod/test.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/pathod/test.py 2020-12-15 16:41:27.000000000 +0000 @@ -16,7 +16,7 @@ self.thread = _PaThread(self.IFACE, self.q, ssl, daemonargs) self.thread.start() self.port = self.q.get(True, 5) - self.urlbase = "%s://%s:%s" % ( + self.urlbase = "{}://{}:{}".format( "https" if ssl else "http", self.IFACE, self.port @@ -34,7 +34,7 @@ """ Return a URL that will render the response in spec. """ - return "%s/p/%s" % (self.urlbase, spec) + return f"{self.urlbase}/p/{spec}" def text_log(self) -> str: return self.logfp.getvalue() @@ -96,7 +96,7 @@ ssl=self.ssl, **self.daemonargs ) - self.name = "PathodThread (%s:%s)" % ( + self.name = "PathodThread ({}:{})".format( self.server.address[0], self.server.address[1], ) diff -Nru mitmproxy-5.1.1/README.rst mitmproxy-6.0.2/README.rst --- mitmproxy-5.1.1/README.rst 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/README.rst 2020-12-15 16:41:27.000000000 +0000 @@ -120,7 +120,7 @@ The following tools are required to build the mitmproxy docs: -- Hugo_ +- Hugo_ (the extended version ``hugo_extended`` is required) - modd_ .. code-block:: bash @@ -163,7 +163,7 @@ :target: https://stackoverflow.com/questions/tagged/mitmproxy :alt: StackOverflow: mitmproxy -.. |slack| image:: http://slack.mitmproxy.org/badge.svg +.. |slack| image:: https://shields.mitmproxy.org/badge/slack-mitmproxy-E01563.svg :target: http://slack.mitmproxy.org/ :alt: Slack Developer Chat diff -Nru mitmproxy-5.1.1/release/cibuild.py mitmproxy-6.0.2/release/cibuild.py --- mitmproxy-5.1.1/release/cibuild.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/release/cibuild.py 2020-12-15 16:41:27.000000000 +0000 @@ -181,7 +181,7 @@ t = "dev" else: t = self.version - return "mitmproxy/mitmproxy:{}".format(t) + return f"mitmproxy/mitmproxy:{t}" def dump_info(self, fp=sys.stdout) -> None: lst = [ @@ -339,23 +339,35 @@ "--dist-dir", be.dist_dir, ]) whl, = glob.glob(os.path.join(be.dist_dir, 'mitmproxy-*-py3-none-any.whl')) - click.echo("Found wheel package: {}".format(whl)) + click.echo(f"Found wheel package: {whl}") subprocess.check_call(["tox", "-e", "wheeltest", "--", whl]) return whl def build_docker_image(be: BuildEnviron): # pragma: no cover whl, = glob.glob(os.path.join(be.dist_dir, 'mitmproxy-*-py3-none-any.whl')) + whl = str(pathlib.Path(whl).relative_to(pathlib.Path(".").absolute())) click.echo("Building Docker images...") subprocess.check_call([ "docker", "build", "--tag", be.docker_tag, - "--build-arg", "WHEEL_MITMPROXY={}".format(whl), + "--build-arg", f"WHEEL_MITMPROXY={whl}", "--build-arg", "WHEEL_BASENAME_MITMPROXY={}".format(os.path.basename(whl)), "--file", "release/docker/Dockerfile", "." ]) + # smoke-test the newly built docker image + r = subprocess.run([ + "docker", + "run", + "--rm", + be.docker_tag, + "mitmdump", + "--version", + ], check=True, capture_output=True) + print(r.stdout.decode()) + assert "Mitmproxy: " in r.stdout.decode() def build_pyinstaller(be: BuildEnviron): # pragma: no cover @@ -417,7 +429,7 @@ + [tool] ) # Delete the spec file - we're good without. - os.remove("{}.spec".format(tool)) + os.remove(f"{tool}.spec") # Test if it works at all O:-) executable = os.path.join(PYINSTALLER_DIST, tool) @@ -445,7 +457,7 @@ return click.echo("Building wininstaller package...") - IB_VERSION = "20.3.0" + IB_VERSION = "20.9.0" IB_DIR = pathlib.Path(be.release_dir) / "installbuilder" IB_SETUP = IB_DIR / "setup" / f"{IB_VERSION}-installer.exe" IB_CLI = fr"C:\Program Files (x86)\VMware InstallBuilder Enterprise {IB_VERSION}\bin\builder-cli.exe" @@ -461,7 +473,7 @@ click.secho(f"Downloading... {round(100 * done / total)}%") urllib.request.urlretrieve( - f"https://installbuilder.com/installbuilder-enterprise-{IB_VERSION}-windows-installer.exe", + f"https://clients.bitrock.com/installbuilder/installbuilder-enterprise-{IB_VERSION}-windows-installer.exe", IB_SETUP.with_suffix(".tmp"), reporthook=report ) @@ -543,17 +555,17 @@ "aws", "s3", "cp", "--acl", "public-read", be.dist_dir + "/", - "s3://snapshots.mitmproxy.org/{}/".format(be.upload_dir), + f"s3://snapshots.mitmproxy.org/{be.upload_dir}/", "--recursive", ]) if be.should_upload_pypi: whl = glob.glob(os.path.join(be.dist_dir, 'mitmproxy-*-py3-none-any.whl'))[0] - click.echo("Uploading {} to PyPi...".format(whl)) + click.echo(f"Uploading {whl} to PyPi...") subprocess.check_call(["twine", "upload", whl]) if be.should_upload_docker: - click.echo("Uploading Docker image to tag={}...".format(be.docker_tag)) + click.echo(f"Uploading Docker image to tag={be.docker_tag}...") subprocess.check_call([ "docker", "login", diff -Nru mitmproxy-5.1.1/release/docker/Dockerfile mitmproxy-6.0.2/release/docker/Dockerfile --- mitmproxy-5.1.1/release/docker/Dockerfile 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/release/docker/Dockerfile 2020-12-15 16:41:27.000000000 +0000 @@ -1,4 +1,4 @@ -FROM alpine:3.11 +FROM alpine:3.12 ENV LANG=en_US.UTF-8 diff -Nru mitmproxy-5.1.1/release/hooks/hook-cryptography.py mitmproxy-6.0.2/release/hooks/hook-cryptography.py --- mitmproxy-5.1.1/release/hooks/hook-cryptography.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/release/hooks/hook-cryptography.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,44 +0,0 @@ -# Taken from the latest pyinstaller master on 2016-11-27 (0729a2b). -# flake8: noqa - -#----------------------------------------------------------------------------- -# Copyright (c) 2005-2016, PyInstaller Development Team. -# -# Distributed under the terms of the GNU General Public License with exception -# for distributing bootloader. -# -# The full license is in the file COPYING.txt, distributed with this software. -#----------------------------------------------------------------------------- - - -""" -Hook for cryptography module from the Python Cryptography Authority. -""" - -import os.path -import glob - -from PyInstaller.compat import EXTENSION_SUFFIXES -from PyInstaller.utils.hooks import collect_submodules, get_module_file_attribute -from PyInstaller.utils.hooks import copy_metadata - -# get the package data so we can load the backends -datas = copy_metadata('cryptography') - -# Add the backends as hidden imports -hiddenimports = collect_submodules('cryptography.hazmat.backends') - -# Add the OpenSSL FFI binding modules as hidden imports -hiddenimports += collect_submodules('cryptography.hazmat.bindings.openssl') + ['_cffi_backend'] - - -# Include the cffi extensions as binaries in a subfolder named like the package. -# The cffi verifier expects to find them inside the package directory for -# the main module. We cannot use hiddenimports because that would add the modules -# outside the package. -binaries = [] -cryptography_dir = os.path.dirname(get_module_file_attribute('cryptography')) -for ext in EXTENSION_SUFFIXES: - ffimods = glob.glob(os.path.join(cryptography_dir, '*_cffi_*%s*' % ext)) - for f in ffimods: - binaries.append((f, 'cryptography')) diff -Nru mitmproxy-5.1.1/release/hooks/hook-passlib.py mitmproxy-6.0.2/release/hooks/hook-passlib.py --- mitmproxy-5.1.1/release/hooks/hook-passlib.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/release/hooks/hook-passlib.py 1970-01-01 00:00:00.000000000 +0000 @@ -1 +0,0 @@ -hiddenimports = ["configparser"] diff -Nru mitmproxy-5.1.1/release/hooks/hook-pkg_resources.py mitmproxy-6.0.2/release/hooks/hook-pkg_resources.py --- mitmproxy-5.1.1/release/hooks/hook-pkg_resources.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/release/hooks/hook-pkg_resources.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,7 +0,0 @@ -# flake8: noqa - -# temporary fix for https://github.com/pypa/setuptools/issues/1963 -# can be removed when we upgrade to PyInstaller 3.7. -hiddenimports = collect_submodules('pkg_resources._vendor') -hiddenimports.append('pkg_resources.py2_warn') -excludedimports = ['__main__'] diff -Nru mitmproxy-5.1.1/release/hooks/hook-publicsuffix2.py mitmproxy-6.0.2/release/hooks/hook-publicsuffix2.py --- mitmproxy-5.1.1/release/hooks/hook-publicsuffix2.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/release/hooks/hook-publicsuffix2.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,3 +0,0 @@ -from PyInstaller.utils.hooks import collect_data_files - -datas = collect_data_files('publicsuffix2') diff -Nru mitmproxy-5.1.1/release/hooks/hook-pydivert.py mitmproxy-6.0.2/release/hooks/hook-pydivert.py --- mitmproxy-5.1.1/release/hooks/hook-pydivert.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/release/hooks/hook-pydivert.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,3 +0,0 @@ -from PyInstaller.utils.hooks import collect_data_files - -datas = collect_data_files('pydivert.windivert_dll') diff -Nru mitmproxy-5.1.1/requirements.txt mitmproxy-6.0.2/requirements.txt --- mitmproxy-5.1.1/requirements.txt 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/requirements.txt 2020-12-15 16:41:27.000000000 +0000 @@ -1 +1 @@ --e .[dev,examples] +-e .[dev] diff -Nru mitmproxy-5.1.1/setup.cfg mitmproxy-6.0.2/setup.cfg --- mitmproxy-5.1.1/setup.cfg 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/setup.cfg 2020-12-15 16:41:27.000000000 +0000 @@ -1,7 +1,7 @@ [flake8] max-line-length = 140 max-complexity = 25 -ignore = E251,E252,C901,W292,W503,W504,W605,E722,E741,E126 +ignore = E251,E252,C901,W292,W503,W504,W605,E722,E741,E126,F541 exclude = mitmproxy/contrib/*,test/mitmproxy/data/*,release/build/*,mitmproxy/io/proto/* addons = file,open,basestring,xrange,unicode,long,cmp @@ -17,9 +17,11 @@ show_missing = True exclude_lines = pragma: no cover - raise NotImplementedError() + raise NotImplementedError + raise AssertionError if typing.TYPE_CHECKING: if TYPE_CHECKING: + @overload [mypy] ignore_missing_imports = True @@ -55,13 +57,16 @@ [tool:individual_coverage] exclude = mitmproxy/addons/onboardingapp/app.py + mitmproxy/addons/session.py mitmproxy/addons/termlog.py mitmproxy/contentviews/base.py mitmproxy/controller.py mitmproxy/ctx.py mitmproxy/exceptions.py mitmproxy/flow.py + mitmproxy/io/db.py mitmproxy/io/io.py + mitmproxy/io/protobuf.py mitmproxy/io/tnetstring.py mitmproxy/log.py mitmproxy/master.py diff -Nru mitmproxy-5.1.1/setup.py mitmproxy-6.0.2/setup.py --- mitmproxy-5.1.1/setup.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/setup.py 2020-12-15 16:41:27.000000000 +0000 @@ -34,9 +34,8 @@ "Operating System :: POSIX", "Operating System :: Microsoft :: Windows", "Programming Language :: Python :: 3 :: Only", - "Programming Language :: Python :: 3.6", - "Programming Language :: Python :: 3.7", "Programming Language :: Python :: 3.8", + "Programming Language :: Python :: 3.9", "Programming Language :: Python :: Implementation :: CPython", "Topic :: Security", "Topic :: Internet :: WWW/HTTP", @@ -45,6 +44,11 @@ "Topic :: Software Development :: Testing", "Typing :: Typed", ], + project_urls={ + 'Documentation': 'https://docs.mitmproxy.org/stable/', + 'Source': 'https://github.com/mitmproxy/mitmproxy/', + 'Tracker': 'https://github.com/mitmproxy/mitmproxy/issues', + }, packages=find_packages(include=[ "mitmproxy", "mitmproxy.*", "pathod", "pathod.*", @@ -59,52 +63,50 @@ "pathoc = pathod.pathoc_cmdline:go_pathoc" ] }, + python_requires='>=3.8', # https://packaging.python.org/en/latest/requirements/#install-requires # It is not considered best practice to use install_requires to pin dependencies to specific versions. install_requires=[ + "asgiref>=3.2.10,<3.4", "blinker>=1.4, <1.5", "Brotli>=1.0,<1.1", "certifi>=2019.9.11", # no semver here - this should always be on the last release! "click>=7.0,<8", - "cryptography>=2.9,<3.0", + "cryptography>=3.3,<3.4", "flask>=1.1.1,<1.2", - "h2>=3.2.0,<4", - "hyperframe>=5.1.0,<6", - "kaitaistruct>=0.7,<0.9", - "ldap3>=2.6.1,<2.8", + "h2>=4.0,<5", + "hyperframe>=6.0,<7", + "kaitaistruct>=0.7,<0.10", + "ldap3>=2.8,<2.9", + "msgpack>=1.0.0, <1.1.0", "passlib>=1.6.5, <1.8", - "protobuf>=3.6.0, <3.12", + "protobuf>=3.14,<3.15", "pyasn1>=0.3.1,<0.5", - "pyOpenSSL>=19.1.0,<19.2", + "pyOpenSSL>=20.0,<20.1", "pyparsing>=2.4.2,<2.5", "pyperclip>=1.6.0,<1.9", "ruamel.yaml>=0.16,<0.17", - "sortedcontainers>=2.1.0,<2.2", + "sortedcontainers>=2.3,<2.4", "tornado>=4.3,<7", - "urwid>=2.1.0,<2.2", - "wsproto>=0.14,<0.16", + "urwid>=2.1.1,<2.2", + "wsproto>=1.0,<1.1", "publicsuffix2>=2.20190812,<3", - "zstandard>=0.11,<0.14", + "zstandard>=0.11,<0.15", ], extras_require={ ':sys_platform == "win32"': [ "pydivert>=2.0.3,<2.2", ], 'dev': [ - "asynctest>=0.12.0", - "Flask>=1.0,<1.2", - "hypothesis>=5.8,<5.9", + "hypothesis>=5.8,<6", "parver>=0.1,<2.0", - "pytest-asyncio>=0.10.0,<0.11", + "pytest-asyncio>=0.10.0,<0.14,!=0.14", "pytest-cov>=2.7.1,<3", "pytest-timeout>=1.3.3,<2", - "pytest-xdist>=1.29,<2", - "pytest>=5.1.3,<6", + "pytest-xdist>=2.1.0,<3", + "pytest>=6.1.0,<7", "requests>=2.9.1,<3", - "tox>=3.5,<3.15", - ], - 'examples': [ - "beautifulsoup4>=4.4.1,<4.9" + "tox>=3.5,<4", ] } ) diff -Nru mitmproxy-5.1.1/test/bench/benchmark.py mitmproxy-6.0.2/test/bench/benchmark.py --- mitmproxy-5.1.1/test/bench/benchmark.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/bench/benchmark.py 2020-12-15 16:41:27.000000000 +0000 @@ -33,7 +33,7 @@ stdout, _ = await traf.communicate() with open(ctx.options.benchmark_save_path + ".bench", mode="wb") as f: f.write(stdout) - ctx.log.error("Proxy saw %s requests, %s responses" % (self.reqs, self.resps)) + ctx.log.error(f"Proxy saw {self.reqs} requests, {self.resps} responses") ctx.log.error(stdout.decode("ascii")) backend.kill() ctx.master.shutdown() diff -Nru mitmproxy-5.1.1/test/conftest.py mitmproxy-6.0.2/test/conftest.py --- mitmproxy-5.1.1/test/conftest.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/conftest.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,7 +1,7 @@ import os import socket -from mitmproxy.utils import data +from mitmproxy.utils import data, compat import pytest @@ -22,6 +22,11 @@ reason='Skipping due to Appveyor' ) +skip_new_proxy_core = pytest.mark.skipif( + compat.new_proxy_core, + reason='Skipping legacy test for old proxy core' +) + try: s = socket.socket(socket.AF_INET6, socket.SOCK_STREAM) s.bind(("::1", 0)) diff -Nru mitmproxy-5.1.1/test/examples/test_examples.py mitmproxy-6.0.2/test/examples/test_examples.py --- mitmproxy-5.1.1/test/examples/test_examples.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/examples/test_examples.py 2020-12-15 16:41:27.000000000 +0000 @@ -10,35 +10,21 @@ class TestScripts(tservers.MasterTest): def test_add_header(self, tdata): with taddons.context() as tctx: - a = tctx.script(tdata.path("../examples/simple/add_header.py")) - f = tflow.tflow(resp=tutils.tresp()) - a.response(f) - assert f.response.headers["newheader"] == "foo" + a = tctx.script(tdata.path("../examples/addons/scripting-minimal-example.py")) + f = tflow.tflow() + a.request(f) + assert f.request.headers["myheader"] == "value" def test_custom_contentviews(self, tdata): with taddons.context() as tctx: - tctx.script(tdata.path("../examples/simple/custom_contentview.py")) + tctx.script(tdata.path("../examples/addons/contentview.py")) swapcase = contentviews.get("swapcase") _, fmt = swapcase(b"Test!") assert any(b'tEST!' in val[0][1] for val in fmt) - def test_iframe_injector(self, tdata): - with taddons.context() as tctx: - sc = tctx.script(tdata.path("../examples/simple/modify_body_inject_iframe.py")) - tctx.configure( - sc, - iframe = "http://example.org/evil_iframe" - ) - f = tflow.tflow( - resp=tutils.tresp(content=b"mitmproxy") - ) - tctx.master.addons.invoke_addon(sc, "response", f) - content = f.response.content - assert b'iframe' in content and b'evil_iframe' in content - def test_modify_form(self, tdata): with taddons.context() as tctx: - sc = tctx.script(tdata.path("../examples/simple/modify_form.py")) + sc = tctx.script(tdata.path("../examples/addons/http-modify-form.py")) form_header = Headers(content_type="application/x-www-form-urlencoded") f = tflow.tflow(req=tutils.treq(headers=form_header)) @@ -52,7 +38,7 @@ def test_modify_querystring(self, tdata): with taddons.context() as tctx: - sc = tctx.script(tdata.path("../examples/simple/modify_querystring.py")) + sc = tctx.script(tdata.path("../examples/addons/http-modify-query-string.py")) f = tflow.tflow(req=tutils.treq(path="/search?q=term")) sc.request(f) @@ -64,36 +50,14 @@ def test_redirect_requests(self, tdata): with taddons.context() as tctx: - sc = tctx.script(tdata.path("../examples/simple/redirect_requests.py")) + sc = tctx.script(tdata.path("../examples/addons/http-redirect-requests.py")) f = tflow.tflow(req=tutils.treq(host="example.org")) sc.request(f) assert f.request.host == "mitmproxy.org" def test_send_reply_from_proxy(self, tdata): with taddons.context() as tctx: - sc = tctx.script(tdata.path("../examples/simple/send_reply_from_proxy.py")) + sc = tctx.script(tdata.path("../examples/addons/http-reply-from-proxy.py")) f = tflow.tflow(req=tutils.treq(host="example.com", port=80)) sc.request(f) assert f.response.content == b"Hello World" - - def test_dns_spoofing(self, tdata): - with taddons.context() as tctx: - sc = tctx.script(tdata.path("../examples/complex/dns_spoofing.py")) - - original_host = "example.com" - - host_header = Headers(host=original_host) - f = tflow.tflow(req=tutils.treq(headers=host_header, port=80)) - - tctx.master.addons.invoke_addon(sc, "requestheaders", f) - - # Rewrite by reverse proxy mode - f.request.scheme = "https" - f.request.port = 443 - - tctx.master.addons.invoke_addon(sc, "request", f) - - assert f.request.scheme == "http" - assert f.request.port == 80 - - assert f.request.headers["Host"] == original_host diff -Nru mitmproxy-5.1.1/test/examples/test_har_dump.py mitmproxy-6.0.2/test/examples/test_har_dump.py --- mitmproxy-5.1.1/test/examples/test_har_dump.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/examples/test_har_dump.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,84 +0,0 @@ -import json - -from mitmproxy.test import tflow -from mitmproxy.test import tutils -from mitmproxy.test import taddons -from mitmproxy.net.http import cookies - - -class TestHARDump: - def flow(self, resp_content=b'message'): - times = dict( - timestamp_start=746203272, - timestamp_end=746203272, - ) - - # Create a dummy flow for testing - return tflow.tflow( - req=tutils.treq(method=b'GET', **times), - resp=tutils.tresp(content=resp_content, **times) - ) - - def test_simple(self, tmpdir, tdata): - with taddons.context() as tctx: - a = tctx.script(tdata.path("../examples/complex/har_dump.py")) - path = str(tmpdir.join("somefile")) - tctx.configure(a, hardump=path) - tctx.invoke(a, "response", self.flow()) - tctx.invoke(a, "done") - with open(path, "r") as inp: - har = json.load(inp) - assert len(har["log"]["entries"]) == 1 - - def test_base64(self, tmpdir, tdata): - with taddons.context() as tctx: - a = tctx.script(tdata.path("../examples/complex/har_dump.py")) - path = str(tmpdir.join("somefile")) - tctx.configure(a, hardump=path) - - tctx.invoke( - a, "response", self.flow(resp_content=b"foo" + b"\xFF" * 10) - ) - tctx.invoke(a, "done") - with open(path, "r") as inp: - har = json.load(inp) - assert har["log"]["entries"][0]["response"]["content"]["encoding"] == "base64" - - def test_format_cookies(self, tdata): - with taddons.context() as tctx: - a = tctx.script(tdata.path("../examples/complex/har_dump.py")) - - CA = cookies.CookieAttrs - - f = a.format_cookies([("n", "v", CA([("k", "v")]))])[0] - assert f['name'] == "n" - assert f['value'] == "v" - assert not f['httpOnly'] - assert not f['secure'] - - f = a.format_cookies([("n", "v", CA([("httponly", None), ("secure", None)]))])[0] - assert f['httpOnly'] - assert f['secure'] - - f = a.format_cookies([("n", "v", CA([("expires", "Mon, 24-Aug-2037 00:00:00 GMT")]))])[0] - assert f['expires'] - - def test_binary(self, tmpdir, tdata): - with taddons.context() as tctx: - a = tctx.script(tdata.path("../examples/complex/har_dump.py")) - path = str(tmpdir.join("somefile")) - tctx.configure(a, hardump=path) - - f = self.flow() - f.request.method = "POST" - f.request.headers["content-type"] = "application/x-www-form-urlencoded" - f.request.content = b"foo=bar&baz=s%c3%bc%c3%9f" - f.response.headers["random-junk"] = bytes(range(256)) - f.response.content = bytes(range(256)) - - tctx.invoke(a, "response", f) - tctx.invoke(a, "done") - - with open(path, "r") as inp: - har = json.load(inp) - assert len(har["log"]["entries"]) == 1 diff -Nru mitmproxy-5.1.1/test/examples/test_xss_scanner.py mitmproxy-6.0.2/test/examples/test_xss_scanner.py --- mitmproxy-5.1.1/test/examples/test_xss_scanner.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/examples/test_xss_scanner.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,391 +0,0 @@ -import pytest -import requests -from examples.complex import xss_scanner as xss -from mitmproxy.test import tflow, tutils - - -class TestXSSScanner(): - def test_get_XSS_info(self): - # First type of exploit: - # Exploitable: - xss_info = xss.get_XSS_data(b"" % - xss.FULL_PAYLOAD, - "https://example.com", - "End of URL") - expected_xss_info = xss.XSSData('https://example.com', - "End of URL", - '" % - xss.FULL_PAYLOAD.replace(b"'", b"%27").replace(b'"', b"%22"), - "https://example.com", - "End of URL") - expected_xss_info = xss.XSSData("https://example.com", - "End of URL", - '" % - xss.FULL_PAYLOAD.replace(b"'", b"%27").replace(b'"', b"%22").replace(b"/", b"%2F"), - "https://example.com", - "End of URL") - assert xss_info is None - # Second type of exploit: - # Exploitable: - xss_info = xss.get_XSS_data(b"" % - xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b">", b"%3E").replace(b"\"", b"%22"), - "https://example.com", - "End of URL") - expected_xss_info = xss.XSSData("https://example.com", - "End of URL", - "';alert(0);g='", - xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b">", b"%3E") - .replace(b"\"", b"%22").decode('utf-8')) - assert xss_info == expected_xss_info - # Non-Exploitable: - xss_info = xss.get_XSS_data(b"" % - xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b"\"", b"%22").replace(b"'", b"%22"), - "https://example.com", - "End of URL") - assert xss_info is None - # Third type of exploit: - # Exploitable: - xss_info = xss.get_XSS_data(b"" % - xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b">", b"%3E").replace(b"'", b"%27"), - "https://example.com", - "End of URL") - expected_xss_info = xss.XSSData("https://example.com", - "End of URL", - '";alert(0);g="', - xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b">", b"%3E") - .replace(b"'", b"%27").decode('utf-8')) - assert xss_info == expected_xss_info - # Non-Exploitable: - xss_info = xss.get_XSS_data(b"" % - xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b"'", b"%27").replace(b"\"", b"%22"), - "https://example.com", - "End of URL") - assert xss_info is None - # Fourth type of exploit: Test - # Exploitable: - xss_info = xss.get_XSS_data(b"Test" % - xss.FULL_PAYLOAD, - "https://example.com", - "End of URL") - expected_xss_info = xss.XSSData("https://example.com", - "End of URL", - "'>", - xss.FULL_PAYLOAD.decode('utf-8')) - assert xss_info == expected_xss_info - # Non-Exploitable: - xss_info = xss.get_XSS_data(b"Test" % - xss.FULL_PAYLOAD.replace(b"'", b"%27"), - "https://example.com", - "End of URL") - assert xss_info is None - # Fifth type of exploit: Test - # Exploitable: - xss_info = xss.get_XSS_data(b"Test" % - xss.FULL_PAYLOAD.replace(b"'", b"%27"), - "https://example.com", - "End of URL") - expected_xss_info = xss.XSSData("https://example.com", - "End of URL", - "\">", - xss.FULL_PAYLOAD.replace(b"'", b"%27").decode('utf-8')) - assert xss_info == expected_xss_info - # Non-Exploitable: - xss_info = xss.get_XSS_data(b"Test" % - xss.FULL_PAYLOAD.replace(b"'", b"%27").replace(b"\"", b"%22"), - "https://example.com", - "End of URL") - assert xss_info is None - # Sixth type of exploit: Test - # Exploitable: - xss_info = xss.get_XSS_data(b"Test" % - xss.FULL_PAYLOAD, - "https://example.com", - "End of URL") - expected_xss_info = xss.XSSData("https://example.com", - "End of URL", - ">", - xss.FULL_PAYLOAD.decode('utf-8')) - assert xss_info == expected_xss_info - # Non-Exploitable - xss_info = xss.get_XSS_data(b"Test" % - xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b">", b"%3E") - .replace(b"=", b"%3D"), - "https://example.com", - "End of URL") - assert xss_info is None - # Seventh type of exploit: PAYLOAD - # Exploitable: - xss_info = xss.get_XSS_data(b"%s" % - xss.FULL_PAYLOAD, - "https://example.com", - "End of URL") - expected_xss_info = xss.XSSData("https://example.com", - "End of URL", - "", - xss.FULL_PAYLOAD.decode('utf-8')) - assert xss_info == expected_xss_info - # Non-Exploitable - xss_info = xss.get_XSS_data(b"%s" % - xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b">", b"%3E").replace(b"/", b"%2F"), - "https://example.com", - "End of URL") - assert xss_info is None - # Eighth type of exploit: Test - # Exploitable: - xss_info = xss.get_XSS_data(b"Test" % - xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b">", b"%3E"), - "https://example.com", - "End of URL") - expected_xss_info = xss.XSSData("https://example.com", - "End of URL", - "Javascript:alert(0)", - xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b">", b"%3E").decode('utf-8')) - assert xss_info == expected_xss_info - # Non-Exploitable: - xss_info = xss.get_XSS_data(b"Test" % - xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b">", b"%3E") - .replace(b"=", b"%3D"), - "https://example.com", - "End of URL") - assert xss_info is None - # Ninth type of exploit: Test - # Exploitable: - xss_info = xss.get_XSS_data(b"Test" % - xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b">", b"%3E"), - "https://example.com", - "End of URL") - expected_xss_info = xss.XSSData("https://example.com", - "End of URL", - '" onmouseover="alert(0)" t="', - xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b">", b"%3E").decode('utf-8')) - assert xss_info == expected_xss_info - # Non-Exploitable: - xss_info = xss.get_XSS_data(b"Test" % - xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b">", b"%3E") - .replace(b'"', b"%22"), - "https://example.com", - "End of URL") - assert xss_info is None - # Tenth type of exploit: Test - # Exploitable: - xss_info = xss.get_XSS_data(b"Test" % - xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b">", b"%3E"), - "https://example.com", - "End of URL") - expected_xss_info = xss.XSSData("https://example.com", - "End of URL", - "' onmouseover='alert(0)' t='", - xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b">", b"%3E").decode('utf-8')) - assert xss_info == expected_xss_info - # Non-Exploitable: - xss_info = xss.get_XSS_data(b"Test" % - xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b">", b"%3E") - .replace(b"'", b"%22"), - "https://example.com", - "End of URL") - assert xss_info is None - # Eleventh type of exploit: Test - # Exploitable: - xss_info = xss.get_XSS_data(b"Test" % - xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b">", b"%3E"), - "https://example.com", - "End of URL") - expected_xss_info = xss.XSSData("https://example.com", - "End of URL", - " onmouseover=alert(0) t=", - xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b">", b"%3E").decode('utf-8')) - assert xss_info == expected_xss_info - # Non-Exploitable: - xss_info = xss.get_XSS_data(b"Test" % - xss.FULL_PAYLOAD.replace(b"<", b"%3C").replace(b">", b"%3E") - .replace(b"=", b"%3D"), - "https://example.com", - "End of URL") - assert xss_info is None - - def test_get_SQLi_data(self): - sqli_data = xss.get_SQLi_data("SQL syntax MySQL", - "", - "https://example.com", - "End of URL") - expected_sqli_data = xss.SQLiData("https://example.com", - "End of URL", - "SQL syntax.*MySQL", - "MySQL") - assert sqli_data == expected_sqli_data - sqli_data = xss.get_SQLi_data("SQL syntax MySQL", - "SQL syntax MySQL", - "https://example.com", - "End of URL") - assert sqli_data is None - - def test_inside_quote(self): - assert not xss.inside_quote("'", b"no", 0, b"no") - assert xss.inside_quote("'", b"yes", 0, b"'yes'") - assert xss.inside_quote("'", b"yes", 1, b"'yes'otherJunk'yes'more") - assert not xss.inside_quote("'", b"longStringNotInIt", 1, b"short") - - def test_paths_to_text(self): - text = xss.paths_to_text("""

STRING

- - """, "STRING") - expected_text = ["/html/head/h1", "/html/script"] - assert text == expected_text - assert xss.paths_to_text("""""", "STRING") == [] - - def mocked_requests_vuln(*args, headers=None, cookies=None): - class MockResponse: - def __init__(self, html, headers=None, cookies=None): - self.text = html - return MockResponse("%s" % xss.FULL_PAYLOAD) - - def mocked_requests_invuln(*args, headers=None, cookies=None): - class MockResponse: - def __init__(self, html, headers=None, cookies=None): - self.text = html - return MockResponse("") - - def test_test_end_of_url_injection(self, get_request_vuln): - xss_info = xss.test_end_of_URL_injection("", "https://example.com/index.html", {})[0] - expected_xss_info = xss.XSSData('https://example.com/index.html/1029zxcs\'d"aoso[sb]po(pc)se;sl/bsl\\eq=3847asd', - 'End of URL', - '', - '1029zxcs\\\'d"aoso[sb]po(pc)se;sl/bsl\\\\eq=3847asd') - sqli_info = xss.test_end_of_URL_injection("", "https://example.com/", {})[1] - assert xss_info == expected_xss_info - assert sqli_info is None - - def test_test_referer_injection(self, get_request_vuln): - xss_info = xss.test_referer_injection("", "https://example.com/", {})[0] - expected_xss_info = xss.XSSData('https://example.com/', - 'Referer', - '', - '1029zxcs\\\'d"aoso[sb]po(pc)se;sl/bsl\\\\eq=3847asd') - sqli_info = xss.test_referer_injection("", "https://example.com/", {})[1] - assert xss_info == expected_xss_info - assert sqli_info is None - - def test_test_user_agent_injection(self, get_request_vuln): - xss_info = xss.test_user_agent_injection("", "https://example.com/", {})[0] - expected_xss_info = xss.XSSData('https://example.com/', - 'User Agent', - '', - '1029zxcs\\\'d"aoso[sb]po(pc)se;sl/bsl\\\\eq=3847asd') - sqli_info = xss.test_user_agent_injection("", "https://example.com/", {})[1] - assert xss_info == expected_xss_info - assert sqli_info is None - - def test_test_query_injection(self, get_request_vuln): - - xss_info = xss.test_query_injection("", "https://example.com/vuln.php?cmd=ls", {})[0] - expected_xss_info = xss.XSSData('https://example.com/vuln.php?cmd=1029zxcs\'d"aoso[sb]po(pc)se;sl/bsl\\eq=3847asd', - 'Query', - '', - '1029zxcs\\\'d"aoso[sb]po(pc)se;sl/bsl\\\\eq=3847asd') - sqli_info = xss.test_query_injection("", "https://example.com/vuln.php?cmd=ls", {})[1] - assert xss_info == expected_xss_info - assert sqli_info is None - - @pytest.fixture(scope='function') - def logger(self, monkeypatch): - class Logger(): - def __init__(self): - self.args = [] - - def info(self, str): - self.args.append(str) - - def error(self, str): - self.args.append(str) - - logger = Logger() - monkeypatch.setattr("mitmproxy.ctx.log", logger) - yield logger - - @pytest.fixture(scope='function') - def get_request_vuln(self, monkeypatch): - monkeypatch.setattr(requests, 'get', self.mocked_requests_vuln) - - @pytest.fixture(scope='function') - def get_request_invuln(self, monkeypatch): - monkeypatch.setattr(requests, 'get', self.mocked_requests_invuln) - - @pytest.fixture(scope='function') - def mock_gethostbyname(self, monkeypatch): - def gethostbyname(domain): - claimed_domains = ["google.com"] - if domain not in claimed_domains: - from socket import gaierror - raise gaierror("[Errno -2] Name or service not known") - else: - return '216.58.221.46' - - monkeypatch.setattr("socket.gethostbyname", gethostbyname) - - def test_find_unclaimed_URLs(self, logger, mock_gethostbyname): - xss.find_unclaimed_URLs("", - "https://example.com") - assert logger.args == [] - xss.find_unclaimed_URLs("", - "https://example.com") - assert logger.args[0] == 'XSS found in https://example.com due to unclaimed URL "http://unclaimedDomainName.com".' - xss.find_unclaimed_URLs("", - "https://example.com") - assert logger.args[1] == 'XSS found in https://example.com due to unclaimed URL "http://unclaimedDomainName.com".' - xss.find_unclaimed_URLs("", - "https://example.com") - assert logger.args[2] == 'XSS found in https://example.com due to unclaimed URL "http://unclaimedDomainName.com".' - - def test_log_XSS_data(self, logger): - xss.log_XSS_data(None) - assert logger.args == [] - # self, url: str, injection_point: str, exploit: str, line: str - xss.log_XSS_data(xss.XSSData('https://example.com', - 'Location', - 'String', - 'Line of HTML')) - assert logger.args[0] == '===== XSS Found ====' - assert logger.args[1] == 'XSS URL: https://example.com' - assert logger.args[2] == 'Injection Point: Location' - assert logger.args[3] == 'Suggested Exploit: String' - assert logger.args[4] == 'Line: Line of HTML' - - def test_log_SQLi_data(self, logger): - xss.log_SQLi_data(None) - assert logger.args == [] - xss.log_SQLi_data(xss.SQLiData('https://example.com', - 'Location', - 'Oracle.*Driver', - 'Oracle')) - assert logger.args[0] == '===== SQLi Found =====' - assert logger.args[1] == 'SQLi URL: https://example.com' - assert logger.args[2] == 'Injection Point: Location' - assert logger.args[3] == 'Regex used: Oracle.*Driver' - - def test_get_cookies(self): - mocked_req = tutils.treq() - mocked_req.cookies = [("cookieName2", "cookieValue2")] - mocked_flow = tflow.tflow(req=mocked_req) - # It only uses the request cookies - assert xss.get_cookies(mocked_flow) == {"cookieName2": "cookieValue2"} - - def test_response(self, get_request_invuln, logger): - mocked_flow = tflow.tflow( - req=tutils.treq(path=b"index.html?q=1"), - resp=tutils.tresp(content=b'') - ) - xss.response(mocked_flow) - assert logger.args == [] - - def test_data_equals(self): - xssData = xss.XSSData("a", "b", "c", "d") - sqliData = xss.SQLiData("a", "b", "c", "d") - assert xssData == xssData - assert sqliData == sqliData diff -Nru mitmproxy-5.1.1/test/filename_matching.py mitmproxy-6.0.2/test/filename_matching.py --- mitmproxy-5.1.1/test/filename_matching.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/filename_matching.py 2020-12-15 16:41:27.000000000 +0000 @@ -44,14 +44,14 @@ if missing_test_files: exitcode += 1 for f, p in sorted(missing_test_files): - print("{} MUST have a matching test file: {}".format(f, p)) + print(f"{f} MUST have a matching test file: {p}") unknown_test_files = check_test_files_have_src() if unknown_test_files: # TODO: enable this in the future # exitcode += 1 for f, p in sorted(unknown_test_files): - print("{} DOES NOT MATCH a source file! Expected to find: {}".format(f, p)) + print(f"{f} DOES NOT MATCH a source file! Expected to find: {p}") sys.exit(exitcode) diff -Nru mitmproxy-5.1.1/test/full_coverage_plugin.py mitmproxy-6.0.2/test/full_coverage_plugin.py --- mitmproxy-5.1.1/test/full_coverage_plugin.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/full_coverage_plugin.py 2020-12-15 16:41:27.000000000 +0000 @@ -68,7 +68,7 @@ yield - coverage_values = dict([(name, 0) for name in session.config.option.full_cov]) + coverage_values = {name: 0 for name in session.config.option.full_cov} prefix = os.getcwd() @@ -113,7 +113,7 @@ markup = {'red': True, 'bold': True} for s, v in sorted(coverage_values[name][1]): if v < 100: - msg += ' {}: {:.2f}%\n'.format(s, v) + msg += f' {s}: {v:.2f}%\n' else: markup = {'green': True} terminalreporter.write(msg, **markup) @@ -124,5 +124,5 @@ msg = '\nExcluded files:\n' for s in sorted(no_full_cov): - msg += " {}\n".format(s) + msg += f" {s}\n" terminalreporter.write(msg) diff -Nru mitmproxy-5.1.1/test/helper_tools/memoryleak.py mitmproxy-6.0.2/test/helper_tools/memoryleak.py --- mitmproxy-5.1.1/test/helper_tools/memoryleak.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/helper_tools/memoryleak.py 2020-12-15 16:41:27.000000000 +0000 @@ -24,8 +24,8 @@ def request(ctx, flow): global step, ssl print("==========") - print("GC: {}".format(gc.collect())) - print("Threads: {}".format(threading.active_count())) + print(f"GC: {gc.collect()}") + print(f"Threads: {threading.active_count()}") step += 1 if step == 1: diff -Nru mitmproxy-5.1.1/test/individual_coverage.py mitmproxy-6.0.2/test/individual_coverage.py --- mitmproxy-5.1.1/test/individual_coverage.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/individual_coverage.py 2020-12-15 16:41:27.000000000 +0000 @@ -31,7 +31,7 @@ print("FAIL DUE TO UNEXPECTED SUCCESS:", src, "Please remove this file from setup.cfg tool:individual_coverage/exclude.") e = 42 else: - print("Success:", src) + print(".") else: if fail: print("Ignoring allowed fail:", src) diff -Nru mitmproxy-5.1.1/test/mitmproxy/addons/test_asgiapp.py mitmproxy-6.0.2/test/mitmproxy/addons/test_asgiapp.py --- mitmproxy-5.1.1/test/mitmproxy/addons/test_asgiapp.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/addons/test_asgiapp.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,84 @@ +import asyncio +import json +import sys +from unittest import mock + +import flask +import pytest +from flask import request + +from .. import tservers +from mitmproxy.addons import asgiapp +from mitmproxy.test import tflow + +tapp = flask.Flask(__name__) + + +@tapp.route("/") +def hello(): + return "testapp" + + +@tapp.route("/parameters") +def request_check(): + args = {} + for k in request.args.keys(): + args[k] = request.args[k] + return json.dumps(args) + + +@tapp.route("/error") +def error(): + raise ValueError("An exception...") + + +async def errapp(scope, receive, send): + raise ValueError("errapp") + + +async def noresponseapp(scope, receive, send): + return + + +class TestApp(tservers.HTTPProxyTest): + def addons(self): + return [ + asgiapp.WSGIApp(tapp, "testapp", 80), + asgiapp.ASGIApp(errapp, "errapp", 80), + asgiapp.ASGIApp(noresponseapp, "noresponseapp", 80), + ] + + def test_simple(self): + p = self.pathoc() + with p.connect(): + ret = p.request("get:'http://testapp/'") + assert b"testapp" in ret.content + + def test_parameters(self): + p = self.pathoc() + with p.connect(): + ret = p.request("get:'http://testapp/parameters?param1=1¶m2=2'") + assert b'{"param1": "1", "param2": "2"}' == ret.data.content + + def test_app_err(self): + p = self.pathoc() + with p.connect(): + ret = p.request("get:'http://errapp/?foo=bar'") + assert ret.status_code == 500 + assert b"ASGI Error" in ret.content + + def test_app_no_response(self): + p = self.pathoc() + with p.connect(): + ret = p.request("get:'http://noresponseapp/'") + assert ret.status_code == 500 + assert b"ASGI Error" in ret.content + + @pytest.mark.skipif(sys.version_info < (3, 8), reason='requires Python 3.8 or higher') + def test_app_not_serve_loading_flows(self): + with mock.patch('mitmproxy.addons.asgiapp.serve') as mck: + flow = tflow.tflow() + flow.request.host = "testapp" + flow.request.port = 80 + asyncio.run_coroutine_threadsafe(self.master.load_flow(flow), self.master.channel.loop).result() + mck.assert_not_awaited() diff -Nru mitmproxy-5.1.1/test/mitmproxy/addons/test_block.py mitmproxy-6.0.2/test/mitmproxy/addons/test_block.py --- mitmproxy-5.1.1/test/mitmproxy/addons/test_block.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/addons/test_block.py 2020-12-15 16:41:27.000000000 +0000 @@ -3,6 +3,7 @@ from mitmproxy.addons import block from mitmproxy.test import taddons +from mitmproxy.utils import compat @pytest.mark.parametrize("block_global, block_private, should_be_killed, address", [ @@ -55,6 +56,15 @@ async def test_block_global(block_global, block_private, should_be_killed, address): ar = block.Block() with taddons.context(ar) as tctx: + if compat.new_proxy_core: + from mitmproxy.proxy2 import context + + tctx.configure(ar, block_global=block_global, block_private=block_private) + client = context.Client(address, ("127.0.0.1", 8080), 1607699500) + ar.client_connected(client) + assert bool(client.error) == should_be_killed + return + tctx.options.block_global = block_global tctx.options.block_private = block_private with mock.patch('mitmproxy.proxy.protocol.base.Layer') as layer: diff -Nru mitmproxy-5.1.1/test/mitmproxy/addons/test_clientplayback.py mitmproxy-6.0.2/test/mitmproxy/addons/test_clientplayback.py --- mitmproxy-5.1.1/test/mitmproxy/addons/test_clientplayback.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/addons/test_clientplayback.py 2020-12-15 16:41:27.000000000 +0000 @@ -10,6 +10,7 @@ from mitmproxy.test import taddons from .. import tservers +from ...conftest import skip_new_proxy_core def tdump(path, flows): @@ -48,6 +49,7 @@ def addons(self): return [clientplayback.ClientPlayback()] + @skip_new_proxy_core def test_replay(self): cr = self.master.addons.get("clientplayback") @@ -144,6 +146,9 @@ f.request.raw_content = None assert "missing content" in cp.check(f) + f = tflow.ttcpflow() + assert "Can only replay HTTP" in cp.check(f) + @pytest.mark.asyncio async def test_playback(self): cp = clientplayback.ClientPlayback() @@ -161,6 +166,7 @@ assert cp.count() == 0 await ctx.master.await_log("live") + @skip_new_proxy_core def test_http2(self): cp = clientplayback.ClientPlayback() with taddons.context(cp): diff -Nru mitmproxy-5.1.1/test/mitmproxy/addons/test_command_history.py mitmproxy-6.0.2/test/mitmproxy/addons/test_command_history.py --- mitmproxy-5.1.1/test/mitmproxy/addons/test_command_history.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/addons/test_command_history.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,4 +1,8 @@ import os +from unittest.mock import patch +from pathlib import Path + +import pytest from mitmproxy.addons import command_history from mitmproxy.test import taddons @@ -19,19 +23,38 @@ ch.add_command("cmd4") ch.done() - with open(history_file, "r") as f: + with open(history_file) as f: assert f.read() == "cmd3\ncmd4\n" + @pytest.mark.asyncio + async def test_done_writing_failed(self): + ch = command_history.CommandHistory() + ch.VACUUM_SIZE = 1 + with taddons.context(ch) as tctx: + ch.history.append('cmd1') + ch.history.append('cmd2') + ch.history.append('cmd3') + tctx.options.confdir = '/non/existent/path/foobar1234/' + ch.done() + assert await tctx.master.await_log(f"Failed writing to {ch.history_file}") + def test_add_command(self): - history = command_history.CommandHistory() + ch = command_history.CommandHistory() - history.add_command('cmd1') - history.add_command('cmd2') + ch.add_command('cmd1') + ch.add_command('cmd2') + assert ch.history == ['cmd1', 'cmd2'] - assert history.history == ['cmd1', 'cmd2'] + ch.add_command('') + assert ch.history == ['cmd1', 'cmd2'] - history.add_command('') - assert history.history == ['cmd1', 'cmd2'] + @pytest.mark.asyncio + async def test_add_command_failed(self): + ch = command_history.CommandHistory() + with taddons.context(ch) as tctx: + tctx.options.confdir = '/non/existent/path/foobar1234/' + ch.add_command('cmd1') + assert await tctx.master.await_log(f"Failed writing to {ch.history_file}") def test_get_next_and_prev(self, tmpdir): ch = command_history.CommandHistory() @@ -133,6 +156,20 @@ ch.clear_history() + @pytest.mark.asyncio + async def test_clear_failed(self, monkeypatch): + ch = command_history.CommandHistory() + + with taddons.context(ch) as tctx: + tctx.options.confdir = '/non/existent/path/foobar1234/' + + with patch.object(Path, 'exists') as mock_exists: + mock_exists.return_value = True + with patch.object(Path, 'unlink') as mock_unlink: + mock_unlink.side_effect = IOError() + ch.clear_history() + assert await tctx.master.await_log(f"Failed deleting {ch.history_file}") + def test_filter(self, tmpdir): ch = command_history.CommandHistory() @@ -239,7 +276,7 @@ instances.pop(0).done() _path = os.path.join(tctx.options.confdir, 'command_history') - lines = open(_path, 'r').readlines() + lines = open(_path).readlines() saved_commands = [cmd.strip() for cmd in lines] assert saved_commands == ['cmd1', 'cmd2', 'cmd3', 'cmd4', 'cmd_before_close', 'new_cmd'] @@ -267,6 +304,6 @@ instances.pop().done() _path = os.path.join(tctx.options.confdir, 'command_history') - lines = open(_path, 'r').readlines() + lines = open(_path).readlines() saved_commands = [cmd.strip() for cmd in lines] assert saved_commands == ['cmd1', 'cmd2', 'cmd3', 'cmd4', 'cmd5'] diff -Nru mitmproxy-5.1.1/test/mitmproxy/addons/test_core.py mitmproxy-6.0.2/test/mitmproxy/addons/test_core.py --- mitmproxy-5.1.1/test/mitmproxy/addons/test_core.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/addons/test_core.py 2020-12-15 16:41:27.000000000 +0000 @@ -161,18 +161,12 @@ tctx.configure(sa, body_size_limit = "invalid") tctx.configure(sa, body_size_limit = "1m") - with pytest.raises(exceptions.OptionsError, match="mutually exclusive"): + with pytest.raises(exceptions.OptionsError, match="requires the upstream_cert option to be enabled"): tctx.configure( sa, add_upstream_certs_to_client_chain = True, upstream_cert = False ) - with pytest.raises(exceptions.OptionsError, match="requires certificate verification to be disabled"): - tctx.configure( - sa, - add_upstream_certs_to_client_chain = True, - ssl_insecure = False - ) with pytest.raises(exceptions.OptionsError, match="Invalid mode"): tctx.configure( sa, diff -Nru mitmproxy-5.1.1/test/mitmproxy/addons/test_cut.py mitmproxy-6.0.2/test/mitmproxy/addons/test_cut.py --- mitmproxy-5.1.1/test/mitmproxy/addons/test_cut.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/addons/test_cut.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,4 +1,3 @@ - from mitmproxy.addons import cut from mitmproxy.addons import view from mitmproxy import exceptions @@ -40,14 +39,14 @@ ["client_conn.address.host", "127.0.0.1"], ["client_conn.tls_version", "TLSv1.2"], ["client_conn.sni", "address"], - ["client_conn.tls_established", "false"], + ["client_conn.tls_established", "true"], ["server_conn.address.port", "22"], ["server_conn.address.host", "address"], ["server_conn.ip_address.host", "192.168.0.1"], ["server_conn.tls_version", "TLSv1.2"], ["server_conn.sni", "address"], - ["server_conn.tls_established", "false"], + ["server_conn.tls_established", "true"], ] for spec, expected in tests: ret = cut.extract(spec, tf) diff -Nru mitmproxy-5.1.1/test/mitmproxy/addons/test_disable_h2c.py mitmproxy-6.0.2/test/mitmproxy/addons/test_disable_h2c.py --- mitmproxy-5.1.1/test/mitmproxy/addons/test_disable_h2c.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/addons/test_disable_h2c.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,10 +1,10 @@ import io -from mitmproxy import http + from mitmproxy.addons import disable_h2c -from mitmproxy.net.http import http1 from mitmproxy.exceptions import Kill -from mitmproxy.test import tflow +from mitmproxy.net.http import http1 from mitmproxy.test import taddons +from mitmproxy.test import tflow class TestDisableH2CleartextUpgrade: @@ -30,7 +30,7 @@ b = io.BytesIO(b"PRI * HTTP/2.0\r\n\r\nSM\r\n\r\n") f = tflow.tflow() - f.request = http.HTTPRequest.wrap(http1.read_request(b)) + f.request = http1.read_request(b) f.intercept() a.request(f) diff -Nru mitmproxy-5.1.1/test/mitmproxy/addons/test_dumper.py mitmproxy-6.0.2/test/mitmproxy/addons/test_dumper.py --- mitmproxy-5.1.1/test/mitmproxy/addons/test_dumper.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/addons/test_dumper.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,15 +1,15 @@ import io import shutil -import pytest from unittest import mock -from mitmproxy.test import tflow -from mitmproxy.test import taddons -from mitmproxy.test import tutils +import pytest -from mitmproxy.addons import dumper from mitmproxy import exceptions -from mitmproxy import http +from mitmproxy.addons import dumper +from mitmproxy.net.http import Headers +from mitmproxy.test import taddons +from mitmproxy.test import tflow +from mitmproxy.test import tutils def test_configure(): @@ -83,7 +83,7 @@ flow.client_conn = mock.MagicMock() flow.client_conn.address[0] = "foo" flow.response = tutils.tresp(content=None) - flow.response.is_replay = True + flow.is_replay = "response" flow.response.status_code = 300 d.response(flow) assert sio.getvalue() @@ -104,8 +104,7 @@ ctx.configure(d, flow_detail=4) flow = tflow.tflow() flow.request.content = None - flow.response = http.HTTPResponse.wrap(tutils.tresp()) - flow.response.content = None + flow.response = tutils.tresp(content=None) d.response(flow) assert "content missing" in sio.getvalue() sio.truncate(0) @@ -128,6 +127,35 @@ assert "cut off" in t +def test_echo_trailer(): + sio = io.StringIO() + sio_err = io.StringIO() + d = dumper.Dumper(sio, sio_err) + with taddons.context(d) as ctx: + ctx.configure(d, flow_detail=3) + f = tflow.tflow(client_conn=True, server_conn=True, resp=True) + + f.request.headers["content-type"] = "text/html" + f.request.headers["transfer-encoding"] = "chunked" + f.request.headers["trailer"] = "my-little-request-trailer" + f.request.content = b"some request content\n" * 100 + f.request.trailers = Headers([(b"my-little-request-trailer", b"foobar-request-trailer")]) + + f.response.headers["transfer-encoding"] = "chunked" + f.response.headers["trailer"] = "my-little-response-trailer" + f.response.content = b"some response content\n" * 100 + f.response.trailers = Headers([(b"my-little-response-trailer", b"foobar-response-trailer")]) + + d.echo_flow(f) + t = sio.getvalue() + assert "content-type" in t + assert "cut off" in t + assert "some request content" in t + assert "foobar-request-trailer" in t + assert "some response content" in t + assert "foobar-response-trailer" in t + + def test_echo_request_line(): sio = io.StringIO() sio_err = io.StringIO() @@ -135,13 +163,13 @@ with taddons.context(d) as ctx: ctx.configure(d, flow_detail=3, showhost=True) f = tflow.tflow(client_conn=None, server_conn=True, resp=True) - f.request.is_replay = True + f.is_replay = "request" d._echo_request_line(f) assert "[replay]" in sio.getvalue() sio.truncate(0) f = tflow.tflow(client_conn=None, server_conn=True, resp=True) - f.request.is_replay = False + f.is_replay = None d._echo_request_line(f) assert "[replay]" not in sio.getvalue() sio.truncate(0) @@ -208,3 +236,14 @@ f = tflow.twebsocketflow(client_conn=True, err=True) d.websocket_error(f) assert "Error in WebSocket" in sio_err.getvalue() + + +def test_http2(): + sio = io.StringIO() + sio_err = io.StringIO() + d = dumper.Dumper(sio, sio_err) + with taddons.context(d): + f = tflow.tflow(resp=True) + f.response.http_version = b"HTTP/2.0" + d.response(f) + assert "HTTP/2.0 200 OK" in sio.getvalue() diff -Nru mitmproxy-5.1.1/test/mitmproxy/addons/test_export.py mitmproxy-6.0.2/test/mitmproxy/addons/test_export.py --- mitmproxy-5.1.1/test/mitmproxy/addons/test_export.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/addons/test_export.py 2020-12-15 16:41:27.000000000 +0000 @@ -144,6 +144,7 @@ def test_get_request_present(self, get_request): assert b"header: qvalue" in export.raw(get_request) + assert b"content-length: 0" in export.raw_request(get_request) def test_get_response_present(self, get_response): delattr(get_response, 'request') @@ -163,6 +164,7 @@ class TestRawRequest: def test_get(self, get_request): assert b"header: qvalue" in export.raw_request(get_request) + assert b"content-length: 0" in export.raw_request(get_request) def test_no_request(self, get_response): delattr(get_response, 'request') diff -Nru mitmproxy-5.1.1/test/mitmproxy/addons/test_intercept.py mitmproxy-6.0.2/test/mitmproxy/addons/test_intercept.py --- mitmproxy-5.1.1/test/mitmproxy/addons/test_intercept.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/addons/test_intercept.py 2020-12-15 16:41:27.000000000 +0000 @@ -42,3 +42,32 @@ f = tflow.tflow(resp=True) tctx.cycle(r, f) assert f.intercepted + + +def test_tcp(): + r = intercept.Intercept() + with taddons.context(r) as tctx: + tctx.configure(r, intercept="~tcp") + f = tflow.ttcpflow() + tctx.cycle(r, f) + assert f.intercepted + + tctx.configure(r, intercept_active=False) + f = tflow.ttcpflow() + tctx.cycle(r, f) + assert not f.intercepted + + +def test_already_taken(): + r = intercept.Intercept() + with taddons.context(r) as tctx: + tctx.configure(r, intercept="~q") + + f = tflow.tflow() + tctx.invoke(r, "request", f) + assert f.intercepted + + f = tflow.tflow() + f.reply.take() + tctx.invoke(r, "request", f) + assert not f.intercepted diff -Nru mitmproxy-5.1.1/test/mitmproxy/addons/test_maplocal.py mitmproxy-6.0.2/test/mitmproxy/addons/test_maplocal.py --- mitmproxy-5.1.1/test/mitmproxy/addons/test_maplocal.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/addons/test_maplocal.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,186 @@ +import sys +from pathlib import Path + +import pytest + +from mitmproxy.addons.maplocal import MapLocal, MapLocalSpec, file_candidates +from mitmproxy.utils.spec import parse_spec +from mitmproxy.test import taddons +from mitmproxy.test import tflow + + +@pytest.mark.parametrize( + "url,spec,expected_candidates", + [ + # trailing slashes + ("https://example.com/foo", ":example.com/foo:/tmp", ["/tmp/index.html"]), + ("https://example.com/foo/", ":example.com/foo:/tmp", ["/tmp/index.html"]), + ("https://example.com/foo", ":example.com/foo:/tmp/", ["/tmp/index.html"]), + ] + [ + # simple prefixes + ("http://example.com/foo/bar.jpg", ":example.com/foo:/tmp", ["/tmp/bar.jpg", "/tmp/bar.jpg/index.html"]), + ("https://example.com/foo/bar.jpg", ":example.com/foo:/tmp", ["/tmp/bar.jpg", "/tmp/bar.jpg/index.html"]), + ("https://example.com/foo/bar.jpg?query", ":example.com/foo:/tmp", ["/tmp/bar.jpg", "/tmp/bar.jpg/index.html"]), + ("https://example.com/foo/bar/baz.jpg", ":example.com/foo:/tmp", + ["/tmp/bar/baz.jpg", "/tmp/bar/baz.jpg/index.html"]), + ("https://example.com/foo/bar.jpg", ":/foo/bar.jpg:/tmp", ["/tmp/index.html"]), + ] + [ + # URL decode and special characters + ("http://example.com/foo%20bar.jpg", ":example.com:/tmp", [ + "/tmp/foo bar.jpg", + "/tmp/foo bar.jpg/index.html", + "/tmp/foo_bar.jpg", + "/tmp/foo_bar.jpg/index.html" + ]), + ("http://example.com/fóobår.jpg", ":example.com:/tmp", [ + "/tmp/fóobår.jpg", + "/tmp/fóobår.jpg/index.html", + "/tmp/f_ob_r.jpg", + "/tmp/f_ob_r.jpg/index.html" + ]), + ] + [ + # index.html + ("https://example.com/foo", ":example.com/foo:/tmp", ["/tmp/index.html"]), + ("https://example.com/foo/", ":example.com/foo:/tmp", ["/tmp/index.html"]), + ("https://example.com/foo/bar", ":example.com/foo:/tmp", ["/tmp/bar", "/tmp/bar/index.html"]), + ("https://example.com/foo/bar/", ":example.com/foo:/tmp", ["/tmp/bar", "/tmp/bar/index.html"]), + ] + [ + # regex + ( + "https://example/view.php?f=foo.jpg", + ":example/view.php\\?f=(.+):/tmp", + ["/tmp/foo.jpg", "/tmp/foo.jpg/index.html"] + ), ( + "https://example/results?id=1&foo=2", + ":example/(results\\?id=.+):/tmp", + [ + "/tmp/results?id=1&foo=2", + "/tmp/results?id=1&foo=2/index.html", + "/tmp/results_id=1_foo=2", + "/tmp/results_id=1_foo=2/index.html" + ] + ), + ] + [ + # test directory traversal detection + ("https://example.com/../../../../../../etc/passwd", ":example.com:/tmp", []), + # this is slightly hacky, but werkzeug's behavior differs per system. + ("https://example.com/C:\\foo.txt", ":example.com:/tmp", [] if sys.platform == "win32" else [ + "/tmp/C:\\foo.txt", + "/tmp/C:\\foo.txt/index.html", + "/tmp/C__foo.txt", + "/tmp/C__foo.txt/index.html" + ]), + ("https://example.com//etc/passwd", ":example.com:/tmp", ["/tmp/etc/passwd", "/tmp/etc/passwd/index.html"]), + ] +) +def test_file_candidates(url, spec, expected_candidates): + # we circumvent the path existence checks here to simplify testing + filt, subj, repl = parse_spec(spec) + spec = MapLocalSpec(filt, subj, Path(repl)) + + candidates = file_candidates(url, spec) + assert [x.as_posix() for x in candidates] == expected_candidates + + +class TestMapLocal: + + def test_configure(self, tmpdir): + ml = MapLocal() + with taddons.context(ml) as tctx: + tctx.configure(ml, map_local=["/foo/bar/" + str(tmpdir)]) + with pytest.raises(Exception, match="Invalid regular expression"): + tctx.configure(ml, map_local=["/foo/+/" + str(tmpdir)]) + with pytest.raises(Exception, match="Invalid file path"): + tctx.configure(ml, map_local=["/foo/.+/three"]) + + def test_simple(self, tmpdir): + ml = MapLocal() + + with taddons.context(ml) as tctx: + tmpfile = tmpdir.join("foo.jpg") + tmpfile.write("foo") + tctx.configure( + ml, + map_local=[ + "|//example.org/images|" + str(tmpdir) + ] + ) + f = tflow.tflow() + f.request.url = b"https://example.org/images/foo.jpg" + ml.request(f) + assert f.response.content == b"foo" + + tmpfile = tmpdir.join("images", "bar.jpg") + tmpfile.write("bar", ensure=True) + tctx.configure( + ml, + map_local=[ + "|//example.org|" + str(tmpdir) + ] + ) + f = tflow.tflow() + f.request.url = b"https://example.org/images/bar.jpg" + ml.request(f) + assert f.response.content == b"bar" + + tmpfile = tmpdir.join("foofoobar.jpg") + tmpfile.write("foofoobar", ensure=True) + tctx.configure( + ml, + map_local=[ + "|example.org/foo/foo/bar.jpg|" + str(tmpfile) + ] + ) + f = tflow.tflow() + f.request.url = b"https://example.org/foo/foo/bar.jpg" + ml.request(f) + assert f.response.content == b"foofoobar" + + @pytest.mark.asyncio + async def test_nonexistent_files(self, tmpdir, monkeypatch): + ml = MapLocal() + + with taddons.context(ml) as tctx: + tctx.configure( + ml, + map_local=[ + "|example.org/css|" + str(tmpdir) + ] + ) + f = tflow.tflow() + f.request.url = b"https://example.org/css/nonexistent" + ml.request(f) + assert f.response.status_code == 404 + assert await tctx.master.await_log("None of the local file candidates exist") + + tmpfile = tmpdir.join("foo.jpg") + tmpfile.write("foo") + tctx.configure( + ml, + map_local=[ + "|//example.org/images|" + str(tmpfile) + ] + ) + tmpfile.remove() + monkeypatch.setattr(Path, "is_file", lambda x: True) + f = tflow.tflow() + f.request.url = b"https://example.org/images/foo.jpg" + ml.request(f) + assert await tctx.master.await_log("could not read file") + + def test_has_reply(self, tmpdir): + ml = MapLocal() + with taddons.context(ml) as tctx: + tmpfile = tmpdir.join("foo.jpg") + tmpfile.write("foo") + tctx.configure( + ml, + map_local=[ + "|//example.org/images|" + str(tmpfile) + ] + ) + f = tflow.tflow() + f.request.url = b"https://example.org/images/foo.jpg" + f.kill() + ml.request(f) + assert not f.response diff -Nru mitmproxy-5.1.1/test/mitmproxy/addons/test_mapremote.py mitmproxy-6.0.2/test/mitmproxy/addons/test_mapremote.py --- mitmproxy-5.1.1/test/mitmproxy/addons/test_mapremote.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/addons/test_mapremote.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,39 @@ +import pytest + +from mitmproxy.addons import mapremote +from mitmproxy.test import taddons +from mitmproxy.test import tflow + + +class TestMapRemote: + + def test_configure(self): + mr = mapremote.MapRemote() + with taddons.context(mr) as tctx: + tctx.configure(mr, map_remote=["one/two/three"]) + with pytest.raises(Exception, match="Invalid regular expression"): + tctx.configure(mr, map_remote=["/foo/+/three"]) + + def test_simple(self): + mr = mapremote.MapRemote() + with taddons.context(mr) as tctx: + tctx.configure( + mr, + map_remote=[ + ":example.org/images/:mitmproxy.org/img/", + ] + ) + f = tflow.tflow() + f.request.url = b"https://example.org/images/test.jpg" + mr.request(f) + assert f.request.url == "https://mitmproxy.org/img/test.jpg" + + def test_has_reply(self): + mr = mapremote.MapRemote() + with taddons.context(mr) as tctx: + tctx.configure(mr, map_remote=[":example.org:mitmproxy.org"]) + f = tflow.tflow() + f.request.url = b"https://example.org/images/test.jpg" + f.kill() + mr.request(f) + assert f.request.url == "https://example.org/images/test.jpg" diff -Nru mitmproxy-5.1.1/test/mitmproxy/addons/test_modifybody.py mitmproxy-6.0.2/test/mitmproxy/addons/test_modifybody.py --- mitmproxy-5.1.1/test/mitmproxy/addons/test_modifybody.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/addons/test_modifybody.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,89 @@ +import pytest + +from mitmproxy.addons import modifybody +from mitmproxy.test import taddons +from mitmproxy.test import tflow + + +class TestModifyBody: + def test_configure(self): + mb = modifybody.ModifyBody() + with taddons.context(mb) as tctx: + tctx.configure(mb, modify_body=["one/two/three"]) + with pytest.raises(Exception, match="Cannot parse modify_body"): + tctx.configure(mb, modify_body=["/"]) + + def test_simple(self): + mb = modifybody.ModifyBody() + with taddons.context(mb) as tctx: + tctx.configure( + mb, + modify_body=[ + "/~q/foo/bar", + "/~s/foo/bar", + ] + ) + f = tflow.tflow() + f.request.content = b"foo" + mb.request(f) + assert f.request.content == b"bar" + + f = tflow.tflow(resp=True) + f.response.content = b"foo" + mb.response(f) + assert f.response.content == b"bar" + + def test_order(self): + mb = modifybody.ModifyBody() + with taddons.context(mb) as tctx: + tctx.configure( + mb, + modify_body=[ + "/foo/bar", + "/bar/baz", + "/foo/oh noes!", + "/bar/oh noes!", + ] + ) + f = tflow.tflow() + f.request.content = b"foo" + mb.request(f) + assert f.request.content == b"baz" + + +class TestModifyBodyFile: + def test_simple(self, tmpdir): + mb = modifybody.ModifyBody() + with taddons.context(mb) as tctx: + tmpfile = tmpdir.join("replacement") + tmpfile.write("bar") + tctx.configure( + mb, + modify_body=["/~q/foo/@" + str(tmpfile)] + ) + f = tflow.tflow() + f.request.content = b"foo" + mb.request(f) + assert f.request.content == b"bar" + + @pytest.mark.asyncio + async def test_nonexistent(self, tmpdir): + mb = modifybody.ModifyBody() + with taddons.context(mb) as tctx: + with pytest.raises(Exception, match="Invalid file path"): + tctx.configure( + mb, + modify_body=["/~q/foo/@nonexistent"] + ) + + tmpfile = tmpdir.join("replacement") + tmpfile.write("bar") + tctx.configure( + mb, + modify_body=["/~q/foo/@" + str(tmpfile)] + ) + tmpfile.remove() + f = tflow.tflow() + f.request.content = b"foo" + mb.request(f) + assert await tctx.master.await_log("could not read") diff -Nru mitmproxy-5.1.1/test/mitmproxy/addons/test_modifyheaders.py mitmproxy-6.0.2/test/mitmproxy/addons/test_modifyheaders.py --- mitmproxy-5.1.1/test/mitmproxy/addons/test_modifyheaders.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/addons/test_modifyheaders.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,153 @@ +import pytest + +from mitmproxy.test import tflow +from mitmproxy.test import taddons + +from mitmproxy.addons.modifyheaders import parse_modify_spec, ModifyHeaders + + +def test_parse_modify_spec(): + spec = parse_modify_spec("/foo/bar/voing", True) + assert spec.matches.pattern == "foo" + assert spec.subject == b"bar" + assert spec.read_replacement() == b"voing" + + spec = parse_modify_spec("/foo/bar/vo/ing/", False) + assert spec.matches.pattern == "foo" + assert spec.subject == b"bar" + assert spec.read_replacement() == b"vo/ing/" + + spec = parse_modify_spec("/bar/voing", False) + assert spec.matches(tflow.tflow()) + assert spec.subject == b"bar" + assert spec.read_replacement() == b"voing" + + with pytest.raises(ValueError, match="Invalid regular expression"): + parse_modify_spec("/[/two", True) + + +class TestModifyHeaders: + + def test_configure(self): + mh = ModifyHeaders() + with taddons.context(mh) as tctx: + with pytest.raises(Exception, match="Cannot parse modify_headers"): + tctx.configure(mh, modify_headers=["/"]) + tctx.configure(mh, modify_headers=["/foo/bar/voing"]) + + def test_modify_headers(self): + mh = ModifyHeaders() + with taddons.context(mh) as tctx: + tctx.configure( + mh, + modify_headers=[ + "/~q/one/two", + "/~s/one/three" + ] + ) + f = tflow.tflow() + f.request.headers["one"] = "xxx" + mh.request(f) + assert f.request.headers["one"] == "two" + + f = tflow.tflow(resp=True) + f.response.headers["one"] = "xxx" + mh.response(f) + assert f.response.headers["one"] == "three" + + tctx.configure( + mh, + modify_headers=[ + "/~s/one/two", + "/~s/one/three" + ] + ) + f = tflow.tflow(resp=True) + f.request.headers["one"] = "xxx" + f.response.headers["one"] = "xxx" + mh.response(f) + assert f.response.headers.get_all("one") == ["two", "three"] + + tctx.configure( + mh, + modify_headers=[ + "/~q/one/two", + "/~q/one/three" + ] + ) + f = tflow.tflow() + f.request.headers["one"] = "xxx" + mh.request(f) + assert f.request.headers.get_all("one") == ["two", "three"] + + # test removal of existing headers + tctx.configure( + mh, + modify_headers=[ + "/~q/one/", + "/~s/one/" + ] + ) + f = tflow.tflow() + f.request.headers["one"] = "xxx" + mh.request(f) + assert "one" not in f.request.headers + + f = tflow.tflow(resp=True) + f.response.headers["one"] = "xxx" + mh.response(f) + assert "one" not in f.response.headers + + tctx.configure( + mh, + modify_headers=[ + "/one/" + ] + ) + f = tflow.tflow() + f.request.headers["one"] = "xxx" + mh.request(f) + assert "one" not in f.request.headers + + f = tflow.tflow(resp=True) + f.response.headers["one"] = "xxx" + mh.response(f) + assert "one" not in f.response.headers + + +class TestModifyHeadersFile: + def test_simple(self, tmpdir): + mh = ModifyHeaders() + with taddons.context(mh) as tctx: + tmpfile = tmpdir.join("replacement") + tmpfile.write("two") + tctx.configure( + mh, + modify_headers=["/~q/one/@" + str(tmpfile)] + ) + f = tflow.tflow() + f.request.headers["one"] = "xxx" + mh.request(f) + assert f.request.headers["one"] == "two" + + @pytest.mark.asyncio + async def test_nonexistent(self, tmpdir): + mh = ModifyHeaders() + with taddons.context(mh) as tctx: + with pytest.raises(Exception, match="Cannot parse modify_headers .* Invalid file path"): + tctx.configure( + mh, + modify_headers=["/~q/foo/@nonexistent"] + ) + + tmpfile = tmpdir.join("replacement") + tmpfile.write("bar") + tctx.configure( + mh, + modify_headers=["/~q/foo/@" + str(tmpfile)] + ) + tmpfile.remove() + f = tflow.tflow() + f.request.content = b"foo" + mh.request(f) + assert await tctx.master.await_log("could not read") diff -Nru mitmproxy-5.1.1/test/mitmproxy/addons/test_onboarding.py mitmproxy-6.0.2/test/mitmproxy/addons/test_onboarding.py --- mitmproxy-5.1.1/test/mitmproxy/addons/test_onboarding.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/addons/test_onboarding.py 2020-12-15 16:41:27.000000000 +0000 @@ -16,7 +16,7 @@ tctx.configure(ob) assert self.app("/").status_code == 200 - @pytest.mark.parametrize("ext", ["pem", "p12"]) + @pytest.mark.parametrize("ext", ["pem", "p12", "cer"]) @pytest.mark.asyncio async def test_cert(self, ext): ob = onboarding.Onboarding() @@ -26,7 +26,7 @@ assert resp.status_code == 200 assert resp.content - @pytest.mark.parametrize("ext", ["pem", "p12"]) + @pytest.mark.parametrize("ext", ["pem", "p12", "cer"]) @pytest.mark.asyncio async def test_head(self, ext): ob = onboarding.Onboarding() @@ -34,7 +34,7 @@ tctx.configure(ob) p = self.pathoc() with p.connect(): - resp = p.request("head:'http://%s/cert/%s'" % (tctx.options.onboarding_host, ext)) + resp = p.request(f"head:'http://{tctx.options.onboarding_host}/cert/{ext}'") assert resp.status_code == 200 assert "Content-Length" in resp.headers assert not resp.content diff -Nru mitmproxy-5.1.1/test/mitmproxy/addons/test_readfile.py mitmproxy-6.0.2/test/mitmproxy/addons/test_readfile.py --- mitmproxy-5.1.1/test/mitmproxy/addons/test_readfile.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/addons/test_readfile.py 2020-12-15 16:41:27.000000000 +0000 @@ -2,7 +2,7 @@ import io import pytest -import asynctest +from unittest import mock import mitmproxy.io from mitmproxy import exceptions @@ -54,17 +54,17 @@ tf = tmpdir.join("tfile") - with asynctest.patch('mitmproxy.master.Master.load_flow') as mck: + with mock.patch('mitmproxy.master.Master.load_flow') as mck: tf.write(data.getvalue()) tctx.configure( rf, rfile = str(tf), readfile_filter = ".*" ) - assert not mck.awaited + mck.assert_not_awaited() rf.running() await asyncio.sleep(0) - assert mck.awaited + mck.assert_awaited() tf.write(corrupt_data.getvalue()) tctx.configure(rf, rfile=str(tf)) @@ -93,16 +93,16 @@ class TestReadFileStdin: - @asynctest.patch('sys.stdin') + @mock.patch('sys.stdin') @pytest.mark.asyncio async def test_stdin(self, stdin, data, corrupt_data): rf = readfile.ReadFileStdin() with taddons.context(rf): - with asynctest.patch('mitmproxy.master.Master.load_flow') as mck: + with mock.patch('mitmproxy.master.Master.load_flow') as mck: stdin.buffer = data - assert not mck.awaited + mck.assert_not_awaited() await rf.load_flows(stdin.buffer) - assert mck.awaited + mck.assert_awaited() stdin.buffer = corrupt_data with pytest.raises(exceptions.FlowReadException): @@ -113,10 +113,10 @@ rf = readfile.ReadFileStdin() with taddons.context(rf) as tctx: tf = tmpdir.join("tfile") - with asynctest.patch('mitmproxy.master.Master.load_flow') as mck: + with mock.patch('mitmproxy.master.Master.load_flow') as mck: tf.write(data.getvalue()) tctx.configure(rf, rfile=str(tf)) - assert not mck.awaited + mck.assert_not_awaited() rf.running() await asyncio.sleep(0) - assert mck.awaited + mck.assert_awaited() diff -Nru mitmproxy-5.1.1/test/mitmproxy/addons/test_replace.py mitmproxy-6.0.2/test/mitmproxy/addons/test_replace.py --- mitmproxy-5.1.1/test/mitmproxy/addons/test_replace.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/addons/test_replace.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,102 +0,0 @@ -import pytest - -from mitmproxy.addons import replace -from mitmproxy.test import taddons -from mitmproxy.test import tflow - - -class TestReplace: - def test_parse_hook(self): - x = replace.parse_hook("/foo/bar/voing") - assert x == ("foo", "bar", "voing") - x = replace.parse_hook("/foo/bar/vo/ing/") - assert x == ("foo", "bar", "vo/ing/") - x = replace.parse_hook("/bar/voing") - assert x == (".*", "bar", "voing") - with pytest.raises(Exception, match="Invalid replacement"): - replace.parse_hook("/") - - def test_configure(self): - r = replace.Replace() - with taddons.context(r) as tctx: - tctx.configure(r, replacements=["one/two/three"]) - with pytest.raises(Exception, match="Invalid filter pattern"): - tctx.configure(r, replacements=["/~b/two/three"]) - with pytest.raises(Exception, match="Invalid regular expression"): - tctx.configure(r, replacements=["/foo/+/three"]) - tctx.configure(r, replacements=["/a/b/c/"]) - - def test_simple(self): - r = replace.Replace() - with taddons.context(r) as tctx: - tctx.configure( - r, - replacements=[ - "/~q/foo/bar", - "/~s/foo/bar", - ] - ) - f = tflow.tflow() - f.request.content = b"foo" - r.request(f) - assert f.request.content == b"bar" - - f = tflow.tflow(resp=True) - f.response.content = b"foo" - r.response(f) - assert f.response.content == b"bar" - - def test_order(self): - r = replace.Replace() - with taddons.context(r) as tctx: - tctx.configure( - r, - replacements=[ - "/foo/bar", - "/bar/baz", - "/foo/oh noes!", - "/bar/oh noes!", - ] - ) - f = tflow.tflow() - f.request.content = b"foo" - r.request(f) - assert f.request.content == b"baz" - - -class TestReplaceFile: - def test_simple(self, tmpdir): - r = replace.Replace() - with taddons.context(r) as tctx: - tmpfile = tmpdir.join("replacement") - tmpfile.write("bar") - tctx.configure( - r, - replacements=["/~q/foo/@" + str(tmpfile)] - ) - f = tflow.tflow() - f.request.content = b"foo" - r.request(f) - assert f.request.content == b"bar" - - @pytest.mark.asyncio - async def test_nonexistent(self, tmpdir): - r = replace.Replace() - with taddons.context(r) as tctx: - with pytest.raises(Exception, match="Invalid file path"): - tctx.configure( - r, - replacements=["/~q/foo/@nonexistent"] - ) - - tmpfile = tmpdir.join("replacement") - tmpfile.write("bar") - tctx.configure( - r, - replacements=["/~q/foo/@" + str(tmpfile)] - ) - tmpfile.remove() - f = tflow.tflow() - f.request.content = b"foo" - r.request(f) - assert await tctx.master.await_log("could not read") diff -Nru mitmproxy-5.1.1/test/mitmproxy/addons/test_script.py mitmproxy-6.0.2/test/mitmproxy/addons/test_script.py --- mitmproxy-5.1.1/test/mitmproxy/addons/test_script.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/addons/test_script.py 2020-12-15 16:41:27.000000000 +0000 @@ -74,7 +74,7 @@ path = tdata.path("mitmproxy/data/addonscripts/recorder/recorder.py") s = script.Script( - '"{}"'.format(path), + f'"{path}"', False ) assert '"' not in s.fullpath diff -Nru mitmproxy-5.1.1/test/mitmproxy/addons/test_serverplayback.py mitmproxy-6.0.2/test/mitmproxy/addons/test_serverplayback.py --- mitmproxy-5.1.1/test/mitmproxy/addons/test_serverplayback.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/addons/test_serverplayback.py 2020-12-15 16:41:27.000000000 +0000 @@ -299,7 +299,7 @@ boundary = 'somefancyboundary' def multipart_setter(r, **kwargs): - b = "--{0}\n".format(boundary) + b = f"--{boundary}\n" parts = [] for k, v in kwargs.items(): parts.append( @@ -331,7 +331,7 @@ tf = tflow.tflow() assert not tf.response s.request(tf) - assert tf.response == f.response + assert tf.response.data == f.response.data tf = tflow.tflow() tf.request.content = b"gibble" diff -Nru mitmproxy-5.1.1/test/mitmproxy/addons/test_session.py mitmproxy-6.0.2/test/mitmproxy/addons/test_session.py --- mitmproxy-5.1.1/test/mitmproxy/addons/test_session.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/addons/test_session.py 2020-12-15 16:41:27.000000000 +0000 @@ -160,6 +160,7 @@ assert len(s._view) == 4 @pytest.mark.asyncio + @pytest.mark.skip async def test_storage_flush_with_specials(self): s = self.start_session(fp=0.5) f = self.tft() @@ -187,6 +188,7 @@ assert s._flush_period == s._FP_DEFAULT @pytest.mark.asyncio + @pytest.mark.skip async def test_storage_bodies(self): # Need to test for configure # Need to test for set_order @@ -202,8 +204,8 @@ ).fetchall()[0] assert content == (1, b"A" * 1001) assert s.db_store.body_ledger == {f.id} - f.response = http.HTTPResponse.wrap(tutils.tresp(content=b"A" * 1001)) - f2.response = http.HTTPResponse.wrap(tutils.tresp(content=b"A" * 1001)) + f.response = tutils.tresp(content=b"A" * 1001) + f2.response = tutils.tresp(content=b"A" * 1001) # Content length is wrong for some reason -- quick fix f.response.headers['content-length'] = b"1001" f2.response.headers['content-length'] = b"1001" @@ -222,6 +224,7 @@ assert all([lf.__dict__ == rf.__dict__ for lf, rf in list(zip(s.load_view(), [f, f2]))]) @pytest.mark.asyncio + @pytest.mark.skip async def test_storage_order(self): s = self.start_session(fp=0.5) s.request(self.tft(method="GET", start=4)) diff -Nru mitmproxy-5.1.1/test/mitmproxy/addons/test_setheaders.py mitmproxy-6.0.2/test/mitmproxy/addons/test_setheaders.py --- mitmproxy-5.1.1/test/mitmproxy/addons/test_setheaders.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/addons/test_setheaders.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,70 +0,0 @@ -import pytest - -from mitmproxy.test import tflow -from mitmproxy.test import taddons - -from mitmproxy.addons import setheaders - - -class TestSetHeaders: - def test_parse_setheaders(self): - x = setheaders.parse_setheader("/foo/bar/voing") - assert x == ("foo", "bar", "voing") - x = setheaders.parse_setheader("/foo/bar/vo/ing/") - assert x == ("foo", "bar", "vo/ing/") - x = setheaders.parse_setheader("/bar/voing") - assert x == (".*", "bar", "voing") - with pytest.raises(Exception, match="Invalid replacement"): - setheaders.parse_setheader("/") - - def test_configure(self): - sh = setheaders.SetHeaders() - with taddons.context(sh) as tctx: - with pytest.raises(Exception, match="Invalid setheader filter pattern"): - tctx.configure(sh, setheaders = ["/~b/one/two"]) - tctx.configure(sh, setheaders = ["/foo/bar/voing"]) - - def test_setheaders(self): - sh = setheaders.SetHeaders() - with taddons.context(sh) as tctx: - tctx.configure( - sh, - setheaders = [ - "/~q/one/two", - "/~s/one/three" - ] - ) - f = tflow.tflow() - f.request.headers["one"] = "xxx" - sh.request(f) - assert f.request.headers["one"] == "two" - - f = tflow.tflow(resp=True) - f.response.headers["one"] = "xxx" - sh.response(f) - assert f.response.headers["one"] == "three" - - tctx.configure( - sh, - setheaders = [ - "/~s/one/two", - "/~s/one/three" - ] - ) - f = tflow.tflow(resp=True) - f.request.headers["one"] = "xxx" - f.response.headers["one"] = "xxx" - sh.response(f) - assert f.response.headers.get_all("one") == ["two", "three"] - - tctx.configure( - sh, - setheaders = [ - "/~q/one/two", - "/~q/one/three" - ] - ) - f = tflow.tflow() - f.request.headers["one"] = "xxx" - sh.request(f) - assert f.request.headers.get_all("one") == ["two", "three"] diff -Nru mitmproxy-5.1.1/test/mitmproxy/addons/test_termstatus.py mitmproxy-6.0.2/test/mitmproxy/addons/test_termstatus.py --- mitmproxy-5.1.1/test/mitmproxy/addons/test_termstatus.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/addons/test_termstatus.py 2020-12-15 16:41:27.000000000 +0000 @@ -10,6 +10,7 @@ ts = termstatus.TermStatus() with taddons.context() as ctx: ctx.master.server = proxy.DummyServer() + ctx.master.server.bound = True ctx.configure(ts, server=False) ts.running() ctx.configure(ts, server=True) diff -Nru mitmproxy-5.1.1/test/mitmproxy/addons/test_view.py mitmproxy-6.0.2/test/mitmproxy/addons/test_view.py --- mitmproxy-5.1.1/test/mitmproxy/addons/test_view.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/addons/test_view.py 2020-12-15 16:41:27.000000000 +0000 @@ -36,7 +36,7 @@ assert sargs -def test_order_generators(): +def test_order_generators_http(): v = view.View() tf = tflow.tflow(resp=True) @@ -53,6 +53,23 @@ assert sz.generate(tf) == len(tf.request.raw_content) + len(tf.response.raw_content) +def test_order_generators_tcp(): + v = view.View() + tf = tflow.ttcpflow() + + rs = view.OrderRequestStart(v) + assert rs.generate(tf) == 946681200 + + rm = view.OrderRequestMethod(v) + assert rm.generate(tf) == "TCP" + + ru = view.OrderRequestURL(v) + assert ru.generate(tf) == "address:22" + + sz = view.OrderKeySize(v) + assert sz.generate(tf) == sum(len(m.content) for m in tf.messages) + + def test_simple(): v = view.View() f = tft(start=1) @@ -105,6 +122,21 @@ assert len(v._store) == 0 +def test_simple_tcp(): + v = view.View() + f = tflow.ttcpflow() + assert v.store_count() == 0 + v.tcp_start(f) + assert list(v) == [f] + + # These all just call update + v.tcp_start(f) + v.tcp_message(f) + v.tcp_error(f) + v.tcp_end(f) + assert list(v) == [f] + + def test_filter(): v = view.View() v.request(tft(method="get")) @@ -180,7 +212,7 @@ assert len(v) == 4 try: v.load_file("nonexistent_file_path") - except IOError: + except OSError: assert False with open(path, "wb") as f: f.write(b"invalidflows") diff -Nru mitmproxy-5.1.1/test/mitmproxy/addons/test_wsgiapp.py mitmproxy-6.0.2/test/mitmproxy/addons/test_wsgiapp.py --- mitmproxy-5.1.1/test/mitmproxy/addons/test_wsgiapp.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/addons/test_wsgiapp.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,41 +0,0 @@ -import flask - -from .. import tservers -from mitmproxy.addons import wsgiapp - -tapp = flask.Flask(__name__) - - -@tapp.route("/") -def hello(): - return "testapp" - - -@tapp.route("/error") -def error(): - raise ValueError("An exception...") - - -def errapp(environ, start_response): - raise ValueError("errapp") - - -class TestApp(tservers.HTTPProxyTest): - def addons(self): - return [ - wsgiapp.WSGIApp(tapp, "testapp", 80), - wsgiapp.WSGIApp(errapp, "errapp", 80) - ] - - def test_simple(self): - p = self.pathoc() - with p.connect(): - ret = p.request("get:'http://testapp/'") - assert ret.status_code == 200 - - def test_app_err(self): - p = self.pathoc() - with p.connect(): - ret = p.request("get:'http://errapp/'") - assert ret.status_code == 500 - assert b"ValueError" in ret.content diff -Nru mitmproxy-5.1.1/test/mitmproxy/contentviews/test_json.py mitmproxy-6.0.2/test/mitmproxy/contentviews/test_json.py --- mitmproxy-5.1.1/test/mitmproxy/contentviews/test_json.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/contentviews/test_json.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,16 +1,43 @@ +from hypothesis import given +from hypothesis.strategies import binary + from mitmproxy.contentviews import json from . import full_eval -def test_pretty_json(): - assert json.pretty_json(b'{"foo": 1}') - assert not json.pretty_json(b"moo") - assert json.pretty_json(b'{"foo" : "\xe4\xb8\x96\xe7\x95\x8c"}') # utf8 with chinese characters - assert not json.pretty_json(b'{"foo" : "\xFF"}') +def test_parse_json(): + assert json.parse_json(b'{"foo": 1}') + assert json.parse_json(b'null') is None + assert json.parse_json(b"moo") is json.PARSE_ERROR + assert json.parse_json(b'{"foo" : "\xe4\xb8\x96\xe7\x95\x8c"}') # utf8 with chinese characters + assert json.parse_json(b'{"foo" : "\xFF"}') is json.PARSE_ERROR + + +def test_format_json(): + assert list(json.format_json({ + "data": [ + "str", + 42, + True, + False, + None, + {}, + [] + ] + })) def test_view_json(): v = full_eval(json.ViewJSON()) + assert v(b"null") assert v(b"{}") assert not v(b"{") assert v(b"[1, 2, 3, 4, 5]") + assert v(b'{"foo" : 3}') + assert v(b'{"foo": true, "nullvalue": null}') + + +@given(binary()) +def test_view_json_doesnt_crash(data): + v = full_eval(json.ViewJSON()) + v(data) diff -Nru mitmproxy-5.1.1/test/mitmproxy/contentviews/test_msgpack.py mitmproxy-6.0.2/test/mitmproxy/contentviews/test_msgpack.py --- mitmproxy-5.1.1/test/mitmproxy/contentviews/test_msgpack.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/contentviews/test_msgpack.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,46 @@ +from hypothesis import given +from hypothesis.strategies import binary + +from msgpack import packb + +from mitmproxy.contentviews import msgpack +from . import full_eval + + +def msgpack_encode(content): + return packb(content, use_bin_type=True) + + +def test_parse_msgpack(): + assert msgpack.parse_msgpack(msgpack_encode({"foo": 1})) + assert msgpack.parse_msgpack(b"aoesuteoahu") is msgpack.PARSE_ERROR + assert msgpack.parse_msgpack(msgpack_encode({"foo": "\xe4\xb8\x96\xe7\x95\x8c"})) + + +def test_format_msgpack(): + assert list(msgpack.format_msgpack({ + "data": [ + "str", + 42, + True, + False, + None, + {}, + [] + ] + })) + + +def test_view_msgpack(): + v = full_eval(msgpack.ViewMsgPack()) + assert v(msgpack_encode({})) + assert not v(b"aoesuteoahu") + assert v(msgpack_encode([1, 2, 3, 4, 5])) + assert v(msgpack_encode({"foo": 3})) + assert v(msgpack_encode({"foo": True, "nullvalue": None})) + + +@given(binary()) +def test_view_msgpack_doesnt_crash(data): + v = full_eval(msgpack.ViewMsgPack()) + v(data) diff -Nru mitmproxy-5.1.1/test/mitmproxy/data/addonscripts/recorder/recorder.py mitmproxy-6.0.2/test/mitmproxy/data/addonscripts/recorder/recorder.py --- mitmproxy-5.1.1/test/mitmproxy/data/addonscripts/recorder/recorder.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/data/addonscripts/recorder/recorder.py 2020-12-15 16:41:27.000000000 +0000 @@ -16,7 +16,7 @@ if attr != "log": ctx.log.info(str(lg)) self.call_log.append(lg) - ctx.log.debug("%s %s" % (self.name, attr)) + ctx.log.debug(f"{self.name} {attr}") return prox raise AttributeError Binary files /tmp/tmp0HPLb0/g07x9iCg1Z/mitmproxy-5.1.1/test/mitmproxy/data/dumpfile-019.bin and /tmp/tmp0HPLb0/lh4usDLsJr/mitmproxy-6.0.2/test/mitmproxy/data/dumpfile-019.bin differ diff -Nru mitmproxy-5.1.1/test/mitmproxy/data/mitmproxy.pem mitmproxy-6.0.2/test/mitmproxy/data/mitmproxy.pem --- mitmproxy-5.1.1/test/mitmproxy/data/mitmproxy.pem 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/data/mitmproxy.pem 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,51 @@ +-----BEGIN ENCRYPTED PRIVATE KEY----- +MIIFHDBOBgkqhkiG9w0BBQ0wQTApBgkqhkiG9w0BBQwwHAQI9fSurwMcOA4CAggA +MAwGCCqGSIb3DQIJBQAwFAYIKoZIhvcNAwcECKALBF48zvQnBIIEyKDIy72IMJoZ +4q7LsG0dCSa8oGI/CtAnC9YqRlDj+paWoGKDUkzxnMloUbJkpQlTEYRHXp0xKtdP +IcCjWFqWeQjsaJUlwILNLiliVpbyW/0PLmNQmRSfvLhlZ77rRk08DyLU0mcW2zRX +DuKHuxGhdlmte7EKsNf8czch9hDXqrCLqxlzr86K0pwT40W1r32TgQdX68edluoj +acggWuEzeTTKy1BKkVtlCq63dflgRfSo0as+dYX38wxzC6O6hxKpax277ijoJZHc +QsXxi/zREa+gtVOq9D6Vz5E+MmmIIAzVrXsFe1Uj4wYb3XUSO6WnJ9GlrixqWeu3 +9lkOZOEKyyDgIY+twn06kyZBspKnXvQMMPjeiSSeaqI9LA0qpvRsxuWCxyTJ2YZI +s+xab8j5g5RKOmrt1bGtLl66tcrGNP9jYC5pjMNl6fz3c8+oxC0Bun4q+yOA9QzG +4GaiA834x+9wtsEBSjlMB5AMwYH+1ODo6Q+VUAWH1qBvCm/gQT2mvSgcrz6bcGJI +gimfzl/IbqVuVkWl7yFqNN/renE47pvy34Dbymb0FBK/5Gb1FImno3CcAkCuaEJ1 +sWdx2Ej0Ezit8v1iJN2q29xlD7MrxB0uPvklUPRlD9RVcDJ15GwBPA8ugN/Fjj50 +2BiMJ2/uqBoEnAjMyStINArS5PWL6gthIXenVJ4w0wegBciCsGo4G7UFQ0z/w2Je +7NJ8TjwKdTYJdAfgO5Rr8u6j0ybn72T/+QJfjugNLufRx4sakvPZR90/AFb2YX+L +kgCVS3ySOfom9p5JcxdnI8omelBIi1Qa9xwPKMPaV6oYkqBVjmcDDZocC6qN15PD +jCrgGryV3Fsn5OLYTB+EQDLNqmo+qd1O0pNY2THwD/DGGlx6VhmeQnWdt534g5lo +clQOmLXEeUWIb2u5PanakqNpY5mBQcOJ88/RS+oGAjTGU0e3I1zLb6EN/Ftndjv1 +sfEh+HMwHxIWxdnJb6z6m73XJr4z30VGN8e+f1lC8c9SJ9aTQ/9vH3bsaXLW6GFY +DBisBg2/+vMwRSG9PkYrp1p6rGAhwbaofnZE5zApT7PFEX2RVNPU7lgXn84ycRHw +gZ89Mpa9zShL4T1PS8BrKwS7AH/se7ofKW/s8Z9SgngTWj0Efd4hZmn/EenVHBWf +kjAkvKIgGE8FJF1QlmU5dHDFhRiUGXIaB1rYAcwwuwB06fxRqEL3pU6jkHSru3ry +sYaY/cfpd5D5PT+FlxkzAPH1iiC3knXpcotWpJ2iQshsw9ifwg/vVJB0n20+Rxeu +XTgwiT+X5mJNAQUCj6aExWUg+D5gPnJPwFmzAWBGKWrvwI+vI6zIv4MJywzU+Ei8 +1lU5rezPovAbGSTwUBPDydhORua0P8tVT8KPMmPJhza6IORTPpzdEOCXCOH17CWg +VWKjYvEul8CdNh4O3CJDU4lN8yn6RXCBPK4NKDea17GCIEBgnOnpFny+jdfNT+Ce +9aNh8ah61vbPag9EM2okmBlbnpkhUO+x8K8prZHZE7qRgUbmn1cJwIP6pNN/263q +S2uKZMnoaT65BaQh9wpgSvWmDup3/lGG/C2+m0k087QBVHMSfpTK9WcZ94BbzoeR +S9rWCU2k/woEUOv3hssY5w== +-----END ENCRYPTED PRIVATE KEY----- +-----BEGIN CERTIFICATE----- +MIIDdTCCAl2gAwIBAgIURm3Wk48Uc3RtqZ0FFxCaE33TvjcwDQYJKoZIhvcNAQEL +BQAwSTELMAkGA1UEBhMCVVMxDTALBgNVBAgMBENhbGkxDTALBgNVBAcMBHRlc3Qx +DTALBgNVBAoMBHRlc3QxDTALBgNVBAMMBHRlc3QwIBcNMjAwODI4MTIyNDU4WhgP +MjI5NDA2MTIxMjI0NThaMEkxCzAJBgNVBAYTAlVTMQ0wCwYDVQQIDARDYWxpMQ0w +CwYDVQQHDAR0ZXN0MQ0wCwYDVQQKDAR0ZXN0MQ0wCwYDVQQDDAR0ZXN0MIIBIjAN +BgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAvyrgqfiwRh2EcPYk/GFMgF2k8yJq +4mkQcjpWlowp/fL99W/c33ySEtV/LyMJ6LtPAIKN4SambUFcx97pEgJi9YEDLRGW +aN9jznss5AAe03uXN2gizpq9LmdPxr/vmH1DnqdI5MKwa8g9phpe9tT6ik3f2qkm +1V9Ka38GlkHbB+w743ytz20jM6ifTtrX0SuDqDbAppandv5Ix3CHdlllRS/MKNEw +LAs7LVkct0UNTp+soTIhGcASmbf24dvJnO+Msfuqw60mHJpoUP/xDcRGcXnjsgAZ +zAi0UXlV9QiItQeOKxLBHIlMSAEd9oEejPCi6uN+zjKb3De7LUD2Vxu7iwIDAQAB +o1MwUTAdBgNVHQ4EFgQU6tvbiSgA6pujKJBSFw/j+QRZOiwwHwYDVR0jBBgwFoAU +6tvbiSgA6pujKJBSFw/j+QRZOiwwDwYDVR0TAQH/BAUwAwEB/zANBgkqhkiG9w0B +AQsFAAOCAQEAgVa/jUdGqElv3EzxzSqtYkGxmeY5m2aLPKgsUl2zOPWwgyE1Synu +TuFS+B5pyTT1KZ6IDwRQ+hA9jOnEopNK33lVelz7XBuw7485qVidG4o6QH1uo/J9 +GlxjM5SCY6yQ4frCI8lCY6+LA0NbI05qtVNTS1zgdOBnC/IOlMFpp0oDaf5FZHxc +Ci1I/g32ES3rvKiAGBY2m6hy138GzYpZTXnKS03MaTfUCFfsOvqq/z2KBCeCd4mH +VDO7adjhw4I7EYYXjmly2um6NaqyXtT6/AARY3JuQgFoW7W3XBV6TCsYmsGSeUTH +JrhnGnHiNi06IuBwOXYZDID+orBMr9NDKw== +-----END CERTIFICATE----- diff -Nru mitmproxy-5.1.1/test/mitmproxy/data/servercert/9da13359.0 mitmproxy-6.0.2/test/mitmproxy/data/servercert/9da13359.0 --- mitmproxy-5.1.1/test/mitmproxy/data/servercert/9da13359.0 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/data/servercert/9da13359.0 2020-12-15 16:41:27.000000000 +0000 @@ -1,21 +1,21 @@ -----BEGIN CERTIFICATE----- -MIIDazCCAlOgAwIBAgIUTE2oFYATbkkytGOt0+CIAOPpaeEwDQYJKoZIhvcNAQEL +MIIDazCCAlOgAwIBAgIUN1abbPeW/FETKzGlISahLuBhKEwwDQYJKoZIhvcNAQEL BQAwRTELMAkGA1UEBhMCQVUxEzARBgNVBAgMClNvbWUtU3RhdGUxITAfBgNVBAoM -GEludGVybmV0IFdpZGdpdHMgUHR5IEx0ZDAeFw0xOTExMjMwMDA1MDlaFw0zOTEx -MTgwMDA1MDlaMEUxCzAJBgNVBAYTAkFVMRMwEQYDVQQIDApTb21lLVN0YXRlMSEw +GEludGVybmV0IFdpZGdpdHMgUHR5IEx0ZDAeFw0yMDExMDEwMDI0NTNaFw00MDEw +MjcwMDI0NTNaMEUxCzAJBgNVBAYTAkFVMRMwEQYDVQQIDApTb21lLVN0YXRlMSEw HwYDVQQKDBhJbnRlcm5ldCBXaWRnaXRzIFB0eSBMdGQwggEiMA0GCSqGSIb3DQEB -AQUAA4IBDwAwggEKAoIBAQDZ7fqv4iC6738OvOC/ONFmhOVmMNe2SrGtk+7KIv/M -d7MJJKpIygwc99EfL3m6WFjvSQ8CABO9GnLIeoB6RzX5X904dPQzIekMkE27J4D5 -wzYWAmYsu9md//9rjadaA01zHvxtpc30AsToOrRa2Rv7Lwotvcop2bBCXXZMdgvD -BpCuCOl3cQ346LZBmED41q77P7SgnI98gCPuyyMVAFTZE4NnFUloVERTcMMo5Vwv -tXFT41B6FOqtyhZFY89W1SeNKlm+n8zu+/aUF5c9usxwvQzj79pzBFYS9xMZzbxl -BmU91GbLFNU32O+wp0jX6c1MfluGvRjcHQpK+gHRRJdZAgMBAAGjUzBRMB0GA1Ud -DgQWBBSTmtfFP/zQDiCbte3AG3vchjSekjAfBgNVHSMEGDAWgBSTmtfFP/zQDiCb -te3AG3vchjSekjAPBgNVHRMBAf8EBTADAQH/MA0GCSqGSIb3DQEBCwUAA4IBAQB6 -CnGmyJKVdD41Qs8QbyZ5VvrAkSq7DkL4CAIDy0pClOYjx51R16g4/dXPd8nuDjVB -GbRuO/ZHlKeO+RAVbV4s74KeBmQ4uyuZMvuiJGCDtTgu+fbgp1l41vj21bkDnz/X -0aVrwC4clh6O1/AFCY7cYygp22QJ0zGpIb9d0ly6QDMEocHJKasuQ8/0GPvv6wkY -hwNub2ScsVFHPrFzzufZblBk3Ks0YZw+IsCMTY7QtGb1TZhGeX9xAydqNMnmIxNj -XOE2FKx4ISQNFwf/U1LEAruO7PjbbsWx5FHr4oYV9TCLoERZFofwcGGM9o/cPc4P -CH8fGvFYoU30PG+xX0Pc +AQUAA4IBDwAwggEKAoIBAQDOyhGCoQjxdBmDKcU8NDyilLm33sKMhVMaKiv44n+s +UovY92Ie/GQ5C63H0LPyctQfCDXatx2SzGRXy4ii5QsD0Jz0mva9M/Y15iW8+9Ng +wjYXQBBRkin0ll2QXrxnqvRExiVnRZ5mJsUHyTG/6krFkA+O0kOwP40vkD0joKp1 +L0mN5jFHUJcnidwBujyt832zFgRnKZejMBOm1t5ihwtdQDzCtLvfmm6C/+4jyypJ +7HW+yjGQXj7uM5hD72azf5PXmfNZ+WrGHXsSXoJSPAYHQnH1gLhYIfcMa4yH+HIr +/BCk2BNggqKnyOWIKAYo5e3H8JYatGIpzoisU9Pm//cjAgMBAAGjUzBRMB0GA1Ud +DgQWBBRwRGTY59eiUG6soqnj4ZyjIAYwaDAfBgNVHSMEGDAWgBRwRGTY59eiUG6s +oqnj4ZyjIAYwaDAPBgNVHRMBAf8EBTADAQH/MA0GCSqGSIb3DQEBCwUAA4IBAQCI +cyriRs4fySXNG3SP/od3BbUulKHC37DvV8r/0vMPD6r5znjX08KvjIEiSNQkGpyN +G06/Jt1p1mucPwVf2VrSmC7RV4/5NpzOIUNfjMaA3EASl+Mtz8Cinp0e/MECi7Tz +VtBtmia1pbPIjr8/GSZP74kbysNccz06c8OyrI+91vq3isa+xJgjXYcyLnY5rNok +bet7QHb1QE2oDPTZlrmZ9Rsh6K6xcwfKtXbsjMDFrdROKwdP1zGpQQIqOelaOixO +9qGk0Y9Tq/T4ItTdQJYvCSxOvU8LDVDeYfOYwW3kbvclyyZ7CnieWxlzCY41As1u +2OuNWUhfNcoqamAkv0nH -----END CERTIFICATE----- diff -Nru mitmproxy-5.1.1/test/mitmproxy/data/servercert/self-signed.pem mitmproxy-6.0.2/test/mitmproxy/data/servercert/self-signed.pem --- mitmproxy-5.1.1/test/mitmproxy/data/servercert/self-signed.pem 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/data/servercert/self-signed.pem 2020-12-15 16:41:27.000000000 +0000 @@ -1,49 +1,49 @@ -----BEGIN CERTIFICATE----- -MIIDjTCCAnWgAwIBAgIUb0mIVHKB+bu2PGObggokU3Re7xMwDQYJKoZIhvcNAQEL +MIIDjTCCAnWgAwIBAgIUR5bIx5hi9BsJ9j2CF4yeWvQrHQEwDQYJKoZIhvcNAQEL BQAwRTELMAkGA1UEBhMCQVUxEzARBgNVBAgMClNvbWUtU3RhdGUxITAfBgNVBAoM -GEludGVybmV0IFdpZGdpdHMgUHR5IEx0ZDAeFw0xOTExMjMwMDA1MDlaFw0zOTEx -MTgwMDA1MDlaMEUxCzAJBgNVBAYTAkFVMRMwEQYDVQQIDApTb21lLVN0YXRlMSEw +GEludGVybmV0IFdpZGdpdHMgUHR5IEx0ZDAeFw0yMDExMDEwMDI0NTNaFw00MDEw +MjcwMDI0NTNaMEUxCzAJBgNVBAYTAkFVMRMwEQYDVQQIDApTb21lLVN0YXRlMSEw HwYDVQQKDBhJbnRlcm5ldCBXaWRnaXRzIFB0eSBMdGQwggEiMA0GCSqGSIb3DQEB -AQUAA4IBDwAwggEKAoIBAQDF6eqyZD2z83buUF4T+6AY0Zoe925a2AHOhGtHJMLo -9AD7FF1Xi1iksEvxbOI6mreHtvYKzUpfNsA3DkFdSO91HMSkdvWcDcExpW62sNK9 -gQQrpcCx7DogOrFiGSIHI1LIy1y6YEJma3G71SgGbw7g1QF64dTX9+BzVQJsloT2 -H3ZxTi8Fb6APJq6d/Tp67GTM8U82vM+FjLKzfH7RMSEPvSyvWSibw3AZP6owFaxz -DJtXpR7evOZbiZxqXmGOBl5OQPu9GdDA3Fyi9Drp7xa234loqd1a8PYyL8qWV/2G -HM3IJOzG8Y5PUJhL8CkDbx4LJ9LfzeSuBnQPUf2ZNalZAgMBAAGjdTBzMB0GA1Ud -DgQWBBQbkS0mAGPD4XjOv9GKzGMyR9ix3TAfBgNVHSMEGDAWgBQbkS0mAGPD4XjO -v9GKzGMyR9ix3TAPBgNVHRMBAf8EBTADAQH/MCAGA1UdEQQZMBeCFWV4YW1wbGUu -bWl0bXByb3h5Lm9yZzANBgkqhkiG9w0BAQsFAAOCAQEAqZ64xpu3qrTlCc555OoP -GgobUFP3qv07d0/r48cOyYdAdlEHhvmDPlqWTB9e4ZYtWZMlocY9DpCywzKTa7F7 -Ad8BwS0a/No3wVdl1UEkIGYxuD//jbd77Mrpf5URvQco85o/bicn+H0GAOchYt1P -jP1VShqsRv6WiTs5kn1/JwVoafddl1jBlMDmCqDv4loAZJYHzie0CqdjjSeorFfE -8FG8OLwmEnmIW6VnanRH8coH9MBbZ+dRtCavS+Q8s0R77dJM1sCp5/4yKcr5D/PD -+dQN9f+iugLxDdBQjiRyadWX9/l4n/h8ezabZ4cNsiWbRXrp5VS0nGNmAK3XvPAu -ow== +AQUAA4IBDwAwggEKAoIBAQDLpkN07c5l5Llw3fSkLcsb0VoGTVMa7Z7YiU2Dpdnv +gQoI6JzUDXHuXsItaUk/V5nqiLQaNto/Hfsx7q/NmPAHoLFVxxgQc6Z9t0fE6Ko2 +X6TLXNwRtUM38OIdMSobJgwWZaa5HmBZlO+WcvfCCp01gsx69HhVzAWHtX+HEl+j +XXCujerj6bOHUYhItbkUD8EqVmEyyyrzitTd7xN4HLsECI2yAaC+laNWpAoYKSVD +P+G6kyzV3UlpqN/h6sOxWFVcmgxZrIwCpRKxYciO9OrOPPW/VCggXNfX2R6JT6Bi +TwjdhChWmpyOg196Vyz/fTNMECNBA9S1YXvYX4SW3CxXAgMBAAGjdTBzMB0GA1Ud +DgQWBBSYrvmp80kdLiIeEw3LzI8caaAIGDAfBgNVHSMEGDAWgBSYrvmp80kdLiIe +Ew3LzI8caaAIGDAPBgNVHRMBAf8EBTADAQH/MCAGA1UdEQQZMBeCFWV4YW1wbGUu +bWl0bXByb3h5Lm9yZzANBgkqhkiG9w0BAQsFAAOCAQEAtgnRA72vhXqoarlFNhbv +ghkEEi5f6auAkfZmITMOdXE3oI4QuPvZRtjYxtRM8cu17wf8W/h2lOsXDyg0Ndx0 +56XO7K17lV/HA73is85ikchk4KnbOxyrMKD/ssOO8eUWz07vnzM+nfWny8xeeAmV +DT2tC/l6gG3SYYZKfp/wkHw503OJS8ZtWQviTeRJCDFcOzrI8OVBdb/lybNJ7uEi +aPEqViFlPnMFYLqquej5D/pRo4B664++6f4UVXqudGPYXy9TSTjw+R0mdaQKNX70 +Jz8HH31q3puDMUgP9X3LTxNhbvvK1JJDOuOCBJdY3ayr8MHzn43f6CYwfEirGJ4u +Ng== -----END CERTIFICATE----- -----BEGIN RSA PRIVATE KEY----- -MIIEpAIBAAKCAQEAxenqsmQ9s/N27lBeE/ugGNGaHvduWtgBzoRrRyTC6PQA+xRd -V4tYpLBL8WziOpq3h7b2Cs1KXzbANw5BXUjvdRzEpHb1nA3BMaVutrDSvYEEK6XA -sew6IDqxYhkiByNSyMtcumBCZmtxu9UoBm8O4NUBeuHU1/fgc1UCbJaE9h92cU4v -BW+gDyaunf06euxkzPFPNrzPhYyys3x+0TEhD70sr1kom8NwGT+qMBWscwybV6Ue -3rzmW4mcal5hjgZeTkD7vRnQwNxcovQ66e8Wtt+JaKndWvD2Mi/Kllf9hhzNyCTs -xvGOT1CYS/ApA28eCyfS383krgZ0D1H9mTWpWQIDAQABAoIBAGl4H8+TZeJ5E18q -ywfhJ08ym+x2tYOJ62SP4s+WApy8M62aC6g0pTeWj9IH0YOjobycPwBAqKqW9dYh -Lao1zQ5fF1gB4R+ZoOQBIkAPeS7uCzfrbAYlOlCklpUNibm+FEbXQQI9fAUyqviL -Pno3QvmD6fb/VDsHaMBthA40JIU4C3yUUcSooKKUC3XKmbRDg5/stKbVdi6co5AQ -OMNFj4smts3rXtk5tKMBep4AgKKdClf4IuhpWEqZDxN6e+hZA7g1VOmOIE+hKEEO -ODBkjHNgHIFbsr8GTlZ+GSkHBxMNVdHxntJTU5a2jPiTNTTnhqu2hcVdkyDwLb9x -TGUCdyUCgYEA/ZPwI2KaT4Lj8b+6DCV4v3nZF8nLCtt1LiXxJWexdnr+3q/ZfNjN -5sKVl9rIynz2zoJDVr1MI8yjO6OmV7GBW2wgJsSFwg0vnV/bs75KqPZWZXqy/mrp -HnXd82XUweyyOpmBg4cCjmcnMTrdvCHvfgx7JafnUDqeXNwriWeC5Y8CgYEAx83d -qHgAT2rlnCQYRmCOGb5fuyQ+pSM4p8h1AjueLrTNiWcqX57ct9eVWjymD+AgevFZ -hZH9m6TI4nCb/5ukcvjQb4g1kyY3l7rZiiJ3kf/c22kVsVUagJGeiW6Ypnox4r1o -oFXEwAw1qSGIqOIjYVB2DqvjUl3+UjWCl9SOnpcCgYEA0TSdWUQ/VTwCvW9VmjHM -FgT8I5Ebj+CRI7qv4hFTqxE8dxKTl1nzPd/ptTgOkmhY4vU7gzN3vs1VGp4gXZcX -xwpE2Fcol3lzgB4Wz4s+Y3mgu+ZoCFjB7ZyGugmYZ0nVnV0KKi5X4I6gGhCb4VwK -D29SpjWJNHq4LpqC3MDmkGcCgYAxnKOKXmmtTpy+3ZONfhIqwEOjA0fu10UNHFA5 -grYvYMOcd5pk7dxeZdB2/JI7ZOqLvHv/F5YCXLNozo9ds7bsuW2AFDFBXX72VPYJ -P6+y9/ZOINS7GKeg/wd/lo+e3r6eT2u4TDOzgBSe722wiZ5BXqpB0Fp8rEwm+5R2 -wNe89wKBgQC59SOa4EW3gfjLRaH6hEPcMEzjr0Dt5ilBgntYbI+DsCePwEX28hAK -fd7A6XoCqjnwLpAmh0cmNRsYjtPCzg6TXUdrhfnR/3r7DHzvsxaC06BfkPZJ0+Fu -rP3el/oKSPaDsZKjW6RA5oqlIF82o45MNl2oCG82LgVtu25e3HbJSA== +MIIEowIBAAKCAQEAy6ZDdO3OZeS5cN30pC3LG9FaBk1TGu2e2IlNg6XZ74EKCOic +1A1x7l7CLWlJP1eZ6oi0GjbaPx37Me6vzZjwB6CxVccYEHOmfbdHxOiqNl+ky1zc +EbVDN/DiHTEqGyYMFmWmuR5gWZTvlnL3wgqdNYLMevR4VcwFh7V/hxJfo11wro3q +4+mzh1GISLW5FA/BKlZhMssq84rU3e8TeBy7BAiNsgGgvpWjVqQKGCklQz/hupMs +1d1Jaajf4erDsVhVXJoMWayMAqUSsWHIjvTqzjz1v1QoIFzX19keiU+gYk8I3YQo +VpqcjoNfelcs/30zTBAjQQPUtWF72F+EltwsVwIDAQABAoIBAH2Z2djwInAtlUHL +YusvYymRARkJ78uX6nRamngrfjJ9P5cnADK46B/ASbqASTDL8p34GjxgwFJVR2wQ +blU7ki7iKP1igbED4xd+RsFnTjOi4ZybdQ9m4EJMSuj+PZ3o5WOa5i0eUwZtGSeN +WMlQio1KUpsy1FT2NCLzCehgqKO0D9i3qpSZ7V+Oz/ZDTKhfOs2P13XJDPzfzOoW +CtD0GDAsKGQmXkgwdkiVzgsNz4deLZpFNQhz3DLKx3fl6xih/Fd5KsLv5EQbCvq6 +vPZPz4mrEDl95KUM5t9i+ErW2kHLjkg5upsuMT1dCbefowD+0ufLkzlPoYegQo9G +7psLgsECgYEA60p1BYSDSDcA8Q8zMiBaY9udUGLJhBM5mijVt94Qskbofny99xJX +Z8vwRUmtu7Or7B5A6zmOiO/6e664IjBKyHtpFQ6SwPWm1tLtTWL6eBFqThqQbWZY +PnUvLYkBGQZsM7rYSY3cFT3CCoSFoM7XRKcnN1Apx+eRHQUau/ReX+kCgYEA3ZLd +7EMF1WC0aOHIbjPnz0e2x3XW69K7uNvXYeQUcBMOUnFJ1DFBJ9nE4L6ZPUt+pbrO +Psm5zyb44mWhBHfGGi71Avi/m/Ry5xcXR4JWRa8zuQTrQyGsxEYmw5daer0OR6V+ +/F5WH+ErfLR/uj5EK0B4XnHikc/nw+KwCTLUwj8CgYEAmeiBdZOBoJpCJmzv6ykY +l1tPowaLlLZSppFxgy1iNIuHhz2nsVcxD3dqephAaZiUrAX6JyPmJZL8gSz/y3VV +ZaC8UwbOXiQR91HP4iANgL67j812jek1bd2Pd7S3oBwAaTIeSrbdsJNOOz0Gf/Lt +Az5nu5nXSqkh1h7BpxKpLQkCgYB2odjiU6w+560QbrrSjkXyDDKHrscoskbCHiq+ +hdCPhRRo/bEKxGhfPtCTiFkhoVcowtEEextaF0KwbLt+WeaeRD/4nFhy1rniktFq +/xTT8I9f44tXASOwXuMIW/QcvTZ1E93en1+7Z+LRvQ6aGBaFldocrxsg+NDdZ3Tb +J/UUHwKBgF9h8mbPapp6yMOSnm1mr6VMRpn3AqBffaN0SoCKfFEwIDRD7nDpyqDq +0X26F6galBZEX1oMn/BgYCb/PwXsQ3nIzT8fYrOy56bvGV04VIL6gWfj2L+oGISn +zgAPifo4u6q8nmx3LQvvtJjmHgCPVAGIWCPAt8spbtXTLtLUv5AA -----END RSA PRIVATE KEY----- diff -Nru mitmproxy-5.1.1/test/mitmproxy/data/servercert/trusted-leaf.pem mitmproxy-6.0.2/test/mitmproxy/data/servercert/trusted-leaf.pem --- mitmproxy-5.1.1/test/mitmproxy/data/servercert/trusted-leaf.pem 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/data/servercert/trusted-leaf.pem 2020-12-15 16:41:27.000000000 +0000 @@ -1,48 +1,47 @@ -----BEGIN CERTIFICATE----- -MIIDajCCAlKgAwIBAgIJAKPoP2hXKVv4MA0GCSqGSIb3DQEBCwUAMEUxCzAJBgNV -BAYTAkFVMRMwEQYDVQQIDApTb21lLVN0YXRlMSEwHwYDVQQKDBhJbnRlcm5ldCBX -aWRnaXRzIFB0eSBMdGQwHhcNMTkxMTIzMDAwNTA5WhcNMzkxMTE4MDAwNTA5WjBF -MQswCQYDVQQGEwJBVTETMBEGA1UECAwKU29tZS1TdGF0ZTEhMB8GA1UECgwYSW50 -ZXJuZXQgV2lkZ2l0cyBQdHkgTHRkMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIB -CgKCAQEA4BZUGKkf52583Kps2SinCzRtGFaGIqtP1YfjaN1H21c08hEERRWf0GqW -PIbZz0NRyqOVM0O+4A9cxKrdqDpALBNXdgB62Ven1NNE31ZjsbFgwbORPTiOQo+6 -Xsu+2KHQdqAGOXSqOdNSoZV/HqAKa41iweg6lUf3yrO/hpfnlXTFbuM/oYFARrr9 -YWY7qH5BEvcZBf4Sg3cWGjdbzYg8S6hnFBBkHPDMbS3xra7H8uru8aG9L70aiq0C -3WsGf1/Qctz3cccB6BdnbiXji0QlP0miT+b52hEzDvNjdXvoObgbOZsUUcLrbz8q -LqaQPj6TA2PYpKcxEdJLDcbN++tglQIDAQABo10wWzAfBgNVHSMEGDAWgBSTmtfF -P/zQDiCbte3AG3vchjSekjAJBgNVHRMEAjAAMAsGA1UdDwQEAwIFoDAgBgNVHREE -GTAXghVleGFtcGxlLm1pdG1wcm94eS5vcmcwDQYJKoZIhvcNAQELBQADggEBAMoK -8+KTYF10dbQ8Hl3Fq4Iux7eutAPuKUXUz9dnCfDRhZukdl+gahRoNvkyMBdXOvvx -R9Z2+Guo8NOYKgT1mJtS/c8IxTkjZj9doujJOVD2wTowSfoaT9z+/EJa+6Fp9+xd -YVsO3E/2Vxai8PCNx8JTXr2axcnBDvpHPRXF21hOI8N94SPAcmLTZsdsTELjrGGa -/BA0y+pCEwW6cY9mMHVAAvRoMoqfocBVI7nrYBaQfFoKuwxscxO679eEv+lbSjul -z/VNdWfqrDhFFSzwRSVchapQ9q1EeTzv++wZRwI5bT0Ib6DFTJB3J5+9RihlFYfU -GI17CI0D//DsFic7QJ0= +MIIDUDCCAjigAwIBAgIUSWoLwl7Rc//TKPwKwv25Vhb6uE8wDQYJKoZIhvcNAQEL +BQAwRTELMAkGA1UEBhMCQVUxEzARBgNVBAgMClNvbWUtU3RhdGUxITAfBgNVBAoM +GEludGVybmV0IFdpZGdpdHMgUHR5IEx0ZDAeFw0yMDExMDEwMDI0NTNaFw00MDEw +MjcwMDI0NTNaMCAxHjAcBgNVBAMMFWV4YW1wbGUubWl0bXByb3h5Lm9yZzCCASIw +DQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBALVyHPsi0UUkdv94efFUihuDpJt+ +6+AmwM/QCXiQz6c042oqEeAFfuANWqJiJKrWBP4+waezPT48iKtn597lV09R3gDH +hLGzEDHoDKY7+2gxT0ETYc6UcrYlRkDOYy+twMcj8I4xdU1Qt0TZhHlLxvDvO/jS +BLQke7xymJr29YPUpRLtvigyihT/zRx1zKKSG9nObUoRq+I73i46baC55fWL3aS7 +u1W+/auqnPLVKAlGS/kgGScuk/SnwMrhfcSLizEjYUt1JzRfyHg88tGY6pzJEnUh +UGYHBxetRlD7Rt2pChrkHHngNbgDkh7gSD4Qm/PwcQEzmWuYP6fcO60o9RkCAwEA +AaNdMFswHwYDVR0jBBgwFoAUcERk2OfXolBurKKp4+GcoyAGMGgwCQYDVR0TBAIw +ADALBgNVHQ8EBAMCBaAwIAYDVR0RBBkwF4IVZXhhbXBsZS5taXRtcHJveHkub3Jn +MA0GCSqGSIb3DQEBCwUAA4IBAQCAD7jv+dTx4Dr5jqtIPFjKmeuUiqti6mx1bpY9 +/e9J923+7xWU50DHxDnwnPKNqYS4jr2Hmht9PRYE+ZxgkxKZzHDzHX+zBMph5TUr +8H84pbV1WuLjWpOC2hBobNHS1Rq707M7wtL+RV6EDKW3T9Foo/F+AUficubOmBF6 +XFD+oXcxRYYGdUaMBzd3tpJMRrUYlGRL+7aFKCM+gMKqFzJwZdcfENjIw/jj40SM +pyTauRIxouwPTxpAYeKZ9JGLgzoJviuq+he7S1m6eiN/NAvK6M+E22FZcYMHsEOV +sTsNwqivVdb0eFVjsAIFgAwJ2BMO6ye13/6QJ3WI6RJOKEDG -----END CERTIFICATE----- -----BEGIN RSA PRIVATE KEY----- -MIIEowIBAAKCAQEA4BZUGKkf52583Kps2SinCzRtGFaGIqtP1YfjaN1H21c08hEE -RRWf0GqWPIbZz0NRyqOVM0O+4A9cxKrdqDpALBNXdgB62Ven1NNE31ZjsbFgwbOR -PTiOQo+6Xsu+2KHQdqAGOXSqOdNSoZV/HqAKa41iweg6lUf3yrO/hpfnlXTFbuM/ -oYFARrr9YWY7qH5BEvcZBf4Sg3cWGjdbzYg8S6hnFBBkHPDMbS3xra7H8uru8aG9 -L70aiq0C3WsGf1/Qctz3cccB6BdnbiXji0QlP0miT+b52hEzDvNjdXvoObgbOZsU -UcLrbz8qLqaQPj6TA2PYpKcxEdJLDcbN++tglQIDAQABAoIBAG5V8DB4TdI5T9ej -Ppcqch2NQc5DBCbb7SI5l5qRogj5BoPOJykQ/bC0WqcQyvxHrGU3aIZma/yM8+OO -Mjfb/q71ExJyKAsOIwAiyn2hXtMmgHq/vNrFFx7lACIe9ihafHd8UbRGom54g+41 -2vKsYJUWd7L8cqQAXJz9JmfSMeAfRBVVLyOpENSVT8gM/LgPokmZbg0wRJTA5oa3 -10f4Ax1HqXO3cQgKKnrvIU72gB7MQ9Y2G+P2eAmiAlzGr41N5sv7uFz0JrFHhqtC -qc7QrGVVNtxanu0s/LWgfQ+LrKldTysX16j/j6J0tg7Pfuq9cMFPklU98dv2LF5J -jMB8eQECgYEA84+Yy/IGVjUPVb3HjmCmWUoxt4Aqop/12/4+1cBNS6h58iLJpcWc -FVh8m3UbuE7shimEg0fZ1rPPkqepKlDw8/sKXDni/khBASOAh2v1RhvOxNGjxDfn -ZvcK+BG99kU8ZWQHpKmn8OGfYswL2Fvo67L9XFg/wDnk1QpJx7E1GVUCgYEA64gg -G6qnsah1NKSW64BPVKi/Gby83l4oa1amjHqq/Unq6iuGB4/QhK4BzW0aOGI7RMTE -SPh5NF5ZhfT3LWD/EnaMm6QxMTSsL9TDIg36WTVZTkbODFOJczg19EdTs1pa6OXW -0NM6psLu6Nf/QmHJzyYxSZ83uqPOM305xgooKkECgYEAnA0XMySglr9sUd1EbJ7U -NkVpUU8XAhdHKWrey4lofN83MsLDPCk+dha5z8jat94pgVQ8iPiSRBP1HNu7cVdm -6ouf+bNFEvMsYxRiF2I+RmsuscA4E1JWOwxxxLtpYM6/gZ7znrbs2VNWEbD2retF -cy69UltgjUMKsMzktMN/Z/kCgYAbzV285lAVMIVlSWhnNCYpICIur5C7zvGGehv+ -yRwV+fu42JphmiBLCR89WHuX3ECSxYdF9c6Y1+pJXbkvqhtx2nyOgrsry8PngX3n -Ly82CI4aJ1F7MwEukJwN0b2XljrU8wyAae6qcKgy5AxFkbV4tlFrF1hEt8FHYqjH -L7u+AQKBgBnN2vjOVagtlZjJadg5ueR7uKnfk7/TGMtZjNf89C1UZH3TqWnnj6p6 -rLLe4tnA3EebFtGeVWlmnw5k166HRBEd7KuebOCq3hkNlf71aHwhDJ0bIdDwo1g+ -FTpG2vqkqpuEb+2FzDjZT5EdPnj8tUGthREDXpXqsdjFb6+2Enjx +MIIEogIBAAKCAQEAtXIc+yLRRSR2/3h58VSKG4Okm37r4CbAz9AJeJDPpzTjaioR +4AV+4A1aomIkqtYE/j7Bp7M9PjyIq2fn3uVXT1HeAMeEsbMQMegMpjv7aDFPQRNh +zpRytiVGQM5jL63AxyPwjjF1TVC3RNmEeUvG8O87+NIEtCR7vHKYmvb1g9SlEu2+ +KDKKFP/NHHXMopIb2c5tShGr4jveLjptoLnl9YvdpLu7Vb79q6qc8tUoCUZL+SAZ +Jy6T9KfAyuF9xIuLMSNhS3UnNF/IeDzy0ZjqnMkSdSFQZgcHF61GUPtG3akKGuQc +eeA1uAOSHuBIPhCb8/BxATOZa5g/p9w7rSj1GQIDAQABAoIBAFx1GIVz+XUlHqoR +RvoNXQ6mJxPBOgqPVMLQPHM+P0HxtkxbwZ4Iztz2kQ+buRmAUy0G54E/2V5EVFQb +b1DqRnI5wkldYaV3HDKCDygJ+pLeRYdgi/9LL6TXG5RcVrht7oTLMIIUe7R5A9pG +iLWSQn51OXcR9amhyZd+QXr9ip+wt/knCeSCOgYhPigleIXFSYkgi9gwkg6CjkLp +zx48QDQmAFcbRS+tPnbgJiB302I5l9RCz6ZoBvl8tVWVV3vSz8khZ8PL4oL9Qlz4 +EnIJLmwXd1odLSOdG1APjGooRwRU7cBjqncinEkw7IvDsIpACchcP7+gF2VskUJX +bqj3Zi0CgYEA7Ytc0VV6sQYcnVIs8vIrE7FGl0fP/vxcF1C0Y42t191uoSxuhEJZ +7f2l3ZrSyRLjV7FyqngOcBYv4NSTrkYNyVu5/ZHf7sJNCgDpg02BtUNrX5w6TEL3 +s0V2GgDBeVCVPsDFg1LQPzSDB9r0A7TERrD48b7FFetr5JOpKjk0v9sCgYEAw4r6 +MRc4D/FTdUMzS+5QJaVCsMUx8242s4dOO2FvVuDJ1gXqTvpqjnqpGEsP4w1A/I9o +xylKXkFvuEQrC6d4brLCvj3KT8QXcMGHF6ZIdyZ/GlTU5l9H2Rr+8NBFM18ZIQ2u +MgqLgIXehmlsG7VpGE6/uENz1WX4xmDIi0CH+xsCgYBZuCZxhliV3hoWrX/+rsro +YC+qWdxMkaJyx++qHkwH2UG8rFx9pdXzlrb8EUsF2RV2/LWzfy7s1OR2hbPVwe3Z +HMUN6ffrZQXV/mw0RKT2AqXecdJvFWgbFOQ7hAePO4lzLzoqlK/E/59x6xf5AmCT +1qngHM6xrb6Nkdv1769luwKBgAZa8/q4eHf8Lew3vOkQ28X9cwdvwUnaISkAkshX +SFkYgTbVhmH+vYvX7Lr/ZYrJmX0b5Bnk+6fIlKr2fDeqv71JTg8Ezxh8lQ5zG8Ln +Ap/svmDKtQivvBQQTozhF/6tbpGmcizqLtKEh1DzCDJ2WO4TUSPZ/V/IKqe36Yyt +Ej5VAoGAIeXj6eECrVyW/q1NUpt/Tmvn2ubzuJfg4KruLOPpLsi05EKeCmjE6Puw +7GxZFfrfxv3IGl12UdegTkYL9eTjte+RqUm/Ofre9aD/hVDoWrUzJkD91uMP/sdh +PmXA4vx5VvMPbQzoMsWrJlVygRw2jQHqL1GRhRkV/NnQVYTegd4= -----END RSA PRIVATE KEY----- diff -Nru mitmproxy-5.1.1/test/mitmproxy/data/servercert/trusted-root.pem mitmproxy-6.0.2/test/mitmproxy/data/servercert/trusted-root.pem --- mitmproxy-5.1.1/test/mitmproxy/data/servercert/trusted-root.pem 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/data/servercert/trusted-root.pem 2020-12-15 16:41:27.000000000 +0000 @@ -1,48 +1,48 @@ -----BEGIN CERTIFICATE----- -MIIDazCCAlOgAwIBAgIUTE2oFYATbkkytGOt0+CIAOPpaeEwDQYJKoZIhvcNAQEL +MIIDazCCAlOgAwIBAgIUN1abbPeW/FETKzGlISahLuBhKEwwDQYJKoZIhvcNAQEL BQAwRTELMAkGA1UEBhMCQVUxEzARBgNVBAgMClNvbWUtU3RhdGUxITAfBgNVBAoM -GEludGVybmV0IFdpZGdpdHMgUHR5IEx0ZDAeFw0xOTExMjMwMDA1MDlaFw0zOTEx -MTgwMDA1MDlaMEUxCzAJBgNVBAYTAkFVMRMwEQYDVQQIDApTb21lLVN0YXRlMSEw +GEludGVybmV0IFdpZGdpdHMgUHR5IEx0ZDAeFw0yMDExMDEwMDI0NTNaFw00MDEw +MjcwMDI0NTNaMEUxCzAJBgNVBAYTAkFVMRMwEQYDVQQIDApTb21lLVN0YXRlMSEw HwYDVQQKDBhJbnRlcm5ldCBXaWRnaXRzIFB0eSBMdGQwggEiMA0GCSqGSIb3DQEB -AQUAA4IBDwAwggEKAoIBAQDZ7fqv4iC6738OvOC/ONFmhOVmMNe2SrGtk+7KIv/M -d7MJJKpIygwc99EfL3m6WFjvSQ8CABO9GnLIeoB6RzX5X904dPQzIekMkE27J4D5 -wzYWAmYsu9md//9rjadaA01zHvxtpc30AsToOrRa2Rv7Lwotvcop2bBCXXZMdgvD -BpCuCOl3cQ346LZBmED41q77P7SgnI98gCPuyyMVAFTZE4NnFUloVERTcMMo5Vwv -tXFT41B6FOqtyhZFY89W1SeNKlm+n8zu+/aUF5c9usxwvQzj79pzBFYS9xMZzbxl -BmU91GbLFNU32O+wp0jX6c1MfluGvRjcHQpK+gHRRJdZAgMBAAGjUzBRMB0GA1Ud -DgQWBBSTmtfFP/zQDiCbte3AG3vchjSekjAfBgNVHSMEGDAWgBSTmtfFP/zQDiCb -te3AG3vchjSekjAPBgNVHRMBAf8EBTADAQH/MA0GCSqGSIb3DQEBCwUAA4IBAQB6 -CnGmyJKVdD41Qs8QbyZ5VvrAkSq7DkL4CAIDy0pClOYjx51R16g4/dXPd8nuDjVB -GbRuO/ZHlKeO+RAVbV4s74KeBmQ4uyuZMvuiJGCDtTgu+fbgp1l41vj21bkDnz/X -0aVrwC4clh6O1/AFCY7cYygp22QJ0zGpIb9d0ly6QDMEocHJKasuQ8/0GPvv6wkY -hwNub2ScsVFHPrFzzufZblBk3Ks0YZw+IsCMTY7QtGb1TZhGeX9xAydqNMnmIxNj -XOE2FKx4ISQNFwf/U1LEAruO7PjbbsWx5FHr4oYV9TCLoERZFofwcGGM9o/cPc4P -CH8fGvFYoU30PG+xX0Pc +AQUAA4IBDwAwggEKAoIBAQDOyhGCoQjxdBmDKcU8NDyilLm33sKMhVMaKiv44n+s +UovY92Ie/GQ5C63H0LPyctQfCDXatx2SzGRXy4ii5QsD0Jz0mva9M/Y15iW8+9Ng +wjYXQBBRkin0ll2QXrxnqvRExiVnRZ5mJsUHyTG/6krFkA+O0kOwP40vkD0joKp1 +L0mN5jFHUJcnidwBujyt832zFgRnKZejMBOm1t5ihwtdQDzCtLvfmm6C/+4jyypJ +7HW+yjGQXj7uM5hD72azf5PXmfNZ+WrGHXsSXoJSPAYHQnH1gLhYIfcMa4yH+HIr +/BCk2BNggqKnyOWIKAYo5e3H8JYatGIpzoisU9Pm//cjAgMBAAGjUzBRMB0GA1Ud +DgQWBBRwRGTY59eiUG6soqnj4ZyjIAYwaDAfBgNVHSMEGDAWgBRwRGTY59eiUG6s +oqnj4ZyjIAYwaDAPBgNVHRMBAf8EBTADAQH/MA0GCSqGSIb3DQEBCwUAA4IBAQCI +cyriRs4fySXNG3SP/od3BbUulKHC37DvV8r/0vMPD6r5znjX08KvjIEiSNQkGpyN +G06/Jt1p1mucPwVf2VrSmC7RV4/5NpzOIUNfjMaA3EASl+Mtz8Cinp0e/MECi7Tz +VtBtmia1pbPIjr8/GSZP74kbysNccz06c8OyrI+91vq3isa+xJgjXYcyLnY5rNok +bet7QHb1QE2oDPTZlrmZ9Rsh6K6xcwfKtXbsjMDFrdROKwdP1zGpQQIqOelaOixO +9qGk0Y9Tq/T4ItTdQJYvCSxOvU8LDVDeYfOYwW3kbvclyyZ7CnieWxlzCY41As1u +2OuNWUhfNcoqamAkv0nH -----END CERTIFICATE----- -----BEGIN RSA PRIVATE KEY----- -MIIEpAIBAAKCAQEA2e36r+Iguu9/DrzgvzjRZoTlZjDXtkqxrZPuyiL/zHezCSSq -SMoMHPfRHy95ulhY70kPAgATvRpyyHqAekc1+V/dOHT0MyHpDJBNuyeA+cM2FgJm -LLvZnf//a42nWgNNcx78baXN9ALE6Dq0Wtkb+y8KLb3KKdmwQl12THYLwwaQrgjp -d3EN+Oi2QZhA+Nau+z+0oJyPfIAj7ssjFQBU2RODZxVJaFREU3DDKOVcL7VxU+NQ -ehTqrcoWRWPPVtUnjSpZvp/M7vv2lBeXPbrMcL0M4+/acwRWEvcTGc28ZQZlPdRm -yxTVN9jvsKdI1+nNTH5bhr0Y3B0KSvoB0USXWQIDAQABAoIBAQCWWYbgHSPzlBOW -eVyc0Hg3QGx7aisIStP2Kt9NeYP87oAISNFqUmq0+Yu+9iQHGbiRrVe7S45SopKa -GVnWApcMKsUWlCl9tWFxF4VpH0HuDm2cFZ+kMR1b0ifHbf0NLsYaLEB+7Sr/s4Fh -rk6LdsnFK5jcIdn9sX/W6WAaND69Ft45sMXuAKPeqwySm7t77nlIEp4lJy/F/rvS -UUV77UVL/5GqC1HOP1kXXEJDxHtNrDPh0fF49Qo/6JPRmLIOia0yKUS5+8lKHh6V -J7nWTucyvviKoYvdJHpi9w2+DTI7mT33Go1Ss4QgHihzcURTtfjsn7yOelS6z8XM -YOxURwoZAoGBAPZpod0Zpjm9trf547reFPLfAJUiKDj1wYXnlpSFAzOf7pSXDrfW -L5dc3c5x2DiBXUlBu03ZWo7wVOKv7d9ByTrvXEqT1/KTqmHoI6Mzy0rn9wU0FmL4 -ASPcpPMnNStOSmRfe9HITDVvgLYo2wQEu1MXf5rGs5HjTRpEGVdWRkRPAoGBAOJo -pQ9uRtaLlCF3KXoRr8HCBKI64IWDki7pbbLTglOBbmptQLfE2GPc/FPBsQwR3KBX -m18A2pc7CRUKzsDavb7SyXTqtP3NF/AguTjGXIv/45QD6A96SStaYR0ZXSA3/uhZ -rAaSSXxaI3mDTc32pXoXJDp7K+CPc5w00I8u1/fXAoGAe1UPoPyPiGL+K0M1yngR -gCZBwmMgQrIutHjfk2Kn4ZTw8wpQYY8grt/aXNP6Zv3I1TvDJgneG6EKu5NWueHR -eGAJj4JEGbPzGaH5BFyOKeXEa6RQeCStXWe4X8OGBzDeZzKrZKqeCjjO8V2tkWtU -3xfp1GwTwLdGBhmDnYUfEl0CgYEA1vzVF6T4gQtDGtADQ5V91je8nKvZvQ4lhoRD -lVZAX7j8tvSNSrMRYypZM9Mtoi9n1524vGqcJpR5WFDN6NUM7iFMCMhCGupgO7Vn -DCFXidzvJgLbna7Zwd/tbWtDQa/KTqmvrwHD49/X5a+n9tapZRiKXznMfUzaU87W -589sZjsCgYBQvtZpNQRqTDNOSuIJGsloMNp1HG2Keoj/nBQ9idkw29rUqhae7ZAa -Csk91ix+AMR8nzQSkpHWOk2+RjdVAQ7m87bnQM6mmAJTcipikDfFxNqOWt7JkM2h -Ydk36LHBIBVLOeqMXrhCHvai3h6efUz7wjGhNxRJ2OGrzWFfLgVsqA== +MIIEpQIBAAKCAQEAzsoRgqEI8XQZgynFPDQ8opS5t97CjIVTGior+OJ/rFKL2Pdi +HvxkOQutx9Cz8nLUHwg12rcdksxkV8uIouULA9Cc9Jr2vTP2NeYlvPvTYMI2F0AQ +UZIp9JZdkF68Z6r0RMYlZ0WeZibFB8kxv+pKxZAPjtJDsD+NL5A9I6CqdS9JjeYx +R1CXJ4ncAbo8rfN9sxYEZymXozATptbeYocLXUA8wrS735pugv/uI8sqSex1vsox +kF4+7jOYQ+9ms3+T15nzWflqxh17El6CUjwGB0Jx9YC4WCH3DGuMh/hyK/wQpNgT +YIKip8jliCgGKOXtx/CWGrRiKc6IrFPT5v/3IwIDAQABAoIBAEawNqoj6E25tVrr +9XtuE+gz1QSwxmqIzO6dyehLbP94PEwPzN/wwUXq4x3yDwFrFW4CPX6lqBtdeVO1 +xOCCUXCdaHoJWx00XQ7xvKbMouZpnVn3UVzBDUyJ9jP63obFnri/z1ttJgefay0N +Ls9/BM5iP58/61mdb+L5eJzsODjUMjaxvgJ8DPQWEuticz3s5sEE6TPCoIZH9Vc2 +Ri8kWy2/tpCLZpJbrrKZNdAFnc8zgSSXwwu97m8KpXNnhRFA1Qu0y3U3NP6oFYnL +57qR+Kqdi7QG7GcvUXsHhsYYTiusJu9PaEomuBqyYFuhyFtHtLPX65nWZMQSK3pu +Eu9Hk9ECgYEA/z8eTD+uMqC7Hz+YUWPMENynAy5KVOSqkBzHg8ePuW/gyTRrT2rQ +wXp6l/nTO5A5jzrAdvMIqnEmyhdnB1AoRM4OtA2/wblsNFykWecVQyD3uplafVtS +D3JgAN3FId/x2fZ8U2rSvHW2gTbel9HS3yhGtHN9giMKVtHTUs4FLlsCgYEAz2ZV +HQf5+deYr/da/XFkkEA9i8deiCcy+wqVlTW6TrEzBPwczLtV36bgubDZLy0NHO4j +oHPMGWtn343MzcOx6/hZGoCoBTLkZghipMMexWgldeBgOqqWGV+7mq8uhc9j2g9E +aLJKUkGhRmPXvlcS396P9bjKCX1nFLgZqw26xNkCgYEArzrl0fLl5Fv0Gr/OEhIx +RmI1RjnBTgauEcGwNwgJ44SrDXavNRZfunQbNbqNCZc57NqoxwCSMnD78EGguaSO +AK516aaGvrhNUKVJL281aaB4ceJV/Tl0XKZowi995oOXxcsyDxKHGlq1sH+60P97 +hThhUgD3aSAlOPVT3sURSYECgYEAkJKQ+i+5PUPgl6khsFH++98MCAfuCZLpor16 +EAzn1jsHiY4EtWJRNhopxMdWXeBVp7kI9KxDteR9AATkovqraAYydpsAhL85Hzbx +LFEnfgxfMJA8BCktgHqPWACNuZEb1cGWujSuQjClr1+h4HjqG0wVpgAuNfdIh9Td +b7OeYeECgYEAjLrVoYhgqiG6qW+UNgzhr2EM4my0Q9dda4DjVIWZW+19XCmqPkEd +ryaqBpolHKzD0GCiutI5S68W9wil3MBgV05qTbFhnTqWGJ3twGNykpYXx6F0aBTa +6hwH9Q/IQ5AONkmSipMRnVWAuDwvdutX/knhEULe8TdGl5CmGkJ/OZs= -----END RSA PRIVATE KEY----- diff -Nru mitmproxy-5.1.1/test/mitmproxy/io/test_compat.py mitmproxy-6.0.2/test/mitmproxy/io/test_compat.py --- mitmproxy-5.1.1/test/mitmproxy/io/test_compat.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/io/test_compat.py 2020-12-15 16:41:27.000000000 +0000 @@ -4,20 +4,17 @@ from mitmproxy import exceptions -def test_load(tdata): - with open(tdata.path("mitmproxy/data/dumpfile-011.bin"), "rb") as f: +@pytest.mark.parametrize("dumpfile, url", [ + ["dumpfile-011.bin", "https://example.com/"], + ["dumpfile-018.bin", "https://www.example.com/"], + ["dumpfile-019.bin", "https://webrv.rtb-seller.com/"], +]) +def test_load(tdata, dumpfile, url): + with open(tdata.path("mitmproxy/data/" + dumpfile), "rb") as f: flow_reader = io.FlowReader(f) flows = list(flow_reader.stream()) assert len(flows) == 1 - assert flows[0].request.url == "https://example.com/" - - -def test_load_018(tdata): - with open(tdata.path("mitmproxy/data/dumpfile-018.bin"), "rb") as f: - flow_reader = io.FlowReader(f) - flows = list(flow_reader.stream()) - assert len(flows) == 1 - assert flows[0].request.url == "https://www.example.com/" + assert flows[0].request.url.startswith(url) def test_cannot_convert(tdata): diff -Nru mitmproxy-5.1.1/test/mitmproxy/io/test_db.py mitmproxy-6.0.2/test/mitmproxy/io/test_db.py --- mitmproxy-5.1.1/test/mitmproxy/io/test_db.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/io/test_db.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,7 +1,10 @@ +import pytest + from mitmproxy.io import db from mitmproxy.test import tflow +@pytest.mark.skip class TestDB: def test_create(self, tdata): diff -Nru mitmproxy-5.1.1/test/mitmproxy/io/test_protobuf.py mitmproxy-6.0.2/test/mitmproxy/io/test_protobuf.py --- mitmproxy-5.1.1/test/mitmproxy/io/test_protobuf.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/io/test_protobuf.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,12 +1,12 @@ import pytest from mitmproxy import certs -from mitmproxy import http from mitmproxy import exceptions -from mitmproxy.test import tflow, tutils from mitmproxy.io import protobuf +from mitmproxy.test import tflow, tutils +@pytest.mark.skip class TestProtobuf: def test_roundtrip_client(self): @@ -66,25 +66,25 @@ assert s.via.__dict__ == ls.via.__dict__ def test_roundtrip_http_request(self): - req = http.HTTPRequest.wrap(tutils.treq()) + req = tutils.treq() preq = protobuf._dump_http_request(req) lreq = protobuf._load_http_request(preq) assert req.__dict__ == lreq.__dict__ def test_roundtrip_http_request_empty_content(self): - req = http.HTTPRequest.wrap(tutils.treq(content=b"")) + req = tutils.treq(content=b"") preq = protobuf._dump_http_request(req) lreq = protobuf._load_http_request(preq) assert req.__dict__ == lreq.__dict__ def test_roundtrip_http_response(self): - res = http.HTTPResponse.wrap(tutils.tresp()) + res = tutils.tresp() pres = protobuf._dump_http_response(res) lres = protobuf._load_http_response(pres) assert res.__dict__ == lres.__dict__ def test_roundtrip_http_response_empty_content(self): - res = http.HTTPResponse.wrap(tutils.tresp(content=b"")) + res = tutils.tresp(content=b"") pres = protobuf._dump_http_response(res) lres = protobuf._load_http_response(pres) assert res.__dict__ == lres.__dict__ diff -Nru mitmproxy-5.1.1/test/mitmproxy/io/test_tnetstring.py mitmproxy-6.0.2/test/mitmproxy/io/test_tnetstring.py --- mitmproxy-5.1.1/test/mitmproxy/io/test_tnetstring.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/io/test_tnetstring.py 2020-12-15 16:41:27.000000000 +0000 @@ -15,9 +15,9 @@ {b'hello': [12345678901, b'this', True, None, b'\x00\x00\x00\x00']}, b'5:12345#': 12345, b'12:this is cool,': b'this is cool', - b'19:this is unicode \xe2\x98\x85;': u'this is unicode \u2605', + b'19:this is unicode \xe2\x98\x85;': 'this is unicode \u2605', b'0:,': b'', - b'0:;': u'', + b'0:;': '', b'0:~': None, b'4:true!': True, b'5:false!': False, diff -Nru mitmproxy-5.1.1/test/mitmproxy/net/data/verificationcerts/9da13359.0 mitmproxy-6.0.2/test/mitmproxy/net/data/verificationcerts/9da13359.0 --- mitmproxy-5.1.1/test/mitmproxy/net/data/verificationcerts/9da13359.0 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/net/data/verificationcerts/9da13359.0 2020-12-15 16:41:27.000000000 +0000 @@ -1,21 +1,21 @@ -----BEGIN CERTIFICATE----- -MIIDazCCAlOgAwIBAgIUTE2oFYATbkkytGOt0+CIAOPpaeEwDQYJKoZIhvcNAQEL +MIIDazCCAlOgAwIBAgIUB5WLNdtHRvciUwDfySAEHYVSNe0wDQYJKoZIhvcNAQEL BQAwRTELMAkGA1UEBhMCQVUxEzARBgNVBAgMClNvbWUtU3RhdGUxITAfBgNVBAoM -GEludGVybmV0IFdpZGdpdHMgUHR5IEx0ZDAeFw0xOTExMjMwMDA1MDlaFw0zOTEx -MTgwMDA1MDlaMEUxCzAJBgNVBAYTAkFVMRMwEQYDVQQIDApTb21lLVN0YXRlMSEw +GEludGVybmV0IFdpZGdpdHMgUHR5IEx0ZDAeFw0yMDExMDMwNzAzMjZaFw00MDEw +MjkwNzAzMjZaMEUxCzAJBgNVBAYTAkFVMRMwEQYDVQQIDApTb21lLVN0YXRlMSEw HwYDVQQKDBhJbnRlcm5ldCBXaWRnaXRzIFB0eSBMdGQwggEiMA0GCSqGSIb3DQEB -AQUAA4IBDwAwggEKAoIBAQDZ7fqv4iC6738OvOC/ONFmhOVmMNe2SrGtk+7KIv/M -d7MJJKpIygwc99EfL3m6WFjvSQ8CABO9GnLIeoB6RzX5X904dPQzIekMkE27J4D5 -wzYWAmYsu9md//9rjadaA01zHvxtpc30AsToOrRa2Rv7Lwotvcop2bBCXXZMdgvD -BpCuCOl3cQ346LZBmED41q77P7SgnI98gCPuyyMVAFTZE4NnFUloVERTcMMo5Vwv -tXFT41B6FOqtyhZFY89W1SeNKlm+n8zu+/aUF5c9usxwvQzj79pzBFYS9xMZzbxl -BmU91GbLFNU32O+wp0jX6c1MfluGvRjcHQpK+gHRRJdZAgMBAAGjUzBRMB0GA1Ud -DgQWBBSTmtfFP/zQDiCbte3AG3vchjSekjAfBgNVHSMEGDAWgBSTmtfFP/zQDiCb -te3AG3vchjSekjAPBgNVHRMBAf8EBTADAQH/MA0GCSqGSIb3DQEBCwUAA4IBAQB6 -CnGmyJKVdD41Qs8QbyZ5VvrAkSq7DkL4CAIDy0pClOYjx51R16g4/dXPd8nuDjVB -GbRuO/ZHlKeO+RAVbV4s74KeBmQ4uyuZMvuiJGCDtTgu+fbgp1l41vj21bkDnz/X -0aVrwC4clh6O1/AFCY7cYygp22QJ0zGpIb9d0ly6QDMEocHJKasuQ8/0GPvv6wkY -hwNub2ScsVFHPrFzzufZblBk3Ks0YZw+IsCMTY7QtGb1TZhGeX9xAydqNMnmIxNj -XOE2FKx4ISQNFwf/U1LEAruO7PjbbsWx5FHr4oYV9TCLoERZFofwcGGM9o/cPc4P -CH8fGvFYoU30PG+xX0Pc +AQUAA4IBDwAwggEKAoIBAQDVWUg2YdsztWIZpsWdbIB6Aewa1Igt7mNkKMQ1NJ6v +priVlzWhZr0FfSVTid2m3Od2f5A751Q+IXLkNJ8d6Y4qVirAstEM57sCH/Hmt1E3 +cZOAzLQY2xCIrKPmbu6d8eSGO1q+Y8KstCK/V94/QRahjjoHxkLcSAmdg6PAkbhB +7MwOxiJslIbsBY4UCXP0l4kUUIuMi8W2Y4M1VgvLRpUjaKVo1NQ0hLi4XBnU7Bsq +A8/PZDuVeP0NcJtAxRicqqLX/MjgkTrA2tPnSj9m60lO79S40GSzB238o2zJQ5ON +jHJdAhLlrsrkxZmDmKM59IdmV2OhN7O844ktbXDOqvq5AgMBAAGjUzBRMB0GA1Ud +DgQWBBTkDj66Wmaf9YP0YMqGKG51PejxWzAfBgNVHSMEGDAWgBTkDj66Wmaf9YP0 +YMqGKG51PejxWzAPBgNVHRMBAf8EBTADAQH/MA0GCSqGSIb3DQEBCwUAA4IBAQAo +9FwbGob1WRAo7ORShaDhopIDAYMCL1oc9YZ2ZQK8aDuZbqIZ/7+1LbWCeMAV4PTH +Tcx+n9Zg/g/RkoSNu8KqFoQGFmdBZnKOMU4vlITu/ORpDu1sjSA+Eo9YbipemX+n +jv+YHI4STFAWnyext3IUZVkT6wpU3pwUjX0fbk4LJfVLR41y4iD11XGergxAcpjj +T03txkJcrTiX65SnB041Y4exUMLOUn5lTs/q2rBNkiLNljXQ6l+8L1rdQEN/j0Mx +OYdc6FIUIESC1mMOf80+YOwxPJ862SyTv/cJB3npwTj/DdQu7blf/z2hMP7a7w+X +l5d31XzcDOrCf/bTthvb -----END CERTIFICATE----- diff -Nru mitmproxy-5.1.1/test/mitmproxy/net/data/verificationcerts/example.mitmproxy.org.pem mitmproxy-6.0.2/test/mitmproxy/net/data/verificationcerts/example.mitmproxy.org.pem --- mitmproxy-5.1.1/test/mitmproxy/net/data/verificationcerts/example.mitmproxy.org.pem 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/net/data/verificationcerts/example.mitmproxy.org.pem 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,48 @@ +-----BEGIN CERTIFICATE----- +MIIDZDCCAkygAwIBAgIUSWoLwl7Rc//TKPwKwv25Vhb6uFAwDQYJKoZIhvcNAQEL +BQAwRTELMAkGA1UEBhMCQVUxEzARBgNVBAgMClNvbWUtU3RhdGUxITAfBgNVBAoM +GEludGVybmV0IFdpZGdpdHMgUHR5IEx0ZDAeFw0yMDExMDMwNzAzMjZaFw00MDEw +MjkwNzAzMjZaMDQxHjAcBgNVBAMMFWV4YW1wbGUubWl0bXByb3h5Lm9yZzESMBAG +A1UECgwJbWl0bXByb3h5MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA +qcEdB7TZD7DxJlJH/ZTyrovP0din2vLCRZRZ07ix48arITGWj8wFJ/XpZsFmAWlH +2LriXGC8BB54EJt8Epix5tcdKvCZ/7vTgaBQ+DQVdCNiscaKwvDbigy037OY97sT +6Ou9xYjjgg0cb+WTkQGMOTMbCW2e5iuSvyMPdZ2+2RzbTcNV1/6O/5cQ7TOW2E83 +xeaN4stBMfRmsQRoHXBri0tqtQ3bHd3V4H9iai7HFBOcB/nsJUwLliZpz3j/dUD0 +yZAgdgB9XBlGvs/e746bGctyvl2QZu2JeACEVPmVrAUhqe2TVMVLDT17uzNeH1x5 +UXxdB4cexTOm4amvWSwvTQIDAQABo10wWzAfBgNVHSMEGDAWgBTkDj66Wmaf9YP0 +YMqGKG51PejxWzAJBgNVHRMEAjAAMAsGA1UdDwQEAwIFoDAgBgNVHREEGTAXghVl +eGFtcGxlLm1pdG1wcm94eS5vcmcwDQYJKoZIhvcNAQELBQADggEBABwKrf+Gy1I5 +BnfhAqbDUYj8OVcjbB5esbvVYAADNNNQ10RREyz5OTTo2dSKy2hIVQeGxaBy5W6d +cNk0CoLzOgpgkwQPQ351+Y0C5yQztXIg/JM5aSeFuAbQg+NtzOQISrcDB390Xr2f +7YhyYL0XLX4NCP7ek7+lAClrf+lL9MysHTTP0pmFWLPP6AIR8JIsLpNpTDV10c8p +QcnJ8+5q6oLHNKN+M/HRcdKzj/XyJ538UzSOlmU+izhNT1j7mbszS0tc9yh0dmpP +9uuBY0wcD/NAHBpSX9v9ZJcOuWntsAxTbMXj1eWNRyt0QJLURlkqeVn6ZrE1/BDS +BqDdOKYJbkw= +-----END CERTIFICATE----- +-----BEGIN RSA PRIVATE KEY----- +MIIEowIBAAKCAQEAqcEdB7TZD7DxJlJH/ZTyrovP0din2vLCRZRZ07ix48arITGW +j8wFJ/XpZsFmAWlH2LriXGC8BB54EJt8Epix5tcdKvCZ/7vTgaBQ+DQVdCNiscaK +wvDbigy037OY97sT6Ou9xYjjgg0cb+WTkQGMOTMbCW2e5iuSvyMPdZ2+2RzbTcNV +1/6O/5cQ7TOW2E83xeaN4stBMfRmsQRoHXBri0tqtQ3bHd3V4H9iai7HFBOcB/ns +JUwLliZpz3j/dUD0yZAgdgB9XBlGvs/e746bGctyvl2QZu2JeACEVPmVrAUhqe2T +VMVLDT17uzNeH1x5UXxdB4cexTOm4amvWSwvTQIDAQABAoIBAEb3arF8E3qR2F7S +6zHCASqjXIA3+QR5lGoOOPdgMU4uEgDQgEchXc506dyBYamZX+XlSxifgDqgmkUn +G1mS6Fy+9XysFVVqKmP4p6D79TQWTv5PKFeS5dTytvMGXB7E4O/xDeb08Ve/L3JA +Ic7vPLX0/YqVf2ZuNO0fNSlQhyawUVlSS6yq/B1GUo44PX/PXrWYZd66mc29g+KF +7D0nnbloLY+5QmOEVYbf99RHJ64Cwxzwxi4ic9AxLaNMfkrydjbGQ0VhJq14WEux +/TJLvinzIDyslnx4aYstno3ryoxLtR/9ALysrdQ8vVMaVCdbf0aMu832R2dI4s9K +ixQlgmkCgYEA0Ho0a4qzwk9oLWYk1x16kOCqmfuenlEz3Tv8uBB/jdz7UM63sS3h +OUQYKhhtzOGf6CDA3CE11ie6SX7zHrowPLJBSpacJZJcrHPGZxdBZ/As/QpGnZll +1eTMlVRXk06k/fEYplU1dS5IDbsvUIaaF9CF9Qd3MkDRglhbM19Iy+sCgYEA0HM1 +d7OBsjSi09zFDE+Joq6Gih8LqqFLtMsIHEKAJ0gJgmt6MTfjtdPo55ZKVLCR2pbP +yRHagK4LRUFY9Pl5eGlEUF9cKEKKtZOURUFVz3yBMYPBje/B1o/UCCj+nr2reWgx +qj/jwZrph6QKCkEsnQXi98tE7XSgihUh6UEQO6cCgYASVe0uWDCfMmSzOXyb/te8 +zkWy7VJyEipBlvkPJ0RQsdLYtJWrW6Gna7nEWgmuL1nlDJxpv/IAN9ZGiIfReAau +D+92I/DvzQOhlz0n6/+wqIsMZk73pXozacAkkhpxtkUEoKPOXUgqWju0GXZ72prK +5WgiuNle7hx/Hk5HImZAqQKBgQCK7W4iRGpZeklXiNlvtgcWfNlAbyaYZ34MlhDm +vM+q3pEv8i/zY7uJcR3WU81gmnnrRP5hlVuazeTHGKGQTEFQJmCYbKYAUzEdiamV +atElQ2bbuGOlFLmNJjj7406oP+NsPCx1urUyUOv6MjNa2EtCsCywWDKtTEC/Jwx9 +6JZIGwKBgDa4Ephl+lcyGC8D9Tdc4Nme6n/10pIN53ZCplqVSuma+kOOQX7ZXpJb +0csHR9KODcpQuo2yOnUfZ1z4+tsjms2Uv97GgcZ9sO84I/3RGJCvxmzAEiRmJMfH +y38GtEYCnat5VOKdo//eIYtiu9YSXuuS1ENSWUPZ9YInrkUk0j8S +-----END RSA PRIVATE KEY----- diff -Nru mitmproxy-5.1.1/test/mitmproxy/net/data/verificationcerts/generate.py mitmproxy-6.0.2/test/mitmproxy/net/data/verificationcerts/generate.py --- mitmproxy-5.1.1/test/mitmproxy/net/data/verificationcerts/generate.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/net/data/verificationcerts/generate.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,10 +1,10 @@ """ Generate SSL test certificates. """ -import subprocess -import shlex import os +import shlex import shutil +import subprocess import textwrap ROOT_CA = "trusted-root" @@ -28,7 +28,7 @@ authorityKeyIdentifier=keyid,issuer basicConstraints=CA:FALSE keyUsage = digitalSignature, keyEncipherment - subjectAltName = {subject} + subjectAltName = DNS:{subject} """)) do(f"openssl x509 -req -in {cert}.csr " f"-CA {ROOT_CA}.crt " @@ -46,7 +46,8 @@ genrsa(cert) do(f"openssl req -new -nodes -batch " f"-key {cert}.key " - f"-addext \"subjectAltName = {subject}\" " + f"-subj /CN={subject}/O=mitmproxy " + f"-addext \"subjectAltName = DNS:{subject}\" " f"-out {cert}.csr" ) sign(cert, subject) @@ -61,10 +62,10 @@ "-out trusted-root.crt" ) h = do("openssl x509 -hash -noout -in trusted-root.crt").decode("ascii").strip() -shutil.copyfile("trusted-root.crt", "{}.0".format(h)) +shutil.copyfile("trusted-root.crt", f"{h}.0") # create trusted leaf cert. -mkcert("trusted-leaf", f'DNS:{SUBJECT}') +mkcert("trusted-leaf", SUBJECT) # create self-signed cert genrsa("self-signed") @@ -73,4 +74,11 @@ f'-addext "subjectAltName = DNS:{SUBJECT}" ' "-days 7300 " "-out self-signed.crt" - ) \ No newline at end of file + ) + +for x in ["self-signed", "trusted-leaf", "trusted-root"]: + with open(f"{x}.crt") as crt, open(f"{x}.key") as key, open(f"{x}.pem", "w") as pem: + pem.write(crt.read()) + pem.write(key.read()) + +shutil.copyfile("trusted-leaf.pem", "example.mitmproxy.org.pem") diff -Nru mitmproxy-5.1.1/test/mitmproxy/net/data/verificationcerts/self-signed.crt mitmproxy-6.0.2/test/mitmproxy/net/data/verificationcerts/self-signed.crt --- mitmproxy-5.1.1/test/mitmproxy/net/data/verificationcerts/self-signed.crt 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/net/data/verificationcerts/self-signed.crt 2020-12-15 16:41:27.000000000 +0000 @@ -1,22 +1,22 @@ -----BEGIN CERTIFICATE----- -MIIDjTCCAnWgAwIBAgIUb0mIVHKB+bu2PGObggokU3Re7xMwDQYJKoZIhvcNAQEL +MIIDjTCCAnWgAwIBAgIUK0t9k7btz5P97e+UgXtU8L2fhkUwDQYJKoZIhvcNAQEL BQAwRTELMAkGA1UEBhMCQVUxEzARBgNVBAgMClNvbWUtU3RhdGUxITAfBgNVBAoM -GEludGVybmV0IFdpZGdpdHMgUHR5IEx0ZDAeFw0xOTExMjMwMDA1MDlaFw0zOTEx -MTgwMDA1MDlaMEUxCzAJBgNVBAYTAkFVMRMwEQYDVQQIDApTb21lLVN0YXRlMSEw +GEludGVybmV0IFdpZGdpdHMgUHR5IEx0ZDAeFw0yMDExMDMwNzAzMjdaFw00MDEw +MjkwNzAzMjdaMEUxCzAJBgNVBAYTAkFVMRMwEQYDVQQIDApTb21lLVN0YXRlMSEw HwYDVQQKDBhJbnRlcm5ldCBXaWRnaXRzIFB0eSBMdGQwggEiMA0GCSqGSIb3DQEB -AQUAA4IBDwAwggEKAoIBAQDF6eqyZD2z83buUF4T+6AY0Zoe925a2AHOhGtHJMLo -9AD7FF1Xi1iksEvxbOI6mreHtvYKzUpfNsA3DkFdSO91HMSkdvWcDcExpW62sNK9 -gQQrpcCx7DogOrFiGSIHI1LIy1y6YEJma3G71SgGbw7g1QF64dTX9+BzVQJsloT2 -H3ZxTi8Fb6APJq6d/Tp67GTM8U82vM+FjLKzfH7RMSEPvSyvWSibw3AZP6owFaxz -DJtXpR7evOZbiZxqXmGOBl5OQPu9GdDA3Fyi9Drp7xa234loqd1a8PYyL8qWV/2G -HM3IJOzG8Y5PUJhL8CkDbx4LJ9LfzeSuBnQPUf2ZNalZAgMBAAGjdTBzMB0GA1Ud -DgQWBBQbkS0mAGPD4XjOv9GKzGMyR9ix3TAfBgNVHSMEGDAWgBQbkS0mAGPD4XjO -v9GKzGMyR9ix3TAPBgNVHRMBAf8EBTADAQH/MCAGA1UdEQQZMBeCFWV4YW1wbGUu -bWl0bXByb3h5Lm9yZzANBgkqhkiG9w0BAQsFAAOCAQEAqZ64xpu3qrTlCc555OoP -GgobUFP3qv07d0/r48cOyYdAdlEHhvmDPlqWTB9e4ZYtWZMlocY9DpCywzKTa7F7 -Ad8BwS0a/No3wVdl1UEkIGYxuD//jbd77Mrpf5URvQco85o/bicn+H0GAOchYt1P -jP1VShqsRv6WiTs5kn1/JwVoafddl1jBlMDmCqDv4loAZJYHzie0CqdjjSeorFfE -8FG8OLwmEnmIW6VnanRH8coH9MBbZ+dRtCavS+Q8s0R77dJM1sCp5/4yKcr5D/PD -+dQN9f+iugLxDdBQjiRyadWX9/l4n/h8ezabZ4cNsiWbRXrp5VS0nGNmAK3XvPAu -ow== +AQUAA4IBDwAwggEKAoIBAQDDUhk2OdZk5TyMmqUdUrjk8s6mv7LxTSSznY5PqkOF +MFYYWSeER98cB7L+RLBKizN1vt7X1UXRIuPjBevfpfsIqQl8WHMesU6ODXpyI2jQ +ZGJzzYrhW+ULSpElVdqRNWE39Ob/M0v3QYktMEy2LjuOYO6nnYjyPf389Jvyf/oh +Yx80yq3p+pINh94BODN4rSPEb5ISkKb6Kp4B3MsTdmi1kky1ncJOJB9nl4lqYdIn +ylRGVkN+FrwnJL4PN54hkS8DZAZaANGKBLifbSnvKSvBTFpRsuHZFXaf6RXL4LCB +2yAV8TXjQcjkwndZTv9N+jFwpQkXmAHfOaSHAP6zbaLlAgMBAAGjdTBzMB0GA1Ud +DgQWBBRABTSt0uxVpXTkRpgn8Foqei4p2jAfBgNVHSMEGDAWgBRABTSt0uxVpXTk +Rpgn8Foqei4p2jAPBgNVHRMBAf8EBTADAQH/MCAGA1UdEQQZMBeCFWV4YW1wbGUu +bWl0bXByb3h5Lm9yZzANBgkqhkiG9w0BAQsFAAOCAQEAAX8CP7pbdO8flMvCeqFK +S/yBKJxlqR0ycLtXAkvx+xxu8pgTikKnagm6Jc/q9utz2SQ2wPjakmucAOQW14XL +LPPRyX9hpsQcv7J75Q3o01UiukEZ26flMTcABL6U++IsXStdGgvtihGiL3lOBne3 +cApGH7C36+OI0gak3231m6TqDbx/KczNspFTjtSBllEQsnhE6DI1XB54J9e29xd1 +WNwjFcz55iEwu9E6wEBHB3YdUGorHsc67MyE3MbO1PgmQvq4P3pwi8UhHJwnnNjF +HkEXU0xICwHGV8pn/IXs30Z4nAw+N5T0ltW29Ovo+uhVeVYiS2G2BaTP7EtSQivV +LA== -----END CERTIFICATE----- diff -Nru mitmproxy-5.1.1/test/mitmproxy/net/data/verificationcerts/self-signed.key mitmproxy-6.0.2/test/mitmproxy/net/data/verificationcerts/self-signed.key --- mitmproxy-5.1.1/test/mitmproxy/net/data/verificationcerts/self-signed.key 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/net/data/verificationcerts/self-signed.key 2020-12-15 16:41:27.000000000 +0000 @@ -1,27 +1,27 @@ -----BEGIN RSA PRIVATE KEY----- -MIIEpAIBAAKCAQEAxenqsmQ9s/N27lBeE/ugGNGaHvduWtgBzoRrRyTC6PQA+xRd -V4tYpLBL8WziOpq3h7b2Cs1KXzbANw5BXUjvdRzEpHb1nA3BMaVutrDSvYEEK6XA -sew6IDqxYhkiByNSyMtcumBCZmtxu9UoBm8O4NUBeuHU1/fgc1UCbJaE9h92cU4v -BW+gDyaunf06euxkzPFPNrzPhYyys3x+0TEhD70sr1kom8NwGT+qMBWscwybV6Ue -3rzmW4mcal5hjgZeTkD7vRnQwNxcovQ66e8Wtt+JaKndWvD2Mi/Kllf9hhzNyCTs -xvGOT1CYS/ApA28eCyfS383krgZ0D1H9mTWpWQIDAQABAoIBAGl4H8+TZeJ5E18q -ywfhJ08ym+x2tYOJ62SP4s+WApy8M62aC6g0pTeWj9IH0YOjobycPwBAqKqW9dYh -Lao1zQ5fF1gB4R+ZoOQBIkAPeS7uCzfrbAYlOlCklpUNibm+FEbXQQI9fAUyqviL -Pno3QvmD6fb/VDsHaMBthA40JIU4C3yUUcSooKKUC3XKmbRDg5/stKbVdi6co5AQ -OMNFj4smts3rXtk5tKMBep4AgKKdClf4IuhpWEqZDxN6e+hZA7g1VOmOIE+hKEEO -ODBkjHNgHIFbsr8GTlZ+GSkHBxMNVdHxntJTU5a2jPiTNTTnhqu2hcVdkyDwLb9x -TGUCdyUCgYEA/ZPwI2KaT4Lj8b+6DCV4v3nZF8nLCtt1LiXxJWexdnr+3q/ZfNjN -5sKVl9rIynz2zoJDVr1MI8yjO6OmV7GBW2wgJsSFwg0vnV/bs75KqPZWZXqy/mrp -HnXd82XUweyyOpmBg4cCjmcnMTrdvCHvfgx7JafnUDqeXNwriWeC5Y8CgYEAx83d -qHgAT2rlnCQYRmCOGb5fuyQ+pSM4p8h1AjueLrTNiWcqX57ct9eVWjymD+AgevFZ -hZH9m6TI4nCb/5ukcvjQb4g1kyY3l7rZiiJ3kf/c22kVsVUagJGeiW6Ypnox4r1o -oFXEwAw1qSGIqOIjYVB2DqvjUl3+UjWCl9SOnpcCgYEA0TSdWUQ/VTwCvW9VmjHM -FgT8I5Ebj+CRI7qv4hFTqxE8dxKTl1nzPd/ptTgOkmhY4vU7gzN3vs1VGp4gXZcX -xwpE2Fcol3lzgB4Wz4s+Y3mgu+ZoCFjB7ZyGugmYZ0nVnV0KKi5X4I6gGhCb4VwK -D29SpjWJNHq4LpqC3MDmkGcCgYAxnKOKXmmtTpy+3ZONfhIqwEOjA0fu10UNHFA5 -grYvYMOcd5pk7dxeZdB2/JI7ZOqLvHv/F5YCXLNozo9ds7bsuW2AFDFBXX72VPYJ -P6+y9/ZOINS7GKeg/wd/lo+e3r6eT2u4TDOzgBSe722wiZ5BXqpB0Fp8rEwm+5R2 -wNe89wKBgQC59SOa4EW3gfjLRaH6hEPcMEzjr0Dt5ilBgntYbI+DsCePwEX28hAK -fd7A6XoCqjnwLpAmh0cmNRsYjtPCzg6TXUdrhfnR/3r7DHzvsxaC06BfkPZJ0+Fu -rP3el/oKSPaDsZKjW6RA5oqlIF82o45MNl2oCG82LgVtu25e3HbJSA== +MIIEpAIBAAKCAQEAw1IZNjnWZOU8jJqlHVK45PLOpr+y8U0ks52OT6pDhTBWGFkn +hEffHAey/kSwSoszdb7e19VF0SLj4wXr36X7CKkJfFhzHrFOjg16ciNo0GRic82K +4VvlC0qRJVXakTVhN/Tm/zNL90GJLTBMti47jmDup52I8j39/PSb8n/6IWMfNMqt +6fqSDYfeATgzeK0jxG+SEpCm+iqeAdzLE3ZotZJMtZ3CTiQfZ5eJamHSJ8pURlZD +fha8JyS+DzeeIZEvA2QGWgDRigS4n20p7ykrwUxaUbLh2RV2n+kVy+CwgdsgFfE1 +40HI5MJ3WU7/TfoxcKUJF5gB3zmkhwD+s22i5QIDAQABAoIBADK7Gi1JbHQcTlO+ +vvAU0k00+5O36sRd4xB79cCfWpY3bcU5Mthayoo/PbBpKtjRuvX0M3EfxdiCFWqb +2R3nwIIJVZtkZdIs/1hKC+mlZM3rpN6rHk1WTvFV1sk5uWFJ2gxsoarbKfn4naaN +Cv+ulm1uo84JTs6MZ3HSHscnklIlNnDG3piC8JnzXVgS7kzbJPXTccZJI1Mwgyho +D1HoWatIep5XAp3OZePlApfRpKn/+2Hg6USkbcHYzX/XpA3b3/YEBqsXVUFxAs5m +nChdLpKUCIjcARInAXoULh0AQHGcH9dfJk1QF1Lw6nCoY12P1sZfdGU5trcslA1O +W28syoECgYEA6GjaqFYffceJHlODukHgpLNfa1EUy49ZWjmdmXTYXJpufzjD4n4H +sUkEnJ00ssaCVdd0XClbN/l/QvIAz3W8ZXXQWtZJHV2wAIyjWpqiOwaiXQFZIZ67 +kxGsLVjkuiST33CX6yl8IKAD+8B67uZhmWTnYBmxGnIZODqUatpqW3ECgYEA1yV+ +CGoNp6VQqej0t0y8C8pXiMGe2DGj5p1WFn00cn/fC1E2Yc0HG+E62OB7tgUFI180 +/nArOQHnTZCzAQAc44M63gesRMeu+0cxuTUqOUCn1lGD+4K/3IA59Rgzfvvnq6xu +tn2jidfqaDPfKavqtCqRbs8wJYaGEhgcb8pUvLUCgYEA4pjpKFvgFGip/lF7C+0T +NEJXdHD3j4lSmy+1w1szYQaJWa1k/73VjjsdLf3w1aXKihuprfn8oFS4ifMeayfl +6h62aPqpCuK/qal10+8U4ewT/g5Ecw0q4bfHYedcC0mCi8ZhuL0X809Q0vLWaXti +CYdiOEaUcK5yfGpRLuWJ8WECgYAWWXa2OQ4iFDJE9EY3pGkEcIiXVEXD/6QfGMkQ +nQENw+rPqigUENBkPQl37hnr1qmp+wHuTIiw61mz3Qw7Vl+p4sACwJlMq9GpmMO5 +kaRJPkYxJVaokfSMW2Wp6FGxJ0nxs3/sxTBv6VYYbQsJsSo4fROOh0dhHpBe4NJT +aplS4QKBgQCaMUN88HNaVr0+DPQKeFoflqwCCYsdK43QTWEirrFDWDliJ0k81DZJ +k7N3oR5UujsTMgfYDJ6Cp3enZVYyuoFnOlVT8v4XzvzAGTehyo4rMiE2TbMir7Xn +agPTZYSuQc4Iy7BPwh/PMlX6bDOTeq/ftyWrDuF9+pC5w3v4SkuMew== -----END RSA PRIVATE KEY----- diff -Nru mitmproxy-5.1.1/test/mitmproxy/net/data/verificationcerts/self-signed.pem mitmproxy-6.0.2/test/mitmproxy/net/data/verificationcerts/self-signed.pem --- mitmproxy-5.1.1/test/mitmproxy/net/data/verificationcerts/self-signed.pem 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/net/data/verificationcerts/self-signed.pem 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,49 @@ +-----BEGIN CERTIFICATE----- +MIIDjTCCAnWgAwIBAgIUK0t9k7btz5P97e+UgXtU8L2fhkUwDQYJKoZIhvcNAQEL +BQAwRTELMAkGA1UEBhMCQVUxEzARBgNVBAgMClNvbWUtU3RhdGUxITAfBgNVBAoM +GEludGVybmV0IFdpZGdpdHMgUHR5IEx0ZDAeFw0yMDExMDMwNzAzMjdaFw00MDEw +MjkwNzAzMjdaMEUxCzAJBgNVBAYTAkFVMRMwEQYDVQQIDApTb21lLVN0YXRlMSEw +HwYDVQQKDBhJbnRlcm5ldCBXaWRnaXRzIFB0eSBMdGQwggEiMA0GCSqGSIb3DQEB +AQUAA4IBDwAwggEKAoIBAQDDUhk2OdZk5TyMmqUdUrjk8s6mv7LxTSSznY5PqkOF +MFYYWSeER98cB7L+RLBKizN1vt7X1UXRIuPjBevfpfsIqQl8WHMesU6ODXpyI2jQ +ZGJzzYrhW+ULSpElVdqRNWE39Ob/M0v3QYktMEy2LjuOYO6nnYjyPf389Jvyf/oh +Yx80yq3p+pINh94BODN4rSPEb5ISkKb6Kp4B3MsTdmi1kky1ncJOJB9nl4lqYdIn +ylRGVkN+FrwnJL4PN54hkS8DZAZaANGKBLifbSnvKSvBTFpRsuHZFXaf6RXL4LCB +2yAV8TXjQcjkwndZTv9N+jFwpQkXmAHfOaSHAP6zbaLlAgMBAAGjdTBzMB0GA1Ud +DgQWBBRABTSt0uxVpXTkRpgn8Foqei4p2jAfBgNVHSMEGDAWgBRABTSt0uxVpXTk +Rpgn8Foqei4p2jAPBgNVHRMBAf8EBTADAQH/MCAGA1UdEQQZMBeCFWV4YW1wbGUu +bWl0bXByb3h5Lm9yZzANBgkqhkiG9w0BAQsFAAOCAQEAAX8CP7pbdO8flMvCeqFK +S/yBKJxlqR0ycLtXAkvx+xxu8pgTikKnagm6Jc/q9utz2SQ2wPjakmucAOQW14XL +LPPRyX9hpsQcv7J75Q3o01UiukEZ26flMTcABL6U++IsXStdGgvtihGiL3lOBne3 +cApGH7C36+OI0gak3231m6TqDbx/KczNspFTjtSBllEQsnhE6DI1XB54J9e29xd1 +WNwjFcz55iEwu9E6wEBHB3YdUGorHsc67MyE3MbO1PgmQvq4P3pwi8UhHJwnnNjF +HkEXU0xICwHGV8pn/IXs30Z4nAw+N5T0ltW29Ovo+uhVeVYiS2G2BaTP7EtSQivV +LA== +-----END CERTIFICATE----- +-----BEGIN RSA PRIVATE KEY----- +MIIEpAIBAAKCAQEAw1IZNjnWZOU8jJqlHVK45PLOpr+y8U0ks52OT6pDhTBWGFkn +hEffHAey/kSwSoszdb7e19VF0SLj4wXr36X7CKkJfFhzHrFOjg16ciNo0GRic82K +4VvlC0qRJVXakTVhN/Tm/zNL90GJLTBMti47jmDup52I8j39/PSb8n/6IWMfNMqt +6fqSDYfeATgzeK0jxG+SEpCm+iqeAdzLE3ZotZJMtZ3CTiQfZ5eJamHSJ8pURlZD +fha8JyS+DzeeIZEvA2QGWgDRigS4n20p7ykrwUxaUbLh2RV2n+kVy+CwgdsgFfE1 +40HI5MJ3WU7/TfoxcKUJF5gB3zmkhwD+s22i5QIDAQABAoIBADK7Gi1JbHQcTlO+ +vvAU0k00+5O36sRd4xB79cCfWpY3bcU5Mthayoo/PbBpKtjRuvX0M3EfxdiCFWqb +2R3nwIIJVZtkZdIs/1hKC+mlZM3rpN6rHk1WTvFV1sk5uWFJ2gxsoarbKfn4naaN +Cv+ulm1uo84JTs6MZ3HSHscnklIlNnDG3piC8JnzXVgS7kzbJPXTccZJI1Mwgyho +D1HoWatIep5XAp3OZePlApfRpKn/+2Hg6USkbcHYzX/XpA3b3/YEBqsXVUFxAs5m +nChdLpKUCIjcARInAXoULh0AQHGcH9dfJk1QF1Lw6nCoY12P1sZfdGU5trcslA1O +W28syoECgYEA6GjaqFYffceJHlODukHgpLNfa1EUy49ZWjmdmXTYXJpufzjD4n4H +sUkEnJ00ssaCVdd0XClbN/l/QvIAz3W8ZXXQWtZJHV2wAIyjWpqiOwaiXQFZIZ67 +kxGsLVjkuiST33CX6yl8IKAD+8B67uZhmWTnYBmxGnIZODqUatpqW3ECgYEA1yV+ +CGoNp6VQqej0t0y8C8pXiMGe2DGj5p1WFn00cn/fC1E2Yc0HG+E62OB7tgUFI180 +/nArOQHnTZCzAQAc44M63gesRMeu+0cxuTUqOUCn1lGD+4K/3IA59Rgzfvvnq6xu +tn2jidfqaDPfKavqtCqRbs8wJYaGEhgcb8pUvLUCgYEA4pjpKFvgFGip/lF7C+0T +NEJXdHD3j4lSmy+1w1szYQaJWa1k/73VjjsdLf3w1aXKihuprfn8oFS4ifMeayfl +6h62aPqpCuK/qal10+8U4ewT/g5Ecw0q4bfHYedcC0mCi8ZhuL0X809Q0vLWaXti +CYdiOEaUcK5yfGpRLuWJ8WECgYAWWXa2OQ4iFDJE9EY3pGkEcIiXVEXD/6QfGMkQ +nQENw+rPqigUENBkPQl37hnr1qmp+wHuTIiw61mz3Qw7Vl+p4sACwJlMq9GpmMO5 +kaRJPkYxJVaokfSMW2Wp6FGxJ0nxs3/sxTBv6VYYbQsJsSo4fROOh0dhHpBe4NJT +aplS4QKBgQCaMUN88HNaVr0+DPQKeFoflqwCCYsdK43QTWEirrFDWDliJ0k81DZJ +k7N3oR5UujsTMgfYDJ6Cp3enZVYyuoFnOlVT8v4XzvzAGTehyo4rMiE2TbMir7Xn +agPTZYSuQc4Iy7BPwh/PMlX6bDOTeq/ftyWrDuF9+pC5w3v4SkuMew== +-----END RSA PRIVATE KEY----- diff -Nru mitmproxy-5.1.1/test/mitmproxy/net/data/verificationcerts/trusted-leaf.crt mitmproxy-6.0.2/test/mitmproxy/net/data/verificationcerts/trusted-leaf.crt --- mitmproxy-5.1.1/test/mitmproxy/net/data/verificationcerts/trusted-leaf.crt 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/net/data/verificationcerts/trusted-leaf.crt 2020-12-15 16:41:27.000000000 +0000 @@ -1,21 +1,21 @@ -----BEGIN CERTIFICATE----- -MIIDajCCAlKgAwIBAgIJAKPoP2hXKVv4MA0GCSqGSIb3DQEBCwUAMEUxCzAJBgNV -BAYTAkFVMRMwEQYDVQQIDApTb21lLVN0YXRlMSEwHwYDVQQKDBhJbnRlcm5ldCBX -aWRnaXRzIFB0eSBMdGQwHhcNMTkxMTIzMDAwNTA5WhcNMzkxMTE4MDAwNTA5WjBF -MQswCQYDVQQGEwJBVTETMBEGA1UECAwKU29tZS1TdGF0ZTEhMB8GA1UECgwYSW50 -ZXJuZXQgV2lkZ2l0cyBQdHkgTHRkMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIB -CgKCAQEA4BZUGKkf52583Kps2SinCzRtGFaGIqtP1YfjaN1H21c08hEERRWf0GqW -PIbZz0NRyqOVM0O+4A9cxKrdqDpALBNXdgB62Ven1NNE31ZjsbFgwbORPTiOQo+6 -Xsu+2KHQdqAGOXSqOdNSoZV/HqAKa41iweg6lUf3yrO/hpfnlXTFbuM/oYFARrr9 -YWY7qH5BEvcZBf4Sg3cWGjdbzYg8S6hnFBBkHPDMbS3xra7H8uru8aG9L70aiq0C -3WsGf1/Qctz3cccB6BdnbiXji0QlP0miT+b52hEzDvNjdXvoObgbOZsUUcLrbz8q -LqaQPj6TA2PYpKcxEdJLDcbN++tglQIDAQABo10wWzAfBgNVHSMEGDAWgBSTmtfF -P/zQDiCbte3AG3vchjSekjAJBgNVHRMEAjAAMAsGA1UdDwQEAwIFoDAgBgNVHREE -GTAXghVleGFtcGxlLm1pdG1wcm94eS5vcmcwDQYJKoZIhvcNAQELBQADggEBAMoK -8+KTYF10dbQ8Hl3Fq4Iux7eutAPuKUXUz9dnCfDRhZukdl+gahRoNvkyMBdXOvvx -R9Z2+Guo8NOYKgT1mJtS/c8IxTkjZj9doujJOVD2wTowSfoaT9z+/EJa+6Fp9+xd -YVsO3E/2Vxai8PCNx8JTXr2axcnBDvpHPRXF21hOI8N94SPAcmLTZsdsTELjrGGa -/BA0y+pCEwW6cY9mMHVAAvRoMoqfocBVI7nrYBaQfFoKuwxscxO679eEv+lbSjul -z/VNdWfqrDhFFSzwRSVchapQ9q1EeTzv++wZRwI5bT0Ib6DFTJB3J5+9RihlFYfU -GI17CI0D//DsFic7QJ0= +MIIDZDCCAkygAwIBAgIUSWoLwl7Rc//TKPwKwv25Vhb6uFAwDQYJKoZIhvcNAQEL +BQAwRTELMAkGA1UEBhMCQVUxEzARBgNVBAgMClNvbWUtU3RhdGUxITAfBgNVBAoM +GEludGVybmV0IFdpZGdpdHMgUHR5IEx0ZDAeFw0yMDExMDMwNzAzMjZaFw00MDEw +MjkwNzAzMjZaMDQxHjAcBgNVBAMMFWV4YW1wbGUubWl0bXByb3h5Lm9yZzESMBAG +A1UECgwJbWl0bXByb3h5MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA +qcEdB7TZD7DxJlJH/ZTyrovP0din2vLCRZRZ07ix48arITGWj8wFJ/XpZsFmAWlH +2LriXGC8BB54EJt8Epix5tcdKvCZ/7vTgaBQ+DQVdCNiscaKwvDbigy037OY97sT +6Ou9xYjjgg0cb+WTkQGMOTMbCW2e5iuSvyMPdZ2+2RzbTcNV1/6O/5cQ7TOW2E83 +xeaN4stBMfRmsQRoHXBri0tqtQ3bHd3V4H9iai7HFBOcB/nsJUwLliZpz3j/dUD0 +yZAgdgB9XBlGvs/e746bGctyvl2QZu2JeACEVPmVrAUhqe2TVMVLDT17uzNeH1x5 +UXxdB4cexTOm4amvWSwvTQIDAQABo10wWzAfBgNVHSMEGDAWgBTkDj66Wmaf9YP0 +YMqGKG51PejxWzAJBgNVHRMEAjAAMAsGA1UdDwQEAwIFoDAgBgNVHREEGTAXghVl +eGFtcGxlLm1pdG1wcm94eS5vcmcwDQYJKoZIhvcNAQELBQADggEBABwKrf+Gy1I5 +BnfhAqbDUYj8OVcjbB5esbvVYAADNNNQ10RREyz5OTTo2dSKy2hIVQeGxaBy5W6d +cNk0CoLzOgpgkwQPQ351+Y0C5yQztXIg/JM5aSeFuAbQg+NtzOQISrcDB390Xr2f +7YhyYL0XLX4NCP7ek7+lAClrf+lL9MysHTTP0pmFWLPP6AIR8JIsLpNpTDV10c8p +QcnJ8+5q6oLHNKN+M/HRcdKzj/XyJ538UzSOlmU+izhNT1j7mbszS0tc9yh0dmpP +9uuBY0wcD/NAHBpSX9v9ZJcOuWntsAxTbMXj1eWNRyt0QJLURlkqeVn6ZrE1/BDS +BqDdOKYJbkw= -----END CERTIFICATE----- diff -Nru mitmproxy-5.1.1/test/mitmproxy/net/data/verificationcerts/trusted-leaf.key mitmproxy-6.0.2/test/mitmproxy/net/data/verificationcerts/trusted-leaf.key --- mitmproxy-5.1.1/test/mitmproxy/net/data/verificationcerts/trusted-leaf.key 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/net/data/verificationcerts/trusted-leaf.key 2020-12-15 16:41:27.000000000 +0000 @@ -1,27 +1,27 @@ -----BEGIN RSA PRIVATE KEY----- -MIIEowIBAAKCAQEA4BZUGKkf52583Kps2SinCzRtGFaGIqtP1YfjaN1H21c08hEE -RRWf0GqWPIbZz0NRyqOVM0O+4A9cxKrdqDpALBNXdgB62Ven1NNE31ZjsbFgwbOR -PTiOQo+6Xsu+2KHQdqAGOXSqOdNSoZV/HqAKa41iweg6lUf3yrO/hpfnlXTFbuM/ -oYFARrr9YWY7qH5BEvcZBf4Sg3cWGjdbzYg8S6hnFBBkHPDMbS3xra7H8uru8aG9 -L70aiq0C3WsGf1/Qctz3cccB6BdnbiXji0QlP0miT+b52hEzDvNjdXvoObgbOZsU -UcLrbz8qLqaQPj6TA2PYpKcxEdJLDcbN++tglQIDAQABAoIBAG5V8DB4TdI5T9ej -Ppcqch2NQc5DBCbb7SI5l5qRogj5BoPOJykQ/bC0WqcQyvxHrGU3aIZma/yM8+OO -Mjfb/q71ExJyKAsOIwAiyn2hXtMmgHq/vNrFFx7lACIe9ihafHd8UbRGom54g+41 -2vKsYJUWd7L8cqQAXJz9JmfSMeAfRBVVLyOpENSVT8gM/LgPokmZbg0wRJTA5oa3 -10f4Ax1HqXO3cQgKKnrvIU72gB7MQ9Y2G+P2eAmiAlzGr41N5sv7uFz0JrFHhqtC -qc7QrGVVNtxanu0s/LWgfQ+LrKldTysX16j/j6J0tg7Pfuq9cMFPklU98dv2LF5J -jMB8eQECgYEA84+Yy/IGVjUPVb3HjmCmWUoxt4Aqop/12/4+1cBNS6h58iLJpcWc -FVh8m3UbuE7shimEg0fZ1rPPkqepKlDw8/sKXDni/khBASOAh2v1RhvOxNGjxDfn -ZvcK+BG99kU8ZWQHpKmn8OGfYswL2Fvo67L9XFg/wDnk1QpJx7E1GVUCgYEA64gg -G6qnsah1NKSW64BPVKi/Gby83l4oa1amjHqq/Unq6iuGB4/QhK4BzW0aOGI7RMTE -SPh5NF5ZhfT3LWD/EnaMm6QxMTSsL9TDIg36WTVZTkbODFOJczg19EdTs1pa6OXW -0NM6psLu6Nf/QmHJzyYxSZ83uqPOM305xgooKkECgYEAnA0XMySglr9sUd1EbJ7U -NkVpUU8XAhdHKWrey4lofN83MsLDPCk+dha5z8jat94pgVQ8iPiSRBP1HNu7cVdm -6ouf+bNFEvMsYxRiF2I+RmsuscA4E1JWOwxxxLtpYM6/gZ7znrbs2VNWEbD2retF -cy69UltgjUMKsMzktMN/Z/kCgYAbzV285lAVMIVlSWhnNCYpICIur5C7zvGGehv+ -yRwV+fu42JphmiBLCR89WHuX3ECSxYdF9c6Y1+pJXbkvqhtx2nyOgrsry8PngX3n -Ly82CI4aJ1F7MwEukJwN0b2XljrU8wyAae6qcKgy5AxFkbV4tlFrF1hEt8FHYqjH -L7u+AQKBgBnN2vjOVagtlZjJadg5ueR7uKnfk7/TGMtZjNf89C1UZH3TqWnnj6p6 -rLLe4tnA3EebFtGeVWlmnw5k166HRBEd7KuebOCq3hkNlf71aHwhDJ0bIdDwo1g+ -FTpG2vqkqpuEb+2FzDjZT5EdPnj8tUGthREDXpXqsdjFb6+2Enjx +MIIEowIBAAKCAQEAqcEdB7TZD7DxJlJH/ZTyrovP0din2vLCRZRZ07ix48arITGW +j8wFJ/XpZsFmAWlH2LriXGC8BB54EJt8Epix5tcdKvCZ/7vTgaBQ+DQVdCNiscaK +wvDbigy037OY97sT6Ou9xYjjgg0cb+WTkQGMOTMbCW2e5iuSvyMPdZ2+2RzbTcNV +1/6O/5cQ7TOW2E83xeaN4stBMfRmsQRoHXBri0tqtQ3bHd3V4H9iai7HFBOcB/ns +JUwLliZpz3j/dUD0yZAgdgB9XBlGvs/e746bGctyvl2QZu2JeACEVPmVrAUhqe2T +VMVLDT17uzNeH1x5UXxdB4cexTOm4amvWSwvTQIDAQABAoIBAEb3arF8E3qR2F7S +6zHCASqjXIA3+QR5lGoOOPdgMU4uEgDQgEchXc506dyBYamZX+XlSxifgDqgmkUn +G1mS6Fy+9XysFVVqKmP4p6D79TQWTv5PKFeS5dTytvMGXB7E4O/xDeb08Ve/L3JA +Ic7vPLX0/YqVf2ZuNO0fNSlQhyawUVlSS6yq/B1GUo44PX/PXrWYZd66mc29g+KF +7D0nnbloLY+5QmOEVYbf99RHJ64Cwxzwxi4ic9AxLaNMfkrydjbGQ0VhJq14WEux +/TJLvinzIDyslnx4aYstno3ryoxLtR/9ALysrdQ8vVMaVCdbf0aMu832R2dI4s9K +ixQlgmkCgYEA0Ho0a4qzwk9oLWYk1x16kOCqmfuenlEz3Tv8uBB/jdz7UM63sS3h +OUQYKhhtzOGf6CDA3CE11ie6SX7zHrowPLJBSpacJZJcrHPGZxdBZ/As/QpGnZll +1eTMlVRXk06k/fEYplU1dS5IDbsvUIaaF9CF9Qd3MkDRglhbM19Iy+sCgYEA0HM1 +d7OBsjSi09zFDE+Joq6Gih8LqqFLtMsIHEKAJ0gJgmt6MTfjtdPo55ZKVLCR2pbP +yRHagK4LRUFY9Pl5eGlEUF9cKEKKtZOURUFVz3yBMYPBje/B1o/UCCj+nr2reWgx +qj/jwZrph6QKCkEsnQXi98tE7XSgihUh6UEQO6cCgYASVe0uWDCfMmSzOXyb/te8 +zkWy7VJyEipBlvkPJ0RQsdLYtJWrW6Gna7nEWgmuL1nlDJxpv/IAN9ZGiIfReAau +D+92I/DvzQOhlz0n6/+wqIsMZk73pXozacAkkhpxtkUEoKPOXUgqWju0GXZ72prK +5WgiuNle7hx/Hk5HImZAqQKBgQCK7W4iRGpZeklXiNlvtgcWfNlAbyaYZ34MlhDm +vM+q3pEv8i/zY7uJcR3WU81gmnnrRP5hlVuazeTHGKGQTEFQJmCYbKYAUzEdiamV +atElQ2bbuGOlFLmNJjj7406oP+NsPCx1urUyUOv6MjNa2EtCsCywWDKtTEC/Jwx9 +6JZIGwKBgDa4Ephl+lcyGC8D9Tdc4Nme6n/10pIN53ZCplqVSuma+kOOQX7ZXpJb +0csHR9KODcpQuo2yOnUfZ1z4+tsjms2Uv97GgcZ9sO84I/3RGJCvxmzAEiRmJMfH +y38GtEYCnat5VOKdo//eIYtiu9YSXuuS1ENSWUPZ9YInrkUk0j8S -----END RSA PRIVATE KEY----- diff -Nru mitmproxy-5.1.1/test/mitmproxy/net/data/verificationcerts/trusted-leaf.pem mitmproxy-6.0.2/test/mitmproxy/net/data/verificationcerts/trusted-leaf.pem --- mitmproxy-5.1.1/test/mitmproxy/net/data/verificationcerts/trusted-leaf.pem 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/net/data/verificationcerts/trusted-leaf.pem 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,48 @@ +-----BEGIN CERTIFICATE----- +MIIDZDCCAkygAwIBAgIUSWoLwl7Rc//TKPwKwv25Vhb6uFAwDQYJKoZIhvcNAQEL +BQAwRTELMAkGA1UEBhMCQVUxEzARBgNVBAgMClNvbWUtU3RhdGUxITAfBgNVBAoM +GEludGVybmV0IFdpZGdpdHMgUHR5IEx0ZDAeFw0yMDExMDMwNzAzMjZaFw00MDEw +MjkwNzAzMjZaMDQxHjAcBgNVBAMMFWV4YW1wbGUubWl0bXByb3h5Lm9yZzESMBAG +A1UECgwJbWl0bXByb3h5MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA +qcEdB7TZD7DxJlJH/ZTyrovP0din2vLCRZRZ07ix48arITGWj8wFJ/XpZsFmAWlH +2LriXGC8BB54EJt8Epix5tcdKvCZ/7vTgaBQ+DQVdCNiscaKwvDbigy037OY97sT +6Ou9xYjjgg0cb+WTkQGMOTMbCW2e5iuSvyMPdZ2+2RzbTcNV1/6O/5cQ7TOW2E83 +xeaN4stBMfRmsQRoHXBri0tqtQ3bHd3V4H9iai7HFBOcB/nsJUwLliZpz3j/dUD0 +yZAgdgB9XBlGvs/e746bGctyvl2QZu2JeACEVPmVrAUhqe2TVMVLDT17uzNeH1x5 +UXxdB4cexTOm4amvWSwvTQIDAQABo10wWzAfBgNVHSMEGDAWgBTkDj66Wmaf9YP0 +YMqGKG51PejxWzAJBgNVHRMEAjAAMAsGA1UdDwQEAwIFoDAgBgNVHREEGTAXghVl +eGFtcGxlLm1pdG1wcm94eS5vcmcwDQYJKoZIhvcNAQELBQADggEBABwKrf+Gy1I5 +BnfhAqbDUYj8OVcjbB5esbvVYAADNNNQ10RREyz5OTTo2dSKy2hIVQeGxaBy5W6d +cNk0CoLzOgpgkwQPQ351+Y0C5yQztXIg/JM5aSeFuAbQg+NtzOQISrcDB390Xr2f +7YhyYL0XLX4NCP7ek7+lAClrf+lL9MysHTTP0pmFWLPP6AIR8JIsLpNpTDV10c8p +QcnJ8+5q6oLHNKN+M/HRcdKzj/XyJ538UzSOlmU+izhNT1j7mbszS0tc9yh0dmpP +9uuBY0wcD/NAHBpSX9v9ZJcOuWntsAxTbMXj1eWNRyt0QJLURlkqeVn6ZrE1/BDS +BqDdOKYJbkw= +-----END CERTIFICATE----- +-----BEGIN RSA PRIVATE KEY----- +MIIEowIBAAKCAQEAqcEdB7TZD7DxJlJH/ZTyrovP0din2vLCRZRZ07ix48arITGW +j8wFJ/XpZsFmAWlH2LriXGC8BB54EJt8Epix5tcdKvCZ/7vTgaBQ+DQVdCNiscaK +wvDbigy037OY97sT6Ou9xYjjgg0cb+WTkQGMOTMbCW2e5iuSvyMPdZ2+2RzbTcNV +1/6O/5cQ7TOW2E83xeaN4stBMfRmsQRoHXBri0tqtQ3bHd3V4H9iai7HFBOcB/ns +JUwLliZpz3j/dUD0yZAgdgB9XBlGvs/e746bGctyvl2QZu2JeACEVPmVrAUhqe2T +VMVLDT17uzNeH1x5UXxdB4cexTOm4amvWSwvTQIDAQABAoIBAEb3arF8E3qR2F7S +6zHCASqjXIA3+QR5lGoOOPdgMU4uEgDQgEchXc506dyBYamZX+XlSxifgDqgmkUn +G1mS6Fy+9XysFVVqKmP4p6D79TQWTv5PKFeS5dTytvMGXB7E4O/xDeb08Ve/L3JA +Ic7vPLX0/YqVf2ZuNO0fNSlQhyawUVlSS6yq/B1GUo44PX/PXrWYZd66mc29g+KF +7D0nnbloLY+5QmOEVYbf99RHJ64Cwxzwxi4ic9AxLaNMfkrydjbGQ0VhJq14WEux +/TJLvinzIDyslnx4aYstno3ryoxLtR/9ALysrdQ8vVMaVCdbf0aMu832R2dI4s9K +ixQlgmkCgYEA0Ho0a4qzwk9oLWYk1x16kOCqmfuenlEz3Tv8uBB/jdz7UM63sS3h +OUQYKhhtzOGf6CDA3CE11ie6SX7zHrowPLJBSpacJZJcrHPGZxdBZ/As/QpGnZll +1eTMlVRXk06k/fEYplU1dS5IDbsvUIaaF9CF9Qd3MkDRglhbM19Iy+sCgYEA0HM1 +d7OBsjSi09zFDE+Joq6Gih8LqqFLtMsIHEKAJ0gJgmt6MTfjtdPo55ZKVLCR2pbP +yRHagK4LRUFY9Pl5eGlEUF9cKEKKtZOURUFVz3yBMYPBje/B1o/UCCj+nr2reWgx +qj/jwZrph6QKCkEsnQXi98tE7XSgihUh6UEQO6cCgYASVe0uWDCfMmSzOXyb/te8 +zkWy7VJyEipBlvkPJ0RQsdLYtJWrW6Gna7nEWgmuL1nlDJxpv/IAN9ZGiIfReAau +D+92I/DvzQOhlz0n6/+wqIsMZk73pXozacAkkhpxtkUEoKPOXUgqWju0GXZ72prK +5WgiuNle7hx/Hk5HImZAqQKBgQCK7W4iRGpZeklXiNlvtgcWfNlAbyaYZ34MlhDm +vM+q3pEv8i/zY7uJcR3WU81gmnnrRP5hlVuazeTHGKGQTEFQJmCYbKYAUzEdiamV +atElQ2bbuGOlFLmNJjj7406oP+NsPCx1urUyUOv6MjNa2EtCsCywWDKtTEC/Jwx9 +6JZIGwKBgDa4Ephl+lcyGC8D9Tdc4Nme6n/10pIN53ZCplqVSuma+kOOQX7ZXpJb +0csHR9KODcpQuo2yOnUfZ1z4+tsjms2Uv97GgcZ9sO84I/3RGJCvxmzAEiRmJMfH +y38GtEYCnat5VOKdo//eIYtiu9YSXuuS1ENSWUPZ9YInrkUk0j8S +-----END RSA PRIVATE KEY----- diff -Nru mitmproxy-5.1.1/test/mitmproxy/net/data/verificationcerts/trusted-root.crt mitmproxy-6.0.2/test/mitmproxy/net/data/verificationcerts/trusted-root.crt --- mitmproxy-5.1.1/test/mitmproxy/net/data/verificationcerts/trusted-root.crt 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/net/data/verificationcerts/trusted-root.crt 2020-12-15 16:41:27.000000000 +0000 @@ -1,21 +1,21 @@ -----BEGIN CERTIFICATE----- -MIIDazCCAlOgAwIBAgIUTE2oFYATbkkytGOt0+CIAOPpaeEwDQYJKoZIhvcNAQEL +MIIDazCCAlOgAwIBAgIUB5WLNdtHRvciUwDfySAEHYVSNe0wDQYJKoZIhvcNAQEL BQAwRTELMAkGA1UEBhMCQVUxEzARBgNVBAgMClNvbWUtU3RhdGUxITAfBgNVBAoM -GEludGVybmV0IFdpZGdpdHMgUHR5IEx0ZDAeFw0xOTExMjMwMDA1MDlaFw0zOTEx -MTgwMDA1MDlaMEUxCzAJBgNVBAYTAkFVMRMwEQYDVQQIDApTb21lLVN0YXRlMSEw +GEludGVybmV0IFdpZGdpdHMgUHR5IEx0ZDAeFw0yMDExMDMwNzAzMjZaFw00MDEw +MjkwNzAzMjZaMEUxCzAJBgNVBAYTAkFVMRMwEQYDVQQIDApTb21lLVN0YXRlMSEw HwYDVQQKDBhJbnRlcm5ldCBXaWRnaXRzIFB0eSBMdGQwggEiMA0GCSqGSIb3DQEB -AQUAA4IBDwAwggEKAoIBAQDZ7fqv4iC6738OvOC/ONFmhOVmMNe2SrGtk+7KIv/M -d7MJJKpIygwc99EfL3m6WFjvSQ8CABO9GnLIeoB6RzX5X904dPQzIekMkE27J4D5 -wzYWAmYsu9md//9rjadaA01zHvxtpc30AsToOrRa2Rv7Lwotvcop2bBCXXZMdgvD -BpCuCOl3cQ346LZBmED41q77P7SgnI98gCPuyyMVAFTZE4NnFUloVERTcMMo5Vwv -tXFT41B6FOqtyhZFY89W1SeNKlm+n8zu+/aUF5c9usxwvQzj79pzBFYS9xMZzbxl -BmU91GbLFNU32O+wp0jX6c1MfluGvRjcHQpK+gHRRJdZAgMBAAGjUzBRMB0GA1Ud -DgQWBBSTmtfFP/zQDiCbte3AG3vchjSekjAfBgNVHSMEGDAWgBSTmtfFP/zQDiCb -te3AG3vchjSekjAPBgNVHRMBAf8EBTADAQH/MA0GCSqGSIb3DQEBCwUAA4IBAQB6 -CnGmyJKVdD41Qs8QbyZ5VvrAkSq7DkL4CAIDy0pClOYjx51R16g4/dXPd8nuDjVB -GbRuO/ZHlKeO+RAVbV4s74KeBmQ4uyuZMvuiJGCDtTgu+fbgp1l41vj21bkDnz/X -0aVrwC4clh6O1/AFCY7cYygp22QJ0zGpIb9d0ly6QDMEocHJKasuQ8/0GPvv6wkY -hwNub2ScsVFHPrFzzufZblBk3Ks0YZw+IsCMTY7QtGb1TZhGeX9xAydqNMnmIxNj -XOE2FKx4ISQNFwf/U1LEAruO7PjbbsWx5FHr4oYV9TCLoERZFofwcGGM9o/cPc4P -CH8fGvFYoU30PG+xX0Pc +AQUAA4IBDwAwggEKAoIBAQDVWUg2YdsztWIZpsWdbIB6Aewa1Igt7mNkKMQ1NJ6v +priVlzWhZr0FfSVTid2m3Od2f5A751Q+IXLkNJ8d6Y4qVirAstEM57sCH/Hmt1E3 +cZOAzLQY2xCIrKPmbu6d8eSGO1q+Y8KstCK/V94/QRahjjoHxkLcSAmdg6PAkbhB +7MwOxiJslIbsBY4UCXP0l4kUUIuMi8W2Y4M1VgvLRpUjaKVo1NQ0hLi4XBnU7Bsq +A8/PZDuVeP0NcJtAxRicqqLX/MjgkTrA2tPnSj9m60lO79S40GSzB238o2zJQ5ON +jHJdAhLlrsrkxZmDmKM59IdmV2OhN7O844ktbXDOqvq5AgMBAAGjUzBRMB0GA1Ud +DgQWBBTkDj66Wmaf9YP0YMqGKG51PejxWzAfBgNVHSMEGDAWgBTkDj66Wmaf9YP0 +YMqGKG51PejxWzAPBgNVHRMBAf8EBTADAQH/MA0GCSqGSIb3DQEBCwUAA4IBAQAo +9FwbGob1WRAo7ORShaDhopIDAYMCL1oc9YZ2ZQK8aDuZbqIZ/7+1LbWCeMAV4PTH +Tcx+n9Zg/g/RkoSNu8KqFoQGFmdBZnKOMU4vlITu/ORpDu1sjSA+Eo9YbipemX+n +jv+YHI4STFAWnyext3IUZVkT6wpU3pwUjX0fbk4LJfVLR41y4iD11XGergxAcpjj +T03txkJcrTiX65SnB041Y4exUMLOUn5lTs/q2rBNkiLNljXQ6l+8L1rdQEN/j0Mx +OYdc6FIUIESC1mMOf80+YOwxPJ862SyTv/cJB3npwTj/DdQu7blf/z2hMP7a7w+X +l5d31XzcDOrCf/bTthvb -----END CERTIFICATE----- diff -Nru mitmproxy-5.1.1/test/mitmproxy/net/data/verificationcerts/trusted-root.key mitmproxy-6.0.2/test/mitmproxy/net/data/verificationcerts/trusted-root.key --- mitmproxy-5.1.1/test/mitmproxy/net/data/verificationcerts/trusted-root.key 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/net/data/verificationcerts/trusted-root.key 2020-12-15 16:41:27.000000000 +0000 @@ -1,27 +1,27 @@ -----BEGIN RSA PRIVATE KEY----- -MIIEpAIBAAKCAQEA2e36r+Iguu9/DrzgvzjRZoTlZjDXtkqxrZPuyiL/zHezCSSq -SMoMHPfRHy95ulhY70kPAgATvRpyyHqAekc1+V/dOHT0MyHpDJBNuyeA+cM2FgJm -LLvZnf//a42nWgNNcx78baXN9ALE6Dq0Wtkb+y8KLb3KKdmwQl12THYLwwaQrgjp -d3EN+Oi2QZhA+Nau+z+0oJyPfIAj7ssjFQBU2RODZxVJaFREU3DDKOVcL7VxU+NQ -ehTqrcoWRWPPVtUnjSpZvp/M7vv2lBeXPbrMcL0M4+/acwRWEvcTGc28ZQZlPdRm -yxTVN9jvsKdI1+nNTH5bhr0Y3B0KSvoB0USXWQIDAQABAoIBAQCWWYbgHSPzlBOW -eVyc0Hg3QGx7aisIStP2Kt9NeYP87oAISNFqUmq0+Yu+9iQHGbiRrVe7S45SopKa -GVnWApcMKsUWlCl9tWFxF4VpH0HuDm2cFZ+kMR1b0ifHbf0NLsYaLEB+7Sr/s4Fh -rk6LdsnFK5jcIdn9sX/W6WAaND69Ft45sMXuAKPeqwySm7t77nlIEp4lJy/F/rvS -UUV77UVL/5GqC1HOP1kXXEJDxHtNrDPh0fF49Qo/6JPRmLIOia0yKUS5+8lKHh6V -J7nWTucyvviKoYvdJHpi9w2+DTI7mT33Go1Ss4QgHihzcURTtfjsn7yOelS6z8XM -YOxURwoZAoGBAPZpod0Zpjm9trf547reFPLfAJUiKDj1wYXnlpSFAzOf7pSXDrfW -L5dc3c5x2DiBXUlBu03ZWo7wVOKv7d9ByTrvXEqT1/KTqmHoI6Mzy0rn9wU0FmL4 -ASPcpPMnNStOSmRfe9HITDVvgLYo2wQEu1MXf5rGs5HjTRpEGVdWRkRPAoGBAOJo -pQ9uRtaLlCF3KXoRr8HCBKI64IWDki7pbbLTglOBbmptQLfE2GPc/FPBsQwR3KBX -m18A2pc7CRUKzsDavb7SyXTqtP3NF/AguTjGXIv/45QD6A96SStaYR0ZXSA3/uhZ -rAaSSXxaI3mDTc32pXoXJDp7K+CPc5w00I8u1/fXAoGAe1UPoPyPiGL+K0M1yngR -gCZBwmMgQrIutHjfk2Kn4ZTw8wpQYY8grt/aXNP6Zv3I1TvDJgneG6EKu5NWueHR -eGAJj4JEGbPzGaH5BFyOKeXEa6RQeCStXWe4X8OGBzDeZzKrZKqeCjjO8V2tkWtU -3xfp1GwTwLdGBhmDnYUfEl0CgYEA1vzVF6T4gQtDGtADQ5V91je8nKvZvQ4lhoRD -lVZAX7j8tvSNSrMRYypZM9Mtoi9n1524vGqcJpR5WFDN6NUM7iFMCMhCGupgO7Vn -DCFXidzvJgLbna7Zwd/tbWtDQa/KTqmvrwHD49/X5a+n9tapZRiKXznMfUzaU87W -589sZjsCgYBQvtZpNQRqTDNOSuIJGsloMNp1HG2Keoj/nBQ9idkw29rUqhae7ZAa -Csk91ix+AMR8nzQSkpHWOk2+RjdVAQ7m87bnQM6mmAJTcipikDfFxNqOWt7JkM2h -Ydk36LHBIBVLOeqMXrhCHvai3h6efUz7wjGhNxRJ2OGrzWFfLgVsqA== +MIIEpAIBAAKCAQEA1VlINmHbM7ViGabFnWyAegHsGtSILe5jZCjENTSer6a4lZc1 +oWa9BX0lU4ndptzndn+QO+dUPiFy5DSfHemOKlYqwLLRDOe7Ah/x5rdRN3GTgMy0 +GNsQiKyj5m7unfHkhjtavmPCrLQiv1feP0EWoY46B8ZC3EgJnYOjwJG4QezMDsYi +bJSG7AWOFAlz9JeJFFCLjIvFtmODNVYLy0aVI2ilaNTUNIS4uFwZ1OwbKgPPz2Q7 +lXj9DXCbQMUYnKqi1/zI4JE6wNrT50o/ZutJTu/UuNBkswdt/KNsyUOTjYxyXQIS +5a7K5MWZg5ijOfSHZldjoTezvOOJLW1wzqr6uQIDAQABAoIBAQCZ5hSkPgSskTwZ +uua0P2+SE07or8UoNN13g7y9kJ6joIVoSzJBomXZzsOdNkcwtgs1vKEjxSRMyT4i +hh79qduGiK/AX31TbNnGl5qczkIE16YKdB+6ClQr8zUr5Xu1badmeOxliHsGr86c +7lqAkIgts/YCY68ZTdG50RrPr44V5tSqfJdLv7e5/eYYMrmae1xQSBDYgyoSxkmn +A/u/dqI2/eZUIeqzUxplCf+c5Fstw5MVvgiMBtLYYq7Uia1SDV6aQovbFItde31e +nnwhlTsY/BCfCugu/nvkEUbV09TtvYoSiYlByzIEunX2BQ9RYeawnwsGoXXexmE/ +5ZcFLCtxAoGBAO+SL6R04/pAgmiKPMzDSIflAMbbwtZGmJrtPxXYOaX275PLAvB7 +lvL2TlTInbFlUoLHymD0ZQ33Z5yrmjJbdfTHlOOouKjV8tOLXku7AoejbIXQD6P2 +/x4i4xER4dpnx/tPar7RLvoA/+qtwnXH/AXJVmGfqjV0aWC3/EJ4RQetAoGBAOP6 +v38efo3zEz9/c7zibTujOcbdQ8IKwrvOOAg9Qd/QI5bfJusxpjPK5mSP3e4INxDs +7k0jGTFVZQdcAhjkm9yQ1eXIiTK20wpOWigJEEnlp3vAUh2AZGAEZQQ323vOrGgb +X+7N/IsmFU2e0aCCP1XyxSalhSEvXas/rn81T5C9AoGAPDl4ppQH95Id9l/B9wuk +gR0Ne/Ak467tkAzSDhDENTeFdJbVT1USVlOaz297TOwWuO+2rXbirIckUdtFsSES +3w0JHc39Vdwq5gkkFE+XzDdGAcGTHPnrmkzU3iarqzbprACD6iI19/0mWH2D5A35 +Cg9aGsKDjFqOXxUBYkhPuiUCgYEAiPw7IpPdMUgUkbvwfFeFkScj6VU8iJTUq+pe +irzcHzK+n8IYbYwYtNg/Fb/Id2WiSN5E1j63vYf1O7XSYMjEyoSQshCMN9fY+UFZ +NOM1UWHYACjwbl+ecH9Tf3RcwojN2YRRYXmOPMI2XLyIcyywB+HnCrC8TNNcMjEC +ldcfpvUCgYBeiT3tRbfUqeW/6XX50S4YrDwm+5hWmGTJ9fwi0fvu0by/FwP22K4p +xXVrLNiNDuJ/meHuRFXRkFdGummltFLXwZHxIo3AxP+8nRMDvPyt2Pcp3Sf1kidR +z3PpsTVkEDsj5Cgq7JcpTxU9eiHth3bEZtsA4H94UeuIuZo4KI3KOA== -----END RSA PRIVATE KEY----- diff -Nru mitmproxy-5.1.1/test/mitmproxy/net/data/verificationcerts/trusted-root.pem mitmproxy-6.0.2/test/mitmproxy/net/data/verificationcerts/trusted-root.pem --- mitmproxy-5.1.1/test/mitmproxy/net/data/verificationcerts/trusted-root.pem 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/net/data/verificationcerts/trusted-root.pem 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,48 @@ +-----BEGIN CERTIFICATE----- +MIIDazCCAlOgAwIBAgIUB5WLNdtHRvciUwDfySAEHYVSNe0wDQYJKoZIhvcNAQEL +BQAwRTELMAkGA1UEBhMCQVUxEzARBgNVBAgMClNvbWUtU3RhdGUxITAfBgNVBAoM +GEludGVybmV0IFdpZGdpdHMgUHR5IEx0ZDAeFw0yMDExMDMwNzAzMjZaFw00MDEw +MjkwNzAzMjZaMEUxCzAJBgNVBAYTAkFVMRMwEQYDVQQIDApTb21lLVN0YXRlMSEw +HwYDVQQKDBhJbnRlcm5ldCBXaWRnaXRzIFB0eSBMdGQwggEiMA0GCSqGSIb3DQEB +AQUAA4IBDwAwggEKAoIBAQDVWUg2YdsztWIZpsWdbIB6Aewa1Igt7mNkKMQ1NJ6v +priVlzWhZr0FfSVTid2m3Od2f5A751Q+IXLkNJ8d6Y4qVirAstEM57sCH/Hmt1E3 +cZOAzLQY2xCIrKPmbu6d8eSGO1q+Y8KstCK/V94/QRahjjoHxkLcSAmdg6PAkbhB +7MwOxiJslIbsBY4UCXP0l4kUUIuMi8W2Y4M1VgvLRpUjaKVo1NQ0hLi4XBnU7Bsq +A8/PZDuVeP0NcJtAxRicqqLX/MjgkTrA2tPnSj9m60lO79S40GSzB238o2zJQ5ON +jHJdAhLlrsrkxZmDmKM59IdmV2OhN7O844ktbXDOqvq5AgMBAAGjUzBRMB0GA1Ud +DgQWBBTkDj66Wmaf9YP0YMqGKG51PejxWzAfBgNVHSMEGDAWgBTkDj66Wmaf9YP0 +YMqGKG51PejxWzAPBgNVHRMBAf8EBTADAQH/MA0GCSqGSIb3DQEBCwUAA4IBAQAo +9FwbGob1WRAo7ORShaDhopIDAYMCL1oc9YZ2ZQK8aDuZbqIZ/7+1LbWCeMAV4PTH +Tcx+n9Zg/g/RkoSNu8KqFoQGFmdBZnKOMU4vlITu/ORpDu1sjSA+Eo9YbipemX+n +jv+YHI4STFAWnyext3IUZVkT6wpU3pwUjX0fbk4LJfVLR41y4iD11XGergxAcpjj +T03txkJcrTiX65SnB041Y4exUMLOUn5lTs/q2rBNkiLNljXQ6l+8L1rdQEN/j0Mx +OYdc6FIUIESC1mMOf80+YOwxPJ862SyTv/cJB3npwTj/DdQu7blf/z2hMP7a7w+X +l5d31XzcDOrCf/bTthvb +-----END CERTIFICATE----- +-----BEGIN RSA PRIVATE KEY----- +MIIEpAIBAAKCAQEA1VlINmHbM7ViGabFnWyAegHsGtSILe5jZCjENTSer6a4lZc1 +oWa9BX0lU4ndptzndn+QO+dUPiFy5DSfHemOKlYqwLLRDOe7Ah/x5rdRN3GTgMy0 +GNsQiKyj5m7unfHkhjtavmPCrLQiv1feP0EWoY46B8ZC3EgJnYOjwJG4QezMDsYi +bJSG7AWOFAlz9JeJFFCLjIvFtmODNVYLy0aVI2ilaNTUNIS4uFwZ1OwbKgPPz2Q7 +lXj9DXCbQMUYnKqi1/zI4JE6wNrT50o/ZutJTu/UuNBkswdt/KNsyUOTjYxyXQIS +5a7K5MWZg5ijOfSHZldjoTezvOOJLW1wzqr6uQIDAQABAoIBAQCZ5hSkPgSskTwZ +uua0P2+SE07or8UoNN13g7y9kJ6joIVoSzJBomXZzsOdNkcwtgs1vKEjxSRMyT4i +hh79qduGiK/AX31TbNnGl5qczkIE16YKdB+6ClQr8zUr5Xu1badmeOxliHsGr86c +7lqAkIgts/YCY68ZTdG50RrPr44V5tSqfJdLv7e5/eYYMrmae1xQSBDYgyoSxkmn +A/u/dqI2/eZUIeqzUxplCf+c5Fstw5MVvgiMBtLYYq7Uia1SDV6aQovbFItde31e +nnwhlTsY/BCfCugu/nvkEUbV09TtvYoSiYlByzIEunX2BQ9RYeawnwsGoXXexmE/ +5ZcFLCtxAoGBAO+SL6R04/pAgmiKPMzDSIflAMbbwtZGmJrtPxXYOaX275PLAvB7 +lvL2TlTInbFlUoLHymD0ZQ33Z5yrmjJbdfTHlOOouKjV8tOLXku7AoejbIXQD6P2 +/x4i4xER4dpnx/tPar7RLvoA/+qtwnXH/AXJVmGfqjV0aWC3/EJ4RQetAoGBAOP6 +v38efo3zEz9/c7zibTujOcbdQ8IKwrvOOAg9Qd/QI5bfJusxpjPK5mSP3e4INxDs +7k0jGTFVZQdcAhjkm9yQ1eXIiTK20wpOWigJEEnlp3vAUh2AZGAEZQQ323vOrGgb +X+7N/IsmFU2e0aCCP1XyxSalhSEvXas/rn81T5C9AoGAPDl4ppQH95Id9l/B9wuk +gR0Ne/Ak467tkAzSDhDENTeFdJbVT1USVlOaz297TOwWuO+2rXbirIckUdtFsSES +3w0JHc39Vdwq5gkkFE+XzDdGAcGTHPnrmkzU3iarqzbprACD6iI19/0mWH2D5A35 +Cg9aGsKDjFqOXxUBYkhPuiUCgYEAiPw7IpPdMUgUkbvwfFeFkScj6VU8iJTUq+pe +irzcHzK+n8IYbYwYtNg/Fb/Id2WiSN5E1j63vYf1O7XSYMjEyoSQshCMN9fY+UFZ +NOM1UWHYACjwbl+ecH9Tf3RcwojN2YRRYXmOPMI2XLyIcyywB+HnCrC8TNNcMjEC +ldcfpvUCgYBeiT3tRbfUqeW/6XX50S4YrDwm+5hWmGTJ9fwi0fvu0by/FwP22K4p +xXVrLNiNDuJ/meHuRFXRkFdGummltFLXwZHxIo3AxP+8nRMDvPyt2Pcp3Sf1kidR +z3PpsTVkEDsj5Cgq7JcpTxU9eiHth3bEZtsA4H94UeuIuZo4KI3KOA== +-----END RSA PRIVATE KEY----- diff -Nru mitmproxy-5.1.1/test/mitmproxy/net/data/verificationcerts/trusted-root.srl mitmproxy-6.0.2/test/mitmproxy/net/data/verificationcerts/trusted-root.srl --- mitmproxy-5.1.1/test/mitmproxy/net/data/verificationcerts/trusted-root.srl 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/net/data/verificationcerts/trusted-root.srl 2020-12-15 16:41:27.000000000 +0000 @@ -1 +1 @@ -A3E83F6857295BF8 +496A0BC25ED173FFD328FC0AC2FDB95616FAB850 diff -Nru mitmproxy-5.1.1/test/mitmproxy/net/http/http1/test_assemble.py mitmproxy-6.0.2/test/mitmproxy/net/http/http1/test_assemble.py --- mitmproxy-5.1.1/test/mitmproxy/net/http/http1/test_assemble.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/net/http/http1/test_assemble.py 2020-12-15 16:41:27.000000000 +0000 @@ -40,6 +40,22 @@ b"message" ) + resp = tresp() + resp.headers["transfer-encoding"] = "chunked" + resp.headers["trailer"] = "my-little-trailer" + resp.trailers = Headers([(b"my-little-trailer", b"foobar")]) + assert assemble_response(resp) == ( + b"HTTP/1.1 200 OK\r\n" + b"header-response: svalue\r\n" + b"content-length: 7\r\n" + b"transfer-encoding: chunked\r\n" + b"trailer: my-little-trailer\r\n" + b"\r\n7\r\n" + b"message" + b"\r\n0\r\n" + b"my-little-trailer: foobar\r\n\r\n" + ) + with pytest.raises(exceptions.HttpException): assemble_response(tresp(content=None)) @@ -52,28 +68,31 @@ def test_assemble_body(): - c = list(assemble_body(Headers(), [b"body"])) + c = list(assemble_body(Headers(), [b"body"], Headers())) assert c == [b"body"] - c = list(assemble_body(Headers(transfer_encoding="chunked"), [b"123456789a", b""])) + c = list(assemble_body(Headers(transfer_encoding="chunked"), [b"123456789a", b""], Headers())) assert c == [b"a\r\n123456789a\r\n", b"0\r\n\r\n"] - c = list(assemble_body(Headers(transfer_encoding="chunked"), [b"123456789a"])) + c = list(assemble_body(Headers(transfer_encoding="chunked"), [b"123456789a"], Headers())) assert c == [b"a\r\n123456789a\r\n", b"0\r\n\r\n"] + c = list(assemble_body(Headers(transfer_encoding="chunked"), [b"123456789a"], Headers(trailer="trailer"))) + assert c == [b"a\r\n123456789a\r\n", b"0\r\ntrailer: trailer\r\n\r\n"] + + with pytest.raises(exceptions.HttpException): + list(assemble_body(Headers(), [b"body"], Headers(trailer="trailer"))) + def test_assemble_request_line(): assert _assemble_request_line(treq().data) == b"GET /path HTTP/1.1" - authority_request = treq(method=b"CONNECT", first_line_format="authority").data + authority_request = treq(method=b"CONNECT", authority=b"address:22").data assert _assemble_request_line(authority_request) == b"CONNECT address:22 HTTP/1.1" - absolute_request = treq(first_line_format="absolute").data + absolute_request = treq(scheme=b"http", authority=b"address:22").data assert _assemble_request_line(absolute_request) == b"GET http://address:22/path HTTP/1.1" - with pytest.raises(RuntimeError): - _assemble_request_line(treq(first_line_format="invalid_form").data) - def test_assemble_request_headers(): # https://github.com/mitmproxy/mitmproxy/issues/186 diff -Nru mitmproxy-5.1.1/test/mitmproxy/net/http/http1/test_read.py mitmproxy-6.0.2/test/mitmproxy/net/http/http1/test_read.py --- mitmproxy-5.1.1/test/mitmproxy/net/http/http1/test_read.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/net/http/http1/test_read.py 2020-12-15 16:41:27.000000000 +0000 @@ -7,7 +7,7 @@ from mitmproxy.net.http.http1.read import ( read_request, read_response, read_request_head, read_response_head, read_body, connection_close, expected_http_body_size, _get_first_line, - _read_request_line, _parse_authority_form, _read_response_line, _check_http_version, + _read_request_line, _read_response_line, _check_http_version, _read_headers, _read_chunked, get_header_tokens ) from mitmproxy.test.tutils import treq, tresp @@ -146,6 +146,7 @@ headers = Headers() assert connection_close(b"HTTP/1.0", headers) assert not connection_close(b"HTTP/1.1", headers) + assert not connection_close(b"HTTP/2.0", headers) headers["connection"] = "keep-alive" assert not connection_close(b"HTTP/1.1", headers) @@ -242,35 +243,26 @@ return _read_request_line(BytesIO(b)) assert (t(b"GET / HTTP/1.1") == - ("relative", b"GET", None, None, None, b"/", b"HTTP/1.1")) + ("", 0, b"GET", b"", b"", b"/", b"HTTP/1.1")) assert (t(b"OPTIONS * HTTP/1.1") == - ("relative", b"OPTIONS", None, None, None, b"*", b"HTTP/1.1")) + ("", 0, b"OPTIONS", b"", b"", b"*", b"HTTP/1.1")) assert (t(b"CONNECT foo:42 HTTP/1.1") == - ("authority", b"CONNECT", None, b"foo", 42, None, b"HTTP/1.1")) + ("foo", 42, b"CONNECT", b"", b"foo:42", b"", b"HTTP/1.1")) assert (t(b"GET http://foo:42/bar HTTP/1.1") == - ("absolute", b"GET", b"http", b"foo", 42, b"/bar", b"HTTP/1.1")) + ("foo", 42, b"GET", b"http", b"foo:42", b"/bar", b"HTTP/1.1")) with pytest.raises(exceptions.HttpSyntaxException): t(b"GET / WTF/1.1") with pytest.raises(exceptions.HttpSyntaxException): + t(b"CONNECT example.com HTTP/1.1") # port missing + with pytest.raises(exceptions.HttpSyntaxException): + t(b"GET ws://example.com/ HTTP/1.1") # port missing + with pytest.raises(exceptions.HttpSyntaxException): t(b"this is not http") with pytest.raises(exceptions.HttpReadDisconnect): t(b"") -def test_parse_authority_form(): - assert _parse_authority_form(b"foo:42") == (b"foo", 42) - assert _parse_authority_form(b"[2001:db8:42::]:443") == (b"2001:db8:42::", 443) - with pytest.raises(exceptions.HttpSyntaxException): - _parse_authority_form(b"foo") - with pytest.raises(exceptions.HttpSyntaxException): - _parse_authority_form(b"foo:bar") - with pytest.raises(exceptions.HttpSyntaxException): - _parse_authority_form(b"foo:99999999") - with pytest.raises(exceptions.HttpSyntaxException): - _parse_authority_form(b"f\x00oo:80") - - def test_read_response_line(): def t(b): return _read_response_line(BytesIO(b)) diff -Nru mitmproxy-5.1.1/test/mitmproxy/net/http/http2/test_framereader.py mitmproxy-6.0.2/test/mitmproxy/net/http/http2/test_framereader.py --- mitmproxy-5.1.1/test/mitmproxy/net/http/http2/test_framereader.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/net/http/http2/test_framereader.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,41 +0,0 @@ -import pytest -import codecs -from io import BytesIO -import hyperframe.frame - -from mitmproxy import exceptions -from mitmproxy.net.http.http2 import read_raw_frame, parse_frame - - -def test_read_raw_frame(): - raw = codecs.decode('000006000101234567666f6f626172', 'hex_codec') - bio = BytesIO(raw) - bio.safe_read = bio.read - - header, body = read_raw_frame(bio) - assert header - assert body - - -def test_read_raw_frame_failed(): - raw = codecs.decode('485454000000000000', 'hex_codec') - bio = BytesIO(raw) - bio.safe_read = bio.read - - with pytest.raises(exceptions.HttpException): - read_raw_frame(bio) - - -def test_parse_frame(): - f = parse_frame( - codecs.decode('000006000101234567', 'hex_codec'), - codecs.decode('666f6f626172', 'hex_codec') - ) - assert isinstance(f, hyperframe.frame.Frame) - - -def test_parse_frame_combined(): - f = parse_frame( - codecs.decode('000006000101234567666f6f626172', 'hex_codec'), - ) - assert isinstance(f, hyperframe.frame.Frame) diff -Nru mitmproxy-5.1.1/test/mitmproxy/net/http/http2/test_utils.py mitmproxy-6.0.2/test/mitmproxy/net/http/http2/test_utils.py --- mitmproxy-5.1.1/test/mitmproxy/net/http/http2/test_utils.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/net/http/http2/test_utils.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,70 +0,0 @@ -import pytest - -from mitmproxy.net.http.http2 import parse_headers - - -class TestHttp2ParseHeaders: - - def test_relative(self): - h = dict([ - (':authority', "127.0.0.1:1234"), - (':method', 'GET'), - (':scheme', 'https'), - (':path', '/'), - ]) - first_line_format, method, scheme, host, port, path = parse_headers(h) - assert first_line_format == 'relative' - assert method == b'GET' - assert scheme == b'https' - assert host == b'127.0.0.1' - assert port == 1234 - assert path == b'/' - - def test_absolute(self): - h = dict([ - (':authority', "127.0.0.1:1234"), - (':method', 'GET'), - (':scheme', 'https'), - (':path', 'https://127.0.0.1:4321'), - ]) - first_line_format, method, scheme, host, port, path = parse_headers(h) - assert first_line_format == 'absolute' - assert method == b'GET' - assert scheme == b'https' - assert host == b'127.0.0.1' - assert port == 1234 - assert path == b'https://127.0.0.1:4321' - - @pytest.mark.parametrize("scheme, expected_port", [ - ('http', 80), - ('https', 443), - ]) - def test_without_port(self, scheme, expected_port): - h = dict([ - (':authority', "127.0.0.1"), - (':method', 'GET'), - (':scheme', scheme), - (':path', '/'), - ]) - _, _, _, _, port, _ = parse_headers(h) - assert port == expected_port - - def test_without_authority(self): - h = dict([ - (':method', 'GET'), - (':scheme', 'https'), - (':path', '/'), - ]) - _, _, _, host, _, _ = parse_headers(h) - assert host == b'localhost' - - def test_connect(self): - h = dict([ - (':authority', "127.0.0.1"), - (':method', 'CONNECT'), - (':scheme', 'https'), - (':path', '/'), - ]) - - with pytest.raises(NotImplementedError): - parse_headers(h) diff -Nru mitmproxy-5.1.1/test/mitmproxy/net/http/test_headers.py mitmproxy-6.0.2/test/mitmproxy/net/http/test_headers.py --- mitmproxy-5.1.1/test/mitmproxy/net/http/test_headers.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/net/http/test_headers.py 2020-12-15 16:41:27.000000000 +0000 @@ -41,11 +41,11 @@ assert headers["Accept"] == "text/plain" with pytest.raises(TypeError): - Headers([[b"Host", u"not-bytes"]]) + Headers([[b"Host", "not-bytes"]]) def test_set(self): headers = Headers() - headers[u"foo"] = u"1" + headers["foo"] = "1" headers[b"bar"] = b"2" headers["baz"] = b"3" with pytest.raises(TypeError): @@ -65,32 +65,6 @@ headers = Headers() assert bytes(headers) == b"" - def test_replace_simple(self): - headers = Headers(Host="example.com", Accept="text/plain") - replacements = headers.replace("Host: ", "X-Host: ") - assert replacements == 1 - assert headers["X-Host"] == "example.com" - assert "Host" not in headers - assert headers["Accept"] == "text/plain" - - def test_replace_multi(self): - headers = self._2host() - headers.replace(r"Host: example.com", r"Host: example.de") - assert headers.get_all("Host") == ["example.de", "example.org"] - - def test_replace_remove_spacer(self): - headers = Headers(Host="example.com") - replacements = headers.replace(r"Host: ", "X-Host ") - assert replacements == 0 - assert headers["Host"] == "example.com" - - def test_replace_with_count(self): - headers = Headers(Host="foobarfoo.com", Accept="foo/bar") - replacements = headers.replace("foo", "bar", count=1) - assert replacements == 1 - assert headers["Host"] == "barbarfoo.com" - assert headers["Accept"] == "foo/bar" - def test_parse_content_type(): p = parse_content_type diff -Nru mitmproxy-5.1.1/test/mitmproxy/net/http/test_http2.py mitmproxy-6.0.2/test/mitmproxy/net/http/test_http2.py --- mitmproxy-5.1.1/test/mitmproxy/net/http/test_http2.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/net/http/test_http2.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,37 @@ +import pytest +import codecs +from io import BytesIO + +import hyperframe + +from mitmproxy import exceptions +from mitmproxy.net.http import http2 + + +def test_read_frame(): + raw = codecs.decode('000006000101234567666f6f626172', 'hex_codec') + bio = BytesIO(raw) + bio.safe_read = bio.read + + frame, consumed_bytes = http2.read_frame(bio) + assert isinstance(frame, hyperframe.frame.DataFrame) + assert frame.stream_id == 19088743 + assert 'END_STREAM' in frame.flags + assert len(frame.flags) == 1 + assert frame.data == b'foobar' + assert consumed_bytes == raw + + bio = BytesIO(raw) + bio.safe_read = bio.read + frame, consumed_bytes = http2.read_frame(bio, False) + assert frame is None + assert consumed_bytes == raw + + +def test_read_frame_failed(): + raw = codecs.decode('485454000000000000', 'hex_codec') + bio = BytesIO(raw) + bio.safe_read = bio.read + + with pytest.raises(exceptions.HttpException): + _ = http2.read_frame(bio, False) diff -Nru mitmproxy-5.1.1/test/mitmproxy/net/http/test_message.py mitmproxy-6.0.2/test/mitmproxy/net/http/test_message.py --- mitmproxy-5.1.1/test/mitmproxy/net/http/test_message.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/net/http/test_message.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,5 +1,3 @@ -# -*- coding: utf-8 -*- - import pytest from mitmproxy.test import tutils @@ -64,17 +62,17 @@ def test_eq_ne(self): resp = tutils.tresp(timestamp_start=42, timestamp_end=42) same = tutils.tresp(timestamp_start=42, timestamp_end=42) - assert resp == same + assert resp.data == same.data other = tutils.tresp(timestamp_start=0, timestamp_end=0) - assert resp != other + assert resp.data != other.data assert resp != 0 def test_serializable(self): resp = tutils.tresp() resp2 = http.Response.from_state(resp.get_state()) - assert resp == resp2 + assert resp.data == resp2.data def test_content_length_update(self): resp = tutils.tresp() @@ -99,16 +97,9 @@ def test_http_version(self): _test_decoded_attr(tutils.tresp(), "http_version") - - def test_replace(self): - r = tutils.tresp() - r.content = b"foofootoo" - r.replace(b"foo", "gg") - assert r.content == b"ggggtoo" - - r.content = b"foofootoo" - r.replace(b"foo", "gg", count=1) - assert r.content == b"ggfootoo" + assert tutils.tresp(http_version=b"HTTP/1.0").is_http10 + assert tutils.tresp(http_version=b"HTTP/1.1").is_http11 + assert tutils.tresp(http_version=b"HTTP/2.0").is_http2 class TestMessageContentEncoding: @@ -142,7 +133,7 @@ assert r.raw_content == b"foo" with pytest.raises(TypeError): - r.content = u"foo" + r.content = "foo" def test_unknown_ce(self): r = tutils.tresp() @@ -211,38 +202,59 @@ r = tutils.tresp(content=b'\xfc') assert r.raw_content == b"\xfc" assert r.content == b"\xfc" - assert r.text == u"ü" + assert r.text == "ü" r.encode("gzip") - assert r.text == u"ü" + assert r.text == "ü" r.decode() - assert r.text == u"ü" + assert r.text == "ü" r.headers["content-type"] = "text/html; charset=latin1" r.content = b"\xc3\xbc" - assert r.text == u"ü" + assert r.text == "ü" r.headers["content-type"] = "text/html; charset=utf8" - assert r.text == u"ü" + assert r.text == "ü" def test_guess_json(self): r = tutils.tresp(content=b'"\xc3\xbc"') r.headers["content-type"] = "application/json" - assert r.text == u'"ü"' + assert r.text == '"ü"' def test_guess_meta_charset(self): r = tutils.tresp(content=b'\xe6\x98\x8e\xe4\xbc\xaf') # "鏄庝集" is decoded form of \xe6\x98\x8e\xe4\xbc\xaf in gb18030 - assert u"鏄庝集" in r.text + assert "鏄庝集" in r.text + + def test_guess_css_charset(self): + # @charset but not text/css + r = tutils.tresp(content=b'@charset "gb2312";' + b'#foo::before {content: "\xe6\x98\x8e\xe4\xbc\xaf"}') + # "鏄庝集" is decoded form of \xe6\x98\x8e\xe4\xbc\xaf in gb18030 + assert "鏄庝集" not in r.text + + # @charset not at the beginning + r = tutils.tresp(content=b'foo@charset "gb2312";' + b'#foo::before {content: "\xe6\x98\x8e\xe4\xbc\xaf"}') + r.headers["content-type"] = "text/css" + # "鏄庝集" is decoded form of \xe6\x98\x8e\xe4\xbc\xaf in gb18030 + assert "鏄庝集" not in r.text + + # @charset and text/css + r = tutils.tresp(content=b'@charset "gb2312";' + b'#foo::before {content: "\xe6\x98\x8e\xe4\xbc\xaf"}') + r.headers["content-type"] = "text/css" + # "鏄庝集" is decoded form of \xe6\x98\x8e\xe4\xbc\xaf in gb18030 + assert "鏄庝集" in r.text def test_guess_latin_1(self): r = tutils.tresp(content=b"\xF0\xE2") - assert r.text == u"ðâ" + assert r.text == "ðâ" def test_none(self): r = tutils.tresp(content=None) assert r.text is None - r.text = u"foo" + r.text = "foo" assert r.text is not None r.text = None assert r.text is None @@ -250,11 +262,11 @@ def test_modify(self): r = tutils.tresp() - r.text = u"ü" + r.text = "ü" assert r.raw_content == b"\xfc" r.headers["content-type"] = "text/html; charset=utf8" - r.text = u"ü" + r.text = "ü" assert r.raw_content == b"\xc3\xbc" assert r.headers["content-length"] == "2" @@ -263,8 +275,8 @@ r.headers["content-type"] = "text/html; charset=wtf" r.raw_content = b"foo" with pytest.raises(ValueError): - assert r.text == u"foo" - assert r.get_text(strict=False) == u"foo" + assert r.text == "foo" + assert r.get_text(strict=False) == "foo" def test_cannot_decode(self): r = tutils.tresp() @@ -282,21 +294,21 @@ assert r.raw_content is None r.headers["content-type"] = "text/html; charset=latin1; foo=bar" - r.text = u"☃" + r.text = "☃" assert r.headers["content-type"] == "text/html; charset=utf-8; foo=bar" assert r.raw_content == b'\xe2\x98\x83' r.headers["content-type"] = "gibberish" - r.text = u"☃" + r.text = "☃" assert r.headers["content-type"] == "text/plain; charset=utf-8" assert r.raw_content == b'\xe2\x98\x83' del r.headers["content-type"] - r.text = u"☃" + r.text = "☃" assert r.headers["content-type"] == "text/plain; charset=utf-8" assert r.raw_content == b'\xe2\x98\x83' r.headers["content-type"] = "text/html; charset=latin1" - r.text = u'\udcff' + r.text = '\udcff' assert r.headers["content-type"] == "text/html; charset=utf-8" assert r.raw_content == b"\xFF" diff -Nru mitmproxy-5.1.1/test/mitmproxy/net/http/test_multipart.py mitmproxy-6.0.2/test/mitmproxy/net/http/test_multipart.py --- mitmproxy-5.1.1/test/mitmproxy/net/http/test_multipart.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/net/http/test_multipart.py 2020-12-15 16:41:27.000000000 +0000 @@ -38,8 +38,8 @@ def test_encode(): - data = [("file".encode('utf-8'), "shell.jpg".encode('utf-8')), - ("file_size".encode('utf-8'), "1000".encode('utf-8'))] + data = [(b"file", b"shell.jpg"), + (b"file_size", b"1000")] headers = Headers( content_type='multipart/form-data; boundary=127824672498' ) @@ -51,7 +51,7 @@ assert len(content) == 252 with pytest.raises(ValueError, match=r"boundary found in encoded string"): - multipart.encode(headers, [("key".encode('utf-8'), "--127824672498".encode('utf-8'))]) + multipart.encode(headers, [(b"key", b"--127824672498")]) boundary = 'boundary茅莽' headers = Headers( diff -Nru mitmproxy-5.1.1/test/mitmproxy/net/http/test_request.py mitmproxy-6.0.2/test/mitmproxy/net/http/test_request.py --- mitmproxy-5.1.1/test/mitmproxy/net/http/test_request.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/net/http/test_request.py 2020-12-15 16:41:27.000000000 +0000 @@ -21,20 +21,40 @@ treq(headers="foobar") with pytest.raises(ValueError): treq(content="foobar") + with pytest.raises(ValueError): + treq(trailers="foobar") assert isinstance(treq(headers=()).headers, Headers) + assert isinstance(treq(trailers=()).trailers, Headers) class TestRequestCore: """ Tests for addons and the attributes that are directly proxied from the data structure """ + def test_repr(self): request = treq() assert repr(request) == "Request(GET address:22/path)" request.host = None assert repr(request) == "Request(GET /path)" + def test_init_conv(self): + assert Request( + b"example.com", + 80, + "GET", + "http", + "example.com", + "/", + "HTTP/1.1", + (), + None, + (), + 0, + 0, + ) # type: ignore + def test_make(self): r = Request.make("GET", "https://example.com/") assert r.method == "GET" @@ -58,70 +78,55 @@ r = Request.make("GET", "https://example.com/", headers=({"foo": "baz"})) assert r.headers["foo"] == "baz" + r = Request.make("GET", "https://example.com/", headers=Headers(foo="qux")) + assert r.headers["foo"] == "qux" + with pytest.raises(TypeError): Request.make("GET", "https://example.com/", headers=42) - def test_replace(self): - r = treq() - r.path = b"foobarfoo" - r.replace(b"foo", "bar") - assert r.path == "barbarbar" - - r.path = b"foobarfoo" - r.replace(b"foo", "bar", count=1) - assert r.path == "barbarfoo" - - r.path = "foobarfoo" - r.replace("foo", "bar", count=1) - assert r.path == "barbarfoo" - def test_first_line_format(self): - _test_passthrough_attr(treq(), "first_line_format") + assert treq(method=b"CONNECT").first_line_format == "authority" + assert treq(authority=b"example.com").first_line_format == "absolute" + assert treq(authority=b"").first_line_format == "relative" def test_method(self): _test_decoded_attr(treq(), "method") def test_scheme(self): _test_decoded_attr(treq(), "scheme") - assert treq(scheme=None).scheme is None def test_port(self): _test_passthrough_attr(treq(), "port") def test_path(self): - req = treq() - _test_decoded_attr(req, "path") - # path can also be None. - req.path = None - assert req.path is None - assert req.data.path is None + _test_decoded_attr(treq(), "path") - def test_host(self): + def test_authority(self): request = treq() - assert request.host == request.data.host.decode("idna") + assert request.authority == request.data.authority.decode("idna") # Test IDNA encoding # Set str, get raw bytes - request.host = "ídna.example" - assert request.data.host == b"xn--dna-qma.example" + request.authority = "ídna.example" + assert request.data.authority == b"xn--dna-qma.example" # Set raw bytes, get decoded - request.data.host = b"xn--idn-gla.example" - assert request.host == "idná.example" + request.data.authority = b"xn--idn-gla.example" + assert request.authority == "idná.example" # Set bytes, get raw bytes - request.host = b"xn--dn-qia9b.example" - assert request.data.host == b"xn--dn-qia9b.example" + request.authority = b"xn--dn-qia9b.example" + assert request.data.authority == b"xn--dn-qia9b.example" # IDNA encoding is not bijective - request.host = "fußball" - assert request.host == "fussball" + request.authority = "fußball" + assert request.authority == "fussball" # Don't fail on garbage - request.data.host = b"foo\xFF\x00bar" - assert request.host.startswith("foo") - assert request.host.endswith("bar") + request.data.authority = b"foo\xFF\x00bar" + assert request.authority.startswith("foo") + assert request.authority.endswith("bar") # foo.bar = foo.bar should not cause any side effects. - d = request.host - request.host = d - assert request.data.host == b"foo\xFF\x00bar" + d = request.authority + request.authority = d + assert request.data.authority == b"foo\xFF\x00bar" def test_host_update_also_updates_header(self): request = treq() @@ -130,59 +135,61 @@ assert "host" not in request.headers request.headers["Host"] = "foo" + request.authority = "foo" request.host = "example.org" assert request.headers["Host"] == "example.org" + assert request.authority == "example.org:22" def test_get_host_header(self): no_hdr = treq() assert no_hdr.host_header is None - h1 = treq(headers=( - (b"host", b"example.com"), - )) - assert h1.host_header == "example.com" - - h2 = treq(headers=( - (b":authority", b"example.org"), - )) - assert h2.host_header == "example.org" - - both_hdrs = treq(headers=( - (b"host", b"example.org"), - (b":authority", b"example.com"), - )) - assert both_hdrs.host_header == "example.com" + h1 = treq( + headers=((b"host", b"header.example.com"),), + authority=b"authority.example.com" + ) + assert h1.host_header == "header.example.com" + + h2 = h1.copy() + h2.http_version = "HTTP/2.0" + assert h2.host_header == "authority.example.com" + + h2_host_only = h2.copy() + h2_host_only.authority = "" + assert h2_host_only.host_header == "header.example.com" def test_modify_host_header(self): h1 = treq() assert "host" not in h1.headers - assert ":authority" not in h1.headers + h1.host_header = "example.com" - assert "host" in h1.headers - assert ":authority" not in h1.headers + assert h1.headers["Host"] == "example.com" + assert not h1.authority + h1.host_header = None assert "host" not in h1.headers + assert not h1.authority h2 = treq(http_version=b"HTTP/2.0") h2.host_header = "example.org" assert "host" not in h2.headers - assert ":authority" in h2.headers - del h2.host_header - assert ":authority" not in h2.headers - - both_hdrs = treq(headers=( - (b":authority", b"example.com"), - (b"host", b"example.org"), - )) - both_hdrs.host_header = "foo.example.com" - assert both_hdrs.headers["Host"] == "foo.example.com" - assert both_hdrs.headers[":authority"] == "foo.example.com" + assert h2.authority == "example.org" + + h2.headers["Host"] = "example.org" + h2.host_header = "foo.example.com" + assert h2.headers["Host"] == "foo.example.com" + assert h2.authority == "foo.example.com" + + h2.host_header = None + assert "host" not in h2.headers + assert not h2.authority class TestRequestUtils: """ Tests for additional convenience methods. """ + def test_url(self): request = treq() assert request.url == "http://address:22/path" @@ -201,7 +208,7 @@ assert request.url == "http://address:22" def test_url_authority(self): - request = treq(first_line_format="authority") + request = treq(method=b"CONNECT") assert request.url == "address:22" def test_pretty_host(self): @@ -212,17 +219,9 @@ # Same port as self.port (22) request.headers["host"] = "other:22" assert request.pretty_host == "other" - # Different ports - request.headers["host"] = "other" - assert request.pretty_host == "address" - assert request.host == "address" - # Empty host - request.host = None - assert request.pretty_host is None - assert request.host is None # Invalid IDNA - request.headers["host"] = ".disqus.com:22" + request.headers["host"] = ".disqus.com" assert request.pretty_host == ".disqus.com" def test_pretty_url(self): @@ -230,19 +229,19 @@ # Without host header assert request.url == "http://address:22/path" assert request.pretty_url == "http://address:22/path" - # Same port as self.port (22) + request.headers["host"] = "other:22" assert request.pretty_url == "http://other:22/path" - # Different ports - request.headers["host"] = "other" - assert request.pretty_url == "http://address:22/path" + + request = treq(method=b"CONNECT", authority=b"example:44") + assert request.pretty_url == "example:44" def test_pretty_url_options(self): request = treq(method=b"OPTIONS", path=b"*") assert request.pretty_url == "http://address:22" def test_pretty_url_authority(self): - request = treq(first_line_format="authority") + request = treq(method=b"CONNECT", authority="address:22") assert request.pretty_url == "address:22" def test_get_query(self): diff -Nru mitmproxy-5.1.1/test/mitmproxy/net/http/test_response.py mitmproxy-6.0.2/test/mitmproxy/net/http/test_response.py --- mitmproxy-5.1.1/test/mitmproxy/net/http/test_response.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/net/http/test_response.py 2020-12-15 16:41:27.000000000 +0000 @@ -20,8 +20,11 @@ tresp(reason="fööbär") with pytest.raises(ValueError): tresp(content="foobar") + with pytest.raises(ValueError): + tresp(trailers="foobar") assert isinstance(tresp(headers=()).headers, Headers) + assert isinstance(tresp(trailers=()).trailers, Headers) class TestResponseCore: @@ -30,9 +33,9 @@ """ def test_repr(self): response = tresp() - assert repr(response) == "Response(200 OK, unknown content type, 7b)" + assert repr(response) == "Response(200, unknown content type, 7b)" response.content = None - assert repr(response) == "Response(200 OK, no content)" + assert repr(response) == "Response(200, no content)" def test_make(self): r = Response.make() @@ -55,6 +58,9 @@ r = Response.make(headers=({"foo": "baz"})) assert r.headers["foo"] == "baz" + r = Response.make(headers=Headers(foo="qux")) + assert r.headers["foo"] == "qux" + with pytest.raises(TypeError): Response.make(headers=42) @@ -71,9 +77,6 @@ resp.reason = b"DEF" assert resp.data.reason == b"DEF" - resp.reason = None - assert resp.data.reason is None - resp.data.reason = b'cr\xe9e' assert resp.reason == "crée" diff -Nru mitmproxy-5.1.1/test/mitmproxy/net/http/test_url.py mitmproxy-6.0.2/test/mitmproxy/net/http/test_url.py --- mitmproxy-5.1.1/test/mitmproxy/net/http/test_url.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/net/http/test_url.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,7 +1,9 @@ +from typing import AnyStr + import pytest -import sys from mitmproxy.net.http import url +from mitmproxy.net.http.url import parse_authority def test_parse(): @@ -50,7 +52,6 @@ def test_ascii_check(): - test_url = "https://xyz.tax-edu.net?flag=selectCourse&lc_id=42825&lc_name=茅莽莽猫氓猫氓".encode() scheme, host, port, full_path = url.parse(test_url) assert scheme == b'https' @@ -60,7 +61,6 @@ b'%BD%E7%8C%AB%E6%B0%93%E7%8C%AB%E6%B0%93' -@pytest.mark.skipif(sys.version_info < (3, 6), reason='requires Python 3.6 or higher') def test_parse_port_range(): # Port out of range with pytest.raises(ValueError): @@ -115,10 +115,10 @@ post_data_empty_key_middle = [('one', 'two'), ('emptykey', ''), ('three', 'four')] post_data_empty_key_end = [('one', 'two'), ('three', 'four'), ('emptykey', '')] - assert url.encode(post_data_empty_key_middle, similar_to = reference_with_equal) == "one=two&emptykey=&three=four" - assert url.encode(post_data_empty_key_end, similar_to = reference_with_equal) == "one=two&three=four&emptykey=" - assert url.encode(post_data_empty_key_middle, similar_to = reference_without_equal) == "one=two&emptykey&three=four" - assert url.encode(post_data_empty_key_end, similar_to = reference_without_equal) == "one=two&three=four&emptykey" + assert url.encode(post_data_empty_key_middle, similar_to=reference_with_equal) == "one=two&emptykey=&three=four" + assert url.encode(post_data_empty_key_end, similar_to=reference_with_equal) == "one=two&three=four&emptykey=" + assert url.encode(post_data_empty_key_middle, similar_to=reference_without_equal) == "one=two&emptykey&three=four" + assert url.encode(post_data_empty_key_end, similar_to=reference_without_equal) == "one=two&three=four&emptykey" def test_encode(): @@ -147,3 +147,33 @@ def test_hostport(): assert url.hostport(b"https", b"foo.com", 8080) == b"foo.com:8080" + + +def test_default_port(): + assert url.default_port("http") == 80 + assert url.default_port(b"https") == 443 + assert url.default_port(b"qux") is None + + +@pytest.mark.parametrize( + "authority,valid,out", [ + ["foo:42", True, ("foo", 42)], + [b"foo:42", True, ("foo", 42)], + ["127.0.0.1:443", True, ("127.0.0.1", 443)], + ["[2001:db8:42::]:443", True, ("2001:db8:42::", 443)], + [b"xn--aaa-pla.example:80", True, ("äaaa.example", 80)], + ["foo", True, ("foo", None)], + ["foo..bar", False, ("foo..bar", None)], + ["foo:bar", False, ("foo:bar", None)], + ["foo:999999999", False, ("foo:999999999", None)], + [b"\xff", False, ('\udcff', None)] + ] +) +def test_parse_authority(authority: AnyStr, valid: bool, out): + assert parse_authority(authority, False) == out + + if valid: + assert parse_authority(authority, True) == out + else: + with pytest.raises(ValueError): + parse_authority(authority, True) diff -Nru mitmproxy-5.1.1/test/mitmproxy/net/test_check.py mitmproxy-6.0.2/test/mitmproxy/net/test_check.py --- mitmproxy-5.1.1/test/mitmproxy/net/test_check.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/net/test_check.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,5 +1,3 @@ -# coding=utf-8 - from mitmproxy.net import check @@ -69,4 +67,8 @@ assert check.is_valid_host(b'api-.a.example.com') assert check.is_valid_host(b'api-._a.example.com') assert check.is_valid_host(b'api-.a_.example.com') - assert check.is_valid_host(b'api-.ab.example.com') \ No newline at end of file + assert check.is_valid_host(b'api-.ab.example.com') + + # Test str + assert check.is_valid_host('example.tld') + assert not check.is_valid_host("foo..bar") # cannot be idna-encoded. \ No newline at end of file diff -Nru mitmproxy-5.1.1/test/mitmproxy/net/test_socks.py mitmproxy-6.0.2/test/mitmproxy/net/test_socks.py --- mitmproxy-5.1.1/test/mitmproxy/net/test_socks.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/net/test_socks.py 2020-12-15 16:41:27.000000000 +0000 @@ -163,7 +163,7 @@ def test_message_ipv6(): # Test ATYP=0x04 (IPV6) - ipv6_addr = u"2001:db8:85a3:8d3:1319:8a2e:370:7344" + ipv6_addr = "2001:db8:85a3:8d3:1319:8a2e:370:7344" raw = tutils.treader( b"\x05\x01\x00\x04" + diff -Nru mitmproxy-5.1.1/test/mitmproxy/net/test_tcp.py mitmproxy-6.0.2/test/mitmproxy/net/test_tcp.py --- mitmproxy-5.1.1/test/mitmproxy/net/test_tcp.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/net/test_tcp.py 2020-12-15 16:41:27.000000000 +0000 @@ -52,7 +52,7 @@ # Client connection is dead... if ret == "" or ret == b"": return - except socket.error: + except OSError: pass except SSL.WantReadError: pass diff -Nru mitmproxy-5.1.1/test/mitmproxy/net/test_tls.py mitmproxy-6.0.2/test/mitmproxy/net/test_tls.py --- mitmproxy-5.1.1/test/mitmproxy/net/test_tls.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/net/test_tls.py 2020-12-15 16:41:27.000000000 +0000 @@ -43,7 +43,7 @@ tls.log_master_secret.close() with open(logfile, "rb") as f: - assert f.read().count(b"CLIENT_RANDOM") >= 2 + assert f.read().count(b"SERVER_HANDSHAKE_TRAFFIC_SECRET") >= 2 tls.log_master_secret = _logfun diff -Nru mitmproxy-5.1.1/test/mitmproxy/net/test_websocket.py mitmproxy-6.0.2/test/mitmproxy/net/test_websocket.py --- mitmproxy-5.1.1/test/mitmproxy/net/test_websocket.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/net/test_websocket.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,122 @@ +import pytest +from io import BytesIO +from unittest import mock + +from wsproto.frame_protocol import Opcode, RsvBits, Header, Frame + +from mitmproxy.net import http, websocket + + +@pytest.mark.parametrize("input,masking_key,payload_length", [ + (b'\x01\rserver-foobar', None, 13), + (b'\x01\x8dasdf\x12\x16\x16\x10\x04\x01I\x00\x0e\x1c\x06\x07\x13', b'asdf', 13), + (b'\x01~\x04\x00server-foobar', None, 1024), + (b'\x01\x7f\x00\x00\x00\x00\x00\x02\x00\x00server-foobar', None, 131072), +]) +def test_read_frame(input, masking_key, payload_length): + bio = BytesIO(input) + bio.safe_read = bio.read + + header, frame, consumed_bytes = websocket.read_frame(bio) + assert header == \ + Header( + fin=False, + rsv=RsvBits(rsv1=False, rsv2=False, rsv3=False), + opcode=Opcode.TEXT, + payload_len=payload_length, + masking_key=masking_key, + ) + assert frame == \ + Frame( + opcode=Opcode.TEXT, + payload=b'server-foobar', + frame_finished=False, + message_finished=False, + ) + assert consumed_bytes == input + + bio = BytesIO(input) + bio.safe_read = bio.read + header, frame, consumed_bytes = websocket.read_frame(bio, False) + assert header is None + assert frame is None + assert consumed_bytes == input + + +@mock.patch('os.urandom', return_value=b'pumpkinspumpkins') +def test_client_handshake_headers(_): + assert websocket.client_handshake_headers() == \ + http.Headers([ + (b'connection', b'upgrade'), + (b'upgrade', b'websocket'), + (b'sec-websocket-version', b'13'), + (b'sec-websocket-key', b'cHVtcGtpbnNwdW1wa2lucw=='), + ]) + assert websocket.client_handshake_headers(b"13", b"foobar", b"foo", b"bar") == \ + http.Headers([ + (b'connection', b'upgrade'), + (b'upgrade', b'websocket'), + (b'sec-websocket-version', b'13'), + (b'sec-websocket-key', b'foobar'), + (b'sec-websocket-protocol', b'foo'), + (b'sec-websocket-extensions', b'bar') + ]) + + +def test_server_handshake_headers(): + assert websocket.server_handshake_headers("foobar", "foo", "bar") == \ + http.Headers([ + (b'connection', b'upgrade'), + (b'upgrade', b'websocket'), + (b'sec-websocket-accept', b'AzhRPA4TNwR6I/riJheN0TfR7+I='), + (b'sec-websocket-protocol', b'foo'), + (b'sec-websocket-extensions', b'bar'), + ]) + + +def test_check_handshake(): + assert not websocket.check_handshake({ + "connection": "upgrade", + "upgrade": "webFOOsocket", + "sec-websocket-key": "foo", + }) + assert websocket.check_handshake({ + "connection": "upgrade", + "upgrade": "websocket", + "sec-websocket-key": "foo", + }) + assert websocket.check_handshake({ + "connection": "upgrade", + "upgrade": "websocket", + "sec-websocket-accept": "bar", + }) + + +def test_create_server_nonce(): + assert websocket.create_server_nonce(b"foobar") == b"AzhRPA4TNwR6I/riJheN0TfR7+I=" + + +def test_check_client_version(): + assert not websocket.check_client_version({}) + assert not websocket.check_client_version({"sec-websocket-version": b"42"}) + assert websocket.check_client_version({"sec-websocket-version": b"13"}) + + +def test_get_extensions(): + assert websocket.get_extensions({}) is None + assert websocket.get_extensions({"sec-websocket-extensions": "foo"}) == "foo" + + +def test_get_protocol(): + assert websocket.get_protocol({}) is None + assert websocket.get_protocol({"sec-websocket-protocol": "foo"}) == "foo" + + +def test_get_client_key(): + assert websocket.get_client_key({}) is None + assert websocket.get_client_key({"sec-websocket-key": "foo"}) == "foo" + + +def test_get_server_accept(): + assert websocket.get_server_accept({}) is None + assert websocket.get_server_accept({"sec-websocket-accept": "foo"}) == "foo" diff -Nru mitmproxy-5.1.1/test/mitmproxy/net/test_wsgi.py mitmproxy-6.0.2/test/mitmproxy/net/test_wsgi.py --- mitmproxy-5.1.1/test/mitmproxy/net/test_wsgi.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/net/test_wsgi.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,106 +0,0 @@ -from io import BytesIO -import sys -from mitmproxy.net import wsgi -from mitmproxy.net.http import Headers - - -def tflow(): - headers = Headers(test=b"value") - req = wsgi.Request("http", "GET", "/", "HTTP/1.1", headers, "") - return wsgi.Flow(("127.0.0.1", 8888), req) - - -class ExampleApp: - - def __init__(self): - self.called = False - - def __call__(self, environ, start_response): - self.called = True - status = '200 OK' - response_headers = [('Content-type', 'text/plain')] - start_response(status, response_headers) - return [b'Hello', b' world!\n'] - - -class TestWSGI: - - def test_make_environ(self): - w = wsgi.WSGIAdaptor(None, "foo", 80, "version") - tf = tflow() - assert w.make_environ(tf, None) - - tf.request.path = "/foo?bar=voing" - r = w.make_environ(tf, None) - assert r["QUERY_STRING"] == "bar=voing" - - def test_serve(self): - ta = ExampleApp() - w = wsgi.WSGIAdaptor(ta, "foo", 80, "version") - f = tflow() - f.request.host = "foo" - f.request.port = 80 - - wfile = BytesIO() - err = w.serve(f, wfile) - assert ta.called - assert not err - - val = wfile.getvalue() - assert b"Hello world" in val - assert b"Server:" in val - - def _serve(self, app): - w = wsgi.WSGIAdaptor(app, "foo", 80, "version") - f = tflow() - f.request.host = "foo" - f.request.port = 80 - wfile = BytesIO() - w.serve(f, wfile) - return wfile.getvalue() - - def test_serve_empty_body(self): - def app(environ, start_response): - status = '200 OK' - response_headers = [('Foo', 'bar')] - start_response(status, response_headers) - return [] - assert self._serve(app) - - def test_serve_double_start(self): - def app(environ, start_response): - try: - raise ValueError("foo") - except: - sys.exc_info() - status = '200 OK' - response_headers = [('Content-type', 'text/plain')] - start_response(status, response_headers) - start_response(status, response_headers) - assert b"Internal Server Error" in self._serve(app) - - def test_serve_single_err(self): - def app(environ, start_response): - try: - raise ValueError("foo") - except: - ei = sys.exc_info() - status = '200 OK' - response_headers = [('Content-type', 'text/plain')] - start_response(status, response_headers, ei) - yield b"" - assert b"Internal Server Error" in self._serve(app) - - def test_serve_double_err(self): - def app(environ, start_response): - try: - raise ValueError("foo") - except: - ei = sys.exc_info() - status = '200 OK' - response_headers = [('Content-type', 'text/plain')] - start_response(status, response_headers) - yield b"aaa" - start_response(status, response_headers, ei) - yield b"bbb" - assert b"Internal Server Error" in self._serve(app) diff -Nru mitmproxy-5.1.1/test/mitmproxy/net/websockets/test_frame.py mitmproxy-6.0.2/test/mitmproxy/net/websockets/test_frame.py --- mitmproxy-5.1.1/test/mitmproxy/net/websockets/test_frame.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/net/websockets/test_frame.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,166 +0,0 @@ -import os -import codecs -import pytest - -from mitmproxy.net import websockets -from mitmproxy.test import tutils - - -class TestFrameHeader: - - @pytest.mark.parametrize("input,expected", [ - (0, '0100'), - (125, '017D'), - (126, '017E007E'), - (127, '017E007F'), - (142, '017E008E'), - (65534, '017EFFFE'), - (65535, '017EFFFF'), - (65536, '017F0000000000010000'), - (8589934591, '017F00000001FFFFFFFF'), - (2 ** 64 - 1, '017FFFFFFFFFFFFFFFFF'), - ]) - def test_serialization_length(self, input, expected): - h = websockets.FrameHeader( - opcode=websockets.OPCODE.TEXT, - payload_length=input, - ) - assert bytes(h) == codecs.decode(expected, 'hex') - - def test_serialization_too_large(self): - h = websockets.FrameHeader( - payload_length=2 ** 64 + 1, - ) - with pytest.raises(ValueError): - bytes(h) - - @pytest.mark.parametrize("input,expected", [ - ('0100', 0), - ('017D', 125), - ('017E007E', 126), - ('017E007F', 127), - ('017E008E', 142), - ('017EFFFE', 65534), - ('017EFFFF', 65535), - ('017F0000000000010000', 65536), - ('017F00000001FFFFFFFF', 8589934591), - ('017FFFFFFFFFFFFFFFFF', 2 ** 64 - 1), - ]) - def test_deserialization_length(self, input, expected): - h = websockets.FrameHeader.from_file(tutils.treader(codecs.decode(input, 'hex'))) - assert h.payload_length == expected - - @pytest.mark.parametrize("input,expected", [ - ('0100', (False, None)), - ('018000000000', (True, '00000000')), - ('018012345678', (True, '12345678')), - ]) - def test_deserialization_masking(self, input, expected): - h = websockets.FrameHeader.from_file(tutils.treader(codecs.decode(input, 'hex'))) - assert h.mask == expected[0] - if h.mask: - assert h.masking_key == codecs.decode(expected[1], 'hex') - - def test_equality(self): - h = websockets.FrameHeader(mask=True, masking_key=b'1234') - h2 = websockets.FrameHeader(mask=True, masking_key=b'1234') - assert h == h2 - - h = websockets.FrameHeader(fin=True) - h2 = websockets.FrameHeader(fin=False) - assert h != h2 - - assert h != 'foobar' - - def test_roundtrip(self): - def round(*args, **kwargs): - h = websockets.FrameHeader(*args, **kwargs) - h2 = websockets.FrameHeader.from_file(tutils.treader(bytes(h))) - assert h == h2 - - round() - round(fin=True) - round(rsv1=True) - round(rsv2=True) - round(rsv3=True) - round(payload_length=1) - round(payload_length=100) - round(payload_length=1000) - round(payload_length=10000) - round(opcode=websockets.OPCODE.PING) - round(masking_key=b"test") - - def test_human_readable(self): - f = websockets.FrameHeader( - masking_key=b"test", - fin=True, - payload_length=10 - ) - assert repr(f) - - f = websockets.FrameHeader() - assert repr(f) - - def test_funky(self): - f = websockets.FrameHeader(masking_key=b"test", mask=False) - raw = bytes(f) - f2 = websockets.FrameHeader.from_file(tutils.treader(raw)) - assert not f2.mask - - def test_violations(self): - with pytest.raises(Exception, match="opcode"): - websockets.FrameHeader(opcode=17) - with pytest.raises(Exception, match="Masking key"): - websockets.FrameHeader(masking_key=b"x") - - def test_automask(self): - f = websockets.FrameHeader(mask=True) - assert f.masking_key - - f = websockets.FrameHeader(masking_key=b"foob") - assert f.mask - - f = websockets.FrameHeader(masking_key=b"foob", mask=0) - assert not f.mask - assert not f.masking_key - - -class TestFrame: - def test_equality(self): - f = websockets.Frame(payload=b'1234') - f2 = websockets.Frame(payload=b'1234') - assert f == f2 - - assert f != b'1234' - - def test_roundtrip(self): - def round(*args, **kwargs): - f = websockets.Frame(*args, **kwargs) - raw = bytes(f) - f2 = websockets.Frame.from_file(tutils.treader(raw)) - assert f == f2 - round(b"test") - round(b"test", fin=1) - round(b"test", rsv1=1) - round(b"test", opcode=websockets.OPCODE.PING) - round(b"test", masking_key=b"test") - - def test_human_readable(self): - f = websockets.Frame() - assert repr(f) - - f = websockets.Frame(b"foobar") - assert "foobar" in repr(f) - - @pytest.mark.parametrize("masked", [True, False]) - @pytest.mark.parametrize("length", [100, 50000, 150000]) - def test_serialization_bijection(self, masked, length): - frame = websockets.Frame( - os.urandom(length), - fin=True, - opcode=websockets.OPCODE.TEXT, - mask=int(masked), - masking_key=(os.urandom(4) if masked else None) - ) - serialized = bytes(frame) - assert frame == websockets.Frame.from_bytes(serialized) diff -Nru mitmproxy-5.1.1/test/mitmproxy/net/websockets/test_masker.py mitmproxy-6.0.2/test/mitmproxy/net/websockets/test_masker.py --- mitmproxy-5.1.1/test/mitmproxy/net/websockets/test_masker.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/net/websockets/test_masker.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,23 +0,0 @@ -import codecs -import pytest - -from mitmproxy.net import websockets - - -class TestMasker: - - @pytest.mark.parametrize("input,expected", [ - ([b"a"], '00'), - ([b"four"], '070d1616'), - ([b"fourf"], '070d161607'), - ([b"fourfive"], '070d1616070b1501'), - ([b"a", b"aasdfasdfa", b"asdf"], '000302170504021705040205120605'), - ([b"a" * 50, b"aasdfasdfa", b"asdf"], '00030205000302050003020500030205000302050003020500030205000302050003020500030205000302050003020500030205120605051206050500110702'), # noqa - ]) - def test_masker(self, input, expected): - m = websockets.Masker(b"abcd") - data = b"".join([m(t) for t in input]) - assert data == codecs.decode(expected, 'hex') - - data = websockets.Masker(b"abcd")(data) - assert data == b"".join(input) diff -Nru mitmproxy-5.1.1/test/mitmproxy/net/websockets/test_utils.py mitmproxy-6.0.2/test/mitmproxy/net/websockets/test_utils.py --- mitmproxy-5.1.1/test/mitmproxy/net/websockets/test_utils.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/net/websockets/test_utils.py 1970-01-01 00:00:00.000000000 +0000 @@ -1,105 +0,0 @@ -import pytest - -from mitmproxy.net import http -from mitmproxy.net import websockets - - -class TestUtils: - - def test_client_handshake_headers(self): - h = websockets.client_handshake_headers(version='42') - assert h['sec-websocket-version'] == '42' - - h = websockets.client_handshake_headers(key='some-key') - assert h['sec-websocket-key'] == 'some-key' - - h = websockets.client_handshake_headers(protocol='foobar') - assert h['sec-websocket-protocol'] == 'foobar' - - h = websockets.client_handshake_headers(extensions='foo; bar') - assert h['sec-websocket-extensions'] == 'foo; bar' - - def test_server_handshake_headers(self): - h = websockets.server_handshake_headers('some-key') - assert h['sec-websocket-accept'] == '8iILEZtcVdtFD7MDlPKip9ec9nw=' - assert 'sec-websocket-protocol' not in h - assert 'sec-websocket-extensions' not in h - - h = websockets.server_handshake_headers('some-key', 'foobar', 'foo; bar') - assert h['sec-websocket-accept'] == '8iILEZtcVdtFD7MDlPKip9ec9nw=' - assert h['sec-websocket-protocol'] == 'foobar' - assert h['sec-websocket-extensions'] == 'foo; bar' - - @pytest.mark.parametrize("input,expected", [ - ([(b'connection', b'upgrade'), (b'upgrade', b'websocket'), (b'sec-websocket-key', b'foobar')], True), - ([(b'connection', b'upgrade'), (b'upgrade', b'websocket'), (b'sec-websocket-accept', b'foobar')], True), - ([(b'Connection', b'UpgRaDe'), (b'Upgrade', b'WebSocKeT'), (b'Sec-WebSockeT-KeY', b'foobar')], True), - ([(b'Connection', b'UpgRaDe'), (b'Upgrade', b'WebSocKeT'), (b'Sec-WebSockeT-AccePt', b'foobar')], True), - ([(b'connection', b'foo'), (b'upgrade', b'bar'), (b'sec-websocket-key', b'foobar')], False), - ([(b'connection', b'upgrade'), (b'upgrade', b'websocket')], False), - ([(b'connection', b'upgrade'), (b'sec-websocket-key', b'foobar')], False), - ([(b'upgrade', b'websocket'), (b'sec-websocket-key', b'foobar')], False), - ([], False), - ]) - def test_check_handshake(self, input, expected): - h = http.Headers(input) - assert websockets.check_handshake(h) == expected - - @pytest.mark.parametrize("input,expected", [ - ([(b'sec-websocket-version', b'13')], True), - ([(b'Sec-WebSockeT-VerSion', b'13')], True), - ([(b'sec-websocket-version', b'9')], False), - ([(b'sec-websocket-version', b'42')], False), - ([(b'sec-websocket-version', b'')], False), - ([], False), - ]) - def test_check_client_version(self, input, expected): - h = http.Headers(input) - assert websockets.check_client_version(h) == expected - - @pytest.mark.parametrize("input,expected", [ - ('foobar', b'AzhRPA4TNwR6I/riJheN0TfR7+I='), - (b'foobar', b'AzhRPA4TNwR6I/riJheN0TfR7+I='), - ]) - def test_create_server_nonce(self, input, expected): - assert websockets.create_server_nonce(input) == expected - - @pytest.mark.parametrize("input,expected", [ - ([(b'sec-websocket-extensions', b'foo; bar')], 'foo; bar'), - ([(b'Sec-WebSockeT-ExteNsionS', b'foo; bar')], 'foo; bar'), - ([(b'sec-websocket-extensions', b'')], ''), - ([], None), - ]) - def test_get_extensions(self, input, expected): - h = http.Headers(input) - assert websockets.get_extensions(h) == expected - - @pytest.mark.parametrize("input,expected", [ - ([(b'sec-websocket-protocol', b'foobar')], 'foobar'), - ([(b'Sec-WebSockeT-ProTocoL', b'foobar')], 'foobar'), - ([(b'sec-websocket-protocol', b'')], ''), - ([], None), - ]) - def test_get_protocol(self, input, expected): - h = http.Headers(input) - assert websockets.get_protocol(h) == expected - - @pytest.mark.parametrize("input,expected", [ - ([(b'sec-websocket-key', b'foobar')], 'foobar'), - ([(b'Sec-WebSockeT-KeY', b'foobar')], 'foobar'), - ([(b'sec-websocket-key', b'')], ''), - ([], None), - ]) - def test_get_client_key(self, input, expected): - h = http.Headers(input) - assert websockets.get_client_key(h) == expected - - @pytest.mark.parametrize("input,expected", [ - ([(b'sec-websocket-accept', b'foobar')], 'foobar'), - ([(b'Sec-WebSockeT-AccepT', b'foobar')], 'foobar'), - ([(b'sec-websocket-accept', b'')], ''), - ([], None), - ]) - def test_get_server_accept(self, input, expected): - h = http.Headers(input) - assert websockets.get_server_accept(h) == expected diff -Nru mitmproxy-5.1.1/test/mitmproxy/proxy/protocol/test_http1.py mitmproxy-6.0.2/test/mitmproxy/proxy/protocol/test_http1.py --- mitmproxy-5.1.1/test/mitmproxy/proxy/protocol/test_http1.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/proxy/protocol/test_http1.py 2020-12-15 16:41:27.000000000 +0000 @@ -21,7 +21,7 @@ def test_double_connect(self): p = self.pathoc() with p.connect(): - r = p.request("connect:'%s:%s'" % ("127.0.0.1", self.server2.port)) + r = p.request("connect:'{}:{}'".format("127.0.0.1", self.server2.port)) assert r.status_code == 400 assert b"Unexpected CONNECT" in r.content @@ -59,6 +59,7 @@ client.wfile.flush() assert client.rfile.readline() == b"HTTP/1.1 100 Continue\r\n" + assert client.rfile.readline() == b"content-length: 0\r\n" assert client.rfile.readline() == b"\r\n" client.wfile.write(b"0123456789abcdef\r\n") diff -Nru mitmproxy-5.1.1/test/mitmproxy/proxy/protocol/test_http2.py mitmproxy-6.0.2/test/mitmproxy/proxy/protocol/test_http2.py --- mitmproxy-5.1.1/test/mitmproxy/proxy/protocol/test_http2.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/proxy/protocol/test_http2.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,6 +1,3 @@ -# coding=utf-8 - - import os import tempfile import traceback @@ -10,6 +7,7 @@ from mitmproxy import options import mitmproxy.net +import mitmproxy.http from ...net import tservers as net_tservers from mitmproxy import exceptions from mitmproxy.net.http import http1, http2 @@ -55,8 +53,8 @@ done = False while not done: try: - raw = b''.join(http2.read_raw_frame(self.rfile)) - events = h2_conn.receive_data(raw) + _, consumed_bytes = http2.read_frame(self.rfile, False) + events = h2_conn.receive_data(consumed_bytes) except exceptions.HttpException: print(traceback.format_exc()) assert False @@ -124,17 +122,9 @@ self.client.connect() # send CONNECT request - self.client.wfile.write(http1.assemble_request(mitmproxy.net.http.Request( - 'authority', - b'CONNECT', - b'', - b'localhost', - self.server.server.address[1], - b'/', - b'HTTP/1.1', - [(b'host', b'localhost:%d' % self.server.server.address[1])], - b'', - ))) + self.client.wfile.write(http1.assemble_request( + mitmproxy.http.make_connect_request(("localhost", self.server.server.address[1])) + )) self.client.wfile.flush() # read CONNECT response @@ -199,7 +189,7 @@ _Http2ServerBase.teardown_class() -class TestSimple(_Http2Test): +class TestSimpleRequestWithBody(_Http2Test): request_body_buffer = b'' @classmethod @@ -233,7 +223,7 @@ cls.request_body_buffer += event.data return True - def test_simple(self): + def test_simple_request_with_body(self): response_body_buffer = b'' h2_conn = self.setup_connection() @@ -253,8 +243,8 @@ done = False while not done: try: - raw = b''.join(http2.read_raw_frame(self.client.rfile)) - events = h2_conn.receive_data(raw) + _, consumed_bytes = http2.read_frame(self.client.rfile, False) + events = h2_conn.receive_data(consumed_bytes) except exceptions.HttpException: print(traceback.format_exc()) assert False @@ -281,6 +271,77 @@ assert response_body_buffer == b'response body' +class TestSimpleRequestWithoutBody(_Http2Test): + @classmethod + def handle_server_event(cls, event, h2_conn, rfile, wfile): + if isinstance(event, h2.events.ConnectionTerminated): + return False + elif isinstance(event, h2.events.RequestReceived): + assert (b'self.client-foo', b'self.client-bar-1') in event.headers + elif isinstance(event, h2.events.StreamEnded): + import warnings + with warnings.catch_warnings(): + # Ignore UnicodeWarning: + # h2/utilities.py:64: UnicodeWarning: Unicode equal comparison + # failed to convert both arguments to Unicode - interpreting + # them as being unequal. + # elif header[0] in (b'cookie', u'cookie') and len(header[1]) < 20: + + warnings.simplefilter("ignore") + h2_conn.send_headers(event.stream_id, [ + (':status', '200'), + ]) + h2_conn.send_data(event.stream_id, b'response body') + h2_conn.end_stream(event.stream_id) + wfile.write(h2_conn.data_to_send()) + wfile.flush() + elif isinstance(event, h2.events.DataReceived): + return False + return True + + def test_simple(self): + response_body_buffer = b'' + h2_conn = self.setup_connection() + + self._send_request( + self.client.wfile, + h2_conn, + headers=[ + (':authority', "127.0.0.1:{}".format(self.server.server.address[1])), + (':method', 'GET'), + (':scheme', 'https'), + (':path', '/'), + ('self.client-FoO', 'self.client-bar-1'), + ]) + + done = False + while not done: + try: + _, consumed_bytes = http2.read_frame(self.client.rfile, False) + events = h2_conn.receive_data(consumed_bytes) + except exceptions.HttpException: + print(traceback.format_exc()) + assert False + + self.client.wfile.write(h2_conn.data_to_send()) + self.client.wfile.flush() + + for event in events: + if isinstance(event, h2.events.DataReceived): + response_body_buffer += event.data + elif isinstance(event, h2.events.StreamEnded): + done = True + + h2_conn.close_connection() + self.client.wfile.write(h2_conn.data_to_send()) + self.client.wfile.flush() + + assert len(self.master.state.flows) == 1 + assert self.master.state.flows[0].response.status_code == 200 + assert self.master.state.flows[0].response.content == b'response body' + assert response_body_buffer == b'response body' + + class TestRequestWithPriority(_Http2Test): @classmethod @@ -337,8 +398,8 @@ done = False while not done: try: - raw = b''.join(http2.read_raw_frame(self.client.rfile)) - events = h2_conn.receive_data(raw) + _, consumed_bytes = http2.read_frame(self.client.rfile, False) + events = h2_conn.receive_data(consumed_bytes) except exceptions.HttpException: print(traceback.format_exc()) assert False @@ -425,8 +486,8 @@ done = False while not done: try: - raw = b''.join(http2.read_raw_frame(self.client.rfile)) - events = h2_conn.receive_data(raw) + _, consumed_bytes = http2.read_frame(self.client.rfile, False) + events = h2_conn.receive_data(consumed_bytes) except exceptions.HttpException: print(traceback.format_exc()) assert False @@ -475,8 +536,8 @@ done = False while not done: try: - raw = b''.join(http2.read_raw_frame(self.client.rfile)) - events = h2_conn.receive_data(raw) + _, consumed_bytes = http2.read_frame(self.client.rfile, False) + events = h2_conn.receive_data(consumed_bytes) except exceptions.HttpException: print(traceback.format_exc()) assert False @@ -542,8 +603,8 @@ done = False while not done: try: - raw = b''.join(http2.read_raw_frame(self.client.rfile)) - events = h2_conn.receive_data(raw) + _, consumed_bytes = http2.read_frame(self.client.rfile, False) + events = h2_conn.receive_data(consumed_bytes) except exceptions.HttpException: print(traceback.format_exc()) assert False @@ -589,8 +650,8 @@ done = False while not done: try: - raw = b''.join(http2.read_raw_frame(self.client.rfile)) - events = h2_conn.receive_data(raw) + _, consumed_bytes = http2.read_frame(self.client.rfile, False) + events = h2_conn.receive_data(consumed_bytes) except exceptions.HttpException: print(traceback.format_exc()) assert False @@ -625,7 +686,7 @@ wfile.flush() h2_conn.push_stream(1, 2, [ - (':authority', "127.0.0.1:{}".format(cls.port)), + (':authority', f"127.0.0.1:{cls.port}"), (':method', 'GET'), (':scheme', 'https'), (':path', '/pushed_stream_foo'), @@ -635,7 +696,7 @@ wfile.flush() h2_conn.push_stream(1, 4, [ - (':authority', "127.0.0.1:{}".format(cls.port)), + (':authority', f"127.0.0.1:{cls.port}"), (':method', 'GET'), (':scheme', 'https'), (':path', '/pushed_stream_bar'), @@ -686,8 +747,8 @@ responses = 0 while not done: try: - raw = b''.join(http2.read_raw_frame(self.client.rfile)) - events = h2_conn.receive_data(raw) + _, consumed_bytes = http2.read_frame(self.client.rfile, False) + events = h2_conn.receive_data(consumed_bytes) except exceptions.HttpException: print(traceback.format_exc()) assert False @@ -742,8 +803,8 @@ responses = 0 while not done: try: - raw = b''.join(http2.read_raw_frame(self.client.rfile)) - events = h2_conn.receive_data(raw) + _, consumed_bytes = http2.read_frame(self.client.rfile, False) + events = h2_conn.receive_data(consumed_bytes) except exceptions.HttpException: print(traceback.format_exc()) assert False @@ -801,8 +862,8 @@ done = False while not done: try: - raw = b''.join(http2.read_raw_frame(self.client.rfile)) - h2_conn.receive_data(raw) + _, consumed_bytes = http2.read_frame(self.client.rfile, False) + h2_conn.receive_data(consumed_bytes) except exceptions.HttpException: print(traceback.format_exc()) assert False @@ -834,7 +895,7 @@ (':status', '200'), ('X-Stream-ID', str(event.stream_id)), ]) - h2_conn.send_data(event.stream_id, 'Stream-ID {}'.format(event.stream_id).encode()) + h2_conn.send_data(event.stream_id, f'Stream-ID {event.stream_id}'.encode()) h2_conn.end_stream(event.stream_id) wfile.write(h2_conn.data_to_send()) wfile.flush() @@ -857,8 +918,8 @@ ended_streams = 0 while ended_streams != len(new_streams): try: - header, body = http2.read_raw_frame(self.client.rfile) - events = h2_conn.receive_data(b''.join([header, body])) + _, consumed_bytes = http2.read_frame(self.client.rfile, False) + events = h2_conn.receive_data(consumed_bytes) except: break self.client.wfile.write(h2_conn.data_to_send()) @@ -902,8 +963,8 @@ connection_terminated_event = None while not done: try: - raw = b''.join(http2.read_raw_frame(self.client.rfile)) - events = h2_conn.receive_data(raw) + _, consumed_bytes = http2.read_frame(self.client.rfile, False) + events = h2_conn.receive_data(consumed_bytes) for event in events: if isinstance(event, h2.events.ConnectionTerminated): connection_terminated_event = event @@ -960,15 +1021,21 @@ self.client.rfile.o.settimeout(2) while not done: try: - raw = b''.join(http2.read_raw_frame(self.client.rfile)) - events = h2_conn.receive_data(raw) + _, consumed_bytes = http2.read_frame(self.client.rfile, False) + events = h2_conn.receive_data(consumed_bytes) for event in events: if isinstance(event, h2.events.ConnectionTerminated): connection_terminated_event = event done = True + except mitmproxy.exceptions.TcpTimeout: + if not streaming: + break # this is expected for this test case + else: + assert False except: - break + print(traceback.format_exc()) + assert False if streaming: assert connection_terminated_event.additional_data == body @@ -1017,8 +1084,8 @@ data = None while not done: try: - raw = b''.join(http2.read_raw_frame(self.client.rfile)) - events = h2_conn.receive_data(raw) + _, consumed_bytes = http2.read_frame(self.client.rfile, False) + events = h2_conn.receive_data(consumed_bytes) for event in events: if isinstance(event, h2.events.DataReceived): @@ -1031,3 +1098,147 @@ assert data else: assert data is None + + +class TestRequestTrailers(_Http2Test): + server_trailers_received = False + + @classmethod + def handle_server_event(cls, event, h2_conn, rfile, wfile): + if isinstance(event, h2.events.RequestReceived): + # reset the value for a fresh test + cls.server_trailers_received = False + elif isinstance(event, h2.events.ConnectionTerminated): + return False + elif isinstance(event, h2.events.TrailersReceived): + cls.server_trailers_received = True + + elif isinstance(event, h2.events.StreamEnded): + h2_conn.send_headers(event.stream_id, [ + (':status', '200'), + ('x-my-trailer-request-received', 'success' if cls.server_trailers_received else "failure"), + ], end_stream=True) + wfile.write(h2_conn.data_to_send()) + wfile.flush() + return True + + @pytest.mark.parametrize('announce', [True, False]) + @pytest.mark.parametrize('body', [None, b"foobar"]) + def test_trailers(self, announce, body): + h2_conn = self.setup_connection() + stream_id = 1 + headers = [ + (':authority', "127.0.0.1:{}".format(self.server.server.address[1])), + (':method', 'GET'), + (':scheme', 'https'), + (':path', '/'), + ] + if announce: + headers.append(('trailer', 'x-my-trailers')) + h2_conn.send_headers( + stream_id=stream_id, + headers=headers, + ) + if body: + h2_conn.send_data(stream_id, body) + + # send trailers + h2_conn.send_headers(stream_id, [('x-my-trailers', 'foobar')], end_stream=True) + + self.client.wfile.write(h2_conn.data_to_send()) + self.client.wfile.flush() + + done = False + while not done: + try: + _, consumed_bytes = http2.read_frame(self.client.rfile, False) + events = h2_conn.receive_data(consumed_bytes) + except exceptions.HttpException: + print(traceback.format_exc()) + assert False + + self.client.wfile.write(h2_conn.data_to_send()) + self.client.wfile.flush() + + for event in events: + if isinstance(event, h2.events.StreamEnded): + done = True + + h2_conn.close_connection() + self.client.wfile.write(h2_conn.data_to_send()) + self.client.wfile.flush() + + assert len(self.master.state.flows) == 1 + assert self.master.state.flows[0].request.trailers['x-my-trailers'] == 'foobar' + assert self.master.state.flows[0].response.status_code == 200 + assert self.master.state.flows[0].response.headers['x-my-trailer-request-received'] == 'success' + + +class TestResponseTrailers(_Http2Test): + + @classmethod + def handle_server_event(cls, event, h2_conn, rfile, wfile): + if isinstance(event, h2.events.ConnectionTerminated): + return False + elif isinstance(event, h2.events.StreamEnded): + headers = [ + (':status', '200'), + ] + if event.stream_id == 1: + # special stream_id to activate the Trailer announcement header + headers.append(('trailer', 'x-my-trailers')) + + h2_conn.send_headers(event.stream_id, headers) + h2_conn.send_data(event.stream_id, b'response body') + h2_conn.send_headers(event.stream_id, [('x-my-trailers', 'foobar')], end_stream=True) + wfile.write(h2_conn.data_to_send()) + wfile.flush() + return True + + @pytest.mark.parametrize('announce', [True, False]) + def test_trailers(self, announce): + response_body_buffer = b'' + h2_conn = self.setup_connection() + + self._send_request( + self.client.wfile, + h2_conn, + stream_id=(1 if announce else 3), + headers=[ + (':authority', "127.0.0.1:{}".format(self.server.server.address[1])), + (':method', 'GET'), + (':scheme', 'https'), + (':path', '/'), + ]) + + trailers_buffer = None + done = False + while not done: + try: + _, consumed_bytes = http2.read_frame(self.client.rfile, False) + events = h2_conn.receive_data(consumed_bytes) + except exceptions.HttpException: + print(traceback.format_exc()) + assert False + + self.client.wfile.write(h2_conn.data_to_send()) + self.client.wfile.flush() + + for event in events: + if isinstance(event, h2.events.DataReceived): + response_body_buffer += event.data + elif isinstance(event, h2.events.TrailersReceived): + trailers_buffer = event.headers + elif isinstance(event, h2.events.StreamEnded): + done = True + + h2_conn.close_connection() + self.client.wfile.write(h2_conn.data_to_send()) + self.client.wfile.flush() + + assert len(self.master.state.flows) == 1 + assert self.master.state.flows[0].response.status_code == 200 + assert self.master.state.flows[0].response.content == b'response body' + assert response_body_buffer == b'response body' + assert self.master.state.flows[0].response.trailers['x-my-trailers'] == 'foobar' + assert trailers_buffer == [(b'x-my-trailers', b'foobar')] diff -Nru mitmproxy-5.1.1/test/mitmproxy/proxy/protocol/test_websocket.py mitmproxy-6.0.2/test/mitmproxy/proxy/protocol/test_websocket.py --- mitmproxy-5.1.1/test/mitmproxy/proxy/protocol/test_websocket.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/proxy/protocol/test_websocket.py 2020-12-15 16:41:27.000000000 +0000 @@ -4,18 +4,18 @@ import tempfile import traceback -from mitmproxy import options -from mitmproxy import exceptions -from mitmproxy.http import HTTPFlow +from wsproto.frame_protocol import Opcode + +from mitmproxy import exceptions, options +from mitmproxy.http import HTTPFlow, make_connect_request from mitmproxy.websocket import WebSocketFlow +from mitmproxy.net import http, tcp, websocket + +from pathod.language import websockets_frame -from mitmproxy.net import tcp -from mitmproxy.net import http from ...net import tservers as net_tservers from ... import tservers -from mitmproxy.net import websockets - class _WebSocketServerBase(net_tservers.ServerTestBase): @@ -24,12 +24,12 @@ def handle(self): try: request = http.http1.read_request(self.rfile) - assert websockets.check_handshake(request.headers) + assert websocket.check_handshake(request.headers) response = http.Response( - "HTTP/1.1", - 101, - reason=http.status_codes.RESPONSES.get(101), + http_version=b"HTTP/1.1", + status_code=101, + reason=http.status_codes.RESPONSES.get(101).encode(), headers=http.Headers( connection='upgrade', upgrade='websocket', @@ -37,6 +37,9 @@ sec_websocket_extensions='permessage-deflate' if "permessage-deflate" in request.headers.values() else '' ), content=b'', + trailers=None, + timestamp_start=0, + timestamp_end=0, ) self.wfile.write(http.http1.assemble_response(response)) self.wfile.flush() @@ -86,15 +89,7 @@ self.client = tcp.TCPClient(("127.0.0.1", self.proxy.port)) self.client.connect() - request = http.Request( - "authority", - "CONNECT", - "", - "127.0.0.1", - self.server.server.address[1], - "", - "HTTP/1.1", - content=b'') + request = make_connect_request(("127.0.0.1", self.server.server.address[1])) self.client.wfile.write(http.http1.assemble_request(request)) self.client.wfile.flush() @@ -105,13 +100,13 @@ assert self.client.tls_established request = http.Request( - "relative", - "GET", - "http", - "127.0.0.1", - self.server.server.address[1], - "/ws", - "HTTP/1.1", + host="127.0.0.1", + port=self.server.server.address[1], + method=b"GET", + scheme=b"http", + authority=b"", + path=b"/ws", + http_version=b"HTTP/1.1", headers=http.Headers( connection="upgrade", upgrade="websocket", @@ -119,12 +114,16 @@ sec_websocket_key="1234", sec_websocket_extensions="permessage-deflate" if extension else "" ), - content=b'') + content=b'', + trailers=None, + timestamp_start=0, + timestamp_end=0, + ) self.client.wfile.write(http.http1.assemble_request(request)) self.client.wfile.flush() response = http.http1.read_response(self.client.rfile, request) - assert websockets.check_handshake(response.headers) + assert websocket.check_handshake(response.headers) class _WebSocketTest(_WebSocketTestBase, _WebSocketServerBase): @@ -144,15 +143,15 @@ @classmethod def handle_websockets(cls, rfile, wfile): - wfile.write(bytes(websockets.Frame(fin=1, opcode=websockets.OPCODE.TEXT, payload=b'server-foobar'))) + wfile.write(bytes(websockets_frame.Frame(fin=1, opcode=Opcode.TEXT, payload=b'server-foobar'))) wfile.flush() - frame = websockets.Frame.from_file(rfile) - wfile.write(bytes(websockets.Frame(fin=1, opcode=frame.header.opcode, payload=frame.payload))) + header, frame, _ = websocket.read_frame(rfile) + wfile.write(bytes(websockets_frame.Frame(fin=1, opcode=header.opcode, payload=frame.payload))) wfile.flush() - frame = websockets.Frame.from_file(rfile) - wfile.write(bytes(websockets.Frame(fin=1, opcode=frame.header.opcode, payload=frame.payload))) + header, frame, _ = websocket.read_frame(rfile) + wfile.write(bytes(websockets_frame.Frame(fin=1, opcode=header.opcode, payload=frame.payload))) wfile.flush() @pytest.mark.parametrize('streaming', [True, False]) @@ -164,22 +163,22 @@ self.proxy.set_addons(Stream()) self.setup_connection() - frame = websockets.Frame.from_file(self.client.rfile) + _, frame, _ = websocket.read_frame(self.client.rfile) assert frame.payload == b'server-foobar' - self.client.wfile.write(bytes(websockets.Frame(fin=1, mask=1, opcode=websockets.OPCODE.TEXT, payload=b'self.client-foobar'))) + self.client.wfile.write(bytes(websockets_frame.Frame(fin=1, mask=1, opcode=Opcode.TEXT, payload=b'self.client-foobar'))) self.client.wfile.flush() - frame = websockets.Frame.from_file(self.client.rfile) + _, frame, _ = websocket.read_frame(self.client.rfile) assert frame.payload == b'self.client-foobar' - self.client.wfile.write(bytes(websockets.Frame(fin=1, mask=1, opcode=websockets.OPCODE.BINARY, payload=b'\xde\xad\xbe\xef'))) + self.client.wfile.write(bytes(websockets_frame.Frame(fin=1, mask=1, opcode=Opcode.BINARY, payload=b'\xde\xad\xbe\xef'))) self.client.wfile.flush() - frame = websockets.Frame.from_file(self.client.rfile) + _, frame, _ = websocket.read_frame(self.client.rfile) assert frame.payload == b'\xde\xad\xbe\xef' - self.client.wfile.write(bytes(websockets.Frame(fin=1, mask=1, opcode=websockets.OPCODE.CLOSE))) + self.client.wfile.write(bytes(websockets_frame.Frame(fin=1, mask=1, opcode=Opcode.CLOSE))) self.client.wfile.flush() assert len(self.master.state.flows) == 2 @@ -187,15 +186,15 @@ assert isinstance(self.master.state.flows[1], WebSocketFlow) assert len(self.master.state.flows[1].messages) == 5 assert self.master.state.flows[1].messages[0].content == 'server-foobar' - assert self.master.state.flows[1].messages[0].type == websockets.OPCODE.TEXT + assert self.master.state.flows[1].messages[0].type == Opcode.TEXT assert self.master.state.flows[1].messages[1].content == 'self.client-foobar' - assert self.master.state.flows[1].messages[1].type == websockets.OPCODE.TEXT + assert self.master.state.flows[1].messages[1].type == Opcode.TEXT assert self.master.state.flows[1].messages[2].content == 'self.client-foobar' - assert self.master.state.flows[1].messages[2].type == websockets.OPCODE.TEXT + assert self.master.state.flows[1].messages[2].type == Opcode.TEXT assert self.master.state.flows[1].messages[3].content == b'\xde\xad\xbe\xef' - assert self.master.state.flows[1].messages[3].type == websockets.OPCODE.BINARY + assert self.master.state.flows[1].messages[3].type == Opcode.BINARY assert self.master.state.flows[1].messages[4].content == b'\xde\xad\xbe\xef' - assert self.master.state.flows[1].messages[4].type == websockets.OPCODE.BINARY + assert self.master.state.flows[1].messages[4].type == Opcode.BINARY def test_change_payload(self): class Addon: @@ -205,19 +204,19 @@ self.proxy.set_addons(Addon()) self.setup_connection() - frame = websockets.Frame.from_file(self.client.rfile) + _, frame, _ = websocket.read_frame(self.client.rfile) assert frame.payload == b'foo' - self.client.wfile.write(bytes(websockets.Frame(fin=1, mask=1, opcode=websockets.OPCODE.TEXT, payload=b'self.client-foobar'))) + self.client.wfile.write(bytes(websockets_frame.Frame(fin=1, mask=1, opcode=Opcode.TEXT, payload=b'self.client-foobar'))) self.client.wfile.flush() - frame = websockets.Frame.from_file(self.client.rfile) + _, frame, _ = websocket.read_frame(self.client.rfile) assert frame.payload == b'foo' - self.client.wfile.write(bytes(websockets.Frame(fin=1, mask=1, opcode=websockets.OPCODE.BINARY, payload=b'\xde\xad\xbe\xef'))) + self.client.wfile.write(bytes(websockets_frame.Frame(fin=1, mask=1, opcode=Opcode.BINARY, payload=b'\xde\xad\xbe\xef'))) self.client.wfile.flush() - frame = websockets.Frame.from_file(self.client.rfile) + _, frame, _ = websocket.read_frame(self.client.rfile) assert frame.payload == b'foo' @@ -225,7 +224,7 @@ @classmethod def handle_websockets(cls, rfile, wfile): - wfile.write(bytes(websockets.Frame(fin=1, opcode=websockets.OPCODE.TEXT, payload=b'server-foobar'))) + wfile.write(bytes(websockets_frame.Frame(fin=1, opcode=Opcode.TEXT, payload=b'server-foobar'))) wfile.flush() def test_kill(self): @@ -237,7 +236,7 @@ self.setup_connection() with pytest.raises(exceptions.TcpDisconnect): - websockets.Frame.from_file(self.client.rfile) + _ = websocket.read_frame(self.client.rfile, False) class TestSimpleTLS(_WebSocketTest): @@ -245,26 +244,26 @@ @classmethod def handle_websockets(cls, rfile, wfile): - wfile.write(bytes(websockets.Frame(fin=1, opcode=websockets.OPCODE.TEXT, payload=b'server-foobar'))) + wfile.write(bytes(websockets_frame.Frame(fin=1, opcode=Opcode.TEXT, payload=b'server-foobar'))) wfile.flush() - frame = websockets.Frame.from_file(rfile) - wfile.write(bytes(websockets.Frame(fin=1, opcode=frame.header.opcode, payload=frame.payload))) + header, frame, _ = websocket.read_frame(rfile) + wfile.write(bytes(websockets_frame.Frame(fin=1, opcode=header.opcode, payload=frame.payload))) wfile.flush() def test_simple_tls(self): self.setup_connection() - frame = websockets.Frame.from_file(self.client.rfile) + _, frame, _ = websocket.read_frame(self.client.rfile) assert frame.payload == b'server-foobar' - self.client.wfile.write(bytes(websockets.Frame(fin=1, mask=1, opcode=websockets.OPCODE.TEXT, payload=b'self.client-foobar'))) + self.client.wfile.write(bytes(websockets_frame.Frame(fin=1, mask=1, opcode=Opcode.TEXT, payload=b'self.client-foobar'))) self.client.wfile.flush() - frame = websockets.Frame.from_file(self.client.rfile) + _, frame, _ = websocket.read_frame(self.client.rfile) assert frame.payload == b'self.client-foobar' - self.client.wfile.write(bytes(websockets.Frame(fin=1, mask=1, opcode=websockets.OPCODE.CLOSE))) + self.client.wfile.write(bytes(websockets_frame.Frame(fin=1, mask=1, opcode=Opcode.CLOSE))) self.client.wfile.flush() @@ -272,29 +271,29 @@ @classmethod def handle_websockets(cls, rfile, wfile): - wfile.write(bytes(websockets.Frame(fin=1, opcode=websockets.OPCODE.PING, payload=b'foobar'))) + wfile.write(bytes(websockets_frame.Frame(fin=1, opcode=Opcode.PING, payload=b'foobar'))) wfile.flush() - frame = websockets.Frame.from_file(rfile) - assert frame.header.opcode == websockets.OPCODE.PONG + header, frame, _ = websocket.read_frame(rfile) + assert header.opcode == Opcode.PONG assert frame.payload == b'foobar' - wfile.write(bytes(websockets.Frame(fin=1, opcode=websockets.OPCODE.PONG, payload=b'done'))) + wfile.write(bytes(websockets_frame.Frame(fin=1, opcode=Opcode.PONG, payload=b'done'))) wfile.flush() - wfile.write(bytes(websockets.Frame(fin=1, opcode=websockets.OPCODE.CLOSE))) + wfile.write(bytes(websockets_frame.Frame(fin=1, opcode=Opcode.CLOSE))) wfile.flush() - websockets.Frame.from_file(rfile) + _ = websocket.read_frame(rfile, False) @pytest.mark.asyncio async def test_ping(self): self.setup_connection() - frame = websockets.Frame.from_file(self.client.rfile) - websockets.Frame.from_file(self.client.rfile) - self.client.wfile.write(bytes(websockets.Frame(fin=1, mask=1, opcode=websockets.OPCODE.CLOSE))) + header, frame, _ = websocket.read_frame(self.client.rfile) + _ = websocket.read_frame(self.client.rfile, False) + self.client.wfile.write(bytes(websockets_frame.Frame(fin=1, mask=1, opcode=Opcode.CLOSE))) self.client.wfile.flush() - assert frame.header.opcode == websockets.OPCODE.PING + assert header.opcode == Opcode.PING assert frame.payload == b'' # We don't send payload to other end assert await self.master.await_log("Pong Received from server", "info") @@ -304,30 +303,30 @@ @classmethod def handle_websockets(cls, rfile, wfile): - frame = websockets.Frame.from_file(rfile) - assert frame.header.opcode == websockets.OPCODE.PING + header, frame, _ = websocket.read_frame(rfile) + assert header.opcode == Opcode.PING assert frame.payload == b'' - wfile.write(bytes(websockets.Frame(fin=1, opcode=websockets.OPCODE.PONG, payload=frame.payload))) + wfile.write(bytes(websockets_frame.Frame(fin=1, opcode=Opcode.PONG, payload=frame.payload))) wfile.flush() - wfile.write(bytes(websockets.Frame(fin=1, opcode=websockets.OPCODE.CLOSE))) + wfile.write(bytes(websockets_frame.Frame(fin=1, opcode=Opcode.CLOSE))) wfile.flush() - websockets.Frame.from_file(rfile) + _ = websocket.read_frame(rfile) @pytest.mark.asyncio async def test_pong(self): self.setup_connection() - self.client.wfile.write(bytes(websockets.Frame(fin=1, mask=1, opcode=websockets.OPCODE.PING, payload=b'foobar'))) + self.client.wfile.write(bytes(websockets_frame.Frame(fin=1, mask=1, opcode=Opcode.PING, payload=b'foobar'))) self.client.wfile.flush() - frame = websockets.Frame.from_file(self.client.rfile) - websockets.Frame.from_file(self.client.rfile) - self.client.wfile.write(bytes(websockets.Frame(fin=1, mask=1, opcode=websockets.OPCODE.CLOSE))) + header, frame, _ = websocket.read_frame(self.client.rfile) + _ = websocket.read_frame(self.client.rfile) + self.client.wfile.write(bytes(websockets_frame.Frame(fin=1, mask=1, opcode=Opcode.CLOSE))) self.client.wfile.flush() - assert frame.header.opcode == websockets.OPCODE.PONG + assert header.opcode == Opcode.PONG assert frame.payload == b'foobar' assert await self.master.await_log("pong received") @@ -336,57 +335,56 @@ @classmethod def handle_websockets(cls, rfile, wfile): - frame = websockets.Frame.from_file(rfile) - wfile.write(bytes(websockets.Frame(fin=1, opcode=frame.header.opcode, payload=frame.payload))) - wfile.write(bytes(websockets.Frame(fin=1, opcode=websockets.OPCODE.CLOSE))) + header, frame, _ = websocket.read_frame(rfile) + wfile.write(bytes(websockets_frame.Frame(fin=1, opcode=header.opcode, payload=frame.payload))) + wfile.write(bytes(websockets_frame.Frame(fin=1, opcode=Opcode.CLOSE))) wfile.flush() with pytest.raises(exceptions.TcpDisconnect): - websockets.Frame.from_file(rfile) + _ = websocket.read_frame(rfile) def test_close(self): self.setup_connection() - self.client.wfile.write(bytes(websockets.Frame(fin=1, mask=1, opcode=websockets.OPCODE.CLOSE))) + self.client.wfile.write(bytes(websockets_frame.Frame(fin=1, mask=1, opcode=Opcode.CLOSE))) self.client.wfile.flush() - websockets.Frame.from_file(self.client.rfile) + _ = websocket.read_frame(self.client.rfile) with pytest.raises(exceptions.TcpDisconnect): - websockets.Frame.from_file(self.client.rfile) + _ = websocket.read_frame(self.client.rfile) def test_close_payload_1(self): self.setup_connection() - self.client.wfile.write(bytes(websockets.Frame(fin=1, mask=1, opcode=websockets.OPCODE.CLOSE, payload=b'\00\42'))) + self.client.wfile.write(bytes(websockets_frame.Frame(fin=1, mask=1, opcode=Opcode.CLOSE, payload=b'\00\42'))) self.client.wfile.flush() - websockets.Frame.from_file(self.client.rfile) + _ = websocket.read_frame(self.client.rfile) with pytest.raises(exceptions.TcpDisconnect): - websockets.Frame.from_file(self.client.rfile) + _ = websocket.read_frame(self.client.rfile) def test_close_payload_2(self): self.setup_connection() - self.client.wfile.write(bytes(websockets.Frame(fin=1, mask=1, opcode=websockets.OPCODE.CLOSE, payload=b'\00\42foobar'))) + self.client.wfile.write(bytes(websockets_frame.Frame(fin=1, mask=1, opcode=Opcode.CLOSE, payload=b'\00\42foobar'))) self.client.wfile.flush() - websockets.Frame.from_file(self.client.rfile) + _ = websocket.read_frame(self.client.rfile) with pytest.raises(exceptions.TcpDisconnect): - websockets.Frame.from_file(self.client.rfile) + _ = websocket.read_frame(self.client.rfile) class TestInvalidFrame(_WebSocketTest): @classmethod def handle_websockets(cls, rfile, wfile): - wfile.write(bytes(websockets.Frame(fin=1, opcode=15, payload=b'foobar'))) + wfile.write(bytes(websockets_frame.Frame(fin=1, opcode=15, payload=b'foobar'))) wfile.flush() def test_invalid_frame(self): self.setup_connection() - # with pytest.raises(exceptions.TcpDisconnect): - frame = websockets.Frame.from_file(self.client.rfile) + _, frame, _ = websocket.read_frame(self.client.rfile) code, = struct.unpack('!H', frame.payload[:2]) assert code == 1002 assert frame.payload[2:].startswith(b'Invalid opcode') @@ -396,7 +394,7 @@ @classmethod def handle_websockets(cls, rfile, wfile): - wfile.write(bytes(websockets.Frame(opcode=websockets.OPCODE.TEXT, payload=b'server-foobar'))) + wfile.write(bytes(websockets_frame.Frame(opcode=Opcode.TEXT, payload=b'server-foobar'))) wfile.flush() @pytest.mark.parametrize('streaming', [True, False]) @@ -411,11 +409,11 @@ frame = None if not streaming: with pytest.raises(exceptions.TcpDisconnect): # Reader.safe_read get nothing as result - frame = websockets.Frame.from_file(self.client.rfile) + _, frame, _ = websocket.read_frame(self.client.rfile) assert frame is None else: - frame = websockets.Frame.from_file(self.client.rfile) + _, frame, _ = websocket.read_frame(self.client.rfile) assert frame assert self.master.state.flows[1].messages == [] # Message not appended as the final frame isn't received @@ -428,45 +426,45 @@ wfile.write(b'\xc1\x0f*N-*K-\xd2M\xcb\xcfOJ,\x02\x00') wfile.flush() - frame = websockets.Frame.from_file(rfile) - assert frame.header.rsv1 + header, _, _ = websocket.read_frame(rfile) + assert header.rsv.rsv1 wfile.write(b'\xc1\nJ\xce\xc9L\xcd+\x81r\x00\x00') wfile.flush() - frame = websockets.Frame.from_file(rfile) - assert frame.header.rsv1 + header, _, _ = websocket.read_frame(rfile) + assert header.rsv.rsv1 wfile.write(b'\xc2\x07\xba\xb7v\xdf{\x00\x00') wfile.flush() def test_extension(self): self.setup_connection(True) - frame = websockets.Frame.from_file(self.client.rfile) - assert frame.header.rsv1 + header, _, _ = websocket.read_frame(self.client.rfile) + assert header.rsv.rsv1 self.client.wfile.write(b'\xc1\x8fQ\xb7vX\x1by\xbf\x14\x9c\x9c\xa7\x15\x9ax9\x12}\xb5v') self.client.wfile.flush() - frame = websockets.Frame.from_file(self.client.rfile) - assert frame.header.rsv1 + header, _, _ = websocket.read_frame(self.client.rfile) + assert header.rsv.rsv1 self.client.wfile.write(b'\xc2\x87\xeb\xbb\x0csQ\x0cz\xac\x90\xbb\x0c') self.client.wfile.flush() - frame = websockets.Frame.from_file(self.client.rfile) - assert frame.header.rsv1 + header, _, _ = websocket.read_frame(self.client.rfile) + assert header.rsv.rsv1 assert len(self.master.state.flows[1].messages) == 5 assert self.master.state.flows[1].messages[0].content == 'server-foobar' - assert self.master.state.flows[1].messages[0].type == websockets.OPCODE.TEXT + assert self.master.state.flows[1].messages[0].type == Opcode.TEXT assert self.master.state.flows[1].messages[1].content == 'client-foobar' - assert self.master.state.flows[1].messages[1].type == websockets.OPCODE.TEXT + assert self.master.state.flows[1].messages[1].type == Opcode.TEXT assert self.master.state.flows[1].messages[2].content == 'client-foobar' - assert self.master.state.flows[1].messages[2].type == websockets.OPCODE.TEXT + assert self.master.state.flows[1].messages[2].type == Opcode.TEXT assert self.master.state.flows[1].messages[3].content == b'\xde\xad\xbe\xef' - assert self.master.state.flows[1].messages[3].type == websockets.OPCODE.BINARY + assert self.master.state.flows[1].messages[3].type == Opcode.BINARY assert self.master.state.flows[1].messages[4].content == b'\xde\xad\xbe\xef' - assert self.master.state.flows[1].messages[4].type == websockets.OPCODE.BINARY + assert self.master.state.flows[1].messages[4].type == Opcode.BINARY class TestInjectMessageClient(_WebSocketTest): @@ -483,8 +481,8 @@ self.proxy.set_addons(Inject()) self.setup_connection() - frame = websockets.Frame.from_file(self.client.rfile) - assert frame.header.opcode == websockets.OPCODE.TEXT + header, frame, _ = websocket.read_frame(self.client.rfile) + assert header.opcode == Opcode.TEXT assert frame.payload == b'This is an injected message!' @@ -492,11 +490,11 @@ @classmethod def handle_websockets(cls, rfile, wfile): - frame = websockets.Frame.from_file(rfile) - assert frame.header.opcode == websockets.OPCODE.TEXT + header, frame, _ = websocket.read_frame(rfile) + assert header.opcode == Opcode.TEXT success = frame.payload == b'This is an injected message!' - wfile.write(bytes(websockets.Frame(fin=1, opcode=websockets.OPCODE.TEXT, payload=str(success).encode()))) + wfile.write(bytes(websockets_frame.Frame(fin=1, opcode=Opcode.TEXT, payload=str(success).encode()))) wfile.flush() def test_inject_message_server(self): @@ -507,6 +505,6 @@ self.proxy.set_addons(Inject()) self.setup_connection() - frame = websockets.Frame.from_file(self.client.rfile) - assert frame.header.opcode == websockets.OPCODE.TEXT + header, frame, _ = websocket.read_frame(self.client.rfile) + assert header.opcode == Opcode.TEXT assert frame.payload == b'True' diff -Nru mitmproxy-5.1.1/test/mitmproxy/proxy/test_server.py mitmproxy-6.0.2/test/mitmproxy/proxy/test_server.py --- mitmproxy-5.1.1/test/mitmproxy/proxy/test_server.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/proxy/test_server.py 2020-12-15 16:41:27.000000000 +0000 @@ -206,7 +206,7 @@ p = self.pathoc() with p.connect(): ret = p.request("get:'https://localhost:%s/'" % self.server.port) - assert ret.status_code == 400 + assert ret.status_code == 502 def test_connection_close(self): # Add a body, so we have a content-length header, which combined with @@ -452,7 +452,7 @@ assert resp.status_code == 200 req = self.master.state.flows[0].request - assert req.host_header == "127.0.0.1" + assert req.host_header.startswith("127.0.0.1:") @pytest.mark.asyncio async def test_selfconnection(self): @@ -801,12 +801,14 @@ chunks = list(http1.read_body(fconn, None)) assert chunks == [b"this", b"isatest__reachhex"] + fconn.close() + connection.shutdown(socket.SHUT_RDWR) connection.close() class AFakeResponse: def request(self, f): - f.response = http.HTTPResponse.wrap(mitmproxy.test.tutils.tresp()) + f.response = mitmproxy.test.tutils.tresp() class TestFakeResponse(tservers.HTTPProxyTest): @@ -873,7 +875,7 @@ class AIncomplete: def request(self, f): - resp = http.HTTPResponse.wrap(mitmproxy.test.tutils.tresp()) + resp = mitmproxy.test.tutils.tresp() resp.content = None f.response = resp diff -Nru mitmproxy-5.1.1/test/mitmproxy/test_certs.py mitmproxy-6.0.2/test/mitmproxy/test_certs.py --- mitmproxy-5.1.1/test/mitmproxy/test_certs.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/test_certs.py 2020-12-15 16:41:27.000000000 +0000 @@ -204,3 +204,9 @@ x = certs.Cert('') x.set_state(a) assert x == c + + def test_from_store_with_passphrase(self, tdata, tmpdir): + ca = certs.CertStore.from_store(str(tmpdir), "mitmproxy", 2048, "password") + ca.add_cert_file("*", tdata.path("mitmproxy/data/mitmproxy.pem"), "password") + + assert ca.get_cert(b"foo", []) diff -Nru mitmproxy-5.1.1/test/mitmproxy/test_command_lexer.py mitmproxy-6.0.2/test/mitmproxy/test_command_lexer.py --- mitmproxy-5.1.1/test/mitmproxy/test_command_lexer.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/test_command_lexer.py 2020-12-15 16:41:27.000000000 +0000 @@ -16,6 +16,7 @@ ("'foo'x", False), ('''"foo ''', True), ('''"foo 'bar' ''', True), + ('"foo\\', True), ] ) def test_partial_quoted_string(test_input, valid): @@ -34,6 +35,7 @@ ("'foo'x", ["'foo'", 'x']), ('''"foo''', ['"foo']), ('''"foo 'bar' ''', ['''"foo 'bar' ''']), + ('"foo\\', ['"foo\\']), ] ) def test_expr(test_input, expected): diff -Nru mitmproxy-5.1.1/test/mitmproxy/test_command.py mitmproxy-6.0.2/test/mitmproxy/test_command.py --- mitmproxy-5.1.1/test/mitmproxy/test_command.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/test_command.py 2020-12-15 16:41:27.000000000 +0000 @@ -509,7 +509,7 @@ class TAttr: def __getattr__(self, item): - raise IOError + raise OSError class TAttr2: diff -Nru mitmproxy-5.1.1/test/mitmproxy/test_connections.py mitmproxy-6.0.2/test/mitmproxy/test_connections.py --- mitmproxy-5.1.1/test/mitmproxy/test_connections.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/test_connections.py 2020-12-15 16:41:27.000000000 +0000 @@ -12,10 +12,12 @@ from mitmproxy.test import tflow from .net import tservers from pathod import test +from ..conftest import skip_new_proxy_core class TestClientConnection: + @skip_new_proxy_core def test_send(self): c = tflow.tclient_conn() c.send(b'foobar') @@ -26,20 +28,22 @@ c.send(['string', 'not']) assert c.wfile.getvalue() == b'foobarfoobar' + @skip_new_proxy_core def test_repr(self): c = tflow.tclient_conn() assert '127.0.0.1:22' in repr(c) assert 'ALPN' in repr(c) - assert 'TLS' not in repr(c) + assert 'TLS' in repr(c) c.alpn_proto_negotiated = None - c.tls_established = True + c.tls_established = False assert 'ALPN' not in repr(c) - assert 'TLS' in repr(c) + assert 'TLS' not in repr(c) c.address = None assert repr(c) + @skip_new_proxy_core def test_tls_established_property(self): c = tflow.tclient_conn() c.tls_established = True @@ -82,6 +86,7 @@ class TestServerConnection: + @skip_new_proxy_core def test_send(self): c = tflow.tserver_conn() c.send(b'foobar') @@ -92,6 +97,7 @@ c.send(['string', 'not']) assert c.wfile.getvalue() == b'foobarfoobar' + @skip_new_proxy_core def test_repr(self): c = tflow.tserver_conn() @@ -115,6 +121,7 @@ c.address = None assert repr(c) + @skip_new_proxy_core def test_tls_established_property(self): c = tflow.tserver_conn() c.tls_established = True diff -Nru mitmproxy-5.1.1/test/mitmproxy/test_flowfilter.py mitmproxy-6.0.2/test/mitmproxy/test_flowfilter.py --- mitmproxy-5.1.1/test/mitmproxy/test_flowfilter.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/test_flowfilter.py 2020-12-15 16:41:27.000000000 +0000 @@ -439,6 +439,17 @@ assert not self.q("~tcp", f) assert not self.q("~http", f) + def test_handshake(self): + f = self.flow().handshake_flow + assert self.q("~websocket", f) + assert not self.q("~tcp", f) + assert self.q("~http", f) + + f = tflow.tflow() + assert not self.q("~websocket", f) + f = tflow.tflow(resp=True) + assert not self.q("~websocket", f) + def test_ferr(self): e = self.err() assert self.q("~e", e) diff -Nru mitmproxy-5.1.1/test/mitmproxy/test_flow.py mitmproxy-6.0.2/test/mitmproxy/test_flow.py --- mitmproxy-5.1.1/test/mitmproxy/test_flow.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/test_flow.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,14 +1,14 @@ import io + import pytest -from mitmproxy.test import tflow, taddons import mitmproxy.io +from mitmproxy import flow from mitmproxy import flowfilter from mitmproxy import options -from mitmproxy.io import tnetstring from mitmproxy.exceptions import FlowReadException -from mitmproxy import flow -from mitmproxy import http +from mitmproxy.io import tnetstring +from mitmproxy.test import taddons, tflow from . import tservers @@ -29,7 +29,7 @@ f2 = l[0] assert f2.get_state() == f.get_state() - assert f2.request == f.request + assert f2.request.data == f.request.data assert f2.marked def test_filter(self): @@ -128,11 +128,11 @@ with taddons.context(s, options=opts) as ctx: f = tflow.tflow(req=None) await ctx.master.addons.handle_lifecycle("clientconnect", f.client_conn) - f.request = http.HTTPRequest.wrap(mitmproxy.test.tutils.treq()) + f.request = mitmproxy.test.tutils.treq() await ctx.master.addons.handle_lifecycle("request", f) assert len(s.flows) == 1 - f.response = http.HTTPResponse.wrap(mitmproxy.test.tutils.tresp()) + f.response = mitmproxy.test.tutils.tresp() await ctx.master.addons.handle_lifecycle("response", f) assert len(s.flows) == 1 diff -Nru mitmproxy-5.1.1/test/mitmproxy/test_http.py mitmproxy-6.0.2/test/mitmproxy/test_http.py --- mitmproxy-5.1.1/test/mitmproxy/test_http.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/test_http.py 2020-12-15 16:41:27.000000000 +0000 @@ -24,7 +24,7 @@ assert hash(r) def test_get_url(self): - r = http.HTTPRequest.wrap(mitmproxy.test.tutils.treq()) + r = mitmproxy.test.tutils.treq() assert r.url == "http://address:22/path" @@ -44,18 +44,8 @@ assert r.url == "https://address:22/path" assert r.pretty_url == "https://foo.com:22/path" - def test_replace(self): - r = http.HTTPRequest.wrap(mitmproxy.test.tutils.treq()) - r.path = "path/foo" - r.headers["Foo"] = "fOo" - r.content = b"afoob" - assert r.replace("(?i)foo", "boo") == 4 - assert r.path == "path/boo" - assert b"foo" not in r.content - assert r.headers["boo"] == "boo" - def test_constrain_encoding(self): - r = http.HTTPRequest.wrap(mitmproxy.test.tutils.treq()) + r = mitmproxy.test.tutils.treq() r.headers["accept-encoding"] = "gzip, oink" r.constrain_encoding() assert "oink" not in r.headers["accept-encoding"] @@ -65,7 +55,7 @@ assert "oink" not in r.headers["accept-encoding"] def test_get_content_type(self): - resp = http.HTTPResponse.wrap(mitmproxy.test.tutils.tresp()) + resp = mitmproxy.test.tutils.tresp() resp.headers = Headers(content_type="text/plain") assert resp.headers["content-type"] == "text/plain" @@ -78,16 +68,8 @@ resp2 = resp.copy() assert resp2.get_state() == resp.get_state() - def test_replace(self): - r = http.HTTPResponse.wrap(mitmproxy.test.tutils.tresp()) - r.headers["Foo"] = "fOo" - r.content = b"afoob" - assert r.replace("(?i)foo", "boo") == 3 - assert b"foo" not in r.content - assert r.headers["boo"] == "boo" - def test_get_content_type(self): - resp = http.HTTPResponse.wrap(mitmproxy.test.tutils.tresp()) + resp = mitmproxy.test.tutils.tresp() resp.headers = Headers(content_type="text/plain") assert resp.headers["content-type"] == "text/plain" @@ -136,7 +118,7 @@ def test_backup(self): f = tflow.tflow() - f.response = http.HTTPResponse.wrap(mitmproxy.test.tutils.tresp()) + f.response = mitmproxy.test.tutils.tresp() f.request.content = b"foo" assert not f.modified() f.backup() @@ -212,47 +194,9 @@ f2.resume() assert f.intercepted is f2.intercepted is False - def test_replace_unicode(self): - f = tflow.tflow(resp=True) - f.response.content = b"\xc2foo" - f.replace(b"foo", u"bar") - - def test_replace_no_content(self): + def test_timestamp_start(self): f = tflow.tflow() - f.request.content = None - assert f.replace("foo", "bar") == 0 - - def test_replace(self): - f = tflow.tflow(resp=True) - f.request.headers["foo"] = "foo" - f.request.content = b"afoob" - - f.response.headers["foo"] = "foo" - f.response.content = b"afoob" - - assert f.replace("foo", "bar") == 6 - - assert f.request.headers["bar"] == "bar" - assert f.request.content == b"abarb" - assert f.response.headers["bar"] == "bar" - assert f.response.content == b"abarb" - - def test_replace_encoded(self): - f = tflow.tflow(resp=True) - f.request.content = b"afoob" - f.request.encode("gzip") - f.response.content = b"afoob" - f.response.encode("gzip") - - f.replace("foo", "bar") - - assert f.request.raw_content != b"abarb" - f.request.decode() - assert f.request.raw_content == b"abarb" - - assert f.response.raw_content != b"abarb" - f.response.decode() - assert f.response.raw_content == b"abarb" + assert f.timestamp_start == f.request.timestamp_start def test_make_error_response(): @@ -274,5 +218,6 @@ def test_expect_continue_response(): - assert http.expect_continue_response.http_version == 'HTTP/1.1' - assert http.expect_continue_response.status_code == 100 + resp = http.make_expect_continue_response() + assert resp.http_version == 'HTTP/1.1' + assert resp.status_code == 100 diff -Nru mitmproxy-5.1.1/test/mitmproxy/test_proxy.py mitmproxy-6.0.2/test/mitmproxy/test_proxy.py --- mitmproxy-5.1.1/test/mitmproxy/test_proxy.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/test_proxy.py 2020-12-15 16:41:27.000000000 +0000 @@ -10,7 +10,7 @@ from mitmproxy.proxy.server import ConnectionHandler, DummyServer, ProxyServer from mitmproxy.tools import cmdline from mitmproxy.tools import main -from ..conftest import skip_windows +from ..conftest import skip_windows, skip_new_proxy_core class MockParser(argparse.ArgumentParser): @@ -42,11 +42,10 @@ assert self.p() def test_certs(self, tdata): - self.assert_noerr( - "--cert", - tdata.path("mitmproxy/data/testkey.pem")) - with pytest.raises(Exception, match="does not exist"): - self.p("--cert", "nonexistent") + with pytest.raises(Exception, match="ambiguous option"): + self.assert_noerr( + "--cert", + tdata.path("mitmproxy/data/testkey.pem")) class TestProxyServer: @@ -75,6 +74,7 @@ class TestConnectionHandler: + @skip_new_proxy_core def test_fatal_error(self, capsys): opts = options.Options() pconf = config.ProxyConfig(opts) diff -Nru mitmproxy-5.1.1/test/mitmproxy/test_websocket.py mitmproxy-6.0.2/test/mitmproxy/test_websocket.py --- mitmproxy-5.1.1/test/mitmproxy/test_websocket.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/test_websocket.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,10 +1,12 @@ import io + import pytest -from mitmproxy.io import tnetstring from mitmproxy import flowfilter from mitmproxy.exceptions import Kill, ControlException +from mitmproxy.io import tnetstring from mitmproxy.test import tflow +from ..conftest import skip_new_proxy_core class TestWebSocketFlow: @@ -87,6 +89,7 @@ tnetstring.dump(d, b) assert b.getvalue() + @skip_new_proxy_core def test_message_kill(self): f = tflow.twebsocketflow() assert not f.messages[-1].killed diff -Nru mitmproxy-5.1.1/test/mitmproxy/tools/console/test_commander.py mitmproxy-6.0.2/test/mitmproxy/tools/console/test_commander.py --- mitmproxy-5.1.1/test/mitmproxy/tools/console/test_commander.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/tools/console/test_commander.py 2020-12-15 16:41:27.000000000 +0000 @@ -7,19 +7,19 @@ @pytest.fixture(autouse=True) -def tctx(tmpdir): +def commander_tctx(tmpdir): # This runs before each test dir_name = tmpdir.mkdir('mitmproxy').dirname confdir = dir_name opts = options.Options() opts.set(*[f"confdir={confdir}"]) - tctx = taddons.context(options=opts) + commander_tctx = taddons.context(options=opts) ch = command_history.CommandHistory() - tctx.master.addons.add(ch) + commander_tctx.master.addons.add(ch) ch.configure('command_history') - yield tctx + yield commander_tctx # This runs after each test ch.clear_history() @@ -31,48 +31,60 @@ [ "", ["a", "b", "c"], - ["a", "b", "c", "a"] + ["a", "b", "c", "a"], + ["c", "b", "a", "c"], + ["a", "c", "a", "c"] ], [ "xxx", ["a", "b", "c"], + ["xxx", "xxx", "xxx"], + ["xxx", "xxx", "xxx"], ["xxx", "xxx", "xxx"] ], [ "b", ["a", "b", "ba", "bb", "c"], - ["b", "ba", "bb", "b"] + ["b", "ba", "bb", "b"], + ["bb", "ba", "b", "bb"], + ["b", "bb", "b", "bb"] ], ] - for start, opts, cycle in tests: + for start, opts, cycle, cycle_reverse, cycle_mix in tests: c = commander.ListCompleter(start, opts) for expected in cycle: assert c.cycle() == expected + for expected in cycle_reverse: + assert c.cycle(False) == expected + forward = True + for expected in cycle_mix: + assert c.cycle(forward) == expected + forward = not forward class TestCommandEdit: - def test_open_command_bar(self, tctx): - edit = commander.CommandEdit(tctx.master, '') + def test_open_command_bar(self, commander_tctx): + edit = commander.CommandEdit(commander_tctx.master, '') try: edit.update() except IndexError: pytest.faied("Unexpected IndexError") - def test_insert(self, tctx): - edit = commander.CommandEdit(tctx.master, '') + def test_insert(self, commander_tctx): + edit = commander.CommandEdit(commander_tctx.master, '') edit.keypress(1, 'a') assert edit.get_edit_text() == 'a' # Don't let users type a space before starting a command # as a usability feature - edit = commander.CommandEdit(tctx.master, '') + edit = commander.CommandEdit(commander_tctx.master, '') edit.keypress(1, ' ') assert edit.get_edit_text() == '' - def test_backspace(self, tctx): - edit = commander.CommandEdit(tctx.master, '') + def test_backspace(self, commander_tctx): + edit = commander.CommandEdit(commander_tctx.master, '') edit.keypress(1, 'a') edit.keypress(1, 'b') @@ -81,8 +93,8 @@ edit.keypress(1, 'backspace') assert edit.get_edit_text() == 'a' - def test_left(self, tctx): - edit = commander.CommandEdit(tctx.master, '') + def test_left(self, commander_tctx): + edit = commander.CommandEdit(commander_tctx.master, '') edit.keypress(1, 'a') assert edit.cbuf.cursor == 1 @@ -94,8 +106,8 @@ edit.keypress(1, 'left') assert edit.cbuf.cursor == 0 - def test_right(self, tctx): - edit = commander.CommandEdit(tctx.master, '') + def test_right(self, commander_tctx): + edit = commander.CommandEdit(commander_tctx.master, '') edit.keypress(1, 'a') assert edit.cbuf.cursor == 1 @@ -111,11 +123,11 @@ edit.keypress(1, 'right') assert edit.cbuf.cursor == 1 - def test_up_and_down(self, tctx): - edit = commander.CommandEdit(tctx.master, '') + def test_up_and_down(self, commander_tctx): + edit = commander.CommandEdit(commander_tctx.master, '') - tctx.master.commands.execute('commands.history.clear') - tctx.master.commands.execute('commands.history.add "cmd1"') + commander_tctx.master.commands.execute('commands.history.clear') + commander_tctx.master.commands.execute('commands.history.add "cmd1"') edit.keypress(1, 'up') assert edit.get_edit_text() == 'cmd1' @@ -129,11 +141,11 @@ edit.keypress(1, 'down') assert edit.get_edit_text() == '' - edit = commander.CommandEdit(tctx.master, '') + edit = commander.CommandEdit(commander_tctx.master, '') - tctx.master.commands.execute('commands.history.clear') - tctx.master.commands.execute('commands.history.add "cmd1"') - tctx.master.commands.execute('commands.history.add "cmd2"') + commander_tctx.master.commands.execute('commands.history.clear') + commander_tctx.master.commands.execute('commands.history.add "cmd1"') + commander_tctx.master.commands.execute('commands.history.add "cmd2"') edit.keypress(1, 'up') assert edit.get_edit_text() == 'cmd2' @@ -167,8 +179,8 @@ edit.keypress(1, 'up') assert edit.get_edit_text() == 'abc' - edit = commander.CommandEdit(tctx.master, '') - tctx.master.commands.execute('commands.history.add "cmd3"') + edit = commander.CommandEdit(commander_tctx.master, '') + commander_tctx.master.commands.execute('commands.history.add "cmd3"') edit.keypress(1, 'z') edit.keypress(1, 'up') @@ -288,9 +300,9 @@ [("123", 2), ("13", 1)], [("123", 0), ("123", 0)], ] - with taddons.context() as tctx: + with taddons.context() as commander_tctx: for start, output in tests: - cb = commander.CommandBuffer(tctx.master) + cb = commander.CommandBuffer(commander_tctx.master) cb.text, cb.cursor = start[0], start[1] cb.backspace() assert cb.text == output[0] @@ -298,8 +310,8 @@ def test_left(self): cursors = [3, 2, 1, 0, 0] - with taddons.context() as tctx: - cb = commander.CommandBuffer(tctx.master) + with taddons.context() as commander_tctx: + cb = commander.CommandBuffer(commander_tctx.master) cb.text, cb.cursor = "abcd", 4 for c in cursors: cb.left() @@ -307,8 +319,8 @@ def test_right(self): cursors = [1, 2, 3, 4, 4] - with taddons.context() as tctx: - cb = commander.CommandBuffer(tctx.master) + with taddons.context() as commander_tctx: + cb = commander.CommandBuffer(commander_tctx.master) cb.text, cb.cursor = "abcd", 0 for c in cursors: cb.right() @@ -320,22 +332,22 @@ [("a", 0), ("xa", 1)], [("xa", 2), ("xax", 3)], ] - with taddons.context() as tctx: + with taddons.context() as commander_tctx: for start, output in tests: - cb = commander.CommandBuffer(tctx.master) + cb = commander.CommandBuffer(commander_tctx.master) cb.text, cb.cursor = start[0], start[1] cb.insert("x") assert cb.text == output[0] assert cb.cursor == output[1] def test_cycle_completion(self): - with taddons.context() as tctx: - cb = commander.CommandBuffer(tctx.master) + with taddons.context() as commander_tctx: + cb = commander.CommandBuffer(commander_tctx.master) cb.text = "foo bar" cb.cursor = len(cb.text) cb.cycle_completion() - ce = commander.CommandEdit(tctx.master, "se") + ce = commander.CommandEdit(commander_tctx.master, "se") ce.keypress(1, 'tab') ce.update() ret = ce.cbuf.render() @@ -347,8 +359,8 @@ ] def test_render(self): - with taddons.context() as tctx: - cb = commander.CommandBuffer(tctx.master) + with taddons.context() as commander_tctx: + cb = commander.CommandBuffer(commander_tctx.master) cb.text = "foo" assert cb.render() diff -Nru mitmproxy-5.1.1/test/mitmproxy/tools/console/test_common.py mitmproxy-6.0.2/test/mitmproxy/tools/console/test_common.py --- mitmproxy-5.1.1/test/mitmproxy/tools/console/test_common.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/tools/console/test_common.py 2020-12-15 16:41:27.000000000 +0000 @@ -5,10 +5,16 @@ def test_format_flow(): - f = tflow.tflow(resp=True) - assert common.format_flow(f, True) - assert common.format_flow(f, True, hostheader=True) - assert common.format_flow(f, True, extended=True) + flows = [ + tflow.tflow(resp=True), + tflow.tflow(err=True), + tflow.ttcpflow(), + tflow.ttcpflow(err=True), + ] + for f in flows: + for render_mode in common.RenderMode: + assert common.format_flow(f, render_mode=render_mode) + assert common.format_flow(f, render_mode=render_mode, hostheader=True, focused=False) def test_format_keyvals(): @@ -26,7 +32,7 @@ ) ), 1 ) - assert wrapped.render((30, )) + assert wrapped.render((30,)) assert common.format_keyvals( [ ("aa", wrapped) diff -Nru mitmproxy-5.1.1/test/mitmproxy/tools/console/test_defaultkeys.py mitmproxy-6.0.2/test/mitmproxy/tools/console/test_defaultkeys.py --- mitmproxy-5.1.1/test/mitmproxy/tools/console/test_defaultkeys.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/tools/console/test_defaultkeys.py 2020-12-15 16:41:27.000000000 +0000 @@ -34,4 +34,4 @@ try: cmd_obj.prepare_args(args) except Exception as e: - raise ValueError("Invalid command: {}".format(binding.command)) from e + raise ValueError(f"Invalid command: {binding.command}") from e diff -Nru mitmproxy-5.1.1/test/mitmproxy/tools/console/test_keymap.py mitmproxy-6.0.2/test/mitmproxy/tools/console/test_keymap.py --- mitmproxy-5.1.1/test/mitmproxy/tools/console/test_keymap.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/tools/console/test_keymap.py 2020-12-15 16:41:27.000000000 +0000 @@ -21,7 +21,7 @@ assert km.get("options", "key") assert km.get("commands", "key") assert not km.get("flowlist", "key") - assert len((km.list("commands"))) == 1 + assert len(km.list("commands")) == 1 km.handle("unknown", "unknown") assert not km.executor.called @@ -34,7 +34,7 @@ km.handle("options", "glob") assert km.executor.called - assert len((km.list("global"))) == 1 + assert len(km.list("global")) == 1 def test_join(): diff -Nru mitmproxy-5.1.1/test/mitmproxy/tools/console/test_statusbar.py mitmproxy-6.0.2/test/mitmproxy/tools/console/test_statusbar.py --- mitmproxy-5.1.1/test/mitmproxy/tools/console/test_statusbar.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/tools/console/test_statusbar.py 2020-12-15 16:41:27.000000000 +0000 @@ -8,8 +8,8 @@ o = options.Options() m = master.ConsoleMaster(o) m.options.update( - setheaders=[":~q:foo:bar"], - replacements=[":~q:foo:bar"], + modify_headers=[":~q:foo:bar"], + modify_body=[":~q:foo:bar"], ignore_hosts=["example.com", "example.org"], tcp_hosts=["example.tcp"], intercept="~q", diff -Nru mitmproxy-5.1.1/test/mitmproxy/tools/web/test_app.py mitmproxy-6.0.2/test/mitmproxy/tools/web/test_app.py --- mitmproxy-5.1.1/test/mitmproxy/tools/web/test_app.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/tools/web/test_app.py 2020-12-15 16:41:27.000000000 +0000 @@ -3,16 +3,25 @@ from unittest import mock import os import asyncio +import sys import pytest -import tornado.testing -from tornado import httpclient -from tornado import websocket - -from mitmproxy import options -from mitmproxy.test import tflow -from mitmproxy.tools.web import app -from mitmproxy.tools.web import master as webmaster + +if sys.platform == 'win32': + # workaround for + # https://github.com/tornadoweb/tornado/issues/2751 + # https://www.tornadoweb.org/en/stable/index.html#installation + # (copied multiple times in the codebase, please remove all occurrences) + asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy()) + +import tornado.testing # noqa +from tornado import httpclient # noqa +from tornado import websocket # noqa + +from mitmproxy import options # noqa +from mitmproxy.test import tflow # noqa +from mitmproxy.tools.web import app # noqa +from mitmproxy.tools.web import master as webmaster # noqa @pytest.fixture(scope="module") @@ -281,7 +290,7 @@ @tornado.testing.gen_test def test_websocket(self): - ws_url = "ws://localhost:{}/updates".format(self.get_http_port()) + ws_url = f"ws://localhost:{self.get_http_port()}/updates" ws_client = yield websocket.websocket_connect(ws_url) self.master.options.anticomp = True @@ -330,6 +339,6 @@ here = os.path.abspath(os.path.dirname(__file__)) web_root = os.path.join(here, os.pardir, os.pardir, os.pardir, os.pardir, 'web') tflow_path = os.path.join(web_root, 'src/js/__tests__/ducks/_tflow.js') - content = """export default function(){{\n return {tflow_json}\n}}""".format(tflow_json=tflow_json) + content = f"""export default function(){{\n return {tflow_json}\n}}""" with open(tflow_path, 'w', newline="\n") as f: f.write(content) diff -Nru mitmproxy-5.1.1/test/mitmproxy/tservers.py mitmproxy-6.0.2/test/mitmproxy/tservers.py --- mitmproxy-5.1.1/test/mitmproxy/tservers.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/tservers.py 2020-12-15 16:41:27.000000000 +0000 @@ -228,7 +228,7 @@ if self.ssl: q = "get:'/p/%s'" % spec else: - q = "get:'%s/p/%s'" % (self.server.urlbase, spec) + q = f"get:'{self.server.urlbase}/p/{spec}'" with p.connect(): return p.request(q) @@ -242,7 +242,7 @@ else: p = self.pathoc() with p.connect(): - return p.request("get:'http://%s%s'" % (self.master.options.onboarding_host, page)) + return p.request(f"get:'http://{self.master.options.onboarding_host}{page}'") class TransparentProxyTest(ProxyTestBase): diff -Nru mitmproxy-5.1.1/test/mitmproxy/utils/test_arg_check.py mitmproxy-6.0.2/test/mitmproxy/utils/test_arg_check.py --- mitmproxy-5.1.1/test/mitmproxy/utils/test_arg_check.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/utils/test_arg_check.py 2020-12-15 16:41:27.000000000 +0000 @@ -24,8 +24,10 @@ 'SPEC Format: "username:pass", "any" to accept any user/pass combination,\n' '"@path" to use an Apache htpasswd file, or\n' '"ldap[s]:url_server_ldap:dn_auth:password:dn_subtree" ' - 'for LDAP authentication.') - + 'for LDAP authentication.'), + (["--replacements"], "--replacements is deprecated.\n" + "Please use `--modify-body` or `--modify-headers` instead."), + (["--underscore_option"], "--underscore_option uses underscores, please use hyphens --underscore-option") ]) def test_check_args(arg, output): f = io.StringIO() diff -Nru mitmproxy-5.1.1/test/mitmproxy/utils/test_asyncio_utils.py mitmproxy-6.0.2/test/mitmproxy/utils/test_asyncio_utils.py --- mitmproxy-5.1.1/test/mitmproxy/utils/test_asyncio_utils.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/utils/test_asyncio_utils.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,45 @@ +import asyncio + +import pytest + +from mitmproxy.utils import asyncio_utils + + +async def ttask(): + asyncio_utils.set_task_debug_info( + asyncio.current_task(), + name="newname", + ) + await asyncio.sleep(999) + + +@pytest.mark.asyncio +async def test_simple(): + task = asyncio_utils.create_task( + ttask(), + name="ttask", + client=("127.0.0.1", 42313) + ) + assert asyncio_utils.task_repr(task) == "127.0.0.1:42313: ttask (age: 0s)" + await asyncio.sleep(0) + assert "newname" in asyncio_utils.task_repr(task) + asyncio_utils.cancel_task(task, "bye") + await asyncio.sleep(0) + assert task.cancelled() + + +def test_closed_loop(): + # Crude test for line coverage. + # This should eventually go, see the description in asyncio_utils.create_task for details. + asyncio_utils.create_task( + ttask(), + name="ttask", + ) + t = ttask() + with pytest.raises(RuntimeError): + asyncio_utils.create_task( + t, + name="ttask", + ignore_closed_loop=False, + ) + t.close() # suppress "not awaited" warning \ No newline at end of file diff -Nru mitmproxy-5.1.1/test/mitmproxy/utils/test_compat.py mitmproxy-6.0.2/test/mitmproxy/utils/test_compat.py --- mitmproxy-5.1.1/test/mitmproxy/utils/test_compat.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/utils/test_compat.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,6 @@ +from mitmproxy.utils import compat + + +def test_simple(): + assert compat.Server + assert compat.Client diff -Nru mitmproxy-5.1.1/test/mitmproxy/utils/test_debug.py mitmproxy-6.0.2/test/mitmproxy/utils/test_debug.py --- mitmproxy-5.1.1/test/mitmproxy/utils/test_debug.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/utils/test_debug.py 2020-12-15 16:41:27.000000000 +0000 @@ -17,6 +17,14 @@ cs = io.StringIO() debug.dump_info(None, None, file=cs, testing=True) assert cs.getvalue() + assert "Tasks" not in cs.getvalue() + + +@pytest.mark.asyncio +async def test_dump_info_async(): + cs = io.StringIO() + debug.dump_info(None, None, file=cs, testing=True) + assert "Tasks" in cs.getvalue() def test_dump_stacks(): diff -Nru mitmproxy-5.1.1/test/mitmproxy/utils/test_spec.py mitmproxy-6.0.2/test/mitmproxy/utils/test_spec.py --- mitmproxy-5.1.1/test/mitmproxy/utils/test_spec.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/utils/test_spec.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,20 @@ +import pytest +from mitmproxy.utils.spec import parse_spec + + +def test_parse_spec(): + flow_filter, subject, replacement = parse_spec("/foo/bar/voing") + assert flow_filter.pattern == "foo" + assert subject == "bar" + assert replacement == "voing" + + flow_filter, subject, replacement = parse_spec("/bar/voing") + assert flow_filter(1) is True + assert subject == "bar" + assert replacement == "voing" + + with pytest.raises(ValueError, match="Invalid number of parameters"): + parse_spec("/") + + with pytest.raises(ValueError, match="Invalid filter pattern"): + parse_spec("/~b/one/two") diff -Nru mitmproxy-5.1.1/test/mitmproxy/utils/test_strutils.py mitmproxy-6.0.2/test/mitmproxy/utils/test_strutils.py --- mitmproxy-5.1.1/test/mitmproxy/utils/test_strutils.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/utils/test_strutils.py 2020-12-15 16:41:27.000000000 +0000 @@ -7,7 +7,7 @@ assert strutils.always_bytes(bytes(range(256))) == bytes(range(256)) assert strutils.always_bytes("foo") == b"foo" with pytest.raises(ValueError): - strutils.always_bytes(u"\u2605", "ascii") + strutils.always_bytes("\u2605", "ascii") with pytest.raises(TypeError): strutils.always_bytes(42, "ascii") @@ -21,20 +21,20 @@ def test_escape_control_characters(): - assert strutils.escape_control_characters(u"one") == u"one" - assert strutils.escape_control_characters(u"\00ne") == u".ne" - assert strutils.escape_control_characters(u"\nne") == u"\nne" - assert strutils.escape_control_characters(u"\nne", False) == u".ne" - assert strutils.escape_control_characters(u"\u2605") == u"\u2605" + assert strutils.escape_control_characters("one") == "one" + assert strutils.escape_control_characters("\00ne") == ".ne" + assert strutils.escape_control_characters("\nne") == "\nne" + assert strutils.escape_control_characters("\nne", False) == ".ne" + assert strutils.escape_control_characters("\u2605") == "\u2605" assert ( strutils.escape_control_characters(bytes(bytearray(range(128))).decode()) == - u'.........\t\n..\r.................. !"#$%&\'()*+,-./0123456789:;<' - u'=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\\]^_`abcdefghijklmnopqrstuvwxyz{|}~.' + '.........\t\n..\r.................. !"#$%&\'()*+,-./0123456789:;<' + '=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\\]^_`abcdefghijklmnopqrstuvwxyz{|}~.' ) assert ( strutils.escape_control_characters(bytes(bytearray(range(128))).decode(), False) == - u'................................ !"#$%&\'()*+,-./0123456789:;<' - u'=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\\]^_`abcdefghijklmnopqrstuvwxyz{|}~.' + '................................ !"#$%&\'()*+,-./0123456789:;<' + '=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\\]^_`abcdefghijklmnopqrstuvwxyz{|}~.' ) with pytest.raises(ValueError): @@ -61,16 +61,16 @@ assert strutils.bytes_to_escaped_str(b"\\\\n", True) == "\\ \\ \\ \\ n".replace(" ", "") with pytest.raises(ValueError): - strutils.bytes_to_escaped_str(u"such unicode") + strutils.bytes_to_escaped_str("such unicode") def test_escaped_str_to_bytes(): assert strutils.escaped_str_to_bytes("foo") == b"foo" assert strutils.escaped_str_to_bytes("\x08") == b"\b" assert strutils.escaped_str_to_bytes("&!?=\\\\)") == br"&!?=\)" - assert strutils.escaped_str_to_bytes(u"\\x08") == b"\b" - assert strutils.escaped_str_to_bytes(u"&!?=\\\\)") == br"&!?=\)" - assert strutils.escaped_str_to_bytes(u"\u00fc") == b'\xc3\xbc' + assert strutils.escaped_str_to_bytes("\\x08") == b"\b" + assert strutils.escaped_str_to_bytes("&!?=\\\\)") == br"&!?=\)" + assert strutils.escaped_str_to_bytes("\u00fc") == b'\xc3\xbc' with pytest.raises(ValueError): strutils.escaped_str_to_bytes(b"very byte") diff -Nru mitmproxy-5.1.1/test/mitmproxy/utils/test_typecheck.py mitmproxy-6.0.2/test/mitmproxy/utils/test_typecheck.py --- mitmproxy-5.1.1/test/mitmproxy/utils/test_typecheck.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/mitmproxy/utils/test_typecheck.py 2020-12-15 16:41:27.000000000 +0000 @@ -12,11 +12,12 @@ class T(TBase): def __init__(self, foo: str): - super(T, self).__init__(42) + super().__init__(42) def test_check_option_type(): typecheck.check_option_type("foo", 42, int) + typecheck.check_option_type("foo", 42, float) with pytest.raises(TypeError): typecheck.check_option_type("foo", 42, str) with pytest.raises(TypeError): @@ -72,6 +73,7 @@ assert(typecheck.typespec_to_str(str)) == "str" assert(typecheck.typespec_to_str(typing.Sequence[str])) == "sequence of str" assert(typecheck.typespec_to_str(typing.Optional[str])) == "optional str" + assert(typecheck.typespec_to_str(typing.Optional[int])) == "optional int" with pytest.raises(NotImplementedError): typecheck.typespec_to_str(dict) diff -Nru mitmproxy-5.1.1/test/pathod/language/test_websockets_frame.py mitmproxy-6.0.2/test/pathod/language/test_websockets_frame.py --- mitmproxy-5.1.1/test/pathod/language/test_websockets_frame.py 1970-01-01 00:00:00.000000000 +0000 +++ mitmproxy-6.0.2/test/pathod/language/test_websockets_frame.py 2020-12-15 16:41:27.000000000 +0000 @@ -0,0 +1,187 @@ +import os +import codecs +import pytest + +from wsproto.frame_protocol import Opcode + +from pathod.language import websockets_frame +from mitmproxy.test import tutils + + +class TestMasker: + + @pytest.mark.parametrize("input,expected", [ + ([b"a"], '00'), + ([b"four"], '070d1616'), + ([b"fourf"], '070d161607'), + ([b"fourfive"], '070d1616070b1501'), + ([b"a", b"aasdfasdfa", b"asdf"], '000302170504021705040205120605'), + ([b"a" * 50, b"aasdfasdfa", b"asdf"], '00030205000302050003020500030205000302050003020500030205000302050003020500030205000302050003020500030205120605051206050500110702'), # noqa + ]) + def test_masker(self, input, expected): + m = websockets_frame.Masker(b"abcd") + data = b"".join([m(t) for t in input]) + assert data == codecs.decode(expected, 'hex') + + data = websockets_frame.Masker(b"abcd")(data) + assert data == b"".join(input) + + +class TestFrameHeader: + + @pytest.mark.parametrize("input,expected", [ + (0, '0100'), + (125, '017D'), + (126, '017E007E'), + (127, '017E007F'), + (142, '017E008E'), + (65534, '017EFFFE'), + (65535, '017EFFFF'), + (65536, '017F0000000000010000'), + (8589934591, '017F00000001FFFFFFFF'), + (2 ** 64 - 1, '017FFFFFFFFFFFFFFFFF'), + ]) + def test_serialization_length(self, input, expected): + h = websockets_frame.FrameHeader( + opcode=Opcode.TEXT, + payload_length=input, + ) + assert bytes(h) == codecs.decode(expected, 'hex') + + def test_serialization_too_large(self): + h = websockets_frame.FrameHeader( + payload_length=2 ** 64 + 1, + ) + with pytest.raises(ValueError): + bytes(h) + + @pytest.mark.parametrize("input,expected", [ + ('0100', 0), + ('017D', 125), + ('017E007E', 126), + ('017E007F', 127), + ('017E008E', 142), + ('017EFFFE', 65534), + ('017EFFFF', 65535), + ('017F0000000000010000', 65536), + ('017F00000001FFFFFFFF', 8589934591), + ('017FFFFFFFFFFFFFFFFF', 2 ** 64 - 1), + ]) + def test_deserialization_length(self, input, expected): + h = websockets_frame.FrameHeader.from_file(tutils.treader(codecs.decode(input, 'hex'))) + assert h.payload_length == expected + + @pytest.mark.parametrize("input,expected", [ + ('0100', (False, None)), + ('018000000000', (True, '00000000')), + ('018012345678', (True, '12345678')), + ]) + def test_deserialization_masking(self, input, expected): + h = websockets_frame.FrameHeader.from_file(tutils.treader(codecs.decode(input, 'hex'))) + assert h.mask == expected[0] + if h.mask: + assert h.masking_key == codecs.decode(expected[1], 'hex') + + def test_equality(self): + h = websockets_frame.FrameHeader(mask=True, masking_key=b'1234') + h2 = websockets_frame.FrameHeader(mask=True, masking_key=b'1234') + assert h == h2 + + h = websockets_frame.FrameHeader(fin=True) + h2 = websockets_frame.FrameHeader(fin=False) + assert h != h2 + + assert h != 'foobar' + + def test_roundtrip(self): + def round(*args, **kwargs): + h = websockets_frame.FrameHeader(*args, **kwargs) + h2 = websockets_frame.FrameHeader.from_file(tutils.treader(bytes(h))) + assert h == h2 + + round() + round(fin=True) + round(rsv1=True) + round(rsv2=True) + round(rsv3=True) + round(payload_length=1) + round(payload_length=100) + round(payload_length=1000) + round(payload_length=10000) + round(opcode=Opcode.PING) + round(masking_key=b"test") + + def test_human_readable(self): + f = websockets_frame.FrameHeader( + masking_key=b"test", + fin=True, + payload_length=10 + ) + assert repr(f) + + f = websockets_frame.FrameHeader() + assert repr(f) + + def test_funky(self): + f = websockets_frame.FrameHeader(masking_key=b"test", mask=False) + raw = bytes(f) + f2 = websockets_frame.FrameHeader.from_file(tutils.treader(raw)) + assert not f2.mask + + def test_violations(self): + with pytest.raises(Exception, match="opcode"): + websockets_frame.FrameHeader(opcode=17) + with pytest.raises(Exception, match="Masking key"): + websockets_frame.FrameHeader(masking_key=b"x") + + def test_automask(self): + f = websockets_frame.FrameHeader(mask=True) + assert f.masking_key + + f = websockets_frame.FrameHeader(masking_key=b"foob") + assert f.mask + + f = websockets_frame.FrameHeader(masking_key=b"foob", mask=0) + assert not f.mask + assert not f.masking_key + + +class TestFrame: + def test_equality(self): + f = websockets_frame.Frame(payload=b'1234') + f2 = websockets_frame.Frame(payload=b'1234') + assert f == f2 + + assert f != b'1234' + + def test_roundtrip(self): + def round(*args, **kwargs): + f = websockets_frame.Frame(*args, **kwargs) + raw = bytes(f) + f2 = websockets_frame.Frame.from_file(tutils.treader(raw)) + assert f == f2 + round(b"test") + round(b"test", fin=1) + round(b"test", rsv1=1) + round(b"test", opcode=Opcode.PING) + round(b"test", masking_key=b"test") + + def test_human_readable(self): + f = websockets_frame.Frame() + assert repr(f) + + f = websockets_frame.Frame(b"foobar") + assert "foobar" in repr(f) + + @pytest.mark.parametrize("masked", [True, False]) + @pytest.mark.parametrize("length", [100, 50000, 150000]) + def test_serialization_bijection(self, masked, length): + frame = websockets_frame.Frame( + os.urandom(length), + fin=True, + opcode=Opcode.TEXT, + mask=int(masked), + masking_key=(os.urandom(4) if masked else None) + ) + serialized = bytes(frame) + assert frame == websockets_frame.Frame.from_bytes(serialized) diff -Nru mitmproxy-5.1.1/test/pathod/language/test_websockets.py mitmproxy-6.0.2/test/pathod/language/test_websockets.py --- mitmproxy-5.1.1/test/pathod/language/test_websockets.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/pathod/language/test_websockets.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,8 +1,8 @@ import pytest from pathod import language -from pathod.language import websockets -import mitmproxy.net.websockets + +from wsproto.frame_protocol import Opcode from .. import tservers @@ -41,7 +41,7 @@ "wf:k@4", "wf:x10", ] - self._test_messages(specs, websockets.WebsocketFrame) + self._test_messages(specs, language.websockets.WebsocketFrame) def test_parse_websocket_frames(self): wf = language.parse_websocket_frame("wf:x10") @@ -53,7 +53,7 @@ specs = [ "wf:f'wf'", ] - self._test_messages(specs, websockets.WebsocketClientFrame) + self._test_messages(specs, language.websockets.WebsocketClientFrame) def test_nested_frame(self): wf = parse_request("wf:f'wf'") @@ -61,7 +61,7 @@ def test_flags(self): wf = parse_request("wf:fin:mask:rsv1:rsv2:rsv3") - frm = mitmproxy.net.websockets.Frame.from_bytes(tservers.render(wf)) + frm = language.websockets_frame.Frame.from_bytes(tservers.render(wf)) assert frm.header.fin assert frm.header.mask assert frm.header.rsv1 @@ -69,7 +69,7 @@ assert frm.header.rsv3 wf = parse_request("wf:-fin:-mask:-rsv1:-rsv2:-rsv3") - frm = mitmproxy.net.websockets.Frame.from_bytes(tservers.render(wf)) + frm = language.websockets_frame.Frame.from_bytes(tservers.render(wf)) assert not frm.header.fin assert not frm.header.mask assert not frm.header.rsv1 @@ -79,15 +79,13 @@ def fr(self, spec, **kwargs): settings = language.base.Settings(**kwargs) wf = parse_request(spec) - return mitmproxy.net.websockets.Frame.from_bytes(tservers.render(wf, settings)) + return language.websockets_frame.Frame.from_bytes(tservers.render(wf, settings)) def test_construction(self): assert self.fr("wf:c1").header.opcode == 1 assert self.fr("wf:c0").header.opcode == 0 - assert self.fr("wf:cbinary").header.opcode ==\ - mitmproxy.net.websockets.OPCODE.BINARY - assert self.fr("wf:ctext").header.opcode ==\ - mitmproxy.net.websockets.OPCODE.TEXT + assert self.fr("wf:cbinary").header.opcode == Opcode.BINARY + assert self.fr("wf:ctext").header.opcode == Opcode.TEXT def test_rawbody(self): frm = self.fr("wf:mask:r'foo'") diff -Nru mitmproxy-5.1.1/test/pathod/protocols/test_http2.py mitmproxy-6.0.2/test/pathod/protocols/test_http2.py --- mitmproxy-5.1.1/test/pathod/protocols/test_http2.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/pathod/protocols/test_http2.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,15 +1,13 @@ from unittest import mock -import codecs -import pytest + import hyperframe +import pytest -from mitmproxy.net import tcp, http -from mitmproxy.net.http import http2 from mitmproxy import exceptions - -from ...mitmproxy.net import tservers as net_tservers - +from mitmproxy.net import http, tcp +from mitmproxy.net.http import http2 from pathod.protocols.http2 import HTTP2StateProtocol, TCPHandler +from ...mitmproxy.net import tservers as net_tservers class TestTCPHandlerWrapper: @@ -100,23 +98,23 @@ def handle(self): # send magic - self.wfile.write(codecs.decode('505249202a20485454502f322e300d0a0d0a534d0d0a0d0a', 'hex_codec')) + self.wfile.write(bytes.fromhex("505249202a20485454502f322e300d0a0d0a534d0d0a0d0a")) self.wfile.flush() # send empty settings frame - self.wfile.write(codecs.decode('000000040000000000', 'hex_codec')) + self.wfile.write(bytes.fromhex("000000040000000000")) self.wfile.flush() # check empty settings frame - raw = http2.read_raw_frame(self.rfile) - assert raw == codecs.decode('00000c040000000000000200000000000300000001', 'hex_codec') + _, consumed_bytes = http2.read_frame(self.rfile, False) + assert consumed_bytes == bytes.fromhex("00000c040000000000000200000000000300000001") # check settings acknowledgement - raw = http2.read_raw_frame(self.rfile) - assert raw == codecs.decode('000000040100000000', 'hex_codec') + _, consumed_bytes = http2.read_frame(self.rfile, False) + assert consumed_bytes == bytes.fromhex("000000040100000000") # send settings acknowledgement - self.wfile.write(codecs.decode('000000040100000000', 'hex_codec')) + self.wfile.write(bytes.fromhex("000000040100000000")) self.wfile.flush() def test_perform_server_connection_preface(self): @@ -128,7 +126,7 @@ protocol.perform_server_connection_preface() assert protocol.connection_preface_performed - with pytest.raises(exceptions.TcpDisconnect): + with pytest.raises(exceptions.TcpReadIncomplete): protocol.perform_server_connection_preface(force=True) @@ -141,18 +139,18 @@ # check empty settings frame assert self.rfile.read(9) ==\ - codecs.decode('000000040000000000', 'hex_codec') + bytes.fromhex("000000040000000000") # send empty settings frame - self.wfile.write(codecs.decode('000000040000000000', 'hex_codec')) + self.wfile.write(bytes.fromhex("000000040000000000")) self.wfile.flush() # check settings acknowledgement assert self.rfile.read(9) == \ - codecs.decode('000000040100000000', 'hex_codec') + bytes.fromhex("000000040100000000") # send settings acknowledgement - self.wfile.write(codecs.decode('000000040100000000', 'hex_codec')) + self.wfile.write(bytes.fromhex("000000040100000000")) self.wfile.flush() def test_perform_client_connection_preface(self): @@ -197,7 +195,7 @@ class handler(tcp.BaseHandler): def handle(self): # check settings acknowledgement - assert self.rfile.read(9) == codecs.decode('000000040100000000', 'hex_codec') + assert self.rfile.read(9) == bytes.fromhex("000000040100000000") self.wfile.write(b"OK") self.wfile.flush() self.rfile.safe_read(9) # just to keep the connection alive a bit longer @@ -236,15 +234,13 @@ (b':scheme', b'https'), (b'foo', b'bar')]) - bytes = HTTP2StateProtocol(self.c)._create_headers( + data = HTTP2StateProtocol(self.c)._create_headers( headers, 1, end_stream=True) - assert b''.join(bytes) ==\ - codecs.decode('000014010500000001824488355217caf3a69a3f87408294e7838c767f', 'hex_codec') + assert b''.join(data) == bytes.fromhex("000014010500000001824488355217caf3a69a3f87408294e7838c767f") - bytes = HTTP2StateProtocol(self.c)._create_headers( + data = HTTP2StateProtocol(self.c)._create_headers( headers, 1, end_stream=False) - assert b''.join(bytes) ==\ - codecs.decode('000014010400000001824488355217caf3a69a3f87408294e7838c767f', 'hex_codec') + assert b''.join(data) == bytes.fromhex("000014010400000001824488355217caf3a69a3f87408294e7838c767f") def test_create_headers_multiple_frames(self): headers = http.Headers([ @@ -256,11 +252,11 @@ protocol = HTTP2StateProtocol(self.c) protocol.http2_settings[hyperframe.frame.SettingsFrame.MAX_FRAME_SIZE] = 8 - bytes = protocol._create_headers(headers, 1, end_stream=True) - assert len(bytes) == 3 - assert bytes[0] == codecs.decode('000008010100000001828487408294e783', 'hex_codec') - assert bytes[1] == codecs.decode('0000080900000000018c767f7685ee5b10', 'hex_codec') - assert bytes[2] == codecs.decode('00000209040000000163d5', 'hex_codec') + data = protocol._create_headers(headers, 1, end_stream=True) + assert len(data) == 3 + assert data[0] == bytes.fromhex("000008010100000001828487408294e783") + assert data[1] == bytes.fromhex("0000080900000000018c767f7685ee5b10") + assert data[2] == bytes.fromhex("00000209040000000163d5") class TestCreateBody: @@ -273,17 +269,17 @@ def test_create_body_single_frame(self): protocol = HTTP2StateProtocol(self.c) - bytes = protocol._create_body(b'foobar', 1) - assert b''.join(bytes) == codecs.decode('000006000100000001666f6f626172', 'hex_codec') + data = protocol._create_body(b'foobar', 1) + assert b''.join(data) == bytes.fromhex("000006000100000001666f6f626172") def test_create_body_multiple_frames(self): protocol = HTTP2StateProtocol(self.c) protocol.http2_settings[hyperframe.frame.SettingsFrame.MAX_FRAME_SIZE] = 5 - bytes = protocol._create_body(b'foobarmehm42', 1) - assert len(bytes) == 3 - assert bytes[0] == codecs.decode('000005000000000001666f6f6261', 'hex_codec') - assert bytes[1] == codecs.decode('000005000000000001726d65686d', 'hex_codec') - assert bytes[2] == codecs.decode('0000020001000000013432', 'hex_codec') + data = protocol._create_body(b'foobarmehm42', 1) + assert len(data) == 3 + assert data[0] == bytes.fromhex("000005000000000001666f6f6261") + assert data[1] == bytes.fromhex("000005000000000001726d65686d") + assert data[2] == bytes.fromhex("0000020001000000013432") class TestReadRequest(net_tservers.ServerTestBase): @@ -291,9 +287,9 @@ def handle(self): self.wfile.write( - codecs.decode('000003010400000001828487', 'hex_codec')) + bytes.fromhex("000003010400000001828487")) self.wfile.write( - codecs.decode('000006000100000001666f6f626172', 'hex_codec')) + bytes.fromhex("000006000100000001666f6f626172")) self.wfile.flush() self.rfile.safe_read(9) # just to keep the connection alive a bit longer @@ -320,7 +316,7 @@ class handler(tcp.BaseHandler): def handle(self): self.wfile.write( - codecs.decode('00000c0105000000014287d5af7e4d5a777f4481f9', 'hex_codec')) + bytes.fromhex("00000c0105000000014287d5af7e4d5a777f4481f9")) self.wfile.flush() ssl = True @@ -339,37 +335,13 @@ assert req.path == "*" -class TestReadRequestAbsolute(net_tservers.ServerTestBase): - class handler(tcp.BaseHandler): - def handle(self): - self.wfile.write( - codecs.decode('00001901050000000182448d9d29aee30c0e492c2a1170426366871c92585422e085', 'hex_codec')) - self.wfile.flush() - - ssl = True - - def test_absolute_form(self): - c = tcp.TCPClient(("127.0.0.1", self.port)) - with c.connect(): - c.convert_to_tls() - protocol = HTTP2StateProtocol(c, is_server=True) - protocol.connection_preface_performed = True - - req = protocol.read_request(NotImplemented) - - assert req.first_line_format == "absolute" - assert req.scheme == "http" - assert req.host == "address" - assert req.port == 22 - - class TestReadResponse(net_tservers.ServerTestBase): class handler(tcp.BaseHandler): def handle(self): self.wfile.write( - codecs.decode('00000801040000002a88628594e78c767f', 'hex_codec')) + bytes.fromhex("00000801040000002a88628594e78c767f")) self.wfile.write( - codecs.decode('00000600010000002a666f6f626172', 'hex_codec')) + bytes.fromhex("00000600010000002a666f6f626172")) self.wfile.flush() self.rfile.safe_read(9) # just to keep the connection alive a bit longer @@ -396,7 +368,7 @@ class handler(tcp.BaseHandler): def handle(self): self.wfile.write( - codecs.decode('00000801050000002a88628594e78c767f', 'hex_codec')) + bytes.fromhex("00000801050000002a88628594e78c767f")) self.wfile.flush() ssl = True @@ -422,89 +394,107 @@ c = tcp.TCPClient(("127.0.0.1", 0)) def test_request_simple(self): - bytes = HTTP2StateProtocol(self.c).assemble_request(http.Request( - b'', - b'GET', - b'https', - b'', - b'', - b'/', - b"HTTP/2.0", - (), - None, + data = HTTP2StateProtocol(self.c).assemble_request(http.Request( + host="", + port=0, + method=b'GET', + scheme=b'https', + authority=b'', + path=b'/', + http_version=b"HTTP/2.0", + headers=(), + content=None, + trailers=None, + timestamp_start=0, + timestamp_end=0 )) - assert len(bytes) == 1 - assert bytes[0] == codecs.decode('00000d0105000000018284874188089d5c0b8170dc07', 'hex_codec') + assert len(data) == 1 + assert data[0] == bytes.fromhex('00000d0105000000018284874188089d5c0b8170dc07') def test_request_with_stream_id(self): req = http.Request( - b'', - b'GET', - b'https', - b'', - b'', - b'/', - b"HTTP/2.0", - (), - None, + host="", + port=0, + method=b'GET', + scheme=b'https', + authority=b'', + path=b'/', + http_version=b"HTTP/2.0", + headers=(), + content=None, + trailers=None, + timestamp_start=0, + timestamp_end=0 ) req.stream_id = 0x42 - bytes = HTTP2StateProtocol(self.c).assemble_request(req) - assert len(bytes) == 1 - assert bytes[0] == codecs.decode('00000d0105000000428284874188089d5c0b8170dc07', 'hex_codec') + data = HTTP2StateProtocol(self.c).assemble_request(req) + assert len(data) == 1 + assert data[0] == bytes.fromhex('00000d0105000000428284874188089d5c0b8170dc07') def test_request_with_body(self): - bytes = HTTP2StateProtocol(self.c).assemble_request(http.Request( - b'', - b'GET', - b'https', - b'', - b'', - b'/', - b"HTTP/2.0", - http.Headers([(b'foo', b'bar')]), - b'foobar', + data = HTTP2StateProtocol(self.c).assemble_request(http.Request( + host="", + port=0, + method=b'GET', + scheme=b'https', + authority=b'', + path=b'/', + http_version=b"HTTP/2.0", + headers=http.Headers([(b'foo', b'bar')]), + content=b'foobar', + trailers=None, + timestamp_start=0, + timestamp_end=None, )) - assert len(bytes) == 2 - assert bytes[0] ==\ - codecs.decode('0000150104000000018284874188089d5c0b8170dc07408294e7838c767f', 'hex_codec') - assert bytes[1] ==\ - codecs.decode('000006000100000001666f6f626172', 'hex_codec') + assert len(data) == 2 + assert data[0] == bytes.fromhex("0000150104000000018284874188089d5c0b8170dc07408294e7838c767f") + assert data[1] == bytes.fromhex("000006000100000001666f6f626172") class TestAssembleResponse: c = tcp.TCPClient(("127.0.0.1", 0)) def test_simple(self): - bytes = HTTP2StateProtocol(self.c, is_server=True).assemble_response(http.Response( - b"HTTP/2.0", - 200, + data = HTTP2StateProtocol(self.c, is_server=True).assemble_response(http.Response( + http_version=b"HTTP/2.0", + status_code=200, + reason=b"", + headers=(), + content=b"", + trailers=None, + timestamp_start=0, + timestamp_end=0, )) - assert len(bytes) == 1 - assert bytes[0] ==\ - codecs.decode('00000101050000000288', 'hex_codec') + assert len(data) == 1 + assert data[0] == bytes.fromhex("00000101050000000288") def test_with_stream_id(self): resp = http.Response( - b"HTTP/2.0", - 200, + http_version=b"HTTP/2.0", + status_code=200, + reason=b"", + headers=(), + content=b"", + trailers=None, + timestamp_start=0, + timestamp_end=0, ) resp.stream_id = 0x42 - bytes = HTTP2StateProtocol(self.c, is_server=True).assemble_response(resp) - assert len(bytes) == 1 - assert bytes[0] ==\ - codecs.decode('00000101050000004288', 'hex_codec') + data = HTTP2StateProtocol(self.c, is_server=True).assemble_response(resp) + assert len(data) == 1 + assert data[0] == bytes.fromhex("00000101050000004288") def test_with_body(self): - bytes = HTTP2StateProtocol(self.c, is_server=True).assemble_response(http.Response( - b"HTTP/2.0", - 200, - b'', - http.Headers(foo=b"bar"), - b'foobar' + data = HTTP2StateProtocol(self.c, is_server=True).assemble_response(http.Response( + http_version=b"HTTP/2.0", + status_code=200, + reason=b'', + headers=http.Headers(foo=b"bar"), + content=b'foobar', + trailers=None, + timestamp_start=0, + timestamp_end=0, )) - assert len(bytes) == 2 - assert bytes[0] ==\ - codecs.decode('00000901040000000288408294e7838c767f', 'hex_codec') - assert bytes[1] ==\ - codecs.decode('000006000100000002666f6f626172', 'hex_codec') + assert len(data) == 2 + assert data[0] == bytes.fromhex("00000901040000000288408294e7838c767f") + assert data[1] == bytes.fromhex("000006000100000002666f6f626172") diff -Nru mitmproxy-5.1.1/test/pathod/test_pathoc.py mitmproxy-6.0.2/test/pathod/test_pathoc.py --- mitmproxy-5.1.1/test/pathod/test_pathoc.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/pathod/test_pathoc.py 2020-12-15 16:41:27.000000000 +0000 @@ -1,23 +1,16 @@ import io from unittest.mock import Mock + import pytest -from mitmproxy.net import http -from mitmproxy.net.http import http1 from mitmproxy import exceptions - -from pathod import pathoc, language -from pathod.protocols.http2 import HTTP2StateProtocol - +from mitmproxy.net.http import http1 from mitmproxy.test import tutils +from pathod import language, pathoc +from pathod.protocols.http2 import HTTP2StateProtocol from . import tservers -def test_response(): - r = http.Response(b"HTTP/1.1", 200, b"Message", {}, None, None) - assert repr(r) - - class PathocTestDaemon(tservers.DaemonTests): def tval(self, requests, timeout=None, showssl=False, **kwargs): s = io.StringIO() @@ -243,7 +236,7 @@ c.convert_to_tls.side_effect = tmp_convert_to_tls with c.connect(): _, kwargs = c.convert_to_tls.call_args - assert set(kwargs['alpn_protos']) == set([b'http/1.1', b'h2']) + assert set(kwargs['alpn_protos']) == {b'http/1.1', b'h2'} def test_request(self): c = pathoc.Pathoc( diff -Nru mitmproxy-5.1.1/test/pathod/tservers.py mitmproxy-6.0.2/test/pathod/tservers.py --- mitmproxy-5.1.1/test/pathod/tservers.py 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/test/pathod/tservers.py 2020-12-15 16:41:27.000000000 +0000 @@ -72,7 +72,7 @@ def _getpath(self, path, params=None): scheme = "https" if self.ssl else "http" resp = requests.get( - "%s://localhost:%s/%s" % ( + "{}://localhost:{}/{}".format( scheme, self.d.port, path diff -Nru mitmproxy-5.1.1/tox.ini mitmproxy-6.0.2/tox.ini --- mitmproxy-5.1.1/tox.ini 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/tox.ini 2020-12-15 16:41:27.000000000 +0000 @@ -1,5 +1,5 @@ [tox] -envlist = py35, py36, py37, flake8, filename_matching, mypy, individual_coverage, docs +envlist = py38, py39, flake8, filename_matching, mypy, individual_coverage, docs skipsdist = True toxworkdir={env:TOX_WORK_DIR:.tox} @@ -9,50 +9,43 @@ setenv = HOME = {envtmpdir} commands = mitmdump --version - pytest --timeout 60 --cov-report xml \ + pytest --timeout 60 -vv --cov-report xml \ --cov=mitmproxy --cov=pathod --cov=release \ --full-cov=mitmproxy/ --full-cov=pathod/ \ {posargs} -[testenv:py35] -whitelist_externals = - bash -deps = - -rrequirements.txt -commands = - bash -c "mitmdump --version 2>&1 | grep 'mitmproxy requires Python 3.6'" - [testenv:flake8] -deps = flake8>=3.7.8,<3.8 +deps = flake8==3.8.4 commands = - flake8 --jobs 8 mitmproxy pathod examples test release + flake8 --jobs 8 mitmproxy pathod examples test release {posargs} [testenv:filename_matching] +deps = commands = python ./test/filename_matching.py [testenv:mypy] -deps = mypy>=0.761,<0.762 +deps = mypy==0.782 commands = - mypy . + mypy . {posargs} [testenv:rstcheck] deps = rstcheck>=2.2,<4.0 commands = - rstcheck README.rst + rstcheck README.rst {posargs} [testenv:individual_coverage] deps = -rrequirements.txt commands = - python ./test/individual_coverage.py + python ./test/individual_coverage.py {posargs} [testenv:cibuild] passenv = CI_* GITHUB_* AWS_* TWINE_* DOCKER_* deps = -rrequirements.txt - pyinstaller==3.5 - twine==3.1.1 + pyinstaller==4.1 + twine==3.2.0 awscli commands = mitmdump --version diff -Nru mitmproxy-5.1.1/web/README.md mitmproxy-6.0.2/web/README.md --- mitmproxy-5.1.1/web/README.md 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/web/README.md 2020-12-15 16:41:27.000000000 +0000 @@ -6,3 +6,11 @@ - Run `yarn` to install dependencies - Run `yarn run gulp` to start live-compilation. - Run `mitmweb` and open http://localhost:8081/ + +## Architecture + +There are two components: + +- Server: [`mitmproxy/tools/web`](../mitmproxy/tools/web) + +- Client: `web` diff -Nru mitmproxy-5.1.1/web/src/css/modal.less mitmproxy-6.0.2/web/src/css/modal.less --- mitmproxy-5.1.1/web/src/css/modal.less 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/web/src/css/modal.less 2020-12-15 16:41:27.000000000 +0000 @@ -8,6 +8,6 @@ } .modal-body { - max-height: calc(100vh - 20px); + max-height: calc(100vh - 200px); overflow-y: auto; } diff -Nru mitmproxy-5.1.1/web/src/js/components/common/DocsLink.jsx mitmproxy-6.0.2/web/src/js/components/common/DocsLink.jsx --- mitmproxy-5.1.1/web/src/js/components/common/DocsLink.jsx 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/web/src/js/components/common/DocsLink.jsx 2020-12-15 16:41:27.000000000 +0000 @@ -6,7 +6,7 @@ } export default function DocsLink({ children, resource }) { - let url = `http://docs.mitmproxy.org/en/stable/${resource}` + let url = `https://docs.mitmproxy.org/stable/${resource}` return ( {children || } diff -Nru mitmproxy-5.1.1/web/src/js/components/FlowTable/FlowColumns.jsx mitmproxy-6.0.2/web/src/js/components/FlowTable/FlowColumns.jsx --- mitmproxy-5.1.1/web/src/js/components/FlowTable/FlowColumns.jsx 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/web/src/js/components/FlowTable/FlowColumns.jsx 2020-12-15 16:41:27.000000000 +0000 @@ -57,7 +57,7 @@ let err; if(flow.error){ - if (flow.error.msg === "Connection killed"){ + if (flow.error.msg === "Connection killed."){ err = } else { err = diff -Nru mitmproxy-5.1.1/web/src/js/components/FlowView/Details.jsx mitmproxy-6.0.2/web/src/js/components/FlowView/Details.jsx --- mitmproxy-5.1.1/web/src/js/components/FlowView/Details.jsx 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/web/src/js/components/FlowView/Details.jsx 2020-12-15 16:41:27.000000000 +0000 @@ -20,20 +20,38 @@ ) } -export function ConnectionInfo({ conn }) { +export function ConnectionInfo({ conn }) { return ( - + - + {conn.sni && ( )} + {conn.tls_version && ( + + + + + )} + {conn.cipher_name && ( + + + + + )} + {conn.alpn_proto_negotiated && ( + + + + + )}
Address: {conn.address.join(':')}
TLS SNI: {conn.sni}
TLS version:{conn.tls_version}
cipher name:{conn.cipher_name}
ALPN:{conn.alpn_proto_negotiated}
) @@ -82,7 +100,7 @@ deltaTo: req.timestamp_start }, { title: "First request byte", - t: req.timestamp_start, + t: req.timestamp_start }, { title: "Request complete", t: req.timestamp_end, @@ -118,9 +136,13 @@

Client Connection

-

Server Connection

- - + {flow.server_conn.address && + [ +

Server Connection

, + + ] + } + diff -Nru mitmproxy-5.1.1/web/src/js/components/FlowView/Headers.jsx mitmproxy-6.0.2/web/src/js/components/FlowView/Headers.jsx --- mitmproxy-5.1.1/web/src/js/components/FlowView/Headers.jsx 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/web/src/js/components/FlowView/Headers.jsx 2020-12-15 16:41:27.000000000 +0000 @@ -46,10 +46,15 @@ static propTypes = { onChange: PropTypes.func.isRequired, message: PropTypes.object.isRequired, + type: PropTypes.string.isRequired, + } + + static defaultProps = { + type: 'headers', } onChange(row, col, val) { - const nextHeaders = _.cloneDeep(this.props.message.headers) + const nextHeaders = _.cloneDeep(this.props.message[this.props.type]) nextHeaders[row][col] = val @@ -75,7 +80,7 @@ } onTab(row, col, e) { - const headers = this.props.message.headers + const headers = this.props.message[this.props.type] if (col === 0) { this._nextSel = `${row}-value` @@ -88,7 +93,7 @@ e.preventDefault() - const nextHeaders = _.cloneDeep(this.props.message.headers) + const nextHeaders = _.cloneDeep(this.props.message[this.props.type]) nextHeaders.push(['Name', 'Value']) this.props.onChange(nextHeaders) this._nextSel = `${row + 1}-key` @@ -113,37 +118,45 @@ render() { const { message, readonly } = this.props - - return ( - - - {message.headers.map((header, i) => ( - - - - - ))} - -
- this.onChange(i, 0, val)} - onRemove={event => this.onRemove(i, 0, event)} - onTab={event => this.onTab(i, 0, event)} - /> - : - - this.onChange(i, 1, val)} - onRemove={event => this.onRemove(i, 1, event)} - onTab={event => this.onTab(i, 1, event)} - /> -
- ) + if (message[this.props.type]) { + return ( + + + {message[this.props.type].map((header, i) => ( + + + + + ))} + +
+ this.onChange(i, 0, val)} + onRemove={event => this.onRemove(i, 0, event)} + onTab={event => this.onTab(i, 0, event)} + /> + : + + this.onChange(i, 1, val)} + onRemove={event => this.onRemove(i, 1, event)} + onTab={event => this.onTab(i, 1, event)} + /> +
+ ) + } else { + return ( + + + +
+ ) + } } } diff -Nru mitmproxy-5.1.1/web/src/js/components/FlowView/Messages.jsx mitmproxy-6.0.2/web/src/js/components/FlowView/Messages.jsx --- mitmproxy-5.1.1/web/src/js/components/FlowView/Messages.jsx 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/web/src/js/components/FlowView/Messages.jsx 2020-12-15 16:41:27.000000000 +0000 @@ -105,6 +105,14 @@ flow={flow} onContentChange={content => updateFlow({ request: {content}})} message={flow.request}/> + +
+ updateFlow({ request: { trailers } })} + type='trailers' + /> {!noContent && @@ -150,6 +158,13 @@ onContentChange={content => updateFlow({ response: {content}})} message={flow.response} /> +
+ updateFlow({ response: { trailers } })} + type='trailers' + /> {!noContent && diff -Nru mitmproxy-5.1.1/web/src/js/components/Header/FileMenu.jsx mitmproxy-6.0.2/web/src/js/components/Header/FileMenu.jsx --- mitmproxy-5.1.1/web/src/js/components/Header/FileMenu.jsx 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/web/src/js/components/Header/FileMenu.jsx 2020-12-15 16:41:27.000000000 +0000 @@ -23,8 +23,8 @@ return ( FileMenu.onNewClick(e, clearFlows)}> - -  New + +  Clear All
- Strip cache headers + Strip cache headers Use host header for display diff -Nru mitmproxy-5.1.1/web/src/js/__tests__/components/common/__snapshots__/DocsLinkSpec.js.snap mitmproxy-6.0.2/web/src/js/__tests__/components/common/__snapshots__/DocsLinkSpec.js.snap --- mitmproxy-5.1.1/web/src/js/__tests__/components/common/__snapshots__/DocsLinkSpec.js.snap 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/web/src/js/__tests__/components/common/__snapshots__/DocsLinkSpec.js.snap 2020-12-15 16:41:27.000000000 +0000 @@ -2,7 +2,7 @@ exports[`DocsLink Component should be able to be rendered with children nodes 1`] = ` foo @@ -11,7 +11,7 @@ exports[`DocsLink Component should be able to be rendered without children nodes 1`] = ` ) tree = pathColumn.toJSON() diff -Nru mitmproxy-5.1.1/web/src/js/__tests__/components/FlowView/DetailsSpec.js mitmproxy-6.0.2/web/src/js/__tests__/components/FlowView/DetailsSpec.js --- mitmproxy-5.1.1/web/src/js/__tests__/components/FlowView/DetailsSpec.js 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/web/src/js/__tests__/components/FlowView/DetailsSpec.js 2020-12-15 16:41:27.000000000 +0000 @@ -47,4 +47,24 @@ tree = details.toJSON() expect(tree).toMatchSnapshot() }) + + it('should render correctly when server address is missing', () => { + let tflowServerAddressNull = tflow + + tflowServerAddressNull.server_conn.address = null + tflowServerAddressNull.server_conn.ip_address = null + tflowServerAddressNull.server_conn.alpn_proto_negotiated = null + tflowServerAddressNull.server_conn.sni = null + tflowServerAddressNull.server_conn.ssl_established = false + tflowServerAddressNull.server_conn.tls_version = null + tflowServerAddressNull.server_conn.timestamp_tcp_setup = null + tflowServerAddressNull.server_conn.timestamp_ssl_setup = null + tflowServerAddressNull.server_conn.timestamp_start = null + tflowServerAddressNull.server_conn.timestamp_end = null + + let details = renderer.create(
), + tree = details.toJSON() + expect(tree).toMatchSnapshot() + }) + }) diff -Nru mitmproxy-5.1.1/web/src/js/__tests__/components/FlowView/MessagesSpec.js mitmproxy-6.0.2/web/src/js/__tests__/components/FlowView/MessagesSpec.js --- mitmproxy-5.1.1/web/src/js/__tests__/components/FlowView/MessagesSpec.js 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/web/src/js/__tests__/components/FlowView/MessagesSpec.js 2020-12-15 16:41:27.000000000 +0000 @@ -58,7 +58,7 @@ }) it('should handle change on flow request header', () => { - let headers = TestUtils.findRenderedComponentWithType(provider, Headers) + let headers = TestUtils.scryRenderedComponentsWithType(provider, Headers).filter(headers => headers.props.type === 'headers')[0] headers.props.onChange('foo') expect(store.getActions()).toEqual([updateEdit({ request: { headers: 'foo' }})]) }) @@ -115,7 +115,7 @@ }) it('should handle change on flow response headers', () => { - let headers = TestUtils.findRenderedComponentWithType(provider, Headers) + let headers = TestUtils.scryRenderedComponentsWithType(provider, Headers).filter(headers => headers.props.type === 'headers')[0] headers.props.onChange('foo') expect(store.getActions()).toEqual([updateEdit( { response: { headers: 'foo' }})]) }) diff -Nru mitmproxy-5.1.1/web/src/js/__tests__/components/FlowView/__snapshots__/DetailsSpec.js.snap mitmproxy-6.0.2/web/src/js/__tests__/components/FlowView/__snapshots__/DetailsSpec.js.snap --- mitmproxy-5.1.1/web/src/js/__tests__/components/FlowView/__snapshots__/DetailsSpec.js.snap 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/web/src/js/__tests__/components/FlowView/__snapshots__/DetailsSpec.js.snap 2020-12-15 16:41:27.000000000 +0000 @@ -27,6 +27,34 @@ address + + + TLS version: + + + TLSv1.2 + + + + + cipher name: + + + cipher + + + + + + ALPN: + + + + http/1.1 + + `; @@ -62,6 +90,34 @@ address + + + TLS version: + + + TLSv1.2 + + + + + cipher name: + + + cipher + + + + + + ALPN: + + + + http/1.1 + +

@@ -91,6 +147,14 @@ address + + + TLS version: + + + TLSv1.2 + +
@@ -150,6 +214,123 @@ + + First response byte + : + + + 2017-05-21 12:38:32.481 + + + + + Response complete + : + + + 2017-05-21 12:38:32.481 + + + + +
+ +`; + +exports[`Details Component should render correctly when server address is missing 1`] = ` +
+

+ Client Connection +

+ + + + + + + + + + + + + + + + + + + + + + + +
+ Address: + + 127.0.0.1:22 +
+ + TLS SNI: + + + address +
+ TLS version: + + TLSv1.2 +
+ cipher name: + + cipher +
+ + ALPN: + + + http/1.1 +
+
+
+

+ Timing +

+ + + + + + + + + + + + + + + +
+ Client conn. established + : + + 1970-01-01 00:00:01.000 +
+ Client conn. SSL handshake + : + + 1970-01-01 00:00:02.000 +
First response byte : diff -Nru mitmproxy-5.1.1/web/src/js/__tests__/components/FlowView/__snapshots__/MessagesSpec.js.snap mitmproxy-6.0.2/web/src/js/__tests__/components/FlowView/__snapshots__/MessagesSpec.js.snap --- mitmproxy-5.1.1/web/src/js/__tests__/components/FlowView/__snapshots__/MessagesSpec.js.snap 2020-04-13 11:09:23.000000000 +0000 +++ mitmproxy-6.0.2/web/src/js/__tests__/components/FlowView/__snapshots__/MessagesSpec.js.snap 2020-12-15 16:41:27.000000000 +0000 @@ -199,6 +199,111 @@
+
+ + + + + + + + + + + +
+
+ + : + +
+
+
+
+ + : + +
+
+