Website Evidence Collector 1.0.0

Latest release
10 months ago

1.0.0 / 2021-01-07

  • fix an issue when tool crashes when the browsing of multiple random pages with --max includes PDF files and other downloads: selected links are tested first to have mime type text/html using a HEAD HTTP request outside of the browser.
  • bugfix: stop script and log error in case of unreachable websites
  • feature: allow for custom page load timeouts, example: --page-timeout=200 for a timeout of 200 ms.
  • feature: add option --dnt to send DO-NOT-TRACK (DNT) HTTP header
  • feature: add option --dnt-js to both send DO-NOT-TRACK (DNT) HTTP header and set navigator.doNotTrack property accordingly
  • bugfix: Chromium retains cookies with empty name and value if web servers send empty HTTP Cookie Header, i.e. "Set-Cookie: "
  • feature: add option --task-description to include custom text/JSON in the output
  • bugfix: HTML output generation crashed for some websites with incomplete SSL certificates
  • documentation: include tips for fast/parallel evidence collection in FAQ
  • bugfix: injected JS produces error on some websites (Uncaught TypeError: this.each is not a function at NodeList.collect (prototype.js:293))
  • bugfix: tool crashes sometimes if no redirects from HTTP are found
  • bugfix: catch Error: Protocol error (Page.captureScreenshot): Unable to capture screenshot
  • bugfix: catch Error: Protocol error (DOMStorage.getDOMStorageItems): Frame not found for the given security origin
  • bugfix: tool crashes sometimes when page response is for some reason empty (TypeError: Cannot read property 'request' of null)
  • bugfix: tool hangs when page has forms that contains an element with id action
  • feature: allow to pre-install cookies with --set-cookie to e.g. indicate given/rejected consent
  • documentation: add to FAQ tips on data analysis with shell tool jq
  • breaking change: change commandline option name of website-evidence-reporter from --output to --output-file to mark the distinct meaning and avoid clash when setting WEC_OUTPUT environment variable (short option name stays -o for both)
  • update: node dependencies and filterlists
  • feature: respect environment variables such as WEC_MAX=10 or WEC_headless=false
  • feature: introduce option --browser-options <browser options> as an alternative to the -- <browser options> syntax (required for Docker setup)
  • bugfix: update yargs dependency to get bugfix yargs/yargs-parser#261 for --browser-options starting with double dash --
  • feature: add commandline option --browser-profile (-p) to allow the reuse of existing browser profiles (thanks to Hamburg DPA staff!)
  • feature: include medium.com and tiktok.com links in social media subsection of output link list
  • bugfix: handle non-zero testSSL error codes according to https://github.com/drwetter/testssl.sh/blob/3.1dev/doc/testssl.1.md#exit-status
  • bugfix: catch and log error "Frame not found for the given security origin" (thanks to Hamburg DPA staff!)
  • bugfix: fix CLI option to pass custom templates (#38)