Release 2021.08.02

Add donate links
Add logo and banner
2026-08-08 21:28:38 +03:00 · 2021-08-02 10:37:10 +05:30 · 2021-08-02 08:51:23 +05:30 · 2021-08-02 08:51:22 +05:30 · 2021-08-02 08:00:08 +05:30 · 2021-08-02 07:52:11 +05:30
96 changed files with 4355 additions and 4947 deletions
@@ -0,0 +1,13 @@
+# These are supported funding model platforms
+
+github: # Replace with up to 4 GitHub Sponsors-enabled usernames e.g., [user1, user2]
+patreon: # Replace with a single Patreon username
+open_collective: # Replace with a single Open Collective username
+ko_fi: # Replace with a single Ko-fi username
+tidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel
+community_bridge: # Replace with a single Community Bridge project-name e.g., cloud-foundry
+liberapay: # Replace with a single Liberapay username
+issuehunt: # Replace with a single IssueHunt username
+otechie: # Replace with a single Otechie username
+
+custom: ['https://github.com/yt-dlp/yt-dlp/blob/master/Collaborators.md']
@@ -21,7 +21,7 @@ assignees: ''

 <!--
 Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.06.23. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
+- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.07.24. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
 - Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
 - Make sure that all URLs and arguments with special characters are properly quoted or escaped as explained in https://github.com/yt-dlp/yt-dlp.
 - Search the bugtracker for similar issues: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
@@ -29,7 +29,7 @@ Carefully read and work through this check list in order to prevent the most com
 -->

 - [ ] I'm reporting a broken site support
- [ ] I've verified that I'm running yt-dlp version **2021.06.23**
+- [ ] I've verified that I'm running yt-dlp version **2021.07.24**
 - [ ] I've checked that all provided URLs are alive and playable in a browser
 - [ ] I've checked that all URLs and arguments with special characters are properly quoted or escaped
 - [ ] I've searched the bugtracker for similar issues including closed ones
@@ -44,7 +44,7 @@ Add the `-v` flag to your command line you run yt-dlp with (`yt-dlp -v <your com
 [debug] User config: []
 [debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKc']
 [debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
- [debug] yt-dlp version 2021.06.23
+ [debug] yt-dlp version 2021.07.24
 [debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
 [debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
 [debug] Proxy map: {}
@@ -21,7 +21,7 @@ assignees: ''

 <!--
 Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.06.23. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
+- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.07.24. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
 - Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
 - Make sure that site you are requesting is not dedicated to copyright infringement, see https://github.com/yt-dlp/yt-dlp. yt-dlp does not support such sites. In order for site support request to be accepted all provided example URLs should not violate any copyrights.
 - Search the bugtracker for similar site support requests: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
@@ -29,7 +29,7 @@ Carefully read and work through this check list in order to prevent the most com
 -->

 - [ ] I'm reporting a new site support request
- [ ] I've verified that I'm running yt-dlp version **2021.06.23**
+- [ ] I've verified that I'm running yt-dlp version **2021.07.24**
 - [ ] I've checked that all provided URLs are alive and playable in a browser
 - [ ] I've checked that none of provided URLs violate any copyrights
 - [ ] I've searched the bugtracker for similar site support requests including closed ones
@@ -21,13 +21,13 @@ assignees: ''

 <!--
 Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.06.23. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
+- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.07.24. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
 - Search the bugtracker for similar site feature requests: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
 - Finally, put x into all relevant boxes like this [x] (Dont forget to delete the empty space)
 -->

 - [ ] I'm reporting a site feature request
- [ ] I've verified that I'm running yt-dlp version **2021.06.23**
+- [ ] I've verified that I'm running yt-dlp version **2021.07.24**
 - [ ] I've searched the bugtracker for similar site feature requests including closed ones


@@ -21,7 +21,7 @@ assignees: ''

 <!--
 Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.06.23. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
+- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.07.24. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
 - Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
 - Make sure that all URLs and arguments with special characters are properly quoted or escaped as explained in https://github.com/yt-dlp/yt-dlp.
 - Search the bugtracker for similar issues: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
@@ -30,7 +30,7 @@ Carefully read and work through this check list in order to prevent the most com
 -->

 - [ ] I'm reporting a broken site support issue
- [ ] I've verified that I'm running yt-dlp version **2021.06.23**
+- [ ] I've verified that I'm running yt-dlp version **2021.07.24**
 - [ ] I've checked that all provided URLs are alive and playable in a browser
 - [ ] I've checked that all URLs and arguments with special characters are properly quoted or escaped
 - [ ] I've searched the bugtracker for similar bug reports including closed ones
@@ -46,7 +46,7 @@ Add the `-v` flag to your command line you run yt-dlp with (`yt-dlp -v <your com
 [debug] User config: []
 [debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKc']
 [debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
- [debug] yt-dlp version 2021.06.23
+ [debug] yt-dlp version 2021.07.24
 [debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
 [debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
 [debug] Proxy map: {}
@@ -21,13 +21,13 @@ assignees: ''

 <!--
 Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.06.23. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
+- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.07.24. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
 - Search the bugtracker for similar feature requests: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
 - Finally, put x into all relevant boxes like this [x] (Dont forget to delete the empty space)
 -->

 - [ ] I'm reporting a feature request
- [ ] I've verified that I'm running yt-dlp version **2021.06.23**
+- [ ] I've verified that I'm running yt-dlp version **2021.07.24**
 - [ ] I've searched the bugtracker for similar feature requests including closed ones


@@ -10,7 +10,7 @@ jobs:
      matrix:
        os: [ubuntu-18.04]
        # py3.9 is in quick-test
-        python-version: [3.7, 3.8, pypy-3.6, pypy-3.7]
+        python-version: [3.7, 3.8, 3.10-dev, pypy-3.6, pypy-3.7]
        run-tests-ext: [sh]
        include:
        # atleast one of the tests must be in windows
@@ -23,11 +23,9 @@ jobs:
      uses: actions/setup-python@v2
      with:
        python-version: ${{ matrix.python-version }}
-    - name: Install nose
-      run: pip install nose
+    - name: Install pytest
+      run: pip install pytest
    - name: Run tests
      continue-on-error: False
-      env:
-        YTDL_TEST_SET: core
-      run: ./devscripts/run_tests.${{ matrix.run-tests-ext }}
+      run: ./devscripts/run_tests.${{ matrix.run-tests-ext }} core
  # Linter is in quick-test
@@ -9,7 +9,7 @@ jobs:
      fail-fast: true
      matrix:
        os: [ubuntu-18.04]
-        python-version: [3.7, 3.8, 3.9, pypy-3.6, pypy-3.7]
+        python-version: [3.7, 3.8, 3.9, 3.10-dev, pypy-3.6, pypy-3.7]
        run-tests-ext: [sh]
        include:
        - os: windows-latest
@@ -21,10 +21,8 @@ jobs:
      uses: actions/setup-python@v2
      with:
        python-version: ${{ matrix.python-version }}
-    - name: Install nose
-      run: pip install nose
+    - name: Install pytest
+      run: pip install pytest
    - name: Run tests
      continue-on-error: true
-      env:
-        YTDL_TEST_SET: download
-      run: ./devscripts/run_tests.${{ matrix.run-tests-ext }}
+      run: ./devscripts/run_tests.${{ matrix.run-tests-ext }} download
@@ -11,12 +11,10 @@ jobs:
      uses: actions/setup-python@v2
      with:
        python-version: 3.9
-    - name: Install nose
-      run: pip install nose
+    - name: Install test requirements
+      run: pip install pytest pycryptodome
    - name: Run tests
-      env:
-        YTDL_TEST_SET: core
-      run: ./devscripts/run_tests.sh
+      run: ./devscripts/run_tests.sh core
  flake8:
    name: Linter
    if: "!contains(github.event.head_commit.message, 'ci skip all')"
@@ -30,4 +28,4 @@ jobs:
    - name: Install flake8
      run: pip install flake8
    - name: Run flake8
-      run: flake8 .
+      run: flake8 .
@@ -33,6 +33,7 @@ cookies.txt
 *.info.json
 *.live_chat.json
 *.jpg
+*.jpeg
 *.png
 *.webp
 *.annotations.xml
@@ -44,6 +45,7 @@ cookies.txt
 # Python
 *.pyc
 *.pyo
+.pytest_cache
 wine-py2exe/
 py2exe.log
 build/
@@ -78,6 +80,7 @@ README.txt
 *.tar.gz
 *.zsh
 *.spec
+test/testdata/player-*.js

 # Binary
 /youtube-dl
@@ -81,16 +81,17 @@ To run the test, simply invoke your favorite test runner, or execute a test file
    python -m unittest discover
    python test/test_download.py
    nosetests
+    pytest

 See item 6 of [new extractor tutorial](#adding-support-for-a-new-site) for how to run extractor specific test cases.

 If you want to create a build of youtube-dl yourself, you'll need

-* python
+* python3
 * make (only GNU make is supported)
 * pandoc
 * zip
-* nosetests
+* pytest

 ### Adding support for a new site

@@ -1,6 +1,6 @@
 pukkandan (owner)
 shirt-dev (collaborator)
-colethedj (collaborator)
+coletdjnz/colethedj (collaborator)
 Ashish0804 (collaborator)
 h-h-h-h
 pauldubois98
@@ -58,3 +58,12 @@ krichbanana
 ohmybahgosh
 nyuszika7h
 blackjack4494
+pyx
+TpmKranz
+mzbaulhaque
+zackmark29
+mbway
+zerodytrash
+wesnm
+pento
+rigstot
@@ -19,14 +19,135 @@
 -->


+### 2021.08.02
+
+* Add logo, banner and donate links
+* Expand and escape environment variables correctly in output template
+* Add format types `j` (json), `l` (comma delimited list), `q` (quoted for terminal) in output template
+* [downloader] Allow streaming some unmerged formats to stdout using ffmpeg
+* [youtube] **Age-gate bypass**
+    * Add `agegate` clients by [pukkandan](https://github.com/pukkandan), [MinePlayersPE](https://github.com/MinePlayersPE)
+    * Add `thirdParty` to agegate clients to bypass more videos
+    * Simplify client definitions, expose `embedded` clients
+    * Improve age-gate detection by [coletdjnz](https://github.com/coletdjnz)
+    * Fix default global API key by [coletdjnz](https://github.com/coletdjnz)
+    * Add `creator` clients for age-gate bypass using unverified accounts by [zerodytrash](https://github.com/zerodytrash), [coletdjnz](https://github.com/coletdjnz), [pukkandan](https://github.com/pukkandan)
+* [adobepass] Add MSO Sling TV by [wesnm](https://github.com/wesnm)
+* [CBS] Add ParamountPlusSeriesIE by [Ashish0804](https://github.com/Ashish0804)
+* [dplay] Add `ScienceChannelIE` by [Sipherdrakon](https://github.com/Sipherdrakon)
+* [UtreonIE] Add extractor by [Ashish0804](https://github.com/Ashish0804)
+* [youtube] Add `mweb` client by [coletdjnz](https://github.com/coletdjnz)
+* [youtube] Add `player_client=all`
+* [youtube] Force `hl=en` for comments by [coletdjnz](https://github.com/coletdjnz)
+* [youtube] Fix format sorting when using alternate clients
+* [youtube] Misc cleanup by [pukkandan](https://github.com/pukkandan), [coletdjnz](https://github.com/coletdjnz)
+* [youtube] Extract SAPISID only once
+* [CBS] Add fallback by [llacb47](https://github.com/llacb47), [pukkandan](https://github.com/pukkandan)
+* [Hotstar] Support cookies by [Ashish0804](https://github.com/Ashish0804)
+* [HotStarSeriesIE] Fix regex by [Ashish0804](https://github.com/Ashish0804)
+* [bilibili] Improve `_VALID_URL`
+* [mediaset] Fix extraction by [nixxo](https://github.com/nixxo)
+* [Mxplayer] Add h265 formats by [Ashish0804](https://github.com/Ashish0804)
+* [RCTIPlus] Remove PhantomJS dependency by [MinePlayersPE](https://github.com/MinePlayersPE)
+* [tenplay] Add MA15+ age limit by [pento](https://github.com/pento)
+* [vidio] Fix login error detection by [MinePlayersPE](https://github.com/MinePlayersPE)
+* [vimeo] Better extraction of original file by [Ashish0804](https://github.com/Ashish0804)
+* [generic] Support KVS player (replaces ThisVidIE) by [rigstot](https://github.com/rigstot)
+* Add compat-option `no-clean-infojson`
+* Remove `asr` appearing twice in `-F`
+* Set `home:` as the default key for `-P`
+* [utils] Fix slicing of reversed `LazyList`
+* [FormatSort] Fix bug for audio with unknown codec
+* [test:download] Support testing with `ignore_no_formats_error`
+* [cleanup] Refactor some code
+
+
+### 2021.07.24
+
+* [youtube:tab] Extract video duration early
+* [downloader] Pass `info_dict` to `progress_hook`s
+* [youtube] Fix age-gated videos for API clients when cookies are supplied by [coletdjnz](https://github.com/coletdjnz)
+* [youtube] Disable `get_video_info` age-gate workaround - This endpoint seems to be completely dead
+* [youtube] Try all clients even if age-gated
+* [youtube] Fix subtitles only being extracted from the first client
+* [youtube] Simplify `_get_text`
+* [cookies] bugfix for microsoft edge on macOS
+* [cookies] Handle `sqlite` `ImportError` gracefully by [mbway](https://github.com/mbway)
+* [cookies] Handle errors when importing `keyring`
+
+### 2021.07.21
+
+* **Add option `--cookies-from-browser`** to load cookies from a browser by [mbway](https://github.com/mbway)
+    * Usage: `--cookies-from-browser BROWSER[:PROFILE_NAME_OR_PATH]`
+    * Also added `--no-cookies-from-browser`
+    * To decrypt chromium cookies, `keyring` is needed for UNIX and `pycryptodome` for Windows
+* Add option `--exec-before-download`
+* Add field `live_status`
+* [FFmpegMetadata] Add language of each stream and some refactoring
+* [douyin] Add extractor by [pukkandan](https://github.com/pukkandan), [pyx](https://github.com/pyx)
+* [pornflip] Add extractor by [mzbaulhaque](https://github.com/mzbaulhaque)
+* **[youtube] Extract data from multiple clients** by [pukkandan](https://github.com/pukkandan), [coletdjnz](https://github.com/coletdjnz)
+    * `player_client` now accepts multiple clients
+    * Default `player_client` = `android,web`
+        * This uses twice as many requests, but avoids throttling for most videos while also not losing any formats
+    * Music clients can be specifically requested and is enabled by default if `music.youtube.com`
+    * Added `player_client=ios` (Known issue: formats from ios are not sorted correctly)
+    * Add age-gate bypass for android and ios clients
+* [youtube] Extract more thumbnails
+    * The thumbnail URLs are hard-coded and their actual existence is tested lazily
+    * Added option `--no-check-formats` to not test them
+* [youtube] Misc fixes
+    * Improve extraction of livestream metadata by [pukkandan](https://github.com/pukkandan), [krichbanana](https://github.com/krichbanana)
+    * Hide live dash formats since they can't be downloaded anyway
+    * Fix authentication when using multiple accounts by [coletdjnz](https://github.com/coletdjnz)
+    * Fix controversial videos when requested via API by [coletdjnz](https://github.com/coletdjnz)
+    * Fix session index extraction and headers for non-web player clients by [coletdjnz](https://github.com/coletdjnz)
+    * Make `--extractor-retries` work for more errors
+    * Fix sorting of 3gp format
+    * Sanity check `chapters` (and refactor related code)
+    * Make `parse_time_text` and `_extract_chapters` non-fatal
+    * Misc cleanup and bug fixes by [coletdjnz](https://github.com/coletdjnz)
+* [youtube:tab] Fix channels tab
+* [youtube:tab] Extract playlist availability by [coletdjnz](https://github.com/coletdjnz)
+* **[youtube:comments] Move comment extraction to new API** by [coletdjnz](https://github.com/coletdjnz)
+    * Adds extractor-args `comment_sort` (`top`/`new`), `max_comments`, `max_comment_depth`
+* [youtube:comments] Fix `is_favorited`, improve `like_count` parsing by [coletdjnz](https://github.com/coletdjnz)
+* [BravoTV] Improve metadata extraction by [kevinoconnor7](https://github.com/kevinoconnor7)
+* [crunchyroll:playlist] Force http
+* [yahoo:gyao:player] Relax `_VALID_URL` by [nao20010128nao](https://github.com/nao20010128nao)
+* [nebula] Authentication via tokens from cookie jar by [hheimbuerger](https://github.com/hheimbuerger), [TpmKranz](https://github.com/TpmKranz)
+* [RTP] Fix extraction and add subtitles by [fstirlitz](https://github.com/fstirlitz)
+* [viki] Rewrite extractors and add extractor-arg `video_types` to `vikichannel` by [zackmark29](https://github.com/zackmark29), [pukkandan](https://github.com/pukkandan)
+* [vlive] Extract thumbnail directly in addition to the one from Naver
+* [generic] Extract previously missed subtitles by [fstirlitz](https://github.com/fstirlitz)
+* [generic] Extract everything in the SMIL manifest and detect discarded subtitles by [fstirlitz](https://github.com/fstirlitz)
+* [embedthumbnail] Fix `_get_thumbnail_resolution`
+* [metadatafromfield] Do not detect numbers as field names
+* Fix selectors `all`, `mergeall` and add tests
+* Errors in playlist extraction should obey `--ignore-errors`
+* Fix bug where `original_url` was not propagated when `_type`=`url`
+* Revert "Merge webm formats into mkv if thumbnails are to be embedded (#173)"
+    * This was wrongly checking for `write_thumbnail`
+* Improve `extractor_args` parsing
+* Rename `NOTE` in `-F` to `MORE INFO` since it's often confused to be the same as `format_note`
+* Add `only_once` param for `write_debug` and `report_warning`
+* [extractor] Allow extracting multiple groups in `_search_regex` by [fstirlitz](https://github.com/fstirlitz)
+* [utils] Improve `traverse_obj`
+* [utils] Add `variadic`
+* [utils] Improve `js_to_json` comment regex by [fstirlitz](https://github.com/fstirlitz)
+* [webtt] Fix timestamps
+* [compat] Remove unnecessary code
+* [docs] fix default of multistreams
+
+
 ### 2021.07.07

 * Merge youtube-dl: Upto [commit/a803582](https://github.com/ytdl-org/youtube-dl/commit/a8035827177d6b59aca03bd717acb6a9bdd75ada)
-* Add `--extractor-args` to pass extractor-specific arguments
+* Add `--extractor-args` to pass some extractor-specific arguments. See [readme](https://github.com/yt-dlp/yt-dlp#extractor-arguments)
    * Add extractor option `skip` for `youtube`. Eg: `--extractor-args youtube:skip=hls,dash`
-    * Deprecates --youtube-skip-dash-manifest, --youtube-skip-hls-manifest, --youtube-include-dash-manifest, --youtube-include-hls-manifest
+    * Deprecates `--youtube-skip-dash-manifest`, `--youtube-skip-hls-manifest`, `--youtube-include-dash-manifest`, `--youtube-include-hls-manifest`
 * Allow `--list...` options to work with `--print`, `--quiet` and other `--list...` options
-* [youtube] Use `player` API for additional video extraction requests by [colethedj](https://github.com/colethedj)
+* [youtube] Use `player` API for additional video extraction requests by [coletdjnz](https://github.com/coletdjnz)
    * **Fixes youtube premium music** (format 141) extraction
    * Adds extractor option `player_client` = `web`/`android`
        * **`--extractor-args youtube:player_client=android` works around the throttling** for the time-being
@@ -34,7 +155,7 @@
    * Adds age-gate fallback using embedded client
 * [youtube] Choose correct Live chat API for upcoming streams by [krichbanana](https://github.com/krichbanana)
 * [youtube] Fix subtitle names for age-gated videos
-* [youtube:comments] Fix error handling and add `itct` to params by [colethedj](https://github.com/colethedj)
+* [youtube:comments] Fix error handling and add `itct` to params by [coletdjnz](https://github.com/coletdjnz)
 * [youtube_live_chat] Fix download with cookies by [siikamiika](https://github.com/siikamiika)
 * [youtube_live_chat] use `clickTrackingParams` by [siikamiika](https://github.com/siikamiika)
 * [Funimation] Rewrite extractor
@@ -81,9 +202,9 @@
 * [downloader/mhtml] Add new downloader for slideshows/storyboards by [fstirlitz](https://github.com/fstirlitz)
 * [youtube] Temporary **fix for age-gate**
 * [youtube] Support ongoing live chat by [siikamiika](https://github.com/siikamiika)
-* [youtube] Improve SAPISID cookie handling by [colethedj](https://github.com/colethedj)
+* [youtube] Improve SAPISID cookie handling by [coletdjnz](https://github.com/coletdjnz)
 * [youtube] Login is not needed for `:ytrec`
-* [youtube] Non-fatal alert reporting for unavailable videos page by [colethedj](https://github.com/colethedj)
+* [youtube] Non-fatal alert reporting for unavailable videos page by [coletdjnz](https://github.com/coletdjnz)
 * [twitcasting] Websocket support by [nao20010128nao](https://github.com/nao20010128nao)
 * [mediasite] Extract slides by [fstirlitz](https://github.com/fstirlitz)
 * [funimation] Extract subtitles 
@@ -141,7 +262,7 @@
 * Merge youtube-dl: Upto [commit/d495292](https://github.com/ytdl-org/youtube-dl/commit/d495292852b6c2f1bd58bc2141ff2b0265c952cf)
 * Pre-check archive and filters during playlist extraction
 * Handle Basic Auth `user:pass` in URLs by [hhirtz](https://github.com/hhirtz) and [pukkandan](https://github.com/pukkandan)
-* [archiveorg] Add YoutubeWebArchiveIE by [colethedj](https://github.com/colethedj) and [alex-gedeon](https://github.com/alex-gedeon)
+* [archiveorg] Add YoutubeWebArchiveIE by [coletdjnz](https://github.com/coletdjnz) and [alex-gedeon](https://github.com/alex-gedeon)
 * [fancode] Add extractor by [rhsmachine](https://github.com/rhsmachine)
 * [patreon] Support vimeo embeds by [rhsmachine](https://github.com/rhsmachine)
 * [Saitosan] Add new extractor by [llacb47](https://github.com/llacb47)
@@ -184,7 +305,7 @@

 * **Youtube improvements**: 
    * Support youtube music `MP`, `VL` and `browse` pages
-    * Extract more formats for youtube music by [craftingmod](https://github.com/craftingmod), [colethedj](https://github.com/colethedj) and [pukkandan](https://github.com/pukkandan)
+    * Extract more formats for youtube music by [craftingmod](https://github.com/craftingmod), [coletdjnz](https://github.com/coletdjnz) and [pukkandan](https://github.com/pukkandan)
    * Extract multiple subtitles in same language by [pukkandan](https://github.com/pukkandan) and [tpikonen](https://github.com/tpikonen)
    * Redirect channels that doesn't have a `videos` tab to their `UU` playlists
    * Support in-channel search
@@ -193,7 +314,7 @@
    * Extract audio language
    * Add subtitle language names by [nixxo](https://github.com/nixxo) and [tpikonen](https://github.com/tpikonen)
    * Show alerts only from the final webpage
-    * Add `html5=1` param to `get_video_info` page requests by [colethedj](https://github.com/colethedj)
+    * Add `html5=1` param to `get_video_info` page requests by [coletdjnz](https://github.com/coletdjnz)
    * Better message when login required
 * **Add option `--print`**: to print any field/template
    * Deprecates: `--get-description`, `--get-duration`, `--get-filename`, `--get-format`, `--get-id`, `--get-thumbnail`, `--get-title`, `--get-url`
@@ -290,11 +411,11 @@
 * [TubiTv] Add TubiTvShowIE by [Ashish0804](https://github.com/Ashish0804)
 * [twitcasting] Fix extractor
 * [viu:ott] Fix extractor and support series by [lkho](https://github.com/lkho) and [pukkandan](https://github.com/pukkandan)
-* [youtube:tab] Show unavailable videos in playlists by [colethedj](https://github.com/colethedj)
+* [youtube:tab] Show unavailable videos in playlists by [coletdjnz](https://github.com/coletdjnz)
 * [youtube:tab] Reload with unavailable videos for all playlists
 * [youtube] Ignore invalid stretch ratio
-* [youtube] Improve channel syncid extraction to support ytcfg by [colethedj](https://github.com/colethedj)
-* [youtube] Standardize API calls for tabs, mixes and search by [colethedj](https://github.com/colethedj)
+* [youtube] Improve channel syncid extraction to support ytcfg by [coletdjnz](https://github.com/coletdjnz)
+* [youtube] Standardize API calls for tabs, mixes and search by [coletdjnz](https://github.com/coletdjnz)
 * [youtube] Bugfix in `_extract_ytcfg`
 * [mildom:user:vod] Download only necessary amount of pages
 * [mildom] Remove proxy completely by [fstirlitz](https://github.com/fstirlitz)
@@ -306,8 +427,8 @@
 * Improve the yt-dlp.sh script by [fstirlitz](https://github.com/fstirlitz)
 * [lazy_extractor] Do not load plugins
 * [ci] Disable fail-fast
-* [documentation] Clarify which deprecated options still work
-* [documentation] Fix typos
+* [docs] Clarify which deprecated options still work
+* [docs] Fix typos


 ### 2021.04.11
@@ -324,17 +445,17 @@
 * [nitter] Fix extraction of reply tweets and update instance list by [B0pol](https://github.com/B0pol)
 * [nitter] Fix thumbnails by [B0pol](https://github.com/B0pol)
 * [youtube] Fix thumbnail URL
-* [youtube] Parse API parameters from initial webpage by [colethedj](https://github.com/colethedj)
-* [youtube] Extract comments' approximate timestamp by [colethedj](https://github.com/colethedj)
+* [youtube] Parse API parameters from initial webpage by [coletdjnz](https://github.com/coletdjnz)
+* [youtube] Extract comments' approximate timestamp by [coletdjnz](https://github.com/coletdjnz)
 * [youtube] Fix alert extraction
 * [bilibili] Fix uploader
-* [utils] Add `datetime_from_str` and `datetime_add_months` by [colethedj](https://github.com/colethedj)
+* [utils] Add `datetime_from_str` and `datetime_add_months` by [coletdjnz](https://github.com/coletdjnz)
 * Run some `postprocessors` before actual download
 * Improve argument parsing for `-P`, `-o`, `-S`
 * Fix some `m3u8` not obeying `--allow-unplayable-formats`
 * Fix default of `dynamic_mpd`
 * Deprecate `--all-formats`, `--include-ads`, `--hls-prefer-native`, `--hls-prefer-ffmpeg`
-* [documentation] Improvements
+* [docs] Improvements

 ### 2021.04.03
 * Merge youtube-dl: Upto [commit/654b4f4](https://github.com/ytdl-org/youtube-dl/commit/654b4f4ff2718f38b3182c1188c5d569c14cc70a)
@@ -345,10 +466,10 @@
 * [mildom] Update extractor with current proxy by [nao20010128nao](https://github.com/nao20010128nao)
 * [ard:mediathek] Fix video id extraction
 * [generic] Detect Invidious' link element
-* [youtube] Show premium state in `availability` by [colethedj](https://github.com/colethedj)
+* [youtube] Show premium state in `availability` by [coletdjnz](https://github.com/coletdjnz)
 * [viewsource] Add extractor to handle `view-source:`
 * [sponskrub] Run before embedding thumbnail
-* [documentation] Improve `--parse-metadata` documentation
+* [docs] Improve `--parse-metadata` documentation


 ### 2021.03.24.1
@@ -380,8 +501,8 @@
 * Use headers and cookies when downloading subtitles by [damianoamatruda](https://github.com/damianoamatruda)
 * Parse resolution in info dictionary by [damianoamatruda](https://github.com/damianoamatruda)
 * More consistent warning messages by [damianoamatruda](https://github.com/damianoamatruda) and [pukkandan](https://github.com/pukkandan)
-* [documentation] Add deprecated options and aliases in readme
-* [documentation] Fix some minor mistakes
+* [docs] Add deprecated options and aliases in readme
+* [docs] Fix some minor mistakes

 * [niconico] Partial fix adapted from [animelover1984/youtube-dl@b5eff52](https://github.com/animelover1984/youtube-dl/commit/b5eff52dd9ed5565672ea1694b38c9296db3fade) (login and smile formats still don't work)
 * [niconico] Add user extractor by [animelover1984](https://github.com/animelover1984)
@@ -390,7 +511,7 @@
 * [stitcher] Merge from youtube-dl by [nixxo](https://github.com/nixxo)
 * [rcs] Improved extraction by [nixxo](https://github.com/nixxo)
 * [linuxacadamy] Improve regex
-* [youtube] Show if video is `private`, `unlisted` etc in info (`availability`) by [colethedj](https://github.com/colethedj) and [pukkandan](https://github.com/pukkandan)
+* [youtube] Show if video is `private`, `unlisted` etc in info (`availability`) by [coletdjnz](https://github.com/coletdjnz) and [pukkandan](https://github.com/pukkandan)
 * [youtube] bugfix for channel playlist extraction
 * [nbc] Improve metadata extraction by [2ShedsJackson](https://github.com/2ShedsJackson)

@@ -407,15 +528,15 @@
 * [wimtv] Add extractor by [nixxo](https://github.com/nixxo)
 * [mtv] Add mtv.it and extract series metadata by [nixxo](https://github.com/nixxo)
 * [pluto.tv] Add extractor by [kevinoconnor7](https://github.com/kevinoconnor7)
-* [youtube] Rewrite comment extraction by [colethedj](https://github.com/colethedj)
+* [youtube] Rewrite comment extraction by [coletdjnz](https://github.com/coletdjnz)
 * [embedthumbnail] Set mtime correctly
 * Refactor some postprocessor/downloader code by [pukkandan](https://github.com/pukkandan) and [shirt](https://github.com/shirt-dev)


 ### 2021.03.07
-* [youtube] Fix history, mixes, community pages and trending by [pukkandan](https://github.com/pukkandan) and [colethedj](https://github.com/colethedj)
-* [youtube] Fix private feeds/playlists on multi-channel accounts by [colethedj](https://github.com/colethedj)
-* [youtube] Extract alerts from continuation by [colethedj](https://github.com/colethedj)
+* [youtube] Fix history, mixes, community pages and trending by [pukkandan](https://github.com/pukkandan) and [coletdjnz](https://github.com/coletdjnz)
+* [youtube] Fix private feeds/playlists on multi-channel accounts by [coletdjnz](https://github.com/coletdjnz)
+* [youtube] Extract alerts from continuation by [coletdjnz](https://github.com/coletdjnz)
 * [cbs] Add support for ParamountPlus by [shirt](https://github.com/shirt-dev)
 * [mxplayer] Rewrite extractor with show support by [pukkandan](https://github.com/pukkandan) and [Ashish0804](https://github.com/Ashish0804)
 * [gedi] Improvements from youtube-dl by [nixxo](https://github.com/nixxo)
@@ -427,7 +548,7 @@
 * [downloader] Fix bug for `ffmpeg`/`httpie`
 * [update] Fix updater removing the executable bit on some UNIX distros
 * [update] Fix current build hash for UNIX
-* [documentation] Include wget/curl/aria2c install instructions for Unix by [Ashish0804](https://github.com/Ashish0804)
+* [docs] Include wget/curl/aria2c install instructions for Unix by [Ashish0804](https://github.com/Ashish0804)
 * Fix some videos downloading with `m3u8` extension
 * Remove "fixup is ignored" warning when fixup wasn't passed by user

@@ -436,7 +557,7 @@
 * [build] Fix bug

 ### 2021.03.03
-* [youtube] Use new browse API for continuation page extraction by [colethedj](https://github.com/colethedj) and [pukkandan](https://github.com/pukkandan)
+* [youtube] Use new browse API for continuation page extraction by [coletdjnz](https://github.com/coletdjnz) and [pukkandan](https://github.com/pukkandan)
 * Fix HLS playlist downloading by [shirt](https://github.com/shirt-dev)
 * Merge youtube-dl: Upto [2021.03.03](https://github.com/ytdl-org/youtube-dl/releases/tag/2021.03.03)
 * [mtv] Fix extractor
@@ -484,7 +605,7 @@
 * [ffmpeg] Allow passing custom arguments before -i using `--ppa "ffmpeg_i1:ARGS"` syntax
 * Fix `--windows-filenames` removing `/` from UNIX paths
 * [hls] Show warning if pycryptodome is not found
-* [documentation] Improvements
+* [docs] Improvements
    * Fix documentation of `Extractor Options`
    * Document `all` in format selection
    * Document `playable_in_embed` in output templates
@@ -512,7 +633,7 @@
    * Exclude `vcruntime140.dll` from UPX by [jbruchon](https://github.com/jbruchon)
    * Set version number based on UTC time, not local time
    * Publish on PyPi only if token is set
-* [documentation] Better document `--prefer-free-formats` and add `--no-prefer-free-format`
+* [docs] Better document `--prefer-free-formats` and add `--no-prefer-free-format`


 ### 2021.02.15
@@ -555,7 +676,7 @@
 * [movefiles] Fix compatibility with python2
 * [remuxvideo] Fix validation of conditional remux
 * [sponskrub] Don't raise error when the video does not exist
-* [documentation] Crypto is an optional dependency
+* [docs] Crypto is an optional dependency


 ### 2021.02.04
@@ -616,10 +737,10 @@
 * Merge youtube-dl: Upto [2021.01.24](https://github.com/ytdl-org/youtube-dl/releases/tag/2021.01.16)
 * Plugin support ([documentation](https://github.com/yt-dlp/yt-dlp#plugins))
 * **Multiple paths**: New option `-P`/`--paths` to give different paths for different types of files
-    * The syntax is `-P "type:path" -P "type:path"` ([documentation](https://github.com/yt-dlp/yt-dlp#:~:text=-P,%20--paths%20TYPE:PATH))
+    * The syntax is `-P "type:path" -P "type:path"`
    * Valid types are: home, temp, description, annotation, subtitle, infojson, thumbnail
-    * Additionally, configuration file is taken from home directory or current directory ([documentation](https://github.com/yt-dlp/yt-dlp#:~:text=Home%20Configuration))
-* Allow passing different arguments to different external downloaders ([documentation](https://github.com/yt-dlp/yt-dlp#:~:text=--downloader-args%20NAME:ARGS))
+    * Additionally, configuration file is taken from home directory or current directory
+* Allow passing different arguments to different external downloaders
 * [mildom] Add extractor by [nao20010128nao](https://github.com/nao20010128nao)
 * Warn when using old style `--external-downloader-args` and `--post-processor-args`
 * Fix `--no-overwrite` when using `--write-link`
@@ -654,9 +775,9 @@
 * [roosterteeth.com] Fix for bonus episodes by [Zocker1999NET](https://github.com/Zocker1999NET)
 * [tiktok] Fix for when share_info is empty
 * [EmbedThumbnail] Fix bug due to incorrect function name
-* [documentation] Changed sponskrub links to point to [yt-dlp/SponSkrub](https://github.com/yt-dlp/SponSkrub) since I am now providing both linux and windows releases
-* [documentation] Change all links to correctly point to new fork URL
-* [documentation] Fixes typos
+* [docs] Changed sponskrub links to point to [yt-dlp/SponSkrub](https://github.com/yt-dlp/SponSkrub) since I am now providing both linux and windows releases
+* [docs] Change all links to correctly point to new fork URL
+* [docs] Fixes typos


 ### 2021.01.12
@@ -752,7 +873,7 @@
    * Redirect channel home to /video
    * Print youtube's warning message
    * Handle Multiple pages for feeds better
-* [youtube] Fix ytsearch not returning results sometimes due to promoted content by [colethedj](https://github.com/colethedj)
+* [youtube] Fix ytsearch not returning results sometimes due to promoted content by [coletdjnz](https://github.com/coletdjnz)
 * [youtube] Temporary fix for automatic captions - disable json3 by [blackjack4494](https://github.com/blackjack4494)
 * Add --break-on-existing by [gergesh](https://github.com/gergesh)
 * Pre-check video IDs in the archive before downloading by [pukkandan](https://github.com/pukkandan)
@@ -0,0 +1,37 @@
+# Collaborators
+
+This is a list of the collaborators of the project and their major contributions. See the [Changelog](Changelog.md) for more details.
+
+You can also find lists of all [contributors of yt-dlp](CONTRIBUTORS) and [authors of youtube-dl](https://github.com/ytdl-org/youtube-dl/blob/master/AUTHORS)
+
+
+## [pukkandan](https://github.com/pukkandan)
+
+[![ko-fi](https://img.shields.io/badge/_-Ko--fi-red.svg?logo=kofi&labelColor=555555&style=for-the-badge)](https://ko-fi.com/pukkandan)
+
+* Owner of the fork
+
+
+
+## [shirt](https://github.com/shirt-dev)
+
+* Multithreading (`-N`) and aria2c support for fragment downloads
+* Support for media initialization and discontinuity in HLS
+* The self-updater (`-U`)
+
+
+
+## [coletdjnz](https://github.com/coletdjnz)
+
+[![gh-sponsor](https://img.shields.io/badge/_-Sponsor-red.svg?logo=githubsponsors&labelColor=555555&style=for-the-badge)](https://github.com/sponsors/coletdjnz)
+
+* YouTube improvements including: age-gate bypass, private playlists, multiple-clients (to avoid throttling) and a lot of under-the-hood improvements
+
+
+
+## [Ashish0804](https://github.com/Ashish0804)
+
+[![ko-fi](https://img.shields.io/badge/_-Ko--fi-red.svg?logo=kofi&labelColor=555555&style=for-the-badge)](https://ko-fi.com/ashish0804)
+
+* Added support for new websites Zee5, MXPlayer, DiscoveryPlusIndia, ShemarooMe, Utreon etc
+* Added playlist/series downloads for TubiTv, SonyLIV, Voot, HotStar etc
@@ -13,7 +13,7 @@ pypi-files: AUTHORS Changelog.md LICENSE README.md README.txt supportedsites com
 .PHONY: all clean install test tar pypi-files completions ot offlinetest codetest supportedsites

 clean-test:
-	rm -rf *.dump *.part* *.ytdl *.info.json *.mp4 *.m4a *.flv *.mp3 *.avi *.mkv *.webm *.3gp *.wav *.ape *.swf *.jpg *.png *.frag *.frag.urls *.frag.aria2
+	rm -rf *.dump *.part* *.ytdl *.info.json *.mp4 *.m4a *.flv *.mp3 *.avi *.mkv *.webm *.3gp *.wav *.ape *.swf *.jpg *.png *.frag *.frag.urls *.frag.aria2 test/testdata/player-*.js
 clean-dist:
 	rm -rf yt-dlp.1.temp.md yt-dlp.1 README.txt MANIFEST build/ dist/ .coverage cover/ yt-dlp.tar.gz completions/ yt_dlp/extractor/lazy_extractors.py *.spec CONTRIBUTING.md.tmp yt-dlp yt-dlp.exe yt_dlp.egg-info/ AUTHORS .mailmap
 clean-cache:
@@ -49,23 +49,11 @@ codetest:
 	flake8 .

 test:
-	#nosetests --with-coverage --cover-package=yt_dlp --cover-html --verbose --processes 4 test
-	nosetests --verbose test
+	$(PYTHON) -m pytest
 	$(MAKE) codetest

-# Keep this list in sync with devscripts/run_tests.sh
 offlinetest: codetest
-	$(PYTHON) -m nose --verbose test \
-		--exclude test_age_restriction.py \
-		--exclude test_download.py \
-		--exclude test_iqiyi_sdk_interpreter.py \
-		--exclude test_overwrites.py \
-		--exclude test_socks.py \
-		--exclude test_subtitles.py \
-		--exclude test_write_annotations.py \
-		--exclude test_youtube_lists.py \
-		--exclude test_youtube_signature.py \
-		--exclude test_post_hooks.py
+	$(PYTHON) -m pytest -k "not download"

 yt-dlp: yt_dlp/*.py yt_dlp/*/*.py
 	mkdir -p zip
@@ -1,17 +1,16 @@
 <div align="center">

-# YT-DLP
-A command-line program to download videos from YouTube and many other [video platforms](supportedsites.md)
+[![YT-DLP](https://github.com/yt-dlp/yt-dlp/tree/master/.github/banner.svg)](#readme)

-<!-- GHA doesn't have for-the-badge style
-[![CI Status](https://github.com/yt-dlp/yt-dlp/workflows/Core%20Tests/badge.svg?branch=master)](https://github.com/yt-dlp/yt-dlp/actions)
-->
-[![Release version](https://img.shields.io/github/v/release/yt-dlp/yt-dlp?color=brightgreen&label=Release&style=for-the-badge)](https://github.com/yt-dlp/yt-dlp/releases/latest)
-[![License: Unlicense](https://img.shields.io/badge/License-Unlicense-blue.svg?style=for-the-badge)](LICENSE)
+[![Release version](https://img.shields.io/github/v/release/yt-dlp/yt-dlp?color=blue&label=&style=for-the-badge)](https://github.com/yt-dlp/yt-dlp/releases/latest)
+[![CI Status](https://img.shields.io/github/workflow/status/yt-dlp/yt-dlp/Core%20Tests/master?label=&style=for-the-badge)](https://github.com/yt-dlp/yt-dlp/actions)
+[![License: Unlicense](https://img.shields.io/badge/-Unlicense-blue.svg?style=for-the-badge)](LICENSE)
+[![Donate](https://img.shields.io/badge/_-Donate-red.svg?logo=githubsponsors&labelColor=555555&style=for-the-badge)](Collaborators.md)
+[![Supported Sites](https://img.shields.io/badge/-Supported_Sites-brightgreen.svg?style=for-the-badge)](supportedsites.md)
+[![Discord](https://img.shields.io/discord/807245652072857610?color=blue&label=&logo=discord&style=for-the-badge)](https://discord.gg/H5MNcFW63r)
 [![Doc Status](https://readthedocs.org/projects/yt-dlp/badge/?version=latest&style=for-the-badge)](https://yt-dlp.readthedocs.io)
-[![Discord](https://img.shields.io/discord/807245652072857610?color=blue&label=discord&logo=discord&style=for-the-badge)](https://discord.gg/H5MNcFW63r)
 [![Commits](https://img.shields.io/github/commit-activity/m/yt-dlp/yt-dlp?label=commits&style=for-the-badge)](https://github.com/yt-dlp/yt-dlp/commits)
-[![Last Commit](https://img.shields.io/github/last-commit/yt-dlp/yt-dlp/master?style=for-the-badge)](https://github.com/yt-dlp/yt-dlp/commits)
+[![Last Commit](https://img.shields.io/github/last-commit/yt-dlp/yt-dlp/master?label=&style=for-the-badge)](https://github.com/yt-dlp/yt-dlp/commits)
 [![Downloads](https://img.shields.io/github/downloads/yt-dlp/yt-dlp/total?style=for-the-badge&color=blue)](https://github.com/yt-dlp/yt-dlp/releases/latest)
 [![PyPi Downloads](https://img.shields.io/pypi/dm/yt-dlp?label=PyPi&style=for-the-badge)](https://pypi.org/project/yt-dlp)

@@ -72,22 +71,26 @@ The major new features from the latest release of [blackjack4494/yt-dlc](https:/
 * **Merged with animelover1984/youtube-dl**: You get most of the features and improvements from [animelover1984/youtube-dl](https://github.com/animelover1984/youtube-dl) including `--write-comments`, `BiliBiliSearch`, `BilibiliChannel`, Embedding thumbnail in mp4/ogg/opus, playlist infojson etc. Note that the NicoNico improvements are not available. See [#31](https://github.com/yt-dlp/yt-dlp/pull/31) for details.

 * **Youtube improvements**:
-    * All Feeds (`:ytfav`, `:ytwatchlater`, `:ytsubs`, `:ythistory`, `:ytrec`) supports downloading multiple pages of content
+    * All Feeds (`:ytfav`, `:ytwatchlater`, `:ytsubs`, `:ythistory`, `:ytrec`) and private playlists supports downloading multiple pages of content
    * Search (`ytsearch:`, `ytsearchdate:`), search URLs and in-channel search works
    * Mixes supports downloading multiple pages of content
+    * Most (but not all) age-gated content can be downloaded without cookies
+    * Partial workaround for throttling issue
    * Redirect channel's home URL automatically to `/video` to preserve the old behaviour
    * `255kbps` audio is extracted from youtube music if premium cookies are given
    * Youtube music Albums, channels etc can be downloaded

+* **Cookies from browser**: Cookies can be automatically extracted from all major web browsers using `--cookies-from-browser BROWSER[:PROFILE]`
+
 * **Split video by chapters**: Videos can be split into multiple files based on chapters using `--split-chapters`

 * **Multi-threaded fragment downloads**: Download multiple fragments of m3u8/mpd videos in parallel. Use `--concurrent-fragments` (`-N`) option to set the number of threads used

 * **Aria2c with HLS/DASH**: You can use `aria2c` as the external downloader for DASH(mpd) and HLS(m3u8) formats

-* **New extractors**: AnimeLab, Philo MSO, Spectrum MSO, Rcs, Gedi, bitwave.tv, mildom, audius, zee5, mtv.it, wimtv, pluto.tv, niconico users, discoveryplus.in, mediathek, NFHSNetwork, nebula, ukcolumn, whowatch, MxplayerShow, parlview (au), YoutubeWebArchive, fancode, Saitosan, ShemarooMe, telemundo, VootSeries, SonyLIVSeries, HotstarSeries, VidioPremier, VidioLive, RCTIPlus, TBS Live
+* **New extractors**: AnimeLab, Philo MSO, Spectrum MSO, SlingTV MSO, Rcs, Gedi, bitwave.tv, mildom, audius, zee5, mtv.it, wimtv, pluto.tv, niconico users, discoveryplus.in, mediathek, NFHSNetwork, nebula, ukcolumn, whowatch, MxplayerShow, parlview (au), YoutubeWebArchive, fancode, Saitosan, ShemarooMe, telemundo, VootSeries, SonyLIVSeries, HotstarSeries, VidioPremier, VidioLive, RCTIPlus, TBS Live, douyin, pornflip, ParamountPlusSeries, ScienceChannel, Utreon

-* **Fixed extractors**: archive.org, roosterteeth.com, skyit, instagram, itv, SouthparkDe, spreaker, Vlive, akamai, ina, rumble, tennistv, amcnetworks, la7 podcasts, linuxacadamy, nitter, twitcasting, viu, crackle, curiositystream, mediasite, rmcdecouverte, sonyliv, tubi, tenplay, patreon, videa, yahoo
+* **Fixed/improved extractors**: archive.org, roosterteeth.com, skyit, instagram, itv, SouthparkDe, spreaker, Vlive, akamai, ina, rumble, tennistv, amcnetworks, la7 podcasts, linuxacadamy, nitter, twitcasting, viu, crackle, curiositystream, mediasite, rmcdecouverte, sonyliv, tubi, tenplay, patreon, videa, yahoo, BravoTV, crunchyroll playlist, RTP, viki, Hotstar, vidio, vimeo, mediaset, Mxplayer

 * **Subtitle extraction from manifests**: Subtitles can be extracted from streaming media manifests. See [commit/be6202f](https://github.com/yt-dlp/yt-dlp/commit/be6202f12b97858b9d716e608394b51065d0419f) for details

@@ -120,7 +123,7 @@ Some of yt-dlp's default options are different from that of youtube-dl and youtu
 * The options `--id`, `--auto-number` (`-A`), `--title` (`-t`) and `--literal` (`-l`), no longer work. See [removed options](#Removed) for details
 * `avconv` is not supported as as an alternative to `ffmpeg`
 * The default [output template](#output-template) is `%(title)s [%(id)s].%(ext)s`. There is no real reason for this change. This was changed before yt-dlp was ever made public and now there are no plans to change it back to `%(title)s.%(id)s.%(ext)s`. Instead, you may use `--compat-options filename`
-* The default [format sorting](sorting-formats) is different from youtube-dl and prefers higher resolution and better codecs rather than higher bitrates. You can use the `--format-sort` option to change this to any order you prefer, or use `--compat-options format-sort` to use youtube-dl's sorting order
+* The default [format sorting](#sorting-formats) is different from youtube-dl and prefers higher resolution and better codecs rather than higher bitrates. You can use the `--format-sort` option to change this to any order you prefer, or use `--compat-options format-sort` to use youtube-dl's sorting order
 * The default format selector is `bv*+ba/b`. This means that if a combined video + audio format that is better than the best video-only format is found, the former will be prefered. Use `-f bv+ba/b` or `--compat-options format-spec` to revert this
 * Unlike youtube-dlc, yt-dlp does not allow merging multiple audio/video streams into one file by default (since this conflicts with the use of `-f bv*+ba`). If needed, this feature must be enabled using `--audio-multistreams` and `--video-multistreams`. You can also use `--compat-options multistreams` to enable both
 * `--ignore-errors` is enabled by default. Use `--abort-on-error` or `--compat-options abort-on-error` to abort on errors instead
@@ -134,6 +137,7 @@ Some of yt-dlp's default options are different from that of youtube-dl and youtu
 * Unavailable videos are also listed for youtube playlists. Use `--compat-options no-youtube-unavailable-videos` to remove this
 * If `ffmpeg` is used as the downloader, the downloading and merging of formats happen in a single step when possible. Use `--compat-options no-direct-merge` to revert this
 * Thumbnail embedding in `mp4` is done with mutagen if possible. Use `--compat-options embed-thumbnail-atomicparsley` to force the use of AtomicParsley instead
+* Some private fields such as filenames are removed by default from the infojson. Use `--no-clean-infojson` or `--compat-options no-clean-infojson` to revert this

 For ease of use, a few more compat options are available:
 * `--compat-options all`: Use all compat options
@@ -185,6 +189,7 @@ While all the other dependancies are optional, `ffmpeg` and `ffprobe` are highly
 * [**mutagen**](https://github.com/quodlibet/mutagen) - For embedding thumbnail in certain formats. Licenced under [GPLv2+](https://github.com/quodlibet/mutagen/blob/master/COPYING)
 * [**pycryptodome**](https://github.com/Legrandin/pycryptodome) - For decrypting various data. Licenced under [BSD2](https://github.com/Legrandin/pycryptodome/blob/master/LICENSE.rst)
 * [**websockets**](https://github.com/aaugustin/websockets) - For downloading over websocket. Licenced under [BSD3](https://github.com/aaugustin/websockets/blob/main/LICENSE)
+* [**keyring**](https://github.com/jaraco/keyring) - For decrypting cookies of chromium-based browsers on Linux. Licenced under [MIT](https://github.com/jaraco/keyring/blob/main/LICENSE)
 * [**AtomicParsley**](https://github.com/wez/atomicparsley) - For embedding thumbnail in mp4/m4a if mutagen is not present. Licenced under [GPLv2+](https://github.com/wez/atomicparsley/blob/master/COPYING)
 * [**rtmpdump**](http://rtmpdump.mplayerhq.hu) - For downloading `rtmp` streams. ffmpeg will be used as a fallback. Licenced under [GPLv2+](http://rtmpdump.mplayerhq.hu)
 * [**mplayer**](http://mplayerhq.hu/design7/info.html) or [**mpv**](https://mpv.io) - For downloading `rstp` streams. ffmpeg will be used as a fallback. Licenced under [GPLv2+](https://github.com/mpv-player/mpv/blob/master/Copyright)
@@ -211,7 +216,7 @@ You can also build the executable without any version info or metadata by using:
 Note that pyinstaller [does not support](https://github.com/pyinstaller/pyinstaller#requirements-and-tested-platforms) Python installed from the Windows store without using a virtual environment

 **For Unix**:
-You will need the required build tools: `python`, `make` (GNU), `pandoc`, `zip`, `nosetests`  
+You will need the required build tools: `python`, `make` (GNU), `pandoc`, `zip`, `pytest`  
 Then simply run `make`. You can also run `make yt-dlp` instead to compile only the binary without updating any of the additional files

 **Note**: In either platform, `devscripts\update-version.py` can be used to automatically update the version number
@@ -444,17 +449,17 @@ Then simply run `make`. You can also run `make yt-dlp` instead to compile only t
                                     stdin), one URL per line. Lines starting
                                     with '#', ';' or ']' are considered as
                                     comments and ignored
-    -P, --paths TYPES:PATH           The paths where the files should be
+    -P, --paths [TYPES:]PATH         The paths where the files should be
                                     downloaded. Specify the type of file and
                                     the path separated by a colon ":". All the
                                     same types as --output are supported.
                                     Additionally, you can also provide "home"
-                                     and "temp" paths. All intermediary files
-                                     are first downloaded to the temp path and
-                                     then the final files are moved over to the
-                                     home path after download is finished. This
-                                     option is ignored if --output is an
-                                     absolute path
+                                     (default) and "temp" paths. All
+                                     intermediary files are first downloaded to
+                                     the temp path and then the final files are
+                                     moved over to the home path after download
+                                     is finished. This option is ignored if
+                                     --output is an absolute path
    -o, --output [TYPES:]TEMPLATE    Output filename template; see "OUTPUT
                                     TEMPLATE" for details
    --output-na-placeholder TEXT     Placeholder value for unavailable meta
@@ -520,7 +525,19 @@ Then simply run `make`. You can also run `make yt-dlp` instead to compile only t
                                     option)
    --cookies FILE                   File to read cookies from and dump cookie
                                     jar in
-    --no-cookies                     Do not read/dump cookies (default)
+    --no-cookies                     Do not read/dump cookies from/to file
+                                     (default)
+    --cookies-from-browser BROWSER[:PROFILE]
+                                     Load cookies from a user profile of the
+                                     given web browser. Currently supported
+                                     browsers are: brave|chrome|chromium|edge|fi
+                                     refox|opera|safari|vivaldi. You can specify
+                                     the user profile name or directory using
+                                     "BROWSER:PROFILE_NAME" or
+                                     "BROWSER:PROFILE_PATH". If no profile is
+                                     given, the most recently accessed one is
+                                     used
+    --no-cookies-from-browser        Do not load cookies from browser (default)
    --cache-dir DIR                  Location in the filesystem where youtube-dl
                                     can store some downloaded information (such
                                     as client ids and signatures) permanently.
@@ -638,7 +655,9 @@ Then simply run `make`. You can also run `make yt-dlp` instead to compile only t
    --no-prefer-free-formats         Don't give any special preference to free
                                     containers (default)
    --check-formats                  Check that the formats selected are
-                                     actually downloadable (Experimental)
+                                     actually downloadable
+    --no-check-formats               Do not check that the formats selected are
+                                     actually downloadable
    -F, --list-formats               List all available formats of requested
                                     videos
    --merge-output-format FORMAT     If a merge is required (e.g.
@@ -771,8 +790,11 @@ Then simply run `make`. You can also run `make yt-dlp` instead to compile only t
                                     command. An additional field "filepath"
                                     that contains the final path of the
                                     downloaded file is also available. If no
-                                     fields are passed, "%(filepath)s" is
-                                     appended to the end of the command
+                                     fields are passed, %(filepath)q is appended
+                                     to the end of the command
+    --exec-before-download CMD       Execute a command before the actual
+                                     download. The syntax is the same as --exec
+                                     but "filepath" is not available
    --convert-subs FORMAT            Convert the subtitles to another format
                                     (currently supported: srt|vtt|ass|lrc)
                                     (Alias: --convert-subtitles)
@@ -822,7 +844,7 @@ Then simply run `make`. You can also run `make yt-dlp` instead to compile only t
    --extractor-args KEY:ARGS        Pass these arguments to the extractor. See
                                     "EXTRACTOR ARGUMENTS" for details. You can
                                     use this option multiple times to give
-                                     different arguments to different extractors
+                                     arguments for different extractors

 # CONFIGURATION

@@ -897,10 +919,11 @@ The simplest usage of `-o` is not to set any template arguments when downloading
 It may however also contain special sequences that will be replaced when downloading each video. The special sequences may be formatted according to [python string formatting operations](https://docs.python.org/2/library/stdtypes.html#string-formatting). For example, `%(NAME)s` or `%(NAME)05d`. To clarify, that is a percent symbol followed by a name in parentheses, followed by formatting operations.

 The field names themselves (the part inside the parenthesis) can also have some special formatting:
-1. **Object traversal**: The dictionaries and lists available in metadata can be traversed by using a `.` (dot) separator. You can also do python slicing using `:`. Eg: `%(tags.0)s`, `%(subtitles.en.-1.ext)`, `%(id.3:7:-1)s`. Note that the fields that become available using this method are not listed below. Use `-j` to see such fields
+1. **Object traversal**: The dictionaries and lists available in metadata can be traversed by using a `.` (dot) separator. You can also do python slicing using `:`. Eg: `%(tags.0)s`, `%(subtitles.en.-1.ext)`, `%(id.3:7:-1)s`, `%(formats.:.format_id)s`. Note that all the fields that become available using this method are not listed below. Use `-j` to see such fields
 1. **Addition**: Addition and subtraction of numeric fields can be done using `+` and `-` respectively. Eg: `%(playlist_index+10)03d`, `%(n_entries+1-playlist_index)d`
 1. **Date/time Formatting**: Date/time fields can be formatted according to [strftime formatting](https://docs.python.org/3/library/datetime.html#strftime-and-strptime-format-codes) by specifying it separated from the field name using a `>`. Eg: `%(duration>%H-%M-%S)s`, `%(upload_date>%Y-%m-%d)s`, `%(epoch-3600>%H-%M-%S)s`
 1. **Default**: A default value can be specified for when the field is empty using a `|` seperator. This overrides `--output-na-template`. Eg: `%(uploader|Unknown)s`
+1. **More Conversions**: In addition to the normal format types `diouxXeEfFgGcrs`, `j`, `l`, `q` can be used for converting to **j**son, a comma seperated **l**ist and a string **q**uoted for the terminal respectively

 To summarize, the general syntax for a field is:
 ```
@@ -937,6 +960,7 @@ The available fields are:
 - `average_rating` (numeric): Average rating give by users, the scale used depends on the webpage
 - `comment_count` (numeric): Number of comments on the video (For some extractors, comments are only downloaded at the end, and so this field cannot be used)
 - `age_limit` (numeric): Age restriction for the video (years)
+ - `live_status` (string): One of 'is_live', 'was_live', 'upcoming', 'not_live'
 - `is_live` (boolean): Whether this video is a live stream or a fixed-length video
 - `was_live` (boolean): Whether this video was originally a live stream
 - `playable_in_embed` (string): Whether this video is allowed to play in embedded players on other sites
@@ -1100,7 +1124,7 @@ If you want to download multiple videos and they don't have the same formats ava

 If you want to download several formats of the same video use a comma as a separator, e.g. `-f 22,17,18` will download all these three formats, of course if they are available. Or a more sophisticated example combined with the precedence feature: `-f 136/137/mp4/bestvideo,140/m4a/bestaudio`.

-You can merge the video and audio of multiple formats into a single file using `-f <format1>+<format2>+...` (requires ffmpeg installed), for example `-f bestvideo+bestaudio` will download the best video-only format, the best audio-only format and mux them together with ffmpeg. If `--no-video-multistreams` is used, all formats with a video stream except the first one are ignored. Similarly, if `--no-audio-multistreams` is used, all formats with an audio stream except the first one are ignored. For example, `-f bestvideo+best+bestaudio` will download and merge all 3 given formats. The resulting file will have 2 video streams and 2 audio streams. But `-f bestvideo+best+bestaudio --no-video-multistreams` will download and merge only `bestvideo` and `bestaudio`. `best` is ignored since another format containing a video stream (`bestvideo`) has already been selected. The order of the formats is therefore important. `-f best+bestaudio --no-audio-multistreams` will download and merge both formats while `-f bestaudio+best --no-audio-multistreams` will ignore `best` and download only `bestaudio`.
+You can merge the video and audio of multiple formats into a single file using `-f <format1>+<format2>+...` (requires ffmpeg installed), for example `-f bestvideo+bestaudio` will download the best video-only format, the best audio-only format and mux them together with ffmpeg. Unless `--video-multistreams` is used, all formats with a video stream except the first one are ignored. Similarly, unless `--audio-multistreams` is used, all formats with an audio stream except the first one are ignored. For example, `-f bestvideo+best+bestaudio --video-multistreams --audio-multistreams` will download and merge all 3 given formats. The resulting file will have 2 video streams and 2 audio streams. But `-f bestvideo+best+bestaudio --no-video-multistreams` will download and merge only `bestvideo` and `bestaudio`. `best` is ignored since another format containing a video stream (`bestvideo`) has already been selected. The order of the formats is therefore important. `-f best+bestaudio --no-audio-multistreams` will download and merge both formats while `-f bestaudio+best --no-audio-multistreams` will ignore `best` and download only `bestaudio`.

 ## Filtering Formats

@@ -1333,13 +1357,19 @@ Some extractors accept additional arguments which can be passed using `--extract
 The following extractors use this feature:
 * **youtube**
    * `skip`: `hls` or `dash` (or both) to skip download of the respective manifests
-    * `player_client`: `web` (default) or `android` (force use the android client fallbacks for video extraction)
-    * `player_skip`: `configs` - skip requests if applicable for client configs and use defaults
+    * `player_client`: Clients to extract video data from. The main clients are `web`, `android`, `ios`, `mweb`. These also have `_music`, `_embedded`, `_agegate`, and `_creator` variants (Eg: `web_embedded`) (`mweb` has only `_agegate`). By default, `android,web` is used, but the agegate and creator variants are added as required for age-gated videos. Similarly the music variants are added for `music.youtube.com` urls. You can also use `all` to use all the clients
+    * `player_skip`: `configs` - skip any requests for client configs and use defaults
+    * `comment_sort`: `top` or `new` (default) - choose comment sorting mode (on YouTube's side).
+    * `max_comments`: maximum amount of comments to download (default all).
+    * `max_comment_depth`: maximum depth for nested comments. YouTube supports depths 1 or 2 (default). 

 * **funimation**
    * `language`: Languages to extract. Eg: `funimation:language=english,japanese`
    * `version`: The video version to extract - `uncut` or `simulcast`

+* **vikiChannel**
+    * `video_types`: Types of videos to download - one or more of `episodes`, `movies`, `clips`, `trailers`
+
 NOTE: These options may be changed/removed in the future without concern for backward compatibility


@@ -1353,8 +1383,8 @@ Plugins are loaded from `<root-dir>/ytdlp_plugins/<type>/__init__.py`. Currently

 These are all the deprecated options and the current alternative to achieve the same effect

-#### Not recommended
-While these options still work, their use is not recommended since there are other alternatives to achieve the same
+#### Redundant options
+While these options are redundant, they are still expected to be used due to their ease of use

    --get-description                --print description
    --get-duration                   --print duration_string
@@ -1364,6 +1394,11 @@ While these options still work, their use is not recommended since there are oth
    --get-thumbnail                  --print thumbnail
    -e, --get-title                  --print title
    -g, --get-url                    --print urls
+
+
+#### Not recommended
+While these options still work, their use is not recommended since there are other alternatives to achieve the same
+
    --all-formats                    -f all
    --all-subs                       --sub-langs all --write-subs
    --autonumber-size NUMBER         Use string formatting. Eg: %(autonumber)03d
@@ -1378,8 +1413,13 @@ While these options still work, their use is not recommended since there are oth
    --youtube-skip-hls-manifest      --extractor-args "youtube:skip=hls" (Alias: --no-youtube-include-hls-manifest)
    --youtube-include-dash-manifest  Default (Alias: --no-youtube-skip-dash-manifest)
    --youtube-include-hls-manifest   Default (Alias: --no-youtube-skip-hls-manifest)
-    --test                           Used by developers for testing extractors. Not intended for the end user
-    --youtube-print-sig-code         Used for testing youtube signatures
+
+
+#### Developer options
+These options are not intended to be used by the end-user
+
+    --test                           For testing extractors
+    --youtube-print-sig-code         For testing youtube signatures


 #### Old aliases
@@ -1,17 +1,16 @@
+@setlocal
@echo off
+cd /d %~dp0..

-rem Keep this list in sync with the `offlinetest` target in Makefile
-set DOWNLOAD_TESTS="age_restriction^|download^|iqiyi_sdk_interpreter^|socks^|subtitles^|write_annotations^|youtube_lists^|youtube_signature^|post_hooks"
-
-if "%YTDL_TEST_SET%" == "core" (
-    set test_set="-I test_("%DOWNLOAD_TESTS%")\.py"
-    set multiprocess_args=""
-) else if "%YTDL_TEST_SET%" == "download" (
-    set test_set="-I test_(?!"%DOWNLOAD_TESTS%").+\.py"
-    set multiprocess_args="--processes=4 --process-timeout=540"
+if ["%~1"]==[""] (
+    set "test_set="
+) else if ["%~1"]==["core"] (
+    set "test_set=-k "not download""
+) else if ["%~1"]==["download"] (
+    set "test_set=-k download"
 ) else (
-    echo YTDL_TEST_SET is not set or invalid
+    echo.Invalid test type "%~1". Use "core" ^| "download"
    exit /b 1
 )

-nosetests test --verbose %test_set:"=% %multiprocess_args:"=%
+pytest %test_set%
@@ -1,22 +1,15 @@
-#!/bin/bash
+#!/bin/sh

-# Keep this list in sync with the `offlinetest` target in Makefile
-DOWNLOAD_TESTS="age_restriction|download|iqiyi_sdk_interpreter|overwrites|socks|subtitles|write_annotations|youtube_lists|youtube_signature|post_hooks"
+if [ -z $1 ]; then
+    test_set='test'
+elif [ $1 = 'core' ]; then
+    test_set='not download'
+elif [ $1 = 'download' ]; then
+    test_set='download'
+else
+    echo 'Invalid test type "'$1'". Use "core" | "download"'
+    exit 1
+fi

-test_set=""
-multiprocess_args=""
-
-case "$YTDL_TEST_SET" in
-    core)
-        test_set="-I test_($DOWNLOAD_TESTS)\.py"
-    ;;
-    download)
-        test_set="-I test_(?!$DOWNLOAD_TESTS).+\.py"
-        multiprocess_args="--processes=4 --process-timeout=540"
-    ;;
-    *)
-        break
-    ;;
-esac
-
-nosetests test --verbose $test_set $multiprocess_args
+echo python3 -m pytest -k $test_set
+python3 -m pytest -k "$test_set"
@@ -0,0 +1,5 @@
+---
+orphan: true
+---
+```{include} ../Collaborators.md
+```
@@ -73,7 +73,7 @@ excluded_modules = ['test', 'ytdlp_plugins', 'youtube-dl', 'youtube-dlc']
 PyInstaller.__main__.run([
    '--name=yt-dlp%s' % _x86,
    '--onefile',
-    '--icon=devscripts/cloud.ico',
+    '--icon=devscripts/logo.ico',
    *[f'--exclude-module={module}' for module in excluded_modules],
    *[f'--hidden-import={module}' for module in dependancies],
    '--upx-exclude=vcruntime140.dll',
@@ -0,0 +1,4 @@
+[pytest]
+addopts = -ra -v --strict-markers
+markers =
+    download
@@ -259,6 +259,7 @@
 - **dlive:vod**
 - **DoodStream**
 - **Dotsub**
+ - **Douyin**
 - **DouyuShow**
 - **DouyuTV**: 斗鱼
 - **DPlay**
@@ -306,6 +307,7 @@
 - **EyedoTV**
 - **facebook**
 - **FacebookPluginsVideo**
+ - **fancode:live**
 - **fancode:vod**
 - **faz.net**
 - **fc2**
@@ -343,6 +345,8 @@
 - **FrontendMastersLesson**
 - **FujiTVFODPlus7**
 - **Funimation**
+ - **funimation:page**
+ - **funimation:show**
 - **Funk**
 - **Fusion**
 - **Fux**
@@ -724,6 +728,7 @@
 - **PalcoMP3:video**
 - **pandora.tv**: 판도라TV
 - **ParamountNetwork**
+ - **ParamountPlusSeries**
 - **parliamentlive.tv**: UK parliament videos
 - **Parlview**
 - **Patreon**
@@ -766,6 +771,7 @@
 - **PopcornTV**
 - **PornCom**
 - **PornerBros**
+ - **PornFlip**
 - **PornHd**
 - **PornHub**: PornHub and Thumbzilla
 - **PornHubPagedVideoList**
@@ -808,6 +814,8 @@
 - **RCS**
 - **RCSEmbeds**
 - **RCSVarious**
+ - **RCTIPlus**
+ - **RCTIPlusSeries**
 - **RDS**: RDS.ca
 - **RedBull**
 - **RedBullEmbed**
@@ -866,6 +874,7 @@
 - **savefrom.net**
 - **SBS**: sbs.com.au
 - **schooltv**
+ - **ScienceChannel**
 - **screen.yahoo:search**: Yahoo screen search
 - **Screencast**
 - **ScreencastOMatic**
@@ -1004,7 +1013,6 @@
 - **ThisAmericanLife**
 - **ThisAV**
 - **ThisOldHouse**
- - **ThisVid**
 - **TikTok**
 - **tinypic**: tinypic.com videos
 - **TMZ**
@@ -1101,6 +1109,7 @@
 - **ustream:channel**
 - **ustudio**
 - **ustudio:embed**
+ - **Utreon**
 - **Varzesh3**
 - **Vbox7**
 - **VeeHD**
@@ -22,6 +22,14 @@ from yt_dlp.utils import (
 )


+if "pytest" in sys.modules:
+    import pytest
+    is_download_test = pytest.mark.download
+else:
+    def is_download_test(testClass):
+        return testClass
+
+
 def get_params(override=None):
    PARAMETERS_FILE = os.path.join(os.path.dirname(os.path.abspath(__file__)),
                                   "parameters.json")
@@ -190,7 +198,10 @@ def expect_info_dict(self, got_dict, expected_dict):
    expect_dict(self, got_dict, expected_dict)
    # Check for the presence of mandatory fields
    if got_dict.get('_type') not in ('playlist', 'multi_video'):
-        for key in ('id', 'url', 'title', 'ext'):
+        mandatory_fields = ['id', 'title']
+        if expected_dict.get('ext'):
+            mandatory_fields.extend(('url', 'ext'))
+        for key in mandatory_fields:
            self.assertTrue(got_dict.get(key), 'Missing mandatory field %s' % key)
    # Check for mandatory fields that are automatically set by YoutubeDL
    for key in ['webpage_url', 'extractor', 'extractor_key']:
@@ -1,4 +1,5 @@
 {
+    "check_formats": false,
    "consoletitle": false,
    "continuedl": true,
    "forcedescription": false,
@@ -35,13 +35,13 @@ class InfoExtractorTestRequestHandler(compat_http_server.BaseHTTPRequestHandler)
            assert False


-class TestIE(InfoExtractor):
+class DummyIE(InfoExtractor):
    pass


 class TestInfoExtractor(unittest.TestCase):
    def setUp(self):
-        self.ie = TestIE(FakeYDL())
+        self.ie = DummyIE(FakeYDL())

    def test_ie_key(self):
        self.assertEqual(get_info_extractor(YoutubeIE.ie_key()), YoutubeIE)
@@ -10,10 +10,11 @@ import unittest
 sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))

 import copy
+import json

 from test.helper import FakeYDL, assertRegexpMatches
 from yt_dlp import YoutubeDL
-from yt_dlp.compat import compat_str, compat_urllib_error
+from yt_dlp.compat import compat_os_name, compat_setenv, compat_str, compat_urllib_error
 from yt_dlp.extractor import YoutubeIE
 from yt_dlp.extractor.common import InfoExtractor
 from yt_dlp.postprocessor.common import PostProcessor
@@ -35,6 +36,9 @@ class YDL(FakeYDL):
    def to_screen(self, msg):
        self.msgs.append(msg)

+    def dl(self, *args, **kwargs):
+        assert False, 'Downloader must not be invoked for test_YoutubeDL'
+

 def _make_result(formats, **kwargs):
    res = {
@@ -117,35 +121,24 @@ class TestFormatSelection(unittest.TestCase):
        ]
        info_dict = _make_result(formats)

-        ydl = YDL({'format': '20/47'})
-        ydl.process_ie_result(info_dict.copy())
-        downloaded = ydl.downloaded_info_dicts[0]
-        self.assertEqual(downloaded['format_id'], '47')
+        def test(inp, *expected, multi=False):
+            ydl = YDL({
+                'format': inp,
+                'allow_multiple_video_streams': multi,
+                'allow_multiple_audio_streams': multi,
+            })
+            ydl.process_ie_result(info_dict.copy())
+            downloaded = map(lambda x: x['format_id'], ydl.downloaded_info_dicts)
+            self.assertEqual(list(downloaded), list(expected))

-        ydl = YDL({'format': '20/71/worst'})
-        ydl.process_ie_result(info_dict.copy())
-        downloaded = ydl.downloaded_info_dicts[0]
-        self.assertEqual(downloaded['format_id'], '35')
-
-        ydl = YDL()
-        ydl.process_ie_result(info_dict.copy())
-        downloaded = ydl.downloaded_info_dicts[0]
-        self.assertEqual(downloaded['format_id'], '2')
-
-        ydl = YDL({'format': 'webm/mp4'})
-        ydl.process_ie_result(info_dict.copy())
-        downloaded = ydl.downloaded_info_dicts[0]
-        self.assertEqual(downloaded['format_id'], '47')
-
-        ydl = YDL({'format': '3gp/40/mp4'})
-        ydl.process_ie_result(info_dict.copy())
-        downloaded = ydl.downloaded_info_dicts[0]
-        self.assertEqual(downloaded['format_id'], '35')
-
-        ydl = YDL({'format': 'example-with-dashes'})
-        ydl.process_ie_result(info_dict.copy())
-        downloaded = ydl.downloaded_info_dicts[0]
-        self.assertEqual(downloaded['format_id'], 'example-with-dashes')
+        test('20/47', '47')
+        test('20/71/worst', '35')
+        test(None, '2')
+        test('webm/mp4', '47')
+        test('3gp/40/mp4', '35')
+        test('example-with-dashes', 'example-with-dashes')
+        test('all', '35', 'example-with-dashes', '45', '47', '2')  # Order doesn't actually matter for this
+        test('mergeall', '2+47+45+example-with-dashes+35', multi=True)

    def test_format_selection_audio(self):
        formats = [
@@ -655,6 +648,7 @@ class TestYoutubeDL(unittest.TestCase):
        'title1': '$PATH',
        'title2': '%PATH%',
        'title3': 'foo/bar\\test',
+        'title4': 'foo "bar" test',
        'timestamp': 1618488000,
        'duration': 100000,
        'playlist_index': 1,
@@ -671,16 +665,18 @@ class TestYoutubeDL(unittest.TestCase):
            self.assertEqual(ydl.validate_outtmpl(tmpl), None)

            outtmpl, tmpl_dict = ydl.prepare_outtmpl(tmpl, info or self.outtmpl_info)
-            out = outtmpl % tmpl_dict
+            out = ydl.escape_outtmpl(outtmpl) % tmpl_dict
            fname = ydl.prepare_filename(info or self.outtmpl_info)

            if callable(expected):
                self.assertTrue(expected(out))
                self.assertTrue(expected(fname))
-            elif isinstance(expected, compat_str):
-                self.assertEqual((out, fname), (expected, expected))
+            elif isinstance(expected, str):
+                self.assertEqual(out, expected)
+                self.assertEqual(fname, expected)
            else:
-                self.assertEqual((out, fname), expected)
+                self.assertEqual(out, expected[0])
+                self.assertEqual(fname, expected[1])

        # Auto-generated fields
        test('%(id)s.%(ext)s', '1234.mp4')
@@ -693,9 +689,15 @@ class TestYoutubeDL(unittest.TestCase):
        test('%(autonumber)s', '001', autonumber_size=3)

        # Escaping %
+        test('%', '%')
        test('%%', '%')
        test('%%%%', '%%')
+        test('%s', '%s')
+        test('%%%s', '%%s')
+        test('%d', '%d')
+        test('%abc%', '%abc%')
        test('%%(width)06d.%(ext)s', '%(width)06d.mp4')
+        test('%%%(height)s', '%1080')
        test('%(width)06d.%(ext)s', 'NA.mp4')
        test('%(width)06d.%%(ext)s', 'NA.%(ext)s')
        test('%%(width)06d.%(ext)s', '%(width)06d.mp4')
@@ -710,12 +712,9 @@ class TestYoutubeDL(unittest.TestCase):
        test('%(id)s', ('ab:cd', 'ab -cd'), info={'id': 'ab:cd'})

        # Invalid templates
-        self.assertTrue(isinstance(YoutubeDL.validate_outtmpl('%'), ValueError))
        self.assertTrue(isinstance(YoutubeDL.validate_outtmpl('%(title)'), ValueError))
        test('%(invalid@tmpl|def)s', 'none', outtmpl_na_placeholder='none')
        test('%()s', 'NA')
-        test('%s', '%s')
-        test('%d', '%d')

        # NA placeholder
        NA_TEST_OUTTMPL = '%(uploader_date)s-%(width)d-%(x|def)s-%(id)s.%(ext)s'
@@ -746,13 +745,26 @@ class TestYoutubeDL(unittest.TestCase):
        test('%(width|0)04d', '0000')
        test('a%(width|)d', 'a', outtmpl_na_placeholder='none')

-        # Internal formatting
        FORMATS = self.outtmpl_info['formats']
+        sanitize = lambda x: x.replace(':', ' -').replace('"', "'")
+
+        # Custom type casting
+        test('%(formats.:.id)l', 'id1, id2, id3')
+        test('%(ext)l', 'mp4')
+        test('%(formats.:.id) 15l', '  id1, id2, id3')
+        test('%(formats)j', (json.dumps(FORMATS), sanitize(json.dumps(FORMATS))))
+        if compat_os_name == 'nt':
+            test('%(title4)q', ('"foo \\"bar\\" test"', "'foo _'bar_' test'"))
+        else:
+            test('%(title4)q', ('\'foo "bar" test\'', "'foo 'bar' test'"))
+
+        # Internal formatting
        test('%(timestamp-1000>%H-%M-%S)s', '11-43-20')
+        test('%(title|%)s %(title|%%)s', '% %%')
        test('%(id+1-height+3)05d', '00158')
        test('%(width+100)05d', 'NA')
-        test('%(formats.0) 15s', ('% 15s' % FORMATS[0], '% 15s' % str(FORMATS[0]).replace(':', ' -')))
-        test('%(formats.0)r', (repr(FORMATS[0]), repr(FORMATS[0]).replace(':', ' -')))
+        test('%(formats.0) 15s', ('% 15s' % FORMATS[0], '% 15s' % sanitize(str(FORMATS[0]))))
+        test('%(formats.0)r', (repr(FORMATS[0]), sanitize(repr(FORMATS[0]))))
        test('%(height.0)03d', '001')
        test('%(-height.0)04d', '-001')
        test('%(formats.-1.id)s', FORMATS[-1]['id'])
@@ -767,6 +779,11 @@ class TestYoutubeDL(unittest.TestCase):
        # test('%(foo|)s.%(ext)s', ('.mp4', '_.mp4'))  # fixme
        # test('%(foo|)s', ('', '_'))  # fixme

+        # Environment variable expansion for prepare_filename
+        compat_setenv('__yt_dlp_var', 'expanded')
+        envvar = '%__yt_dlp_var%' if compat_os_name == 'nt' else '$__yt_dlp_var'
+        test(envvar, (envvar, 'expanded'))
+
        # Path expansion and escaping
        test('Hello %(title1)s', 'Hello $PATH')
        test('Hello %(title2)s', 'Hello %PATH%')
@@ -7,8 +7,7 @@ import sys
 import unittest
 sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))

-from test.helper import try_rm
-
+from test.helper import try_rm, is_download_test

 from yt_dlp import YoutubeDL

@@ -32,6 +31,7 @@ def _download_restricted(url, filename, age):
    return res


+@is_download_test
 class TestAgeRestriction(unittest.TestCase):
    def _assert_restricted(self, url, filename, age, old_age=None):
        self.assertTrue(_download_restricted(url, filename, old_age))
@@ -0,0 +1,96 @@
+import unittest
+from datetime import datetime, timezone
+
+from yt_dlp import cookies
+from yt_dlp.cookies import (
+    CRYPTO_AVAILABLE,
+    LinuxChromeCookieDecryptor,
+    MacChromeCookieDecryptor,
+    WindowsChromeCookieDecryptor,
+    YDLLogger,
+    parse_safari_cookies,
+    pbkdf2_sha1,
+)
+
+
+class MonkeyPatch:
+    def __init__(self, module, temporary_values):
+        self._module = module
+        self._temporary_values = temporary_values
+        self._backup_values = {}
+
+    def __enter__(self):
+        for name, temp_value in self._temporary_values.items():
+            self._backup_values[name] = getattr(self._module, name)
+            setattr(self._module, name, temp_value)
+
+    def __exit__(self, exc_type, exc_val, exc_tb):
+        for name, backup_value in self._backup_values.items():
+            setattr(self._module, name, backup_value)
+
+
+class TestCookies(unittest.TestCase):
+    def test_chrome_cookie_decryptor_linux_derive_key(self):
+        key = LinuxChromeCookieDecryptor.derive_key(b'abc')
+        self.assertEqual(key, b'7\xa1\xec\xd4m\xfcA\xc7\xb19Z\xd0\x19\xdcM\x17')
+
+    def test_chrome_cookie_decryptor_mac_derive_key(self):
+        key = MacChromeCookieDecryptor.derive_key(b'abc')
+        self.assertEqual(key, b'Y\xe2\xc0\xd0P\xf6\xf4\xe1l\xc1\x8cQ\xcb|\xcdY')
+
+    def test_chrome_cookie_decryptor_linux_v10(self):
+        with MonkeyPatch(cookies, {'_get_linux_keyring_password': lambda *args, **kwargs: b''}):
+            encrypted_value = b'v10\xccW%\xcd\xe6\xe6\x9fM" \xa7\xb0\xca\xe4\x07\xd6'
+            value = 'USD'
+            decryptor = LinuxChromeCookieDecryptor('Chrome', YDLLogger())
+            self.assertEqual(decryptor.decrypt(encrypted_value), value)
+
+    def test_chrome_cookie_decryptor_linux_v11(self):
+        with MonkeyPatch(cookies, {'_get_linux_keyring_password': lambda *args, **kwargs: b'',
+                                   'KEYRING_AVAILABLE': True}):
+            encrypted_value = b'v11#\x81\x10>`w\x8f)\xc0\xb2\xc1\r\xf4\x1al\xdd\x93\xfd\xf8\xf8N\xf2\xa9\x83\xf1\xe9o\x0elVQd'
+            value = 'tz=Europe.London'
+            decryptor = LinuxChromeCookieDecryptor('Chrome', YDLLogger())
+            self.assertEqual(decryptor.decrypt(encrypted_value), value)
+
+    @unittest.skipIf(not CRYPTO_AVAILABLE, 'cryptography library not available')
+    def test_chrome_cookie_decryptor_windows_v10(self):
+        with MonkeyPatch(cookies, {
+            '_get_windows_v10_key': lambda *args, **kwargs: b'Y\xef\xad\xad\xeerp\xf0Y\xe6\x9b\x12\xc2<z\x16]\n\xbb\xb8\xcb\xd7\x9bA\xc3\x14e\x99{\xd6\xf4&'
+        }):
+            encrypted_value = b'v10T\xb8\xf3\xb8\x01\xa7TtcV\xfc\x88\xb8\xb8\xef\x05\xb5\xfd\x18\xc90\x009\xab\xb1\x893\x85)\x87\xe1\xa9-\xa3\xad='
+            value = '32101439'
+            decryptor = WindowsChromeCookieDecryptor('', YDLLogger())
+            self.assertEqual(decryptor.decrypt(encrypted_value), value)
+
+    def test_chrome_cookie_decryptor_mac_v10(self):
+        with MonkeyPatch(cookies, {'_get_mac_keyring_password': lambda *args, **kwargs: b'6eIDUdtKAacvlHwBVwvg/Q=='}):
+            encrypted_value = b'v10\xb3\xbe\xad\xa1[\x9fC\xa1\x98\xe0\x9a\x01\xd9\xcf\xbfc'
+            value = '2021-06-01-22'
+            decryptor = MacChromeCookieDecryptor('', YDLLogger())
+            self.assertEqual(decryptor.decrypt(encrypted_value), value)
+
+    def test_safari_cookie_parsing(self):
+        cookies = \
+            b'cook\x00\x00\x00\x01\x00\x00\x00i\x00\x00\x01\x00\x01\x00\x00\x00\x10\x00\x00\x00\x00\x00\x00\x00Y' \
+            b'\x00\x00\x00\x00\x00\x00\x00 \x00\x00\x00\x00\x00\x00\x008\x00\x00\x00B\x00\x00\x00F\x00\x00\x00H' \
+            b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x80\x03\xa5>\xc3A\x00\x00\x80\xc3\x07:\xc3A' \
+            b'localhost\x00foo\x00/\x00test%20%3Bcookie\x00\x00\x00\x054\x07\x17 \x05\x00\x00\x00Kbplist00\xd1\x01' \
+            b'\x02_\x10\x18NSHTTPCookieAcceptPolicy\x10\x02\x08\x0b&\x00\x00\x00\x00\x00\x00\x01\x01\x00\x00\x00' \
+            b'\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00('
+
+        jar = parse_safari_cookies(cookies)
+        self.assertEqual(len(jar), 1)
+        cookie = list(jar)[0]
+        self.assertEqual(cookie.domain, 'localhost')
+        self.assertEqual(cookie.port, None)
+        self.assertEqual(cookie.path, '/')
+        self.assertEqual(cookie.name, 'foo')
+        self.assertEqual(cookie.value, 'test%20%3Bcookie')
+        self.assertFalse(cookie.secure)
+        expected_expiration = datetime(2021, 6, 18, 21, 39, 19, tzinfo=timezone.utc)
+        self.assertEqual(cookie.expires, int(expected_expiration.timestamp()))
+
+    def test_pbkdf2_sha1(self):
+        key = pbkdf2_sha1(b'peanuts', b' ' * 16, 1, 16)
+        self.assertEqual(key, b'g\xe1\x8e\x0fQ\x1c\x9b\xf3\xc9`!\xaa\x90\xd9\xd34')
@@ -10,12 +10,13 @@ sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))

 from test.helper import (
    assertGreaterEqual,
+    expect_info_dict,
    expect_warnings,
    get_params,
    gettestcases,
-    expect_info_dict,
-    try_rm,
+    is_download_test,
    report_warning,
+    try_rm,
 )


@@ -64,6 +65,7 @@ def _file_md5(fn):
 defs = gettestcases()


+@is_download_test
 class TestDownload(unittest.TestCase):
    # Parallel testing in nosetests. See
    # http://nose.readthedocs.org/en/latest/doc_tests/test_multiprocess/multiprocess.html
@@ -106,8 +108,13 @@ def generator(test_case, tname):

        for tc in test_cases:
            info_dict = tc.get('info_dict', {})
-            if not (info_dict.get('id') and info_dict.get('ext')):
-                raise Exception('Test definition incorrect. The output file cannot be known. Are both \'id\' and \'ext\' keys present?')
+            params = tc.get('params', {})
+            if not info_dict.get('id'):
+                raise Exception('Test definition incorrect. \'id\' key is not present')
+            elif not info_dict.get('ext'):
+                if params.get('skip_download') and params.get('ignore_no_formats_error'):
+                    continue
+                raise Exception('Test definition incorrect. The output file cannot be known. \'ext\' key is not present')

        if 'skip' in test_case:
            print_skipping(test_case['skip'])
@@ -8,7 +8,7 @@ import sys
 import unittest
 sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))

-from test.helper import FakeYDL
+from test.helper import FakeYDL, is_download_test
 from yt_dlp.extractor import IqiyiIE


@@ -31,6 +31,7 @@ class WarningLogger(object):
        pass


+@is_download_test
 class TestIqiyiSDKInterpreter(unittest.TestCase):
    def test_iqiyi_sdk_interpreter(self):
        '''
@@ -7,7 +7,7 @@ import sys
 import unittest
 sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))

-from test.helper import get_params, try_rm
+from test.helper import get_params, try_rm, is_download_test
 import yt_dlp.YoutubeDL
 from yt_dlp.utils import DownloadError

@@ -22,6 +22,7 @@ TEST_ID = 'gr51aVj-mLg'
 EXPECTED_NAME = 'gr51aVj-mLg'


+@is_download_test
 class TestPostHooks(unittest.TestCase):
    def setUp(self):
        self.stored_name_1 = None
@@ -14,6 +14,7 @@ import subprocess
 from test.helper import (
    FakeYDL,
    get_params,
+    is_download_test,
 )
 from yt_dlp.compat import (
    compat_str,
@@ -21,6 +22,7 @@ from yt_dlp.compat import (
 )


+@is_download_test
 class TestMultipleSocks(unittest.TestCase):
    @staticmethod
    def _check_params(attrs):
@@ -76,6 +78,7 @@ class TestMultipleSocks(unittest.TestCase):
            params['secondary_server_ip'])


+@is_download_test
 class TestSocks(unittest.TestCase):
    _SKIP_SOCKS_TEST = True

@@ -7,7 +7,7 @@ import sys
 import unittest
 sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))

-from test.helper import FakeYDL, md5
+from test.helper import FakeYDL, md5, is_download_test


 from yt_dlp.extractor import (
@@ -30,6 +30,7 @@ from yt_dlp.extractor import (
 )


+@is_download_test
 class BaseTestSubtitles(unittest.TestCase):
    url = None
    IE = None
@@ -55,6 +56,7 @@ class BaseTestSubtitles(unittest.TestCase):
        return dict((l, sub_info['data']) for l, sub_info in subtitles.items())


+@is_download_test
 class TestYoutubeSubtitles(BaseTestSubtitles):
    url = 'QRS8MkLhQmM'
    IE = YoutubeIE
@@ -111,6 +113,7 @@ class TestYoutubeSubtitles(BaseTestSubtitles):
        self.assertFalse(subtitles)


+@is_download_test
 class TestDailymotionSubtitles(BaseTestSubtitles):
    url = 'http://www.dailymotion.com/video/xczg00'
    IE = DailymotionIE
@@ -134,6 +137,7 @@ class TestDailymotionSubtitles(BaseTestSubtitles):
        self.assertFalse(subtitles)


+@is_download_test
 class TestTedSubtitles(BaseTestSubtitles):
    url = 'http://www.ted.com/talks/dan_dennett_on_our_consciousness.html'
    IE = TEDIE
@@ -149,6 +153,7 @@ class TestTedSubtitles(BaseTestSubtitles):
            self.assertTrue(subtitles.get(lang) is not None, 'Subtitles for \'%s\' not extracted' % lang)


+@is_download_test
 class TestVimeoSubtitles(BaseTestSubtitles):
    url = 'http://vimeo.com/76979871'
    IE = VimeoIE
@@ -170,6 +175,7 @@ class TestVimeoSubtitles(BaseTestSubtitles):
        self.assertFalse(subtitles)


+@is_download_test
 class TestWallaSubtitles(BaseTestSubtitles):
    url = 'http://vod.walla.co.il/movie/2705958/the-yes-men'
    IE = WallaIE
@@ -191,6 +197,7 @@ class TestWallaSubtitles(BaseTestSubtitles):
        self.assertFalse(subtitles)


+@is_download_test
 class TestCeskaTelevizeSubtitles(BaseTestSubtitles):
    url = 'http://www.ceskatelevize.cz/ivysilani/10600540290-u6-uzasny-svet-techniky'
    IE = CeskaTelevizeIE
@@ -212,6 +219,7 @@ class TestCeskaTelevizeSubtitles(BaseTestSubtitles):
        self.assertFalse(subtitles)


+@is_download_test
 class TestLyndaSubtitles(BaseTestSubtitles):
    url = 'http://www.lynda.com/Bootstrap-tutorials/Using-exercise-files/110885/114408-4.html'
    IE = LyndaIE
@@ -224,6 +232,7 @@ class TestLyndaSubtitles(BaseTestSubtitles):
        self.assertEqual(md5(subtitles['en']), '09bbe67222259bed60deaa26997d73a7')


+@is_download_test
 class TestNPOSubtitles(BaseTestSubtitles):
    url = 'http://www.npo.nl/nos-journaal/28-08-2014/POW_00722860'
    IE = NPOIE
@@ -236,6 +245,7 @@ class TestNPOSubtitles(BaseTestSubtitles):
        self.assertEqual(md5(subtitles['nl']), 'fc6435027572b63fb4ab143abd5ad3f4')


+@is_download_test
 class TestMTVSubtitles(BaseTestSubtitles):
    url = 'http://www.cc.com/video-clips/p63lk0/adam-devine-s-house-party-chasing-white-swans'
    IE = ComedyCentralIE
@@ -251,6 +261,7 @@ class TestMTVSubtitles(BaseTestSubtitles):
        self.assertEqual(md5(subtitles['en']), '78206b8d8a0cfa9da64dc026eea48961')


+@is_download_test
 class TestNRKSubtitles(BaseTestSubtitles):
    url = 'http://tv.nrk.no/serie/ikke-gjoer-dette-hjemme/DMPV73000411/sesong-2/episode-1'
    IE = NRKTVIE
@@ -263,6 +274,7 @@ class TestNRKSubtitles(BaseTestSubtitles):
        self.assertEqual(md5(subtitles['no']), '544fa917d3197fcbee64634559221cc2')


+@is_download_test
 class TestRaiPlaySubtitles(BaseTestSubtitles):
    IE = RaiPlayIE

@@ -283,6 +295,7 @@ class TestRaiPlaySubtitles(BaseTestSubtitles):
        self.assertEqual(md5(subtitles['it']), '4b3264186fbb103508abe5311cfcb9cd')


+@is_download_test
 class TestVikiSubtitles(BaseTestSubtitles):
    url = 'http://www.viki.com/videos/1060846v-punch-episode-18'
    IE = VikiIE
@@ -295,6 +308,7 @@ class TestVikiSubtitles(BaseTestSubtitles):
        self.assertEqual(md5(subtitles['en']), '53cb083a5914b2d84ef1ab67b880d18a')


+@is_download_test
 class TestThePlatformSubtitles(BaseTestSubtitles):
    # from http://www.3playmedia.com/services-features/tools/integrations/theplatform/
    # (see http://theplatform.com/about/partners/type/subtitles-closed-captioning/)
@@ -309,6 +323,7 @@ class TestThePlatformSubtitles(BaseTestSubtitles):
        self.assertEqual(md5(subtitles['en']), '97e7670cbae3c4d26ae8bcc7fdd78d4b')


+@is_download_test
 class TestThePlatformFeedSubtitles(BaseTestSubtitles):
    url = 'http://feed.theplatform.com/f/7wvmTC/msnbc_video-p-test?form=json&pretty=true&range=-40&byGuid=n_hardball_5biden_140207'
    IE = ThePlatformFeedIE
@@ -321,6 +336,7 @@ class TestThePlatformFeedSubtitles(BaseTestSubtitles):
        self.assertEqual(md5(subtitles['en']), '48649a22e82b2da21c9a67a395eedade')


+@is_download_test
 class TestRtveSubtitles(BaseTestSubtitles):
    url = 'http://www.rtve.es/alacarta/videos/los-misterios-de-laura/misterios-laura-capitulo-32-misterio-del-numero-17-2-parte/2428621/'
    IE = RTVEALaCartaIE
@@ -335,6 +351,7 @@ class TestRtveSubtitles(BaseTestSubtitles):
        self.assertEqual(md5(subtitles['es']), '69e70cae2d40574fb7316f31d6eb7fca')


+@is_download_test
 class TestDemocracynowSubtitles(BaseTestSubtitles):
    url = 'http://www.democracynow.org/shows/2015/7/3'
    IE = DemocracynowIE
@@ -1054,6 +1054,9 @@ class TestUtil(unittest.TestCase):
        on = js_to_json('{ "040": "040" }')
        self.assertEqual(json.loads(on), {'040': '040'})

+        on = js_to_json('[1,//{},\n2]')
+        self.assertEqual(json.loads(on), [1, 2])
+
    def test_js_to_json_malformed(self):
        self.assertEqual(js_to_json('42a1'), '42"a1"')
        self.assertEqual(js_to_json('42a-1'), '42"a"-1')
@@ -1534,8 +1537,11 @@ Line 1
        self.assertEqual(LazyList(it).exhaust(), it)
        self.assertEqual(LazyList(it)[5], it[5])

+        self.assertEqual(LazyList(it)[5:], it[5:])
+        self.assertEqual(LazyList(it)[:5], it[:5])
        self.assertEqual(LazyList(it)[::2], it[::2])
        self.assertEqual(LazyList(it)[1::2], it[1::2])
+        self.assertEqual(LazyList(it)[5::-1], it[5::-1])
        self.assertEqual(LazyList(it)[6:2:-2], it[6:2:-2])
        self.assertEqual(LazyList(it)[::-1], it[::-1])

@@ -1547,6 +1553,7 @@ Line 1

        self.assertEqual(list(LazyList(it).reverse()), it[::-1])
        self.assertEqual(list(LazyList(it).reverse()[1:3:7]), it[::-1][1:3:7])
+        self.assertEqual(list(LazyList(it).reverse()[::-1]), it)

    def test_LazyList_laziness(self):

@@ -8,7 +8,7 @@ import sys
 import unittest
 sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))

-from test.helper import get_params, try_rm
+from test.helper import get_params, try_rm, is_download_test


 import io
@@ -38,6 +38,7 @@ ANNOTATIONS_FILE = TEST_ID + '.annotations.xml'
 EXPECTED_ANNOTATIONS = ['Speech bubble', 'Note', 'Title', 'Spotlight', 'Label']


+@is_download_test
 class TestAnnotations(unittest.TestCase):
    def setUp(self):
        # Clear old files
@@ -7,7 +7,7 @@ import sys
 import unittest
 sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))

-from test.helper import FakeYDL
+from test.helper import FakeYDL, is_download_test


 from yt_dlp.extractor import (
@@ -17,6 +17,7 @@ from yt_dlp.extractor import (
 )


+@is_download_test
 class TestYoutubeLists(unittest.TestCase):
    def assertIsPlaylist(self, info):
        """Make sure the info has '_type' set to 'playlist'"""
@@ -12,7 +12,7 @@ import io
 import re
 import string

-from test.helper import FakeYDL
+from test.helper import FakeYDL, is_download_test
 from yt_dlp.extractor import YoutubeIE
 from yt_dlp.compat import compat_str, compat_urlretrieve

@@ -65,6 +65,7 @@ _TESTS = [
 ]


+@is_download_test
 class TestPlayerInfo(unittest.TestCase):
    def test_youtube_extract_player_info(self):
        PLAYER_URLS = (
@@ -87,6 +88,7 @@ class TestPlayerInfo(unittest.TestCase):
            self.assertEqual(player_id, expected_player_id)


+@is_download_test
 class TestSignature(unittest.TestCase):
    def setUp(self):
        TEST_DIR = os.path.dirname(os.path.abspath(__file__))
@@ -1,5 +1,7 @@
 [tox]
 envlist = py26,py27,py33,py34,py35
+
+# Needed?
 [testenv]
 deps =
   nose
@@ -31,17 +31,18 @@ from zipimport import zipimporter

 from .compat import (
    compat_basestring,
-    compat_cookiejar,
    compat_get_terminal_size,
    compat_kwargs,
    compat_numeric_types,
    compat_os_name,
+    compat_shlex_quote,
    compat_str,
    compat_tokenize_tokenize,
    compat_urllib_error,
    compat_urllib_request,
    compat_urllib_request_DataHandler,
 )
+from .cookies import load_cookies
 from .utils import (
    age_restricted,
    args_to_str,
@@ -65,7 +66,8 @@ from .utils import (
    float_or_none,
    format_bytes,
    format_field,
-    STR_FORMAT_RE,
+    STR_FORMAT_RE_TMPL,
+    STR_FORMAT_TYPES,
    formatSeconds,
    GeoRestrictedError,
    HEADRequest,
@@ -107,10 +109,10 @@ from .utils import (
    try_get,
    UnavailableVideoError,
    url_basename,
+    variadic,
    version_tuple,
    write_json_file,
    write_string,
-    YoutubeDLCookieJar,
    YoutubeDLCookieProcessor,
    YoutubeDLHandler,
    YoutubeDLRedirectHandler,
@@ -124,6 +126,7 @@ from .extractor import (
 )
 from .extractor.openload import PhantomJSwrapper
 from .downloader import (
+    FFmpegFD,
    get_suitable_downloader,
    shorten_protocol_name
 )
@@ -209,6 +212,9 @@ class YoutubeDL(object):
                       into a single file
    allow_multiple_audio_streams:   Allow multiple audio streams to be merged
                       into a single file
+    check_formats      Whether to test if the formats are downloadable.
+                       Can be True (check all), False (check none)
+                       or None (check only if requested by extractor)
    paths:             Dictionary of output paths. The allowed keys are 'home'
                       'temp' and the keys of OUTTMPL_TYPES (in utils.py)
    outtmpl:           Dictionary of templates for output names. Allowed keys
@@ -253,7 +259,7 @@ class YoutubeDL(object):
    writedesktoplink:  Write a Linux internet shortcut file (.desktop)
    writesubtitles:    Write the video subtitles to a file
    writeautomaticsub: Write the automatically generated subtitles to a file
-    allsubtitles:      Deprecated - Use subtitlelangs = ['all']
+    allsubtitles:      Deprecated - Use subtitleslangs = ['all']
                       Downloads all the subtitles of the video
                       (requires writesubtitles or writeautomaticsub)
    listsubtitles:     Lists all available subtitles for the video
@@ -287,6 +293,9 @@ class YoutubeDL(object):
    break_on_reject:   Stop the download process when encountering a video that
                       has been filtered out.
    cookiefile:        File name where cookies should be read from and dumped to
+    cookiesfrombrowser: A tuple containing the name of the browser and the profile
+                       name/path from where cookies are loaded.
+                       Eg: ('chrome', ) or (vivaldi, 'default')
    nocheckcertificate:Do not verify SSL certificates
    prefer_insecure:   Use HTTP instead of HTTPS to retrieve information.
                       At the moment, this is only supported by YouTube.
@@ -317,6 +326,7 @@ class YoutubeDL(object):
                       progress, with a dictionary with the entries
                       * status: One of "downloading", "error", or "finished".
                                 Check this first and ignore unknown values.
+                       * info_dict: The extracted info_dict

                       If status is one of "downloading", or "finished", the
                       following properties may also be present:
@@ -394,7 +404,8 @@ class YoutubeDL(object):
    compat_opts:       Compatibility options. See "Differences in default behavior".
                       The following options do not work when used through the API:
                       filename, abort-on-error, multistreams, no-live-chat,
-                       no-playlist-metafiles. Refer __init__.py for their implementation
+                       no-clean-infojson, no-playlist-metafiles.
+                       Refer __init__.py for their implementation

    The following parameters are not used by YoutubeDL itself, they are used by
    the downloader (see yt_dlp/downloader/common.py):
@@ -447,7 +458,7 @@ class YoutubeDL(object):
    params = None
    _ies = []
    _pps = {'pre_process': [], 'before_dl': [], 'after_move': [], 'post_process': []}
-    __prepare_filename_warned = False
+    _printed_messages = set()
    _first_webpage_request = True
    _download_retcode = None
    _num_downloads = None
@@ -462,7 +473,7 @@ class YoutubeDL(object):
        self._ies = []
        self._ies_instances = {}
        self._pps = {'pre_process': [], 'before_dl': [], 'after_move': [], 'post_process': []}
-        self.__prepare_filename_warned = False
+        self._printed_messages = set()
        self._first_webpage_request = True
        self._post_hooks = []
        self._progress_hooks = []
@@ -657,8 +668,12 @@ class YoutubeDL(object):
                      for _ in range(line_count))
        return res[:-len('\n')]

-    def _write_string(self, s, out=None):
-        write_string(s, out=out, encoding=self.params.get('encoding'))
+    def _write_string(self, message, out=None, only_once=False):
+        if only_once:
+            if message in self._printed_messages:
+                return
+            self._printed_messages.add(message)
+        write_string(message, out=out, encoding=self.params.get('encoding'))

    def to_stdout(self, message, skip_eol=False, quiet=False):
        """Print message to stdout"""
@@ -669,13 +684,13 @@ class YoutubeDL(object):
                '%s%s' % (self._bidi_workaround(message), ('' if skip_eol else '\n')),
                self._err_file if quiet else self._screen_file)

-    def to_stderr(self, message):
+    def to_stderr(self, message, only_once=False):
        """Print message to stderr"""
        assert isinstance(message, compat_str)
        if self.params.get('logger'):
            self.params['logger'].error(message)
        else:
-            self._write_string('%s\n' % self._bidi_workaround(message), self._err_file)
+            self._write_string('%s\n' % self._bidi_workaround(message), self._err_file, only_once=only_once)

    def to_console_title(self, message):
        if not self.params.get('consoletitle', False):
@@ -752,7 +767,7 @@ class YoutubeDL(object):
        self.to_stdout(
            message, skip_eol, quiet=self.params.get('quiet', False))

-    def report_warning(self, message):
+    def report_warning(self, message, only_once=False):
        '''
        Print the message to stderr, it will be prefixed with 'WARNING:'
        If stderr is a tty file the 'WARNING:' will be colored
@@ -767,7 +782,7 @@ class YoutubeDL(object):
            else:
                _msg_header = 'WARNING:'
            warning_message = '%s %s' % (_msg_header, message)
-            self.to_stderr(warning_message)
+            self.to_stderr(warning_message, only_once)

    def report_error(self, message, tb=None):
        '''
@@ -781,7 +796,7 @@ class YoutubeDL(object):
        error_message = '%s %s' % (_msg_header, message)
        self.trouble(error_message, tb)

-    def write_debug(self, message):
+    def write_debug(self, message, only_once=False):
        '''Log debug message or Print message to stderr'''
        if not self.params.get('verbose', False):
            return
@@ -789,7 +804,7 @@ class YoutubeDL(object):
        if self.params.get('logger'):
            self.params['logger'].debug(message)
        else:
-            self._write_string('%s\n' % message)
+            self.to_stderr(message, only_once)

    def report_file_already_downloaded(self, file_name):
        """Report file has already been fully downloaded."""
@@ -835,20 +850,43 @@ class YoutubeDL(object):
        return sanitize_path(path, force=self.params.get('windowsfilenames'))

    @staticmethod
-    def validate_outtmpl(tmpl):
+    def _outtmpl_expandpath(outtmpl):
+        # expand_path translates '%%' into '%' and '$$' into '$'
+        # correspondingly that is not what we want since we need to keep
+        # '%%' intact for template dict substitution step. Working around
+        # with boundary-alike separator hack.
+        sep = ''.join([random.choice(ascii_letters) for _ in range(32)])
+        outtmpl = outtmpl.replace('%%', '%{0}%'.format(sep)).replace('$$', '${0}$'.format(sep))
+
+        # outtmpl should be expand_path'ed before template dict substitution
+        # because meta fields may contain env variables we don't want to
+        # be expanded. For example, for outtmpl "%(title)s.%(ext)s" and
+        # title "Hello $PATH", we don't want `$PATH` to be expanded.
+        return expand_path(outtmpl).replace(sep, '')
+
+    @staticmethod
+    def escape_outtmpl(outtmpl):
+        ''' Escape any remaining strings like %s, %abc% etc. '''
+        return re.sub(
+            STR_FORMAT_RE_TMPL.format('', '(?![%(\0])'),
+            lambda mobj: ('' if mobj.group('has_key') else '%') + mobj.group(0),
+            outtmpl)
+
+    @classmethod
+    def validate_outtmpl(cls, outtmpl):
        ''' @return None or Exception object '''
+        outtmpl = re.sub(
+            STR_FORMAT_RE_TMPL.format('[^)]*', '[ljq]'),
+            lambda mobj: f'{mobj.group(0)[:-1]}s',
+            cls._outtmpl_expandpath(outtmpl))
        try:
-            re.sub(
-                STR_FORMAT_RE.format(''),
-                lambda mobj: ('%' if not mobj.group('has_key') else '') + mobj.group(0),
-                tmpl
-            ) % collections.defaultdict(int)
+            cls.escape_outtmpl(outtmpl) % collections.defaultdict(int)
            return None
        except ValueError as err:
            return err

    def prepare_outtmpl(self, outtmpl, info_dict, sanitize=None):
-        """ Make the template and info_dict suitable for substitution (outtmpl % info_dict)"""
+        """ Make the template and info_dict suitable for substitution : ydl.outtmpl_escape(outtmpl) % info_dict """
        info_dict = dict(info_dict)
        na = self.params.get('outtmpl_na_placeholder', 'NA')

@@ -869,7 +907,7 @@ class YoutubeDL(object):
        }

        TMPL_DICT = {}
-        EXTERNAL_FORMAT_RE = re.compile(STR_FORMAT_RE.format('[^)]*'))
+        EXTERNAL_FORMAT_RE = re.compile(STR_FORMAT_RE_TMPL.format('[^)]*', f'[{STR_FORMAT_TYPES}ljq]'))
        MATH_FUNCTIONS = {
            '+': float.__add__,
            '-': float.__sub__,
@@ -928,10 +966,11 @@ class YoutubeDL(object):

        def create_key(outer_mobj):
            if not outer_mobj.group('has_key'):
-                return '%{}'.format(outer_mobj.group(0))
+                return f'%{outer_mobj.group(0)}'

+            prefix = outer_mobj.group('prefix')
            key = outer_mobj.group('key')
-            fmt = outer_mobj.group('format')
+            original_fmt = fmt = outer_mobj.group('format')
            mobj = re.match(INTERNAL_FORMAT_RE, key)
            if mobj is None:
                value, default, mobj = None, na, {'fields': ''}
@@ -945,8 +984,15 @@ class YoutubeDL(object):

            value = default if value is None else value

-            if fmt == 'c':
-                value = compat_str(value)
+            str_fmt = f'{fmt[:-1]}s'
+            if fmt[-1] == 'l':
+                value, fmt = ', '.join(variadic(value)), str_fmt
+            elif fmt[-1] == 'j':
+                value, fmt = json.dumps(value), str_fmt
+            elif fmt[-1] == 'q':
+                value, fmt = compat_shlex_quote(str(value)), str_fmt
+            elif fmt[-1] == 'c':
+                value = str(value)
                if value is None:
                    value, fmt = default, 's'
                else:
@@ -955,16 +1001,18 @@ class YoutubeDL(object):
                value = float_or_none(value)
                if value is None:
                    value, fmt = default, 's'
+
            if sanitize:
                if fmt[-1] == 'r':
                    # If value is an object, sanitize might convert it to a string
                    # So we convert it to repr first
-                    value, fmt = repr(value), '%ss' % fmt[:-1]
+                    value, fmt = repr(value), str_fmt
                if fmt[-1] in 'csr':
                    value = sanitize(mobj['fields'].split('.')[-1], value)
-            key += '\0%s' % fmt
+
+            key = '%s\0%s' % (key.replace('%', '%\0'), original_fmt)
            TMPL_DICT[key] = value
-            return '%({key}){fmt}'.format(key=key, fmt=fmt)
+            return f'{prefix}%({key}){fmt}'

        return EXTERNAL_FORMAT_RE.sub(create_key, outtmpl), TMPL_DICT

@@ -976,19 +1024,8 @@ class YoutubeDL(object):
                is_id=(k == 'id' or k.endswith('_id')))
            outtmpl = self.outtmpl_dict.get(tmpl_type, self.outtmpl_dict['default'])
            outtmpl, template_dict = self.prepare_outtmpl(outtmpl, info_dict, sanitize)
-
-            # expand_path translates '%%' into '%' and '$$' into '$'
-            # correspondingly that is not what we want since we need to keep
-            # '%%' intact for template dict substitution step. Working around
-            # with boundary-alike separator hack.
-            sep = ''.join([random.choice(ascii_letters) for _ in range(32)])
-            outtmpl = outtmpl.replace('%%', '%{0}%'.format(sep)).replace('$$', '${0}$'.format(sep))
-
-            # outtmpl should be expand_path'ed before template dict substitution
-            # because meta fields may contain env variables we don't want to
-            # be expanded. For example, for outtmpl "%(title)s.%(ext)s" and
-            # title "Hello $PATH", we don't want `$PATH` to be expanded.
-            filename = expand_path(outtmpl).replace(sep, '') % template_dict
+            outtmpl = self.escape_outtmpl(self._outtmpl_expandpath(outtmpl))
+            filename = outtmpl % template_dict

            force_ext = OUTTMPL_TYPES.get(tmpl_type)
            if force_ext is not None:
@@ -1014,13 +1051,13 @@ class YoutubeDL(object):

        filename = self._prepare_filename(info_dict, dir_type or 'default')

-        if warn and not self.__prepare_filename_warned:
+        if warn:
            if not self.params.get('paths'):
                pass
            elif filename == '-':
-                self.report_warning('--paths is ignored when an outputting to stdout')
+                self.report_warning('--paths is ignored when an outputting to stdout', only_once=True)
            elif os.path.isabs(filename):
-                self.report_warning('--paths is ignored since an absolute path is given in output template')
+                self.report_warning('--paths is ignored since an absolute path is given in output template', only_once=True)
            self.__prepare_filename_warned = True
        if filename == '-' or not filename:
            return filename
@@ -1136,7 +1173,7 @@ class YoutubeDL(object):
        else:
            self.report_error('no suitable InfoExtractor for URL %s' % url)

-    def __handle_extraction_exceptions(func):
+    def __handle_extraction_exceptions(func, handle_all_errors=True):
        def wrapper(self, *args, **kwargs):
            try:
                return func(self, *args, **kwargs)
@@ -1156,7 +1193,7 @@ class YoutubeDL(object):
            except (MaxDownloadsReached, ExistingVideoReached, RejectedVideoReached):
                raise
            except Exception as e:
-                if self.params.get('ignoreerrors', False):
+                if handle_all_errors and self.params.get('ignoreerrors', False):
                    self.report_error(error_to_compat_str(e), tb=encode_compat_str(traceback.format_exc()))
                else:
                    raise
@@ -1173,6 +1210,8 @@ class YoutubeDL(object):
                '_type': 'compat_list',
                'entries': ie_result,
            }
+        if extra_info.get('original_url'):
+            ie_result.setdefault('original_url', extra_info['original_url'])
        self.add_default_extra_info(ie_result, ie, url)
        if process:
            return self.process_ie_result(ie_result, download, extra_info)
@@ -1204,6 +1243,9 @@ class YoutubeDL(object):

        if result_type in ('url', 'url_transparent'):
            ie_result['url'] = sanitize_url(ie_result['url'])
+            if ie_result.get('original_url'):
+                extra_info.setdefault('original_url', ie_result['original_url'])
+
            extract_flat = self.params.get('extract_flat', False)
            if ((extract_flat == 'in_playlist' and 'playlist' in extra_info)
                    or extract_flat is True):
@@ -1360,13 +1402,19 @@ class YoutubeDL(object):
        if not isinstance(ie_entries, (list, PagedList)):
            ie_entries = LazyList(ie_entries)

+        def get_entry(i):
+            return YoutubeDL.__handle_extraction_exceptions(
+                lambda self, i: ie_entries[i - 1],
+                False
+            )(self, i)
+
        entries = []
        for i in playlistitems or itertools.count(playliststart):
            if playlistitems is None and playlistend is not None and playlistend < i:
                break
            entry = None
            try:
-                entry = ie_entries[i - 1]
+                entry = get_entry(i)
                if entry is None:
                    raise EntryNotInPlaylist()
            except (IndexError, EntryNotInPlaylist):
@@ -1709,6 +1757,7 @@ class YoutubeDL(object):
                        if not allow_multiple_streams[aud_vid] and fmt_info.get(aud_vid[0] + 'codec') != 'none':
                            if get_no_more[aud_vid]:
                                formats_info.pop(i)
+                                break
                            get_no_more[aud_vid] = True

            if len(formats_info) == 1:
@@ -1758,6 +1807,7 @@ class YoutubeDL(object):
        def _check_formats(formats):
            if not check_formats:
                yield from formats
+                return
            for f in formats:
                self.to_screen('[info] Testing format %s' % f['format_id'])
                temp_file = tempfile.NamedTemporaryFile(
@@ -1943,15 +1993,27 @@ class YoutubeDL(object):
                t.get('id') if t.get('id') is not None else '',
                t.get('url')))

-            def test_thumbnail(t):
-                self.to_screen('[info] Testing thumbnail %s' % t['id'])
-                try:
-                    self.urlopen(HEADRequest(t['url']))
-                except network_exceptions as err:
-                    self.to_screen('[info] Unable to connect to thumbnail %s URL "%s" - %s. Skipping...' % (
-                        t['id'], t['url'], error_to_compat_str(err)))
-                    return False
-                return True
+            def thumbnail_tester():
+                if self.params.get('check_formats'):
+                    test_all = True
+                    to_screen = lambda msg: self.to_screen(f'[info] {msg}')
+                else:
+                    test_all = False
+                    to_screen = self.write_debug
+
+                def test_thumbnail(t):
+                    if not test_all and not t.get('_test_url'):
+                        return True
+                    to_screen('Testing thumbnail %s' % t['id'])
+                    try:
+                        self.urlopen(HEADRequest(t['url']))
+                    except network_exceptions as err:
+                        to_screen('Unable to connect to thumbnail %s URL "%s" - %s. Skipping...' % (
+                            t['id'], t['url'], error_to_compat_str(err)))
+                        return False
+                    return True
+
+                return test_thumbnail

            for i, t in enumerate(thumbnails):
                if t.get('id') is None:
@@ -1959,8 +2021,11 @@ class YoutubeDL(object):
                if t.get('width') and t.get('height'):
                    t['resolution'] = '%dx%d' % (t['width'], t['height'])
                t['url'] = sanitize_url(t['url'])
-            if self.params.get('check_formats'):
-                info_dict['thumbnails'] = LazyList(filter(test_thumbnail, thumbnails[::-1])).reverse()
+
+            if self.params.get('check_formats') is not False:
+                info_dict['thumbnails'] = LazyList(filter(thumbnail_tester(), thumbnails[::-1])).reverse()
+            else:
+                info_dict['thumbnails'] = thumbnails

    def process_video_result(self, info_dict, download=True):
        assert info_dict.get('_type', 'video') == 'video'
@@ -2007,7 +2072,7 @@ class YoutubeDL(object):
        elif thumbnails:
            info_dict['thumbnail'] = thumbnails[-1]['url']

-        if 'display_id' not in info_dict and 'id' in info_dict:
+        if info_dict.get('display_id') is None and 'id' in info_dict:
            info_dict['display_id'] = info_dict['id']

        for ts_key, date_key in (
@@ -2023,6 +2088,23 @@ class YoutubeDL(object):
                except (ValueError, OverflowError, OSError):
                    pass

+        live_keys = ('is_live', 'was_live')
+        live_status = info_dict.get('live_status')
+        if live_status is None:
+            for key in live_keys:
+                if info_dict.get(key) is False:
+                    continue
+                if info_dict.get(key):
+                    live_status = key
+                break
+            if all(info_dict.get(key) is False for key in live_keys):
+                live_status = 'not_live'
+        if live_status:
+            info_dict['live_status'] = live_status
+            for key in live_keys:
+                if info_dict.get(key) is None:
+                    info_dict[key] = (live_status == key)
+
        # Auto generate title fields corresponding to the *_number fields when missing
        # in order to always have clean titles. This is very common for TV series.
        for field in ('chapter', 'season', 'episode'):
@@ -2290,7 +2372,7 @@ class YoutubeDL(object):
            if re.match(r'\w+$', tmpl):
                tmpl = '%({})s'.format(tmpl)
            tmpl, info_copy = self.prepare_outtmpl(tmpl, info_dict)
-            self.to_stdout(tmpl % info_copy)
+            self.to_stdout(self.escape_outtmpl(tmpl) % info_copy)

        print_mandatory('title')
        print_mandatory('id')
@@ -2323,7 +2405,7 @@ class YoutubeDL(object):
            }
        else:
            params = self.params
-        fd = get_suitable_downloader(info, params)(self, params)
+        fd = get_suitable_downloader(info, params, to_stdout=(name == '-'))(self, params)
        if not test:
            for ph in self._progress_hooks:
                fd.add_progress_hook(ph)
@@ -2589,19 +2671,14 @@ class YoutubeDL(object):

                    requested_formats = info_dict['requested_formats']
                    old_ext = info_dict['ext']
-                    if self.params.get('merge_output_format') is None:
-                        if not compatible_formats(requested_formats):
-                            info_dict['ext'] = 'mkv'
-                            self.report_warning(
-                                'Requested formats are incompatible for merge and will be merged into mkv.')
-                        if (info_dict['ext'] == 'webm'
-                                and self.params.get('writethumbnail', False)
-                                and info_dict.get('thumbnails')):
-                            info_dict['ext'] = 'mkv'
-                            self.report_warning(
-                                'webm doesn\'t support embedding a thumbnail, mkv will be used.')
+                    if self.params.get('merge_output_format') is None and not compatible_formats(requested_formats):
+                        info_dict['ext'] = 'mkv'
+                        self.report_warning(
+                            'Requested formats are incompatible for merge and will be merged into mkv.')

                    def correct_ext(filename):
+                        if filename == '-':
+                            return filename
                        filename_real_ext = os.path.splitext(filename)[1][1:]
                        filename_wo_ext = (
                            os.path.splitext(filename)[0]
@@ -2616,20 +2693,16 @@ class YoutubeDL(object):
                    info_dict['__real_download'] = False

                    _protocols = set(determine_protocol(f) for f in requested_formats)
-                    if len(_protocols) == 1:
+                    if len(_protocols) == 1:  # All requested formats have same protocol
                        info_dict['protocol'] = _protocols.pop()
-                    directly_mergable = (
-                        'no-direct-merge' not in self.params.get('compat_opts', [])
-                        and info_dict.get('protocol') is not None  # All requested formats have same protocol
-                        and not self.params.get('allow_unplayable_formats')
-                        and get_suitable_downloader(info_dict, self.params).__name__ == 'FFmpegFD')
-                    if directly_mergable:
-                        info_dict['url'] = requested_formats[0]['url']
-                        # Treat it as a single download
-                        dl_filename = existing_file(full_filename, temp_filename)
-                        if dl_filename is None:
-                            success, real_download = self.dl(temp_filename, info_dict)
-                            info_dict['__real_download'] = real_download
+                    directly_mergable = FFmpegFD.can_merge_formats(info_dict)
+                    if dl_filename is not None:
+                        pass
+                    elif (directly_mergable and get_suitable_downloader(
+                            info_dict, self.params, to_stdout=(temp_filename == '-')) == FFmpegFD):
+                        info_dict['url'] = '\n'.join(f['url'] for f in requested_formats)
+                        success, real_download = self.dl(temp_filename, info_dict)
+                        info_dict['__real_download'] = real_download
                    else:
                        downloaded = []
                        merger = FFmpegMergerPP(self)
@@ -2643,28 +2716,34 @@ class YoutubeDL(object):
                                'You have requested merging of multiple formats but ffmpeg is not installed. '
                                'The formats won\'t be merged.')

-                        if dl_filename is None:
-                            for f in requested_formats:
-                                new_info = dict(info_dict)
-                                del new_info['requested_formats']
-                                new_info.update(f)
-                                fname = prepend_extension(
-                                    self.prepare_filename(new_info, 'temp'),
-                                    'f%s' % f['format_id'], new_info['ext'])
+                        if temp_filename == '-':
+                            reason = ('using a downloader other than ffmpeg' if directly_mergable
+                                      else 'but the formats are incompatible for simultaneous download' if merger.available
+                                      else 'but ffmpeg is not installed')
+                            self.report_warning(
+                                f'You have requested downloading multiple formats to stdout {reason}. '
+                                'The formats will be streamed one after the other')
+                            fname = temp_filename
+                        for f in requested_formats:
+                            new_info = dict(info_dict)
+                            del new_info['requested_formats']
+                            new_info.update(f)
+                            if temp_filename != '-':
+                                fname = prepend_extension(temp_filename, 'f%s' % f['format_id'], new_info['ext'])
                                if not self._ensure_dir_exists(fname):
                                    return
                                downloaded.append(fname)
-                                partial_success, real_download = self.dl(fname, new_info)
-                                info_dict['__real_download'] = info_dict['__real_download'] or real_download
-                                success = success and partial_success
-                            if merger.available and not self.params.get('allow_unplayable_formats'):
-                                info_dict['__postprocessors'].append(merger)
-                                info_dict['__files_to_merge'] = downloaded
-                                # Even if there were no downloads, it is being merged only now
-                                info_dict['__real_download'] = True
-                            else:
-                                for file in downloaded:
-                                    files_to_move[file] = None
+                            partial_success, real_download = self.dl(fname, new_info)
+                            info_dict['__real_download'] = info_dict['__real_download'] or real_download
+                            success = success and partial_success
+                        if merger.available and not self.params.get('allow_unplayable_formats'):
+                            info_dict['__postprocessors'].append(merger)
+                            info_dict['__files_to_merge'] = downloaded
+                            # Even if there were no downloads, it is being merged only now
+                            info_dict['__real_download'] = True
+                        else:
+                            for file in downloaded:
+                                files_to_move[file] = None
                else:
                    # Just a single file
                    dl_filename = existing_file(full_filename, temp_filename)
@@ -2995,17 +3074,6 @@ class YoutubeDL(object):
            res += '~' + format_bytes(fdict['filesize_approx'])
        return res

-    def _format_note_table(self, f):
-        def join_fields(*vargs):
-            return ', '.join((val for val in vargs if val != ''))
-
-        return join_fields(
-            'UNSUPPORTED' if f.get('ext') in ('f4f', 'f4m') else '',
-            format_field(f, 'language', '[%s]'),
-            format_field(f, 'format_note'),
-            format_field(f, 'container', ignore=(None, f.get('ext'))),
-            format_field(f, 'asr', '%5dHz'))
-
    def list_formats(self, info_dict):
        formats = info_dict.get('formats', [info_dict])
        new_format = (
@@ -3028,11 +3096,15 @@ class YoutubeDL(object):
                    format_field(f, 'acodec', default='unknown').replace('none', ''),
                    format_field(f, 'abr', '%3dk'),
                    format_field(f, 'asr', '%5dHz'),
-                    self._format_note_table(f)]
-                for f in formats
-                if f.get('preference') is None or f['preference'] >= -1000]
+                    ', '.join(filter(None, (
+                        'UNSUPPORTED' if f.get('ext') in ('f4f', 'f4m') else '',
+                        format_field(f, 'language', '[%s]'),
+                        format_field(f, 'format_note'),
+                        format_field(f, 'container', ignore=(None, f.get('ext'))),
+                    ))),
+                ] for f in formats if f.get('preference') is None or f['preference'] >= -1000]
            header_line = ['ID', 'EXT', 'RESOLUTION', 'FPS', '|', ' FILESIZE', '  TBR', 'PROTO',
-                           '|', 'VCODEC', '  VBR', 'ACODEC', ' ABR', ' ASR', 'NOTE']
+                           '|', 'VCODEC', '  VBR', 'ACODEC', ' ABR', ' ASR', 'MORE INFO']
        else:
            table = [
                [
@@ -3179,16 +3251,11 @@ class YoutubeDL(object):
        timeout_val = self.params.get('socket_timeout')
        self._socket_timeout = 600 if timeout_val is None else float(timeout_val)

+        opts_cookiesfrombrowser = self.params.get('cookiesfrombrowser')
        opts_cookiefile = self.params.get('cookiefile')
        opts_proxy = self.params.get('proxy')

-        if opts_cookiefile is None:
-            self.cookiejar = compat_cookiejar.CookieJar()
-        else:
-            opts_cookiefile = expand_path(opts_cookiefile)
-            self.cookiejar = YoutubeDLCookieJar(opts_cookiefile)
-            if os.access(opts_cookiefile, os.R_OK):
-                self.cookiejar.load(ignore_discard=True, ignore_expires=True)
+        self.cookiejar = load_cookies(opts_cookiefile, opts_cookiesfrombrowser, self)

        cookie_processor = YoutubeDLCookieProcessor(self.cookiejar)
        if opts_proxy is not None:
@@ -20,6 +20,7 @@ from .compat import (
    compat_getpass,
    workaround_optparse_bug9161,
 )
+from .cookies import SUPPORTED_BROWSERS
 from .utils import (
    DateRange,
    decodeOption,
@@ -242,6 +243,12 @@ def _real_main(argv=None):
        if opts.convertthumbnails not in FFmpegThumbnailsConvertorPP.SUPPORTED_EXTS:
            parser.error('invalid thumbnail format specified')

+    if opts.cookiesfrombrowser is not None:
+        opts.cookiesfrombrowser = [
+            part.strip() or None for part in opts.cookiesfrombrowser.split(':', 1)]
+        if opts.cookiesfrombrowser[0] not in SUPPORTED_BROWSERS:
+            parser.error('unsupported browser specified for cookies')
+
    if opts.date is not None:
        date = DateRange.day(opts.date)
    else:
@@ -273,7 +280,7 @@ def _real_main(argv=None):
        'filename', 'format-sort', 'abort-on-error', 'format-spec', 'no-playlist-metafiles',
        'multistreams', 'no-live-chat', 'playlist-index', 'list-formats', 'no-direct-merge',
        'no-youtube-channel-redirect', 'no-youtube-unavailable-videos', 'no-attach-info-json',
-        'embed-thumbnail-atomicparsley', 'seperate-video-versions',
+        'embed-thumbnail-atomicparsley', 'seperate-video-versions', 'no-clean-infojson',
    ]
    compat_opts = parse_compat_opts()

@@ -284,7 +291,7 @@ def _real_main(argv=None):
        compat_opts.update(['*%s' % name])
        return True

-    def set_default_compat(compat_name, opt_name, default=True, remove_compat=False):
+    def set_default_compat(compat_name, opt_name, default=True, remove_compat=True):
        attr = getattr(opts, opt_name)
        if compat_name in compat_opts:
            if attr is None:
@@ -300,6 +307,7 @@ def _real_main(argv=None):

    set_default_compat('abort-on-error', 'ignoreerrors')
    set_default_compat('no-playlist-metafiles', 'allow_playlist_files')
+    set_default_compat('no-clean-infojson', 'clean_infojson')
    if 'format-sort' in compat_opts:
        opts.format_sort.extend(InfoExtractor.FormatSort.ytdl_default)
    _video_multistreams_set = set_default_compat('multistreams', 'allow_multiple_video_streams', False, remove_compat=False)
@@ -415,6 +423,13 @@ def _real_main(argv=None):
            # Run this before the actual video download
            'when': 'before_dl'
        })
+    # Must be after all other before_dl
+    if opts.exec_before_dl_cmd:
+        postprocessors.append({
+            'key': 'ExecAfterDownload',
+            'exec_cmd': opts.exec_before_dl_cmd,
+            'when': 'before_dl'
+        })
    if opts.extractaudio:
        postprocessors.append({
            'key': 'FFmpegExtractAudio',
@@ -621,6 +636,7 @@ def _real_main(argv=None):
        'break_on_reject': opts.break_on_reject,
        'skip_playlist_after_errors': opts.skip_playlist_after_errors,
        'cookiefile': opts.cookiefile,
+        'cookiesfrombrowser': opts.cookiesfrombrowser,
        'nocheckcertificate': opts.no_check_certificate,
        'prefer_insecure': opts.prefer_insecure,
        'proxy': opts.proxy,
@@ -0,0 +1,755 @@
+import ctypes
+import json
+import os
+import shutil
+import struct
+import subprocess
+import sys
+import tempfile
+from datetime import datetime, timedelta, timezone
+from hashlib import pbkdf2_hmac
+
+from yt_dlp.aes import aes_cbc_decrypt
+from yt_dlp.compat import (
+    compat_b64decode,
+    compat_cookiejar_Cookie,
+)
+from yt_dlp.utils import (
+    bug_reports_message,
+    bytes_to_intlist,
+    expand_path,
+    intlist_to_bytes,
+    process_communicate_or_kill,
+    YoutubeDLCookieJar,
+)
+
+try:
+    import sqlite3
+    SQLITE_AVAILABLE = True
+except ImportError:
+    # although sqlite3 is part of the standard library, it is possible to compile python without
+    # sqlite support. See: https://github.com/yt-dlp/yt-dlp/issues/544
+    SQLITE_AVAILABLE = False
+
+
+try:
+    from Crypto.Cipher import AES
+    CRYPTO_AVAILABLE = True
+except ImportError:
+    CRYPTO_AVAILABLE = False
+
+try:
+    import keyring
+    KEYRING_AVAILABLE = True
+    KEYRING_UNAVAILABLE_REASON = f'due to unknown reasons{bug_reports_message()}'
+except ImportError:
+    KEYRING_AVAILABLE = False
+    KEYRING_UNAVAILABLE_REASON = (
+        'as the `keyring` module is not installed. '
+        'Please install by running `python3 -m pip install keyring`. '
+        'Depending on your platform, additional packages may be required '
+        'to access the keyring; see  https://pypi.org/project/keyring')
+except Exception as _err:
+    KEYRING_AVAILABLE = False
+    KEYRING_UNAVAILABLE_REASON = 'as the `keyring` module could not be initialized: %s' % _err
+
+
+CHROMIUM_BASED_BROWSERS = {'brave', 'chrome', 'chromium', 'edge', 'opera', 'vivaldi'}
+SUPPORTED_BROWSERS = CHROMIUM_BASED_BROWSERS | {'firefox', 'safari'}
+
+
+class YDLLogger:
+    def __init__(self, ydl=None):
+        self._ydl = ydl
+
+    def debug(self, message):
+        if self._ydl:
+            self._ydl.write_debug(message)
+
+    def info(self, message):
+        if self._ydl:
+            self._ydl.to_screen(f'[Cookies] {message}')
+
+    def warning(self, message, only_once=False):
+        if self._ydl:
+            self._ydl.report_warning(message, only_once)
+
+    def error(self, message):
+        if self._ydl:
+            self._ydl.report_error(message)
+
+
+def load_cookies(cookie_file, browser_specification, ydl):
+    cookie_jars = []
+    if browser_specification is not None:
+        browser_name, profile = _parse_browser_specification(*browser_specification)
+        cookie_jars.append(extract_cookies_from_browser(browser_name, profile, YDLLogger(ydl)))
+
+    if cookie_file is not None:
+        cookie_file = expand_path(cookie_file)
+        jar = YoutubeDLCookieJar(cookie_file)
+        if os.access(cookie_file, os.R_OK):
+            jar.load(ignore_discard=True, ignore_expires=True)
+        cookie_jars.append(jar)
+
+    return _merge_cookie_jars(cookie_jars)
+
+
+def extract_cookies_from_browser(browser_name, profile=None, logger=YDLLogger()):
+    if browser_name == 'firefox':
+        return _extract_firefox_cookies(profile, logger)
+    elif browser_name == 'safari':
+        return _extract_safari_cookies(profile, logger)
+    elif browser_name in CHROMIUM_BASED_BROWSERS:
+        return _extract_chrome_cookies(browser_name, profile, logger)
+    else:
+        raise ValueError('unknown browser: {}'.format(browser_name))
+
+
+def _extract_firefox_cookies(profile, logger):
+    logger.info('Extracting cookies from firefox')
+    if not SQLITE_AVAILABLE:
+        logger.warning('Cannot extract cookies from firefox without sqlite3 support. '
+                       'Please use a python interpreter compiled with sqlite3 support')
+        return YoutubeDLCookieJar()
+
+    if profile is None:
+        search_root = _firefox_browser_dir()
+    elif _is_path(profile):
+        search_root = profile
+    else:
+        search_root = os.path.join(_firefox_browser_dir(), profile)
+
+    cookie_database_path = _find_most_recently_used_file(search_root, 'cookies.sqlite')
+    if cookie_database_path is None:
+        raise FileNotFoundError('could not find firefox cookies database in {}'.format(search_root))
+    logger.debug('extracting from: "{}"'.format(cookie_database_path))
+
+    with tempfile.TemporaryDirectory(prefix='youtube_dl') as tmpdir:
+        cursor = None
+        try:
+            cursor = _open_database_copy(cookie_database_path, tmpdir)
+            cursor.execute('SELECT host, name, value, path, expiry, isSecure FROM moz_cookies')
+            jar = YoutubeDLCookieJar()
+            for host, name, value, path, expiry, is_secure in cursor.fetchall():
+                cookie = compat_cookiejar_Cookie(
+                    version=0, name=name, value=value, port=None, port_specified=False,
+                    domain=host, domain_specified=bool(host), domain_initial_dot=host.startswith('.'),
+                    path=path, path_specified=bool(path), secure=is_secure, expires=expiry, discard=False,
+                    comment=None, comment_url=None, rest={})
+                jar.set_cookie(cookie)
+            logger.info('Extracted {} cookies from firefox'.format(len(jar)))
+            return jar
+        finally:
+            if cursor is not None:
+                cursor.connection.close()
+
+
+def _firefox_browser_dir():
+    if sys.platform in ('linux', 'linux2'):
+        return os.path.expanduser('~/.mozilla/firefox')
+    elif sys.platform == 'win32':
+        return os.path.expandvars(r'%APPDATA%\Mozilla\Firefox\Profiles')
+    elif sys.platform == 'darwin':
+        return os.path.expanduser('~/Library/Application Support/Firefox')
+    else:
+        raise ValueError('unsupported platform: {}'.format(sys.platform))
+
+
+def _get_chromium_based_browser_settings(browser_name):
+    # https://chromium.googlesource.com/chromium/src/+/HEAD/docs/user_data_dir.md
+    if sys.platform in ('linux', 'linux2'):
+        config = _config_home()
+        browser_dir = {
+            'brave': os.path.join(config, 'BraveSoftware/Brave-Browser'),
+            'chrome': os.path.join(config, 'google-chrome'),
+            'chromium': os.path.join(config, 'chromium'),
+            'edge': os.path.join(config, 'microsoft-edge'),
+            'opera': os.path.join(config, 'opera'),
+            'vivaldi': os.path.join(config, 'vivaldi'),
+        }[browser_name]
+
+    elif sys.platform == 'win32':
+        appdata_local = os.path.expandvars('%LOCALAPPDATA%')
+        appdata_roaming = os.path.expandvars('%APPDATA%')
+        browser_dir = {
+            'brave': os.path.join(appdata_local, r'BraveSoftware\Brave-Browser\User Data'),
+            'chrome': os.path.join(appdata_local, r'Google\Chrome\User Data'),
+            'chromium': os.path.join(appdata_local, r'Chromium\User Data'),
+            'edge': os.path.join(appdata_local, r'Microsoft\Edge\User Data'),
+            'opera': os.path.join(appdata_roaming, r'Opera Software\Opera Stable'),
+            'vivaldi': os.path.join(appdata_local, r'Vivaldi\User Data'),
+        }[browser_name]
+
+    elif sys.platform == 'darwin':
+        appdata = os.path.expanduser('~/Library/Application Support')
+        browser_dir = {
+            'brave': os.path.join(appdata, 'BraveSoftware/Brave-Browser'),
+            'chrome': os.path.join(appdata, 'Google/Chrome'),
+            'chromium': os.path.join(appdata, 'Chromium'),
+            'edge': os.path.join(appdata, 'Microsoft Edge'),
+            'opera': os.path.join(appdata, 'com.operasoftware.Opera'),
+            'vivaldi': os.path.join(appdata, 'Vivaldi'),
+        }[browser_name]
+
+    else:
+        raise ValueError('unsupported platform: {}'.format(sys.platform))
+
+    # Linux keyring names can be determined by snooping on dbus while opening the browser in KDE:
+    # dbus-monitor "interface='org.kde.KWallet'" "type=method_return"
+    keyring_name = {
+        'brave': 'Brave',
+        'chrome': 'Chrome',
+        'chromium': 'Chromium',
+        'edge': 'Microsoft Edge' if sys.platform == 'darwin' else 'Chromium',
+        'opera': 'Opera' if sys.platform == 'darwin' else 'Chromium',
+        'vivaldi': 'Vivaldi' if sys.platform == 'darwin' else 'Chrome',
+    }[browser_name]
+
+    browsers_without_profiles = {'opera'}
+
+    return {
+        'browser_dir': browser_dir,
+        'keyring_name': keyring_name,
+        'supports_profiles': browser_name not in browsers_without_profiles
+    }
+
+
+def _extract_chrome_cookies(browser_name, profile, logger):
+    logger.info('Extracting cookies from {}'.format(browser_name))
+
+    if not SQLITE_AVAILABLE:
+        logger.warning(('Cannot extract cookies from {} without sqlite3 support. '
+                        'Please use a python interpreter compiled with sqlite3 support').format(browser_name))
+        return YoutubeDLCookieJar()
+
+    config = _get_chromium_based_browser_settings(browser_name)
+
+    if profile is None:
+        search_root = config['browser_dir']
+    elif _is_path(profile):
+        search_root = profile
+        config['browser_dir'] = os.path.dirname(profile) if config['supports_profiles'] else profile
+    else:
+        if config['supports_profiles']:
+            search_root = os.path.join(config['browser_dir'], profile)
+        else:
+            logger.error('{} does not support profiles'.format(browser_name))
+            search_root = config['browser_dir']
+
+    cookie_database_path = _find_most_recently_used_file(search_root, 'Cookies')
+    if cookie_database_path is None:
+        raise FileNotFoundError('could not find {} cookies database in "{}"'.format(browser_name, search_root))
+    logger.debug('extracting from: "{}"'.format(cookie_database_path))
+
+    decryptor = get_cookie_decryptor(config['browser_dir'], config['keyring_name'], logger)
+
+    with tempfile.TemporaryDirectory(prefix='youtube_dl') as tmpdir:
+        cursor = None
+        try:
+            cursor = _open_database_copy(cookie_database_path, tmpdir)
+            cursor.connection.text_factory = bytes
+            column_names = _get_column_names(cursor, 'cookies')
+            secure_column = 'is_secure' if 'is_secure' in column_names else 'secure'
+            cursor.execute('SELECT host_key, name, value, encrypted_value, path, '
+                           'expires_utc, {} FROM cookies'.format(secure_column))
+            jar = YoutubeDLCookieJar()
+            failed_cookies = 0
+            for host_key, name, value, encrypted_value, path, expires_utc, is_secure in cursor.fetchall():
+                host_key = host_key.decode('utf-8')
+                name = name.decode('utf-8')
+                value = value.decode('utf-8')
+                path = path.decode('utf-8')
+
+                if not value and encrypted_value:
+                    value = decryptor.decrypt(encrypted_value)
+                    if value is None:
+                        failed_cookies += 1
+                        continue
+
+                cookie = compat_cookiejar_Cookie(
+                    version=0, name=name, value=value, port=None, port_specified=False,
+                    domain=host_key, domain_specified=bool(host_key), domain_initial_dot=host_key.startswith('.'),
+                    path=path, path_specified=bool(path), secure=is_secure, expires=expires_utc, discard=False,
+                    comment=None, comment_url=None, rest={})
+                jar.set_cookie(cookie)
+            if failed_cookies > 0:
+                failed_message = ' ({} could not be decrypted)'.format(failed_cookies)
+            else:
+                failed_message = ''
+            logger.info('Extracted {} cookies from {}{}'.format(len(jar), browser_name, failed_message))
+            return jar
+        finally:
+            if cursor is not None:
+                cursor.connection.close()
+
+
+class ChromeCookieDecryptor:
+    """
+    Overview:
+
+        Linux:
+        - cookies are either v10 or v11
+            - v10: AES-CBC encrypted with a fixed key
+            - v11: AES-CBC encrypted with an OS protected key (keyring)
+            - v11 keys can be stored in various places depending on the activate desktop environment [2]
+
+        Mac:
+        - cookies are either v10 or not v10
+            - v10: AES-CBC encrypted with an OS protected key (keyring) and more key derivation iterations than linux
+            - not v10: 'old data' stored as plaintext
+
+        Windows:
+        - cookies are either v10 or not v10
+            - v10: AES-GCM encrypted with a key which is encrypted with DPAPI
+            - not v10: encrypted with DPAPI
+
+    Sources:
+    - [1] https://chromium.googlesource.com/chromium/src/+/refs/heads/main/components/os_crypt/
+    - [2] https://chromium.googlesource.com/chromium/src/+/refs/heads/main/components/os_crypt/key_storage_linux.cc
+        - KeyStorageLinux::CreateService
+    """
+
+    def decrypt(self, encrypted_value):
+        raise NotImplementedError
+
+
+def get_cookie_decryptor(browser_root, browser_keyring_name, logger):
+    if sys.platform in ('linux', 'linux2'):
+        return LinuxChromeCookieDecryptor(browser_keyring_name, logger)
+    elif sys.platform == 'darwin':
+        return MacChromeCookieDecryptor(browser_keyring_name, logger)
+    elif sys.platform == 'win32':
+        return WindowsChromeCookieDecryptor(browser_root, logger)
+    else:
+        raise NotImplementedError('Chrome cookie decryption is not supported '
+                                  'on this platform: {}'.format(sys.platform))
+
+
+class LinuxChromeCookieDecryptor(ChromeCookieDecryptor):
+    def __init__(self, browser_keyring_name, logger):
+        self._logger = logger
+        self._v10_key = self.derive_key(b'peanuts')
+        if KEYRING_AVAILABLE:
+            self._v11_key = self.derive_key(_get_linux_keyring_password(browser_keyring_name))
+        else:
+            self._v11_key = None
+
+    @staticmethod
+    def derive_key(password):
+        # values from
+        # https://chromium.googlesource.com/chromium/src/+/refs/heads/main/components/os_crypt/os_crypt_linux.cc
+        return pbkdf2_sha1(password, salt=b'saltysalt', iterations=1, key_length=16)
+
+    def decrypt(self, encrypted_value):
+        version = encrypted_value[:3]
+        ciphertext = encrypted_value[3:]
+
+        if version == b'v10':
+            return _decrypt_aes_cbc(ciphertext, self._v10_key, self._logger)
+
+        elif version == b'v11':
+            if self._v11_key is None:
+                self._logger.warning(f'cannot decrypt cookie {KEYRING_UNAVAILABLE_REASON}', only_once=True)
+                return None
+            return _decrypt_aes_cbc(ciphertext, self._v11_key, self._logger)
+
+        else:
+            return None
+
+
+class MacChromeCookieDecryptor(ChromeCookieDecryptor):
+    def __init__(self, browser_keyring_name, logger):
+        self._logger = logger
+        password = _get_mac_keyring_password(browser_keyring_name)
+        self._v10_key = None if password is None else self.derive_key(password)
+
+    @staticmethod
+    def derive_key(password):
+        # values from
+        # https://chromium.googlesource.com/chromium/src/+/refs/heads/main/components/os_crypt/os_crypt_mac.mm
+        return pbkdf2_sha1(password, salt=b'saltysalt', iterations=1003, key_length=16)
+
+    def decrypt(self, encrypted_value):
+        version = encrypted_value[:3]
+        ciphertext = encrypted_value[3:]
+
+        if version == b'v10':
+            if self._v10_key is None:
+                self._logger.warning('cannot decrypt v10 cookies: no key found', only_once=True)
+                return None
+
+            return _decrypt_aes_cbc(ciphertext, self._v10_key, self._logger)
+
+        else:
+            # other prefixes are considered 'old data' which were stored as plaintext
+            # https://chromium.googlesource.com/chromium/src/+/refs/heads/main/components/os_crypt/os_crypt_mac.mm
+            return encrypted_value
+
+
+class WindowsChromeCookieDecryptor(ChromeCookieDecryptor):
+    def __init__(self, browser_root, logger):
+        self._logger = logger
+        self._v10_key = _get_windows_v10_key(browser_root, logger)
+
+    def decrypt(self, encrypted_value):
+        version = encrypted_value[:3]
+        ciphertext = encrypted_value[3:]
+
+        if version == b'v10':
+            if self._v10_key is None:
+                self._logger.warning('cannot decrypt v10 cookies: no key found', only_once=True)
+                return None
+            elif not CRYPTO_AVAILABLE:
+                self._logger.warning('cannot decrypt cookie as the `pycryptodome` module is not installed. '
+                                     'Please install by running `python3 -m pip install pycryptodome`',
+                                     only_once=True)
+                return None
+
+            # https://chromium.googlesource.com/chromium/src/+/refs/heads/main/components/os_crypt/os_crypt_win.cc
+            #   kNonceLength
+            nonce_length = 96 // 8
+            # boringssl
+            #   EVP_AEAD_AES_GCM_TAG_LEN
+            authentication_tag_length = 16
+
+            raw_ciphertext = ciphertext
+            nonce = raw_ciphertext[:nonce_length]
+            ciphertext = raw_ciphertext[nonce_length:-authentication_tag_length]
+            authentication_tag = raw_ciphertext[-authentication_tag_length:]
+
+            return _decrypt_aes_gcm(ciphertext, self._v10_key, nonce, authentication_tag, self._logger)
+
+        else:
+            # any other prefix means the data is DPAPI encrypted
+            # https://chromium.googlesource.com/chromium/src/+/refs/heads/main/components/os_crypt/os_crypt_win.cc
+            return _decrypt_windows_dpapi(encrypted_value, self._logger).decode('utf-8')
+
+
+def _extract_safari_cookies(profile, logger):
+    if profile is not None:
+        logger.error('safari does not support profiles')
+    if sys.platform != 'darwin':
+        raise ValueError('unsupported platform: {}'.format(sys.platform))
+
+    cookies_path = os.path.expanduser('~/Library/Cookies/Cookies.binarycookies')
+
+    if not os.path.isfile(cookies_path):
+        raise FileNotFoundError('could not find safari cookies database')
+
+    with open(cookies_path, 'rb') as f:
+        cookies_data = f.read()
+
+    jar = parse_safari_cookies(cookies_data, logger=logger)
+    logger.info('Extracted {} cookies from safari'.format(len(jar)))
+    return jar
+
+
+class ParserError(Exception):
+    pass
+
+
+class DataParser:
+    def __init__(self, data, logger):
+        self._data = data
+        self.cursor = 0
+        self._logger = logger
+
+    def read_bytes(self, num_bytes):
+        if num_bytes < 0:
+            raise ParserError('invalid read of {} bytes'.format(num_bytes))
+        end = self.cursor + num_bytes
+        if end > len(self._data):
+            raise ParserError('reached end of input')
+        data = self._data[self.cursor:end]
+        self.cursor = end
+        return data
+
+    def expect_bytes(self, expected_value, message):
+        value = self.read_bytes(len(expected_value))
+        if value != expected_value:
+            raise ParserError('unexpected value: {} != {} ({})'.format(value, expected_value, message))
+
+    def read_uint(self, big_endian=False):
+        data_format = '>I' if big_endian else '<I'
+        return struct.unpack(data_format, self.read_bytes(4))[0]
+
+    def read_double(self, big_endian=False):
+        data_format = '>d' if big_endian else '<d'
+        return struct.unpack(data_format, self.read_bytes(8))[0]
+
+    def read_cstring(self):
+        buffer = []
+        while True:
+            c = self.read_bytes(1)
+            if c == b'\x00':
+                return b''.join(buffer).decode('utf-8')
+            else:
+                buffer.append(c)
+
+    def skip(self, num_bytes, description='unknown'):
+        if num_bytes > 0:
+            self._logger.debug('skipping {} bytes ({}): {}'.format(
+                num_bytes, description, self.read_bytes(num_bytes)))
+        elif num_bytes < 0:
+            raise ParserError('invalid skip of {} bytes'.format(num_bytes))
+
+    def skip_to(self, offset, description='unknown'):
+        self.skip(offset - self.cursor, description)
+
+    def skip_to_end(self, description='unknown'):
+        self.skip_to(len(self._data), description)
+
+
+def _mac_absolute_time_to_posix(timestamp):
+    return int((datetime(2001, 1, 1, 0, 0, tzinfo=timezone.utc) + timedelta(seconds=timestamp)).timestamp())
+
+
+def _parse_safari_cookies_header(data, logger):
+    p = DataParser(data, logger)
+    p.expect_bytes(b'cook', 'database signature')
+    number_of_pages = p.read_uint(big_endian=True)
+    page_sizes = [p.read_uint(big_endian=True) for _ in range(number_of_pages)]
+    return page_sizes, p.cursor
+
+
+def _parse_safari_cookies_page(data, jar, logger):
+    p = DataParser(data, logger)
+    p.expect_bytes(b'\x00\x00\x01\x00', 'page signature')
+    number_of_cookies = p.read_uint()
+    record_offsets = [p.read_uint() for _ in range(number_of_cookies)]
+    if number_of_cookies == 0:
+        logger.debug('a cookies page of size {} has no cookies'.format(len(data)))
+        return
+
+    p.skip_to(record_offsets[0], 'unknown page header field')
+
+    for record_offset in record_offsets:
+        p.skip_to(record_offset, 'space between records')
+        record_length = _parse_safari_cookies_record(data[record_offset:], jar, logger)
+        p.read_bytes(record_length)
+    p.skip_to_end('space in between pages')
+
+
+def _parse_safari_cookies_record(data, jar, logger):
+    p = DataParser(data, logger)
+    record_size = p.read_uint()
+    p.skip(4, 'unknown record field 1')
+    flags = p.read_uint()
+    is_secure = bool(flags & 0x0001)
+    p.skip(4, 'unknown record field 2')
+    domain_offset = p.read_uint()
+    name_offset = p.read_uint()
+    path_offset = p.read_uint()
+    value_offset = p.read_uint()
+    p.skip(8, 'unknown record field 3')
+    expiration_date = _mac_absolute_time_to_posix(p.read_double())
+    _creation_date = _mac_absolute_time_to_posix(p.read_double())  # noqa: F841
+
+    try:
+        p.skip_to(domain_offset)
+        domain = p.read_cstring()
+
+        p.skip_to(name_offset)
+        name = p.read_cstring()
+
+        p.skip_to(path_offset)
+        path = p.read_cstring()
+
+        p.skip_to(value_offset)
+        value = p.read_cstring()
+    except UnicodeDecodeError:
+        logger.warning('failed to parse cookie because UTF-8 decoding failed')
+        return record_size
+
+    p.skip_to(record_size, 'space at the end of the record')
+
+    cookie = compat_cookiejar_Cookie(
+        version=0, name=name, value=value, port=None, port_specified=False,
+        domain=domain, domain_specified=bool(domain), domain_initial_dot=domain.startswith('.'),
+        path=path, path_specified=bool(path), secure=is_secure, expires=expiration_date, discard=False,
+        comment=None, comment_url=None, rest={})
+    jar.set_cookie(cookie)
+    return record_size
+
+
+def parse_safari_cookies(data, jar=None, logger=YDLLogger()):
+    """
+    References:
+        - https://github.com/libyal/dtformats/blob/main/documentation/Safari%20Cookies.asciidoc
+            - this data appears to be out of date but the important parts of the database structure is the same
+            - there are a few bytes here and there which are skipped during parsing
+    """
+    if jar is None:
+        jar = YoutubeDLCookieJar()
+    page_sizes, body_start = _parse_safari_cookies_header(data, logger)
+    p = DataParser(data[body_start:], logger)
+    for page_size in page_sizes:
+        _parse_safari_cookies_page(p.read_bytes(page_size), jar, logger)
+    p.skip_to_end('footer')
+    return jar
+
+
+def _get_linux_keyring_password(browser_keyring_name):
+    password = keyring.get_password('{} Keys'.format(browser_keyring_name),
+                                    '{} Safe Storage'.format(browser_keyring_name))
+    if password is None:
+        # this sometimes occurs in KDE because chrome does not check hasEntry and instead
+        # just tries to read the value (which kwallet returns "") whereas keyring checks hasEntry
+        # to verify this:
+        # dbus-monitor "interface='org.kde.KWallet'" "type=method_return"
+        # while starting chrome.
+        # this may be a bug as the intended behaviour is to generate a random password and store
+        # it, but that doesn't matter here.
+        password = ''
+    return password.encode('utf-8')
+
+
+def _get_mac_keyring_password(browser_keyring_name):
+    if KEYRING_AVAILABLE:
+        password = keyring.get_password('{} Safe Storage'.format(browser_keyring_name), browser_keyring_name)
+        return password.encode('utf-8')
+    else:
+        proc = subprocess.Popen(['security', 'find-generic-password',
+                                 '-w',  # write password to stdout
+                                 '-a', browser_keyring_name,  # match 'account'
+                                 '-s', '{} Safe Storage'.format(browser_keyring_name)],  # match 'service'
+                                stdout=subprocess.PIPE,
+                                stderr=subprocess.DEVNULL)
+        try:
+            stdout, stderr = process_communicate_or_kill(proc)
+            return stdout
+        except BaseException:
+            return None
+
+
+def _get_windows_v10_key(browser_root, logger):
+    path = _find_most_recently_used_file(browser_root, 'Local State')
+    if path is None:
+        logger.error('could not find local state file')
+        return None
+    with open(path, 'r') as f:
+        data = json.load(f)
+    try:
+        base64_key = data['os_crypt']['encrypted_key']
+    except KeyError:
+        logger.error('no encrypted key in Local State')
+        return None
+    encrypted_key = compat_b64decode(base64_key)
+    prefix = b'DPAPI'
+    if not encrypted_key.startswith(prefix):
+        logger.error('invalid key')
+        return None
+    return _decrypt_windows_dpapi(encrypted_key[len(prefix):], logger)
+
+
+def pbkdf2_sha1(password, salt, iterations, key_length):
+    return pbkdf2_hmac('sha1', password, salt, iterations, key_length)
+
+
+def _decrypt_aes_cbc(ciphertext, key, logger, initialization_vector=b' ' * 16):
+    plaintext = aes_cbc_decrypt(bytes_to_intlist(ciphertext),
+                                bytes_to_intlist(key),
+                                bytes_to_intlist(initialization_vector))
+    padding_length = plaintext[-1]
+    try:
+        return intlist_to_bytes(plaintext[:-padding_length]).decode('utf-8')
+    except UnicodeDecodeError:
+        logger.warning('failed to decrypt cookie because UTF-8 decoding failed. Possibly the key is wrong?')
+        return None
+
+
+def _decrypt_aes_gcm(ciphertext, key, nonce, authentication_tag, logger):
+    cipher = AES.new(key, AES.MODE_GCM, nonce)
+    try:
+        plaintext = cipher.decrypt_and_verify(ciphertext, authentication_tag)
+    except ValueError:
+        logger.warning('failed to decrypt cookie because the MAC check failed. Possibly the key is wrong?')
+        return None
+
+    try:
+        return plaintext.decode('utf-8')
+    except UnicodeDecodeError:
+        logger.warning('failed to decrypt cookie because UTF-8 decoding failed. Possibly the key is wrong?')
+        return None
+
+
+def _decrypt_windows_dpapi(ciphertext, logger):
+    """
+    References:
+        - https://docs.microsoft.com/en-us/windows/win32/api/dpapi/nf-dpapi-cryptunprotectdata
+    """
+    from ctypes.wintypes import DWORD
+
+    class DATA_BLOB(ctypes.Structure):
+        _fields_ = [('cbData', DWORD),
+                    ('pbData', ctypes.POINTER(ctypes.c_char))]
+
+    buffer = ctypes.create_string_buffer(ciphertext)
+    blob_in = DATA_BLOB(ctypes.sizeof(buffer), buffer)
+    blob_out = DATA_BLOB()
+    ret = ctypes.windll.crypt32.CryptUnprotectData(
+        ctypes.byref(blob_in),  # pDataIn
+        None,  # ppszDataDescr: human readable description of pDataIn
+        None,  # pOptionalEntropy: salt?
+        None,  # pvReserved: must be NULL
+        None,  # pPromptStruct: information about prompts to display
+        0,  # dwFlags
+        ctypes.byref(blob_out)  # pDataOut
+    )
+    if not ret:
+        logger.warning('failed to decrypt with DPAPI')
+        return None
+
+    result = ctypes.string_at(blob_out.pbData, blob_out.cbData)
+    ctypes.windll.kernel32.LocalFree(blob_out.pbData)
+    return result
+
+
+def _config_home():
+    return os.environ.get('XDG_CONFIG_HOME', os.path.expanduser('~/.config'))
+
+
+def _open_database_copy(database_path, tmpdir):
+    # cannot open sqlite databases if they are already in use (e.g. by the browser)
+    database_copy_path = os.path.join(tmpdir, 'temporary.sqlite')
+    shutil.copy(database_path, database_copy_path)
+    conn = sqlite3.connect(database_copy_path)
+    return conn.cursor()
+
+
+def _get_column_names(cursor, table_name):
+    table_info = cursor.execute('PRAGMA table_info({})'.format(table_name)).fetchall()
+    return [row[1].decode('utf-8') for row in table_info]
+
+
+def _find_most_recently_used_file(root, filename):
+    # if there are multiple browser profiles, take the most recently used one
+    paths = []
+    for root, dirs, files in os.walk(root):
+        for file in files:
+            if file == filename:
+                paths.append(os.path.join(root, file))
+    return None if not paths else max(paths, key=lambda path: os.lstat(path).st_mtime)
+
+
+def _merge_cookie_jars(jars):
+    output_jar = YoutubeDLCookieJar()
+    for jar in jars:
+        for cookie in jar:
+            output_jar.set_cookie(cookie)
+        if jar.filename is not None:
+            output_jar.filename = jar.filename
+    return output_jar
+
+
+def _is_path(value):
+    return os.path.sep in value
+
+
+def _parse_browser_specification(browser_name, profile=None):
+    if browser_name not in SUPPORTED_BROWSERS:
+        raise ValueError(f'unsupported browser: "{browser_name}"')
+    if profile is not None and _is_path(profile):
+        profile = os.path.expanduser(profile)
+    return browser_name, profile
@@ -3,17 +3,20 @@ from __future__ import unicode_literals
 from ..compat import compat_str
 from ..utils import (
    determine_protocol,
+    NO_DEFAULT
 )


-def _get_real_downloader(info_dict, protocol=None, *args, **kwargs):
+def get_suitable_downloader(info_dict, params={}, default=NO_DEFAULT, protocol=None, to_stdout=False):
+    info_dict['protocol'] = determine_protocol(info_dict)
    info_copy = info_dict.copy()
    if protocol:
        info_copy['protocol'] = protocol
-    return get_suitable_downloader(info_copy, *args, **kwargs)
+    info_copy['to_stdout'] = to_stdout
+    return _get_suitable_downloader(info_copy, params, default)


-# Some of these require _get_real_downloader
+# Some of these require get_suitable_downloader
 from .common import FileDownloader
 from .dash import DashSegmentsFD
 from .f4m import F4mFD
@@ -69,22 +72,24 @@ def shorten_protocol_name(proto, simplify=False):
    return short_protocol_names.get(proto, proto)


-def get_suitable_downloader(info_dict, params={}, default=HttpFD):
+def _get_suitable_downloader(info_dict, params, default):
    """Get the downloader class that can handle the info dict."""
-    protocol = determine_protocol(info_dict)
-    info_dict['protocol'] = protocol
+    if default is NO_DEFAULT:
+        default = HttpFD

    # if (info_dict.get('start_time') or info_dict.get('end_time')) and not info_dict.get('requested_formats') and FFmpegFD.can_download(info_dict):
    #     return FFmpegFD

+    protocol = info_dict['protocol']
    downloaders = params.get('external_downloader')
    external_downloader = (
        downloaders if isinstance(downloaders, compat_str) or downloaders is None
        else downloaders.get(shorten_protocol_name(protocol, True), downloaders.get('default')))
-    if external_downloader and external_downloader.lower() == 'native':
-        external_downloader = 'native'

-    if external_downloader not in (None, 'native'):
+    if external_downloader is None:
+        if info_dict['to_stdout'] and FFmpegFD.can_merge_formats(info_dict, params):
+            return FFmpegFD
+    elif external_downloader.lower() != 'native':
        ed = get_external_downloader(external_downloader)
        if ed.can_download(info_dict, external_downloader):
            return ed
@@ -92,9 +97,10 @@ def get_suitable_downloader(info_dict, params={}, default=HttpFD):
    if protocol in ('m3u8', 'm3u8_native'):
        if info_dict.get('is_live'):
            return FFmpegFD
-        elif external_downloader == 'native':
+        elif (external_downloader or '').lower() == 'native':
            return HlsFD
-        elif _get_real_downloader(info_dict, 'm3u8_frag_urls', params, None):
+        elif get_suitable_downloader(
+                info_dict, params, None, protocol='m3u8_frag_urls', to_stdout=info_dict['to_stdout']):
            return HlsFD
        elif params.get('hls_prefer_native') is True:
            return HlsFD
@@ -1,5 +1,6 @@
 from __future__ import division, unicode_literals

+import copy
 import os
 import re
 import sys
@@ -360,7 +361,7 @@ class FileDownloader(object):
                    'filename': filename,
                    'status': 'finished',
                    'total_bytes': os.path.getsize(encodeFilename(filename)),
-                })
+                }, info_dict)
                return True, False

        if subtitle is False:
@@ -388,7 +389,16 @@ class FileDownloader(object):
        """Real download process. Redefine in subclasses."""
        raise NotImplementedError('This method must be implemented by subclasses')

-    def _hook_progress(self, status):
+    def _hook_progress(self, status, info_dict):
+        if not self._progress_hooks:
+            return
+        info_dict = dict(info_dict)
+        for key in ('__original_infodict', '__postprocessors'):
+            info_dict.pop(key, None)
+        # youtube-dl passes the same status object to all the hooks.
+        # Some third party scripts seems to be relying on this.
+        # So keep this behavior if possible
+        status['info_dict'] = copy.deepcopy(info_dict)
        for ph in self._progress_hooks:
            ph(status)

@@ -1,6 +1,6 @@
 from __future__ import unicode_literals

-from ..downloader import _get_real_downloader
+from ..downloader import get_suitable_downloader
 from .fragment import FragmentFD

 from ..utils import urljoin
@@ -15,11 +15,15 @@ class DashSegmentsFD(FragmentFD):
    FD_NAME = 'dashsegments'

    def real_download(self, filename, info_dict):
+        if info_dict.get('is_live'):
+            self.report_error('Live DASH videos are not supported')
+
        fragment_base_url = info_dict.get('fragment_base_url')
        fragments = info_dict['fragments'][:1] if self.params.get(
            'test', False) else info_dict['fragments']

-        real_downloader = _get_real_downloader(info_dict, 'dash_frag_urls', self.params, None)
+        real_downloader = get_suitable_downloader(
+            info_dict, self.params, None, protocol='dash_frag_urls', to_stdout=(filename == '-'))

        ctx = {
            'filename': filename,
@@ -29,7 +33,7 @@ class DashSegmentsFD(FragmentFD):
        if real_downloader:
            self._prepare_external_frag_download(ctx)
        else:
-            self._prepare_and_start_frag_download(ctx)
+            self._prepare_and_start_frag_download(ctx, info_dict)

        fragments_to_download = []
        frag_index = 0
@@ -54,9 +58,6 @@ class DashSegmentsFD(FragmentFD):
            info_copy = info_dict.copy()
            info_copy['fragments'] = fragments_to_download
            fd = real_downloader(self.ydl, self.params)
-            # TODO: Make progress updates work without hooking twice
-            # for ph in self._progress_hooks:
-            #     fd.add_progress_hook(ph)
            return fd.real_download(filename, info_copy)

        return self.download_and_append_fragments(ctx, fragments_to_download, info_dict)
@@ -36,6 +36,7 @@ from ..utils import (

 class ExternalFD(FileDownloader):
    SUPPORTED_PROTOCOLS = ('http', 'https', 'ftp', 'ftps')
+    can_download_to_stdout = False

    def real_download(self, filename, info_dict):
        self.report_destination(filename)
@@ -67,7 +68,7 @@ class ExternalFD(FileDownloader):
                    'downloaded_bytes': fsize,
                    'total_bytes': fsize,
                })
-            self._hook_progress(status)
+            self._hook_progress(status, info_dict)
            return True
        else:
            self.to_stderr('\n')
@@ -93,7 +94,9 @@ class ExternalFD(FileDownloader):

    @classmethod
    def supports(cls, info_dict):
-        return info_dict['protocol'] in cls.SUPPORTED_PROTOCOLS
+        return (
+            (cls.can_download_to_stdout or not info_dict.get('to_stdout'))
+            and info_dict['protocol'] in cls.SUPPORTED_PROTOCOLS)

    @classmethod
    def can_download(cls, info_dict, path=None):
@@ -341,16 +344,27 @@ class HttpieFD(ExternalFD):

 class FFmpegFD(ExternalFD):
    SUPPORTED_PROTOCOLS = ('http', 'https', 'ftp', 'ftps', 'm3u8', 'm3u8_native', 'rtsp', 'rtmp', 'rtmp_ffmpeg', 'mms')
+    can_download_to_stdout = True

    @classmethod
    def available(cls, path=None):
        # TODO: Fix path for ffmpeg
+        # Fixme: This may be wrong when --ffmpeg-location is used
        return FFmpegPostProcessor().available

    def on_process_started(self, proc, stdin):
        """ Override this in subclasses  """
        pass

+    @classmethod
+    def can_merge_formats(cls, info_dict, params={}):
+        return (
+            info_dict.get('requested_formats')
+            and info_dict.get('protocol')
+            and not params.get('allow_unplayable_formats')
+            and 'no-direct-merge' not in params.get('compat_opts', [])
+            and cls.can_download(info_dict))
+
    def _call_downloader(self, tmpfilename, info_dict):
        urls = [f['url'] for f in info_dict.get('requested_formats', [])] or [info_dict['url']]
        ffpp = FFmpegPostProcessor(downloader=self)
@@ -380,7 +380,7 @@ class F4mFD(FragmentFD):

        base_url_parsed = compat_urllib_parse_urlparse(base_url)

-        self._start_frag_download(ctx)
+        self._start_frag_download(ctx, info_dict)

        frag_index = 0
        while fragments_list:
@@ -434,6 +434,6 @@ class F4mFD(FragmentFD):
                    msg = 'Missed %d fragments' % (fragments_list[0][1] - (frag_i + 1))
                    self.report_warning(msg)

-        self._finish_frag_download(ctx)
+        self._finish_frag_download(ctx, info_dict)

        return True
@@ -83,9 +83,9 @@ class FragmentFD(FileDownloader):
        headers = info_dict.get('http_headers')
        return sanitized_Request(url, None, headers) if headers else url

-    def _prepare_and_start_frag_download(self, ctx):
+    def _prepare_and_start_frag_download(self, ctx, info_dict):
        self._prepare_frag_download(ctx)
-        self._start_frag_download(ctx)
+        self._start_frag_download(ctx, info_dict)

    def __do_ytdl_file(self, ctx):
        return not ctx['live'] and not ctx['tmpfilename'] == '-' and not self.params.get('_no_ytdl_file')
@@ -219,7 +219,7 @@ class FragmentFD(FileDownloader):
            'complete_frags_downloaded_bytes': resume_len,
        })

-    def _start_frag_download(self, ctx):
+    def _start_frag_download(self, ctx, info_dict):
        resume_len = ctx['complete_frags_downloaded_bytes']
        total_frags = ctx['total_frags']
        # This dict stores the download progress, it's updated by the progress
@@ -248,6 +248,7 @@ class FragmentFD(FileDownloader):
            time_now = time.time()
            state['elapsed'] = time_now - start
            frag_total_bytes = s.get('total_bytes') or 0
+            s['fragment_info_dict'] = s.pop('info_dict', {})
            if not ctx['live']:
                estimated_size = (
                    (ctx['complete_frags_downloaded_bytes'] + frag_total_bytes)
@@ -270,13 +271,13 @@ class FragmentFD(FileDownloader):
                state['speed'] = s.get('speed') or ctx.get('speed')
                ctx['speed'] = state['speed']
                ctx['prev_frag_downloaded_bytes'] = frag_downloaded_bytes
-            self._hook_progress(state)
+            self._hook_progress(state, info_dict)

        ctx['dl'].add_progress_hook(frag_progress_hook)

        return start

-    def _finish_frag_download(self, ctx):
+    def _finish_frag_download(self, ctx, info_dict):
        ctx['dest_stream'].close()
        if self.__do_ytdl_file(ctx):
            ytdl_filename = encodeFilename(self.ytdl_filename(ctx['filename']))
@@ -303,7 +304,7 @@ class FragmentFD(FileDownloader):
            'filename': ctx['filename'],
            'status': 'finished',
            'elapsed': elapsed,
-        })
+        }, info_dict)

    def _prepare_external_frag_download(self, ctx):
        if 'live' not in ctx:
@@ -421,5 +422,5 @@ class FragmentFD(FileDownloader):
                if not result:
                    return False

-        self._finish_frag_download(ctx)
+        self._finish_frag_download(ctx, info_dict)
        return True
@@ -4,7 +4,7 @@ import re
 import io
 import binascii

-from ..downloader import _get_real_downloader
+from ..downloader import get_suitable_downloader
 from .fragment import FragmentFD, can_decrypt_frag
 from .external import FFmpegFD

@@ -80,16 +80,14 @@ class HlsFD(FragmentFD):
            fd = FFmpegFD(self.ydl, self.params)
            self.report_warning(
                '%s detected unsupported features; extraction will be delegated to %s' % (self.FD_NAME, fd.get_basename()))
-            # TODO: Make progress updates work without hooking twice
-            # for ph in self._progress_hooks:
-            #     fd.add_progress_hook(ph)
            return fd.real_download(filename, info_dict)

        is_webvtt = info_dict['ext'] == 'vtt'
        if is_webvtt:
            real_downloader = None  # Packing the fragments is not currently supported for external downloader
        else:
-            real_downloader = _get_real_downloader(info_dict, 'm3u8_frag_urls', self.params, None)
+            real_downloader = get_suitable_downloader(
+                info_dict, self.params, None, protocol='m3u8_frag_urls', to_stdout=(filename == '-'))
        if real_downloader and not real_downloader.supports_manifest(s):
            real_downloader = None
        if real_downloader:
@@ -133,7 +131,7 @@ class HlsFD(FragmentFD):
        if real_downloader:
            self._prepare_external_frag_download(ctx)
        else:
-            self._prepare_and_start_frag_download(ctx)
+            self._prepare_and_start_frag_download(ctx, info_dict)

        extra_state = ctx.setdefault('extra_state', {})

@@ -177,7 +177,7 @@ class HttpFD(FileDownloader):
                                'status': 'finished',
                                'downloaded_bytes': ctx.resume_len,
                                'total_bytes': ctx.resume_len,
-                            })
+                            }, info_dict)
                            raise SucceedDownload()
                        else:
                            # The length does not match, we start the download over
@@ -310,7 +310,7 @@ class HttpFD(FileDownloader):
                    'eta': eta,
                    'speed': speed,
                    'elapsed': now - ctx.start_time,
-                })
+                }, info_dict)

                if data_len is not None and byte_counter == data_len:
                    break
@@ -357,7 +357,7 @@ class HttpFD(FileDownloader):
                'filename': ctx.filename,
                'status': 'finished',
                'elapsed': time.time() - ctx.start_time,
-            })
+            }, info_dict)

            return True

@@ -246,7 +246,7 @@ class IsmFD(FragmentFD):
            'total_frags': len(segments),
        }

-        self._prepare_and_start_frag_download(ctx)
+        self._prepare_and_start_frag_download(ctx, info_dict)

        extra_state = ctx.setdefault('extra_state', {
            'ism_track_written': False,
@@ -284,6 +284,6 @@ class IsmFD(FragmentFD):
                self.report_error('giving up after %s fragment retries' % fragment_retries)
                return False

-        self._finish_frag_download(ctx)
+        self._finish_frag_download(ctx, info_dict)

        return True
@@ -122,7 +122,7 @@ body > figure > img {
            'total_frags': len(fragments),
        }

-        self._prepare_and_start_frag_download(ctx)
+        self._prepare_and_start_frag_download(ctx, info_dict)

        extra_state = ctx.setdefault('extra_state', {
            'header_written': False,
@@ -198,5 +198,5 @@ body > figure > img {

        ctx['dest_stream'].write(
            b'--%b--\r\n\r\n' % frag_boundary.encode('us-ascii'))
-        self._finish_frag_download(ctx)
+        self._finish_frag_download(ctx, info_dict)
        return True
@@ -4,7 +4,7 @@ from __future__ import unicode_literals
 import threading

 from .common import FileDownloader
-from ..downloader import _get_real_downloader
+from ..downloader import get_suitable_downloader
 from ..extractor.niconico import NiconicoIE
 from ..compat import compat_urllib_request

@@ -20,7 +20,7 @@ class NiconicoDmcFD(FileDownloader):
        ie = NiconicoIE(self.ydl)
        info_dict, heartbeat_info_dict = ie._get_heartbeat_info(info_dict)

-        fd = _get_real_downloader(info_dict, params=self.params)(self.ydl, self.params)
+        fd = get_suitable_downloader(info_dict, params=self.params)(self.ydl, self.params)

        success = download_complete = False
        timer = [None]
@@ -66,7 +66,7 @@ class RtmpFD(FileDownloader):
                            'eta': eta,
                            'elapsed': time_now - start,
                            'speed': speed,
-                        })
+                        }, info_dict)
                        cursor_in_new_line = False
                    else:
                        # no percent for live streams
@@ -82,7 +82,7 @@ class RtmpFD(FileDownloader):
                                'status': 'downloading',
                                'elapsed': time_now - start,
                                'speed': speed,
-                            })
+                            }, info_dict)
                            cursor_in_new_line = False
                        elif self.params.get('verbose', False):
                            if not cursor_in_new_line:
@@ -208,7 +208,7 @@ class RtmpFD(FileDownloader):
                'filename': filename,
                'status': 'finished',
                'elapsed': time.time() - started,
-            })
+            }, info_dict)
            return True
        else:
            self.to_stderr('\n')
@@ -39,7 +39,7 @@ class RtspFD(FileDownloader):
                'total_bytes': fsize,
                'filename': filename,
                'status': 'finished',
-            })
+            }, info_dict)
            return True
        else:
            self.to_stderr('\n')
@@ -116,7 +116,7 @@ class YoutubeLiveChatFD(FragmentFD):
                    if not success:
                        return False, None, None, None
                    try:
-                        data = ie._extract_yt_initial_data(video_id, raw_fragment.decode('utf-8', 'replace'))
+                        data = ie.extract_yt_initial_data(video_id, raw_fragment.decode('utf-8', 'replace'))
                    except RegexNotFoundError:
                        data = None
                    if not data:
@@ -140,13 +140,13 @@ class YoutubeLiveChatFD(FragmentFD):
                self.report_error('giving up after %s fragment retries' % fragment_retries)
                return False, None, None, None

-        self._prepare_and_start_frag_download(ctx)
+        self._prepare_and_start_frag_download(ctx, info_dict)

        success, raw_fragment = dl_fragment(info_dict['url'])
        if not success:
            return False
        try:
-            data = ie._extract_yt_initial_data(video_id, raw_fragment.decode('utf-8', 'replace'))
+            data = ie.extract_yt_initial_data(video_id, raw_fragment.decode('utf-8', 'replace'))
        except RegexNotFoundError:
            return False
        continuation_id = try_get(
@@ -155,7 +155,7 @@ class YoutubeLiveChatFD(FragmentFD):
        # no data yet but required to call _append_fragment
        self._append_fragment(ctx, b'')

-        ytcfg = ie._extract_ytcfg(video_id, raw_fragment.decode('utf-8', 'replace'))
+        ytcfg = ie.extract_ytcfg(video_id, raw_fragment.decode('utf-8', 'replace'))

        if not ytcfg:
            return False
@@ -183,7 +183,7 @@ class YoutubeLiveChatFD(FragmentFD):
                request_data['currentPlayerState'] = {'playerOffsetMs': str(max(offset - 5000, 0))}
                if click_tracking_params:
                    request_data['context']['clickTracking'] = {'clickTrackingParams': click_tracking_params}
-                headers = ie._generate_api_headers(ytcfg, visitor_data=visitor_data)
+                headers = ie.generate_api_headers(ytcfg, visitor_data=visitor_data)
                headers.update({'content-type': 'application/json'})
                fragment_request_data = json.dumps(request_data, ensure_ascii=False).encode('utf-8') + b'\n'
                success, continuation_id, offset, click_tracking_params = download_and_parse_fragment(
@@ -196,7 +196,7 @@ class YoutubeLiveChatFD(FragmentFD):
            if test:
                break

-        self._finish_frag_download(ctx)
+        self._finish_frag_download(ctx, info_dict)
        return True

    @staticmethod
@@ -1330,6 +1330,11 @@ MSO_INFO = {
    'cou060': {
        'name': 'Zito Media'
    },
+    'slingtv': {
+        'name': 'Sling TV',
+        'username_field': 'username',
+        'password_field': 'password',
+    },
 }


@@ -1565,6 +1570,40 @@ class AdobePassIE(InfoExtractor):
                        }), headers={
                            'Content-Type': 'application/x-www-form-urlencoded'
                        })
+                elif mso_id == 'slingtv':
+                    # SlingTV has a meta-refresh based authentication, but also
+                    # looks at the tab history to count the number of times the
+                    # browser has been on a page
+
+                    first_bookend_page, urlh = provider_redirect_page_res
+
+                    hidden_data = self._hidden_inputs(first_bookend_page)
+                    hidden_data['history'] = 1
+
+                    provider_login_page_res = self._download_webpage_handle(
+                        urlh.geturl(), video_id, 'Sending first bookend.',
+                        query=hidden_data)
+
+                    provider_association_redirect, urlh = post_form(
+                        provider_login_page_res, 'Logging in', {
+                            mso_info['username_field']: username,
+                            mso_info['password_field']: password
+                        })
+
+                    provider_refresh_redirect_url = extract_redirect_url(
+                        provider_association_redirect, url=urlh.geturl())
+
+                    last_bookend_page, urlh = self._download_webpage_handle(
+                        provider_refresh_redirect_url, video_id,
+                        'Downloading Auth Association Redirect Page')
+                    hidden_data = self._hidden_inputs(last_bookend_page)
+                    hidden_data['history'] = 3
+
+                    mvpd_confirm_page_res = self._download_webpage_handle(
+                        urlh.geturl(), video_id, 'Sending final bookend.',
+                        query=hidden_data)
+
+                    post_form(mvpd_confirm_page_res, 'Confirming Login')
                else:
                    # Some providers (e.g. DIRECTV NOW) have another meta refresh
                    # based redirect that should be followed.
@@ -37,7 +37,7 @@ class BiliBiliIE(InfoExtractor):
                                video/[aA][vV]|
                                anime/(?P<anime_id>\d+)/play\#
                            )(?P<id>\d+)|
-                            video/[bB][vV](?P<id_bv>[^/?#&]+)
+                            (s/)?video/[bB][vV](?P<id_bv>[^/?#&]+)
                        )
                        (?:/?\?p=(?P<page>\d+))?
                    '''
@@ -8,6 +8,9 @@ from ..utils import (
    smuggle_url,
    update_url_query,
    int_or_none,
+    float_or_none,
+    try_get,
+    dict_get,
 )


@@ -24,6 +27,11 @@ class BravoTVIE(AdobePassIE):
            'uploader': 'NBCU-BRAV',
            'upload_date': '20190314',
            'timestamp': 1552591860,
+            'season_number': 16,
+            'episode_number': 15,
+            'series': 'Top Chef',
+            'episode': 'The Top Chef Season 16 Winner Is...',
+            'duration': 190.0,
        }
    }, {
        'url': 'http://www.bravotv.com/below-deck/season-3/ep-14-reunion-part-1',
@@ -79,12 +87,34 @@ class BravoTVIE(AdobePassIE):
                'episode_number': int_or_none(metadata.get('episode_num')),
            })
            query['switch'] = 'progressive'
+
+        tp_url = 'http://link.theplatform.com/s/%s/%s' % (account_pid, tp_path)
+
+        tp_metadata = self._download_json(
+            update_url_query(tp_url, {'format': 'preview'}),
+            display_id, fatal=False)
+        if tp_metadata:
+            info.update({
+                'title': tp_metadata.get('title'),
+                'description': tp_metadata.get('description'),
+                'duration': float_or_none(tp_metadata.get('duration'), 1000),
+                'season_number': int_or_none(
+                    dict_get(tp_metadata, ('pl1$seasonNumber', 'nbcu$seasonNumber'))),
+                'episode_number': int_or_none(
+                    dict_get(tp_metadata, ('pl1$episodeNumber', 'nbcu$episodeNumber'))),
+                # For some reason the series is sometimes wrapped into a single element array.
+                'series': try_get(
+                    dict_get(tp_metadata, ('pl1$show', 'nbcu$show')),
+                    lambda x: x[0] if isinstance(x, list) else x,
+                    expected_type=str),
+                'episode': dict_get(
+                    tp_metadata, ('pl1$episodeName', 'nbcu$episodeName', 'title')),
+            })
+
        info.update({
            '_type': 'url_transparent',
            'id': release_pid,
-            'url': smuggle_url(update_url_query(
-                'http://link.theplatform.com/s/%s/%s' % (account_pid, tp_path),
-                query), {'force_smil_url': True}),
+            'url': smuggle_url(update_url_query(tp_url, query), {'force_smil_url': True}),
            'ie_key': 'ThePlatform',
        })
        return info
@@ -1,5 +1,6 @@
 from __future__ import unicode_literals

+from .common import InfoExtractor
 from .theplatform import ThePlatformFeedIE
 from ..utils import (
    ExtractorError,
@@ -52,6 +53,54 @@ class CBSIE(CBSBaseIE):
            'skip_download': True,
        },
        '_skip': 'Blocked outside the US',
+    }, {
+        'url': 'https://www.paramountplus.com/shows/catdog/video/Oe44g5_NrlgiZE3aQVONleD6vXc8kP0k/catdog-climb-every-catdog-the-canine-mutiny/',
+        'info_dict': {
+            'id': 'Oe44g5_NrlgiZE3aQVONleD6vXc8kP0k',
+            'ext': 'mp4',
+            'title': 'CatDog - Climb Every CatDog/The Canine Mutiny',
+            'description': 'md5:7ac835000645a69933df226940e3c859',
+            'duration': 1418,
+            'timestamp': 920264400,
+            'upload_date': '19990301',
+            'uploader': 'CBSI-NEW',
+        },
+        'params': {
+            'skip_download': 'm3u8',
+        },
+        '_skip': 'Blocked outside the US',
+    }, {
+        'url': 'https://www.paramountplus.com/shows/tooning-out-the-news/video/6hSWYWRrR9EUTz7IEe5fJKBhYvSUfexd/7-23-21-week-in-review-rep-jahana-hayes-howard-fineman-sen-michael-bennet-sheera-frenkel-cecilia-kang-/',
+        'info_dict': {
+            'id': '6hSWYWRrR9EUTz7IEe5fJKBhYvSUfexd',
+            'ext': 'mp4',
+            'title': '7/23/21 WEEK IN REVIEW (Rep. Jahana Hayes/Howard Fineman/Sen. Michael Bennet/Sheera Frenkel & Cecilia Kang)',
+            'description': 'md5:f4adcea3e8b106192022e121f1565bae',
+            'duration': 2506,
+            'timestamp': 1627063200,
+            'upload_date': '20210723',
+            'uploader': 'CBSI-NEW',
+        },
+        'params': {
+            'skip_download': 'm3u8',
+        },
+        '_skip': 'Blocked outside the US',
+    }, {
+        'url': 'https://www.cbs.com/shows/the-late-show-with-stephen-colbert/video/60icOhMb9NcjbcWnF_gub9XXHdeBcNk2/the-late-show-6-23-21-christine-baranski-joy-oladokun-',
+        'info_dict': {
+            'id': '60icOhMb9NcjbcWnF_gub9XXHdeBcNk2',
+            'title': 'The Late Show - 6/23/21 (Christine Baranski, Joy Oladokun)',
+            'timestamp': 1624507140,
+            'description': 'md5:e01af24e95c74d55e8775aef86117b95',
+            'uploader': 'CBSI-NEW',
+            'upload_date': '20210624',
+        },
+        'params': {
+            'ignore_no_formats_error': True,
+            'skip_download': True,
+        },
+        'expected_warnings': [
+            'This content expired on', 'No video formats found', 'Requested format is not available'],
    }, {
        'url': 'http://colbertlateshow.com/video/8GmB0oY0McANFvp2aEffk9jZZZ2YyXxy/the-colbeard/',
        'only_matching': True,
@@ -78,17 +127,26 @@ class CBSIE(CBSBaseIE):
        asset_types = []
        subtitles = {}
        formats = []
+        useXMLmetadata = True
        last_e = None
        for item in items_data.findall('.//item'):
            asset_type = xpath_text(item, 'assetType')
-            if not asset_type or asset_type in asset_types or 'HLS_FPS' in asset_type or 'DASH_CENC' in asset_type:
-                continue
-            asset_types.append(asset_type)
            query = {
                'mbr': 'true',
                'assetTypes': asset_type,
            }
-            if asset_type.startswith('HLS') or asset_type in ('OnceURL', 'StreamPack'):
+            if not asset_type:
+                # fallback for content_ids that videoPlayerService doesn't return anything for
+                useXMLmetadata = False
+                asset_type = 'fallback'
+                query['formats'] = 'M3U+none,MPEG4,M3U+appleHlsEncryption,MP3'
+                del query['assetTypes']
+            elif asset_type in asset_types:
+                continue
+            elif any(excluded in asset_type for excluded in ('HLS_FPS', 'DASH_CENC', 'OnceURL')):
+                continue
+            asset_types.append(asset_type)
+            if asset_type.startswith('HLS') or 'StreamPack' in asset_type:
                query['formats'] = 'MPEG4,M3U'
            elif asset_type in ('RTMP', 'WIFI', '3G'):
                query['formats'] = 'MPEG4,FLV'
@@ -98,27 +156,75 @@ class CBSIE(CBSBaseIE):
                    'Downloading %s SMIL data' % asset_type)
            except ExtractorError as e:
                last_e = e
-                continue
+                if useXMLmetadata:
+                    continue
+                query['formats'] = ''  # blank query to check if expired
+                try:
+                    tp_formats, tp_subtitles = self._extract_theplatform_smil(
+                        update_url_query(tp_release_url, query), content_id,
+                        'Downloading %s SMIL data, trying again with another format' % asset_type)
+                except ExtractorError as e:
+                    last_e = e
+                    continue
            formats.extend(tp_formats)
            subtitles = self._merge_subtitles(subtitles, tp_subtitles)
        if last_e and not formats:
-            raise last_e
+            self.raise_no_formats(last_e, True, content_id)
        self._sort_formats(formats)

        info = self._extract_theplatform_metadata(tp_path, content_id)
        info.update({
-            'id': content_id,
-            'title': title,
-            'series': xpath_text(video_data, 'seriesTitle'),
-            'season_number': int_or_none(xpath_text(video_data, 'seasonNumber')),
-            'episode_number': int_or_none(xpath_text(video_data, 'episodeNumber')),
-            'duration': int_or_none(xpath_text(video_data, 'videoLength'), 1000),
-            'thumbnail': xpath_text(video_data, 'previewImageURL'),
            'formats': formats,
            'subtitles': subtitles,
+            'id': content_id
        })
+        if useXMLmetadata:
+            info.update({
+                'title': title,
+                'series': xpath_text(video_data, 'seriesTitle'),
+                'season_number': int_or_none(xpath_text(video_data, 'seasonNumber')),
+                'episode_number': int_or_none(xpath_text(video_data, 'episodeNumber')),
+                'duration': int_or_none(xpath_text(video_data, 'videoLength'), 1000),
+                'thumbnail': xpath_text(video_data, 'previewImageURL')
+            })
        return info

    def _real_extract(self, url):
        content_id = self._match_id(url)
        return self._extract_video_info(content_id)
+
+
+class ParamountPlusSeriesIE(InfoExtractor):
+    _VALID_URL = r'https?://(?:www\.)?paramountplus\.com/shows/(?P<id>[a-zA-Z0-9-_]+)/?(?:[#?]|$)'
+    _TESTS = [{
+        'url': 'https://www.paramountplus.com/shows/drake-josh',
+        'playlist_mincount': 50,
+        'info_dict': {
+            'id': 'drake-josh',
+        }
+    }, {
+        'url': 'https://www.paramountplus.com/shows/hawaii_five_0/',
+        'playlist_mincount': 240,
+        'info_dict': {
+            'id': 'hawaii_five_0',
+        }
+    }, {
+        'url': 'https://www.paramountplus.com/shows/spongebob-squarepants/',
+        'playlist_mincount': 248,
+        'info_dict': {
+            'id': 'spongebob-squarepants',
+        }
+    }]
+    _API_URL = 'https://www.paramountplus.com/shows/{}/xhr/episodes/page/0/size/100000/xs/0/season/0/'
+
+    def _entries(self, show_name):
+        show_json = self._download_json(self._API_URL.format(show_name), video_id=show_name)
+        if show_json.get('success'):
+            for episode in show_json['result']['data']:
+                yield self.url_result(
+                    'https://www.paramountplus.com%s' % episode['url'],
+                    ie=CBSIE.ie_key(), video_id=episode['content_id'])
+
+    def _real_extract(self, url):
+        show_name = self._match_id(url)
+        return self.playlist_result(self._entries(show_name), playlist_id=show_name)
@@ -19,7 +19,6 @@ from ..compat import (
    compat_etree_Element,
    compat_etree_fromstring,
    compat_getpass,
-    compat_integer_types,
    compat_http_client,
    compat_os_name,
    compat_str,
@@ -79,6 +78,7 @@ from ..utils import (
    urljoin,
    url_basename,
    url_or_none,
+    variadic,
    xpath_element,
    xpath_text,
    xpath_with_ns,
@@ -229,6 +229,7 @@ class InfoExtractor(object):
                        * "resolution" (optional, string "{width}x{height}",
                                        deprecated)
                        * "filesize" (optional, int)
+                        * "_test_url" (optional, bool) - If true, test the URL
    thumbnail:      Full URL to a video thumbnail image.
    description:    Full video description.
    uploader:       Full name of the video uploader.
@@ -296,6 +297,8 @@ class InfoExtractor(object):
                    live stream that goes on instead of a fixed-length video.
    was_live:       True, False, or None (=unknown). Whether this video was
                    originally a live stream.
+    live_status:    'is_live', 'upcoming', 'was_live', 'not_live' or None (=unknown)
+                    If absent, automatically set from is_live, was_live
    start_time:     Time in seconds where the reproduction should start, as
                    specified in the URL.
    end_time:       Time in seconds where the reproduction should end, as
@@ -628,14 +631,10 @@ class InfoExtractor(object):
        assert isinstance(err, compat_urllib_error.HTTPError)
        if expected_status is None:
            return False
-        if isinstance(expected_status, compat_integer_types):
-            return err.code == expected_status
-        elif isinstance(expected_status, (list, tuple)):
-            return err.code in expected_status
        elif callable(expected_status):
            return expected_status(err.code) is True
        else:
-            assert False
+            return err.code in variadic(expected_status)

    def _request_webpage(self, url_or_request, video_id, note=None, errnote=None, fatal=True, data=None, headers={}, query={}, expected_status=None):
        """
@@ -1053,6 +1052,8 @@ class InfoExtractor(object):
    def raise_no_formats(self, msg, expected=False, video_id=None):
        if expected and self.get_param('ignore_no_formats_error'):
            self.report_warning(msg, video_id)
+        elif isinstance(msg, ExtractorError):
+            raise msg
        else:
            raise ExtractorError(msg, expected=expected, video_id=video_id)

@@ -1115,6 +1116,8 @@ class InfoExtractor(object):
            if group is None:
                # return the first matching group
                return next(g for g in mobj.groups() if g is not None)
+            elif isinstance(group, (list, tuple)):
+                return tuple(mobj.group(g) for g in group)
            else:
                return mobj.group(group)
        elif default is not NO_DEFAULT:
@@ -1207,8 +1210,7 @@ class InfoExtractor(object):
                    [^>]+?content=(["\'])(?P<content>.*?)\2''' % re.escape(prop)

    def _og_search_property(self, prop, html, name=None, **kargs):
-        if not isinstance(prop, (list, tuple)):
-            prop = [prop]
+        prop = variadic(prop)
        if name is None:
            name = 'OpenGraph %s' % prop[0]
        og_regexes = []
@@ -1238,8 +1240,7 @@ class InfoExtractor(object):
        return self._og_search_property('url', html, **kargs)

    def _html_search_meta(self, name, html, display_name=None, fatal=False, **kwargs):
-        if not isinstance(name, (list, tuple)):
-            name = [name]
+        name = variadic(name)
        if display_name is None:
            display_name = name[0]
        return self._html_search_regex(
@@ -1299,7 +1300,7 @@ class InfoExtractor(object):
        # JSON-LD may be malformed and thus `fatal` should be respected.
        # At the same time `default` may be passed that assumes `fatal=False`
        # for _search_regex. Let's simulate the same behavior here as well.
-        fatal = kwargs.get('fatal', True) if default == NO_DEFAULT else False
+        fatal = kwargs.get('fatal', True) if default is NO_DEFAULT else False
        json_ld = []
        for mobj in json_ld_list:
            json_ld_item = self._parse_json(
@@ -1498,7 +1499,7 @@ class InfoExtractor(object):
                     'order': ('m4a', 'aac', 'mp3', 'ogg', 'opus', 'webm', '', 'none'),
                     'order_free': ('opus', 'ogg', 'webm', 'm4a', 'mp3', 'aac', '', 'none')},
            'hidden': {'visible': False, 'forced': True, 'type': 'extractor', 'max': -1000},
-            'aud_or_vid': {'visible': False, 'forced': True, 'type': 'multiple', 'default': 1,
+            'aud_or_vid': {'visible': False, 'forced': True, 'type': 'multiple',
                           'field': ('vcodec', 'acodec'),
                           'function': lambda it: int(any(v != 'none' for v in it))},
            'ie_pref': {'priority': True, 'type': 'extractor'},
@@ -1522,7 +1523,8 @@ class InfoExtractor(object):
            'br': {'type': 'combined', 'field': ('tbr', 'vbr', 'abr'), 'same_limit': True},
            'size': {'type': 'combined', 'same_limit': True, 'field': ('filesize', 'fs_approx')},
            'ext': {'type': 'combined', 'field': ('vext', 'aext')},
-            'res': {'type': 'multiple', 'field': ('height', 'width'), 'function': min},
+            'res': {'type': 'multiple', 'field': ('height', 'width'),
+                    'function': lambda it: (lambda l: min(l) if l else 0)(tuple(filter(None, it)))},

            # Most of these exist only for compatibility reasons
            'dimension': {'type': 'alias', 'field': 'res'},
@@ -1566,7 +1568,7 @@ class InfoExtractor(object):
                elif key == 'convert':
                    default = 'order' if type == 'ordered' else 'float_string' if field else 'ignore'
                else:
-                    default = {'type': 'field', 'visible': True, 'order': [], 'not_in_list': (None,), 'function': max}.get(key, None)
+                    default = {'type': 'field', 'visible': True, 'order': [], 'not_in_list': (None,)}.get(key, None)
                propObj[key] = default
            return propObj[key]

@@ -1706,11 +1708,7 @@ class InfoExtractor(object):
                type = 'field'  # Only 'field' is allowed in multiple for now
                actual_fields = self._get_field_setting(field, 'field')

-                def wrapped_function(values):
-                    values = tuple(filter(lambda x: x is not None, values))
-                    return self._get_field_setting(field, 'function')(values) if values else None
-
-                value = wrapped_function((get_value(f) for f in actual_fields))
+                value = self._get_field_setting(field, 'function')(get_value(f) for f in actual_fields)
            else:
                value = get_value(field)
            return self._calculate_field_preference_from_value(format, field, type, value)
@@ -2210,7 +2208,7 @@ class InfoExtractor(object):
                out.append('{%s}%s' % (namespace, c))
        return '/'.join(out)

-    def _extract_smil_formats(self, smil_url, video_id, fatal=True, f4m_params=None, transform_source=None):
+    def _extract_smil_formats_and_subtitles(self, smil_url, video_id, fatal=True, f4m_params=None, transform_source=None):
        smil = self._download_smil(smil_url, video_id, fatal=fatal, transform_source=transform_source)

        if smil is False:
@@ -2219,8 +2217,21 @@ class InfoExtractor(object):

        namespace = self._parse_smil_namespace(smil)

-        return self._parse_smil_formats(
+        fmts = self._parse_smil_formats(
            smil, smil_url, video_id, namespace=namespace, f4m_params=f4m_params)
+        subs = self._parse_smil_subtitles(
+            smil, namespace=namespace)
+
+        return fmts, subs
+
+    def _extract_smil_formats(self, *args, **kwargs):
+        fmts, subs = self._extract_smil_formats_and_subtitles(*args, **kwargs)
+        if subs:
+            self.report_warning(bug_reports_message(
+                "Ignoring subtitle tracks found in the SMIL manifest; "
+                "if any subtitle tracks are missing,"
+            ))
+        return fmts

    def _extract_smil_info(self, smil_url, video_id, fatal=True, f4m_params=None):
        smil = self._download_smil(smil_url, video_id, fatal=fatal)
@@ -3498,9 +3509,18 @@ class InfoExtractor(object):
            else 'public' if all_known
            else None)

-    def _configuration_arg(self, key):
-        return traverse_obj(
+    def _configuration_arg(self, key, default=NO_DEFAULT, casesense=False):
+        '''
+        @returns            A list of values for the extractor argument given by "key"
+                            or "default" if no such key is present
+        @param default      The default value to return when the key is not present (default: [])
+        @param casesense    When false, the values are converted to lower case
+        '''
+        val = traverse_obj(
            self._downloader.params, ('extractor_args', self.ie_key().lower(), key))
+        if val is None:
+            return [] if default is NO_DEFAULT else default
+        return list(val) if casesense else [x.lower() for x in val]


 class SearchInfoExtractor(InfoExtractor):
@@ -636,7 +636,7 @@ class CrunchyrollShowPlaylistIE(CrunchyrollBaseIE):
    _VALID_URL = r'https?://(?:(?P<prefix>www|m)\.)?(?P<url>crunchyroll\.com/(?!(?:news|anime-news|library|forum|launchcalendar|lineup|store|comics|freetrial|login|media-\d+))(?P<id>[\w\-]+))/?(?:\?|$)'

    _TESTS = [{
-        'url': 'http://www.crunchyroll.com/a-bridge-to-the-starry-skies-hoshizora-e-kakaru-hashi',
+        'url': 'https://www.crunchyroll.com/a-bridge-to-the-starry-skies-hoshizora-e-kakaru-hashi',
        'info_dict': {
            'id': 'a-bridge-to-the-starry-skies-hoshizora-e-kakaru-hashi',
            'title': 'A Bridge to the Starry Skies - Hoshizora e Kakaru Hashi'
@@ -661,7 +661,8 @@ class CrunchyrollShowPlaylistIE(CrunchyrollBaseIE):
        show_id = self._match_id(url)

        webpage = self._download_webpage(
-            self._add_skip_wall(url), show_id,
+            # https:// gives a 403, but http:// does not
+            self._add_skip_wall(url).replace('https://', 'http://'), show_id,
            headers=self.geo_verification_headers())
        title = self._html_search_meta('name', webpage, default=None)

@@ -63,8 +63,7 @@ class DiscoveryPlusIndiaShowIE(InfoExtractor):
        'info_dict': {
            'id': 'how-do-they-do-it',
        },
-    }
-    ]
+    }]

    def _entries(self, show_name):
        headers = {
@@ -0,0 +1,145 @@
+# coding: utf-8
+
+from ..utils import (
+    int_or_none,
+    traverse_obj,
+    url_or_none,
+)
+from .common import (
+    InfoExtractor,
+    compat_urllib_parse_unquote,
+)
+
+
+class DouyinIE(InfoExtractor):
+    _VALID_URL = r'https?://(?:www\.)?douyin\.com/video/(?P<id>[0-9]+)'
+    _TESTS = [{
+        'url': 'https://www.douyin.com/video/6961737553342991651',
+        'md5': '10523312c8b8100f353620ac9dc8f067',
+        'info_dict': {
+            'id': '6961737553342991651',
+            'ext': 'mp4',
+            'title': '#杨超越  小小水手带你去远航❤️',
+            'uploader': '杨超越',
+            'upload_date': '20210513',
+            'timestamp': 1620905839,
+            'uploader_id': '110403406559',
+            'view_count': int,
+            'like_count': int,
+            'repost_count': int,
+            'comment_count': int,
+        }
+    }, {
+        'url': 'https://www.douyin.com/video/6982497745948921092',
+        'md5': 'd78408c984b9b5102904cf6b6bc2d712',
+        'info_dict': {
+            'id': '6982497745948921092',
+            'ext': 'mp4',
+            'title': '这个夏日和小羊@杨超越 一起遇见白色幻想',
+            'uploader': '杨超越工作室',
+            'upload_date': '20210708',
+            'timestamp': 1625739481,
+            'uploader_id': '408654318141572',
+            'view_count': int,
+            'like_count': int,
+            'repost_count': int,
+            'comment_count': int,
+        }
+    }, {
+        'url': 'https://www.douyin.com/video/6953975910773099811',
+        'md5': '72e882e24f75064c218b76c8b713c185',
+        'info_dict': {
+            'id': '6953975910773099811',
+            'ext': 'mp4',
+            'title': '#一起看海  出现在你的夏日里',
+            'uploader': '杨超越',
+            'upload_date': '20210422',
+            'timestamp': 1619098692,
+            'uploader_id': '110403406559',
+            'view_count': int,
+            'like_count': int,
+            'repost_count': int,
+            'comment_count': int,
+        }
+    }, {
+        'url': 'https://www.douyin.com/video/6950251282489675042',
+        'md5': 'b4db86aec367ef810ddd38b1737d2fed',
+        'info_dict': {
+            'id': '6950251282489675042',
+            'ext': 'mp4',
+            'title': '哈哈哈，成功了哈哈哈哈哈哈',
+            'uploader': '杨超越',
+            'upload_date': '20210412',
+            'timestamp': 1618231483,
+            'uploader_id': '110403406559',
+            'view_count': int,
+            'like_count': int,
+            'repost_count': int,
+            'comment_count': int,
+        }
+    }, {
+        'url': 'https://www.douyin.com/video/6963263655114722595',
+        'md5': '1abe1c477d05ee62efb40bf2329957cf',
+        'info_dict': {
+            'id': '6963263655114722595',
+            'ext': 'mp4',
+            'title': '#哪个爱豆的105度最甜 换个角度看看我哈哈',
+            'uploader': '杨超越',
+            'upload_date': '20210517',
+            'timestamp': 1621261163,
+            'uploader_id': '110403406559',
+            'view_count': int,
+            'like_count': int,
+            'repost_count': int,
+            'comment_count': int,
+        }
+    }]
+
+    def _real_extract(self, url):
+        video_id = self._match_id(url)
+        webpage = self._download_webpage(url, video_id)
+        render_data = self._parse_json(
+            self._search_regex(
+                r'<script [^>]*\bid=[\'"]RENDER_DATA[\'"][^>]*>(%7B.+%7D)</script>',
+                webpage, 'render data'),
+            video_id, transform_source=compat_urllib_parse_unquote)
+        details = traverse_obj(render_data, (..., 'aweme', 'detail'), get_all=False)
+
+        thumbnails = [{'url': self._proto_relative_url(url)} for url in traverse_obj(
+            details, ('video', ('cover', 'dynamicCover', 'originCover')), expected_type=url_or_none, default=[])]
+
+        common = {
+            'width': traverse_obj(details, ('video', 'width'), expected_type=int),
+            'height': traverse_obj(details, ('video', 'height'), expected_type=int),
+            'ext': 'mp4',
+        }
+        formats = [{**common, 'url': self._proto_relative_url(url)} for url in traverse_obj(
+            details, ('video', 'playAddr', ..., 'src'), expected_type=url_or_none, default=[]) if url]
+        self._remove_duplicate_formats(formats)
+
+        download_url = traverse_obj(details, ('download', 'url'), expected_type=url_or_none)
+        if download_url:
+            formats.append({
+                **common,
+                'format_id': 'download',
+                'url': self._proto_relative_url(download_url),
+                'quality': 1,
+            })
+        self._sort_formats(formats)
+
+        return {
+            'id': video_id,
+            'title': details.get('desc') or self._html_search_meta('title', webpage),
+            'formats': formats,
+            'thumbnails': thumbnails,
+            'uploader': traverse_obj(details, ('authorInfo', 'nickname'), expected_type=str),
+            'uploader_id': traverse_obj(details, ('authorInfo', 'uid'), expected_type=str),
+            'uploader_url': 'https://www.douyin.com/user/%s' % traverse_obj(
+                details, ('authorInfo', 'secUid'), expected_type=str),
+            'timestamp': int_or_none(details.get('createTime')),
+            'duration': traverse_obj(details, ('video', 'duration'), expected_type=int),
+            'view_count': traverse_obj(details, ('stats', 'playCount'), expected_type=int),
+            'like_count': traverse_obj(details, ('stats', 'diggCount'), expected_type=int),
+            'repost_count': traverse_obj(details, ('stats', 'shareCount'), expected_type=int),
+            'comment_count': traverse_obj(details, ('stats', 'commentCount'), expected_type=int),
+        }
@@ -296,50 +296,6 @@ class DPlayIE(InfoExtractor):
            url, display_id, host, 'dplay' + country, country)


-class DiscoveryPlusIE(DPlayIE):
-    _VALID_URL = r'https?://(?:www\.)?discoveryplus\.com/video' + DPlayIE._PATH_REGEX
-    _TESTS = [{
-        'url': 'https://www.discoveryplus.com/video/property-brothers-forever-home/food-and-family',
-        'info_dict': {
-            'id': '1140794',
-            'display_id': 'property-brothers-forever-home/food-and-family',
-            'ext': 'mp4',
-            'title': 'Food and Family',
-            'description': 'The brothers help a Richmond family expand their single-level home.',
-            'duration': 2583.113,
-            'timestamp': 1609304400,
-            'upload_date': '20201230',
-            'creator': 'HGTV',
-            'series': 'Property Brothers: Forever Home',
-            'season_number': 1,
-            'episode_number': 1,
-        },
-        'skip': 'Available for Premium users',
-    }]
-
-    def _update_disco_api_headers(self, headers, disco_base, display_id, realm):
-        headers['x-disco-client'] = 'WEB:UNKNOWN:dplus_us:15.0.0'
-
-    def _download_video_playback_info(self, disco_base, video_id, headers):
-        return self._download_json(
-            disco_base + 'playback/v3/videoPlaybackInfo',
-            video_id, headers=headers, data=json.dumps({
-                'deviceInfo': {
-                    'adBlocker': False,
-                },
-                'videoId': video_id,
-                'wisteriaProperties': {
-                    'platform': 'desktop',
-                    'product': 'dplus_us',
-                },
-            }).encode('utf-8'))['data']['attributes']['streaming']
-
-    def _real_extract(self, url):
-        display_id = self._match_id(url)
-        return self._get_disco_api_info(
-            url, display_id, 'us1-prod-direct.discoveryplus.com', 'go', 'us')
-
-
 class HGTVDeIE(DPlayIE):
    _VALID_URL = r'https?://de\.hgtv\.com/sendungen' + DPlayIE._PATH_REGEX
    _TESTS = [{
@@ -367,3 +323,70 @@ class HGTVDeIE(DPlayIE):
        display_id = self._match_id(url)
        return self._get_disco_api_info(
            url, display_id, 'eu1-prod.disco-api.com', 'hgtv', 'de')
+
+
+class DiscoveryPlusIE(DPlayIE):
+    _VALID_URL = r'https?://(?:www\.)?discoveryplus\.com/video' + DPlayIE._PATH_REGEX
+    _TESTS = [{
+        'url': 'https://www.discoveryplus.com/video/property-brothers-forever-home/food-and-family',
+        'info_dict': {
+            'id': '1140794',
+            'display_id': 'property-brothers-forever-home/food-and-family',
+            'ext': 'mp4',
+            'title': 'Food and Family',
+            'description': 'The brothers help a Richmond family expand their single-level home.',
+            'duration': 2583.113,
+            'timestamp': 1609304400,
+            'upload_date': '20201230',
+            'creator': 'HGTV',
+            'series': 'Property Brothers: Forever Home',
+            'season_number': 1,
+            'episode_number': 1,
+        },
+        'skip': 'Available for Premium users',
+    }]
+
+    _PRODUCT = 'dplus_us'
+    _API_URL = 'us1-prod-direct.discoveryplus.com'
+
+    def _update_disco_api_headers(self, headers, disco_base, display_id, realm):
+        headers['x-disco-client'] = f'WEB:UNKNOWN:{self._PRODUCT}:15.0.0'
+
+    def _download_video_playback_info(self, disco_base, video_id, headers):
+        return self._download_json(
+            disco_base + 'playback/v3/videoPlaybackInfo',
+            video_id, headers=headers, data=json.dumps({
+                'deviceInfo': {
+                    'adBlocker': False,
+                },
+                'videoId': video_id,
+                'wisteriaProperties': {
+                    'platform': 'desktop',
+                    'product': self._PRODUCT,
+                },
+            }).encode('utf-8'))['data']['attributes']['streaming']
+
+    def _real_extract(self, url):
+        display_id = self._match_id(url)
+        return self._get_disco_api_info(
+            url, display_id, self._API_URL, 'go', 'us')
+
+
+class ScienceChannelIE(DiscoveryPlusIE):
+    _VALID_URL = r'https?://(?:www\.)?sciencechannel\.com/video' + DPlayIE._PATH_REGEX
+    _TESTS = [{
+        'url': 'https://www.sciencechannel.com/video/strangest-things-science-atve-us/nazi-mystery-machine',
+        'info_dict': {
+            'id': '2842849',
+            'display_id': 'strangest-things-science-atve-us/nazi-mystery-machine',
+            'ext': 'mp4',
+            'title': 'Nazi Mystery Machine',
+            'description': 'Experts investigate the secrets of a revolutionary encryption machine.',
+            'season_number': 1,
+            'episode_number': 1,
+        },
+        'skip': 'Available for Premium users',
+    }]
+
+    _PRODUCT = 'sci'
+    _API_URL = 'us1-prod-direct.sciencechannel.com'
@@ -202,7 +202,10 @@ from .cbc import (
    CBCWatchIE,
    CBCOlympicsIE,
 )
-from .cbs import CBSIE
+from .cbs import (
+    CBSIE,
+    ParamountPlusSeriesIE,
+)
 from .cbslocal import (
    CBSLocalIE,
    CBSLocalArticleIE,
@@ -321,6 +324,7 @@ from .discoveryplusindia import (
    DiscoveryPlusIndiaShowIE,
 )
 from .dotsub import DotsubIE
+from .douyin import DouyinIE
 from .douyutv import (
    DouyuShowIE,
    DouyuTVIE,
@@ -329,6 +333,7 @@ from .dplay import (
    DPlayIE,
    DiscoveryPlusIE,
    HGTVDeIE,
+    ScienceChannelIE
 )
 from .dreisat import DreiSatIE
 from .drbonanza import DRBonanzaIE
@@ -1015,6 +1020,7 @@ from .popcorntimes import PopcorntimesIE
 from .popcorntv import PopcornTVIE
 from .porn91 import Porn91IE
 from .porncom import PornComIE
+from .pornflip import PornFlipIE
 from .pornhd import PornHdIE
 from .pornhub import (
    PornHubIE,
@@ -1336,7 +1342,6 @@ from .theweatherchannel import TheWeatherChannelIE
 from .thisamericanlife import ThisAmericanLifeIE
 from .thisav import ThisAVIE
 from .thisoldhouse import ThisOldHouseIE
-from .thisvid import ThisVidIE
 from .threeqsdn import ThreeQSDNIE
 from .tiktok import TikTokIE
 from .tinypic import TinyPicIE
@@ -1490,6 +1495,7 @@ from .ustudio import (
    UstudioIE,
    UstudioEmbedIE,
 )
+from .utreon import UtreonIE
 from .varzesh3 import Varzesh3IE
 from .vbox7 import Vbox7IE
 from .veehd import VeeHDIE
@@ -186,8 +186,8 @@ class FunimationIE(InfoExtractor):
        for lang, version, fmt in self._get_experiences(episode):
            experience_id = str(fmt['experienceId'])
            if (only_initial_experience and experience_id != initial_experience_id
-                    or requested_languages and lang not in requested_languages
-                    or requested_versions and version not in requested_versions):
+                    or requested_languages and lang.lower() not in requested_languages
+                    or requested_versions and version.lower() not in requested_versions):
                continue
            thumbnails.append({'url': fmt.get('poster')})
            duration = max(duration, fmt.get('duration', 0))
@@ -2238,6 +2238,87 @@ class GenericIE(InfoExtractor):
                'title': '#WEAREFMI – PT.2 – 2021 – MsMotorTV',
            },
            'playlist_count': 1,
+        }, {
+            # KVS Player
+            'url': 'https://www.kvs-demo.com/videos/105/kelis-4th-of-july/',
+            'info_dict': {
+                'id': '105',
+                'display_id': 'kelis-4th-of-july',
+                'ext': 'mp4',
+                'title': 'Kelis - 4th Of July',
+                'thumbnail': 'https://kvs-demo.com/contents/videos_screenshots/0/105/preview.jpg',
+            },
+            'params': {
+                'skip_download': True,
+            },
+        }, {
+            # KVS Player
+            'url': 'https://www.kvs-demo.com/embed/105/',
+            'info_dict': {
+                'id': '105',
+                'display_id': 'kelis-4th-of-july',
+                'ext': 'mp4',
+                'title': 'Kelis - 4th Of July / Embed Player',
+                'thumbnail': 'https://kvs-demo.com/contents/videos_screenshots/0/105/preview.jpg',
+            },
+            'params': {
+                'skip_download': True,
+            },
+        }, {
+            # KVS Player
+            'url': 'https://thisvid.com/videos/french-boy-pantsed/',
+            'md5': '3397979512c682f6b85b3b04989df224',
+            'info_dict': {
+                'id': '2400174',
+                'display_id': 'french-boy-pantsed',
+                'ext': 'mp4',
+                'title': 'French Boy Pantsed - ThisVid.com',
+                'thumbnail': 'https://media.thisvid.com/contents/videos_screenshots/2400000/2400174/preview.mp4.jpg',
+            }
+        }, {
+            # KVS Player
+            'url': 'https://thisvid.com/embed/2400174/',
+            'md5': '3397979512c682f6b85b3b04989df224',
+            'info_dict': {
+                'id': '2400174',
+                'display_id': 'french-boy-pantsed',
+                'ext': 'mp4',
+                'title': 'French Boy Pantsed - ThisVid.com',
+                'thumbnail': 'https://media.thisvid.com/contents/videos_screenshots/2400000/2400174/preview.mp4.jpg',
+            }
+        }, {
+            # KVS Player
+            'url': 'https://youix.com/video/leningrad-zoj/',
+            'md5': '94f96ba95706dc3880812b27b7d8a2b8',
+            'info_dict': {
+                'id': '18485',
+                'display_id': 'leningrad-zoj',
+                'ext': 'mp4',
+                'title': 'Клип: Ленинград - ЗОЖ скачать, смотреть онлайн | Youix.com',
+                'thumbnail': 'https://youix.com/contents/videos_screenshots/18000/18485/preview_480x320_youix_com.mp4.jpg',
+            }
+        }, {
+            # KVS Player
+            'url': 'https://youix.com/embed/18485',
+            'md5': '94f96ba95706dc3880812b27b7d8a2b8',
+            'info_dict': {
+                'id': '18485',
+                'display_id': 'leningrad-zoj',
+                'ext': 'mp4',
+                'title': 'Ленинград - ЗОЖ',
+                'thumbnail': 'https://youix.com/contents/videos_screenshots/18000/18485/preview_480x320_youix_com.mp4.jpg',
+            }
+        }, {
+            # KVS Player
+            'url': 'https://bogmedia.org/videos/21217/40-nochey-40-nights-2016/',
+            'md5': '94166bdb26b4cb1fb9214319a629fc51',
+            'info_dict': {
+                'id': '21217',
+                'display_id': '40-nochey-40-nights-2016',
+                'ext': 'mp4',
+                'title': '40 ночей (2016) - BogMedia.org',
+                'thumbnail': 'https://bogmedia.org/contents/videos_screenshots/21000/21217/preview_480p.mp4.jpg',
+            }
        },
    ]

@@ -2343,6 +2424,44 @@ class GenericIE(InfoExtractor):
            'title': title,
        }

+    def _kvs_getrealurl(self, video_url, license_code):
+        if not video_url.startswith('function/0/'):
+            return video_url  # not obfuscated
+
+        url_path, _, url_query = video_url.partition('?')
+        urlparts = url_path.split('/')[2:]
+        license = self._kvs_getlicensetoken(license_code)
+        newmagic = urlparts[5][:32]
+
+        for o in range(len(newmagic) - 1, -1, -1):
+            new = ''
+            l = (o + sum([int(n) for n in license[o:]])) % 32
+
+            for i in range(0, len(newmagic)):
+                if i == o:
+                    new += newmagic[l]
+                elif i == l:
+                    new += newmagic[o]
+                else:
+                    new += newmagic[i]
+            newmagic = new
+
+        urlparts[5] = newmagic + urlparts[5][32:]
+        return '/'.join(urlparts) + '?' + url_query
+
+    def _kvs_getlicensetoken(self, license):
+        modlicense = license.replace('$', '').replace('0', '1')
+        center = int(len(modlicense) / 2)
+        fronthalf = int(modlicense[:center + 1])
+        backhalf = int(modlicense[center:])
+
+        modlicense = str(4 * abs(fronthalf - backhalf))
+        retval = ''
+        for o in range(0, center + 1):
+            for i in range(1, 5):
+                retval += str((int(license[o + i]) + int(modlicense[o])) % 10)
+        return retval
+
    def _real_extract(self, url):
        if url.startswith('//'):
            return self.url_result(self.http_scheme() + url)
@@ -2462,7 +2581,7 @@ class GenericIE(InfoExtractor):

        # Is it an M3U playlist?
        if first_bytes.startswith(b'#EXTM3U'):
-            info_dict['formats'] = self._extract_m3u8_formats(url, video_id, 'mp4')
+            info_dict['formats'], info_dict['subtitles'] = self._extract_m3u8_formats_and_subtitles(url, video_id, 'mp4')
            self._sort_formats(info_dict['formats'])
            return info_dict

@@ -3410,6 +3529,7 @@ class GenericIE(InfoExtractor):
            if not isinstance(sources, list):
                sources = [sources]
            formats = []
+            subtitles = {}
            for source in sources:
                src = source.get('src')
                if not src or not isinstance(src, compat_str):
@@ -3422,12 +3542,16 @@ class GenericIE(InfoExtractor):
                if src_type == 'video/youtube':
                    return self.url_result(src, YoutubeIE.ie_key())
                if src_type == 'application/dash+xml' or ext == 'mpd':
-                    formats.extend(self._extract_mpd_formats(
-                        src, video_id, mpd_id='dash', fatal=False))
+                    fmts, subs = self._extract_mpd_formats_and_subtitles(
+                        src, video_id, mpd_id='dash', fatal=False)
+                    formats.extend(fmts)
+                    self._merge_subtitles(subs, target=subtitles)
                elif src_type == 'application/x-mpegurl' or ext == 'm3u8':
-                    formats.extend(self._extract_m3u8_formats(
+                    fmts, subs = self._extract_m3u8_formats_and_subtitles(
                        src, video_id, 'mp4', entry_protocol='m3u8_native',
-                        m3u8_id='hls', fatal=False))
+                        m3u8_id='hls', fatal=False)
+                    formats.extend(fmts)
+                    self._merge_subtitles(subs, target=subtitles)
                else:
                    formats.append({
                        'url': src,
@@ -3437,9 +3561,10 @@ class GenericIE(InfoExtractor):
                            'Referer': full_response.geturl(),
                        },
                    })
-            if formats:
+            if formats or subtitles:
                self._sort_formats(formats)
                info_dict['formats'] = formats
+                info_dict['subtitles'] = subtitles
                return info_dict

        # Looking for http://schema.org/VideoObject
@@ -3472,6 +3597,52 @@ class GenericIE(InfoExtractor):
                )
                .*?
                ['"]?file['"]?\s*:\s*["\'](.*?)["\']''', webpage))
+        if not found:
+            # Look for generic KVS player
+            found = re.search(r'<script [^>]*?src="https://.+?/kt_player\.js\?v=(?P<ver>(?P<maj_ver>\d+)(\.\d+)+)".*?>', webpage)
+            if found:
+                if found.group('maj_ver') not in ['4', '5']:
+                    self.report_warning('Untested major version (%s) in player engine--Download may fail.' % found.group('ver'))
+                flashvars = re.search(r'(?ms)<script.*?>.*?var\s+flashvars\s*=\s*(\{.*?\});.*?</script>', webpage)
+                flashvars = self._parse_json(flashvars.group(1), video_id, transform_source=js_to_json)
+
+                # extract the part after the last / as the display_id from the
+                # canonical URL.
+                display_id = self._search_regex(
+                    r'(?:<link href="https?://[^"]+/(.+?)/?" rel="canonical"\s*/?>'
+                    r'|<link rel="canonical" href="https?://[^"]+/(.+?)/?"\s*/?>)',
+                    webpage, 'display_id', fatal=False
+                )
+                title = self._html_search_regex(r'<(?:h1|title)>(?:Video: )?(.+?)</(?:h1|title)>', webpage, 'title')
+
+                thumbnail = flashvars['preview_url']
+                if thumbnail.startswith('//'):
+                    protocol, _, _ = url.partition('/')
+                    thumbnail = protocol + thumbnail
+
+                formats = []
+                for key in ('video_url', 'video_alt_url', 'video_alt_url2'):
+                    if key in flashvars and '/get_file/' in flashvars[key]:
+                        next_format = {
+                            'url': self._kvs_getrealurl(flashvars[key], flashvars['license_code']),
+                            'format_id': flashvars.get(key + '_text', key),
+                            'ext': 'mp4',
+                        }
+                        height = re.search(r'%s_(\d+)p\.mp4(?:/[?].*)?$' % flashvars['video_id'], flashvars[key])
+                        if height:
+                            next_format['height'] = int(height.group(1))
+                        else:
+                            next_format['quality'] = 1
+                        formats.append(next_format)
+                self._sort_formats(formats)
+
+                return {
+                    'id': flashvars['video_id'],
+                    'display_id': display_id,
+                    'title': title,
+                    'thumbnail': thumbnail,
+                    'formats': formats,
+                }
        if not found:
            # Broaden the search a little bit
            found = filter_video(re.findall(r'[^A-Za-z0-9]?(?:file|source)=(http[^\'"&]*)', webpage))
@@ -3574,13 +3745,13 @@ class GenericIE(InfoExtractor):

            ext = determine_ext(video_url)
            if ext == 'smil':
-                entry_info_dict['formats'] = self._extract_smil_formats(video_url, video_id)
+                entry_info_dict = {**self._extract_smil_info(video_url, video_id), **entry_info_dict}
            elif ext == 'xspf':
                return self.playlist_result(self._extract_xspf_playlist(video_url, video_id), video_id)
            elif ext == 'm3u8':
-                entry_info_dict['formats'] = self._extract_m3u8_formats(video_url, video_id, ext='mp4')
+                entry_info_dict['formats'], entry_info_dict['subtitles'] = self._extract_m3u8_formats_and_subtitles(video_url, video_id, ext='mp4')
            elif ext == 'mpd':
-                entry_info_dict['formats'] = self._extract_mpd_formats(video_url, video_id)
+                entry_info_dict['formats'], entry_info_dict['subtitles'] = self._extract_mpd_formats_and_subtitles(video_url, video_id)
            elif ext == 'f4m':
                entry_info_dict['formats'] = self._extract_f4m_formats(video_url, video_id)
            elif re.search(r'(?i)\.(?:ism|smil)/manifest', video_url) and video_url != url:
@@ -7,7 +7,6 @@ import re
 import time
 import uuid
 import json
-import random

 from .common import InfoExtractor
 from ..compat import (
@@ -27,34 +26,24 @@ from ..utils import (
 class HotStarBaseIE(InfoExtractor):
    _AKAMAI_ENCRYPTION_KEY = b'\x05\xfc\x1a\x01\xca\xc9\x4b\xc4\x12\xfc\x53\x12\x07\x75\xf9\xee'

-    def _call_api_impl(self, path, video_id, query, st=None):
+    def _call_api_impl(self, path, video_id, query, st=None, cookies=None):
        st = int_or_none(st) or int(time.time())
        exp = st + 6000
        auth = 'st=%d~exp=%d~acl=/*' % (st, exp)
        auth += '~hmac=' + hmac.new(self._AKAMAI_ENCRYPTION_KEY, auth.encode(), hashlib.sha256).hexdigest()

-        def _generate_device_id():
-            """
-            Reversed from javascript library.
-            JS function is generateUUID
-            """
-            t = int(round(time.time() * 1000))
-            e = "xxxxxxxx-xxxx-4xxx-xxxx-xxxxxxxxxxxx"  # 4 seems to be interchangeable
-
-            def _replacer():
-                n = int((t + 16 * random.random())) % 16 | 0
-                return hex(n if "x" == e else 3 & n | 8)[2:]
-            return "".join([_.replace('x', _replacer()) for _ in e])
-
-        token = self._download_json(
-            'https://api.hotstar.com/um/v3/users',
-            video_id, note='Downloading token',
-            data=json.dumps({"device_ids": [{"id": compat_str(uuid.uuid4()), "type": "device_id"}]}).encode('utf-8'),
-            headers={
-                'hotstarauth': auth,
-                'x-hs-platform': 'PCTV',  # or 'web'
-                'Content-Type': 'application/json',
-            })['user_identity']
+        if cookies.get('userUP'):
+            token = cookies.get('userUP').value
+        else:
+            token = self._download_json(
+                'https://api.hotstar.com/um/v3/users',
+                video_id, note='Downloading token',
+                data=json.dumps({"device_ids": [{"id": compat_str(uuid.uuid4()), "type": "device_id"}]}).encode('utf-8'),
+                headers={
+                    'hotstarauth': auth,
+                    'x-hs-platform': 'PCTV',  # or 'web'
+                    'Content-Type': 'application/json',
+                })['user_identity']

        response = self._download_json(
            'https://api.hotstar.com/' + path, video_id, headers={
@@ -75,11 +64,11 @@ class HotStarBaseIE(InfoExtractor):
            'tas': 10000,
        })

-    def _call_api_v2(self, path, video_id, st=None):
+    def _call_api_v2(self, path, video_id, st=None, cookies=None):
        return self._call_api_impl(
-            '%s/content/%s' % (path, video_id), video_id, st=st, query={
+            '%s/content/%s' % (path, video_id), video_id, st=st, cookies=cookies, query={
                'desired-config': 'audio_channel:stereo|dynamic_range:sdr|encryption:plain|ladder:tv|package:dash|resolution:hd|subs-tag:HotstarVIP|video_codec:vp9',
-                'device-id': compat_str(uuid.uuid4()),
+                'device-id': cookies.get('device_id').value if cookies.get('device_id') else compat_str(uuid.uuid4()),
                'os-name': 'Windows',
                'os-version': '10',
            })
@@ -130,7 +119,7 @@ class HotStarIE(HotStarBaseIE):

    def _real_extract(self, url):
        video_id = self._match_id(url)
-
+        cookies = self._get_cookies(url)
        webpage, urlh = self._download_webpage_handle(url, video_id)
        st = urlh.headers.get('x-origin-date')
        app_state = self._parse_json(self._search_regex(
@@ -156,7 +145,7 @@ class HotStarIE(HotStarBaseIE):
        formats = []
        geo_restricted = False
        # change to v2 in the future
-        playback_sets = self._call_api_v2('play/v1/playback', video_id, st=st)['playBackSets']
+        playback_sets = self._call_api_v2('play/v1/playback', video_id, st=st, cookies=cookies)['playBackSets']
        for playback_set in playback_sets:
            if not isinstance(playback_set, dict):
                continue
@@ -247,7 +236,7 @@ class HotStarPlaylistIE(HotStarBaseIE):

 class HotStarSeriesIE(HotStarBaseIE):
    IE_NAME = 'hotstar:series'
-    _VALID_URL = r'(?:https?://)(?:www\.)?hotstar\.com(?:/in)?/tv/[^/]+/(?P<id>\d{10})$'
+    _VALID_URL = r'(?:https?://)(?:www\.)?hotstar\.com(?:/in)?/tv/[^/]+/(?P<id>\d+)'
    _TESTS = [{
        'url': 'https://www.hotstar.com/in/tv/radhakrishn/1260000646',
        'info_dict': {
@@ -260,6 +249,12 @@ class HotStarSeriesIE(HotStarBaseIE):
            'id': '1260050431',
        },
        'playlist_mincount': 43,
+    }, {
+        'url': 'https://www.hotstar.com/in/tv/mahabharat/435/',
+        'info_dict': {
+            'id': '435',
+        },
+        'playlist_mincount': 269,
    }]

    def _real_extract(self, url):
@@ -19,6 +19,7 @@ from ..utils import (
    std_headers,
    try_get,
    url_or_none,
+    variadic,
 )


@@ -188,9 +189,7 @@ class InstagramIE(InfoExtractor):
            uploader_id = media.get('owner', {}).get('username')

            def get_count(keys, kind):
-                if not isinstance(keys, (list, tuple)):
-                    keys = [keys]
-                for key in keys:
+                for key in variadic(keys):
                    count = int_or_none(try_get(
                        media, (lambda x: x['edge_media_%s' % key]['count'],
                                lambda x: x['%ss' % kind]['count'])))
@@ -30,20 +30,20 @@ class MediasetIE(ThePlatformBaseIE):
                    '''
    _TESTS = [{
        # full episode
-        'url': 'https://www.mediasetplay.mediaset.it/video/hellogoodbye/quarta-puntata_FAFU000000661824',
-        'md5': '9b75534d42c44ecef7bf1ffeacb7f85d',
+        'url': 'https://www.mediasetplay.mediaset.it/video/mrwronglezionidamore/episodio-1_F310575103000102',
+        'md5': 'a7e75c6384871f322adb781d3bd72c26',
        'info_dict': {
-            'id': 'FAFU000000661824',
+            'id': 'F310575103000102',
            'ext': 'mp4',
-            'title': 'Quarta puntata',
+            'title': 'Episodio 1',
            'description': 'md5:d41d8cd98f00b204e9800998ecf8427e',
            'thumbnail': r're:^https?://.*\.jpg$',
-            'duration': 1414.26,
-            'upload_date': '20161107',
-            'series': 'Hello Goodbye',
-            'timestamp': 1478532900,
-            'uploader': 'Rete 4',
-            'uploader_id': 'R4',
+            'duration': 2682.0,
+            'upload_date': '20210530',
+            'series': 'Mr Wrong - Lezioni d\'amore',
+            'timestamp': 1622413946,
+            'uploader': 'Canale 5',
+            'uploader_id': 'C5',
        },
    }, {
        'url': 'https://www.mediasetplay.mediaset.it/video/matrix/puntata-del-25-maggio_F309013801000501',
@@ -54,10 +54,10 @@ class MediasetIE(ThePlatformBaseIE):
            'title': 'Puntata del 25 maggio',
            'description': 'md5:d41d8cd98f00b204e9800998ecf8427e',
            'thumbnail': r're:^https?://.*\.jpg$',
-            'duration': 6565.007,
-            'upload_date': '20180526',
+            'duration': 6565.008,
+            'upload_date': '20200903',
            'series': 'Matrix',
-            'timestamp': 1527326245,
+            'timestamp': 1599172492,
            'uploader': 'Canale 5',
            'uploader_id': 'C5',
        },
@@ -135,36 +135,38 @@ class MediasetIE(ThePlatformBaseIE):
        formats = []
        subtitles = {}
        first_e = None
-        for asset_type in ('SD', 'HD'):
-            # TODO: fixup ISM+none manifest URLs
-            for f in ('MPEG4', 'MPEG-DASH+none', 'M3U+none'):
-                try:
-                    tp_formats, tp_subtitles = self._extract_theplatform_smil(
-                        update_url_query('http://link.theplatform.%s/s/%s' % (self._TP_TLD, tp_path), {
-                            'mbr': 'true',
-                            'formats': f,
-                            'assetTypes': asset_type,
-                        }), guid, 'Downloading %s %s SMIL data' % (f.split('+')[0], asset_type))
-                except ExtractorError as e:
-                    if not first_e:
-                        first_e = e
-                    break
-                for tp_f in tp_formats:
-                    tp_f['quality'] = 1 if asset_type == 'HD' else 0
-                formats.extend(tp_formats)
-                subtitles = self._merge_subtitles(subtitles, tp_subtitles)
+        asset_type = 'HD,browser,geoIT|SD,browser,geoIT|geoNo:HD,browser,geoIT|geoNo:SD,browser,geoIT|geoNo'
+        # TODO: fixup ISM+none manifest URLs
+        for f in ('MPEG4', 'MPEG-DASH+none', 'M3U+none'):
+            try:
+                tp_formats, tp_subtitles = self._extract_theplatform_smil(
+                    update_url_query('http://link.theplatform.%s/s/%s' % (self._TP_TLD, tp_path), {
+                        'mbr': 'true',
+                        'formats': f,
+                        'assetTypes': asset_type,
+                    }), guid, 'Downloading %s SMIL data' % (f.split('+')[0]))
+            except ExtractorError as e:
+                if not first_e:
+                    first_e = e
+                break
+            formats.extend(tp_formats)
+            subtitles = self._merge_subtitles(subtitles, tp_subtitles)
        if first_e and not formats:
            raise first_e
        self._sort_formats(formats)

-        fields = []
-        for templ, repls in (('tvSeason%sNumber', ('', 'Episode')), ('mediasetprogram$%s', ('brandTitle', 'numberOfViews', 'publishInfo'))):
-            fields.extend(templ % repl for repl in repls)
        feed_data = self._download_json(
-            'https://feed.entertainment.tv.theplatform.eu/f/PR1GhC/mediaset-prod-all-programs/guid/-/' + guid,
-            guid, fatal=False, query={'fields': ','.join(fields)})
+            'https://feed.entertainment.tv.theplatform.eu/f/PR1GhC/mediaset-prod-all-programs-v2/guid/-/' + guid,
+            guid, fatal=False)
        if feed_data:
            publish_info = feed_data.get('mediasetprogram$publishInfo') or {}
+            thumbnails = feed_data.get('thumbnails') or {}
+            thumbnail = None
+            for key, value in thumbnails.items():
+                if key.startswith('image_keyframe_poster-'):
+                    thumbnail = value.get('url')
+                    break
+
            info.update({
                'episode_number': int_or_none(feed_data.get('tvSeasonEpisodeNumber')),
                'season_number': int_or_none(feed_data.get('tvSeasonNumber')),
@@ -172,6 +174,7 @@ class MediasetIE(ThePlatformBaseIE):
                'uploader': publish_info.get('description'),
                'uploader_id': publish_info.get('channel'),
                'view_count': int_or_none(feed_data.get('mediasetprogram$numberOfViews')),
+                'thumbnail': thumbnail,
            })

        info.update({
@@ -110,10 +110,15 @@ class MxplayerIE(InfoExtractor):
                for frmt in dash_formats:
                    frmt['quality'] = get_quality(quality)
                formats.extend(dash_formats)
+                dash_formats_h265 = self._extract_mpd_formats(
+                    format_url.replace('h264_high', 'h265_main'), video_id, mpd_id='dash-%s' % quality, headers={'Referer': url}, fatal=False)
+                for frmt in dash_formats_h265:
+                    frmt['quality'] = get_quality(quality)
+                formats.extend(dash_formats_h265)
            elif stream_type == 'hls':
                formats.extend(self._extract_m3u8_formats(
                    format_url, video_id, fatal=False,
-                    m3u8_id='hls-%s' % quality, quality=get_quality(quality)))
+                    m3u8_id='hls-%s' % quality, quality=get_quality(quality), ext='mp4'))

        self._sort_formats(formats)
        return {
@@ -2,9 +2,11 @@
 from __future__ import unicode_literals

 import json
+import time

+from urllib.error import HTTPError
 from .common import InfoExtractor
-from ..compat import compat_str
+from ..compat import compat_str, compat_urllib_parse_unquote, compat_urllib_parse_quote
 from ..utils import (
    ExtractorError,
    parse_iso8601,
@@ -78,7 +80,9 @@ class NebulaIE(InfoExtractor):
    ]
    _NETRC_MACHINE = 'watchnebula'

-    def _retrieve_nebula_auth(self, video_id):
+    _nebula_token = None
+
+    def _retrieve_nebula_auth(self):
        """
        Log in to Nebula, and returns a Nebula API token
        """
@@ -91,7 +95,7 @@ class NebulaIE(InfoExtractor):
        data = json.dumps({'email': username, 'password': password}).encode('utf8')
        response = self._download_json(
            'https://api.watchnebula.com/api/v1/auth/login/',
-            data=data, fatal=False, video_id=video_id,
+            data=data, fatal=False, video_id=None,
            headers={
                'content-type': 'application/json',
                # Submitting the 'sessionid' cookie always causes a 403 on auth endpoint
@@ -101,6 +105,19 @@ class NebulaIE(InfoExtractor):
            errnote='Authentication failed or rejected')
        if not response or not response.get('key'):
            self.raise_login_required()
+
+        # save nebula token as cookie
+        self._set_cookie(
+            'nebula.app', 'nebula-auth',
+            compat_urllib_parse_quote(
+                json.dumps({
+                    "apiToken": response["key"],
+                    "isLoggingIn": False,
+                    "isLoggingOut": False,
+                }, separators=(",", ":"))),
+            expire_time=int(time.time()) + 86400 * 365,
+        )
+
        return response['key']

    def _retrieve_zype_api_key(self, page_url, display_id):
@@ -139,8 +156,17 @@ class NebulaIE(InfoExtractor):
            'Authorization': 'Token {access_token}'.format(access_token=access_token)
        }, note=note)

-    def _fetch_zype_access_token(self, video_id, nebula_token):
-        user_object = self._call_nebula_api('/auth/user/', video_id, nebula_token, note='Retrieving Zype access token')
+    def _fetch_zype_access_token(self, video_id):
+        try:
+            user_object = self._call_nebula_api('/auth/user/', video_id, self._nebula_token, note='Retrieving Zype access token')
+        except ExtractorError as exc:
+            # if 401, attempt credential auth and retry
+            if exc.cause and isinstance(exc.cause, HTTPError) and exc.cause.code == 401:
+                self._nebula_token = self._retrieve_nebula_auth()
+                user_object = self._call_nebula_api('/auth/user/', video_id, self._nebula_token, note='Retrieving Zype access token')
+            else:
+                raise
+
        access_token = try_get(user_object, lambda x: x['zype_auth_info']['access_token'], compat_str)
        if not access_token:
            if try_get(user_object, lambda x: x['is_subscribed'], bool):
@@ -162,9 +188,21 @@ class NebulaIE(InfoExtractor):
            if category.get('value'):
                return category['value'][0]

+    def _real_initialize(self):
+        # check cookie jar for valid token
+        nebula_cookies = self._get_cookies('https://nebula.app')
+        nebula_cookie = nebula_cookies.get('nebula-auth')
+        if nebula_cookie:
+            self.to_screen('Authenticating to Nebula with token from cookie jar')
+            nebula_cookie_value = compat_urllib_parse_unquote(nebula_cookie.value)
+            self._nebula_token = self._parse_json(nebula_cookie_value, None).get('apiToken')
+
+        # try to authenticate using credentials if no valid token has been found
+        if not self._nebula_token:
+            self._nebula_token = self._retrieve_nebula_auth()
+
    def _real_extract(self, url):
        display_id = self._match_id(url)
-        nebula_token = self._retrieve_nebula_auth(display_id)
        api_key = self._retrieve_zype_api_key(url, display_id)

        response = self._call_zype_api('/videos', {'friendly_title': display_id},
@@ -174,7 +212,7 @@ class NebulaIE(InfoExtractor):
        video_meta = response['response'][0]

        video_id = video_meta['_id']
-        zype_access_token = self._fetch_zype_access_token(display_id, nebula_token=nebula_token)
+        zype_access_token = self._fetch_zype_access_token(display_id)

        channel_title = self._extract_channel_title(video_meta)

@@ -187,13 +225,12 @@ class NebulaIE(InfoExtractor):
            'title': video_meta.get('title'),
            'description': video_meta.get('description'),
            'timestamp': parse_iso8601(video_meta.get('published_at')),
-            'thumbnails': [
-                {
-                    'id': tn.get('name'),  # this appears to be null
-                    'url': tn['url'],
-                    'width': tn.get('width'),
-                    'height': tn.get('height'),
-                } for tn in video_meta.get('thumbnails', [])],
+            'thumbnails': [{
+                'id': tn.get('name'),  # this appears to be null
+                'url': tn['url'],
+                'width': tn.get('width'),
+                'height': tn.get('height'),
+            } for tn in video_meta.get('thumbnails', [])],
            'duration': video_meta.get('duration'),
            'channel': channel_title,
            'uploader': channel_title,  # we chose uploader = channel name
@@ -0,0 +1,82 @@
+# coding: utf-8
+from __future__ import unicode_literals
+
+from .common import InfoExtractor
+from ..utils import (
+    int_or_none,
+    parse_duration,
+    parse_iso8601
+)
+
+
+class PornFlipIE(InfoExtractor):
+    _VALID_URL = r'https?://(?:www\.)?pornflip\.com/(?:(embed|sv|v)/)?(?P<id>[^/]+)'
+    _TESTS = [
+        {
+            'url': 'https://www.pornflip.com/dzv9Mtw1qj2/sv/brazzers-double-dare-two-couples-fucked-jenna-reid-maya-bijou',
+            'info_dict': {
+                'id': 'dzv9Mtw1qj2',
+                'ext': 'mp4',
+                'title': 'Brazzers - Double Dare Two couples fucked Jenna Reid Maya Bijou',
+                'description': 'md5:d2b69e6cc743c5fd158e162aa7f05821',
+                'duration': 476,
+                'like_count': int,
+                'dislike_count': int,
+                'view_count': int,
+                'timestamp': 1617846819,
+                'upload_date': '20210408',
+                'uploader': 'Brazzers',
+                'age_limit': 18,
+            },
+            'params': {
+                'format': 'bestvideo',
+                'skip_download': True,
+            },
+        },
+        {
+            'url': 'https://www.pornflip.com/v/IrJEC40i21L',
+            'only_matching': True,
+        },
+        {
+            'url': 'https://www.pornflip.com/Z3jzbChC5-P/sexintaxi-e-sereyna-gomez-czech-naked-couple',
+            'only_matching': True,
+        },
+        {
+            'url': 'https://www.pornflip.com/embed/bLcDFxnrZnU',
+            'only_matching': True,
+        },
+    ]
+    _HOST = 'www.pornflip.com'
+
+    def _real_extract(self, url):
+        video_id = self._match_id(url)
+        webpage = self._download_webpage(
+            'https://{}/sv/{}'.format(self._HOST, video_id), video_id, headers={'host': self._HOST})
+        description = self._html_search_regex(r'&p\[summary\]=(.*?)\s*&p', webpage, 'description', fatal=False)
+        duration = self._search_regex(r'"duration":\s+"([^"]+)",', webpage, 'duration', fatal=False)
+        view_count = self._search_regex(r'"interactionCount":\s+"([^"]+)"', webpage, 'view_count', fatal=False)
+        title = self._html_search_regex(r'id="mediaPlayerTitleLink"[^>]*>(.+)</a>', webpage, 'title', fatal=False)
+        uploader = self._html_search_regex(r'class="title-chanel"[^>]*>[^<]*<a[^>]*>([^<]+)<', webpage, 'uploader', fatal=False)
+        upload_date = self._search_regex(r'"uploadDate":\s+"([^"]+)",', webpage, 'upload_date', fatal=False)
+        likes = self._html_search_regex(
+            r'class="btn btn-up-rating[^>]*>[^<]*<i[^>]*>[^<]*</i>[^>]*<span[^>]*>[^0-9]*([0-9]+)[^<0-9]*<', webpage, 'like_count', fatal=False)
+        dislikes = self._html_search_regex(
+            r'class="btn btn-down-rating[^>]*>[^<]*<i[^>]*>[^<]*</i>[^>]*<span[^>]*>[^0-9]*([0-9]+)[^<0-9]*<', webpage, 'dislike_count', fatal=False)
+        mpd_url = self._search_regex(r'"([^"]+userscontent.net/dash/[0-9]+/manifest.mpd[^"]*)"', webpage, 'mpd_url').replace('&amp;', '&')
+        formats = self._extract_mpd_formats(mpd_url, video_id, mpd_id='dash')
+        self._sort_formats(formats)
+
+        return {
+            'age_limit': 18,
+            'description': description,
+            'dislike_count': int_or_none(dislikes),
+            'duration': parse_duration(duration),
+            'formats': formats,
+            'id': video_id,
+            'like_count': int_or_none(likes),
+            'timestamp': parse_iso8601(upload_date),
+            'thumbnail': self._og_search_thumbnail(webpage),
+            'title': title,
+            'uploader': uploader,
+            'view_count': int_or_none(view_count),
+        }
@@ -2,9 +2,9 @@
 from __future__ import unicode_literals

 import itertools
+import json
 import re
-
-from .openload import PhantomJSwrapper
+import time

 from .common import InfoExtractor
 from ..utils import (
@@ -88,6 +88,16 @@ class RCTIPlusIE(RCTIPlusBaseIE):
            'format': 'bestvideo',
        },
    }]
+    _CONVIVA_JSON_TEMPLATE = {
+        't': 'CwsSessionHb',
+        'cid': 'ff84ae928c3b33064b76dec08f12500465e59a6f',
+        'clid': '0',
+        'sid': 0,
+        'seq': 0,
+        'caps': 0,
+        'sf': 7,
+        'sdk': True,
+    }

    def _search_auth_key(self, webpage):
        try:
@@ -105,15 +115,18 @@ class RCTIPlusIE(RCTIPlusBaseIE):
            'https://api.rctiplus.com/api/v1/%s/%s/url?appierid=.1' % (video_type, video_id), display_id, 'Downloading video URL JSON')[0]
        video_url = video_json['url']
        if 'akamaized' in video_url:
-            # Akamai's CDN requires a session to at least be made via Conviva's API
-            # TODO: Reverse-engineer Conviva's heartbeat code to avoid phantomJS
-            phantom = None
-            try:
-                phantom = PhantomJSwrapper(self)
-                phantom.get(url, webpage, display_id, note2='Initiating video session')
-            except ExtractorError:
-                self.report_warning('PhantomJS is highly recommended for this video, as it might load incredibly slowly otherwise.'
-                                    'You can also try opening the page in this device\'s browser first')
+            # For some videos hosted on Akamai's CDN (possibly AES-encrypted ones?), a session needs to at least be made via Conviva's API
+            conviva_json_data = {
+                **self._CONVIVA_JSON_TEMPLATE,
+                'url': video_url,
+                'sst': int(time.time())
+            }
+            conviva_json_res = self._download_json(
+                'https://ff84ae928c3b33064b76dec08f12500465e59a6f.cws.conviva.com/0/wsg', display_id,
+                'Creating Conviva session', 'Failed to create Conviva session',
+                fatal=False, data=json.dumps(conviva_json_data).encode('utf-8'))
+            if conviva_json_res and conviva_json_res.get('err') != 'ok':
+                self.report_warning('Conviva said: %s' % str(conviva_json_res.get('err')))

        video_meta, meta_paths = self._call_api(
            'https://api.rctiplus.com/api/v1/%s/%s' % (video_type, video_id), display_id, 'Downloading video metadata')
@@ -2,10 +2,11 @@
 from __future__ import unicode_literals

 from .common import InfoExtractor
-from ..utils import (
-    determine_ext,
-    js_to_json,
-)
+from ..utils import js_to_json
+import re
+import json
+import urllib.parse
+import base64


 class RTPIE(InfoExtractor):
@@ -25,6 +26,22 @@ class RTPIE(InfoExtractor):
        'only_matching': True,
    }]

+    _RX_OBFUSCATION = re.compile(r'''(?xs)
+        atob\s*\(\s*decodeURIComponent\s*\(\s*
+            (\[[0-9A-Za-z%,'"]*\])
+        \s*\.\s*join\(\s*(?:""|'')\s*\)\s*\)\s*\)
+    ''')
+
+    def __unobfuscate(self, data, *, video_id):
+        if data.startswith('{'):
+            data = self._RX_OBFUSCATION.sub(
+                lambda m: json.dumps(
+                    base64.b64decode(urllib.parse.unquote(
+                        ''.join(self._parse_json(m.group(1), video_id))
+                    )).decode('iso-8859-1')),
+                data)
+        return js_to_json(data)
+
    def _real_extract(self, url):
        video_id = self._match_id(url)

@@ -32,30 +49,46 @@ class RTPIE(InfoExtractor):
        title = self._html_search_meta(
            'twitter:title', webpage, display_name='title', fatal=True)

-        config = self._parse_json(self._search_regex(
-            r'(?s)RTPPlayer\(({.+?})\);', webpage,
-            'player config'), video_id, js_to_json)
-        file_url = config['file']
-        ext = determine_ext(file_url)
-        if ext == 'm3u8':
-            file_key = config.get('fileKey')
-            formats = self._extract_m3u8_formats(
-                file_url, video_id, 'mp4', 'm3u8_native',
-                m3u8_id='hls', fatal=file_key)
-            if file_key:
-                formats.append({
-                    'url': 'https://cdn-ondemand.rtp.pt' + file_key,
-                    'quality': 1,
-                })
-            self._sort_formats(formats)
+        f, config = self._search_regex(
+            r'''(?sx)
+                var\s+f\s*=\s*(?P<f>".*?"|{[^;]+?});\s*
+                var\s+player1\s+=\s+new\s+RTPPlayer\s*\((?P<config>{(?:(?!\*/).)+?})\);(?!\s*\*/)
+            ''', webpage,
+            'player config', group=('f', 'config'))
+
+        f = self._parse_json(
+            f, video_id,
+            lambda data: self.__unobfuscate(data, video_id=video_id))
+        config = self._parse_json(
+            config, video_id,
+            lambda data: self.__unobfuscate(data, video_id=video_id))
+
+        formats = []
+        if isinstance(f, dict):
+            f_hls = f.get('hls')
+            if f_hls is not None:
+                formats.extend(self._extract_m3u8_formats(
+                    f_hls, video_id, 'mp4', 'm3u8_native', m3u8_id='hls'))
+
+            f_dash = f.get('dash')
+            if f_dash is not None:
+                formats.extend(self._extract_mpd_formats(f_dash, video_id, mpd_id='dash'))
        else:
-            formats = [{
-                'url': file_url,
-                'ext': ext,
-            }]
-        if config.get('mediaType') == 'audio':
-            for f in formats:
-                f['vcodec'] = 'none'
+            formats.append({
+                'format_id': 'f',
+                'url': f,
+                'vcodec': 'none' if config.get('mediaType') == 'audio' else None,
+            })
+
+        subtitles = {}
+
+        vtt = config.get('vtt')
+        if vtt is not None:
+            for lcode, lname, url in vtt:
+                subtitles.setdefault(lcode, []).append({
+                    'name': lname,
+                    'url': url,
+                })

        return {
            'id': video_id,
@@ -63,4 +96,5 @@ class RTPIE(InfoExtractor):
            'formats': formats,
            'description': self._html_search_meta(['description', 'twitter:description'], webpage),
            'thumbnail': config.get('poster') or self._og_search_thumbnail(webpage),
+            'subtitles': subtitles,
        }
@@ -41,6 +41,7 @@ class TenPlayIE(InfoExtractor):
        'PG': 15,
        'M': 15,
        'MA': 15,
+        'MA15+': 15,
        'R': 18,
        'X': 18
    }
@@ -79,7 +80,7 @@ class TenPlayIE(InfoExtractor):
            'id': data.get('altId') or content_id,
            'title': data.get('title'),
            'description': data.get('description'),
-            'age_limit': self._AUS_AGES[data.get('classification')],
+            'age_limit': self._AUS_AGES.get(data.get('classification')),
            'series': data.get('showName'),
            'season': data.get('showContentSeason'),
            'timestamp': data.get('published'),
@@ -1,97 +0,0 @@
-# coding: utf-8
-from __future__ import unicode_literals
-import re
-
-from .common import InfoExtractor
-
-
-class ThisVidIE(InfoExtractor):
-    _VALID_URL = r'https?://(?:www\.)?thisvid\.com/(?P<type>videos|embed)/(?P<id>[A-Za-z0-9-]+/?)'
-    _TESTS = [{
-        'url': 'https://thisvid.com/videos/french-boy-pantsed/',
-        'md5': '3397979512c682f6b85b3b04989df224',
-        'info_dict': {
-            'id': '2400174',
-            'ext': 'mp4',
-            'title': 'French Boy Pantsed',
-            'thumbnail': 'https://media.thisvid.com/contents/videos_screenshots/2400000/2400174/preview.mp4.jpg',
-            'age_limit': 18,
-        }
-    }, {
-        'url': 'https://thisvid.com/embed/2400174/',
-        'md5': '3397979512c682f6b85b3b04989df224',
-        'info_dict': {
-            'id': '2400174',
-            'ext': 'mp4',
-            'title': 'French Boy Pantsed',
-            'thumbnail': 'https://media.thisvid.com/contents/videos_screenshots/2400000/2400174/preview.mp4.jpg',
-            'age_limit': 18,
-        }
-    }]
-
-    def _real_extract(self, url):
-        main_id = self._match_id(url)
-        webpage = self._download_webpage(url, main_id)
-
-        # URL decryptor was reversed from version 4.0.4, later verified working with 5.2.0 and may change in the future.
-        kvs_version = self._html_search_regex(r'<script [^>]+?src="https://thisvid\.com/player/kt_player\.js\?v=(\d+(\.\d+)+)">', webpage, 'kvs_version', fatal=False)
-        if not kvs_version.startswith("5."):
-            self.report_warning("Major version change (" + kvs_version + ") in player engine--Download may fail.")
-
-        title = self._html_search_regex(r'<title>(?:Video: )?(.+?)(?: - (?:\w+ porn at )?ThisVid(?:.com| tube))?</title>', webpage, 'title')
-        # video_id, video_url and license_code from the 'flashvars' JSON object:
-        video_id = self._html_search_regex(r"video_id: '([0-9]+)',", webpage, 'video_id')
-        video_url = self._html_search_regex(r"video_url: '(function/0/.+?)',", webpage, 'video_url')
-        license_code = self._html_search_regex(r"license_code: '([0-9$]{16})',", webpage, 'license_code')
-        thumbnail = self._html_search_regex(r"preview_url: '((?:https?:)?//media.thisvid.com/.+?.jpg)',", webpage, 'thumbnail', fatal=False)
-        if thumbnail.startswith("//"):
-            thumbnail = "https:" + thumbnail
-        if (re.match(self._VALID_URL, url).group('type') == "videos"):
-            display_id = main_id
-        else:
-            display_id = self._search_regex(r'<link rel="canonical" href="' + self._VALID_URL + r'">', webpage, 'display_id', fatal=False),
-
-        return {
-            'id': video_id,
-            'display_id': display_id,
-            'title': title,
-            'url': getrealurl(video_url, license_code),
-            'thumbnail': thumbnail,
-            'age_limit': 18,
-        }
-
-
-def getrealurl(video_url, license_code):
-    urlparts = video_url.split('/')[2:]
-    license = getlicensetoken(license_code)
-    newmagic = urlparts[5][:32]
-
-    for o in range(len(newmagic) - 1, -1, -1):
-        new = ""
-        l = (o + sum([int(n) for n in license[o:]])) % 32
-
-        for i in range(0, len(newmagic)):
-            if i == o:
-                new += newmagic[l]
-            elif i == l:
-                new += newmagic[o]
-            else:
-                new += newmagic[i]
-        newmagic = new
-
-    urlparts[5] = newmagic + urlparts[5][32:]
-    return "/".join(urlparts)
-
-
-def getlicensetoken(license):
-    modlicense = license.replace("$", "").replace("0", "1")
-    center = int(len(modlicense) / 2)
-    fronthalf = int(modlicense[:center + 1])
-    backhalf = int(modlicense[center:])
-
-    modlicense = str(4 * abs(fronthalf - backhalf))
-    retval = ""
-    for o in range(0, center + 1):
-        for i in range(1, 5):
-            retval += str((int(license[o + i]) + int(modlicense[o])) % 10)
-    return retval
@@ -0,0 +1,85 @@
+# coding: utf-8
+from __future__ import unicode_literals
+
+from .common import InfoExtractor
+from ..utils import (
+    dict_get,
+    int_or_none,
+    str_or_none,
+    try_get,
+    unified_strdate,
+    url_or_none,
+)
+
+
+class UtreonIE(InfoExtractor):
+    _VALID_URL = r'(?:https?://)(?:www\.)?utreon.com/v/(?P<id>[a-zA-Z0-9_-]+)'
+    _TESTS = [{
+        'url': 'https://utreon.com/v/z_I7ikQbuDw',
+        'info_dict': {
+            'id': 'z_I7ikQbuDw',
+            'ext': 'mp4',
+            'title': 'Freedom Friday meditation - Rising in the wind',
+            'description': 'md5:a9bf15a42434a062fe313b938343ad1b',
+            'uploader': 'Heather Dawn Elemental Health',
+            'thumbnail': 'https://data-1.utreon.com/v/MG/M2/NT/z_I7ikQbuDw/z_I7ikQbuDw_preview.jpg',
+            'release_date': '20210723',
+        }
+    }, {
+        'url': 'https://utreon.com/v/jerJw5EOOVU',
+        'info_dict': {
+            'id': 'jerJw5EOOVU',
+            'ext': 'mp4',
+            'title': 'When I\'m alone, I love to reflect in peace, to make my dreams come true... [Quotes and Poems]',
+            'description': 'md5:61ee6c2da98be51b04b969ca80273aaa',
+            'uploader': 'Frases e Poemas Quotes and Poems',
+            'thumbnail': 'https://data-1.utreon.com/v/Mz/Zh/ND/jerJw5EOOVU/jerJw5EOOVU_89af85470a4b16eededde7f8674c96d9_cover.jpg',
+            'release_date': '20210723',
+        }
+    }, {
+        'url': 'https://utreon.com/v/C4ZxXhYBBmE',
+        'info_dict': {
+            'id': 'C4ZxXhYBBmE',
+            'ext': 'mp4',
+            'title': 'Biden’s Capital Gains Tax Rate to Test World’s Highest',
+            'description': 'md5:fb5a6c2e506f013cc76f133f673bc5c8',
+            'uploader': 'Nomad Capitalist',
+            'thumbnail': 'https://data-1.utreon.com/v/ZD/k1/Mj/C4ZxXhYBBmE/C4ZxXhYBBmE_628342076198c9c06dd6b2c665978584_cover.jpg',
+            'release_date': '20210723',
+        }
+    }, {
+        'url': 'https://utreon.com/v/Y-stEH-FBm8',
+        'info_dict': {
+            'id': 'Y-stEH-FBm8',
+            'ext': 'mp4',
+            'title': 'Creeper-Chan Pranks Steve! 💚 [MINECRAFT ANIME]',
+            'description': 'md5:7a48450b0d761b96dec194be0c5ecb5f',
+            'uploader': 'Merryweather Comics',
+            'thumbnail': 'https://data-1.utreon.com/v/MT/E4/Zj/Y-stEH-FBm8/Y-stEH-FBm8_5290676a41a4a1096db133b09f54f77b_cover.jpg',
+            'release_date': '20210718',
+        }},
+    ]
+
+    def _real_extract(self, url):
+        video_id = self._match_id(url)
+        json_data = self._download_json(
+            'https://api.utreon.com/v1/videos/' + video_id,
+            video_id)
+        videos_json = json_data['videos']
+        formats = [{
+            'url': format_url,
+            'format_id': format_key.split('_')[1],
+            'height': int(format_key.split('_')[1][:-1]),
+        } for format_key, format_url in videos_json.items() if url_or_none(format_url)]
+        self._sort_formats(formats)
+        thumbnail = url_or_none(dict_get(json_data, ('cover_image_url', 'preview_image_url')))
+        return {
+            'id': video_id,
+            'title': json_data['title'],
+            'formats': formats,
+            'description': str_or_none(json_data.get('description')),
+            'duration': int_or_none(json_data.get('duration')),
+            'uploader': str_or_none(try_get(json_data, lambda x: x['channel']['title'])),
+            'thumbnail': thumbnail,
+            'release_date': unified_strdate(json_data.get('published_datetime')),
+        }
@@ -5,6 +5,7 @@ import re

 from .common import InfoExtractor
 from ..utils import (
+    clean_html,
    ExtractorError,
    get_element_by_class,
    int_or_none,
@@ -47,10 +48,19 @@ class VidioBaseIE(InfoExtractor):
            self._LOGIN_URL, None, 'Logging in', data=urlencode_postdata(login_form), expected_status=[302, 401])

        if login_post_urlh.status == 401:
-            reason = get_element_by_class('onboarding-form__general-error', login_post)
-            if reason:
+            if get_element_by_class('onboarding-content-register-popup__title', login_post):
                raise ExtractorError(
-                    'Unable to log in: %s' % reason, expected=True)
+                    'Unable to log in: The provided email has not registered yet.', expected=True)
+
+            reason = get_element_by_class('onboarding-form__general-error', login_post) or get_element_by_class('onboarding-modal__title', login_post)
+            if 'Akun terhubung ke' in reason:
+                raise ExtractorError(
+                    'Unable to log in: Your account is linked to a social media account. '
+                    'Use --cookies to provide account credentials instead', expected=True)
+            elif reason:
+                subreason = get_element_by_class('onboarding-modal__description-text', login_post) or ''
+                raise ExtractorError(
+                    'Unable to log in: %s. %s' % (reason, clean_html(subreason)), expected=True)
            raise ExtractorError('Unable to log in')

    def _real_initialize(self):
@@ -1,39 +1,28 @@
 # coding: utf-8
 from __future__ import unicode_literals
-
-import base64
 import hashlib
 import hmac
-import itertools
 import json
-import re
 import time

 from .common import InfoExtractor
-from ..compat import (
-    compat_parse_qs,
-    compat_urllib_parse_urlparse,
-)
 from ..utils import (
    ExtractorError,
    int_or_none,
-    HEADRequest,
    parse_age_limit,
    parse_iso8601,
-    sanitized_Request,
-    std_headers,
    try_get,
 )


 class VikiBaseIE(InfoExtractor):
    _VALID_URL_BASE = r'https?://(?:www\.)?viki\.(?:com|net|mx|jp|fr)/'
-    _API_QUERY_TEMPLATE = '/v4/%sapp=%s&t=%s&site=www.viki.com'
-    _API_URL_TEMPLATE = 'https://api.viki.io%s&sig=%s'
+    _API_URL_TEMPLATE = 'https://api.viki.io%s'

+    _DEVICE_ID = '86085977d'  # used for android api
    _APP = '100005a'
-    _APP_VERSION = '6.0.0'
-    _APP_SECRET = 'MM_d*yP@`&1@]@!AVrXf_o-HVEnoTnm$O-ti4[G~$JDI/Dc-&piU&z&5.;:}95=Iad'
+    _APP_VERSION = '6.11.3'
+    _APP_SECRET = 'd96704b180208dbb2efa30fe44c48bd8690441af9f567ba8fd710a72badc85198f7472'

    _GEO_BYPASS = False
    _NETRC_MACHINE = 'viki'
@@ -46,53 +35,57 @@ class VikiBaseIE(InfoExtractor):
        'paywall': 'Sorry, this content is only available to Viki Pass Plus subscribers',
    }

-    def _prepare_call(self, path, timestamp=None, post_data=None):
+    def _stream_headers(self, timestamp, sig):
+        return {
+            'X-Viki-manufacturer': 'vivo',
+            'X-Viki-device-model': 'vivo 1606',
+            'X-Viki-device-os-ver': '6.0.1',
+            'X-Viki-connection-type': 'WIFI',
+            'X-Viki-carrier': '',
+            'X-Viki-as-id': '100005a-1625321982-3932',
+            'timestamp': str(timestamp),
+            'signature': str(sig),
+            'x-viki-app-ver': self._APP_VERSION
+        }
+
+    def _api_query(self, path, version=4, **kwargs):
        path += '?' if '?' not in path else '&'
-        if not timestamp:
-            timestamp = int(time.time())
-        query = self._API_QUERY_TEMPLATE % (path, self._APP, timestamp)
+        query = f'/v{version}/{path}app={self._APP}'
        if self._token:
            query += '&token=%s' % self._token
+        return query + ''.join(f'&{name}={val}' for name, val in kwargs.items())
+
+    def _sign_query(self, path):
+        timestamp = int(time.time())
+        query = self._api_query(path, version=5)
        sig = hmac.new(
-            self._APP_SECRET.encode('ascii'),
-            query.encode('ascii'),
-            hashlib.sha1
-        ).hexdigest()
-        url = self._API_URL_TEMPLATE % (query, sig)
-        return sanitized_Request(
-            url, json.dumps(post_data).encode('utf-8')) if post_data else url
+            self._APP_SECRET.encode('ascii'), f'{query}&t={timestamp}'.encode('ascii'), hashlib.sha1).hexdigest()
+        return timestamp, sig, self._API_URL_TEMPLATE % query

-    def _call_api(self, path, video_id, note, timestamp=None, post_data=None):
+    def _call_api(
+            self, path, video_id, note='Downloading JSON metadata', data=None, query=None, fatal=True):
+        if query is None:
+            timestamp, sig, url = self._sign_query(path)
+        else:
+            url = self._API_URL_TEMPLATE % self._api_query(path, version=4)
        resp = self._download_json(
-            self._prepare_call(path, timestamp, post_data),
-            video_id, note,
-            headers={
-                'x-client-user-agent': std_headers['User-Agent'],
-                'x-viki-as-id': self._APP,
-                'x-viki-app-ver': self._APP_VERSION,
-            })
-
-        error = resp.get('error')
-        if error:
-            if error == 'invalid timestamp':
-                resp = self._download_json(
-                    self._prepare_call(path, int(resp['current_timestamp']), post_data),
-                    video_id, '%s (retry)' % note,
-                    headers={
-                        'x-client-user-agent': std_headers['User-Agent'],
-                        'x-viki-as-id': self._APP,
-                        'x-viki-app-ver': self._APP_VERSION,
-                    })
-                error = resp.get('error')
-            if error:
-                self._raise_error(resp['error'])
+            url, video_id, note, fatal=fatal, query=query,
+            data=json.dumps(data).encode('utf-8') if data else None,
+            headers=({'x-viki-app-ver': self._APP_VERSION} if data
+                     else self._stream_headers(timestamp, sig) if query is None
+                     else None)) or {}

+        self._raise_error(resp.get('error'), fatal)
        return resp

-    def _raise_error(self, error):
-        raise ExtractorError(
-            '%s returned error: %s' % (self.IE_NAME, error),
-            expected=True)
+    def _raise_error(self, error, fatal=True):
+        if error is None:
+            return
+        msg = '%s said: %s' % (self.IE_NAME, error)
+        if fatal:
+            raise ExtractorError(msg, expected=True)
+        else:
+            self.report_warning(msg)

    def _check_errors(self, data):
        for reason, status in (data.get('blocking') or {}).items():
@@ -101,9 +94,10 @@ class VikiBaseIE(InfoExtractor):
                if reason == 'geo':
                    self.raise_geo_restricted(msg=message)
                elif reason == 'paywall':
+                    if try_get(data, lambda x: x['paywallable']['tvod']):
+                        self._raise_error('This video is for rent only or TVOD (Transactional Video On demand)')
                    self.raise_login_required(message)
-                raise ExtractorError('%s said: %s' % (
-                    self.IE_NAME, message), expected=True)
+                self._raise_error(message)

    def _real_initialize(self):
        self._login()
@@ -113,29 +107,17 @@ class VikiBaseIE(InfoExtractor):
        if username is None:
            return

-        login_form = {
-            'login_id': username,
-            'password': password,
-        }
-
-        login = self._call_api(
-            'sessions.json', None,
-            'Logging in', post_data=login_form)
-
-        self._token = login.get('token')
+        self._token = self._call_api(
+            'sessions.json', None, 'Logging in', fatal=False,
+            data={'username': username, 'password': password}).get('token')
        if not self._token:
-            self.report_warning('Unable to get session token, login has probably failed')
+            self.report_warning('Login Failed: Unable to get session token')

    @staticmethod
-    def dict_selection(dict_obj, preferred_key, allow_fallback=True):
+    def dict_selection(dict_obj, preferred_key):
        if preferred_key in dict_obj:
-            return dict_obj.get(preferred_key)
-
-        if not allow_fallback:
-            return
-
-        filtered_dict = list(filter(None, [dict_obj.get(k) for k in dict_obj.keys()]))
-        return filtered_dict[0] if filtered_dict else None
+            return dict_obj[preferred_key]
+        return (list(filter(None, dict_obj.values())) or [None])[0]


 class VikiIE(VikiBaseIE):
@@ -266,18 +248,10 @@ class VikiIE(VikiBaseIE):

    def _real_extract(self, url):
        video_id = self._match_id(url)
-
-        resp = self._download_json(
-            'https://www.viki.com/api/videos/' + video_id,
-            video_id, 'Downloading video JSON', headers={
-                'x-client-user-agent': std_headers['User-Agent'],
-                'x-viki-app-ver': '3.0.0',
-            })
-        video = resp['video']
-
+        video = self._call_api(f'videos/{video_id}.json', video_id, 'Downloading video JSON', query={})
        self._check_errors(video)

-        title = self.dict_selection(video.get('titles', {}), 'en', allow_fallback=False)
+        title = try_get(video, lambda x: x['titles']['en'], str)
        episode_number = int_or_none(video.get('number'))
        if not title:
            title = 'Episode %d' % episode_number if video.get('type') == 'episode' else video.get('id') or video_id
@@ -285,116 +259,46 @@ class VikiIE(VikiBaseIE):
            container_title = self.dict_selection(container_titles, 'en')
            title = '%s - %s' % (container_title, title)

-        description = self.dict_selection(video.get('descriptions', {}), 'en')
+        thumbnails = [{
+            'id': thumbnail_id,
+            'url': thumbnail['url'],
+        } for thumbnail_id, thumbnail in (video.get('images') or {}).items() if thumbnail.get('url')]

-        like_count = int_or_none(try_get(video, lambda x: x['likes']['count']))
+        resp = self._call_api(
+            'playback_streams/%s.json?drms=dt1,dt2&device_id=%s' % (video_id, self._DEVICE_ID),
+            video_id, 'Downloading video streams JSON')['main'][0]

-        thumbnails = []
-        for thumbnail_id, thumbnail in (video.get('images') or {}).items():
-            thumbnails.append({
-                'id': thumbnail_id,
-                'url': thumbnail.get('url'),
-            })
+        stream_id = try_get(resp, lambda x: x['properties']['track']['stream_id'])
+        subtitles = dict((lang, [{
+            'ext': ext,
+            'url': self._API_URL_TEMPLATE % self._api_query(
+                f'videos/{video_id}/auth_subtitles/{lang}.{ext}', stream_id=stream_id)
+        } for ext in ('srt', 'vtt')]) for lang in (video.get('subtitle_completions') or {}).keys())

-        subtitles = {}
-        for subtitle_lang, _ in (video.get('subtitle_completions') or {}).items():
-            subtitles[subtitle_lang] = [{
-                'ext': subtitles_format,
-                'url': self._prepare_call(
-                    'videos/%s/subtitles/%s.%s' % (video_id, subtitle_lang, subtitles_format)),
-            } for subtitles_format in ('srt', 'vtt')]
+        mpd_url = resp['url']
+        # 1080p is hidden in another mpd which can be found in the current manifest content
+        mpd_content = self._download_webpage(mpd_url, video_id, note='Downloading initial MPD manifest')
+        mpd_url = self._search_regex(
+            r'(?mi)<BaseURL>(http.+.mpd)', mpd_content, 'new manifest', default=mpd_url)
+        formats = self._extract_mpd_formats(mpd_url, video_id)
+        self._sort_formats(formats)

-        result = {
+        return {
            'id': video_id,
+            'formats': formats,
            'title': title,
-            'description': description,
+            'description': self.dict_selection(video.get('descriptions', {}), 'en'),
            'duration': int_or_none(video.get('duration')),
            'timestamp': parse_iso8601(video.get('created_at')),
            'uploader': video.get('author'),
            'uploader_url': video.get('author_url'),
-            'like_count': like_count,
+            'like_count': int_or_none(try_get(video, lambda x: x['likes']['count'])),
            'age_limit': parse_age_limit(video.get('rating')),
            'thumbnails': thumbnails,
            'subtitles': subtitles,
            'episode_number': episode_number,
        }

-        formats = []
-
-        def add_format(format_id, format_dict, protocol='http'):
-            # rtmps URLs does not seem to work
-            if protocol == 'rtmps':
-                return
-            format_url = format_dict.get('url')
-            if not format_url:
-                return
-            qs = compat_parse_qs(compat_urllib_parse_urlparse(format_url).query)
-            stream = qs.get('stream', [None])[0]
-            if stream:
-                format_url = base64.b64decode(stream).decode()
-            if format_id in ('m3u8', 'hls'):
-                m3u8_formats = self._extract_m3u8_formats(
-                    format_url, video_id, 'mp4',
-                    entry_protocol='m3u8_native',
-                    m3u8_id='m3u8-%s' % protocol, fatal=False)
-                # Despite CODECS metadata in m3u8 all video-only formats
-                # are actually video+audio
-                for f in m3u8_formats:
-                    if not self.get_param('allow_unplayable_formats') and '_drm/index_' in f['url']:
-                        continue
-                    if f.get('acodec') == 'none' and f.get('vcodec') != 'none':
-                        f['acodec'] = None
-                    formats.append(f)
-            elif format_id in ('mpd', 'dash'):
-                formats.extend(self._extract_mpd_formats(
-                    format_url, video_id, 'mpd-%s' % protocol, fatal=False))
-            elif format_url.startswith('rtmp'):
-                mobj = re.search(
-                    r'^(?P<url>rtmp://[^/]+/(?P<app>.+?))/(?P<playpath>mp4:.+)$',
-                    format_url)
-                if not mobj:
-                    return
-                formats.append({
-                    'format_id': 'rtmp-%s' % format_id,
-                    'ext': 'flv',
-                    'url': mobj.group('url'),
-                    'play_path': mobj.group('playpath'),
-                    'app': mobj.group('app'),
-                    'page_url': url,
-                })
-            else:
-                urlh = self._request_webpage(
-                    HEADRequest(format_url), video_id, 'Checking file size', fatal=False)
-                formats.append({
-                    'url': format_url,
-                    'format_id': '%s-%s' % (format_id, protocol),
-                    'height': int_or_none(self._search_regex(
-                        r'^(\d+)[pP]$', format_id, 'height', default=None)),
-                    'filesize': int_or_none(urlh.headers.get('Content-Length')),
-                })
-
-        for format_id, format_dict in (resp.get('streams') or {}).items():
-            add_format(format_id, format_dict)
-        if not formats:
-            streams = self._call_api(
-                'videos/%s/streams.json' % video_id, video_id,
-                'Downloading video streams JSON')
-
-            if 'external' in streams:
-                result.update({
-                    '_type': 'url_transparent',
-                    'url': streams['external']['url'],
-                })
-                return result
-
-            for format_id, stream_dict in streams.items():
-                for protocol, format_dict in stream_dict.items():
-                    add_format(format_id, format_dict, protocol)
-        self._sort_formats(formats)
-
-        result['formats'] = formats
-        return result
-

 class VikiChannelIE(VikiBaseIE):
    IE_NAME = 'viki:channel'
@@ -406,7 +310,7 @@ class VikiChannelIE(VikiBaseIE):
            'title': 'Boys Over Flowers',
            'description': 'md5:804ce6e7837e1fd527ad2f25420f4d59',
        },
-        'playlist_mincount': 71,
+        'playlist_mincount': 51,
    }, {
        'url': 'http://www.viki.com/tv/1354c-poor-nastya-complete',
        'info_dict': {
@@ -427,33 +331,35 @@ class VikiChannelIE(VikiBaseIE):
        'only_matching': True,
    }]

-    _PER_PAGE = 25
+    _video_types = ('episodes', 'movies', 'clips', 'trailers')
+
+    def _entries(self, channel_id):
+        params = {
+            'app': self._APP, 'token': self._token, 'only_ids': 'true',
+            'direction': 'asc', 'sort': 'number', 'per_page': 30
+        }
+        video_types = self._configuration_arg('video_types') or self._video_types
+        for video_type in video_types:
+            if video_type not in self._video_types:
+                self.report_warning(f'Unknown video_type: {video_type}')
+            page_num = 0
+            while True:
+                page_num += 1
+                params['page'] = page_num
+                res = self._call_api(
+                    f'containers/{channel_id}/{video_type}.json', channel_id, query=params, fatal=False,
+                    note='Downloading %s JSON page %d' % (video_type.title(), page_num))
+
+                for video_id in res.get('response') or []:
+                    yield self.url_result(f'https://www.viki.com/videos/{video_id}', VikiIE.ie_key(), video_id)
+                if not res.get('more'):
+                    break

    def _real_extract(self, url):
        channel_id = self._match_id(url)
-
-        channel = self._call_api(
-            'containers/%s.json' % channel_id, channel_id,
-            'Downloading channel JSON')
-
+        channel = self._call_api('containers/%s.json' % channel_id, channel_id, 'Downloading channel JSON')
        self._check_errors(channel)
-
-        title = self.dict_selection(channel['titles'], 'en')
-
-        description = self.dict_selection(channel['descriptions'], 'en')
-
-        entries = []
-        for video_type in ('episodes', 'clips', 'movies'):
-            for page_num in itertools.count(1):
-                page = self._call_api(
-                    'containers/%s/%s.json?per_page=%d&sort=number&direction=asc&with_paging=true&page=%d'
-                    % (channel_id, video_type, self._PER_PAGE, page_num), channel_id,
-                    'Downloading %s JSON page #%d' % (video_type, page_num))
-                for video in page['response']:
-                    video_id = video['id']
-                    entries.append(self.url_result(
-                        'https://www.viki.com/videos/%s' % video_id, 'Viki'))
-                if not page['pagination']['next']:
-                    break
-
-        return self.playlist_result(entries, channel_id, title, description)
+        return self.playlist_result(
+            self._entries(channel_id), channel_id,
+            self.dict_selection(channel['titles'], 'en'),
+            self.dict_selection(channel['descriptions'], 'en'))
@@ -253,6 +253,30 @@ class VimeoBaseInfoExtractor(InfoExtractor):
                            'quality': 1,
                        }

+        jwt_response = self._download_json(
+            'https://vimeo.com/_rv/viewer', video_id, note='Downloading jwt token', fatal=False) or {}
+        if not jwt_response.get('jwt'):
+            return
+        headers = {'Authorization': 'jwt %s' % jwt_response['jwt']}
+        original_response = self._download_json(
+            f'https://api.vimeo.com/videos/{video_id}', video_id,
+            headers=headers, fatal=False) or {}
+        for download_data in original_response.get('download') or {}:
+            download_url = download_data.get('link')
+            if not download_url or download_data.get('quality') != 'source':
+                continue
+            query = compat_urlparse.parse_qs(compat_urlparse.urlparse(download_url).query)
+            return {
+                'url': download_url,
+                'ext': determine_ext(query.get('filename', [''])[0].lower()),
+                'format_id': download_data.get('public_name', 'Original'),
+                'width': int_or_none(download_data.get('width')),
+                'height': int_or_none(download_data.get('height')),
+                'fps': int_or_none(download_data.get('fps')),
+                'filesize': int_or_none(download_data.get('size')),
+                'quality': 1,
+            }
+

 class VimeoIE(VimeoBaseInfoExtractor):
    """Information extractor for vimeo.com."""
@@ -426,6 +450,22 @@ class VimeoIE(VimeoBaseInfoExtractor):
                'description': 'md5:ae23671e82d05415868f7ad1aec21147',
            },
        },
+        {
+            'note': 'Contains original format not accessible in webpage',
+            'url': 'https://vimeo.com/393756517',
+            'md5': 'c464af248b592190a5ffbb5d33f382b0',
+            'info_dict': {
+                'id': '393756517',
+                'ext': 'mov',
+                'timestamp': 1582642091,
+                'uploader_id': 'frameworkla',
+                'title': 'Straight To Hell - Sabrina: Netflix',
+                'uploader': 'Framework Studio',
+                'description': 'md5:f2edc61af3ea7a5592681ddbb683db73',
+                'upload_date': '20200225',
+            },
+            'expected_warnings': ['Unable to download JSON metadata'],
+        },
        {
            # only available via https://vimeo.com/channels/tributes/6213729 and
            # not via https://vimeo.com/6213729
@@ -178,9 +178,15 @@ class VLiveIE(VLiveBaseIE):
        if video_type == 'VOD':
            inkey = self._call_api('video/v1.0/vod/%s/inkey', video_id)['inkey']
            vod_id = video['vodId']
-            return merge_dicts(
+            info_dict = merge_dicts(
                get_common_fields(),
                self._extract_video_info(video_id, vod_id, inkey))
+            thumbnail = video.get('thumb')
+            if thumbnail:
+                if not info_dict.get('thumbnails') and info_dict.get('thumbnail'):
+                    info_dict['thumbnails'] = [{'url': info_dict.pop('thumbnail')}]
+                info_dict.setdefault('thumbnails', []).append({'url': thumbnail, 'preference': 1})
+            return info_dict
        elif video_type == 'LIVE':
            status = video.get('status')
            if status == 'ON_AIR':
@@ -362,7 +362,7 @@ class YahooSearchIE(SearchInfoExtractor):

 class YahooGyaOPlayerIE(InfoExtractor):
    IE_NAME = 'yahoo:gyao:player'
-    _VALID_URL = r'https?://(?:gyao\.yahoo\.co\.jp/(?:player|episode/[^/]+)|streaming\.yahoo\.co\.jp/c/y)/(?P<id>\d+/v\d+/v\d+|[\da-f]{8}-[\da-f]{4}-[\da-f]{4}-[\da-f]{4}-[\da-f]{12})'
+    _VALID_URL = r'https?://(?:gyao\.yahoo\.co\.jp/(?:player|episode(?:/[^/]+)?)|streaming\.yahoo\.co\.jp/c/y)/(?P<id>\d+/v\d+/v\d+|[\da-f]{8}-[\da-f]{4}-[\da-f]{4}-[\da-f]{4}-[\da-f]{12})'
    _TESTS = [{
        'url': 'https://gyao.yahoo.co.jp/player/00998/v00818/v0000000000000008564/',
        'info_dict': {
@@ -384,6 +384,9 @@ class YahooGyaOPlayerIE(InfoExtractor):
    }, {
        'url': 'https://gyao.yahoo.co.jp/episode/%E3%81%8D%E3%81%AE%E3%81%86%E4%BD%95%E9%A3%9F%E3%81%B9%E3%81%9F%EF%BC%9F%20%E7%AC%AC2%E8%A9%B1%202019%2F4%2F12%E6%94%BE%E9%80%81%E5%88%86/5cb02352-b725-409e-9f8d-88f947a9f682',
        'only_matching': True,
+    }, {
+        'url': 'https://gyao.yahoo.co.jp/episode/5fa1226c-ef8d-4e93-af7a-fd92f4e30597',
+        'only_matching': True,
    }]
    _GEO_BYPASS = False

@@ -19,6 +19,7 @@ from .utils import (
    preferredencoding,
    write_string,
 )
+from .cookies import SUPPORTED_BROWSERS
 from .version import __version__

 from .downloader.external import list_external_downloaders
@@ -137,14 +138,21 @@ def parseOpts(overrideArguments=None):
        else:
            raise optparse.OptionValueError(
                'wrong %s formatting; it should be %s, not "%s"' % (opt_str, option.metavar, value))
-        val = process(val) if callable(process) else val
+        try:
+            val = process(val) if process else val
+        except Exception as err:
+            raise optparse.OptionValueError(
+                'wrong %s formatting; %s' % (opt_str, err))
        for key in keys:
            out_dict[key] = val

    # No need to wrap help messages if we're on a wide console
    columns = compat_get_terminal_size().columns
    max_width = columns if columns else 80
-    max_help_position = 80
+    # 47% is chosen because that is how README.md is currently formatted
+    # and moving help text even further to the right is undesirable.
+    # This can be reduced in the future to get a prettier output
+    max_help_position = int(0.47 * max_width)

    fmt = optparse.IndentedHelpFormatter(width=max_width, max_help_position=max_help_position)
    fmt.format_option_strings = _format_option_string
@@ -520,8 +528,12 @@ def parseOpts(overrideArguments=None):
        help="Don't give any special preference to free containers (default)")
    video_format.add_option(
        '--check-formats',
-        action='store_true', dest='check_formats', default=False,
-        help="Check that the formats selected are actually downloadable (Experimental)")
+        action='store_true', dest='check_formats', default=None,
+        help='Check that the formats selected are actually downloadable')
+    video_format.add_option(
+        '--no-check-formats',
+        action='store_false', dest='check_formats',
+        help='Do not check that the formats selected are actually downloadable')
    video_format.add_option(
        '-F', '--list-formats',
        action='store_true', dest='listformats',
@@ -694,9 +706,8 @@ def parseOpts(overrideArguments=None):
        callback_kwargs={
            'allowed_keys': 'http|ftp|m3u8|dash|rtsp|rtmp|mms',
            'default_key': 'default',
-            'process': lambda x: x.strip()
-        },
-        help=(
+            'process': str.strip
+        }, help=(
            'Name or path of the external downloader to use (optionally) prefixed by '
            'the protocols (http, ftp, m3u8, dash, rstp, rtmp, mms) to use it for. '
            'Currently supports native, %s (Recommended: aria2c). '
@@ -712,8 +723,7 @@ def parseOpts(overrideArguments=None):
            'allowed_keys': '|'.join(list_external_downloaders()),
            'default_key': 'default',
            'process': compat_shlex_split
-        },
-        help=(
+        }, help=(
            'Give these arguments to the external downloader. '
            'Specify the downloader name and the arguments separated by a colon ":". '
            'You can use this option multiple times to give different arguments to different downloaders '
@@ -912,14 +922,16 @@ def parseOpts(overrideArguments=None):
        action='store_true', dest='useid', help=optparse.SUPPRESS_HELP)
    filesystem.add_option(
        '-P', '--paths',
-        metavar='TYPES:PATH', dest='paths', default={}, type='str',
+        metavar='[TYPES:]PATH', dest='paths', default={}, type='str',
        action='callback', callback=_dict_from_options_callback,
-        callback_kwargs={'allowed_keys': 'home|temp|%s' % '|'.join(OUTTMPL_TYPES.keys())},
-        help=(
+        callback_kwargs={
+            'allowed_keys': 'home|temp|%s' % '|'.join(OUTTMPL_TYPES.keys()),
+            'default_key': 'home'
+        }, help=(
            'The paths where the files should be downloaded. '
            'Specify the type of file and the path separated by a colon ":". '
            'All the same types as --output are supported. '
-            'Additionally, you can also provide "home" and "temp" paths. '
+            'Additionally, you can also provide "home" (default) and "temp" paths. '
            'All intermediary files are first downloaded to the temp path and '
            'then the final files are moved over to the home path after download is finished. '
            'This option is ignored if --output is an absolute path'))
@@ -930,8 +942,7 @@ def parseOpts(overrideArguments=None):
        callback_kwargs={
            'allowed_keys': '|'.join(OUTTMPL_TYPES.keys()),
            'default_key': 'default'
-        },
-        help='Output filename template; see "OUTPUT TEMPLATE" for details')
+        }, help='Output filename template; see "OUTPUT TEMPLATE" for details')
    filesystem.add_option(
        '--output-na-placeholder',
        dest='outtmpl_na_placeholder', metavar='TEXT', default='NA',
@@ -1050,7 +1061,7 @@ def parseOpts(overrideArguments=None):
        help='Do not write playlist metadata when using --write-info-json, --write-description etc.')
    filesystem.add_option(
        '--clean-infojson',
-        action='store_true', dest='clean_infojson', default=True,
+        action='store_true', dest='clean_infojson', default=None,
        help=(
            'Remove some private fields such as filenames from the infojson. '
            'Note that it could still contain some personal information (default)'))
@@ -1079,7 +1090,21 @@ def parseOpts(overrideArguments=None):
    filesystem.add_option(
        '--no-cookies',
        action='store_const', const=None, dest='cookiefile', metavar='FILE',
-        help='Do not read/dump cookies (default)')
+        help='Do not read/dump cookies from/to file (default)')
+    filesystem.add_option(
+        '--cookies-from-browser',
+        dest='cookiesfrombrowser', metavar='BROWSER[:PROFILE]',
+        help=(
+            'Load cookies from a user profile of the given web browser. '
+            'Currently supported browsers are: {}. '
+            'You can specify the user profile name or directory using '
+            '"BROWSER:PROFILE_NAME" or "BROWSER:PROFILE_PATH". '
+            'If no profile is given, the most recently accessed one is used'.format(
+                '|'.join(sorted(SUPPORTED_BROWSERS)))))
+    filesystem.add_option(
+        '--no-cookies-from-browser',
+        action='store_const', const=None, dest='cookiesfrombrowser',
+        help='Do not load cookies from browser (default)')
    filesystem.add_option(
        '--cache-dir', dest='cachedir', default=None, metavar='DIR',
        help='Location in the filesystem where youtube-dl can store some downloaded information (such as client ids and signatures) permanently. By default $XDG_CACHE_HOME/youtube-dl or ~/.cache/youtube-dl')
@@ -1163,8 +1188,7 @@ def parseOpts(overrideArguments=None):
            'allowed_keys': r'\w+(?:\+\w+)?', 'default_key': 'default-compat',
            'process': compat_shlex_split,
            'multiple_keys': False
-        },
-        help=(
+        }, help=(
            'Give these arguments to the postprocessors. '
            'Specify the postprocessor/executable name and the arguments separated by a colon ":" '
            'to give the argument to the specified postprocessor/executable. Supported PP are: '
@@ -1260,7 +1284,11 @@ def parseOpts(overrideArguments=None):
            'Execute a command on the file after downloading and post-processing. '
            'Similar syntax to the output template can be used to pass any field as arguments to the command. '
            'An additional field "filepath" that contains the final path of the downloaded file is also available. '
-            'If no fields are passed, "%(filepath)s" is appended to the end of the command'))
+            'If no fields are passed, %(filepath)q is appended to the end of the command'))
+    postproc.add_option(
+        '--exec-before-download',
+        metavar='CMD', dest='exec_before_dl_cmd',
+        help='Execute a command before the actual download. The syntax is the same as --exec but "filepath" is not available')
    postproc.add_option(
        '--convert-subs', '--convert-sub', '--convert-subtitles',
        metavar='FORMAT', dest='convertsubtitles', default=None,
@@ -1344,6 +1372,7 @@ def parseOpts(overrideArguments=None):
        '--no-hls-split-discontinuity',
        dest='hls_split_discontinuity', action='store_false',
        help='Do not split HLS playlists to different formats at discontinuities such as ad breaks (default)')
+    _extractor_arg_parser = lambda key, vals='': (key.strip().lower(), [val.strip() for val in vals.split(',')])
    extractor.add_option(
        '--extractor-args',
        metavar='KEY:ARGS', dest='extractor_args', default={}, type='str',
@@ -1351,11 +1380,10 @@ def parseOpts(overrideArguments=None):
        callback_kwargs={
            'multiple_keys': False,
            'process': lambda val: dict(
-                (lambda x: (x[0], x[1].split(',')))(arg.split('=', 1) + ['', '']) for arg in val.split(';'))
-        },
-        help=(
+                _extractor_arg_parser(*arg.split('=', 1)) for arg in val.split(';'))
+        }, help=(
            'Pass these arguments to the extractor. See "EXTRACTOR ARGUMENTS" for details. '
-            'You can use this option multiple times to give different arguments to different extractors'))
+            'You can use this option multiple times to give arguments for different extractors'))
    extractor.add_option(
        '--youtube-include-dash-manifest', '--no-youtube-skip-dash-manifest',
        action='store_true', dest='youtube_include_dash_manifest', default=True,
@@ -51,7 +51,7 @@ class EmbedThumbnailPP(FFmpegPostProcessor):

        try:
            size_regex = r',\s*(?P<w>\d+)x(?P<h>\d+)\s*[,\[]'
-            size_result = self.run_ffmpeg(filename, filename, ['-hide_banner'])
+            size_result = self.run_ffmpeg(filename, None, ['-hide_banner'], expected_retcodes=(1,))
            mobj = re.search(size_regex, size_result)
            if mobj is None:
                return guess()
@@ -23,12 +23,13 @@ class ExecAfterDownloadPP(PostProcessor):
    def parse_cmd(self, cmd, info):
        tmpl, tmpl_dict = self._downloader.prepare_outtmpl(cmd, info)
        if tmpl_dict:  # if there are no replacements, tmpl_dict = {}
-            return tmpl % tmpl_dict
+            return self._downloader.escape_outtmpl(tmpl) % tmpl_dict

        # If no replacements are found, replace {} for backard compatibility
        if '{}' not in cmd:
            cmd += ' {}'
-        return cmd.replace('{}', compat_shlex_quote(info['filepath']))
+        return cmd.replace('{}', compat_shlex_quote(
+            info.get('filepath') or info['_filename']))

    def run(self, info):
        cmd = self.parse_cmd(self.exec_cmd, info)
@@ -1,6 +1,7 @@
 from __future__ import unicode_literals

 import io
+import itertools
 import os
 import subprocess
 import time
@@ -24,6 +25,7 @@ from ..utils import (
    process_communicate_or_kill,
    replace_extension,
    traverse_obj,
+    variadic,
 )


@@ -233,16 +235,16 @@ class FFmpegPostProcessor(PostProcessor):
            None)
        return num, len(streams)

-    def run_ffmpeg_multiple_files(self, input_paths, out_path, opts):
+    def run_ffmpeg_multiple_files(self, input_paths, out_path, opts, **kwargs):
        return self.real_run_ffmpeg(
            [(path, []) for path in input_paths],
-            [(out_path, opts)])
+            [(out_path, opts)], **kwargs)

-    def real_run_ffmpeg(self, input_path_opts, output_path_opts):
+    def real_run_ffmpeg(self, input_path_opts, output_path_opts, *, expected_retcodes=(0,)):
        self.check_version()

        oldest_mtime = min(
-            os.stat(encodeFilename(path)).st_mtime for path, _ in input_path_opts)
+            os.stat(encodeFilename(path)).st_mtime for path, _ in input_path_opts if path)

        cmd = [encodeFilename(self.executable, True), encodeArgument('-y')]
        # avconv does not have repeat option
@@ -261,23 +263,25 @@ class FFmpegPostProcessor(PostProcessor):
                + [encodeFilename(self._ffmpeg_filename_argument(file), True)])

        for arg_type, path_opts in (('i', input_path_opts), ('o', output_path_opts)):
-            cmd += [arg for i, o in enumerate(path_opts)
-                    for arg in make_args(o[0], o[1], arg_type, i + 1)]
+            cmd += itertools.chain.from_iterable(
+                make_args(path, list(opts), arg_type, i + 1)
+                for i, (path, opts) in enumerate(path_opts) if path)

        self.write_debug('ffmpeg command line: %s' % shell_quote(cmd))
        p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, stdin=subprocess.PIPE)
        stdout, stderr = process_communicate_or_kill(p)
-        if p.returncode != 0:
+        if p.returncode not in variadic(expected_retcodes):
            stderr = stderr.decode('utf-8', 'replace').strip()
            if self.get_param('verbose', False):
                self.report_error(stderr)
            raise FFmpegPostProcessorError(stderr.split('\n')[-1])
        for out_path, _ in output_path_opts:
-            self.try_utime(out_path, oldest_mtime, oldest_mtime)
+            if out_path:
+                self.try_utime(out_path, oldest_mtime, oldest_mtime)
        return stderr.decode('utf-8', 'replace')

-    def run_ffmpeg(self, path, out_path, opts):
-        return self.run_ffmpeg_multiple_files([path], out_path, opts)
+    def run_ffmpeg(self, path, out_path, opts, **kwargs):
+        return self.run_ffmpeg_multiple_files([path], out_path, opts, **kwargs)

    def _ffmpeg_filename_argument(self, fn):
        # Always use 'file:' because the filename may contain ':' (ffmpeg
@@ -526,6 +530,15 @@ class FFmpegEmbedSubtitlePP(FFmpegPostProcessor):


 class FFmpegMetadataPP(FFmpegPostProcessor):
+
+    @staticmethod
+    def _options(target_ext):
+        yield from ('-map', '0', '-dn')
+        if target_ext == 'm4a':
+            yield from ('-vn', '-acodec', 'copy')
+        else:
+            yield from ('-c', 'copy')
+
    @PostProcessor._restrict_to(images=False)
    def run(self, info):
        metadata = {}
@@ -533,15 +546,9 @@ class FFmpegMetadataPP(FFmpegPostProcessor):
        def add(meta_list, info_list=None):
            if not meta_list:
                return
-            if not info_list:
-                info_list = meta_list
-            if not isinstance(meta_list, (list, tuple)):
-                meta_list = (meta_list,)
-            if not isinstance(info_list, (list, tuple)):
-                info_list = (info_list,)
-            for info_f in info_list:
+            for info_f in variadic(info_list or meta_list):
                if isinstance(info.get(info_f), (compat_str, compat_numeric_types)):
-                    for meta_f in meta_list:
+                    for meta_f in variadic(meta_list):
                        metadata[meta_f] = info[info_f]
                    break

@@ -570,22 +577,17 @@ class FFmpegMetadataPP(FFmpegPostProcessor):
        for key in filter(lambda k: k.startswith(prefix), info.keys()):
            add(key[len(prefix):], key)

-        if not metadata:
-            self.to_screen('There isn\'t any metadata to add')
-            return [], info
+        filename, metadata_filename = info['filepath'], None
+        options = [('-metadata', f'{name}={value}') for name, value in metadata.items()]

-        filename = info['filepath']
-        temp_filename = prepend_extension(filename, 'temp')
-        in_filenames = [filename]
-        options = ['-map', '0', '-dn']
-
-        if info['ext'] == 'm4a':
-            options.extend(['-vn', '-acodec', 'copy'])
-        else:
-            options.extend(['-c', 'copy'])
-
-        for name, value in metadata.items():
-            options.extend(['-metadata', '%s=%s' % (name, value)])
+        stream_idx = 0
+        for fmt in info.get('requested_formats') or []:
+            stream_count = 2 if 'none' not in (fmt.get('vcodec'), fmt.get('acodec')) else 1
+            if fmt.get('language'):
+                lang = ISO639Utils.short2long(fmt['language']) or fmt['language']
+                options.extend(('-metadata:s:%d' % (stream_idx + i), 'language=%s' % lang)
+                               for i in range(stream_count))
+            stream_idx += stream_count

        chapters = info.get('chapters', [])
        if chapters:
@@ -603,24 +605,29 @@ class FFmpegMetadataPP(FFmpegPostProcessor):
                    if chapter_title:
                        metadata_file_content += 'title=%s\n' % ffmpeg_escape(chapter_title)
                f.write(metadata_file_content)
-                in_filenames.append(metadata_filename)
-                options.extend(['-map_metadata', '1'])
+                options.append(('-map_metadata', '1'))

        if ('no-attach-info-json' not in self.get_param('compat_opts', [])
                and '__infojson_filename' in info and info['ext'] in ('mkv', 'mka')):
-            old_stream, new_stream = self.get_stream_number(
-                filename, ('tags', 'mimetype'), 'application/json')
+            old_stream, new_stream = self.get_stream_number(filename, ('tags', 'mimetype'), 'application/json')
            if old_stream is not None:
-                options.extend(['-map', '-0:%d' % old_stream])
+                options.append(('-map', '-0:%d' % old_stream))
                new_stream -= 1

-            options.extend([
+            options.append((
                '-attach', info['__infojson_filename'],
                '-metadata:s:%d' % new_stream, 'mimetype=application/json'
-            ])
+            ))

-        self.to_screen('Adding metadata to \'%s\'' % filename)
-        self.run_ffmpeg_multiple_files(in_filenames, temp_filename, options)
+        if not options:
+            self.to_screen('There isn\'t any metadata to add')
+            return [], info
+
+        temp_filename = prepend_extension(filename, 'temp')
+        self.to_screen('Adding metadata to "%s"' % filename)
+        self.run_ffmpeg_multiple_files(
+            (filename, metadata_filename), temp_filename,
+            itertools.chain(self._options(info['ext']), *options))
        if chapters:
            os.remove(metadata_filename)
        os.remove(encodeFilename(filename))
@@ -27,7 +27,7 @@ class MetadataFromFieldPP(PostProcessor):

    @staticmethod
    def field_to_template(tmpl):
-        if re.match(r'\w+$', tmpl):
+        if re.match(r'[a-zA-Z_]+$', tmpl):
            return '%%(%s)s' % tmpl
        return tmpl

@@ -55,7 +55,7 @@ class MetadataFromFieldPP(PostProcessor):
    def run(self, info):
        for dictn in self._data:
            tmpl, tmpl_dict = self._downloader.prepare_outtmpl(dictn['tmpl'], info)
-            data_to_parse = tmpl % tmpl_dict
+            data_to_parse = self._downloader.escape_outtmpl(tmpl) % tmpl_dict
            self.write_debug('Searching for r"%s" in %s' % (dictn['regex'], dictn['tmpl']))
            match = re.search(dictn['regex'], data_to_parse)
            if match is None:
@@ -63,7 +63,7 @@ class MetadataFromFieldPP(PostProcessor):
                continue
            for attribute, value in match.groupdict().items():
                info[attribute] = value
-                self.to_screen('parsed %s from "%s": %s' % (attribute, dictn['in'], value if value is not None else 'NA'))
+                self.to_screen('parsed %s from "%s": %s' % (attribute, dictn['tmpl'], value if value is not None else 'NA'))
        return [], info


@@ -3964,7 +3964,7 @@ def detect_exe_version(output, version_re=None, unrecognized='present'):
        return unrecognized


-class LazyList(collections.Sequence):
+class LazyList(collections.abc.Sequence):
    ''' Lazy immutable list from an iterable
    Note that slices of a LazyList are lists and not LazyList'''

@@ -3993,28 +3993,27 @@ class LazyList(collections.Sequence):

    @staticmethod
    def __reverse_index(x):
-        return -(x + 1)
+        return None if x is None else -(x + 1)

    def __getitem__(self, idx):
        if isinstance(idx, slice):
-            step = idx.step or 1
-            start = idx.start if idx.start is not None else 0 if step > 0 else -1
-            stop = idx.stop if idx.stop is not None else -1 if step > 0 else 0
            if self.__reversed:
-                (start, stop), step = map(self.__reverse_index, (start, stop)), -step
-                idx = slice(start, stop, step)
+                idx = slice(self.__reverse_index(idx.start), self.__reverse_index(idx.stop), -(idx.step or 1))
+            start, stop, step = idx.start, idx.stop, idx.step or 1
        elif isinstance(idx, int):
            if self.__reversed:
                idx = self.__reverse_index(idx)
-            start = stop = idx
+            start, stop, step = idx, idx, 0
        else:
            raise TypeError('indices must be integers or slices')
-        if start < 0 or stop < 0:
+        if ((start or 0) < 0 or (stop or 0) < 0
+                or (start is None and step < 0)
+                or (stop is None and step > 0)):
            # We need to consume the entire iterable to be able to slice from the end
            # Obviously, never use this with infinite iterables
            return self.__exhaust()[idx]

-        n = max(start, stop) - len(self.__cache) + 1
+        n = max(start or 0, stop or 0) - len(self.__cache) + 1
        if n > 0:
            self.__cache.extend(itertools.islice(self.__iterable, n))
        return self.__cache[idx]
@@ -4289,9 +4288,7 @@ def dict_get(d, key_or_keys, default=None, skip_false_values=True):


 def try_get(src, getter, expected_type=None):
-    if not isinstance(getter, (list, tuple)):
-        getter = [getter]
-    for get in getter:
+    for get in variadic(getter):
        try:
            v = get(src)
        except (AttributeError, KeyError, TypeError, IndexError):
@@ -4367,7 +4364,7 @@ def strip_jsonp(code):

 def js_to_json(code, vars={}):
    # vars is a dict of var, val pairs to substitute
-    COMMENT_RE = r'/\*(?:(?!\*/).)*?\*/|//[^\n]*'
+    COMMENT_RE = r'/\*(?:(?!\*/).)*?\*/|//[^\n]*\n'
    SKIP_RE = r'\s*(?:{comment})?\s*'.format(comment=COMMENT_RE)
    INTEGER_TABLE = (
        (r'(?s)^(0[xX][0-9a-fA-F]+){skip}:?$'.format(skip=SKIP_RE), 16),
@@ -4440,8 +4437,8 @@ OUTTMPL_TYPES = {
 # As of [1] format syntax is:
 #  %[mapping_key][conversion_flags][minimum_width][.precision][length_modifier]type
 # 1. https://docs.python.org/2/library/stdtypes.html#string-formatting
-STR_FORMAT_RE = r'''(?x)
-    (?<!%)
+STR_FORMAT_RE_TMPL = r'''(?x)
+    (?<!%)(?P<prefix>(?:%%)*)
    %
    (?P<has_key>\((?P<key>{0})\))?  # mapping key
    (?P<format>
@@ -4449,11 +4446,14 @@ STR_FORMAT_RE = r'''(?x)
        (?:\d+)?  # minimum field width (optional)
        (?:\.\d+)?  # precision (optional)
        [hlL]?  # length modifier (optional)
-        [diouxXeEfFgGcrs]  # conversion type
+        {1}  # conversion type
    )
 '''


+STR_FORMAT_TYPES = 'diouxXeEfFgGcrs'
+
+
 def limit_length(s, length):
    """ Add ellipses to overly long strings """
    if s is None:
@@ -4544,7 +4544,7 @@ def parse_codecs(codecs_str):
    if not codecs_str:
        return {}
    split_codecs = list(filter(None, map(
-        lambda str: str.strip(), codecs_str.strip().strip(',').split(','))))
+        str.strip, codecs_str.strip().strip(',').split(','))))
    vcodec, acodec = None, None
    for full_codec in split_codecs:
        codec = full_codec.split('.')[0]
@@ -4964,11 +4964,9 @@ def cli_configuration_args(argdict, keys, default=[], use_compat=True):

    assert isinstance(keys, (list, tuple))
    for key_list in keys:
-        if isinstance(key_list, compat_str):
-            key_list = (key_list,)
        arg_list = list(filter(
            lambda x: x is not None,
-            [argdict.get(key.lower()) for key in key_list]))
+            [argdict.get(key.lower()) for key in variadic(key_list)]))
        if arg_list:
            return [arg for args in arg_list for arg in args]
    return default
@@ -6228,40 +6226,93 @@ def load_plugins(name, suffix, namespace):
    return classes


-def traverse_obj(obj, keys, *, casesense=True, is_user_input=False, traverse_string=False):
+def traverse_obj(
+        obj, *path_list, default=None, expected_type=None, get_all=True,
+        casesense=True, is_user_input=False, traverse_string=False):
    ''' Traverse nested list/dict/tuple
+    @param path_list        A list of paths which are checked one by one.
+                            Each path is a list of keys where each key is a string,
+                            a tuple of strings or "...". When a tuple is given,
+                            all the keys given in the tuple are traversed, and
+                            "..." traverses all the keys in the object
+    @param default          Default value to return
+    @param expected_type    Only accept final value of this type (Can also be any callable)
+    @param get_all          Return all the values obtained from a path or only the first one
    @param casesense        Whether to consider dictionary keys as case sensitive
    @param is_user_input    Whether the keys are generated from user input. If True,
                            strings are converted to int/slice if necessary
    @param traverse_string  Whether to traverse inside strings. If True, any
                            non-compatible object will also be converted into a string
+    # TODO: Write tests
    '''
-    keys = list(keys)[::-1]
-    while keys:
-        key = keys.pop()
-        if isinstance(obj, dict):
-            assert isinstance(key, compat_str)
-            if not casesense:
-                obj = {k.lower(): v for k, v in obj.items()}
-                key = key.lower()
-            obj = obj.get(key)
-        else:
-            if is_user_input:
-                key = (int_or_none(key) if ':' not in key
-                       else slice(*map(int_or_none, key.split(':'))))
-                if key is None:
+    if not casesense:
+        _lower = lambda k: (k.lower() if isinstance(k, str) else k)
+        path_list = (map(_lower, variadic(path)) for path in path_list)
+
+    def _traverse_obj(obj, path, _current_depth=0):
+        nonlocal depth
+        path = tuple(variadic(path))
+        for i, key in enumerate(path):
+            if isinstance(key, (list, tuple)):
+                obj = [_traverse_obj(obj, sub_key, _current_depth) for sub_key in key]
+                key = ...
+            if key is ...:
+                obj = (obj.values() if isinstance(obj, dict)
+                       else obj if isinstance(obj, (list, tuple, LazyList))
+                       else str(obj) if traverse_string else [])
+                _current_depth += 1
+                depth = max(depth, _current_depth)
+                return [_traverse_obj(inner_obj, path[i + 1:], _current_depth) for inner_obj in obj]
+            elif isinstance(obj, dict):
+                obj = (obj.get(key) if casesense or (key in obj)
+                       else next((v for k, v in obj.items() if _lower(k) == key), None))
+            else:
+                if is_user_input:
+                    key = (int_or_none(key) if ':' not in key
+                           else slice(*map(int_or_none, key.split(':'))))
+                    if key == slice(None):
+                        return _traverse_obj(obj, (..., *path[i + 1:]))
+                if not isinstance(key, (int, slice)):
                    return None
-            if not isinstance(obj, (list, tuple)):
-                if traverse_string:
-                    obj = compat_str(obj)
-                else:
+                if not isinstance(obj, (list, tuple, LazyList)):
+                    if not traverse_string:
+                        return None
+                    obj = str(obj)
+                try:
+                    obj = obj[key]
+                except IndexError:
                    return None
-            assert isinstance(key, (int, slice))
-            obj = try_get(obj, lambda x: x[key])
-    return obj
+        return obj
+
+    if isinstance(expected_type, type):
+        type_test = lambda val: val if isinstance(val, expected_type) else None
+    elif expected_type is not None:
+        type_test = expected_type
+    else:
+        type_test = lambda val: val
+
+    for path in path_list:
+        depth = 0
+        val = _traverse_obj(obj, path)
+        if val is not None:
+            if depth:
+                for _ in range(depth - 1):
+                    val = itertools.chain.from_iterable(v for v in val if v is not None)
+                val = [v for v in map(type_test, val) if v is not None]
+                if val:
+                    return val if get_all else val[0]
+            else:
+                val = type_test(val)
+                if val is not None:
+                    return val
+    return default


 def traverse_dict(dictn, keys, casesense=True):
    ''' For backward compatibility. Do not use '''
    return traverse_obj(dictn, keys, casesense=casesense,
                        is_user_input=True, traverse_string=True)
+
+
+def variadic(x, allowed_types=(str, bytes)):
+    return x if isinstance(x, collections.abc.Iterable) and not isinstance(x, allowed_types) else (x,)
@@ -1,3 +1,3 @@
 from __future__ import unicode_literals

-__version__ = '2021.06.23'
+__version__ = '2021.07.24'
@@ -120,12 +120,11 @@ def _format_ts(ts):
    Convert an MPEG PES timestamp into a WebVTT timestamp.
    This will lose sub-millisecond precision.
    """
-
-    ts = int((ts + 45) // 90)
-    ms , ts = divmod(ts, 1000)  # noqa: W504,E221,E222,E203
-    s  , ts = divmod(ts, 60)    # noqa: W504,E221,E222,E203
-    min, h  = divmod(ts, 60)    # noqa: W504,E221,E222
-    return '%02u:%02u:%02u.%03u' % (h, min, s, ms)
+    msec = int((ts + 45) // 90)
+    secs, msec = divmod(msec, 1000)
+    mins, secs = divmod(secs, 60)
+    hrs, mins = divmod(mins, 60)
+    return '%02u:%02u:%02u.%03u' % (hrs, mins, secs, msec)


 class Block(object):
Author	SHA1	Message	Date
pukkandan	6ccf351a87	Release 2021.08.02	2021-08-02 10:37:10 +05:30
pukkandan	28dff70b51	Add donate links	2021-08-02 08:51:23 +05:30
pukkandan	1aebc0f79e	Add logo and banner	2021-08-02 08:51:22 +05:30
pukkandan	cf87314d4e	[youtube] Extract SAPISID only once	2021-08-02 08:00:08 +05:30
pukkandan	1bd3639f69	[tenplay] Add MA15+ age limit (#606 ) Authored by: pento	2021-08-02 07:52:11 +05:30
LEandGitHub	68f5867cf0	[CBS] Add fallback (#579 ) Related: https://github.com/ytdl-org/youtube-dl/issues/29564 Authored-by: llacb47, pukkandan	2021-08-02 07:46:12 +05:30
AshishandGitHub	605cad0be7	[Vimeo] Better extraction of original file (#599 ) Authored by: Ashish0804	2021-08-02 07:23:12 +05:30
pukkandan	0855702f3f	[test:download] Support testing with `ignore_no_formats_error`	2021-08-02 03:47:31 +05:30
AshishandGitHub	e8384376c0	[CBS] Add ParamountPlusSeriesIE (#603 ) Authored by: Ashish0804	2021-08-02 02:58:47 +05:30
DavidandGitHub	e7e94f2a5c	[youtube] Add age-gate bypass for unverified accounts (#600 ) Adds `_creator` variants for each client Authored by: zerodytrash, colethedj, pukkandan	2021-08-02 02:43:46 +05:30
pukkandan	a46a815b05	[cleanup] Fix linter in `96fccc101f`	2021-08-01 12:52:09 +05:30
pukkandan	96fccc101f	[downloader] Allow streaming unmerged formats to stdout using ffmpeg For this to work: 1. The downloader must be ffmpeg 2. The selected formats must have the same protocol 3. The formats must be downloadable by ffmpeg to stdout Partial solution for: https://github.com/ytdl-org/youtube-dl/issues/28146, https://github.com/ytdl-org/youtube-dl/issues/27265	2021-08-01 12:38:06 +05:30
pukkandan	dbf5416a20	[cleanup] Refactor some code	2021-08-01 12:38:05 +05:30
pukkandan	d74a58a186	Set `home:` as the default key for `-P`	2021-08-01 12:13:40 +05:30
pukkandan	f5510afef0	[FormatSort] Fix bug for audio with unknown codec	2021-08-01 12:13:40 +05:30
pukkandan	e4f0275711	Add compat-option `no-clean-infojson`	2021-08-01 12:13:40 +05:30
pukkandan	e0f2b4b47d	[utils] Fix slicing of reversed `LazyList` Closes #589	2021-08-01 12:13:40 +05:30
coletdjnzandGitHub	eca330cb88	[youtube] Fix default global API key bug introduced in https://github.com/yt-dlp/yt-dlp/commit/000c15a4cad1079fe91609a5d5a0ff8d20cca865	2021-08-01 06:12:26 +00:00
WesandGitHub	d24734daea	[adobepass] Add MSO Sling TV (#596 ) Original PR: ytdl-org/youtube-dl#29686 Closes: #300, ytdl-org/youtube-dl#18132 Authored by: wesnm	2021-07-31 03:35:56 +05:30
MinePlayersPEandGitHub	d9e6e9481e	[RCTIPlus] Remove PhantomJS dependency (#595 ) Authored by: MinePlayersPE	2021-07-31 03:22:52 +05:30
pukkandan	3619f78d2c	[youtube] Misc cleanup (#577 ) Authored by: pukkandan, colethedj	2021-07-31 03:01:49 +05:30
pukkandan	65c2fde23f	[youtube] Add `thirdParty` to agegate clients (#577 ) * This allows more videos like `tf2U5Vyj0oU` to become embeddable See https://github.com/yt-dlp/yt-dlp/pull/575#issuecomment-888837000 * Also added tests for all types of age-gate Closes #581	2021-07-31 02:20:21 +05:30
pukkandan	000c15a4ca	[youtube] simplify and de-duplicate client definitions (#577 )	2021-07-31 02:14:15 +05:30
colethedjandpukkandan	9275f62cf8	[youtube] Improve age-gate detection (#577 ) Authored by: colethedj	2021-07-31 02:13:55 +05:30
coletdjnzandGitHub	6552469433	[youtube] Force `hl=en` for comments (#594 ) Closes #532	2021-07-31 01:06:00 +05:30
MinePlayersPEandGitHub	11cc45718c	[vidio] Fix login error detection (#582 ) Authored by: MinePlayersPE	2021-07-29 10:11:05 +05:30
AshishandGitHub	fe07e2c69f	[Hotstar] Support cookies (#584 ) Closes #583 Authored by: Ashish0804	2021-07-29 10:06:38 +05:30
AshishandGitHub	89ce723edd	[Mxplayer] Add h265 formats (#572 ) Authored by: Ashish0804	2021-07-29 09:57:09 +05:30
SipherdrakonandGitHub	45d1f15725	[dplay] Add `ScienceChannelIE` (#567 ) Authored by: Sipherdrakon	2021-07-29 09:55:00 +05:30
rigstotandGitHub	a318f59d14	[generic] Support KVS player (#549 ) * Replaces the extractor for thisvid Fixes: https://github.com/ytdl-org/youtube-dl/issues/2077 Authored-by: rigstot	2021-07-29 09:33:01 +05:30
pukkandan	7d1eb38af1	Add format types `j`, `l`, `q` for outtmpl Closes #345	2021-07-29 08:47:25 +05:30
pukkandan	901130bbcf	Expand and escape environment variables correctly in outtmpl Fixes: https://www.reddit.com/r/youtubedl/comments/otfmq3/ytdlp_same_parameters_different_results	2021-07-29 08:38:18 +05:30
MinePlayersPEandGitHub	c0bc527bca	[YouTube] Age-gate bypass implementation (#575 ) * Calling the API with `clientScreen=EMBED` allows access to most age-gated videos - discovered by @ccdffddfddfdsfedeee (https://github.com/yt-dlp/yt-dlp/issues/574#issuecomment-887171136) * Adds clients: (web/android/ios)_(embedded/agegate), mweb_embedded * Renamed mobile_web to mweb Closes #574 Authored by pukkandan, MinePlayersPE	2021-07-27 15:10:44 +05:30
pukkandan	2a9c6dcd22	[youtube] Fix format sorting when using alternate clients	2021-07-26 03:50:13 +05:30
coletdjnzandGitHub	5a1fc62b41	[youtube] Add `mobile_web` client (#557 ) Authored by: colethedj	2021-07-26 03:48:36 +05:30
pukkandan	b4c055bac2	[youtube] Add `player_client=all`	2021-07-26 03:38:18 +05:30
pukkandan	ea05b3020d	Remove `asr` appearing twice in `-F`	2021-07-26 03:38:15 +05:30
pukkandan	9536bc072d	[bilibili] Improve `_VALID_URL`	2021-07-26 03:38:10 +05:30
AshishandGitHub	8242bf220d	[HotStarSeriesIE] Fix regex (#569 ) Authored by: Ashish0804	2021-07-25 22:43:43 +05:30
AshishandGitHub	4bfa401d40	[UtreonIE] Add extractor (#562 ) Authored by: Ashish0804	2021-07-25 22:41:45 +05:30
nixxoandGitHub	0222620725	[mediaset] Fix extraction (#564 ) Closes #365 Authored by: nixxo	2021-07-24 20:06:55 +05:30
pukkandan	1fe3c4c27e	[version] update :ci skip all	2021-07-24 20:02:12 +05:30
pukkandan	f703a88055	Release 2021.07.24	2021-07-24 07:03:14 +05:30
pukkandan	a353beba83	[youtube:tab] Extract video duration early Based on: https://github.com/ytdl-org/youtube-dl/pull/29487 by glenn-slayden	2021-07-24 06:59:20 +05:30
pukkandan	052e135029	[youtube] Simplify `_get_text` early	2021-07-24 06:59:20 +05:30
xtkobaandGitHub	cb89cfc14b	[test] Add Python 3.10 (#480 ) Authored-by: pukkandan, xtkoba	2021-07-23 20:32:48 +05:30
pukkandanandGitHub	060ac76257	[test] Use `pytest` instead of `nosetests` (#482 ) `nosetests` is no longer being maintained : https://github.com/nose-devs/nose/issues/1099 and will stop working in py 3.10 as can be seen in #480	2021-07-23 20:18:15 +05:30
pukkandan	063c409dfb	[cookies] Handle errors when importing `keyring` Workaround for #551	2021-07-23 19:58:27 +05:30
Matt BroadwayandGitHub	767b02a99b	[cookies] Handle `sqlite` `ImportError` gracefully (#554 ) Closes #544 Authored by: mbway	2021-07-23 19:56:19 +05:30
pukkandan	f45e6c1126	[downloader] Pass same status object to all `progress_hooks`	2021-07-23 09:46:55 +05:30
pukkandan	3944e7af92	[youtube] Fix subtitles only being extracted from the first client Closes #547	2021-07-23 09:46:55 +05:30
pukkandan	ad34b2951e	Try all clients even if age-gated Reverts: https://github.com/yt-dlp/yt-dlp/pull/536/commits/892e31ce7c5f66418fee26a24eda30e5fe79d901 If some API calls have any issue, saving the state will cause unnecessary errors	2021-07-23 09:46:54 +05:30
pukkandan	c8fa48fd94	[youtube] Disable `get_video_info` age-gate workaround This now seems to be completely dead Closes: #553	2021-07-23 09:46:52 +05:30
coletdjnzandGitHub	2fd226f6a7	[youtube] Fix age-gated videos for API clients when cookies are supplied (#545 ) Fixes #543 Authored by: colethedj	2021-07-22 08:11:04 +00:00
pukkandan	3ba7740dd8	[downloader] Pass `info_dict` to `progress_hook`s	2021-07-22 04:30:11 +05:30
pukkandan	29b208f6f9	[cookies] bugfix Fixes: https://github.com/yt-dlp/yt-dlp/pull/488#discussion_r674352059	2021-07-22 03:00:21 +05:30
pukkandan	e4d666d27b	[version] update :ci skip all	2021-07-22 02:37:51 +05:30
pukkandan	245524e6a3	Release 2021.07.21 and fix some typos Closes #538	2021-07-22 02:33:28 +05:30
pukkandan	9c0d7f4951	[youtube] Make `--extractor-retries` work for more errors Closes #507	2021-07-22 02:32:20 +05:30
pukkandan	e37d0efbd9	Fix bug where `original_url` was not propagated when `_type`=`url`	2021-07-22 02:32:19 +05:30
coletdjnzandGitHub	c926c9541f	[youtube] Add debug message for SAPISID cookie extraction (#540 ) Authored by: colethedj	2021-07-21 20:45:05 +00:00
Matt BroadwayandGitHub	982ee69a74	Add option `--cookies-from-browser` to load cookies from a browser (#488 ) * also adds `--no-cookies-from-browser` Original PR: https://github.com/ytdl-org/youtube-dl/pull/29201 Authored by: mbway	2021-07-22 02:02:49 +05:30
pukkandan	7ea6541124	[youtube] Improve extraction of livestream metadata Modified from and closes #441 Authored by: pukkandan, krichbanana	2021-07-21 20:50:59 +05:30
pukkandan	ae30b84072	Add field `live_status`	2021-07-21 20:50:58 +05:30
pukkandan	cc9d1493c6	bugfix for `50fed816dd`	2021-07-21 20:50:49 +05:30
Philip XuandGitHub	f6755419d1	[douyin] Add extractor (#513 ) Authored-by: pukkandan, pyx	2021-07-21 20:49:27 +05:30
Henrik HeimbuergerandGitHub	145bd631c5	[nebula] Authentication via tokens from cookie jar (#537 ) Closes #496 Co-authored-by: hheimbuerger, TpmKranz	2021-07-21 18:12:43 +05:30
pukkandan	b35496d825	Add `only_once` param for `write_debug`	2021-07-21 18:06:34 +05:30
pukkandan	352d63fdb5	[utils] Improve `traverse_obj`	2021-07-21 11:30:06 +05:30
pukkandanandGitHub	11f9be0912	[youtube] Extract data from multiple clients (#536 ) * `player_client` accepts multiple clients * default `player_client` = `android,web` * music clients can be specifically requested * Add IOS `player_client` * Hide live dash since they can't be downloaded Closes #501 Authored-by: pukkandan, colethedj	2021-07-21 09:22:34 +05:30
pukkandan	c84aeac6b5	Add `only_once` param for `report_warning` Related: https://github.com/yt-dlp/yt-dlp/pull/488#discussion_r667527297	2021-07-21 01:39:58 +05:30
pukkandan	50fed816dd	Errors in playlist extraction should obey `--ignore-errors` Related: https://github.com/yt-dlp/yt-dlp/issues/535#issuecomment-883277272, https://github.com/yt-dlp/yt-dlp/issues/518#issuecomment-881794754	2021-07-21 01:04:53 +05:30
coletdjnzandGitHub	a1a7907bc0	[youtube] Fix controversial videos when requested via API (#533 ) Closes: https://github.com/yt-dlp/yt-dlp/issues/511#issuecomment-883024350 Authored by: colethedj	2021-07-20 23:31:28 +05:30
pukkandan	d61fc64618	[youtube:tab] Fix channels tab	2021-07-20 23:22:34 +05:30
pukkandan	6586bca9b9	[utils] Fix LazyList for Falsey values	2021-07-20 23:22:26 +05:30
pukkandan	da503b7a52	[youtube] Make `parse_time_text` and `_extract_chapters` non-fatal Related: #532, `7c365c2109`	2021-07-20 07:22:26 +05:30
pukkandan	7c365c2109	[youtube] Sanity check `chapters` (and refactor related code) Closes #520	2021-07-20 05:39:02 +05:30
pukkandan	3f698246b2	Rename `NOTE` in `-F` to `MORE INFO` since it's often confused to be the same as `format_note`	2021-07-20 05:30:28 +05:30
pukkandan	cca80fe611	[youtube] Extract even more thumbnails and reduce testing * Also fix bug where `_test_url` was being ignored Ref: https://stackoverflow.com/a/20542029 Related: #340	2021-07-20 03:46:06 +05:30
pukkandan	c634ad2a3c	[compat] Remove unnecessary code	2021-07-20 03:46:05 +05:30
pukkandan	8f3343809e	[utils] Improve `traverse_obj` * Allow skipping a level: `traverse_obj([{k:v1}, {k:v2}], (None, k))` => `[v1, v2]` * Make keys variadic: `traverse_obj(obj, k1: str, k2: str)` => `traverse_obj(obj, (k1,), (k2,))` * Fetch from multiple keys: `traverse_obj([{k1:[1], k2:[2], k3:[3]}], (0, (k1, k2), 0))` => `[1, 2]` TODO: Add tests	2021-07-20 02:42:11 +05:30
pukkandan	0ba692acc8	[youtube] Extract more thumbnails * The thumbnail URLs are hard-coded and their actual existence is tested lazily * Added option `--no-check-formats` to not test them Closes #340, Related: #402, #337, https://github.com/ytdl-org/youtube-dl/issues/29049	2021-07-20 02:42:11 +05:30
pukkandan	d9488f69c1	[crunchyroll:playlist] Force http Closes #495	2021-07-20 02:42:11 +05:30
pukkandan	dce8743677	[docs] fix default of multistreams	2021-07-19 23:47:57 +05:30
pukkandan	5520aa2dc9	Add option `--exec-before-download` Closes #530	2021-07-19 23:47:45 +05:30
mzbaulhaqueandGitHub	8d9b902243	[pornflip] Add new extractor (#523 ) Authored-by: mzbaulhaque	2021-07-19 23:46:21 +05:30
coletdjnzandGitHub	fe93e2c4cf	[youtube] misc cleanup and bug fixes (#505 ) * Update some `_extract_response` calls to keep them consistent * Cleanup continuation extraction related code using new API format * Improve `_extract_account_syncid` to support multiple parameters * Generalize `get_text` and related functions into one * Update `INNERTUBE_CONTEXT_CLIENT_NAME` with integer values Authored by: colethedj	2021-07-19 10:25:07 +05:30
coletdjnzandGitHub	314ee30548	[youtube] Fix session index extraction and headers for non-web player clients (#526 ) Fixes #522	2021-07-18 06:23:32 +00:00
coletdjnzandGitHub	34917076ad	[youtube] Fix authentication when using multiple accounts `SESSION_INDEX` in `ytcfg` is the index of the active account and should be sent as `X-Goog-AuthUser` header Closes #518 Authored by @colethedj	2021-07-17 11:50:05 +05:30
The Hatsune DaishiandGitHub	ccc7795ca3	[yahoo:gyao:player] Relax `_VALID_URL` (#503 ) Authored by: nao20010128nao	2021-07-16 20:06:53 +05:30
Felix SandGitHub	da1c94ee45	[generic] Extract previously missed subtitles (#515 ) * [generic] Extract subtitles in cases missed previously * [common] Detect discarded subtitles in SMIL manifests * [generic] Extract everything in the SMIL manifest Authored by: fstirlitz	2021-07-16 19:52:56 +05:30
pukkandan	3b297919e0	Revert "Merge webm formats into mkv if thumbnails are to be embedded (#173 )" This reverts commit `4d971a16b8` by @damianoamatruda Closes #500 This was wrongly checking for `write_thumbnail`	2021-07-15 23:34:52 +05:30
coletdjnzandGitHub	47193e0298	[youtube:tab] Extract playlist availability (#504 ) Authored by: colethedj	2021-07-15 02:42:30 +00:00
coletdjnzandGitHub	49bd8c66d3	[youtube:comments] Improve comment vote count parsing (fixes #506 ) (#508 ) Authored by: colethedj	2021-07-14 23:24:42 +00:00
Felix SandGitHub	182b6ae8a6	[RTP] Fix extraction and add subtitles (#497 ) Authored by: fstirlitz	2021-07-14 05:06:18 +05:30
felixandpukkandan	c843e68588	[utils] Improve `js_to_json` comment regex Capture the newline character as part of a single-line comment From #497, Authored by: fstirlitz	2021-07-14 05:02:43 +05:30
felixandpukkandan	198f7ea89e	[extractor] Allow extracting multiple groups in `_search_regex` From #497, Authored by: fstirlitz	2021-07-14 05:02:42 +05:30
coletdjnzandGitHub	c888ffb95a	[youtube] Use android client as default and add age-gate bypass for it (#492 ) Authored by: colethedj	2021-07-14 03:58:51 +05:30
coletdjnzandGitHub	9752433221	[youtube:comments] Fix `is_favorited` (#491 ) Authored by colethedj	2021-07-12 06:50:03 +05:30
pukkandan	f0ff9979c6	[vlive] Extract thumbnail directly in addition to the one from Naver Closes #477	2021-07-12 06:07:23 +05:30
pukkandan	501dd1ad55	[metadatafromfield] Do not detect numbers as field names Related: https://github.com/yt-dlp/yt-dlp/issues/486#issuecomment-877820394	2021-07-12 05:20:12 +05:30
pukkandan	75722b037d	[webtt] Fix timestamps Closes #474	2021-07-12 05:20:12 +05:30
coletdjnzandGitHub	2d6659b9ea	[youtube:comments] Move comment extraction to new API (#466 ) Closes #438, #481, #485 Authored by: colethedj	2021-07-12 04:48:40 +05:30
Kevin O'ConnorandGitHub	c5370857b3	[BravoTV] Improve metadata extraction (#483 ) Authored by: kevinoconnor7	2021-07-11 16:36:26 +05:30
pukkandan	00034c146a	[embedthumbnail] Fix `_get_thumbnail_resolution`	2021-07-11 04:46:53 +05:30
pukkandan	325ebc1703	Improve `traverse_obj`	2021-07-11 04:46:53 +05:30
pukkandan	7dde84f3c9	[FFmpegMetadata] Add language of each stream and some refactoring	2021-07-11 04:46:52 +05:30
pukkandan	6606817a86	[utils] Add `variadic`	2021-07-11 04:46:51 +05:30
zackmark29 GitHubzackmark29, pukkandan	73d829c144	[VIKI] Rewrite extractors (#475 ) Closes #462 Also added extractor-arg `video_types` to `vikichannel` Co-authored-by: zackmark29, pukkandan	2021-07-10 02:08:09 +05:30
pukkandan	60bdb7bd9e	[youtube] Fix sorting of 3gp format	2021-07-08 22:33:33 +05:30
pukkandan	4bb6b02f93	Improve `extractor_args` parsing	2021-07-08 21:22:35 +05:30
pukkandan	b5ac45b197	Fix selectors `all`, `mergeall` and add tests Bug from: `981052c9c6`	2021-07-07 21:10:43 +05:30
pukkandan	38a40c9e16	[version] update :ci skip all	2021-07-07 05:43:58 +05:30