Release 2021.04.22

[documentation] Fix typos
[ci] Disable fail-fast
2026-08-09 05:38:38 +03:00 · 2021-04-22 16:58:28 +05:30 · 2021-04-22 16:54:44 +05:30 · 2021-04-22 16:54:41 +05:30 · 2021-04-22 16:54:08 +05:30 · 2021-04-22 16:54:07 +05:30
82 changed files with 1287 additions and 507 deletions
@@ -21,7 +21,7 @@ assignees: ''

 <!--
 Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.04.03. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
+- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.04.11. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
 - Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
 - Make sure that all URLs and arguments with special characters are properly quoted or escaped as explained in https://github.com/yt-dlp/yt-dlp.
 - Search the bugtracker for similar issues: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
@@ -29,7 +29,7 @@ Carefully read and work through this check list in order to prevent the most com
 -->

 - [ ] I'm reporting a broken site support
- [ ] I've verified that I'm running yt-dlp version **2021.04.03**
+- [ ] I've verified that I'm running yt-dlp version **2021.04.11**
 - [ ] I've checked that all provided URLs are alive and playable in a browser
 - [ ] I've checked that all URLs and arguments with special characters are properly quoted or escaped
 - [ ] I've searched the bugtracker for similar issues including closed ones
@@ -44,7 +44,7 @@ Add the `-v` flag to your command line you run yt-dlp with (`yt-dlp -v <your com
 [debug] User config: []
 [debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
 [debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
- [debug] yt-dlp version 2021.04.03
+ [debug] yt-dlp version 2021.04.11
 [debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
 [debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
 [debug] Proxy map: {}
@@ -21,7 +21,7 @@ assignees: ''

 <!--
 Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.04.03. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
+- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.04.11. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
 - Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
 - Make sure that site you are requesting is not dedicated to copyright infringement, see https://github.com/yt-dlp/yt-dlp. yt-dlp does not support such sites. In order for site support request to be accepted all provided example URLs should not violate any copyrights.
 - Search the bugtracker for similar site support requests: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
@@ -29,7 +29,7 @@ Carefully read and work through this check list in order to prevent the most com
 -->

 - [ ] I'm reporting a new site support request
- [ ] I've verified that I'm running yt-dlp version **2021.04.03**
+- [ ] I've verified that I'm running yt-dlp version **2021.04.11**
 - [ ] I've checked that all provided URLs are alive and playable in a browser
 - [ ] I've checked that none of provided URLs violate any copyrights
 - [ ] I've searched the bugtracker for similar site support requests including closed ones
@@ -21,13 +21,13 @@ assignees: ''

 <!--
 Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.04.03. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
+- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.04.11. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
 - Search the bugtracker for similar site feature requests: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
 - Finally, put x into all relevant boxes like this [x] (Dont forget to delete the empty space)
 -->

 - [ ] I'm reporting a site feature request
- [ ] I've verified that I'm running yt-dlp version **2021.04.03**
+- [ ] I've verified that I'm running yt-dlp version **2021.04.11**
 - [ ] I've searched the bugtracker for similar site feature requests including closed ones


@@ -21,7 +21,7 @@ assignees: ''

 <!--
 Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.04.03. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
+- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.04.11. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
 - Make sure that all provided video/audio/playlist URLs (if any) are alive and playable in a browser.
 - Make sure that all URLs and arguments with special characters are properly quoted or escaped as explained in https://github.com/yt-dlp/yt-dlp.
 - Search the bugtracker for similar issues: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
@@ -30,7 +30,7 @@ Carefully read and work through this check list in order to prevent the most com
 -->

 - [ ] I'm reporting a broken site support issue
- [ ] I've verified that I'm running yt-dlp version **2021.04.03**
+- [ ] I've verified that I'm running yt-dlp version **2021.04.11**
 - [ ] I've checked that all provided URLs are alive and playable in a browser
 - [ ] I've checked that all URLs and arguments with special characters are properly quoted or escaped
 - [ ] I've searched the bugtracker for similar bug reports including closed ones
@@ -46,7 +46,7 @@ Add the `-v` flag to your command line you run yt-dlp with (`yt-dlp -v <your com
 [debug] User config: []
 [debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
 [debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
- [debug] yt-dlp version 2021.04.03
+ [debug] yt-dlp version 2021.04.11
 [debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
 [debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
 [debug] Proxy map: {}
@@ -21,13 +21,13 @@ assignees: ''

 <!--
 Carefully read and work through this check list in order to prevent the most common mistakes and misuse of yt-dlp:
- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.04.03. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
+- First of, make sure you are using the latest version of yt-dlp. Run `yt-dlp --version` and ensure your version is 2021.04.11. If it's not, see https://github.com/yt-dlp/yt-dlp on how to update. Issues with outdated version will be REJECTED.
 - Search the bugtracker for similar feature requests: https://github.com/yt-dlp/yt-dlp. DO NOT post duplicates.
 - Finally, put x into all relevant boxes like this [x] (Dont forget to delete the empty space)
 -->

 - [ ] I'm reporting a feature request
- [ ] I've verified that I'm running yt-dlp version **2021.04.03**
+- [ ] I've verified that I'm running yt-dlp version **2021.04.11**
 - [ ] I've searched the bugtracker for similar feature requests including closed ones


@@ -6,7 +6,7 @@ jobs:
    if: "!contains(github.event.head_commit.message, 'ci skip')"
    runs-on: ${{ matrix.os }}
    strategy:
-      fail-fast: true
+      fail-fast: false
      matrix:
        os: [ubuntu-18.04]
        # TODO: python 2.6
@@ -37,4 +37,6 @@ CXwudi
 xtkoba
 llacb47
 hheimbuerger
-B0pol
+B0pol
+lkho
+fstirlitz
@@ -19,6 +19,45 @@
 -->


+### 2021.04.22
+* **Improve output template:**
+    * Objects can be traversed like `%(field.key1.key2)s`
+    * An offset can be added to numeric fields as `%(field+N)s`
+    * Deprecates `--autonumber-start`
+* **Improve `--sub-langs`:**
+    * Treat `--sub-langs` entries as regex
+    * `all` can be used to refer to all the subtitles
+    * language codes can be prefixed with `-` to exclude it
+    * Deprecates `--all-subs`
+* Add option `--ignore-no-formats-error` to ignore the "no video format" and similar errors
+* Add option `--skip-playlist-after-errors` to skip the rest of a playlist after a given number of errors are encountered
+* Merge youtube-dl: Upto [commit/7e8b3f9](https://github.com/ytdl-org/youtube-dl/commit/7e8b3f9439ebefb3a3a4e5da9c0bd2b595976438)
+* [downloader] Fix bug in downloader selection
+* [BilibiliChannel] Fix pagination by [nao20010128nao](https://github.com/nao20010128nao) and[pukkandan](https://github.com/pukkandan)
+* [rai] Add support for http formats by [nixxo](https://github.com/nixxo)
+* [TubiTv] Add TubiTvShowIE by [Ashish0804](https://github.com/Ashish0804)
+* [twitcasting] Fix extractor
+* [viu:ott] Fix extractor and support series by [lkho](https://github.com/lkho) and[pukkandan](https://github.com/pukkandan)
+* [youtube:tab] Show unavailable videos in playlists by [colethedj](https://github.com/colethedj)
+* [youtube:tab] Reload with unavailable videos for all playlists
+* [youtube] Ignore invalid stretch ratio
+* [youtube] Improve channel syncid extraction to support ytcfg by [colethedj](https://github.com/colethedj)
+* [youtube] Standardize API calls for tabs, mixes and search by [colethedj](https://github.com/colethedj)
+* [youtube] Bugfix in `_extract_ytcfg`
+* [mildom:user:vod] Download only necessary amount of pages
+* [mildom] Remove proxy completely by [fstirlitz](https://github.com/fstirlitz)
+* [go] Fix `_VALID_URL`
+* [MetadataFromField] Improve regex and add tests
+* [Exec] Ensure backward compatibility when the command contains `%`
+* [extractor] Fix inconsistent use of `report_warning`
+* Ensure `mergeall` selects best format when multistreams are disabled
+* Improve the yt-dlp.sh script by [fstirlitz](https://github.com/fstirlitz)
+* [lazy_extractor] Do not load plugins
+* [ci] Disable fail-fast
+* [documentation] Clarify which deprecated options still work
+* [documentation] Fix typos
+
+
 ### 2021.04.11
 * Add option `--convert-thumbnails` (only jpg currently supported)
 * Format selector `mergeall` to download and merge all formats
@@ -35,7 +74,7 @@
 * [youtube] Fix thumbnail URL
 * [youtube] Parse API parameters from initial webpage by [colethedj](https://github.com/colethedj)
 * [youtube] Extract comments' approximate timestamp by [colethedj](https://github.com/colethedj)
-* [youtube] Fix `\_extract_alerts`
+* [youtube] Fix alert extraction
 * [bilibili] Fix uploader
 * [utils] Add `datetime_from_str` and `datetime_add_months` by [colethedj](https://github.com/colethedj)
 * Run some `postprocessors` before actual download
@@ -3,17 +3,16 @@
 # YT-DLP
 A command-line program to download videos from YouTube and many other [video platforms](supportedsites.md)

-<!-- GHA doesnot have for-the-badge style
+<!-- GHA doesn't have for-the-badge style
 [![CI Status](https://github.com/yt-dlp/yt-dlp/workflows/Core%20Tests/badge.svg?branch=master)](https://github.com/yt-dlp/yt-dlp/actions)
 -->
 [![Release version](https://img.shields.io/github/v/release/yt-dlp/yt-dlp?color=brightgreen&label=Release&style=for-the-badge)](https://github.com/yt-dlp/yt-dlp/releases/latest)
 [![License: Unlicense](https://img.shields.io/badge/License-Unlicense-blue.svg?style=for-the-badge)](LICENSE)
 [![Doc Status](https://readthedocs.org/projects/yt-dlp/badge/?version=latest&style=for-the-badge)](https://yt-dlp.readthedocs.io)
-[![Discord](https://img.shields.io/discord/807245652072857610?color=blue&label=discord&logo=discord&style=for-the-badge)](https://discord.gg/H5MNcFW63r)  
-
+[![Discord](https://img.shields.io/discord/807245652072857610?color=blue&label=discord&logo=discord&style=for-the-badge)](https://discord.gg/H5MNcFW63r)
 [![Commits](https://img.shields.io/github/commit-activity/m/yt-dlp/yt-dlp?label=commits&style=for-the-badge)](https://github.com/yt-dlp/yt-dlp/commits)
 [![Last Commit](https://img.shields.io/github/last-commit/yt-dlp/yt-dlp/master?style=for-the-badge)](https://github.com/yt-dlp/yt-dlp/commits)
-[![Downloads](https://img.shields.io/github/downloads/yt-dlp/yt-dlp/total?style=for-the-badge)](https://github.com/yt-dlp/yt-dlp/releases/latest)
+[![Downloads](https://img.shields.io/github/downloads/yt-dlp/yt-dlp/total?style=for-the-badge&color=blue)](https://github.com/yt-dlp/yt-dlp/releases/latest)
 [![PyPi Downloads](https://img.shields.io/pypi/dm/yt-dlp?label=PyPi&style=for-the-badge)](https://pypi.org/project/yt-dlp)

 </div>
@@ -22,7 +21,7 @@ yt-dlp is a [youtube-dl](https://github.com/ytdl-org/youtube-dl) fork based on t

 * [NEW FEATURES](#new-features)
 * [INSTALLATION](#installation)
-    * [Dependancies](#dependancies)
+    * [Dependencies](#dependencies)
    * [Update](#update)
    * [Compile](#compile)
 * [USAGE AND OPTIONS](#usage-and-options)
@@ -66,13 +65,14 @@ The major new features from the latest release of [blackjack4494/yt-dlc](https:/

 * **[Format Sorting](#sorting-formats)**: The default format sorting options have been changed so that higher resolution and better codecs will be now preferred instead of simply using larger bitrate. Furthermore, you can now specify the sort order using `-S`. This allows for much easier format selection that what is possible by simply using `--format` ([examples](#format-selection-examples))

-* **Merged with youtube-dl v2021.04.07**: You get all the latest features and patches of [youtube-dl](https://github.com/ytdl-org/youtube-dl) in addition to all the features of [youtube-dlc](https://github.com/blackjack4494/yt-dlc)
+* **Merged with youtube-dl v2021.04.17**: You get all the latest features and patches of [youtube-dl](https://github.com/ytdl-org/youtube-dl) in addition to all the features of [youtube-dlc](https://github.com/blackjack4494/yt-dlc)

 * **Merged with animelover1984/youtube-dl**: You get most of the features and improvements from [animelover1984/youtube-dl](https://github.com/animelover1984/youtube-dl) including `--get-comments`, `BiliBiliSearch`, `BilibiliChannel`, Embedding thumbnail in mp4/ogg/opus, Playlist infojson etc. Note that the NicoNico improvements are not available. See [#31](https://github.com/yt-dlp/yt-dlp/pull/31) for details.

 * **Youtube improvements**:
-    * All Youtube Feeds (`:ytfav`, `:ytwatchlater`, `:ytsubs`, `:ythistory`, `:ytrec`) works correctly and supports downloading multiple pages of content
-    * Youtube search (`ytsearch:`, `ytsearchdate:`) along with Search URLs works correctly
+    * All Youtube Feeds (`:ytfav`, `:ytwatchlater`, `:ytsubs`, `:ythistory`, `:ytrec`) works and supports downloading multiple pages of content
+    * Youtube search (`ytsearch:`, `ytsearchdate:`) along with Search URLs work
+    * Youtube mixes supports downloading multiple pages of content
    * Redirect channel's home URL automatically to `/video` to preserve the old behaviour

 * **Split video by chapters**: Videos can be split into multiple files based on chapters using `--split-chapters`
@@ -83,7 +83,7 @@ The major new features from the latest release of [blackjack4494/yt-dlc](https:/

 * **New extractors**: AnimeLab, Philo MSO, Rcs, Gedi, bitwave.tv, mildom, audius, zee5, mtv.it, wimtv, pluto.tv, niconico users, discoveryplus.in, mediathek, NFHSNetwork, nebula

-* **Fixed extractors**: archive.org, roosterteeth.com, skyit, instagram, itv, SouthparkDe, spreaker, Vlive, akamai, ina, rumble, tennistv, amcnetworks, la7 podcasts, linuxacadamy, nitter
+* **Fixed extractors**: archive.org, roosterteeth.com, skyit, instagram, itv, SouthparkDe, spreaker, Vlive, akamai, ina, rumble, tennistv, amcnetworks, la7 podcasts, linuxacadamy, nitter, twitcasting, viu

 * **Multiple paths and output templates**: You can give different [output templates](#output-template) and download paths for different types of files. You can also set a temporary path where intermediary files are downloaded to using `--paths` (`-P`)

@@ -132,11 +132,10 @@ sudo aria2c https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp -o
 sudo chmod a+rx /usr/local/bin/yt-dlp
 ```

-### DEPENDANCIES
-
+### DEPENDENCIES
 Python versions 2.6, 2.7, or 3.2+ are currently supported. However, 3.2+ is strongly recommended and python2 support will be deprecated in the future.

-Although there are no required dependancies, `ffmpeg` and `ffprobe` are highly recommended. Other optional dependancies are `sponskrub`, `AtomicParsley`, `mutagen`, `pycryptodome` and any of the supported external downloaders. Note that the windows releases are already built with the python interpreter, mutagen and pycryptodome included.
+Although there are no required dependencies, `ffmpeg` and `ffprobe` are highly recommended. Other optional dependencies are `sponskrub`, `AtomicParsley`, `mutagen`, `pycryptodome` and any of the supported external downloaders. Note that the windows releases are already built with the python interpreter, mutagen and pycryptodome included.

 ### UPDATE
 You can use `yt-dlp -U` to update if you are using the provided release.
@@ -149,7 +148,7 @@ To build the Windows executable, you must have pyinstaller (and optionally mutag

    python -m pip install --upgrade pyinstaller mutagen pycryptodome

-Once you have all the necessary dependancies installed, just run `py pyinst.py`. The executable will be built for the same architecture (32/64 bit) as the python used to build it. It is strongly reccomended to use python3 although python2.6+ is supported.
+Once you have all the necessary dependencies installed, just run `py pyinst.py`. The executable will be built for the same architecture (32/64 bit) as the python used to build it. It is strongly recommended to use python3 although python2.6+ is supported.

 You can also build the executable without any version info or metadata by using:

@@ -166,7 +165,7 @@ Then simply run `make`. You can also run `make yt-dlp` instead to compile only t
    yt-dlp [OPTIONS] [--] URL [URL...]

 `Ctrl+F` is your friend :D
-<!-- Autogenerated -->
+<!-- Auto generated -->

 ## General Options:
    -h, --help                       Print this help text and exit
@@ -306,6 +305,8 @@ Then simply run `make`. You can also run `make yt-dlp` instead to compile only t
                                     a file that is in the archive
    --break-on-reject                Stop the download process when encountering
                                     a file that has been filtered out
+    --skip-playlist-after-errors N   Number of allowed failures until the rest
+                                     of the playlist is skipped
    --no-download-archive            Do not use archive file (default)

 ## Download Options:
@@ -394,8 +395,6 @@ Then simply run `make`. You can also run `make yt-dlp` instead to compile only t
    --output-na-placeholder TEXT     Placeholder value for unavailable meta
                                     fields in output filename template
                                     (default: "NA")
-    --autonumber-start NUMBER        Specify the start value for %(autonumber)s
-                                     (default is 1)
    --restrict-filenames             Restrict filenames to only ASCII
                                     characters, and avoid "&" and spaces in
                                     filenames
@@ -415,7 +414,7 @@ Then simply run `make`. You can also run `make yt-dlp` instead to compile only t
    -c, --continue                   Resume partially downloaded files/fragments
                                     (default)
    --no-continue                    Do not resume partially downloaded
-                                     fragments. If the file is unfragmented,
+                                     fragments. If the file is not fragmented,
                                     restart download of the entire file
    --part                           Use .part files instead of writing directly
                                     into output file (default)
@@ -487,6 +486,12 @@ Then simply run `make`. You can also run `make yt-dlp` instead to compile only t
    --no-warnings                    Ignore warnings
    -s, --simulate                   Do not download the video and do not write
                                     anything to disk
+    --ignore-no-formats-error        Ignore "No video formats" error. Usefull
+                                     for extracting metadata even if the video
+                                     is not actually available for download
+                                     (experimental)
+    --no-ignore-no-formats-error     Throw error when no downloadable video
+                                     formats are found (default)
    --skip-download                  Do not download the video but write all
                                     related files (Alias: --no-download)
    -g, --get-url                    Simulate, quiet but print URL
@@ -507,8 +512,8 @@ Then simply run `make`. You can also run `make yt-dlp` instead to compile only t
    --print-json                     Be quiet and print the video information as
                                     JSON (video is still being downloaded)
    --force-write-archive            Force download archive entries to be
-                                     written as far as no errors occur,even if
-                                     -s or another simulation switch is used
+                                     written as far as no errors occur, even if
+                                     -s or another simulation option is used
                                     (Alias: --force-download-archive)
    --newline                        Output progress bar as new lines
    --no-progress                    Do not print progress bar
@@ -583,7 +588,7 @@ Then simply run `make`. You can also run `make yt-dlp` instead to compile only t
                                     container format. One of mkv, mp4, ogg,
                                     webm, flv. Ignored if no merge is required
    --allow-unplayable-formats       Allow unplayable formats to be listed and
-                                     downloaded. All video postprocessing will
+                                     downloaded. All video post-processing will
                                     also be turned off
    --no-allow-unplayable-formats    Do not allow unplayable formats to be
                                     listed or downloaded (default)
@@ -595,15 +600,17 @@ Then simply run `make`. You can also run `make yt-dlp` instead to compile only t
                                     (Alias: --write-automatic-subs)
    --no-write-auto-subs             Do not write auto-generated subtitles
                                     (default) (Alias: --no-write-automatic-subs)
-    --all-subs                       Download all the available subtitles of the
-                                     video
    --list-subs                      List all available subtitles for the video
    --sub-format FORMAT              Subtitle format, accepts formats
                                     preference, for example: "srt" or
                                     "ass/srt/best"
-    --sub-langs LANGS                Languages of the subtitles to download
-                                     (optional) separated by commas, use --list-
-                                     subs for available language tags
+    --sub-langs LANGS                Languages of the subtitles to download (can
+                                     be regex) or "all" separated by commas.
+                                     (Eg: --sub-langs en.*,ja) You can prefix
+                                     the language code with a "-" to exempt it
+                                     from the requested languages. (Eg: --sub-
+                                     langs all,-live_chat) Use --list-subs for a
+                                     list of available language tags

 ## Authentication Options:
    -u, --username USERNAME          Login with this account ID
@@ -830,9 +837,21 @@ The `-o` option is used to indicate a template for the output file names while `

 **tl;dr:** [navigate me to examples](#output-template-examples).

-The basic usage of `-o` is not to set any template arguments when downloading a single file, like in `yt-dlp -o funny_video.flv "https://some/video"` (hard-coding file extension like this is not recommended). However, it may contain special sequences that will be replaced when downloading each video. The special sequences may be formatted according to [python string formatting operations](https://docs.python.org/2/library/stdtypes.html#string-formatting). For example, `%(NAME)s` or `%(NAME)05d`. To clarify, that is a percent symbol followed by a name in parentheses, followed by formatting operations. Date/time fields can also be formatted according to [strftime formatting](https://docs.python.org/3/library/datetime.html#strftime-and-strptime-format-codes) by specifying it inside the parantheses separated from the field name using a `>`. For example, `%(duration>%H-%M-%S)s`.
+The simplest usage of `-o` is not to set any template arguments when downloading a single file, like in `yt-dlp -o funny_video.flv "https://some/video"` (hard-coding file extension like this is _not_ recommended and could break some post-processing).

-Additionally, you can set different output templates for the various metadata files separately from the general output template by specifying the type of file followed by the template separated by a colon `:`. The different filetypes supported are `subtitle`, `thumbnail`, `description`, `annotation`, `infojson`, `pl_description`, `pl_infojson`, `chapter`. For example, `-o '%(title)s.%(ext)s' -o 'thumbnail:%(title)s\%(title)s.%(ext)s'`  will put the thumbnails in a folder with the same name as the video.
+It may however also contain special sequences that will be replaced when downloading each video. The special sequences may be formatted according to [python string formatting operations](https://docs.python.org/2/library/stdtypes.html#string-formatting). For example, `%(NAME)s` or `%(NAME)05d`. To clarify, that is a percent symbol followed by a name in parentheses, followed by formatting operations.
+
+The field names themselves (the part inside the parenthesis) can also have some special formatting:
+1. **Date/time Formatting**: Date/time fields can be formatted according to [strftime formatting](https://docs.python.org/3/library/datetime.html#strftime-and-strptime-format-codes) by specifying it separated from the field name using a `>`. Eg: `%(duration>%H-%M-%S)s` or `%(upload_date>%Y-%m-%d)s`
+2. **Offset numbers**: Numeric fields can have an initial offset specified by using a `+` separator. Eg: `%(playlist_index+10)03d`. This can also be used in conjunction with the date-time formatting. Eg: `%(epoch+-3600>%H-%M-%S)s`
+3. **Object traversal**: The dictionaries and lists available in metadata can be traversed by using a `.` (dot) separator. Eg: `%(tags.0)s` or `%(subtitles.en.-1.ext)`. Note that the fields that become available using this method are not listed below. Use `-j` to see such fields
+
+To summarize, the general syntax for a field is:
+```
+%(name[.keys][+offset][>strf])[flags][width][.precision][length]type
+```
+
+Additionally, you can set different output templates for the various metadata files separately from the general output template by specifying the type of file followed by the template separated by a colon `:`. The different file types supported are `subtitle`, `thumbnail`, `description`, `annotation`, `infojson`, `pl_description`, `pl_infojson`, `chapter`. For example, `-o '%(title)s.%(ext)s' -o 'thumbnail:%(title)s\%(title)s.%(ext)s'`  will put the thumbnails in a folder with the same name as the video.

 The available fields are:

@@ -989,7 +1008,7 @@ The general syntax for format selection is `-f FORMAT` (or `--format FORMAT`) wh

 **tl;dr:** [navigate me to examples](#format-selection-examples).

-The simplest case is requesting a specific format, for example with `-f 22` you can download the format with format code equal to 22. You can get the list of available format codes for particular video using `--list-formats` or `-F`. Note that these format codes are extractor specific. 
+The simplest case is requesting a specific format, for example with `-f 22` you can download the format with format code equal to 22. You can get the list of available format codes for particular video using `--list-formats` or `-F`. Note that these format codes are extractor specific.

 You can also use a file extension (currently `3gp`, `aac`, `flv`, `m4a`, `mp3`, `mp4`, `ogg`, `wav`, `webm` are supported) to download the best quality format of a particular file extension served as a single file, e.g. `-f webm` will download the best quality format with the `webm` extension served as a single file.

@@ -1010,7 +1029,7 @@ You can also use special names to select particular edge case formats:
 - `ba*`, `bestaudio*`: Select the best quality format that contains audio. It may also contain video. Equivalent to `best*[acodec!=none]`
 - `wa*`, `worstaudio*`: Select the worst quality format that contains audio. It may also contain video. Equivalent to `worst*[acodec!=none]`

-For example, to download the worst quality video-only format you can use `-f worstvideo`. It is however recomended not to use `worst` and related options. When your format selector is `worst`, the format which is worst in all respects is selected. Most of the time, what you actually want is the video with the smallest filesize instead. So it is generally better to use `-f best -S +size,+br,+res,+fps` instead of `-f worst`. See [sorting formats](#sorting-formats) for more details.
+For example, to download the worst quality video-only format you can use `-f worstvideo`. It is however recommended not to use `worst` and related options. When your format selector is `worst`, the format which is worst in all respects is selected. Most of the time, what you actually want is the video with the smallest filesize instead. So it is generally better to use `-f best -S +size,+br,+res,+fps` instead of `-f worst`. See [sorting formats](#sorting-formats) for more details.

 You can select the n'th best format of a type by using `best<type>.<n>`. For example, `best.2` will select the 2nd best combined format. Similarly, `bv*.3` will select the 3rd best format that contains a video stream.

@@ -1047,7 +1066,7 @@ Also filtering work for comparisons `=` (equals), `^=` (starts with), `$=` (ends

 Any string comparison may be prefixed with negation `!` in order to produce an opposite comparison, e.g. `!*=` (does not contain).

-Note that none of the aforementioned meta fields are guaranteed to be present since this solely depends on the metadata obtained by particular extractor, i.e. the metadata offered by the video hoster. Any other field made available by the extractor can also be used for filtering.
+Note that none of the aforementioned meta fields are guaranteed to be present since this solely depends on the metadata obtained by particular extractor, i.e. the metadata offered by the website. Any other field made available by the extractor can also be used for filtering.

 Formats for which the value is not known are excluded unless you put a question mark (`?`) after the operator. You can combine format filters, so `-f "[height<=?720][tbr>500]"` selects up to 720p videos (or videos where the height is not known) with a bitrate of at least 500 KBit/s. You can also use the filters with `all` to download all formats that satisfy the filter. For example, `-f "all[vcodec=none]"` selects all audio-only formats.

@@ -1063,7 +1082,7 @@ You can change the criteria for being considered the `best` by using `-S` (`--fo
 - `lang`: Language preference as given by the extractor
 - `quality`: The quality of the format as given by the extractor
 - `source`: Preference of the source as given by the extractor
- - `proto`: Protocol used for download (`https`/`ftps` > `http`/`ftp` > `m3u8-native` > `m3u8` > `http-dash-segments` > other > `mms`/`rtsp` > unknown > `f4f`/`f4m`)
+ - `proto`: Protocol used for download (`https`/`ftps` > `http`/`ftp` > `m3u8_native` > `m3u8` > `http_dash_segments` > other > `mms`/`rtsp` > unknown > `f4f`/`f4m`)
 - `vcodec`: Video Codec (`av01` > `vp9.2` > `vp9` > `h265` > `h264` > `vp8` > `h263` > `theora` > other > unknown)
 - `acodec`: Audio Codec (`opus` > `vorbis` > `aac` > `mp4a` > `mp3` > `ac3` > `dts` > other > unknown)
 - `codec`: Equivalent to `vcodec,acodec`
@@ -1083,11 +1102,11 @@ You can change the criteria for being considered the `best` by using `-S` (`--fo
 - `br`: Equivalent to using `tbr,vbr,abr`
 - `asr`: Audio sample rate in Hz

-Note that any other **numerical** field made available by the extractor can also be used. All fields, unless specified otherwise, are sorted in decending order. To reverse this, prefix the field with a `+`. Eg: `+res` prefers format with the smallest resolution. Additionally, you can suffix a prefered value for the fields, separated by a `:`. Eg: `res:720` prefers larger videos, but no larger than 720p and the smallest video if there are no videos less than 720p. For `codec` and `ext`, you can provide two prefered values, the first for video and the second for audio. Eg: `+codec:avc:m4a` (equivalent to `+vcodec:avc,+acodec:m4a`) sets the video codec preference to `h264` > `h265` > `vp9` > `vp9.2` > `av01` > `vp8` > `h263` > `theora` and audio codec preference to `mp4a` > `aac` > `vorbis` > `opus` > `mp3` > `ac3` > `dts`. You can also make the sorting prefer the nearest values to the provided by using `~` as the delimiter. Eg: `filesize~1G` prefers the format with filesize closest to 1 GiB.
+Note that any other **numerical** field made available by the extractor can also be used. All fields, unless specified otherwise, are sorted in descending order. To reverse this, prefix the field with a `+`. Eg: `+res` prefers format with the smallest resolution. Additionally, you can suffix a preferred value for the fields, separated by a `:`. Eg: `res:720` prefers larger videos, but no larger than 720p and the smallest video if there are no videos less than 720p. For `codec` and `ext`, you can provide two preferred values, the first for video and the second for audio. Eg: `+codec:avc:m4a` (equivalent to `+vcodec:avc,+acodec:m4a`) sets the video codec preference to `h264` > `h265` > `vp9` > `vp9.2` > `av01` > `vp8` > `h263` > `theora` and audio codec preference to `mp4a` > `aac` > `vorbis` > `opus` > `mp3` > `ac3` > `dts`. You can also make the sorting prefer the nearest values to the provided by using `~` as the delimiter. Eg: `filesize~1G` prefers the format with filesize closest to 1 GiB.

 The fields `hasvid`, `ie_pref`, `lang` are always given highest priority in sorting, irrespective of the user-defined order. This behaviour can be changed by using `--force-format-sort`. Apart from these, the default order used is: `quality,res,fps,codec:vp9.2,size,br,asr,proto,ext,hasaud,source,id`. Note that the extractors may override this default order, but they cannot override the user-provided order.

-If your format selector is `worst`, the last item is selected after sorting. This means it will select the format that is worst in all repects. Most of the time, what you actually want is the video with the smallest filesize instead. So it is generally better to use `-f best -S +size,+br,+res,+fps`.
+If your format selector is `worst`, the last item is selected after sorting. This means it will select the format that is worst in all respects. Most of the time, what you actually want is the video with the smallest filesize instead. So it is generally better to use `-f best -S +size,+br,+res,+fps`.

 **Tip**: You can use the `-v -F` to see how the formats have been sorted (worst to best).

@@ -1096,7 +1115,7 @@ If your format selector is `worst`, the last item is selected after sorting. Thi
 Note that on Windows you may need to use double quotes instead of single.

 ```bash
-# Download and merge the best best video-only format and the best audio-only format,
+# Download and merge the best video-only format and the best audio-only format,
 # or download the best combined format if video-only format is not available
 $ yt-dlp -f 'bv+ba/b'

@@ -1198,20 +1217,20 @@ $ yt-dlp -S '+codec:h264'

 # More complex examples

-# Download the best video no better than 720p prefering framerate greater than 30,
-# or the worst video (still prefering framerate greater than 30) if there is no such video
+# Download the best video no better than 720p preferring framerate greater than 30,
+# or the worst video (still preferring framerate greater than 30) if there is no such video
 $ yt-dlp -f '((bv*[fps>30]/bv*)[height<=720]/(wv*[fps>30]/wv*)) + ba / (b[fps>30]/b)[height<=720]/(w[fps>30]/w)'

 # Download the video with the largest resolution no better than 720p,
 # or the video with the smallest resolution available if there is no such video,
-# prefering larger framerate for formats with the same resolution
+# preferring larger framerate for formats with the same resolution
 $ yt-dlp -S 'res:720,fps'



 # Download the video with smallest resolution no worse than 480p,
 # or the video with the largest resolution available if there is no such video,
-# prefering better codec and then larger total bitrate for the same resolution
+# preferring better codec and then larger total bitrate for the same resolution
 $ yt-dlp -S '+res:480,codec,br'
 ```

@@ -1252,39 +1271,57 @@ Plugins are loaded from `<root-dir>/ytdlp_plugins/<type>/__init__.py`. Currently

 These are all the deprecated options and the current alternative to achieve the same effect

+#### Not recommended
+While these options still work, their use is not recommended since there are other alternatives to achieve the same
+
+    --all-formats                    -f all
+    --all-subs                       --sub-langs all --write-subs
+    --autonumber-size NUMBER         Use string formatting. Eg: %(autonumber)03d
+    --autonumber-start NUMBER        Use internal field formatting like %(autonumber+NUMBER)s
+    --metadata-from-title FORMAT     --parse-metadata "%(title)s:FORMAT"
+    --hls-prefer-native              --downloader "m3u8:native"
+    --hls-prefer-ffmpeg              --downloader "m3u8:ffmpeg"
+    --sponskrub-args ARGS            --ppa "sponskrub:ARGS"
+    --test                           Used by developers for testing extractors. Not intended for the end user
+
+
+#### Old aliases
+These are aliases that are no longer documented for various reasons
+
+    --avconv-location                --ffmpeg-location
    --cn-verification-proxy URL      --geo-verification-proxy URL
+    --dump-headers                   --print-traffic
+    --dump-intermediate-pages        --dump-pages
+    --force-write-download-archive   --force-write-archive
+    --load-info                      --load-info-json
+    --no-split-tracks                --no-split-chapters
+    --no-write-srt                   --no-write-subs
+    --prefer-unsecure                --prefer-insecure
+    --rate-limit RATE                --limit-rate RATE
+    --split-tracks                   --split-chapters
+    --srt-lang LANGS                 --sub-langs LANGS
+    --trim-file-names LENGTH         --trim-filenames LENGTH
+    --write-srt                      --write-subs
+    --yes-overwrites                 --force-overwrites
+
+#### No longer supported
+These options may no longer work as intended
+
+    --prefer-avconv                  avconv is not officially supported by yt-dlp (Alias: --no-prefer-ffmpeg)
+    --prefer-ffmpeg                  Default (Alias: --no-prefer-avconv)
+    -C, --call-home                  Not implemented
+    --no-call-home                   Default
+    --include-ads                    No longer supported
+    --no-include-ads                 Default
+    --youtube-print-sig-code         No longer supported
    --id                             -o "%(id)s.%(ext)s"
    -A, --auto-number                -o "%(autonumber)s-%(id)s.%(ext)s"
    -t, --title                      -o "%(title)s-%(id)s.%(ext)s"
    -l, --literal                    -o accepts literal names
-    --all-formats                    -f all
-    --autonumber-size NUMBER         Use string formatting. Eg: %(autonumber)03d
-    --metadata-from-title FORMAT     --parse-metadata "%(title)s:FORMAT"
-    --prefer-avconv                  avconv is no longer officially supported (Alias: --no-prefer-ffmpeg)
-    --prefer-ffmpeg                  Default (Alias: --no-prefer-avconv)
-    --hls-prefer-native              --downloader "m3u8:native"
-    --hls-prefer-ffmpeg              --downloader "m3u8:ffmpeg"
-    --avconv-location                avconv is no longer officially supported
-    -C, --call-home                  Not implemented
-    --no-call-home                   Default
-    --include-ads                    Not implemented
-    --no-include-ads                 Default
-    --write-srt                      --write-subs
-    --no-write-srt                   --no-write-subs
-    --srt-lang LANGS                 --sub-langs LANGS
-    --prefer-unsecure                --prefer-insecure
-    --rate-limit RATE                --limit-rate RATE
-    --force-write-download-archive   --force-write-archive
-    --dump-intermediate-pages        --dump-pages
-    --dump-headers                   --print-traffic
-    --youtube-print-sig-code         No longer supported
-    --trim-file-names LENGTH         --trim-filenames LENGTH
-    --yes-overwrites                 --force-overwrites
-    --load-info                      --load-info-json
-    --split-tracks                   --split-chapters
-    --no-split-tracks                --no-split-chapters
-    --sponskrub-args ARGS            --ppa "sponskrub:ARGS"
-    --test                           Only used for testing extractors
+
+#### Removed
+Currently, there are no options that have been completely removed. But there are plans to remove the old output options `-A`,`-t`, `-l`, `--id` (which have been deprecated since 2014) in the near future. If you are still using these, please move to using `--output` instead
+


 # MORE
@@ -14,9 +14,14 @@ lazy_extractors_filename = sys.argv[1]
 if os.path.exists(lazy_extractors_filename):
    os.remove(lazy_extractors_filename)

+# Block plugins from loading
+os.rename('ytdlp_plugins', 'ytdlp_plugins_blocked')
+
 from yt_dlp.extractor import _ALL_CLASSES
 from yt_dlp.extractor.common import InfoExtractor, SearchInfoExtractor

+os.rename('ytdlp_plugins_blocked', 'ytdlp_plugins')
+
 with open('devscripts/lazy_load_template.py', 'rt') as f:
    module_template = f.read()

@@ -69,6 +69,7 @@ PyInstaller.__main__.run([
    '--onefile',
    '--icon=devscripts/cloud.ico',
    '--exclude-module=youtube_dl',
+    '--exclude-module=youtube_dlc',
    '--exclude-module=test',
    '--exclude-module=ytdlp_plugins',
    '--hidden-import=mutagen',
@@ -1016,6 +1016,7 @@
 - **TruTV**
 - **Tube8**
 - **TubiTv**
+ - **TubiTvShow**
 - **Tumblr**
 - **tunein:clip**
 - **tunein:program**
@@ -311,8 +311,8 @@ class TestFormatSelection(unittest.TestCase):
        self.assertRaises(ExtractorError, ydl.process_ie_result, info_dict.copy())

    def test_youtube_format_selection(self):
+        # FIXME: Rewrite in accordance with the new format sorting options
        return
-        # disabled for now - this needs some changes

        order = [
            '38', '37', '46', '22', '45', '35', '44', '18', '34', '43', '6', '5', '17', '36', '13',
@@ -601,6 +601,26 @@ class TestYoutubeDL(unittest.TestCase):
        self.assertTrue(subs)
        self.assertEqual(set(subs.keys()), set(['es', 'fr']))

+        result = get_info({'writesubtitles': True, 'subtitleslangs': ['all', '-en']})
+        subs = result['requested_subtitles']
+        self.assertTrue(subs)
+        self.assertEqual(set(subs.keys()), set(['es', 'fr']))
+
+        result = get_info({'writesubtitles': True, 'subtitleslangs': ['en', 'fr', '-en']})
+        subs = result['requested_subtitles']
+        self.assertTrue(subs)
+        self.assertEqual(set(subs.keys()), set(['fr']))
+
+        result = get_info({'writesubtitles': True, 'subtitleslangs': ['-en', 'en']})
+        subs = result['requested_subtitles']
+        self.assertTrue(subs)
+        self.assertEqual(set(subs.keys()), set(['en']))
+
+        result = get_info({'writesubtitles': True, 'subtitleslangs': ['e.+']})
+        subs = result['requested_subtitles']
+        self.assertTrue(subs)
+        self.assertEqual(set(subs.keys()), set(['es', 'en']))
+
        result = get_info({'writesubtitles': True, 'writeautomaticsub': True, 'subtitleslangs': ['es', 'pt']})
        subs = result['requested_subtitles']
        self.assertTrue(subs)
@@ -635,6 +655,8 @@ class TestYoutubeDL(unittest.TestCase):
            'height': 1080,
            'title1': '$PATH',
            'title2': '%PATH%',
+            'timestamp': 1618488000,
+            'formats': [{'id': 'id1'}, {'id': 'id2'}]
        }

        def fname(templ, na_placeholder='NA'):
@@ -651,6 +673,7 @@ class TestYoutubeDL(unittest.TestCase):
        # Or by provided placeholder
        self.assertEqual(fname(NA_TEST_OUTTMPL, na_placeholder='none'), 'none-none-1234.mp4')
        self.assertEqual(fname(NA_TEST_OUTTMPL, na_placeholder=''), '--1234.mp4')
+        self.assertEqual(fname('%(height)s.%(ext)s'), '1080.mp4')
        self.assertEqual(fname('%(height)d.%(ext)s'), '1080.mp4')
        self.assertEqual(fname('%(height)6d.%(ext)s'), '  1080.mp4')
        self.assertEqual(fname('%(height)-6d.%(ext)s'), '1080  .mp4')
@@ -668,6 +691,12 @@ class TestYoutubeDL(unittest.TestCase):
        self.assertEqual(fname('%%(width)06d.%(ext)s'), '%(width)06d.mp4')
        self.assertEqual(fname('Hello %(title1)s'), 'Hello $PATH')
        self.assertEqual(fname('Hello %(title2)s'), 'Hello %PATH%')
+        self.assertEqual(fname('%(timestamp+-1000>%H-%M-%S)s'), '11-43-20')
+        self.assertEqual(fname('%(id+1)05d'), '01235')
+        self.assertEqual(fname('%(width+100)05d'), 'NA')
+        self.assertEqual(fname('%(formats.0)s').replace("u", ""), "{'id' - 'id1'}")
+        self.assertEqual(fname('%(formats.-1.id)s'), 'id2')
+        self.assertEqual(fname('%(formats.2)s'), 'NA')

    def test_format_note(self):
        ydl = YoutubeDL()
@@ -72,15 +72,6 @@ class TestAllURLsMatching(unittest.TestCase):
        self.assertMatch('http://www.youtube.com/results?search_query=making+mustard', ['youtube:search_url'])
        self.assertMatch('https://www.youtube.com/results?baz=bar&search_query=youtube-dl+test+video&filters=video&lclk=video', ['youtube:search_url'])

-    def test_youtube_extract(self):
-        assertExtractId = lambda url, id: self.assertEqual(YoutubeIE.extract_id(url), id)
-        assertExtractId('http://www.youtube.com/watch?&v=BaW_jenozKc', 'BaW_jenozKc')
-        assertExtractId('https://www.youtube.com/watch?&v=BaW_jenozKc', 'BaW_jenozKc')
-        assertExtractId('https://www.youtube.com/watch?feature=player_embedded&v=BaW_jenozKc', 'BaW_jenozKc')
-        assertExtractId('https://www.youtube.com/watch_popup?v=BaW_jenozKc', 'BaW_jenozKc')
-        assertExtractId('http://www.youtube.com/watch?v=BaW_jenozKcsharePLED17F32AD9753930', 'BaW_jenozKc')
-        assertExtractId('BaW_jenozKc', 'BaW_jenozKc')
-
    def test_facebook_matching(self):
        self.assertTrue(FacebookIE.suitable('https://www.facebook.com/Shiniknoh#!/photo.php?v=10153317450565268'))
        self.assertTrue(FacebookIE.suitable('https://www.facebook.com/cindyweather?fref=ts#!/photo.php?v=10152183998945793'))
@@ -39,6 +39,16 @@ class TestExecution(unittest.TestCase):
        _, stderr = p.communicate()
        self.assertFalse(stderr)

+    def test_lazy_extractors(self):
+        try:
+            subprocess.check_call([sys.executable, 'devscripts/make_lazy_extractors.py', 'yt_dlp/extractor/lazy_extractors.py'], cwd=rootDir, stdout=_DEV_NULL)
+            subprocess.check_call([sys.executable, 'test/test_all_urls.py'], cwd=rootDir, stdout=_DEV_NULL)
+        finally:
+            try:
+                os.remove('yt_dlp/extractor/lazy_extractors.py')
+            except (IOError, OSError):
+                pass
+

 if __name__ == '__main__':
    unittest.main()
@@ -16,6 +16,15 @@ class TestMetadataFromField(unittest.TestCase):
        pp = MetadataFromFieldPP(None, ['title:%(title)s - %(artist)s'])
        self.assertEqual(pp._data[0]['regex'], r'(?P<title>.+)\ \-\ (?P<artist>.+)')

+    def test_field_to_outtmpl(self):
+        pp = MetadataFromFieldPP(None, ['title:%(title)s : %(artist)s'])
+        self.assertEqual(pp._data[0]['tmpl'], '%(title)s')
+
+    def test_in_out_seperation(self):
+        pp = MetadataFromFieldPP(None, ['%(title)s \\: %(artist)s:%(title)s : %(artist)s'])
+        self.assertEqual(pp._data[0]['in'], '%(title)s : %(artist)s')
+        self.assertEqual(pp._data[0]['out'], '%(title)s : %(artist)s')
+

 class TestMetadataFromTitle(unittest.TestCase):
    def test_format_to_regex(self):
@@ -0,0 +1,26 @@
+#!/usr/bin/env python
+from __future__ import unicode_literals
+
+# Allow direct execution
+import os
+import sys
+import unittest
+sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+
+
+from yt_dlp.extractor import YoutubeIE
+
+
+class TestYoutubeMisc(unittest.TestCase):
+    def test_youtube_extract(self):
+        assertExtractId = lambda url, id: self.assertEqual(YoutubeIE.extract_id(url), id)
+        assertExtractId('http://www.youtube.com/watch?&v=BaW_jenozKc', 'BaW_jenozKc')
+        assertExtractId('https://www.youtube.com/watch?&v=BaW_jenozKc', 'BaW_jenozKc')
+        assertExtractId('https://www.youtube.com/watch?feature=player_embedded&v=BaW_jenozKc', 'BaW_jenozKc')
+        assertExtractId('https://www.youtube.com/watch_popup?v=BaW_jenozKc', 'BaW_jenozKc')
+        assertExtractId('http://www.youtube.com/watch?v=BaW_jenozKcsharePLED17F32AD9753930', 'BaW_jenozKc')
+        assertExtractId('BaW_jenozKc', 'BaW_jenozKc')
+
+
+if __name__ == '__main__':
+    unittest.main()
@@ -1,2 +1,2 @@
-#!/bin/bash
-python3 "$(dirname $(realpath $0))/yt_dlp/__main__.py" "$@"
+#!/bin/sh
+exec python3 "$(dirname "$(realpath "$0")")/yt_dlp/__main__.py" "$@"
@@ -99,6 +99,7 @@ from .utils import (
    strftime_or_none,
    subtitles_filename,
    to_high_limit_path,
+    traverse_dict,
    UnavailableVideoError,
    url_basename,
    version_tuple,
@@ -191,6 +192,9 @@ class YoutubeDL(object):
    simulate:          Do not download the video files.
    format:            Video format code. see "FORMAT SELECTION" for more details.
    allow_unplayable_formats:   Allow unplayable formats to be extracted and downloaded.
+    ignore_no_formats_error: Ignore "No video formats" error. Usefull for
+                       extracting metadata even if the video is not actually
+                       available for download (experimental)
    format_sort:       How to sort the video formats. see "Sorting Formats"
                       for more details.
    format_sort_force: Force the given format_sort. see "Sorting Formats"
@@ -211,6 +215,8 @@ class YoutubeDL(object):
    ignoreerrors:      Do not stop on download errors
                       (Default True when running yt-dlp,
                       but False when directly accessing YoutubeDL class)
+    skip_playlist_after_errors: Number of allowed failures until the rest of
+                       the playlist is skipped
    force_generic_extractor: Force downloader to use the generic extractor
    overwrites:        Overwrite all video and metadata files if True,
                       overwrite only non-video files if None
@@ -241,11 +247,15 @@ class YoutubeDL(object):
    writedesktoplink:  Write a Linux internet shortcut file (.desktop)
    writesubtitles:    Write the video subtitles to a file
    writeautomaticsub: Write the automatically generated subtitles to a file
-    allsubtitles:      Downloads all the subtitles of the video
+    allsubtitles:      Deprecated - Use subtitlelangs = ['all']
+                       Downloads all the subtitles of the video
                       (requires writesubtitles or writeautomaticsub)
    listsubtitles:     Lists all available subtitles for the video
    subtitlesformat:   The format code for subtitles
-    subtitleslangs:    List of languages of the subtitles to download
+    subtitleslangs:    List of languages of the subtitles to download (can be regex).
+                       The list may contain "all" to refer to all the available
+                       subtitles. The language can be prefixed with a "-" to
+                       exclude it from the requested languages. Eg: ['all', '-live_chat']
    keepvideo:         Keep the video file after post-processing
    daterange:         A DateRange object, download only if the upload_date is in the range.
    skip_download:     Skip the actual download of the video file
@@ -787,6 +797,7 @@ class YoutubeDL(object):
    def prepare_outtmpl(self, outtmpl, info_dict, sanitize=None):
        """ Make the template and info_dict suitable for substitution (outtmpl % info_dict)"""
        template_dict = dict(info_dict)
+        na = self.params.get('outtmpl_na_placeholder', 'NA')

        # duration_string
        template_dict['duration_string'] = (  # %(duration>%H-%M-%S)s is wrong if duration > 24hrs
@@ -812,18 +823,10 @@ class YoutubeDL(object):
            elif template_dict.get('width'):
                template_dict['resolution'] = '%dx?' % template_dict['width']

-        if sanitize is None:
-            sanitize = lambda k, v: v
-        template_dict = dict((k, v if isinstance(v, compat_numeric_types) else sanitize(k, v))
-                             for k, v in template_dict.items()
-                             if v is not None and not isinstance(v, (list, tuple, dict)))
-        na = self.params.get('outtmpl_na_placeholder', 'NA')
-        template_dict = collections.defaultdict(lambda: na, template_dict)
-
        # For fields playlist_index and autonumber convert all occurrences
        # of %(field)s to %(field)0Nd for backward compatibility
        field_size_compat_map = {
-            'playlist_index': len(str(template_dict['n_entries'])),
+            'playlist_index': len(str(template_dict.get('n_entries', na))),
            'autonumber': autonumber_size,
        }
        FIELD_SIZE_COMPAT_RE = r'(?<!%)%\((?P<field>autonumber|playlist_index)\)s'
@@ -835,32 +838,51 @@ class YoutubeDL(object):
                outtmpl)

        numeric_fields = list(self._NUMERIC_FIELDS)
+        if sanitize is None:
+            sanitize = lambda k, v: v

-        # Format date
-        FORMAT_DATE_RE = FORMAT_RE.format(r'(?P<key>(?P<field>\w+)>(?P<format>.+?))')
-        for mobj in re.finditer(FORMAT_DATE_RE, outtmpl):
-            conv_type, field, frmt, key = mobj.group('type', 'field', 'format', 'key')
-            if key in template_dict:
-                continue
-            value = strftime_or_none(template_dict.get(field), frmt, na)
-            if conv_type in 'crs':  # string
-                value = sanitize(field, value)
-            else:  # number
-                numeric_fields.append(key)
-                value = float_or_none(value, default=None)
+        # Internal Formatting = name.key1.key2+number>strf
+        INTERNAL_FORMAT_RE = FORMAT_RE.format(
+            r'''(?P<final_key>
+                        (?P<fields>\w+(?:\.[-\w]+)*)
+                        (?:\+(?P<add>-?\d+(?:\.\d+)?))?
+                        (?:>(?P<strf_format>.+?))?
+            )''')
+        for mobj in re.finditer(INTERNAL_FORMAT_RE, outtmpl):
+            mobj = mobj.groupdict()
+            # Object traversal
+            fields = mobj['fields'].split('.')
+            final_key = mobj['final_key']
+            value = traverse_dict(template_dict, fields)
+            # Offset the value
+            if mobj['add']:
+                value = float_or_none(value)
+                if value is not None:
+                    value = value + float(mobj['add'])
+            # Datetime formatting
+            if mobj['strf_format']:
+                value = strftime_or_none(value, mobj['strf_format'])
+            if mobj['type'] in 'crs' and value is not None:  # string
+                value = sanitize('%{}'.format(mobj['type']) % fields[-1], value)
+            else:  # numeric
+                numeric_fields.append(final_key)
+                value = float_or_none(value)
            if value is not None:
-                template_dict[key] = value
+                template_dict[final_key] = value

        # Missing numeric fields used together with integer presentation types
        # in format specification will break the argument substitution since
        # string NA placeholder is returned for missing fields. We will patch
        # output template for missing fields to meet string presentation type.
        for numeric_field in numeric_fields:
-            if numeric_field not in template_dict:
+            if template_dict.get(numeric_field) is None:
                outtmpl = re.sub(
                    FORMAT_RE.format(re.escape(numeric_field)),
                    r'%({0})s'.format(numeric_field), outtmpl)

+        template_dict = collections.defaultdict(lambda: na, (
+            (k, v if isinstance(v, compat_numeric_types) else sanitize(k, v))
+            for k, v in template_dict.items() if v is not None))
        return outtmpl, template_dict

    def _prepare_filename(self, info_dict, tmpl_type='default'):
@@ -1320,6 +1342,8 @@ class YoutubeDL(object):
        x_forwarded_for = ie_result.get('__x_forwarded_for_ip')

        self.to_screen('[%s] playlist %s: %s' % (ie_result['extractor'], playlist, msg))
+        failures = 0
+        max_failures = self.params.get('skip_playlist_after_errors') or float('inf')
        for i, entry in enumerate(entries, 1):
            self.to_screen('[download] Downloading video %s of %s' % (i, n_entries))
            # This __x_forwarded_for_ip thing is a bit ugly but requires
@@ -1344,6 +1368,12 @@ class YoutubeDL(object):
                continue

            entry_result = self.__process_iterable_entry(entry, download, extra)
+            if not entry_result:
+                failures += 1
+            if failures >= max_failures:
+                self.report_error(
+                    'Skipping the remaining entries in playlist "%s" since %d items failed extraction' % (playlist, failures))
+                break
            # TODO: skip failed (empty) entries?
            playlist_results.append(entry_result)
        ie_result['entries'] = playlist_results
@@ -1650,8 +1680,8 @@ class YoutubeDL(object):
                        formats = list(ctx['formats'])
                        if not formats:
                            return
-                        merged_format = formats[0]
-                        for f in formats[1:]:
+                        merged_format = formats[-1]
+                        for f in formats[-2::-1]:
                            merged_format = _merge((merged_format, f))
                        yield merged_format

@@ -1884,7 +1914,10 @@ class YoutubeDL(object):
            formats = info_dict['formats']

        if not formats:
-            raise ExtractorError('No video formats found!')
+            if not self.params.get('ignore_no_formats_error'):
+                raise ExtractorError('No video formats found!')
+            else:
+                self.report_warning('No video formats found!')

        def is_wellformed(f):
            url = f.get('url')
@@ -1948,13 +1981,15 @@ class YoutubeDL(object):

        # TODO Central sorting goes here

-        if formats[0] is not info_dict:
+        if formats and formats[0] is not info_dict:
            # only set the 'formats' fields if the original info_dict list them
            # otherwise we end up with a circular reference, the first (and unique)
            # element in the 'formats' field in info_dict is info_dict itself,
            # which can't be exported to json
            info_dict['formats'] = formats
        if self.params.get('listformats'):
+            if not info_dict.get('formats'):
+                raise ExtractorError('No video formats found', expected=True)
            self.list_formats(info_dict)
            return

@@ -1994,19 +2029,25 @@ class YoutubeDL(object):

        formats_to_download = list(format_selector(ctx))
        if not formats_to_download:
-            raise ExtractorError('requested format not available',
-                                 expected=True)
-
-        if download:
-            self.to_screen('[info] Downloading format(s) %s' % ", ".join([f['format_id'] for f in formats_to_download]))
+            if not self.params.get('ignore_no_formats_error'):
+                raise ExtractorError('Requested format is not available', expected=True)
+            else:
+                self.report_warning('Requested format is not available')
+        elif download:
+            self.to_screen(
+                '[info] %s: Downloading format(s) %s'
+                % (info_dict['id'], ", ".join([f['format_id'] for f in formats_to_download])))
            if len(formats_to_download) > 1:
-                self.to_screen('[info] %s: downloading video in %s formats' % (info_dict['id'], len(formats_to_download)))
-            for format in formats_to_download:
+                self.to_screen(
+                    '[info] %s: Downloading video in %s formats'
+                    % (info_dict['id'], len(formats_to_download)))
+            for fmt in formats_to_download:
                new_info = dict(info_dict)
-                new_info.update(format)
+                new_info.update(fmt)
                self.process_info(new_info)
        # We update the info dict with the best quality format (backwards compatibility)
-        info_dict.update(formats_to_download[-1])
+        if formats_to_download:
+            info_dict.update(formats_to_download[-1])
        return info_dict

    def process_subtitles(self, video_id, normal_subtitles, automatic_captions):
@@ -2024,15 +2065,28 @@ class YoutubeDL(object):
                available_subs):
            return None

+        all_sub_langs = available_subs.keys()
        if self.params.get('allsubtitles', False):
-            requested_langs = available_subs.keys()
+            requested_langs = all_sub_langs
+        elif self.params.get('subtitleslangs', False):
+            requested_langs = set()
+            for lang in self.params.get('subtitleslangs'):
+                if lang == 'all':
+                    requested_langs.update(all_sub_langs)
+                    continue
+                discard = lang[0] == '-'
+                if discard:
+                    lang = lang[1:]
+                current_langs = filter(re.compile(lang + '$').match, all_sub_langs)
+                if discard:
+                    for lang in current_langs:
+                        requested_langs.discard(lang)
+                else:
+                    requested_langs.update(current_langs)
+        elif 'en' in available_subs:
+            requested_langs = ['en']
        else:
-            if self.params.get('subtitleslangs', False):
-                requested_langs = self.params.get('subtitleslangs')
-            elif 'en' in available_subs:
-                requested_langs = ['en']
-            else:
-                requested_langs = [list(available_subs.keys())[0]]
+            requested_langs = [list(all_sub_langs)[0]]

        formats_query = self.params.get('subtitlesformat', 'best')
        formats_preference = formats_query.split('/') if formats_query else []
@@ -466,6 +466,7 @@ def _real_main(argv=None):
        'skip_download': opts.skip_download,
        'format': opts.format,
        'allow_unplayable_formats': opts.allow_unplayable_formats,
+        'ignore_no_formats_error': opts.ignore_no_formats_error,
        'format_sort': opts.format_sort,
        'format_sort_force': opts.format_sort_force,
        'allow_multiple_video_streams': opts.allow_multiple_video_streams,
@@ -543,6 +544,7 @@ def _real_main(argv=None):
        'download_archive': download_archive_fn,
        'break_on_existing': opts.break_on_existing,
        'break_on_reject': opts.break_on_reject,
+        'skip_playlist_after_errors': opts.skip_playlist_after_errors,
        'cookiefile': opts.cookiefile,
        'nocheckcertificate': opts.no_check_certificate,
        'prefer_insecure': opts.prefer_insecure,
@@ -80,12 +80,12 @@ def get_suitable_downloader(info_dict, params={}, default=HttpFD):
        if ed.can_download(info_dict, external_downloader):
            return ed

-    if protocol.startswith('m3u8'):
+    if protocol in ('m3u8', 'm3u8_native'):
        if info_dict.get('is_live'):
            return FFmpegFD
        elif external_downloader == 'native':
            return HlsFD
-        elif _get_real_downloader(info_dict, 'frag_urls', params, None):
+        elif _get_real_downloader(info_dict, 'm3u8_frag_urls', params, None):
            return HlsFD
        elif params.get('hls_prefer_native') is True:
            return HlsFD
@@ -330,7 +330,7 @@ class HttpieFD(ExternalFD):


 class FFmpegFD(ExternalFD):
-    SUPPORTED_PROTOCOLS = ('http', 'https', 'ftp', 'ftps', 'm3u8', 'rtsp', 'rtmp', 'mms')
+    SUPPORTED_PROTOCOLS = ('http', 'https', 'ftp', 'ftps', 'm3u8', 'm3u8_native', 'rtsp', 'rtmp', 'mms')

    @classmethod
    def available(cls, path=None):
@@ -257,7 +257,7 @@ class AfreecaTVIE(InfoExtractor):
            if flag and flag == 'SUCCEED':
                break
            if flag == 'PARTIAL_ADULT':
-                self._downloader.report_warning(
+                self.report_warning(
                    'In accordance with local laws and regulations, underage users are restricted from watching adult content. '
                    'Only content suitable for all ages will be downloaded. '
                    'Provide account credentials if you wish to download restricted content.')
@@ -323,7 +323,7 @@ class AfreecaTVIE(InfoExtractor):
                        'url': file_url,
                        'format_id': 'http',
                    }]
-                if not formats:
+                if not formats and not self._downloader.params.get('ignore_no_formats'):
                    continue
                self._sort_formats(formats)
                file_info = common_entry.copy()
@@ -36,12 +36,12 @@ class ARDMediathekBaseIE(InfoExtractor):

        if not formats:
            if fsk:
-                raise ExtractorError(
+                self.raise_no_formats(
                    'This video is only available after 20:00', expected=True)
            elif media_info.get('_geoblocked'):
                self.raise_geo_restricted(
                    'This video is not available due to geoblocking',
-                    countries=self._GEO_COUNTRIES)
+                    countries=self._GEO_COUNTRIES, metadata_available=True)

        self._sort_formats(formats)

@@ -11,6 +11,7 @@ from ..compat import (
    compat_etree_Element,
    compat_HTTPError,
    compat_parse_qs,
+    compat_str,
    compat_urllib_parse_urlparse,
    compat_urlparse,
 )
@@ -25,8 +26,10 @@ from ..utils import (
    js_to_json,
    parse_duration,
    parse_iso8601,
+    strip_or_none,
    try_get,
    unescapeHTML,
+    unified_timestamp,
    url_or_none,
    urlencode_postdata,
    urljoin,
@@ -761,8 +764,17 @@ class BBCIE(BBCCoUkIE):
        'only_matching': True,
    }, {
        # custom redirection to www.bbc.com
+        # also, video with window.__INITIAL_DATA__
        'url': 'http://www.bbc.co.uk/news/science-environment-33661876',
-        'only_matching': True,
+        'info_dict': {
+            'id': 'p02xzws1',
+            'ext': 'mp4',
+            'title': "Pluto may have 'nitrogen glaciers'",
+            'description': 'md5:6a95b593f528d7a5f2605221bc56912f',
+            'thumbnail': r're:https?://.+/.+\.jpg',
+            'timestamp': 1437785037,
+            'upload_date': '20150725',
+        },
    }, {
        # single video article embedded with data-media-vpid
        'url': 'http://www.bbc.co.uk/sport/rowing/35908187',
@@ -1164,12 +1176,29 @@ class BBCIE(BBCCoUkIE):
                        continue
                    formats, subtitles = self._download_media_selector(item_id)
                    self._sort_formats(formats)
+                    item_desc = None
+                    blocks = try_get(media, lambda x: x['summary']['blocks'], list)
+                    if blocks:
+                        summary = []
+                        for block in blocks:
+                            text = try_get(block, lambda x: x['model']['text'], compat_str)
+                            if text:
+                                summary.append(text)
+                        if summary:
+                            item_desc = '\n\n'.join(summary)
+                    item_time = None
+                    for meta in try_get(media, lambda x: x['metadata']['items'], list) or []:
+                        if try_get(meta, lambda x: x['label']) == 'Published':
+                            item_time = unified_timestamp(meta.get('timestamp'))
+                            break
                    entries.append({
                        'id': item_id,
                        'title': item_title,
                        'thumbnail': item.get('holdingImageUrl'),
                        'formats': formats,
                        'subtitles': subtitles,
+                        'timestamp': item_time,
+                        'description': strip_or_none(item_desc),
                    })
            for resp in (initial_data.get('data') or {}).values():
                name = resp.get('name')
@@ -1242,7 +1271,7 @@ class BBCIE(BBCCoUkIE):
        entries = []
        for num, media_meta in enumerate(medias, start=1):
            formats, subtitles = self._extract_from_media_meta(media_meta, playlist_id)
-            if not formats:
+            if not formats and not self._downloader.params.get('ignore_no_formats'):
                continue
            self._sort_formats(formats)

@@ -2,6 +2,7 @@
 from __future__ import unicode_literals

 import hashlib
+import itertools
 import json
 import re

@@ -498,28 +499,40 @@ class BiliBiliBangumiIE(InfoExtractor):

 class BilibiliChannelIE(InfoExtractor):
    _VALID_URL = r'https?://space.bilibili\.com/(?P<id>\d+)'
-    # May need to add support for pagination? Need to find a user with many video uploads to test
-    _API_URL = "https://api.bilibili.com/x/space/arc/search?mid=%s&pn=1&ps=25&jsonp=jsonp"
-    _TEST = {}  # TODO: Add tests
+    _API_URL = "https://api.bilibili.com/x/space/arc/search?mid=%s&pn=%d&jsonp=jsonp"
+    _TESTS = [{
+        'url': 'https://space.bilibili.com/3985676/video',
+        'info_dict': {},
+        'playlist_mincount': 112,
+    }]
+
+    def _entries(self, list_id):
+        count, max_count = 0, None
+
+        for page_num in itertools.count(1):
+            data = self._parse_json(
+                self._download_webpage(
+                    self._API_URL % (list_id, page_num), list_id,
+                    note='Downloading page %d' % page_num),
+                list_id)['data']
+
+            max_count = max_count or try_get(data, lambda x: x['page']['count'])
+
+            entries = try_get(data, lambda x: x['list']['vlist'])
+            if not entries:
+                return
+            for entry in entries:
+                yield self.url_result(
+                    'https://www.bilibili.com/video/%s' % entry['bvid'],
+                    BiliBiliIE.ie_key(), entry['bvid'])
+
+            count += len(entries)
+            if max_count and count >= max_count:
+                return

    def _real_extract(self, url):
        list_id = self._match_id(url)
-        json_str = self._download_webpage(self._API_URL % list_id, "None")
-
-        json_parsed = json.loads(json_str)
-        entries = [{
-            '_type': 'url',
-            'ie_key': BiliBiliIE.ie_key(),
-            'url': ('https://www.bilibili.com/video/%s' %
-                    entry['bvid']),
-            'id': entry['bvid'],
-        } for entry in json_parsed['data']['list']['vlist']]
-
-        return {
-            '_type': 'playlist',
-            'id': list_id,
-            'entries': entries
-        }
+        return self.playlist_result(self._entries(list_id), list_id)


 class BiliBiliSearchIE(SearchInfoExtractor):
@@ -114,7 +114,7 @@ class BRIE(InfoExtractor):
            medias.append(media)

        if len(medias) > 1:
-            self._downloader.report_warning(
+            self.report_warning(
                'found multiple medias; please '
                'report this with the video URL to http://yt-dl.org/bug')
        if not medias:
@@ -545,9 +545,9 @@ class BrightcoveNewIE(AdobePassIE):
            errors = json_data.get('errors')
            if errors:
                error = errors[0]
-                raise ExtractorError(
+                self.raise_no_formats(
                    error.get('message') or error.get('error_subcode') or error['error_code'], expected=True)
-            if (not self._downloader.params.get('allow_unplayable_formats')
+            elif (not self._downloader.params.get('allow_unplayable_formats')
                    and sources and num_drm_sources == len(sources)):
                raise ExtractorError('This video is DRM protected.', expected=True)

@@ -26,7 +26,7 @@ class CBSNewsEmbedIE(CBSIE):
    def _real_extract(self, url):
        item = self._parse_json(zlib.decompress(compat_b64decode(
            compat_urllib_parse_unquote(self._match_id(url))),
-            -zlib.MAX_WBITS), None)['video']['items'][0]
+            -zlib.MAX_WBITS).decode('utf-8'), None)['video']['items'][0]
        return self._extract_video_info(item['mpxRefId'], 'cbsnews')


@@ -5,7 +5,6 @@ import re
 from .common import InfoExtractor
 from ..utils import (
    clean_html,
-    ExtractorError,
    int_or_none,
    parse_iso8601,
    qualities,
@@ -187,14 +186,13 @@ class Channel9IE(InfoExtractor):
                    'quality': quality(q, q_url),
                })

-            self._sort_formats(formats)
-
            slides = content_data.get('Slides')
            zip_file = content_data.get('ZipFile')

            if not formats and not slides and not zip_file:
-                raise ExtractorError(
+                self.raise_no_formats(
                    'None of recording, slides or zip are available for %s' % content_path)
+            self._sort_formats(formats)

            subtitles = {}
            for caption in content_data.get('Captions', []):
@@ -683,7 +683,7 @@ class InfoExtractor(object):
            if fatal:
                raise ExtractorError(errmsg, sys.exc_info()[2], cause=err)
            else:
-                self._downloader.report_warning(errmsg)
+                self.report_warning(errmsg)
                return False

    def _download_webpage_handle(self, url_or_request, video_id, note=None, errnote=None, fatal=True, encoding=None, data=None, headers={}, query={}, expected_status=None):
@@ -968,15 +968,27 @@ class InfoExtractor(object):
        """Report attempt to log in."""
        self.to_screen('Logging in')

-    @staticmethod
-    def raise_login_required(msg='This video is only available for registered users'):
+    def raise_login_required(
+            self, msg='This video is only available for registered users', metadata_available=False):
+        if metadata_available and self._downloader.params.get('ignore_no_formats_error'):
+            self.report_warning(msg)
        raise ExtractorError(
-            '%s. Use --username and --password or --netrc to provide account credentials.' % msg,
+            '%s. Use --cookies, --username and --password or --netrc to provide account credentials' % msg,
            expected=True)

-    @staticmethod
-    def raise_geo_restricted(msg='This video is not available from your location due to geo restriction', countries=None):
-        raise GeoRestrictedError(msg, countries=countries)
+    def raise_geo_restricted(
+            self, msg='This video is not available from your location due to geo restriction',
+            countries=None, metadata_available=False):
+        if metadata_available and self._downloader.params.get('ignore_no_formats_error'):
+            self.report_warning(msg)
+        else:
+            raise GeoRestrictedError(msg, countries=countries)
+
+    def raise_no_formats(self, msg, expected=False, video_id=None):
+        if expected and self._downloader.params.get('ignore_no_formats_error'):
+            self.report_warning(msg, video_id)
+        else:
+            raise ExtractorError(msg, expected=expected, video_id=video_id)

    # Methods for following #608
    @staticmethod
@@ -1044,7 +1056,7 @@ class InfoExtractor(object):
        elif fatal:
            raise RegexNotFoundError('Unable to extract %s' % _name)
        else:
-            self._downloader.report_warning('unable to extract %s' % _name + bug_reports_message())
+            self.report_warning('unable to extract %s' % _name + bug_reports_message())
            return None

    def _html_search_regex(self, pattern, string, name, default=NO_DEFAULT, fatal=True, flags=0, group=None):
@@ -1072,7 +1084,7 @@ class InfoExtractor(object):
                    raise netrc.NetrcParseError(
                        'No authenticators for %s' % netrc_machine)
            except (IOError, netrc.NetrcParseError) as err:
-                self._downloader.report_warning(
+                self.report_warning(
                    'parsing .netrc: %s' % error_to_compat_str(err))

        return username, password
@@ -1247,7 +1259,7 @@ class InfoExtractor(object):
        elif fatal:
            raise RegexNotFoundError('Unable to extract JSON-LD')
        else:
-            self._downloader.report_warning('unable to extract JSON-LD %s' % bug_reports_message())
+            self.report_warning('unable to extract JSON-LD %s' % bug_reports_message())
            return {}

    def _json_ld(self, json_ld, video_id, fatal=True, expected_type=None):
@@ -1670,6 +1682,8 @@ class InfoExtractor(object):

    def _sort_formats(self, formats, field_preference=[]):
        if not formats:
+            if self._downloader.params.get('ignore_no_formats_error'):
+                return
            raise ExtractorError('No video formats found')
        format_sort = self.FormatSort()  # params and to_screen are taken from the downloader
        format_sort.evaluate_params(self._downloader.params, field_preference)
@@ -3203,7 +3217,7 @@ class InfoExtractor(object):
            if fatal:
                raise ExtractorError(msg)
            else:
-                self._downloader.report_warning(msg)
+                self.report_warning(msg)
        return res

    def _float(self, v, name, fatal=False, **kwargs):
@@ -3213,7 +3227,7 @@ class InfoExtractor(object):
            if fatal:
                raise ExtractorError(msg)
            else:
-                self._downloader.report_warning(msg)
+                self.report_warning(msg)
        return res

    def _set_cookie(self, domain, name, value, expire_time=None, port=None,
@@ -3389,7 +3403,7 @@ class SearchInfoExtractor(InfoExtractor):
            if n <= 0:
                raise ExtractorError('invalid download number %s for query "%s"' % (n, query))
            elif n > self._MAX_RESULTS:
-                self._downloader.report_warning('%s returns max %i results (you requested %i)' % (self._SEARCH_KEY, self._MAX_RESULTS, n))
+                self.report_warning('%s returns max %i results (you requested %i)' % (self._SEARCH_KEY, self._MAX_RESULTS, n))
                n = self._MAX_RESULTS
            return self._get_n_results(query, n)

@@ -131,7 +131,7 @@ class CorusIE(ThePlatformFeedIE):
            formats.extend(self._parse_smil_formats(
                smil, smil_url, video_id, namespace))
        if not formats and video.get('drm'):
-            raise ExtractorError('This video is DRM protected.', expected=True)
+            self.raise_no_formats('This video is DRM protected.', expected=True)
        self._sort_formats(formats)

        subtitles = {}
@@ -428,7 +428,7 @@ Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text
            r'<div class="showmedia-trailer-notice">(.+?)</div>',
            webpage, 'trailer-notice', default='')
        if note_m:
-            raise ExtractorError(note_m)
+            raise ExtractorError(note_m, expected=True)

        mobj = re.search(r'Page\.messaging_box_controller\.addItems\(\[(?P<msg>{.+?})\]\)', webpage)
        if mobj:
@@ -14,7 +14,7 @@ from ..utils import (
 class DeezerBaseInfoExtractor(InfoExtractor):
    def get_data(self, url):
        if not self._downloader.params.get('test'):
-            self._downloader.report_warning('For now, this extractor only supports the 30 second previews. Patches welcome!')
+            self.report_warning('For now, this extractor only supports the 30 second previews. Patches welcome!')

        mobj = re.match(self._VALID_URL, url)
        data_id = mobj.group('id')
@@ -9,7 +9,6 @@ from ..utils import (
    unified_strdate,
    compat_str,
    determine_ext,
-    ExtractorError,
    update_url_query,
 )

@@ -140,7 +139,7 @@ class DisneyIE(InfoExtractor):
                'vcodec': 'none' if (width == 0 and height == 0) else None,
            })
        if not formats and video_data.get('expired'):
-            raise ExtractorError(
+            self.raise_no_formats(
                '%s said: %s' % (self.IE_NAME, page_data['translations']['video_expired']),
                expected=True)
        self._sort_formats(formats)
@@ -79,7 +79,7 @@ class ElonetIE(InfoExtractor):
        elif fmt == 'mpd':
            subs = self._parse_mpd_subtitles(doc)
        else:
-            self._downloader.report_warning(
+            self.report_warning(
                "Cannot download subtitles from '%s' streams." % (fmt))
            subs = {}
        return subs
@@ -1340,7 +1340,10 @@ from .trovo import (
 from .trunews import TruNewsIE
 from .trutv import TruTVIE
 from .tube8 import Tube8IE
-from .tubitv import TubiTvIE
+from .tubitv import (
+    TubiTvIE,
+    TubiTvShowIE,
+)
 from .tumblr import TumblrIE
 from .tunein import (
    TuneInClipIE,
@@ -348,7 +348,7 @@ class FacebookIE(InfoExtractor):
                    login_results, 'login error', default=None, group='error')
                if error:
                    raise ExtractorError('Unable to login: %s' % error, expected=True)
-                self._downloader.report_warning('unable to log in: bad username/password, or exceeded login rate limit (~3/min). Check credentials or wait.')
+                self.report_warning('unable to log in: bad username/password, or exceeded login rate limit (~3/min). Check credentials or wait.')
                return

            fb_dtsg = self._search_regex(
@@ -369,9 +369,9 @@ class FacebookIE(InfoExtractor):
            check_response = self._download_webpage(check_req, None,
                                                    note='Confirming login')
            if re.search(r'id="checkpointSubmitButton"', check_response) is not None:
-                self._downloader.report_warning('Unable to confirm login, you have to login in your browser and authorize the login.')
+                self.report_warning('Unable to confirm login, you have to login in your browser and authorize the login.')
        except (compat_urllib_error.URLError, compat_http_client.HTTPException, socket.error) as err:
-            self._downloader.report_warning('unable to log in: %s' % error_to_compat_str(err))
+            self.report_warning('unable to log in: %s' % error_to_compat_str(err))
            return

    def _real_initialize(self):
@@ -625,8 +625,6 @@ class FacebookIE(InfoExtractor):
            subtitles_src = f[0].get('subtitles_src')
            if subtitles_src:
                subtitles.setdefault('en', []).append({'url': subtitles_src})
-        if not formats:
-            raise ExtractorError('Cannot find video formats')

        process_formats(formats)

@@ -2376,7 +2376,7 @@ class GenericIE(InfoExtractor):

            if default_search in ('auto', 'auto_warning', 'fixup_error'):
                if re.match(r'^[^\s/]+\.[^\s/]+/', url):
-                    self._downloader.report_warning('The url doesn\'t specify the protocol, trying with http')
+                    self.report_warning('The url doesn\'t specify the protocol, trying with http')
                    return self.url_result('http://' + url)
                elif default_search != 'fixup_error':
                    if default_search == 'auto_warning':
@@ -2385,7 +2385,7 @@ class GenericIE(InfoExtractor):
                                'Invalid URL:  %r . Call yt-dlp like this:  yt-dlp -v "https://www.youtube.com/watch?v=BaW_jenozKc"  ' % url,
                                expected=True)
                        else:
-                            self._downloader.report_warning(
+                            self.report_warning(
                                'Falling back to youtube search for  %s . Set --default-search "auto" to suppress this warning.' % url)
                    return self.url_result('ytsearch:' + url)

@@ -2461,7 +2461,7 @@ class GenericIE(InfoExtractor):

        if not self._downloader.params.get('test', False) and not is_intentional:
            force = self._downloader.params.get('force_generic_extractor', False)
-            self._downloader.report_warning(
+            self.report_warning(
                '%s on generic information extractor.' % ('Forcing' if force else 'Falling back'))

        if not full_response:
@@ -2488,7 +2488,7 @@ class GenericIE(InfoExtractor):
        # Maybe it's a direct link to a video?
        # Be careful not to download the whole thing!
        if not is_html(first_bytes):
-            self._downloader.report_warning(
+            self.report_warning(
                'URL could be a direct video link, returning it as such.')
            info_dict.update({
                'direct': True,
@@ -4,10 +4,14 @@ from __future__ import unicode_literals
 import re

 from .adobepass import AdobePassIE
+from ..compat import compat_str
 from ..utils import (
    int_or_none,
    determine_ext,
    parse_age_limit,
+    remove_start,
+    remove_end,
+    try_get,
    urlencode_postdata,
    ExtractorError,
 )
@@ -46,15 +50,15 @@ class GoIE(AdobePassIE):
    }
    _VALID_URL = r'''(?x)
                    https?://
-                        (?:
-                            (?:(?P<sub_domain>%s)\.)?go|
-                            (?P<sub_domain_2>abc|freeform|disneynow|fxnow\.fxnetworks)
+                        (?P<sub_domain>
+                            (?:%s\.)?go|fxnow\.fxnetworks|
+                            (?:www\.)?(?:abc|freeform|disneynow)
                        )\.com/
                        (?:
                            (?:[^/]+/)*(?P<id>[Vv][Dd][Kk][Aa]\w+)|
                            (?:[^/]+/)*(?P<display_id>[^/?\#]+)
                        )
-                    ''' % '|'.join(list(_SITE_INFO.keys()))
+                    ''' % r'\.|'.join(list(_SITE_INFO.keys()))
    _TESTS = [{
        'url': 'http://abc.go.com/shows/designated-survivor/video/most-recent/VDKA3807643',
        'info_dict': {
@@ -116,6 +120,18 @@ class GoIE(AdobePassIE):
            # m3u8 download
            'skip_download': True,
        },
+    }, {
+        'url': 'https://abc.com/shows/modern-family/episode-guide/season-01/101-pilot',
+        'info_dict': {
+            'id': 'VDKA22600213',
+            'ext': 'mp4',
+            'title': 'Pilot',
+            'description': 'md5:74306df917cfc199d76d061d66bebdb4',
+        },
+        'params': {
+            # m3u8 download
+            'skip_download': True,
+        },
    }, {
        'url': 'http://abc.go.com/shows/the-catch/episode-guide/season-01/10-the-wedding',
        'only_matching': True,
@@ -133,6 +149,9 @@ class GoIE(AdobePassIE):
    }, {
        'url': 'https://disneynow.com/shows/minnies-bow-toons/video/happy-campers/vdka4872013',
        'only_matching': True,
+    }, {
+        'url': 'https://www.freeform.com/shows/cruel-summer/episode-guide/season-01/01-happy-birthday-jeanette-turner',
+        'only_matching': True,
    }]

    def _extract_videos(self, brand, video_id='-1', show_id='-1'):
@@ -143,24 +162,36 @@ class GoIE(AdobePassIE):

    def _real_extract(self, url):
        mobj = re.match(self._VALID_URL, url)
-        sub_domain = mobj.group('sub_domain') or mobj.group('sub_domain_2')
+        sub_domain = remove_start(remove_end(mobj.group('sub_domain') or '', '.go'), 'www.')
        video_id, display_id = mobj.group('id', 'display_id')
        site_info = self._SITE_INFO.get(sub_domain, {})
        brand = site_info.get('brand')
        if not video_id or not site_info:
            webpage = self._download_webpage(url, display_id or video_id)
-            video_id = self._search_regex(
-                (
-                    # There may be inner quotes, e.g. data-video-id="'VDKA3609139'"
-                    # from http://freeform.go.com/shows/shadowhunters/episodes/season-2/1-this-guilty-blood
-                    r'data-video-id=["\']*(VDKA\w+)',
-                    # https://github.com/ytdl-org/youtube-dl/pull/25216/files
-                    #  The following is based on the pull request on the line above. Changed the ABC.com URL to a show available now.
-                    # https://abc.com/shows/the-rookie/episode-guide/season-02/19-the-q-word
-                    r'\bvideoIdCode["\']\s*:\s*["\'](vdka\w+)',
-                    # Deprecated  fallback pattern
-                    r'\b(?:video)?id["\']\s*:\s*["\'](VDKA\w+)'
-                ), webpage, 'video id', default=video_id)
+            data = self._parse_json(
+                self._search_regex(
+                    r'["\']__abc_com__["\']\s*\]\s*=\s*({.+?})\s*;', webpage,
+                    'data', default='{}'),
+                display_id or video_id, fatal=False)
+            # https://abc.com/shows/modern-family/episode-guide/season-01/101-pilot
+            layout = try_get(data, lambda x: x['page']['content']['video']['layout'], dict)
+            video_id = None
+            if layout:
+                video_id = try_get(
+                    layout,
+                    (lambda x: x['videoid'], lambda x: x['video']['id']),
+                    compat_str)
+            if not video_id:
+                video_id = self._search_regex(
+                    (
+                        # There may be inner quotes, e.g. data-video-id="'VDKA3609139'"
+                        # from http://freeform.go.com/shows/shadowhunters/episodes/season-2/1-this-guilty-blood
+                        r'data-video-id=["\']*(VDKA\w+)',
+                        # page.analytics.videoIdCode
+                        r'\bvideoIdCode["\']\s*:\s*["\']((?:vdka|VDKA)\w+)',
+                        # https://abc.com/shows/the-rookie/episode-guide/season-02/03-the-bet
+                        r'\b(?:video)?id["\']\s*:\s*["\'](VDKA\w+)'
+                    ), webpage, 'video id', default=video_id)
            if not site_info:
                brand = self._search_regex(
                    (r'data-brand=\s*["\']\s*(\d+)',
@@ -253,7 +253,7 @@ class GoogleDriveIE(InfoExtractor):
                            or 'unable to extract confirmation code')

        if not formats and reason:
-            raise ExtractorError(reason, expected=True)
+            self.raise_no_formats(reason, expected=True)

        self._sort_formats(formats)

@@ -184,7 +184,7 @@ class HotStarIE(HotStarBaseIE):
                    geo_restricted = True
                continue
        if not formats and geo_restricted:
-            self.raise_geo_restricted(countries=['IN'])
+            self.raise_geo_restricted(countries=['IN'], metadata_available=True)
        self._sort_formats(formats)

        for f in formats:
@@ -136,7 +136,7 @@ class IPrimaIE(InfoExtractor):
                extract_formats(src)

        if not formats and '>GEO_IP_NOT_ALLOWED<' in playerpage:
-            self.raise_geo_restricted(countries=['CZ'])
+            self.raise_geo_restricted(countries=['CZ'], metadata_available=True)

        self._sort_formats(formats)

@@ -280,7 +280,7 @@ class IqiyiIE(InfoExtractor):
                msg = 'error %s' % code
                if validation_result.get('msg'):
                    msg += ': ' + validation_result['msg']
-            self._downloader.report_warning('unable to log in: ' + msg)
+            self.report_warning('unable to log in: ' + msg)
            return False

        return True
@@ -101,7 +101,7 @@ class KeezMoviesIE(InfoExtractor):

        if not formats:
            if 'title="This video is no longer available"' in webpage:
-                raise ExtractorError(
+                self.raise_no_formats(
                    'Video %s is no longer available' % video_id, expected=True)

        try:
@@ -122,6 +122,26 @@ class LBRYIE(LBRYBaseIE):
            'channel_url': 'https://lbry.tv/@LBRYFoundation:0ed629d2b9c601300cacf7eabe9da0be79010212',
            'vcodec': 'none',
        }
+    }, {
+        # HLS
+        'url': 'https://odysee.com/@gardeningincanada:b/plants-i-will-never-grow-again.-the:e',
+        'md5': 'fc82f45ea54915b1495dd7cb5cc1289f',
+        'info_dict': {
+            'id': 'e51671357333fe22ae88aad320bde2f6f96b1410',
+            'ext': 'mp4',
+            'title': 'PLANTS I WILL NEVER GROW AGAIN. THE BLACK LIST PLANTS FOR A CANADIAN GARDEN | Gardening in Canada 🍁',
+            'description': 'md5:9c539c6a03fb843956de61a4d5288d5e',
+            'timestamp': 1618254123,
+            'upload_date': '20210412',
+            'release_timestamp': 1618254002,
+            'release_date': '20210412',
+            'tags': list,
+            'duration': 554,
+            'channel': 'Gardening In Canada',
+            'channel_id': 'b8be0e93b423dad221abe29545fbe8ec36e806bc',
+            'channel_url': 'https://odysee.com/@gardeningincanada:b8be0e93b423dad221abe29545fbe8ec36e806bc',
+            'formats': 'mincount:3',
+        }
    }, {
        'url': 'https://odysee.com/@BrodieRobertson:5/apple-is-tracking-everything-you-do-on:e',
        'only_matching': True,
@@ -168,10 +188,18 @@ class LBRYIE(LBRYBaseIE):
        streaming_url = self._call_api_proxy(
            'get', claim_id, {'uri': uri}, 'streaming url')['streaming_url']
        info = self._parse_stream(result, url)
+        urlh = self._request_webpage(
+            streaming_url, display_id, note='Downloading streaming redirect url info')
+        if determine_ext(urlh.geturl()) == 'm3u8':
+            info['formats'] = self._extract_m3u8_formats(
+                urlh.geturl(), display_id, 'mp4', entry_protocol='m3u8_native',
+                m3u8_id='hls')
+            self._sort_formats(info['formats'])
+        else:
+            info['url'] = streaming_url
        info.update({
            'id': claim_id,
            'title': title,
-            'url': streaming_url,
        })
        return info

@@ -6,7 +6,6 @@ import re
 from .common import InfoExtractor
 from ..compat import compat_str
 from ..utils import (
-    ExtractorError,
    int_or_none,
    js_to_json,
    str_or_none,
@@ -77,7 +76,7 @@ class LineTVIE(InfoExtractor):

        self._sort_formats(formats)

-        if not formats[0].get('width'):
+        if formats and not formats[0].get('width'):
            formats[0]['vcodec'] = 'none'

        title = self._og_search_title(webpage)
@@ -183,7 +182,7 @@ class LineLiveIE(LineLiveBaseIE):
        if not formats:
            archive_status = item.get('archiveStatus')
            if archive_status != 'ARCHIVED':
-                raise ExtractorError('this video has been ' + archive_status.lower(), expected=True)
+                self.raise_no_formats('this video has been ' + archive_status.lower(), expected=True)
        self._sort_formats(formats)
        info['formats'] = formats
        return info
@@ -331,7 +331,7 @@ class LyndaCourseIE(LyndaBaseIE):
                    })

        if unaccessible_videos > 0:
-            self._downloader.report_warning(
+            self.report_warning(
                '%s videos are only available for members (or paid members) and will not be downloaded. '
                % unaccessible_videos + self._ACCOUNT_CREDENTIALS_HINT)

@@ -97,11 +97,11 @@ class MedalTVIE(InfoExtractor):
        error = clip.get('error')
        if not formats and error:
            if error == 404:
-                raise ExtractorError(
+                self.raise_no_formats(
                    'That clip does not exist.',
                    expected=True, video_id=video_id)
            else:
-                raise ExtractorError(
+                self.raise_no_formats(
                    'An unknown error occurred ({0}).'.format(error),
                    video_id=video_id)

@@ -1,22 +1,20 @@
 # coding: utf-8
 from __future__ import unicode_literals

+import base64
 from datetime import datetime
 import itertools
 import json
-import base64
 import re

 from .common import InfoExtractor
 from ..utils import (
-    ExtractorError, std_headers,
+    std_headers,
    update_url_query,
    random_uuidv4,
    try_get,
 )
 from ..compat import (
-    compat_urlparse,
-    compat_urllib_parse_urlencode,
    compat_str,
 )

@@ -48,29 +46,24 @@ class MildomBaseIE(InfoExtractor):

    def _fetch_dispatcher_config(self):
        if not self._DISPATCHER_CONFIG:
-            try:
-                tmp = self._download_json(
-                    'https://disp.mildom.com/serverListV2', 'initialization',
-                    note='Downloading dispatcher_config', data=json.dumps({
-                        'protover': 0,
-                        'data': base64.b64encode(json.dumps({
-                            'fr': 'web',
-                            'sfr': 'pc',
-                            'devi': 'Windows',
-                            'la': 'ja',
-                            'gid': None,
-                            'loc': '',
-                            'clu': '',
-                            'wh': '1919*810',
-                            'rtm': self.iso_timestamp(),
-                            'ua': std_headers['User-Agent'],
-                        }).encode('utf8')).decode('utf8').replace('\n', ''),
-                    }).encode('utf8'))
-                self._DISPATCHER_CONFIG = self._parse_json(base64.b64decode(tmp['data']), 'initialization')
-            except ExtractorError:
-                self._DISPATCHER_CONFIG = self._download_json(
-                    'https://bookish-octo-barnacle.vercel.app/api/mildom/dispatcher_config', 'initialization',
-                    note='Downloading dispatcher_config fallback')
+            tmp = self._download_json(
+                'https://disp.mildom.com/serverListV2', 'initialization',
+                note='Downloading dispatcher_config', data=json.dumps({
+                    'protover': 0,
+                    'data': base64.b64encode(json.dumps({
+                        'fr': 'web',
+                        'sfr': 'pc',
+                        'devi': 'Windows',
+                        'la': 'ja',
+                        'gid': None,
+                        'loc': '',
+                        'clu': '',
+                        'wh': '1919*810',
+                        'rtm': self.iso_timestamp(),
+                        'ua': std_headers['User-Agent'],
+                    }).encode('utf8')).decode('utf8').replace('\n', ''),
+                }).encode('utf8'))
+            self._DISPATCHER_CONFIG = self._parse_json(base64.b64decode(tmp['data']), 'initialization')
        return self._DISPATCHER_CONFIG

    @staticmethod
@@ -145,15 +138,10 @@ class MildomIE(MildomBaseIE):
            'Referer': 'https://www.mildom.com/',
            'Origin': 'https://www.mildom.com',
        }, note='Downloading m3u8 information')
+
        del stream_query['streamReqId'], stream_query['timestamp']
        for fmt in formats:
-            # Uses https://github.com/nao20010128nao/bookish-octo-barnacle by @nao20010128nao as a proxy
-            parsed = compat_urlparse.urlparse(fmt['url'])
-            parsed = parsed._replace(
-                netloc='bookish-octo-barnacle.vercel.app',
-                query=compat_urllib_parse_urlencode(stream_query, True),
-                path='/api/mildom' + parsed.path)
-            fmt['url'] = compat_urlparse.urlunparse(parsed)
+            fmt.setdefault('http_headers', {})['Referer'] = 'https://www.mildom.com/'

        self._sort_formats(formats)

@@ -200,16 +188,16 @@ class MildomVodIE(MildomBaseIE):
                lambda x: x['author_info']['login_name'],
            ), compat_str)

-        audio_formats = [{
+        formats = [{
            'url': autoplay['audio_url'],
            'format_id': 'audio',
            'protocol': 'm3u8_native',
            'vcodec': 'none',
            'acodec': 'aac',
+            'ext': 'm4a'
        }]
-        video_formats = []
        for fmt in autoplay['video_link']:
-            video_formats.append({
+            formats.append({
                'format_id': 'video-%s' % fmt['name'],
                'url': fmt['url'],
                'protocol': 'm3u8_native',
@@ -217,23 +205,9 @@ class MildomVodIE(MildomBaseIE):
                'height': fmt['level'],
                'vcodec': 'h264',
                'acodec': 'aac',
+                'ext': 'mp4'
            })

-        stream_query = self._common_queries({
-            'is_lhls': '0',
-        })
-        del stream_query['timestamp']
-        formats = audio_formats + video_formats
-        for fmt in formats:
-            fmt['ext'] = 'mp4'
-            parsed = compat_urlparse.urlparse(fmt['url'])
-            stream_query['path'] = parsed.path[5:]
-            parsed = parsed._replace(
-                netloc='bookish-octo-barnacle.vercel.app',
-                query=compat_urllib_parse_urlencode(stream_query, True),
-                path='/api/mildom/vod2/proxy')
-            fmt['url'] = compat_urlparse.urlunparse(parsed)
-
        self._sort_formats(formats)

        return {
@@ -259,16 +233,7 @@ class MildomUserVodIE(MildomBaseIE):
        'playlist_mincount': 351,
    }]

-    def _real_extract(self, url):
-        user_id = self._match_id(url)
-
-        self._downloader.report_warning('To download ongoing live, please use "https://www.mildom.com/%s" instead. This will list up VODs belonging to user.' % user_id)
-
-        profile = self._call_api(
-            'https://cloudac.mildom.com/nonolive/gappserv/user/profileV2', user_id,
-            query={'user_id': user_id}, note='Downloading user profile')['user_info']
-
-        results = []
+    def _entries(self, user_id):
        for page in itertools.count(1):
            reply = self._call_api(
                'https://cloudac.mildom.com/nonolive/videocontent/profile/playbackList',
@@ -279,7 +244,16 @@ class MildomUserVodIE(MildomBaseIE):
                })
            if not reply:
                break
-            results.extend('https://www.mildom.com/playback/%s/%s' % (user_id, x['v_id']) for x in reply)
-        return self.playlist_result([
-            self.url_result(u, ie=MildomVodIE.ie_key()) for u in results
-        ], user_id, 'Uploads from %s' % profile['loginname'])
+            for x in reply:
+                yield self.url_result('https://www.mildom.com/playback/%s/%s' % (user_id, x['v_id']))
+
+    def _real_extract(self, url):
+        user_id = self._match_id(url)
+        self.to_screen('This will download all VODs belonging to user. To download ongoing live video, use "https://www.mildom.com/%s" instead' % user_id)
+
+        profile = self._call_api(
+            'https://cloudac.mildom.com/nonolive/gappserv/user/profileV2', user_id,
+            query={'user_id': user_id}, note='Downloading user profile')['user_info']
+
+        return self.playlist_result(
+            self._entries(user_id), user_id, 'Uploads from %s' % profile['loginname'])
@@ -157,7 +157,7 @@ class MixcloudIE(MixcloudBaseIE):
                })

        if not formats and cloudcast.get('isExclusive'):
-            self.raise_login_required()
+            self.raise_login_required(metadata_available=True)

        self._sort_formats(formats)

@@ -190,7 +190,7 @@ class NiconicoIE(InfoExtractor):
            if compat_parse_qs(parts.query).get('message', [None])[0] == 'cant_login':
                login_ok = False
        if not login_ok:
-            self._downloader.report_warning('unable to log in: bad username or password')
+            self.report_warning('unable to log in: bad username or password')
        return login_ok

    def _get_heartbeat_info(self, info_dict):
@@ -247,7 +247,7 @@ class NPOIE(NPOBaseIE):

        if not formats:
            if not self._downloader.params.get('allow_unplayable_formats') and drm:
-                raise ExtractorError('This video is DRM protected.', expected=True)
+                self.raise_no_formats('This video is DRM protected.', expected=True)
            return

        self._sort_formats(formats)
@@ -260,7 +260,7 @@ class OdnoklassnikiIE(InfoExtractor):
        if not formats:
            payment_info = metadata.get('paymentInfo')
            if payment_info:
-                raise ExtractorError('This video is paid, subscribe to download it', expected=True)
+                self.raise_no_formats('This video is paid, subscribe to download it', expected=True)

        self._sort_formats(formats)

@@ -10,7 +10,6 @@ from ..compat import (
 )
 from ..utils import (
    determine_ext,
-    ExtractorError,
    float_or_none,
    int_or_none,
    try_get,
@@ -85,7 +84,7 @@ class OoyalaBaseIE(InfoExtractor):
                    'fps': float_or_none(stream.get('framerate')),
                })
        if not formats and not auth_data.get('authorized'):
-            raise ExtractorError('%s said: %s' % (
+            self.raise_no_formats('%s said: %s' % (
                self.IE_NAME, auth_data['message']), expected=True)
        self._sort_formats(formats)

@@ -79,7 +79,7 @@ class PhilharmonieDeParisIE(InfoExtractor):
                formats.extend(self._extract_m3u8_formats(
                    m3u8_url, video_id, 'mp4', entry_protocol='m3u8_native',
                    m3u8_id='hls', fatal=False))
-            if not formats:
+            if not formats and not self._downloader.params.get('ignore_no_formats'):
                return
            self._sort_formats(formats)
            return {
@@ -393,7 +393,7 @@ query viewClip {
                # To somewhat reduce the probability of these consequences
                # we will sleep random amount of time before each call to ViewClip.
                self._sleep(
-                    random.randint(2, 5), display_id,
+                    random.randint(5, 10), display_id,
                    '%(video_id)s: Waiting for %(timeout)s seconds to avoid throttling')

                if not viewclip:
@@ -5,15 +5,16 @@ import re

 from .common import InfoExtractor
 from ..compat import (
-    compat_urlparse,
    compat_str,
+    compat_urlparse,
 )
 from ..utils import (
-    ExtractorError,
    determine_ext,
+    ExtractorError,
    find_xpath_attr,
    fix_xml_ampersands,
    GeoRestrictedError,
+    HEADRequest,
    int_or_none,
    parse_duration,
    remove_start,
@@ -94,7 +95,9 @@ class RaiBaseIE(InfoExtractor):
                })

        if not formats and geoprotection is True:
-            self.raise_geo_restricted(countries=self._GEO_COUNTRIES)
+            self.raise_geo_restricted(countries=self._GEO_COUNTRIES, metadata_available=True)
+
+        formats.extend(self._create_http_urls(relinker_url, formats))

        return dict((k, v) for k, v in {
            'is_live': is_live,
@@ -102,6 +105,92 @@ class RaiBaseIE(InfoExtractor):
            'formats': formats,
        }.items() if v is not None)

+    def _create_http_urls(self, relinker_url, fmts):
+        _RELINKER_REG = r'https?://(?P<host>[^/]+?)/(?:i/)?(?P<extra>[^/]+?)/(?P<path>.+?)/(?P<id>\d+)(?:_(?P<quality>[\d\,]+))?(?:\.mp4|/playlist\.m3u8).+?'
+        _MP4_TMPL = '%s&overrideUserAgentRule=mp4-%s'
+        _QUALITY = {
+            # tbr: w, h
+            '250': [352, 198],
+            '400': [512, 288],
+            '700': [512, 288],
+            '800': [700, 394],
+            '1200': [736, 414],
+            '1800': [1024, 576],
+            '2400': [1280, 720],
+            '3200': [1440, 810],
+            '3600': [1440, 810],
+            '5000': [1920, 1080],
+            '10000': [1920, 1080],
+        }
+
+        def test_url(url):
+            resp = self._request_webpage(
+                HEADRequest(url), None, headers={'User-Agent': 'Rai'},
+                fatal=False, errnote=False, note=False)
+
+            if resp is False:
+                return False
+
+            if resp.code == 200:
+                return False if resp.url == url else resp.url
+            return None
+
+        def get_format_info(tbr):
+            import math
+            br = int_or_none(tbr)
+            if len(fmts) == 1 and not br:
+                br = fmts[0].get('tbr')
+            if br > 300:
+                tbr = compat_str(math.floor(br / 100) * 100)
+            else:
+                tbr = '250'
+
+            # try extracting info from available m3u8 formats
+            format_copy = None
+            for f in fmts:
+                if f.get('tbr'):
+                    br_limit = math.floor(br / 100)
+                    if br_limit - 1 <= math.floor(f['tbr'] / 100) <= br_limit + 1:
+                        format_copy = f.copy()
+            return {
+                'width': format_copy.get('width'),
+                'height': format_copy.get('height'),
+                'tbr': format_copy.get('tbr'),
+                'vcodec': format_copy.get('vcodec'),
+                'acodec': format_copy.get('acodec'),
+                'fps': format_copy.get('fps'),
+                'format_id': 'https-%s' % tbr,
+            } if format_copy else {
+                'width': _QUALITY[tbr][0],
+                'height': _QUALITY[tbr][1],
+                'format_id': 'https-%s' % tbr,
+                'tbr': int(tbr),
+            }
+
+        loc = test_url(_MP4_TMPL % (relinker_url, '*'))
+        if not isinstance(loc, compat_str):
+            return []
+
+        mobj = re.match(
+            _RELINKER_REG,
+            test_url(relinker_url) or '')
+        if not mobj:
+            return []
+
+        available_qualities = mobj.group('quality').split(',') if mobj.group('quality') else ['*']
+        available_qualities = [i for i in available_qualities if i]
+
+        formats = []
+        for q in available_qualities:
+            fmt = {
+                'url': _MP4_TMPL % (relinker_url, q),
+                'protocol': 'https',
+                'ext': 'mp4',
+            }
+            fmt.update(get_format_info(q))
+            formats.append(fmt)
+        return formats
+
    @staticmethod
    def _extract_subtitles(url, video_data):
        STL_EXT = 'stl'
@@ -151,6 +240,22 @@ class RaiPlayIE(RaiBaseIE):
        'params': {
            'skip_download': True,
        },
+    }, {
+        # 1080p direct mp4 url
+        'url': 'https://www.raiplay.it/video/2021/03/Leonardo-S1E1-b5703b02-82ee-475a-85b6-c9e4a8adf642.html',
+        'md5': '2e501e8651d72f05ffe8f5d286ad560b',
+        'info_dict': {
+            'id': 'b5703b02-82ee-475a-85b6-c9e4a8adf642',
+            'ext': 'mp4',
+            'title': 'Leonardo - S1E1',
+            'alt_title': 'St 1 Ep 1 - Episodio 1',
+            'description': 'md5:f5360cd267d2de146e4e3879a5a47d31',
+            'thumbnail': r're:^https?://.*\.jpg$',
+            'uploader': 'Rai 1',
+            'duration': 3229,
+            'series': 'Leonardo',
+            'season': 'Season 1',
+        },
    }, {
        'url': 'http://www.raiplay.it/video/2016/11/gazebotraindesi-efebe701-969c-4593-92f3-285f0d1ce750.html?',
        'only_matching': True,
@@ -318,7 +423,7 @@ class RaiIE(RaiBaseIE):
    }, {
        # with ContentItem in og:url
        'url': 'http://www.rai.it/dl/RaiTV/programmi/media/ContentItem-efb17665-691c-45d5-a60c-5301333cbb0c.html',
-        'md5': '6865dd00cf0bbf5772fdd89d59bd768a',
+        'md5': '06345bd97c932f19ffb129973d07a020',
        'info_dict': {
            'id': 'efb17665-691c-45d5-a60c-5301333cbb0c',
            'ext': 'mp4',
@@ -350,22 +455,6 @@ class RaiIE(RaiBaseIE):
        'params': {
            'skip_download': True,
        },
-    }, {
-        # ContentItem in iframe (see #12652) and subtitle at 'subtitlesUrl' key
-        'url': 'http://www.presadiretta.rai.it/dl/portali/site/puntata/ContentItem-3ed19d13-26c2-46ff-a551-b10828262f1b.html',
-        'info_dict': {
-            'id': '1ad6dc64-444a-42a4-9bea-e5419ad2f5fd',
-            'ext': 'mp4',
-            'title': 'Partiti acchiappavoti - Presa diretta del 13/09/2015',
-            'description': 'md5:d291b03407ec505f95f27970c0b025f4',
-            'upload_date': '20150913',
-            'subtitles': {
-                'it': 'count:2',
-            },
-        },
-        'params': {
-            'skip_download': True,
-        },
    }, {
        # Direct MMS URL
        'url': 'http://www.rai.it/dl/RaiTV/programmi/media/ContentItem-b63a4089-ac28-48cf-bca5-9f5b5bc46df5.html',
@@ -202,7 +202,7 @@ class RuutuIE(InfoExtractor):
        if not formats:
            if (not self._downloader.params.get('allow_unplayable_formats')
                    and xpath_text(video_xml, './Clip/DRM', default=None)):
-                raise ExtractorError('This video is DRM protected.', expected=True)
+                self.raise_no_formats('This video is DRM protected.', expected=True)
            ns_st_cds = pv('ns_st_cds')
            if ns_st_cds != 'free':
                raise ExtractorError('This video is %s.' % ns_st_cds, expected=True)
@@ -312,7 +312,7 @@ class SoundcloudIE(InfoExtractor):
                    self._update_client_id()
                    continue
                elif non_fatal:
-                    self._downloader.report_warning(error_to_compat_str(e))
+                    self.report_warning(error_to_compat_str(e))
                    return False
                raise

@@ -498,7 +498,7 @@ class SoundcloudIE(InfoExtractor):
            f['vcodec'] = 'none'

        if not formats and info.get('policy') == 'BLOCK':
-            self.raise_geo_restricted()
+            self.raise_geo_restricted(metadata_available=True)
        self._sort_formats(formats)

        user = info.get('user') or {}
@@ -77,7 +77,7 @@ class SportDeutschlandIE(InfoExtractor):
                        continue
                    formats = self._extract_m3u8_formats(
                        video_url.replace('.smil', '.m3u8'), video_id, 'mp4', fatal=False)
-                    if not formats:
+                    if not formats and not self._downloader.params.get('ignore_no_formats'):
                        continue
                    yield {
                        'id': video_id,
@@ -139,7 +139,7 @@ class SteamIE(InfoExtractor):
                                        'format_id': ext + quality,
                                        'url': video_url,
                                    })
-                if not formats:
+                if not formats and not self._downloader.params.get('ignore_no_formats'):
                    continue
                entry['formats'] = formats
                entries.append(entry)
@@ -49,7 +49,7 @@ class SVTBaseIE(InfoExtractor):
        if not formats and rights.get('geoBlockedSweden'):
            self.raise_geo_restricted(
                'This video is only available in Sweden',
-                countries=self._GEO_COUNTRIES)
+                countries=self._GEO_COUNTRIES, metadata_available=True)
        self._sort_formats(formats)

        subtitles = {}
@@ -7,7 +7,6 @@ import re
 from .common import InfoExtractor
 from ..utils import (
    determine_ext,
-    ExtractorError,
    float_or_none,
    int_or_none,
    parse_iso8601,
@@ -156,10 +155,9 @@ class ToggleIE(InfoExtractor):
            for meta in (info.get('Metas') or []):
                if (not self._downloader.params.get('allow_unplayable_formats')
                        and meta.get('Key') == 'Encryption' and meta.get('Value') == '1'):
-                    raise ExtractorError(
+                    self.raise_no_formats(
                        'This video is DRM protected.', expected=True)
-            # Most likely because geo-blocked
-            raise ExtractorError('No downloadable videos found', expected=True)
+            # Most likely because geo-blocked if no formats and no DRM
        self._sort_formats(formats)

        thumbnails = []
@@ -7,13 +7,19 @@ from .common import InfoExtractor
 from ..utils import (
    ExtractorError,
    int_or_none,
+    js_to_json,
    sanitized_Request,
    urlencode_postdata,
 )


 class TubiTvIE(InfoExtractor):
-    _VALID_URL = r'https?://(?:www\.)?tubitv\.com/(?:video|movies|tv-shows)/(?P<id>[0-9]+)'
+    _VALID_URL = r'''(?x)
+                    (?:
+                        tubitv:|
+                        https?://(?:www\.)?tubitv\.com/(?:video|movies|tv-shows)/
+                    )
+                    (?P<id>[0-9]+)'''
    _LOGIN_URL = 'http://tubitv.com/login'
    _NETRC_MACHINE = 'tubitv'
    _GEO_COUNTRIES = ['US']
@@ -108,3 +114,28 @@ class TubiTvIE(InfoExtractor):
            'uploader_id': video_data.get('publisher_id'),
            'release_year': int_or_none(video_data.get('year')),
        }
+
+
+class TubiTvShowIE(InfoExtractor):
+    _VALID_URL = r'https?://(?:www\.)?tubitv\.com/series/[0-9]+/(?P<show_name>[^/?#]+)'
+    _TESTS = [{
+        'url': 'https://tubitv.com/series/3936/the-joy-of-painting-with-bob-ross?start=true',
+        'playlist_mincount': 390,
+        'info_dict': {
+            'id': 'the-joy-of-painting-with-bob-ross',
+        }
+    }]
+
+    def _entries(self, show_url, show_name):
+        show_webpage = self._download_webpage(show_url, show_name)
+        show_json = self._parse_json(self._search_regex(
+            r"window\.__data\s*=\s*({.+?});\s*</script>",
+            show_webpage, 'data',), show_name, transform_source=js_to_json)['video']
+        for episode_id in show_json['fullContentById'].keys():
+            yield self.url_result(
+                'tubitv:%s' % episode_id,
+                ie=TubiTvIE.ie_key(), video_id=episode_id)
+
+    def _real_extract(self, url):
+        show_name = re.match(self._VALID_URL, url).group('show_name')
+        return self.playlist_result(self._entries(url, show_name), playlist_id=show_name)
@@ -103,7 +103,7 @@ class TV2IE(InfoExtractor):
                        'filesize': int_or_none(item.get('fileSize')),
                    })
        if not formats and data.get('drmProtected'):
-            raise ExtractorError('This video is DRM protected.', expected=True)
+            self.raise_no_formats('This video is DRM protected.', expected=True)
        self._sort_formats(formats)

        thumbnails = [{
@@ -107,7 +107,7 @@ class TV4IE(InfoExtractor):
            video_id, ism_id='mss', fatal=False))

        if not formats and info.get('is_geo_restricted'):
-            self.raise_geo_restricted(countries=self._GEO_COUNTRIES)
+            self.raise_geo_restricted(countries=self._GEO_COUNTRIES, metadata_available=True)

        self._sort_formats(formats)

@@ -298,7 +298,8 @@ class TVPlayIE(InfoExtractor):

        if not formats and video.get('is_geo_blocked'):
            self.raise_geo_restricted(
-                'This content might not be available in your country due to copyright reasons')
+                'This content might not be available in your country due to copyright reasons',
+                metadata_available=True)

        self._sort_formats(formats)

@@ -65,7 +65,9 @@ class TwitCastingIE(InfoExtractor):
            request_data = urlencode_postdata({
                'password': video_password,
            })
-        webpage = self._download_webpage(url, video_id, data=request_data)
+        webpage = self._download_webpage(
+            url, video_id, data=request_data,
+            headers={'Origin': 'https://twitcasting.tv'})

        title = clean_html(get_element_by_id(
            'movietitle', webpage)) or self._html_search_meta(
@@ -77,14 +79,15 @@ class TwitCastingIE(InfoExtractor):
            webpage, 'm3u8 url', group='url', default=None)
        if not m3u8_url:
            video_js_data = self._parse_json(self._search_regex(
-                r"data-movie-playlist='(\[[^']+\])'",
-                webpage, 'movie playlist'), video_id)[0]
+                r'data-movie-playlist=(["\'])(?P<url>(?:(?!\1).)+)',
+                webpage, 'movie playlist', group='url'), video_id)
+            if isinstance(video_js_data, dict):
+                video_js_data = list(video_js_data.values())[0]
+            video_js_data = video_js_data[0]
            m3u8_url = video_js_data['source']['url']

-        # use `m3u8` entry_protocol until EXT-X-MAP is properly supported by `m3u8_native` entry_protocol
        formats = self._extract_m3u8_formats(
-            m3u8_url, video_id, 'mp4', m3u8_id='hls')
-
+            m3u8_url, video_id, 'mp4', 'm3u8_native', m3u8_id='hls')
        thumbnail = video_js_data.get('thumbnailUrl') or self._og_search_thumbnail(webpage)
        description = clean_html(get_element_by_id(
            'authorcomment', webpage)) or self._html_search_meta(
@@ -10,7 +10,6 @@ from ..compat import (
    compat_urllib_parse_urlparse,
 )
 from ..utils import (
-    ExtractorError,
    int_or_none,
 )

@@ -193,8 +192,8 @@ class VideomoreIE(InfoExtractor):
            error = item.get('error')
            if error:
                if error in ('Данное видео недоступно для просмотра на территории этой страны', 'Данное видео доступно для просмотра только на территории России'):
-                    self.raise_geo_restricted(countries=['RU'])
-                raise ExtractorError(error, expected=True)
+                    self.raise_geo_restricted(countries=['RU'], metadata_available=True)
+                self.raise_no_formats(error, expected=True)
        self._sort_formats(formats)

        return {
@@ -804,7 +804,7 @@ class VimeoIE(VimeoBaseInfoExtractor):
                video_description = self._html_search_meta(
                    'description', orig_webpage, default=None)
        if not video_description and not is_player:
-            self._downloader.report_warning('Cannot find video description')
+            self.report_warning('Cannot find video description')

        # Extract upload date
        if not timestamp:
@@ -1,16 +1,22 @@
 # coding: utf-8
 from __future__ import unicode_literals

+import json
 import re

 from .common import InfoExtractor
 from ..compat import (
    compat_kwargs,
    compat_str,
+    compat_urlparse,
+    compat_urllib_request,
 )
 from ..utils import (
    ExtractorError,
    int_or_none,
+    try_get,
+    smuggle_url,
+    unsmuggle_url,
 )


@@ -168,7 +174,8 @@ class ViuPlaylistIE(ViuBaseIE):

 class ViuOTTIE(InfoExtractor):
    IE_NAME = 'viu:ott'
-    _VALID_URL = r'https?://(?:www\.)?viu\.com/ott/(?P<country_code>[a-z]{2})/[a-z]{2}-[a-z]{2}/vod/(?P<id>\d+)'
+    _NETRC_MACHINE = 'viu'
+    _VALID_URL = r'https?://(?:www\.)?viu\.com/ott/(?P<country_code>[a-z]{2})/(?P<lang_code>[a-z]{2}-[a-z]{2})/vod/(?P<id>\d+)'
    _TESTS = [{
        'url': 'http://www.viu.com/ott/sg/en-us/vod/3421/The%20Prime%20Minister%20and%20I',
        'info_dict': {
@@ -179,6 +186,7 @@ class ViuOTTIE(InfoExtractor):
        },
        'params': {
            'skip_download': 'm3u8 download',
+            'noplaylist': True,
        },
        'skip': 'Geo-restricted to Singapore',
    }, {
@@ -191,6 +199,19 @@ class ViuOTTIE(InfoExtractor):
        },
        'params': {
            'skip_download': 'm3u8 download',
+            'noplaylist': True,
+        },
+        'skip': 'Geo-restricted to Hong Kong',
+    }, {
+        'url': 'https://www.viu.com/ott/hk/zh-hk/vod/68776/%E6%99%82%E5%B0%9A%E5%AA%BD%E5%92%AA',
+        'playlist_count': 12,
+        'info_dict': {
+            'id': '3916',
+            'title': '時尚媽咪',
+        },
+        'params': {
+            'skip_download': 'm3u8 download',
+            'noplaylist': False,
        },
        'skip': 'Geo-restricted to Hong Kong',
    }]
@@ -201,9 +222,51 @@ class ViuOTTIE(InfoExtractor):
        'TH': 4,
        'PH': 5,
    }
+    _LANGUAGE_FLAG = {
+        'zh-hk': 1,
+        'zh-cn': 2,
+        'en-us': 3,
+    }
+    _user_info = None
+
+    def _detect_error(self, response):
+        code = response.get('status', {}).get('code')
+        if code > 0:
+            message = try_get(response, lambda x: x['status']['message'])
+            raise ExtractorError('%s said: %s (%s)' % (
+                self.IE_NAME, message, code), expected=True)
+        return response['data']
+
+    def _raise_login_required(self):
+        raise ExtractorError(
+            'This video requires login. '
+            'Specify --username and --password or --netrc (machine: %s) '
+            'to provide account credentials.' % self._NETRC_MACHINE,
+            expected=True)
+
+    def _login(self, country_code, video_id):
+        if not self._user_info:
+            username, password = self._get_login_info()
+            if username is None or password is None:
+                return
+
+            data = self._download_json(
+                compat_urllib_request.Request(
+                    'https://www.viu.com/ott/%s/index.php' % country_code, method='POST'),
+                video_id, 'Logging in', errnote=False, fatal=False,
+                query={'r': 'user/login'},
+                data=json.dumps({
+                    'username': username,
+                    'password': password,
+                    'platform_flag_label': 'web',
+                }).encode())
+            self._user_info = self._detect_error(data)['user']
+
+        return self._user_info

    def _real_extract(self, url):
-        country_code, video_id = re.match(self._VALID_URL, url).groups()
+        url, idata = unsmuggle_url(url, {})
+        country_code, lang_code, video_id = re.match(self._VALID_URL, url).groups()

        query = {
            'r': 'vod/ajax-detail',
@@ -223,20 +286,88 @@ class ViuOTTIE(InfoExtractor):
        if not video_data:
            raise ExtractorError('This video is not available in your region.', expected=True)

-        stream_data = self._download_json(
-            'https://d1k2us671qcoau.cloudfront.net/distribute_web_%s.php' % country_code,
-            video_id, 'Downloading stream info', query={
-                'ccs_product_id': video_data['ccs_product_id'],
-            }, headers={
-                'Referer': url,
-                'Origin': re.search(r'https?://[^/]+', url).group(0),
-            })['data']['stream']
+        series_id = video_data.get('series_id')
+        if not self._downloader.params.get('noplaylist') and not idata.get('force_noplaylist'):
+            self.to_screen('Downloading playlist %s - add --no-playlist to just download video' % series_id)
+            series = product_data.get('series', {})
+            product = series.get('product')
+            if product:
+                entries = []
+                for entry in sorted(product, key=lambda x: int_or_none(x.get('number', 0))):
+                    item_id = entry.get('product_id')
+                    if not item_id:
+                        continue
+                    item_id = compat_str(item_id)
+                    entries.append(self.url_result(
+                        smuggle_url(
+                            'http://www.viu.com/ott/%s/%s/vod/%s/' % (country_code, lang_code, item_id),
+                            {'force_noplaylist': True}),  # prevent infinite recursion
+                        'ViuOTT',
+                        item_id,
+                        entry.get('synopsis', '').strip()))
+
+                return self.playlist_result(entries, series_id, series.get('name'), series.get('description'))
+
+        if self._downloader.params.get('noplaylist'):
+            self.to_screen('Downloading just video %s because of --no-playlist' % video_id)
+
+        duration_limit = False
+        query = {
+            'ccs_product_id': video_data['ccs_product_id'],
+            'language_flag_id': self._LANGUAGE_FLAG.get(lang_code.lower()) or '3',
+        }
+        headers = {
+            'Referer': url,
+            'Origin': url,
+        }
+        try:
+            stream_data = self._download_json(
+                'https://d1k2us671qcoau.cloudfront.net/distribute_web_%s.php' % country_code,
+                video_id, 'Downloading stream info', query=query, headers=headers)
+            stream_data = self._detect_error(stream_data)['stream']
+        except (ExtractorError, KeyError):
+            stream_data = None
+            if video_data.get('user_level', 0) > 0:
+                user = self._login(country_code, video_id)
+                if user:
+                    query['identity'] = user['identity']
+                    stream_data = self._download_json(
+                        'https://d1k2us671qcoau.cloudfront.net/distribute_web_%s.php' % country_code,
+                        video_id, 'Downloading stream info', query=query, headers=headers)
+                    stream_data = self._detect_error(stream_data).get('stream')
+                else:
+                    # preview is limited to 3min for non-members
+                    # try to bypass the duration limit
+                    duration_limit = True
+                    query['duration'] = '180'
+                    stream_data = self._download_json(
+                        'https://d1k2us671qcoau.cloudfront.net/distribute_web_%s.php' % country_code,
+                        video_id, 'Downloading stream info', query=query, headers=headers)
+                    try:
+                        stream_data = self._detect_error(stream_data)['stream']
+                    except (ExtractorError, KeyError):  # if still not working, give up
+                        self._raise_login_required()
+
+        if not stream_data:
+            raise ExtractorError('Cannot get stream info', expected=True)

        stream_sizes = stream_data.get('size', {})
        formats = []
        for vid_format, stream_url in stream_data.get('url', {}).items():
            height = int_or_none(self._search_regex(
                r's(\d+)p', vid_format, 'height', default=None))
+
+            # bypass preview duration limit
+            if duration_limit:
+                stream_url = compat_urlparse.urlparse(stream_url)
+                query = dict(compat_urlparse.parse_qsl(stream_url.query, keep_blank_values=True))
+                time_duration = int_or_none(video_data.get('time_duration'))
+                query.update({
+                    'duration': time_duration if time_duration > 0 else '9999999',
+                    'duration_start': '0',
+                })
+                stream_url = stream_url._replace(query=compat_urlparse.urlencode(query)).geturl()
+
            formats.append({
                'format_id': vid_format,
                'url': stream_url,
@@ -8,7 +8,6 @@ from ..compat import (
 )
 from ..utils import (
    int_or_none,
-    ExtractorError,
 )


@@ -125,13 +124,13 @@ class VubeIE(InfoExtractor):
                })
            formats.append(fmt)

-        self._sort_formats(formats)
-
        if not formats and video.get('vst') == 'dmca':
-            raise ExtractorError(
+            self.raise_no_formats(
                'This video has been removed in response to a complaint received under the US Digital Millennium Copyright Act.',
                expected=True)

+        self._sort_formats(formats)
+
        title = video['title']
        description = video.get('description')
        thumbnail = self._proto_relative_url(video.get('thumbnail_src'), scheme='http:')
@@ -87,7 +87,7 @@ class WatIE(InfoExtractor):
        extract_formats({delivery.get('format'): delivery.get('url')})
        if not formats:
            if delivery.get('drm'):
-                raise ExtractorError('This video is DRM protected.', expected=True)
+                self.raise_no_formats('This video is DRM protected.', expected=True)
            manifest_urls = self._download_json(
                'http://www.wat.tv/get/webhtml/' + video_id, video_id, fatal=False)
            if manifest_urls:
@@ -239,7 +239,7 @@ class YahooIE(InfoExtractor):
                'm3u8_native', m3u8_id='hls', fatal=False))

        if not formats and msg == 'geo restricted':
-            self.raise_geo_restricted()
+            self.raise_geo_restricted(metadata_available=True)

        self._sort_formats(formats)

@@ -29,6 +29,7 @@ from ..utils import (
    clean_html,
    dict_get,
    datetime_from_str,
+    error_to_compat_str,
    ExtractorError,
    format_field,
    float_or_none,
@@ -52,6 +53,10 @@ from ..utils import (
 )


+def parse_qs(url):
+    return compat_urlparse.parse_qs(compat_urlparse.urlparse(url).query)
+
+
 class YoutubeBaseInfoExtractor(InfoExtractor):
    """Provide base functions for Youtube extractors"""
    _LOGIN_URL = 'https://accounts.google.com/ServiceLogin'
@@ -72,11 +77,6 @@ class YoutubeBaseInfoExtractor(InfoExtractor):

    _PLAYLIST_ID_RE = r'(?:(?:PL|LL|EC|UU|FL|RD|UL|TL|PU|OLAK5uy_)[0-9A-Za-z-_]{10,}|RDMM|WL|LL|LM)'

-    def _ids_to_results(self, ids):
-        return [
-            self.url_result(vid_id, 'Youtube', video_id=vid_id)
-            for vid_id in ids]
-
    def _login(self):
        """
        Attempt to log in to YouTube.
@@ -127,7 +127,7 @@ class YoutubeBaseInfoExtractor(InfoExtractor):
                })

        def warn(message):
-            self._downloader.report_warning(message)
+            self.report_warning(message)

        lookup_req = [
            username,
@@ -336,16 +336,23 @@ class YoutubeBaseInfoExtractor(InfoExtractor):

    @staticmethod
    def _extract_account_syncid(data):
-        """Extract syncId required to download private playlists of secondary channels"""
-        sync_ids = (
-            try_get(data, lambda x: x['responseContext']['mainAppWebResponseContext']['datasyncId'], compat_str)
-            or '').split("||")
+        """
+        Extract syncId required to download private playlists of secondary channels
+        @param data Either response or ytcfg
+        """
+        sync_ids = (try_get(
+            data, (lambda x: x['responseContext']['mainAppWebResponseContext']['datasyncId'],
+                   lambda x: x['DATASYNC_ID']), compat_str) or '').split("||")
        if len(sync_ids) >= 2 and sync_ids[1]:
            # datasyncid is of the form "channel_syncid||user_syncid" for secondary channel
            # and just "user_syncid||" for primary channel. We only want the channel_syncid
            return sync_ids[0]
+        # ytcfg includes channel_syncid if on secondary channel
+        return data.get('DELEGATED_SESSION_ID')

    def _extract_ytcfg(self, video_id, webpage):
+        if not webpage:
+            return {}
        return self._parse_json(
            self._search_regex(
                r'ytcfg\.set\s*\(\s*({.+?})\s*\)\s*;', webpage, 'ytcfg',
@@ -432,14 +439,10 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
        r'(?:(?:www|dev)\.)?invidio\.us',
        # Invidious instances taken from https://github.com/iv-org/documentation/blob/master/Invidious-Instances.md
        r'(?:www\.)?invidious\.pussthecat\.org',
-        r'(?:www\.)?invidious\.048596\.xyz',
        r'(?:www\.)?invidious\.zee\.li',
-        r'(?:www\.)?vid\.puffyan\.us',
        r'(?:(?:www|au)\.)?ytprivate\.com',
        r'(?:www\.)?invidious\.namazso\.eu',
        r'(?:www\.)?invidious\.ethibox\.fr',
-        r'(?:www\.)?inv\.skyn3t\.in',
-        r'(?:www\.)?invidious\.himiko\.cloud',
        r'(?:www\.)?w6ijuptxiku4xpnnaetxvnkc5vqcdu7mgns2u77qefoixi63vbvnpnqd\.onion',
        r'(?:www\.)?kbjggqkzv65ivcqj6bumvp337z6264huv5kpkwuv6gu5yjiskvan7fad\.onion',
        r'(?:www\.)?invidious\.3o7z6yfxhbw7n3za4rss6l434kmv55cgw2vuziwuigpwegswvwzqipyd\.onion',
@@ -448,25 +451,32 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
        r'(?:(?:www|no)\.)?invidiou\.sh',
        r'(?:(?:www|fi)\.)?invidious\.snopyta\.org',
        r'(?:www\.)?invidious\.kabi\.tk',
-        r'(?:www\.)?invidious\.13ad\.de',
        r'(?:www\.)?invidious\.mastodon\.host',
        r'(?:www\.)?invidious\.zapashcanon\.fr',
        r'(?:www\.)?invidious\.kavin\.rocks',
+        r'(?:www\.)?invidious\.tinfoil-hat\.net',
+        r'(?:www\.)?invidious\.himiko\.cloud',
+        r'(?:www\.)?invidious\.reallyancient\.tech',
        r'(?:www\.)?invidious\.tube',
        r'(?:www\.)?invidiou\.site',
        r'(?:www\.)?invidious\.site',
        r'(?:www\.)?invidious\.xyz',
        r'(?:www\.)?invidious\.nixnet\.xyz',
+        r'(?:www\.)?invidious\.048596\.xyz',
        r'(?:www\.)?invidious\.drycat\.fr',
+        r'(?:www\.)?inv\.skyn3t\.in',
        r'(?:www\.)?tube\.poal\.co',
        r'(?:www\.)?tube\.connect\.cafe',
        r'(?:www\.)?vid\.wxzm\.sx',
        r'(?:www\.)?vid\.mint\.lgbt',
+        r'(?:www\.)?vid\.puffyan\.us',
        r'(?:www\.)?yewtu\.be',
        r'(?:www\.)?yt\.elukerio\.org',
        r'(?:www\.)?yt\.lelux\.fi',
        r'(?:www\.)?invidious\.ggc-project\.de',
        r'(?:www\.)?yt\.maisputain\.ovh',
+        r'(?:www\.)?ytprivate\.com',
+        r'(?:www\.)?invidious\.13ad\.de',
        r'(?:www\.)?invidious\.toot\.koeln',
        r'(?:www\.)?invidious\.fdn\.fr',
        r'(?:www\.)?watch\.nettohikari\.com',
@@ -509,16 +519,9 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
                         |(?:www\.)?cleanvideosearch\.com/media/action/yt/watch\?videoId=
                         )
                     )?                                                       # all until now is optional -> you can pass the naked ID
-                     (?P<id>[0-9A-Za-z_-]{11})                                      # here is it! the YouTube video ID
-                     (?!.*?\blist=
-                        (?:
-                            %(playlist_id)s|                                  # combined list/video URLs are handled by the playlist IE
-                            WL                                                # WL are handled by the watch later IE
-                        )
-                     )
+                     (?P<id>[0-9A-Za-z_-]{11})                                # here is it! the YouTube video ID
                     (?(1).+)?                                                # if we found the ID, everything can follow
                     $""" % {
-        'playlist_id': YoutubeBaseInfoExtractor._PLAYLIST_ID_RE,
        'invidious': '|'.join(_INVIDIOUS_SITES),
    }
    _PLAYER_INFO_RE = (
@@ -1003,6 +1006,11 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
            },
            'skip': 'This video does not exist.',
        },
+        {
+            # Video with incomplete 'yt:stretch=16:'
+            'url': 'https://www.youtube.com/watch?v=FRhJzUSJbGI',
+            'only_matching': True,
+        },
        {
            # Video licensed under Creative Commons
            'url': 'https://www.youtube.com/watch?v=M4gD1WSo5mA',
@@ -1298,6 +1306,16 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
        },
    ]

+    @classmethod
+    def suitable(cls, url):
+        # Hack for lazy extractors until more generic solution is implemented
+        # (see #28780)
+        from .youtube import parse_qs
+        qs = parse_qs(url)
+        if qs.get('list', [None])[0]:
+            return False
+        return super(YoutubeIE, cls).suitable(url)
+
    def __init__(self, *args, **kwargs):
        super(YoutubeIE, self).__init__(*args, **kwargs)
        self._code_cache = {}
@@ -1733,7 +1751,7 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
                    # See: https://github.com/ytdl-org/youtube-dl/issues/28194
                    last_error = 'Incomplete data received'
                    if count >= retries:
-                        self._downloader.report_error(last_error)
+                        raise ExtractorError(last_error)

            if not response:
                break
@@ -2044,7 +2062,7 @@ class YoutubeIE(YoutubeBaseInfoExtractor):

        if not formats:
            if not self._downloader.params.get('allow_unplayable_formats') and streaming_data.get('licenseInfos'):
-                raise ExtractorError(
+                self.raise_no_formats(
                    'This video is DRM protected.', expected=True)
            pemr = try_get(
                playability_status,
@@ -2059,11 +2077,10 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
                    if not countries:
                        regions_allowed = search_meta('regionsAllowed')
                        countries = regions_allowed.split(',') if regions_allowed else None
-                    self.raise_geo_restricted(
-                        subreason, countries)
+                    self.raise_geo_restricted(subreason, countries, metadata_available=True)
                reason += '\n' + subreason
            if reason:
-                raise ExtractorError(reason, expected=True)
+                self.raise_no_formats(reason, expected=True)

        self._sort_formats(formats)

@@ -2074,13 +2091,16 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
                for m in re.finditer(self._meta_regex('og:video:tag'), webpage)]
        for keyword in keywords:
            if keyword.startswith('yt:stretch='):
-                w, h = keyword.split('=')[1].split(':')
-                w, h = int(w), int(h)
-                if w > 0 and h > 0:
-                    ratio = w / h
-                    for f in formats:
-                        if f.get('vcodec') != 'none':
-                            f['stretched_ratio'] = ratio
+                mobj = re.search(r'(\d+)\s*:\s*(\d+)', keyword)
+                if mobj:
+                    # NB: float is intentional for forcing float division
+                    w, h = (float(v) for v in mobj.groups())
+                    if w > 0 and h > 0:
+                        ratio = w / h
+                        for f in formats:
+                            if f.get('vcodec') != 'none':
+                                f['stretched_ratio'] = ratio
+                        break

        thumbnails = []
        for container in (video_details, microformat):
@@ -2477,6 +2497,15 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
            'uploader_id': 'UCqj7Cz7revf5maW9g5pgNcg',
            'uploader': 'Игорь Клейнер',
        },
+    }, {
+        # playlists, series
+        'url': 'https://www.youtube.com/c/3blue1brown/playlists?view=50&sort=dd&shelf_id=3',
+        'playlist_mincount': 5,
+        'info_dict': {
+            'id': 'UCYO_jab_esuFRV4b17AJtAw',
+            'title': '3Blue1Brown - Playlists',
+            'description': 'md5:e1384e8a133307dd10edee76e875d62f',
+        },
    }, {
        # playlists, singlepage
        'url': 'https://www.youtube.com/user/ThirstForScience/playlists',
@@ -2621,6 +2650,29 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
            'uploader_id': 'UCXw-G3eDE9trcvY2sBMM_aA',
        },
        'playlist_mincount': 21,
+    }, {
+        'note': 'Playlist with "show unavailable videos" button',
+        'url': 'https://www.youtube.com/playlist?list=UUTYLiWFZy8xtPwxFwX9rV7Q',
+        'info_dict': {
+            'title': 'Uploads from Phim Siêu Nhân Nhật Bản',
+            'id': 'UUTYLiWFZy8xtPwxFwX9rV7Q',
+            'uploader': 'Phim Siêu Nhân Nhật Bản',
+            'uploader_id': 'UCTYLiWFZy8xtPwxFwX9rV7Q',
+        },
+        'playlist_mincount': 1400,
+        'expected_warnings': [
+            'YouTube said: INFO - Unavailable videos are hidden',
+        ]
+    }, {
+        'note': 'Playlist with unavailable videos in a later page',
+        'url': 'https://www.youtube.com/playlist?list=UU8l9frL61Yl5KFOl87nIm2w',
+        'info_dict': {
+            'title': 'Uploads from BlankTV',
+            'id': 'UU8l9frL61Yl5KFOl87nIm2w',
+            'uploader': 'BlankTV',
+            'uploader_id': 'UC8l9frL61Yl5KFOl87nIm2w',
+        },
+        'playlist_mincount': 20000,
    }, {
        # https://github.com/ytdl-org/youtube-dl/issues/21844
        'url': 'https://www.youtube.com/playlist?list=PLzH6n4zXuckpfMu_4Ff8E7Z1behQks5ba',
@@ -2760,6 +2812,9 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
            'title': '#cctv9',
        },
        'playlist_mincount': 350,
+    }, {
+        'url': 'https://www.youtube.com/watch?list=PLW4dVinRY435CBE_JD3t-0SRXKfnZHS1P&feature=youtu.be&v=M9cJMXmQ_ZU',
+        'only_matching': True,
    }]

    @classmethod
@@ -2783,14 +2838,16 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
    @staticmethod
    def _extract_basic_item_renderer(item):
        # Modified from _extract_grid_item_renderer
-        known_renderers = (
-            'playlistRenderer', 'videoRenderer', 'channelRenderer',
-            'gridPlaylistRenderer', 'gridVideoRenderer', 'gridChannelRenderer'
+        known_basic_renderers = (
+            'playlistRenderer', 'videoRenderer', 'channelRenderer', 'showRenderer'
        )
        for key, renderer in item.items():
-            if key not in known_renderers:
+            if not isinstance(renderer, dict):
                continue
-            return renderer
+            elif key in known_basic_renderers:
+                return renderer
+            elif key.startswith('grid') and key.endswith('Renderer'):
+                return renderer

    def _grid_entries(self, grid_renderer):
        for item in grid_renderer['items']:
@@ -2800,7 +2857,8 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
            if not isinstance(renderer, dict):
                continue
            title = try_get(
-                renderer, lambda x: x['title']['runs'][0]['text'], compat_str)
+                renderer, (lambda x: x['title']['runs'][0]['text'],
+                           lambda x: x['title']['simpleText']), compat_str)
            # playlist
            playlist_id = renderer.get('playlistId')
            if playlist_id:
@@ -2808,10 +2866,12 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
                    'https://www.youtube.com/playlist?list=%s' % playlist_id,
                    ie=YoutubeTabIE.ie_key(), video_id=playlist_id,
                    video_title=title)
+                continue
            # video
            video_id = renderer.get('videoId')
            if video_id:
                yield self._extract_video(renderer)
+                continue
            # channel
            channel_id = renderer.get('channelId')
            if channel_id:
@@ -2820,6 +2880,17 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
                yield self.url_result(
                    'https://www.youtube.com/channel/%s' % channel_id,
                    ie=YoutubeTabIE.ie_key(), video_title=title)
+                continue
+            # generic endpoint URL support
+            ep_url = urljoin('https://www.youtube.com/', try_get(
+                renderer, lambda x: x['navigationEndpoint']['commandMetadata']['webCommandMetadata']['url'],
+                compat_str))
+            if ep_url:
+                for ie in (YoutubeTabIE, YoutubePlaylistIE, YoutubeIE):
+                    if ie.suitable(ep_url):
+                        yield self.url_result(
+                            ep_url, ie=ie.ie_key(), video_id=ie._match_id(ep_url), video_title=title)
+                        break

    def _shelf_entries_from_content(self, shelf_renderer):
        content = shelf_renderer.get('content')
@@ -3033,46 +3104,15 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
        for page_num in itertools.count(1):
            if not continuation:
                break
+            query = {
+                'continuation': continuation['continuation'],
+                'clickTracking': {'clickTrackingParams': continuation['itct']}
+            }
            headers = self._generate_api_headers(ytcfg, identity_token, account_syncid, visitor_data)
-            retries = self._downloader.params.get('extractor_retries', 3)
-            count = -1
-            last_error = None
-            while count < retries:
-                count += 1
-                if last_error:
-                    self.report_warning('%s. Retrying ...' % last_error)
-                try:
-                    response = self._call_api(
-                        ep='browse', fatal=True, headers=headers,
-                        video_id='%s page %s' % (item_id, page_num),
-                        query={
-                            'continuation': continuation['continuation'],
-                            'clickTracking': {'clickTrackingParams': continuation['itct']},
-                        },
-                        context=context,
-                        api_key=self._extract_api_key(ytcfg),
-                        note='Downloading API JSON%s' % (' (retry #%d)' % count if count else ''))
-                except ExtractorError as e:
-                    if isinstance(e.cause, compat_HTTPError) and e.cause.code in (500, 503, 404):
-                        # Downloading page may result in intermittent 5xx HTTP error
-                        # Sometimes a 404 is also recieved. See: https://github.com/ytdl-org/youtube-dl/issues/28289
-                        last_error = 'HTTP Error %s' % e.cause.code
-                        if count < retries:
-                            continue
-                    raise
-                else:
-                    # Youtube sometimes sends incomplete data
-                    # See: https://github.com/ytdl-org/youtube-dl/issues/28194
-                    if dict_get(response,
-                                ('continuationContents', 'onResponseReceivedActions', 'onResponseReceivedEndpoints')):
-                        break
-
-                    # Youtube may send alerts if there was an issue with the continuation page
-                    self._extract_alerts(response, expected=False)
-
-                    last_error = 'Incomplete data received'
-                    if count >= retries:
-                        self._downloader.report_error(last_error)
+            response = self._extract_response(
+                item_id='%s page %s' % (item_id, page_num),
+                query=query, headers=headers, ytcfg=ytcfg,
+                check_get_keys=('continuationContents', 'onResponseReceivedActions', 'onResponseReceivedEndpoints'))

            if not response:
                break
@@ -3229,8 +3269,13 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
                self._extract_ytcfg(item_id, webpage)),
            **metadata)

-    def _extract_mix_playlist(self, playlist, playlist_id):
+    def _extract_mix_playlist(self, playlist, playlist_id, data, webpage):
        first_id = last_id = None
+        ytcfg = self._extract_ytcfg(playlist_id, webpage)
+        headers = self._generate_api_headers(
+            ytcfg, account_syncid=self._extract_account_syncid(data),
+            identity_token=self._extract_identity_token(webpage, item_id=playlist_id),
+            visitor_data=try_get(self._extract_context(ytcfg), lambda x: x['client']['visitorData'], compat_str))
        for page_num in itertools.count(1):
            videos = list(self._playlist_entries(playlist))
            if not videos:
@@ -3245,14 +3290,25 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
                yield video
            first_id = first_id or videos[0]['id']
            last_id = videos[-1]['id']
-
-            _, data = self._extract_webpage(
-                'https://www.youtube.com/watch?list=%s&v=%s' % (playlist_id, last_id),
-                '%s page %d' % (playlist_id, page_num))
+            watch_endpoint = try_get(
+                playlist, lambda x: x['contents'][-1]['playlistPanelVideoRenderer']['navigationEndpoint']['watchEndpoint'])
+            query = {
+                'playlistId': playlist_id,
+                'videoId': watch_endpoint.get('videoId') or last_id,
+                'index': watch_endpoint.get('index') or len(videos),
+                'params': watch_endpoint.get('params') or 'OAE%3D'
+            }
+            response = self._extract_response(
+                item_id='%s page %d' % (playlist_id, page_num),
+                query=query,
+                ep='next',
+                headers=headers,
+                check_get_keys='contents'
+            )
            playlist = try_get(
-                data, lambda x: x['contents']['twoColumnWatchNextResults']['playlist']['playlist'], dict)
+                response, lambda x: x['contents']['twoColumnWatchNextResults']['playlist']['playlist'], dict)

-    def _extract_from_playlist(self, item_id, url, data, playlist):
+    def _extract_from_playlist(self, item_id, url, data, playlist, webpage):
        title = playlist.get('title') or try_get(
            data, lambda x: x['titleText']['simpleText'], compat_str)
        playlist_id = playlist.get('playlistId') or item_id
@@ -3267,7 +3323,7 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
                video_title=title)

        return self.playlist_result(
-            self._extract_mix_playlist(playlist, playlist_id),
+            self._extract_mix_playlist(playlist, playlist_id, data, webpage),
            playlist_id=playlist_id, playlist_title=title)

    def _extract_alerts(self, data, expected=False):
@@ -3297,10 +3353,102 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
                warnings.append([alert_type, alert_message])

        for alert_type, alert_message in (warnings + errors[:-1]):
-            self._downloader.report_warning('YouTube said: %s - %s' % (alert_type, alert_message))
+            self.report_warning('YouTube said: %s - %s' % (alert_type, alert_message))
        if errors:
            raise ExtractorError('YouTube said: %s' % errors[-1][1], expected=expected)

+    def _reload_with_unavailable_videos(self, item_id, data, webpage):
+        """
+        Get playlist with unavailable videos if the 'show unavailable videos' button exists.
+        """
+        sidebar_renderer = try_get(
+            data, lambda x: x['sidebar']['playlistSidebarRenderer']['items'], list)
+        if not sidebar_renderer:
+            return
+        browse_id = params = None
+        for item in sidebar_renderer:
+            if not isinstance(item, dict):
+                continue
+            renderer = item.get('playlistSidebarPrimaryInfoRenderer')
+            menu_renderer = try_get(
+                renderer, lambda x: x['menu']['menuRenderer']['items'], list) or []
+            for menu_item in menu_renderer:
+                if not isinstance(menu_item, dict):
+                    continue
+                nav_item_renderer = menu_item.get('menuNavigationItemRenderer')
+                text = try_get(
+                    nav_item_renderer, lambda x: x['text']['simpleText'], compat_str)
+                if not text or text.lower() != 'show unavailable videos':
+                    continue
+                browse_endpoint = try_get(
+                    nav_item_renderer, lambda x: x['navigationEndpoint']['browseEndpoint'], dict) or {}
+                browse_id = browse_endpoint.get('browseId')
+                params = browse_endpoint.get('params')
+                break
+
+            ytcfg = self._extract_ytcfg(item_id, webpage)
+            headers = self._generate_api_headers(
+                ytcfg, account_syncid=self._extract_account_syncid(ytcfg),
+                identity_token=self._extract_identity_token(webpage, item_id=item_id),
+                visitor_data=try_get(
+                    self._extract_context(ytcfg), lambda x: x['client']['visitorData'], compat_str))
+            query = {
+                'params': params or 'wgYCCAA=',
+                'browseId': browse_id or 'VL%s' % item_id
+            }
+            return self._extract_response(
+                item_id=item_id, headers=headers, query=query,
+                check_get_keys='contents', fatal=False,
+                note='Downloading API JSON with unavailable videos')
+
+    def _extract_response(self, item_id, query, note='Downloading API JSON', headers=None,
+                          ytcfg=None, check_get_keys=None, ep='browse', fatal=True):
+        response = None
+        last_error = None
+        count = -1
+        retries = self._downloader.params.get('extractor_retries', 3)
+        if check_get_keys is None:
+            check_get_keys = []
+        while count < retries:
+            count += 1
+            if last_error:
+                self.report_warning('%s. Retrying ...' % last_error)
+            try:
+                response = self._call_api(
+                    ep=ep, fatal=True, headers=headers,
+                    video_id=item_id, query=query,
+                    context=self._extract_context(ytcfg),
+                    api_key=self._extract_api_key(ytcfg),
+                    note='%s%s' % (note, ' (retry #%d)' % count if count else ''))
+            except ExtractorError as e:
+                if isinstance(e.cause, compat_HTTPError) and e.cause.code in (500, 503, 404):
+                    # Downloading page may result in intermittent 5xx HTTP error
+                    # Sometimes a 404 is also recieved. See: https://github.com/ytdl-org/youtube-dl/issues/28289
+                    last_error = 'HTTP Error %s' % e.cause.code
+                    if count < retries:
+                        continue
+                if fatal:
+                    raise
+                else:
+                    self.report_warning(error_to_compat_str(e))
+                    return
+
+            else:
+                # Youtube may send alerts if there was an issue with the continuation page
+                self._extract_alerts(response, expected=False)
+                if not check_get_keys or dict_get(response, check_get_keys):
+                    break
+                # Youtube sometimes sends incomplete data
+                # See: https://github.com/ytdl-org/youtube-dl/issues/28194
+                last_error = 'Incomplete data received'
+                if count >= retries:
+                    if fatal:
+                        raise ExtractorError(last_error)
+                    else:
+                        self.report_warning(last_error)
+                        return
+        return response
+
    def _extract_webpage(self, url, item_id):
        retries = self._downloader.params.get('extractor_retries', 3)
        count = -1
@@ -3319,7 +3467,7 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
            if data.get('contents') or data.get('currentVideoEndpoint'):
                break
            if count >= retries:
-                self._downloader.report_error(last_error)
+                raise ExtractorError(last_error)
        return webpage, data

    def _real_extract(self, url):
@@ -3331,13 +3479,13 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
        mobj = re.match(r'(?P<pre>%s)(?P<post>/?(?![^#?]).*$)' % self._VALID_URL, url)
        mobj = mobj.groupdict() if mobj else {}
        if mobj and not mobj.get('not_channel'):
-            self._downloader.report_warning(
+            self.report_warning(
                'A channel/user page was given. All the channel\'s videos will be downloaded. '
                'To download only the videos in the home page, add a "/featured" to the URL')
            url = '%s/videos%s' % (mobj.get('pre'), mobj.get('post') or '')

        # Handle both video/playlist URLs
-        qs = compat_urlparse.parse_qs(compat_urlparse.urlparse(url).query)
+        qs = parse_qs(url)
        video_id = qs.get('v', [None])[0]
        playlist_id = qs.get('list', [None])[0]

@@ -3346,7 +3494,7 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
                # If there is neither video or playlist ids,
                # youtube redirects to home page, which is undesirable
                raise ExtractorError('Unable to recognize tab page')
-            self._downloader.report_warning('A video URL was given without video ID. Trying to download playlist %s' % playlist_id)
+            self.report_warning('A video URL was given without video ID. Trying to download playlist %s' % playlist_id)
            url = 'https://www.youtube.com/playlist?list=%s' % playlist_id

        if video_id and playlist_id:
@@ -3357,6 +3505,9 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):

        webpage, data = self._extract_webpage(url, item_id)

+        # YouTube sometimes provides a button to reload playlist with unavailable videos.
+        data = self._reload_with_unavailable_videos(item_id, data, webpage) or data
+
        tabs = try_get(
            data, lambda x: x['contents']['twoColumnBrowseResultsRenderer']['tabs'], list)
        if tabs:
@@ -3365,13 +3516,13 @@ class YoutubeTabIE(YoutubeBaseInfoExtractor):
        playlist = try_get(
            data, lambda x: x['contents']['twoColumnWatchNextResults']['playlist']['playlist'], dict)
        if playlist:
-            return self._extract_from_playlist(item_id, url, data, playlist)
+            return self._extract_from_playlist(item_id, url, data, playlist, webpage)

        video_id = try_get(
            data, lambda x: x['currentVideoEndpoint']['watchEndpoint']['videoId'],
            compat_str) or video_id
        if video_id:
-            self._downloader.report_warning('Unable to recognize playlist. Downloading just video %s' % video_id)
+            self.report_warning('Unable to recognize playlist. Downloading just video %s' % video_id)
            return self.url_result(video_id, ie=YoutubeIE.ie_key(), video_id=video_id)

        raise ExtractorError('Unable to recognize tab page')
@@ -3440,12 +3591,19 @@ class YoutubePlaylistIE(InfoExtractor):

    @classmethod
    def suitable(cls, url):
-        return False if YoutubeTabIE.suitable(url) else super(
-            YoutubePlaylistIE, cls).suitable(url)
+        if YoutubeTabIE.suitable(url):
+            return False
+        # Hack for lazy extractors until more generic solution is implemented
+        # (see #28780)
+        from .youtube import parse_qs
+        qs = parse_qs(url)
+        if qs.get('v', [None])[0]:
+            return False
+        return super(YoutubePlaylistIE, cls).suitable(url)

    def _real_extract(self, url):
        playlist_id = self._match_id(url)
-        qs = compat_urlparse.parse_qs(compat_urlparse.urlparse(url).query)
+        qs = parse_qs(url)
        if not qs:
            qs = {'list': playlist_id}
        return self.url_result(
@@ -3527,7 +3685,7 @@ class YoutubeFavouritesIE(YoutubeBaseInfoExtractor):
            ie=YoutubeTabIE.ie_key())


-class YoutubeSearchIE(SearchInfoExtractor, YoutubeBaseInfoExtractor):
+class YoutubeSearchIE(SearchInfoExtractor, YoutubeTabIE):
    IE_DESC = 'YouTube.com searches, "ytsearch" keyword'
    # there doesn't appear to be a real limit, for example if you search for
    # 'python' you get more than 8.000.000 results
@@ -3543,9 +3701,10 @@ class YoutubeSearchIE(SearchInfoExtractor, YoutubeBaseInfoExtractor):
            data['params'] = self._SEARCH_PARAMS
        total = 0
        for page_num in itertools.count(1):
-            search = self._call_api(
-                ep='search', video_id='query "%s"' % query, fatal=False,
-                note='Downloading page %s' % page_num, query=data)
+            search = self._extract_response(
+                item_id='query "%s" page %s' % (query, page_num), ep='search', query=data,
+                check_get_keys=('contents', 'onResponseReceivedCommands')
+            )
            if not search:
                break
            slr_contents = try_get(
@@ -3,7 +3,6 @@ from __future__ import unicode_literals

 from .common import InfoExtractor
 from ..utils import (
-    ExtractorError,
    int_or_none,
 )

@@ -48,8 +47,8 @@ class ZingMp3BaseIE(InfoExtractor):
                return
            msg = item['msg']
            if msg == 'Sorry, this content is not available in your country.':
-                self.raise_geo_restricted(countries=self._GEO_COUNTRIES)
-            raise ExtractorError(msg, expected=True)
+                self.raise_geo_restricted(countries=self._GEO_COUNTRIES, metadata_available=True)
+            self.raise_no_formats(msg, expected=True)
        self._sort_formats(formats)

        subtitles = None
@@ -395,6 +395,10 @@ def parseOpts(overrideArguments=None):
        '--break-on-reject',
        action='store_true', dest='break_on_reject', default=False,
        help='Stop the download process when encountering a file that has been filtered out')
+    selection.add_option(
+        '--skip-playlist-after-errors', metavar='N',
+        dest='skip_playlist_after_errors', default=None, type=int,
+        help='Number of allowed failures until the rest of the playlist is skipped')
    selection.add_option(
        '--no-download-archive',
        dest='download_archive', action="store_const", const=None,
@@ -522,7 +526,7 @@ def parseOpts(overrideArguments=None):
        action='store_true', dest='allow_unplayable_formats', default=False,
        help=(
            'Allow unplayable formats to be listed and downloaded. '
-            'All video postprocessing will also be turned off'))
+            'All video post-processing will also be turned off'))
    video_format.add_option(
        '--no-allow-unplayable-formats',
        action='store_false', dest='allow_unplayable_formats',
@@ -548,7 +552,7 @@ def parseOpts(overrideArguments=None):
    subtitles.add_option(
        '--all-subs',
        action='store_true', dest='allsubtitles', default=False,
-        help='Download all the available subtitles of the video')
+        help=optparse.SUPPRESS_HELP)
    subtitles.add_option(
        '--list-subs',
        action='store_true', dest='listsubtitles', default=False,
@@ -561,7 +565,10 @@ def parseOpts(overrideArguments=None):
        '--sub-langs', '--srt-langs',
        action='callback', dest='subtitleslangs', metavar='LANGS', type='str',
        default=[], callback=_comma_separated_values_options_callback,
-        help='Languages of the subtitles to download (optional) separated by commas, use --list-subs for available language tags')
+        help=(
+            'Languages of the subtitles to download (can be regex) or "all" separated by commas. (Eg: --sub-langs en.*,ja) '
+            'You can prefix the language code with a "-" to exempt it from the requested languages. (Eg: --sub-langs all,-live_chat) '
+            'Use --list-subs for a list of available language tags'))

    downloader = optparse.OptionGroup(parser, 'Download Options')
    downloader.add_option(
@@ -749,6 +756,16 @@ def parseOpts(overrideArguments=None):
        '-s', '--simulate',
        action='store_true', dest='simulate', default=False,
        help='Do not download the video and do not write anything to disk')
+    verbosity.add_option(
+        '--ignore-no-formats-error',
+        action='store_true', dest='ignore_no_formats_error', default=False,
+        help=(
+            'Ignore "No video formats" error. Usefull for extracting metadata '
+            'even if the video is not actually available for download (experimental)'))
+    verbosity.add_option(
+        '--no-ignore-no-formats-error',
+        action='store_false', dest='ignore_no_formats_error',
+        help='Throw error when no downloadable video formats are found (default)')
    verbosity.add_option(
        '--skip-download', '--no-download',
        action='store_true', dest='skip_download', default=False,
@@ -803,8 +820,8 @@ def parseOpts(overrideArguments=None):
        '--force-write-archive', '--force-write-download-archive', '--force-download-archive',
        action='store_true', dest='force_write_download_archive', default=False,
        help=(
-            'Force download archive entries to be written as far as no errors occur,'
-            'even if -s or another simulation switch is used (Alias: --force-download-archive)'))
+            'Force download archive entries to be written as far as no errors occur, '
+            'even if -s or another simulation option is used (Alias: --force-download-archive)'))
    verbosity.add_option(
        '--newline',
        action='store_true', dest='progress_with_newline', default=False,
@@ -891,7 +908,7 @@ def parseOpts(overrideArguments=None):
    filesystem.add_option(
        '--autonumber-start',
        dest='autonumber_start', metavar='NUMBER', default=1, type=int,
-        help='Specify the start value for %(autonumber)s (default is %default)')
+        help=optparse.SUPPRESS_HELP)
    filesystem.add_option(
        '--restrict-filenames',
        action='store_true', dest='restrictfilenames', default=False,
@@ -945,7 +962,7 @@ def parseOpts(overrideArguments=None):
        action='store_false', dest='continue_dl',
        help=(
            'Do not resume partially downloaded fragments. '
-            'If the file is unfragmented, restart download of the entire file'))
+            'If the file is not fragmented, restart download of the entire file'))
    filesystem.add_option(
        '--part',
        action='store_false', dest='nopart', default=False,
@@ -1,11 +1,13 @@
 from __future__ import unicode_literals

+import re
 import subprocess

 from .common import PostProcessor
 from ..compat import compat_shlex_quote
 from ..utils import (
    encodeArgument,
+    FORMAT_RE,
    PostProcessingError,
 )

@@ -20,18 +22,20 @@ class ExecAfterDownloadPP(PostProcessor):
    def pp_key(cls):
        return 'Exec'

-    def run(self, info):
-        tmpl, info_copy = self._downloader.prepare_outtmpl(self.exec_cmd, info)
-        cmd = tmpl % info_copy
-        if cmd == self.exec_cmd:  # No replacements were made
-            if '{}' not in self.exec_cmd:
-                self.exec_cmd += ' {}'
-            cmd = self.exec_cmd.replace('{}', compat_shlex_quote(info['filepath']))
+    def parse_cmd(self, cmd, info):
+        # If no %(key)s is found, replace {} for backard compatibility
+        if not re.search(FORMAT_RE.format(r'[-\w>.+]+'), cmd):
+            if '{}' not in cmd:
+                cmd += ' {}'
+            return cmd.replace('{}', compat_shlex_quote(info['filepath']))

+        tmpl, info_copy = self._downloader.prepare_outtmpl(cmd, info)
+        return tmpl % info_copy
+
+    def run(self, info):
+        cmd = self.parse_cmd(self.exec_cmd, info)
        self.to_screen('Executing command: %s' % cmd)
        retCode = subprocess.call(encodeArgument(cmd), shell=True)
        if retCode != 0:
-            raise PostProcessingError(
-                'Command returned error code %d' % retCode)
-
+            raise PostProcessingError('Command returned error code %d' % retCode)
        return [], info
@@ -7,7 +7,7 @@ from ..compat import compat_str


 class MetadataFromFieldPP(PostProcessor):
-    regex = r'(?P<in>.+):(?P<out>.+)$'
+    regex = r'(?P<in>.*?)(?<!\\):(?P<out>.+)$'

    def __init__(self, downloader, formats):
        PostProcessor.__init__(self, downloader)
@@ -17,10 +17,11 @@ class MetadataFromFieldPP(PostProcessor):
            assert isinstance(f, compat_str)
            match = re.match(self.regex, f)
            assert match is not None
+            inp = match.group('in').replace('\\:', ':')
            self._data.append({
-                'in': match.group('in'),
+                'in': inp,
                'out': match.group('out'),
-                'tmpl': self.field_to_template(match.group('in')),
+                'tmpl': self.field_to_template(inp),
                'regex': self.format_to_regex(match.group('out')),
            })

@@ -68,6 +69,6 @@ class MetadataFromFieldPP(PostProcessor):

 class MetadataFromTitlePP(MetadataFromFieldPP):  # for backward compatibility
    def __init__(self, downloader, titleformat):
-        super(MetadataFromTitlePP, self).__init__(downloader, ['title:%s' % titleformat])
+        super(MetadataFromTitlePP, self).__init__(downloader, ['%%(title)s:%s' % titleformat])
        self._titleformat = titleformat
        self._titleregex = self._data[0]['regex']
@@ -40,6 +40,7 @@ import zlib
 from .compat import (
    compat_HTMLParseError,
    compat_HTMLParser,
+    compat_HTTPError,
    compat_basestring,
    compat_chr,
    compat_cookiejar,
@@ -2925,12 +2926,60 @@ class YoutubeDLCookieProcessor(compat_urllib_request.HTTPCookieProcessor):


 class YoutubeDLRedirectHandler(compat_urllib_request.HTTPRedirectHandler):
-    if sys.version_info[0] < 3:
-        def redirect_request(self, req, fp, code, msg, headers, newurl):
-            # On python 2 urlh.geturl() may sometimes return redirect URL
-            # as byte string instead of unicode. This workaround allows
-            # to force it always return unicode.
-            return compat_urllib_request.HTTPRedirectHandler.redirect_request(self, req, fp, code, msg, headers, compat_str(newurl))
+    """YoutubeDL redirect handler
+
+    The code is based on HTTPRedirectHandler implementation from CPython [1].
+
+    This redirect handler solves two issues:
+     - ensures redirect URL is always unicode under python 2
+     - introduces support for experimental HTTP response status code
+       308 Permanent Redirect [2] used by some sites [3]
+
+    1. https://github.com/python/cpython/blob/master/Lib/urllib/request.py
+    2. https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/308
+    3. https://github.com/ytdl-org/youtube-dl/issues/28768
+    """
+
+    http_error_301 = http_error_303 = http_error_307 = http_error_308 = compat_urllib_request.HTTPRedirectHandler.http_error_302
+
+    def redirect_request(self, req, fp, code, msg, headers, newurl):
+        """Return a Request or None in response to a redirect.
+
+        This is called by the http_error_30x methods when a
+        redirection response is received.  If a redirection should
+        take place, return a new Request to allow http_error_30x to
+        perform the redirect.  Otherwise, raise HTTPError if no-one
+        else should try to handle this url.  Return None if you can't
+        but another Handler might.
+        """
+        m = req.get_method()
+        if (not (code in (301, 302, 303, 307, 308) and m in ("GET", "HEAD")
+                 or code in (301, 302, 303) and m == "POST")):
+            raise compat_HTTPError(req.full_url, code, msg, headers, fp)
+        # Strictly (according to RFC 2616), 301 or 302 in response to
+        # a POST MUST NOT cause a redirection without confirmation
+        # from the user (of urllib.request, in this case).  In practice,
+        # essentially all clients do redirect in this case, so we do
+        # the same.
+
+        # On python 2 urlh.geturl() may sometimes return redirect URL
+        # as byte string instead of unicode. This workaround allows
+        # to force it always return unicode.
+        if sys.version_info[0] < 3:
+            newurl = compat_str(newurl)
+
+        # Be conciliant with URIs containing a space.  This is mainly
+        # redundant with the more complete encoding done in http_error_302(),
+        # but it is kept for compatibility with other callers.
+        newurl = newurl.replace(' ', '%20')
+
+        CONTENT_HEADERS = ("content-length", "content-type")
+        # NB: don't use dict comprehension for python 2.6 compatibility
+        newheaders = dict((k, v) for k, v in req.headers.items()
+                          if k.lower() not in CONTENT_HEADERS)
+        return compat_urllib_request.Request(
+            newurl, headers=newheaders, origin_req_host=req.origin_req_host,
+            unverifiable=True)


 def extract_timezone(date_str):
@@ -6043,11 +6092,20 @@ def load_plugins(name, type, namespace):


 def traverse_dict(dictn, keys, casesense=True):
-    if not isinstance(dictn, dict):
-        return None
-    first_key = keys[0]
-    if not casesense:
-        dictn = {key.lower(): val for key, val in dictn.items()}
-        first_key = first_key.lower()
-    value = dictn.get(first_key, None)
-    return value if len(keys) < 2 else traverse_dict(value, keys[1:], casesense)
+    keys = list(keys)[::-1]
+    while keys:
+        key = keys.pop()
+        if isinstance(dictn, dict):
+            if not casesense:
+                dictn = {k.lower(): v for k, v in dictn.items()}
+                key = key.lower()
+            dictn = dictn.get(key)
+        elif isinstance(dictn, (list, tuple, compat_str)):
+            key, n = int_or_none(key), len(dictn)
+            if key is not None and -n <= key < n:
+                dictn = dictn[key]
+            else:
+                dictn = None
+        else:
+            return None
+    return dictn
@@ -1,3 +1,3 @@
 from __future__ import unicode_literals

-__version__ = '2021.04.03'
+__version__ = '2021.04.11'
Author	SHA1	Message	Date
pukkandan	0cf0571560	Release 2021.04.22	2021-04-22 16:58:28 +05:30
pukkandan	e58c22a0f6	[documentation] Fix typos	2021-04-22 16:54:44 +05:30
pukkandan	e4bdd3377d	[ci] Disable fail-fast	2021-04-22 16:54:41 +05:30
pukkandan	0b2e9d2c30	[lazy_extractor] Do not load plugins	2021-04-22 16:54:08 +05:30
pukkandan	1bdae7d312	Update to ytdl-commit-7e8b3f9 [youtube] Remove unused code https://github.com/ytdl-org/youtube-dl/commit/7e8b3f9439ebefb3a3a4e5da9c0bd2b595976438	2021-04-22 16:54:07 +05:30
Felix SandGitHub	a471f21da6	[mildom] Remove proxy (#260 ) Closes #251 Makes `2cff495997`, `ab406a1c0e`, #252 obsolete Authored by: fstirlitz	2021-04-22 16:52:22 +05:30
pukkandan	6efb071135	[BilibiliChannel] Fix pagination Closes #222 https://github.com/nao20010128nao/ytdl-patched/commit/ccca21d7f57f1330a205f07d369e31883ba006c2 Coauthored by: nao20010128nao, pukkandan	2021-04-22 04:19:33 +05:30
pukkandan	f4536226c1	[documentation] Clarify which deprecated options still work	2021-04-22 04:19:33 +05:30
pukkandan	a439a3a45c	Improve output template (see desc) * Objects can be traversed like `%(field.key1.key2)s` * A number can be added to the field as `%(field+n)s` * Deprecates `--autonumber-start`	2021-04-22 04:19:33 +05:30
pukkandan	26e2805c3f	Add option `--skip-playlist-after-errors` Allows to skip the rest of a playlist after a given number of errors are encountered	2021-04-22 02:16:31 +05:30
pukkandan	3b4775e021	[go] Fix `_VALID_URL` Closes #255	2021-04-21 15:43:53 +05:30
pukkandan	ab406a1c0e	[mildom] Warn user of proxy	2021-04-21 15:43:22 +05:30
pukkandan	a3faeb7de4	[MetadataFromField] Improve regex and add tests	2021-04-21 11:12:04 +05:30
pukkandan	8c54a3051d	[youtube] Bugfix in `_extract_ytcfg`	2021-04-21 10:37:24 +05:30
pukkandan	c32b0aab8a	Improve --sub-langs (see desc) * Treat `--sub-langs` entries as regex * `all` can be used to refer to all the subtitles * the language code can be prefixed with `-` to exclude it * Deprecates `--all-subs` Closes #253	2021-04-20 02:58:03 +05:30
pukkandan	3097d9e512	[mildom:user:vod] Download only necessary amount of pages	2021-04-19 11:41:49 +05:30
pukkandan	c1df120eda	[mildom:vod] Remove proxy * Proxy is needed only for live videos	2021-04-19 11:41:40 +05:30
pukkandan	2cff495997	[mildom] Change proxy Related: #251 Closes #252	2021-04-19 11:41:33 +05:30
pukkandan	d0491a1ebe	[twitcasting] Fix extractor * `Origin: https://twitcasting.tv` must be sent when requesting the webpage. Otherwise the extracted `m3u8` will always give a `502` * Fix regex for when `data-movie-playlist` is a dict containing the needed list * media initialization is fully supported; so change downloader to native Closes #220	2021-04-18 17:57:46 +05:30
nixxoandGitHub	b9d68c199b	[rai] Add support for http formats (#208 ) Authored by: nixxo	2021-04-17 22:42:28 +05:30
Felix SandGitHub	155510fe81	Improve the yt-dlp.sh script (#248 ) * Quote the `$0` variable to correctly handle spaces * Change the shebang line to `/bin/sh` to avoid unnecessarily depending on bash * Use the `exec` command to avoid having the shell process linger unnecessarily * Change the mode to make the script directly executable Authored by: fstirlitz :ci skip all	2021-04-17 19:21:25 +05:30
pukkandan	201c145953	Update to ytdl-commit-9f6c03 [cbsnews] Fix extraction for python <3.6 https://github.com/ytdl-org/youtube-dl/commit/9f6c03a00602eb1119e43a522cf50682f6d6a6dd	2021-04-17 08:40:31 +05:30
pukkandan	5d34200268	[youtube:tab] Reload with unavailable videos for all playlists If the unavailable video is in a later page, the warning and button are not shown in the initial webpage So we force all playlists' initial page to reload with the correct params	2021-04-17 08:40:30 +05:30
pukkandan	b7da73eb19	Add option `--ignore-no-formats-error` * Ignores the "no video format" and similar errors * Experimental - Some extractors may still throw these errors	2021-04-17 08:40:30 +05:30
pukkandan	6a39ee13f7	Fix inconsistent use of `report_warning`	2021-04-17 04:16:41 +05:30
pukkandan	33245766ab	[downloader] Fix ffmpeg selection for m3u8_native	2021-04-17 04:15:56 +05:30
coletdjnzandGitHub	358de58c4d	[youtube:tab] Show unavailable videos in playlists (#242 ) Closes #231 Authored by: colethedj	2021-04-17 04:09:08 +05:30
pukkandan	a7191c6f57	Fix some linter and typos	2021-04-16 05:31:47 +05:30
lkhoandpukkandan	baa5873942	[viu:ott] Fix extractor (see desc) * add language_flag_id query param * add support for premium account (untested since I dont have a premium account) * support entire series Code from: https://github.com/blackjack4494/youtube-dlc/pull/211 https://github.com/ytdl-org/youtube-dl/pull/15182 https://github.com/ytdl-org/youtube-dl/pull/26775 Fixes: https://github.com/yt-dlp/yt-dlp/issues/219 https://github.com/ytdl-org/youtube-dl/issues/27946 https://github.com/ytdl-org/youtube-dl/issues/27863 https://github.com/ytdl-org/youtube-dl/issues/27812 https://github.com/ytdl-org/youtube-dl/issues/27464 https://github.com/ytdl-org/youtube-dl/issues/26788 https://github.com/blackjack4494/yt-dlc/issues/136 Possibly also fixes (untested): https://github.com/ytdl-org/youtube-dl/issues/16992 https://github.com/ytdl-org/youtube-dl/issues/26701 Co-authored by: lkho, pukkandan	2021-04-16 05:19:46 +05:30
pukkandan	c6ce815461	[Exec] Ensure backward compatibility when the command contains `%`	2021-04-16 05:19:44 +05:30
coletdjnzandGitHub	79360d99d3	[youtube] Standardize API calls for tabs, mixes and search (#245 ) Authored by: colethedj	2021-04-15 16:52:59 +05:30
pukkandan	46fff7105e	[youtube] Ignore invalid stretch ratio Closes #244	2021-04-14 15:22:17 +05:30
pukkandan	72e1fe969f	[downloader] Fix downloader selection for m3u8 Bug introduced by: `52a8a1e1b9` and `a31953b0e6`	2021-04-14 12:25:42 +05:30
AshishandGitHub	b5be6dd504	[TubiTv] Add TubiTvShowIE (#243 ) Authored by: Ashish0804	2021-04-14 12:22:28 +05:30
coletdjnzandGitHub	8ea3f7b909	[youtube] Improve channel syncid extraction to support ytcfg (#241 ) Authored by: colethedj	2021-04-14 10:37:03 +05:30
pukkandan	921b76cab8	Ensure `mergeall` selects best format when multistreams are disabled	2021-04-13 10:53:25 +05:30
pukkandan	a31953b0e6	[downloader] Fix external downloader selection for m3u8 Closes #239	2021-04-12 22:34:11 +05:30
pukkandan	54670cf084	[version] update :ci skip all	2021-04-12 03:30:55 +05:30